To remain competitive, every modern enterprise must adopt the latest generation of new operational and business support systems. These systems are designed to stream-line operations and enable the enterprise to rapidly adapt when faced with competitive threats and disruptive market events. For example, the telecoms, data and entertainment industries have rapidly converged, forcing all players to adapt their businesses, and their way of doing business, at a ferocious pace.
The systems, which are needed to support these new business trends are very different from their bespoke, often home grown predecessors. The systems are very much more flexible and bring together data, from many disparate systems. The problem is further compounded by their need to be populated with data which may never previously have been recorded.
Halogence has developed HT fusion, designed specifically to ease the population of these new operational and business support systems at the time of their inception. HT fusion provides a framework within which the data sources, target model, mapping, validation and inference rules, are all specified and agreed with the stakeholders. Once these have been defined, HT fusion performs bulk data processing to produce a data payload for the target system, supported by a set of data quality reports.
HT fusion has an easy to use graphical user interface, which guides the consultant through a series of steps:
- Specify the target model
Define the hierarchy of data containers in the model, and the attributes held by each. For example a telecoms service provider may model their subscribers as having a number of sites. Each site, will deliver a number of services. This model reflects the way the data is designed within the target system.
- Specify the attributes for each of the containers
Each container (e.g. subscriber) holds a series of attributes, which will be loaded into the target database. For example a subscriber may have a contact telephone number, a billing address, a level of service etc.
- Specify the data mapping between the source data sets and the target attributes
Where the data exists in a legacy system, and can be mapped directly, this simple relationship is specified. Where the relationship does not exist, it will be inferred or defaulted according to a series of rules/algorithms. The product supports complex, attribute specific programmability using the widely supported Python language.
- Specify the data validation rules
The source data and the target values can be audited against a set of selectable predefined rules and/or custom rules, which can be programmed using the widely supported Python language.
The process typically involves multiple stakeholder groups. To allow progress towards a convergent solution that can be tracked, the status of each attribute is reported using red, amber, green reporting. Comments can also be added to the mapping.
Once the transformation and validation rules have been specified, the data is bulk processed. The resulting validation reports can be used to direct a data cleanse program or to tune the rules. This cycle may be repeated several times. When an acceptable level of data quality is achieved, the exported payload can be used to populate the target system.
Below is a diagram of the overall process. A user will log in to the HT fusion portal to download and install the HT fusion client, on a local machine. The user will load a sample of the data into the client and specify the mapping, transformation and validation rules using the client. Once the rules have been completed the user will then export a blueprint via the client. The blueprint contains all of the necessary instructions regarding the data transformation.
The user will upload the data and the blueprint to the portal. Once this has been completed the user will select these files to submit a job. The HT fusion engines will pick up the job from the workstack and process the uploaded data using the instructions set within the blueprint.
The job management console within the portal will indicate to the user the current status of the job. When the HT fusion engine has completed the job the user is able to download the newly transformed files. In addition to this, validation reports are exported from the HT fusion engine to allow the user to understand which source data was not processed as per the validation rules set within the blueprint.
The Halogence Advantage
- Web based migration capability and easy to use mapping software
- Free "Proof of Concept", mapping, validation and migration of your data using our Bronze service
- Easy to understand and flexible pricing model to allow budgeting
- Migration capability in a product not a service
- No services/vendor lock in
- Easy to use Integrated Development Environment for data transformation and validation projects
- Leverages the well publicised and powerful "Python" language to create custom validation and transformation rules
- No need for temporary migration databases, for example migration staging environments
- Offers a standard framework to migration projects
- Utilises cloud technology for easy integration
- Bronze, Silver and Gold levels of service