MDM - SoA Marriage

After a long break from blogging, I am starting my series of explorations with MDM and SoA. When would somebody go for a SoA implementation for an MDM solution? Is it because it's a enterprise-wide initiative to make everything the SoA way? My current project proved to be a big failure on this front. We had to build an MDM solution and the enterprise architecture team had a clear focus to make anything and everything SoA enabled. The MDM solution was built on Kalido. Who are the consumers of this MDM data? Answer is . A series of downstream CRM applications. Sounds good. Where would the SoA architecture fit in? Is it just in the Consumer world or also in the Sourcing world? We had to design the reporting solution completely SoA. But during a period of stress testing, it proved that the SoA framework just couldn't handle the volumes the downstream applications were streaming. The MDM Solution would get a huge number of updates from the ERP stack everyday and all these changes had to be donated to the consumers. The SOAP message was just too big to be parsed by the reporting solution. The users had to wait for a considerable amount of time to get their reports out.

So my question is whether you design the MDM solution the SoA way expecting in future that things with performance would get solved or wait till things get solved and then re-architect the solution?

Currently, we have the SoA suite disabled and reports are being fired out from the databases directly to the reporting solution.

MDM - Part 2 - Key capabilities of an MDM framework

Last year, one of my colleagues, was deployed to a leading FMCG company's workplace to understand their global reference data and consolidate it. She started off with interviewing the data management heads from various countries and after 6 weeks of tough grind, she came up with a very good logical data model. But when she started materializing her E-R model into the tool, she started facing problems. On further investigation with many of my colleagues who have worked on MDM implementations and also with my own experience, I have collated a few key points that are essential for the smooth running of an MDM engine.

Note: Broadly they have been categorized as Must-to-have(bold red) and Nice-to-have(bold green).

1. Data Governance and Stewardship: Identifying the right people to own the right data. This team is responsible for setting up the security access, correcting the erroneous data, defining the work-flow and acting on the notifications and submitting a report on the usage of the data.
   
2. Data Quality Management: Bad data is as good as not having the data at all. Processes and frameworks constantly working on the business rules, to furnish out sanity to the master data is a must. This is one of the most complex points in the whole MDM cycle. Does the tool possess adequate data validation techniques or does it rely on the ETL tool?
   
3. Flexible Data Modeling Capability: The tool should be as adaptive as the business process. A flexible data model to quickly prototype and develop is the ideal tool for such an implementation.
   
4. Integration Engine Maturity: The Data Integration drivers that get shipped along with the tool play a key role in the tool evaluation. Look for a tool which has good integration capability. Some of the tools stop with a flat file feature; though this might be be enough to start developing your repository, there might be added ETL effort if your sources are completely heterogeneous in nature.
5. Work-flow enabled Authorization Model: How does the authorization and the publishing of data happen? Is it through mails or through a sophisticated work-flow engine? Based on my acquaintances with the tools in the market, I have found that much of the MDM analyst's time in occupied in composing mails about the next action items that have to be taken on the data. This is where a tool with a 'cool' work flow feature takes its upper hand.
6. Security & Access control: Can the users of the Indian market control Australian customers? Probably yes, maybe no. Security and access driven capability of the MDM system is a must for an organization trying to consolidate its world-wide master information.
   
7. Search & UI Customization: In this search-driven world, (thanks to Google), a tool without search capabilities is a failure written all over it. The UI should be customizable and the framework should have inherent APIs to achieve the same.
   
8. Data Enrichment: Some of the tools have the means to integrate with the market research data vendors to enrich their data. A good example could be enriching the customer data for D&B related fields. Though this is not a MUST feature, it certainly is a feature for tool differentiation.
   
9. Service Oriented in Nature: SoA utilizes loosely coupled, reusable, and interoperable software services to support business process requirements. Though this is not very specific to the tool, it is more of a framework question - Can the MDM solution easily get positioned into the SoA architecture? For example, if the tool has capability to talk to different sources, integrate the data, present the data as services; YES it has capabilities to marry SoA.
   
10. Distributed system: This probably is one of the last items to be ever evaluated. If your master data runs into Tera bytes, then this feature of the tool might be worth visiting.
    

These 10 points sum up the different capabilities/components of an MDM solution. There are few other points like cost and platform dependency which I would like to place it to the discretion of the organization's policies.