Release Early Release Often

Release Early Release Often (RERO) technique proposes to have releases early and often, instead of a big bang release. This approach is typically followed in tech startups, working on Open source projects. That’s the reason we see many of Google’s products still in beta version and their updates getting released once in a month or so. We planned to experiment the strategy for a big Master Data Management (MDM) project. The experimentation turned out to be successful. The rest of the essay discusses the experience details of such an implementation.

User Thrill

Important features of the application were phased out for various distinct releases. Some of them were Hierarchy & Workflow management, Security and Exception reporting. And the duration between releases were as close as 2 weeks. That meant, the user saw features getting added once in 2 weeks. We captured the user feedback about the releases and made sure we corrected it in the immediate ones. This approach had a two prong benefit. User experienced the application very, very early and we experienced the bugs. By the time, the UAT phase reached us, the application had reached a near-to-zero defect zone. We were a bit skeptical whether the user participation would be high, but since the product was there to be played with, it naturally attracted them.

Incremental Application testing

The application was getting tested from the day the first beta was released; rather from the “Go Live” day. Although this created few negative impressions on the user experience due to few unpleasant bugs; they knew that it was in its beta stages and the next release would have the patched version. In fact, our testing team grew from a 3 member team to a 6 member virtual team (There were 3 business users).

Support framework

To enable such a dynamic release process, the revision control and the code review/release systems should be efficient; there would be multiple releases instead of one. The integration testing should be solid. And the unit testing before the releases should be good enough not to distract your users completely; dissolving the purpose. Meticulous planning of the releases will also form a key to the success. The development tools that you use should be agile and adaptable enough to accept and implement the user’s feedback for the next release.

Conclusion

The experiment turned out to be a success. This strategy would work for most of your implementations, unless it’s a maintenance project with less than a week’s duration of deliverable.

Which MDM approach is right for you?

MDM, in the past 5 years, has come a long way in its maturity model. Most of the MDM implementations fall under 2 different kinds of approaches.

1. Operational MDM (the tougher among the two)
2. Analytical MDM

Operational MDM enables synchronization of master entities and their attributes between the transaction processing systems. Why does one need such an MDM? Let's take an example. ABC Corporation is a manufacturing firm. It conducts roadshows and marketing campaigns to advertise its products. The salesperson collect customer information during those roadshows and feed it into their IT systems for further followup. There are a different set of sales representatives who conduct feedback on their products sold, with their customers. They too enter the customer feedback into their IT systems. These are 2 different sets of CRM processes.

Typically what happens in a mature company is, there are a set of batch processes which pick up the master data from one system and transfer it to the other. Now this introduces delay, inconsistency, inaccuracy of data and lot of manual reconciliation (same customer name can be entered by 2 different salesperson or the latest survey from a salesperson can erase previously collected information about the customer). So the IT develops custom programs to clean up the data, write reconciliation programs but still cannot manage to do all this in real time.

This mess can be reduced or eliminated by deploying an operational MDM. Operational MDM tools solve the synchronization problem using complex match-merge algorthims. Some of the tools currently in the market are Siperian, IBM, Purisma, Oracle and SAP.

Analytical MDM is an architectural approach if the problem revolves around inconsistent reporting for business performance management. In simple terms, inconsistent hierarchies are getting reported out. This needs for a unified reporting view of the master data. The audience for this system would be the downstream data warehousing and business intelligence applications. Some of the MDM vendors selling their expertise in this area are Kalido, Oracle, IBM.

It is essential that an organization has to build both these models to address their MDM needs. But which one to chose first depends on which problem is in their high priority list.

Teradata's reseller alliance with Trillium

Teradata Corporation announced its reseller alliance partnership with Trillium Software. Teradata will now combine its warehouse product with Trillium's Data quality tools and its own MDM products. Overall, this seems to be a good strategy for Teradata, because now Teradata's customers can leverage Trillium's data quality abilities on their huge databases.

Because of this alliance, the customers will enjoy a powerpacked database, Data Quality tools and a MDM suite. Information Difference has ranked Teradata's MDM low in the quadrant though compared to the likes of SAP, Oracle and Siperian.

Buy or Make - Financial Analytics

Today, I had a consulting assignment with a company focussing on Server Virtualization. The objective was to narrate the factors influencing a Make vs Buy (mVb) decision and their risk quotients for a Financial Analytics Solution.

Some of them are :

1. What is the business requirement and is the requirement very unique?
2. How urgent is the application?
3. What is the technology Strategy of the Organization?
4. Does the off-the-shelf product address most of the requirements and does it have flexibility to customize it?
5. How does the present make-buy decision relate to the strategy?
6. Are their right people and support systems to support the application, in case of a build?
7. Does the financial tool address internationalization needs?
8. Are their security measures in-built in the tool, because it hosts sensitive data?
9. Can the Integration of the Packaged Solution into the process control system be done seamlessly?
10. What is the underlying technology? In this case, what is the ERP system? It would make sense to buy the analytical solution from the same vendor of the ERP system, if it addresses your requirements
11. Will the TCO be reduced because of the Buy approach?
12. Are their right people and support systems to support the application, in case of a build?
13. Does it reduce cost?

After these questions were answered, the following matrices were prepared which summed up the decision.

1. High Level Requirement x Priority x Effort Estimation Matrix
2. Benefit Comparison Matrix
3. Risk Comparison Matrix

Statistics and Data

I was reading an excellent text "Statistics for Business and Economics" written by Anderson & Sweeney. It highlights the importance of statistical measures in decision making. Many of the existing predictive analytical tools use most of the principles covered in the text. It also highlights the importance of collecting and preserving data.

One such example covered was to calculate the average wait time of a queue in a particular ATM in New York. Using this data, the bank would then decide to position a new ATM to balance the load in that busy place. The predictive model uses probability distribution and helps the analyst in making a decision. The models have to be refined so that they don't reflect any false positives.

Why should I make my MDM SoA enabled?

A Car Sales Manager is capturing the details of a customer, who visited his showroom. After jotting down the client's address details, the Manager wants to check out if the address is a valid one. How can he achieve it?

A PRO in the same organization receives a call from the customer that he wants to change his address in the system records. The PRO logs in to the silo-ed application and enters the new address. When entering the address, the PRO wants to check if the new address is a valid one. How can she achieve it?

MDM and SoA make this happen. MDM is more like a service provider and SoA is a framework helping the consumers to access the service with ease. A Location Master Repository tied up a SoA framework makes any consumer to use the services of the Location Master. This way, the same master data gets reused throughout the organization for multiple purposes.

In one of our projects, the client had a unique requirement to assess if a product is worth promoting and if found promotable, what is the promotion to be given to it. Such a requirement was addressed using a SoA framework on a product portfolio MDM. The architecture had BEA AquaLogic Service Bus interacting with Kalido MDM to provide the services. The challenges lied in identifying the streams which would consume this service and evaluating if its worth considering a service. The service found its usage in many CRM applications in the organization.

MDM - SoA Marriage

After a long break from blogging, I am starting my series of explorations with MDM and SoA. When would somebody go for a SoA implementation for an MDM solution? Is it because it's a enterprise-wide initiative to make everything the SoA way? My current project proved to be a big failure on this front. We had to build an MDM solution and the enterprise architecture team had a clear focus to make anything and everything SoA enabled. The MDM solution was built on Kalido. Who are the consumers of this MDM data? Answer is . A series of downstream CRM applications. Sounds good. Where would the SoA architecture fit in? Is it just in the Consumer world or also in the Sourcing world? We had to design the reporting solution completely SoA. But during a period of stress testing, it proved that the SoA framework just couldn't handle the volumes the downstream applications were streaming. The MDM Solution would get a huge number of updates from the ERP stack everyday and all these changes had to be donated to the consumers. The SOAP message was just too big to be parsed by the reporting solution. The users had to wait for a considerable amount of time to get their reports out.

So my question is whether you design the MDM solution the SoA way expecting in future that things with performance would get solved or wait till things get solved and then re-architect the solution?

Currently, we have the SoA suite disabled and reports are being fired out from the databases directly to the reporting solution.

XML-based MDM

After a brief hiatus, I am writing this article on Orchestra networks' EBX.Platform - an XML based approach to Master Data Management. The EBX.Platform is based on J2EE and XML. The whole architecture is based on 3 items - Models, Services and Modules (Models + Services).

So how are they able to achieve their MDM framework?

The Data Model is developed based on a simple XML schema standard and this is termed as an Adaptation Model. Services such as import/export can be added on top of the adaptation models. It can also be maintenance features provided by the UI. And finally Modules, nothing but Models and Services are deployed as Web applications.

They also support quite a unique feature - Branches & versions of Master Data. This helps a company to maintain its current version of master data, when it is working on a futuristic model.

I would like to monitor the progress of this interesting tool, given that it is being used in some big companies.

Data Modeling a Maze

Couple of weeks back, my friend took me to a maze. I was lost in a couple of minutes and was getting really frustrated after a while. I wasn't sure what algorithm they had used to construct the maze.

The only algorithm that I knew was the "Wall follower". All you have to do is it to follow either your right-hand or left hand touching the wall and you will reach either the exit or the entrance. I did take the longest path, but eventually reached the EXIT.

This algorithm would work only if all the walls are connected to form a loop. From that point, I was quite fascinated with the algorithms associated with the maze. There are also a few other efficient algorithms like Tremaux's algorithm. Visit Think Labyrinth for more fun.

After that, I thought of simulating a maze. Unfortunately, I am quite inept at programming languages, so decided to do what I know best. I thought of creating a simple data model for a Wall Follower Maze. It turned out to be quite an interesting problem. It took me around 30 minutes to come up with a decent logical data model, that would work for quite some scenarios.

So, the first model that I came up with is shown below. (Click on the picture to enlarge)

Let me give a quick explanation of the model

1. Design Co-ordinate: Super-type Entity
2. Entry, Exit and In-Maze Coordinate: Sub-Type Entities of DESIGN CO-ORDINATE which contains the co-ordinates of the location, where a player has to take a decision.
3. Decision: Entity which holds whether to turn LEFT,RIGHT,UP,DOWN or ABORT.
4. Decision Map: Entity which holds the map of a START-COORDINATE, DECISION TAKEN(whether to move left,right,up,down (or) abort) and an END-COORDINATE (the co-ordinate where he lands after he takes the decision).
5. Player: Entity which holds information about the player of the maze.
6. Movement: Entity which tracks the movement of the player.

There is one interesting phenomenon happening in this model. If an intelligent player has to play this model, this model would work, because the association between MOVEMENT-PLAYER-DECISION MAP has been modeled as an Identifying relationship. It means if a player, tries to navigate the same path twice, the system will spit out an error, (simulating an INTELLIGENT player, because he would never do the same mistake again).

But if a DUMB player had to play this maze, then this model wouldn't work, because a DUMB player would make the mistake of traversing a path with no fruits, again and again. So the association should be made as a Non-Identifying relationship.

How can I model both the scenarios at one-shot, without introducing redundancy in the entities or associations?

One of the ways to model both the scenarios is to track both their movements as 2 different MOVEMENT entities and include the constraint in the INTELLIGENT PLAYER MOVEMENT's entity. But this introduces an extra associative entity. It's easy for a toggle situation, Yes or No, DUMB or INTELLIGENT.

But, if I were to model differing levels of intelligence, how would I do it in a data model, without writing any procedural code to do it? How can E-R data models be efficiently designed for Fuzzy Logic Systems?

I found this as an interesting exercise to showcase that E-R data models are still long way from being truly a self sufficient tool.

We need a modern day E.F.Codd

Kalido's Business Information Modeler

Today, I received an update from Kalido on the Business Information Modeler Engine. This is what Kalido claims about the product.

"Kalido Business Information Modeler provides a graphical design interface that can be used to develop and refine business requirements for new and existing information. Instead of modeling data and their structures, the Kalido Business Information Modeler allows you to model the actual parts of your business; customers, products, assets, transactions, even people – and define how you want to see information in context. Even better, the Kalido Business Information Modeler can be used to change and update your model directly against the Kalido Dynamic Information Warehouse, allowing you ultimate flexibility in meeting the information needs of your business. The Kalido Business Information Modeler dramatically improves your ability to meet the needs of your business when it requires it – not when how it’s stored determines it".

The product is due for March 2008. I am waiting to experiment on the new features it claims. I will be evaluating the product on the following questions.

1. Can an in-house data warehouse be easily migrated into Kalido?
2. Will the business layer completely abstract the data layer?
3. Is it just a visual aid for creating/maintaining your data model?
4. Will the data in the warehouse be used by the tool to help the modeler provide real-time feedback on the errors and the inconsistencies of the new model that he plans to implement?

I will be writing more on this interesting product after I get a practical hands-on. Visit www.kalido.com for more details.