Merlin - Legacy Migration to Microservices.
The Goals of Merlin
We code named the project Merlin because the thought was it will take a wizard to make all this work. The business goals of Merlin were to take a slow moving Property and Casualty (P&C) company and transform their technical capabilities so they could support the strategic goal of selling P&C software and services to other companies in the industry while using the new software for their own internal customers. This would have the effect of turning their technology group from a cost center into a revenue generating arm for the company. We built a business case that supported a financially neutral scenario in two years and profitability in four. After conducting business assessments and market analysis, numerous senior executive and board level meetings the project was green lighted and we were under the gun to hit our financial targets.
There were three main technical parts to this effort - first, a development component which required us to create a new more modern system that would offer better services at higher quality to existing internal and external clients. The second component was customer transition - moving existing customers and their data to the new systems. Lastly, a business development element that would sell the the system to new clients. I led the first two components and interfaced with a colleague who was responsible for sales and marketing of the system. This was a fairly large effort with off shore development teams, legacy business and systems staff and new systems technical and support people.
Creating The New System & Migrating to Microservices
I have had a significant number of legacy migration projects in my career and find them challenging and enjoyable. Here we had to develop new functionality and migrate the legacy systems to the new microservices architecture. Microservices was the design choice because there was a need for fast frequent changes for each customer with client configurability, speed and flexibility being of paramount importance for the new business. In a series of blog post I will be going into detail on how we performed the migration to microservices but at a high level it involved using DDD (Domain Driven Design) to analyze legacy systems, review and decompose current code base to begin to break down a monolith code base. While a separate team took on product owner responsibilities related to new functionality. The new functionality was then incorporated into the migrated system.
In addition to new microservices based architecture the CTO took the opportunity for a full technical tool and methodology refresh. Moving from a waterfall delivery methodology to Agile DevOps using an all Microsoft based dev stack.
Moving Existing Customers to the New Platform
Moving data and processes from the old system to the new environment is always tricky business. One of the most common mistakes clients make is they move too slowly and the “scaffolding” or “bridge” layer needed for the migration i.e. the tools, staff and processes; remain in place too long. Our mantra was; get the scaffolding up, make the move and tear down the scaffolding as soon as possible. The cost associated with the scaffolding becomes unbearable over an extended period of time. To minimize the amount of time the scaffolding was required we did the following:
OCM and Training - Prepare the users of the system for the change. A robust Organizational Change Management (OCM) program combined with communications and training helps to prepare customers and decrease fear uncertainty and doubt.
Automate - We spent significant time upfront to automate as much as possible this included data migration, data verification, testing, and data cleansing/ETL.
SWAT Team - We put a separate SWAT time in place to address any concerns customers may have. Most of the member of this team started off on the project as analyst or entry level developers and moved to a SWAT team. This gave them strong familiarity with new and old systems and fixed issues early.
Business Development
As with any new development effort sales and marketing were out there making promises and looking for new customers. This was of course necessary but put an added burden on engineering as must have features poured in. Eventually a small pre-sales engineering group was created to support the business development team and act as a bridge to product engineering.
Merlin was not without hiccups but ultimately is was successful. Learnings from this project have provided experience that I have applied in many other like projects.