Insights as a Service - Using External Partners for AI/ML

Whether you are just starting to think about implementing an AI practice using machine learning (ML) or want to expand your existing ML operations, insights as a service is an alternative that corporate CxO’s should consider. Insights as a service is a methodology that allows business units to pay by the drink for data insights related to specific high value business problems.

Definition of Insights as a Service

Insights as a service is a methodology to leverage an external partner for some of your ML requirements. The partner will provide tools, ML models, compute resources, trained data scientists and domain experts, and most importantly access to a trove of external data for you to leverage during your analysis. The buyer pays by the drink for this service, meaning you pay for the insights you receive. We have designed deals between parties where the buyer only pays for actionable results, if there is a dry hole i.e. no actionable information then you don’t pay for that effort. There are many other ways to set up contracts to incentivize the partner to produce what you ultimately want - results that move the needle for the business. Further on in this post I have a case study showing some of the details of a successful implementation.

Why use Insights as a Service

It’s not an entirely hands off process when you invest in insights as a service I will cover the buyer’s responsibilities in the implementation sections below. You should consider using Insights as a Service for one reason - speed to market, getting actionable insights sooner. If you choose your vendor/partner wisely you will be able to get insights sooner because that partner is bringing their own data including licenses for external data, pre-build and tested ML models to be refined for you, and they should have trained staff ready to deploy to your project. For companies just getting started with AI I have seen implementations that typically take 8 to 12 months from start to finish get cut to 2 or 3 months. Even companies with a mature AI/ML practice an insights as a service provider can augment your internal team providing additional expertise, new data sources and staff that can cut your project backlog providing the business areas with valuable insights for decision making.

Implementing Insights as a Service

Insights as a Service programs that I have put together generally fall into one of two groups; the first is a mature organization that wants to add additional throughput into their organization to meet increasing demand or to provide some expertise that they lack. The more challenging case is the second situation where the organization wants to get started with ML and is looking for a way to show real progress in a reasonable amount of time. It is this second scenario that I am going to focus on here.

Getting Started in AI

The process of starting an AI program is the same as introducing any other significant and new technology into the enterprise with the difference being that AI will bring greater opportunities and implications for your business. AI is considered by many to be a general-purpose technology which are technologies that have profound and lasting changes that ripple across the globe spawning new industries, ways of working and changing how people live. For example, electricity was such a technology - it significantly changed how people lived and worked. New products were created and it spurred explosive growth and disruption. Introducing AI and machine learning into the organization takes a significant amount of up front work.

For this post I will approach AI/ML introduction from the perspective of the Chief Technology Office (CTO) as they are a likely advocate for the technology. As a senior leader, the CTO must take on the role of educator of her peers at the senior leadership level. Helping them understand in business terms what AI/ML is and is not, the kinds of problems that it can be best applied to, what kind of returns and new opportunities could open up because of the new technology. At the heart of doing the hard work and sometimes long process of this education is good old fashion marketing. In the past I have assisted CTO’s with this effort by leading tours and holding Q and A’s with other companies who have successfully and recently started this journey, brought vendors in for short “art of the possible” meetings, engaged speakers (technical and business) to discuss industry trends, email short pieces that explain AI/ML. I would recommend reading Andrew Ng AI Playbook which I have followed in the past for successful POC/MVP projects.

Thinking “stractically”; not too strategic not overly tactical

There is a trade off between strategy and tactical — in the beginning phase I recommend a happy medium which we call “stractical”. You want to think strategically but don’t get too ivory tower, you want to be tactical but not short sighted. The CTO needs to help senior leadership balance these two competing interests. We help customers think about digital transformations and if there ever was a technology that will drive transformation it’s AI/ML. Be sure to think about the strategic direction you want to pursue, for example; how will this new and powerful technology allow me to service my current customers better and open new opportunities, what are my competitors doing in the space, who are my competitors now and does AI change the competitive landscape, how will I avail myself of new data and new sources of data, what new business innovations does this open up for us, and how do I adapt to my customers changing needs and values, how will I build an adaptive value proposition. As you, the CTO think through these areas you will simultaneously be thinking of which project(s) you should choose as a pilot aka proof of concepts or as I prefer to think of it, as a minimal viable product (MVP).

Conducting the training and education sessions while guiding the conversation between a strategic and tactical plan is necessary for success in your organization. You will uncover opportunities and build trust and gain supporters, which you will need to be successful.

Choosing the Right Project

Choosing the right project for your first implementation is important as you want to maximize your chances for success as well as your impact on the business. Some project characteristics to think about:

• Transparency of the model: If you need to understand why your models made a certain prediction or why they didn’t make a different prediction you most likely won’t be able to get this information using current ML algorithms - for the most part ML algorithms in use today are black boxes even to the inventors.

• Data availability: Is the data you will need available, accurate, usable, and sizable? Do you need and can you get rights to external data sources?

• Industry related project: Don’t pick a project that is unrelated to the business you are in. If you work for a medical device company don’t pick a project that uses ML to make staffing/hiring decisions more quickly. Look for a project that moves the needle for the business domain you are in so when you are “selling” the ideas and techniques demonstrated from a successful implementation it’s readily apparent how it applies to the business at hand.

• Too large vs trivial: As you think about your first foray into machine learning you will need to balance the trade off between a project that is too large vs one that is just trivial. Look for something that will produce a relatively quick win and demonstrate to senior leadership that this is a direction that is worth investing in for future growth. Ideally you will look for a project that will positively impact revenue growth.

Choosing the Right Partner

This post is focused on Insights as a Service, the crux of which is partnering with a company that can augment your organization’s skills and tools so you can quickly maximize your chances of success in implementing a new technology. With machine learning you need to look for a company that fits culturally, brings strengths where you have deficits and vice versa, has relevant business domain experience, brings pre-trained models, has labeled data from external sources along with the rights to use that data, a deep bench of data scientists, software engineers, and statisticians, and brings the compute power and tools necessary to run a machine learning project.

A good partner will be able to assist you with all phases in the lifecycle of a ML project. In a coming post I will discuss the ML project lifecycle in detail but for know understand that this is a multi-step process that begins with business goals and direction, data collection and analysis, feature engineering, model development, training and maintenance, through model deployment. When you are choosing a partner it is incumbent on you to evaluate them across this lifecycle to ensure that they have the wherewithal to help you in this journey.

Be aware of your financial goals as you choose a ML partner and how you wish to incentivize them for your success. Seek out a partner who is open to new more modern financial arrangements such as gain share or risk/reward share models for success.

Reserved Data: Test Set

When using an Insights as a Service partner the most important item to remember is to hold data back from your partner to use as a final evaluation after they have completed their model building, training and validation testing.

Data for ML is split into at least two sets - a training set and a validation set. The training set is used for training your models. The validation set is used to evaluate the the model’s performance by the ML developers. In theory the model is being validated on data it has never seen before so if it is making accurate predictions on the data it is because it has learned the characteristics of the data not because it has seen the data before. It is very similar to studying for a test in college - if you study by reviewing practice problems it is generally not a good idea to give the students the same questions on the test that they used in practice — it will obviously bias the results.

A test set is a third data set. This is a highly reserved data set - it has never been used by the models for updating weights or biases nor has it been used for validation, as a matter of fact, it has been kept hidden from the model builders themselves. It is used solely to evaluate the model at the end of your efforts. You will want to pick a metric that is useful to your business need and requirement set to ensure that the model will meet all your expectations. This step, of holding back a test data set is absolutely necessary for a successful Insights as a Service program.

Case Study

We were contracted by an entertainment content provider to assist them modernize their business intelligence area, this included creating a AI/ML group. The wanted to move quickly, get quick wins and influence the business to show the promise of ML. We proposed an Insights as a Service model where they would pay for useful insights and not be charged for unsuccessful attempts. We implemented a vetting process to evaluate candidate ideas for projects, reviewed those candidates against financial, technical and business criteria to then rank them to be added to the backlog accordingly. This process then fed the ML project lifecycle (discussed above) for eventual deployment. Simultaneously we were evaluating potential partners who fit the criteria we defined - criteria based on the characteristics outlined earlier. The chosen provider was eager to prove that they had the experience and ability to work with the client and was amenable to a financial approach based on success criteria that was a win/win for both parties.

Choosing the first project was a difficult decision. We wanted it to be impactful, in the companies primary business area and not too big or unwieldy. We settled on the somewhat risky decision to focus our effort on churn analysis and how to augment the current churn models with new data sources and model updates that would recommend changes to customer offers based on this new information. Churn and how to minimize it with hyper-targeted customer offers was a key measure for success in the industry so any changes to the churn model were scrutinized very carefully. In the end, our MVP/POC was successful, churn was significantly lowered and we did it with a lower overall spend on customer offers.

The above project proved successful, we implemented subsequent project for further evaluation and we eventually built up the program to run at enterprise scale.

Whether augmenting an existing practice or just starting an MVP insights as a service should be considered as part of your business intelligence or machine learning strategy.

From Domain Driven Design to Microservices

As many of you may recall, the software design and architecture style known as service-oriented architecture (SOA) emerged in the mid 1990’s. Since then, we have discovered better ways to build systems, including advances in cloud-based virtualization, continuous integration and delivery, and microservices. In the process, these technologies have made SOA and all the associated benefits a reality. 

In March, I published the first post of this three-part blog series – How to Introduce Microservices in a Legacy Environment – explaining how microservices can be introduced into a large organization with well-established legacy systems. In this post, I will cover domain-driven design (DDD) and how this development philosophy can be used to represent the real world in code while being well-suited to a microservices implementation.

Domain-driven design 

 Cohesion is an early tenet of software design and refers to the degree of functional relatedness that exists inside a module or class. In this context, cohesion was first described in the late 1970’s by Tom DeMarco and has come to mean grouping and keeping together those things that change for the same reasons and separate the functionality that changes for different reasons.

DDD provides a method to facilitate the development of highly cohesive systems through bounded contexts. Microservices is an implementation approach that encourages you to focus your service boundaries on the business domain boundaries. DDD and microservices can be used together as you move your organization to a service-oriented design and reap the benefits of continuous integration and delivery.

 The seminal work in DDD was defined in a 2003 book by Eric Evans called Domain-Driven Design: Tackling Complexity in the Heart of Software. The overarching philosophy of DDD is to use the notion of bounded contexts which form protective layers around models that define the business domain. Bounded contexts are analogous to departments in a company – the legal department has certain specific responsibilities (contexts) that are different than the IT department and those responsibilities are enforced by rules (boundaries) for interaction and obtaining services from the departments.  

 This is the same for bounded contexts that we model using DDD. To facilitate a common understanding of the problem domain and translate that domain knowledge into a computer system the business and technical team must develop a common language. In DDD this common language is called the ubiquitous language (UL). As the technical staff develops their models and code they use the UL to decrease the risk of misunderstanding between the business analysts and the engineering staff as the project progresses.

This also serves to provide an additional layer of documentation of the systems and enhances the organization’s understanding of how a system was designed and intended to work. The analysis models that are used to understand and define the domain are tied to the code models that are used to create software by the UL.

Other key principles of DDD include:

• Iterative creation of the analysis and code models. As the team learns more about the domain they iterate on their analysis and code models, keeping both in sync. DDD does not specify tools, databases or languages, but I have used UML (universal modeling language) to create analysis models and my code or implementation models were done in C++ and Java.

• Collaboration of the business and technical teams requires close, face-to-face collaboration to create the relevant models. This is a heavy commitment for all parties to develop the UL, use it to define the domain, iterate through the definition of the domain, and focus on the problem instead of jumping directly to a solution.

• Focus on the core domains – the core domains are those domains that will make the product a success. It is a core domain if it is absolutely essential to the success of the business. You should be asking yourselves how this domain increases revenue, cut costs, or increase efficiency, and why and how this domain is critical to the business.

 The problem you are solving must be substantial. There is no use in implementing DDD for problems that are insignificant, won’t move the needle for the business or are better solved with a COTS (commercial off-the-shelf) solution.

After you have begun to understand the business problem and developed models to define it you will have to think about how to integrate bounded contexts. In their book Enterprise Integration Patterns (Addison Wesley Signature Series) Gregor Hohpe and Bobby Woolf define four integration styles: file transfer, shared database, remote procedure invocation, and messaging. In most applications of substantial size and for reasons of cohesion your DDD and technical team will most likely settle on remote procedure invocation and/or messaging for integration.

From domain-driven design to microservices, pairing these two approaches to solve large and complex problems makes good business sense.

The third and final post in this series is an overview of microservices tooling and can be found here.

Entrepreneurs and Innovation

“Today, it is truer than ever that basic research is the pacemaker of technological progress”. This sentence was written in 1945 by Vannevar Bush in A Report to the President entitled Science the Endless Frontier. Bush lays out an argument for why and how the US Government can foster research activities by public and private institutions to maintain and enhance its competitive position. Bush did not want to conduct research just for the pursuit of knowledge he knew that it would lead to new products, practical usages and bolster our competitive position on the world stage — “New products and new processes do not appear full grown. They are founded on new principles and new conceptions, which in turn are painstakingly developed by researching the purest reals of science”. I read this report while I was in grad school and thinking about working in the public or private sector and I had this in mind while I was reading Mariana Mazzucato’s book The Entrepreneurial State.

Mazzucato reminds us of the importance of Vannevar Bush’s idea of government investment in R&D but she pushes beyond the notion of the government as a behind the scenes deep pocketed investor funding basic research but she advocates for governments to be recognized for their risk taking entrepreneurial market making activities they engage in. She provides some very interesting and compelling examples from domains such computing, Pharma, bio-tech, and clean energy to demonstrate that “innovations” from the private sector were really built on top of the hard work and investments done by and/or funded by the State, Mazzucato takes nothing away from the success of Apple but spends a chapter discussing the iPhone and how it was built based on technologies pioneered by the US government - see diagram below from The Entrepreneurial State.

Taking nothing away from Apple’s technical, design and business success Mazzucato demonstrate the State’s role in the critical underpinnings that make the iPhone possible.

The US government in particular seems to suffer from an inferiority complex and thus has a marketing problem. The State has ceded the narrative to others and allowed the tax payers to forget how some major technologies were developed. Mazzucato goes on to discuss fiscal/tax policies that could allow the government to become more of an active investor in technologies it develops to fund the next ventures and recoup some expenses and who knows, maybe have some money left over. She does have her critics for sure, but she also has supports on both ends of the political spectrum. It is definitely something to think about.

I have started a number of technology businesses over the years and must admit I continue to love the idea of the lone genius inventor out to change the world … maybe we have been taking too much credit?

Introducing Microservices into a Legacy Environment.

While currently no consensus exists on how to define microservices it’s generally agreed that they are an architectural pattern that is composed of loosely coupled, autonomous, fine-grained services that are independently deployable and communicate using a lightweight mechanism such as HTTP/REST.  Now is the time for companies -- particularly enterprises that need to make frequent changes to their systems and where time to market is paramount -- to be investigating how best to introduce microservices in their legacy environments if they expect to realize a digital transformation that drives tangible business results.

The benefits and potential hurdles associated with adopting microservices are well documented. On the plus side, the modular and independent nature of microservices enables improvements in efficiency, scalability, speed and flexibility. Detractors, however, frequently point to management and security challenges, especially when they pertain to customer-facing applications and services. 

Like virtually all technology decisions, it’s critical to balance risk with reward and, when it comes to microservices, embracing an evolutionary approach and process. After all, lessons can be learned from both success and failure, and the same is true for implementing microservices that can increase product and service quality, ensure systems are more resilient and secure, and drive revenue growth. In the first of this three-part series, I’m going to explain how business and technology leaders can smoothly and successfully introduce microservices in a legacy environment.

It’s all about the monkey

A key requirement of microservices design is to focus service boundaries around application business boundaries. A keen awareness and understanding of service and business boundaries helps right-size services and keeps technology professionals focused on doing one thing and doing it very well. 

In my experience, I’m finding that the larger the organization the greater value microservices architecture can deliver, but only if executed in a systematic, enterprise-wide fashion. Fortune 500 organizations tend to have a significant proliferation of legacy technologies and should strive to simplify deployment, along with applying continuous integration and delivery of microservices. All too often, enterprises focus their efforts on buying tools, implementing a small proof-of-concept or other “quick wins” that likely aren’t the most effective place to initiate microservices strategies.

Astro Teller, the “Captain of Google Moonshots” has a humorous anecdote about where to begin when solving a large and complex problem and advocates that companies should avoid allocating all of their resources on the easy stuff and instead start by addressing the hard problems; he calls it “tackling the monkey first.”  The monkey, when deploying microservices in a large, established environment, is understanding and decomposing the legacy systems.

Decompose the legacy environment by identifying seams

In the second part of this series I’ll cover domain-driven design (DDD), but for now it’s important to understand two concepts found in DDD: bounded contexts and domain models. 

Any problem domain is composed of a number of bounded contexts with models sitting inside them. The bounded contexts provide a level of protection and isolation of the models. In addition, the bounded context provides an interface to the model and controls what information is shared with other bounded contexts. For example, in an e-commerce application some of the bounded contexts may be ordering, pricing, or promotions.   

Years ago I enjoyed reading “Working Effectively with Legacy Code” by Michael Feathers. In his book, the author presented the idea of a seam as a way to identify portions of code that can be modified without affecting the rest of the code base. This notion of seams can be extended as a method to divide a monolithic system into bounded contexts from which services can be quickly and seamlessly created.  

Uncovering seams in applications and building bounded contexts is an important first step in breaking down the monolith. Seam identification can be accomplished by reviewing the current code base, interviewing domain experts, and understanding the organizational structure. A few suggestions:

• Review the current code base. When reviewing the current code base and any artifacts it’s critical to realize that this is only a starting point. The code is often redundant and difficult to understand. 

• Interview domain experts. This is a key step to learning where the seams are and identifying bounded contexts. Having domain experts that understand what the business should be doing not just what the system currently does is critically important.

• Understand the organizational structure – Often, organizational structure will provide clues to where the seams can be found.

Once bounded contexts are identified, along with the programming language and environment that support them, creating packages and sub-packages that contain these bounded contexts should closely follow. This approach will afford a careful analysis of package usage and dependencies, which are paramount to fully and quickly understanding and ensuring that testing and instrumenting code is being done properly.  In addition, there are some standard design patterns that should be followed:

•  Open Host Pattern - Exposing legacy systems via JSON/HTTP service.  Here the isolated legacy system is exposed through an API that returns JSON.

• Anti-Corruption Layer (ACL) Pattern – A translation layer or sometimes called bridge layer is built between the legacy environment and the microservices code.  This pattern can be effective for short durations but it can be costly to maintain over time. We have also called this layer “scaffolding” it’s needed during the transition but will be taken down after the job is done.

 That’s how microservices should be introduced in a legacy environment.

Read the second post in this series here