Insights as a Service - Using External Partners for AI/ML
Whether you are just starting to think about implementing an AI practice using machine learning (ML) or want to expand your existing ML operations, insights as a service is an alternative that corporate CxO’s should consider. Insights as a service is a methodology that allows business units to pay by the drink for data insights related to specific high value business problems.
Definition of Insights as a Service
Insights as a service is a methodology to leverage an external partner for some of your ML requirements. The partner will provide tools, ML models, compute resources, trained data scientists and domain experts, and most importantly access to a trove of external data for you to leverage during your analysis. The buyer pays by the drink for this service, meaning you pay for the insights you receive. We have designed deals between parties where the buyer only pays for actionable results, if there is a dry hole i.e. no actionable information then you don’t pay for that effort. There are many other ways to set up contracts to incentivize the partner to produce what you ultimately want - results that move the needle for the business. Further on in this post I have a case study showing some of the details of a successful implementation.
Why use Insights as a Service
It’s not an entirely hands off process when you invest in insights as a service I will cover the buyer’s responsibilities in the implementation sections below. You should consider using Insights as a Service for one reason - speed to market, getting actionable insights sooner. If you choose your vendor/partner wisely you will be able to get insights sooner because that partner is bringing their own data including licenses for external data, pre-build and tested ML models to be refined for you, and they should have trained staff ready to deploy to your project. For companies just getting started with AI I have seen implementations that typically take 8 to 12 months from start to finish get cut to 2 or 3 months. Even companies with a mature AI/ML practice an insights as a service provider can augment your internal team providing additional expertise, new data sources and staff that can cut your project backlog providing the business areas with valuable insights for decision making.
Implementing Insights as a Service
Insights as a Service programs that I have put together generally fall into one of two groups; the first is a mature organization that wants to add additional throughput into their organization to meet increasing demand or to provide some expertise that they lack. The more challenging case is the second situation where the organization wants to get started with ML and is looking for a way to show real progress in a reasonable amount of time. It is this second scenario that I am going to focus on here.
Getting Started in AI
The process of starting an AI program is the same as introducing any other significant and new technology into the enterprise with the difference being that AI will bring greater opportunities and implications for your business. AI is considered by many to be a general-purpose technology which are technologies that have profound and lasting changes that ripple across the globe spawning new industries, ways of working and changing how people live. For example, electricity was such a technology - it significantly changed how people lived and worked. New products were created and it spurred explosive growth and disruption. Introducing AI and machine learning into the organization takes a significant amount of up front work.
For this post I will approach AI/ML introduction from the perspective of the Chief Technology Office (CTO) as they are a likely advocate for the technology. As a senior leader, the CTO must take on the role of educator of her peers at the senior leadership level. Helping them understand in business terms what AI/ML is and is not, the kinds of problems that it can be best applied to, what kind of returns and new opportunities could open up because of the new technology. At the heart of doing the hard work and sometimes long process of this education is good old fashion marketing. In the past I have assisted CTO’s with this effort by leading tours and holding Q and A’s with other companies who have successfully and recently started this journey, brought vendors in for short “art of the possible” meetings, engaged speakers (technical and business) to discuss industry trends, email short pieces that explain AI/ML. I would recommend reading Andrew Ng AI Playbook which I have followed in the past for successful POC/MVP projects.
Thinking “stractically”; not too strategic not overly tactical
There is a trade off between strategy and tactical — in the beginning phase I recommend a happy medium which we call “stractical”. You want to think strategically but don’t get too ivory tower, you want to be tactical but not short sighted. The CTO needs to help senior leadership balance these two competing interests. We help customers think about digital transformations and if there ever was a technology that will drive transformation it’s AI/ML. Be sure to think about the strategic direction you want to pursue, for example; how will this new and powerful technology allow me to service my current customers better and open new opportunities, what are my competitors doing in the space, who are my competitors now and does AI change the competitive landscape, how will I avail myself of new data and new sources of data, what new business innovations does this open up for us, and how do I adapt to my customers changing needs and values, how will I build an adaptive value proposition. As you, the CTO think through these areas you will simultaneously be thinking of which project(s) you should choose as a pilot aka proof of concepts or as I prefer to think of it, as a minimal viable product (MVP).
Conducting the training and education sessions while guiding the conversation between a strategic and tactical plan is necessary for success in your organization. You will uncover opportunities and build trust and gain supporters, which you will need to be successful.
Choosing the Right Project
Choosing the right project for your first implementation is important as you want to maximize your chances for success as well as your impact on the business. Some project characteristics to think about:
Transparency of the model: If you need to understand why your models made a certain prediction or why they didn’t make a different prediction you most likely won’t be able to get this information using current ML algorithms - for the most part ML algorithms in use today are black boxes even to the inventors.
Data availability: Is the data you will need available, accurate, usable, and sizable? Do you need and can you get rights to external data sources?
Industry related project: Don’t pick a project that is unrelated to the business you are in. If you work for a medical device company don’t pick a project that uses ML to make staffing/hiring decisions more quickly. Look for a project that moves the needle for the business domain you are in so when you are “selling” the ideas and techniques demonstrated from a successful implementation it’s readily apparent how it applies to the business at hand.
Too large vs trivial: As you think about your first foray into machine learning you will need to balance the trade off between a project that is too large vs one that is just trivial. Look for something that will produce a relatively quick win and demonstrate to senior leadership that this is a direction that is worth investing in for future growth. Ideally you will look for a project that will positively impact revenue growth.
Choosing the Right Partner
This post is focused on Insights as a Service, the crux of which is partnering with a company that can augment your organization’s skills and tools so you can quickly maximize your chances of success in implementing a new technology. With machine learning you need to look for a company that fits culturally, brings strengths where you have deficits and vice versa, has relevant business domain experience, brings pre-trained models, has labeled data from external sources along with the rights to use that data, a deep bench of data scientists, software engineers, and statisticians, and brings the compute power and tools necessary to run a machine learning project.
A good partner will be able to assist you with all phases in the lifecycle of a ML project. In a coming post I will discuss the ML project lifecycle in detail but for know understand that this is a multi-step process that begins with business goals and direction, data collection and analysis, feature engineering, model development, training and maintenance, through model deployment. When you are choosing a partner it is incumbent on you to evaluate them across this lifecycle to ensure that they have the wherewithal to help you in this journey.
Be aware of your financial goals as you choose a ML partner and how you wish to incentivize them for your success. Seek out a partner who is open to new more modern financial arrangements such as gain share or risk/reward share models for success.
Reserved Data: Test Set
When using an Insights as a Service partner the most important item to remember is to hold data back from your partner to use as a final evaluation after they have completed their model building, training and validation testing.
Data for ML is split into at least two sets - a training set and a validation set. The training set is used for training your models. The validation set is used to evaluate the the model’s performance by the ML developers. In theory the model is being validated on data it has never seen before so if it is making accurate predictions on the data it is because it has learned the characteristics of the data not because it has seen the data before. It is very similar to studying for a test in college - if you study by reviewing practice problems it is generally not a good idea to give the students the same questions on the test that they used in practice — it will obviously bias the results.
A test set is a third data set. This is a highly reserved data set - it has never been used by the models for updating weights or biases nor has it been used for validation, as a matter of fact, it has been kept hidden from the model builders themselves. It is used solely to evaluate the model at the end of your efforts. You will want to pick a metric that is useful to your business need and requirement set to ensure that the model will meet all your expectations. This step, of holding back a test data set is absolutely necessary for a successful Insights as a Service program.
Case Study
We were contracted by an entertainment content provider to assist them modernize their business intelligence area, this included creating a AI/ML group. The wanted to move quickly, get quick wins and influence the business to show the promise of ML. We proposed an Insights as a Service model where they would pay for useful insights and not be charged for unsuccessful attempts. We implemented a vetting process to evaluate candidate ideas for projects, reviewed those candidates against financial, technical and business criteria to then rank them to be added to the backlog accordingly. This process then fed the ML project lifecycle (discussed above) for eventual deployment. Simultaneously we were evaluating potential partners who fit the criteria we defined - criteria based on the characteristics outlined earlier. The chosen provider was eager to prove that they had the experience and ability to work with the client and was amenable to a financial approach based on success criteria that was a win/win for both parties.
Choosing the first project was a difficult decision. We wanted it to be impactful, in the companies primary business area and not too big or unwieldy. We settled on the somewhat risky decision to focus our effort on churn analysis and how to augment the current churn models with new data sources and model updates that would recommend changes to customer offers based on this new information. Churn and how to minimize it with hyper-targeted customer offers was a key measure for success in the industry so any changes to the churn model were scrutinized very carefully. In the end, our MVP/POC was successful, churn was significantly lowered and we did it with a lower overall spend on customer offers.
The above project proved successful, we implemented subsequent project for further evaluation and we eventually built up the program to run at enterprise scale.
Whether augmenting an existing practice or just starting an MVP insights as a service should be considered as part of your business intelligence or machine learning strategy.