Machine Learning Consulting: Does Your Business Really Need It?

You’re probably familiar with the hype surrounding the business value of applied data science and machine learning. You’ve heard things like, "your data can become your competitive edge" and "your data can be turned into actionable insights." Well, that's true, but only if you're well-prepared with actual data in hand and if you learn the basics of typical project flow.

These are some of the questions to be answered. 

Here is a story of two companies operating across different industries that applied machine learning (ML) algorithms to their business data and got valuable insights that resulted in new product development.

A SaaS ERP solution provides for retailers in the U.S. Having been gathering warehouse and sales statistics for years, the owner and a machine learning consultant explored the data and what value it could bring. The consultant helped to set and achieve proper business goals by adding Data Science and Machine Learning features. The data was turned into a source of sales forecasting.

Another case is about a U.S. insurance company needed to secure access to sensitive healthcare data with the help of custom-built software. Since high reliability was needed, the company was advised to enable multimodal biometric authentication using voice and facial recognition. Working with a machine learning consulting company resulted in additional security measures, such as deep learning-based face anti-spoofing, OCR for driver’s license recognition, and an NLP-driven question generation module. The enhancements eventually turned the product into a cutting edge enterprise verification-as-a-service (EVaaS) solution.

Preparedness is half the battle. Here are several stages of a successful machine learning consulting and development process.

In the SaaS retail ERP case, statistics were gathered over several years, so there was plenty of data to work with. The first step was to explore and extract the data. Then, the quality must be evaluated and the structure identified. Lastly, a consultant has to spend some time to understand what this data means from a business perspective.

However, sometimes there is no existing data. In such cases, a machine learning consultant has to explore the client's aims and main use cases. Then, they must come up with ideas and technical roadmap. For example: what are the options to gather the data, and are there any existing data sets on the market? Perhaps it’s possible to use any of already pre-trained models — it would significantly reduce the amount of data, time, and effort needed. In some cases, even synthetic data could be generated.

For the case of the insurance company and securing healthcare data, already pre-trained models were used for voice and facial recognition. Then, its accuracy was adjusted based on additionally gathered data.

First, a machine learning consultant has to understand the business landscape and how it all works for the client and their customers. The consultant must identify the problems the client is aiming to solve and what value this serves to the client’s customers.

As a result, you can expect a consultant to have a good understanding of the business and get a clear roadmap of where exactly Machine Learning could be applied and what is needed for the implementation. 

It’s true that in most cases at this stage there’s no way to predict what will be the end result in terms of quality and accuracy of a certain machine learning model. It can only be done with a series of experiments along the road of development. 

As for the timeline for exploration of business inputs and aims - it depends on the scale and complexity of each business. However, in most cases, it should not take more than a week. Clients can even expect to get this stage done free of charge as part of the pre-contract stage.

Simple business goals such as "increase sales by 10%" must be transformed into something tangible and manageable from a technology perspective. In other words, something that could be influenced directly by technologies. However, we need to know what the metrics are to succeed.

Success metrics can be a combination of your business metric (e.g. SCOR, FIFO, LIFO) and ML team’s math metric such as an absolute percentage error.

The next step is setting objectives that could be achieved with the help of technologies. For example, we might need to build a model that predicts demand for 1 week with, say, 80% accuracy, or predict how many goods of a particular type will be purchased next month based on available statistics over the past 6 months. Down the road, those metrics will be adjusted and accuracy will be improved, but it’s crucial to start with something. 

In the retail case, the first focus was on optimizing warehouse supply and minimizing purchases. Having studied sales statistics, machine learning consultants found they could also predict how many people would purchase different types of goods. Thus, they were able to solve several related tasks with a single dataset and get desirable accuracy. 

Every task needs its own hypothesis. Consider it as a roadmap for the consultants to follow and also as a way for business owners to understand the route. Once results are achieved on a certain hypothesis, there’s a choice to be made. If the results are promising, machine learning engineers should focus on improving those results. A lot of effort is placed here.

However, often the first results show that there’s no value in this particular hypothesis with certain data and the approach taken. In this case, either the hypothesis, data, or approach must be changed. This is the most optimal way to run machine learning projects.

How Much Accuracy Is Enough?
Accuracy depends on the data and can’t be predicted without running experiments. Yet, keep in mind that you need to keep it cost-effective according to your business goals. Sometimes the effort to get an additional 5% boost can take more time than the efforts spent on the previous 90% — yet hold no tangible value for the business.

There are different approaches to manage machine learning projects. The one that we often use is a combination of classic Agile principles with the CRISP-DM model for data mining and analysis. A typical cycle takes roughly up to 4 week-long sprints. During the first two sprints — Business & Data Understanding — we generate our first hypothesis and form a baseline to build future models and improve results.

The scheme below presents 2 cycles spanning 8 sprints.

With a hypothesis, it’s possible to make raw data actionable. For this, the following steps are needed:

• Understand, explore and assess data, describing its format and quantity

• Make an exploration report with initial insights and findings

• Verify data quality and completeness

• Construct an actionable dataset for modeling

For the demand forecasting case, the data was examined, cleaned, and prepared to get baseline sales forecasting for the most popular goods. To get meaningful results, machine learning consultants have to understand which data is relevant to the hypothesis and which data should be used for analysis. This means not all of the data should be analyzed at the same time.

In the case of multimodal biometric verification, there was a goal to identify the user by comparing photos from a OCR-processed driver's license and a selfie. For the task of facial recognition, it was fine to use an already pre-trained model to get a baseline. 

However, pictures on the driver’s licenses were often of poor quality. So, most of the datasets with faces were not acceptable. It was crucial to carefully choose each dataset used. In a case like this, it’s very important to collect datasets relevant to the task.

This is probably the second most complicated question. The first one - how much time it will take to create the end-product? 

Before the data could be used for machine learning operations, it has to be cleaned and checked. It often leads to data decreasing. After being verified for relevance to the hypothesis and cleaned, it could be applied to get a baseline result. This is the point where the conclusion satisfies the objectives and metrics can be made. With insufficient data, it’s impossible to pass the modeling stage and the model fails to learn. When we understand that we need more data, we can either collect/generate it or move on to a second hypothesis. 

If the hypothesis fails, we have to go back and start a new cycle with a new hypothesis. This is probably the single most important risk of a machine learning project. Despite this risk, this is the only way that it can function. 

In the case of analyzing retail data, it was vital to understand how certain types of goods were sold. For this, it was needed to know how many were typically purchased during a certain time period, and even the average duration of warehouse storage. 

The more goods covered at a time, the less accuracy that can be achieved. However, it was possible to re-train the model for different types of goods, eventually getting better results for each. To illustrate this, when predicting sales with at least 95% accuracy in a monthly cycle, it was needed to have at least a six-month period of historical data. If the hypothesis passes the test, it means that the model works and can be deployed.

Before using data to train a model, it's good to think about how to test model accuracy. That's why initial datasets often can be randomly divided into two datasets: one for training and one for testing.

When the model has been verified, it is deployed on a test server to work with real data. It is reasonable to make predictions and monitor their accuracy over short periods. You can't have statistics gathered over 6 months and predict something accurate for the next 12 months; 1 month or even 1 week would be more reasonable. Eventually, if the model successfully achieves your business objectives in the test environment, it can be deployed in production.

In the retail case, the demand prediction model was run on a test server and then integrated with the system on the adopter level, using relevant retailer-specific data. This allows the model to generate predictions with an average accuracy of 95%.

It is worth noting that changing business processes and new statistics can make models obsolete over time. However, we can always gather the latest data for more relevant studies. Data science and machine learning tools keep advancing, which makes it easier for developers to build and test models. This, in turn, facilitates integration with your existing software products, enables faster deliveries, and reduces your time to market.

Now you know the typical flow of a machine learning project. Hopefully, this article has helped you understand the role of a machine learning consulting company and the value it can bring to your business: to provide assistance in designing a system from requirements to deployment.