Proof of value: predictive analytics and trade investment
Over the past three months we've been collaborating with Alistair Ferag to design and build a proof of value that demonstrated bleeding-edge data science techniques (“machine learning”) applied to selected consumer business challenges:
- Can we model a mass of sales and promotional data and use the result for prediction and optimisation?
- Which promotional tactics most influence the sales of a given product set and to what degree?
- What is the impact on product sales when other products are promoted and to what degree?
A set of commercial data was obtained from a client within the consumer business industry, which ensured the results had real business value.
“Machine learning” (ML) is a term that is bandied around a lot. It is something that I have worked to understand as it moves to the forefront of the newest enterprise solutions. In our collaboration I challenged Alistair to build a solution that utilised innovative ML algorithms to derive insight from a mass of customer data. In this case we used electronic “point of sale” (EPOS) data coupled with market intelligence on trade promotions.
“Machine learning” = learning from data
In essence, machine learning comprises a wide range of techniques that aim to enable a computer programme to learn from data with little or no input. The algorithms look for patterns in data and use the results to refine their understanding of the data to improve further analysis.
Machine learning cannot tell us why a pattern exists (that’s real science!), but it can identify complex patterns in a vast ocean of data and use these patterns to make predictions.
Blowing the lid off the black box with SAP HANA
I’m fed up with “black box” solutions. Part of the remit of the project was to build something I could understand and thus have confidence in how it worked. We agreed Alistair would use the ‘R’ statistical programming language, since the code could easily be integrated with SAP HANA for future enterprise applications. The result was an elegant design* that worked as follows:
- Data load and cleansing
- Testing of a set of ML models to determine best fit (learning from data)
- Use of modelling results to analyse promotional tactic effectiveness vs. product across multiple categories
- Use of modelling results to analyse effect of promotions on products on other product across multiple categories (cannibalisation).
Some example results for item 3 on a bar chart and item 4 are shown below on a “chord diagram”. The chord diagram shows that promotions on ‘product 1’ had a far stronger influence on sales of ‘product 3’ than those of ‘product 3’ on ‘product 1’. This representation of the data may not be to everyone’s tastes, but it demonstrates an innovative way in which this kind of data can be visualised.
Item 3: relative impacts of promotional tactics on each product in dataset
Item 4: relative impacts of all promotional tactics by product on each other product in dataset
Rise of the machines
Machine learning is an area that you are guaranteed to hear a lot about in the coming months and years, and it’s certainly something to consider when future-proofing an analytics capability. Ignoring for the moment Stephen Hawking’s doom-saying about a machine learning robot rebellion, this technology is almost certainly going to make a positive and profound impact on enterprise solutions very soon. We really showed that it was possible to do something innovative and of genuine business benefit in an extremely short space of time.
* If you would like more information on Alistair’s work, a copy of his Master’s thesis can be provided on request.