How Retailers Moved from Rules to Artificial Intelligence

Artificial Intelligence (AI) is a broad category that includes several different types of analysis or methods of creating a system in which the program makes a decision based on the available information. Self-driving cars are a glamorous example of a system that collects data from its environment, compares them to objectives, and decides what action will produce the best outcome. AI in retail is not so glamorous. It does not involve shiny new cars or crash test dummies. Instead, it involves behind-the-scenes analysis of transactions that happen so fast, it can render accurate decisions in milliseconds.

For us to understand the “how” of this evolution in retail analytics, we first need to define some of the common terms.

Predictive models and machine learning

Predictive models and machine learning are shown in the diagram above as almost completely overlapping circles. The primary difference between the two is machine learning originated in computer science, whereas predictive modeling was developed in the field of statistics. The two fields have essentially merged and now share a highly overlapping definition. Both are concerned with using data to predict or model outcomes, and both are tools used in artificial intelligence.

Deep learning

To model relationships, a mathematical equation is defined that has variables that are known and parameters that are unknown. The unknown parameters are estimated using a mathematical optimization process to best fit the data. Typically, variables used in machine learning models are defined by a person as candidate variables for a prediction and are derived from a raw set of data. For example, the raw data may be something like the point-of-sale data for a retailer and an example of a derived variable would be the number of line voids in the past three months. Deep learning is a process that uses highly complex model structures to learn patterns in the raw data without any human intervention, thus eliminating the need for derived variables.

Dynamic learning

A static model is fit to a static pattern in data. The predictions from a static model can change as the data it is presented with changes, but if the model is presented with the same exact data, it will produce the same exact answer. By contrast, a dynamic model’s prediction can change from day to day even if it is presented with the same information. In dynamic learning, models are re-fit or updated dynamically as new data presents itself. This approach is useful when a system is continually bombarded with additional information and the relationships in the data are dynamic and the model itself needs to change to adapt.

Rules became data mining queries

Until the 1990s, a retailer’s approach to analytics was manual comparison and calculation. It was a rules-based system. In the late 1990s and early 2000s, retailers began to use data mining which could bombard each piece of data with dozens of queries and then display the exceptions. One use of these data mining queries was exception based reporting (EBR), a technique the credit card industry had moved to a decade before.

Today, most Loss Prevention departments use traditional EBR tools to monitor their employees’ behaviors. EBR is usually viewed as a query run on point-of-sale (POS) data that flags suspicious activities; such as, who has the most line voids? Who processes a large number of returns whenever the boss is on lunch break? EBR greatly improved productivity, but it also highlighted every exception. LP investigators had to learn to discern legitimate behavior or honest mistakes quickly. There was so much to review.

The move to predicative models and machine learning

Recently, retail began to use the analytical techniques pioneered in other industries and started to incorporate predictive models. Through machine learning, software could infer what would happen next.

When used to identify fraud, the software solution learned to highlight the behaviors that usually indicated fraud that the retailer would pursue. With a machine learning model, the top ranked exceptions will have a much higher rate of success than those that are generated from traditional EBR. Over time, an approach with a higher success rate will make Loss Prevention departments far more efficient and will decrease employee fraud rates.

A final objective of an AI-based employee deviance detection system is a feedback loop from the investigations back into the models. During an investigation, the investigator will perform many actions that can be captured in a system and later used to improve system accuracy. For example, the investigator will click on related transactions and individuals, will open a case, will abandon a case or a transaction, and so on. The captured usage activity can be converted into information that a machine learning model can utilize to improve the models.

Summary

The move from rules to AI did not happen overnight. The technology, proven in other industries, has been gradually applied to retail over the course of the last twenty years. Retailers who embrace machine learning and other forms of AI, see gains in overall performance as well as higher profits.