In the mid-1960s, editors at Time Magazine predicted that by 2000, the “machines will be producing so much that everyone in the US will, in effect, be independently wealthy” and our only real problem would be finding new ways to “leisure meaningfully.” Their forecast, as we can all agree, was a bit off.
Forecasting is a practice as old as civilization itself, and there are records of ancient Babylonians charting cloud patterns and keeping astronomical diaries as far back as 650 BC. It’s only natural for humans to be curious (or apprehensive) about the future, and our brains are hardwired to constantly process what’s happening, why it happened, and what’s happening next. For those in the retail industry, that mentality helps keep you ahead of the competition, but nowadays, too many variables can impact outcomes.
Winning the Game of Variables
As difficult as it is to accurately forecast, it can be equally as difficult to select the right forecasting method. In other words, there are many paths that might likely lead to the same wrong place. There are two main methods of forecasting: time series models, and associative models. Retail teams should understand both methods to decide which is appropriate and when.
Time series forecasting is a technique for “predicting future events by analyzing past trends,” but because historical trends are less reliable during times of constant flux, which is an accurate (if mild) way of describing the past few years, its exactness is sometimes limited.
That’s why associative forecasting has gained in popularity of late as it looks at the relationship between two or more variables over time, an independent variable and a dependent one, and hinges on the argument that their relationship to each other is the best predictor of future results.
Even the best Big Box retailers failed at predicting consumer demand in the second half of 2023, despite using advanced analytics and machine learning in their forecasting practices. They expected consumers to keep spending, but this did not happen. Even today there is close to $732B of additional inventory that consumers don’t want to buy across retail.
Many retailers tracked savings rates as a key driver for consumer spending in 2022. This factor remained high throughout much of COVID-19 and resulted in stimulus-driven spending. However high savings in March 2023 was a sign of consumers saving money in anticipation of a recession, which counterintuitively slowed demand. The outcomes of these seemingly opposing economic variables can often be predicted by underlying interactions and causal inferences through associative forecasting.
Apart from these macro variables, there are category-specific unknowns in retail that should be considered. In cosmetics and personal care, mascara sales experienced a sharp uptick during early 2023, while lipstick sales fell amongst female shoppers in response to global mask mandates. Much of these segments need to be treated with pre-pandemic-related demand drivers today instead of immediate history.
In fashion and apparel, we have seen customers moving from more structured styles like jeans to comfortable athleisure—a change that had nothing to do with COVID-19 and rendering forecasting such changes nearly impossible while using historic machine learning or AI models.
GenZ and millennial groups are also adopting unisex-style clothing more openly today. Each customer is looking for a touch of individual preferences, which can be predicted better through associative forecasting because clustering consumer segment-specific attributes and studying how they relate results in more accurate demand predictions.
Developing a More Robust Approach to Associative Forecasting
Associative forecasting has a range of advantages over traditional time series methods because of its inherent mixture of internal and external variables. In retail, internal variables would include all the strategies within a retailer’s sphere of control, such as things like shelf space allocation, shelf utilization, personalized bundling of products, promotions, and channel mix spend optimization. These are the variables they can manipulate to influence consumer behavior. External variables are those factors that drive behavior outside of a retailer’s control, like a global pandemic or recession.
In the past, retailers were somewhat limited by the number of variables they could factor into their forecast. At a certain point, it was inefficient (and costly) to spend time considering all possible scenarios. But now, thanks to advancements in AI/ML technology, more data points mean better opportunities for unique insights and correlations.
Here are some common steps and best practices to consider when applying the associative forecasting technique:
Typical Scenario: Product A sales have dipped; Product B sales have risen.
- Select multiple internal and external variables to better understand the motivations behind the change in behavior. Have customers made the switch from Product A to B due to company actions, or has the product become less necessary due to external factors or a change in trends?
- Google Trends provides access to a largely unfiltered sample of actual search requests made to Google, which can be a good leading indicator of demand. By adding in specific external variables from your product’s point of sale (socioeconomic conditions, regional weather patterns, etc.) along with Google Trends data, into the associative forecasting equation, you can get a better sense of whether this change in demand is an anomaly or part of a larger trend.
- Use advanced clustering techniques to group similar products within a category/sub-category. (Pro Tip: graph databases like Neo4j provide more robust and distinctive clusters.)
- Take an “ensemble approach” that includes bespoke assumptions, interpretations, and objectives across all teams within the organization to foster an inclusive forecasting practice. You can do this by creating an insight platform with a simulator to understand the “what-if” scenarios offered by leaders across teams and take appropriate actions.
The Way Ahead
The lifeblood of the retail business is making products available to customers when they want them. However, traditional forecasting methods have become obsolete. Integrating AI/ML technology with associative forecasting has become a strategic imperative to navigate the uncertainty surrounding the complex retail landscape. This powerful synergy will enable retailers to analyze vast datasets and uncover the intricate web of connections that drive consumer behavior. As we look ahead, it is clear that retailers who embrace associative forecasting will be agile in a volatile market.
