Forecasting the Future: AI and ML Reshape Artisanal Sales

Charting the Course of Dairy Sales with Advanced Analytical Solutions.

Company Description

Our client, a renowned artisan in the dairy industry, has established a legacy of crafting exceptional Italian cheeses. Their expertise extends from traditional varieties to contemporary favorites, catering to top-tier pizzerias and Italian eateries nationwide. Beyond their famed cheese products, they also innovate in specialty whey ingredients, enhancing the nutritional and textural qualities of various food products. At the core of their operations lies a commitment to sustainable practices, community engagement, and environmental stewardship, making them a distinguished example of excellence and responsible business in the dairy sector.

Challenge

Problem

The client faced the challenge of modernizing their sales forecasting process. Their existing system, heavily reliant on traditional methods and manual inputs, was not equipped to efficiently handle the complex variables impacting sales. This included adjusting for unique market events and integrating intricate market intelligence data. The need was for a sophisticated, yet user-friendly forecasting model that could accurately predict future sales, taking into account a myriad of factors such as seasonal trends, special market events, and evolving consumer preferences.

Project Goals

· Develop a web-based application to automate the sales forecasting process, replacing the existing manual Excel-based system.

· Integrate advanced statistical models capable of handling complex forecasting requirements, including adjustments for ‘black swan’ events and other non-historical factors.

· Ensure the new system is flexible enough to incorporate various levels of market intelligence assumptions (MIA) and capable of providing forecasts at multiple hierarchical levels (e.g., style, major group, minor group, SKU).

· Achieve a user-friendly interface that allows seamless management of forecasting projects, making it accessible for various stakeholders with different technical backgrounds.

· Implement a system that not only enhances the accuracy of sales forecasts but also supports the client’s decision-making process by providing insightful data analytics.

Solution

In response to the complex needs of the client, a robust and intuitive web-based application was developed to revolutionize their sales forecasting process. This solution was intricately designed to encompass the full spectrum of forecasting requirements, from data ingestion and processing to advanced analytics and user interaction. Here’s a detailed breakdown of the key components of the solution:

· Web-Based Application Development: A sophisticated web application was developed to replace the existing manual, Excel-based system. This platform was engineered to handle large datasets, complex calculations, and provide comprehensive forecasting capabilities.

· Data Processing and Integration: The application was designed to automatically pull historical sales data from the client’s data warehouse. It seamlessly integrates this data with user-inputted market intelligence assumptions (MIA), ensuring that all relevant factors influencing sales are considered.

· Advanced Forecasting Models: Utilizing a suite of time-series forecasting models (SES, HWES, MA, ARIMA, VARMA), the application can generate accurate and detailed forecasts. These models were selected for their ability to handle different data patterns and forecasting needs. They can forecast at various hierarchical levels, from overall trends down to individual SKU levels.

· Adjustments for Market Events: A key feature of the solution is its ability to adjust historical data for significant market events, such as the COVID-19 pandemic. This ensures that forecasts are not skewed by extraordinary, non-recurring events.

· Integration of Market Intelligence Assumptions: The application allows users to input specific MIA at different levels (e.g., All, Style, Major Group, Minor Group, SKU), ensuring that forecasts are tailored to reflect anticipated market changes and trends.

· User Interface Design: The interface was crafted to be intuitive and user-friendly, catering to users with varying levels of technical expertise. This design approach ensures that the complex process of data input, model selection, and result interpretation is straightforward and accessible.

Innovations

· Custom Forecasting Algorithms: The development of bespoke algorithms tailored specifically to the client’s unique forecasting needs stands out as a key innovation. These algorithms are designed to handle complex data patterns and integrate various forecasting models seamlessly.

· Dynamic Market Intelligence Integration: A standout feature of the solution is its ability to dynamically incorporate Market Intelligence Assumptions (MIA) into the forecasting models. This innovation allows the client to input and adjust for non-historical data and trends, ensuring that the forecasts are not only data-driven but also contextually relevant.

· Adaptive Forecasting Models: The use of a range of time-series models like SES, HWES, MA, ARIMA, and VARMA, which are automatically selected based on the data characteristics, represents a significant advancement. This adaptability ensures optimal accuracy and relevance of the forecasts for different product & group levels as well as market conditions.

· Interactive Data Visualization and Reporting: The application includes advanced data visualization tools, enabling users to interact with the data and forecasts in a more meaningful way. This feature aids in better understanding and decision-making based on the forecast outputs.

· User-Centric Design Approach: The design and development of the application with a focus on user experience is an innovation in itself. It makes complex data processing and forecast generation accessible to a broader range of users, irrespective of their technical expertise.

High-Level Architecture

Components

· Data Integration and Management: Handles data flow from the data warehouse, ensuring efficient data processing and storage.

· Forecasting Engine: Implements various statistical models for accurate forecasting.

· User Interface (UI): Provides an intuitive platform for interaction with forecasting tools and data management.

· Security Layer: Ensures application security through user authentication and data protection.

· Market Intelligence Assumption (MIA) Handler: Manages the input and integration of MIAs into forecasts.

· Reporting and Visualization: Offers tools for detailed reporting and interactive data visualization.

· Administrative Dashboard: Enables user role management and application monitoring.

· Notification System: Keeps users updated on system status and important forecasting alerts.

Interactions

· The Data Integration Module feeds historical sales and MIA data into the Forecasting Engine for processing.

· The Forecasting Engine analyzes this data and outputs forecasts, which are then visualized and presented through the UI Module.

· The Security and Authentication Module interacts with all other modules to ensure data integrity and secure access.

· The Reporting and Visualization Module pulls processed data from the Forecasting Engine to create insightful reports and visual aids.

Core Technologies

· Backend: .NET 6 Web API, Azure Functions for data modeling

· Frontend: React for web applications

· Database: Microsoft SQL Server

· Deployment: Manual

· API Integration: REST API

· Authentication & Security: Custom JWT Authentication

Process

Team

The project was brought to fruition by a multidisciplinary team comprising data scientists, software developers, project managers, and other key stakeholders. The data scientists played a crucial role in statistical modeling and data analysis, while the developers focused on building the web application and integrating the models into it. Project managers oversaw the workflow, ensuring timely delivery and alignment with project goals.

General Development

Utilizing an agile methodology, the development process was organized into iterations, each one producing an increment of the product. Regular meetings and reviews ensured the product met the client’s needs and expectations.

Testing

The testing strategy encompassed various stages, including unit testing, integration testing, and user acceptance testing. Data scientists were involved in testing the accuracy and performance of the forecasting models, while developers focused on the application’s functionality and user interface.

Rigorous testing ensured the reliability of the forecasts and the overall performance of the web application. Feedback loops were established to gather insights from users and stakeholders, further refining the application.

Results

Results Overview

· Model Performance and Diversity: The integration of a variety of models, including Simple Exponential Smoothing, Holt Winters’ Exponential Smoothing, Moving Average, VARMA, and ARIMA, met the clients expectations for accuracy of forecasts. The application’s ability to choose the optimal model based on specific product or group data led to reliable predictions.

· Efficiency in Forecasting: The forecasting process saw a remarkable increase in efficiency. What previously took around 45 minutes in Excel for processing a single model has now been dramatically streamlined. Running all 10 models across the full range of products and groups completes in just 10-15 minutes.

· Operational Impact: Feedback from management indicated a substantial improvement in the forecasting process’s effectiveness. The enhanced speed and accuracy of the forecasting models positively impacted decision-making processes, making them more data-driven and reliable.