Module: Analysis of extremes¶
Executive summary¶
The module detects extremes in sales on the one hand and orders received from individual customers on the other. The received orders need to be cleaned, as they enter into sales forecast as well.
The basic principle is to identify sales that are so extreme that they are not likely to be repeated – and even if, we do not want them to be our base for sale forecasts and stock levels computations. The reason behind this is that if we were to keep the safety stock at such a level to satisfy the extreme sales, the stock would have to be enormously high.
The module is able to take into account promotions and it also works with seasonality of sales (including moveable feasts).
Functional description¶
Automatic computation of extremes¶
The module cleans sales forecast at three levels of sensitivity: - 1 – the lowest sensitivity, - 2 – medium sensitivity set as default and - 3 – the highest sensitivity. The setting affects two components: cleaning seasonality of a month and adjustment of the error component (value) for a given month.
Sales labeled as promotions are never cleaned.
Extreme sales can be evaluated for each product or for a group of products. When analyzing a group of products, an extreme is only such an extreme that applies to the whole group.
To clean extremes, we use two algorithms. The first one is used for products with a very low number of sales; the second one is used for standard products. It is possible to set the limit for the numbers of sales (that is, to define what is the ‘low number of sales’); it is 10 sales as default. For products with less than 4 sales, extremes are never cleaned.
Algorithm for products with a small number of sales¶
The algorithm uses two principles:
The first principle is just a simple conjecture: the largest sale cannot be x-times bigger than the second largest. The ‘x’ is different for different levels of sensitivity and it can be set.
The second principle is based on the idea that if a sale constitutes a certain percentage of the total numbers of sales, the quantity of the product sold in the sale cannot constitute a significantly higher percentage of the total quantity of the sold product. Suppose, for example, that we have five sales (hence, each sale represents 20% of the total number of sales) and suppose further that in the first sale, we sell 1 item, in the second 3 items, in the third 5 items, in the fourth 10 items, and in the fifth 1000 items (of the same product). The last sale, i.e. the 1,000-item sale, is disproportionate with respect to the other sales, since we sold 98% of the product in a single sale (1000 items of a total 1,019 items sold). The limits for these disproportions are set by sensitivity.
Notice that the last algorithm does not take seasonality of a product into consideration.
Algorithm for products with a higher number of sales¶
The algorithm has two components. The first component identifies months with extremes – and thus different from the average. The second component seeks to identify sales that cause these extremes.
The first component: identifying extreme months
The algorithm analyzes the three components to monthly sales (seasonal, trend and random, i.e. an error). If sales in a given month exceed materiality threshold, the month is treated as an extreme and, thus, cleaned. That a month exceeds the pre-set threshold of significance means that it is too different from the average monthly sales (supplemented by seasonal and trend component).
Each level of sensitivity of adjusting sales has a different way of cleaning: while the high sensitivity can clean seasonal sales more effectively, the low sensitivity has almost no need for cleaning in seasonal sales. The moderate level of sensitivity is the average of low and high sensitivity.
The second component: identifying extreme transactions
If sales in a given month exceed the threshold, STOCK finds the suspicious sales and keeps looking for extremes either based on records for the given customer and/or on the extreme’s share on the total sales in the given month. Subsequently, the system evaluates whether the transaction is so big that it requires cleaning. Whether the sale is big enough is determined by a histogram of sales: ‘significant sales’ are the sales with a certain ‘x’-time size of the upper quartile. The value of ‘x’ is different for different sensitivities.
On this basis, STOCK selects only those extreme sales that must be cleaned. A customer’s history is important: if there are two similarly sized extreme sales, the algorithm identifies as an extreme the one done by a customer with a lower frequency of sales (if someone buys often, it is more likely that sales to her/him will not be extreme).
Setting extreme sales using SIDI¶
Despite automatic detection of extreme sales, it is always the best practice to label extreme sales as such already when data from the primary system are imported.
Labeling can be done on two levels:
- There is a flag at the transaction and the transaction is then automatically labeled as extreme.
- A customer is identified as an extreme. Then every sale to the customer is labeled as extreme.
Setting extreme sales using GUI¶
Extreme sales can be labeled as such also directly in STOCK, using the tab ‘Cleaning sales.’ Here, it is possible to filter out all sales in a selected period (sales of a particular size) and then identify specific sales (or an entire month) as extreme.
In the tab ‘Cleaning sales’, it is visible whether the transaction was labeled automatically, i.e. by the STOCK system – it then appears as ‘Autoextrem’. By ticking a checkbox, it is possible to manually label a selected sale as extreme.
Up to 30,000 transactions may be shown on a screen. If a product (a category) pertains to more transactions, the system displays a message saying that the product or category cannot be viewed. In this case, it is necessary to filter to a lower category (i.e., a segment), or to limit products using a date filter.
All sales can be exported to Excel.
Explanation of extremes trimming¶
To clarify reasons behind trimming an extreme, an explanation of the trimming is available for each of the algorithms.
Algorithm for products with a small number of sales¶
The table below illustrates the exact course of the algorithm applied on individual sales; it also shows step-by-step decisions for each of the sales.
Algorithm for products with a higher number of sales¶
As the first approximation, the figure below shows time series decomposed into individual components:
As you can see, the algorithm trims the data series and the months considered extreme according to various levels of sensitivity of trimming. When you mouse over the trimmed month, you can see which particular sales were adjusted.
Extensions¶
Joint analysis of extremes for a product group¶
The basic cleaning of sales applies to a single warehouse item, that is to a single product in a particular warehouse.
But sometimes it is helpful to consider as an extreme just such a sale that is extreme with respect to a larger group of warehouses – or even with respect to the whole company.
There is another case in which it is better to apply the analysis to a group of items: namely once the module Virtual warehouse is used.
A joint analysis of extremes can be set for any group of items. The algorithm then proceeds as if there was just a single item as an input.
Cleaning contractual customers¶
The extension is used in the case a user has no option to label (a group of) sales – which are to be considered extreme – in the ERP system. The STOCK system provides a simple tool: it is easy to set a combination of a Customer, Warehouse, Product and the requested Period so that the relevant sales are considered extreme.
For the extension to be used, you must select a customer – other fields are optional. If nothing further is specified, then the setting applies to all warehouses / products / for the entire sales period.
Settings of exceptions in extremes cleaning¶
Settings of exceptions in extremes cleaning is used in those cases when user needs to clean extremes of various products with different force.
An example of using these settings are products, that have extreme sales in the past, but those sales are not coincidental. If such sales are cleaned, in future its stock level at a specific warehouse will be kept at lower level that it should be - and in the moment when the extreme sales is repeated, a stockout of this product at the warehouse would happen.
In the tab “Exception settings” the user may select a warehouse, where this specific product category is located, product category level, last but not least the supplier, ABC category and a new item, as shown in the following picture.
The most important setting is the extremes cleaning sensitivity. Planning Wizard uses the middle value of sensitivity as a default. The value “do not clean” means that all historical sales will be kept. The sensitivity level is dependent on the number of extremes, that could be cleaned, then on the number of pass-throughs, value of trimming etc. The higher the sensitivity, the more historical data are trimmed.
The table below product filtering shows data about items that are marked as exceptions. In the last column of the table you may edit selected products again, or remove a line from exceptions list.
The next tab of settings, “List of items with exceptions”, shows the complete list of items with an exception.
It is important to mention, that if there is a collision between exceptions settings at related categories, the latest category settings is used. For example, that means that if the user has set an exception of non-cleaning at the category A (and this category includes Resistors sub-category), and now the user demands middle level of cleaning at the Resistors category, the middle level of cleaning for Resistors will be used. For the remaining categories inside category A the original non-cleaning exception is used.