01 / 03

Write down the decision first

Choose the level at which the forecast will be used: product, category, market or channel. Then choose the horizon. A forecast for next week and a forecast for a supplier order due in three months answer different questions.

Put the existing planning method into the comparison. If the team uses last year’s sales plus a planned adjustment, record that version before changing anything. It deserves the same test as a new model.

A seasonal naïve forecast uses the last observation from the matching season, such as the same month in the previous year. It is a useful simple benchmark described by Hyndman and Athanasopoulos in Forecasting: Principles and Practice.

02 / 03

Make each method work with the same information

Pick historical decision dates and recreate what was known at each one. Do not give a model a promotion result or a final order quantity that the planner could not have known then.

Use the same product groups and time periods for every method. Mark launches, stockouts and discontinued products. Keep the difficult periods in view instead of quietly removing the weeks that make the result look worse.

For each decision date, compare the prediction with what followed. Repeating that process across dates gives a more useful picture than judging a model by how neatly it follows the history used to fit it. This follows the rolling evaluation approach in the same textbook.

03 / 03

Look beyond the average error

Inspect the products that account for most of the buying decision. Check whether a method repeatedly buys too much or too little. A small average improvement may be unhelpful if it comes with a large miss on a core product.

Consider the time required to maintain the method and explain its changes. If two approaches perform similarly, the team has a good reason to favour the one it can understand and keep running.

The purpose of this comparison is to choose a better way to buy. Keep that decision in the room when the charts arrive.