01 / 03

Recreate the moment of the decision

A forecast can appear convincing when drawn over a finished year. The harder question is what it would have predicted before the year unfolded.

Hyndman and Athanasopoulos describe time series cross-validation as a sequence of tests in which each forecast uses only earlier observations. The forecast origin moves forward through time. They also explain how the test can cover several steps ahead rather than just the next observation.

That gives a planner a useful way to ask whether a method would have helped at the point an order had to be placed.

02 / 03

Choose the horizon that matches the buy

Our application is to start with a real purchasing decision. If you commit to stock several months ahead, judge the method at that distance. A strong result for the following week may tell you little about the decision you need to make.

For each historical buying date, keep a record of the information available then. That includes planned promotions and expected deliveries, with the uncertainty they had at the time. Final outcomes should not slip back into the inputs.

Compare more than one period. A method that handled an ordinary month may behave differently around a launch or a peak season. Show those differences to the people who will use the forecast.

03 / 03

Keep the simple comparison

The same textbook describes simple benchmark methods, including using the most recent observation or the observation from the matching season. A more complicated method should earn its place against an appropriate starting point.

For a brand, we would add a review of what the errors would have meant for the buy. Where would too much stock have landed? Which important items would have been short? These are business questions to examine alongside statistical error.

This is a reading note and a suggested application, not a forecasting result from Telemyr. A useful test leaves the team knowing what it compared, what each method knew and where it still gets things wrong.