Time-Series Forecasting skill for Claude Code

A forecast is ready when it beats a seasonal naive baseline on a rolling-origin backtest over the real horizon, its prediction intervals cover close to their nominal rate, and every feature used at forecast time would actually be known at that time. A model that was never compared to a baseline is not evaluated.

Workflow

- [ ] 1. Define the decision: target, granularity, horizon, update cadence, cost of errors
- [ ] 2. Inspect and clean the series (frequency, gaps, outliers, structural breaks)
- [ ] 3. Fit baselines: naive, seasonal naive, drift / mean
- [ ] 4. Fit candidate models
- [ ] 5. Backtest all of them with rolling origins over the real horizon
- [ ] 6. Pick by metric and stability; check interval coverage
- [ ] 7. Refit on all data, forecast, report with intervals and caveats

1. Define the decision

Question	Why it matters
Horizon (h) and cadence	backtest must use the same h; a model good at 1 day can be bad at 8 weeks
Granularity (SKU x store x day vs category x week)	aggregate forecasts are more accurate; forecast at the level decisions are made
Is over- or under-forecasting worse?	choose a quantile (e.g. P90 for stockouts) rather than the mean
What is known in the future?	holidays, prices, promotions planned ahead are usable; weather or actual traffic are not unless forecast

2. Inspect

import pandas as pd
df = pd.read_parquet("sales.parquet")                      # columns: unique_id, ds, y
df["ds"] = pd.to_datetime(df["ds"])
full = (df.set_index("ds").groupby("unique_id")["y"]
          .apply(lambda s: s.asfreq("D")).reset_index())     # expose missing dates
print(full["y"].isna().groupby(full["unique_id"]).mean().describe())

• Make the frequency explicit and fill gaps deliberately: zero where absence means no sales, interpolation for sensor gaps, NaN-aware models otherwise. Say which.
• Plot every series (or a sample plus the largest). Look for level shifts (COVID, a pricing change, a tracking bug), trend, weekly and yearly seasonality, outliers.
• Data before a structural break often hurts; consider training only after it or adding a regime indicator.
• Stock-outs censor demand: zero sales while out of stock is not zero demand.

3-4. Baselines and candidates

Series type	Start with
Strong single seasonality, stable	seasonal naive, AutoETS
Trend + seasonality, moderate length	AutoETS, AutoARIMA
Multiple seasonalities (hourly data: daily + weekly)	MSTL, or gradient boosting with calendar features
Many related series (thousands of SKUs)	one global LightGBM model with lag/rolling features, or statsforecast models run per series
Intermittent demand (many zeros)	Croston / ADIDA / IMAPA (statsforecast), evaluate with appropriate metrics
Strong known drivers (price, promo, holidays)	regression with ARIMA errors, gradient boosting, or Prophet with regressors

from statsforecast import StatsForecast
from statsforecast.models import Naive, SeasonalNaive, AutoETS, AutoARIMA

sf = StatsForecast(
    models=[Naive(), SeasonalNaive(season_length=7), AutoETS(season_length=7), AutoARIMA(season_length=7)],
    freq="D", n_jobs=-1,
)
cv = sf.cross_validation(df=train_df, h=28, step_size=7, n_windows=6)   # columns: unique_id, ds, cutoff, y, <models>

Prophet, when holidays and changepoints dominate and interpretability matters:

from prophet import Prophet
m = Prophet(yearly_seasonality=True, weekly_seasonality=True, interval_width=0.8)
m.add_country_holidays(country_name="US")
m.fit(series[["ds", "y"]])
fc = m.predict(m.make_future_dataframe(periods=28))

Prophet is frequently beaten by ETS/ARIMA and seasonal naive on short or noisy series; include it as a candidate, not a default.

Gradient boosting with lag features: create lags at least h steps back (or use a recursive/direct strategy deliberately), rolling stats computed only from past values (shift(h) before rolling), calendar features, and known-future covariates. Libraries such as mlforecast handle the bookkeeping.

5. Backtesting

Rolling origin: train up to cutoff t, forecast t+1..t+h, slide forward by step_size, repeat over at least 4-6 windows that include the seasons that matter (e.g. December).

|---- train ----|-- h --|
|------ train ------|-- h --|
|-------- train --------|-- h --|

Leakage checklist:

• Features at each cutoff use only data available at that cutoff
• Scaling, imputation, outlier caps fitted inside each training window
• Hyperparameters chosen on earlier windows, evaluated on later ones
• No random K-fold splits on time series
• Holiday/promo calendars contain only what was planned in advance

Metrics

import numpy as np

def mase(y, yhat, insample, m=7):
    scale = np.mean(np.abs(insample[m:] - insample[:-m]))      # in-sample seasonal naive MAE
    return np.mean(np.abs(y - yhat)) / scale

def wape(y, yhat):
    return np.sum(np.abs(y - yhat)) / np.sum(np.abs(y))

Metric	Use	Caveat
MASE	comparing across series and against seasonal naive (< 1 beats it in-sample)	needs in-sample scale per series
WAPE	business-friendly aggregate error, volume-weighted	dominated by large series
RMSE	when big misses are disproportionately costly	scale-dependent
MAPE	avoid	undefined at zero, explodes near zero, penalizes over-forecasts more
sMAPE	only if a stakeholder insists	asymmetric despite the name
Pinball loss	quantile forecasts (P10/P50/P90)	report per quantile
Interval coverage	share of actuals inside the 80%/95% interval	should be near nominal

Report metrics per horizon bucket (days 1-7, 8-14, …) and per segment; a single number hides where the model fails.

6. Choosing

• Pick the simplest model within noise of the best (differences under a few percent of MASE across windows are often noise).
• Prefer stable performance across windows over the best average.
• Ensembles (median of ETS, ARIMA, and a global model) are often more reliable than any single model.
• Check interval coverage on the backtest: 80% intervals covering 60% mean intervals are too narrow - widen or use conformal calibration from backtest residuals.

Gotchas

• Forecasting the log or Box-Cox transform then exponentiating gives the median, not the mean; correct if totals matter.
• Aggregating independently forecast children rarely matches the parent forecast; reconcile (bottom-up, top-down, or MinT via hierarchicalforecast) if both are reported.
• Daylight saving time creates 23- and 25-hour days in hourly data; model in UTC or handle explicitly.
• New products have no history; use analogs, category curves, or judgment and flag them.
• Retrain cadence matters as much as the model; monitor production error versus the backtest and alert on drift.

Output: forecast report

## Forecast: daily units by store, horizon 28 days, generated 2026-10-20
**Backtest:** 6 rolling windows, step 7 days, cutoffs 2026-07-28..2026-09-01
| Model | MASE | WAPE | 80% coverage |
|---|---|---|---|
| Seasonal naive (7) | 1.00 | 18.4% | 74% |
| AutoETS | 0.86 | 15.9% | 79% |
| Ensemble (ETS+ARIMA+LGBM) | 0.81 | 15.1% | 81% |
**Chosen:** ensemble. Error grows from 12% (days 1-7) to 19% (days 22-28).
**Known gaps:** 14 new stores use category curves; Black Friday window under-forecast by 9% last year.
**Files:** forecasts.parquet (unique_id, ds, p10, p50, p90)

Verification

• Backtest table includes the seasonal naive baseline and at least one candidate beats it.
• Leakage checklist completed; features recomputed per cutoff.
• Interval coverage within a few points of nominal on backtest windows.
• Plot of the final forecast with intervals over recent history reviewed for obvious nonsense (negative demand, broken seasonality, level jumps).