🔬 Relative accuracy

Relative accuracy#

Forecast accuracy measured against a baseline rather than in absolute terms.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

Relative accuracy measures forecast accuracy relative to a benchmark rather than in absolute units. You normalize the error by a reference method’s error — usually the naïve (or seasonal-naïve) forecast — so results land on a common, scale-free scale comparable across series:

\[\text{relative error} = \frac{\mathrm{MAE}_{\text{model}}}{\mathrm{MAE}_{\text{benchmark}}}.\]

How to read it#

A value < 1 means the model beats the benchmark, = 1 means it matches it, and > 1 means it is worse — a relative error of 0.6 is roughly 40% better than the benchmark. This family includes MASE, Theil’s U (\(<1\) beats a naïve guess), and relative / bounded relative absolute errors.

Why it matters#

Absolute errors like MAE and RMSE are meaningless without a reference — is an MAE of 10 good? depends entirely on the problem — whereas relative accuracy is interpretable and puts easy and hard-to-forecast series on equal footing.

Theme: Model Evaluation & Uncertainty · All terminology

Hint

Mind map — connected ideas

Forecast Error · Naïve Baseline Forecast · Average Absolute Error (AAE) · M-Competitions (Makridakis Competitions) · R² (R-squared) · Forecasting Benchmarks

Hint

More in Model Evaluation & Uncertainty

Average Absolute Error (AAE) · Baseline Heuristics · Bootstrap · Bootstrap Confidence Intervals (CIs) · Coverage · Cramér’s V · DeLong’s Test · KS Statistic (Kolmogorov–Smirnov Statistic) · Likelihood Ratio (LR) · Mann–Whitney U Test (also called the Wilcoxon rank-sum test) · MASE (Mean Absolute Scaled Error) · Mean Absolute Error (MAE) · Mean Absolute Percentage Error (MAPE) · Mean Squared Error (MSE)