📉 Quantile Regression

Quantile Regression#

Regression that estimates conditional quantiles rather than the mean.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

Quantile regression estimates a conditional quantile of the target instead of its mean: for a chosen level \(\tau \in (0, 1)\) it predicts the \(\tau\)-th quantile given the features. Fit every level and you recover the inverse CDF — the whole conditional distribution.

The pinball loss and τ#

It minimizes the pinball loss, which weights over- and under-prediction asymmetrically by \(\tau\):

\[\begin{split}\ell_\tau(y, \hat{y}) = \begin{cases} \tau\,(y - \hat{y}) & y \ge \hat{y}, \\[2pt] (1 - \tau)(\hat{y} - y) & y < \hat{y}. \end{cases}\end{split}\]

At \(\tau = 0.5\) this is symmetric and recovers the median (equivalent to minimizing MAE); \(\tau < 0.5\) pushes the model to under-predict, \(\tau > 0.5\) to over-predict, and the further \(\tau\) is from 0.5 the stronger the asymmetry.

In practice#

It is distribution-free and robust (built on absolute differences), but fits each quantile separately, which can cause quantile crossing (a lower quantile predicted above a higher one) unless constrained. Common estimators: the linear QuantileRegressor, gradient-boosted quantile models, and quantile random forests.

from sklearn.linear_model import QuantileRegressor

lower = QuantileRegressor(quantile=0.05, alpha=0.0).fit(X_train, y_train)
upper = QuantileRegressor(quantile=0.95, alpha=0.0).fit(X_train, y_train)
# [lower.predict(X), upper.predict(X)] is a 90% prediction interval

Theme: Risk & Probabilistic Forecasting · All terminology

Hint

Mind map — connected ideas

Quantile Forecasts · Prediction Intervals (PI) · Quantile Level · Predicting Percentiles · Probabilistic Forecasts · R² (R-squared)

Hint

More in Risk & Probabilistic Forecasting

Continuous Probabilistic Forecasts · Continuous Ranked Probability Score (CRPS) · Deterministic forecasts · Full Distribution · Pinball Loss (a.k.a. Quantile Loss) · Point Forecasts · Predicting Percentiles · Prediction Intervals (PI) · Probabilistic Forecasts · Probabilistic Scoring · Probability Forecasts · Quantile Forecasts · Quantile Level · Return Distribution