Multiple Linear Regression#

Stage 5 · 📈 Regression · Lesson 32 of 56 · intermediate

◀ Previous · Least Squares Regression · Next · Feature Importance in Linear Regression ▶ · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

More than one predictor#

Real outcomes rarely depend on a single cause. A taxi fare responds to distance and duration; a house price to size, location and age. Multiple linear regression extends the line to several features at once:

\[\hat{y} = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \dots + \beta_p x_p.\]

Geometrically this is no longer a line but a hyperplane, yet the fitting principle is unchanged: choose the coefficients that minimise the sum of squared residuals.

Holding others constant#

The coefficients gain a subtle, powerful meaning. Each \(\beta_j\) is the change in \(y\) for a one-unit increase in \(x_j\) while holding all other features fixed. That “holding constant” is what lets regression disentangle overlapping influences — estimating the effect of duration on fare after accounting for distance, rather than confounding the two. It is why multiple regression is a workhorse for interpretation, not just prediction.

The matrix solution#

With many features, the tidy way to write and solve the problem is matrix algebra. Stacking the data into a matrix \(X\) (with a column of ones for the intercept) and outcomes into a vector \(y\), least squares has the closed-form solution

\[\hat{\beta} = (X^{\top} X)^{-1} X^{\top} y.\]

This single expression delivers all the coefficients at once. It rests on assumptions worth remembering — a genuinely linear relationship, independent errors of constant variance — which the evaluation stage checks with residual plots.

Fitting it in Python#

In practice you never invert the matrix by hand. scikit-learn fits the model in three lines — LinearRegression().fit(X, y) — exposing .coef_ and .intercept_; statsmodels OLS adds a full statistical summary with standard errors and p-values for each coefficient. The next lesson uses those coefficients to ask which features matter most.

Hint