Standard generalized linear model likelihoods#

Part 4 · Stage 13 · 🔗 Generalized Linear Models · Lesson 106 of 144 · advanced

◀ Previous · Hierarchical models for batches of variance components · Next · Working with generalized linear models ▶ · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

Beyond the normal outcome#

Linear regression assumes a continuous, unbounded, normally-distributed response. Most real outcomes are none of these: a yes/no, a count, a category, a waiting time. The generalized linear model keeps the linear predictor \(X\beta\) but connects it to the outcome through two changes — a link function and a non-normal likelihood — so the same machinery covers a whole zoo of data types.

The three ingredients#

Every GLM is specified by:

a linear predictor \(\eta = X\beta\), the familiar weighted sum;
a link function \(g\) mapping the mean of the outcome to that unbounded scale, \(g(\mu) = \eta\), so predictions respect the outcome’s range;
a distribution for the outcome given its mean, from the exponential family.

\[g\bigl(\mathrm{E}[y_i]\bigr) = X_i \beta, \qquad y_i \sim \text{ExponentialFamily}(\mu_i).\]

The link is what keeps a probability in \([0, 1]\) and a rate positive: the linear predictor roams the whole real line, and \(g^{-1}\) folds it back into the legal range.

The common members#

Four cover most applied work:

Logistic — binary \(y\). Link: logit, \(\log\frac{\mu}{1-\mu}\). Likelihood: Bernoulli/binomial. Coefficients are log odds ratios.
Poisson — counts. Link: log, \(\log \mu\). Likelihood: Poisson. Coefficients are log rate ratios; an offset handles exposure.
Normal — the identity link, the special case where the whole apparatus collapses back to linear regression.
Multinomial / ordinal — categorical outcomes, via softmax or cumulative-logit links (next stage).

import pymc as pm
# logistic regression
with pm.Model():
    beta = pm.Normal("beta", 0, 2.5, shape=X.shape[1])
    p = pm.Deterministic("p", pm.math.sigmoid(X @ beta))     # inverse logit
    pm.Bernoulli("y", p=p, observed=y)

# Poisson regression with an exposure offset
with pm.Model():
    beta = pm.Normal("beta", 0, 1, shape=X.shape[1])
    rate = pm.math.exp(X @ beta + log_exposure)              # log link + offset
    pm.Poisson("y", mu=rate, observed=counts)

Why one framework#

Unifying these under one structure is not mere tidiness. The same priors, the same hierarchical extensions (varying coefficients, batching), the same diagnostics (PPC, LOO), and the same sampler apply across every outcome type — only the link and likelihood change. Learn the pattern once and binary, count and categorical data are variations on a theme, not separate subjects. The interpretation of coefficients changes with the link, though, and that — along with the priors that keep GLMs well-behaved — is what the rest of this stage is about.

Hint