Models for multivariate and multinomial responses#

Part 4 · Stage 13 · 🔗 Generalized Linear Models · Lesson 111 of 144 · advanced

◀ Previous · State-level opinons from national polls · Next · Loglinear models for multivariate discrete data ▶ · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

When the outcome is not a single number#

So far each observation had one response. Two common cases break that: a multinomial outcome, where the response is one of several unordered (or ordered) categories, and a multivariate outcome, where several correlated responses are measured together. Both extend the GLM, and both hinge on modelling the structure among the outcomes.

Multinomial and categorical#

For an unordered categorical outcome — choice of transport, party, product — the multinomial (softmax) logit gives each category \(k\) its own linear predictor and normalises:

\[\Pr(y_i = k) = \frac{e^{X_i \beta_k}}{\sum_{k'} e^{X_i \beta_{k'}}},\]

with one category fixed as baseline for identifiability. For an ordered outcome — a Likert scale, a severity grade — the ordered logit is better: a single linear predictor with a set of increasing cutpoints slicing a latent continuum, which respects the ordering and spends far fewer parameters.

import pymc as pm
# ordered categorical: shared slope, learned cutpoints
with pm.Model():
    beta = pm.Normal("beta", 0, 1, shape=k)
    cut = pm.Normal("cut", 0, 5, shape=n_cat - 1,
                    transform=pm.distributions.transforms.ordered)   # increasing
    pm.OrderedLogistic("y", eta=X @ beta, cutpoints=cut, observed=y)

Multivariate responses#

When several responses are measured per unit — height and weight, a battery of test scores — they are correlated, and modelling them jointly is more informative than one regression each. Give the vector of responses a multivariate normal likelihood with a covariance to estimate, using the LKJ decomposition from the hierarchy stage:

\[y_i \sim \mathrm{N}(X_i B, \; \Sigma), \qquad \Sigma = \mathrm{diag}(\sigma)\,\Omega\,\mathrm{diag}(\sigma), \quad \Omega \sim \mathrm{LKJ}(\eta).\]

The estimated correlations are often the point — how the outcomes move together, given the predictors.

Why model the joint structure#

Two payoffs. Modelling outcomes jointly borrows information across them: a missing component is predicted from the others through the learned correlation, and estimates are more efficient than separate fits. And the dependence itself is substantive — the correlation between test scores given background, the covariance of symptoms — a quantity separate univariate models simply cannot report. Categorical and multivariate responses complete the GLM’s reach: with them, the framework covers binary, count, ordered, unordered and vector-valued outcomes under one set of tools.

Hint