🎲 Bayesian Data Analysis
Reasoning about uncertainty with priors, likelihoods and posteriors
0.5.dev0+git.20260808.d6e9440 - August 08, 2026 18:54 UTC

Bayesian Data Analysis

Bayesian analysis treats unknown quantities as probability distributions and updates them with data: instead of a single best estimate you get a full posterior. This course follows the arc of Gelman et al., Bayesian Data Analysis — from first principles through computation and regression up to the nonparametric models (mixtures, Dirichlet processes) that close it — as an ordered sequence of 144 lessons across 16 stages in 5 parts.

Read it at any depth:

• newcomers — the intuition of prior → likelihood → posterior;

• practitioners — how to compute, check and compare posteriors;

• researchers — hierarchical, GP and nonparametric (infinite-mixture) models.

Note

Type in the filter box for instant lookup by title or keyword, expand a stage to browse, or open the A–Z index at the bottom. Code snippets use real scipy.stats / PyMC / ArviZ / scikit-learn calls. This course pairs with the Terminology reference (probability and distributions).

Fundamentals

🎲 Stage 1 — The Bayesian Idea · 10 lessons

The three steps, notation, probability as uncertainty, and first worked examples.

• 001 · The three steps of Bayesian data analysis — Set up a full probability model, condition on data, then evaluate the fit.

• 002 · General Notation for Statistical Inference — The notation for parameters, data and distributions used throughout.

• 003 · Bayesian Inference — Drawing conclusions as posterior probability statements about parameters.

• 004 · Discrete Bayesian Examples – Genetics and Spell Checking (with θ) — Bayes’ rule on discrete unknowns, from carrier genetics to spell-checking.

• 005 · Probability as a Measure of Uncertainty — Interpreting probability as a quantified degree of belief.

• 006 · Example — Probabilities from Football Point Spreads — Reading calibrated probabilities off betting point spreads.

• 007 · Example — Calibration for Record Linkage — Assigning match probabilities when merging records across files.

• 008 · Some Useful Results from Probability Theory — Key identities — conditioning, expectation, transformation — the analysis relies on.

• 009 · Computation and Software — The computing tools and workflow for fitting Bayesian models.

• 010 · Bayesian Inference in Applied Statistics — How the Bayesian approach plays out across real applications.

📍 Stage 2 — Single-Parameter Models & Priors · 9 lessons

Binomial and normal models; informative, noninformative and weakly-informative priors.

• 011 · Estimating a Probability from Binomial Data — The Beta–Binomial model: inferring a success probability.

• 012 · Posterior as a Compromise Between Data and Prior Information — How the posterior blends prior belief with observed evidence.

• 013 · Summarizing Posterior Inference — Point estimates, intervals and quantiles from a posterior.

• 014 · Informative Prior Distributions — Priors that encode real prior knowledge about a parameter.

• 015 · Normal Distribution with Known Variance — Conjugate inference for a normal mean when the variance is fixed.

• 016 · Other Standard Single-Parameter Models — Poisson, exponential and related conjugate one-parameter models.

• 017 · Informative Prior Distribution for Cancer Rates — A worked informative-prior analysis of county cancer rates.

• 018 · Noninformative Prior Distributions — Priors meant to let the data dominate the inference.

• 019 · Weakly Informative Prior Distributions — Priors that gently regularize without imposing strong belief.

🧮 Stage 3 — Multiparameter Models · 8 lessons

Nuisance parameters, the multinomial and multivariate normal, and the bioassay example.

• 020 · Averaging Over Nuisance Parameters — Marginalizing out parameters you do not care about.

• 021 · Normal Data with a Noninformative Prior Distribution — Joint inference for a normal mean and variance under a flat prior.

• 022 · Normal Data with a Conjugate Prior Distribution — The normal–inverse-gamma conjugate analysis for mean and variance.

• 023 · Multinomial Model for Categorical Data — Dirichlet–multinomial inference for category probabilities.

• 024 · Multivariate Normal Model with Known Variance — Inferring a mean vector under a known covariance.

• 025 · Multivariate Normal with Unknown Mean and Variance — Joint inference for mean and covariance via the inverse-Wishart.

• 026 · Example: Bayesian analysis of a bioassay experiment (logistic, nonconjugate) — A nonconjugate logistic dose–response fit on a grid and by sampling.

• 027 · Summary of Elementary Modeling and Computation — Consolidating the single- and multi-parameter modeling toolkit.

📏 Stage 4 — Asymptotics & Frequentist Ties · 5 lessons

Normal approximation, large-sample theory, and how Bayesian and frequentist inference relate.

• 028 · Normal Approximations to the Posterior Distribution — Approximating a posterior by a normal around its mode.

• 029 · Large-Sample Theory — How posteriors concentrate and become normal as data grow.

• 030 · Counterexamples to large-sample (asymptotic) Bayesian theorems — When the normal-approximation guarantees fail.

• 031 · Frequency Evaluations of Bayesian Inferences — Judging Bayesian procedures by their long-run frequency behavior.

• 032 · Bayesian interpretations of other statistical methods — Seeing penalized and classical methods as Bayesian in disguise.

🏛️ Stage 5 — Hierarchical Models · 7 lessons

Exchangeability, the eight-schools model, and meta-analysis through partial pooling.

• 033 · Constructing a Parameterized Prior Distribution — Building a prior whose parameters are themselves estimated.

• 034 · Exchangeability and hierarchical models — Exchangeability as the justification for hierarchical structure.

• 035 · Bayesian analysis of conjugate hierarchical models — Fitting hierarchical models that admit conjugate updates.

• 036 · Normal model with exchangeable parameters — The normal hierarchical model with a shared population distribution.

• 037 · Example: parallel experiments in eight schools — The canonical partial-pooling example across eight studies.

• 038 · Hierarchical modeling applied to a meta-analysis — Pooling effect estimates across studies hierarchically.

• 039 · Weakly Informative Priors for Variance Parameters — Sensible priors for group-level variance in hierarchies.

Checking & Deciding

🔍 Stage 6 — Model Checking & Comparison · 10 lessons

Posterior predictive checks, predictive accuracy, Bayes factors, and model expansion.

• 040 · The Place of Model Checking in Applied Bayesian Statistics — Why checking fit is integral, not optional, to modeling.

• 041 · Do the Inferences from the Model Make Sense? — Sanity-checking posterior conclusions against knowledge.

• 042 · Posterior predictive checking — Comparing data replicated from the model to the real data.

• 043 · Graphical posterior predictive checks — Visual comparisons of replicated and observed data.

• 044 · Model checking for the educational testing example — Applying predictive checks to the eight-schools fit.

• 045 · Measures of predictive accuracy — Log predictive density, WAIC and cross-validation scores.

• 046 · Model comparison based on predictive performance — Choosing models by out-of-sample predictive ability.

• 047 · Model comparison using Bayes factors — Comparing models via marginal-likelihood ratios.

• 048 · Continuous model expansion — Embedding a model in a richer family to test assumptions.

• 049 · Implicit assumptions and model expansion: an example — Surfacing hidden assumptions by expanding a model.

🗳️ Stage 7 — Data Collection & Decisions · 12 lessons

Ignorability, surveys and experiments, censoring, and Bayesian decision analysis.

• 050 · Bayesian inference requires a model for data collection — Why the sampling/collection mechanism must be modeled.

• 051 · Data-collection models and ignorability — When the data-collection process can be safely ignored.

• 052 · Sample surveys — Bayesian inference for survey sampling designs.

• 053 · Designed experiments — Modeling randomized experiments the Bayesian way.

• 054 · Sensitivity and the role of randomization — How randomization protects inference from hidden bias.

• 055 · Observational studies — Inference when treatment is not randomized.

• 056 · Censoring and truncation — Modeling data that are cut off or partially observed.

• 057 · Bayesian decision theory in different contexts — Choosing actions to maximize expected utility.

• 058 · Using regression predictions: survey incentives — A decision analysis of incentive choices via regression.

• 059 · Multistage decision making: medical screening — Sequential decisions under uncertainty in screening.

• 060 · Hierarchical decision analysis for home radon — A hierarchical decision model for radon remediation.

• 061 · Personal vs. institutional decision analysis — Whose utilities and information a decision should use.

Computation

🧰 Stage 8 — Simulation Basics · 7 lessons

Numerical integration, rejection and importance sampling, and how many draws are needed.

• 062 · Numerical integration — Quadrature and simulation for intractable integrals.

• 063 · Distributional approximations — Approximating posteriors with tractable distributions.

• 064 · Direct simulation and rejection sampling — Drawing exact samples by proposal and acceptance.

• 065 · Importance sampling — Reweighting draws from a proposal to target the posterior.

• 066 · How many simulation draws are needed? — Sizing Monte Carlo samples for a target accuracy.

• 067 · Computing environments — The software environments for Bayesian computation.

• 068 · Debugging Bayesian computing — Diagnosing and fixing errors in model fitting.

⛓️ Stage 9 — MCMC: Gibbs, Metropolis & HMC · 12 lessons

Samplers, convergence diagnostics, Hamiltonian Monte Carlo, and Stan.

• 069 · Gibbs sampler — Sampling each parameter in turn from its full conditional.

• 070 · Metropolis and Metropolis-Hastings algorithms — Accept–reject proposals that target the posterior.

• 071 · Using Gibbs and Metropolis as building blocks — Composing samplers for complex models.

• 072 · Inference and assessing convergence — R-hat, effective sample size and mixing diagnostics.

• 073 · Effective number of simulation draws — Effective sample size after autocorrelation.

• 074 · Example: hierarchical normal model — A worked Gibbs sampler for the normal hierarchy.

• 075 · Efficient Gibbs samplers — Reparameterizations that speed up Gibbs sampling.

• 076 · Efficient Metropolis jumping rules — Tuning proposal scale for efficient Metropolis moves.

• 077 · Further extensions to Gibbs and Metropolis — Slice, reversible-jump and auxiliary-variable extensions.

• 078 · Hamiltonian Monte Carlo — Gradient-guided proposals for efficient exploration.

• 079 · Hamiltonian Monte Carlo for a hierarchical model — Applying HMC to a hierarchical example.

• 080 · Stan: developing a computing environment — The Stan language and NUTS sampler for Bayesian models.

🎛️ Stage 10 — Modal & Variational Approximation · 10 lessons

Posterior modes, EM, variational inference, and expectation propagation.

• 081 · Finding posterior modes — Optimization to locate the posterior’s peak.

• 082 · Boundary-avoiding priors for modal summaries — Priors that keep modal estimates off the boundary.

• 083 · Normal and related mixture approximations — Approximating posteriors with normal mixtures.

• 084 · Finding marginal posterior modes using EM — The EM algorithm for marginal mode-finding.

• 085 · Conditional and marginal posterior approximations — Approximating conditionals and marginals in stages.

• 086 · Example: hierarchical normal model (continued) — The hierarchical normal model via modal approximation.

• 087 · Variational inference — Fitting the closest tractable distribution to the posterior.

• 088 · Expectation propagation — Iteratively matching moments for posterior approximation.

• 089 · Other approximations — Laplace, INLA and further approximate-inference methods.

• 090 · Unknown normalizing factors — Inference when the likelihood’s constant is intractable.

Regression

📈 Stage 11 — Regression Foundations · 8 lessons

Bayesian classical regression, causal inference, and regularization.

• 091 · Conditional modeling — Modeling an outcome conditional on predictors.

• 092 · Bayesian analysis of classical regression — Posterior inference for the normal linear model.

• 093 · Regression for causal inference: incumbency and voting — Using regression to estimate an incumbency effect.

• 094 · Goals of regression analysis — Prediction, explanation and causal aims of regression.

• 095 · Assembling the matrix of explanatory variables — Coding, transforming and interacting predictors.

• 096 · Regularization and dimension reduction — Shrinkage priors and reducing predictor dimension.

• 097 · Unequal variances and correlations — Modeling heteroscedastic and correlated errors.

• 098 · Including numerical prior information — Adding quantitative prior knowledge to regression.

🏗️ Stage 12 — Hierarchical Regression · 7 lessons

Batched coefficients, varying intercepts and slopes, and election forecasting.

• 099 · Regression coefficients exchangeable in batches — Grouping coefficients into exchangeable batches.

• 100 · Example: forecasting U.S. presidential elections — A hierarchical election-forecasting regression.

• 101 · Interpreting a normal prior distribution as extra data — Reading a coefficient prior as pseudo-observations.

• 102 · Varying intercepts and slopes — Letting regression coefficients vary across groups.

• 103 · Computation: batching and transformation — Efficient computation for hierarchical regressions.

• 104 · Analysis of variance and the batching of coefficients — ANOVA as hierarchical batching of effects.

• 105 · Hierarchical models for batches of variance components — Priors over multiple variance components.

🔗 Stage 13 — Generalized Linear Models · 7 lessons

GLM likelihoods, priors for logistic regression, and overdispersed Poisson models.

• 106 · Standard generalized linear model likelihoods — Binomial, Poisson and other GLM data models.

• 107 · Working with generalized linear models — Fitting, checking and interpreting Bayesian GLMs.

• 108 · Weakly informative priors for logistic regression — Regularizing logistic coefficients to avoid separation.

• 109 · Overdispersed Poisson regression for police stops — A hierarchical Poisson model of stop-and-frisk data.

• 110 · State-level opinons from national polls — Multilevel regression and poststratification of opinion.

• 111 · Models for multivariate and multinomial responses — GLMs for vector and categorical outcomes.

• 112 · Loglinear models for multivariate discrete data — Loglinear structure for contingency tables.

🛡️ Stage 14 — Robustness & Missing Data · 11 lessons

Heavy-tailed errors, overdispersion, and multiple imputation.

• 113 · Aspects of robustness — Making inference resilient to model misspecification.

• 114 · Overdispersed versions of standard models — Heavy-tailed and overdispersed extensions.

• 115 · Posterior inference and computation — Computing posteriors for robust models.

• 116 · Robust inference for the eight schools — A t-based robust reanalysis of eight schools.

• 117 · Robust regression using t-distributed errors — Down-weighting outliers with t errors.

• 118 · Notation — Notation for the missing-data framework.

• 119 · Multiple imputation — Filling in missing values with several posterior draws.

• 120 · Missing data in the multivariate normal and t models — Imputation under multivariate normal and t models.

• 121 · Example: multiple imputation for a series of polls — Imputing missing responses across repeated polls.

• 122 · Missing values with counted data — Handling missingness in count data.

• 123 · Example: an opinion poll in Slovenia — A worked missing-data analysis of a plebiscite poll.

Nonlinear & Nonparametric

🌊 Stage 15 — Basis Functions & Gaussian Processes · 9 lessons

Splines, Gaussian-process regression, and functional data analysis.

• 124 · Example: serial dilution assay — A nonlinear Bayesian calibration for dilution assays.

• 125 · Example: population toxicokinetics — A hierarchical nonlinear pharmacokinetic model.

• 126 · Splines and weighted sums of basis functions — Flexible curves built from weighted basis functions.

• 127 · Basis selection and shrinkage of coefficients — Shrinking basis coefficients to control smoothness.

• 128 · Non-normal models and regression surfaces — Nonlinear and non-normal regression surfaces.

• 129 · Gaussian process regression — Priors over functions for flexible regression.

• 130 · Example: birthdays and birthdates — A Gaussian-process decomposition of a birth time series.

• 131 · Latent Gaussian process models — GPs as latent components inside larger models.

• 132 · Functional data analysis — Modeling whole curves as data objects.

♾️ Stage 16 — Mixtures & Nonparametric Bayes · 12 lessons

Mixtures, label switching, Bayesian histograms, and Dirichlet processes.

• 133 · Density estimation and regression — Estimating whole distributions, not just parameters.

• 134 · Setting up and interpreting mixture models — Modeling populations as blends of components.

• 135 · Example: reaction times and schizophrenia — A mixture model separating response-time subgroups.

• 136 · Label switching and posterior computation — Handling component permutation in mixture posteriors.

• 137 · Unspecified number of mixture components — Letting the number of components be unknown.

• 138 · Mixture models for classification and regression — Mixtures as flexible supervised models.

• 139 · Bayesian histograms — Nonparametric density via random histograms.

• 140 · Dirichlet process prior distributions — Priors over distributions with unbounded support.

• 141 · Dirichlet process mixtures — Infinite mixtures whose clusters grow with the data.

• 142 · Beyond density estimation — Nonparametric priors for broader modeling tasks.

• 143 · Hierarchical dependence — Dependent nonparametric priors across groups.

• 144 · Density regression — Letting an entire density vary with predictors.

🔤 Every lesson, A–Z index

🔠 Open the full alphabetical index

Analysis of variance and the batching of coefficients Aspects of robustness Assembling the matrix of explanatory variables Averaging Over Nuisance Parameters Basis selection and shrinkage of coefficients Bayesian analysis of classical regression Bayesian analysis of conjugate hierarchical models Bayesian decision theory in different contexts Bayesian histograms Bayesian Inference Bayesian Inference in Applied Statistics Bayesian inference requires a model for data collection Bayesian interpretations of other statistical methods Beyond density estimation Boundary-avoiding priors for modal summaries Censoring and truncation Computation and Software Computation: batching and transformation Computing environments Conditional and marginal posterior approximations Conditional modeling Constructing a Parameterized Prior Distribution Continuous model expansion Counterexamples to large-sample (asymptotic) Bayesian theorems Data-collection models and ignorability Debugging Bayesian computing Density estimation and regression Density regression Designed experiments Direct simulation and rejection sampling Dirichlet process mixtures Dirichlet process prior distributions Discrete Bayesian Examples – Genetics and Spell Checking (with θ) Distributional approximations Do the Inferences from the Model Make Sense? Effective number of simulation draws Efficient Gibbs samplers Efficient Metropolis jumping rules Estimating a Probability from Binomial Data Example — Calibration for Record Linkage Example — Probabilities from Football Point Spreads Example: an opinion poll in Slovenia Example: Bayesian analysis of a bioassay experiment (logistic, nonconjugate) Example: birthdays and birthdates Example: forecasting U.S. presidential elections Example: hierarchical normal model Example: hierarchical normal model (continued) Example: multiple imputation for a series of polls Example: parallel experiments in eight schools Example: population toxicokinetics Example: reaction times and schizophrenia Example: serial dilution assay Exchangeability and hierarchical models Expectation propagation Finding marginal posterior modes using EM Finding posterior modes Frequency Evaluations of Bayesian Inferences Functional data analysis Further extensions to Gibbs and Metropolis Gaussian process regression General Notation for Statistical Inference Gibbs sampler Goals of regression analysis Graphical posterior predictive checks Hamiltonian Monte Carlo Hamiltonian Monte Carlo for a hierarchical model Hierarchical decision analysis for home radon Hierarchical dependence Hierarchical modeling applied to a meta-analysis Hierarchical models for batches of variance components How many simulation draws are needed? Implicit assumptions and model expansion: an example

Importance sampling Including numerical prior information Inference and assessing convergence Informative Prior Distribution for Cancer Rates Informative Prior Distributions Interpreting a normal prior distribution as extra data Label switching and posterior computation Large-Sample Theory Latent Gaussian process models Loglinear models for multivariate discrete data Measures of predictive accuracy Metropolis and Metropolis-Hastings algorithms Missing data in the multivariate normal and t models Missing values with counted data Mixture models for classification and regression Model checking for the educational testing example Model comparison based on predictive performance Model comparison using Bayes factors Models for multivariate and multinomial responses Multinomial Model for Categorical Data Multiple imputation Multistage decision making: medical screening Multivariate Normal Model with Known Variance Multivariate Normal with Unknown Mean and Variance Non-normal models and regression surfaces Noninformative Prior Distributions Normal and related mixture approximations Normal Approximations to the Posterior Distribution Normal Data with a Conjugate Prior Distribution Normal Data with a Noninformative Prior Distribution Normal Distribution with Known Variance Normal model with exchangeable parameters Notation Numerical integration Observational studies Other approximations Other Standard Single-Parameter Models Overdispersed Poisson regression for police stops Overdispersed versions of standard models Personal vs. institutional decision analysis Posterior as a Compromise Between Data and Prior Information Posterior inference and computation Posterior predictive checking Probability as a Measure of Uncertainty Regression coefficients exchangeable in batches Regression for causal inference: incumbency and voting Regularization and dimension reduction Robust inference for the eight schools Robust regression using t-distributed errors Sample surveys Sensitivity and the role of randomization Setting up and interpreting mixture models Some Useful Results from Probability Theory Splines and weighted sums of basis functions Stan: developing a computing environment Standard generalized linear model likelihoods State-level opinons from national polls Summarizing Posterior Inference Summary of Elementary Modeling and Computation The Place of Model Checking in Applied Bayesian Statistics The three steps of Bayesian data analysis Unequal variances and correlations Unknown normalizing factors Unspecified number of mixture components Using Gibbs and Metropolis as building blocks Using regression predictions: survey incentives Variational inference Varying intercepts and slopes Weakly Informative Prior Distributions Weakly informative priors for logistic regression Weakly Informative Priors for Variance Parameters Working with generalized linear models

🗺️ scikit-plots & the Bayesian stack

Where scikit-plots and the PPL stack fit

scikit-plots’ role here is diagnostic and model-selection visual support; the heavy lifting is done by the probabilistic-programming stack.

• Gaussian Mixture Models (AIC / BIC) — choose the number of components: https://scikit-plots.github.io/dev/auto_examples/stats/plot_gaussian_mixture_models.html

• Residuals distribution — distributional / Q–Q model checks: https://scikit-plots.github.io/dev/auto_examples/stats/plot_residuals_distribution_script.html

• PyMC — probabilistic programming: https://www.pymc.io/

• ArviZ — diagnostics and plots for Bayesian inference: https://python.arviz.org/

Sources & standard reference

Framing re-expressed in our own words; API calls verified against official docs.

• Source context (144 posts): https://insightful-data-lab.com/category/bayesian-data-analysis/

• SciPy stats: https://docs.scipy.org/doc/scipy/reference/stats.html

• scikit-learn mixtures: https://scikit-learn.org/stable/modules/mixture.html

• Gelman, Carlin, Stern, Dunson, Vehtari & Rubin, Bayesian Data Analysis (3rd ed.): http://www.stat.columbia.edu/~gelman/book/

• Terminology reference

Tags: purpose: reference topic: data analysis domain: bayesian