How many simulation draws are needed?#

Part 3 · Stage 8 · 🧰 Simulation Basics · Lesson 066 of 144 · intermediate

◀ Previous · Importance sampling · Next · Computing environments ▶ · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

Monte Carlo error is not posterior uncertainty#

Two uncertainties coexist in a simulation-based analysis. The posterior standard deviation \(\sigma_{\theta}\) expresses what the data leave unknown — it cannot be reduced by computing harder. The Monte Carlo standard error expresses how imprecisely your finite sample of draws estimates a posterior summary — and it shrinks as you run longer:

\[\mathrm{MCSE}(\bar{\theta}) = \frac{\sigma_{\theta}}{\sqrt{S_{\text{eff}}}} .\]

Note \(S_{\text{eff}}\), not \(S\): MCMC draws are autocorrelated, so a thousand draws may carry the information of a hundred.

Fewer than you think, for the mean#

The classic argument is bracing. Suppose you estimate the posterior mean with \(S = 100\) independent draws. The MCSE is \(\sigma_{\theta}/10\), so the total uncertainty about \(\theta\) — posterior plus simulation — inflates from \(\sigma_{\theta}\) to \(\sigma_{\theta}\sqrt{1 + 1/100} \approx 1.005 \, \sigma_{\theta}\). A 0.5% increase. For reporting a posterior mean and interval, a hundred effective draws is already enough, and the fourth decimal place of a posterior mean was never meaningful anyway.

More than you think, for tails#

The picture changes for quantities that depend on rare draws. A 2.5% quantile is estimated from the draws in that tail; a probability like \(\Pr(\theta > c \mid y) = 0.001\) requires enough draws to see the event repeatedly. As a working rule, modern practice targets \(S_{\text{eff}} \gtrsim 400\) per quantity of interest — enough for stable tail quantiles and for the convergence diagnostics of Stage 9 to be trustworthy themselves.

import arviz as az
az.summary(idata)     # columns: mcse_mean, mcse_sd, ess_bulk, ess_tail, r_hat
# ess_bulk governs the mean/sd; ess_tail governs the 5%/95% quantiles.
# Report a number only to the precision its MCSE supports.

The discipline#

Three habits follow. Report MCSE, or at least check it: a posterior mean of 2.43 with an MCSE of 0.05 should be written as 2.4. Check ess_tail separately from ess_bulk, because a chain that mixes well in the middle can crawl in the tails. And remember what more draws cannot buy: they shrink Monte Carlo error toward zero and leave posterior uncertainty exactly where it was. If the interval is too wide to act on, the remedy is more data or a better model, never a longer chain.

Hint