# Model comparison¶

## Model evidence¶

In the context of Bayesian statistics, the most rigorous way of comparing a set of models is looking at the Bayes evidence associated with each model. Given a dataset $$\mathcal D$$, model $$\mathcal M$$ and a set of parameter $$\theta$$, the evidence or marginal likelihood is given by

$\mathcal{Z} = P(\mathcal D|\mathcal M) = \int_\theta P(\mathcal D | \theta, \mathcal M) P(\theta | \mathcal M) \mathrm{d}\theta = \int_\theta \mathcal{L}_{\mathcal M}(\theta) \pi(\theta) \mathrm{d}\theta$

To compare different models, one wants to compute the Bayes factor, which is given by

$R = \frac{P(\mathcal{M}_1 | d)}{P(\mathcal{M}_2 | d)} = \frac{P(\mathcal{D} | \mathcal{M}_1) P(\mathcal{M}_1)}{P(\mathcal{D} | \mathcal{M}_2) P(\mathcal{M}_2)} = \frac{\mathcal{Z}_1}{\mathcal{Z}_2} \times \frac{P(\mathcal{M}_1)}{P(\mathcal{M}_2)} .$

If there is no a priori reason for preferring one model over the other (as it is often the case), the prior ratio $$P(\mathcal{M}_1)/P(\mathcal{M}_2)$$ becomes unity and the Bayes factor becomes the ratio of evidences.

## Information criteria¶

[ ]: