A note on the evidence and
Bayesian Occam’s razor

Iain Murray and Zoubin Ghahramani.
Gatsby Unit Technical Report GCNU-TR 2005-003. August 2005.

In his thesis MacKay introduced a figure explaining how Bayes rule provides an automatic “Occam’s razor” effect, penalizing unnecessarily complex models. This figure has been adopted by several authors in the same schematic form. Here, after briefly reviewing necessary material, we compute a realization of the plot for a toy data modeling problem. We discuss interesting aspects of this plot and their implications for understanding model complexity.

[PDF, DjVu, GoogleViewer, BibTeX]