Must-read: Cosma Shalizi (2011): “When Bayesians Can’t Handle the Truth”
Must-Read: So I was teaching Acemoglu, Johnson, and Robinson’s “Atlantic Trade” paper last week, and pointing out that (a) eighteenth-century England is a hugely-influential observation at the very edge of the range of the independent variables in the regression, and (b) it carries a huge residual even with a large estimated coefficient on Atlantic trade interacted with representative government. The huge residual, I said, means that the computer is saying: “I really do not like this model”. The rejection of a null hypothesis on the coefficient of interest is the computer saying “even though the model with a large coefficient is very unlikely, the model with a zero coefficient is very very very unlikely”. But, I said, Acemoglu, Johnson, and Robinson do not let their computer say the first statement, but only the second.
And so I thought of Cosma Shalizi and his:
(2011): When Bayesians Can’t Handle the Truth: “When should a frequentist expect Bayesian updating to work?…
…There are elegant results on the consistency of Bayesian updating for well-specified models facing IID or Markovian data, but both completely correct models and fully observed states are vanishingly rare. In this talk, I give conditions for posterior convergence that hold when the prior excludes the truth, which may have complex dependencies. The key dynamical assumption is the convergence of time-averaged log likelihoods (Shannon- McMillan-Breiman property). The main statistical assumption is a building into the prior a form of capacity control related to the method of sieves. With these, I derive posterior convergence and a large deviations principle for the posterior, even in infinite- dimensional hypothesis spaces, extending in some cases to the rates of convergence; and clarify role of the prior and of model averaging as regularization devices. Paper: http://projecteuclid.org/euclid.ejs/1256822130