Must-Read: Robert Waldmann: Benchmark, Model, and Hypothesis

Must-Read: Robert Waldmann: Benchmark, Model, and Hypothesis:

I imagine hopes followed by disappointments in the following order…

P might be true…. P isn’t true, but P might still be a useful approximation…. All the attempts to use P to approximate reality have failed, because each implication is far from the truth…. Work starting with P shares the fault of totally undisciplined empiricism which can describe but not forecast. However, P is a useful benchmark….

Macroeconomics is reaching the 4th stage [with respect to] the DSGE models which have dominated academic work for decades…. My view is that… the claim that a something is a useful benchmark… is similar to the claim that a model is a useful approximation…. The choice of a benchmark is important because one modification is considered at a time. If implications were… linear… it wouldn’t matter…. But they aren’t. The way in which an unrealistic DSGE model differs from the same model with a financial friction can be completely different from the way in which the real world would be different if a financial friction were eliminated….

A more important problem…. The vast majority of models in the literature share many of the implications of the benchmark…. So, for example, if the benchmark model has Ricardian equivalence, so do most of the modified models…. [When] temporary lump sum tax cuts… were proposed in the USA in 2009… many economists argued that policy makers were ignoring the results of decades of academic research. In fact, they were ignoring the implication of the standard benchmark model….

The more diplomatic critics of mainstream academic macroeconomics insist that the models, which they find unsatisfactory, are useful benchmarks…. [But] a simple example illustrat[es] the danger…. [Consider] the original Lucas supply function… [leads to] the policy ineffectiveness proposition which lead Sargent and Wallace to note that, in their model, the optimal policy was to set the inflation rate to some desired target and ignore everything else. Notably this is the policy mandate of the European Central Bank….

The repeated use of the word “hypothesis” in the 50s, 60s, and 70s strongly suggests that the equations in question were not originally considered parts of models which were false by definition. Thomas Sargent’s phrase “take a model seriously” sure seems to imply “treat a model as a null hypothesis.”… Lucas and Prescott were enthusiastic about hypothesis testing until it falsified too many of their hypotheses, and both independently said exactly that:

My recollection is that Bob Lucas and Ed Prescott were initially very enthusiastic about rational expetations econometrics. After all, it simply involved imposing on ourselves the same high standards we had criticized the Keynesians for failing to live up to. But after about five years of doing likelihood ratio tests on rational expectations models, I recall Bob Lucas and Ed Prescott both telling me that those tests were rejecting too many good models. The idea of calibration is to ignore some of the probabilistic implications of your model but to retain others…