Worm Wars!
Live from Bullwinkle Plaza: Ted Miguel is 1… 2… 3… 4… 5… 6… 7… 8… 9… 10… 11… 12… 13… offices down the hall. So be warned!
BUT NEVERTHELESS:
My take on this is the same as the very sharp and very hard working Chris Blattman’s: http://chrisblattman.com/2015/07/23/dear-journalists-and-policymakers-what-you-need-to-know-about-the-worm-wars/ http://chrisblattman.com/2015/07/24/the-10-things-i-learned-in-the-trenches-of-the-worm-wars/
One of the things I love about the Internet is that it brought a lot of very smart people to a key intellectual problem, the discussion brought out the central assumptions and claims, and they were answered within about a day or two. See Berk Ozler, for example. My conclusion is that the Kenya deworming results are relatively robust. Yay hive mind.
On the other hand, the hive mind has a tendency to be grumpy, rude, shrill and angry. I found the debate more dignified than some, but vicious at times.
I am guilty as well. I was too quick to suspect and insinuate bad faith on the part of the replicators. I can hold suspicions but I shouldn’t publicly insinuate or accuse without grounds. I am sorry for that.
I do find any big, coordinated media push of a scientific finding to be problematic, to say the least. This drove and drives my suspicion of bias, even if accidental.
To me the big failure in this entire business was by the editor of the academic journal. The competing claims on whether or not the results are fragile or not, and why, should never have been allowed to remain ambiguous.
To me, a glitzy media push undermined the credibility and intentions of the journal further. This is a general problem in medicine and hard science that I do not see as much in social science (where the journals could care less about news coverage).
On the journalist side, I can’t blame any of the writers for not following the finer statistical points. I had trouble myself. But almost none of the journalists read the reply by Miguel and Kremer (published by the same journal) and maybe none called Miguel or Kremer on the phone. I am told I was the first. Tell me if I’m wrong, but isn’t this the definition of sloppy journalism?
I think the deworm the world movement has also tended to exaggerate or selectively quote the evidence in order to justify the cause. GiveWell has a much more balanced account: the evidence is not that great, but it’s good enough. Sort of. I thought GiveWell had one of the best posts. Do read it. http://blog.givewell.org/2015/07/24/new-deworming-reanalyses-and-cochrane-review/
To me, the real tragedy is that, 18 years after the Kenya deworming experiment (which was not even a real experiment) we do not have large-scale, randomized, multi-country, long-term evidence on the health, education, and labor market impacts of deworming medicine. This is not some schmuck cause. This is touted as one of the most promising development interventions in human history.
I also fear for the reputation of replications in development economics. I imagine some of the problems could be addressed by getting more clarity into pre-analysis plans for replications. But the incentives to make mountains out of molehills is huge. Maybe everyone should sign a “no glitzy media push” pledge.