Piketty’s data deserve better analysis

There was buzz across the economics blogosphere and twitterverse over the Memorial Day weekend and into the work week today about Chris Giles’ Friday afternoon Financial Times piece claiming that data errors in the best-selling “Capital in the 21st Century” invalidate author Thomas Piketty’s results on the relationship between economic growth and the concentration of wealth.

Giles’ claims attracted substantial attention in no small part because his attack strikes at the Paris School of Economics professor’s data, which have been nearly universally praised for thoroughness—even by critics who disagree with the conclusions that he draws from them. While it is clear that Giles spent some time looking over Piketty’s spreadsheets, he jumps to conclusions that are not supported by the points he raises.

Indeed, my examination of Giles’ analysis and the spreadsheets that Piketty provided to the public indicate that perhaps the key claim by Giles is erroneous. Giles bases his argument that there was not an increase in wealth concentration in the United Kingdom but rather a decrease on a single data point from a 2010 wealth survey in the UK. Because that survey did not exist in 2000, it cannot be directly compared to other time series data without harmonization. The entirety of the drop Giles claims is occurring can be explained by switching from one survey to another.

In contrast, Piketty went through the different surveys and sources to stitch together a coherent data set that is presumably free of these discontinuities. In order to do his comprehensive analysis of the change in wealth inequality over time, Piketty had to look at disparate data sources, harmonized them (so that he could compare apples to apples), and draw conclusions. Wealth is notoriously difficult to measure, which makes working with wealth data especially tricky. Piketty has been exceptionally transparent with the data sets used in his book (the data can be found here).

Giles uses the raw, non-harmonized wealth data to claim that wealth inequality in the United States has been flat and that it has been decreasing in the United Kingdom. Yet by combining these non-harmonized data sets, Giles is comparing apples to oranges. To say that this deviates from best data practices would be an understatement. In addition, as Piketty notes in his response to the article, recent work by Emmanuel Saez and Gabriel Zucman using better data and more sophisticated methods show an increase in wealth concentration in the United States. I am not aware of comparable analysis for the United Kingdom to confirm or refute those claims made by Giles.

Many of Giles’ other critiques highlight the difficult choices that economists must make when working with complicated data. Some of his points about data mistakes suggest that Piketty may need to update a few figures and data points in a second edition of his book. Just like many textbooks (and other publications such as the Financial Times) contain errata, so will this 700-page empirical work. Some of Giles’ points do require a more thorough response from Piketty than the one he has already given, especially Giles’ claim that Piketty sometimes cherry-picks his data. None, however, seem likely to ultimately undermine Piketty’s basic empirical insight—that across time, societies tend toward an ever-increasing consolidation of wealth in the hands of the few.

In this light, Giles’ critique of Piketty’s research jumps the shark when he compares it to the Reinhart and Rogoff spreadsheet scandal, where their widely-cited debt study was used to push fiscal austerity policies for debt-burdened economies was debunked when a graduate student got his hands on their spreadsheets and discovered that they had made a summation error that materially altered their conclusions.

Piketty may have made a few errors, and we certainly look forward to future work that corrects any errata and more deeply works through any questionable data decisions. He also did not always provide enough detail about his harmonization methods—his explanation for the wealth data can be found here on pages 56-62—so providing more details in the future would be wise. But the sum total of his work and that of others suggests that the basic insights that have made Piketty’s “Capital in the 21st Century” a phenomenon in economics are solid.

No piece of research is above reproach and, particularly given the volume of work in the book, this research will require extensive study for confirmation. That said, the critique by Giles has much less than meets the eye.

I’m not alone in drawing this conclusion after a careful look at Giles’ points and Piketty’s data. Some of the best analysis on this that I read over the long weekend includes:

  • Neil Irwin and Justin Wolfers at The Upshot both do great work putting the debate in context.
  • Mike Konczal also does a good job responding to the points individually.
  • For more, I also recommend looking at the Twitter dialogue between Gabriel Zucman (@gabriel_zucman) and Scott Winship (@swinshi) on Saturday.
  • Roosevelt Institute: In “The FT Gets Piketty’s Capital Argument Wrong,” the think tank notes that Giles writes, “The central theme of Prof Piketty’s work is that wealth inequalities are heading back up to levels last seen before the first world war.” The institute says “this is incorrect, or at least badly stated. Piketty’s central theme is not that inequality of the ownership of wealth is going to skyrocket.”
  • Washington Post: A piece in Wonkblog titled “Is Piketty’s ‘Capital’ full of mistakes?” starts with this—“The nine most terrifying words in the English language for a researcher are: You made spreadsheet errors like Reinhart and Rogoff did.”  Wonkblog then concludes that “this doesn’t seem to be a Reinhart and Rogoff situation. Their Excel errors really did change their conclusions. Piketty’s don’t.
  • Bloomberg: The wire service published a piece titled “Piketty Rejects ‘Ridiculous Allegations of Data Flaws,’” in which the wire service notes that Scott Winship, a fellow at the New York-based Manhattan Institute for Policy Research, said the newspaper’s allegations aren’t ‘significant for the fundamental question of whether Piketty’s thesis is right or not.’ Bloomberg also noted that James Hamilton, an economics professor at the University of California, San Diego, said there’s ‘abundant evidence’ of widening inequality ‘from a good many sources besides Piketty.’