Hoisted from the Archives: Night Thoughts on Dynamic Scoring

Should-Read: I say it is time to promote this guy to Admiral: AdmiralPAYGO!: Ed Lorenzen: @CaptainPAYGO on Twitter: “The Treasury Department dynamic ‘analysis’ of tax reform makes a mockery of dynamic analysis and does a disservice to those who advocate for serious dynamic estimates https://t.co/PudiRrQzu1…”

Brad DeLong: @de1ong on Twitter: This is a surprise? Static analysis was always about making a bias-variance tradeoff: A static analysis would be biased, but have lower mean-squared error because the “dynamic” terms would inevitably be overwhelmingly large-magnitude political-partisan-lobbyist-ideologue noise:

Hoisted from the Archives from 2015: Night Thoughts on Dynamic Scoring: Live from DuPont Circle: Last Thursday two of the smartest participants at the Brookings Panel on Economic Activity conference—Martin Feldstein and Glenn Hubbard—claimed marvelous things from the enactment of JEB!’s proposed tax cuts and his regulatory reform program. They claimed:

  • that it would boost economic growth over the next ten years by 0.5%/year (for the tax cuts) plus an additional 0.3%/year (for the regulatory reforms).

  • that it would leave the U.S. economy in ten years producing $840 billion more in annual GDP than in their baseline.

  • that over the next ten years faster growth would produce an average of 210 billion dollars a year of additional revenue to offset more than half of the 340 billion dollars a year ‘static’ revenue lost from the tax cuts

  • that the net cost to the Treasury would thus be not 340 but 130 billion dollars a year.

  • that in the tenth year—fiscal 2027—the 400 billion dollar ′static′ cost of the tax cuts would be outweighed by a 420 billion dollar faster-growth revenue gain.

The problem is that if I were doing the numbers I would reverse the sign…

I would say that:

  • On net, deregulatory programs have been very costly to the U.S. economy in unpredictable ways
    witness the subprime boom and the financial crisis.
  • The incentive effects would tend to push up growth by only 0.1%/year
  • That would be more than offset by a drag on the economy that would vary depending on how the tax cuts were financed:
    • If they were financed by issuing debt, I would ballpark the drag at -0.2%/year.
    • If they were financed by cutting public investment, I would ballpark the drag at -0.4%/year.
    • If they were financed by cutting government programs, there might be a small boost to growth–0.1%/year–but any societal welfare benefit-cost calculation would conclude that the growth gain was not worth the cost.

And there is substantial evidence that I am right:

  • You cannot find a boost to potential output growth flowing from either the Reagan or the Bush tax cuts.
  • You cannot find a drag on growth from the Obama tax increases.
  • You can find an effect of the Clinton tax increases—but it is that, thereafter, growth was faster, because the reduction in the deficit powered an investment-led recovery.
  • Over the past thirty years, the agencies that do the government’s accounting have tried to reduce their vulnerability to the imposition of a rosy scenario by their political masters by claiming as a matter of principle that they do not calculate positive growth impacts of policies. This is clearly the wrong thing to do—policies do affect growth rates. But is overestimating growth effects in a way that pleases one’s political masters a less-wrong thing? There is a bias-variance tradeoff here.

[Name Redacted] suggested at the conference that the right thing to do is probably to apply a substantial haircut to the growth-boost claims of political appointees.

The problem is that when I look at the example of ‘dynamic scoring’ that was on the table at Brookings—the 0.8%/year growth boost that I really think should be a -0.1%/year growth drag—the haircut I come up with, for Republican policy proposals at least, is 112.5%.

Yet the near-consensus of the meeting was that dynamic scoring—done properly—was a thing that estimating agencies like JCT and CBO (and Treasury OTA) should do. If there were to be a day on which the news flow was less favorable to such a consensus conclusion, I do not know what that day would have looked like.
Twenty-two years and one month ago, after an OEOB meeting I spent carrying spears for David Cutler in one of his hopeless attempts to warn certain Assistant to the President for Health Policy precisely what reception his policy proposals would get from a CBO where Doug Elmendorf piloted the health-care desk, I returned to my office at the Treasury, and one of our career economists lectured me thus about dynamic scoring:

Brad, you people come in with your exaggerated belief in the productivity benefits of public investment. And so you command us to score your policies as having a very favorable impact on the deficit. They come in with their exaggerated belief in the benefits of tax cuts. They command us to score their policies as having a very favorable impact. We cannot say we disagree with our bosses’ analytic judgments. But by holding the line and stating that we do not consider any macroeconomic effects of policies, we can at least prevent being whipsawed by this partisan rosy-scenario ratchet.

Thus I find myself worrying about this:

  • I find myself thinking of CBO Directors past and future.
  • I think of June O’Neill, talking over and over again about how her model showed substantial disemployment effects of universal health coverage, without ever letting past her lips any acknowledgement that the people whose jobs her model showed as ‘destroyed’ had in fact voted with their feet and moved to a higher utility level by quitting.
  • I find myself thinking of the persistent rumors that after Doug Elmendorf and company had wreaked their analytic wrath on Ira Magaziner, Majority Leader Mitchell had said to Bob Reischauer: ‘You are gone on January 4, 1995’.

One unintended side effect of the budget process introduced in the 1970s and the 1980s has been to give CBO and JCT great power—has given their analytic decisions the importance of the unanimous coordinated votes of twenty senators over and above the impact of their estimates on members’ minds. They have by and large shouldered that great power with great responsibility. But with great power also comes great pressure. And it is not at all clear to me that, given the magnitude of this pressure, we want extra degrees of freedom in which these organizations can respond to the pressures they are under.

Yesterday, after all, I saw estimates of the dynamic revenue impact of Jeb!’s tax proposals that varied from negative—that the reduction in national savings would outweigh any positive incentive effects—to recouping 2/3 of the static revenue loss. And I imminently expect to see an ‘estimate’ today that it will produce 4%/year real growth and thus raise revenue–perhaps from someone at Heritage, perhaps from someone at Cato, perhaps from John Cochrane. It’s opening a can of worms. Doug and Peter may think the worms are dead. I fear they are not…

Doug Elmendorf wrote:

Based on my experience as the director of CBO from January 2009 through March 2015, the principal concerns expressed about estimated macroeconomic effects of proposals apply with equal force to other aspects of budget estimates or can be addressed by CBO and JCT. In my view, including macroeconomic effects in budget estimates for certain legislative proposals would improve the accuracy of those estimates and would provide important information about the economic effects of those proposals. Moreover, if certain key conditions were satisfied, those estimates would meet the general goals of the estimating process that estimates be understandable and resistant to misinterpretation, based on a consistent and credible methodology, produced quickly enough to serve the legislative process, and prepared using the resources available to CBO and JCT.

Doug has it wrong: they do not apply “with equal force”. As we have seen today, Monday, December 11, 2017, with the Treasury tax “reform” “study”.

Mea Culpa: Confidence Proceedings Edition: I Really Do Think Dynamic Scoring Is a Bad Idea

I have committed a grave sin for the first time.

In revising and extending for publication the remarks I made during the discussion of Doug Elmendorf’s paper for the Fall 2015 issue of the Brookings Papers on Political Economy, I have not done so.

Instead, I have rewritten and compressed. The below bears little resemblance to what I did say. But it is what I should have said:

Brad DeLong noted that when he was a Treasury political appointee, one of the Treasury career staff economists lectured me him about dynamic scoring thus:

Brad, you people come in with your exaggerated belief in the productivity benefits of public investment. And so you command us to score your policies as having a very favorable impact on the deficit. They come in with their exaggerated belief in the benefits of tax cuts. They command us to score their policies as having a very favorable impact. We cannot say we disagree with our bosses’ analytic judgments. But by holding the line and stating that we do not consider any macroeconomic effects of policies, we can at least prevent being whipsawed by this partisan rosy-scenario ratchet…

He noted that being whipsawed by the partisan rosy-scenario ratchet is a serious danger, as evidenced most recently by yesterday’s the recent semi-score of the Jeb Bush tax plan by Feldstein et al. There would be an upside if appropriate real technocratic dynamic-scoring corrections were significant. But, he concluded, they mostly likely are not.

I do not know whether to be encouraged or discouraged by the fact that at my age I am still finding myself committing new categories of sin. I wonder what the appropriate penance would be?

Looking back on it, it was a very strange morning indeed. Mankiw, Feldstein, Hubbard, and Cogan–people who were at the very top of the list for the post of CBO Director in Republican majority congresses a generation ago–had just come out with a semi-score of the JEB! tax plan that used dynamic scoring in exactly the way that every reality-based technocrat had always feared that it would be used.

And yet–with the honorable exception of Bill Gale–virtually every other speaker out of the whole BPEA audience agreed that there ws no danger that dynamic scoring would ever be used in the way that every reality-based technocrat had always feared.

What was going on? I did not understand it. I do not understand it…


My previous takes on this subject:

September 14, 2015: Night Thoughts on Dynamic Scoring: Live from DuPont Circle: Last Thursday two of the smartest participants at last Friday’s Brookings Panel on Economic Activity conference–Martin Feldstein and Glenn Hubbard–claimed marvelous things from the enactment of JEB!’s proposed tax cuts and his regulatory reform program.

They claimed it would boost economic growth over the next ten years by 0.5%/year (for the tax cuts) plus an additional 0.3%/year (for the regulatory reforms).

That would leave the U.S. economy in ten years producing $840 billion more in annual GDP than in their baseline. That would mean that over the next ten years faster growth would produce an average of $210 billion a year of additional revenue to offset more than half of the $340 billion a year ‘static’ revenue lost from the tax cuts, making the net cost to the Treasury not $340 billion/year but $130 billion/year. And that would mean that in the tenth year–fiscal 2027–the $400 billion ‘static’ cost of the tax cuts in that year would be outweighed by a $420 billion faster-growth revenue gain.

The problem is that if I were doing the numbers I would reverse the sign:

  • I would say that, on net, deregulatory programs have been very costly to the U.S. economy in unpredictable ways–witness the subprime boom and the financial crisis.
  • I would say that the incentive effects would tend to push up growth by only 0.1%/year, and that would be more than offset by a drag on the economy that would vary depending on how the tax cuts were financed.
    • If they were financed by issuing debt, I would ballpark the drag at -0.2%/year.
    • If they were financed by cutting public investment, I would ballpark the drag at -0.4%/year.
    • If they were financed by cutting government programs, there might be a small boost to growth–0.1%/year–but any societal welfare benefit-cost calculation would conclude that the growth gain was not worth the cost.

And there is substantial evidence that I am right:

  • You cannot find a boost to potential output growth flowing from either the Reagan or the Bush tax cuts.
  • You cannot find a drag on growth from the Obama tax increases.
  • You can find an effect of the Clinton tax increases–but it is that, thereafter, growth was faster, because the reduction in the deficit powered an investment-led recovery.

Over the past thirty years, the agencies that do the government’s accounting have tried to reduce their vulnerability to the imposition of a rosy scenario by their political masters by claiming as a matter of principle that they do not calculate positive growth impacts of policies. This is clearly the wrong thing to do–policies do affect growth rates. But is overestimating growth effects in a way that pleases one’s political masters a less-wrong thing?

[Name Redacted] suggested at the conference that the right thing to do is probably to apply a substantial haircut to the growth-boost claims of political appointees.

The problem is that when I look at the example of ‘dynamic scoring’ that was on the table at Brookings today–the 0.8%/year growth boost that I really think should be a -0.1%/year growth drag–the haircut I come up with, for Republican policy proposals at least, is 112.5%.

Yet the near-consensus of the meeting was that dynamic scoring–done properly–was a thing that estimating agencies like JCT and CBO (and Treasury OTA) should do.

If there were to be a day less favorable to such a consensus conclusion, I do not know what that day would have looked like…


: Fall 2015 Brookings Panel on Economic Activity Weblogging: Doug Elmendorf on Dynamic Scoring: Fall 2015 BPEA 8:30 AM Fr: Twenty-two years and one month ago, after an OEOB meeting I spent carrying spears for David Cutler in one of his hopeless attempts to warn certain Assistant to the President for Health Policy precisely what reception his policy proposals would get from a CBO where Doug Elmendorf piloted the health-care desk, I returned to my office at the Treasury, and one of our career economists lectured me thus about dynamic scoring:

Brad, you people come in with your exaggerated belief in the productivity benefits of public investment. And so you command us to score your policies as having a very favorable impact on the deficit. They come in with their exaggerated belief in the benefits of tax cuts. They command us to score their policies as having a very favorable impact. We cannot say we disagree with our bosses’ analytic judgments. But by holding the line and stating that we do not consider any macroeconomic effects of policies, we can at least prevent being whipsawed by this partisan rosy-scenario ratchet.

Thus I find myself worrying about this:

  • I find myself thinking of CBO Directors past and future.
  • I think of June O’Neill, talking over and over again about how her model showed substantial disemployment effects of universal health coverage, without ever letting past her lips any acknowledgement that the people whose jobs her model showed as ‘destroyed’ had in fact voted with their feet and moved to a higher utility level by quitting.
  • I find myself thinking of the persistent rumors that after Doug Elmendorf and company had wreaked their analytic wrath on Ira Magaziner, Majority Leader Mitchell had said to Bob Reischauer: ‘You are gone on January 4, 1995’.

One unintended side effect of the budget process introduced in the 1970s and the 1980s has been to give CBO and JCT great power–has given their analytic decisions the importance of the unanimous coordinated votes of twenty senators over and above the impact of their estimates on members’ minds. They have by and large shouldered that great power with great responsibility. But with great power also comes great pressure. And it is not at all clear to me that, given the magnitude of this pressure, we want extra degrees of freedom in which these organizations can respond to the pressures they are under.

Yesterday, after all, I saw estimates of the dynamic revenue impact of Jeb!’s tax proposals that varied from negative–that the reduction in national savings would outweigh any positive incentive effects–to recouping 2/3 of the static revenue loss. And I imminently expect to see an ‘estimate’ today that it will produce 4%/year real growth and thus raise revenue–perhaps from someone at Heritage, perhaps from someone at Cato, perhaps from John Cochrane. It’s opening a can of worms. Doug and Peter may think the worms are dead. I fear they are not…


Doug Elmendorf: ‘Based on my experience as the director of CBO from January 2009 through March 2015…

…the principal concerns expressed about estimated macroeconomic effects of proposals apply with equal force to other aspects of budget estimates or can be addressed by CBO and JCT. In my view, including macroeconomic effects in budget estimates for certain legislative proposals would improve the accuracy of those estimates and would provide important information about the economic effects of those proposals. Moreover, if certain key conditions were satisfied, those estimates would meet the general goals of the estimating process that estimates be understandable and resistant to misinterpretation, based on a consistent and credible methodology, produced quickly enough to serve the legislative process, and prepared using the resources available to CBO and JCT.