Over the past 40 years, the observed earnings gap between African American men and their white counterparts closed slowly but steadily. The average black employed worker earned about a quarter less than the average white employed worker with similar experience in 2010 compared to about a third less in 1970. Such enduring earnings inequality is nothing to celebrate, but at least the trend line is encouraging.

Or is it?

Those reported earnings gains among black men fail to take account of different trends in incarceration and employment, which not only skews labor market statistics but also masks the debilitating economic consequences of the mass incarceration of African American men over the past several decades. When properly accounted, there is little reason to believe that the labor market prospects for black men relative to white men have improved over the past 40 years.

Let’s start with the “prison boom,” or more precisely, the trend in incarceration rates, which have more than doubled over the past 30 years. Today, more than 2.3 million people are locked up in local jails, state prisons, or federal prisons. Although this prison boom affected all racial and ethnic groups, it has had a disproportionate effect on African American men. In the 2010 Census, almost one in ten African American men ages 20 to 34 were institutionalized, while the corresponding rate for white men was only about one in fifty.

Further, on any given day in 2010, about one third of African American men who were high school drop-outs between the ages 20 and 34 lived in jails, prisons, mental health institutions, or nursing homes, and there is good reason to believe that the fraction in prison or jail exceeded the employment rate for this group. Of course, this is just at any given point in time. The fraction incarcerated at some point in life is even higher—about two-thirds by age 34, according to a recent book by sociologist Becky Pettit from the University of Washington.

While these statistics are not new to criminologists, they imply that a growing share of the U.S. population is missing from the government’s main source of information about the labor market: the Current Population Survey. The CPS only covers the non-institutionalized population, but the federal government uses it to calculate important measures of labor market outcomes such as wages, labor force participation, and unemployment rates as well as official poverty statistics, including the Census Bureau’s new Supplemental Poverty Measure.

As the missing data problem has become more severe, these measures have become more distorted, in particular with respect to trends in racial inequality. In a recent NBER working paper, economist Derek Neal and I argue that since 1970, the economic progress of African American men relative to white men has been quite anemic. We reach this conclusion by properly accounting for the growth of the prison population over this period, and hence the misleading picture derived from average labor market earnings for employed workers.

In our paper, we treat the median weekly wages of men in their prime working years as a proxy for their overall labor market prospects. Among the employed, the ratio of median weekly wages for African Americans relative to whites increased steadily from around 65 percent in 1970 to well over 75 percent in 2010, the most recent census year. Yet this statistic substantially overstates the recent relative progress of African Americans for two reasons. First, employment rates for working age men have declined much more among blacks than among whites, and growing numbers among the non-employed are incarcerated. Second, earnings prospects are now and have always been worse for those who are not currently employed.

Thus, we estimate what we call median potential wages for blacks and whites, making adjustments for changes in the numbers of non-employed and institutionalized persons over time. We find that the labor market prospects of black men relative to white men have not improved over the past 40 years. There have been slight ups and downs (with some noteworthy progress in the 1990s), but in 2010, the ratios of median potential wages among African American men to the median potential wages of their white peers were roughly at 1970 levels, across groups with different levels of experience.

Black-white economic convergence, then, has come to a halt after substantial progress throughout most of the past century, as documented in a seminal 1989 study by James Smith of the Rand Corporation and Finis Welch, then an economics professor at the University of California-Los Angeles. While it is difficult to quantify the exact contribution of mass incarceration to the lack of black relative progress in recent decades, some studies do find suggestive evidence that incarceration harms employment and earnings opportunities long after prisoners serve their time.

Our results concerning stalled relative progress for African American men are particularly noteworthy because we are also able to demonstrate that the prison boom was primarily the result of policy choices. At first glance, one might suspect that rising incarceration rates reflect increased criminal activity as a consequence of deteriorating legal labor market opportunities for people with little formal education. But the boom in crime is long over. Criminal activity and arrests for all non-drug-related offenses peaked in the early to mid-1990s and have been on the decline ever since. Drug-related arrests increased well into the late 2000s, but due to short average sentences, drug offenses on their own contributed relatively little to the overall boom in incarceration.

Instead, the main driver of the prison boom has been a move toward more punitive corrections policies across all offense categories, not just drug crimes. Such policies include so-called Truth-in-Sentencing laws, “Three Strikes” policies, and mandatory minimum sentences. As a result, arrested alleged offenders in each violent crime category are now at least twice as likely to spend more than five years in prison then they were in the mid-1980s. The pattern is perhaps even more striking for non-violent offenses: conditional on arrest, the probability of any given sentence length has increased—often by a factor of two or more.

Overall, an alleged offender in the 2000s can expect to spend about twice as long in prison as in the 1980s, conditional on the severity of the crime. Of course, not all of this shift necessarily reflects a change in policy. In particular, technological advances such as the use of DNA evidence may have increased the probability that an alleged offender is found guilty. But these new investigative methods have been adopted by other developed countries—and none of them have experienced changes in distributions of time-served among offenders that are even remotely similar to those we have seen in the United States. Therefore, it is hard to avoid the conclusion that sentencing and parole release policies have played the leading role. We estimate that the overall shift toward more punitive corrections policies probably accounts for between 70 and 85 percent of the growth in incarceration rates since 1985.

There is now substantial evidence that the boom in incarceration had an adverse effect on the relative economic progress of African American men, and that this prison boom was primarily a policy choice and not a result of deteriorating labor market conditions. Supporters of tougher corrections policies may argue that these policies have contributed to the decline in criminal activity over the past two decades. But even with our study, the costs of that crime reduction have not been fully counted and may not have been fully realized yet.

Some recent studies provide evidence that more punitive treatment of first offenders increases recidivism rates and prolongs criminal careers, and recent trends in the demographic characteristics of prisoners are consistent with this claim. Crime in our country was once almost exclusively a young man’s game, but arrest rates and prison admission rates for men ages 40 to 49 have risen disproportionately in recent years. In addition, we have not yet seen how policies that promote mass incarceration within particular communities will impact future generations from those communities.

—Armin Rick is Assistant Professor of Economics at Cornell University’s Johnson School of Management. His collaborator on this project is Professor Derek Neal of the University of Chicago Economics Department. Their paper, “The Prison Boom and the Lack of Black Progress after Smith and Welch,” was recently released by the National Bureau of Economic Research.