The 2015 graduating class of Texas Southmost College in Brownsville, Texas.

Education is supposed to be a silver bullet. A great equalizer when it comes to income inequality, and a boost to flagging wages and incomes. But higher levels of education alone may not be able to reduce income inequality. While increasing the share of so-called “marginal students” attending college would have a positive return, it’s unclear just how large that return would be. And the return to higher education may also vary by how much a student’s parents earn.

In a column for the Brookings Institution, Brad Hershbein of the W.E. Upjohn Institute for Employment Research lays out some surprising results on the lifetime earnings of U.S. workers from different family backgrounds. Specifically, he and his colleagues looked at the lifetime earnings measured by the Panel Study of Income Dynamics for workers who grew up in families with incomes above and below 185 percent of the federal poverty level (about $45,000 for a family of four in 2016).

Unsurprisingly, workers with college degrees earn more than workers with just high school degrees, regardless of family background. College grads from families with incomes below 185 percent of the poverty line make more than than high school grads from the same income background, and the same for those from families well above the poverty threshold. But the boost from a college degree varies by background.

College grads from lower-income backgrounds (specifically families with incomes below the 185 percent threshold) earn 91 percent more over the course of their career than high school grads from the same background. But college grads coming from families that were well above poverty (or families earning above the 185 percent threshold) make 162 percent more than high school grads from the same background. So the return on a college degree is significantly higher for a student who grew up far away from poverty.

In other words, if you were to interpret these results as the causal effect of a college degree, a college degree boosts the earnings of students from low-income backgrounds, but college grads from higher-income backgrounds pull away from them as they get a larger boost. Incomes for both groups go up, but inequality rises.

Hershbein notes that he and his colleagues are now trying to dig into the reasons for this differential rate of return on college and making sure the trend holds up in other data sources. If the trend does hold up, understanding the causes will be incredibly important. If we want higher education to be a growth-enhancing and equalizing policy tool, we have to understand how to make sure that students from low-income backgrounds get the same—or perhaps even greater—rate of return.