Value-Added College Rankings

New report from Brookings estimates value added (in terms of economic success) by university, controlling for input factors such as student quality and family income. This is just the first step toward outcomes-driven rankings of universities that will be far more useful than the existing rankings, which are largely based on prestige. Brief summary.

See also Defining Merit.

Brookings: The choice of whether and where to attend college is among the most important investment decisions individuals and families make, yet people know little about how institutions of higher learning compare along important dimensions of quality. This is especially true for the nearly 5,000 colleges granting credentials of two years or fewer, which together graduate nearly 2 million students annually, or about 39 percent of all postsecondary graduates. Moreover, popular rankings of college quality, such as those produced by U.S. News, Forbes, and Money, focus only on a small fraction of the nation’s four-year colleges and tend to reward highly selective institutions over those that contribute the most to student success.

Drawing on a variety of government and private data sources, this report presents a provisional analysis of college value-added with respect to the economic success of the college’s graduates, measured by the incomes graduates earn, the occupations in which they work, and their loan repayment rates. This is not an attempt to measure how much alumni earnings increase compared to forgoing a postsecondary education. Rather, as defined here, a college’s value-added measures the difference between actual alumni outcomes (like salaries) and predicted outcomes for institutions with similar characteristics and students. Value-added, in this sense, captures the benefits that accrue from both measurable aspects of college quality, such as graduation rates and the market value of the skills a college teaches, as well as unmeasurable “x factors,” like exceptional leadership or teaching, that contribute to student success.

While imperfect, the value-added measures introduced here improve on conventional rankings in several ways. They are available for a much larger number of postsecondary institutions; they focus on the factors that best predict objectively measured student economic outcomes; and their goal is to isolate the effect colleges themselves have on those outcomes, above and beyond what students’ backgrounds would predict.

Using a variety of private and public data sources, this analysis finds that:

Graduates of some colleges enjoy much more economic success than their characteristics at time of admission would suggest. Colleges with high value-added in terms of alumni earnings include not only nationally recognized universities such as Cal Tech, MIT, and Stanford, but also less well-known institutions such as Rose-Hulman Institute of Technology in Indiana, Colgate in upstate New York, and Carleton College in Minnesota. Two-year colleges with high-value added scores include the New Hampshire Technical Institute, Lee College near Houston, and Pearl River Community College in Mississippi. ...

On the quality of PayScale compensation figures used in the analysis:

...There are a number of ways one can assess whether or not PayScale accurately captures the earnings of graduates—or whether the sample is statistically biased by the voluntary nature of its data collection.

Broadly, PayScale earnings by major for U.S. residents with bachelor’s degrees can be compared to similar data from the ACS, which annually samples 1 percent of the U.S. population.30 The correlation between the two is what matters most for this analysis, since value-added calculations are based on relative differences between predicted and actual earnings.

The correlation between bachelor’s degree holders on PayScale and median salaries by major for workers in the labor force from the Census Bureau is 0.85 across 158 majors matched between the two databases.

Effects of student ability and family SES:

For each of the three post-attendance outcomes measured here—mid-career salary, loan repayment rate, and occupational earnings power—student test scores, math scores in particular, are highly correlated: 0.76 for mid-career salaries and 0.69 for student loan repayment and occupational earnings power (Figure 3). Other student characteristics, such as the percentage receiving Pell grants, also correlate highly with these outcomes, though not as highly as test scores.

Go Beavers! Note Caltech grads are also much more likely to win Nobel Prizes and be elected to the National Academy of Science or Engineering than graduates of any other university. Claims that a technical education bestows narrow technical skill without conveying deep ideas and teaching critical thinking are silly.

... Alumni from Cal Tech list the highest-value skills on their LinkedIn profiles (Table 3); their skills include algorithm development, machine learning, Python, C++, and startups (that is, starting a new business). Cal Tech is followed closely by Harvey Mudd and MIT. Babson College, also in the top 10, focuses on business rather than science; its course offerings teach many quantitative skills relevant for business-oriented STEM careers. Many graduates from the Air Force Academy are prepared for high-paying engineering jobs in the military and at large defense contractors. They list skills like aerospace and project planning.

Amusingly, you can see by looking at the tables of regression results that Asian share of enrollment is strongly positively correlated with mid-career earnings, whereas female share is negatively correlated. This is not surprising, given that STEM skills are big drivers of compensation.

Colgate's strong performance (and probably that of Carleton and some others) cannot be explained in terms of STEM skills -- see discussion and figure, on page 16.