Thursday, June 09, 2011

GRE and SAT validity

Note Added in response to 2020 Twitter mob attack which attempts to misrepresent my views: Every faculty member involved in admissions has a responsibility to understand the science behind standardized tests. The subject is heavily politicized, but the scientific results are very clear and based on very large datasets -- e.g., decades of student records in large university systems.

I refer you to the detailed 2020 report prepared by a special committee of faculty assigned this task by the University of California system. This committee recommended continued use of the SAT in admissions.

https://infoproc.blogspot.com/2020/02/report-of-university-of-california.html


###################################



If you are a professor at a research university you have probably spent time on graduate admissions. How good is the GRE as an indicator of candidate quality? Is the subject score more useful than the general score? What about relative to undergraduate GPA? Similar questions apply to the SAT and undergraduate admissions.

In both cases the answer is that standardized tests have roughly as much predictive power as GPA (SAT is about as powerful as HS GPA; GRE similar to undergraduate GPA). Not bad for a brief test! When these factors are combined the overall predictive power is increased. My opinion is that standardized tests load more heavily on cognitive ability and less on conscientiousness relative to course grades, hence the non-redundant information in the two measures.



This is a meta-analysis of GRE validity by Kuncel and collaborators. Here are slides from a related talk. See also this blog discussion (social scientists) of the GRE.

I first came across Kuncel's work when I was investigating SAT validity using U Oregon undergraduate data. Unlike many of the people working on this topic, Kuncel is neither an idiot nor an ideologically motivated faker. For example, he understands issues like restriction of range, variation in course and subject difficulty, and self-selection effects. See these slides from his talk on SAT validity. Video and audio here (bonus: Thomas Espenshade; Kuncel starts at 27 minutes in).




Validities for various standardized tests:

Blog Archive

Labels