I refer you to the detailed 2020 report prepared by a special committee of faculty assigned this task by the University of California system. This committee recommended continued use of the SAT in admissions.
https://infoproc.blogspot.com/2020/02/report-of-university-of-california.html
###################################
If you are a professor at a research university you have probably spent time on graduate admissions. How good is the GRE as an indicator of candidate quality? Is the subject score more useful than the general score? What about relative to undergraduate GPA? Similar questions apply to the SAT and undergraduate admissions.
In both cases the answer is that standardized tests have roughly as much predictive power as GPA (SAT is about as powerful as HS GPA; GRE similar to undergraduate GPA). Not bad for a brief test! When these factors are combined the overall predictive power is increased. My opinion is that standardized tests load more heavily on cognitive ability and less on conscientiousness relative to course grades, hence the non-redundant information in the two measures.
This is a meta-analysis of GRE validity by Kuncel and collaborators. Here are slides from a related talk. See also this blog discussion (social scientists) of the GRE.
I first came across Kuncel's work when I was investigating SAT validity using U Oregon undergraduate data. Unlike many of the people working on this topic, Kuncel is neither an idiot nor an ideologically motivated faker. For example, he understands issues like restriction of range, variation in course and subject difficulty, and self-selection effects. See these slides from his talk on SAT validity. Video and audio here (bonus: Thomas Espenshade; Kuncel starts at 27 minutes in).
Validities for various standardized tests:
So why do most science&Engineering Phd programs rely more on research experience and recommendations,and not GRE scores?
ReplyDeleteBecause the candidates are above the range where the GRE-Q (which is the only GRE part relevant to STEM) can distinguish among them.
ReplyDeleteBecause those programs must distinguish among candidates that are above the level that the GRE-Q (the only GRE section that matters for STEM) can separate.
ReplyDeleteBecause those programs must distinguish among candidates that are above the level that the GRE-Q (the only GRE section that matters for STEM) can separate.
ReplyDelete[S.Hsu -- please remove the duplicate (and this message). Sorry.]
ReplyDeleteFollowing that logic you'll have to somehow subtract out the effect of institutional prestige on career outcomes. Better test scores gets you into a better school.
ReplyDeleteBecause:
ReplyDelete1) they don't know what they're doing
2) STEM academics are still academics and far left of center just not as far as non-STEM
3) they themselves may not have have scored that high.
Grad admission people always tell me that the important criteria are (in order),
ReplyDelete1. Proven research experiences
2. Undergrad recommendations
3. Some proof of ability to write competent English
4. Undergrad GPA
They claim not to even know why GRE scores are requested since they don't count.
"works to end the misuses and flaws of standardized testing and to ensure that evaluation of students, teachers and schools is fair, open, valid and educationally beneficial"
ReplyDeleteI suppose Steve might (fairly) paraphrase this: "works to assure that everyone gets the same score or at least that there are no big differences between ethicities or the sexes".
Maybe. In our case, the median score of applicants who make our short list is 800, so the test isn't all that helpful to us. And if we limited ourselves to 800s, we wouldn't eliminate many applicants. Maybe it means the test is useful, but it's value in the right tail is pretty limited: we look at things like what courses applicants have taken, whether they have research experience. What we really want is passion, but that's hard to measure.
ReplyDeleteYou look only at the quantitative section? Did you look at the analytic section (logic problems) in the old days? You don't look at the subject test?
ReplyDeleteIs that the GMAT or the GRE? I thought perfect GMATs were quite rare, like 50 a year.
ReplyDeleteSpeaking of ideological fakers, SJ Gould has been badly exposed.
ReplyDeletehttp://johnhawks.net/weblog/topics/meta/gould-morton-lewis-2011.html
It's hard to ensure there being no big differences between ethicities, when the unethical people cheat.
ReplyDeleteUmm so is 0.37 considered good? Confused there!
ReplyDeleteSee pictures here or in my data mining paper to get an idea of what correlation of .37 looks like. For social science it's considered a relatively strong correlation.
ReplyDeletehttp://en.wikipedia.org/wiki/Correlation_and_dependence
Why then cant they use the undergraduate equivalent of an academic index,finding a composite of all the three sections with allowance for those who come from non english speaking countries?Wont that be a better predictor than research experince?
ReplyDeleteI wonder how much the hormone testosterone affects these types of scores: http://www.nytimes.com/2000/04/02/magazine/the-he-hormone.html
ReplyDeleteThere are times when GPA and SAT/GRE conflict; in such cases, which is a better predictor of success?
ReplyDeleteie, what happens when you have low GPA and high SAT/GRE (or vice versa)?
Gre exam is very important so students must keep in mind
ReplyDeletethat only hardworking can help for passing it
gre practice
Gre exam is very important so
ReplyDeletestudents must keep in mind that only hardworking can help for passing it
gre score
Another interesting thing about the GRE. Most colleges don't care! When I called one of my prosepective graduate schools they said that the GRE is usually the last thing they look at when deciding whether or not to accept you
ReplyDeletegre practice study
I have been obsessed with IQ analysis for the past 1 year. I think using SAT or GRE score for IQ measurement is total nonsense. The math section is ridiculously easy and measures how well you have been trained in specific techniques you learn in high school. India and China perform exceedingly well since engineering is given extreme focus in both the countries and hence most of students get trained very well in math techniques.
ReplyDeleteAlso Indian and Chinese students put in a lot of hours getting trained in those specific techniques. Americans on the other hand enter all kinds of fields like english, psychology, business, history and so on and also do not tend to put much emphasis on math skills and also do not put much emphasis on education in general. Chinese Students have an average GRE score of 780. Does it mean that all the students coming in have an IQ of 145 (3SD above the mean)? I have seen many of my indian friends (comsci grads) who cannot even write a single program and who fail out of most of their college courses score 740-750 without any prep. The SAT and GRE scores should not be taken seriously as told by prominent thinkers like Taleb.
ReplyDeleteI think the SAT english section is a far more better indicator of general intelligence as compared to the verbal section (of course only for people having a strong english background).
ReplyDeleteBoth the GRE scores I have mentioned below (780,740) are the math scores and I got the data from the ETS website itself. I have even seen your slides and like the BGI research program but I think taking the GRE or SAT scores is a big mistake. The math score should not be a factor at all for the input set of students and the verbal score cannot be a factor because of the cultural differences among students. I have seen many Indian Phd students struggling to score even 400 in verbal and I have seen all the exceptional Chinese students ( they dominate the grad college in all the best universities) failing their english courses and many Chinese lectures who have published hundreds of papers still have trouble string a sentence together in their college lectures.
ReplyDeleteThanks for sharing such a nice information . you can also visit GRE Verbal Course
ReplyDeleteThanks for sharing such a nice information . you can also visit. GRE Verbal Course
ReplyDeleteThanks for sharing such a nice information .you can also visit GRE Verbal Course
ReplyDelete