Monday, January 26, 2009

Horsepower matters; psychometrics works

Can psychometrics separate the top .1 percent from the top 1 percent in ability? Yes: SAT-M quartile within top 1 percent predicts future scientific success, even when the testing is done at age 13. The top quartile clearly outperforms the lower quartiles. These results strongly refute the "IQ above 120 doesn't matter" claim, at least in fields like science and engineering; everyone in this sample is above 120 and the top quartile are at the 1 in 10,000 level. The data comes from the Study of Mathematically Precocious Youth (SMPY), a planned 50-year longitudinal study of intellectual talent. For more, see papers here. (Thanks to Bob Williams for reminding me about this data.)

Ability Differences Among People Who Have Commensurate Degrees Matter for Scientific Creativity

Gregory Park, David Lubinski, and Camilla P. Benbow

Vanderbilt University

ABSTRACT—A sample of 1,586 intellectually talented adolescents (top 1%) were assessed on the math portion of the SAT by age 13 and tracked for more than 25 years. Patents and scientific publications were used as criteria for scientific and technological accomplishment. Participants were categorized according to whether their terminal degree was a bachelor’s, master’s, or doctorate degree, and within these degree groupings, the proportion of participants with at least one patent or scientific publication in adulthood increased as a function of this early SAT assessment. Information about individual differences in cognitive ability (even when measured in early adolescence) can predict differential creative potential in science and technology within populations that have advanced educational degrees.


Anonymous said...

Why doesn't it work on the people who make the tests? As PhDs go psychology PhDs are among the dumbest.

Anonymous said...

Did the higher scorers go to better schools? This factor has to be subtracted.

It is interesting that the top 1% of 13 yr olds scored only in the 500s, and that the PhD, B difference was only 46 pts.

SAT-M at age 16 or 17 must be available. How predictive is it? How predictive combined with SAT-M at 13?

But I'm a dummy 740 old SAT-M at 17.

hcl said...

I wish to point out that by definition, to make an invention in any given area (sub-area), one must be smarter than the last guy, let's say n+1 IQ points where the smartest previous guy's IQ = n.

Carson C. Chow said...

What I don't understand is even among the top quartile less than 50% had published one article and for the whole group it was less than 30%. That seems odd to me. They might be missing publications, especially in humanities where the currency is books rather than publications. Also, it is quite astonishing that Google Scholar is counted as a legitimate research tool and the final word on publications. They could have at least gone to the library!

Steve Hsu said...

It's plausible to me that a 13 year old who scores as high as the age 17 average (SAT-M = 500) is in the top 1% of their age cohort.

If you look at some of the other SMPY papers you can find what the age 17 SAT averages of their sample were -- pretty high as you can imagine. The precise correlation I don't know but the problem is the SAT-M has a very low ceiling. It is only useful for looking at the high end if you use it on 13 year olds! (Modal SAT-M score at Caltech is 800, I believe.)

Carson: yes, Google Scholar probably misses lots of publications, and might even have some biases, like missing papers in certain fields in particular. But I suspect the trend is correct, and the patent data is probably OK.

How is Malcolm Gladwell going to explain this? A simple test taken at age 13 strongly differentiates people even within the top 1%.

Anonymous said...

My scores on the GRE I took in 1992 are as follows:

Math 790
Verbal 790
Logic or whatever it was called 740

Does anyone know where I can find a link to the percentiles (individual and total)?


Anonymous said...

How is Malcolm Gladwell going to explain this?

He won't. He doesn't give a damn.

Propagandists of all types know that in repeating a "lie" often enough, people will believe it.

(Classic "advertising 101" in the Josef Goebbels school of propaganda).

Anonymous said...

To anonymous 10:25.

After I got a triple 800 in 1999, I called ETS to ask them your question. I was told they had no such statistics. "Only for the subtests", they told me.

Modal SAT-M score at Caltech is 800,I believe. I don't think that was true when I applied.

The SAT-M "has a really low ceiling"? I'm sure when I took the test less that .33% scored 800. The GRE quantiative ceiling is very low. 800 is only the 97th percentile.

Gladwell is just a propagandist, but what he said might be reinterpreted less literally to mean something Jensen himself has said, namely, with specific attaiments there is a g threshold.
It wasn't that high for Feynman.

"But, but Feynman was given a low ceiling test at too young an age to know,..."


G said...

As anonymous 5:19 said, couldn't this be explained in principle by a differential in the quality of education, both before and after the age 13 test?

I'm not saying that is the explanation, merely that I can't see that they have excluded that possibility.

Anonymous said...

Wonder if anybody has done similar studies on asian test taking data.

Many asian countries have a culture of "test taking" on steroids. In some countries, it is so extreme that "cram schools" designed for test preparation are very common.

David said...

My 2001 GRE scores were 790-M, 710-A, 680-V, which puts me right about 3SD out, in the 1 in 600-ish range. Of the people that I've met that impressed me with how smart they are, I usually thought, "Wow, they are MUCH smarter than me." So, this result isn't so surprising.

Anonymous said...

Of the people that I've met that impressed me with how smart they are, I usually thought, "Wow, they are MUCH smarter than me."

How do you know that these individuals were not just bluffing and/or putting on a very convincing act? A good con-artist can fool a lot of "smart" people.

Anonymous said...

My 2001 GRE scores were 790-M, 710-A, 680-V, which puts me right about 3SD out, in the 1 in 600-ish range.

Where did you get the stats for the aggregate?

David said...

Re: stats - I looked up GRE->IQ equivalency, and used that percentile. It's a WAG.

Re: phonies - I'm not sure if I should be insulted by the quotes around smart. Regardless, I refer you to Steve.

Anonymous said...

On a similar topic, it would be interesting to see studies measuring the intelligence of con artists, fraudsters, mafia, and other professional criminals.

Though one can imagine such a study being fraught with problems, such as the cons conning the shrink and/or "gaming" the data. There's also the problem of determining who's a con and finding ones who have not yet been caught and arrested.

Albert said...

Now, everyone who scored highly on a standardized test is going to state that this is ridiculous but then no one on here has really advanced humanity any measurable amount. Sorry to destroy your egos. They appear to be extremely inflated from the responses.

Creativity is substantially more important than reasoning. Most standardized tests measure reasoning skills. In other words, qualitative factors play a substantial role in substantial progress made by humanity. Creativity cannot be measured by a test because everyone's creativity is truly unique. With Einstein it was visualization and way of thinking not measurable by any standardized test. Why? Because measuring creativity is limited by the creativity of those designing the test.

I have patents to my name, have worked in a field of science that classifies me as probably in the very top of intelligence yet I realize I will truly never impact society as the great names in our past for the very reason that they had a gift that was simply not quantifiable.

Let the critical comments from the egotistical begin. lol.

Anonymous said...


Most people have fragile egos, and are willing to do anything to preserve and maintain it.

High intelligence scores on tests is one of many ways to do so. For people who are not "brainy" types, one can maintain their egos through other ways such as:

- maintaining a highly desirable body image to the opposite sex
- how many sexual conquests they have
- the amount of money they make and have
- their "coolness" factor amongst a particular peer group
- how many other people they have ripped off
- how many people they have physically assaulted and sent to the hospital and/or to the grave
- how "tough" they are in appearance
- how large the amount of alcohol they can drink without falling and stumbling
- etc ...

Albert said...

It seems those huge egos are pretty quiet. Maybe they realize they aren't quite as intellectually superior as they'd like to be. In fact, the discussion actually reminds me of the prior discussions by scientists trying to invalidate the equality of humanity.

The reality is smart or not smart is that we are all human. And, focus on being smarter than someone else denigrates the freedom and purity of science.

Anonymous said...

To Albert.

IQ is like height in basketball. It doesn't make you great, but it helps more than any other factor.

Anonymous said...

reply to Albert's 12:17 PM comment

The smart egotistical people with a superiority complex on this blog are intelligent enough to know that replying to a troll such as yourself, is a complete waste of their time and energy. They have bigger fish to fry than you.

On the same side of the coin, a similar thing can be said about Malcolm Gladwell. He has bigger fish to fry than in dealing with the critical comments of "insignificant" people online (in his own mind). He's just as egotistical and arrogant as any other person who thinks the world revolves only around themselves.

Albert said...

Nice last remark. Ah, the attempts at marginalization. Of course, you have bigger fish to fry than me. I bruised your fragile ego. An ego that somehow finds solace in some attempted rationalization of intellectual superiority. How predictable. Denigrating someone who challenges those who would vote themselves elitists were they able to. Of course, at society's expense. It's an age old trick used by insecure people. One who was quite good at this that we all recognize was Adolph Hitler.

Anonymous said...

The art of trolling.

The world revolves around one's self.

hcl said...

Who's Albert?

Stevie Mac said...

"no one on here has really advanced humanity any measurable amount".

Probably not but do you think you would be looking at a computer screen if it wasn't for very high iq people, or there would be a light bulb in that room? Pretty much all the modern technology and science comes from high iq people. Newton, Einstein, Darwin, any great scientist you can think of probably had a very high IQ. Creativity may have played a substantial role too but somebody with average cognitive ability would never be able to advance physics, no matter how creative they are.

I have quite a high iq but there is nothing special about me. The only reason I would want a very high iq is not to boost my ego but to aid my quest for knowledge and enable me to attain a greater intellectual understanding of the universe.

Blog Archive