This seems like a trivial point (as are most things, when explained clearly), however, it still eludes the vast majority. For example, in the Atlantic article I linked to in the earlier post Creative Minds, the neuroscientist professor who studies creative genius misunderstands the implications of the Terman study. She repeats the common claim that Terman's study fails to support the importance of high cognitive ability to "genius"-level achievement: none of the Termites won a Nobel prize, whereas Shockley and Alvarez, who narrowly missed the (verbally loaded) Stanford-Binet cut for the study, each won for work in experimental physics. But luck, drive, creativity, and other factors, all at least somewhat independent of intelligence, influence success in science. Combine this with the fact that there are exponentially more people a bit below the Terman cut than above it, and Terman's results do little more than confirm that cognitive ability is positively but not perfectly correlated with creative output.
In the SMPY study probability of having published a literary work or earned a patent was increasing with ability even within the top 1%. The "IQ over 120 doesn't matter" meme falls apart if one measures individual likelihood of success, as opposed to the total number of individuals at, e.g., IQ 120 vs IQ 145, who have achieved some milestone.
It is plausible that, e.g., among top execs or scientists or engineers there are roughly equal numbers of IQ 120 and IQ 145 individuals (the actual numbers could vary depending on how the groups are defined). But the base population of the former group is 100 times that of the latter! (IQ 120 is about top 10% and IQ 145 is roughly top 0.1% in the population.) This means, e.g., that the probability that an IQ 145 person becomes a top scientist could be ~100x higher than for an IQ 120 person.
This topic came up last night in Hong Kong, at dinner with two hedge funders (Caltech/MIT guys with PhDs) who have had long careers in finance. Both observed that 20 years ago it was nearly impossible to predict which of their colleagues and peers would go on to make vast fortunes, as opposed to becoming merely rich.
.JPG)
Few Muslims are international terrorists, but most international terrorists are Muslims. Are we talking probability functions, or sets?
ReplyDelete"e.g., IQ 120 vs IQ 145 who have achieved some milestone. There are 100 times more of the former than then the latter!" mixup
ReplyDeleteExcellent post. Over the years I've read every major IQ blogger on the internet, and none of them have made as much sense as you! You're one of those rare people who doesn't suffer from bounded cognition, .
ReplyDeleteNot only should we not expect the highest IQ person to be the most "successful". but we shouldn't even expect the highest IQ person to be the most intelligent, given the imperfect correlation between IQ and g or intelligence.
Given the 0.65 correlation between IQ and academic success (whether defined as grades range corrected or years of education), we should expect one in a million academic achievers (Nobel prize winners) to have about a one in 2000 level IQ,
http://brainsize.wordpress.com/2014/07/12/the-iqs-of-academic-elites/
Conversely, one in a million IQ should perform at the one in 2000 level academically (i.e. maybe get into Harvard).
We should not expect anything spectacular from Terman type intellects (one in 100?) on average (a college degree, but not much beyond that).
It would be interesting to calculate likelihoods for the X (e.g. height, IQ) of the best Y (e.g. basketball player, wealthiest) for a given correlation. I wonder what the MLE would be and how it would compare to Bill Gates' IQ (since we can actually make a decent estimate given his SAT score). It would probably be even better to look at the top n instead.
ReplyDeleteThe shape of that likelihood curve would be quite interesting as well. I would also like to see likelihoods for in top a% of Y given in top b% of X or at c% of X.
One nit. I think your last sentence is misleading. That may be true for people AT the Terman cutoff, but I think the phrase "Terman type intellects" more accurately means ABOVE the Terman cutoff. I think we should expect exceptional (don't know about spectacular, depends on population size) things from that group. As an aside I wish someone could replicate the Terman experiment with an IQ test that was less V dominated (did Shockley or Alvarez take any other tests that would estimate IQ? Anne Roe's papers should have later life data for Alvarez).
"We should not expect anything spectacular from Terman type intellects (one in 100?) on average (a college degree, but not much beyond that)."
ReplyDeleteAt a time when more than thirty percent of Americans over the age of twenty-five have bachelor's degrees and more than ten percent have graduate degrees, we should expect much more than that.
Verbal is hard. You make a g loaded test and you're gonna have a skewed distribution with too many high scorers, similar to how math score on SAT/GREs are always higher than verbal for most of the population. Kinda makes you think twice before dismissing non-STEM as being 'dumb'.
ReplyDeleteThat is interesting, but not the same as what I was proposing. Oprah is definitely an intriguing case--especially so for me since I grew up watching the Baltimore news during the time she came onto the scene there. It is amazing how far she has come since then.
ReplyDeleteWell let's say the average Terman gifted kid is 2.5 SD above average in IQ. Given the 0.65 correlation between IQ and academic success they should by 2.5 SD(0.65) = 1.63 SD above average academically. So roughly the top 5% on average. You're right, that would be more than a college degree, though less than a PhD which is the top 1%. And you also have to factor in that the Terman study was back in the days when a college degree put you in an exclusive club.
ReplyDeleteWhat you're proposing (if I understand you) sounds like it's just the standard error around a prediction. So given a 0.4 correlation between IQ and income, Gates' expected IQ would be 134 with an SE of 13.75, so there would be a 95% chance his IQ is between 107 and 162. If I'm totally missing your point, please forgive me. :-)
ReplyDeleteI did a simulation as a sanity check. Assuming a correlation of 0.65 of outcome Y with independent variable X, the most likely value of X for maximum Y was between 2.5 and 3 SD. Looking at probability of individual success the simulation (unsurprisingly) agrees with Steve's assertion (the probability keeps increasing past 5 SD (at that point I think sampling error takes over).
ReplyDeleteDetails (including the R code) at http://rpubs.com/rseiter/23845
In that link you write "calculate the likelihood that a given X will have the largest Y". If you're asking, what's the likelihood of the richest (self-made) person in America being the smartest person in America, then the likelihood would be about one in 10,000.
ReplyDeletePerhaps, though the point of simulation is to be able to give numbers with at least a little justification. The question I am asking would be better phrased as "what is the likelihood of the richest (I'm not sure explicitly saying self made matters here, is anyone in the top 10 inherited wealth?) person being >3 SD in IQ?" I think likelihood by X is a better way to express it though. And the chance per person by X is quite interesting--and the clearest way to argue against the "over a threshold is irrelevant" meme IMHO.
ReplyDeletewell explained!
ReplyDeletebut i was hoping to see the word "heteroskedasticity":)
Well assuming there are about 242 million American adults in America, the richest American can be viewed as having a (cumulative) income normalized Z score of 5.73 (one in 242 million level). Given the 0.4 correlation between IQ and income, his expected IQ would have a Z score of 5.73(0.4) = 2.29. Given a 0.4 correlation, the standard error of that estimate would be 0.92. So an IQ > 3 SD would be 0.77 SE above the predicted value, so the odds of that occurring would be about 21%. So, yeah, the higher the income, the greater the probability that one would be above a 3 SD IQ.
ReplyDeleteIQ is an imperfect measurement of one of the dimensions needed for "success". Aside from luck, I will bet that there are a number of genetic factors for every kind of success. When a farmer looks for a good dairy cow, he does not just look for a cow with the biggest udder. In fact, he looks for the cow that gives the most milk. The udder size is a by-product of this selection.
ReplyDeleteI think if we are to look for someone with some sort of success, we should do just that, and IQ is a natural by-product of it, recognizing that there are number of other factors, many of which are genetic but only some what correlated to IQ.
? The fact that the V ceiling on SAT/GRE is higher than for M is the choice of the test makers. They could have done it the other way as well.
ReplyDeleteIIRC two of the experimental physicists in her sample had pretty mediocre (~120) V scores. One of them could have been Alvarez. The V mean for all the scientists was very high - 166!
ReplyDeleteSMPY is a good modern update of Terman.
I think you recall correctly about Roe. Pp 162-3 has the relevant table and text but gives raw scores. A version of her chapter 12 tables (including that one) converted to IQs appears in http://psychology.ucdavis.edu/faculty_sites/simonton/greatpsychologistspiiich6.pdf (I think this is worth a look, it appears to be a chapter from Simonton's Great Psychologists book). Also note Roe's comment about the lowest scoring experimental physicist may not have done justice to him?self.
ReplyDeleteHave you seen Anne Roe's 1953 monograph "A Psychological study of eminent psychologists and anthropologists, and as a comparison with biological and physical scientists"? Worldcat says MSU has a copy. There is an excerpt at http://www.physics.umd.edu/~einstein/HONR228K/Roe.pdf but I suspect the monograph is more likely to have data.
Dutton and Lynn's 2014 "Intelligence and Religious and Political Differences Among Members of the U.S. Academic Elite" http://www.religjournal.com/pdf/ijrr10001.pdf (published in a religious journal?!) mentions Gibson and Light (1967) http://psycnet.apa.org/psycinfo/1967-11916-001 as a study similar to Roe's and has some other interesting bits (e.g. the Big 5 in academics).
Good point about the SMPY. The initial 2188 in top 1% cohort is a good basis for comparisons. Your quartile graphic of the top 1% is a great example. Did they break out the top 1% any further than quartiles? 1% to 0.25% is about 2.5 to 3 SD. The 0.01 % cohort (~3.9SD cutoff, 501 people) does this, but the within cohort comparisons seem most helpful. I guess the problem is there would only be ~20 in their first cohort of 2188 above ~4SD. Still, it would be interesting to see odds ratios broken out for 3, 3.5, 4, + SD groups.
I'm still quite baffled by that 166 mean V score Roe obtained. IQ is supposed to be normally distributed so 166 would imply one in 185,000 level ability. It would be somewhat equivalent to scoring near the ceiling of the verbal SAT (new scale) at age 13! It's hard to imagine the average (verbal) IQ of any group of people, no matter how accomplished, being that high given the imperfect correlation between ability and achievement. I speculate that Roe did not have any extremely gifted people in her test norming sample and thus was forced to assign extreme IQ scores through linear extrapolation which works on an interval scale, but I suspect Roe used ordinal scales.
ReplyDeleteThus I think the more meaningful data in Roe's study is the lowest score. If the least brilliant scientist (of the 64) had an IQ of 121 (and assuming he/she wasn't an outlier), then IQ 121 should be 2.13 SD below the mean of all the scientists. If we assume the scientists had an SD of 13.5 (similar to college grads) then the average eminent scientist should have a (verbal) IQ of 2.13(13.5) + 121 = 150.
As I mentioned above, one in a million level academic achievers should AVERAGE IQ's around the one in 2000 level given the 0.65 correlation between IQ and scholastic success, so an IQ of 150 would make perfect sense.
I have two posts on the blog about Roe's results and the identities of the scientists she studied.
ReplyDeleteI think what we can safely conclude from Roe is that her top scientists were unusually smart compared to average scientists.
ReplyDeleteIn the book, she says she normed her high ceiling tests on a population of Columbia Teacher's College graduate students, who also took standard IQ tests. Once she knows the SD in that group she can use it to convert high end raw scores to z scores. Perhaps the student SD is smaller than the general population SD and she failed to adjust for that. But she had their scores on well normed standard tests, so she could and should have done so. She didn't do the psychometric work herself -- she worked with a specialist at Columbia.
Also, there is probably some deviation from normality at the high end.
I agree. The eminent scientists were way smarter than average scientists, and the test was probably properly normed, but the distribution was probably non-Gaussian at the high end, either because the item difficulty increased in a non-linear way or because high verbal ability itself departs from the Gaussian curve.
ReplyDeleteRegarding the low V scoring scientists, Roe makes a statement (pp 162-163) which leads me to think two of the scores (a biologist and experimental physicist) should be treated as outliers (she says she noted something like that even before scoring the tests - "did not do justice to themselves").
ReplyDeleteIt might be worth checking what "scholastic success" metric is used in your correlation. Scientific eminence is different from grades. Also, I'm not sure how much I trust a linear correlation pushed out to a >4 SD tail. Correlations are handy, but they do have limitations as analytical tools IMHO. (but they do very nicely meet my criteria of justifiable numbers so please keep the analysis coming, I appreciate that ;-)
If you have the inclination Roe's methodology for assigning IQs might be worth a closer look. Steve summarized it, and it looks to me like she made a credible effort to "do it right." Elsewhere in this thread I mentioned the 1967 Gibson and Light study which used the WAIS.
Nice. I already wondered how to create two randomly generated normal distributions that are correlated.
ReplyDeleteThe average Termite was 152, sd=15.
ReplyDeleteThanks! Google and Stack Overflow are a great combination for answering questions like that (I had to answer that question to do the simulation).
ReplyDeleteI updated the simulation to include a look at the top 0.1% of Y as well as the largest Y (and comparing the two). Interesting (though perhaps not surprising) to see how the curves shift. The way the top 0.1% density curve peaks near 2SD makes clear how low the X level can be for the high Y's most observed (though I wonder about the actual threshold effect in science, e.g. you need a certain IQ to make it at all, if that is so how would it affect this analysis?) while the per-person curves show how increasing returns to higher X can be consistent with the observed density.
Reasoning statistically is just something that's culturally new to us. The concept of regression toward the mean wasn't worked out until 1885, 198 years after Newton's big book.
ReplyDeleteThat's about, very roughly, where Michael Jordan fell on the height distribution: about 2.5 to 3.0 standard deviations above the mean height. LeBron James is another s.d. out to the right.
ReplyDeleteYes, there are many ways you can measure scholastic success. Elementary school grades in children and years of education among adults both correlate 0.65 with IQ so I use 0.65 as a good estimate for scholastic success in general (in the total population, correlations are much smaller in restricted samples like grad students). So if there were a perfect correlation between IQ and scholastic success, we'd expect the dumbest academic Nobel prize winner to have an IQ of 171 (one in a million on a Gaussian scale) and the average to be maybe 175? (correct me if I'm wrong) But since the correlation is 0.65, their average expected IQ would be 0.65(175 - 100) = 149. The reason I'm so suspicious of Roe's average IQ of 166 is it would imply a 0.88 correlation between IQ and academic success (actually maybe higher since Roe's scientists were below the Nobel prize level) which is higher than IQ tests correlate with each other
ReplyDeleteBut I understand that such calculations have limitations. There are different types of academic success and it's simplistic to just use 0.65 as a one size fits all correlation for criteria as diverse as elementary school grades, years of education, and scientific eminence, and it's also simplistic to assume correlations observed in the general population can be extrapolated to predict one in a million achievements. But occam's razor suggests that we start with simple assumptions and only abandon them when they become untenable I also just find linear relationships aesthetically pleasing and am biased in favor of them which is probably how I got interested in IQ in the first place.
Oops, sorry I didn't realize the monograph might be different from the book. Might be worth looking into.
ReplyDeleteI've read the Cambridge paper and it's not very enlightening. The scores seem very low to me.
The average Termite did indeed score in the 150s, but ratio IQ's used at the time generally had SDs greater than 15 and were non-normally distributed at the extremes because some kids have cognitive growth spurts causing their mental age to leap to twice their chronological age, thus their NORMALIZED average IQ was likely well below the 3 SD level
ReplyDeleteV-IQ tends to increase somewhat with age:
ReplyDeletehttp://qph.is.quoracdn.net/main-qimg-fd832e64253819688a4eaaaf25eac62b?convert_to_webp=true
Let's take that graph at face value and assume the test was normalized in such a way that average 20-somethings would score 100 for V-IQ. At the same age, the theoretical physicists in the group would have scored about 164 V-IQ. That sounds about right to me.
Nobel Prize winners are not noticeably smarter than average faculty at top schools such as MIT and Harvard. When I think of theoretical physicists at that level, a verbal IQ at the 1:100,000 level seems about right. 1:10,000 is probably too low, but it could be a lower bound for those eminent theoretical physicists whose main strength is non-spatial reasoning.
Can you elaborate on this? A reference expanding on this point would be appreciated.
ReplyDeleteInteresting graph. I went looking for it and found http://www.sibille.pitt.edu/Publications/2011_Glorioso_ProgressNeurobiology.pdf
ReplyDeletewhich looks at nervous system aging and disease. Figure 3 emphasizes the heterogeneity of the population underlying the average curves.
Concur with you.
ReplyDeleteWe all noticed that some kids who were so smart in our grade school yet falled behind in high school years. Kids too young yield unreliable intellectual information. People might have very different mental maturation rate just like that of height.