Perverse Incentives and Replication in Science

Here's a depressing but all too common pattern in scientific research:

1. Study reports results which reinforce the dominant, politically correct, narrative.

2. Study is widely cited in other academic work, lionized in the popular press, and used to advance real world agendas.

3. Study fails to replicate, but no one (except a few careful and independent thinkers) notices.

For numerous examples, see, e.g., any of Malcolm Gladwell's books :-(

A recent example: the idea that collective intelligence of groups (i.e., ability to solve problems and accomplish assigned tasks) is not primarily dependent on the cognitive ability of individuals in the group.

It seems plausible to me that by adopting certain best practices for collaboration one can improve group performance, and that diversity of knowledge base and personal experience could also enhance performance on certain tasks. But recent results in this direction were probably oversold, and seem to have failed to replicate.

James Thompson has given a good summary of the situation.

Parts 1 and 2 of our story:

MIT Center for Collective Intelligence: ... group-IQ, or “collective intelligence” is not strongly correlated with the average or maximum individual intelligence of group members but is correlated with the average social sensitivity of group members, the equality in distribution of conversational turn-taking, and the proportion of females in the group.

Is it true? The original paper on this topic, from 2010, has been cited 700+ times. See here for some coverage on this blog when it originally appeared.

Below is the (only independent?) attempt at replication, with strongly negative results. The first author is a regular (and very insightful) commenter here -- I hope he'll add his perspective to the discussion. Have we reached part 3 of the story?

Smart groups of smart people: Evidence for IQ as the origin of collective intelligence in the performance of human groups

Timothy C. Bates a,b,⁎, Shivani Gupta a
a Department of Psychology, University of Edinburgh
b Centre for Cognitive Ageing and Cognitive Epidemiology, University of Edinburgh

What allows groups to behave intelligently? One suggestion is that groups exhibit a collective intelligence accounted for by number of women in the group, turn-taking and emotional empathizing, with group-IQ being only weakly-linked to individual IQ (Woolley, Chabris, Pentland, Hashmi, & Malone, 2010). Here we report tests of this model across three studies with 312 people. Contrary to prediction, individual IQ accounted for around 80% of group-IQ differences. Hypotheses that group-IQ increases with number of women in the group and with turn-taking were not supported. Reading the mind in the eyes (RME) performance was associated with individual IQ, and, in one study, with group-IQ factor scores. However, a well-fitting structural model combining data from studies 2 and 3 indicated that RME exerted no influence on the group-IQ latent factor (instead having a modest impact on a single group test). The experiments instead showed that higher individual IQ enhances group performance such that individual IQ determined 100% of latent group-IQ. Implications for future work on group-based achievement are examined.

From the paper:

Given the ubiquitous importance of group activities (Simon, 1997) these results have wide implications. Rather than hiring individuals with high cognitive skill who command higher salaries (Ritchie & Bates, 2013), organizations might select-for or teach social sensitivity thus raising collective intelligence, or even operate a female gender bias with the expectation of substantial performance gains. While the study has over 700 citations and was widely reported to the public (Woolley, Malone, & Chabris, 2015), to our knowledge only one replication has been reported (Engel, Woolley, Jing, Chabris, & Malone, 2014). This study used online (rather than in-person) tasks and did not include individual IQ. We therefore conducted three replication studies, reported below.

... Rather than a small link of individual IQ to group-IQ, we found that the overlap of these two traits was indistinguishable from 100%. Smart groups are (simply) groups of smart people. ... Across the three studies we saw no significant support for the hypothesized effects of women raising (or men lowering) group-IQ: All male, all female and mixed-sex groups performed equally well. Nor did we see any relationship of some members speaking more than others on either higher or lower group-IQ. These findings were weak in the initial reports, failing to survive incorporation of covariates. We attribute these to false positives. ... The present findings cast important doubt on any policy-style conclusions regarding gender composition changes cast as raising cognitive-efficiency. ...

In conclusion, across three studies groups exhibited a robust cognitive g-factor across diverse tasks. As in individuals, this g-factor accounted for approximately 50% of variance in cognition (Spearman, 1904). In structural tests, this group-IQ factor was indistinguishable from average individual IQ, and social sensitivity exerted no effects via latent group-IQ. Considering the present findings, work directed at developing group-IQ tests to predict team effectiveness would be redundant given the extremely high utility, reliability, validity for this task shown by individual IQ tests. Work seeking to raise group-IQ, like re- search to raise individual IQ might find this task achievable at a task- specific level (Ritchie et al., 2013; Ritchie, Bates, & Plomin, 2015), but less amenable to general change than some have anticipated. Our attempt to manipulate scores suggested that such interventions may even decrease group performance. Instead, work understanding the developmental conditions which maximize expression of individual IQ (Bates et al., 2013) as well as on personality and cultural traits supporting cooperation and cumulation in groups should remain a priority if we are to understand and develop cognitive ability. The present experiments thus provide new evidence for a central, positive role of individual IQ in enhanced group-IQ.

Meta-Observation: Given the 1-2-3 pattern described above, one should be highly skeptical of results in many areas of social science and even biomedical science (see link below). Serious researchers (i.e., those who actually aspire to participate in Science) in fields with low replication rates should (as a demonstration of collective intelligence!) do everything possible to improve the situation. Replication should be considered an important research activity, and should be taken seriously.

Most researchers I know in the relevant areas have not yet grasped that there is a serious problem. They might admit that "some studies fail to replicate" but don't realize the fraction might be in the 50 percent range!

More on the replication crisis in certain fields of science.

IQ prediction from structural MRI

These authors use machine learning techniques to build sparse predictors based on grey/white matter volumes of specific regions. Correlations obtained are ~ 0.7 (see figure).

I predict that genomic estimators of this kind will be available once ~ 1 million genomes and cognitive scores are available for analysis. See also Myths, Sisyphus and g.

MRI-Based Intelligence Quotient (IQ) Estimation with Sparse Learning (PLOS)

In this paper, we propose a novel framework for IQ estimation using Magnetic Resonance Imaging (MRI) data. In particular, we devise a new feature selection method based on an extended dirty model for jointly considering both element-wise sparsity and group-wise sparsity. Meanwhile, due to the absence of large dataset with consistent scanning protocols for the IQ estimation, we integrate multiple datasets scanned from different sites with different scanning parameters and protocols. In this way, there is large variability in these different datasets. To address this issue, we design a two-step procedure for 1) first identifying the possible scanning site for each testing subject and 2) then estimating the testing subject’s IQ by using a specific estimator designed for that scanning site. We perform two experiments to test the performance of our method by using the MRI data collected from 164 typically developing children between 6 and 15 years old. In the first experiment, we use a multi-kernel Support Vector Regression (SVR) for estimating IQ values, and obtain an average correlation coefficient of 0.718 and also an average root mean square error of 8.695 between the true IQs and the estimated ones. In the second experiment, we use a single-kernel SVR for IQ estimation, and achieve an average correlation coefficient of 0.684 and an average root mean square error of 9.166. All these results show the effectiveness of using imaging data for IQ prediction, which is rarely done in the field according to our knowledge.

Training and testing of models was performed as described below. They had only 164 individuals in their sample, so IIUC the average correlation is computed on test samples of ~16 individuals. It would be good to see their predictors tested on larger data sets. I wonder how stable the predictor variables (feature coefficients) were across partitions.

We performed experiments with 10-fold cross-validations. Specifically, we randomly partitioned each dataset into 10 subsets with no replacement, and used 9 out of the 10 subsets for training and the remaining one for testing. To further avoid a possible bias during partitioning, we repeated the experiments 10 times.

Some background from the paper. Strangely, they don't cite the Thompson lab (UCLA) results on brain size and intelligence (21k individuals). IIRC from their results, brain size alone correlates 0.4 with IQ.

... Uncovering human intelligence has always been of major interest in cognitive neuroscience. With the advent of brain imaging, there have been efforts to investigate the relation between brain anatomy and intelligence [3,4], and substantial understanding has been achieved in the field. For example, Supekar et al. showed that the size and circuitry of certain parts of children’s brains could be a potential predictor for how well they would respond to intensive math tutoring [5]. Chen et al. [6] demonstrated that the volumetric analysis of gray matter (GM) from structural Magnetic Resonance Imaging (MRI) could be used to predict a subsequent decline in IQ in children with sickle cell disease. McDaniel et al. [3] found that the volume of the brain is positively correlated with IQ according to MRI-based experiments. Frangou et al. [7] reported positive correlations between IQ score and GM density of the orbitofrontal cortex, cingulate gyrus, cerebellum, and thalamus, but negative correlation between IQ score and the caudate nucleus. On the other hand, Navas-Sanchez et al. [8] investigated the relationship between IQ score and microstructure of white matter (WM) tracts using diffusion tensor imaging (DTI), and found that IQ score is positively correlated with fractional anisotropy (FA). Kim et al. [9] found that lower performance in verbal IQ score is correlated with the decrease of FA values. In another DTI-based study, Welcome et al. [10] discovered that the volume of WM fiber tracts is correlated with nonverbal IQ score. Inspired by these strong correlations between brain anatomy and IQ score, we propose, in this study, a novel framework to estimate IQ by using GM and WM features extracted from structural MRI. ...

Their results might give some indication as to which regions of the brain are responsible for most of the population variation in IQ. Below are the brain regions most commonly identified as "features" by sparse learning methods.

From the comments (55% of variance means a correlation just larger than 0.7):

There are lots of recent studies that have tried to estimate IQ from MRI or EEG readings (sometimes called "neurometric" IQ); many of the teams are based in South Korea and Malaysia. The Malaysian group, based at the MARA University of Technology, has published about a dozen papers over the past two years, involving hundreds of subjects. They can now use EEG readings to sort subjects into one of seven IQ ranges (e.g. 90-100, 120-130) with 83% accuracy; this figure jumps to 98% when subjects are sorted into one of three IQ ranges (low, medium, or high). The South Korean researchers, at Seoul National University, have been combining MRI and fMRI scans to predict IQ scores, and in late 2012 they were granted a patent for their "neurobiological method for measuring human intelligence," which can explain up to 55% of the variance between individual IQ scores. An example (from Dec 2013) is at http://ieeexplore.ieee.org/...

Additional links:
http://dx.doi.org/10.1016/j.cmpb.2014.01.016
http://www.google.com/patents/WO2008018763A1?cl=en

Nature, Nurture, and Invention: analysis of Finnish data

What is the dominant causal mechanism for the results shown above? Is it that better family environments experienced by affluent children make them more likely to invent later in life? Is it that higher income fathers tend to pass on better genes (e.g., for cognitive ability) to their children? Obviously the explanation has important implications for social policy and for models of how the world works.

The authors of the paper below have access to patent, income, education, and military IQ records in Finland. (All males are subject to conscription.) By looking at brothers who are close in age but differ in IQ score, they can estimate the relative importance of common family environment (such as family income level or parental education level, which affect both brothers) versus the IQ difference itself. Their results suggest that cognitive ability has a stronger effect than shared family environment. Again, if one just looks at probability of invention versus family income or SES (see graph), one might mistakenly conclude that family environment is the main cause of increased likelihood of earning a patent later in life. In fact, higher family SES is also correlated to superior genetic endowments which can be passed on to the children.

The Social Origins of Inventors
Philippe Aghion, Ufuk Akcigit, Ari Hyytinen, Otto Toivanen
NBER Working Paper No. 24110
December 2017

In this paper we merge three datasets - individual income data, patenting data, and IQ data - to analyze the determinants of an individual's probability of inventing. We find that: (i) parental income matters even after controlling for other background variables and for IQ, yet the estimated impact of parental income is greatly diminished once parental education and the individual's IQ are controlled for; (ii) IQ has both a direct effect on the probability of inventing an indirect impact through education. The effect of IQ is larger for inventors than for medical doctors or lawyers. The impact of IQ is robust to controlling for unobserved family characteristics by focusing on potential inventors with brothers close in age. We also provide evidence on the importance of social family interactions, by looking at biological versus non-biological parents. Finally, we find a positive and significant interaction effect between IQ and father income, which suggests a misallocation of talents to innovation.

From the paper:

... IQ has both a direct effect on the probability of inventing which is almost five times as large as that of having a high-income father, and an indirect effect through education ...

... an R-squared decomposition shows that IQ matters more than all family background variables combined; moreover, IQ has both a direct and an indirect impact through education on the probability of inventing, and finally the impact of IQ is larger and more convex for inventors than for medical doctors or lawyers. Third, to address the potential endogeneity of IQ, we focused on potential inventors with brothers close in age. This allowed us to control for family-specific time-invariant unobservables. We showed that the effect of visuospatial IQ on the probability of inventing is maintained when adding these controls.

More on the close brothers analysis (p.24).

We look at the effect of an IQ differential between the individual and close brother(s) born at most three years apart.16 This allows us to include family fixed effects and thereby control for family-level time-invariant unobservables, such as genes shared by siblings, parenting style, and fixed family resources. Table 4 shows the results from the regression with family-fixed effects. The first column shows the baseline OLS results using the sample on brothers born at most three years apart. Notice that we include a dummy for the individual being the first born son in the family to account for birth-order effects. The second column shows the results from a regression where we introduce family fixed effects. We lose other parental characteristics than income due to their time-invariant nature.17 The main finding in Table 4 is that the coefficients on "IQ 91-95" and "IQ 96-100" [ these are percentiles, not IQ scores ] in Column 2 (i.e. when we perform the regression with family fixed effects) are the same as in the OLS Column 1. This suggests that these coefficients capture an effect of IQ on the probability of inventing which is largely independent of unobserved family background characteristics, as otherwise the OLS coefficients would be biased and different from the fixed effects estimates.

Note Added: Finland is generally more egalitarian than the US, both in terms of wealth distribution and access to education. But the probability of invention vs family income graph is qualitatively similar in both countries (see Fig 1 in the paper). The figure below is from recent US data; compare to the Finland figure at top.

Thanks to some discussion (see comments) I noticed that in the Finnish data the probability of invention seems to saturate at high incomes (see top figure, red circle), whereas it continues to rise strongly at top IQ scores (middle figure above; also perhaps in the US data above?). It would be interesting to explore this in more detail...

If you're so smart, why aren't you rich?

Does being smart help you become rich?

Folksy Warren Buffett once said that an investor with IQ of 150 should sell 30 points to someone else, as anything above 120 is unnecessary.

Consider the following simple model (we can call it the "Igon Model" in honor of Malcolm Gladwell):

Igon Model: IQ correlates positively with wealth, but the effect goes away for IQ > 120. IQ above 120 provides no advantage, relative to IQ=120, for acquiring wealth.

Were this model to be true, one would expect with overwhelming probability to find that the vast majority of rich people have IQ around 120, but not much higher. This is because IQ is normally distributed: as you go further out the tail the population decreases exponentially. To be specific, IQ = 120 corresponds to the 90th percentile, whereas IQ = 135 is 99th percentile (i.e., only 1 in 10 people with IQ > 120 have IQ > 135) and IQ = 145 is 99.9th percentile (i.e., only 1 in 100 people with IQ > 120 have IQ > 145).

Now let's look at the 2009 Forbes list of richest people in the world:

1 William Gates III 53 40.0 United States
2 Warren Buffett 78 37.0 United States
3 Carlos Slim Helu 69 35.0 Mexico

If the Igon Model were correct, we would not expect to find this list dominated by people with IQ much higher than 120. But in fact we do. Note these three made their money in different ways: Gates founded a software company, Buffett is primarily an investor, and Carlos Slim is an oligarch ;-)

Bill Gates scored 1580 on the pre-1995 SAT. His IQ is clearly >> 145 and possibly as high as 160 or so.

Warren Buffett graduated high school at 16 ranked in the top 5 percent of his class despite devoting substantial effort to entrepreneurial activities. Most people who know him well refer to him as brilliant, that folksy quote above notwithstanding. I would suggest the evidence is strong that his IQ is above 135, perhaps higher than 145.

Carlos Slim studied engineering and taught linear programming while still an undergraduate at UNAM, the top university in Mexico. He reportedly discovered the use of compound interest at age 10. I would suggest his IQ is also at least 135.

So it would appear that the three richest men in the world all have IQs that are higher than 90 percent or even 99 percent of the > 120 IQ population. (Relative to the general population they are all likely in the 99th or even 99.9th percentile.) The probability of this happening in the Igon Model is less than 1 in 1000.

[Here's a basketball analogy: the analogous Igon Model for basketball would say height over 6ft2 (90th percentile) doesn't increase likelihood of success in basketball. Suppose we find the 3 top players in the world are 7ft (Shaq/Gates), 6ft8 (LeBron/Buffett) and 6ft6 (Kobe/Slim). That strongly disfavors the model, as a random draw of 3 people from the set of people over 6ft2 in height has almost zero probability of producing the 3 heights we found.]

Note to angry Gladwell egalitarians: don't take this analysis too seriously :-) It's really an example of "Igon analysis" in the spirit of MG!

There are many factors aside from intelligence that impact success in business or investing. See here for a discussion by money manager and investment theorist William Bernstein, which is very similar to what Buffett has said on various occasions. If you carefully study biographies of the three men listed above, what really stands out (aside from high mental ability) is their determination, drive and fascination with material success beginning at a young age. See also: success vs ability and creators and rulers.

What about the broader population? It's well established that graduates of elite universities earn more than graduates of less selective schools. But, interestingly, controlling for SAT score (IQ) largely eliminates the differential. I wonder why? (See also here for UT Austin data on earnings variation with SAT and major.)

Intergenerational mobility: Bowles and Gintis and Clark

I went back to Bowles and Gintis to compare their results to those of Greg Clark that I posted about recently. The largest correlation reported by Bowles and Gintis for intergenerational earnings is 0.65, obtained when fathers' and sons' earnings are averaged over multiyear periods, whereas Clark finds a (roughly) 0.7 -- 0.8 correlation between parental and children's social and economic status. Clark was studying the past 200 years, using rare surnames, whereas Bowles and Gintis concentrated on the modern era. Even the lower value of 0.42 (more typical of results cited by Bowles and Gintis) implies some persistent stratification, as shown in the figure below.

Bowles and Gintis: ... The relevant facts on which most researchers now agree include the following: brothers’ incomes are much more similar than those of randomly chosen males of the same race and similar age differences; the incomes of identical twins are much more similar than fraternal twins or non-twin brothers; the children of well-off parents obtain more and higher quality schooling; and wealth inheritance makes an important contribution to the wealth owned by the offspring of the very rich. On the basis of these and other empirical regularities, it seems safe to conclude that the intergenerational transmission of economic status is accounted for by a heterogeneous collection of mechanisms, including the genetic and cultural transmission of cognitive skills and non-cognitive personality traits in demand by employers, the inheritance of wealth and income enhancing group memberships such as race, and the superior education and health status enjoyed by the children of higher status families. ...

Here are Bowles and Gintis on IQ and earnings:

We have located 65 estimates of the normalized regression coefficient of a test score in an earnings equation in 24 different studies of U.S. data over a period of three decades. Our meta-analysis of these studies is presented in Bowles, Gintis, and Osborne (2002a). The mean of these estimates is 0.15, indicating that a standard deviation change in the cognitive score, holding constant the remaining variables (including schooling), changes the natural logarithm of earnings by about one-seventh of a standard deviation. By contrast, the mean value of the normalized regression coefficient of years of schooling in the same equation predicting the natural log of earnings in these studies is 0.22, suggesting a somewhat larger independent effect of schooling. We checked to see if these results were dependent on the weight of overrepresented authors, the type of cognitive test used, at what age the test was taken and other differences among the studies and found no significant effects. An estimate of the causal impact of childhood IQ on years of schooling (also normalized) is 0.53 (Winship and Korenman 1999). A rough estimate of the direct and indirect effect of IQ on earnings, call it b, is then b = 0.15+(0.53)(0.22) = 0.266.

... Using the values estimated above, we see that the contribution of genetic inheritance of IQ to the intergenerational transmission of income is (h2 (1+m)/2)(0.266)^2 = .035(1 + m) h2. If the heritability [h2] of IQ were 0.5 and the degree of assortation, m, were 0.2 (both reasonable, if only ball park estimates) and the genetic inheritance of IQ were the only mechanism accounting for intergenerational income transmission,then the intergenerational correlation would be 0.01, or roughly two percent the observed intergenerational correlation. Note the conclusion that the contribution of genetic inheritance of IQ is negligible is not the result of any assumptions concerning assortative mating or the heritability of IQ: the IQ genotype of parents could be perfectly correlated and the heritability of IQ 100 per cent without appreciably changing the qualitative conclusions. The estimate results from the fact that IQ is just not an important enough determinant of economic success.

[THESE RESULTS ARE SURPRISING SMALL. I'D GUESS THAT AVG IQ VARIES FROM ABOUT -.5 to -.75 SD AT 10TH PERCENTILE INCOME TO +.5 to +.75 SD (COLLEGE GRADUATES) AT 90TH PERCENTILE INCOME, SO CORRELATION OF PERHAPS 0.3 -- 0.5. I GUESS THE LITERATURE SUPPORTS 0.3.

BUT NOTE THAT AVERAGING INCOMES OVER EXTENDED PERIODS (LIKE 10 YEARS) INCREASES THE INTERGENERATIONAL CORRELATION SIGNIFICANTLY (I.E., FROM 0.4 TO .65, PRESUMABLY BY ELIMINATING SOME OF THE NOISE IN THE INCOME MEASUREMENT), SO ONE MIGHT EXPECT THE SAME FOR THE IQ-INCOME CORRELATION. (THE INTERGENERATIONAL STUFF IS COMING FROM SOMEWHERE!) SINCE THE IQ-INCOME CORRELATION ENTERS SQUARED, INCREASING IT HAS A BIG EFFECT.

IF WE TAKE THE INCOME-IQ CORRELATION AS 0.5 AND h2 = 0.5, THE CONTRIBUTION TO INTERGENERATIONAL CORRELATION IS (1/2)^3 = 0.125 WHICH IS A FOURTH OR FIFTH OF THE TOTAL.]

In the Terman study of gifted individuals, additional IQ above 135 or so had a relatively small effect on lifetime earnings: an increase of 15-20% for an additional SD of IQ. In contrast, personality factors such as Conscientiousness and Extraversion were strongly correlated with increased earnings (see figures at link above): an increase of about 50% from 10th to 90th percentile was observed. Note that personality factors are heritable, with h2 roughly 0.5 or so. These findings, which come from a high IQ sample and are thus affected by restriction of range, are nevertheless suggestive of effect sizes in the overall population.

While it doesn't appear that IQ alone can account for most of intergenerational earnings transmission, a combination of IQ, Conscientiousness, Extraversion, ambition, and traits related to social intelligence (even, physical attractiveness) likely play an important and heritable role. (Note the strong similarity in earnings of identical twins.) Perhaps this "success bundle" of traits is what elite (Ivy) holistic admissions is searching for? :-)


For more discussion of Bowles and Gintis, see GNXP 2011 and GNXP 2008. A numerical error in B&G is noted, but it doesn't change the qualitative conclusions.

More on psychometrics

I've had some email discussions elaborating on the psychometrics slides I posted earlier. The slides themselves don't convey a lot of the important points I made in the talks so I thought I'd share this message on the blog.

Hi Guys,

I'm very interested in exactly the question Henry is getting at.

I think our simple two factor model

Grades = ability + work ethic = IQ + W

is not too crazy. Note that once you fix the ability level (=SAT score) the remaining variance in GPA has about the same SD regardless of value of SAT score (vertical red lines in the big figure in the slides). That suggests that we can think of IQ and W as largely uncorrelated random variables -- so there are smart lazy people, hard working dumb people, etc. I can't really prove the residual variance after IQ is controlled for is due to work ethic, but my experience in the classroom suggests that it is. (Note work ethic here isn't necessary general work ethic as a personality factor, but how hard the kid worked in the specific course. However, in our data we average over many courses taken by many kids, so perhaps it does get at variation of personality factor(s) in the overall population.) Beyond work ethic, some people are just more "effective" -- they can get themselves organized, are disciplined, can adapt to new challenges, are emotionally robust -- and this is also absorbed in the W factor above.

Now, in some fields there seems to be a minimum cognitive threshold. I've known physics students who worked incredibly hard and just couldn't master the material. That is reflected in our data on pure math and physics majors at UO. For all majors there is a significant positive correlation between SAT and upper GPA (in the range .3-.5).

Whether IQ has a large impact on life outcomes depends on how you ask the question. I do believe that certain professions are almost off-limits for people below a certain IQ threshold. But for most jobs (even engineer or doctor), this threshold is surprisingly low IF the person has a strong work ethic. In other words a +1 SD IQ person can probably still be a doctor or engineer if they have +(2-3) SD work ethic. However, such people, if they are honest with themselves, understand that they have some cognitive disadvantages relative to their peers. I've chosen a profession in which, every so often, I am the dumbest guy in the room -- in fact I put myself in this situation by going to workshops and wanting to talk to the smartest guys I can find :-) For someone of *average* work ethic I think you can easily find jobs for which the IQ threshold is +2 SD or higher. The typical kid admitted to grad school in my middle-tier physics department is probably > +2 SD IQ and at least +1.5 SD in work ethic -- ditto for a top tier law or med school. That's probably also the case these days for any "academic admit" at a top Ivy.

For typical jobs I think the correlation between success/income and IQ isn't very high. Other factors come into play, like work ethic, interpersonal skills, affect, charisma, luck, etc. This may even be true in many "elite" professions once you are talking about a population where everyone is above the minimum IQ threshold -- if returns to IQ above threshold are not that large then the other factors dominate and determine level of success. What is interesting about the Roe and SMPY studies is that they suggest that in science the returns to IQ above the +2 SD threshold (for getting a PhD) are pretty high. ***

Henry is right that for ideological reasons many researchers are happy to present the data so as to minimize the utility of IQ or testing in making life predictions. They might even go so far as to claim that since we use g-loaded tests in admissions, the conclusion that some professions require high IQ is actually circular. The social scientist who walked out of my Sci Foo talk actually made that claim.

Finally, when it comes to *individual* success I think most analysts significantly underestimate the role of pure blind luck (i.e., what remains when all other reasonable, roughly measurable variables have been accounted for; of course this averages out of any large population study). Or perhaps I am just reassuring myself about my limited success in life :-)


Steve

PS In the actual talks I gave I made most of these points. The slides are kind of bare bones...


*** You would have difficulty finding a hard scientist who would disagree with the statement it is a big advantage in my field to be super smart. However, thanks to political correctness, social science indoctrination, or unfamiliarity with psychometrics, it IS common for scientists to deny that being super smart has anything to do with scores on IQ tests. I myself question the validity of IQ tests beyond +(3-4) SD -- I'm more impressed by success on the IMO, Putnam, or in other high level competitions. (Although I realize that training has a big impact on performance in these competitions I do think real talent is a necessary condition for success.)

Earnings effects of personality, education and IQ for the gifted

Thanks to a reader for pointing me to this recent paper by Heckman and collaborators, which makes use of data from the Terman study of gifted individuals (minimum IQ of 135 on the Stanford-Binet).

Of the personality factors, Conscientiousness and Extraversion had the largest (positive) effect on lifetime earnings: the most conscientious or extraverted individuals earned, on average, about 50% more than the least (see figures below). See here for more on Big 5 personality factors and a link to a personality test.

The Effects of Education, Personality, and IQ on Earnings of High-Ability Men

This paper estimates the internal rate of return (IRR) to education for men and women of the Terman sample, a 70-year long prospective cohort study of high-ability individuals. The Terman data is unique in that it not only provides full working-life earnings histories of the participants, but it also includes detailed profiles of each subject, including IQ and measures of latent personality traits. Having information on latent personality traits is significant as it allows us to measure the importance of personality on educational attainment and lifetime earnings.

Our analysis addresses two problems of the literature on returns to education: First, we establish causality of the treatment effect of education on earnings by implementing generalized matching on a full set of observable individual characteristics and unobserved personality traits. Second, since we observe lifetime earnings data, our estimates of the IRR are direct and do not depend on the assumptions that are usually made in order to justify the interpretation of regression coefficients as rates of return.

For the males, the returns to education beyond high school are sizeable. For example, the IRR for obtaining a bachelor's degree over a high school diploma is 11.1%, and for a doctoral degree over a bachelor's degree it is 6.7%. These results are unique because they highlight the returns to high-ability and high-education individuals, who are not well-represented in regular data sets.

Our results highlight the importance of personality and intelligence on our outcome variables. We find that personality traits similar to the Big Five personality traits are significant factors that help determine educational attainment and lifetime earnings. Even holding the level of education constant, measures of personality traits have significant effects on earnings. Similarly, IQ is rewarded in the labor market, independently of education. Most of the effect of personality and IQ on life-time earnings arise late in life, during the prime working years. Therefore, estimates from samples with shorter durations underestimate the treatment effects.

Here are a couple of interesting excerpts from the paper:

... Our third contribution is to show how the effect of personality on earnings varies through-out the men’s working lives. We find that without access to long follow-up data, the estimated effect would be understated. Note that even though the Terman sample has a restricted range of IQ, there is substantial variation in personality. In fact, the Terman men do not differ from the general population in terms of personality.

... note that even when controlling for rich background variables, IQ maintains a statistically significant effect on lifetime earnings. Even though the effect is slightly diminished from the un-controlled association of the first column, it is still sizable. Malcolm Gladwell claims rather generally in his book Outliers that for the Terman men, IQ did not matter once family background and other observable personal characteristics were taken into account. While we do not want to argue that IQ has a larger role for the difference between 50 and 100, for example, than for the difference between 150 and 200, we do want to point out that even at the high end of the ability distribution, IQ has meaningful consequences. [The syntax of this last sentence is strange. Presumably the impact of IQ variation from 50 to 100 (from severely handicapped to average) is larger than for 150 to 200, even though their results show a significant effect even in the very high range.]

Below are some nice figures (click for larger versions). Note the personality factor distribution among Termites was similar to that of the overall population, whereas the IQ range was restricted due to selection. Typical lifetime earnings for this group of exceptionally able men ranged from $2 to $3 million in 2008 dollars.




Compare the bottom right IQ graph with SMPY results which show the impact of ability (SAT-M measured before age 13) on publication and patent rates. Ability in the SMPY graph varies between 99th and 99.99th percentile in quartiles Q1-Q4. The variation in IQ between the bottom and top deciles of the Terman study covers a similar range. The Terman super-smarties (i.e., +4 SD) only earned slightly more (say, 15-20% over a lifetime) than the ordinary smarties (i.e., +2.5 SD), but the probability of earning a patent (SMPY) went up by about 4x over the corresponding ability range.

Jensen on g and genius

I found the excerpt below in the comments here.

Mega Questions for Renowned Psychologist Dr._Arthur R. Jensen - Interview by Christopher Michael Langan and Dr. Gina LoSasso and members of the Mega Foundation, Mega Society East and Ultranet 

Question #1: 

Christopher Langan for the Mega Foundation: It is reported that one of this century’s greatest physicists, Nobelist Richard Feynman, had an IQ of 125 or so. Yet, a careful reading of his work reveals amazing powers of concentration and analysis…powers of thought far in excess of those suggested by a z score of well under two standard deviations above the population mean. Could this be evidence that something might be wrong with the way intelligence is tested? Could it mean that early crystallization of intelligence, or specialization of intelligence in a specific set of (sub-g) factors – i.e., a narrow investment of g based on a lopsided combination of opportunity and proclivity - might put it beyond the reach of g-loaded tests weak in those specific factors, leading to deceptive results?

Arthur Jensen: I don’t take anecdotal report of the IQs of famous persons at all seriously. They are often fictitious and are used to make a point - typically a put-down of IQ test and the whole idea that individual differences in intelligence can be ranked or measured. James Watson once claimed an IQ of 115; the daughter of another very famous Nobelist claimed that her father would absolutely “flunk” any IQ test. It’s all ridiculous. 

Furthermore, the outstanding feature of any famous and accomplished person, especially a reputed genius, such as Feynman, is never their level of g (or their IQ), but some special talent and some other traits (e.g., zeal, persistence). Outstanding achievements(s) depend on these other qualities besides high intelligence. The special talents, such as mathematical musical, artistic, literary, or any other of the various “multiple intelligences” that have been mentioned by Howard Gardner and others are more salient in the achievements of geniuses than is their typically high level of g. Most very high-IQ people, of course, are not recognized as geniuses, because they haven’t any very outstanding creative achievements to their credit. However, there is a threshold property of IQ, or g, below which few if any individuals are even able to develop high-level complex talents or become known for socially significant intellectual or artistic achievements. This bare minimum threshold is probably somewhere between about +1.5 sigma and +2 sigma from the population mean on highly g-loaded tests. 

Childhood IQs that are at least above this threshold can also be misleading. There are two famous scientific geniuses, both Nobelists in physics, whose childhood IQs are very well authenticated to have been in the mid-130s. They are on record and were tested by none other than Lewis Terman himself, in his search for subjects in his well-known study of gifted children with IQs of 140 or above on the Stanford-Binet intelligence test. Although these two boys were brought to Terman’s attention because they were mathematical prodigies, they failed by a few IQ points to meet the one and only criterion (IQ > 139) for inclusion in Terman’s study. Although Terman was impressed by them, as a good scientist he had to exclude them from his sample of high-IQ kids. Yet none of the 1,500+ subjects in the study ever won a Nobel Prize or has a biography in the Encyclopedia Britannica as these two fellows did. Not only were they gifted mathematically, they had a combination of other traits without which they probably would not have become generally recognized as scientific and inventive geniuses. So-called intelligence tests, or IQ, are not intended to assess these special abilities unrelated to IQ or any other traits involved in outstanding achievement. It would be undesirable for IQ tests to attempt to do so, as it would be undesirable for a clinical thermometer to measure not just temperature but some combination of temperature, blood count, metabolic rate, etc. A good IQ test attempts to estimate the g factor, which isn’t a mixture, but a distillate of the one factor (i.e., a unitary source of individual differences variance) that is common to all cognitive tests, however diverse. 

I have had personal encounters with three Nobelists in science, including Feynman, who attended a lecture I gave at Cal Tech and later discussed it with me. He, like the other two Nobelists I’ve known (Francis Crick and William Shockley), not only came across as extremely sharp, especially in mathematical reasoning, but they were also rather obsessive about making sure they thoroughly understood the topic under immediate discussion. They at times transformed my verbal statements into graphical or mathematical forms and relationships. Two of these men knew each other very well and often discussed problems with each other. Each thought the other was very smart. I got a chance to test one of these Nobelists with Terman’s Concept Mastery Test, which was developed to test the Terman gifted group as adults, and he obtained an exceptionally high score even compared to the Terman group all with IQ > 139 and a mean of 152. 

I have written an essay relevant to this whole question: “Giftedness and genius: Crucial differences.” In C. P. Benbow & D. Lubinski (Eds.) Intellectual Talent: Psychometric and Social Issues, pp. 393-411. Baltimore: Johns Hopkins University Press.

See here for data relevant to this topic and the discussion in the comments.

Brutal, just brutal

Via Gene Expression, three essays on intelligence and education by Charles Murray. Yes, I know they appeared on the WSJ editorial page, and yes, I know what book Murray co-authored back in the 90's. However, he's willing to raise some important issues that others dare not. Some excerpts below.

Warning, not for the politically correct ability egalitarians who think anyone can play basketball like Michael Jordan or compose music like Mozart if they just try hard enough.

If you don't like his continued use of IQ as a measure of intelligence, just pretend you have your own measure (or even measures) and substitute it each time he writes "IQ", keeping in mind your measure will have an average - let's call it "100" for simplicity - and probably be roughly normally distributed, as most human traits are.

1) What fraction of the population is capable of absorbing a university education or mastering college-level material? It might be smaller than the proportion attending college today.

To have an IQ of 100 means that a tough high-school course pushes you about as far as your academic talents will take you. If you are average in math ability, you may struggle with algebra and probably fail a calculus course. If you are average in verbal skills, you often misinterpret complex text and make errors in logic.

These are not devastating shortcomings. You are smart enough to engage in any of hundreds of occupations. You can acquire more knowledge if it is presented in a format commensurate with your intellectual skills. But a genuine college education in the arts and sciences begins where your skills leave off.

In engineering and most of the natural sciences, the demarcation between high-school material and college-level material is brutally obvious. If you cannot handle the math, you cannot pass the courses. In the humanities and social sciences, the demarcation is fuzzier. It is possible for someone with an IQ of 100 to sit in the lectures of Economics 1, read the textbook, and write answers in an examination book. But students who cannot follow complex arguments accurately are not really learning economics. They are taking away a mishmash of half-understood information and outright misunderstandings that probably leave them under the illusion that they know something they do not. (A depressing research literature documents one's inability to recognize one's own incompetence.) Traditionally and properly understood, a four-year college education teaches advanced analytic skills and information at a level that exceeds the intellectual capacity of most people.

There is no magic point at which a genuine college-level education becomes an option, but anything below an IQ of 110 is problematic. If you want to do well, you should have an IQ of 115 or higher. Put another way, it makes sense for only about 15% of the population, 25% if one stretches it, to get a college education. And yet more than 45% of recent high school graduates enroll in four-year colleges. Adjust that percentage to account for high-school dropouts, and more than 40% of all persons in their late teens are trying to go to a four-year college--enough people to absorb everyone down through an IQ of 104.

2) On the highly intelligent fraction and their contributions to society.

If "intellectually gifted" is defined to mean people who can become theoretical physicists, then we're talking about no more than a few people per thousand and perhaps many fewer. They are cognitive curiosities, too rare to have that much impact on the functioning of society from day to day. But if "intellectually gifted" is defined to mean people who can stand out in almost any profession short of theoretical physics, then research about IQ and job performance indicates that an IQ of at least 120 is usually needed. That number demarcates the top 10% of the IQ distribution, or about 15 million people in today's labor force--a lot of people.

In professions screened for IQ by educational requirements--medicine, engineering, law, the sciences and academia--the great majority of people must, by the nature of the selection process, have IQs over 120. Evidence about who enters occupations where the screening is not directly linked to IQ indicates that people with IQs of 120 or higher also occupy large proportions of positions in the upper reaches of corporate America and the senior ranks of government. People in the top 10% of intelligence produce most of the books and newspaper articles we read and the television programs and movies we watch. They are the people in the laboratories and at workstations who invent our new pharmaceuticals, computer chips, software and every other form of advanced technology.

Combine these groups, and the top 10% of the intelligence distribution has a huge influence on whether our economy is vital or stagnant, our culture healthy or sick, our institutions secure or endangered. Of the simple truths about intelligence and its relationship to education, this is the most important and least acknowledged: Our future depends crucially on how we educate the next generation of people gifted with unusually high intelligence.

How assiduously does our federal government work to see that this precious raw material is properly developed? In 2006, the Department of Education spent about $84 billion. The only program to improve the education of the gifted got $9.6 million, one-hundredth of 1% of expenditures. In the 2007 budget, President Bush zeroed it out.

Just to clarify for Murray: he's not proposing a hard cutoff in IQ for any particular achievement (becoming a chip designer, writing a senior thesis in French Literature), but merely that the fraction of people capable of that achievement with IQ below the value he gives is very small (e.g., very few medical researchers with IQ less than 120). Therefore, we can use the estimated value as a way to guess what percentage of the general population is sufficiently capable. There are of course other factors involved in success at a particular task than raw cognitive ability (motivation, organization, communication skills, ...). See earler post on success vs ability.

For more on theoretical physicists, see here. This will sound terribly arrogant (so shoot me), but Murray's estimate of few per thousand is way too high. That's about the ability level of the average Caltech undergrad, and I would guess only the top 5-10% of students there could be theoretical physicists.

SES and IQ

It is sometimes claimed that IQ is just a proxy for SES (Socioeconomic Status): high IQ kids are merely the beneficiaries of better home environments, and correlations between IQ and life outcomes are merely a proxy for correlations between childhood environment and outcomes. Of course, this claim does not address the significant variations in IQ within families. Does IQ have predictive power once SES is controlled for? The answer is obvious from anecdotal experience: we all know siblings who, by definition, shared the same family SES, but with different IQs and life outcomes.

Vanderbilt psychologist David Lubinski sent me a recent paper addressing this issue -- Cognitive Epidemiology: With emphasis on untangling cognitive ability and socioeconomic status, Intelligence 37 (2009) 625-633. Among the figures is the following one, based on NLSY longitudinal data (the analysis was originally done by Charles Murray). It compares life outcomes of paired siblings (one of each pair required to be in the "normal" reference range of IQ), thereby controlling for SES. As you can see, IQ has a strong impact on education, earnings and social status even after family SES is controlled for. (Click for larger version of figure.)



Similar analysis shows that the correlation between SAT scores and college grades only decreases slightly when SES is controlled for. See also related post on non-shared environmental effects.

Below is a useful SES-SAT (or SES-IQ) syllogism, well supported by the studies at the two links above.

SES does not cause SAT (weakly at most). ***
SES does not predict college success, SAT does.

*** Someone recently asked me about the non-genetic or environmental variation in adult IQ. If we use a heritability range of .7-.9 (as deduced in twin studies; applicable for IQ in late adulthood and over a large range of childhood environments), then the resulting distribution of IQ differences for monozygotic twins would have SD = 5-8 points. That is, clones raised in different families would typically have IQ differences of around 5-8 points. Note, though, that very little of this difference is accounted for by SES -- most of it is due to non-shared environmental effects. For a lower heritability value such as .5 (perhaps more appropriate for adolescents), the expected variation between clones would be about 10 points.

Physics still pulls them in...

There is much concern among physicists that our field is no longer getting the same share of the brightest students as in the glory days (e.g., during the cold war). I was pleased to find here that average GRE-general scores in physics PhD programs were still the highest as of 2002, with math and CS close behind. According to these tables, the average IQ of a physics PhD student is about 130. For comparison, english/literature PhD students average about 120 and sociology PhD students only 115. (115 is the average for college students in general, so the typical sociology TA is no smarter than the students in his or her class ;-) Amazingly, the average grad student in education (109) is below the average for college students!

You may find it odd that I converted the results from GRE score to IQ, widely regarded by the politically correct as a discredited measure of intelligence. For a rebuttal of this often repeated, but scientifically unsupported, point of view, see, e.g., Steve Pinker's book The Blank Slate, or any academic research of the type excerpted below.

g on the Job


Although the evidence of genetic and physiological correlates of g argues powerfully for the existence of global intelligence, it has not quelled the critics of intelligence testing. These skeptics argue that even if such a global entity exists, it has no intrinsic functional value and becomes important only to the extent that people treat it as such: for example, by using IQ scores to sort, label and assign students and employees. Such concerns over the proper use of mental tests have prompted a great deal of research in recent decades. This research shows that although IQ tests can indeed be misused, they measure a capability that does in fact affect many kinds of performance and many life outcomes, independent of the tests' interpretations or applications. Moreover, the research shows that intelligence tests measure the capability equally well for all native-born English-speaking groups in the U.S.

If we consider that intelligence manifests itself in everyday life as the ability to deal with complexity, then it is easy to see why it has great functional or practical importance. Children, for example, are regularly exposed to complex tasks once they begin school. Schooling requires above all that students learn, solve problems and think abstractly. That IQ is quite a good predictor of differences in educational achievement is therefore not surprising. When scores on both IQ and standardized achievement tests in different subjects are averaged over several years, the two averages correlate as highly as different IQ tests from the same individual do. High-ability students also master material at many times the rate of their low-ability peers. Many investigations have helped quantify this discrepancy. For example, a 1969 study done for the U.S. Army by the Human Resources Research Office found that enlistees in the bottom fifth of the ability distribution required two to six times as many teaching trials and prompts as did their higher-ability peers to attain minimal proficiency in rifle assembly, monitoring signals, combat plotting and other basic military tasks. Similarly, in school settings the ratio of learning rates between "fast" and "slow" students is typically five to one.

...The existence of biological correlates of intelligence does not necessarily mean that intelligence is dictated by genes. Decades of genetics research have shown, however, that people are born with different hereditary potentials for intelligence and that these genetic endowments are responsible for much of the variation in mental ability among individuals. Last spring an international team of scientists headed by Robert Plomin of the Institute of Psychiatry in London announced the discovery of the first gene linked to intelligence. Of course, genes have their effects only in interaction with environments, partly by enhancing an individual's exposure or sensitivity to formative experiences. Differences in general intelligence, whether measured as IQ or, more accurately, as g are both genetic and environmental in origin--just as are all other psychological traits and attitudes studied so far, including personality, vocational interests and societal attitudes. This is old news among the experts. The experts have, however, been startled by more recent discoveries.

One is that the heritability of IQ rises with age--that is to say, the extent to which genetics accounts for differences in IQ among individuals increases as people get older. Studies comparing identical and fraternal twins, published in the past decade by a group led by Thomas J. Bouchard, Jr., of the University of Minnesota and other scholars, show that about 40 percent of IQ differences among preschoolers stems from genetic differences but that heritability rises to 60 percent by adolescence and to 80 percent by late adulthood. With age, differences among individuals in their developed intelligence come to mirror more closely their genetic differences. It appears that the effects of environment on intelligence fade rather than grow with time. In hindsight, perhaps this should have come as no surprise. Young children have the circumstances of their lives imposed on them by parents, schools and other agents of society, but as people get older they become more independent and tend to seek out the life niches that are most congenial to their genetic proclivities.

A second big surprise for intelligence experts was the discovery that environments shared by siblings have little to do with IQ. Many people still mistakenly believe that social, psychological and economic differences among families create lasting and marked differences in IQ. Behavioral geneticists refer to such environmental effects as "shared" because they are common to siblings who grow up together. Research has shown that although shared environments do have a modest influence on IQ in childhood, their effects dissipate by adolescence. The IQs of adopted children, for example, lose all resemblance to those of their adoptive family members and become more like the IQs of the biological parents they have never known. Such findings suggest that siblings either do not share influential aspects of the rearing environment or do not experience them in the same way. Much behavioral genetics research currently focuses on the still mysterious processes by which environments make members of a household less alike.

Flynn on the Flynn effect

Via GNXP, this lecture by James Flynn. The lecture is long but worth reading in full. His discussion of factor loading in the context of the decathalon, 100m speed, hurdling, etc. is excellent and very relevant to the subject of intelligence and its various cognitive components. The Flynn effect refers to a significant increase in raw IQ scores over the last 100 years or so -- equivalent of 30 points or more, in some cases. This raises a number of thorny issues. Were our ancestors idiots? Is IQ really that malleable under environmental influence (contrary to recent twin studies)? Is the principal component identified as "g" actually time dependent? See Flynn's answers below. A couple of comments: (i) Dramatic gains are seen only in certain areas of intelligence (see table from Sailer discussion), which are plausibly the areas in which modern life provides much more stimulation. (ii) Average people 100 years ago were massively deprived by modern standards -- much more than we could ever reproduce in a modern twin study. US GDP per capita is 10x higher now, and average years of schooling has increased dramatically. It's not surprising that a child who only received (say) 6 years of formal schooling would be far behind someone with 12. Modern twin studies only include kids raised in a much smaller range of environments -- no twin in any recent study had less than the legally mandated US high school education. With a smaller range of environmental effects, the genetic component plays a larger role, leading to the high (.5-.7 or so) heritability result for IQ (similar to height). (Flynn notes: In the America of 1900, adults had an average of about 7 years of schooling, a median of 6.5 years, and 25 percent had completed 4 years or less.) (iii) The analogy with height is quite appropriate. While taller parents tend to have taller children (i.e., height is heritable), we've seen siginificant gains in average height as nutrition and diet have improved. (iv) I think Flynn would agree that variance in adult IQ must have been much larger in the past. People like Newton or Thomas Jefferson obviously had tremendously more exposure to ideas and abstract thinking than a kid on the farm with little or no education, few books, no TV and no radio. The Flynn effect does not imply that the great geniuses of the past were necessarily inferior to those of today.

Lecture by Professor James Flynn at The Psychometrics Centre, Cambridge Assessment, University of Cambridge, 15th December 2006 Naming the paradoxes (1) The factor analysis paradox: Factor analysis shows a first principal component called "g" or general intelligence that seems to bind performance on the various WISC subtests together. However, IQ gains over time show score gains on the WISC subtests occurring independently of one another. How can intelligence be both one and many? (2) The intelligence paradox: If huge IQ gains are intelligence gains, why are we not stuck by the extraordinary subtlety of our children's conversation? Why do we not have to make allowances for the limitations of our parents? A difference of some 18 points in the average IQ over two generations ought to be highly visible. (3) The MR paradox: In 1900, the average IQ scored against current norms was somewhere between 50 and 70. If IQ gains are in any sense real, we are driven to the absurd conclusion that a majority of our ancestors were mentally retarded. (4) The identical twins paradox: Twin studies show that genes dominate individual differences in IQ and that environmental effects are feeble. IQ gains are so great as to signal the existence of environmental factors of enormous potency. How can environment be both so feeble and so potent? The solutions in shorthand (1) The WISC subtests measure a variety of cognitive skills that are functionally independent and responsive to changes in social priorities over time. The inter-correlations that engender "g" are binding only when comparing individuals within a static social context. (2) Asking whether IQ gains are intelligence gains is the wrong question because it implies all or nothing cognitive progress. The 20th century has seen some cognitive skills make great gains, while others have been in the doldrums. To assess cognitive trends, we must dissect "intelligence" into solving mathematical problems, interpreting the great works of literature, finding on-the-spot solutions, assimilating the scientific world view, critical acumen, and wisdom. (3) Our ancestors in 1900 were not mentally retarded. Their intelligence was anchored in everyday reality. We differ from them in that we can use abstractions and logic and the hypothetical to attack the formal problems that arise when science liberates thought from concrete referents. Since 1950, we have become more ingenious in going beyond previously learned rules to solve problems on the spot. (4) At a given time, genetic differences between individuals (within a cohort) are dominant but only because they have hitched powerful environmental factors to their star. Trends over time (between cohorts) liberate environmental factors from the sway of genes and once unleashed, they can have a powerful cumulative effect.

Intelligence, nature and nurture

Some new data on genetic vs environmental influences on IQ in this recent Times magazine article. Until recently, twins studies could only examine the effect of environmental variation within a limited range -- from working to upper class -- because very poor families are generally not allowed to adopt babies. The effect of family background has been found to recede to almost nothing by late adulthood in these twins studies, but the possibility that severe deprivation might have a stronger effect has not been ruled out. Recent investigations, as detailed below, have focused on very poor families and found a significant effect. We might characterize this as discovering the non-linear region of gene--family environment interaction :-)

NYTimes: A century’s worth of quantitative-genetics literature concludes that a person’s I.Q. is remarkably stable and that about three-quarters of I.Q. differences between individuals are attributable to heredity. This is how I.Q. is widely understood — as being mainly “in the genes” — and that understanding has been used as a rationale for doing nothing about seemingly intractable social problems like the black-white school-achievement gap and the widening income disparity. If nature disposes, the argument goes, there is little to be gained by intervening. In their 1994 best seller, “The Bell Curve,” Richard Herrnstein and Charles Murray relied on this research to argue that the United States is a genetic meritocracy and to urge an end to affirmative action. Since there is no way to significantly boost I.Q., prominent geneticists like Arthur Jensen of Berkeley have contended, compensatory education is a bad bet.

...When quantitative geneticists estimate the heritability of I.Q., they are generally relying on studies of twins. Identical twins are in effect clones who share all their genes; fraternal twins are siblings born together — just half of their genes are identical. If heredity explains most of the difference in intelligence, the logic goes, the I.Q. scores of identical twins will be far more similar than the I.Q.’s of fraternal twins. And this is what the research has typically shown. Only when children have spent their earliest years in the most wretched of circumstances, as in the infamous case of the Romanian orphans, treated like animals during the misrule of Nicolae Ceausescu, has it been thought that the environment makes a notable difference. Otherwise, genes rule.

Then along came Eric Turkheimer to shake things up. Turkheimer, a psychology professor at the University of Virginia, is the kind of irreverent academic who gives his papers user-friendly titles like “Spinach and Ice Cream” and “Mobiles.” He also has a reputation as a methodologist’s methodologist. In combing through the research, he noticed that the twins being studied had middle-class backgrounds. The explanation was simple — poor people don’t volunteer for research projects — but he wondered whether this omission mattered.

Together with several colleagues, Turkheimer searched for data on twins from a wider range of families. He found what he needed in a sample from the 1970’s of more than 50,000 American infants, many from poor families, who had taken I.Q. tests at age 7. In a widely-discussed 2003 article, he found that, as anticipated, virtually all the variation in I.Q. scores for twins in the sample with wealthy parents can be attributed to genetics. The big surprise is among the poorest families. Contrary to what you might expect, for those children, the I.Q.’s of identical twins vary just as much as the I.Q.’s of fraternal twins. The impact of growing up impoverished overwhelms these children’s genetic capacities. In other words, home life is the critical factor for youngsters at the bottom of the economic barrel. “If you have a chaotic environment, kids’ genetic potential doesn’t have a chance to be expressed,” Turkheimer explains. “Well-off families can provide the mental stimulation needed for genes to build the brain circuitry for intelligence.”

Income, wealth, and IQ

I'm occasionally asked about financial returns to cognitive ability. As a rough rule of thumb, judging from the graphs below (obtained here), I would say:

On average, an increase of IQ by one SD corresponds to  ~ $30k per annum of additional income. (Somewhat less than 1 SD in income; the distribution is far from normal.)

By early middle age, individuals > 90th percentile in IQ have, typically, more than twice the wealth of individuals who are of average IQ.

If you can find better data than what is shown below, please let me know. (How do bottom decile adults manage to earn ~ $40k per annum, on average? Does this include transfer payments?)

Of course, you can turn this around to estimate the increased (heritable) cognitive ability endowments of high income parents relative to average parents. That might help to clarify causality in results such as this one: "Studies show that children from low-income families have smaller brains and lower cognitive abilities." (Even Nature susceptible to faulty logic.)

Note, there is good evidence that positive returns to IQ persist above high thresholds (e.g., IQ=120, or even top 1 percent ability). See here and here.

Income mobility is strongly affected by IQ. In fact, IQ is a much stronger predictor variable than race for escaping the bottom quintile of income (Pew Trust report; NLSY again, AFQT=IQ scores):

This last figure is very problematic for the "Social Status/Wealth causes IQ" position. It seems to be the other way around: the kids escaping bottom quintile childhoods all experienced poverty, but the ones with higher cognitive ability were more likely to move up. (Recall that adopted children tend to resemble their biological parents much more than their adoptive ones; family environment has a limited effect on IQ, which is highly heritable.)

Pew: Individuals with higher test scores in adolescence are more likely to move out of the bottom quintile, and test scores can explain virtually the entire black-white mobility gap. Figure 13 plots the transition rates against percentiles of the AFQT test score distribution. The upward-sloping lines indicate that, as might be expected, individuals with higher test scores are much more likely to leave the bottom income quintile. For example, for whites, moving from the first percentile of the AFQT distribution to the median roughly doubles the likelihood from 42 percent to 81 percent. The comparable increase for blacks is even more dramatic, rising from 33 percent to 78 percent. Perhaps the most stunning finding is that once one accounts for the AFQT score, the entire racial gap in mobility is eliminated for a broad portion of the distribution. At the very bottom and in the top half of the distribution a small gap remains, but it is not statistically significant.

Supercomputers and the mystery of IQ

This article appeared today in the South China Morning Post, one of the leading newspapers in HK. I fixed a couple of typos in what appears below, but can neither confirm nor deny the accuracy of the entire article ;-)

Scientists seek to unravel the mystery of IQ: Hong Kong supercomputers will power unique genetic study of students

Fiona Tam
Updated on Dec 04, 2010

Some of the world's fastest supercomputers are being set up in Hong Kong to address the age-old mystery of human intelligence.

The study of intelligence quotient (IQ) is being conducted by BGI Hong Kong, [formerly] known as the Beijing Genomics Institute. It will survey DNA samples from 1,000 child prodigies from China's best high schools, comparing them with samples from 1,000 children of average intelligence, searching for genetic variations.

The study will examine protein coding genes of the extremely smart children, many of whom are expected to enroll at Harvard, Yale or Cambridge. The results will be correlated with each youngster's school test scores, in hopes of learning how specific genetic variations affect intelligence.

The study, which started in 2009 in Shenzhen, is moving to a new facility in Tai Po. By the end of this month, 115 of the world's fastest sequencers - the HiSeq 2000 - will have relocated to the city. They will be able to sequence the equivalent of 1,000 human genomes a day, and soon surpass the entire sequencing output of the United States to become the world's largest sequencing centre.

The study by BGI, which receives strong financial backing from the Shenzhen and mainland governments, will be the largest-scale examination of its kind. Ethical and privacy concerns have hindered such work in America and Europe.

In fact, ethical concerns haunt this entire subject. Ever since Nazi Germany misused science to support its murderous racist and anti-Semitic theories, Western societies have been extremely sensitive about linking genetics to IQ. Nevertheless, much scientific research suggests that IQ is strongly affected by heredity, although environment, education and nutrition also play a significant role.

According to Professor Steve Hsu, who comes to the study from the University of Oregon, scientists have identified several candidate genes that may relate to IQ, although researchers are not yet sure. He said about 50 per cent of humans' IQ is substantially [heritable]. "Scientists believe there are many genes that affect IQ, but none of them has been definitively discovered so far," he said. "Although there are some candidates in published work, so far the results have not been replicated." So-called IQ genes are those that influence cognitive capabilities, including verbal and spatial abilities, perceptual speed, memory and math skills.

The BGI study, however, did not begin by focusing on suspected IQ genes nor on candidate areas. Instead, geneticists from BGI, together with researchers from Harvard, Oregon and San Diego, are surveying markers across complete sets of genomes, using a method called GWAS (Genome Wide Association Study).

The GWAS method has proved particularly useful in finding genetic variations that contribute to complex diseases such as asthma, cancer, diabetes, heart disease and mental illnesses. Researchers first obtain DNA samples from each child by rubbing a swab inside his or her mouth to collect cells. DNA is extracted from the cells and spread on tiny gene chips, which can be later read and automatically surveyed by sequencers in the laboratory.

Specialists known as bioinformaticians will then build a genetic model and calculate the differences in genetic variations between the two groups of youngsters using supercomputers. When variations of brain-related genomes are found to be significantly more frequent in smart children than average ones, these variations will be associated with intelligence. Researchers will use these variations to identify the genetic differences that determine IQ.

Hsu said: "Typical genotyping [or analysis] scans the markers of genetic variation, known as single-nucleotide polymorphism[s], or SNPs. However, scanning the SNPs only gives researchers a crude picture. BGI's sequencing capacity enables us to scan the whole genome rather than just the SNPs." Hsu, a theoretical physicist, is helping the team improve its model design and calculations.

Yin Ye, BGI's vice-president, said researchers have already made some discoveries about IQ genes, but "it's too early to make public our results". While reluctant to give further details about the experiment, he took pains to say the study was essentially focused on studying students' aptitudes and suggesting occupations that would make the best use of their capabilities.

Hsu said the genetic basis for intelligence and other characteristics or traits - known as phenotypes - was likely to be discovered in the next five or ten years.

"Even if our planned study fails," Hsu said, "it's clear on the basis of trends in cost and capacity of sequencing [that] massive GWAS involving 100,000 or 1,000,000 individuals are right around the corner."

The idea of probing the genetic basis for human intelligence came after Beijing high school student Zhao Bowen, 17, who came to BGI on a summer internship last year to work on cucumbers, solved an assignment within a few hours that scientists expected to take him weeks.

Zhao is working as a full-time researcher now, and he will study the genes of 1,000 of his best-performing schoolmates from the affiliated high school of China's prestigious Renmin University, where some of the smartest children from across the country have been sent. It's a collaboration project between the institute and the high school.

Liu Pengzhi , principal of the affiliated high school, told parents in a forum that prodigies accounted for 1 to 3 per cent of the population, and the school was working closely with BGI to develop talented children, in addition to the high school's own experimental classes for nurturing prodigies.

Annals of psychometry: IQs of eminent scientists

[ See this 2016 post for the original papers / book and the identities of the 64 scientists. ]

I recently came across a 1950s study of eminent scientists by Harvard psychologist Anne Roe: The Making of a Scientist (1952). Her study is by far the most systematic and sophisticated that I am aware of. She selected 64 eminent scientists -- well known, but not quite at the Nobel level -- in a more or less random fashion, using, e.g., membership lists of scholarly organizations and expert evaluators in the particular subfields. Roughly speaking, there were three groups: physicists (divided into experimental and theoretical subgroups), biologists (including biochemists and geneticists) and social scientists (psychologists, anthropologists).

The Making of a Scientist devotes only one chapter to psychometrics. The other chapters describe the motivation for the study, how the 64 scientists were selected, interviews with the scientists, details of their family history, work life, etc.

Roe devised her own high-end intelligence tests as follows: she obtained difficult problems in verbal, spatial and mathematical reasoning from the Educational Testing Service, which administers the SAT, but also performs bespoke testing research for, e.g., the US military. Using these problems, she created three tests (V, S and M), which were administered to the 64 scientists, and also to a cohort of PhD students at Columbia Teacher's College. The PhD students also took standard IQ tests and the results were used to norm the high-end VSM tests using an SD = 15. Most IQ tests are not good indicators of true high level ability (e.g., beyond +3 SD or so).

Average ages of subjects: mid-40s for physicists, somewhat older for other scientists

Overall normed scores:

Test (Low / Median / High)

V 121 / 166 / 177

S 123 / 137 / 164

M 128 / 154 / 194

Roe comments: (1) V test was too easy for some takers, so top score no ceiling. (2) S scores tend to decrease with age (correlation .4). Peak (younger) performance would have been higher. (3) M test was found to be too easy for the physicists; only administered to other groups.

It is unlikely that any single individual obtained all of the low scores, so each of the 64 would have been strongly superior in at least one or more areas.

Median scores (raw) by group:

group (V / S / M)

Biologists 56.6 / 9.4 / 16.8
Exp. Physics 46.6 / 11.7 / *
Theo. Physics 64.2 / 13.8 / *
Psychologists 57.7 / 11.3 / 15.6
Anthropologists 61.1 / 8.2 / 9.2

The lowest score in each category among the 12 theoretical physicists would have been roughly V 160 (!) S 130 M >> 150. (Ranges for all groups are given, but I'm too lazy to reproduce them here.) It is hard to estimate the M scores of the physicists since when Roe tried the test on a few of them they more or less solved every problem modulo some careless mistakes. Note the top raw scores (27 out of 30 problems solved) among the non-physicists (obtained by 2 geneticists and a psychologist), are quite high but short of a full score. The corresponding normed score is 194!

The lowest V scores in the 120-range were only obtained by 2 experimental physicists, all other scientists scored well above this level -- note the median is 166.

My comments:

The results strongly suggest that high IQ provides a significant advantage in science. It is sometimes claimed that IQ is irrelevant beyond a threshold: more precisely, that the advantage conferred by IQ above some threshold (e.g., 120) decreases significantly as other factors like drive or creativity take precedence. But, if that were the case it would be unlikely to have found such high scores in this group. The average IQ of a science PhD is roughly 130, and individuals with IQs in the higher range described above constitute a tiny fraction of all scientists. If IQ were irrelevant above 130 we would expect the most eminent group to have an average similar to the overall population of scientists.

Conversely, I think one should be impressed that a simple test which can be administered in a short period of time (e.g., 30 minutes for Roe's high-end exams) offers significant predictive power. While it is not true that anyone with a high IQ can or will become a great scientist (certainly other factors like drive, luck, creativity play a role), one can nevertheless easily identify the 95 percent (even 99 percent) of the population for whom success in science is highly improbable. Psychometrics works!

The scores for theoretical physicists confirm an estimate made to me by a famous colleague many years ago, that only 1 in 100,000 people could do high level theoretical physics.

Feynman's 124: in this context one often hears of Feynman's modest grade school IQ score of 124. To understand this score we have to remember that typical IQ tests (e.g., administered to public school children) tend to have low ceilings. They are not of the kind that Roe used in her study. One can imagine that the ceiling on Feynman's exam was roughly 135 (say, 99th percentile). If Feynman received the highest score on the mathematical portion, and a modest score of 115 on the verbal, we can easily understand the resulting average of 124. However, it is well known that Feynman was extremely strong mathematically. He was asked on short notice to take the Putnam exam for MIT as a senior, and received the top score in the country that year! On Roe's test Feynman's math score would presumably have been > 190, with a correspondingly higher composite IQ.

I thought I should put this post up now, as the new book by Malcolm Gladwell, Outliers: Why Some People Succeed and Some Don’t is out soon and will surely handicap the discourse on this subject for years to come :-)