PISA 2023 and the Gloomy Prospect

I'm in the Philippines now. I flew here after the semester ended, in order to meet with outsourcing (BPO = Business Process Outsourcing) companies that run call centers for global brands. This industry accounts for ~8% of Philippine GPD (~$40B per annum), driven by comparative advantages such as the widespread use of English here and relatively low wages. 

I predict that AIs of the type produced by my startup SuperFocus.ai will disrupt the BPO industry in coming years, with dramatic effects on the numbers of humans employed in areas like customer support. I was just interviewed for the podcast of the AI expert at IBPAP, the BPO trade association - he is tasked with helping local companies adopt AI technology, and adapt to a world with generative LLMs like GPT4. I'll publish a link to that interview when it goes live. 

During my visit the latest PISA results were released. This year they provided data with students grouped by Socio-Economic Status [1], so that students in different countries, but with similar levels of wealth and access to educational resources, can be compared directly. See figures below - OECD mean ~500, SD~100. 

https://www.oecd.org/publication/pisa-2022-results/country-notes/

Quintiles are defined using the *entire* international PISA student pool. These figures allow us to compare equivalent SES cohorts across countries and to project how developing countries will perform as they get richer and improve schooling.

In some countries, such as Turkey or Vietnam, the small subset of students that are in the top quintile of SES (among all PISA students tested) already score better than the OECD average for students with similar SES. On the other hand, for most developing countries, such as the Philippines, Indonesia, Saudi Arabia, Brazil, Mexico, etc. even the highest quintile SES students score similarly to or worse than the most deprived students in, e.g., Turkey, Vietnam, Japan, etc.

Note the top 20% SES quintile among all PISA takers is equivalent to roughly top ~30% SES among Japanese. If the SES variable is even crudely accurate, typical kids in this category are not deprived in any way and should be able to achieve their full cognitive potential. In developing countries only a small percentage of students are in this quintile - they are among the elites with access to good schools, nutrition, and potentially with educated parents. Thus it is very bad news that even this subgroup of students score so poorly in almost all developing countries (with exceptions like Turkey and Vietnam). It leads to gloomy projections regarding human capital, economic development, etc. in most of the developing world. 

I had not seen a similar SES analysis before this most recent PISA report. I was hoping to see data showing catch up in cognitive ability with increasing SES in developing countries. The results indicate that cognitive gaps will be very difficult to ameliorate.

In summary, the results suggest that many of these countries will not reach OECD-average levels of human capital density even if they somehow catch up in per capita GDP.

This suggests a Gloomy Prospect for development economics. Catch up in human capital density looks difficult for most developing countries, with only a few exceptions (e.g., Turkey, Vietnam, Iran, etc.).
 

Here is the obligatory US students by ancestry group vs Rest of World graph that reflects: 1. strong US spending on education (vs Rest of World) and 2. selective immigration to the US, at least for some groups.

 

SMPY 65: Help support the SMPY Longitudinal Study

The Study of Mathematically Precocious Youth (SMPY) needs your help to support the Age-65 phase of their unique longitudinal study. 

For decades, co-directed by David Lubinski and Camilla P. Benbow, SMPY has been a beacon of enlightenment, tracking five cohorts comprising over 5,000 remarkably gifted individuals. In doing so, we have unraveled the secrets to nurturing brilliance. However, we are confronted with a disconcerting reality: the effective methods to identify and cultivate intellectual talent are under siege, threatened by political ideology. 

Our 14-minute documentary and the 3-page feature in Nature underscore the dire need to provide our most gifted youths with the educational opportunities they deserve. They are the architects of solutions and the architects of the future itself. 

Here are some compelling longitudinal findings from SMPY's extensive research:

• Prodigies destined for eminent careers can be identified as early as age 13. 

• There is no plateau of ability; even within the top 1%, variations in mathematical, spatial, and verbal abilities profoundly impact educational, occupational, and creative outcomes. 

• The blend of specific abilities, such as mathematical, spatial, and verbal aptitudes, shapes the nature of one's accomplishments and career trajectory.

More information:

Long

Short

DONATE HERE

Indicate "Please designate this gift to Study of Mathematically Precocious Youth" in the Special Instructions.

Meritocracy, SAT Scores, and Laundering Prestige at Elite Universities — Manifold #43

 

I discuss 10 key graphs related to meritocracy and university admissions. Predictive power of SATs and other factors in elite admissions decisions. College learning outcomes - what do students learn? The four paths to elite college admission. Laundering prestige at the Ivies. 

Slides: 

https://docs.google.com/presentation/d/1n-nwoeKe_DcA5tJxTwqTeZBEY7nObxkujKLxVfAzRAY/edit?usp=sharing 

Audio Only and Transcript: 

https://www.manifold1.com/episodes/meritocracy-sat-scores-and-laundering-prestige-at-elite-universities-43

CLA and college learning outcomes

https://infoproc.blogspot.com/2015/01/measuring-college-learning-outcomes.html 

Harvard Veritas: Interview with a recent graduate 

https://infoproc.blogspot.com/2022/08/harvard-veritas-interview-with-recent.html 

Defining Merit - Human Capital and Harvard University

https://infoproc.blogspot.com/2009/11/defining-merit.html 

Chapter markers: 

0:00 Introduction 

1:28 University of California system report and the use of SAT scores admissions 

8:04 Longitudinal study on gifted students and SAT scores (SMPY) 

12:53 Unprecedented data on earnings outcomes and SAT scores 

15:43 How SAT scores and university pedigree influence opportunities at elite firms 

17:35 Non-academic factors fail to predict student success 

20:49 Predicted earnings 

24:24 Measured benefit of Ivy Plus attendance 

28:25 CLA: 13 university study on college learning outcomes 

32:34 Does college education improve generalist skills and critical thinking? 

42:15 The composition of elite universities: 4 paths to admission 

48:12 What happened to meritocracy? 

51:48 Hard versus Soft career tracks 

54:43 Cognitive elite at Ivies vs state flagship universities 

57:11 What happened to Caltech?

Kathryn Paige Harden Profile in The New Yorker (Behavior Genetics)

This is a good profile of behavior geneticist Paige Harden (UT Austin professor of psychology, former student of Eric Turkheimer), with a balanced discussion of polygenic prediction of cognitive traits and the culture war context in which it (unfortunately) exists.

Can Progressives Be Convinced That Genetics Matters? 

The behavior geneticist Kathryn Paige Harden is waging a two-front campaign: on her left are those who assume that genes are irrelevant, on her right those who insist that they’re everything. 

Gideon Lewis-Kraus

Gideon Lewis-Kraus is a talented writer who also wrote a very nice article on the NYTimes / Slate Star Codex hysteria last summer.

Some references related to the New Yorker profile:

1. The paper Harden was attacked for sharing while a visiting scholar at the Russell Sage Foundation: Game Over: Genomic Prediction of Social Mobility 

2. Harden's paper on polygenic scores and mathematics progression in high school: Genomic prediction of student flow through high school math curriculum 

3. Vox article; Turkheimer and Harden drawn into debate including Charles Murray and Sam Harris: Scientific Consensus on Cognitive Ability?

A recent talk by Harden, based on her forthcoming book The Genetic Lottery: Why DNA Matters for Social Equality

Regarding polygenic prediction of complex traits 

I first met Eric Turkheimer in person (we had corresponded online prior to that) at the Behavior Genetics Association annual meeting in 2012, which was back to back with the International Conference on Quantitative Genetics, both held in Edinburgh that year (photos and slides [1] [2] [3]). I was completely new to the field but they allowed me to give a keynote presentation (if memory serves, together with Peter Visscher). Harden may have been at the meeting but I don't recall whether we met. 

At the time, people were still doing underpowered candidate gene studies (there were many talks on this at BGA although fewer at ICQG) and struggling to understand GCTA (Visscher group's work showing one can estimate heritability from modestly large GWAS datasets, results consistent with earlier twins and adoption work). Consequently a theoretical physicist talking about genomic prediction using AI/ML and a million genomes seemed like an alien time traveler from the future. Indeed, I was.

My talk is largely summarized here:

On the genetic architecture of intelligence and other quantitative traits 

https://arxiv.org/abs/1408.3421 

How do genes affect cognitive ability or other human quantitative traits such as height or disease risk? Progress on this challenging question is likely to be significant in the near future. I begin with a brief review of psychometric measurements of intelligence, introducing the idea of a "general factor" or g score. The main results concern the stability, validity (predictive power), and heritability of adult g. The largest component of genetic variance for both height and intelligence is additive (linear), leading to important simplifications in predictive modeling and statistical estimation. Due mainly to the rapidly decreasing cost of genotyping, it is possible that within the coming decade researchers will identify loci which account for a significant fraction of total g variation. In the case of height analogous efforts are well under way. I describe some unpublished results concerning the genetic architecture of height and cognitive ability, which suggest that roughly 10k moderately rare causal variants of mostly negative effect are responsible for normal population variation. Using results from Compressed Sensing (L1-penalized regression), I estimate the statistical power required to characterize both linear and nonlinear models for quantitative traits. The main unknown parameter s (sparsity) is the number of loci which account for the bulk of the genetic variation. The required sample size is of order 100s, or roughly a million in the case of cognitive ability.

The predictions in my 2012 BGA talk and in the 2014 review article above have mostly been validated. Research advances often pass through the following phases of reaction from the scientific community:

1. It's wrong ("genes don't affect intelligence! anyway too complex to figure out... we hope")

2. It's trivial ("ofc with lots of data you can do anything... knew it all along")

3. I did it first ("please cite my important paper on this")

Or, as sometimes attributed to Gandhi: "First they ignore you, then they laugh at you, then they fight you, then you win.”

See also To Predict the Future it is useful to understand the Past.

Technical note: 

In 2014 I estimated that ~1 million genotype | phenotype pairs would be enough to capture most of the common SNP heritability for height and cognitive ability. This was accomplished for height in 2017. However, the sample size of well-phenotyped individuals is much smaller for cognitive ability, even in 2021, than for height in 2017. For example, in UK Biobank the cognitive test is very brief (~5 minutes IIRC, a dozen or so questions), but it has not even been administered to the full cohort as yet. In the Educational Attainment studies the phenotype EA is only moderately correlated (~0.3 ?) or so with actual cognitive ability.

Hence, although the most recent EA4 results use 3 million individuals [1], and produce a predictor which correlates ~0.4 with actual EA, the statistical power available is still less than what I predicted would be required to train a really good cognitive ability predictor.

In our 2017 height paper, which also briefly discussed bone density and cognitive ability prediction, we built a cognitve ability predictor roughly as powerful as EA3 using only ~100k individuals with the noisy UKB test data. So I remain confident that  ~million individuals with good cognitive scores (e.g., SAT, AFQT, full IQ test) would deliver results far beyond what we currently have available. We also found that our predictor, built using actual (albeit noisy) cognitive scores exhibits less power reduction in within-family (sibling) analyses compared to EA. So there is evidence that (no surprise) EA is more influenced by environmental factors, including so-called genetic nurture effects, than is cognitive ability.

A predictor which captures most of the common SNP heritability for cognitive ability might correlate ~0.5 or 0.6 with actual ability. Applications of this predictor in, e.g., studies of social mobility or educational success or even longevity using existing datasets would be extremely dramatic.

The Inheritors and The Grisly Folk: H.G. Wells and William Golding on Neanderthals

Some time ago I posted about The Grisly Folk by H.G. Wells, an essay on Neanderthals and their encounters with modern humans. See also The Neanderthal Problem, about the potential resurrection of early hominids via genomic technology, and the associated ethical problems. 

The Grisly Folk: ... Many and obstinate were the duels and battles these two sorts of men fought for this world in that bleak age of the windy steppes, thirty or forty thousand years ago. The two races were intolerable to each other. They both wanted the eaves and the banks by the rivers where the big flints were got. They fought over the dead mammoths that had been bogged in the marshes, and over the reindeer stags that had been killed in the rutting season. When a human tribe found signs of the grisly folk near their cave and squatting place, they had perforce to track them down and kill them; their own safety and the safety of their little ones was only to be secured by that killing. The Neandertalers thought the little children of men fair game and pleasant eating. ...

William Golding was inspired by Wells to write The Inheritors (his second book, after Lord of the Flies), which is rendered mostly (until the end, at which point the perspective is reversed) from the Neanderthal point of view. Both Wells and Golding assume that Neanderthals were not as cognitively capable as modern humans, but Golding's primitives are peaceful quasi-vegetarians, quite unlike the Grisly Folk of Wells.

The Inheritors 

Golding considered this his finest novel and it is a beautifully realised tale about the last days of the Neanderthal people and our fear of the ‘other’ and the unfamiliar. The action is revealed through the eyes of the Neanderthals whose peaceful world is threatened by the emergence of Homo sapiens. 

The struggle between the simple Neanderthals and the malevolent modern humans ends in helpless despair ... 

From the book jacket: "When the spring came the people - what was left of them - moved back by the old paths from the sea. But this year strange things were happening, terrifying things that had never happened before. Inexplicable sounds and smells; new, unimaginable creatures half glimpsed through the leaves. What the people didn't, and perhaps never would, know, was that the day of their people was already over."

See this episode of the podcast Backlisted for an excellent discussion of the book. 

I am particularly interested in how Golding captures the perspective of pre-humans with limited cognitive abilities. He conveys the strangeness and incomprehensibility of modern humans as perceived by Neanderthals. In this sense, the book is a type of Science Fiction: it describes a first encounter with Aliens of superior capability.

We are approaching the day when modern humans will encounter a new and quasi-alien intelligence: it may be AI, or it may be genetically enhanced versions of ourselves.

On a scientific note, can someone provide an update to this 2013 work: "... high quality genome sequence obtained from the toe of a female Neanderthal who lived in the Altai mountains in Siberia. Interestingly, copy number variation at 16p11.2 is one of the structural variants identified in a recent deCODE study as related to IQ depression"? Here is an interesting follow up paper: Nature 2016 Aug 11; 536(7615): 205–209.

   

Audiobook:

 

A.J. Robison on the Neural Basis of Sex Differences in Depression - Manifold #37

Corey and Steve talk with MSU Neuroscientist A.J. Robison about why females may be more likely to suffer from depression than males. A.J. reviews past findings that low testosterone and having a smaller hippocampus may predict depression risk. He explains how a serendipitous observation opened up his current line of research and describes tools he uses to study neural circuits. Steve asks about the politics of studying sex differences and tells of a start up using CRISPR to attack heart disease. The three end with a discussion of the psychological effects of ketamine, testosterone and deep brain stimulation.

01:18 - Link between antidepressants, neurogenesis and reducing risk of depression

13:54 - Nature of Mouse models

23:19 - Depressive symptoms in mouse

32:36 - Liz Williams' serendipitous finding and the issue of biological sex

45:47 - AJ’s research plans for circuit specific gene editing in the mouse brain and a start up’s plan to use it to tackle human cardiovascular disease

59:07 - Psychological and Neurological Effects of Ketamine. Testosterone and Deep Brain Stimulation

Transcript

Robison Lab at MSU

Androgen-dependent excitability of mouse ventral hippocampal afferents to nucleus accumbens underlies sex-specific susceptibility to stress

Emerging role of viral vectors for circuit-specific gene interrogation and manipulation in rodent brain


man·i·fold /ˈmanəˌfōld/ many and various.

In mathematics, a manifold is a topological space that locally resembles Euclidean space near each point.

Steve Hsu and Corey Washington have been friends for almost 30 years, and between them hold PhDs in Neuroscience, Philosophy, and Theoretical Physics. Join them for wide ranging and unfiltered conversations with leading writers, scientists, technologists, academics, entrepreneurs, investors, and more.

Steve Hsu is VP for Research and Professor of Theoretical Physics at Michigan State University. He is also a researcher in computational genomics and founder of several Silicon Valley startups, ranging from information security to biotech. Educated at Caltech and Berkeley, he was a Harvard Junior Fellow and held faculty positions at Yale and the University of Oregon before joining MSU.

Corey Washington is Director of Analytics in the Office of Research and Innovation at Michigan State University. He was educated at Amherst College and MIT before receiving a PhD in Philosophy from Stanford and a PhD in a Neuroscience from Columbia. He held faculty positions at the University Washington and the University of Maryland. Prior to MSU, Corey worked as a biotech consultant and is founder of a medical diagnostics startup.

Physics as a Strange Attractor

Almost every student who attends a decent high school will be exposed to Special Relativity. Their science/physics teacher may not really understand it very well, may do a terrible job trying to explain it. But the kid will have to read a textbook discussion and (in the internet age) can easily find more with a simple search.

Wikipedia entry on Special Relativity:

In Albert Einstein's original pedagogical treatment, it is based on two postulates:

1. The laws of physics are invariant (i.e., identical) in all inertial systems (i.e., non-accelerating frames of reference).

2. The speed of light in a vacuum is the same for all observers, regardless of the motion of the light source.

What happens next depends, of course, on the kid. I posit that above a certain (perhaps very high) threshold in g and in intellectual curiosity, almost everyone will invest some hours to think about this particular topic. Special Relativity is fundamental to our understanding of space and time and causality, and has a certain intellectual and cultural glamour. Furthermore, it is amazing that a simple empirical observation like 2 above has such deep and significant consequences. A bright individual who invests those few hours is likely to come away with an appreciation of the beauty and power of physics and the mathematical approach to natural science.

I suspect that Special Relativity, because it is easy to introduce (no mathematics beyond algebra is required), yet deep and beautiful and counterintuitive, stimulates many people of high ability to become interested in physics.

So what can one conclude about an educated adult who does not understand Special Relativity? Does it suggest an upper bound (albeit perhaps very high) on a combination of their cognitive ability and intellectual curiosity? I mention curiosity (perhaps better to say interest in first principles or deep knowledge) because of course some (how many?) people of high ability will simply not be interested in the topic. However, as ability level increases the amount of effort necessary to learn and retain the information decreases. So someone with very off-scale ability would have to be quite incurious not to absorb and retain some basic understanding of relativity, if only from school days.

Years ago I was discussing a particle accelerator facility with a distinguished (internationally renowned) engineering professor. I mentioned that the particles in the beam would reach a certain fraction of the speed of light. He asked me why they could not reach or surpass the speed of light. It became obvious that he had essentially zero understanding of Special Relativity, and I was shocked.

We could go a bit further. General Relativity (also an invention of Einstein) describes the dynamics of spacetime (sound interesting?), and is connected to topics in popular culture such as black holes, time travel, wormholes, galactic empires, etc. General Relativity is far more complex than Special Relativity, but can be introduced to someone who has a good understanding of multivariable calculus. For example, Dirac's lecture notes on the subject provide a pedagogical introduction in only 62 pages. Yet what fraction of adults have even a modest grasp of this topic? Perhaps one in ten or a hundred thousand at best.

What is the cognitive threshold to learn Special or General Relativity? What is the cognitive threshold to remember something about it ten or twenty years later? Is the cognitive threshold higher, or the threshold in intellectual curiosity required to ponder such things?

See also One hundred thousand brains and Quantum GDP.

Intuition and the two brains, revisited

Iain McGilchrist, author of The Master and His Emissary: The Divided Brain and the Making of the Western World, in conversation with Jordan Peterson.

I wrote about McGilchrist in 2012: Intuition and the two brains.

Albert Einstein:
“The intuitive mind is a sacred gift and the rational mind is a faithful servant. We have created a society that honors the servant and has forgotten the gift.”

Wigner on Einstein and von Neumann:
"But Einstein's understanding was deeper even than von Neumann's. His mind was both more penetrating and more original than von Neumann's. And that is a very remarkable statement. Einstein took an extraordinary pleasure in invention. Two of his greatest inventions are the Special and General Theories of Relativity; and for all of Jansci's brilliance, he never produced anything as original."

From Schwinger's Feynman eulogy:
"An honest man, the outstanding intuitionist of our age..."

Feynman:
"We know a lot more than we can prove."


... "if the brain is all about making connections, why is it that it's evolved with this whopping divide down the middle?"

... [chicks] use the eye connected to the left hemisphere to attend to the fine detail of picking seeds from amongst grit, whilst the other eye attends to the broader threat from predators. According to the author, "The left hemisphere has its own agenda, to manipulate and use the world"; its world view is essentially that of a mechanism. The right has a broader outlook, "has no preconceptions, and simply looks out to the world for whatever might be. In other words it does not have any allegiance to any particular set of values."

... "The right hemisphere sees a great deal, but in order to refine it, and to make sense of it in certain ways---in order to be able to use what it understands of the world and to be able to manipulate the world---it needs to delegate the job of simplifying it and turning it into a usable form to another part of the brain" [the left hemisphere]. ... the left hemisphere has a "narrow, decontextualised and theoretically based model of the world which is self consistent and is therefore quite powerful" and to the problem of the left hemisphere's lack of awareness of its own shortcomings; whilst in contrast, the right hemisphere is aware that it is in a symbiotic relationship.

Roger Sperry: ... each hemisphere is "indeed a conscious system in its own right, perceiving, thinking, remembering, reasoning, willing, and emoting, all at a characteristically human level, and . . . both the left and the right hemisphere may be conscious simultaneously in different, even in mutually conflicting, mental experiences that run along in parallel."

Much more here.

Split-brain structure (with the different hemispheres having very distinct structures and morphologies) is common to all higher organisms (as far as I know). Is this structure just an accident of evolution? Or does the (putative) split between a systematizing core and a big-picture intuitive core play an important role in higher cognition?

AGI optimists sometimes claim that deep learning and existing neural net structures are capable of taking us all the way to AGI (human-like cognition and beyond). I think there is a significant chance that neural-architectural structures necessary for, e.g., recurrent memory, meta-reasoning, theory of mind, creative generation of ideas, integration of inferences developed from observation into more general hypotheses/models, etc. still need to be developed. Any step requiring development of novel neural architecture could easily take researchers a decade to accomplish. So a timescale > 30-50 years for AGI, even in highly optimistic scenarios, seems quite possible to me.

Game Over: Genomic Prediction of Social Mobility

[ NOTE: The PNAS paper discussed below uses the SSGAC EA3 genomic predictor, trained on over a million genomes. The EA3 paper has now appeared in Nature Genetics. ]

The figure below shows SNP-based polygenic score and life outcome (socioeconomic index, on vertical axis) in four longitudinal cohorts, one from New Zealand (Dunedin) and three from the US. Each cohort (varying somewhat in size) has thousands of individuals, ~20k in total (all of European ancestry). The points displayed are averages over bins containing 10-50 individuals. For each cohort, the individuals have been grouped by childhood (family) social economic status. Social mobility can be predicted from polygenic score. Note that higher SES families tend to have higher polygenic scores on average -- which is what one might expect from a society that is at least somewhat meritocratic. The cohorts have not been used in training -- this is true out-of-sample validation. Furthermore, the four cohorts represent different geographic regions (even, different continents) and individuals born in different decades.

Everyone should stop for a moment and think carefully about the implications of the paragraph above and the figure below.

Caption from the PNAS paper.

Fig. 4. Education polygenic score associations with social attainment for Add Health Study, WLS, Dunedin Study, and HRS participants with low-, middle-, and high-socioeconomic status (SES) social origins. The figure plots polygenic score associations with socioeconomic attainment for Add Health Study (A), Dunedin Study (B), WLS (C), and HRS (D) participants who grew up in low-, middle-, and high-SES households. For the figure, low- middle-, and high-SES households were defined as the bottom quartile, middle 50%, and top quartile of the social origins score distributions for the Add Health Study, WLS, and HRS. For the Dunedin Study, low SES was defined as a childhood NZSEI of two or lower (20% of the sample), middle SES was defined as childhood NZSEI of three to four (63% of the sample), and high SES was defined as childhood NZSEI of five or six (17% of the sample). Attainment is graphed in terms of socioeconomic index scores for the Add Health Study, Dunedin Study, and WLS and in terms of household wealth in the HRS. Add Health Study and WLS socioeconomic index scores were calculated from Hauser and Warren (34) occupational income and occupational education scores. Dunedin Study socioeconomic index scores were calculated similarly, according to the Statistics New Zealand NZSEI (38). HRS household wealth was measured from structured interviews about assets. All measures were z-transformed to have mean = 0, SD = 1 for analysis. The individual graphs show binned scatterplots in which each plotted point reflects average x and y coordinates for a bin of 50 participants for the Add Health Study, WLS, and HRS and for a bin of 10 participants for the Dunedin Study. The red regression lines are plotted from the raw data. The box-and-whisker plots at the bottom of the graphs show the distribution of the education polygenic score for each childhood SES category. The blue diamond in the middle of the box shows the median; the box shows the interquartile range; and the whiskers show upper and lower bounds defined by the 25th percentile minus 1.5× the interquartile range and the 75th percentile plus 1.5× the interquartile range, respectively. The vertical line intersecting the x axis shows the cohort average polygenic score. The figure illustrates three findings observed consistently across cohorts: (i) participants who grew up in higher-SES households tended to have higher socioeconomic attainment independent of their genetics compared with peers who grew up in lower-SES households; (ii) participants’ polygenic scores were correlated with their social origins such that those who grew up in higher-SES households tended to have higher polygenic scores compared with peers who grew up in lower-SES households; (iii) participants with higher polygenic scores tended to achieve higher levels of attainment across strata of social origins, including those born into low-SES families.

The paper:

Genetic analysis of social-class mobility in five longitudinal studies, Belsky et al.

PNAS July 9, 2018. 201801238; published ahead of print July 9, 2018. https://doi.org/10.1073/pnas.1801238115

A summary genetic measure, called a “polygenic score,” derived from a genome-wide association study (GWAS) of education can modestly predict a person’s educational and economic success. This prediction could signal a biological mechanism: Education-linked genetics could encode characteristics that help people get ahead in life. Alternatively, prediction could reflect social history: People from well-off families might stay well-off for social reasons, and these families might also look alike genetically. A key test to distinguish biological mechanism from social history is if people with higher education polygenic scores tend to climb the social ladder beyond their parents’ position. Upward mobility would indicate education-linked genetics encodes characteristics that foster success. We tested if education-linked polygenic scores predicted social mobility in >20,000 individuals in five longitudinal studies in the United States, Britain, and New Zealand. Participants with higher polygenic scores achieved more education and career success and accumulated more wealth. However, they also tended to come from better-off families. In the key test, participants with higher polygenic scores tended to be upwardly mobile compared with their parents. Moreover, in sibling-difference analysis, the sibling with the higher polygenic score was more upwardly mobile. Thus, education GWAS discoveries are not mere correlates of privilege; they influence social mobility within a life. Additional analyses revealed that a mother’s polygenic score predicted her child’s attainment over and above the child’s own polygenic score, suggesting parents’ genetics can also affect their children’s attainment through environmental pathways. Education GWAS discoveries affect socioeconomic attainment through influence on individuals’ family-of-origin environments and their social mobility.

Note Added from comments: Plots would look much noisier if not for averaging many individuals into single point. Keep in mind that socioeconomic success depends on a lot more than just cognitive ability, or even cognitive ability + conscientiousness.

But, underlying predictor correlates ~0.35 with actual educational attainment, IIRC. That is, the polygenic score predicts EA about as well as standardized tests predict success in schooling.

This means you can at least use it to identify outliers: just as a very high/low test score (SAT, ACT, GRE) does not *guarantee* success/failure in school, nevertheless the signal is useful for selection = admissions.

Seven Years, Two Tweets

Is anyone keeping score?

See On the Genetic Architecture of Cognitive Ability (2014) and Nautilus Magazine: Super Intelligent Humans.

This 2011 Google Talk with @hsu_steve changed my life more than any book, political movement, or event https://t.co/SXCmoGMqNV

— Nishiki (@NishikiPrestige) July 5, 2018

2011: we sought *first* genetic variant correlated to g at genome wide statistical significance. 2018: found thousands of such SNPs and can crudely predict g from genotype alone. 2011 predictions: additive effects, highly polygenic architecture, large samples, etc. all came true!

— steve hsu (@hsu_steve) July 6, 2018

Cognitive ability predicted from fMRI (Caltech Neuroscience)

Caltech researchers used elastic net (L1 and L2 penalization) to train a predictor using cognitive scores and fMRI data from ~900 individuals. The predictor captures about 20% of variance in intelligence; the score correlates a bit more than 0.45 with actual intelligence. This may validate earlier work by Korean researchers in 2015, although the Korean group claimed much higher predictive correlations.

Press release:

In a new study, researchers from Caltech, Cedars-Sinai Medical Center, and the University of Salerno show that their new computing tool can predict a person's intelligence from functional magnetic resonance imaging (fMRI) scans of their resting state brain activity. Functional MRI develops a map of brain activity by detecting changes in blood flow to specific brain regions. In other words, an individual's intelligence can be gleaned from patterns of activity in their brain when they're not doing or thinking anything in particular—no math problems, no vocabulary quizzes, no puzzles.

"We found if we just have people lie in the scanner and do nothing while we measure the pattern of activity in their brain, we can use the data to predict their intelligence," says Ralph Adolphs (PhD '92), Bren Professor of Psychology, Neuroscience, and Biology, and director and Allen V. C. Davis and Lenabelle Davis Leadership Chair of the Caltech Brain Imaging Center.

To train their algorithm on the complex patterns of activity in the human brain, Adolphs and his team used data collected by the Human Connectome Project (HCP), a scientific endeavor funded by the National Institutes of Health (NIH) that seeks to improve understanding of the many connections in the human brain. Adolphs and his colleagues downloaded the brain scans and intelligence scores from almost 900 individuals who had participated in the HCP, fed these into their algorithm, and set it to work.

After processing the data, the team's algorithm was able to predict intelligence at statistically significant levels across these 900 subjects, says Julien Dubois (PhD '13), a postdoctoral fellow at Cedars-Sinai Medical Center. But there is a lot of room for improvement, he adds. The scans are coarse and noisy measures of what is actually happening in the brain, and a lot of potentially useful information is still being discarded.

"The information that we derive from the brain measurements can be used to account for about 20 percent of the variance in intelligence we observed in our subjects," Dubois says. "We are doing very well, but we are still quite far from being able to match the results of hour-long intelligence tests, like the Wechsler Adult Intelligence Scale,"

Dubois also points out a sort of philosophical conundrum inherent in the work. "Since the algorithm is trained on intelligence scores to begin with, how do we know that the intelligence scores are correct?" The researchers addressed this issue by extracting a more precise estimate of intelligence across 10 different cognitive tasks that the subjects had taken, not only from an IQ test. ...

Paper:

A distributed brain network predicts general intelligence from resting-state human neuroimaging data

Individual people differ in their ability to reason, solve problems, think abstractly, plan and learn. A reliable measure of this general ability, also known as intelligence, can be derived from scores across a diverse set of cognitive tasks. There is great interest in understanding the neural underpinnings of individual differences in intelligence, since it is the single best predictor of long-term life success, and since individual differences in a similar broad ability are found across animal species. The most replicated neural correlate of human intelligence to date is total brain volume. However, this coarse morphometric correlate gives no insights into mechanisms; it says little about function. Here we ask whether measurements of the activity of the resting brain (resting-state fMRI) might also carry information about intelligence. We used the final release of the Young Adult Human Connectome Project dataset (N=884 subjects after exclusions), providing a full hour of resting-state fMRI per subject; controlled for gender, age, and brain volume; and derived a reliable estimate of general intelligence from scores on multiple cognitive tasks. Using a cross-validated predictive framework, we predicted 20% of the variance in general intelligence in the sampled population from their resting-state fMRI data. Interestingly, no single anatomical structure or network was responsible or necessary for this prediction, which instead relied on redundant information distributed across the brain.

Dominic Cummings on Fighting, Physics, and Learning from tight feedback loops

Another great post from Dom.

Once something has become widely understood, it is difficult to recreate or fully grasp the mindset that prevailed before. But I can attest to the fact that until the 1990s and the advent of MMA, even "experts" (like boxing coaches, karate and kung fu instructors, Navy SEALs) did not know how to fight -- they were deeply confused as to which techniques were most effective in unarmed combat.

Soon our ability to predict heritable outcomes using DNA alone (i.e., Genomic Prediction) will be well-established. Future generations will have difficulty understanding the mindset of people (even, scientists) today who deny that it is possible.

The same will be true of AGI... For example, see the well-known "Chinese Room" argument against AGI, advanced by Berkeley Philosopher John Searle (discussed before in The Mechanical Turk and Searle's Chinese Room). Searle's confusion as to where, exactly, the understanding resides inside a complex computation seems silly to us today given recent developments with deep neural nets and, e.g., machine translation (the very problem used in his thought experiment). Understanding doesn't exist in any sub-portion of the network, it is embodied in the network. (See also Thought vectors and the dimensionality of the space of concepts :-)

Effective action #4a: ‘Expertise’ from fighting and physics to economics, politics and government

Extreme sports: fast feedback = real expertise

In the 1980s and early 1990s, there was an interesting case study in how useful new knowledge jumped from a tiny isolated group to the general population with big effects on performance in a community. Expertise in Brazilian jiu-jitsu was taken from Brazil to southern California by the Gracie family. There were many sceptics but they vanished rapidly because the Gracies were empiricists. They issued ‘the Gracie challenge’.

All sorts of tough guys, trained in all sorts of ways, were invited to come to their garage/academy in Los Angeles to fight one of the Gracies or their trainees. Very quickly it became obvious that the Gracie training system was revolutionary and they were real experts because they always won. There was very fast and clear feedback on predictions. Gracie jiujitsu quickly jumped from an LA garage to TV. At the televised UFC 1 event in 1993 Royce Gracie defeated everyone and a multi-billion dollar business was born.

People could see how training in this new skill could transform performance. Unarmed combat changed across the world. Disciplines other than jiu jitsu have had to make a choice: either isolate themselves and not compete with jiu jitsu or learn from it. If interested watch the first twenty minutes of this documentary (via professor Steve Hsu, physicist, amateur jiu jitsu practitioner, and predictive genomics expert).

...

[[ On politics, a field in which Dom has few peers: ]]

... The faster the feedback cycle, the more likely you are to develop a qualitative improvement in speed that destroys an opponent’s decision-making cycle. If you can reorient yourself faster to the ever-changing environment than your opponent, then you operate inside their ‘OODA loop’ (Observe-Orient-Decide-Act) and the opponent’s performance can quickly degrade and collapse.

This lesson is vital in politics. You can read it in Sun Tzu and see it with Alexander the Great. Everybody can read such lessons and most people will nod along. But it is very hard to apply because most political/government organisations are programmed by their incentives to prioritise seniority, process and prestige over high performance and this slows and degrades decisions. Most organisations don’t do it. Further, political organisations tend to make too slowly those decisions that should be fast and too quickly those decisions that should be slow — they are simultaneously both too sluggish and too impetuous, which closes off favourable branching histories of the future.

See also Kosen Judo and the origins of MMA.

Choking out a Judo black belt in the tatami room at the Payne Whitney gymnasium at Yale. My favorite gi choke is Okuri eri jime.

Training in Hawaii at Relson Gracie's and Enson Inoue's schools. The shirt says Yale Brazilian Jiujitsu -- a club I founded. I was also the faculty advisor to the already existing Judo Club :-)

New Statesman: "like it or not, the debate about whether genes affect intelligence is over"

Science writer Philip Ball, a longtime editor at Nature, writes a sensible article about the implications of rapidly improving genomic prediction for cognitive ability.

Philip Ball is a freelance science writer. He worked previously at Nature for over 20 years, first as an editor for physical sciences (for which his brief extended from biochemistry to quantum physics and materials science) and then as a Consultant Editor. His writings on science for the popular press have covered topical issues ranging from cosmology to the future of molecular biology.

Philip is the author of many popular books on science, including works on the nature of water, pattern formation in the natural world, colour in art, the science of social and political philosophy, the cognition of music, and physics in Nazi Germany.

... Philip has a BA in Chemistry from the University of Oxford and a PhD in Physics from the University of Bristol.

I recommend the whole article -- perhaps it will stimulate a badly needed discussion of this rapidly advancing area of science.

The IQ trap: how the study of genetics could transform education (New Statesman)

The study of the genes which affect intelligence could revolutionise education. But, haunted by the spectre of eugenics, the science risks being lost in a political battle.

... Researchers are now becoming confident enough to claim that the information available from sequencing a person’s genome – the instructions encoded in our DNA that influence our physical and behavioural traits – can be used to make predictions about their potential to achieve academic success. “The speed of this research has surprised me,” says the psychologist Kathryn Asbury of the University of York, “and I think that it is probable that pretty soon someone – probably a commercial company – will start to try to sell it in some way.” Asbury believes “it is vital that we have regulations in place for the use of genetic information in education and that we prepare legal, social and ethical cases for how it could and should be used.”

... Some kids pick things up in a flash, others struggle with the basics. This doesn’t mean it’s all in their genes: no one researching genes and intelligence denies that a child’s environment can play a big role in educational attainment. Of course kids with supportive, stimulating families and motivated peers have an advantage, while in some extreme cases the effects of trauma or malnutrition can compromise brain development.

... Robert Plomin of King’s College London, one of the leading experts on the genetic basis of intelligence, and his colleague Sheila Walker. They surveyed almost 2,000 primary school teachers and parents about their perceptions of genetic influence on a number of traits, including intelligence, and found that on the whole, both teachers and parents rated genetics as being just as important as the environment. This was despite the fact that 80 per cent of the teachers said there was no mention of genetics in their training. Plomin and Walker concluded that educators do seem to accept that genes influence intelligence.

Kathryn Asbury supports that view. When her PhD student Madeline Crosswaite investigated teachers’ beliefs about intelligence, Asbury says she found that “teachers, on average, believe that genetic factors are at least as important as environmental factors” and say they are “open to a role for genetic information in education one day, and that they would like to know more”.

... But now it’s possible to look directly at people’s genomes: to read the molecular code (sequence) of large proportions of an individual’s DNA. Over the past decade the cost of genome sequencing has fallen sharply, making it possible to look more directly at how genes correlate with intelligence. The data both from twin studies and DNA analysis are unambiguous: intelligence is strongly heritable. Typically around 50 per cent of variations in intelligence between individuals can be ascribed to genes, although these gene-induced differences become markedly more apparent as we age. As Ritchie says: like it or not, the debate about whether genes affect intelligence is over.

... Genome-wide polygenic scores can now be used to make such predictions about intelligence. They’re not really reliable at the moment, but will surely become better as the sample sizes for genome-wide studies increase. They will always be about probabilities, though: “Mrs Larkin, there is a 67 per cent chance that your son will be capable of reaching the top 10 per cent of GCSE grades.” Such exam results were indeed the measure Plomin and colleagues used for one recent study of genome-based prediction. They found that there was a stronger correlation between GPS and GCSE results for extreme outcomes – for particularly high or low marks.

... Using GPSs from nearly 5,000 pupils, the report assesses how exam results from different types of school – non-selective state, selective state grammar, and private – are correlated with gene-based estimates of ability for the different pupil sets. The results might offer pause for thought among parents stumping up eyewatering school fees: the distribution of exam results at age 16 could be almost wholly explained by heritable differences, with less than 1 per cent being due to the type of schooling received. In other words, as far as academic achievement is concerned, selective schools seem to add next to nothing to the inherent abilities of their pupils. ...

The Genetics of Human Behavior (The Insight podcast)

Intelligence researcher Stuart Ritchie interviewed by genomicists Razib Khan and Spencer Wells. Highly recommended! Thanks to a commenter for the link.

Allen Institute meeting on Genetics of Complex Traits

You can probably tell by all the photos below that I love their new building :-)

I was a participant in this event: What Makes Us Human? The Genetics of Complex Traits (Allen Frontiers Group), including in a small second day workshop with just the speakers and the AI leadership. This workshop will, I hope, result in some interesting new initiatives in complex trait genomics!

I'd like to thank the Allen Institute organizers for making this such a pleasant and productive 2 days. I learned some incredible things from the other speakers and I recommend all of their talks -- available here.

My talk:




Action photos:

Working hard on day 2 in the little conference room :-)

What Makes Us Human? The Genetics of Complex Traits (Allen Frontiers Group)

I'll be attending this meeting in Seattle the next few days.

Recent research has led to new insights on how genes shape brain structure and development, and their impact on individual variation. Although significant inroads have been made in understanding the genetics underlying disease risk, what about the complex traits of extraordinary variation - such as cognition, superior memory, etc.? Can current advances shed light on genetic components underpinning these variations?

Personal genomics, biobank resources, emerging statistical genetics methods and neuroimaging capabilities are opening new frontiers in the field of complex trait analysis. This symposium will highlight experts using diverse approaches to explore a spectrum of individual variation of the human mind.

Paul Allen (MSFT co-founder) is a major supporter of scientific research, including the Allen Institute for Brain Science. Excerpts from his memoir, Idea Man.

We are at a unique moment in bioscience. New ideas, combined with emerging technologies, will create unprecedented and transformational insights into living systems. Accelerating the pace of this change requires a thoughtful and agile exploration of the entire landscape of bioscience, across disciplines and spheres of research. Launched in 2016 with a $100 million commitment toward a larger 10-year plan, The Paul G. Allen Frontiers Group will discover and support scientific ideas that change the world. We are committed to a continuous conversation with the scientific community that allows us to remain at the ever-changing frontiers of science and reimagine what is possible.

My talk is scheduled for 3:55 PM Pacific Weds 1/17. All talks will be streamed on the Allen Institute Facebook page.

The Vector Institute

I've waxed enthusiastic before about Thought Vectors:

... the space of concepts (primitives) used in human language (or equivalently, in human thought) ... has only ~1000 dimensions, and has some qualities similar to an actual vector space. Indeed, one can speak of some primitives being closer or further from others, leading to a notion of distance, and one can also rescale a vector to increase or decrease the intensity of meaning.

... we now have an automated method to extract an abstract representation of human thought from samples of ordinary language. This abstract representation will allow machines to improve dramatically in their ability to process language, dealing appropriately with semantics (i.e., meaning), which is represented geometrically.

Apparently I am not the only one (MIT Technology Review):

... The Vector Institute, this monument to the ascent of ­Hinton’s ideas, is a research center where companies from around the U.S. and Canada—like Google, and Uber, and Nvidia—will sponsor efforts to commercialize AI technologies. Money has poured in faster than Jacobs could ask for it; two of his cofounders surveyed companies in the Toronto area, and the demand for AI experts ended up being 10 times what Canada produces every year. Vector is in a sense ground zero for the now-worldwide attempt to mobilize around deep learning: to cash in on the technique, to teach it, to refine and apply it. Data centers are being built, towers are being filled with startups, a whole generation of students is going into the field.

... words that have similar meanings start showing up near one another in the space. That is, “insane” and “unhinged” will have coordinates close to each other, as will “three” and “seven,” and so on. What’s more, so-called vector arithmetic makes it possible to, say, subtract the vector for “France” from the vector for “Paris,” add the vector for “Italy,” and end up in the neighborhood of “Rome.” It works without anyone telling the network explicitly that Rome is to Italy as Paris is to France.

... Neural nets can be thought of as trying to take things—images, words, recordings of someone talking, medical data—and put them into what mathematicians call a high-dimensional vector space, where the closeness or distance of the things reflects some important feature of the actual world. Hinton believes this is what the brain itself does. “If you want to know what a thought is,” he says, “I can express it for you in a string of words. I can say ‘John thought, “Whoops.”’ But if you ask, ‘What is the thought? What does it mean for John to have that thought?’ It’s not that inside his head there’s an opening quote, and a ‘Whoops,’ and a closing quote, or even a cleaned-up version of that. Inside his head there’s some big pattern of neural activity.” Big patterns of neural activity, if you’re a mathematician, can be captured in a vector space, with each neuron’s activity corresponding to a number, and each number to a coordinate of a really big vector. In Hinton’s view, that’s what thought is: a dance of vectors.

... It is no coincidence that Toronto’s flagship AI institution was named for this fact. Hinton was the one who came up with the name Vector Institute.

See also Geoff Hinton on Deep Learning (discusses thought vectors).

Feynman, Schwinger, and Psychometrics

Slate Star Codex has a new post entitled Against Individual IQ Worries.

I write a lot about the importance of IQ research, and I try to debunk pseudoscientific claims that IQ “isn’t real” or “doesn’t matter” or “just shows how well you do on a test”. IQ is one of the best-studied ideas in psychology, one of our best predictors of job performance, future income, and various other forms of success, etc.

But every so often, I get comments/emails saying something like “Help! I just took an IQ test and learned that my IQ is x! This is much lower than I thought, and so obviously I will be a failure in everything I do in life. Can you direct me to the best cliff to jump off of?”

So I want to clarify: IQ is very useful and powerful for research purposes. It’s not nearly as interesting for you personally.

I agree with Scott's point that while g is useful as a crude measurement of cognitive ability, and a statistical predictor of life outcomes, one is better off adopting the so-called growth mindset. ("Individuals who believe their talents can be developed through hard work, good strategies, and input from others have a growth mindset.")

Inevitably the question of Feynman's IQ came up in the discussion. I wrote to Scott about this (slightly edited):

Dear Scott,

I enjoyed your most recent SSC post and I agree with you that g is better applied at a statistical level (e.g., by the Army to place recruits) than at an individual level.

I notice Feynman came up again in the discussion. I have written more on this topic (and have done more research as well). My conclusions are as follows:

1. There is no doubt Feynman would have scored near the top of any math-loaded test (and he did -- e.g., the Putnam).

2. I doubt Feynman would have scored near the ceiling on many verbally loaded tests. He often made grammatical mistakes, spelling mistakes (even of words commonly used in physics), etc. He occasionally did not know the *meanings* of terms used by other people around him (even words commonly used in physics).

3. By contrast, his contemporary and rival Julian Schwinger wrote and spoke in elegant, impeccable language. People often said that Schwinger "spoke in entire paragraphs" that emerged well-formed from his mouth. My guess is that Schwinger was a more balanced type for that level of cognitive ability. Feynman was verbally creative, colorful, a master communicator, etc. But his score on the old SAT-V might not have been above top few percentile.

More people know about Feynman than Schwinger, but not just because Feynman was more colorful and charismatic. In fact, very little that Schwinger ever said or wrote was comprehensible to people below a pretty high IQ threshold, whereas Feynman expressed himself simply and intuitively. I think this has a bit to do with their verbal IQs. Even really smart physics students have an easier time understanding Feynman's articles and lectures than Schwinger's!

Schwinger had read (and understood) all of the existing literature on quantum mechanics while still a HS student -- this loads on V, not just M. Feynman's development path was different, partially because he had trouble reading other people's papers.

Schwinger was one of the subjects in Anne Roe's study of top scientists. His verbal score was above +4 SD. I think it's extremely unlikely that Feynman would have scored that high.

See links below for more discussion, examples, etc.

Hope you are enjoying Berkeley!

Best,
Steve

Feynman's Cognitive Style

Feynman and the Secret of Magic

Feynman's War

Schwinger meets Rabi

Roe's Scientists

Here are some (accessible) Schwinger quotes I like.

The pressure for conformity is enormous. I have experienced it in editors’ rejection of submitted papers, based on venomous criticism of anonymous referees. The replacement of impartial reviewing by censorship will be the death of science.


Is the purpose of theoretical physics to be no more than a cataloging of all the things that can happen when particles interact with each other and separate? Or is it to be an understanding at a deeper level in which there are things that are not directly observable (as the underlying quantized fields are) but in terms of which we shall have a more fundamental understanding?


To me, the formalism of quantum mechanics is not just mathematics; rather it is a symbolic account of the realities of atomic measurements. That being so, no independent quantum theory of measurement is required -- it is part and parcel of the formalism.

[ ... recapitulates usual von Neumann formulation: unitary evolution of wavefunction under "normal" circumstances; non-unitary collapse due to measurement ... discusses paper hypothesizing stochastic (dynamical) wavefunction collapse ... ]

In my opinion, this is a desperate attempt to solve a non-existent problem, one that flows from a false premise, namely the vN dichotomization of quantum mechanics. Surely physicists can agree that a microscopic measurement is a physical process, to be described as would any physical process, that is distinguished only by the effective irreversibility produced by amplification to the macroscopic level. ...

(See Schwinger on Quantum Foundations ;-)

Schwinger survived both Feynman and Tomonaga, with whom he shared the Nobel prize for quantum electrodynamics. He began his eulogy for Feynman: "I am the last of the triumvirate ..."

Phase Transitions and Genomic Prediction of Cognitive Ability

James Thompson (University College London) recently blogged about my prediction that with sample size of order a million genotypes|phenotypes, one could construct a good genomic predictor for cognitive ability and identify most of the associated common SNPs.

The Hsu Boundary

... The “Hsu boundary” is Steve Hsu’s estimate that a sample size of roughly 1 million people may be required to reliably identify the genetic signals of intelligence.

... the behaviour of an optimization algorithm involving a million variables can change suddenly as the amount of data available increases. We see this behavior in the case of Compressed Sensing applied to genomes, and it allows us to predict that something interesting will happen with complex traits like cognitive ability at a sample size of the order of a million individuals.

Machine learning is now providing new methods of data analysis, and this may eventually simplify the search for the genes which underpin intelligence.

There are many comments on Thompson's blog post, some of them confused. Comments from a user "Donoho-Student" are mostly correct -- he or she seems to understand the subject. (The phase transition discussed is related to the Donoho-Tanner phase transition. More from Igor Carron.)

The chain of logic leading to this prediction has been discussed here before. The excerpt below is from a 2013 post The human genome as a compressed sensor:

Compressed sensing (see also here) is a method for efficient solution of underdetermined linear systems: y = Ax + noise , using a form of penalized regression (L1 penalization, or LASSO). In the context of genomics, y is the phenotype, A is a matrix of genotypes, x a vector of effect sizes, and the noise is due to nonlinear gene-gene interactions and the effect of the environment. (Note the figure above, which I found on the web, uses different notation than the discussion here and the paper below.)

Let p be the number of variables (i.e., genetic loci = dimensionality of x), s the sparsity (number of variables or loci with nonzero effect on the phenotype = nonzero entries in x) and n the number of measurements of the phenotype (i.e., the number of individuals in the sample = dimensionality of y). Then  A  is an  n x p  dimensional matrix. Traditional statistical thinking suggests that  n > p  is required to fully reconstruct the solution  x  (i.e., reconstruct the effect sizes of each of the loci). But recent theorems in compressed sensing show that  n > C s log p  is sufficient if the matrix A has the right properties (is a good compressed sensor). These theorems guarantee that the performance of a compressed sensor is nearly optimal -- within an overall constant of what is possible if an oracle were to reveal in advance which  s  loci out of  p  have nonzero effect. In fact, one expects a phase transition in the behavior of the method as  n  crosses a critical threshold given by the inequality. In the good phase, full recovery of  x  is possible.

In the paper below, available on arxiv, we show that

1. Matrices of human SNP genotypes are good compressed sensors and are in the universality class of random matrices. The phase behavior is controlled by scaling variables such as  rho = s/n  and our simulation results predict the sample size threshold for future genomic analyses.

2. In applications with real data the phase transition can be detected from the behavior of the algorithm as the amount of data  n  is varied. A priori knowledge of  s  is not required; in fact one deduces the value of  s  this way.

3.  For heritability h2 = 0.5 and p ~ 1E06 SNPs, the value of  C log p  is ~ 30. For example, a trait which is controlled by s = 10k loci would require a sample size of n ~ 300k individuals to determine the (linear) genetic architecture.

For more posts on compressed sensing, L1-penalized optimization, etc. see here. Because s could be larger than 10k, the common SNP heritability of cognitive ability might be less than 0.5, and the phenotype measurements are noisy, and because a million is a nice round figure, I usually give that as my rough estimate of the critical sample size for good results. The estimate that s ~ 10k for cognitive ability and height originates here, but is now supported by other work: see, e.g., Estimation of genetic architecture for complex traits using GWAS data.

We have recently finished analyzing height using L1-penalization and the phase transition technique on a very large data set (many hundreds of thousands of individuals). The paper has been submitted for review, and the results support the claims made above with s ~ 10k, h2 ~ 0.5 for height.


Added: Here are comments from "Donoho-Student":

Donoho-Student says:
September 14, 2017 at 8:27 pm GMT • 100 Words

The Donoho-Tanner transition describes the noise-free (h2=1) case, which has a direct analog in the geometry of polytopes.

The n = 30s result from Hsu et al. (specifically the value of the coefficient, 30, when p is the appropriate number of SNPs on an array and h2 = 0.5) is obtained via simulation using actual genome matrices, and is original to them. (There is no simple formula that gives this number.) The D-T transition had only been established in the past for certain classes of matrices, like random matrices with specific distributions. Those results cannot be immediately applied to genomes.

The estimate that s is (order of magnitude) 10k is also a key input.

I think Hsu refers to n = 1 million instead of 30 * 10k = 300k because the effective SNP heritability of IQ might be less than h2 = 0.5 — there is noise in the phenotype measurement, etc.


Donoho-Student says:
September 15, 2017 at 11:27 am GMT • 200 Words

Lasso is a common statistical method but most people who use it are not familiar with the mathematical theorems from compressed sensing. These results give performance guarantees and describe phase transition behavior, but because they are rigorous theorems they only apply to specific classes of sensor matrices, such as simple random matrices. Genomes have correlation structure, so the theorems do not directly apply to the real world case of interest, as is often true.

What the Hsu paper shows is that the exact D-T phase transition appears in the noiseless (h2 = 1) problem using genome matrices, and a smoothed version appears in the problem with realistic h2. These are new results, as is the prediction for how much data is required to cross the boundary. I don’t think most gwas people are familiar with these results. If they did understand the results they would fund/design adequately powered studies capable of solving lots of complex phenotypes, medical conditions as well as IQ, that have significant h2.

Most ML people who use lasso, as opposed to people who prove theorems, are not aware of the D-T transition. Even most people who prove theorems have followed the Candes-Tao line of attack (restricted isometry property) and don’t think much about D-T. Although D eventually proved some things about the phase transition using high dimensional geometry, it was initially discovered via simulation using simple random matrices.