Information Processing

Pessimism of the Intellect, Optimism of the Will     Archive   Favorite posts   Twitter: @steve_hsu

Wednesday, September 30, 2015

Disruptive mutations and the genetic architecture of autism

New results on the genetic architecture of autism support Mike Wigler's Unified Theory. See earlier post De Novo Mutations and Autism. Recent increases in the incidence of autism could be mainly due to greater diagnostic awareness. However, the new result that women can be carriers of autism-linked variants without exhibiting the same kinds of symptoms as men might alter the usual analysis of the role of assortative mating. Perhaps women who are carriers are predisposed to marry nerdy (but mostly asymptomatic) males who also carry above average mutational load in autism genes?

I suspect many of the ~200 genes identified in this study will overlap with the ~80 SNPs recently found by SSGAC to be associated with cognitive ability. The principle of continuity suggests that in addition to ultra-rare variants with "devastating" effects, there are many moderately rare variants (also under negative, but weaker, selection due to smaller effect size) affecting the same pathways. These would contribute to variance in cognitive ability within the normal population. More discussion in section 3 of On the Genetic Architecture of Intelligence.
Neuroscience News: Quantitative study identifies 239 genes whose ‘vulnerability’ to devastating de novo mutation makes them priority research targets.

... devastating “ultra-rare” mutations of genes that they classify as “vulnerable” play a causal role in roughly half of all ASD cases. The vulnerable genes to which they refer harbor what they call an LGD, or likely gene-disruption. These LGD mutations can occur “spontaneously” between generations, and when that happens they are found in the affected child but not found in either parent.

Although LGDs can impair the function of key genes, and in this way have a deleterious impact on health, this is not always the case. The study, whose first author is the quantitative biologist Ivan Iossifov, a CSHL assistant professor and on faculty at the New York Genome Center, finds that “autism genes” – i.e., those that, when mutated, may contribute to an ASD diagnosis – tend to have fewer mutations than most genes in the human gene pool.

This seems paradoxical, but only on the surface. Iossifov explains that genes with devastating de novo LGD mutations, when they occur in a child and give rise to autism, usually don’t remain in the gene pool for more than one generation before they are, in evolutionary terms, purged. This is because those born with severe autism rarely reproduce.

The team’s data helps the research community prioritize which genes with LGDs are most likely to play a causal role in ASD. The team pares down a list of about 500 likely causal genes to slightly more than 200 best “candidate” autism genes.

The current study also sheds new light on the transmission to children of LGDs that are carried by parents who harbor them but whose health is nevertheless not severely affected. Such transmission events were observed and documented in the families used in the study, comprising the Simons Simplex Collection (SSC). When parents carry potentially devastating LGD mutations, these are more frequently found in the ASD-affected children than in their unaffected children, and most often come from the mother.

This result supports a theory first published in 2007 by senior author Michael Wigler, a CSHL professor, and Dr. Kenny Ye, a statistician at Albert Einstein College of Medicine. They predicted that unaffected mothers are “carriers” of devastating mutations that are preferentially transmitted to children affected with severe ASD. Females have an as yet unexplained factor that protects them from mutations which, when they occur in males, will be significantly more likely to cause ASD. It is well known that at least four times as many males as females have ASD.

Wigler’s 2007 “unified theory” of sporadic autism causation predicted precisely this effect. “Devastating de novo mutations in autism genes should be under strong negative selection pressure,” he explains. “And that is among the findings of the paper we’re publishing today. Our analysis also revealed that a surprising proportion of rare devastating mutations transmitted by parents occurs in genes expressed in the embryonic brain.” This finding tends to support theories suggesting that at least some of the gene mutations with the power to cause ASD occur in genes that are indispensable for normal brain development.
Here is the paper at PNAS:
Low load for disruptive mutations in autism genes and their biased transmission

We previously computed that genes with de novo (DN) likely gene-disruptive (LGD) mutations in children with autism spectrum disorders (ASD) have high vulnerability: disruptive mutations in many of these genes, the vulnerable autism genes, will have a high likelihood of resulting in ASD. Because individuals with ASD have lower fecundity, such mutations in autism genes would be under strong negative selection pressure. An immediate prediction is that these genes will have a lower LGD load than typical genes in the human gene pool. We confirm this hypothesis in an explicit test by measuring the load of disruptive mutations in whole-exome sequence databases from two cohorts. We use information about mutational load to show that lower and higher intelligence quotients (IQ) affected individuals can be distinguished by the mutational load in their respective gene targets, as well as to help prioritize gene targets by their likelihood of being autism genes. Moreover, we demonstrate that transmission of rare disruptions in genes with a lower LGD load occurs more often to affected offspring; we show transmission originates most often from the mother, and transmission of such variants is seen more often in offspring with lower IQ. A surprising proportion of transmission of these rare events comes from genes expressed in the embryonic brain that show sharply reduced expression shortly after birth.

Saturday, September 26, 2015

Expert Prediction: hard and soft

Jason Zweig writes about Philip Tetlock's Good Judgement Project below. See also Expert Predictions, Perils of Prediction, and this podcast talk by Tetlock.

A quick summary: good amateurs (i.e., smart people who think probabilistically and are well read) typically perform as well as or better than area experts (e.g., PhDs in Social Science, History, Government; MBAs) when it comes to predicting real world outcomes. The marginal returns (in predictive power) to special "expertise" in soft subjects are small. (Most of the returns are in the form of credentialing or signaling ;-)
WSJ: ... I think Philip Tetlock’s “Superforecasting: The Art and Science of Prediction,” co-written with the journalist Dan Gardner, is the most important book on decision making since Daniel Kahneman’s “Thinking, Fast and Slow.” (I helped write and edit the Kahneman book but receive no royalties from it.) Prof. Kahneman agrees. “It’s a manual to systematic thinking in the real world,” he told me. “This book shows that under the right conditions regular people are capable of improving their judgment enough to beat the professionals at their own game.”

The book is so powerful because Prof. Tetlock, a psychologist and professor of management at the University of Pennsylvania’s Wharton School, has a remarkable trove of data. He has just concluded the first stage of what he calls the Good Judgment Project, which pitted some 20,000 amateur forecasters against some of the most knowledgeable experts in the world.

The amateurs won — hands down. Their forecasts were more accurate more often, and the confidence they had in their forecasts — as measured by the odds they set on being right — was more accurately tuned.

The top 2%, whom Prof. Tetlock dubs “superforecasters,” have above-average — but rarely genius-level — intelligence. Many are mathematicians, scientists or software engineers; but among the others are a pharmacist, a Pilates instructor, a caseworker for the Pennsylvania state welfare department and a Canadian underwater-hockey coach.

The forecasters competed online against four other teams and against government intelligence experts to answer nearly 500 questions over the course of four years: Will the president of Tunisia go into exile in the next month? Will the gold price exceed $1,850 on Sept. 30, 2011? Will OPEC agree to cut its oil output at or before its November 2014 meeting?

It turned out that, after rigorous statistical controls, the elite amateurs were on average about 30% more accurate than the experts with access to classified information. What’s more, the full pool of amateurs also outperformed the experts. ...
In technical subjects, such as chemistry or physics or mathematics, experts vastly outperform lay people even on questions related to everyday natural phenomena (let alone specialized topics). See, e.g., examples in Thinking Physics or Physics for Future Presidents. Because these fields have access to deep and challenging questions with demonstrably correct answers, the ability to answer these questions (a combination of cognitive ability and knowledge) is an obviously real and useful construct. See earlier post The Differences are Enormous:
Luis Alvarez laid it out bluntly:
The world of mathematics and theoretical physics is hierarchical. That was my first exposure to it. There's a limit beyond which one cannot progress. The differences between the limiting abilities of those on successively higher steps of the pyramid are enormous.
... People who work in "soft" fields (even in science) don't seem to understand this stark reality. I believe it is because their fields do not have ready access to right and wrong answers to deep questions. When those are available, huge differences in cognitive power are undeniable, as is the utility of this power.
Thought experiment for physicists: imagine a professor throwing copies of Jackson's Classical Electrodynamics at a group of students with the order, "Work out the last problem in each chapter and hand in your solutions to me on Monday!" I suspect that this exercise produces a highly useful rank ordering within the group, with huge differences in number of correct solutions.

Friday, September 25, 2015

Largest repositories of genomic data

This list of the largest repositories of genetic data appeared in the 25 September 2015 issue of Science. Note that the quality and extent of phenotyping varies significantly.


This popular personal genomics company now hopes to apply its data to drug discovery (see main story, p. 1472).



This genealogy firm now has a collaboration with the Google-funded biotech Calico to look for longevity genes.


SIZE: 1 million planned GENETIC DATA: whole genomes

Founded by genome pioneer Craig Venter, this company plans to sequence 100,000 people a year to look for aging-related genes.


SIZE: 107 (100,000 planned) GENETIC DATA: whole genomes

Led by another sequencing leader, Leroy Hood, this project is taking a systems approach to genetics and health.


SIZE: 390,000 (1 million planned) GENETIC DATA: SNPs, exomes, whole genomes

This U.S. Department of Defense–funded effort is probing the genetics of kidney and heart disease and substance abuse.


SIZE: 1 million planned GENETIC DATA: to be determined

Part of President Obama's Precision Medicine Initiative, this project will use genetics to tailor health care to individuals.



Study of middle-aged British is probing links between lifestyle, genes, and common diseases.


SIZE: 5500 (75,000 normal + 25,000 tumor genes planned) GENETIC DATA: whole genomes

This U.K.-funded project focusing on cancer and rare diseases aims to integrate whole genomes into clinical care.


SIZE: 140,000 GENETIC DATA: SNPs, whole genomes

Now owned by Amgen, this pioneering Icelandic company hunted for disease-related genes in the island country.


SIZE: 200,000 (500,000 planned) GENETIC DATA: SNPs

This health maintenance organization has published on telomeres and disease risks.


SIZE: 60,000 (250,000 planned) GENETIC DATA: exomes

Geisinger, a Pennsylvania health care provider, works with Regeneron Pharmaceuticals to study DNA links to disease.



Focused on genes that affect common diseases and drug response, BioVU data have been permanently deidentified.



This study collected DNA from volunteers between 2003 and 2007 and is now looking at genetics of common diseases.



This study is probing links between genetics, lifestyle and common diseases.


SIZE: 100,000 planned GENETIC DATA: exomes

One aim is to find healthy “human knockouts”—people who lack a specific gene—in a population in which marrying relatives is common.


SIZE: 100,000 planned GENETIC DATA: exomes

One aim of this national project is to find genes underlying rare inherited conditions.


SIZE: 100,000 GENETIC DATA: SNPs, exomes

The world's largest pediatric biorepository connects DNA to the hospital's health records for studies of childhood diseases.

Wednesday, September 23, 2015

Understanding Genius: roundtable at the Helix Center, NYC

I'll be part of this roundtable discussion Saturday, Oct 3 in NYC. It's open to the general public and will be live streamed at the YouTube link above. I'm pleased to be on the panel with (among others) Dean Simonton, a UC Davis psychology professor and author of numerous books related to the theme of this meeting.
The Helix Center for Interdisciplinary Investigation
The Marianne & Nicholas Young Auditorium
247 East 82nd Street
New York, NY 10028
Understanding Genius

Schopenhauer defined genius in relation to the more conventional quality of talent. “Talent hits a target others miss. Genius hits a target no one sees.” Is originality indeed the sine qua non of genius? Is there, following Kant, a radical separation of the aesthetic genius from the brilliant scientific mind? What further distinctions might be made between different types of genius? If “The Child is father of the Man,” why don’t child prodigies always grow up to become adult geniuses?

Saturday, September 19, 2015

SNP hits on cognitive ability from 300k individuals

James Lee talk at ISIR 2015 (via James Thompson) reports on 74 hits at genome-wide statistical significance (p < 5E-8) using educational attainment as the phenotype. Most of these will also turn out to be hits on cognitive ability.

To quote James: "Shock and Awe" for those who doubt that cognitive ability is influenced by genetic variants. This is just the tip of the iceberg, though. I expect thousands more such variants to be discovered before we have accounted for all of the heritability.
James J Lee

University of Minnesota Twin Cities
Social Science Genetic Association Consortium

Genome-wide association studies (GWAS) have revealed much about the biological pathways responsible for phenotypic variation in many anthropometric traits and diseases. Such studies also have the potential to shed light on the developmental and mechanistic bases of behavioral traits.

Toward this end we have undertaken a GWAS of educational attainment (EA), an outcome that shows phenotypic and genetic correlations with cognitive performance, personality traits, and other psychological phenotypes. We performed a GWAS meta-analysis of ~293,000 individuals, applying a variety of methods to address quality control and potential confounding. We estimated the genetic correlations of several different traits with EA, in essence by determining whether single-nucleotide polymorphisms (SNPs) showing large statistical signals in a GWAS meta-analysis of one trait also tend to show such signals in a meta-analysis of another. We used a variety of bio-informatic tools to shed light on the biological mechanisms giving rise to variation in EA and the mediating traits affecting this outcome. We identified 74 independent SNPs associated with EA (p < 5E-8). The ability of the polygenic score to predict within-family differences suggests that very little of this signal is due to confounding. We found that both cognitive performance (0.82) and intracranial volume (0.39) show substantial genetic correlations with EA. Many of the biological pathways significantly enriched by our signals are active in early development, affecting the proliferation of neural progenitors, neuron migration, axonogenesis, dendrite growth, and synaptic communication. We nominate a number of individual genes of likely importance in the etiology of EA and mediating phenotypes such as cognitive performance.
For a hint at what to expect as more data become available, see Five Years of GWAS Discovery and On the genetic architecture of intelligence and other quantitative traits.

What was once science fiction will soon be reality.
Long ago I sketched out a science fiction story involving two Junior Fellows, one a bioengineer (a former physicist, building the next generation of sequencing machines) and the other a mathematician. The latter, an eccentric, was known for collecting signatures -- signed copies of papers and books authored by visiting geniuses (Nobelists, Fields Medalists, Turing Award winners) attending the Society's Monday dinners. He would present each luminary with an ornate (strangely sticky) fountain pen and a copy of the object to be signed. Little did anyone suspect the real purpose: collecting DNA samples to be turned over to his friend for sequencing! The mathematician is later found dead under strange circumstances. Perhaps he knew too much! ...

Friday, September 18, 2015

Bourdieu and the Economy of Symbolic Exchange

From Bobos in Paradise by David Brooks. This part of Bourdieu's oeuvre is, of course, required reading for all academics. By academics, I don't just mean humanists and social scientists. Even those in the hardest of sciences and technology would benefit from considering the political / symbolic economy of their field. Why, exactly, did most positions in top theoretical physics groups go to string theorists over a 20+ year period? See String Theory Quotes , String Theory and All That , Voting and Weighing.
The Economy of Symbolic Exchange

If a university were to offer a course of study on the marketplace of ideas, the writer who would be at the heart of the curriculum would be Pierre Bourdieu. Bourdieu is a French sociologist who is influential among his colleagues but almost entirely unread outside academia because of his atrocious prose style. Bourdieu’s aim is to develop an economy of symbolic exchanges, to delineate the rules and patterns of the cultural and intellectual marketplace. His basic thesis is that all intellectual and cultural players enter the marketplace with certain forms of capital. They may have academic capital (the right degrees), cultural capital (knowledge of a field or art form, a feel for the proper etiquette), linguistic capital (the ability to use language), political capital (the approved positions or affiliations), or symbolic capital (a famous fellowship or award). Intellectuals spend their careers trying to augment their capital and convert one form of capital into another. One intellectual might try to convert knowledge into a lucrative job; another might convert symbolic capital into invitations to exclusive conferences at tony locales; a third might seek to use linguistic ability to destroy the reputations of colleagues so as to become famous or at least controversial.

Ultimately, Bourdieu writes, intellectuals compete to gain a monopoly over the power to consecrate. Certain people and institutions at the top of each specialty have the power to confer prestige and honor on favored individuals, subjects, and styles of discourse. Those who hold this consecration of power influence taste, favor certain methodologies, and define the boundary of their discipline. To be chief consecrator is the intellectual’s dream.

Bourdieu doesn’t just look at the position an intellectual may hold at a given moment; he looks at the trajectory of a career, the successive attitudes, positions, and strategies a thinker adopts while rising or competing in the marketplace. A young intellectual may enter the world armed only with personal convictions. He or she will be confronted, Bourdieu says, with a diverse “field.” There will be daring radical magazines over on one side, staid establishment journals on another, dull but worthy publishing houses here, vanguard but underfunded houses over there. The intellectual will be confronted with rivalries between schools and between established figures. The complex relationships between these and other players in the field will be the tricky and shifting environment in which the intellectual will try to make his or her name. Bourdieu is quite rigorous about the interplay of these forces, drawing elaborate charts of the various fields of French intellectual life, indicating the power and prestige levels of each institution. He identifies which institutions have consecration power over which sections of the field.

Young intellectuals will have to know how to invest their capital to derive maximum “profit,” and they will have to devise strategies for ascent—whom to kiss up to and whom to criticize and climb over. Bourdieu’s books detail a dazzling array of strategies intellectuals use to get ahead. Bourdieu is not saying that the symbolic field can be understood strictly by economic principles. Often, he says, the “loser wins” rule applies. Those who most vociferously and publicly renounce material success win prestige and honor that can be converted into lucre. Nor does Bourdieu even claim that all of the strategies are self-conscious. He says that each intellectual possesses a “habitus,” or personality and disposition, that leads him or her in certain directions and toward certain fields. Moreover, the intellectual will be influenced, often unwillingly or unknowingly, by the gravitational pull of the rivalries and controversies of the field. Jobs will open up, grants will appear, furies will rage. In some ways the field dominates and the intellectuals are blown about within it.

Bourdieu hasn’t quite established himself as the Adam Smith of the symbolic economy. And it probably wouldn’t be very useful for a young intellectual to read him in hopes of picking up career tips, as a sort of Machiavellian Guide for Nobel Prize Wannabes. Rather, Bourdieu is most useful because he puts into prose some of the concepts that most other intellectuals have observed but have not systematized. Intellectual life is a mixture of careerism and altruism (like most other professions). Today the Bobo intellectual reconciles the quest for knowledge with the quest for the summer house.
See this comment, made 11 years ago!
Steve: We are living through a very bad time in particle theory. Without significant experimental guidance all we are left with is speculation and social dynamics driving the field. I hope things will get better when LHC data starts coming in - at least, most of the models currrently under consideration will be ruled out (although, of course, not string theory :-)

I will probably write a post at some point about how scientific fields which run through fallow experimental periods longer than 20 years (the length of a person's academic career) are in danger of falling into the traps which beset the humanities and social sciences. These were all discussed by Bourdieu long ago.

Wednesday, September 16, 2015

Gun Crazy

I grew up out in the country in Iowa. Our address was RR1 = "Rural Route 1" :-)  We had a creek, pond, dirtbike (motorcycle) track, and other fun stuff on our property. One of the things I enjoyed most was target shooting and plinking with my .22 -- I'd just walk out the back door and start shooting. With my scope zeroed in I could easily hit a squirrel at 50-100 yards from a standing position.

I haven't had a gun since I left for college, but now that I have kids and live in a gun friendly state, I thought I might get back into shooting a bit. There are two good ranges (one free) near my house, and my kids are at an age where they can learn gun safety and how to shoot. My wife disagrees, but to me knowing how to handle a gun is a basic life skill.

Gun technology has matured quite a bit since I was a kid. There is amazing stuff available at reasonable prices (red dot scopes!). With YouTube I got up to speed on the new gear really fast. You can't get a look at the internals of most guns while they are at the store, but online you can find complete disassembly videos.

This is a .22lr in an AR15 pattern (Smith and Wesson M&P 15-22, 5.5 lbs):

This is a 3 lb semi-auto .22lr (all polymer except the barrel and firing mechanism) available for just over $100 (Mossberg Blaze):

Ruger pistol, 25 shots at 5m--15m:

Thursday, September 10, 2015

Colleges ranked by Nobel, Fields, Turing and National Academies output

This Quartz article describes Jonathan Wai's research on the rate at which different universities produce alumni who make great contributions to science, technology, medicine, and mathematics. I think the most striking result is the range of outcomes: the top school outperforms good state flagships (R1 universities) by as much as a thousand times. In my opinion the main causative factor is simply filtering by cognitive ability and other personality traits like drive. Psychometrics works!
Quartz: Few individuals will be remembered in history for discovering a new law of nature, revolutionizing a new technology or captivating the world with their ideas. But perhaps these contributions say more about the impact of a university or college than test scores and future earnings. Which universities are most likely to produce individuals with lasting effect on our world?

The US News college rankings emphasize subjective reputation, student retention, selectivity, graduation rate, faculty and financial resources and alumni giving. Recently, other rankings have proliferated, including some based on objective long-term metrics such as individual earning potential. Yet, we know of no evaluations of colleges based on lasting contributions to society. Of course, such contributions are difficult to judge. In the analysis below, we focus primarily on STEM (science, technology, engineering and medicine/mathematics) contributions, which are arguably the least subjective to evaluate, and increasingly more valued in today’s workforce.

We examined six groups of exceptional achievers divided into two tiers, looking only at winners who attended college in the US. Our goal is to create a ranking among US colleges, but of course one could broaden the analysis if desired. The first level included all winners of the Nobel Prize (physics, chemistry, medicine, economics, literature, and peace), Fields Medal (mathematics) and the Turing Award (computer science). The second level included individuals elected to the National Academy of Sciences (NAS), National Academy of Engineering (NAE) or Institute of Medicine (IOM). The National Academies are representative of the top few thousand individuals in all of STEM.

We then traced each of these individuals back to their undergraduate days, creating two lists to examine whether the same or different schools rose to the top. We wanted to compare results across these two lists to see if findings in the first tier of achievement replicated in the second tier of achievement and to increase sample size to avoid the problem of statistical flukes.

Simply counting up the number of awards likely favors larger schools and alumni populations. We corrected for this by computing a per capita rate of production, dividing the number of winners from a given university by an estimate of the relative size of the alumni population. Specifically, we used the total number of graduates over the period 1966-2013 (an alternative method of estimating base population over 100 to 150 years led to very similar lists). This allowed us to objectively compare newer and smaller schools with older and larger schools.

In order to reduce statistical noise, we eliminated schools with only one or two winners of the Nobel, Fields or Turing prize. This resulted in only 25 schools remaining, which are shown below ...
The vast majority of schools have never produced a winner. #114 Ohio State and #115 Penn State, which have highly ranked research programs in many disciplines, have each produced one winner. Despite being top tier research universities, their per capita rate of production is over 400 times lower than that of the highest ranked school, Caltech. Of course, our ranking doesn’t capture all the ways individuals can impact the world. However, achievements in the Nobel categories, plus math and computer science, are of great importance and have helped shaped the modern world.

As a replication check with a larger sample, we move to the second category of achievement: National Academy of Science, Engineering or Medicine membership. The National Academies originated in an Act of Congress, signed by President Abraham Lincoln in 1863. Lifetime membership is conferred through a rigorous election process and is considered one of the highest honors a researcher can receive.
The results are strikingly similar across the two lists. If we had included schools with two winners in the Nobel/Fields/Turing list, Haverford, Oberlin, Rice, and Johns Hopkins would have been in the top 25 on both. For comparison, very good research universities such as #394 Arizona State, #396 Florida State and #411 University of Georgia are outperformed by the top school (Caltech) by 600 to 900 times. To give a sense of the full range: the per capita rate of production of top school to bottom school was about 449 to one for the Nobel/Fields/Turing list and 1788 to one for the National Academies list. These lists include only schools that produced at least one winner—the majority of colleges have produced zero.

What causes these drastically different odds ratios across a wide variety of leading schools? The top schools on our lists tend to be private, with significant financial resources. However, the top public university, UC Berkeley, is ranked highly on both lists: #13 on the Nobel/Fields/Turing and #31 on the National Academies. Perhaps surprisingly, many elite liberal arts colleges, even those not focused on STEM education, such as Swarthmore and Amherst, rose to the top. One could argue that the playing field here is fairly even: accomplished students at Ohio State, Penn State, Arizona State, Florida State and University of Georgia, which lag the leaders by factors of hundreds or almost a thousand, are likely to end up at the same highly ranked graduate programs as individuals who attended top schools on our list. It seems reasonable to conclude that large differences in concentration or density of highly able students are at least partly responsible for these differences in outcome.

Sports fans are unlikely to be surprised by our results. Among all college athletes only a few will win professional or world championships. Some collegiate programs undoubtedly produce champions at a rate far in excess of others. It would be uncontroversial to attribute this differential rate of production both to differences in ability of recruited athletes as well as the impact of coaching and preparation during college. Just as Harvard has a far higher percentage of students scoring 1600 on the SAT than most schools and provides advanced courses suited to those individuals, Alabama may have more freshman defensive ends who can run the forty yard dash in under 4.6 seconds, and the coaches who can prepare them for the NFL.

One intriguing result is the strong correlation (r ~ 0.5) between our ranking (over all universities) and the average SAT score of each student population, which suggests that cognitive ability, as measured by standardized tests, likely has something to do with great contributions later in life. By selecting heavily on measurable characteristics such as cognitive ability, an institution obtains a student body with a much higher likelihood of achievement. The identification of ability here is probably not primarily due to “holistic review” by admissions committees: Caltech is famously numbers-driven in its selection (it has the highest SAT/ACT scores), and outperforms the other top schools by a sizeable margin. While admission to one of the colleges on the lists above is no guarantee of important achievements later in life, the probability is much higher for these select matriculants.

We cannot say whether outstanding achievement should be attributed to the personal traits of the individual which unlocked the door to admission, the education and experiences obtained at the school, or benefits from alumni networks and reputation. These are questions worthy of continued investigation. Our findings identify schools that excel at producing impact, and our method introduces a new way of thinking about and evaluating what makes a college or university great. Perhaps college rankings should be less subjective and more focused on objective real world achievements of graduates.
For analogous results in college football, see here, here and here. Four and Five star recruits almost always end up at the powerhouse programs, and they are 100x to 1000x more likely to make it as pros than lightly recruited athletes who are nevertheless offered college scholarships.

Wednesday, September 09, 2015

Frozen in time

Although we have whole genome reconstructions, we don't really know what Neanderthals looked like, exactly. It would be amazing to discover an intact individual.
National Geographic: As global warming causes ice sheets and glaciers to retreat, it will be “very, very likely” that a well-preserved Neanderthal will one day emerge, says Hiebert, much like the 40,000-year-old baby mammoth found in Siberia.
See also The Neanderthal Problem.

Monday, September 07, 2015

Meritocracy and DNA

Read Toby Young's new article The Fall of the Meritocracy. The adoption of civil service and educational placement examinations in England and France in the 19th century was consciously based on the Chinese model (see Les Grandes Ecoles Chinoises). Although imperfect, exams work: we have no better system for filtering talent. But few are willing to acknowledge this basic fact.
Quadrant Magazine: In 1958, my father, Michael Young, published a short book called The Rise of the Meritocracy, 1870–2023: An Essay on Education and Equality. It purported to be a paper written by a sociologist in 2034 about the transformation of Britain from a feudal society in which people’s social position and level of income were largely determined by the socio-economic status of their parents into a modern Shangri-La in which status is based solely on merit. He invented the word meritocracy to describe this principle for allocating wealth and prestige and the new society it gave rise to.

The essay begins with the introduction of open examinations for entry into the civil service in the 1870s—hailed as “the beginning of the modern era”—and continues to discuss real events up until the late 1950s, at which point it veers off into fantasy, describing the emergence of a fully-fledged meritocracy in Britain in the second half of the twentieth century. In spite of being semi-fictional, the book is clearly intended to be prophetic—or, rather, a warning. Like George Orwell’s Nineteen Eighty-Four (1949), The Rise of the Meritocracy is a dystopian satire that identifies various aspects of the contemporary world and describes a future they might lead to if left unchallenged. Michael was particularly concerned about the introduction of the 11+ by Britain’s wartime coalition government in 1944, an intelligence test that was used to determine which children should go to grammar schools (the top 15 per cent) and which to secondary moderns and technical schools (the remaining 85 per cent). It wasn’t just the sorting of children into sheep and goats at the age of eleven that my father objected to. As a socialist, he disapproved of equality of opportunity on the grounds that it gave the appearance of fairness to the massive inequalities created by capitalism. He feared that the meritocratic principle would help to legitimise the pyramid-like structure of British society.

In the short term, the book achieved its political aim. It was widely read by Michael’s colleagues in the Labour Party (he ran the party’s research department from 1945 to 1951) and helped persuade his friend Anthony Crosland, who became Labour Education Secretary in 1965, that the 11+ should be phased out and the different types of school created by the 1944 Education Act should be replaced by non-selective, one-size-fits-all comprehensives. Crosland famously declared: “If it’s the last thing I do, I’m going to destroy every f***ing grammar school in England. And Wales and Northern Ireland.” Today, there are only 164 grammar schools in England and sixty-eight in Northern Ireland. There are none in Wales.

But even though my father’s book helped to win the battle over selective education, he lost the war. The term “meritocracy” has now entered the language, and while its meaning hasn’t changed—it is still used to describe the organising principle Michael identified in his book—it has come to be seen as something good rather than bad.[1] The debate about grammar schools rumbles on in Britain, but their opponents no longer argue that a society in which status is determined by merit is undesirable. Rather, they embrace this principle and claim that a universal comprehensive system will lead to higher levels of social mobility than a system that allows some schools to “cream skim” the most intelligent children at the age of eleven.[2]

We are all meritocrats now

Not only do pundits and politicians on all sides claim to be meritocrats—and this is true of most developed countries, not just Britain—they also agree that the principle remains stillborn. In Britain and America there is a continuing debate about whether the rate of inter-generational social mobility has remained stagnant or declined in the past fifty years, but few think it has increased.[3] The absence of opportunities for socio-economic advancement is now seen as one of the key political problems facing Western democracies, leading to the moral collapse of the indigenous white working class, the alienation of economically unsuccessful migrant groups, and unsustainable levels of welfare dependency. This cluster of issues is the subject of several recent books by prominent political scientists, most notably Our Kids: The American Dream in Crisis (2015) by Robert Putnam.

Unlike my father, I’m not an egalitarian. As Friedrich Hayek and others have pointed out, the difficulty with end-state equality is that it can only be achieved at too great a human cost. Left to their own devices, some men will inevitably accumulate more wealth than others, whether through ability or luck, and the only way to “correct” this is through the state’s use of coercive power. If the history of the twentieth century teaches us anything, it is that the dream of creating a socialist utopia often leads to the suppression of free speech, the imprisonment of a significant percentage of the population and, in some extreme cases, state-organised mass murder.

Having said that, I recognise that a lack of social mobility poses a threat to the sustainability of liberal democracies and, in common with many others, believe the solution lies in improving our education systems. There is a consensus among most participants in the debate about education reform that the ideal schools are those that manage to eliminate the attainment gap between the children of the rich and the poor. That is, an education system in which children’s exam results don’t vary according to the neighbourhood they’ve grown up in, the income or education of their parents, or the number of books in the family home. Interestingly, there is a reluctance on the part of many liberal educationalists to accept the corollary of this, which is that attainment in these ideal schools would correspond much more strongly with children’s natural abilities. [4] This is partly because it doesn’t sit well with their egalitarian instincts and partly because they reject the idea that intelligence has a genetic basis. But I’m less troubled by this. I want the clever, hard-working children of those in the bottom half of income distribution to move up, and the less able children of those in the top half to move down. ...
Young follows the meritocracy and social justice argument where it ultimately leads: to DNA.
... However, there’s a problem here—let’s call it the challenge posed by behavioural genetics—which is that cognitive ability and other characteristics that lead to success, such as conscientiousness, impulse control and a willingness to defer gratification, are between 40 per cent and 80 per cent heritable.[5] I know that many people will be reluctant to accept that, but the evidence from numerous studies of identical twins separated at birth, as well as non-biological siblings raised in the same household, is pretty overwhelming. And it’s probable that in the next few years genetic research scientists will produce even more evidence that important aspects of people’s personalities—including those that determine whether they succeed or fail—are linked to their genes, with the relevant variants being physically identified. The implication is that a society in which status is allocated according to merit isn’t much fairer than one in which it’s inherited—or, rather, it is partly inherited, but via parental DNA rather than tax-efficient trusts. This is an argument against meritocracy made by John Rawls in A Theory of Justice (1971): You’ve done nothing to deserve the talents you’re born with—they’re distributed according to a “natural lottery”—so you don’t deserve what flows from them.[6]

It’s worth pausing here to note that Rawls accepts that not all men are born equal, genetically speaking. Some do better out of the “natural lottery” than others and that, in turn, has an impact on their life chances. This is far from universally accepted by liberal commentators and policy-makers, most of whom prefer to think of man as a tabula rasa, forged by society rather than nature. ...

... The reason liberals are so hostile to the concept of IQ—and particularly the claim that it helps to determine socio-economic status, rather than vice versa—is because they have an almost religious attachment to the idea that man is a piece of clay that can be moulded into any shape by society. After all, it’s only if human beings are infinitely malleable and not bound by their inner nature that the various utopias they dream of can become a reality, from William Morris’s Earthly Paradise to the New Jerusalem of my father’s Labour Party. ...

... But the new technologies thrown up by genetic research will mean they no longer have to deny this obvious truth. If it becomes possible to select human embryos according to their possession of genes associated with certain character traits, such as intelligence, the Left’s utopian political projects can be resurrected. Margaret Mead was right after all: human nature is almost unbelievably malleable, you just have to start a lot further back. It is not through changing the culture that we will be able to solve the chronic social problems besetting the advanced societies of the West, but through changing people’s genes. ...

Thursday, September 03, 2015

Don’t Worry, Smart Machines Will Take Us With Them: Why human intelligence and AI will co-evolve.

I hope you enjoy my essay in the new issue of the science magazine Nautilus (theme: the year 2050), which discusses the co-evolution of humans and machines as we advance in both AI and genetic technologies. My Nautilus article from 2014: Super-Intelligent Humans Are Coming.
Nautilus: ... AI can be thought of as a search problem over an effectively infinite, high-dimensional landscape of possible programs. Nature solved this search problem by brute force, effectively performing a huge computation involving trillions of evolving agents of varying information processing capability in a complex environment (the Earth). It took billions of years to go from the first tiny DNA replicators to Homo Sapiens. What evolution accomplished required tremendous resources. While silicon-based technologies are increasingly capable of simulating a mammalian or even human brain, we have little idea of how to find the tiny subset of all possible programs running on this hardware that would exhibit intelligent behavior.

But there is hope. By 2050, there will be another rapidly evolving and advancing intelligence besides that of machines: our own. The cost to sequence a human genome has fallen below $1,000, and powerful methods have been developed to unravel the genetic architecture of complex traits such as human cognitive ability. Technologies already exist which allow genomic selection of embryos during in vitro fertilization—an embryo’s DNA can be sequenced from a single extracted cell. Recent advances such as CRISPR allow highly targeted editing of genomes, and will eventually find their uses in human reproduction.

... These two threads—smarter people and smarter machines—will inevitably intersect. Just as machines will be much smarter in 2050, we can expect that the humans who design, build, and program them will also be smarter. Naively, one would expect the rate of advance of machine intelligence to outstrip that of biological intelligence. Tinkering with a machine seems easier than modifying a living species, one generation at a time. But advances in genomics—both in our ability to relate complex traits to the underlying genetic codes, and the ability to make direct edits to genomes—will allow rapid advances in biologically-based cognition. Also, once machines reach human levels of intelligence, our ability to tinker starts to be limited by ethical considerations. Rebooting an operating system is one thing, but what about a sentient being with memories and a sense of free will?

... AI research also pushes even very bright humans to their limits. The frontier machine intelligence architecture of the moment uses deep neural nets: multilayered networks of simulated neurons inspired by their biological counterparts. Silicon brains of this kind, running on huge clusters of GPUs (graphical processor units made cheap by research and development and economies of scale in the video game industry), have recently surpassed human performance on a number of narrowly defined tasks, such as image or character recognition. We are learning how to tune deep neural nets using large samples of training data, but the resulting structures are mysterious to us. The theoretical basis for this work is still primitive, and it remains largely an empirical black art. The neural networks researcher and physicist Michael Nielsen puts it this way:
... in neural networks there are large numbers of parameters and hyper-parameters, and extremely complex interactions between them. In such extraordinarily complex systems it’s exceedingly difficult to establish reliable general statements. Understanding neural networks in their full generality is a problem that, like quantum foundations, tests the limits of the human mind.
... It may seem incredible, or even disturbing, to predict that ordinary humans will lose touch with the most consequential developments on planet Earth, developments that determine the ultimate fate of our civilization and species. Yet consider the early 20th-century development of quantum mechanics. The first physicists studying quantum mechanics in Berlin—men like Albert Einstein and Max Planck—worried that human minds might not be capable of understanding the physics of the atomic realm. Today, no more than a fraction of a percent of the population has a good understanding of quantum physics, although it underlies many of our most important technologies: Some have estimated that 10-30 percent of modern gross domestic product is based on quantum mechanics. In the same way, ordinary humans of the future will come to accept machine intelligence as everyday technological magic, like the flat screen TV or smartphone, but with no deeper understanding of how it is possible.

New gods will arise, as mysterious and familiar as the old.


I was asked recently to write something about my leadership style / management philosophy. As a startup CEO I led a team of ~35, and now my office has something like 350 FTEs. Eventually, hands on leadership becomes impossible and one needs general principles that can be broadly conveyed.
I have a “no drama” leadership style. We try to be as rational and unbiased as possible in making decisions, always working in the long term interests of the institution and to advance human knowledge. I ask that everyone on my team try to understand all sides of a difficult issue to the point that they can, if asked, effectively argue other perspectives. This exercise helps overcome cognitive biases. My unit tries to be entirely “transparent” -- we want other players at the university to understand the rationale and evidence behind our specific decisions. We want our resource allocations to be predictable, justifiable, and as free from petty politics as possible. Other units view members of my team as effective professionals who can be relied on to do the right thing.
One of the toughest aspects of my current job is the wide variety of things I have to look at -- technologies and research projects across the spectrum from biomedical to engineering to fundamental physics to social science and the humanities. Total NSF + DOE funding at MSU ranks in the top 10 (very close to top 5) among US universities.

The most important principle I advance to my senior staff is epistemic caution together with pragmatism.

See also this interview (startups) and Dale Carnegie: How to Win Friends and Influence People :-)

Monday, August 31, 2015

No genomic dark matter

Let me put it very simply: there is NO genomic "dark matter" or "missing heritability" -- it's merely a matter of sample size (statistical power) to identify the specific variants that account for the total expected heritability. The paper below (see also HaploSNPs and missing heritability) suggests that essentially all of the expected heritability can be accounted for once rare (MAF < 0.01) and common SNPs are taken into account. I suspect the small remaining gap in heritability is accounted for by nonlinear effects.

We don't yet know which specific variants are responsible for, e.g., population variation in height, but we expect that they can be found given sufficient statistical power. See Genetic architecture and predictive modeling of quantitative traits.
Genetic variance estimation with imputed variants finds negligible missing heritability for human height and body mass index

Nature Genetics (2015) doi:10.1038/ng.3390

We propose a method (GREML-LDMS) to estimate heritability for human complex traits in unrelated individuals using whole-genome sequencing data. We demonstrate using simulations based on whole-genome sequencing data that ~97% and ~68% of variation at common and rare variants, respectively, can be captured by imputation. Using the GREML-LDMS method, we estimate from 44,126 unrelated individuals that all ~17 million imputed variants explain 56% (standard error (s.e.) = 2.3%) of variance for height and 27% (s.e. = 2.5%) of variance for body mass index (BMI), and we find evidence that height- and BMI-associated variants have been under natural selection. Considering the imperfect tagging of imputation and potential overestimation of heritability from previous family-based studies, heritability is likely to be 60–70% for height and 30–40% for BMI. Therefore, the missing heritability is small for both traits. For further discovery of genes associated with complex traits, a study design with SNP arrays followed by imputation is more cost-effective than whole-genome sequencing at current prices.
From the paper (click for larger image):

... Under a model of neutral evolution, most variants segregating in the population are rare, whereas most genetic variation underlying traits is due to common variants18. The neutral evolutionary model predicts that the cumulative contribution of variants with MAF ≤θ to the total genetic variance is linearly proportional to θ, where θ is a MAF threshold. However, our observed results for height strongly deviated from this model (Fig. 4a), suggesting that height-associated variants have been under natural selection. Such deviation would be even stronger with whole-genome sequencing data because variation at rare sequence variants is less well captured by 1000 Genomes Project imputation than that at common variants (Fig. 3 and Supplementary Fig. 4). ... Equivalently, the neutral evolutionary model also predicts that variance explained is uniformly distributed as a function of MAF18, such that the variance explained by variants with MAF ≤0.1 equals that of variants with MAF >0.4. However, we observed that, although the variance explained per variant (defined as , with m being the number of variants) for rare variants was much smaller than that for common variants for both height and BMI (Supplementary Fig. 8), the variants with MAF ≤0.1 in total explained a significantly larger proportion of variance than those with MAF >0.4 (21.0% versus 8.8%, Pdifference = 9.2 × 10−7) for height (Fig. 4b and Supplementary Table 3), consistent with height-associated variants being under selection.

... Theoretical studies on variation in complex traits based on models of natural selection suggest that rare variants only explain a substantial amount of variance under strong assumptions about the relationship between effect size and selection strength19, 20, 21. We performed genome-wide association analyses for height and BMI in the combined data set (Online Methods) and found that the minor alleles of variants with lower MAF tended to have stronger and negative effects on height and stronger but positive effects on BMI (Fig. 4c). The correlation between minor allele effect and MAF was highly significant for both height (P < 1.0 × 10−6) and BMI (P = 8.0 × 10−5) and was even stronger for both traits in the data from the latest GIANT Consortium meta-analyses5, 22 (Fig. 4d); these correlations were not driven by population stratification (Supplementary Fig. 10). All these results suggest that height- and BMI-associated variants have been under selection. These results are consistent with the hypothesis that new mutations that decrease height or increase obesity tend to be deleterious to fitness and are hence kept at low frequencies in the population by purifying selection.
See also Deleterious variants affecting traits that have been under selection are rare and of small effect -- the results above support my conjecture from several years ago.

Sunday, August 30, 2015

Jiujitsu renaissance

John Danaher discusses his coaching philosophy. Danaher trained UFC champions Georges St. Pierre and Chris Weidman, among others.

Danaher student Garry Tonon on wrestling and jiujitsu. He's shown rolling with AJ Agazarm, a former All-Big10 wrestler and no-gi BJJ world champ.

These fights are from a no-time-limit submission tournament a few years ago, featuring the top brown belts in the world. Some of the matches lasted over an hour, others ended after only 5 or 10 minutes.  I like this style of competition much more than fighting for points.

Thursday, August 27, 2015

Trump on carried interest and hedge funds: "They didn't build this country."

Say what you want about Trump, he's one of the only candidates who isn't beholden to oligarch campaign contributors. Below he goes after the crazy tax break that hedge fund managers enjoy.
Bloomberg: ... “I know a lot of bad people in this country that are making a hell of a lot of money and not paying taxes,” Trump said in an interview with Time, in apparent reference to hedge fund and private equity fund managers. “The tax law is totally screwed up.”

"They're paying nothing and it's ridiculous," he added on CBS a few days later. “The hedge fund guys didn't build this country. These are guys that shift paper around and they get lucky." He went on: “They’re energetic, they’re very smart. But a lot of them, it’s like they’re paper pushers. They make a fortune, they pay no tax... The hedge funds guys are getting away with murder.”

Trump was apparently referring to carried interest. Most hedge funds and private equity funds are structured as partnerships where the fund managers serve as general partners and the investors as limited partners. Carried interest represents the fund managers’ share of the income generated by the fund, which is typically 20 percent of the fund’s profits at the end of the year. For most funds, this share of the profits, called an “incentive fee,” makes up most of the fund managers’ income, and, depending on the size and performance of the fund, it can stretch into the hundreds of millions of dollars. It’s largely what pays for 40,000 square foot mansions in Greenwich, Conn., and major league baseball teams and $100 million works of art. Under current tax rules, much of that incentive fee income is taxed at the long-term capital gains rate of 20 percent. If it was taxed as ordinary income, the top rate would be 39.6 percent. For hedge fund managers, the carried interest tax provision is something of a third rail, the one thing that unites them in furious opposition.

Monday, August 24, 2015

Man and Superman

These are some of my favorite panels from Frank Miller's graphic novel The Dark Knight Returns (1986). See also I Love Jack Kirby. Click for larger versions.

Education and Achievement Gaps

This recent talk by Harvard economist and education researcher Roland Fryer reviews studies of student incentives, charter schools, best educational practices, and their effects on achievement gaps.  Audio  Slides (the features in the image below are not clickable).

A very recent preprint on a study of parental incentives:
Parental Incentives and Early Childhood Achievement: A Field Experiment in Chicago Heights

Roland G. Fryer, Jr.
Harvard University and NBER

Steven D. Levitt
University of Chicago and NBER

John A. List
University of Chicago and NBER

August 2015

This article describes a randomized field experiment in which parents were provided financial incentives to engage in behaviors designed to increase early childhood cognitive and executive function skills through a parent academy. Parents were rewarded for attendance at early childhood sessions, completing homework assignments with their children, and for their child’s demonstration of mastery on interim assessments. This intervention had large and statistically significant positive impacts on both cognitive and non-cognitive test scores of Hispanics and Whites, but no impact on Blacks. These differential outcomes across races are not attributable to differences in observable characteristics (e.g. family size, mother’s age, mother’s education) or to the intensity of engagement with the program. Children with above median (pre-treatment) non cognitive scores accrue the most benefits from treatment.

Saturday, August 22, 2015

Now go train jiujitsu: choked out terrorist edition

Spencer Stone (left) is a blue belt at Gracie Lisboa. He choked out the terrorist gunman on the Amsterdam-Paris train yesterday.
NYTimes: ... Alek Skarlatos, a specialist in the National Guard from Oregon vacationing in Europe with a friend in the Air Force, Airman First Class Spencer Stone and another American, Anthony Sadler, looked up and saw the gunman. Mr. Skarlatos, who was returning from a deployment in Afghanistan, looked over at the powerfully built Mr. Stone, a martial arts enthusiast. “Let’s go, go!” he shouted.

... In the train carriage, Mr. Stone was the first to act, jumping up at the command of Mr. Skarlatos. He sprinted through the carriage toward the gunman, running “a good 10 meters to get to the guy,” Mr. Skarlatos said. Mr. Stone was unarmed; his target was visibly bristling with weapons.

With Mr. Skarlatos close behind, Mr. Stone grabbed the gunman’s neck, stunning him. But the gunman fought back furiously, slashing with his blade, slicing Mr. Stone in the neck and hand and nearly severing his thumb. Mr. Stone did not let go.

The gunman “pulled out a cutter, started cutting Spencer,” Mr. Norman, the British consultant, told television interviewers. “He cut Spencer behind the neck. He nearly cut his thumb off.”

Mr. Skarlatos grabbed the gunman’s Luger pistol and threw it to the side. Incongruously, the gunman yelled at the men to return it, even as Mr. Stone was choking him. A train conductor rushed up and grabbed the gunman’s left arm, Mr. Norman recalled.

... Mr. Stone, wounded and bleeding, kept the suspect in a chokehold. “Spencer Stone is a very strong guy,” Mr. Norman said. The suspect passed out.

Wednesday, August 19, 2015

Lackeys of the plutocracy?

This essay is an entertaining read, if somewhat wrong headed. See here for an earlier post that discusses Steve Pinker's response to Deresiewicz's earlier article Don’t Send Your Kid to the Ivy League.
The Neoliberal Arts (Harpers): ... Now that the customer-service mentality has conquered academia, colleges are falling all over themselves to give their students what they think they think they want. Which means that administrators are trying to retrofit an institution that was designed to teach analytic skills — and, not incidentally, to provide young people with an opportunity to reflect on the big questions — for an age that wants a very different set of abilities. That is how the president of a top liberal-arts college can end up telling me that he’s not interested in teaching students to make arguments but is interested in leadership. That is why, around the country, even as they cut departments, starve traditional fields, freeze professorial salaries, and turn their classrooms over to adjuncts, colleges and universities are establishing centers and offices and institutes, and hiring coordinators and deanlets, and launching initiatives, and creating courses and programs, for the inculcation of leadership, the promotion of service, and the fostering of creativity. Like their students, they are busy constructing a parallel college. What will happen to the old one now is anybody’s guess.

So what’s so bad about leadership, service, and creativity? What’s bad about them is that, as they’re understood on campus and beyond, they are all encased in neoliberal assumptions. Neoliberalism, which dovetails perfectly with meritocracy, has generated a caste system: “winners and losers,” “makers and takers,” “the best and the brightest,” the whole gospel of Ayn Rand and her Übermenschen. That’s what “leadership” is finally about. There are leaders, and then there is everyone else: the led, presumably — the followers, the little people. Leaders get things done; leaders take command. When colleges promise to make their students leaders, they’re telling them they’re going to be in charge. ...

We have always been, in the United States, what Lionel Trilling called a business civilization. But we have also always had a range of counterbalancing institutions, countercultural institutions, to advance a different set of values: the churches, the arts, the democratic tradition itself. When the pendulum has swung too far in one direction (and it’s always the same direction), new institutions or movements have emerged, or old ones have renewed their mission. Education in general, and higher education in particular, has always been one of those institutions. But now the market has become so powerful that it’s swallowing the very things that are supposed to keep it in check. Artists are becoming “creatives.” Journalism has become “the media.” Government is bought and paid for. The prosperity gospel has arisen as one of the most prominent movements in American Christianity. And colleges and universities are acting like businesses, and in the service of businesses.

What is to be done? Those very same WASP aristocrats — enough of them, at least, including several presidents of Harvard and Yale — when facing the failure of their own class in the form of the Great Depression, succeeded in superseding themselves and creating a new system, the meritocracy we live with now. But I’m not sure we possess the moral resources to do the same. The WASPs had been taught that leadership meant putting the collective good ahead of your own. But meritocracy means looking out for number one, and neoliberalism doesn’t believe in the collective. As Margaret Thatcher famously said about society, “There’s no such thing. There are individual men and women, and there are families.” As for elite university presidents, they are little more these days than lackeys of the plutocracy, with all the moral stature of the butler in a country house.

Neoliberalism disarms us in another sense as well. For all its rhetoric of freedom and individual initiative, the culture of the market is exceptionally good at inculcating a sense of helplessness. So much of the language around college today, and so much of the negative response to my suggestion that students ought to worry less about pursuing wealth and more about constructing a sense of purpose for themselves, presumes that young people are the passive objects of economic forces. That they have no agency, no options. That they have to do what the market tells them. A Princeton student literally made this argument to me: If the market is incentivizing me to go to Wall Street, he said, then who am I to argue?

I have also had the pleasure, over the past year, of hearing from a lot of people who are pushing back against the dictates of neoliberal education: starting high schools, starting colleges, creating alternatives to high school and college, making documentaries, launching nonprofits, parenting in different ways, conducting their lives in different ways. I welcome these efforts, but none of them address the fundamental problem, which is that we no longer believe in public solutions. We only believe in market solutions, or at least private-sector solutions: one-at-a-time solutions, individual solutions.

The worst thing about “leadership,” the notion that society should be run by highly trained elites, is that it has usurped the place of “citizenship,” the notion that society should be run by everyone together. Not coincidentally, citizenship — the creation of an informed populace for the sake of maintaining a free society, a self-governing society — was long the guiding principle of education in the United States. ...

Crossfit Games 2015

Some great highlights.

Friday, August 14, 2015

Pinker on bioethics

Progress in biomedical research is slow enough. It does not need to be slowed down even further.
Boston Globe: A POWERFUL NEW technique for editing genomes, CRISPR-Cas9, is the latest in a series of advances in biotechnology that have raised concerns about the ethics of biomedical research and inspired calls for moratoria and new regulations. Indeed, biotechnology has moral implications that are nothing short of stupendous. But they are not the ones that worry the worriers.

... A truly ethical bioethics should not bog down research in red tape, moratoria, or threats of prosecution based on nebulous but sweeping principles such as “dignity,” “sacredness,” or “social justice.” Nor should it thwart research that has likely benefits now or in the near future by sowing panic about speculative harms in the distant future. These include perverse analogies with nuclear weapons and Nazi atrocities, science-fiction dystopias like “Brave New World’’ and “Gattaca,’’ and freak-show scenarios like armies of cloned Hitlers, people selling their eyeballs on eBay, or warehouses of zombies to supply people with spare organs. Of course, individuals must be protected from identifiable harm, but we already have ample safeguards for the safety and informed consent of patients and research subjects.

Some say that it’s simple prudence to pause and consider the long-term implications of research before it rushes headlong into changing the human condition. But this is an illusion.

First, slowing down research has a massive human cost. Even a one-year delay in implementing an effective treatment could spell death, suffering, or disability for millions of people.

Second, technological prediction beyond a horizon of a few years is so futile that any policy based on it is almost certain to do more harm than good. Contrary to confident predictions during my childhood, the turn of the 21st century did not bring domed cities, jetpack commuting, robot maids, mechanical hearts, or regularly scheduled flights to the moon. This ignorance, of course, cuts both ways: few visionaries foresaw the disruptive effects of the World Wide Web, digital music, ubiquitous smartphones, social media, or fracking. ...

Tuesday, August 11, 2015

Explain it to me like I'm five years old

An MIT Technology Review reporter interviewed me yesterday about my Nautilus Magazine article Super-Intelligent Humans Are Coming. I had to do the interview by gchat because my voice is recovering from a terrible cold and too much yakking with brain scientists at the Allen Institute in Seattle.

I realized I need to find an explanation for the thesis of the article which is as simple as possible -- so that MIT graduates can understand it ;-)

Let me know what you think of the following.
1. Cognitive ability is highly heritable. At least half the variance is genetic in origin.

2. It is influenced by many (probably thousands) of common variants (see GCTA estimates of heritability due to common SNPs). We know there are many because the fewer there are the larger the (average) individual effect size of each variant would have to be. But then the SNPs would be easy to detect with small sample size.

Recent studies with large sample sizes detected ~70 SNP hits, but would have detected many more if effect sizes were consistent with, e.g., only hundreds of causal variants in total.

3. Since these are common variants the probability of having the negative variant -- with (-) effect on g score -- is not small (e.g., like 10% or more).

4. So each individual is carrying around many hundreds (if not thousands) of (-) variants.

5. As long as effects are roughly additive, we know that changing ALL or MOST of these (-) variants into (+) variants would push an individual many standard deviations (SDs) above the population mean. Such an individual would be far beyond any historical figure in cognitive ability. 
Given more details we can estimate the average number of (-) variants carried by individuals, and how many SDs are up for grabs from flipping (-) to (+). As is the case with most domesticated plants and animals, we expect that the existing variation in the population allows for many SDs of improvement (see figure below).
For references and more detailed explanation, see On the Genetic Architecture of Cognitive Ability and Other Heritable Traits.

Monday, August 10, 2015


I watched this on the flight back from Asia. It's a kid movie but it operates at more than one level. The girl robot Athena is really fun.

Saturday, August 08, 2015

Deep Learning in Nature

When I travel I often carry a stack of issues of Nature and Science to read (and then discard) on the plane.

The article below is a nice review of the current state of the art in deep neural networks. See earlier posts Neural Networks and Deep Learning 1 and 2, and Back to the Deep.
Deep learning
Yann LeCun, Yoshua Bengio, Geoffrey Hinton
Nature 521, 436–444 (28 May 2015) doi:10.1038/nature14539 
Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction. These methods have dramatically improved the state-of-the-art in speech recognition, visual object recognition, object detection and many other domains such as drug discovery and genomics. Deep learning discovers intricate structure in large data sets by using the backpropagation algorithm to indicate how a machine should change its internal parameters that are used to compute the representation in each layer from the representation in the previous layer. Deep convolutional nets have brought about breakthroughs in processing images, video, speech and audio, whereas recurrent nets have shone light on sequential data such as text and speech.
The article seems to give a somewhat, er, compressed, version of the history of the field. See these comments by Schmidhuber:
Machine learning is the science of credit assignment. The machine learning community itself profits from proper credit assignment to its members. The inventor of an important method should get credit for inventing it. She may not always be the one who popularizes it. Then the popularizer should get credit for popularizing it (but not for inventing it). Relatively young research areas such as machine learning should adopt the honor code of mature fields such as mathematics: if you have a new theorem, but use a proof technique similar to somebody else's, you must make this very clear. If you "re-invent" something that was already known, and only later become aware of this, you must at least make it clear later.

As a case in point, let me now comment on a recent article in Nature (2015) about "deep learning" in artificial neural networks (NNs), by LeCun & Bengio & Hinton (LBH for short), three CIFAR-funded collaborators who call themselves the "deep learning conspiracy" (e.g., LeCun, 2015). They heavily cite each other. Unfortunately, however, they fail to credit the pioneers of the field, which originated half a century ago. All references below are taken from the recent deep learning overview (Schmidhuber, 2015), except for a few papers listed beneath this critique focusing on nine items.

1. LBH's survey does not even mention the father of deep learning, Alexey Grigorevich Ivakhnenko, who published the first general, working learning algorithms for deep networks (e.g., Ivakhnenko and Lapa, 1965). A paper from 1971 already described a deep learning net with 8 layers (Ivakhnenko, 1971), trained by a highly cited method still popular in the new millennium. Given a training set of input vectors with corresponding target output vectors, layers of additive and multiplicative neuron-like nodes are incrementally grown and trained by regression analysis, then pruned with the help of a separate validation set, where regularisation is used to weed out superfluous nodes. The numbers of layers and nodes per layer can be learned in problem-dependent fashion.

2. LBH discuss the importance and problems of gradient descent-based learning through backpropagation (BP), and cite their own papers on BP, plus a few others, but fail to mention BP's inventors. BP's continuous form was derived in the early 1960s (Bryson, 1961; Kelley, 1960; Bryson and Ho, 1969). Dreyfus (1962) published the elegant derivation of BP based on the chain rule only. BP's modern efficient version for discrete sparse networks (including FORTRAN code) was published by Linnainmaa (1970). Dreyfus (1973) used BP to change weights of controllers in proportion to such gradients. By 1980, automatic differentiation could derive BP for any differentiable graph (Speelpenning, 1980). Werbos (1982) published the first application of BP to NNs, extending thoughts in his 1974 thesis (cited by LBH), which did not have Linnainmaa's (1970) modern, efficient form of BP. BP for NNs on computers 10,000 times faster per Dollar than those of the 1960s can yield useful internal representations, as shown by Rumelhart et al. (1986), who also did not cite BP's inventors. [ THERE ARE 9 POINTS IN THIS CRITIQUE ]

... LBH may be backed by the best PR machines of the Western world (Google hired Hinton; Facebook hired LeCun). In the long run, however, historic scientific facts (as evident from the published record) will be stronger than any PR. There is a long tradition of insights into deep learning, and the community as a whole will benefit from appreciating the historical foundations. 
One very striking aspect of the history of deep neural nets, which is acknowledged both by Schmidhuber and LeCun et al., is that the subject was marginal to "mainstream" AI and CS research for a long time, and that new technologies (i.e., GPUs) were crucial to its current flourishing in terms of practical results. The theoretical results, such as they are, appeared decades ago! It is clear that there are many unanswered questions concerning guarantees of optimal solutions, the relative merits of alternative architectures, use of memory networks, etc.

Some additional points:

1. Prevalence of saddle points over local minima in high dimensional geometries: apparently early researchers were concerned about incomplete optimization of DNNs due to local minima in parameter space. But saddle points are much more common in high dimensional spaces and local minima have turned out not to be a big problem.

2. Optimized neural networks are similar in important ways to biological (e.g., monkey) brains! When monkeys and ConvNet are shown the same pictures, the activation of high-level units in the ConvNet explains half of the variance of random sets of 160 neurons in the monkey's inferotemporal cortex.

Some comments on the relevance of all this to the quest for human-level AI from an earlier post:
.. evolution has encoded the results of a huge environment-dependent optimization in the structure of our brains (and genes), a process that AI would have to somehow replicate. A very crude estimate of the amount of computational power used by nature in this process leads to a pessimistic prognosis for AI even if one is willing to extrapolate Moore's Law well into the future. [ Moore's Law (Dennard scaling) may be toast for the next decade or so! ] Most naive analyses of AI and computational power only ask what is required to simulate a human brain, but do not ask what is required to evolve one. I would guess that our best hope is to cheat by using what nature has already given us -- emulating the human brain as much as possible.

If indeed there are good (deep) generalized learning architectures to be discovered, that will take time. Even with such a learning architecture at hand, training it will require interaction with a rich exterior world -- either the real world (via sensors and appendages capable of manipulation) or a computationally expensive virtual world. Either way, I feel confident in my bet that a strong version of the Turing test (allowing, e.g., me to communicate with the counterpart over weeks or months; to try to teach it things like physics and watch its progress; eventually for it to teach me) won't be passed until at least 2050 and probably well beyond.
Relevant remarks from Schmidhuber:
[Link] ...Ancient algorithms running on modern hardware can already achieve superhuman results in limited domains, and this trend will accelerate. But current commercial AI algorithms are still missing something fundamental. They are no self-referential general purpose learning algorithms. They improve some system’s performance in a given limited domain, but they are unable to inspect and improve their own learning algorithm. They do not learn the way they learn, and the way they learn the way they learn, and so on (limited only by the fundamental limits of computability). As I wrote in the earlier reply: "I have been dreaming about and working on this all-encompassing stuff since my 1987 diploma thesis on this topic." However, additional algorithmic breakthroughs may be necessary to make this a practical reality.
[Link] The world of RNNs is such a big world because RNNs (the deepest of all NNs) are general computers, and because efficient computing hardware in general is becoming more and more RNN-like, as dictated by physics: lots of processors connected through many short and few long wires. It does not take a genius to predict that in the near future, both supervised learning RNNs and reinforcement learning RNNs will be greatly scaled up. Current large, supervised LSTM RNNs have on the order of a billion connections; soon that will be a trillion, at the same price. (Human brains have maybe a thousand trillion, much slower, connections - to match this economically may require another decade of hardware development or so). In the supervised learning department, many tasks in natural language processing, speech recognition, automatic video analysis and combinations of all three will perhaps soon become trivial through large RNNs (the vision part augmented by CNN front-ends). The commercially less advanced but more general reinforcement learning department will see significant progress in RNN-driven adaptive robots in partially observable environments. Perhaps much of this won’t really mean breakthroughs in the scientific sense, because many of the basic methods already exist. However, much of this will SEEM like a big thing for those who focus on applications. (It also seemed like a big thing when in 2011 our team achieved the first superhuman visual classification performance in a controlled contest, although none of the basic algorithms was younger than two decades:

So what will be the real big thing? I like to believe that it will be self-referential general purpose learning algorithms that improve not only some system’s performance in a given domain, but also the way they learn, and the way they learn the way they learn, etc., limited only by the fundamental limits of computability. I have been dreaming about and working on this all-encompassing stuff since my 1987 diploma thesis on this topic, but now I can see how it is starting to become a practical reality. Previous work on this is collected here:
See also Solomonoff universal induction. I don't believe that completely general purpose learning algorithms have to become practical before we achieve human-level AI. Humans are quite limited, after all! When was the last time you introspected to learn about the way you learn you learn ...? Perhaps it is happening "under the hood" to some extent, but not in maximum generality; we have hardwired limits.
Do we really need Solomonoff? Did Nature make use of his Universal Prior in producing us? It seems like cheaper tricks can produce "intelligence" ;-)

Blog Archive