New Yorker: Practice Doesn't Make Perfect (Zach Hambrick, MSU Psychology)

MSU Psychology Professor Zach Hambrick is featured in this New Yorker article about Nature vs Nurture. How far the pendulum has swung since the naive days of Malcolm Gladwell's Outliers and its credulous embrace of Anders Ericsson's nurturist claims. David Lubinski and SMPY also make an appearance.

New Yorker: Practice Doesn't Make Perfect

... So how much did practice actually explain? In a 2014 meta-analysis that looked specifically at the relationship between deliberate practice and performance in music, games like chess, sports, education, and other professions, Hambrick and his team found a relationship that was even more complex than they had expected. For some things, like games, practice explained about a quarter of variance in expertise. For music and sports, the explanatory power accounted for about a fifth. But for education and professions like computer science, military-aircraft piloting, and sales, the effect ranged from small to tiny. For all of these professions, you obviously need to practice, but natural abilities matter more.

What’s more, the explanatory power of practice fell even further when Hambrick took exact level of expertise into account. In sports—one of the areas in which deliberate practice seems to make the most difference—it turned out that the more advanced the athlete, the less of a role practice plays. Training an average athlete for a set number of hours yields far more results than training an élite athlete, which, in turn, yields greater results than training a super-élite athlete. Put differently, someone like me is going to improve a great deal with even a few hundred hours of training. But within an Olympic team tiny differences in performance are unlikely to be the result of training: these athletes train together, with the same coach, day in and day out. Those milliseconds come from somewhere else. Some may be due to the fact that genetic differences can account for some of the response to training. ...

So where else, exactly, do performance differences come from? While Hambrick’s work has been focussed more explicitly on practice and genetics, David Lubinski, a professor of psychology at Vanderbilt University, has been approaching the question from a slightly different angle: through what’s called the Study of Mathematically Precocious Youth (smpy), a longitudinal study of the lives of students who, by the age of thirteen, had scored in the top one per cent of mathematical-reasoning ability and were then selected to take part in an enriched educational environment. (The study, co-directed for many years by Lubinski and his wife, Vanderbilt’s education-school dean, Camilla Benbow, was described in detail in a recent article in Nature.) It’s a crucial supplement to work like Hambrick’s; the data you get from close observation of the same sample and the same individuals over time can answer questions other approaches can’t. “What kinds of practice are more effective? What approaches more effective for some people than others?” Hambrick asks. “We need all the pieces to the puzzle to maximize people’s potential. Lubinski’s work on mathematically precocious youth is an essential piece.”

Technical innovation and spatial ability

A new paper from David Lubinski and collaborators looks at spatial ability measured at age 13 to see whether it adds predictive power to (SAT) Math and Verbal ability scores. The blobs in the figure above (click for larger version) represent subgroups of individuals who have published peer reviewed work in STEM, Humanities or Biomedical research, or (separately) have been awarded a patent. Units in the figure are SDs within the SMPY population.

Creativity and Technical Innovation: Spatial Ability’s Unique Role
DOI: 10.1177/0956797613478615

In the late 1970s, 563 intellectually talented 13-year-olds (identified by the SAT as in the top 0.5% of ability) were assessed on spatial ability. More than 30 years later, the present study evaluated whether spatial ability provided incremental validity (beyond the SAT’s mathematical and verbal reasoning subtests) for differentially predicting which of these individuals had patents and three classes of refereed publications. A two-step discriminant-function analysis revealed that the SAT subtests jointly accounted for 10.8% of the variance among these outcomes (p < .01); when spatial ability was added, an additional 7.6% was accounted for—a statistically significant increase (p < .01). The findings indicate that spatial ability has a unique role in the development of creativity, beyond the roles played by the abilities traditionally measured in educational selection, counseling, and industrial-organizational psychology. Spatial ability plays a key and unique role in structuring many important psychological phenomena and should be examined more broadly across the applied and basic psychological sciences.

Note that SAT composite accounted for 10 percent of variance in research success even within this already gifted subpopulation. This non-zero result, despite the restriction of range, contradicts the Gladwellian claim that IQ above 120 does not provide additional returns. In fact, the higher the IQ score above the 99.5 percentile cutoff for this group, the greater the likelihood that an individual has been awarded a patent or has published a research paper.

If you're so smart, why aren't you rich?

Does being smart help you become rich?

Folksy Warren Buffett once said that an investor with IQ of 150 should sell 30 points to someone else, as anything above 120 is unnecessary.

Consider the following simple model (we can call it the "Igon Model" in honor of Malcolm Gladwell):

Igon Model: IQ correlates positively with wealth, but the effect goes away for IQ > 120. IQ above 120 provides no advantage, relative to IQ=120, for acquiring wealth.

Were this model to be true, one would expect with overwhelming probability to find that the vast majority of rich people have IQ around 120, but not much higher. This is because IQ is normally distributed: as you go further out the tail the population decreases exponentially. To be specific, IQ = 120 corresponds to the 90th percentile, whereas IQ = 135 is 99th percentile (i.e., only 1 in 10 people with IQ > 120 have IQ > 135) and IQ = 145 is 99.9th percentile (i.e., only 1 in 100 people with IQ > 120 have IQ > 145).

Now let's look at the 2009 Forbes list of richest people in the world:

1 William Gates III 53 40.0 United States
2 Warren Buffett 78 37.0 United States
3 Carlos Slim Helu 69 35.0 Mexico

If the Igon Model were correct, we would not expect to find this list dominated by people with IQ much higher than 120. But in fact we do. Note these three made their money in different ways: Gates founded a software company, Buffett is primarily an investor, and Carlos Slim is an oligarch ;-)

Bill Gates scored 1580 on the pre-1995 SAT. His IQ is clearly >> 145 and possibly as high as 160 or so.

Warren Buffett graduated high school at 16 ranked in the top 5 percent of his class despite devoting substantial effort to entrepreneurial activities. Most people who know him well refer to him as brilliant, that folksy quote above notwithstanding. I would suggest the evidence is strong that his IQ is above 135, perhaps higher than 145.

Carlos Slim studied engineering and taught linear programming while still an undergraduate at UNAM, the top university in Mexico. He reportedly discovered the use of compound interest at age 10. I would suggest his IQ is also at least 135.

So it would appear that the three richest men in the world all have IQs that are higher than 90 percent or even 99 percent of the > 120 IQ population. (Relative to the general population they are all likely in the 99th or even 99.9th percentile.) The probability of this happening in the Igon Model is less than 1 in 1000.

[Here's a basketball analogy: the analogous Igon Model for basketball would say height over 6ft2 (90th percentile) doesn't increase likelihood of success in basketball. Suppose we find the 3 top players in the world are 7ft (Shaq/Gates), 6ft8 (LeBron/Buffett) and 6ft6 (Kobe/Slim). That strongly disfavors the model, as a random draw of 3 people from the set of people over 6ft2 in height has almost zero probability of producing the 3 heights we found.]

Note to angry Gladwell egalitarians: don't take this analysis too seriously :-) It's really an example of "Igon analysis" in the spirit of MG!

There are many factors aside from intelligence that impact success in business or investing. See here for a discussion by money manager and investment theorist William Bernstein, which is very similar to what Buffett has said on various occasions. If you carefully study biographies of the three men listed above, what really stands out (aside from high mental ability) is their determination, drive and fascination with material success beginning at a young age. See also: success vs ability and creators and rulers.

What about the broader population? It's well established that graduates of elite universities earn more than graduates of less selective schools. But, interestingly, controlling for SAT score (IQ) largely eliminates the differential. I wonder why? (See also here for UT Austin data on earnings variation with SAT and major.)

Follow that igon value!

Perhaps fittingly, the first use of "igon value" was in a profile of (then obscure) hedge fund philosopher Nassim Taleb. (See earlier post Pinker on Gladwell.)

New Yorker, April 22 & 29, 2002: [this version retrieved from gladwell.com] ... As the day came to an end, Taleb and his team turned their attention once again to the problem of the square root of n. Taleb was back at the whiteboard. Spitznagel was looking on. Pallop was idly peeling a banana. Outside, the sun was beginning to settle behind the trees. "You do a conversion to p1 and p2," Taleb said. His marker was once again squeaking across the whiteboard. "We say we have a Gaussian distribution, and you have the market switching from a low-volume regime to a high-volume. P21. P22. You have your igon value." He frowned and stared at his handiwork. The markets were now closed. Empirica had lost money, which meant that somewhere off in the woods of Connecticut Niederhoffer had no doubt made money. That hurt, but if you steeled yourself, and thought about the problem at hand, and kept in mind that someday the market would do something utterly unexpected because in the world we live in something utterly unexpected always happens, then the hurt was not so bad. Taleb eyed his equations on the whiteboard, and arched an eyebrow. It was a very difficult problem. "Where is Dr. Wu? Should we call in Dr. Wu?"

I doubt the New Yorker and its famous fact checkers caught the error. Possibly not a single New Yorker employee knows any linear algebra. Who needs all that geeky math stuff? [Update: Apparently the New Yorker did correct the electronic version now available on its site, although one can find references to the error online in 2003. See here and comments below for more.]

Leave it for the Asians like Dr. Wu... :-)

... a man whom Taleb refers to, somewhat mysteriously, as Dr. Wu wandered in. Dr. Wu works for another hedge fund, down the hall, and is said to be brilliant. He is thin and squints through black-rimmed glasses. He was asked his opinion on the square root of n but declined to answer. "Dr. Wu comes here for intellectual kicks and to borrow books and to talk music with Mark," Taleb explained after their visitor had drifted away. He added darkly, "Dr. Wu is a Mahlerian."

Pinker on Gladwell

Ouch!

Thanks to a reader for pointing out this Steve Pinker review of Malcolm Gladwell's latest collection in the Sunday Times. I had more or less stopped reading stuff on Gladwell, as the uncritical acceptance of many of his claims is just too depressing a reminder of the mediocrity of our commentariat.

An eclectic essayist is necessarily a dilettante, which is not in itself a bad thing. But Gladwell frequently holds forth about statistics and psychology, and his lack of technical grounding in these subjects can be jarring. He provides misleading definitions of “homology,” “saggital plane” and “power law” and quotes an expert speaking about an “igon value” (that’s eigenvalue, a basic concept in linear algebra). In the spirit of Gladwell, who likes to give portentous names to his aperçus, I will call this the Igon Value Problem: when a writer’s education on a topic consists in interviewing an expert, he is apt to offer generalizations that are banal, obtuse or flat wrong.

...

The common thread in Gladwell’s writing is a kind of populism, which seeks to undermine the ideals of talent, intelligence and analytical prowess in favor of luck, opportunity, experience and intuition. For an apolitical writer like Gladwell, this has the advantage of appealing both to the Horatio Alger right and to the egalitarian left. Unfortunately he wildly overstates his empirical case. It is simply not true that a quarter­back’s rank in the draft is uncorrelated with his success in the pros, that cognitive skills don’t predict a teacher’s effectiveness, that intelligence scores are poorly related to job performance or (the major claim in “Outliers”) that above a minimum I.Q. of 120, higher intelligence does not bring greater intellectual achievements.

My opinion of Malcolm Gladwell was expressed here:

Malcolm Gladwell shows exquisite taste in the subjects he writes and talks about -- he has a nose for great topics. I just wish his logical and analytical capabilities were better ... My feeling is that Gladwell's work appeals most to people who can't quite understand what he is talking about.

What Pinker refers to as the major claim of Outliers: IQ above 120 doesn't matter, is easily shown to be false. Randomly selected eminent scientists have IQs much higher than 120 and also much higher than the average science PhD (120-130); math ability within the top percentile measured in childhood is predictive of future success in science and engineering; advanced education and a challenging career do not enhance adult IQs relative to childhood IQ.

So, accomplished scientists tend to have high IQs, and their IQs were already high before they became scientists -- the causality is clear. 10,000 hours of practice may be necessary but is certainly not sufficient to become a world class expert.

I recently remarked to a friend that many aspects of psychometrics which were well established by the 1950s now seem to have been completely forgotten due to political correctness. This leads to the jarring observation that recent social science articles (the kind that Gladwell is likely to cover) are sometimes completely wrong headed (even, contradicted by existing data of which the authors are unaware) whereas many 50 year old articles are clearly reasoned and correct. The data I cite in the links above comes from the Roe study of eminent scientists and the Terman longitudinal study of gifted individuals, both of which were conducted long ago, and the SMPY longitudinal study of mathematically precocious youth, which is ongoing. I've interacted with many social scientists whose worldview is inconsistent with the established results of these studies, of which they are unaware.

Gladwell on Outliers on Charlie Rose

The interview was better than I had expected, but then my expectations were not high. At least Gladwell stops short of completely embracing the "there is no talent, it's all effort" line.

However, at about 13 minutes in we get a huge dose of politically correct pseudoscience and poor logic: Asians are good at math because -- get this -- rice farming was labor intensive. Tolerance for hard work was transmitted culturally and had no impact on genes -- "we know this" says Gladwell :-) I don't suppose Gladwell has looked at any transnational adoption studies, which remove the cultural component...

At 23 and a half minutes we get the "IQ above 120 doesn't matter" claim -- see here for some pretty strong evidence against that.

See here for earlier comments on Outliers, and here for a discussion of success and talent (the meaning of correlation).

Teaching effectiveness

The two figures below (click for larger versions) are taken from the Brookings report by Gordon, Kane and Staiger: Identifying Effective Teachers Using Performance on the Job. The report has received a lot of attention recently thanks to Malcolm Gladwell's New Yorker article. Both are worth a look if you are interested in education. The top figure shows that certification has no impact on teaching effectiveness. The second shows that effectiveness measured in the years 1 and 2 is predictive of effectiveness in the subsequent year. In this case effectiveness is defined by the average change in percentile ranking of students in the teacher's class. Good teachers help their students to improve their mastery, hence percentile ranking, relative to the average student studying the same material.






It's obvious to me that there is gigantic variation in effectiveness among teachers. Gladwell emphasizes how difficult it is to evaluate teaching capability in initial hiring, and how the single most important impact on overall school effectiveness is due to individual teachers (he also makes the analogy to scouting college QBs for pro football -- it's very hard to predict NFL performance based on college performance). The Brookings paper has many policy suggestions, but the basic idea is that if we were disciplined and data-driven we could easily determine which teachers are good and which ones are not.

New Yorker: ...One of the most important tools in contemporary educational research is “value added” analysis. It uses standardized test scores to look at how much the academic performance of students in a given teacher’s classroom changes between the beginning and the end of the school year. Suppose that Mrs. Brown and Mr. Smith both teach a classroom of third graders who score at the fiftieth percentile on math and reading tests on the first day of school, in September. When the students are retested, in June, Mrs. Brown’s class scores at the seventieth percentile, while Mr. Smith’s students have fallen to the fortieth percentile. That change in the students’ rankings, value-added theory says, is a meaningful indicator of how much more effective Mrs. Brown is as a teacher than Mr. Smith.

It’s only a crude measure, of course. A teacher is not solely responsible for how much is learned in a classroom, and not everything of value that a teacher imparts to his or her students can be captured on a standardized test.

Nonetheless, if you follow Brown and Smith for three or four years, their effect on their students’ test scores starts to become predictable: with enough data, it is possible to identify who the very good teachers are and who the very poor teachers are. What’s more—and this is the finding that has galvanized the educational world—the difference between good teachers and poor teachers turns out to be vast.

Eric Hanushek, an economist at Stanford, estimates that the students of a very bad teacher will learn, on average, half a year’s worth of material in one school year. The students in the class of a very good teacher will learn a year and a half’s worth of material. That difference amounts to a year’s worth of learning in a single year. Teacher effects dwarf school effects: your child is actually better off in a “bad” school with an excellent teacher than in an excellent school with a bad teacher. Teacher effects are also much stronger than class-size effects. You’d have to cut the average class almost in half to get the same boost that you’d get if you switched from an average teacher to a teacher in the eighty-fifth percentile. And remember that a good teacher costs as much as an average one, whereas halving class size would require that you build twice as many classrooms and hire twice as many teachers.

Hanushek recently did a back-of-the-envelope calculation about what even a rudimentary focus on teacher quality could mean for the United States. If you rank the countries of the world in terms of the academic performance of their schoolchildren, the U.S. is just below average, half a standard deviation below a clump of relatively high-performing countries like Canada and Belgium. According to Hanushek, the U.S. could close that gap simply by replacing the bottom six per cent to ten per cent of public-school teachers with teachers of average quality. After years of worrying about issues like school funding levels, class size, and curriculum design, many reformers have come to the conclusion that nothing matters more than finding people with the potential to be great teachers. But there’s a hitch: no one knows what a person with the potential to be a great teacher looks like. The school system has a quarterback problem.

In my experience as a university professor I find that most colleagues think of themselves as above-average teachers, even when they are not. Essentially no "value-added" analysis is ever done, so people can have a 30 year teaching career without ever realizing that they aren't effective in the classroom. I've done many dozens of business presentations, to venture capitalists, technology partners, customers, analysts and even potential M&A acquirers, which has helped me improve my own teaching and communication skills. Despite the business setting such meetings are 90 percent teaching -- trying to convey key points to the audience in a limited time. I'm usually there with a team and my team isn't shy about telling me afterwards what worked and what didn't work, so I've had a lot of honest feedback that most professors never get.



The New Yorker cartoon and article capture some essential aspects of teaching and communication that are not widely understood. The teacher has to be simultaneously on top of the material itself and aware of what the class is doing / thinking / confused about. The big neglected factors in teaching are the ability to be a kind of air traffic controller (or symphony conductor) for the class, and the ability to empathize with (read the mind of) an individual student, to see what, exactly, is confusing them.

Kakutani on Gladwell

Michiku Kakutani of the Times reviews Malcolm Gladwell's new book Outliers. She finds it poorly reasoned -- my usual complaint about Gladwell's work.

Much of what Mr. Gladwell has to say about superstars is little more than common sense: that talent alone is not enough to ensure success, that opportunity, hard work, timing and luck play important roles as well. The problem is that he then tries to extrapolate these observations into broader hypotheses about success. These hypotheses not only rely heavily on suggestion and innuendo, but they also pivot deceptively around various anecdotes and studies that are selective in the extreme: the reader has no idea how representative such examples are, or how reliable — or dated — any particular study might be.

Gladwell highlights the claim of psychologist Anders Ericsson, that effort dominates ability (the 10,000 hours of practice thesis). My opinion on this can be found here, deep in the comments. The evidence is pretty strong in the case of science that native cognitive ability is a prerequisite for success. Practice (effort) is necessary also, but neither alone are sufficient.

...that quote sounds like it could be from Anders Ericsson's research on expertise. I disagree with his conclusions. His studies only show that effortful practice (about 10 years worth) is typically required to reach the highest level of capability. But he then confuses the logic and asserts that practice alone is *sufficient*, when in fact it is only necessary. You need raw ability *and* lengthy practice to reach expertise.

Of course it is appealing for most people to think that Ericsson's model is correct and that effort is all that is required to produce capability, but this claim is very controversial in the psychology community, and I think implausible to anyone who has been around gifted kids/adults.

The Roe study, combined with other studies showing the age stability of IQ (certainly once adulthood is reached), also serves to refute Ericsson. There's clearly some measurable quality, usually present already at an early age, that is advantageous for intellectual achievement. Most people don't have it.

Anders is refuted quite well in papers by leading psychologists like Sternberg (Yale) and in Eysenck's book Genius.

By the way, also contra Ericsson, there are many credible examples of supreme raw talent that didn't require development through 10 years of practice (e.g., Mozart).

Annals of psychometry: IQs of eminent scientists

[ See this 2016 post for the original papers / book and the identities of the 64 scientists. ]

I recently came across a 1950s study of eminent scientists by Harvard psychologist Anne Roe: The Making of a Scientist (1952). Her study is by far the most systematic and sophisticated that I am aware of. She selected 64 eminent scientists -- well known, but not quite at the Nobel level -- in a more or less random fashion, using, e.g., membership lists of scholarly organizations and expert evaluators in the particular subfields. Roughly speaking, there were three groups: physicists (divided into experimental and theoretical subgroups), biologists (including biochemists and geneticists) and social scientists (psychologists, anthropologists).

The Making of a Scientist devotes only one chapter to psychometrics. The other chapters describe the motivation for the study, how the 64 scientists were selected, interviews with the scientists, details of their family history, work life, etc.

Roe devised her own high-end intelligence tests as follows: she obtained difficult problems in verbal, spatial and mathematical reasoning from the Educational Testing Service, which administers the SAT, but also performs bespoke testing research for, e.g., the US military. Using these problems, she created three tests (V, S and M), which were administered to the 64 scientists, and also to a cohort of PhD students at Columbia Teacher's College. The PhD students also took standard IQ tests and the results were used to norm the high-end VSM tests using an SD = 15. Most IQ tests are not good indicators of true high level ability (e.g., beyond +3 SD or so).

Average ages of subjects: mid-40s for physicists, somewhat older for other scientists

Overall normed scores:

Test (Low / Median / High)

V 121 / 166 / 177

S 123 / 137 / 164

M 128 / 154 / 194

Roe comments: (1) V test was too easy for some takers, so top score no ceiling. (2) S scores tend to decrease with age (correlation .4). Peak (younger) performance would have been higher. (3) M test was found to be too easy for the physicists; only administered to other groups.

It is unlikely that any single individual obtained all of the low scores, so each of the 64 would have been strongly superior in at least one or more areas.

Median scores (raw) by group:

group (V / S / M)

Biologists 56.6 / 9.4 / 16.8
Exp. Physics 46.6 / 11.7 / *
Theo. Physics 64.2 / 13.8 / *
Psychologists 57.7 / 11.3 / 15.6
Anthropologists 61.1 / 8.2 / 9.2

The lowest score in each category among the 12 theoretical physicists would have been roughly V 160 (!) S 130 M >> 150. (Ranges for all groups are given, but I'm too lazy to reproduce them here.) It is hard to estimate the M scores of the physicists since when Roe tried the test on a few of them they more or less solved every problem modulo some careless mistakes. Note the top raw scores (27 out of 30 problems solved) among the non-physicists (obtained by 2 geneticists and a psychologist), are quite high but short of a full score. The corresponding normed score is 194!

The lowest V scores in the 120-range were only obtained by 2 experimental physicists, all other scientists scored well above this level -- note the median is 166.

My comments:

The results strongly suggest that high IQ provides a significant advantage in science. It is sometimes claimed that IQ is irrelevant beyond a threshold: more precisely, that the advantage conferred by IQ above some threshold (e.g., 120) decreases significantly as other factors like drive or creativity take precedence. But, if that were the case it would be unlikely to have found such high scores in this group. The average IQ of a science PhD is roughly 130, and individuals with IQs in the higher range described above constitute a tiny fraction of all scientists. If IQ were irrelevant above 130 we would expect the most eminent group to have an average similar to the overall population of scientists.

Conversely, I think one should be impressed that a simple test which can be administered in a short period of time (e.g., 30 minutes for Roe's high-end exams) offers significant predictive power. While it is not true that anyone with a high IQ can or will become a great scientist (certainly other factors like drive, luck, creativity play a role), one can nevertheless easily identify the 95 percent (even 99 percent) of the population for whom success in science is highly improbable. Psychometrics works!

The scores for theoretical physicists confirm an estimate made to me by a famous colleague many years ago, that only 1 in 100,000 people could do high level theoretical physics.

Feynman's 124: in this context one often hears of Feynman's modest grade school IQ score of 124. To understand this score we have to remember that typical IQ tests (e.g., administered to public school children) tend to have low ceilings. They are not of the kind that Roe used in her study. One can imagine that the ceiling on Feynman's exam was roughly 135 (say, 99th percentile). If Feynman received the highest score on the mathematical portion, and a modest score of 115 on the verbal, we can easily understand the resulting average of 124. However, it is well known that Feynman was extremely strong mathematically. He was asked on short notice to take the Putnam exam for MIT as a senior, and received the top score in the country that year! On Roe's test Feynman's math score would presumably have been > 190, with a correspondingly higher composite IQ.

I thought I should put this post up now, as the new book by Malcolm Gladwell, Outliers: Why Some People Succeed and Some Don’t is out soon and will surely handicap the discourse on this subject for years to come :-)

Gladwell amongst the patent trolls

Malcolm Gladwell writes about Nathan Myhrvold's company Intellectual Ventures in the recent New Yorker. (Myhrvold is the former cosmologist who left physics and eventually became consigliere to Bill Gates, founding Microsoft Research and charting Microsoft's blue sky research direction. He famously missed the importance of the Internet until the mid 90's.) If you read this blog often, you know my opinion about Gladwell: he has a good nose for interesting topics, but not enough brainpower or common sense for reliable analysis. The same is true here: he produces an interesting profile of Myhrvold (although see here for a much better one from 1997 by Ken Auletta) and friends, but seems to entirely miss a number of important points. Intellectual Ventures is not about real inventions, but about patenting around ideas so that they have a future claim on the ones that turn out the be useful. In other words, they are patent trolls. Gladwell does not seem to realize the difference between rampant speculation and true invention: the hours of painstaking work in the lab required to convert an idea into reality.

Here's an excerpt about how the "invention" process works -- get some smart guys in a room and let them talk (every theory group lounge is a fount of commercializable ideas ;-). Yes! if your inventors are smart enough, they can produce 36 new inventions at dinner! Is this a statement about real innovation, or about what a patent attorney might manage to get the understaffed, overburdened USPTO to approve? It makes a mockery of what real inventors and innovators do. Why start a company and hire engineers to build a prototype? Just get a few lawyers and patent everything in sight...

How useful is it to have a group of really smart people brainstorm for a day? When Myhrvold started out, his expectations were modest. Although he wanted insights like Alexander Graham Bell’s, Bell was clearly one in a million, a genius who went on to have ideas in an extraordinary number of areas—sound recording, flight, lasers, tetrahedral construction, and hydrofoil boats, to name a few. ...

But then, in August of 2003, I.V. held its first invention session, and it was a revelation. “Afterward, Nathan kept saying, ‘There are so many inventions,’ ” Wood recalled. “He thought if we came up with a half-dozen good ideas it would be great, and we came up with somewhere between fifty and a hundred. I said to him, ‘But you had eight people in that room who are seasoned inventors. Weren’t you expecting a multiplier effect?’ And he said, ‘Yeah, but it was more than multiplicity.’ Not even Nathan had any idea of what it was going to be like.”

The original expectation was that I.V. would file a hundred patents a year. Currently, it’s filing five hundred a year. It has a backlog of three thousand ideas. Wood said that he once attended a two-day invention session presided over by Jung, and after the first day the group went out to dinner. “So Edward took his people out, plus me,” Wood said. “And the eight of us sat down at a table and the attorney said, ‘Do you mind if I record the evening?’ And we all said no, of course not. We sat there. It was a long dinner. I thought we were lightly chewing the rag. But the next day the attorney comes up with eight single-spaced pages flagging thirty-six different inventions from dinner. Dinner.”

For the cognoscenti out there, yes, the Wood mentioned in the article is none other than Star Warrior Lowell Wood, former head of the zany (useless?) O Group (NYTimes 1984) at Livermore. Wood is perfect for Myhrvold's purposes -- for decades his group bamboozled the US defense establishment with wild ideas that cost taxpayers billions of dollars. Follow the link to the Times article and tell me how many of the ideas mentioned turned into something useful, now almost a quarter century later.

Rather than leave you with a completely negative impression of the article, I include the following excerpt, which has Wood noticing something about cancer cells in the bloodstream that seems to have eluded biologists and medical researchers for some time. It is true that there are great ideas out there just waiting to be discovered, but lots of people can have the same idea. The hard part is making the idea into a practical, commercially viable reality.

...Last March, Myhrvold decided to do an invention session with Eric Leuthardt and several other physicians in St. Louis. Rod Hyde came, along with a scientist from M.I.T. named Ed Boyden. Wood was there as well.

“Lowell came in looking like the Cheshire Cat,” Myhrvold recalled. “He said, ‘I have a question for everyone. You have a tumor, and the tumor becomes metastatic, and it sheds metastatic cancer cells. How long do those circulate in the bloodstream before they land?’ And we all said, ‘We don’t know. Ten times?’ ‘No,’ he said. ‘As many as a million times.’ Isn’t that amazing? If you had no time, you’d be screwed. But it turns out that these cells are in your blood for as long as a year before they land somewhere. What that says is that you’ve got a chance to intercept them.”

How did Wood come to this conclusion? He had run across a stray fact in a recent issue of The New England Journal of Medicine. “It was an article that talked about, at one point, the number of cancer cells per millilitre of blood,” he said. “And I looked at that figure and said, ‘Something’s wrong here. That can’t possibly be true.’ The number was incredibly high. Too high. It has to be one cell in a hundred litres, not what they were saying—one cell in a millilitre. Yet they spoke of it so confidently. I clicked through to the references. It was a commonplace. There really were that many cancer cells.”

Wood did some arithmetic. He knew that human beings have only about five litres of blood. He knew that the heart pumps close to a hundred millilitres of blood per beat, which means that all of our blood circulates through our bloodstream in a matter of minutes. The New England Journal article was about metastatic breast cancer, and it seemed to Wood that when women die of metastatic breast cancer they don’t die with thousands of tumors. The vast majority of circulating cancer cells don’t do anything.

“It turns out that some small per cent of tumor cells are actually the deadly ones,” he went on. “Tumor stem cells are what really initiate metastases. And isn’t it astonishing that they have to turn over at least ten thousand times before they can find a happy home? You naïvely think it’s once or twice or three times. Maybe five times at most. It isn’t. In other words, metastatic cancer—the brand of cancer that kills us—is an amazingly hard thing to initiate. Which strongly suggests that if you tip things just a little bit you essentially turn off the process.”

That was the idea that Wood presented to the room in St. Louis. From there, the discussion raced ahead. Myhrvold and his inventors had already done a lot of thinking about using tiny optical filters capable of identifying and zapping microscopic particles. They also knew that finding cancer cells in blood is not hard. They’re often the wrong size or the wrong shape. So what if you slid a tiny filter into a blood vessel of a cancer patient? “You don’t have to intercept very much of the blood for it to work,” Wood went on. “Maybe one ten-thousandth of it. The filter could be put in a little tiny vein in the back of the hand, because that’s all you need. Or maybe I intercept all of the blood, but then it doesn’t have to be a particularly efficient filter.”

Wood was a physicist, not a doctor, but that wasn’t necessarily a liability, at this stage. “People in biology and medicine don’t do arithmetic,” he said. He wasn’t being critical of biologists and physicians: this was, after all, a man who read medical journals for fun. He meant that the traditions of medicine encouraged qualitative observation and interpretation. But what physicists do—out of sheer force of habit and training—is measure things and compare measurements, and do the math to put measurements in context. At that moment, while reading The New England Journal, Wood had the advantages of someone looking at a familiar fact with a fresh perspective.

That was also why Myhrvold had wanted to take his crew to St. Louis to meet with the surgeons. He likes to say that the only time a physicist and a brain surgeon meet is when the physicist is about to be cut open—and to his mind that made no sense. Surgeons had all kinds of problems that they didn’t realize had solutions, and physicists had all kinds of solutions to things that they didn’t realize were problems. At one point, Myhrvold asked the surgeons what, in a perfect world, would make their lives easier, and they said that they wanted an X-ray that went only skin deep. They wanted to know, before they made their first incision, what was just below the surface. When the Intellectual Ventures crew heard that, their response was amazement. “That’s your dream? A subcutaneous X-ray? We can do that.”

Let me close with my usual observation (specifically aimed at venture capitalists, research lab directors and university administrators) concerning an asymmetry in cognitive depth: yes, physicists can casually read the New England Journal of Medicine and come up with interesting insights, but, no, biologists and medical doctors cannot read Physical Review.

Gladwell and genius

Malcolm Gladwell shows exquisite taste in the subjects he writes and talks about -- he has a nose for great topics. I just wish his logical and analytical capabilities were better (see also here). This talk at the New Yorker's recent Genius 2012 conference is entertaining, but I disagree completely with his conclusion. Ribet, Wiles, Taniyama and Shimura are probably the real geniuses, not Michael Ventris, the guy who decoded Linear B. (Gladwell also can't seem to remember that it's the Taniyama-Shimura conjecture, not Tanimara. He says it incorrectly about 10 times.) My feeling is that Gladwell's work appeals most to people who can't quite understand what he is talking about.

Gladwell is confused about the exact topic discussed in James Gleick's book Genius. In a field where sampling of talents is sparse (e.g., decoding ancient codexes) you might find one giant (even an amateur like Michael Ventris) towering above the others, able to do things others cannot. In a well-developed, highly competitive field like modern mathematics, all the top players are "geniuses" in some sense (rare talents, one in a million), even though they don't stand out very much from each other. In Gleick's book, Feynman, discussing how long it might have taken to develop general relativity had Einstein not done it, says "We are not that much smarter than each other"!

To put it simply, if I sample sparsely from a Gaussian distribution, I might find a super-outlier in the resulting set. If I sample densely and have a high minimum cutoff for acceptable points, I will end up with a set entirely composed of outliers, but who do not stand out much from each other. Every guard in the NBA is an athletic freak of nature, even though they are relatively evenly matched when playing against each other.

To counteract the intelligence-damping effect of Gladwell's talk, I suggest this podcast interview with Nassim Taleb, about his new book The Black Swan. Warning: may be psychologically damaging to people who fool themselves and others about their ability to predict the behavior of nonlinear systems.

Are we getting smarter? Why?

It is well known that raw scores on IQ tests have been increasing at a rate of what would be about 3 IQ points per decade (the so-called Flynn effect). This means that, were the result of the test not rescaled so that the average is 100 by definition, the average IQ would have risen to 130 over the last century - i.e., the average person today scores better than all but 2 percent or so of the population in 1900. Malcolm Gladwell discusses this effect and its possible causes in this week's New Yorker, reviewing a new book claiming that modern society, with its fast-paced multimedia entertainment (including video games, computers, TV, etc.) actually improves our cognitive skills.

Gladwell: "Twenty years ago, a political philosopher named James Flynn uncovered a curious fact. Americans—at least, as measured by I.Q. tests—were getting smarter. This fact had been obscured for years, because the people who give I.Q. tests continually recalibrate the scoring system to keep the average at 100. But if you took out the recalibration, Flynn found, I.Q. scores showed a steady upward trajectory, rising by about three points per decade, which means that a person whose I.Q. placed him in the top ten per cent of the American population in 1920 would today fall in the bottom third. Some of that effect, no doubt, is a simple by-product of economic progress: in the surge of prosperity during the middle part of the last century, people in the West became better fed, better educated, and more familiar with things like I.Q. tests. But, even as that wave of change has subsided, test scores have continued to rise—not just in America but all over the developed world. What’s more, the increases have not been confined to children who go to enriched day-care centers and private schools. The middle part of the curve—the people who have supposedly been suffering from a deteriorating public-school system and a steady diet of lowest-common-denominator television and mindless pop music—has increased just as much. What on earth is happening? In the wonderfully entertaining “Everything Bad Is Good for You” (Riverhead; $23.95), Steven Johnson proposes that what is making us smarter is precisely what we thought was making us dumber: popular culture.

...As Johnson points out, television is very different now from what it was thirty years ago. It’s harder. A typical episode of “Starsky and Hutch,” in the nineteen-seventies, followed an essentially linear path: two characters, engaged in a single story line, moving toward a decisive conclusion. To watch an episode of “Dallas” today is to be stunned by its glacial pace—by the arduous attempts to establish social relationships, by the excruciating simplicity of the plotline, by how obvious it was. A single episode of “The Sopranos,” by contrast, might follow five narrative threads, involving a dozen characters who weave in and out of the plot. Modern television also requires the viewer to do a lot of what Johnson calls “filling in,” as in a “Seinfeld” episode that subtly parodies the Kennedy assassination conspiracists, or a typical “Simpsons” episode, which may contain numerous allusions to politics or cinema or pop culture. The extraordinary amount of money now being made in the television aftermarket—DVD sales and syndication—means that the creators of television shows now have an incentive to make programming that can sustain two or three or four viewings.

...It doesn’t seem right, of course, that watching “24” or playing a video game could be as important cognitively as reading a book. Isn’t the extraordinary success of the “Harry Potter” novels better news for the culture than the equivalent success of “Grand Theft Auto III”? Johnson’s response is to imagine what cultural critics might have said had video games been invented hundreds of years ago, and only recently had something called the book been marketed aggressively to children: (Johnson) Reading books chronically understimulates the senses. Unlike the longstanding tradition of gameplaying—which engages the child in a vivid, three-dimensional world filled with moving images and musical sound-scapes, navigated and controlled with complex muscular movements—books are simply a barren string of words on the page. . . .

Books are also tragically isolating. While games have for many years engaged the young in complex social relationships with their peers, building and exploring worlds together, books force the child to sequester him or herself in a quiet space, shut off from interaction with other children. . . .

But perhaps the most dangerous property of these books is the fact that they follow a fixed linear path. You can’t control their narratives in any fashion—you simply sit back and have the story dictated to you. . . . This risks instilling a general passivity in our children, making them feel as though they’re powerless to change their circumstances. Reading is not an active, participatory process; it’s a submissive one.

He’s joking, of course, but only in part. The point is that books and video games represent two very different kinds of learning. When you read a biology textbook, the content of what you read is what matters. Reading is a form of explicit learning. When you play a video game, the value is in how it makes you think. Video games are an example of collateral learning, which is no less important."