There is nothing new under the sun

... except that the Jews, unlike Asian Americans, made a fuss about it.

About The Chosen: ... But the admissions policies of elite universities have long been both tightly controlled and shrouded in secrecy. In The Chosen, the Berkeley sociologist Jerome Karabel lifts the veil on a century of admission and exclusion at Harvard, Yale, and Princeton. How did the policies of our elite schools evolve? Whom have they let in and why? And what do those policies say about America?

p. 76: ... Harvard, Yale, and Princeton thus faced a painful choice: either maintain the almost exclusively objective academic standards for admission and face the arrival of increasing numbers of Jews or replace them with more subjective criteria that could be deployed to produce the desired outcome. Their decision to do the latter was a great departure from their historic practices and bequeathed to us the peculiar admissions process that we now take for granted.

From an earlier post Affirmative Action: the numbers:

US News ... Translating the advantages into SAT scores, study author Thomas Espenshade, a Princeton sociologist, calculated that African-Americans who achieved 1150 scores on the two original SAT tests had the same chances of getting accepted to top private colleges in 1997 as whites who scored 1460s and Asians who scored perfect 1600s.

Espenshade found that when comparing applicants with similar grades, scores, athletic qualifications, and family history for seven elite private colleges and universities:

Whites were three times as likely to get fat envelopes as Asians. Hispanics were twice as likely to win admission as whites. African-Americans were at least five times as likely to be accepted as whites.

More from The Chosen below. OCR = Department of Education's Office of Civil Rights, which conducted an investigation of anti-Asian bias in Harvard admissions around 1990.

The Chosen, p.510: ... Asian Americans had the highest SATs of all [among groups admitted to Harvard]: 1450 out of a possible 1600. In 1991 the Asian-American/white admission ratio [ratio of percentages of applicants from each group admitted] stood at 84 percent -- a sharp downturn from 98 percent in 1990, when the scrutiny from OCR was at its peak. Though [this ratio] never dropped again to the 64 percent level of 1986, it never returned to its 1990 zenith. Despite Asian Americans' growing proportion of the national population, their enrollment also peaked in 1990 at 20 percent, where it more or less remained until 1994. ... by 2001 it had dropped below 15 percent.

So the "subjective but fair" measures used in admissions resulted in a record high admit rate for Asians during the year Harvard was under investigation by the federal government. But mysteriously the admit rate (relative to that of white applicants) went down significantly after the investigation ended, and the overall Asian enrollment has not increased despite the increasing US population fraction of Asians.

Terence Tao

We have Fields medalist Terence Tao on campus to give the Moursund Lectures this week. His blog is here.

At dinner I got him to sign my copy of his book Solving Mathematical Problems, which I bought hoping it might someday help interest my kids in math :-)

This article describes Tao's remarkable intellectual development, as well as that of another mathematical prodigy, Lenny Ng (son of UNC physicist Jack Ng, who did his PhD with Julian Schwinger).

Questions for Dyson

I will be participating in a public Q&A session with Freeman Dyson later this term. Any reader of this blog will know that I'm an admirer of both his work in theoretical physics and his popular writing. (Related posts here.) In preparing for the event, I've been reading and re-reading all sorts of things by and about Dyson. Below is something I found quite striking:

Disturbing the Universe: ... In that spring of 1948 there was another memorable event. Hans [Bethe] received a small package from Japan containing the first two issues of a new physics journal. Progress of Theoretical Physics, published in Kyoto. The two issues were printed in English on brownish paper of poor quality. They contained a total of six short articles. The first article in issue No. 2 was called "On a Relativistically Invariant Formulation of the Quantum Theory of Wave Fields," by S. Tomonaga of Tokyo University. Underneath it was a footnote saying, "Translated from the paper . . . (1943) appeared originally in Japanese." Hans gave me the article to read. It contained, set out simply and lucidly without any mathematical elaboration, the central idea of Julian Schwinger's theory. The implications of this were astonishing. Somehow or other, amid the ruin and turmoil of the war, totally isolated from the rest of the world, Tomonaga had maintained in Japan a school of research in theoretical physics that was in some respects ahead of anything existing anywhere else at that time. He had pushed on alone and laid the foundations of the new quantum electrodynamics, five years before Schwinger and without any help from the Columbia experiments. He had not, in 1943, completed the theory and developed it as a practical tool. To Schwinger rightly belongs the credit for making the theory into a coherent mathematical structure. But Tomonaga had taken the first essential Step. There he was, in the spring of 1948, sitting amid the ashes and rubble of Tokyo and sending us that pathetic little package. It came to us as a voice out of the deep.

A few weeks later, Oppy received a personal letter from Tomonaga describing the more recent work of the Japanese physicists. They had been moving ahead fast in the same direction as Schwinger. Regular communications were soon established. Oppy invited Tomonaga to visit Princeton, and a succession of Tomonaga's students later came to work with us at Princeton and at Cornell. When I met Tomonaga for the first time, a letter to my parents recorded my immediate impression of him "He is more able than either Schwinger or Feynman to talk about ideas other than his own. And he has enough of his own too. He is an exceptionally unselfish person." On his table among the physics journals was a copy of the New Testament.

Ironically, Schweber, in his magisterial book QED and the Men Who Made It, advocates that Dyson deserved a share of the Nobel awarded to Feynman, Schwinger and Tomonaga, and somewhat downplays the role of Tomonaga.

Below are a list of questions I am considering for Dyson (I doubt he'll see it beforehand; does he read my blog? :-). Any suggestions are welcome!

You've written about how depressed you became over your war work analyzing Allied strategic bombing. Yet later you were a Jason, doing top secret military work for the US government. Could you talk about those two experiences, and your opinion about scientists working on weapons and advising the military?


Of the bomb designer turned disarmament activist Ted Taylor, who was the subject of a book called The Curve of Binding Energy, you once said "Very few people have Ted's imagination. ... I think he is perhaps the greatest man that I ever knew well. And he is completely unknown." Could you tell us more about Taylor?


You had a close association with many of the giants of the past -- Feynman, Dirac, Oppenheimer, Bethe. How do you compare them to the best people working today? Would they still be giants?


You advised Francis Crick, while he was still a physicist, that moving into biology might be premature. You thought that biology would eventually be more interesting than physics, but that Crick was too early. What would you be working on today if you were 25 years old?


You wrote that since childhood, some part of you had always known that the “Americans held the future in their hands and that the smart thing for me to do would be to join them.” Do Americans still hold the future in their hands, or will the future be made somewhere else -- for example in Asia or once again in Europe?


You've proposed that genetic engineering might be used for many purposes, from green energy to adapting humans for life in space. What about engineering ourselves for greater intelligence; could that be the next leap forward in human evolution?


You were at Princeton when Everett proposed his "Many Worlds" interpretation of quantum mechanics. Could you describe the reaction to his ideas then (including your own), and your present opinion? Any thoughts on the foundational questions of quantum mechanics?


How well did Feynman understand Second Quantization (or the idea of a quantum field) when he developed his approach to QED? At what point did he really understand the Schwinger / Tomonaga approach?


How much did Dirac understand about the path integral formulation of quantum mechanics before Feynman came along? Feynman was inspired by a formula in one of Dirac's papers, but has claimed that Dirac later acknowledged not knowing whether or how the analogy between amplitude and exponential of action could be made into an equality. Do you have any insight on this?

Zygote!

Electron microscope photograph of human sperm and egg. Is that a flagellum, or are you just happy to see me? :-)

(Someone sent me a bunch of electron microscope photos and I found this one really striking!)

Hard to believe that thing eventually grows into one (or two!) of these: :-)

Do advanced education and a challenging career make you smarter?

Take a population of children with similar (high) IQ scores. Follow them for the next 40 years. Consider two subsets:

Group A, those who obtain advanced (e.g., graduate) education and achieve exceptional career success as scientists, professionals or business leaders, and

Group C, another group that end up in less (intellectually) challenging jobs and with much less formal education (often no more than a high school diploma).

On the re-tests of these adults, did Group A outperform Group C relative to their childhood scores? No.

What does this mean? Enrichment is again seen as unlikely to drastically alter cognitive ability. An 1150 SAT kid is not going to become a 1460 or 1600 kid as a result of their college education. Yes, those funny little tests are measuring something real and relatively stable.

These results have been known for many years, thanks to the Terman study of 1,538 gifted individuals (see here and here). Note that overall the "Termites" tended to be very successful in life; we focus on the most and least successful outliers (Groups A and C) to test the effect of enrichment on cognitive ability.

Terman Study, volume 5: ... Perhaps the most direct way of determining whether the higher and lower occupational groups differed in the way they changed from childhood to young adulthood and then to middle age is to compare the average rank order of [each group] ... at these three stages of life. There was no difference in the amount of change in intellectual performance (IQ) to young or middle adulthood or in Concept Mastery score over the decade from early to later middle age. We must conclude that ... there was no tendency for the lower-level group to slip downwards during their careers.

Stanford alumni magazine: ... the 100 most successful and 100 least successful men in the group, defining success as holding jobs that required their intellectual gifts. The successes, predictably, included professors, scientists, doctors and lawyers. The non-successes included electronics technicians, police, carpenters and pool cleaners, plus a smattering of failed lawyers, doctors and academics. But here's the catch: the successes and non-successes barely differed in average IQ. [All Termites had high childhood IQs as a consequence of the selection process.] The big differences turned out to be in confidence, persistence and early parental encouragement.

10,000 hours of practice won't make you a genius, but being good at something might make you more likely to pursue it for 10,000 hours!

The high IQ scores of eminent scientists (measured late in life) were probably not caused by their scientific activities. Rather, their high cognitive ability aided their scientific success. This ability, while necessary for success, is clearly not by itself sufficient.

Admissions angst

More coverage of elite college admissions in the links below.

Inside Higher Ed on the Espenshade study (see comments!). NYTimes blog on "merit" and admissions angst.

Whistle blower report by a UCLA professor Timothy Groseclose on administration attempts to circumvent Proposition 209. If you read this report you will have a good idea of the micro-level dynamics and political economy of UC admissions. The title is pretty strong: Report on suspected malfeasance in UCLA admissions and the accompanying cover-up; let's see how long Groseclose retains the Marvin Hoffenberg Chair of American Politics :-/

Grand unification through gravitational effects

New paper! This is a followup to our earlier work 0805.0145, which characterized the size of uncertainties in coupling constant unification due to unknown short distance physics such as quantum gravity. In the new paper we show that non-supersymmetric SU(5) and SO(10) models can be made to unify for reasonable (natural) sizes of short distance effects. This raises the question of whether successful unification in supersymmetric models should be taken as strong evidence in favor of low-energy supersymmetry, as has been argued.

Grand unification through gravitational effects

Xavier Calmet, Stephen D. H. Hsu, David Reeb

We systematically study the unification of gauge couplings in the presence of (one or more) effective dimension-5 operators cHGG/4MPl, induced into the grand unified theory by gravitational interactions at the Planck scale MPl. These operators alter the usual condition for gauge coupling unification, which can, depending on the Higgs content H and vacuum expectation value, result in unification at scales MX significantly different than naively expected. We find non-supersymmetric models of SU(5) and SO(10) unification, with natural Wilson coefficients c, that easily satisfy the constraints from proton decay. Furthermore, gauge coupling unification at scales as high as the Planck scale seems feasible, possibly hinting at simultaneous unification of gauge and gravitational interactions. In an appendix we work out the group theoretical aspects of this scenario for SU(5) and SO(10) unified groups in detail; this material is also relevant in the analysis of non-universal gaugino masses obtained from supergravity.

From the introduction to the paper:

What are the boundary conditions for grand unification? One typically assumes that the gauge couplings of the broken subgroups must become numerically equal at the unification scale MX [1]. However, effects from physics above the unification scale can alter the gauge coupling unification condition. In an effective field theory approach, such effects can be caused by dimension-5 operators of the form c H Gµν Gµν /4MPl , which shift the coefficients of the gauge kinetic terms in the low-energy theory after the Higgs H acquires a vacuum expectation value in grand unified symmetry breaking [2, 3]; one obvious source of such operators is quantum gravitational effects. Indeed, it would be unnatural (or require some special explanation) to assume that the Wilson coefficients c above be zero or especially small [4]; the default assumption should be that these coefficients are of order unity in grand unified models, with consequent unification conditions.

In conventional unification models, one might expect $< H > \sim 10^16$ GeV, plausibly leading to effects from quantum gravity of order a fraction of a percent, $< H > / MPl \sim 10^{−3}$, on the gauge coupling unification condition. In [5] we showed that these dimension-5 operators can be even more relevant than previously suspected since the Planck mass MPl tends to be smaller than naively assumed due to its renormalization group evolution [6, 7] under the influence of the large number of fields in supersymmetric grand unified theories. It was noted [5] that these dimension-5 operators introduce in supersymmetric unification models an uncertainty that can be bigger than the two-loop effects which are considered to be necessary to obtain good numerical unification of the gauge couplings.

The aim of this paper is different. We study whether the dimension-5 operators discussed above can lead to perfect gauge coupling unification without supersymmetry by their modifying the gauge coupling unification condition. This unification scheme has been studied previously in the literature for models with and without supersymmetry, e.g. in [2, 3, 5, 8–13], but in less detail and generality and mostly only the effect from a single gravitational operator has been considered. ...

Qian Xuesen, father of Chinese missile program and of JPL, dies at 98

Caption: Left to right: Ludwig Prandtl (German scientist), Qian Xuesen, Theodore von Kármán. Prandtl served for Germany during the World War II; von Kármán and Qian served [for the] US Army; after 1956, Qian served for China. Notice that at that time Qian had US Army rank. Interestingly, Prandtl was doctoral advisor for von Kármán; von Kármán was doctoral advisor for Qian. (Picture and caption from this Wikipedia entry.) "It was the stupidest thing this country ever did," former Navy Secretary Dan Kimball later said, according to Aviation Week. "He was no more a Communist than I was, and we forced him to go." See also this biography by the late Iris Chang. My father, also a professor of aerospace engineering, was an admirer of Qian and of von Karman. He was quite pleased that I decided to attend Caltech as an undergraduate -- although he would have preferred, for practical reasons, that I study EE or CS rather than theoretical physics!

Qian Xuesen dies at 98; rocket scientist helped establish Jet Propulsion Laboratory Deported in 1955 on suspicion of being a Communist, the aeronautical engineer educated at Caltech became known as the father of China's space and missile programs. November 1, 2009 Qian Xuesen, a former Caltech rocket scientist who helped establish the Jet Propulsion Laboratory before being deported in 1955 on suspicion of being a Communist and who became known as the father of China's space and missile programs, has died. He was 98. Qian, also known as Tsien Hsue-shen, died Saturday in Beijing, China's state news agency reported. The cause was not given. Honored in his homeland for his "eminent contributions to science," Qian was credited with leading China to launch intercontinental ballistic missiles, Silkworm anti-ship missiles, weather and reconnaissance satellites and to put a human in space in 2003. The man deemed responsible for these technological feats also was labeled a spy in the 1999 Cox Report issued by Congress after an investigation into how classified information had been obtained by the Chinese. Qian, a Chinese-born aeronautical engineer educated at Caltech and the Massachusetts Institute of Technology, was a protege of Caltech's eminent professor Theodore von Karman, who recognized him as an outstanding mathematician and "undisputed genius." Qian's research contributed to the development of "jet-assisted takeoff" technology that the military began using in the 1940s. He was the founding director of the Daniel and Florence Guggenheim Jet Propulsion Center at Caltech and a member of the university's so-called Suicide Squad of rocket experimenters who laid the groundwork for testing done by JPL. But his brilliant career in the United States came to a screeching halt in 1950, when the FBI accused him of being a member of a subversive organization. Qian packed up eight crates of belongings and set off for Shanghai, saying he and his wife and two young children wanted to visit his aging parents back home. Federal agents seized the containers, which they claimed contained classified materials, and arrested him on suspicion of subversive activity. Qian denied any Communist leanings, rejected the accusation that he was trying to spirit away secret information and initially fought deportation. He later changed course, however, and sought to return to China. Five years after his arrest, he was shipped off in an apparent exchange for 11 American airmen captured during the Korean War. "I do not plan to come back," Qian told reporters. "I have no reason to come back. . . . I plan to do my best to help the Chinese people build up the nation to where they can live with dignity and happiness." Welcomed as a national hero in China, where the Communist regime had defeated the Nationalist forces, Qian became director of China's rocket research and was named to the Central Committee of the Communist Party. China, whose scientific development lagged during the Communist revolution, quickly began making strides. Qian was born in the eastern city of Hangzhou, and in 1934 graduated from Jiaotong University in Shanghai, where he studied mechanical engineering. He won a scholarship to MIT and, after earning a master's degree in aeronautical engineering there, continued his doctoral studies at Caltech. He taught at MIT and Caltech and, having received a security clearance, served on the Scientific Advisory Board that advised the U.S. military during and after World War II. Sent to Germany to interrogate Nazi scientists, Qian interviewed rocket scientist Wernher von Braun. As the trade magazine Aviation Week put it in 2007, upon naming Qian its person of the year, "No one then knew that the father of the future U.S. space program was being quizzed by the father of the future Chinese space program." Qian returned to Caltech in 1949 and a year later faced the accusation by two former members of the Los Angeles Police Department's "Red Squad" that he was a card-carrying member of the Communist Party. He admitted that while a graduate student in the 1930s he had been present at social gatherings organized by colleagues who also were accused of party membership, but he denied any political involvement. Few can agree on the question of whether Qian was a spy. An examination of the papers Qian packed away failed to turn up any classified documents. Colleagues at Caltech firmly stood behind him, and he continued to do research there after he lost his security clearance. In fact, the university gave him its distinguished alumni award in 1979 in recognition of his pioneering work in rocket science. Although federal officials started deportation procedures in 1950, he was prevented from leaving the country because it was decided that he knew too much about sensitive military matters that could be of use to an enemy. For years, Qian was in a sort of limbo, being watched closely by the U.S. government and living under partial house arrest. Eventually he quit fighting his expulsion and actively worked to return to China. Some associates said that he was insulted because his loyalty to this country was questioned and that he initially wanted to clear his name. Once he returned home in 1955, he threw himself into his research with what some saw as calculated revenge. "It was the stupidest thing this country ever did," former Navy Secretary Dan Kimball later said, according to Aviation Week. "He was no more a Communist than I was, and we forced him to go."

Selectivity of US colleges and returns to elite education

Below is some work on US college admissions by economist Caroline Hoxby. There has been a recent increase in the number of American students who are interested in attending an elite university, possibly far from home. This accounts for almost all of the increased competitiveness of college admissions, and the effect is confined to only a small subset of elite schools. Selectivity at typical state universities has not increased.

These observations are consistent with my own experience. Growing up in Iowa, I remember many academically talented students from my high school (often children of professors) simply attending the local state university by default. Today, those same kids would be angling for admission to the Ivy league, Stanford, MIT, Caltech, etc.

However, it's not clear what caused this increased interest in elite education...

Near the end of the paper Hoxby reviews the literature on the returns to elite higher education. Her estimates of the rate of return are on the high side (definitely worthwhile, in her Harvard summa and Rhodes opinion ;-), and she casts some doubt on the Dale and Krueger study which compared elite graduates against others who were accepted to an elite school but chose not to attend. Dale and Krueger concluded that the choice of college had little impact on future earnings, but Hoxby suggests there was something "odd" about the 1 in 10 kids who declined elite admission :-)

See related posts returns to elite education , higher education and human capital.

Note to efficient market / rational agent / Chicago School believers: apparently even leading economists cannot agree on the size of returns to elite education. The economists, who are able to look backward in time and use massive statistics, have a much easier task than an individual student, who has to look forward 30 years to decide which choice is better. (Sound like projecting the cash flows from a bundle of mortgages over the next 30 years?) Therefore, it is hard to believe that individuals are clever enough to act like "rational agents" in solving this problem, or making any other life decision.

The Changing Selectivity of American Colleges

Caroline M. Hoxby
NBER Working Paper No. 15446*
Issued in October 2009

This paper shows that although the top ten percent of colleges are substantially more selective now than they were 5 decades ago, most colleges are not more selective. Moreover, at least 50 percent of colleges are substantially less selective now than they were then. This paper demonstrates that competition for space--the number of students who wish to attend college growing faster than the number of spaces available--does not explain changing selectivity. The explanation is, instead, that the elasticity of a student's preference for a college with respect to its proximity to his home has fallen substantially over time and there has been a corresponding increase in the elasticity of his preference for a college with respect to its resources and peers. In other words, students used to attend a local college regardless of their abilities and its characteristics. Now, their choices are driven far less by distance and far more by a college's resources and student body. It is the consequent re-sorting of students among colleges that has, at once, caused selectivity to rise in a small number of colleges while simultaneously causing it to fall in other colleges. I show that the integration of the market for college education has had profound implications on the peers whom college students experience, the resources invested in their education, the tuition they pay, and the subsidies they enjoy. An important finding is that, even though tuition has been rising rapidly at the most selective schools, the deal students get there has arguably improved greatly. The result is that the "stakes" associated with admission to these colleges are much higher now than in the past.

More here from the Atlantic Monthly:

... The researchers Alan Krueger and Stacy Berg Dale began investigating this question, and in 1999 produced a study that dropped a bomb on the notion of elite-college attendance as essential to success later in life. Krueger, a Princeton economist, and Dale, affiliated with the Andrew Mellon Foundation, began by comparing students who entered Ivy League and similar schools in 1976 with students who entered less prestigious colleges the same year. They found, for instance, that by 1995 Yale graduates were earning 30 percent more than Tulane graduates, which seemed to support the assumption that attending an elite college smoothes one's path in life.

But maybe the kids who got into Yale were simply more talented or hardworking than those who got into Tulane. To adjust for this, Krueger and Dale studied what happened to students who were accepted at an Ivy or a similar institution, but chose instead to attend a less sexy, "moderately selective" school. It turned out that such students had, on average, the same income twenty years later as graduates of the elite colleges. Krueger and Dale found that for students bright enough to win admission to a top school, later income "varied little, no matter which type of college they attended." In other words, the student, not the school, was responsible for the success.

... Which brings us back to the Krueger-Dale thesis. Can we really be sure Hamilton is nearly as good as Harvard?

Some analysts maintain that there are indeed significant advantages to the most selective schools. For instance, a study by Caroline Hoxby, a Harvard economist who has researched college outcomes, suggests that graduates of elite schools do earn more than those of comparable ability who attended other colleges. Hoxby studied male students who entered college in 1982, and adjusted for aptitude, though she used criteria different from those employed by Krueger and Dale. She projected that among students of similar aptitude, those who attended the most selective colleges would earn an average of $2.9 million during their careers; those who attended the next most selective colleges would earn $2.8 million; and those who attended all other colleges would average $2.5 million. This helped convince Hoxby that top applicants should, in fact, lust after the most exclusive possibilities.

"There's a clear benefit to the top fifty or so colleges," she says. "Connections made at the top schools matter. It's not so much that you meet the son of a wealthy banker and his father offers you a job, but that you meet specialists and experts who are on campus for conferences and speeches. The conference networking scene is much better at the elite universities." Hoxby estimates that about three quarters of the educational benefit a student receives is determined by his or her effort and abilities, and should be more or less the same at any good college. The remaining quarter, she thinks, is determined by the status of the school—higher-status schools have more resources and better networking opportunities, and surround top students with other top students.

"Today there are large numbers of colleges with good faculty, so faculty probably isn't the explanation for the advantage at the top," Hoxby says. "Probably there is not much difference between the quality of the faculty at Princeton and at Rutgers. But there's a lot of difference between the students at those places, and some of every person's education comes from interaction with other students." Being in a super-competitive environment may cause a few students to have nervous breakdowns, but many do their best work under pressure, and the contest is keenest at the Gotta-Get-Ins. Hoxby notes that some medium-rated public universities have established internal "honors colleges" to attract top performers who might qualify for the best destinations. "Students at honors colleges in the public universities do okay, but not as well as they would do at the elite schools," Hoxby argues. The reason, she feels, is that they're not surrounded by other top-performing students.

Do flu vaccines work?

The more I learn about the quality standards of medical research, the less I believe what my doctor tells me! Notice how hard it is, even in a "scientific" context to oppose the conventional wisdom.

Related posts: bounded cognition, Bouchard against group think, the future of innovation.

More from Gary Taubes on diet and nutrition issues, and WIRED on placebo mysteries.

As a postdoc I briefly dated a woman who was doing graduate work at Harvard's school of public health. I was a bit surprised at her low opinion of the scientific and statistical prowess of medical doctors, at least until I thought back to all the pre-med students I had taught while in grad school ;-) She pointed out to me that most MD's (as opposed to MD/PhD's) aren't scientists at all -- they simply apply what they're taught in medical school.

Atlantic Monthly: ... When Lisa Jackson, a physician and senior investigator with the Group Health Research Center, in Seattle, began wondering aloud to colleagues if maybe something was amiss with the estimate of 50 percent mortality reduction for people who get flu vaccine, the response she got sounded more like doctrine than science. “People told me, ‘No good can come of [asking] this,’” she says. “‘Potentially a lot of bad could happen’ for me professionally by raising any criticism that might dissuade people from getting vaccinated, because of course, ‘We know that vaccine works.’ This was the prevailing wisdom.”

Nonetheless, in 2004, Jackson and three colleagues set out to determine whether the mortality difference between the vaccinated and the unvaccinated might be caused by a phenomenon known as the “healthy user effect.” They hypothesized that on average, people who get vaccinated are simply healthier than those who don’t, and thus less liable to die over the short term. People who don’t get vaccinated may be bedridden or otherwise too sick to go get a shot. They may also be more likely to succumb to flu or any other illness, because they are generally older and sicker. To test their thesis, Jackson and her colleagues combed through eight years of medical data on more than 72,000 people 65 and older. They looked at who got flu shots and who didn’t. Then they examined which group’s members were more likely to die of any cause when it was not flu season.

Jackson’s findings showed that outside of flu season, the baseline risk of death among people who did not get vaccinated was approximately 60 percent higher than among those who did, lending support to the hypothesis that on average, healthy people chose to get the vaccine, while the “frail elderly” didn’t or couldn’t. In fact, the healthy-user effect explained the entire benefit that other researchers were attributing to flu vaccine, suggesting that the vaccine itself might not reduce mortality at all. Jackson’s papers “are beautiful,” says Lone Simonsen, who is a professor of global health at George Washington University, in Washington, D.C., and an internationally recognized expert in influenza and vaccine epidemiology. “They are classic studies in epidemiology, they are so carefully done.”

The results were also so unexpected that many experts simply refused to believe them. Jackson’s papers were turned down for publication in the top-ranked medical journals. One flu expert who reviewed her studies for the Journal of the American Medical Association wrote, “To accept these results would be to say that the earth is flat!” When the papers were finally published in 2006, in the less prominent International Journal of Epidemiology, they were largely ignored by doctors and public-health officials. “The answer I got,” says Jackson, “was not the right answer.”

... THE MOST vocal—and undoubtedly most vexing—critic of the gospel of flu vaccine is the Cochrane Collaboration’s Jefferson, who’s also an epidemiologist trained at the famed London School of Tropical Hygiene, and who, in Lisa Jackson’s view, makes other skeptics seem “moderate by comparison.” Among his fellow flu researchers, Jefferson’s outspokenness has made him something of a pariah. At a 2007 meeting on pandemic preparedness at a hotel in Bethesda, Maryland, Jefferson, who’d been invited to speak at the conference, was not greeted by any of the colleagues milling about the lobby. He ate his meals in the hotel restaurant alone, surrounded by scientists chatting amiably at other tables. He shrugs off such treatment. As a medical officer working for the United Nations in 1992, during the siege of Sarajevo, he and other peacekeepers were captured and held for more than a month by militiamen brandishing AK-47s and reeking of alcohol. Professional shunning seems trivial by comparison, he says.

“Tom Jefferson has taken a lot of heat just for saying, ‘Here’s the evidence: it’s not very good,’” says Majumdar. “The reaction has been so dogmatic and even hysterical that you’d think he was advocating stealing babies.” Yet while other flu researchers may not like what Jefferson has to say, they cannot ignore the fact that he knows the flu-vaccine literature better than anyone else on the planet. He leads an international team of researchers who have combed through hundreds of flu-vaccine studies. The vast majority of the studies were deeply flawed, says Jefferson. “Rubbish is not a scientific term, but I think it’s the term that applies.” Only four studies were properly designed to pin down the effectiveness of flu vaccine, he says, and two of those showed that it might be effective in certain groups of patients, such as school-age children with no underlying health issues like asthma. The other two showed equivocal results or no benefit.

... In the flu-vaccine world, Jefferson’s call for placebo-controlled studies is considered so radical that even some of his fellow skeptics oppose it.

Eric Baum: What is Thought?

Last week we had AI researcher and former physicist Eric Baum here as our colloquium speaker. (See here for an 11 minute video of a similar, but shorter, talk he gave at the 2008 Singularity Summit.)

Here's what I wrote about Baum and his book What is Thought back in 2008:

My favorite book on AI is Eric Baum's What is Thought? (Google books version). Baum (former theoretical physicist retooled as computer scientist) notes that evolution has compressed a huge amount of information in the structure of our brains (and genes), a process that AI would have to somehow replicate. A very crude estimate of the amount of computational power used by nature in this process leads to a pessimistic prognosis for AI even if one is willing to extrapolate Moore's Law well into the future. Most naive analyses of AI and computational power only ask what is required to simulate a human brain, but do not ask what is required to evolve one. I would guess that our best hope is to cheat by using what nature has already given us -- emulating the human brain as much as possible.

This perspective seems quite obvious now that I have kids -- their rate of learning about the world is clearly enhanced by pre-evolved capabilities. They're not generalized learning engines -- they're optimized to do things like recognize patterns (e.g., faces), use specific concepts (e.g., integers), communicate using language, etc.

What is Thought?

In What Is Thought? Eric Baum proposes a computational explanation of thought. Just as Erwin Schrodinger in his classic 1944 work What Is Life? argued ten years before the discovery of DNA that life must be explainable at a fundamental level by physics and chemistry, Baum contends that the present-day inability of computer science to explain thought and meaning is no reason to doubt there can be such an explanation. Baum argues that the complexity of mind is the outcome of evolution, which has built thought processes that act unlike the standard algorithms of computer science and that to understand the mind we need to understand these thought processes and the evolutionary process that produced them in computational terms.

Baum proposes that underlying mind is a complex but compact program that exploits the underlying structure of the world. He argues further that the mind is essentially programmed by DNA. We learn more rapidly than computer scientists have so far been able to explain because the DNA code has programmed the mind to deal only with meaningful possibilities. Thus the mind understands by exploiting semantics, or meaning, for the purposes of computation; constraints are built in so that although there are myriad possibilities, only a few make sense. Evolution discovered corresponding subroutines or shortcuts to speed up its processes and to construct creatures whose survival depends on making the right choice quickly. Baum argues that the structure and nature of thought, meaning, sensation, and consciousness therefore arise naturally from the evolution of programs that exploit the compact structure of the world.

When I first looked at What is Thought? I was under the impression that Baum's meaning, underlying structure and compact program were defined in terms of algorithmic complexity. However, it's more complicated than that. While Nature is governed by an algorithmically simple program (the Standard Model Hamiltonian can, after all, be written down on a single sheet of paper) a useful evolved program has to run in a reasonable amount of time, under resource (memory, CPU) constraints that Nature itself does not face. Compressible does not imply tractable -- all of physics might reduce to a compact Theory of Everything, but it probably won't be very useful for designing jet airplanes.

Useful programs have to be efficient in many ways -- algorithmically and computationally. So it's not a tautology that Nature is very compressible, therefore there must exist compact (useful) programs that exploit this compressibility. It's important that there are many intermediate levels of compression (i.e., description -- as in quarks vs molecules vs bulk solids vs people), and computationally effective programs to deal with those levels. I'm not sure what measure is used in computer science to encompass both algorithmic and computational complexity. Baum discusses something called minimum description length, but it's not clear to me exactly how the requirement of effective means of computation is formalized. In the language of physicists, Baum's compact (useful) programs are like effective field theories incorporating the relevant degrees of freedom for a certain problem -- they are not only a compressed model of the phenomena, but also allow simple computations.

Evolution has, using a tremendous amount of computational power, found these programs, and our best hope for AI is to exploit their existence to speed our progress. If Baum is correct, the future may be machine learning guided by human Mechanical Turk workers.

Baum has recently relocated to Berkeley to pursue a startup based on his ideas. (Ah, the excitement! I did the same in 2000 ...) His first project is to develop a world class Go program (no lack of ambition :-), with more practical applications down the road. Best of Luck!

Pessimism of the Intellect, Optimism of the Will

I often get questions about the slogan on my homepage:

Pessimism of the Intellect, Optimism of the Will.

Some recognize that it's a quote from Antonio Gramsci and mistakenly think it means I'm a Marxist.

In fact, the meaning of the slogan is quite simple.

Pessimism of the Intellect means, simply, be a scientist: see the world as it really is, not as you might like it to be. Try to identify and overcome hidden biases or prior assumptions. Always ask yourself: What assumption am I making? What if it is incorrect? How do I know what I know?

In many cases, the correct answer is: I don't know. Never be afraid to admit you don't know.

Optimism of the Will means, have the courage to attempt difficult things. Sometimes, Will can overcome the odds.

If people bring so much courage to this world the world has to kill them to break them, so of course it kills them. The world breaks everyone and afterward many are strong in the broken places. But those that will not break it kills. It kills the very good and the very gentle and the very brave impartially. If you are none of these you can be sure it will kill you too but there will be no special hurry. --- A Farewell to Arms, Ernest Hemingway

I'll fight them, I'll fight them until I die. --- The Old Man and the Sea, Ernest Hemingway

Evolution, Design and the Fermi Paradox

What is the time scale for evolution of complex organisms such as ourselves? On Earth complex life evolved in about 5 billion years (5 Gyr), but one can make an argument that we were probably lucky and that the typical time scale T under similar circumstances is much longer.

There is an interesting coincidence at work: 5 Gyr is remarkably close to the 10 Gyr lifetime of main sequence stars (and to the 14 Gyr age of the universe). This is unexpected, as evolution proceeds by molecular processes and natural selection among complex organisms, whereas stellar lifetimes are determined by nuclear physics.

If T were much smaller than 5 Gyr then it would be improbable for evolution to have been so slow on Earth.

It seems more plausible that T is much larger than 5 Gyr, in which case we were lucky, in a sense I will explain. Inflationary cosmology predicts a very large universe (much larger than what is currently visible to us), so that complex life is likely to exist somewhere in the universe. Conditioning on our own existence (a use of the weak anthropic principle), we should not be surprised to find ourselves lucky -- the few Earth-like planets that manage to evolve life must do so before their suns die. Intelligent beings, while not likely to evolve on any particular Earth-like planet, are likely to observe an evolutionary history that took place over a fraction of 10 Gyr.

Why should T be so large? At present we are unable to make quantitative estimates for the rate of evolution from first principles. It is entirely possible that certain evolutionary steps were highly improbable, such as the appearance of the first self-replicating complex molecules. One can also imagine abstract fitness surfaces with local maxima that trap the system for exponentially long periods of time.

I would not be surprised to find that T is exponentially larger than 5 Gyr. Godel went so far as to propose: "... a mathematical theorem to the effect that the formation within geological times of a human body by the laws of physics (or any other law of a similar nature) starting from a random distribution of the elementary particles and the field, is about as unlikely as the separation by chance of the atmosphere into its components."

The framework described above makes the following predictions:

1. The overwhelming majority of Earth-like planets are devoid of life, thereby resolving the Fermi Paradox.

2. Improved understanding of evolution will uncover highly improbable steps -- that is, improbable even over billions (or perhaps 10^100 !) of years. The fact that life on Earth climbed these steps might suggest intelligent design or divine intervention, but is better explained by the anthropic principle.

See related post evolutionary time scales.

Note added: This idea came to me after reading some discussion of ID and the question of improbable steps in evolution. (Here improbable means, for example, that even in an Earth-size population the multiple simultaneous mutations required jump across a particular fitness valley are unlikely to occur in 5 Gyr given the known mutation rate.) It occurred to me that under certain assumptions what might appear to be ID could actually be due to selection bias -- not taking into account the possibility that complex life is rare even on Earth-like planets. It turns out that the idea is not new -- it goes all the way back to Brandon Carter's 1983 paper which (I believe) coined the term "anthropic principle"! See John D. Barrow and Frank J. Tipler, The Anthropic Cosmological Principle, p. 557 for more. Also see this 1998 paper by Robin Hanson (thanks to a commenter for pointing it out). If one assumes many improbable required steps, one can deduce an upper estimate on the remaining time over which favorable conditions can persist on Earth. Note it appears the remaining lifetime of the sun is less than 1 Gyr, so the coincidence is tighter than I suggested in the original post. As the number of required improbable steps increases the likelihood that we would evolve just before time runs out (favorable conditions end) becomes very high -- this is quantified in the two references above.

The point which I have (still) not seen discussed much is that biologists need not be so defensive about improbable steps in evolution. I sense an almost reflexive -- i.e., prior-driven -- response to any claims of improbability. (On the other hand, perhaps biologists just know more about the details: the claim would be that the historical record does not resemble a typical one that would be generated by a chain of improbable events. I think this question requires further study.) But the main takeaway from this analysis is that improbability does not imply design or intervention.

Affirmative action: the numbers

Princeton sociologist Thomas Espenshade has a new book on elite college admissions. He is an author of an earlier study which quantified the advantages and disadvantages conferred by ethnic group at virtually all universities (with only a few notable exceptions such as Caltech). Already, some results from the analysis in the book have leaked out, and the numbers are not pretty -- see below.

US News: ... Translating the advantages into SAT scores, study author Thomas Espenshade, a Princeton sociologist, calculated that African-Americans who achieved 1150 scores on the two original SAT tests had the same chances of getting accepted to top private colleges in 1997 as whites who scored 1460s and Asians who scored perfect 1600s.

Espenshade found that when comparing applicants with similar grades, scores, athletic qualifications, and family history for seven elite private colleges and universities:

Whites were three times as likely to get fat envelopes as Asians. Hispanics were twice as likely to win admission as whites. African-Americans were at least five times as likely to be accepted as whites.

To clarify, 1150 is similar to the average at a typical state university, 1460 similar to the average at a top Ivy, and 1600 is obtained by only a few thousand high school seniors each year (note, this is post-1995 re-centering of the SAT). Cognitively, these are very different groups of kids.

Related posts: Asian-Americans hurt by affirmative action, the ugly truth, the price of admission, Asians at Berkeley, the new white flight.

See these slides for more results. The bottom slide below suggests that if race-blind admissions were enacted the percentage of Asian students at elite universities would jump from 24 percent to 39 percent. Note that 39 percent is similar to the current Asian populations at Caltech and Berkeley, two elite institutions with (roughly) race-blind admissions; the former due to meritocratic idealism, the latter thanks to Proposition 209.

(Click for larger versions.)

Universe Splitter -- iPhone app

New! For your iPhone :-)

Thanks to a correspondent who runs a quant hedge fund and who once worked on phase transitions in the early universe ;-)

Universe Splitter

Tough decision? There's no need to choose — Split the universe and do both!

Scientists say that every quantum event plays out simultaneously in every possible way, with each possibility becoming real in a separate universe. You can now harness this powerful and mysterious effect right from your iPhone or iPod Touch!

How? Whenever you're faced with a choice — for example, whether to accept a job offer or to turn it down — just type both of these actions into Universe Splitter©, and press the button.

Universe Splitter© will immediately contact a laboratory in Geneva, Switzerland, and connect to a Quantis brand quantum device, which releases single photons into a partially-silvered mirror. Each photon will simultaneously bounce off the mirror and pass through it — but in separate universes.*

Within seconds, Universe Splitter© will receive the experiment's result and tell you which of the two universes you're in, and therefore which action to take. Think of it — two entire universes, complete with every last planet and galaxy, and in one, a version of you who took the new job, and in the other, a version of you who didn't!*

Posner: How I became a Keynesian

Somehow I missed this! Thanks to a reader for pointing it out to me.

Posner was as captured by Chicago School nonsense as anyone else, but at least we learn that he can perform a Bayesian update (i.e., learn from reality) -- posteriors need not be wholly determined by priors :-)

Strangely, I don't see any discussion of this article on the Becker-Posner blog. How do Gary Becker and Robert Lucas feel about the recent apostasy of their colleague?

How I Became a Keynesian, by Richard Posner

... I had never thought to read The General Theory of Employment, Interest, and Money, despite my interest in economics.

... We have learned since September that the present generation of economists has not figured out how the economy works. The vast majority of them were blindsided by the housing bubble and the ensuing banking crisis; and misjudged the gravity of the economic downturn that resulted; and were perplexed by the inability of orthodox monetary policy administered by the Federal Reserve to prevent such a steep downturn; and could not agree on what, if anything, the government should do to halt it and put the economy on the road to recovery. By now a majority of economists are in general agreement with the Obama administration's exceedingly Keynesian strategy for digging the economy out of its deep hole.

... The dominant conception of economics today, and one that has guided my own academic work in the economics of law, is that economics is the study of rational choice. People are assumed to make rational decisions across the entire range of human choice, including but not limited to market transactions, by employing a form (usually truncated and informal) of cost-benefit analysis. The older view was that economics is the study of the economy, employing whatever assumptions seem realistic and whatever analytical methods come to hand. Keynes wanted to be realistic about decision-making rather than explore how far an economist could get by assuming that people really do base decisions on some approximation to cost-benefit analysis.

... It is an especially difficult read for present-day academic economists, because it is based on a conception of economics remote from theirs. This is what made the book seem "outdated" to Mankiw--and has made it, indeed, a largely unread classic. (Another very distinguished macroeconomist, Robert Lucas, writing a few years after Mankiw, dismissed The General Theory as "an ideological event.") The dominant conception of economics today, and one that has guided my own academic work in the economics of law, is that economics is the study of rational choice. People are assumed to make rational decisions across the entire range of human choice, including but not limited to market transactions, by employing a form (usually truncated and informal) of cost-benefit analysis. The older view was that economics is the study of the economy, employing whatever assumptions seem realistic and whatever analytical methods come to hand. Keynes wanted to be realistic about decision-making rather than explore how far an economist could get by assuming that people really do base decisions on some approximation to cost-benefit analysis.

The General Theory is full of interesting psychological observations--the word "psychological" is ubiquitous--as when Keynes notes that "during a boom the popular estimation of [risk] is apt to become unusually and imprudently low," while during a bust the "animal spirits" of entrepreneurs droop. He uses such insights without trying to fit them into a model of rational decision-making.

An eclectic approach to economic behavior came naturally to Keynes, because he was not an academic economist in the modern sense. He had no degree in economics, and wrote extensively in other fields (such as probability theory--on which he wrote a treatise that does not mention economics). He combined a fellowship at Cambridge with extensive government service as an adviser and high-level civil servant, and was an active speculator, polemicist, and journalist. He lived in the company of writers and was an ardent balletomane.

... The third claim that I am calling foundational for Keynes's theory--that the business environment is marked by uncertainty in the sense of risk that cannot be calculated--now enters the picture. Savers do not direct how their savings will be used by entrepreneurs; entrepreneurs do, guided by the hope of making profits. But when an investment project will take years to complete before it begins to generate a profit, its prospects for success will be shadowed by all sorts of unpredictable contingencies, having to do with costs, consumer preferences, actions by competitors, government policy, and economic conditions generally. Skidelsky puts this well in his new book: "An unmanaged capitalist economy is inherently unstable. Neither profit expectations nor the rate of interest are solidly anchored in the underlying forces of productivity and thrift. They are driven by uncertain and fluctuating expectations about the future." Only what Keynes called "animal spirits," or the "urge to action," will persuade businessmen to embark on such a sea of uncertainty. "If human nature felt no temptation to take a chance, no satisfaction (profit apart) in constructing a factory, a railway, a mine or a farm, there might not be much investment merely as a result of cold calculation."

But however high-spirited a businessman may be, often the uncertainty of the business environment will make him reluctant to invest. His reluctance will be all the greater if savers are hesitant to part with their money because of their own uncertainties about future interest rates, default risks, and possible emergency needs for cash to pay off debts or to meet unexpected expenses. The greater the propensity to hoard, the higher the interest rate that a businessman will have to pay for the capital that he requires for investment. And since interest expense is greater the longer a loan is outstanding, a high interest rate will have an especially dampening effect on projects that, being intended to meet consumption needs beyond the immediate future, take a long time to complete.

... An ambitious public-works program can be a confidence builder. It shows that government means (to help) business. "The return of confidence," Keynes explains, "is the aspect of the slump which bankers and businessmen have been right in emphasizing, and which the economists who have put their faith in a ‘purely monetary' remedy have underestimated." In a possible gesture toward Roosevelt's first inaugural ("we have nothing to fear but fear itself"), Keynes remarks upon "the uncontrollable and disobedient psychology of the business world."

See also my talk (for physicists) on the financial crisis. Some related posts on Keynes. Even more on Keynes (12th Wrangler) here.

Efficient Frontier and William Bernstein

Last week we had polymath William Bernstein (finance thinker, author, portfolio manager, former neurologist and Berkeley PhD in chemistry) on campus for a seminar. I'd been a reader of his site Efficient Frontier for many years, and when I learned by chance that he lives in Oregon, invited him to visit.

Here is an Amazon link to Bernstein's books -- ranging from practical investing advice to histories of global trade and of the birth of prosperity in the modern world.

Below is an excerpt from the preface of his new book.

... I have come to the sad conclusion that only a tiny minority will ever succeed in managing their money even tolerably well.

Successful investors need four abilities. First, they must possess an interest in the process. It is no different from carpentry, gardening, or parenting. If money management is not enjoyable, then a lousy job inevitably results, and, unfortunately, most people enjoy finance about as much as they do root canal work.

Second, investors need more than a bit of math horsepower, far beyond simple arithmetic and algebra, or even the ability to manipulate a spreadsheet. Mastering the basics of investment theory requires an understanding of the laws of probability and a working knowledge of statistics. Sadly, as one financial columnist explained to me more than a decade ago, fractions are a stretch for 90 percent of the population.

Third, investors need a firm grasp of financial history, from the South Sea Bubble to the Great Depression. Alas, as we shall soon see, this is something that even professionals have real trouble with.

Even if investors possess all three of these abilities, it will all be for naught if they do not have a fourth one: the emotional discipline to execute their planned strategy faithfully, come hell, high water, or the apparent end of capitalism as we know it. “ Stay the course ” : It sounds so easy when uttered at high tide. Unfortunately, when the water recedes, it is not. I expect no more than 10 percent of the population passes muster on each of the above counts. This suggests that as few as one person in ten thousand (10 percent to the fourth power) has the full skill set. Perhaps I am being overly pessimistic. After all, these four abilities may not be entirely independent: if someone is smart enough, it is also more likely he or she will be interested in finance and be driven to delve into financial history.

But even the most optimistic assumptions — increase the odds at any of the four steps to 30 percent and link them -- suggests that no more than a few percent of the population is qualified to manage their own money. And even with the requisite skill set, more than a little moxie is involved. This last requirement — the ability to deploy what legendary investor Charley Ellis calls “ the emotional game ” — is completely independent of the other three; Wall Street is littered with the bones of those who knew just what to do, but could not bring themselves to do it.

It's a sobering thought that so few people make competent money managers, given that most Americans now manage their own retirement accounts. During dinner, we refined his estimate (taking into account correlation) of the fraction of competent investors to about 1 in 1000.

Although I had thought about this question in the past, and even identified the various factors, it hadn't occurred to me until dinner with Bernstein that this was a good reason to suggest finance as a career to someone who has the right interests (history, finance theory, markets -- relatively easily acquired, as these subjects are fascinating), personality factors (discipline, controlled risk taking, decisiveness -- not so easily acquired, but can be improved over time) and intelligence (not easily acquired, but perhaps the threshold isn't that high at 90th percentile). No one factor is particularly rare, but the combination certainly is. As long as the signal to noise ratio in a financial career is reasonable, a person with all these qualities can be confident of success. While only a few people will ever run a hedge fund, it would seem that almost everyone could benefit from help managing their money -- if the advisor has all the qualities listed above.

Calvin Trillin on Wall Street smarts

Calvin Trillin, one of my favorite New Yorker writers and a keen observer of American society, writes in the Times. It's true that the quants built machines (mortgage finance) that were too complex for their masters to operate. Related posts: The new new meme, The best and brightest, The bubble algorithm ...

(This may sound crazy, but does Trillin read this blog?)

NYTimes: Wall Street Smarts

By CALVIN TRILLIN

“IF you really want to know why the financial system nearly collapsed in the fall of 2008, I can tell you in one simple sentence.”

The statement came from a man sitting three or four stools away from me in a sparsely populated Midtown bar, where I was waiting for a friend. “But I have to buy you a drink to hear it?” I asked.

“Absolutely not,” he said. “I can buy my own drinks. My 401(k) is intact. I got out of the market 8 or 10 years ago, when I saw what was happening.”

He did indeed look capable of buying his own drinks — one of which, a dry martini, straight up, was on the bar in front of him. He was a well-preserved, gray-haired man of about retirement age, dressed in the same sort of clothes he must have worn on some Ivy League campus in the late ’50s or early ’60s — a tweed jacket, gray pants, a blue button-down shirt and a club tie that, seen from a distance, seemed adorned with tiny brussels sprouts.

“O.K.,” I said. “Let’s hear it.”

“The financial system nearly collapsed,” he said, “because smart guys had started working on Wall Street.” He took a sip of his martini, and stared straight at the row of bottles behind the bar, as if the conversation was now over.

“But weren’t there smart guys on Wall Street in the first place?” I asked.

He looked at me the way a mathematics teacher might look at a child who, despite heroic efforts by the teacher, seemed incapable of learning the most rudimentary principles of long division. “You are either a lot younger than you look or you don’t have much of a memory,” he said. “One of the speakers at my 25th reunion said that, according to a survey he had done of those attending, income was now precisely in inverse proportion to academic standing in the class, and that was partly because everyone in the lower third of the class had become a Wall Street millionaire.”

I reflected on my own college class, of roughly the same era. The top student had been appointed a federal appeals court judge — earning, by Wall Street standards, tip money. A lot of the people with similarly impressive academic records became professors. I could picture the future titans of Wall Street dozing in the back rows of some gut course like Geology 101, popularly known as Rocks for Jocks.

“That actually sounds more or less accurate,” I said.

“Of course it’s accurate,” he said. “Don’t get me wrong: the guys from the lower third of the class who went to Wall Street had a lot of nice qualities. Most of them were pleasant enough. They made a good impression. And now we realize that by the standards that came later, they weren’t really greedy. They just wanted a nice house in Greenwich and maybe a sailboat. A lot of them were from families that had always been on Wall Street, so they were accustomed to nice houses in Greenwich. They didn’t feel the need to leverage the entire business so they could make the sort of money that easily supports the second oceangoing yacht.”

“So what happened?”

“I told you what happened. Smart guys started going to Wall Street.”

“Why?”

“I thought you’d never ask,” he said, making a practiced gesture with his eyebrows that caused the bartender to get started mixing another martini.

“Two things happened. One is that the amount of money that could be made on Wall Street with hedge fund and private equity operations became just mind-blowing. At the same time, college was getting so expensive that people from reasonably prosperous families were graduating with huge debts. So even the smart guys went to Wall Street, maybe telling themselves that in a few years they’d have so much money they could then become professors or legal-services lawyers or whatever they’d wanted to be in the first place. That’s when you started reading stories about the percentage of the graduating class of Harvard College who planned to go into the financial industry or go to business school so they could then go into the financial industry. That’s when you started reading about these geniuses from M.I.T. and Caltech who instead of going to graduate school in physics went to Wall Street to calculate arbitrage odds.”

“But you still haven’t told me how that brought on the financial crisis.”

“Did you ever hear the word ‘derivatives’?” he said. “Do you think our guys could have invented, say, credit default swaps? Give me a break! They couldn’t have done the math.”

“Why do I get the feeling that there’s one more step in this scenario?” I said.

“Because there is,” he said. “When the smart guys started this business of securitizing things that didn’t even exist in the first place, who was running the firms they worked for? Our guys! The lower third of the class! Guys who didn’t have the foggiest notion of what a credit default swap was. All our guys knew was that they were getting disgustingly rich, and they had gotten to like that. All of that easy money had eaten away at their sense of enoughness.”

“So having smart guys there almost caused Wall Street to collapse.”

“You got it,” he said. “It took you awhile, but you got it.”

The theory sounded too simple to be true, but right offhand I couldn’t find any flaws in it. I found myself contemplating the sort of havoc a horde of smart guys could wreak in other industries. I saw those industries falling one by one, done in by superior intelligence. “I think I need a drink,” I said.

He nodded at my glass and made another one of those eyebrow gestures to the bartender. “Please,” he said. “Allow me.”

I highly recommend Trillin's book Remembering Denny, a meditation on life, success, complexity, the unknowability of others, and fate. Also see Amazon reviews.

Calvin Trillin's "Remembering Denny" is a Cheeveresque rumination on the unfulfilled potential of Trillin's Yale classmate, Denny Hansen. While at Yale, Hansen was so highly thought of that he was profiled in LIFE magazine and his classmates used to kid each other about which cabinet position they'd fill once Hansen had been elected President. After Yale, however, Hansen failed to live up to the high expectations everyone--friends, family, teachers, coaches--had for him. Trillin's book is a delicate examination of what that meant, both for Denny and for his constellation of friends and well-wishers.

Denny doesn't come alive as vividly as might be hoped here, but Trillin does an outstanding job of sketching this young man's life in terms of a larger picture about America. In a country where success on every level is much prized, Trillin subtly but thoroughly plumbs the reasons why Denny didn't succeed--at least not to the extent everyone thought he would. This uncharacteristically somber book is absorbing and thought-provoking, even if it doesn't quite reach the goals Trillin seems to have set for himself in the beginning chapters.

Signals from the market in Guangzhou

This NYTimes article, reported from the world's largest export tradeshow in Guangzhou, China, illustrates that

1. we're past the bottom (at least globally) for the recession

2. the dollar is in trouble, and everybody knows it

3. low-skill labor in China still has little pricing power; manufacturing outside of China is in for more tough times.

NYTimes: Throngs of buyers from around the world swarmed through the world’s largest trade show at its opening Thursday, underscoring how China’s combination of a cheap currency and low wages is producing a resurgence of exports this autumn.

As wholesalers and distributors from dozens of countries began snatching brochures and exchanging business cards at booths in exhibition halls with five times the floor area of the Empire State Building, Chinese exporters said they were convinced that sales were finally rebounding.

In one of the most telling signs of the recovery, container shipping prices from China to the West have jumped 50 percent in the past two months as exporters have scrambled for space on suddenly crowded decks. Chinese factories are hiring temporary workers to cope with a last-minute surge in orders, many of them from emerging markets but some from Europe and even from the economically weakened United States.

“We are very confident,” said Liu En Tian, the marketing manager of the Huasheng Jiangquan Group, a manufacturer of ceramic tiles in Linyi City. “Already the buyers who are coming this morning are more than last year.”

Like those of many Chinese manufacturers, his company’s exports fell by nearly half last winter because of the global economic slowdown, but they are now down only 20 percent from their peak more than a year ago because of a surge in sales to South America and the Middle East. “The economy will get better very soon” around the world, Mr. Liu said.

China’s many policies to help exporters, from tax breaks to currency market intervention, have relieved unemployment in China — but at the expense of contributing to it in other countries, and that is starting to fan trade tensions. President Barack Obama imposed tariffs starting at 35 percent on tires from China last month, and anti-dumping and anti-subsidy cases against China are piling up in front of trade tribunals around the world.

But as exporters here reviewed their orders for the coming months, they described a consistent pattern: Sales to emerging markets are recovering rapidly, demand from Europe is starting to rebound as the Chinese currency falls against the euro, and buying interest from the United States remains fairly weak.

The dollar has fallen steeply against the euro, the yen and most other currencies over the past two weeks, with a broad index of the U.S. currency dropping Thursday to its lowest level in 14 months. But the Chinese government has intervened heavily in currency markets to make sure that its currency, the yuan, falls with the dollar so that Chinese goods can maintain their competitive edge.

The result has been a steep slide in the value of the yuan against the euro, the yen, the Australian dollar and many other currencies that is making it cheaper for businesses from Helsinki to Sydney to order from China. The yuan has fallen 16 percent against the euro since early March and 31 percent against the Australian dollar.

“We export a lot to Australia,” said Linda Zhang, a sales manager at Hebei Wanlong Steel Structure, a maker of prefabricated housing, adding that demand from there “is coming back.”

Kimmo Tarkkonen, the chairman of SRS Fenno-El, a distributor of lamps and space heaters based on the outskirts of Helsinki, said that he was signing all his contracts with Chinese suppliers in dollars after concluding that the dollar would continue to fall against the euro, making his purchases even cheaper.

“We’d rather take the risk,” he said, adding that the dollar “is declining all the time, so the risk is minimal.”

Chinese exporters’ representatives at the fair said that they had not been able to raise prices for European customers as the euro rose because there was so much competition from other Chinese companies.

“Our price depends on our cost,” not the euro, said Nicola Hou, a marketer at Meizhou Koway Electronics, a clock radio manufacturer in Meizhou City in southeastern China.

Some Chinese companies, in another effort to remain competitive, have even been cutting prices after wages fell during the recent economic downturn — a decline from which they have not fully recovered. Most of the country has not yet seen a return of the labor shortages that started appearing shortly before the global economic downturn began.

“It’s easy to find workers — China has too much labor,” said Helen Chen, general manager of Yuyao Panasia International Trading in Yuyao in east-central China.

Transport costs are a small fraction of labor costs, but they are now rebounding from a vertiginous plunge over the winter. ...

Skidelsky at LSE

I highly recommend this brilliant LSE lecture by Robert Skidelsky. Among the topics covered: risk versus uncertainty; Chicago school as the academic scribblers responsible for recent disastrous ideological capture; rational expectations, efficient markets and all that nonsense; economics as a science (not) and the role of mathematics; fiscal versus monetary stimulus; Glass-Steagall; Capital ascendant over Labor and Government: the renewed relevance of Marx; the role of globalization and the neoliberal agenda.

Before attacking me as a socialist pinko (I am not), listen to the talk or at least read the earlier post linked to below.

Keynes and the Crisis of Capitalism
(podcast and video available)

Date: Wednesday 7 October 2009
Speaker: Professor Lord Skidelsky

Robert Skidelsky is Emeritus Professor of Political Economy at the University of Warwick. His three-volume biography of the economist John Maynard Keynes (1983, 1992, 2000) received numerous prizes, including the Lionel Gelber Prize for International Relations and the Council on Foreign Relations Prize for International Relations. He is the author of The World After Communism (1995) (American edition called The Road from Serfdom). He was made a life peer in 1991, and was elected Fellow of the British Academy in 1994.

This event celebrates his latest book, Keynes: The Return of the Master.

For more from Skidelsky on Keynes and the current crisis, see this earlier post.

Keynes: ... the ideas of economists and political philosophers, both when they are right and when they are wrong, are more powerful than is commonly understood. Indeed the world is ruled by little else. Practical men, who believe themselves to be quite exempt from intellectual influences, are usually the slaves of some defunct economist. Madmen in authority, who hear voices in the air, are distilling their frenzy from some academic scribbler of a few years back.

Spooks drowning in data

Almost every technical endeavor, from finance to high energy physics to biology to internet security to spycraft, is either already or soon to be drowning in Big Data. This is an inevitable consequence of exponential Moore's Laws in bandwidth, processing power, and storage, combined with improved "sensing" capability. The challenge is extracting meaning from all that data.

My impression is that the limiting factor at the moment is the human brainpower necessary to understand the idiosyncrasies of the particular problem, and, simultaneously, develop the appropriate algorithms. There are simply not enough people around who are good at this; it's not just a matter of algorithms, you need insight into the specific situation. Of equal importance is that the (usually non-technical) decision makers who have to act on the data need to have some rough grasp of the strengths and limitations of the methods, so as not to have to treat the results as coming from a black box.

To give you my little example of big data, on my desk (in Oakland, not in Eugene) I have stacks of terabyte drives with copies of essentially every Windows executable (program that runs on a flavor of Windows) that has appeared on the web in the past few years (about 5 percent of this is malware; also stored in our data is what each executable does once it's installed). Gathering this data was only modestly hard; analyzing it in a meaningful way is a lot harder!

NY Review of Books: On a remote edge of Utah's dry and arid high desert, where temperatures often zoom past 100 degrees, hard-hatted construction workers with top-secret clearances are preparing to build what may become America's equivalent of Jorge Luis Borges's "Library of Babel," a place where the collection of information is both infinite and at the same time monstrous, where the entire world's knowledge is stored, but not a single word is understood. At a million square feet, the mammoth $2 billion structure will be one-third larger than the US Capitol and will use the same amount of energy as every house in Salt Lake City combined.

Unlike Borges's "labyrinth of letters," this library expects few visitors. It's being built by the ultra-secret National Security Agency—which is primarily responsible for "signals intelligence," the collection and analysis of various forms of communication—to house trillions of phone calls, e-mail messages, and data trails: Web searches, parking receipts, bookstore visits, and other digital "pocket litter." Lacking adequate space and power at its city-sized Fort Meade, Maryland, headquarters, the NSA is also completing work on another data archive, this one in San Antonio, Texas, which will be nearly the size of the Alamodome.

Just how much information will be stored in these windowless cybertemples? A clue comes from a recent report prepared by the MITRE Corporation, a Pentagon think tank. "As the sensors associated with the various surveillance missions improve," says the report, referring to a variety of technical collection methods, "the data volumes are increasing with a projection that sensor data volume could potentially increase to the level of Yottabytes (1024 Bytes) by 2015."[1] Roughly equal to about a septillion (1,000,000,000,000,000,000,000,000) pages of text, numbers beyond Yottabytes haven't yet been named. Once vacuumed up and stored in these near-infinite "libraries," the data are then analyzed by powerful infoweapons, supercomputers running complex algorithmic programs, to determine who among us may be—or may one day become—a terrorist. In the NSA's world of automated surveillance on steroids, every bit has a history and every keystroke tells a story.

... Where does all this leave us? Aid concludes that the biggest problem facing the agency is not the fact that it's drowning in untranslated, indecipherable, and mostly unusable data, problems that the troubled new modernization plan, Turbulence, is supposed to eventually fix. "These problems may, in fact, be the tip of the iceberg," he writes. Instead, what the agency needs most, Aid says, is more power. But the type of power to which he is referring is the kind that comes from electrical substations, not statutes. "As strange as it may sound," he writes, "one of the most urgent problems facing NSA is a severe shortage of electrical power." With supercomputers measured by the acre and estimated $70 million annual electricity bills for its headquarters, the agency has begun browning out, which is the reason for locating its new data centers in Utah and Texas. And as it pleads for more money to construct newer and bigger power generators, Aid notes, Congress is balking.

The issue is critical because at the NSA, electrical power is political power. In its top-secret world, the coin of the realm is the kilowatt. More electrical power ensures bigger data centers. Bigger data centers, in turn, generate a need for more access to phone calls and e-mail and, conversely, less privacy. The more data that comes in, the more reports flow out. And the more reports that flow out, the more political power for the agency.

Rather than give the NSA more money for more power—electrical and political—some have instead suggested just pulling the plug. "NSA can point to things they have obtained that have been useful," Aid quotes former senior State Department official Herbert Levin, a longtime customer of the agency, "but whether they're worth the billions that are spent, is a genuine question in my mind."

Based on the NSA's history of often being on the wrong end of a surprise and a tendency to mistakenly get the country into, rather than out of, wars, it seems to have a rather disastrous cost-benefit ratio. Were it a corporation, it would likely have gone belly-up years ago. The September 11 attacks are a case in point. For more than a year and a half the NSA was eavesdropping on two of the lead hijackers, knowing they had been sent by bin Laden, while they were in the US preparing for the attacks. The terrorists even chose as their command center a motel in Laurel, Maryland, almost within eyesight of the director's office. Yet the agency never once sought an easy-to-obtain FISA warrant to pinpoint their locations, or even informed the CIA or FBI of their presence.

But pulling the plug, or even allowing the lights to dim, seems unlikely given President Obama's hawkish policies in Afghanistan. However, if the war there turns out to be the train wreck many predict, then Obama may decide to take a much closer look at the spy world's most lavish spender. It is a prospect that has some in the Library of Babel very nervous. "It was a great ride while it lasted," said one.

A Nobel for Barry?

Huh? In a few years he might have actually earned one. This just demonstrates the outright political biases of the Nobel Committee. The Nobel citation reads like this (without actually mentioning GWB): "Bush era bad, Obama good. Hope good."

Note I speak as an Obama supporter!

Will pundits on the left have the guts to point out that the emperor has no clothes? This kind of outcome just reinforces the paranoid fantasies of the far-right: that Obama's Harvard magna is fake, that the World Government has been grooming him for leadership since his student days, that Bill Ayers wrote Dreams From My Father, etc.

Speaking more broadly, it seems to me that our obsession with prizes (an offshoot of winner-take-all culture) is unhealthy, and that prizes these days are less and less correlated with actual achievement.

While my faith in the Nobel process is shaken (although commenters have already pointed out the Kissinger prize and of course there is always Modigliani-Miller ;-), my confidence in Obama is not -- he reacted properly.

Mr. Obama said he was "surprised and deeply humbled" by the committee’s decision, ... he said he would accept it as “a call to action.”

“To be honest,” the president said “I do not feel that I deserve to be in the company of so many of the transformative figures who have been honored by this prize, men and women who’ve inspired me and inspired the entire world through their courageous pursuit of peace.”

Someone just pointed out to me that this opens the door for a string theorist to win the physics prize ;-)