Pessimism of the Intellect, Optimism of the Will     Archive   Favorite posts   Twitter: @steve_hsu

Thursday, September 30, 2010

US to nearly break even on TARP?

Is anyone paying attention to the facts? Do voters' attention spans extend back two years? Can academic economists learn from actual events?

The main costs from TARP will come from bailing out the auto companies and insurer AIG, not banks.

NYTimes: Even as voters rage and candidates put up ads against government bailouts, the reviled mother of them all — the $700 billion lifeline to banks, insurance and auto companies — will expire after Sunday at a fraction of that cost and could conceivably earn taxpayers a profit.

... The Treasury never tapped the full $700 billion. It committed $470 billion and to date has disbursed $387 billion, mostly to hundreds of banks and later to A.I.G., to the auto industry — Chrysler, General Motors, the G.M. financing company and suppliers — and to what is, so far, an unsuccessful effort to help homeowners avoid foreclosures.

When Mr. Obama took office, the financial system remained so weak that his first budget indicated the Treasury might need another $750 billion for TARP. The administration soon dropped that idea as Mr. Geithner overhauled the rescue program and the banking system stabilized. Still, by mid-2009, the administration projected that TARP could lose $341 billion, a figure that reflected new commitments to A.I.G. and the auto industry.

The Congressional Budget Office, which had a slightly higher loss estimate initially, in August reduced that to $66 billion.

Now Treasury reckons that taxpayers will lose less than $50 billion at worst, but at best could break even or even make money. Its best-case scenarios, however, assume that A.I.G. and the auto companies will remain profitable and that Treasury will get a good price as it sells its corporate shares in coming years.

“We’d have to be very lucky to have both A.I.G. and the auto companies pay us back in full,” Mr. Elliott said.

... By any measure, TARP’s final tally will be less than even its advocates expected amid the crisis. But the program remains a big loser politically.

See this post from September 2008, during the heat of the TARP debate:

... The following false conundrum has been stated recently by numerous analysts, including Paul Krugman: "if Treasury wants to recapitalize banks it has to overpay for toxic assets, to the detriment of taxpayers; if it wants to pay fair prices for the assets then banks won't benefit." There is no conundrum if markets, at this instant in time, are systematically underpricing mortgage assets.

When the Internet bubble burst in the early years of this century, investors were so gun shy and under so much pressure that they would not pay even rationally justifiable prices for stakes in technology companies. Smart investors who were willing to put capital at risk could buy assets at fire sale prices and made huge profits. This is nothing more than fear and herd mentality at work. If herd thinking can lead to overpricing of assets, why not underpricing immediately following a collapse?

Markets overshoot on both the up- and the down-side!

These points are obvious to any trader... it's the academics with equilibrium intuitions who are struggling to understand!

... To understand, it helps to have seen the collapse of a financial bubble firsthand. If you haven't (as, I suspect is the case with most academic economists), you are likely to cling to the idea that the market price of an asset is a good forecast of its actual value. However, this is completely wrong in the wake of a collapse. (And, certainly, the predictive power of the market price cannot hold at all times -- it is likely to be most wrong at the peak and in the aftermath of a bubble.)

Buy high, sell low

A contrarian strategy ("the average investor is stupid") would have paid off pretty well over the last decade. Ah, the wisdom of markets :-)

WSJ: [A recent research report] calculates that mutual fund investors [over the last decade] bought into the Standard & Poor's 500-stock index at an average of 1,434. That's close to its record high of 1,565. If investors had invested at random times instead, their average purchase price would have been 1,171. [Note an actual contrarian strategy would have performed better than random buying.]

... Human beings are hard-wired to run with the herd. For millions of years, when the herd stampeded, the smartest move wasn't the hang around and wait to see why. It was to run.

And that's how they act on the stock market as well. But when it comes to investing, it's a bad idea. Your feelings are a bad guide. And there is no safety in numbers.

I am frequently surprised at how many people still give in to their instincts in these matters. During the housing boom, anything I wrote questioning house prices automatically drew scathing reactions. ...

Living like kings

Physics library, LeConte Hall, Berkeley, 1987. Studying string theory and Calabi-Yau tomfoolery about 100m from the Campanile in the picture above. We'll never have it better than that.

Me: Mike, I can't believe we're in here working on such a beautiful afternoon. Look at that sunshine!

Mike C. (the pride of Jadwin Hall): Hsu, we're doing exactly what we want to be doing. We're livin' like kings, man! Livin' like kings (big grin).

Sunday, September 26, 2010

Slumdog brainpower

I found the comment below interesting. Currently I think the effective population of India for producing elite brainpower (i.e., comparing to a reference population with average IQ = 100 and access to first world education and training) is roughly 100 million. For China this number might be 300+ million, but we don't feel their presence in the West as strongly because of language barriers and because of the ability of the Chinese economy to absorb many of the high achievers domestically. (Despite all the Chinese scientists and engineers emigrating to the West there are just as many staying at home to develop a continent-sized economy with a space program, high speed trains, tech and defense industry, etc.) Both the China and India numbers should increase drastically in another generation, although see below for challenges confronting India.

See these related posts, or click the India label below.

More where those came from

IIT uber alles

To have and have not

Shanghai from an Indian perspective

Battle for brainpower

Background: I come from Southern India, high achiever, made a pile as they would say, so broadening my attention to study these subjects to understand the cause of our underdevelopment and what we can do about it. I only have experience with Southern India, where I live and travel extensively; I also get to travel the world extensively so I can give detailed comparisons. While I obviously do want to help my country, I am of the "Face Reality As It Is" persuasion, so I don't worry about political correctness of any discussion here.

First, the good news: thanks to better nutrition, I have witnessed IQ go up substantially in the past 30 years. The typical South Indian immigrant you see in the US today (and they do seem to come mostly from the South these days) comes from a vastly lower social scale than the one you saw 40 years ago. The typical immigrant has parents or grandparents who would not be able to function well in Western society, because they lack the cognitive skills. So the Flynn effect seems very visible to me, from my vantage point in South India.

Second, there are even more gains to be had. Malnutrition is slowly going away in the South, but in the North, it is still very very prevalent - we are talking more than 50% of kids malnourished.

Third, a pacific temperament and a traditional respect for authority keeps lower IQ populations from the pervasive social breakdown you see in the US in similarly situated communities. Families are staying intact; if that changes in India in any large scale, I would write India off, because collectively, we Indians are still too dumb to handle "sophisticated" mating behavior (though I think even the Swedish men stay together with their mates to support their kids, in spite of the promiscuous reputation of their society) - I hope our Hindu Gods literally save us on this one!

Now the bad news. Democracy empowers the dumb, probably true everywhere, but truer in India. The elected need to be skillful at manipulating the dumb, so they tend to be smarter than those electing them. Still, all this manipulating exacts a toll. One of the manipulating they do is jobs-for-dumb-constituents, which begets appallingly bad government, particularly at local levels.

This has all sort of bad effects. With such bad local governance, much of India looks like an ungoverned mess - your senses get physically assaulted in almost any urban space in India, disgusting levels of filth and squalor. This is the result of the utter inability to plan, explained by the IQ of those in charge at local levels.

This is more serious than even that. As India urbanizes, we risk going back on Flynn effect, because poor sanitation leads to *urban* malnutrition - not lack of food, but gastrointestinal illness among kids - which lowers IQ. So not having functional lower levels of government puts India's progress at risk. This, I attribute, directly to democracy.

Given these two opposing forces, which was is it going to go? I believe the good edges out the bad, but that would be a hard case to make when you see the filth.

Only good news I can gather is that London and Hong Kong and so on were extremely filthy at a point in their development too.

Taipei Fine Arts Museum

Saturday, September 25, 2010

Machines and bilingualism

I had a terrifying thought the other day. I would guess that at 90 percent confidence level machine translation and voice recognition will be good enough in 20 years that people will be able to communicate pretty well across most language barriers using cheap and unobtrusive devices. If so, is it worth all this effort to make sure my kids are bilingual?

I say it's terrifying because of the significant effort we're expending on our Bilingual Kids Project -- including relocating to Taiwan for this sabbatical. Another point of clarification: I'm not saying in 20 years we'll have AI (far from it). But something that translates basic phrases and simple content (surely we'll have that: Moore's law, massive corpora of translated text, statistical machine learning, yada yada) would reduce significantly the value of all but the most sophisticated language skills.

Friday, September 24, 2010

The fundamental asymmetry of MMA

I once almost fought a Golden Gloves boxer. Almost.

"F@ck jiujitsu -- I'm gonna break your nose."

"Maybe, but if I get my hands on you there's no tapping. I'm tearing your arm off."

In MMA, it's unfair that the striker gets to hit the other guy at full power, but the grappler has to release the hold when the other guy taps. If there was no referee the striker would think very, very hard before mixing it up.

When I was faculty advisor for the Yale jiujitsu club we considered t-shirts with "Snap, Crackle, Pop" on them, but went with something more conservative like "Yale Brazilian Jiujitsu" :-)

Monday, September 20, 2010

Physicist(s) and the flash crash

Flash crash -- mystery solved?

This article profiles former physicist Gregg Berman and his investigation of the flash crash of May 6. Berman works for the SEC now after a long career in finance as a quant and risk manager. Earlier I posted a job ad for his group.

I predict when the dust settles Gregg won't be the only ex-physicist appearing in the story ;-)

NYTimes: ... In investigating the crash, Mr. Berman says he finds himself in a position similar to his physics work 20 years ago, when he was collecting huge amounts of data and comparing the competing views of many laboratories on a question dividing particle physics — whether the neutrino, one of the least known and most common elementary particles, actually had mass.

Today he finds himself in familiar territory, sifting through huge amounts of messy and disjointed data, and at the same time reading blogs and e-mails from a wide range of observers, each with a theory about what happened on May 6.

Despite his formal training as a physicist, Mr. Berman is no stranger to stock markets. After academia, he spent 16 years on Wall Street, first devising algorithmic trading strategies for hedge funds, then working for RiskMetrics Group, where he created software and dispensed risk-management advice to asset managers, banks and hedge funds.

Having worked with hedge funds and high-frequency traders, Mr. Berman came to his current job a year ago with practical market knowledge and a familiarity with the world of stock trading. Several prominent market players say they found Mr. Berman’s rare combination of experiences refreshing — and reassuring.

... “Many market participants told us, ‘We’re not quite sure what happened over all, but this is what my firm saw and the actions we took,’ ” Mr. Berman said. “It was like ‘C.S.I.’ We wanted to interview everyone around.”

Mr. Berman said the level of detail gleaned from his investigation will help provide the explanation for what occurred on May 6, even if it may not delivery the simple answer that many people would like.

“This level of fact proved to be very, very telling,” he said. “We started to build up a complete picture.”

Sunday, September 19, 2010

Back to work

Now it's back to work in Taiwan. I still had several administrative issues to take care of this morning (health insurance, travel reimbursement, opening a bank account, etc.), which made me stop and estimate the total amount of effort that it has taken to set up this sabbatical. I'd guess we expended a person-month of work to book travel, sublet our house, move in, set things up here, enroll the kids in a new school, etc., etc. It's pretty daunting considering our planned stay here is slightly less than a year!

I've had really good luck with jet lag in the last month. I adjusted fairly rapidly from Pacific to Taiwan time (-9 hrs), then to Spain (-6 hrs), and now (seemingly) back to Taiwan. I find melatonin works well for me, although individual responses vary. (I learned about it from a Japanese physicist who travels a lot.)

Below are some photos of my office and the Institute of Physics here. I've gotten in the habit of taking photos all the time using my iPhone. As I mentioned in an earlier post, it helps me remember what I've been doing during this life. ("The days are long, but the years are short.")

Saturday, September 18, 2010

Barcelona photos

I spent part of a day in Barcelona on the way back, mostly in the Gothic Quarter near the Ramblas. If you're looking for a reasonable place to stay, try the Residencia d'Investigadors, which is run by the Catalonia government and is open to professors and researchers. Some random photos below (click for larger versions).

Courtyard of the Picasso museum (no photos allowed of the collection).

An Olmec head at a small museum of indigenous American art across the street from the Picasso. Many of Barcelona's museums are free for professors.

A bookstore and a book that caught my eye.

The oldest magic store in Spain. I went in, hoping to see some good sleights, but was disappointed to see tourists (albeit Spanish) buying simple props and tricks. They did have some beautiful posters on the wall from old time conjurors.

Tourists eating dinner.

A cafe.

VIP lounge, Barcelona airport.

Thursday, September 16, 2010

They got (some) game

Will Jeremy Lin become the first (full) Asian-American and first Harvard grad to play in the modern NBA?

NYTimes: ... A little-known 6-foot-3 point guard, Lin wowed an N.B.A. summer league crowd by upstaging the league’s No. 1 draft choice. He signed a two-year contract with the Golden State Warriors and is edging toward becoming Harvard’s first N.B.A. player in more than 50 years.

... There was a time when Lin was on the junior varsity that Diepenbrock marveled that a 13-year-old would implore teammates, “There’s a double screen!” There was the state championship game against powerful Mater Dei when, in the waning seconds, Lin dribbled toward a screen, calmly retreated and restarted a play that ended in his clinching layup. There was the state tournament game when his Palo Alto teammates looked listless, so Lin uncharacteristically scored 35 points.

My cousin Richard Chang had a shot at it 20 years ago. A 6 foot 7 forward, he was heavily recruited out of Edison high school in Los Angeles, and played at Cal. If it weren't for a knee injury the summer after a freshman season where he saw a lot of playing time, I think he could have made the NBA. A torn ACL back then was a big deal -- he has a scar the length of his lower leg from the surgery. Richard was cut by the Warriors before the beginning of the season (could still happen to Lin), back in the late 80s. When I was a grad student at Berkeley I overlapped with Richard on campus, and undergraduates were always amazed to learn he was my cousin. "RICHARD CHANG is your cousin?!?" they'd ask. I learned a lot about big time sports culture and the sense of entitlement that athletes have. When I visited him in Huntington Beach the ushers would let us into the local movie theaters for free!

Anyone remember Harvard running back Eion Hu (all-time Harvard rushing leader in 1996)? I think he's a doctor now ;-)

Wednesday, September 15, 2010

Four types of men

A reader writes in response to my earlier post on psychometrics and life outcomes.

I saw your comments on the Technology Review site and found them pretty interesting ... the observations are roughly consistent with my experience on Wall Street. I just want to point out an old quote which you might not know that's relevant to the topic. According to legend (the quote is used often but I have never seen the original source cited) Field Marshall Erich von Manstein said that there were only 4 types of men. Smart hardworking men, who make excellent staff officers; smart lazy men who make the best general because they can make the right decision without all the information; stupid lazy men for whom uses could be found, and stupid hardworking men who must be avoided because of the problems they create.

Note: MIT Technology Review hosts a mirror version of my blog, and occasionally good comments appear there although I tend to follow the ones submitted at more closely.

Battle of the Brains

Speaking of quantum theorist Seth Lloyd, here (50 min video) is a funny BBC program called Battle of the Brains in which he outperforms six other brainiacs on a variety of tests :-) When I asked him about it, he downplayed his success, noting that the kinds of puzzles he was asked to solve are similar to what theoretical physicists do every day.

BBC: Horizon takes seven people who are some of the highest flyers in their field - a musical prodigy, a quantum physicist, an artist, a dramatist, an RAF fighter pilot, a chess grandmaster and a Wall Street trader. Each is put through a series of tests to discover who is the most intelligent.

Seth's talk the other evening on Quantum Biology was fantastic, but he used the blackboard so there won't be any slides posted online.

Tuesday, September 14, 2010

More on psychometrics

I've had some email discussions elaborating on the psychometrics slides I posted earlier. The slides themselves don't convey a lot of the important points I made in the talks so I thought I'd share this message on the blog.

Hi Guys,

I'm very interested in exactly the question Henry is getting at.

I think our simple two factor model

Grades = ability + work ethic = IQ + W

is not too crazy. Note that once you fix the ability level (=SAT score) the remaining variance in GPA has about the same SD regardless of value of SAT score (vertical red lines in the big figure in the slides). That suggests that we can think of IQ and W as largely uncorrelated random variables -- so there are smart lazy people, hard working dumb people, etc. I can't really prove the residual variance after IQ is controlled for is due to work ethic, but my experience in the classroom suggests that it is. (Note work ethic here isn't necessary general work ethic as a personality factor, but how hard the kid worked in the specific course. However, in our data we average over many courses taken by many kids, so perhaps it does get at variation of personality factor(s) in the overall population.) Beyond work ethic, some people are just more "effective" -- they can get themselves organized, are disciplined, can adapt to new challenges, are emotionally robust -- and this is also absorbed in the W factor above.

Now, in some fields there seems to be a minimum cognitive threshold. I've known physics students who worked incredibly hard and just couldn't master the material. That is reflected in our data on pure math and physics majors at UO. For all majors there is a significant positive correlation between SAT and upper GPA (in the range .3-.5).

Whether IQ has a large impact on life outcomes depends on how you ask the question. I do believe that certain professions are almost off-limits for people below a certain IQ threshold. But for most jobs (even engineer or doctor), this threshold is surprisingly low IF the person has a strong work ethic. In other words a +1 SD IQ person can probably still be a doctor or engineer if they have +(2-3) SD work ethic. However, such people, if they are honest with themselves, understand that they have some cognitive disadvantages relative to their peers. I've chosen a profession in which, every so often, I am the dumbest guy in the room -- in fact I put myself in this situation by going to workshops and wanting to talk to the smartest guys I can find :-) For someone of *average* work ethic I think you can easily find jobs for which the IQ threshold is +2 SD or higher. The typical kid admitted to grad school in my middle-tier physics department is probably > +2 SD IQ and at least +1.5 SD in work ethic -- ditto for a top tier law or med school. That's probably also the case these days for any "academic admit" at a top Ivy.

For typical jobs I think the correlation between success/income and IQ isn't very high. Other factors come into play, like work ethic, interpersonal skills, affect, charisma, luck, etc. This may even be true in many "elite" professions once you are talking about a population where everyone is above the minimum IQ threshold -- if returns to IQ above threshold are not that large then the other factors dominate and determine level of success. What is interesting about the Roe and SMPY studies is that they suggest that in science the returns to IQ above the +2 SD threshold (for getting a PhD) are pretty high. ***

Henry is right that for ideological reasons many researchers are happy to present the data so as to minimize the utility of IQ or testing in making life predictions. They might even go so far as to claim that since we use g-loaded tests in admissions, the conclusion that some professions require high IQ is actually circular. The social scientist who walked out of my Sci Foo talk actually made that claim.

Finally, when it comes to *individual* success I think most analysts significantly underestimate the role of pure blind luck (i.e., what remains when all other reasonable, roughly measurable variables have been accounted for; of course this averages out of any large population study). Or perhaps I am just reassuring myself about my limited success in life :-)


PS In the actual talks I gave I made most of these points. The slides are kind of bare bones...

*** You would have difficulty finding a hard scientist who would disagree with the statement it is a big advantage in my field to be super smart. However, thanks to political correctness, social science indoctrination, or unfamiliarity with psychometrics, it IS common for scientists to deny that being super smart has anything to do with scores on IQ tests. I myself question the validity of IQ tests beyond +(3-4) SD -- I'm more impressed by success on the IMO, Putnam, or in other high level competitions. (Although I realize that training has a big impact on performance in these competitions I do think real talent is a necessary condition for success.)

Monday, September 13, 2010

Pronto moda

Gritty reporting on the impact of immigrant Chinese on the garment industry in Italy -- click through and read the whole article. Some of this is depicted in the excellent movie Gommorrah. Pronto moda or fast fashion means that manufacturers follow what is hot on the street and get it into stores as fast as possible. More from FT.

See earlier post here (from 2005!).

NYTimes: ... Chinese laborers, first a few immigrants, then tens of thousands, began settling in Prato in the late 1980s. They transformed the textile hub into a low-end garment manufacturing capital — enriching many, stoking resentment and prompting recent crackdowns that in turn have brought cries of bigotry and hypocrisy.

The city is now home to the largest concentration of Chinese in Europe — some legal, many more not. Here in the heart of Tuscany, Chinese laborers work round the clock in some 3,200 businesses making low-end clothes, shoes and accessories, often with materials imported from China, for sale at midprice and low-end retailers worldwide.

It is a “Made in Italy” problem: Enabled by Italy’s weak institutions and high tolerance for rule-bending, the Chinese have blurred the line between “Made in China” and “Made in Italy,” undermining Italy’s cachet and ability to market its goods exclusively as high end.

Part of the resentment is cultural: The city’s classic Italian feel is giving way to that of a Chinatown, with signs in Italian and Chinese, and groceries that sell food imported from China.

But what seems to gall some Italians most is that the Chinese are beating them at their own game — tax evasion and brilliant ways of navigating Italy’s notoriously complex bureaucracy — and have created a thriving, if largely underground, new sector while many Prato businesses have gone under. The result is a toxic combination of residual fears about immigration and the economy.

... The rest of Italy is watching closely. “Lots of businesses from Emilia Romagna, Puglia and the Veneto say, ‘We don’t want to wind up like Prato,’ ” said Silvia Pieraccini, the author of “The Chinese Siege,” a book about the rise of the “pronto moda” or “fast fashion” economy.

Tensions have been running high since the Italian authorities stepped up raids this spring on workshops that use illegal labor, and grew even more when Italian prosecutors arrested 24 people and investigated 100 businesses in the Prato area in late June. The charges included money laundering, prostitution, counterfeiting and classifying foreign-made products as “Made in Italy.”

Yet many Chinese in Prato are offended at the idea that they have ruined the city. Instead, some argue, they have helped rescue Prato from total economic irrelevance, another way of saying that if the Italian state fails to innovate and modernize the economy, somebody else just might.

“If the Chinese hadn’t gone to Prato, would there be pronto moda?” asked Matteo Wong, 30, who was born in China and raised in Prato and runs a consulting office for Chinese immigrants. “Did the Chinese take jobs away from Italians? If anything, they brought lots of jobs to Italians.”

... Resentment runs high. “You take someone from Prato with two unemployed kids and when a Chinese person drives by in a Porsche Cayenne or a Mercedes bought with money earned from illegally exploiting immigrant workers, and this climate is risky,” said Domenico Savi, Prato’s chief of police until June.

According to the Prato mayor’s office, there are 11,500 legal Chinese immigrants, out of Prato’s total population of 187,000. But the office estimates the city has an additional 25,000 illegal immigrants, a majority of them Chinese.

With its bureaucracy, protectionist policies and organized crime, Italy is arguably Western Europe’s least business-friendly country. Yet in Prato, the Chinese have managed to create an entirely new economy from scratch in a matter of years.

A common technique used, often with the aid of knowledgeable Italian tax consultants and lawyers, is to open a business, close it before the tax police can catch up, then reopen the same workspace with a new tax code number.

“The Chinese are very clever. They’re not like other immigrants, who can be pretty thick,” said Riccardo Marini, a textile manufacturer and the head of the Prato branch of Confindustria, the Italian industrialists’ organization.

“The difficulty,” he added ruefully, “is in finding a shared understanding of the rules of the game.”

Prato’s streets have slowly become more and more Chinese, as the Chinese have bought out Italian-owned shops and apartments, often paying in cash. Public schools are increasingly filled with Chinese pupils.

Hypocrisy abounds. “The people in Prato are ostriches,” said Patrizia Bardazzi, who with her husband has run a high-end clothing shop in downtown Prato for 40 years. “I know people who rent space to the Chinese and then say, ‘I don’t come into the center because there are too many Chinese.’ They rent out the space and take the money and go to Forte dei Marmi,” she added, referring to the Tuscan resort town. ...

On the origin of probability in quantum mechanics

Here are the slides for my talk today at Benasque: On the origin of probability in quantum mechanics.

At the end I took a poll of the workshop participants and found that over half agreed with the following statement. About 20 percent were strongly opposed. Note this is a meeting on quantum coherence and decoherence, so there are a lot of practical types here, including experimentalists.

It is plausible (but of course unproven) that unitary evolution of a pure state in a closed system can reproduce, for semi-classical creatures inside the system, all of the phenomenology of the Copenhagen interpretation.

As one insightful participant pointed out while I was taking the poll, this is really a mathematical question (if not entirely well-posed), not a physics question.

More Benasque photos below. Went hiking with Seth Lloyd and some other guys, who kicked my butt! Seth's evening talk, entitled Quantum Biology, covered several biological phenomena in which quantum effects play an important role. There is evidence that evolution has fine-tuned structures so as to optimize efficiency, balancing effects like quantum coherence and decoherence.

Sunday, September 12, 2010

My overview of psychometrics

I used these slides in two talks given at Foo Camp 2010 and Sci Foo 2010. At these "self-organized" meetings attendees are encouraged to talk about whatever they find interesting, and I usually choose to talk about wacky stuff rather than my main research, which tends to be a bit too specialized for the audience. In previous years I've talked about ultimate fighting, internet security, startups, etc.

At Foo, which has a Silicon Valley flavor, I had several CEOs and a bunch of technologists in the audience, and didn't receive any objections to the material. One CEO (an IIT grad with a PhD in engineering from Princeton) who runs a software company employing 1000 developers in India, was very interested in my results and has since agreed to run some experiments (stay tuned!) related to personnel selection and the relation between g and coding ability. At Sci Foo, which has a more scientific or academic flavor, the audience consisted of science writers, Google engineers, physical and computer scientists, a neuroscientist, and (I think) a social scientist. Only the last two voiced objections -- the social scientist actually got up and walked out after 15 minutes. Others in the audience found these objections rather amusing -- How could they argue? All you did was show data; the conclusions are obvious!

See additional comments (elaboration from the talks) here.

East, West, and tests

I'm experiencing something very similar to what NY Times writer Elisabeth Rosenthal discusses below. In Eugene our kids attended a fancy university-run preschool, which was highly unstructured. The teachers and administrators were a bit defensive about this -- Don't say they're just playing all day, that's how kids learn! But surely in a 6 or 8 hour school day there is room for an hour or so of structured learning? Our Taipei kindergarten is much more scheduled, with phys ed, homework, and a detailed workbook that tracks each kid's progress. Interestingly, the fad in Asia is toward more Western style education, so perhaps the two will meet in the middle.

NYTimes: Testing, the Chinese Way

When my children were 6 and 8, taking tests was as much a part of the rhythm of their school day as tag at recess or listening to stories at circle time. There were the “mad minute” math quizzes twice each week, with the results elaborately graphed. There were regular spelling quizzes. Even today I have my daughter’s minutely graded third-grade science exams, with grades like 23/25 or A minus.

We were living in China, where their school blended a mostly Western elementary school curriculum with the emphasis on discipline and testing that typifies Asian educational styles. In Asia, such a march of tests for young children was regarded as normal, and not evil or particularly anxiety provoking. That made for some interesting culture clashes. I remember nearly constant tension between the Asian parents, who wanted still more tests and homework, and the Western parents, who were more concerned with whether their kids were having fun — and wanted less.

... Some education experts hail the change as a step forward from the ideological dark ages. “Research has long shown that more frequent testing is beneficial to kids, but educators have resisted this finding,” said Gregory J. Cizek, a professor of educational measurement and evaluation at the University of North Carolina at Chapel Hill.

Of course, the tests have to be age-appropriate, Professor Cizek notes, and the Race to the Top program includes funds for research to develop new exams. Filling in three pages of multiple-choice bubbles may not be appropriate for young children. Likewise “high stakes” tests — like the Chinese university entrance exam, which alone determines university placement — create anxiety and may unfairly derail a youngster’s future based on poor performance on a single day.

But Professor Cizek, who started his career as a second-grade teacher, said the prevailing philosophy of offering young children unconditional praise and support was probably not the best prescription for successful education. “What’s best for kids is frequent testing, where even if they do badly, they can get help and improve and have the satisfaction of doing better,” he said. “Kids don’t get self-esteem by people just telling them they are wonderful.”

... When testing is commonplace and the teachers are supportive — as my children’s were, for the most part — the tests felt like so many puzzles; not so much a judgment on your being, but an interesting challenge. It is a testament to the International School of Beijing — or to the malleability of childhood memory — that Andrew now says he did not realize that he was being tested. Will tests be like that in a national program, like Race to the Top?

When we moved back to New York City, my children, then 9 and 11, started at a progressive school with no real tests, no grades, not even auditions for the annual school musical. They didn’t last long. It turned out they had come to like the feedback of testing.

“How do I know if I get what’s going on in math class?” my daughter asked with obvious discomfort after a month. Primed with Beijing test-taking experience, they each soon tested into New York City’s academic public schools — where they have had tests aplenty and (probably not surprisingly) a high proportion of Asian classmates.

Saturday, September 11, 2010

Engineering right wing terror

Hmm... autistic geeks fight for abstract ideas whereas empathic humanists fight to end human suffering?

NYTimes: ... In a paper published last year in The European Journal of Sociology, Gambetta and Hertog argue that the engineer-terrorist connection is part of the answer: it is a new window onto what Gambetta calls the “hidden logic” of society. Though the difference in susceptibility is very small — “it’s like saying the probability that you will be struck by lightning is one in a million,” Gambetta says, “and the probability for an engineer to be struck by lightning is four in a million” — it is, they say, real.

For their recent study, the two men collected records on 404 men who belonged to violent Islamist groups active over the past few decades (some in jail, some not). Had those groups reflected the working-age populations of their countries, engineers would have made up about 3.5 percent of the membership. Instead, nearly 20 percent of the militants had engineering degrees. When Gambetta and Hertog looked at only the militants whose education was known for certain to have gone beyond high school, close to half (44 percent) had trained in engineering. Among those with advanced degrees in the militants’ homelands, only 18 percent are engineers.

The two authors found the same high ratio of engineers in most of the 21 organizations they examined, including Jemaah Islamiya in Southeast Asia and Hamas and Islamic Jihad in the Middle East. Sorting the militants according to their 30 homelands showed the same pattern: engineers represented a fifth of all militants from every nation except one, and nearly half of those with advanced degrees.

One seemingly obvious explanation for the presence of engineers in violent groups lies in the terrorist’s job description. Who, after all, is least likely to confuse the radio with the landing gear, as Gambetta puts it, or the red wire with the green? But if groups need geeks for political violence, then engineering degrees ought to turn up in the rosters of all terrorist groups that plant bombs, hijack planes and stage kidnappings. And that’s not the case.

Gambetta and Hertog found engineers only in right-wing groups — the ones that claim to fight for the pious past of Islamic fundamentalists or the white-supremacy America of the Aryan Nations (founder: Richard Butler, engineer) or the minimal pre-modern U.S. government that Stack and Bedell extolled.

Among Communists, anarchists and other groups whose shining ideal lies in the future, the researchers found almost no engineers. Yet these organizations mastered the same technical skills as the right-wingers. Between 1970 and 1978, for instance, the Baader-Meinhof gang in Germany staged kidnappings, assassinations, bank robberies and bombings. Seventeen of its members had college or graduate degrees, mostly in law or the humanities. Not one studied engineering.

The engineer mind-set, Gambetta and Hertog suggest, might be a mix of emotional conservatism and intellectual habits that prefers clear answers to ambiguous questions — “the combination of a sharp mind with a loyal acceptance of authority.” Do people become engineers because they are this way? Or does engineering work shape them? It’s probably a feedback loop of both, Gambetta says.

Economic frustration also matters, Gambetta says. In their sample of militants, there was only one homeland out of 30 in which engineers were less common: Saudi Arabia — where engineers have always had plenty of work. But “engineers’ peculiar cognitive traits and dispositions” made them slightly more likely than accountants, waiters or philosophers to react to career frustration by adopting violent, right-wing beliefs.

Friday, September 10, 2010

Los Pirineos

Wow, it doesn't get any better :-)

This morning we heard two beautiful lectures, one on the creation of macroscopic superpositions (Schrodinger cat states) of millions of photons, by Francesco De Martini of La Sapienza, and another by Adan Cabello on the Kochen-Specker theorem and related experimental tests. I'll post a link to slides once the organizers have them up.

And in the afternoon...

(Los Pirineos = Les Pyrennes. We are about 10 km south of the border with France.)

Thursday, September 09, 2010

Coordinating mediocrity

Seth Roberts (a psychology professor who splits his time between Berkeley and Tsinghua in Beijing) points me to an interesting paper by two Italian sociologists.

If you speak to young Italians, particularly scientists and other highly educated people, you will hear terrible stories about just how dysfunctional their academic and research systems have become. At meetings like the one I am currently attending, I often hear the comment that Italy's number one export is talented people, trained at the expense of taxpayers. "Look at all the Italians at this conference -- but they are all working in other countries!"

The paper characterizes the current situation in Italy as a collective outcome (equilibrium) with its own cynical and corrosive social norms. I can easily think of other examples! There is tremendous value to being an efficient "high trust" society (what the authors would call an H-world), and once the equilibrium has shifted in the L direction it is very hard to correct.

One very jarring thing about science and academia is that students entering the field are among the most idealistic of people, yet a significant fraction of senior researchers are among the most cynical. The proportions vary by discipline.

L-worlds: the curious preference for low quality and its norms

Diego Gambetta and Gloria Origgi
Department of Sociology, Oxford University

Abstract. We investigate a phenomenon which we have experienced as common when dealing with an assortment of Italian public and private institutions: people promise to exchange high quality goods and services (H), but then something goes wrong and the quality delivered is lower than promised (L). While this is perceived as ‘cheating’ by outsiders, insiders seem not only to adapt but to rely on this outcome. They do not resent low quality exchanges, in fact they seem to resent high quality ones, and are inclined to ostracise and avoid dealing with agents who deliver high quality. This equilibrium violates the standard preference ranking associated to the prisoner’s dilemma and similar games, whereby self-interested rational agents prefer to dish out low quality in exchange for high quality. While equally ‘lazy’, agents in our L-worlds are nonetheless oddly ‘pro-social’: to the advantage of maximizing their raw self-interest, they prefer to receive low quality provided that they too can in exchange deliver low quality without embarrassment. They develop a set of oblique social norms to sustain their preferred equilibrium when threatened by intrusions of high quality. We argue that cooperation is not always for the better: high quality collective outcomes are not only endangered by self-interested individual defectors, but by ‘cartels’ of mutually satisfied mediocrities.

Here is a nice example:

... When Federico Varese (1996) revealed that Stefano Zamagni, a well-established Italian economist, had plagiarised verbatim several pages from Robert Nozick, Varese was criticised by several Italian colleagues who together evoked nine norms or reasons that he would have violated by blowing the whistle. None of these include a justification of plagiarism per se. Varese discusses them in an unpublished article (“Economia d’idee II”). They are worth listing, their range is staggering:

1. There is nothing original, everyone plagiarises, so why bother? [journalist]

2. Whistle blowers are always worse than their targets [sociologist]

3. What is the point of targeting Zamagni? They will never punish him anyway.

4. What is the point of blowing the whistle as you will pay the consequences

5. He is a good “barone”, much better than many others, so why target him?

6. Zamagni is a member of the left and you should not weaken the left during election times [economist; various friends]

7. Zamagni shows good intellectual tastes as he plagiarises very good authors, so he does not deserve to be attacked [philosopher]

8. Given that many are guilty of plagiarism, targeting one in particular shows that the whistle blower is driven by base motives.

9. In addition, an economist suggested an explanation rather than a justification saying that the real author of the plagiarism was probably a student of Zamagni who wrote the paper for him. This would, funnily enough, imply that Zamagni was innocent of the plagiarism, but still that he signed a paper he did not write, written by someone who also did not write it!

From the conclusions:

... Whatever its origins, the cost of the L-propensity is proving over time more detrimental than helpful – flexibility shifts to laxness, tolerance to sloppiness, and confusion to breaches of trust – and standards in Italian education, politics, media and cultural creativity in general, although blessed by the occasional geniuses, have never risen and have quite possibly declined further. One does not need to be an incurable perfectionist to appreciate how sadly this is the case.

An implication of this paper is that the threat to good collective outcomes is not just free-riding. There are subtler ones. The L-world we described is not an extra-normative one populated by isolate individual predators free to roam around, but one governed by its own ‘perverse’ social norms. The social sciences have focused on cooperation and on the social norms that sustain it while narrowly conceiving of anti-cooperators as individualistic predators, acting free of normative constraints. Social norms, in the dominant interpretation, would exist as an antidote to our natural antisocial proclivities. The interest of our case is to suggest that this distinction does not stand up, and that those whom we think of as free-riders too operate within a normative structure – a special “cement of society” that glues L-doers together to the detriment of the common good.

See also footnote 3:

Recent experimental research carried out by Herrmann, Thöni & Gächter (2007) has come up with unexpected evidence which may be germane to our case. They ran the so called public-good game with university students in 15 cities in the developed and developing world, from the US to China, from the UK to Russia, Turkey and Saudi Arabia. ... In other words, there were some L-doers who punish H-doers. ... Going in descending order of size of punishment, punishment of more generous contributors was found in Muscat, Athens, Riyadh, Samara, Minsk, Istanbul, Seoul and Dnipropetrovs'k. Although not entirely absent, this type of punishment proved negligible in Boston, Nottingham, St. Gallen, Zurich, Copenhagen, Bonn, and Chengdu.

Wednesday, September 08, 2010

Eternal black holes

The article below appeared recently at New Scientist. See also

abstract and link to paper

discussion stimulated by Lubos Motl (more focused on white than black holes)

more discussion (on the entropy of isolated / exploding white holes)

I should note that I have no idea whether these "eternal" black holes are stable against perturbations.

Eternal black holes are the ultimate cosmic safes

18:59 03 September 2010 by Stephen Battersby

If you wanted to hide something away for all eternity, where could you put it? Black holes might seem like a safe bet, but Stephen Hawking famously calculated that they leak radiation, and most physicists now think that this radiation contains information about their contents. Now, there may be a way to make an "eternal" black hole that would act as the ultimate cosmic lockbox.

The recipe for this unlikely object was discovered by looking at an even more abstruse entity, the white hole. White holes are black holes that run backwards in time, throwing out matter instead of sucking it in. Where a black hole might form from a collapsing star, a white hole would explode and leave a star in its place. White holes have never been observed, though general relativity predicts they could exist in principle.

Stephen Hsu of the University of Oregon in Eugene wanted to caculate whether a white hole would emit radiation like a black hole. He considered the special case of a white hole sitting in a perfect vacuum, and calculated that when it spits out its contents, there is a burst of radiation essentially identical to a black hole's Hawking radiation (

'Exquisitely difficult'

Hsu realised that running the process backwards would be equivalent to a black hole forming and then existing in a perfect vacuum, with no Hawking radiation. "It becomes a black hole that's not radiating, which is a very weird thing," Hsu says.

The snag is that to run this process backwards and make the eternal black hole, you would need to send in a precisely crafted burst of radiation as the hole forms. The radiation would have to be "exactly tuned to interfere with the Hawking radiation that would otherwise come out", says Hsu.

"Maybe in a highly advanced civilisation, physicists could create a black hole that didn't evaporate," he told New Scientist. "It would be exquisitely difficult, but mathematically you can do it."

Encoded information

If one did build an eternal black hole, it would be the perfect place to store sensitive information.

Normal black holes are thought to gradually release information about their contents through Hawking radiation. "Most theorists have come to the conclusion that black hole evaporation is analogous to the burning of a book," says Martin Einhorn of the University of California, Santa Barbara. "All the information in the book must be encoded in special properties of the outgoing radiation." In principle, it would be possible to recreate the original book – if you could collect all the outgoing radiation and understood the quantum properties of gravity.

But with eternal black holes, "it's as if you just put the information inside a box and at the end you still have the box", says Hsu.

More Benasque photos

Click for larger version.

Tuesday, September 07, 2010

Raw g and AI

You can probably guess why I liked this paragraph, from a job description for a Research Fellow at the Singularity Institute.

So what does it take to get that job done? Well, for starters, sheer raw fluid intelligence, plain old-fashioned Spearman's g. You'll need to know things that aren't in textbooks and apply skills that aren't taught in classes. You'll have to pick things up rapidly, from a few hints, without them being hammered into you. I attended the inaugural symposium of the Redwood Center for Theoretical Neuroscience, and they asked a panel of prestigious experimental neuroscientists what kind of experience they'd most like to see in a hiree. And one said "Neuroscience", and one said "Electrical engineering", and then one said, "I'd rather hire a physicist, because they can learn anything," and the rest all nodded. That's the indispensable quality we're looking for, whether it appears in a physicist or not.

Why I think AI is very hard:

... evolution has compressed a huge amount of information in the structure of our brains (and genes), a process that AI would have to somehow replicate. A very crude estimate of the amount of computational power used by nature in this process leads to a pessimistic prognosis for AI even if one is willing to extrapolate Moore's Law well into the future. Most naive analyses of AI and computational power only ask what is required to simulate a human brain, but do not ask what is required to evolve one. I would guess that our best hope is to cheat by using what nature has already given us -- emulating the human brain as much as possible.

Greetings from Benasque

I'm attending a workshop on quantum coherence and decoherence. Disorienting to go from Oregon to Taiwan to Spain in just a few weeks! The crisp mountain air here couldn't be more different from the sub-tropical humidity of Taipei. Super travel hint: the Star Alliance lounge at Frankfurt has shower facilities :-)

I haven't been here since the first Benasque scientific meeting in the 90s, which was held in an empty elementary school. This morning I went for a run and tried to find the school (to do some pull ups on the playground), but failed. I found out later that they built the new center where the school used to be :-)

My Spanish is rusty, but it's starting to come back. Nada y pues nada, pero no por nada yo he vivido.

The view from the Center.


One of the lounges. The basket is full of brain teasers :-)

Even the signage here is cool!

Saturday, September 04, 2010

The measure of success

This article in the Oregon Quarterly (alumni magazine) describes my research with colleague Jim Schombert on college GPA predictions from SAT scores.

“Freshman GPA is not a satisfactory metric of academic success,” Hsu explains. “There is simply too much variation in the difficulty of courses taken by freshmen.” More able freshmen typically take more difficult courses, whereas less able freshmen take introductory courses “not very different from high school classes,” he says. Under these circumstances, academic success—an “A” in an introductory course versus a “B” in an advanced course—becomes too relative to accurately measure. Course variation decreases in later years, as students settle into their respective majors, working hard in required classes.

The new approach bore fruit: SAT and ACT scores, their analysis showed, predict upper-level much better than lower-level college grades, “a significant and entirely new result,” Schombert says.

To be precise, the correlation between SAT and GPA in upper division, in-major courses (presumably the most important courses in a particular student's college career) is much higher than the often reported correlation with freshman GPA.

The feel-good conclusion from our work is that in most majors (e.g., History, English, Biology, social sciences, ...) students with modest SAT scores can still obtain high GPAs, presumably through hard work. However, we found almost no cases of SAT-M scorers below about 90th percentile who obtained high upper division GPAs in physics or pure mathematics (second link below).

Related posts:

Data mining the university , Psychometric thresholds for physics and mathematics

I was invited to participate in an August 26 WGBH radio show on Affirmative Action, but couldn't because it coincided with my travel to Taiwan. (You have to search around a bit at the link for the audio; AA discussion starts about 12 minutes in.) Really a pity because the participants got caught up on the issue of whether SAT is a decent measure of academic ability, with Lani Guinier and Oiyan Poon both asserting the (incorrect) claim that it is not, based on low correlation between SAT and freshman GPA at schools with a significant restriction of range.

Affirmative Action In Education: We’ll discuss the merits and pitfalls of affirmative action, its broader implications in the admissions process, and the untold story of those not just being excluded – but indirectly penalized in the process. We’re joined by Dan Golden, author of The Price of Admission; Harvey Mansfield, a political philosophy and government professor at Harvard University; Oiyan A. Poon, a research associate at UMass Boston’s Institute for Asian American Studies; and Harvard Law School Professor Lani Guinier.

Here's my email to Kara Miller, who hosted the radio segment.


I'm really sorry I missed participating in the program, because I've recently done some research on exactly the question that occupied a central place in the heated discussion that ended your show.

It turns out Mansfield is correct -- SATs and ACTs do predict college performance. In fact, in fields like physics and mathematics there appears to be a hard threshold -- students below 90th percentile or so on standardized tests of math ability almost never do well enough to be admitted to a PhD program, no matter how hard they work. Also, probability of success in mastering the undergraduate curriculum in these fields rises rapidly with SAT math score. Mansfield is right to distrust the studies on this topic by scholars in education or social science. My own analysis of 10 years of detailed U Oregon records yields very different conclusions than the ones cited by Guinier and Poon (see below). I can explain why their conclusions are erroneous, but not without discussing statistical issues that require a high SAT math score to understand ;-)


Thursday, September 02, 2010

Bertrand Russell

At the bookstore yesterday I came across the autobiography of Bertrand Russell, which I became engrossed in for some time. Below is the prologue, written when Russell was 84.


Three passions, simple but overwhelmingly strong, have governed my life: the longing for love, the search for knowledge, and unbearable pity for the suffering of mankind. These passions, like great winds, have blown me hither and thither, in a wayward course, over a deep ocean of anguish, reaching to the very verge of despair.

I have sought love, first, because it brings ecstasy -- ecstasy so great that I would often have sacrificed all the rest of life for a few hours of this joy. I have sought it, next, because it relieves loneliness -- that terrible loneliness in which one shivering consciousness looks over the rim of the world into the cold unfathomable lifeless abyss. I have sought it, finally, because in the union of love I have seen, in a mystic miniature, the prefiguring vision of the heaven that saints and poets have imagined. This is what I sought, and though it might seem too good for human life, this is what -- at last -- I have found.

With equal passion I have sought knowledge. I have wished to understand the hearts of men. I have wished to know why the stars shine. And I have tried to apprehend the Pythagorean power by which number holds sway above the flux. A little of this, but not much, I have achieved.

Love and knowledge, so far as they were possible, led upward toward the heavens. But always pity brought me back to earth. Echoes of cries of pain reverberate in my heart. Children in famine, victims tortured by oppressors, helpless old people a hated burden to their sons, and the whole world of loneliness, poverty, and pain make a mockery of what human life should be. I long to alleviate the evil, but I cannot, and I too suffer.

This has been my life. I have found it worth living, and would gladly live it again if the chance were offered me.

I found Russell's comments on Keynes quite interesting.

Keynes's intellect was the sharpest and clearest that I have ever known. When I argued with him, I felt that I took my own life in my hands, and I seldom emerged without feeling something of a fool. I was sometimes inclined to think that so much cleverness must be incompatible with depth, but I do not think this feeling was justified.

The bookstore had quite a nice section of Russell books. He wrote what could be classified as a "self help" book called The Conquest of Happiness, which anticipated a lot of recent work in positive psychology. See here for a nice overview.

... For those who find that even “the exercise of choice is in itself tiresome,” (147) Russell has a remedy that anticipates the smart unconscious. “I have found, for example, that if I have to write upon some rather difficult topic the best plan is to think about it with very great intensity—the greatest intensity of which I am capable—for a few hours or days, and at the end of that time to give orders, so to speak, that the work is to proceed underground. After some months I return consciously to the topic and find that the work has been done.” (49-50)

Perhaps the greatest obstacle to happiness is “the disease of self-absorption.” (173) Russell offers that his own conquest of happiness was due “very largely [. . . ] to a diminishing preoccupation with myself.” (6) A happy person knows that “one’s ego is no very large part of the world.” (48) ...

To the self-absorbed person, other people primarily serve as objects of comparison. “What people fear [. . .] is not that they will fail to get their breakfast next morning, but that they will fail to outshine their neighbors.” (27) Russell warns that “the habit of thinking in terms of comparisons is a fatal one.” (57) To overcome it, “teach yourself that life would still be worth living even if you were not, as of course you are, immeasurably superior to all your friends in virtue and intelligence.” (173) “You can get away from envy by enjoying the pleasures that come your way, by doing the work that you have to do, and by avoiding comparisons with those whom you imagine, perhaps quite falsely, to be more fortunate than yourself.” (58-59)

Likewise, Russell advises not to worry too much about what others think of you. On the one hand, he suspects that “if we were all given by magic the power to read each other’s thoughts, I suppose the first effect would be that almost all friendships would be dissolved.” (76) On the other hand, he doubts that “most people give enough thought to you to have any special desire to persecute you.” (79) This is a nice example of regression to the mean: Chances are you overestimate the love of your friends and the disdain of your foes.

Blog Archive


Web Statistics