Reflexive bubbles

Paul McCulley of PIMCO discusses the feedback mechanism at work in the housing bubble (and seems to call the top). I recommend the whole article (whimsical as it is), which also touches on Bretton Woods II, Asian and OPEC mercantilism, etc. He also makes an interesting observation that bond markets now assume that the Fed has tamed and will continue to tame inflation, and long term rates reflect projections of the real interest rates necessary to do so.

If Fed-engineered changes in nominal rates no longer reflect changes in long-term inflationary expectations, but rather changes in real rates, then logic suggests that long-dated income-producing assets – bonds, stocks and real estate – will become inherently more prone to bubble and burst.

Accordingly, it seems to me, asset prices inevitably must take on a higher priority in the Fed’s reaction function than during the War Against Inflation. As long as inflation was too high for the Fed’s secular taste, bubbles and their bursting didn’t carry grave harm. They misallocated resources, to be sure, like all good Austrians properly preach.

...when home price appreciation is running higher than mortgage rates, the market booms, if not bubbles, as momentum players chase the market higher, a text book example of what George Soros calls reflexive demand. But once the momentum breaks – and again, Morgan, declining affordability is the fundamental break – reflexive demand becomes reflexive supply, as former speculative buyers become eager sellers.

Reflexive markets – and property is one if there ever was one – inherently tend to have V-shaped tops, not rolling tops. Thus, both volumes in total home sales, particularly existing home sales, and MEW are set to fall sharply in the year head. Not the stuff of recession, I hasten to add, Morgan, but clearly the stuff of a serious slowdown in consumer spending.

MLF: Won’t that be a problem for foreign members of the BW II arrangement? Didn’t you say that they are as addicted to our spending as we are to their financing of our spending?

PM: Yes, Morgan, I did say that. Which implies that when the American property market comes off the boil, maybe turning tepid, the world will feel the impact, not just American homeowners. Such is the nature of globalization, when the ex-USA world has a shortage of aggregate demand or, put differently, runs a surplus of savings relative to desired domestic investment.

Orwellian Internet spying

Some good blog posts on this subject: Bruce Schneier (security expert and author of Applied Cryptography) and some legal analysis by Dan Solove. Quick summary: probably in violation of existing FISA laws, probably not explicitly unconstitutional under the 4th Amendment, but Bush's justification under Article 2 is weak, bordering on "frivolous" according to one legal scholar.

John Yoo's memo (the wishful thinking of a thirty-something junior DOJ official, who nevertheless was the strongest intellectual voice on this subject in the administration?!#?) comes close to asserting that there are no limits on Presidential power in the unending "war" on terror. How will we decide when this "war" is over? Was there ever even a declaration of war? Several scholars assert that the 9/14/01 Congressional authorization for use of force falls far, far short of a declaration of war. It is miles away from the declarations of war in WWI and WWII (and the threat is much less serious, as I pointed out in the previous post). If one defines a dictator as a ruler who is beyond the law, then Yoo's memo seeks to justify dictatorial power for Bush-Cheney.


Nineteen Eighty-Four

“Remember our boys on the Malabar front! And the sailors in the Floating Fortresses! Just think what they have to put up with.”

“The rocket bombs which fell daily on London were probably fired by the government of Oceania itself, 'just to keep the people frightened'.”

“To tell deliberate lies while genuinely believing in them, to forget any fact that has become inconvenient, and then when it becomes necessary again, to draw it back from oblivion for just so long as it is needed.”

WAR IS PEACE

FREEDOM IS SLAVERY

IGNORANCE IS STRENGTH

I told you so...

Confirmation of massive, illegal violation of privacy rights of US citizens through telco and Internet monitoring, authorized by the "war preznit". The legal position of this administration (I am not kidding) seems to be that the "war preznit" is free to disregard existing US law in "times of war" like these. If Congress pushes the issue, as it should, we may end up with a full-blown constitutional crisis.

Bush is no more a war president than Ronald Reagan or Jimmy Carter, who faced down a much more formidable foreign adversary. You might argue that Al Qaeda is more dangerous than the USSR and eastern bloc, with their hundreds of ICBMs and thousands of nuclear warheads, but you'd be crazy. Let me offer the following analogy. While walking home you are confronted by a man with a loaded shotgun. By staring him down and pointing out that you yourself are armed, you avoid having your head blown off. Continuing on your way home, a small dog bites your ankle. Is the dog really a greater threat, just because it bit you, than the guy with the shotgun? If not, why should we allow Bush to unilaterally claim greater security powers than Reagan or Carter had? (Indeed, contravening the existing FISA law of 1978.)

In an op-ed article published Friday in The Washington Post, Mr. Daschle said he rejected a White House effort three days after the attacks to grant Mr. Bush specific authority to conduct antiterrorism operations within the United States as part of a broader resolution backing the use of force.

In seeking the specific authority for a domestic response, Mr. Daschle said, the White House was effectively acknowledging that the resolution did not cover domestic actions like spying on Americans.

"The Bush administration now argues those powers were inherently contained in the resolution adopted by Congress - but at the time, the administration clearly felt they weren't or it wouldn't have tried to insert the additional language," Mr. Daschle said in the article.

The White House has asserted that the resolution, adopted by Congress on Sept. 14, 2001, freed Mr. Bush from the requirement to get warrants to monitor international phone calls and e-mail of Americans and others in the United States. That resolution authorized the president to employ "all necessary and appropriate force" in response to the attacks on New York and the Pentagon.

But by Mr. Daschle's new account, which appears not to have been made public previously, the White House sought within minutes before the vote on the resolution to alter it to include new wording specifically granting power to carry out the antiterrorism campaign within the United States.

The White House, Mr. Daschle said, wanted the resolution to give Mr. Bush authority to use "all necessary and appropriate force in the United States and against those nations, organizations and persons" responsible for the attacks.

Mr. Daschle said he had turned aside the White House's effort to include "in the United States and" in that sentence, leaving the focus of the resolution on fighting terrorism abroad.

China tech and innovation

Nice discussion (Talk of the Nation; sorry no podcast) with John Seely Brown (former director of Xerox PARC) on innovation and technology in China. Brown has obviously spent some time with startups over there, particular in the cellphone sector, where he claims they are ahead of the US and Europe.

This is the kind of media I can still consume while grappling with two babies ;-)

Sniffing around

It's very plausible to me that the latest Bush perfidy - secretly approving widespread spying by the NSA on US citizens in the wake of 9/11 - is not about phone wiretapping at all. It's about large scale monitoring of Internet communications.

The press has pointed out repeatedly that the 1978 FISA law allows the government to wiretap on short notice, even to wiretap first and then ask a FISA court for permission retroactively. So, the Bush administration's claim that existing legal requirements slowed things down and "jeopardized national security" is just wrong, and informed people know this.

I suspect what is really going on is that in the wake of 9/11 Bush authorized large scale monitoring of Internet communications, probably by allowing NSA to tap into the backbone. There was a lot of discussion of similar programs, such as Poindexter's Total Information Awareness (TIA) system. Technically, it would not be hard to sample Internet traffic, looking for email or Web activity with certain key words or patterns. However, 100% coverage is probably beyond anyone's capability at the moment.

The problem with this is that you only catch dumb terrorists (maybe that's good enough). As I pointed out before, even widely available communication tools like Skype allow for unbreakable encrypted communication. Don't all terrorists, even ones who don't understand how the Internet works, simply assume that it is being monitored (at least in some weak way)? If so, how can Bush claim that whistleblowers jeopardized national security by leaking information about this illegal program?

BTW, good thing Rockefeller (Senate Intel. Cmmte.) kept a copy of the letter he sent to Cheney. The Bushies claim (lying again) that they got congressional approval, conveniently leaving out that Rockefeller protested immediately about the legality of the program (as did Daschle, who claims the briefings may also have been technically misleading).

Note Added: Administration officials are very careful to state that only "communications" between the US and foreign countries are being monitored. I suppose this means that if your packets don't leave the US, they aren't sniffed. However, all Blackberry users should be aware that their email likely travels through servers in Canada, so is potentially subject to monitoring :-) This Times article seems to confirm that email is intercepted.

TalkLeft: Why do Gonzales and Condi Rice keep mentioning the "technical" aspects of the program as a dodge around FISA?

Why this seemingly inconsequential parsing by Bush of the difference between "monitoring and detection"? Bush says they use FISA if they're monitoring, but this is about "detection."

Why, in his letter, does Rockefeller state that he's "not a technician."?
Why the mention of TIA in Rockefeller's letter?
And why the mention of "large batches of numbers all at once"?

Why?

These are not phone numbers we're talking about...These are IP addresses, email addresses.

A system is in place that basically filters on certain triggers (text, phoneme, etc.) within Internet "conversations." This is "detection" or at least its tortured definition that was placed in this idiot Bush's mind. "Monitoring" would be recording an entire conversation, like in a phone conversation.

That system then collects information on those conversations including...ta da...source and destination IP addresses. Those IP addresses can then be stored for further investigation on other "conversations."

The next wave in India outsourcing

Nice summary from the economist. The point that legal services, like accounting, can be outsourced rings particularly true to me. Most legal services (from IP to contracts to employment law) that we consume as a startup are delivered over the phone or Internet, at exorbitant prices ($300 per hour is what you are charged for an associate; a partner bills even more). The effectiveness of the attorney is largely driven by raw brainpower, once some basic knowledge is acquired. I can easily see a model with a US-based partner and half or more of his or her staff (including associates) based in India. $30 per hour would be very good compensation for a highly intelligent Indian attorney.

Should we be concerned for the future of our young friends in law school? See here for related discussion.

ASKED for a sound-check at a function in Delhi this month, Bill Gates eschewed the “1,2,3...” favoured by ordinary mortals. “One billion, 2 billion...,” he counted. They think big, these IT moguls, and especially, these days, in India.

Microsoft later announced plans to invest $1.7 billion in India over the next four years, about half of it in adding to its existing research and development (R&D) and technical-support operations. “The only thing that limits us in India,” Mr Gates told the local press, “is the speed at which we can recruit.” A few days earlier, Intel, a giant chipmaker, had unveiled plans to invest more than $1 billion over five years, much of it in expanding its R&D centre in Bangalore. In October Cisco Systems, the world's largest maker of the routers and switches that direct internet traffic, announced its own plans to invest $1.1 billion in India.

The euphoria is confined neither to American multinationals, nor to information technology. It also encompasses India's own IT industries, and the expanding range of other back-office services that can now be performed remotely. The three biggest Indian IT-services firms—Tata Consultancy Services (TCS), Infosys and Wipro—are each recruiting more than 1,000 people a month. And to take just one example of the other services now moving to India, J.P. Morgan Chase, a big investment bank, this month revealed it is to double, to about 9,000, its staff there. Anyone who assumes J.P. Morgan will simply be doing low-level “back office” tasks in the country—a bit of data entry and paper-shuffling—would be flat wrong. One task for the new recruits is to settle complex structured-finance and derivative deals, what one insider calls “some of the most sophisticated transactions in the world”.

All these investments illustrate that a third stage of the great Indian services-export boom is well underway. In the first, firms such as TCS developed world-class expertise in software “application development and maintenance”, and their low-cost developers became the preferred partners of many Western IT firms. In the second, Indian firms and the local “captive” operations of multinationals started offering low-end back-office services that could take place a continent away—telephone call-centres, transcribing medical records, processing insurance claims and so on. In the third, in both IT and the broader spectrum of other “business processes”, ever-more sophisticated functions are happening in India.

So strong are the forces driving this shift that what seemed improbably rosy projections by NASSCOM, the Indian software- and service-industry lobby, and McKinsey, a consultancy, back in 1999, are coming true. This week NASSCOM and McKinsey produced the second full-scale update of their study. It argues that exports from India's IT industry and from “Business Process Offshoring” (BPO)—both from services “outsourced” to Indian firms and those performed by captives—are on track to reach $60 billion a year by 2010.

That would be a huge surge from the $17.2 billion in the year ending in March 2005. But it implies a compounded annual growth rate of 28%—below that achieved in recent years. Moreover, according to McKinsey's estimates, it requires India merely to maintain its present shares of the markets for offshore IT services (65%) and BPO (46%). This is because the study predicts a massive rise in the size of the overall market, estimated at present to make up just one-tenth of those services that could be sent offshore. The proportion is expected to rise as demography—a western labour shortage—becomes more pressing than protectionism.

In IT the growth in Indian exports is expected to come both from the software market, and from “traditional IT outsourcing”—such as the remote management of whole systems, a market now dominated by the big global IT consultancies. This is expected to rise from 8% of Indian sales now to about 30% in 2010, while software-development's share will fall from 55% to 39%. In business-process-offshoring, the big industries will remain banking and insurance. But rapid expansion is also expected in other areas, like legal services.

The law, in fact, illustrates how vast is the untapped potential market. About $250 billion is spent on legal services world-wide, about two-thirds of it in America, and as yet only a tiny proportion goes offshore. Forrester, a research outfit, has estimated that, by last year, 12,000 legal jobs had moved offshore, and forecast that this will increase to 35,000 by 2010. India, with its English-language skills and common-law tradition is well-placed to secure a big share of the business. It is not just a question of “paralegal” hack work such as document-preparation. Sanjay Kamlani, of Pangea3, a small Indian firm, calls it “real lawyering”—drafting contracts and patent applications, research and negotiation. His clients are both big law firms and in-house legal teams.

India's fundamental attraction has not changed since it first drew software developers: fantastic cost savings. With American lawyers costing $300 an hour or more, Indian firms can cut bills by 75%. Across the board, despite climbing rates of pay in IT and BPO, where rapid expansion has brought frantic job-hopping, India remains, say NASSCOM and McKinsey, the lowest-cost of all the main outsourcing destinations. It also has, among these countries, by far, the largest pool of employable people—those with the necessary language and technical skills. On this measure, India, which produces 2.5m graduates a year, 250,000 of whom are engineers, has 28% of the global available workforce, compared with 11% in China.

Yet the supply of talent may be the biggest constraint on the Indian industry's growth. On these latest projections, the number of people working in IT and business-process exports in India will increase from about 700,000 now to 2.3m by 2010. But on today's estimates only 1.05m suitably qualified people will graduate from college between now and then, so there will be a shortfall of nearly 500,000, with business-processing the worst affected. McKinsey's Jayant Sinha believes the education system can be fixed in time to plug the gap. A bigger worry, he says, is India's creaking urban infrastructure. IT firms in Bangalore, for example, are in revolt against the local government for its neglect of basic amenities. Yet India's IT and business-process industries will need about 14m square metres (150m square feet) of office space by 2010: “a new Manhattan”.

Hectic building is under way, and not just in the big IT and business-processing centres (Bangalore, Mumbai and around Delhi) or the “second tier” of cities such as Pune, Hyderabad and Chennai (Madras). The industry's worry over infrastructure, as over education, is that it cannot do everything by itself. Having thrived by keeping government at arm's length, business now needs help.

Workplace discrimination

From a recent Gallup survey, as reported by the EEOC.

The Gallup data indicate that 15% of all workers perceived that they had been subjected to some sort of discriminatory or unfair treatment. When broken down into sub-groups, 31% of Asians surveyed reported incidents of discrimination, the largest percentage of any ethnic group, with African Americans constituting the second largest group at 26%.

This finding is probably only surprising to white Americans. Asian-Americans know very well that they are subject to stereotyping (albeit perhaps of the unconscious type) and often passed over for leadership roles. Thanks to Confucian values, they are also less likely to complain about it. You won't hear them complaining about being shortchanged by affirmative action, either.

Within the context of race filings, 82.5% of charges were brought by African Americans, with Asian/Pacific Islanders filing only 3% -- a sharp contrast with the 30% of Asians employees who responding to the Gallup survey that they perceived discrimination on-the-job.

Wall Street

Wall Street (1987). Directed by Oliver Stone, starring Michael Douglas (Gekko), Martin and Charlie Sheen (father and son Carl and Bud Fox). A classic that fully anticipated our age of hyper-finance. Gekko and company look tame compared to LTCM, Enron and our current hedge fund overlords!

Gekko: Greed, for lack of a better word, is good. Greed is right. Greed works. Greed clarifies and cuts through and captures the essence of evolutionary spirit. Greed in all of its forms, greed for life, for money, for love, knowledge has marked the upward surge of mankind. And greed, you mark my words, will not only save Teldar Paper, but that other malfunctioning corporation called the USA.

Gekko: The richest one percent of this country owns half our country's wealth, five trillion dollars. One third of that comes from hard work, two thirds comes from inheritance, interest on interest accumulating to widows and idiot sons and what I do, stock and real estate speculation. It's bullshit. You got ninety percent of the American public out there with little or no net worth. I create nothing. I own.

Carl Fox: Stop going for the easy buck and start producing something with your life. Create, instead of living off the buying and selling of others.

Personally, I followed the Create path, but I'm not sure I came out ahead :-)

The view from the PBOC

From an interview with Yu Yongding, an Oxford-trained economist who sits on the Monetary Policy Committee of the People's Bank of China. Who can doubt that everyone sees the handwriting on the wall? He clearly understands the inevitability of dollar devaluation, the need for Asian countries to dump those dollars, the prisoner's dilemma problem that central banks of China, Taiwan, Japan and Korea have, etc. Barring a disaster in Asia (China invades Taiwan, N-S Korea conflict, etc.), I predict a 2x devaluation of the dollar against E. Asian currencies over the next decade. Sorry that prediction is not precise enough to trade on :-)

(See here for previous posts on Bretton Woods II and dollar devaluation.)

"...in the first stage we must reduce accumulation, then later we should reduce our reserves....[China and Asian countries] don't need that large an amount- more than $2 trillion- of foreign exchange reserves.... This is a very big problem and I think the Chinese government should take some action to reduce the growth rate of the accumulation of foreign exchange reserves as we're still facing the possibility of a big devaluation of the US dollar, so the capital losses will be huge. If that happens, it will be tremendous hit to the Chinese economy."

This is hardly the statement of a gentleman with a benign view toward the US dollar's valuation. It is instead a gentleman, in a position of authority, with a great deal of concern. He went on: "The trouble is, with such a huge amount of foreign exchange reserves, that there is no way to spend it very quickly and there's no plan to sell it of course-- otherwise that inflicts damage on ourselves. You don't want to dump shares when the stock market has not collapsed yet and you are the biggest shareholder." Then, he said "all east Asian countries have tremendous foreign exchange reserves and they all want to get rid of them, but if you do this then you cause competitive devaluation, not of their own currencies, but of the US dollar. So we should do this in an orderly fashion. If Asian countries moved too fast, everyone would lose... It would be utterly unfortunate if Japan sells a proportion [of their reserves, for] that causes problems. Then China panics and China sells a proportion -- it would be very damaging."

The "nicest possibility" for China, Japan and the US to escape this problem was for further "tightening of US monetary policy so that further dramatic devaluation of the US dollar can be stopped. Then, because of the slowdown in the economy, the US current account deficit would reduce and in this way will create conditions for East Asian countries to get off the hook."

2 percent of the Caltech class

A friend of mine who runs a derivatives desk made job offers to 2 percent of the Caltech class of 2006. That's only 4 people, but still a pretty high percentage! 3 have accepted, the other went to a well-known software company. It makes me wonder how many kids who otherwise would have gone on to grad school (e.g., in engineering, math or physics) are heading right into finance these days. Smart move, if you ask me :-)

Most of my colleagues still don't take the theory of modern finance seriously at all. Even researchers who work on complex systems (modeling traffic, sandpiles, networks, ants, etc.) show surprisingly little interest. I guess the future belongs to the young!

Physics still pulls them in...

There is much concern among physicists that our field is no longer getting the same share of the brightest students as in the glory days (e.g., during the cold war). I was pleased to find here that average GRE-general scores in physics PhD programs were still the highest as of 2002, with math and CS close behind. According to these tables, the average IQ of a physics PhD student is about 130. For comparison, english/literature PhD students average about 120 and sociology PhD students only 115. (115 is the average for college students in general, so the typical sociology TA is no smarter than the students in his or her class ;-) Amazingly, the average grad student in education (109) is below the average for college students!

You may find it odd that I converted the results from GRE score to IQ, widely regarded by the politically correct as a discredited measure of intelligence. For a rebuttal of this often repeated, but scientifically unsupported, point of view, see, e.g., Steve Pinker's book The Blank Slate, or any academic research of the type excerpted below.

g on the Job


Although the evidence of genetic and physiological correlates of g argues powerfully for the existence of global intelligence, it has not quelled the critics of intelligence testing. These skeptics argue that even if such a global entity exists, it has no intrinsic functional value and becomes important only to the extent that people treat it as such: for example, by using IQ scores to sort, label and assign students and employees. Such concerns over the proper use of mental tests have prompted a great deal of research in recent decades. This research shows that although IQ tests can indeed be misused, they measure a capability that does in fact affect many kinds of performance and many life outcomes, independent of the tests' interpretations or applications. Moreover, the research shows that intelligence tests measure the capability equally well for all native-born English-speaking groups in the U.S.

If we consider that intelligence manifests itself in everyday life as the ability to deal with complexity, then it is easy to see why it has great functional or practical importance. Children, for example, are regularly exposed to complex tasks once they begin school. Schooling requires above all that students learn, solve problems and think abstractly. That IQ is quite a good predictor of differences in educational achievement is therefore not surprising. When scores on both IQ and standardized achievement tests in different subjects are averaged over several years, the two averages correlate as highly as different IQ tests from the same individual do. High-ability students also master material at many times the rate of their low-ability peers. Many investigations have helped quantify this discrepancy. For example, a 1969 study done for the U.S. Army by the Human Resources Research Office found that enlistees in the bottom fifth of the ability distribution required two to six times as many teaching trials and prompts as did their higher-ability peers to attain minimal proficiency in rifle assembly, monitoring signals, combat plotting and other basic military tasks. Similarly, in school settings the ratio of learning rates between "fast" and "slow" students is typically five to one.

...The existence of biological correlates of intelligence does not necessarily mean that intelligence is dictated by genes. Decades of genetics research have shown, however, that people are born with different hereditary potentials for intelligence and that these genetic endowments are responsible for much of the variation in mental ability among individuals. Last spring an international team of scientists headed by Robert Plomin of the Institute of Psychiatry in London announced the discovery of the first gene linked to intelligence. Of course, genes have their effects only in interaction with environments, partly by enhancing an individual's exposure or sensitivity to formative experiences. Differences in general intelligence, whether measured as IQ or, more accurately, as g are both genetic and environmental in origin--just as are all other psychological traits and attitudes studied so far, including personality, vocational interests and societal attitudes. This is old news among the experts. The experts have, however, been startled by more recent discoveries.

One is that the heritability of IQ rises with age--that is to say, the extent to which genetics accounts for differences in IQ among individuals increases as people get older. Studies comparing identical and fraternal twins, published in the past decade by a group led by Thomas J. Bouchard, Jr., of the University of Minnesota and other scholars, show that about 40 percent of IQ differences among preschoolers stems from genetic differences but that heritability rises to 60 percent by adolescence and to 80 percent by late adulthood. With age, differences among individuals in their developed intelligence come to mirror more closely their genetic differences. It appears that the effects of environment on intelligence fade rather than grow with time. In hindsight, perhaps this should have come as no surprise. Young children have the circumstances of their lives imposed on them by parents, schools and other agents of society, but as people get older they become more independent and tend to seek out the life niches that are most congenial to their genetic proclivities.

A second big surprise for intelligence experts was the discovery that environments shared by siblings have little to do with IQ. Many people still mistakenly believe that social, psychological and economic differences among families create lasting and marked differences in IQ. Behavioral geneticists refer to such environmental effects as "shared" because they are common to siblings who grow up together. Research has shown that although shared environments do have a modest influence on IQ in childhood, their effects dissipate by adolescence. The IQs of adopted children, for example, lose all resemblance to those of their adoptive family members and become more like the IQs of the biological parents they have never known. Such findings suggest that siblings either do not share influential aspects of the rearing environment or do not experience them in the same way. Much behavioral genetics research currently focuses on the still mysterious processes by which environments make members of a household less alike.

Scientific method at Google

Great podcast of an interview with Google busdev VP Omid Kordestani. Kordestani is credited with coming up with the AdSense business model (although it originally came from Overture, and another person gets the credit in a recent NYT article discussed here). There was also a recent WSJ article in which Kordestani is described as one of the Googlers driving up real estate prices in Atherton -- in his case by buying a 16,000 sq ft house for $17.8 million!

What is interesting about the interview is Kordestani's continued references to the scientific methods applied within Google to tasks as varied as screening job applicants, rooting out click fraud, monitoring (and modeling) AdSense auctions, etc. He uses the phrase "teams of PhDs" several times :-) It is amazing to hear a busdev guy talk this way, even considering that Kordestani has an engineering background.

Expert predictions

How good are "experts" at making accurate predictions? Much worse than you think, says psychology professor Philip Tetlock (Haas School of Business at UC Berkeley) in his new book Expert Political Judgment: How Good Is It? How Can We Know? (See New Yorker review.) In detailed studies, in which "experts" were asked to make forecasts about the future (predicting which of three possible futures would occur), it was found that the "experts" did no better than well-informed non-experts! As Tetlock says, “We reach the point of diminishing marginal predictive returns for knowledge disconcertingly quickly,” he reports. “In this age of academic hyperspecialization, there is no reason for supposing that contributors to top journals—distinguished political scientists, area study specialists, economists, and so on—are any better than journalists or attentive readers of the New York Times in ‘reading’ emerging situations.”

Now, I expect the performance of scientific experts to be somewhat better. Questions like "How hot will that spacecraft get while in orbit around Mercury?" or "How many CPU cycles will it take to compute that integral?" are ones where predictions of real experts will far outperform those of lay people. I guess there is something fundamentally different about scientific versus non-scientific expertise? The last bit below about predicting freshman academic performance is amazing (but not unexpected). Let a simple one or two parameter model pick your freshman class :-)

Finally, what type of "expert" would you trust to run your money (make investment predictions)? As Jim Simons said: "The advantage scientists bring into the game is less their mathematical or computational skills than their ability to think scientifically. They are less likely to accept an apparent winning strategy that might be a mere statistical fluke." In other words, they know when they know something, while others might just be fooling themselves ;-)

New Yorker: Tetlock is a psychologist—he teaches at Berkeley—and his conclusions are based on a long-term study that he began twenty years ago. He picked two hundred and eighty-four people who made their living “commenting or offering advice on political and economic trends,” and he started asking them to assess the probability that various things would or would not come to pass, both in the areas of the world in which they specialized and in areas about which they were not expert. Would there be a nonviolent end to apartheid in South Africa? Would Gorbachev be ousted in a coup? Would the United States go to war in the Persian Gulf? Would Canada disintegrate? (Many experts believed that it would, on the ground that Quebec would succeed in seceding.) And so on. By the end of the study, in 2003, the experts had made 82,361 forecasts.

...Tetlock got a statistical handle on his task by putting most of the forecasting questions into a “three possible futures” form. The respondents were asked to rate the probability of three alternative outcomes: the persistence of the status quo, more of something (political freedom, economic growth), or less of something (repression, recession). And he measured his experts on two dimensions: how good they were at guessing probabilities (did all the things they said had an x per cent chance of happening happen x per cent of the time?), and how accurate they were at predicting specific outcomes. The results were unimpressive. On the first scale, the experts performed worse than they would have if they had simply assigned an equal probability to all three outcomes—if they had given each possible future a thirty-three-per-cent chance of occurring. Human beings who spend their lives studying the state of the world, in other words, are poorer forecasters than dart-throwing monkeys, who would have distributed their picks evenly over the three choices.

Tetlock also found that specialists are not significantly more reliable than non-specialists in guessing what is going to happen in the region they study.

...“Expert Political Judgment” is just one of more than a hundred studies that have pitted experts against statistical or actuarial formulas, and in almost all of those studies the people either do no better than the formulas or do worse. In one study, college counsellors were given information about a group of high-school students and asked to predict their freshman grades in college. The counsellors had access to test scores, grades, the results of personality and vocational tests, and personal statements from the students, whom they were also permitted to interview. Predictions that were produced by a formula using just test scores and grades were more accurate.

Postdoc (no)bargain

This was posted as a comment, but I thought it so good I should share it here.

Note if there is almost one postdoc per permanent researcher (or professor), it means that one 3-5 year cohort of PhDs could replace the entire pool of career researchers. If a career lasts 30 years it means there is a six or ten to one ratio between postdocs that leave the field and those who survive with permanent positions. Even if the postdoc to professor ratio is lower (as in physical science and math -- this number is smaller largely because they don't have much funding for postdocs in math), there may still be a big ratio between PhDs produced and number of professors. Unless the average professor produces no more than one PhD per 30 years, there will continue to be a big oversupply and a tough labor market for scientists.

See related posts here and here.

Thanks for the Great Postdoc Bargain
Richard Freeman
30 August 2002

Postdocs are at the heart of the United States?s extraordinarily successful biological and life scientific research program over the past two-plus decades. In this period, postdocs have produced most of the results in academic laboratories and have come to play an increasing role in industrial and government labs as well. Academic institutions, which engage some 80% of postdocs, aren?t sure whether postdocs are employees, students, or some form of apprentice. With responsibility for hiring and career development resting firmly with the principle investigators who employ the postdocs on their research grants, many universities don?t even know how many postdocs they have or what they are paid, much less how they are progressing toward ? whatever the future holds for them.

Whatever they are, however, postdocs are one of the greatest bargains in the U.S. economy. Where else can one hire Ph.D.s, whose training and smarts put them among the best and brightest in the world, to work 60 hours a week for $30,000 to 40,000 a year, with limited benefits and little power to influence their working conditions and pay? Given the long hours that postdocs work, their hourly pay is on the order of $10 to $13 per hour--on par with the wages paid to custodial and other low-paid workers that have spurred living wage campaigns around the country.


...Huzzah! Huzzah!

Two to three decades ago, the U.S. rewarded postdocs with a reasonably good chance of being hired as a principal investigator. Sorry, but we can no longer carry out that part of the bargain. As Table 1 shows, there are just too many postdocs for us to absorb them as tenured faculty. In 1987, the ratio of postdocs to tenured faculty was already too high at 0.54 for most to obtain faculty jobs at the rate of growth of academic employment. By 1997, the ratio had risen by 43% to 0.77. It has presumably risen further since then. As a result--and as many postdocs have learned to their chagrin--the U.S. does not have a place for them on standard academic tracks.

But don?t get discouraged, postdocs. We need you for our research. How about another postdoctorate--a few more years of long hours at low wages?

Table 1: Ratio of the Number of Postdoctorates in Higher Educational Institutions to the Number of Tenured Faculty, 1987 and 1997

Discipline
1987
1997
% Change

Life Sciences
0.54
0.77
43%

Physical Sciences and Mathematics
0.20
0.23
15%

Engineering
0.11
0.19
73%

Source: The National Academies Committee on Science, Engineering, and Public Policy Enhancing the Postdoctoral Experience for Scientists and Engineers, (NAS, 2000), table B-1 and table B-14.


The forces of supply and demand are unlikely to improve the economic situation of postdocs in any plausible time period. One reason is that the supply of postdocs consists not only of U.S. citizens and permanent residents gaining Ph.D.s but also of U.S.- and foreign-trained Ph.D.s from other countries. Indeed, U.S. scientific research could not proceed at anything like its current pace was it not for the influx of foreign postdocs. Roughly half of postdocs currently come from overseas, many from countries with low personal income rates such as China. Remove foreign postdocs from the nation?s labs and postdoc pay would zoom ... at the cost of short-term chaos and a long-term slower rate of scientific progress. Nevertheless, we should not forget to thank our foreign postdocs for their long hours and hard work on behalf of the rest of our society...
--
Posted by scienceguy11 to Information Processing at 12/05/2005 11:37:13 AM

Is gravity ergodic?

New paper! (Available on arxiv.org.)

The universe is very far from maximizing its entropy. Most egregious is the gravitational state of the universe, as noted years ago by Penrose. Gravity, being a long range, unscreened, attractive force, maximizes its entropy by clumping. This is quite unlike systems with ordinary interactions (generally of limited range, or subject to screening), whose entropy is maximized when matter and energy are uniformly distributed. In the case of gravity the most entropically dense configurations are black holes, with exp(A) microstates (A is the area in Planck units). Why did the big bang begin with a smooth background metric, instead of an agglomeration of black holes, when the latter has much higher entropy and hence occupies an exponentially larger portion of phase space? We explore how a phase transition between the smooth, low entropy spacetimes and the high entropy black hole phase might occur.

Note added: After we posted the paper we discovered calculations using Euclidean path integrals (gravitational instantons) which yield the same results for the black hole free energy and nucleation rate as we obtain from our simple intuitive arguments.


Thermal gravity, black holes and cosmological entropy

Authors: Stephen D. H. Hsu, Brian M. Murray

Taking seriously the interpretation of black hole entropy as the logarithm of the number of microstates, we argue that thermal gravitons may undergo a phase transition to a kind of black hole condensate. The phase transition proceeds via nucleation of black holes at a rate governed by a saddlepoint configuration whose free energy is of order the inverse temperature in Planck units. Whether the universe remains in a low entropy state as opposed to the high entropy black hole condensate depends sensitively on its thermal history. Our results may clarify an old observation of Penrose regarding the very low entropy state of the universe.

Vernor Vinge talk

Nice podcast of a talk given at Accelerating Change 2005. Vinge is a mathematician and sci fi writer who popularized the term Singularity (originally due to Ulam?) to describe the rapid acceleration of technological change that might accompany the development of machine intelligence beyond our own.

Previously I discussed the impending singularity, the relation to the Turing test and AI for kids.

Platform companies

Economist Brad Setser has been discussing platform companies as the future for US industry. The idea is that high value-add activities such as design, engineering and management are performed here, and low value-add manufacturing is done abroad. A good example of this is the iPod, which was designed in Cupertino but is manufactured in Taiwan. Apple sold $1.2 billion worth of iPods in Q1 2005, making iPods almost as big as their computer business! Apple's margin on iPods is healthy - estimated at up to 20% -- while I am sure the manufacturer's margin is very slim. So, the conventional wisdom goes, let those low cost countries have the cutthroat manufacturing business. We clever Americans will simply go upstream, creating new products and markets through innovation.

There is a problem with this, and it is related to the winners and losers issue that arises in the usual story of comparative advantage. Overall, societies may enjoy a net benefit from trade (heck, there are even "theorems" going back to Ricardo "proving" this), but even in the optimistic cases the benefits (and losses) will be unevenly distributed. As autoworkers are displaced by Delphi moving production to China, investment bankers or consultants in the US benefit from demand for globalization strategists. But who ensures that some of the benefits are redistributed? Of course we all benefit from lower prices for manufactured goods, but that is only partial recompense to an unemployed auto worker.

I would go even further, and guess that only a small fraction of the population in advanced countries has the cognitive capabilities to be on the winning end of this process. That is, the Apple iPod designers, investment bankers, CEOs, etc. who win at this game are very able -- in terms of conventional IQ or special abilities such as leadership, design creativity, etc. It is questionable whether the average person will ever be able to contribute on a "high value-add" team! Instead, they'll be caught in a process taking place over a generation in which low-skill wage costs equilibrate worldwide. The theorems might still be satisfied by the huge returns to very able people (whether in China, India, Russia or the US), but the trend for the average American might be negative. Instead of an auto worker in Detroit earning enough for a good retirement and college education for their kids, they'll instead make $10-20 per hour, including benefits. (See earlier post Equilibration can hurt.)

Previous discussion of outsourcing here and here.

Yield curve inverts

The 5 yr yield was briefly below the 2 yr yield today. Is the market predicting a recession? I guess historically the Fed always overtightens...

Imagine you want to model interest rate fluctuations. You have to have a decent model for fluctuations at all maturities. Are there any reasonable constraints? How far can the curve invert? How likely is that? What are the self-consistency constraints? How do we incorporate current option prices into the analysis? A well-known practitioner once told me that modeling the yield curve is to modeling equity prices as quantum field theory is to ordinary quantum mechanics!

See here for Derman's explanation of the BDT (Black, Derman, Toy) yield curve model. I discussed a nice bio of Fischer Black here, and Derman's book here.

Ben Stein on globalization

Ben Stein is most likely familiar to you as a comedian in movies and on TV. Earlier in life he attended Yale Law School and worked in the Nixon White House as a speechwriter. Below he gives the positive spin on globalization. However, at the very end he addresses the winners and losers issue, and his description of losers sounds a lot more like the average american than his description of winners!

China: Friend or Foe?
Wednesday, November 23, 2005

A few nights ago, I had the pleasure of speaking to and MC'ing the annual dinner of the Semiconductor Industry Association (SIA). Attending were the major players in the chip world, the Texas Instruments, AMDs, National Semiconductors, and -- well, you get the picture. Very successful, super smart men and women filled the room -- real geniuses who did things like create programs that read and check immense software codes written in India, only do it in real time over the internet. These are people who started and run incredibly complex and productive businesses.

You might have thought it would be a rollicking good time, but it was not at all.

The three main speakers -- George Scalise, head of the SIA; Charlene Barshefsky, formerly Clinton's Special Trade Representative; and Byron Wien, a major power at Morgan Stanley -- were deeply worried about Far Eastern competition in the chip world. It is simple enough: China, Taiwan, Thailand, and India have far lower labor costs than we do. They have up-to-date machinery and a highly trained, well motivated labor force. They can make chips for less, and they are starting to become players in a big way -- not just in chip manufacturing (or "fabbing," as they say) but in design, which was formerly an American fiefdom.

Trees Do Not Grow to the Sky

The three main speakers showed charts and graphs pointing out Asian inroads and the declining U.S. share of production and design. And one speaker expressed doubt that we would see our kids live as well as we do because of Asian competition.

This prompted several thoughts in my wooly head.

First of all, we start out very far ahead of China in wealth and income. The per capita GDP of the U.S., depending on how you measure it, is as much as 40 times Chinese per capita GDP and not less than 12 times by any measurement.

Yes, China is growing much faster than we are, at roughly 10 percent per year. If current trends last, China will overtake the U.S. in per capita GDP sometime this century. But current rates for China are extremely high. No nation has ever been able to sustain the rates China is experiencing for very long. History argues that China will not be able to do so either. Put simply, trees do not grow to the sky.

However, the Chinese are an amazing, intelligent, hard working, well disciplined people. Possibly they will be able to enjoy super high growth rates for a prolonged period. It would be a historical anomaly, but maybe it will happen.

Is What's Good for China Good for the U.S.?

Even if China became much richer than it is now though, that would not necessarily make us poorer. China is a prodigious buyer of foods and resources, and we have a lot of both. Chinese industry will be partly owned by U.S. investors, and Chinese prosperity will (if no fraud is involved) make them much richer. The fact that a very large nation like China becomes rich makes individual American workers who compete with China worse off, but it does not make America as a nation worse off.

Next, is it not interesting that China, in a state of brutal totalitarian rule, was really not a threat to us except in fiction? But under capitalism, China is making us all run around screaming in fear. I am really not sure what we have to fear from China. The country is our partner in prosperity, not our enemy. The earlier we work out a durable framework of respect and cooperation in trade and policy, the better off everyone will be.

But the main thing I want to express, as I did to the SIA, is that China is getting rich because its people are getting well educated, because they are working hard, because they are saving, and because they are investing. These are exactly the same things that made America rich. And these are exactly the same things that are allowing Hispanic and Asian immigrants to America to move up the economic ladder.

Who Wins, Who Loses?

Young Americans who study hard, learn serious subjects, do not lose themselves in computer games, avoid doomed industries, learn good work habits, save prudently, and invest sensibly will be well off no matter what happens in China or Taiwan or India.

Americans who are slothful, do not pay attention to economic trends, learn no useful skills, do not save, and do not invest wisely will roll downhill fast.

But for the disciplined among us who learn from the Chinese the keys to wealth as the Chinese learned from us and we learned from all of history, the future is bright.

China book reviews

From The NY Review of Books. I briefly mentioned both of these books in earlier posts here and here.

Three Billion New Capitalists: The Great Shift of Wealth and Power to the East
by Clyde Prestowitz
Basic Books, 321 pp., $26.95

China, Inc.: How the Rise of the Next Superpower Challenges America and the World
by Ted C. Fishman
Scribner, 342 pp., $26.00

The "rise" of China has suddenly become the all-absorbing topic for those professionally concerned with the future of the planet. Will the twenty-first century be the Chinese century, and, if so, in what sense? Will China's rise be peaceful or violent? And how will this affect the United States, the current "hyperpower"? In fact, China has been "rising" for some time (after several hundred years of "fall"), but for many years its claim to notice was obscured by more exciting events. Attention in the 1990s concentrated on the fall of Soviet communism, "globalization," the spread of democracy, and the high-tech revolution. These developments, which left America as the world's sole economic and political superpower, seemed to belie Paul Kennedy's prediction in 1987 of relative US decline and "more of a multipolar system."[1]

The attack on the World Trade Center in 2001, together with the concurrent collapse of the high-tech bubble, exposed America's fragility, but this was masked by the hyperactivity of the Bush administration. The "war on terror" planted American armies in Afghanistan and Iraq; the Clinton surpluses were succeeded by the Bush deficits to shore up the economy and finance the military operations. However, as the Iraq escapade foundered and the deficits ballooned, the sense of relative decline reasserted itself. Unlike in 1987, there was now a clear candidate for the succession: China. This was especially so as the US economy became dependent on China's bankrolling its huge trade deficit. The dream of an "American century" receded, to be replaced by the nightmare of a "Chinese century."

Focus on China is overdue. For the last quarter of a century its economy has been growing by over 9 percent a year, increasing eightfold. However, it is not just this long-sustained hyper-growth rate that amazes and alarms the observer. It is the size of the economy which is growing. China's population is officially estimated at 1.3 billion, but is probably larger—one fifth of all the people in the world. This makes its rise much more important than that, say, of Japan in the 1960s. From the economic point of view its cheap labor is much more abundant, so its cost advantage will not quickly be eliminated. The size of an economy obviously matters, too, in measuring power. The Chinese economy, in terms of the purchasing power of the Chinese people, is about two thirds the size of the US economy.[2] If it continues to grow at 9 percent a year, it will overtake the US by 2014. Lee Kwan Yu of Singapore believes that the rise of China will shift the balance of power back to the East for the first time since Portuguese caravels arrived there in the sixteenth century.

China's growth, simply because of its size, is bound to create problems both for itself and others. From the Chinese leadership's point of view, the main problem is how to maintain social cohesion amid the vast socio-economic upheavals going on. Apart from the environmental degradation and rampant corruption, China's pell-mell, and largely uncontrolled, economic growth is disturbing its domestic stability in a profound way: there is a huge floating population without settled jobs or abodes, and a development and income gap between the coastal and inland areas which is as big as between the United States and North Africa. According to one estimate, 30 percent of China's urban workforce, or 200 million people, is currently unemployed or underemployed. The livelihood of another 100 million agricultural workers is threatened as World Trade Organization rules increase China's dependence on foreign food supplies. The specter of chaos frightens the rulers in Beijing.

In international relations, the issue is whether China' impact on the world will be peaceful or violent. Th debate here follows disciplinary lines. "Those who focus on economics tend to see partnership, cooperation and reasons for optimism despite tensions, while security experts are more pessimistic and anticipate strategic conflict as the likely future for two political systems that are so different," writes one commentator [3]. Both views can claim some evidence in their favor...

Google click to call

This is the next step in Internet advertising. Among the text ads that come with your search results you might see a little telephone icon. If you click it and enter your phone number, Google will connect a representative from the advertiser to you via VOIP. (Your phone will ring, and voila, you can discuss how the refrigerator gets delivered or whether the plasma screen is glare resistant. Guess what part of the world the call center rep will be working from :-) This sort of thing is already out there using IM-type clients, but here the user only needs an ordinary phone. Google does not tell the advertiser your phone number, so your anonymity is protected. Very nice!

I'm very envious of the Google people because they actually have the opportunity to implement all of these nifty ideas.

Simons, Thorp and Shannon

Nice article on Jim Simons and Renaissance in Saturday's Times. Part of me wants to get into the new fund (fees are pretty reasonable compared to Medallion), but then again they are embarking on something new, so it's no sure thing. See earlier posts here and here. Most impressive about Medallion is their consistency -- no down years since 1988 and only a single down month in the last 5 years!

BTW, I've received my copy of Fortune's Formula and it's quite good. I learned a number of tidbits about Thorp (the mathematician who wrote Beat the Dealer and invented a system for counting cards in blackjack), Shannon (the father of information theory) and others from this book. Apparently, Thorp and Shannon's investment returns (Thorp ran an early hedge fund called Princeton-Newport, while Shannon invested his own account) rivalled those of the best managers like Buffet and Soros. Buffet and Thorp actually knew each other early on, and had very high opinions of each other. The stories of Thorp testing his card counting system in Nevada are hilarious -- the level of detail after all these years suggests a phenomenal memory!

The bit about optimizing geometric vs arithmetic returns (a subject of controversy between math/physics guys like Kelly, Shannon, Thorp and economists such as Samuelson, and the origin of the title of the book) seems not so interesting to me, as the answer depends on what one wants to achieve. On the subject of hedge funds, it appears everyone is starting one, including information theorist Thomas Cover and former physicist turned AI researcher Eric Baum (author of What is Thought?, the best book I've read on AI).

NYTimes: $100 Billion in the Hands of a Computer

By JOSEPH NOCERA
PEOPLE ask me all the time: What's your secret?" James Simons said. We were sitting in an office in Manhattan that Mr. Simons uses when he's not at the Long Island offices of Renaissance Technologies, the money management firm he founded in 1982. He was wearing an elegant shirt and tie, and loafers with no socks. He took a drag from a cigarette, the second of three he would smoke in the course of a long interview.

I had indeed come to ask him what his secret was. In the hedge fund world, that's what everybody wants to know.

Mr. Simons, 67, who rarely talks to journalists, is hardly a household name like Warren E. Buffett. But Mr. Simons, who got into the hedge fund business after abandoning a stellar career in mathematics, has a track record that is jaw-dropping. This summer, word leaked out that he was starting a new fund - people took to calling it the "$100 billion fund" because its marketing materials say that it could conceivably grow to that enormous size. Not surprisingly, that has caused Wall Street types to be even more curious about him.

Here are Mr. Simons's numbers: from 1990 to 2004, Renaissance's primary hedge fund, called Medallion, has delivered annualized returns of 33.21 percent. (The Standard & Poor's 500-stock index has returned, on average, 10.98 percent during those same years.) Since the end of 2002, the fund, which has $5 billion under management, has disbursed $4.9 billion to its investors - with another $1.5 billion to be delivered at the end of this year.

And these returns are after Medallion's 5 percent management fee and 44 percent share of the profits - surely the highest hedge fund fees in the land. Medallion's returns, and its fees, have helped make Mr. Simons a very wealthy man, with a net worth that Forbes estimates at $2.7 billion.

When I showed Mr. Simons's returns to a hedge fund friend, he looked startled. "Nobody has numbers like those," he said. But here's the real eye-opener: no one outside the firm's 200 or so employees has a clue how he does it.

Medallion, you see, is a quantitative fund. In quant funds, trading activity is generated by complex computer models rather than human judgment. Most quants are secretive about the algorithms that drive their models; after all, that's their investing edge. But of the handful of big-time "black box" investors, as they're often called, Mr. Simons's box may well be the blackest.

HERE'S what we do know. Medallion's portfolio contains literally thousands of stocks and other financial instruments that it trades in rapid-fire fashion. The firm's scientists are constantly searching for repeatable patterns, and other signals, in the enormous amounts of data they compile. The computer models they devise tell them when to make trades based on those signals.

As Mr. Simons put it - and this is about as specific as he would get - "Certain price patterns are nonrandom and will lead to a predictive effect." He also told me that Medallion sticks with highly liquid securities that trade in public markets around the world. Why? "Because there is a lot of data on such instruments, and we're very statistically oriented," he said. He stays away from exotic derivatives.

Not even Mr. Simons's investors know much more than I've just described. "We trust Jim and we think he's smart," said one longtime Medallion investor. "So we stopped caring what the computer was doing." When this investor began describing Mr. Simons's investing approach, he admitted he was guessing.

Mr. Simons shrugged when I suggested to him that his firm's lack of "transparency," as they say in the business, was bound to make people nervous. Humans fail in the market all the time, but somehow we are willing to keep giving our money to human beings to manage because we understand investing based on human judgment. Or at least we think we do. But black box investing feels different. It feels scary somehow, precisely because it is not something most of us can understand.

"How any great investor does it isn't in the least obvious," Mr. Simons responded. "How we do it isn't any more mysterious than how a great fundamental investor does it. In some ways it is less mysterious because what we do can be programmed." Then he stopped, took another drag from his cigarette, and let out a small chuckle. "Well," he conceded, "it's less mysterious to us."

Mr. Simons wasn't always a quant. A former crypt analyst - a code breaker, that is - he did important work in mathematics that helped lay the foundation for string theory. When he began managing money in the 1970's, he did it the same way most investors did: he used his own judgment. "At first," he said, "I didn't think about investing in a scientific fashion. But I was trading currencies, and it gradually occurred to me that there might be some way to create models that would allow you to predict currency movements."

Although Mr. Simons and a partner made an absolute killing in the currency markets the old-fashioned way - they made huge bets that turned out to be right - he began surrounding himself with scientists who developed models for all sorts of tradeable securities. "By the end of the 1980's," he said, "I was a model man, and didn't want to do fundamental analysis." One advantage, he said, is that "models can lower your risk." Another, though, is that "it reduces the daily aggravation." With old-fashioned stock picking, he said: "One day you feel like a hero. The next day you feel like a goat. Either way, most of the time it's just luck."

Indeed, trading the way he does, making thousands of small trades aimed at capturing small price movements, doesn't generate the kind of "10 bagger" that investors love. But when done well, quant investing is less likely to have the kind of disaster that is always the danger when one bets big on a stock.

To those who point to Long-Term Capital Management as an example of the dangers of black box investing, Mr. Simons's defenders point out that his fund has far less leverage than Long-Term Capital, and that in any case, while Long-Term Capital had several Nobel laureates on board, human bets were what caused it to go awry.

Clifford Asness, another well-known quant hedge fund manager, said that while he knew no more about Mr. Simons's methods than anyone else, "It's hard to believe that there isn't a measure of safety in Jim's approach.

"Presumably, he's got a highly diversified portfolio, high turnover, and he's capturing small inefficiencies. It's hard to lose a ton of money doing that. It is always possible that someday his models might stop working. But that's different from 'blowing up.' "

"You know," Mr. Asness added, "human beings have a black box, too. It's called the brain."
As for the new "$100 billion fund," Mr. Simons was even more constrained than usual, thanks to regulatory restrictions that limit what he can say publicly while the fund is raising money. People are buzzing about it nonetheless, for it seems to be a major departure from Medallion. Medallion's investors were almost all wealthy individuals; the new fund, called the Renaissance Institutional Equities Fund, has a $20 million minimum investment and is aimed at institutions. It has a much lower fee structure. It will invest in - or sell short - only publicly traded equities. Instead of making rapid-fire trades, it will be much closer to a buy-and-hold portfolio. And so on.

In one critical way, though, it is similar to Medallion. As the marketing document, which I obtained from a person unconnected to Mr. Simons, put it: "The company's risk control, variance and covariance estimation, execution techniques, slippage models, and predictive signals are all derived from those employed by the managing member in trading the Medallion Funds."

In other words, Mr. Simons believes that computer models similar to those that have worked for Medallion will also work for a fund that can hold $100 billion worth of stocks over long periods of time. It is absolutely audacious.

What interested me most of all was: why? At an age when most men are contemplating retirement, with more money than he can count, why was Mr. Simons still at it? "I enjoy the challenge," he replied.

He then began describing a demonstration he saw recently of a new nuclear accelerator at the Brookhaven National Laboratory, where he is on the board. Two atoms hurtled toward each other, colliding with great force. "A huge number of particles are thrown out," he said, "and the job is to analyze everything that results from the collision."

"Watching the spray of particles on the screen made me think of the stock market," he continued. Every trade, even of a hundred shares of a company, affects every other trade. And every day there are thousands upon thousands of such trades, all of them affecting the rest of the market. His work, as he sees it, is to analyze that incredibly complex mosaic and try to figure out how it all fits together.

"The subject may not be the most important in the world," he concluded, "but the dynamics of the market are really interesting. It's a serious question."

I suddenly understood the motivation behind Mr. Simons's new fund. He's doing it because he wants to see if it can be done. Once a scientist, always a scientist.

The new white flight

Instead of white students fleeing academically weak school districts dominated by disadvantaged blacks and hispanics, in Silicon Valley white students are fleeing overly strong school districts dominated by Asians. Who needs all that math and science anyway?

See related post on Asians and affirmative action here.

WSJ: CUPERTINO, Calif. -- By most measures, Monta Vista High here and Lynbrook High, in nearby San Jose, are among the nation's top public high schools. Both boast stellar test scores, an array of advanced-placement classes and a track record of sending graduates from the affluent suburbs of Silicon Valley to prestigious colleges.

But locally, they're also known for something else: white flight. Over the past 10 years, the proportion of white students at Lynbrook has fallen by nearly half, to 25% of the student body. At Monta Vista, white students make up less than one-third of the population, down from 45% -- this in a town that's half white. Some white Cupertino parents are instead sending their children to private schools or moving them to other, whiter public schools. More commonly, young white families in Silicon Valley say they are avoiding Cupertino altogether.

Whites aren't quitting the schools because the schools are failing academically. Quite the contrary: Many white parents say they're leaving because the schools are too academically driven and too narrowly invested in subjects such as math and science at the expense of liberal arts and extracurriculars like sports and other personal interests.

...In the 1960s, the term "white flight" emerged to describe the rapid exodus of whites from big cities into the suburbs, a process that often resulted in the economic degradation of the remaining community. Back then, the phenomenon was mostly believed to be sparked by the growth in the population of African-Americans, and to a lesser degree Hispanics, in some major cities.

But this modern incarnation is different. Across the country, Asian-Americans have by and large been successful and accepted into middle- and upper-class communities. Silicon Valley has kept Cupertino's economy stable, and the town is almost indistinguishable from many of the suburbs around it. The shrinking number of white students hasn't hurt the academic standards of Cupertino's schools -- in fact the opposite is true.

...white students represented 20% of [Monta Vista's] 29 National Merit Semifinalists this year.

...At Cupertino's top schools, administrators, parents and students say white students end up in the stereotyped role often applied to other minority groups: the underachievers. In one 9th-grade algebra class, Lynbrook's lowest-level math class, the students are an eclectic mix of whites, Asians and other racial and ethnic groups.

"Take a good look," whispered Steve Rowley, superintendent of the Fremont Union High School District, which covers the city of Cupertino as well as portions of other neighboring cities. "This doesn't look like the other classes we're going to."

On the second floor, in advanced-placement chemistry, only a couple of the 32 students are white and the rest are Asian. Some white parents, and even some students, say they suspect teachers don't take white kids as seriously as Asians.

Outsourcing at Conexant

The handwriting is on the wall...

NYT: Dwight Decker, Conexant's chief executive, said that half the semiconductor design and other high-tech engineering work is now done at Conexant India, the division in Hyderabad.

The operation there employs 700 engineers, nearly as many as headquarters does. That is up from 10 percent of such work last year, and Mr. Decker says that figure will jump to 65 percent by the end of 2006, leaving just one-third of the work to be done by the engineers in the United States.

"We are placing a very large bet on our ability to shift a significant part of our development to Asia," Mr. Decker said. "We are doing that more aggressively than any semiconductor company." He emphasizes that no layoffs are planned for Newport Beach, where engineers will work on the "innovation and architecture" of the firm's semiconductor systems.

Conexant's step is enormous, proportionate to its middling size. Giants like General Electric and 3M routinely assign high-tech research to laboratories they own in India and China. And the sprinkling of such work among their global systems is increasing. But most of their research remains in the United States.

Conexant is a fraction of their size, with only 2,400 employees and $900 million in sales of computer and communications equipment last year. Yet it is a technological powerhouse - the inventor 50 years ago of the computer modem - and a major investor in research and development.

Conexant will spend about $250 million this year on research and development because it must keep coming up with new microelectronic wonders for demanding customers like Samsung Electronics, DirecTV and others that are striving to bring movies, music, medical diagnoses and every conceivable service via Internet television to the digital home. Conexant is a strong and rising competitor in the electronics that make possible broadband Internet reception through D.S.L. telephone lines.

In 1955, as a division of North American Aviation, Conexant developed the transistor modem for communicating data over telephone lines, under a contract from the Defense Department. In the early 1980's, as part of Rockwell International, it brought out high-speed fax modems, and in 1996, it introduced high-speed Internet connectivity.

"We have adapted time and again," said Mr. Decker, who has led Conexant since it was spun off from Rockwell in 1999. "And we will continue to adapt and play our part in an expanding world market as long as we innovate." Conexant is putting emphasis on India to keep ahead of it main competitors, the Broadcom Company and the Swiss-based giant, ST Microelectronics Group, said Mr. Decker, a physicist and mathematician with degrees from McGill University in Montreal and a doctorate in math from the California Institute of Technology.

Mr. Decker said that besides financial reasons the export of research jobs is motivated by a looming dearth of engineering talent in the United States. Engineers in India earn one fourth of the pay of their American counterparts, roughly $25,000 a year in salary and benefits, compared with $100,000.

"If we can get two-thirds of our product development at one-fourth the cost, we come close to cutting our overall costs in half," Mr. Decker said. In the first year of large-scale work in India, he said, Conexant reduced costs by $36 million.

Conexant may be a harbinger of developments at corporate research departments across America. "There is a lot more research work being done in India these days," said Shivbir Grewal, a lawyer in Irvine, Calif., who works with companies in America and India.

The attraction is "the huge pool of talent" from the Indian Institutes of Technology - seven major institutions established over the last 54 years - and regional engineering colleges, Mr. Grewal said. But "visas to the United States have been extremely restricted since 9/11," Mr. Grewal said, "so the graduates are staying home and finding work in India." The increase in research overseas is arousing concerns in the United States. The fear is that good jobs will migrate to India and China and that the innovative wellspring of new technology will slowly dry up in the United States.

Terry Opdendyk, a venture capitalist, disagrees with the second concern, but concedes the first.

"It is extremely difficult to achieve and manage innovation in collaborations across oceans and time zones," said Mr. Opdendyk, who has backed more than 100 start-ups as head of Onset Ventures, a Silicon Valley firm he founded in 1984. "But I worry because we're educating too few engineers in America and I don't see us pursuing change-the-world ideas as we used to," Mr. Opdendyk said.

Mr. Decker agrees that at fewer than 60,000 new engineers a year, the United States may not have enough skilled people to handle all the work to be done. However, he said, the United States still leads in information technology, and "we can sustain our world leadership if we continue to innovate."

Fortune's formula

Fortune's Formula: The Untold Story of the Scientific Betting System that Beat the Casinos and Wall Street, by William Poundstone

I just ordered this book from Amazon -- apparently it is in vogue among some of the big institutional investors :-)

The optimization problem described is interesting from an academic point of view, and also as a good rough guide for investors, but of course it assumes that certain quantities, such as correct probabilities of future outcomes, are knowable. That may be the case in casino gambling, but in investing one can only make rough guesses based on past performance. I recall being puzzled at a section on portfolio optimization in the information theory book by Cover and Thomas, but now I see the connection.

BTW, on the topic of finance books, I highly recommend this biography of Fischer Black, which I should have reviewed here long ago. Fischer was yet another outsider (his background was in theoretical physics) to finance who made an important contribution. Unlike Kelly, he was accorded mainstream recognition (professorship at Chicago and partnership at Goldman) during his career. The most impressive thing about Black was his ability to think deeply and independently -- beyond the conventional wisdom. There are some very intriguing passages in the book about his views on money and banking which are, I think, quite unconventional to mainstream economics.

See here for an interesting review of Fortune's Formula by Berkeley math professor E. Berlekamp -- himself a former manager of Jim Simon's Medallion Fund and a collaborator of Claude Shannon!

In a paper published in 1956, John L. Kelly of Bell Labs formulated the asset-allocation problem in terms of an idealized model for which he derived some quantitative results. He used colorful racetrack terminology reminiscent of the classic Damon Runyon movie Guys and Dolls: Suppose that one goes to the racetrack with an available bankroll, B. Suppose further that one knows for each horse the correct probability that it will win the next race. Suppose further that the betting odds are at least slightly inconsistent with this information. And finally, suppose that each race is merely one of a very long sequence of betting opportunities. Kelly found criteria for deciding how much one should then bet on each horse in each race.

Kelly observed that, under similar idealized assumptions, the same formulation could also be applied to investments. In the idealized model, the portfolio manager has an accurate probability distribution on the future performance of each asset in the universe of potential investments. Kelly's methodology then provides a quantitative specification of how big a position to take in each of the candidate assets. Not surprisingly, the fraction of one's portfolio to be invested in any asset that has a negative expected rate of return will be zero. Most assets with positive expected rates of return will merit the investment of some positive fraction of the portfolio. Among assets with similar expected rates of return, those whose returns are relatively stable will be weighted more heavily than those whose future returns have significant risks of substantial losses, even when these risky investments also have some chance of large gains. All of these qualitative features of Kelly's performance criteria concur with conventional wisdom. What distinguishes Kelly's work from that of his predecessors is his quantitative specificity and the fact that he succeeded in proving that, under his assumptions, in the very long run the bankroll of an investor who followed his criteria would eventually surpass the bankroll of anyone following any other strategy.

Kelly also derived a formula for the rate at which this bankroll would grow. This formula is related to a fundamental information-theoretic notion that Claude Shannon (now widely considered to be the father of the information age) had introduced in 1948. Shannon had shown that noise on a communication channel need not impose any bound on the reliability with which information can be communicated across it, because the probability of transmitting a very long file inaccurately can be made arbitrarily small by using sufficiently sophisticated coding techniques, subject to a constraint that the ratio of the length of the source file to the length of the encoded file must be less than a number called the channel capacity. Kelly showed that the asymptotically optimum asset allocation could be determined by solving a system of equations that maximized the log of one's capital. In his horse-track jargon, Kelly also showed that the resulting optimal compound growth rate could be viewed as the capacity of a hypothetical noisy channel over which the bettor was getting the information that distinguished his odds from those of the track. Kelly's betting system, expressed mathematically, is known as the Kelly criterion.

The title of Kelly's paper, "A New Interpretation of the Information Rate," highlighted his discovery of a situation in which Shannon's celebrated capacity theorem applied even though no coding was contemplated. The paper, which appeared in the Bell System Technical Journal, initially attracted a modest audience among information theorists but went unnoticed by economists and professors of finance courses in business schools. Perhaps it would have received more attention if it had had another title. "Information Theory and Gambling" was the title that Kelly himself used for an earlier draft of his paper, but that title was rejected by AT&T executives.

Israeli startups

The Economist discusses the disproportionate impact of Israeli startups in the tech industry. Israelis are especially prominent in computer security, often due to their training in the military. One of the public companies that almost acquired my last startup (since merged/acquired by Juniper) had an extremely bright Israeli CTO who had come aboard from a previous acquisition. Like many other Israeli technologists, he was an alumnus of the IDF (Israel Defense Forces) signal corps training program, which takes in the brightest recruits for intensive IT training. Membership in that unit is sort of equivalent to having a Caltech or MIT degree. At least, the screening process is pretty tough. Of course, there might be other reasons for Israeli success in high tech :-)

Red Herring: Talpiot is a special army training program that puts the best high school graduates through a rigorous curriculum of computer science, physics, and math, then places them in key assignments in, say, intelligence units.

...The selection process for Israel's army-trained technology elite starts when teenagers apply to programs, usually in their last two years of high school. Only volunteers are eligible to be chosen for the army's training programs. The most selective program, Talpiot, accepts only 30 applicants, or 1 in 10, a year. Officers say the army doesn't look for fuzzy traits like creativity and leadership; it focuses on measurable qualities. Extremely high aptitude in math and science, along with success in rigorous exams, are the key qualifications.

REBOOT CAMP

Talpiot's M.O. is total immersion, whether the subject is software coding or the Arabic language. The programming course is just six months long, but classes run from 8 a.m. to 10 p.m., five and a half days a week. Although many soldiers say that the program's immersion approach is an effective way to learn, Col. Tregar says it's simply the only way to cram a lot of information into a very short period of time. There's little time to spend on theory and skills that won't directly relate to the students' army postings later on. "In an academic setting, you'll learn about models for parallel processors and the structure of compilers -- those kinds of things," Col. Tregar says. "It's true it gives you a much broader understanding, but on a practical level, you won't have to deal with them in your first job. People with academic degrees that come to the IDF need to undergo considerable training before being put into practical assignments."

Economist: In 2003, 55% of Israel's exports were high technology, compared with the OECD average of 26%. Tech giants such as IBM, Motorola and Cisco have research centres in Israel, which is also where Intel developed its Centrino chip. Not bad for a country with a population of 6.9m.

Why is Israel—sometimes called the “second Silicon Valley”—so strong in technology? For several reasons, says Mr Mlavsky. First, the pump was primed by government grants in the 1970s, by the BIRD Foundation (a joint American-Israeli initiative that supported many start-ups before VC money was widely available), and by government schemes to encourage Russian immigrants who arrived after the collapse of the Soviet Union.

The second big factor is the army. “The army gets hold of everybody at age 18, and if they have a glimmer of potential, it catalyses their transformation into engineers or scientists,” says Mr Mlavsky. The technically minded are given projects to develop and run, and are allowed to keep any intellectual property that they develop, which results in many spin-outs. It also means that once they get to university, trainee engineers already have practical experience and a problem-solving mentality. Israel has 135 engineers per 10,000 employees, compared with 70 in America, 65 in Japan, and 28 in Britain (see chart).

The small size of Israel's home market is also, paradoxically, an advantage. While a British start-up, say, will look to its home market to get started, Israeli firms cannot. Accordingly, they look to America for customers, so that Israeli start-ups function as “mini-multinationals” from the off—and are instantly exposed to the world's most competitive high-tech market. Similarly, Israel's relative lack of land and resources serves to steer entrepreneurs towards high technology instead.

Naturally, cultural factors play a part too. Around 5% of start-ups in America are headed by repeat entrepreneurs, says Mr Mlavsky, compared with around 30% in Israel. “The whole culture, we're like junkies, and the real kick is success, not the fruits of success, so we want to do it again,” he argues. Israeli entrepreneurs are often workaholics who tend not to change their lifestyles much after becoming successful, he says. Gil Shwed, the boss of Check Point and one of Israel's richest men, still has a regular DJ slot at a Tel Aviv restaurant on Wednesday nights, for example.

The bad news for other countries that wish to encourage the development of their technology industries is that few of these factors can be replicated. Singapore's attempt to establish itself as a biotechnology centre faces the challenge of encouraging risk-taking and entrepreneurialism in a highly conformist society. And Britain is hardly likely to introduce conscription in order to boost the fortunes of the technology cluster around Cambridge University. In technology, as in so many other ways, Israel is a special case.

Chalabi, Hitchens tragicomedy

I don't know whether to laugh or cry. As you may know, Chalabi is the former Iraqi exile (and Chicago math PhD) who conned the neocons with all kinds of crappy pre-war WMD "intelligence," as well as the notion that our troops would be welcomed with flowers (who would you have believed on this, Chalabi or Army Chief of Staff Shinseki?). Rather than being the subject of an FBI/CIA investigation, he is being feted in Washington by Condi and others, and giving a lecture at AEI, apparently attended by Iraq war apologist and self-important blowhard Christopher Hitchens. The little nugget below comes via TalkingPointsMemo.com (see here for more).

Hitchens then turned the subject back to Chalabi, his good friend. I asked him if he thought Chalabi had been passing American intelligence to the Iranians. "No," he insisted. "It's possible that with his training, you know, at [The University of] Chicago that with his own ability he was able to crack the codes. He is a mathematical genius. His expertise is cryptology. It is possible that he broke the codes himself." (This is a paraphrase since I was walking down M Street and crossing Connecticut Avenue all while being amazed that I was having an actual conversation with Christopher Hitchens at the time). Now, I don't believe this for one second. Why would Chalabi be trying to break American codes in his spare time anyway? Who does that if they are friendly to us? Suspicious, I say.

Well, Chalabi's expertise is not cryptography, and no, he didn't single handedly break any Iranian ciphers. What he did was pass on some important intel to the Iranians (that the US had broken their ciphers) that should occasion yet another investigation into neocons leaking classified information.

What makes me most sad is that someone (Hitchens) so ignorant and un-careful in thinking and speaking about topics about which they know nothing could be a public intellectual. But then again in our culture knowledge of basic mathematics or physics is considered geek esoterica whereas an educated person is expected to have read all of Shakespeare.

Baby boom

Invaders: 2, Adults: 0.

The invaders have overwhelmed us! Communications may be interrupted for some time...

VCs, Greenspan and globalization

NYTimes on Silicon Valley VCs and China investments. Note the final paragraph.

WHEN Joe Schoendorf, a Silicon Valley venture capitalist, was in Shanghai a few years ago to hear a pitch from a Chinese start-up company, he sensed something familiar. He interrupted the meeting, walked to the window and pulled back the curtains.

"What are you looking for?" he remembers the would-be entrepreneurs asking.

"I just wanted to make sure I was in China and not back in Palo Alto," he responded.

China's high-technology community, with its brains and competitive spirit, is probably more like its counterpart in Silicon Valley than any other in the world.

Yet Silicon Valley's views of investment in China have tended to swing between wild optimism and deep anxiety - with the anxiety going beyond a fear of losing money. Some worry about helping Chinese start-ups move up the technology food chain.

These days, the Valley venture capitalists are sharply divided in two camps: one rushing into China and one holding back.

...The dominant perspective is that China is a vast sea of opportunity, from its low-cost skilled labor pool to its enormous consumer market that is more than one billion strong.

In fact, it is now routine for venture investors to demand that their start-up firms place the bulk of software development and manufacturing efforts in China or India. (A supply chain problem at a manufacturing arm in China, however, can easily ruin financial results in any given quarter.)

For China skeptics, the concern is that American investment will help energize a formidable competitor, which could come to dominate both markets and technologies.

The fear is based in the Valley's complex relationship with China as supplier, partner, customer and competitor. Most venture capitalists say this evolving relationship will define the future of the Valley and maybe even technology development in the United States.

The Ningbo Bird Company is one case in point. It went from being a contract manufacturing supplier for Motorola to being a serious rival in the Chinese handset market in a matter of a few years.

Still, last year, most of the Valley seemed to throw caution aside as venture firms invested nearly $1.3 billion in China, up nearly 30 percent from 2003, according to Zero2IPO, a venture capital research and consulting company based in Beijing.

But in the first half of this year, investment slowed drastically after several changes in Chinese securities regulations. Those new rules caused "a decline of 50 percent in the first two quarters," said Dixon Doll, managing director of Doll Capital Management, based in Menlo Park, Calif.

The lull is ending, though, in part because of the high-profile success of the initial public offering of Baidu, a Chinese search engine company that was able to raise $86.6 million in August, and a securities rule change in October. In September, Sequoia Capital, a major backer of Google, was reported to be planning a $200 million fund and hiring several employees in China.

That announcement followed an earlier joint agreement this summer by Accel Partners, a leading Silicon Valley firm, and the International Data Group to set up a $250 million fund.

There have even been reports recently that Kleiner Perkins Caufield & Byers, the Valley's highest-profile venture firm, was creating its own China fund, though people briefed on the firm's plans said that was not true. While Kleiner has recently added Colin L. Powell as a partner to serve as a "rainmaker" in Asia, it remains concerned about changes in Chinese security laws that could complicate the return of investment funds to the United States.

Mr. Schoendorf, who is an Accel partner, sees benefits in helping China to become a fierce new competitor. He likens this moment of anxiety and promise to the 1970's, when Japan began to compete successfully with the United States.

"The Chinese graduate more engineers than we do," he said. "They're smart, they work hard, and so the only way to compete with them is to remain more innovative."

WSJ: Greenspan comments on globalization of labor markets. (Holding down inflation is the same as diminished returns to labor...) The bond market reacted negatively to these comments, as Greenspan said the effect would eventually go away. But I think that was a mistake: only the leading edge of those 2 billion workers have been integrated into the world economy -- a distinct minority. It will take decades before that deflationary effect wanes. Not to say that there aren't other inflationary forces at play, like oil and commodity scarcity, but the labor component is deflationary as far as the eye can see. Not only do low-skill workers have little pricing power, but even engineers and service professionals are under pressure from abroad.

The integration of China, India and the former Soviet bloc into the world trading system is helping to hold down inflation but at some point, that effect will fade, Federal Reserve Chairman Alan Greenspan said.

...Mr. Greenspan said the addition of more than 100 million educated workers from former Soviet countries, large segments of China's 750-million strong work force, and workers from India "would approximately double the overall supply of labor once all these workers become fully engaged in competitive world markets," a development that "has restrained the rise of unit labor costs in much of the world and hence has helped to contain inflation."

But while these forces "may well persist for some time," he said, the process and its contribution to inflation control will wane and that "will need to be monitored carefully by the world's central banks."