Reflexive bubbles

Paul McCulley of PIMCO discusses the feedback mechanism at work in the housing bubble (and seems to call the top). I recommend the whole article (whimsical as it is), which also touches on Bretton Woods II, Asian and OPEC mercantilism, etc. He also makes an interesting observation that bond markets now assume that the Fed has tamed and will continue to tame inflation, and long term rates reflect projections of the real interest rates necessary to do so.

If Fed-engineered changes in nominal rates no longer reflect changes in long-term inflationary expectations, but rather changes in real rates, then logic suggests that long-dated income-producing assets – bonds, stocks and real estate – will become inherently more prone to bubble and burst.

Accordingly, it seems to me, asset prices inevitably must take on a higher priority in the Fed’s reaction function than during the War Against Inflation. As long as inflation was too high for the Fed’s secular taste, bubbles and their bursting didn’t carry grave harm. They misallocated resources, to be sure, like all good Austrians properly preach.

...when home price appreciation is running higher than mortgage rates, the market booms, if not bubbles, as momentum players chase the market higher, a text book example of what George Soros calls reflexive demand. But once the momentum breaks – and again, Morgan, declining affordability is the fundamental break – reflexive demand becomes reflexive supply, as former speculative buyers become eager sellers.

Reflexive markets – and property is one if there ever was one – inherently tend to have V-shaped tops, not rolling tops. Thus, both volumes in total home sales, particularly existing home sales, and MEW are set to fall sharply in the year head. Not the stuff of recession, I hasten to add, Morgan, but clearly the stuff of a serious slowdown in consumer spending.

MLF: Won’t that be a problem for foreign members of the BW II arrangement? Didn’t you say that they are as addicted to our spending as we are to their financing of our spending?

PM: Yes, Morgan, I did say that. Which implies that when the American property market comes off the boil, maybe turning tepid, the world will feel the impact, not just American homeowners. Such is the nature of globalization, when the ex-USA world has a shortage of aggregate demand or, put differently, runs a surplus of savings relative to desired domestic investment.

Orwellian Internet spying

Some good blog posts on this subject: Bruce Schneier (security expert and author of Applied Cryptography) and some legal analysis by Dan Solove. Quick summary: probably in violation of existing FISA laws, probably not explicitly unconstitutional under the 4th Amendment, but Bush's justification under Article 2 is weak, bordering on "frivolous" according to one legal scholar.

John Yoo's memo (the wishful thinking of a thirty-something junior DOJ official, who nevertheless was the strongest intellectual voice on this subject in the administration?!#?) comes close to asserting that there are no limits on Presidential power in the unending "war" on terror. How will we decide when this "war" is over? Was there ever even a declaration of war? Several scholars assert that the 9/14/01 Congressional authorization for use of force falls far, far short of a declaration of war. It is miles away from the declarations of war in WWI and WWII (and the threat is much less serious, as I pointed out in the previous post). If one defines a dictator as a ruler who is beyond the law, then Yoo's memo seeks to justify dictatorial power for Bush-Cheney.


Nineteen Eighty-Four

“Remember our boys on the Malabar front! And the sailors in the Floating Fortresses! Just think what they have to put up with.”

“The rocket bombs which fell daily on London were probably fired by the government of Oceania itself, 'just to keep the people frightened'.”

“To tell deliberate lies while genuinely believing in them, to forget any fact that has become inconvenient, and then when it becomes necessary again, to draw it back from oblivion for just so long as it is needed.”

WAR IS PEACE

FREEDOM IS SLAVERY

IGNORANCE IS STRENGTH

I told you so...

Confirmation of massive, illegal violation of privacy rights of US citizens through telco and Internet monitoring, authorized by the "war preznit". The legal position of this administration (I am not kidding) seems to be that the "war preznit" is free to disregard existing US law in "times of war" like these. If Congress pushes the issue, as it should, we may end up with a full-blown constitutional crisis.

Bush is no more a war president than Ronald Reagan or Jimmy Carter, who faced down a much more formidable foreign adversary. You might argue that Al Qaeda is more dangerous than the USSR and eastern bloc, with their hundreds of ICBMs and thousands of nuclear warheads, but you'd be crazy. Let me offer the following analogy. While walking home you are confronted by a man with a loaded shotgun. By staring him down and pointing out that you yourself are armed, you avoid having your head blown off. Continuing on your way home, a small dog bites your ankle. Is the dog really a greater threat, just because it bit you, than the guy with the shotgun? If not, why should we allow Bush to unilaterally claim greater security powers than Reagan or Carter had? (Indeed, contravening the existing FISA law of 1978.)

In an op-ed article published Friday in The Washington Post, Mr. Daschle said he rejected a White House effort three days after the attacks to grant Mr. Bush specific authority to conduct antiterrorism operations within the United States as part of a broader resolution backing the use of force.

In seeking the specific authority for a domestic response, Mr. Daschle said, the White House was effectively acknowledging that the resolution did not cover domestic actions like spying on Americans.

"The Bush administration now argues those powers were inherently contained in the resolution adopted by Congress - but at the time, the administration clearly felt they weren't or it wouldn't have tried to insert the additional language," Mr. Daschle said in the article.

The White House has asserted that the resolution, adopted by Congress on Sept. 14, 2001, freed Mr. Bush from the requirement to get warrants to monitor international phone calls and e-mail of Americans and others in the United States. That resolution authorized the president to employ "all necessary and appropriate force" in response to the attacks on New York and the Pentagon.

But by Mr. Daschle's new account, which appears not to have been made public previously, the White House sought within minutes before the vote on the resolution to alter it to include new wording specifically granting power to carry out the antiterrorism campaign within the United States.

The White House, Mr. Daschle said, wanted the resolution to give Mr. Bush authority to use "all necessary and appropriate force in the United States and against those nations, organizations and persons" responsible for the attacks.

Mr. Daschle said he had turned aside the White House's effort to include "in the United States and" in that sentence, leaving the focus of the resolution on fighting terrorism abroad.

China tech and innovation

Nice discussion (Talk of the Nation; sorry no podcast) with John Seely Brown (former director of Xerox PARC) on innovation and technology in China. Brown has obviously spent some time with startups over there, particular in the cellphone sector, where he claims they are ahead of the US and Europe.

This is the kind of media I can still consume while grappling with two babies ;-)

Sniffing around

It's very plausible to me that the latest Bush perfidy - secretly approving widespread spying by the NSA on US citizens in the wake of 9/11 - is not about phone wiretapping at all. It's about large scale monitoring of Internet communications.

The press has pointed out repeatedly that the 1978 FISA law allows the government to wiretap on short notice, even to wiretap first and then ask a FISA court for permission retroactively. So, the Bush administration's claim that existing legal requirements slowed things down and "jeopardized national security" is just wrong, and informed people know this.

I suspect what is really going on is that in the wake of 9/11 Bush authorized large scale monitoring of Internet communications, probably by allowing NSA to tap into the backbone. There was a lot of discussion of similar programs, such as Poindexter's Total Information Awareness (TIA) system. Technically, it would not be hard to sample Internet traffic, looking for email or Web activity with certain key words or patterns. However, 100% coverage is probably beyond anyone's capability at the moment.

The problem with this is that you only catch dumb terrorists (maybe that's good enough). As I pointed out before, even widely available communication tools like Skype allow for unbreakable encrypted communication. Don't all terrorists, even ones who don't understand how the Internet works, simply assume that it is being monitored (at least in some weak way)? If so, how can Bush claim that whistleblowers jeopardized national security by leaking information about this illegal program?

BTW, good thing Rockefeller (Senate Intel. Cmmte.) kept a copy of the letter he sent to Cheney. The Bushies claim (lying again) that they got congressional approval, conveniently leaving out that Rockefeller protested immediately about the legality of the program (as did Daschle, who claims the briefings may also have been technically misleading).

Note Added: Administration officials are very careful to state that only "communications" between the US and foreign countries are being monitored. I suppose this means that if your packets don't leave the US, they aren't sniffed. However, all Blackberry users should be aware that their email likely travels through servers in Canada, so is potentially subject to monitoring :-) This Times article seems to confirm that email is intercepted.

TalkLeft: Why do Gonzales and Condi Rice keep mentioning the "technical" aspects of the program as a dodge around FISA?

Why this seemingly inconsequential parsing by Bush of the difference between "monitoring and detection"? Bush says they use FISA if they're monitoring, but this is about "detection."

Why, in his letter, does Rockefeller state that he's "not a technician."?
Why the mention of TIA in Rockefeller's letter?
And why the mention of "large batches of numbers all at once"?

Why?

These are not phone numbers we're talking about...These are IP addresses, email addresses.

A system is in place that basically filters on certain triggers (text, phoneme, etc.) within Internet "conversations." This is "detection" or at least its tortured definition that was placed in this idiot Bush's mind. "Monitoring" would be recording an entire conversation, like in a phone conversation.

That system then collects information on those conversations including...ta da...source and destination IP addresses. Those IP addresses can then be stored for further investigation on other "conversations."

The next wave in India outsourcing

Nice summary from the economist. The point that legal services, like accounting, can be outsourced rings particularly true to me. Most legal services (from IP to contracts to employment law) that we consume as a startup are delivered over the phone or Internet, at exorbitant prices ($300 per hour is what you are charged for an associate; a partner bills even more). The effectiveness of the attorney is largely driven by raw brainpower, once some basic knowledge is acquired. I can easily see a model with a US-based partner and half or more of his or her staff (including associates) based in India. $30 per hour would be very good compensation for a highly intelligent Indian attorney.

Should we be concerned for the future of our young friends in law school? See here for related discussion.

ASKED for a sound-check at a function in Delhi this month, Bill Gates eschewed the “1,2,3...” favoured by ordinary mortals. “One billion, 2 billion...,” he counted. They think big, these IT moguls, and especially, these days, in India.

Microsoft later announced plans to invest $1.7 billion in India over the next four years, about half of it in adding to its existing research and development (R&D) and technical-support operations. “The only thing that limits us in India,” Mr Gates told the local press, “is the speed at which we can recruit.” A few days earlier, Intel, a giant chipmaker, had unveiled plans to invest more than $1 billion over five years, much of it in expanding its R&D centre in Bangalore. In October Cisco Systems, the world's largest maker of the routers and switches that direct internet traffic, announced its own plans to invest $1.1 billion in India.

The euphoria is confined neither to American multinationals, nor to information technology. It also encompasses India's own IT industries, and the expanding range of other back-office services that can now be performed remotely. The three biggest Indian IT-services firms—Tata Consultancy Services (TCS), Infosys and Wipro—are each recruiting more than 1,000 people a month. And to take just one example of the other services now moving to India, J.P. Morgan Chase, a big investment bank, this month revealed it is to double, to about 9,000, its staff there. Anyone who assumes J.P. Morgan will simply be doing low-level “back office” tasks in the country—a bit of data entry and paper-shuffling—would be flat wrong. One task for the new recruits is to settle complex structured-finance and derivative deals, what one insider calls “some of the most sophisticated transactions in the world”.

All these investments illustrate that a third stage of the great Indian services-export boom is well underway. In the first, firms such as TCS developed world-class expertise in software “application development and maintenance”, and their low-cost developers became the preferred partners of many Western IT firms. In the second, Indian firms and the local “captive” operations of multinationals started offering low-end back-office services that could take place a continent away—telephone call-centres, transcribing medical records, processing insurance claims and so on. In the third, in both IT and the broader spectrum of other “business processes”, ever-more sophisticated functions are happening in India.

So strong are the forces driving this shift that what seemed improbably rosy projections by NASSCOM, the Indian software- and service-industry lobby, and McKinsey, a consultancy, back in 1999, are coming true. This week NASSCOM and McKinsey produced the second full-scale update of their study. It argues that exports from India's IT industry and from “Business Process Offshoring” (BPO)—both from services “outsourced” to Indian firms and those performed by captives—are on track to reach $60 billion a year by 2010.

That would be a huge surge from the $17.2 billion in the year ending in March 2005. But it implies a compounded annual growth rate of 28%—below that achieved in recent years. Moreover, according to McKinsey's estimates, it requires India merely to maintain its present shares of the markets for offshore IT services (65%) and BPO (46%). This is because the study predicts a massive rise in the size of the overall market, estimated at present to make up just one-tenth of those services that could be sent offshore. The proportion is expected to rise as demography—a western labour shortage—becomes more pressing than protectionism.

In IT the growth in Indian exports is expected to come both from the software market, and from “traditional IT outsourcing”—such as the remote management of whole systems, a market now dominated by the big global IT consultancies. This is expected to rise from 8% of Indian sales now to about 30% in 2010, while software-development's share will fall from 55% to 39%. In business-process-offshoring, the big industries will remain banking and insurance. But rapid expansion is also expected in other areas, like legal services.

The law, in fact, illustrates how vast is the untapped potential market. About $250 billion is spent on legal services world-wide, about two-thirds of it in America, and as yet only a tiny proportion goes offshore. Forrester, a research outfit, has estimated that, by last year, 12,000 legal jobs had moved offshore, and forecast that this will increase to 35,000 by 2010. India, with its English-language skills and common-law tradition is well-placed to secure a big share of the business. It is not just a question of “paralegal” hack work such as document-preparation. Sanjay Kamlani, of Pangea3, a small Indian firm, calls it “real lawyering”—drafting contracts and patent applications, research and negotiation. His clients are both big law firms and in-house legal teams.

India's fundamental attraction has not changed since it first drew software developers: fantastic cost savings. With American lawyers costing $300 an hour or more, Indian firms can cut bills by 75%. Across the board, despite climbing rates of pay in IT and BPO, where rapid expansion has brought frantic job-hopping, India remains, say NASSCOM and McKinsey, the lowest-cost of all the main outsourcing destinations. It also has, among these countries, by far, the largest pool of employable people—those with the necessary language and technical skills. On this measure, India, which produces 2.5m graduates a year, 250,000 of whom are engineers, has 28% of the global available workforce, compared with 11% in China.

Yet the supply of talent may be the biggest constraint on the Indian industry's growth. On these latest projections, the number of people working in IT and business-process exports in India will increase from about 700,000 now to 2.3m by 2010. But on today's estimates only 1.05m suitably qualified people will graduate from college between now and then, so there will be a shortfall of nearly 500,000, with business-processing the worst affected. McKinsey's Jayant Sinha believes the education system can be fixed in time to plug the gap. A bigger worry, he says, is India's creaking urban infrastructure. IT firms in Bangalore, for example, are in revolt against the local government for its neglect of basic amenities. Yet India's IT and business-process industries will need about 14m square metres (150m square feet) of office space by 2010: “a new Manhattan”.

Hectic building is under way, and not just in the big IT and business-processing centres (Bangalore, Mumbai and around Delhi) or the “second tier” of cities such as Pune, Hyderabad and Chennai (Madras). The industry's worry over infrastructure, as over education, is that it cannot do everything by itself. Having thrived by keeping government at arm's length, business now needs help.

Workplace discrimination

From a recent Gallup survey, as reported by the EEOC.

The Gallup data indicate that 15% of all workers perceived that they had been subjected to some sort of discriminatory or unfair treatment. When broken down into sub-groups, 31% of Asians surveyed reported incidents of discrimination, the largest percentage of any ethnic group, with African Americans constituting the second largest group at 26%.

This finding is probably only surprising to white Americans. Asian-Americans know very well that they are subject to stereotyping (albeit perhaps of the unconscious type) and often passed over for leadership roles. Thanks to Confucian values, they are also less likely to complain about it. You won't hear them complaining about being shortchanged by affirmative action, either.

Within the context of race filings, 82.5% of charges were brought by African Americans, with Asian/Pacific Islanders filing only 3% -- a sharp contrast with the 30% of Asians employees who responding to the Gallup survey that they perceived discrimination on-the-job.

Wall Street

Wall Street (1987). Directed by Oliver Stone, starring Michael Douglas (Gekko), Martin and Charlie Sheen (father and son Carl and Bud Fox). A classic that fully anticipated our age of hyper-finance. Gekko and company look tame compared to LTCM, Enron and our current hedge fund overlords!

Gekko: Greed, for lack of a better word, is good. Greed is right. Greed works. Greed clarifies and cuts through and captures the essence of evolutionary spirit. Greed in all of its forms, greed for life, for money, for love, knowledge has marked the upward surge of mankind. And greed, you mark my words, will not only save Teldar Paper, but that other malfunctioning corporation called the USA.

Gekko: The richest one percent of this country owns half our country's wealth, five trillion dollars. One third of that comes from hard work, two thirds comes from inheritance, interest on interest accumulating to widows and idiot sons and what I do, stock and real estate speculation. It's bullshit. You got ninety percent of the American public out there with little or no net worth. I create nothing. I own.

Carl Fox: Stop going for the easy buck and start producing something with your life. Create, instead of living off the buying and selling of others.

Personally, I followed the Create path, but I'm not sure I came out ahead :-)

The view from the PBOC

From an interview with Yu Yongding, an Oxford-trained economist who sits on the Monetary Policy Committee of the People's Bank of China. Who can doubt that everyone sees the handwriting on the wall? He clearly understands the inevitability of dollar devaluation, the need for Asian countries to dump those dollars, the prisoner's dilemma problem that central banks of China, Taiwan, Japan and Korea have, etc. Barring a disaster in Asia (China invades Taiwan, N-S Korea conflict, etc.), I predict a 2x devaluation of the dollar against E. Asian currencies over the next decade. Sorry that prediction is not precise enough to trade on :-)

(See here for previous posts on Bretton Woods II and dollar devaluation.)

"...in the first stage we must reduce accumulation, then later we should reduce our reserves....[China and Asian countries] don't need that large an amount- more than $2 trillion- of foreign exchange reserves.... This is a very big problem and I think the Chinese government should take some action to reduce the growth rate of the accumulation of foreign exchange reserves as we're still facing the possibility of a big devaluation of the US dollar, so the capital losses will be huge. If that happens, it will be tremendous hit to the Chinese economy."

This is hardly the statement of a gentleman with a benign view toward the US dollar's valuation. It is instead a gentleman, in a position of authority, with a great deal of concern. He went on: "The trouble is, with such a huge amount of foreign exchange reserves, that there is no way to spend it very quickly and there's no plan to sell it of course-- otherwise that inflicts damage on ourselves. You don't want to dump shares when the stock market has not collapsed yet and you are the biggest shareholder." Then, he said "all east Asian countries have tremendous foreign exchange reserves and they all want to get rid of them, but if you do this then you cause competitive devaluation, not of their own currencies, but of the US dollar. So we should do this in an orderly fashion. If Asian countries moved too fast, everyone would lose... It would be utterly unfortunate if Japan sells a proportion [of their reserves, for] that causes problems. Then China panics and China sells a proportion -- it would be very damaging."

The "nicest possibility" for China, Japan and the US to escape this problem was for further "tightening of US monetary policy so that further dramatic devaluation of the US dollar can be stopped. Then, because of the slowdown in the economy, the US current account deficit would reduce and in this way will create conditions for East Asian countries to get off the hook."

2 percent of the Caltech class

A friend of mine who runs a derivatives desk made job offers to 2 percent of the Caltech class of 2006. That's only 4 people, but still a pretty high percentage! 3 have accepted, the other went to a well-known software company. It makes me wonder how many kids who otherwise would have gone on to grad school (e.g., in engineering, math or physics) are heading right into finance these days. Smart move, if you ask me :-)

Most of my colleagues still don't take the theory of modern finance seriously at all. Even researchers who work on complex systems (modeling traffic, sandpiles, networks, ants, etc.) show surprisingly little interest. I guess the future belongs to the young!

Physics still pulls them in...

There is much concern among physicists that our field is no longer getting the same share of the brightest students as in the glory days (e.g., during the cold war). I was pleased to find here that average GRE-general scores in physics PhD programs were still the highest as of 2002, with math and CS close behind. According to these tables, the average IQ of a physics PhD student is about 130. For comparison, english/literature PhD students average about 120 and sociology PhD students only 115. (115 is the average for college students in general, so the typical sociology TA is no smarter than the students in his or her class ;-) Amazingly, the average grad student in education (109) is below the average for college students!

You may find it odd that I converted the results from GRE score to IQ, widely regarded by the politically correct as a discredited measure of intelligence. For a rebuttal of this often repeated, but scientifically unsupported, point of view, see, e.g., Steve Pinker's book The Blank Slate, or any academic research of the type excerpted below.

g on the Job


Although the evidence of genetic and physiological correlates of g argues powerfully for the existence of global intelligence, it has not quelled the critics of intelligence testing. These skeptics argue that even if such a global entity exists, it has no intrinsic functional value and becomes important only to the extent that people treat it as such: for example, by using IQ scores to sort, label and assign students and employees. Such concerns over the proper use of mental tests have prompted a great deal of research in recent decades. This research shows that although IQ tests can indeed be misused, they measure a capability that does in fact affect many kinds of performance and many life outcomes, independent of the tests' interpretations or applications. Moreover, the research shows that intelligence tests measure the capability equally well for all native-born English-speaking groups in the U.S.

If we consider that intelligence manifests itself in everyday life as the ability to deal with complexity, then it is easy to see why it has great functional or practical importance. Children, for example, are regularly exposed to complex tasks once they begin school. Schooling requires above all that students learn, solve problems and think abstractly. That IQ is quite a good predictor of differences in educational achievement is therefore not surprising. When scores on both IQ and standardized achievement tests in different subjects are averaged over several years, the two averages correlate as highly as different IQ tests from the same individual do. High-ability students also master material at many times the rate of their low-ability peers. Many investigations have helped quantify this discrepancy. For example, a 1969 study done for the U.S. Army by the Human Resources Research Office found that enlistees in the bottom fifth of the ability distribution required two to six times as many teaching trials and prompts as did their higher-ability peers to attain minimal proficiency in rifle assembly, monitoring signals, combat plotting and other basic military tasks. Similarly, in school settings the ratio of learning rates between "fast" and "slow" students is typically five to one.

...The existence of biological correlates of intelligence does not necessarily mean that intelligence is dictated by genes. Decades of genetics research have shown, however, that people are born with different hereditary potentials for intelligence and that these genetic endowments are responsible for much of the variation in mental ability among individuals. Last spring an international team of scientists headed by Robert Plomin of the Institute of Psychiatry in London announced the discovery of the first gene linked to intelligence. Of course, genes have their effects only in interaction with environments, partly by enhancing an individual's exposure or sensitivity to formative experiences. Differences in general intelligence, whether measured as IQ or, more accurately, as g are both genetic and environmental in origin--just as are all other psychological traits and attitudes studied so far, including personality, vocational interests and societal attitudes. This is old news among the experts. The experts have, however, been startled by more recent discoveries.

One is that the heritability of IQ rises with age--that is to say, the extent to which genetics accounts for differences in IQ among individuals increases as people get older. Studies comparing identical and fraternal twins, published in the past decade by a group led by Thomas J. Bouchard, Jr., of the University of Minnesota and other scholars, show that about 40 percent of IQ differences among preschoolers stems from genetic differences but that heritability rises to 60 percent by adolescence and to 80 percent by late adulthood. With age, differences among individuals in their developed intelligence come to mirror more closely their genetic differences. It appears that the effects of environment on intelligence fade rather than grow with time. In hindsight, perhaps this should have come as no surprise. Young children have the circumstances of their lives imposed on them by parents, schools and other agents of society, but as people get older they become more independent and tend to seek out the life niches that are most congenial to their genetic proclivities.

A second big surprise for intelligence experts was the discovery that environments shared by siblings have little to do with IQ. Many people still mistakenly believe that social, psychological and economic differences among families create lasting and marked differences in IQ. Behavioral geneticists refer to such environmental effects as "shared" because they are common to siblings who grow up together. Research has shown that although shared environments do have a modest influence on IQ in childhood, their effects dissipate by adolescence. The IQs of adopted children, for example, lose all resemblance to those of their adoptive family members and become more like the IQs of the biological parents they have never known. Such findings suggest that siblings either do not share influential aspects of the rearing environment or do not experience them in the same way. Much behavioral genetics research currently focuses on the still mysterious processes by which environments make members of a household less alike.

Scientific method at Google

Great podcast of an interview with Google busdev VP Omid Kordestani. Kordestani is credited with coming up with the AdSense business model (although it originally came from Overture, and another person gets the credit in a recent NYT article discussed here). There was also a recent WSJ article in which Kordestani is described as one of the Googlers driving up real estate prices in Atherton -- in his case by buying a 16,000 sq ft house for $17.8 million!

What is interesting about the interview is Kordestani's continued references to the scientific methods applied within Google to tasks as varied as screening job applicants, rooting out click fraud, monitoring (and modeling) AdSense auctions, etc. He uses the phrase "teams of PhDs" several times :-) It is amazing to hear a busdev guy talk this way, even considering that Kordestani has an engineering background.

Expert predictions

How good are "experts" at making accurate predictions? Much worse than you think, says psychology professor Philip Tetlock (Haas School of Business at UC Berkeley) in his new book Expert Political Judgment: How Good Is It? How Can We Know? (See New Yorker review.) In detailed studies, in which "experts" were asked to make forecasts about the future (predicting which of three possible futures would occur), it was found that the "experts" did no better than well-informed non-experts! As Tetlock says, “We reach the point of diminishing marginal predictive returns for knowledge disconcertingly quickly,” he reports. “In this age of academic hyperspecialization, there is no reason for supposing that contributors to top journals—distinguished political scientists, area study specialists, economists, and so on—are any better than journalists or attentive readers of the New York Times in ‘reading’ emerging situations.”

Now, I expect the performance of scientific experts to be somewhat better. Questions like "How hot will that spacecraft get while in orbit around Mercury?" or "How many CPU cycles will it take to compute that integral?" are ones where predictions of real experts will far outperform those of lay people. I guess there is something fundamentally different about scientific versus non-scientific expertise? The last bit below about predicting freshman academic performance is amazing (but not unexpected). Let a simple one or two parameter model pick your freshman class :-)

Finally, what type of "expert" would you trust to run your money (make investment predictions)? As Jim Simons said: "The advantage scientists bring into the game is less their mathematical or computational skills than their ability to think scientifically. They are less likely to accept an apparent winning strategy that might be a mere statistical fluke." In other words, they know when they know something, while others might just be fooling themselves ;-)

New Yorker: Tetlock is a psychologist—he teaches at Berkeley—and his conclusions are based on a long-term study that he began twenty years ago. He picked two hundred and eighty-four people who made their living “commenting or offering advice on political and economic trends,” and he started asking them to assess the probability that various things would or would not come to pass, both in the areas of the world in which they specialized and in areas about which they were not expert. Would there be a nonviolent end to apartheid in South Africa? Would Gorbachev be ousted in a coup? Would the United States go to war in the Persian Gulf? Would Canada disintegrate? (Many experts believed that it would, on the ground that Quebec would succeed in seceding.) And so on. By the end of the study, in 2003, the experts had made 82,361 forecasts.

...Tetlock got a statistical handle on his task by putting most of the forecasting questions into a “three possible futures” form. The respondents were asked to rate the probability of three alternative outcomes: the persistence of the status quo, more of something (political freedom, economic growth), or less of something (repression, recession). And he measured his experts on two dimensions: how good they were at guessing probabilities (did all the things they said had an x per cent chance of happening happen x per cent of the time?), and how accurate they were at predicting specific outcomes. The results were unimpressive. On the first scale, the experts performed worse than they would have if they had simply assigned an equal probability to all three outcomes—if they had given each possible future a thirty-three-per-cent chance of occurring. Human beings who spend their lives studying the state of the world, in other words, are poorer forecasters than dart-throwing monkeys, who would have distributed their picks evenly over the three choices.

Tetlock also found that specialists are not significantly more reliable than non-specialists in guessing what is going to happen in the region they study.

...“Expert Political Judgment” is just one of more than a hundred studies that have pitted experts against statistical or actuarial formulas, and in almost all of those studies the people either do no better than the formulas or do worse. In one study, college counsellors were given information about a group of high-school students and asked to predict their freshman grades in college. The counsellors had access to test scores, grades, the results of personality and vocational tests, and personal statements from the students, whom they were also permitted to interview. Predictions that were produced by a formula using just test scores and grades were more accurate.

Postdoc (no)bargain

This was posted as a comment, but I thought it so good I should share it here.

Note if there is almost one postdoc per permanent researcher (or professor), it means that one 3-5 year cohort of PhDs could replace the entire pool of career researchers. If a career lasts 30 years it means there is a six or ten to one ratio between postdocs that leave the field and those who survive with permanent positions. Even if the postdoc to professor ratio is lower (as in physical science and math -- this number is smaller largely because they don't have much funding for postdocs in math), there may still be a big ratio between PhDs produced and number of professors. Unless the average professor produces no more than one PhD per 30 years, there will continue to be a big oversupply and a tough labor market for scientists.

See related posts here and here.

Thanks for the Great Postdoc Bargain
Richard Freeman
30 August 2002

Postdocs are at the heart of the United States?s extraordinarily successful biological and life scientific research program over the past two-plus decades. In this period, postdocs have produced most of the results in academic laboratories and have come to play an increasing role in industrial and government labs as well. Academic institutions, which engage some 80% of postdocs, aren?t sure whether postdocs are employees, students, or some form of apprentice. With responsibility for hiring and career development resting firmly with the principle investigators who employ the postdocs on their research grants, many universities don?t even know how many postdocs they have or what they are paid, much less how they are progressing toward ? whatever the future holds for them.

Whatever they are, however, postdocs are one of the greatest bargains in the U.S. economy. Where else can one hire Ph.D.s, whose training and smarts put them among the best and brightest in the world, to work 60 hours a week for $30,000 to 40,000 a year, with limited benefits and little power to influence their working conditions and pay? Given the long hours that postdocs work, their hourly pay is on the order of $10 to $13 per hour--on par with the wages paid to custodial and other low-paid workers that have spurred living wage campaigns around the country.


...Huzzah! Huzzah!

Two to three decades ago, the U.S. rewarded postdocs with a reasonably good chance of being hired as a principal investigator. Sorry, but we can no longer carry out that part of the bargain. As Table 1 shows, there are just too many postdocs for us to absorb them as tenured faculty. In 1987, the ratio of postdocs to tenured faculty was already too high at 0.54 for most to obtain faculty jobs at the rate of growth of academic employment. By 1997, the ratio had risen by 43% to 0.77. It has presumably risen further since then. As a result--and as many postdocs have learned to their chagrin--the U.S. does not have a place for them on standard academic tracks.

But don?t get discouraged, postdocs. We need you for our research. How about another postdoctorate--a few more years of long hours at low wages?

Table 1: Ratio of the Number of Postdoctorates in Higher Educational Institutions to the Number of Tenured Faculty, 1987 and 1997

Discipline
1987
1997
% Change

Life Sciences
0.54
0.77
43%

Physical Sciences and Mathematics
0.20
0.23
15%

Engineering
0.11
0.19
73%

Source: The National Academies Committee on Science, Engineering, and Public Policy Enhancing the Postdoctoral Experience for Scientists and Engineers, (NAS, 2000), table B-1 and table B-14.


The forces of supply and demand are unlikely to improve the economic situation of postdocs in any plausible time period. One reason is that the supply of postdocs consists not only of U.S. citizens and permanent residents gaining Ph.D.s but also of U.S.- and foreign-trained Ph.D.s from other countries. Indeed, U.S. scientific research could not proceed at anything like its current pace was it not for the influx of foreign postdocs. Roughly half of postdocs currently come from overseas, many from countries with low personal income rates such as China. Remove foreign postdocs from the nation?s labs and postdoc pay would zoom ... at the cost of short-term chaos and a long-term slower rate of scientific progress. Nevertheless, we should not forget to thank our foreign postdocs for their long hours and hard work on behalf of the rest of our society...
--
Posted by scienceguy11 to Information Processing at 12/05/2005 11:37:13 AM

Is gravity ergodic?

New paper! (Available on arxiv.org.)

The universe is very far from maximizing its entropy. Most egregious is the gravitational state of the universe, as noted years ago by Penrose. Gravity, being a long range, unscreened, attractive force, maximizes its entropy by clumping. This is quite unlike systems with ordinary interactions (generally of limited range, or subject to screening), whose entropy is maximized when matter and energy are uniformly distributed. In the case of gravity the most entropically dense configurations are black holes, with exp(A) microstates (A is the area in Planck units). Why did the big bang begin with a smooth background metric, instead of an agglomeration of black holes, when the latter has much higher entropy and hence occupies an exponentially larger portion of phase space? We explore how a phase transition between the smooth, low entropy spacetimes and the high entropy black hole phase might occur.

Note added: After we posted the paper we discovered calculations using Euclidean path integrals (gravitational instantons) which yield the same results for the black hole free energy and nucleation rate as we obtain from our simple intuitive arguments.


Thermal gravity, black holes and cosmological entropy

Authors: Stephen D. H. Hsu, Brian M. Murray

Taking seriously the interpretation of black hole entropy as the logarithm of the number of microstates, we argue that thermal gravitons may undergo a phase transition to a kind of black hole condensate. The phase transition proceeds via nucleation of black holes at a rate governed by a saddlepoint configuration whose free energy is of order the inverse temperature in Planck units. Whether the universe remains in a low entropy state as opposed to the high entropy black hole condensate depends sensitively on its thermal history. Our results may clarify an old observation of Penrose regarding the very low entropy state of the universe.

Vernor Vinge talk

Nice podcast of a talk given at Accelerating Change 2005. Vinge is a mathematician and sci fi writer who popularized the term Singularity (originally due to Ulam?) to describe the rapid acceleration of technological change that might accompany the development of machine intelligence beyond our own.

Previously I discussed the impending singularity, the relation to the Turing test and AI for kids.

Platform companies

Economist Brad Setser has been discussing platform companies as the future for US industry. The idea is that high value-add activities such as design, engineering and management are performed here, and low value-add manufacturing is done abroad. A good example of this is the iPod, which was designed in Cupertino but is manufactured in Taiwan. Apple sold $1.2 billion worth of iPods in Q1 2005, making iPods almost as big as their computer business! Apple's margin on iPods is healthy - estimated at up to 20% -- while I am sure the manufacturer's margin is very slim. So, the conventional wisdom goes, let those low cost countries have the cutthroat manufacturing business. We clever Americans will simply go upstream, creating new products and markets through innovation.

There is a problem with this, and it is related to the winners and losers issue that arises in the usual story of comparative advantage. Overall, societies may enjoy a net benefit from trade (heck, there are even "theorems" going back to Ricardo "proving" this), but even in the optimistic cases the benefits (and losses) will be unevenly distributed. As autoworkers are displaced by Delphi moving production to China, investment bankers or consultants in the US benefit from demand for globalization strategists. But who ensures that some of the benefits are redistributed? Of course we all benefit from lower prices for manufactured goods, but that is only partial recompense to an unemployed auto worker.

I would go even further, and guess that only a small fraction of the population in advanced countries has the cognitive capabilities to be on the winning end of this process. That is, the Apple iPod designers, investment bankers, CEOs, etc. who win at this game are very able -- in terms of conventional IQ or special abilities such as leadership, design creativity, etc. It is questionable whether the average person will ever be able to contribute on a "high value-add" team! Instead, they'll be caught in a process taking place over a generation in which low-skill wage costs equilibrate worldwide. The theorems might still be satisfied by the huge returns to very able people (whether in China, India, Russia or the US), but the trend for the average American might be negative. Instead of an auto worker in Detroit earning enough for a good retirement and college education for their kids, they'll instead make $10-20 per hour, including benefits. (See earlier post Equilibration can hurt.)

Previous discussion of outsourcing here and here.