Outsourcing CS homework?

Those creative, enterprising Americans are at it again! They've outsourced boring, low value-added tasks like learning C++ to foreigners! Thanks to yankee ingenuity we'll grow our GDP through real cutting edge innovation, like organizing raves or inventing new ways to dispense shots. Who needs all that math anyway? :-)

WSJ: ...But what the computer-programming student who goes by the handle "Lover Of Nightlife" did last month, as the fall semester raced to a close, could only have happened in the age of the Internet: He went online to outsource his predicament.

"This is homework I did not have time to study for," he said in a message on a Web site devoted to outsourcing computer projects. "I need you guys to help me."

Attached was a take-home final exam for a computer class that Mr. Nightlife Lover wanted to pay someone else -- presumably, someone from a place where people can't afford a lot of night life to begin with -- to take for him.

This bit of commerce took place on Rentacoder.com, a Web site that has been mentioned before in this column as an example of globalization in all its blood-curdling efficiency. Rent A Coder enables people -- usually Americans -- who need computer programs to put them out to bid -- usually for cut-throat prices by Indians and Eastern Europeans.

But if U.S. companies can go online to outsource their programming, why can't U.S. computer students outsource their homework -- which, after all, often involves writing sample programs? Scruples aside, no reason at all. Search for "homework" in the data base of Rent A Coder projects, and you get 1,000 hits. (An impressive number, but still a tiny fraction of all computer students, the vast majority of whom are no doubt an honest and hardworking lot.)

A few examples: "I need a simple console-based program and a PHP script written that uses the openssl library." "I need 2 algorithms filtering -- median and Gaussian." "A C++ program that will implement a billing system using threads. Needs to be completed tonight if possible."

Indeed, some programming students appear to be outsourcing their way through college. "Pascal Rookie," from Colorado Springs, Colo., has put five school projects to bid. And while he may be a plagiarist, at least he treats his helpers well: Mr. Rookie has received the highest marks possible for a buyer in the eBay-like rating system used by Rent A Coder. "A pleasure to work with him," said one.

You can't tell from the site how much was paid for the help, but usually it's well less than $100.

Math rules!

Says the cover of BusinessWeek. It's a pretty lightweight article (see the amusing graphic How much math do you need to know?), but in all seriousness I do believe the economic returns to mathematical ability are going up and up. Yet, oddly, salaries of scientists have been stagnant in real terms over the last decade. (See also here.)

Prediction markets and the LHC

We're nearing a new era in particle physics, when the Large Hadron Collider (LHC) at CERN begins operation, and hopefully we'll finally discover the mechanism by which spontaneous symmetry breaking occurs in the electroweak theory. The vacuum state of our universe chooses a direction in the SU(2) x U(1) gauge space, thereby making the W and Z bosons heavy, and giving masses to spin-1/2 particles like the electron. We've been waiting for decades now to understand the detailed dynamics by which this occurs. Had the US SuperCollider project not been terminated in the early 1990's, we would have known the answer back in the 20th century. Now our hopes rest on a less powerful machine in Geneva.

In an earlier post I noted the inability of "experts" in certain fields (economics, political science, foreign affairs) to predict the future. They barely outperform monkeys throwing darts, and do not outperform well-informed lay people in their predictions. Can particle theorists do better? Over the years, theorists have expended an incredible number of high IQ person-hours investigating certain models of electroweak symmetry breaking (ranging from supersymmetry, to strong QCD-like interactions, to extra dimensions, etc.). These are more or less mutually exclusive possibilities, so it will turn out that much (if not most) of the time spent will have been completely wasted once the dust settles and the data tells us how Nature really works. (You might argue that time spent on exploring a speculative particle physics model is not wasted, even if the model fails to describe reality, and I might accept that point of view for the 10 or 100 best papers written on a particular model or idea, but not for the 1000th!)

I propose that we set up a prediction market where theorists can put their money (inevitably, small sums :-) where their mouths are. In a prediction market one trades outcome contracts which pay off a fixed amount (say, one dollar) in the event that the outcome matches reality (Kerry wins 2004 election, superpartner to gluon discovered with mass less than 100 GeV, etc.). If the set of contracts is exhaustive -- constructed to cover all possibilities (in our case, there may have to be a "none of the above" contract), the prices of each contract will reflect the probability that the community collectively assigns to a given outcome. It would be very interesting to see if this market does a good job of predicting the future, or if proud particle theorists are just as subject to the madness of crowds as stock and real estate speculators.

We might get the Iowa Electronic Markets to help us with this.

Barry Diller interview

Nice podcast of an interview with former media mogul (now Internet mogul) Barry Diller. Diller recently added Ask Jeeves (which bought its search technology from Teoma) to his list of Internet properties, which includes Match and Expedia. Diller reasons very cogently about risk taking and business strategy, and seems to have a much firmer grasp of where Internet technologies are heading than most media-industry transplants. His description of the assumptions behind the Ask acquisition, and of his partnership with Rupert Murdoch in FoxTV are both revealing. I don't agree with everything he says (he's quite negative about the prospects for user-generated content, and overconfident about the efficiency of talent-filtering in our current system), but it is definitely worth a listen. It amuses me greatly that Diller has a better intuitive grasp of the term speculative than many researchers (string theorists) in my field :-) He refers to the $2.6 billion Skype acquisition using that term, as well other recent transactions.

Greenspan gets it, will Bernanke?

Greenspan seems to grasp the effect of globalization on US wages and inflation.

WSJ: Buried in the minutes of the Federal Reserve's Dec. 13 policy meeting, released last week, was this bland observation: Fed officials "noted that robust competition -- including from foreign producers...[was] helping to contain cost and price pressures."

Markets largely ignored the sentence, but it's one of the most important factors guiding Fed Chairman Alan Greenspan in his final weeks at the helm of the nation's central bank. Mr. Greenspan has marveled at how inflation and wage growth have stayed low even as the economy continues to grow robustly and the jobless rate falls below 5%, a level that has often driven up wages and prices. Searching for an explanation, he has hit on globalization.

If the Fed thinks growing imports, outsourcing and international investment are holding a lid on wages and prices, then interest rates can be lower than they otherwise would. Indeed, as the minutes show, this is one reason the Fed seems to think it can stop raising rates soon. Of course, there is no guarantee Ben Bernanke, nominated to succeed Mr. Greenspan, will be guided by the same sentiments.

...Now, with productivity growth leveling off and unemployment declining, Mr. Greenspan has fingered globalization as the missing variable that explains why inflation is so low. He began this avenue of inquiry in 2003 when he was trying to understand how U.S. trade deficits, which must be financed by borrowing overseas, could keep growing with no upward pressure on U.S. interest rates and little downward pressure on the dollar. His conclusion was that the U.S. can run large deficits with its trading partners because investors have become less sensitive to international borders in deciding where to put their money.

Last year, he expanded the explanatory power of globalization to include its influence on inflation and wages. The integration of the former Soviet Union, China and India into world markets would "approximately double the overall supply of labor," he told Congress, and prove a major contributor to "the disinflationary pressures that have been evident in the global economy."

...Often the mere threat that production may move offshore is enough to trigger wage concessions. Auto-parts supplier Delphi Corp., now in bankruptcy protection, has asked for steep wage and benefit cuts from its workers to meet customer demands that it match the prices of low-cost foreign suppliers. Several academic researchers say this "threat effect" has depressed wages and increased inequality in numerous countries, both rich and poor.

Better than the yield curve?

A yield curve inversion has often been an indicator of recession. However, talk is abundant that "this time it's different" -- long yields are low because of foreign central bank buying, not because the market is predicting recession.

Bill Gross of PIMCO touts a better indicator than the yield curve. He compares time-averaged (say, 5 year duration) rates to short term rates, in contrast with the instantaneous values plotted on the yield curve. It appears this indicator has been more reliable in the past, and is currently predicting an economic slowdown and a bull market for bonds.

This 5-year swap concept is important because the U.S. economy operates in much the same way. With close to a 5-year average life, the entire U.S. bond market can be compared to a 5-year fixed swap. That means that companies, homeowners, and consumers that have borrowed money in recent years – (and purchased assets such as a home that are akin in my example to a 5-year swap) – are now being squeezed in a flat yield curve environment. Visualize a real life example in which you have “financed” a home with an adjustable rate mortgage (in my example you finance a 5-year swap with floating 3-month Libor). As the cost of the ARM increases with higher short rates, your excess income available to spend on discretionary items begins to shrink. If that ARM rate goes too high, you hunker down even more by not eating out, going to movies, or taking a vacation to exotic destinations. The economy in other words slows down. How does this translate into a bond market timing tool? Chart I shows but one of a series of graphs PIMCO uses to indicate when enough is enough – the point at which adjustable short rates rise sufficiently to make the owner of a home or a 5-year swap, or more importantly the economy, cry “no más!” That point comes in this example when Fed Funds rise to meet the average cost of intermediate Treasury financing issued over the past 5 years and the spread between the two disappears.

For sophisticates, please note that this is not the same thing as a flat yield curve. A flat yield curve is a concept comparing current short rates to current 5- and 10-year rates. What my chart does is to compare current short rates to the Treasury’s average intermediate term “coupon,” a more reliable and indicative indicator of economic pain or restrictiveness since it uses an average embedded cost of debt concept instead of a current cost. The standard flatness as measured by current market rates in early 1995 (not shown here) never led to a recession, only a slowdown, just as Chart I would have indicated. In other words, this indicator called for a mild slowdown in 1995which is what we got. The standard flat curve theory called for something more extreme which is something we never got. The embedded cost of debt indicator, therefore, shown in Chart I, has been more reliable.

...yields have peaked in the bond market and will soon peak in Fed Funds producing an economic slow-down in 2006. If the Fed goes beyond 41⁄2% and inverts the yield curve, the possibility of recession will increase. Observant readers will have already noted that the current data point in Chart I is not only calling for an end to the bear bond market, but a recession at some point 12-18 months hence. Perhaps. Much will depend on the future condition of the U.S. housing market and of course global economies – primarily of the Asian variety.

Speculative flows, the yuan and deflation in China

This Times article suggests that speculative currency flows into yuan have dropped off drastically as hopes of a significant near-term revaluation have faded. Particularly interesting is the claim that the inflationary forces felt recently in China were more due to "hot money" from outside rather than overheating of the economy. With growth rates in the 10% range there are few signs of inflation now.

...Hundreds of billions of dollars have flowed into China in recent years, driving the country's total reserves beyond Japan's, to $860 billion. But this fall, only half as much money flowed into China as a year earlier.

Investors in the once red-hot Shanghai property market are walking away, and currency speculators who had bet that China would sharply strengthen the yuan have pulled back as it became clear that the authorities favor a slow approach to currency appreciation.

The flow of money has diminished even with the rise in the Chinese trade surplus, which has tripled in the last year to the irritation of politicians and business executives from Washington to Brussels to Tokyo. They want Chinese officials to let the yuan rise faster to make the country's exports more expensive in foreign markets.

In July, authorities revalued the yuan only slightly after trading partners complained that its link to the dollar kept exports cheaper than they would otherwise be. That, combined with very low interest rates, damped investment enthusiasm.

The slowing of speculative investments "takes the pressure off" Chinese authorities to allow appreciation, said William Belchere, chief Asia economist at Macquarie Securities here.

The reduced speculation has brought measurable advantages as well: inflation has almost disappeared, even as the economy grew at a 9.8 percent rate last year. Economists attribute this in part to dwindling amounts of speculative money sloshing around the economy.

China has also enjoyed excellent harvests, and there has been so much investment in new factories that a recent government report estimated that three-quarters of all categories of manufactured goods suffered from oversupply.

"There's no pricing capacity for anything," said Jing Ulrich, a J. P. Morgan economist here. "Food prices are falling," she said, and those for manufactured goods "are seeing stagnation at best."

...The central bank said late Tuesday that it wanted to keep the yuan's exchange rate "basically stable at an adaptive and equilibrium level." It added that big fluctuations "will adversely affect China's economic and financial stability and undermine China's fundamental interests."

Qu Hongbin, a senior economist at HSBC, said speculators had expressed disappointment that China did not let the yuan rise more in July and was keeping short-term interest rates at half the levels in the United States. Speculative inflows have virtually disappeared as a result.

Japan chipmakers: industrial policy gone bad

Proponents of industrial policy should have a careful look at the chip industry in Japan for an example of how badly wrong things can turn out. In the 80's the US formed an industry-government consortium called Sematech to combat what seemed to be an unstoppable Japanese juggernaut. The semiconductor industry was given as a prime example where the US had dropped the ball and where Japanese industrial policy, formulated by MITI, would lead to domination of yet another market. Now, Samsung alone spends more on chip R&D and infrastructure than all of Japan combined!

That the NEC President could say "It's a big risk to limit yourself to a small number of products. Those have to be very strong products," shows the enormous gulf between Japanese corporate thinking and the competitive and entrepreneurial spirit we have here. Innovative companies here are often one (or few) product companies. If your product isn't strong, why are you in that market in the first place?

Choosing technology winners and losers is probably not an area in which government will outperform the market. Is a bureaucrat going to do a better job than entrepreneurs, VCs and big company CEOs with skin in the game? We're better off putting our resources into basic science (the underlying infrastructure for technological advances) and science and engineering education.

NYTimes:
In the late 1980's, Japan dominated the global computer chip industry, overtaking the United States in what was seen as a symbol of American economic decline and Japanese ascendance.

Those roles have been reversed. Japan's global market share is now half of what it was then, while Intel of Santa Clara, Calif., has risen to become the world's largest and most profitable chip maker. Indeed, Intel and Samsung Electronics, a South Korean company that was not even in the picture in Japan's glory days, together have a market share as large as the combined shares of the 20 large Japanese chip makers tracked by the research firm iSuppli.

Japanese chip makers are trying to snap out of this decline by joining forces, either by sharing factory construction costs or through outright mergers. The latest move came Dec. 28, when the Japanese chip makers Hitachi, Toshiba and Renesas Technology announced they were in talks to jointly build a semiconductor factory, a project that would be backed by the government. The media has called the plant the Rising Sun chip factory, after Japan's flag.

Efforts to combine forces have failed in the past: a wave of mergers two years ago produced companies as unprofitable as their predecessors.

At their height in 1988, Japanese companies produced 51 percent of the world's semiconductors, and the top three chip makers by market share - NEC, Toshiba and Hitachi - were all Japanese. Now, Japanese companies have a combined share of 23.4 percent of the $237.3 billion global semiconductor market, according to iSuppli. Just three Japanese companies made the Top 10.

"This has been a lost decade and a half for Japanese semiconductor companies," said Yoshiharu Izumi, an analyst at J. P. Morgan Securities. "Japan has been caught between the United States and Asia, and this middle ground keeps shrinking."

The chip makers' woes have spurred much soul-searching in Japan, where the industry had been a source of national pride. But analysts say an intense sense of national mission in Japan's chip industry has been one cause of its undoing.

For years, chip makers helped the country's export machine by supplying consumer electronics companies with every type of semiconductor imaginable, often at little regard for profits. Much of this was done in-house, as many of today's chip companies started life as divisions of Japanese electronics giants.

Chip sales rose while Japan's consumer electronics were globally dominant, but plunged when the world started buying cheaper televisions, laptop computers and other products made elsewhere in Asia. As losses mounted, many Japanese electronics companies could no longer afford their chip operations and spun them off as separate companies. These new companies lacked the cash to keep pace with the billions of dollars that rivals like Intel and Samsung were spending on new factories and production lines.

Now, many analysts here say, the only way the industry can save itself is by learning from American chip makers like Intel and Texas Instruments, which reinvented themselves two decades ago in response to Japan's strength. These United States companies succeeded by building strong overseas sales networks and concentrating their resources on a small number of products that they made well. Intel focused on building microprocessors, the brains of personal computers, and now dominates the global market. Texas Instruments specialized in chips used in cellphones.

"In the 1980's, the United States figured out a new business strategy," said Toshio Nakajima, president of NEC Electronics, the chip subsidiary of the Japanese electronics giant NEC. NEC fell from being the world's largest chip maker in 1988 to the 10th-largest today. "It is remarkable how these American companies learned to compete."

Mr. Nakajima said his company might eventually focus production on just three types of chips, though it had not decided which three. "It's a big risk to limit yourself to a small number of products. Those have to be very strong products," he said.

Toshiba is doing well focusing production on a specialized product, advanced NAND flash memory chips that are used in digital cameras and music players like the Apple iPod. Toshiba's chip revenues are expected to have grown a healthy 7 percent in 2005, according to iSuppli. (Like most companies, Toshiba does not break out its chip sales figures.)

The picture is not so rosy for the rest of Japan's industry. Of the 20 Japanese chip makers tracked by iSuppli, 12 are expected to report reduced revenues in 2005, including NEC Electronics and Renesas, which was created by the 2003 merger of the chip operations of Hitachi and Mitsubishi Electric.

The Japanese chip makers' problems are not the result of a lack of technology but an overdependence on their home market. Even the three biggest chip makers - Toshiba, Renesas and NEC - still sell about 60 percent of their chips within Japan, according to the Ministry of Economy, Trade and Industry. By contrast, Intel, Samsung and Texas Instruments do about 80 percent of their business outside their home countries.

Another problem is high costs, partly because of outdated and inefficient factories. As sales fell, companies had to cut back on buying new facilities and equipment. In 2002, such spending by all Japanese chip makers totaled 266 billion yen (about $2.3 billion), a third of its level in 1989, according to J. P. Morgan. It is now back up to 741 billion yen ($6.3 billion), still barely enough to keep pace with the $33 billion that Samsung alone plans to spend over the next six years to build nine new semiconductor production lines.

Japan's powerful bureaucrats, who originally helped guide the industry to preeminence, have been urging companies to pool money and technology, with limited success. They originally pressed the largest half-dozen companies to cooperate in building the Rising Sun semiconductor factory, which could cost as much as $3 billion. But the effort was delayed for years as companies failed to agree on what kind of chips Japan should focus its resources on. In the end, just three companies announced that they would join the project.

"Japanese companies have been looking hard for a winning strategy," said Tatsuya Fujiwara, deputy director in charge of the semiconductor industry at the Ministry of Economy, Trade and Industry. "They still haven't found one yet."

Reflexive bubbles

Paul McCulley of PIMCO discusses the feedback mechanism at work in the housing bubble (and seems to call the top). I recommend the whole article (whimsical as it is), which also touches on Bretton Woods II, Asian and OPEC mercantilism, etc. He also makes an interesting observation that bond markets now assume that the Fed has tamed and will continue to tame inflation, and long term rates reflect projections of the real interest rates necessary to do so.

If Fed-engineered changes in nominal rates no longer reflect changes in long-term inflationary expectations, but rather changes in real rates, then logic suggests that long-dated income-producing assets – bonds, stocks and real estate – will become inherently more prone to bubble and burst.

Accordingly, it seems to me, asset prices inevitably must take on a higher priority in the Fed’s reaction function than during the War Against Inflation. As long as inflation was too high for the Fed’s secular taste, bubbles and their bursting didn’t carry grave harm. They misallocated resources, to be sure, like all good Austrians properly preach.

...when home price appreciation is running higher than mortgage rates, the market booms, if not bubbles, as momentum players chase the market higher, a text book example of what George Soros calls reflexive demand. But once the momentum breaks – and again, Morgan, declining affordability is the fundamental break – reflexive demand becomes reflexive supply, as former speculative buyers become eager sellers.

Reflexive markets – and property is one if there ever was one – inherently tend to have V-shaped tops, not rolling tops. Thus, both volumes in total home sales, particularly existing home sales, and MEW are set to fall sharply in the year head. Not the stuff of recession, I hasten to add, Morgan, but clearly the stuff of a serious slowdown in consumer spending.

MLF: Won’t that be a problem for foreign members of the BW II arrangement? Didn’t you say that they are as addicted to our spending as we are to their financing of our spending?

PM: Yes, Morgan, I did say that. Which implies that when the American property market comes off the boil, maybe turning tepid, the world will feel the impact, not just American homeowners. Such is the nature of globalization, when the ex-USA world has a shortage of aggregate demand or, put differently, runs a surplus of savings relative to desired domestic investment.

Orwellian Internet spying

Some good blog posts on this subject: Bruce Schneier (security expert and author of Applied Cryptography) and some legal analysis by Dan Solove. Quick summary: probably in violation of existing FISA laws, probably not explicitly unconstitutional under the 4th Amendment, but Bush's justification under Article 2 is weak, bordering on "frivolous" according to one legal scholar.

John Yoo's memo (the wishful thinking of a thirty-something junior DOJ official, who nevertheless was the strongest intellectual voice on this subject in the administration?!#?) comes close to asserting that there are no limits on Presidential power in the unending "war" on terror. How will we decide when this "war" is over? Was there ever even a declaration of war? Several scholars assert that the 9/14/01 Congressional authorization for use of force falls far, far short of a declaration of war. It is miles away from the declarations of war in WWI and WWII (and the threat is much less serious, as I pointed out in the previous post). If one defines a dictator as a ruler who is beyond the law, then Yoo's memo seeks to justify dictatorial power for Bush-Cheney.


Nineteen Eighty-Four

“Remember our boys on the Malabar front! And the sailors in the Floating Fortresses! Just think what they have to put up with.”

“The rocket bombs which fell daily on London were probably fired by the government of Oceania itself, 'just to keep the people frightened'.”

“To tell deliberate lies while genuinely believing in them, to forget any fact that has become inconvenient, and then when it becomes necessary again, to draw it back from oblivion for just so long as it is needed.”

WAR IS PEACE

FREEDOM IS SLAVERY

IGNORANCE IS STRENGTH

I told you so...

Confirmation of massive, illegal violation of privacy rights of US citizens through telco and Internet monitoring, authorized by the "war preznit". The legal position of this administration (I am not kidding) seems to be that the "war preznit" is free to disregard existing US law in "times of war" like these. If Congress pushes the issue, as it should, we may end up with a full-blown constitutional crisis.

Bush is no more a war president than Ronald Reagan or Jimmy Carter, who faced down a much more formidable foreign adversary. You might argue that Al Qaeda is more dangerous than the USSR and eastern bloc, with their hundreds of ICBMs and thousands of nuclear warheads, but you'd be crazy. Let me offer the following analogy. While walking home you are confronted by a man with a loaded shotgun. By staring him down and pointing out that you yourself are armed, you avoid having your head blown off. Continuing on your way home, a small dog bites your ankle. Is the dog really a greater threat, just because it bit you, than the guy with the shotgun? If not, why should we allow Bush to unilaterally claim greater security powers than Reagan or Carter had? (Indeed, contravening the existing FISA law of 1978.)

In an op-ed article published Friday in The Washington Post, Mr. Daschle said he rejected a White House effort three days after the attacks to grant Mr. Bush specific authority to conduct antiterrorism operations within the United States as part of a broader resolution backing the use of force.

In seeking the specific authority for a domestic response, Mr. Daschle said, the White House was effectively acknowledging that the resolution did not cover domestic actions like spying on Americans.

"The Bush administration now argues those powers were inherently contained in the resolution adopted by Congress - but at the time, the administration clearly felt they weren't or it wouldn't have tried to insert the additional language," Mr. Daschle said in the article.

The White House has asserted that the resolution, adopted by Congress on Sept. 14, 2001, freed Mr. Bush from the requirement to get warrants to monitor international phone calls and e-mail of Americans and others in the United States. That resolution authorized the president to employ "all necessary and appropriate force" in response to the attacks on New York and the Pentagon.

But by Mr. Daschle's new account, which appears not to have been made public previously, the White House sought within minutes before the vote on the resolution to alter it to include new wording specifically granting power to carry out the antiterrorism campaign within the United States.

The White House, Mr. Daschle said, wanted the resolution to give Mr. Bush authority to use "all necessary and appropriate force in the United States and against those nations, organizations and persons" responsible for the attacks.

Mr. Daschle said he had turned aside the White House's effort to include "in the United States and" in that sentence, leaving the focus of the resolution on fighting terrorism abroad.

China tech and innovation

Nice discussion (Talk of the Nation; sorry no podcast) with John Seely Brown (former director of Xerox PARC) on innovation and technology in China. Brown has obviously spent some time with startups over there, particular in the cellphone sector, where he claims they are ahead of the US and Europe.

This is the kind of media I can still consume while grappling with two babies ;-)

Sniffing around

It's very plausible to me that the latest Bush perfidy - secretly approving widespread spying by the NSA on US citizens in the wake of 9/11 - is not about phone wiretapping at all. It's about large scale monitoring of Internet communications.

The press has pointed out repeatedly that the 1978 FISA law allows the government to wiretap on short notice, even to wiretap first and then ask a FISA court for permission retroactively. So, the Bush administration's claim that existing legal requirements slowed things down and "jeopardized national security" is just wrong, and informed people know this.

I suspect what is really going on is that in the wake of 9/11 Bush authorized large scale monitoring of Internet communications, probably by allowing NSA to tap into the backbone. There was a lot of discussion of similar programs, such as Poindexter's Total Information Awareness (TIA) system. Technically, it would not be hard to sample Internet traffic, looking for email or Web activity with certain key words or patterns. However, 100% coverage is probably beyond anyone's capability at the moment.

The problem with this is that you only catch dumb terrorists (maybe that's good enough). As I pointed out before, even widely available communication tools like Skype allow for unbreakable encrypted communication. Don't all terrorists, even ones who don't understand how the Internet works, simply assume that it is being monitored (at least in some weak way)? If so, how can Bush claim that whistleblowers jeopardized national security by leaking information about this illegal program?

BTW, good thing Rockefeller (Senate Intel. Cmmte.) kept a copy of the letter he sent to Cheney. The Bushies claim (lying again) that they got congressional approval, conveniently leaving out that Rockefeller protested immediately about the legality of the program (as did Daschle, who claims the briefings may also have been technically misleading).

Note Added: Administration officials are very careful to state that only "communications" between the US and foreign countries are being monitored. I suppose this means that if your packets don't leave the US, they aren't sniffed. However, all Blackberry users should be aware that their email likely travels through servers in Canada, so is potentially subject to monitoring :-) This Times article seems to confirm that email is intercepted.

TalkLeft: Why do Gonzales and Condi Rice keep mentioning the "technical" aspects of the program as a dodge around FISA?

Why this seemingly inconsequential parsing by Bush of the difference between "monitoring and detection"? Bush says they use FISA if they're monitoring, but this is about "detection."

Why, in his letter, does Rockefeller state that he's "not a technician."?
Why the mention of TIA in Rockefeller's letter?
And why the mention of "large batches of numbers all at once"?

Why?

These are not phone numbers we're talking about...These are IP addresses, email addresses.

A system is in place that basically filters on certain triggers (text, phoneme, etc.) within Internet "conversations." This is "detection" or at least its tortured definition that was placed in this idiot Bush's mind. "Monitoring" would be recording an entire conversation, like in a phone conversation.

That system then collects information on those conversations including...ta da...source and destination IP addresses. Those IP addresses can then be stored for further investigation on other "conversations."

The next wave in India outsourcing

Nice summary from the economist. The point that legal services, like accounting, can be outsourced rings particularly true to me. Most legal services (from IP to contracts to employment law) that we consume as a startup are delivered over the phone or Internet, at exorbitant prices ($300 per hour is what you are charged for an associate; a partner bills even more). The effectiveness of the attorney is largely driven by raw brainpower, once some basic knowledge is acquired. I can easily see a model with a US-based partner and half or more of his or her staff (including associates) based in India. $30 per hour would be very good compensation for a highly intelligent Indian attorney.

Should we be concerned for the future of our young friends in law school? See here for related discussion.

ASKED for a sound-check at a function in Delhi this month, Bill Gates eschewed the “1,2,3...” favoured by ordinary mortals. “One billion, 2 billion...,” he counted. They think big, these IT moguls, and especially, these days, in India.

Microsoft later announced plans to invest $1.7 billion in India over the next four years, about half of it in adding to its existing research and development (R&D) and technical-support operations. “The only thing that limits us in India,” Mr Gates told the local press, “is the speed at which we can recruit.” A few days earlier, Intel, a giant chipmaker, had unveiled plans to invest more than $1 billion over five years, much of it in expanding its R&D centre in Bangalore. In October Cisco Systems, the world's largest maker of the routers and switches that direct internet traffic, announced its own plans to invest $1.1 billion in India.

The euphoria is confined neither to American multinationals, nor to information technology. It also encompasses India's own IT industries, and the expanding range of other back-office services that can now be performed remotely. The three biggest Indian IT-services firms—Tata Consultancy Services (TCS), Infosys and Wipro—are each recruiting more than 1,000 people a month. And to take just one example of the other services now moving to India, J.P. Morgan Chase, a big investment bank, this month revealed it is to double, to about 9,000, its staff there. Anyone who assumes J.P. Morgan will simply be doing low-level “back office” tasks in the country—a bit of data entry and paper-shuffling—would be flat wrong. One task for the new recruits is to settle complex structured-finance and derivative deals, what one insider calls “some of the most sophisticated transactions in the world”.

All these investments illustrate that a third stage of the great Indian services-export boom is well underway. In the first, firms such as TCS developed world-class expertise in software “application development and maintenance”, and their low-cost developers became the preferred partners of many Western IT firms. In the second, Indian firms and the local “captive” operations of multinationals started offering low-end back-office services that could take place a continent away—telephone call-centres, transcribing medical records, processing insurance claims and so on. In the third, in both IT and the broader spectrum of other “business processes”, ever-more sophisticated functions are happening in India.

So strong are the forces driving this shift that what seemed improbably rosy projections by NASSCOM, the Indian software- and service-industry lobby, and McKinsey, a consultancy, back in 1999, are coming true. This week NASSCOM and McKinsey produced the second full-scale update of their study. It argues that exports from India's IT industry and from “Business Process Offshoring” (BPO)—both from services “outsourced” to Indian firms and those performed by captives—are on track to reach $60 billion a year by 2010.

That would be a huge surge from the $17.2 billion in the year ending in March 2005. But it implies a compounded annual growth rate of 28%—below that achieved in recent years. Moreover, according to McKinsey's estimates, it requires India merely to maintain its present shares of the markets for offshore IT services (65%) and BPO (46%). This is because the study predicts a massive rise in the size of the overall market, estimated at present to make up just one-tenth of those services that could be sent offshore. The proportion is expected to rise as demography—a western labour shortage—becomes more pressing than protectionism.

In IT the growth in Indian exports is expected to come both from the software market, and from “traditional IT outsourcing”—such as the remote management of whole systems, a market now dominated by the big global IT consultancies. This is expected to rise from 8% of Indian sales now to about 30% in 2010, while software-development's share will fall from 55% to 39%. In business-process-offshoring, the big industries will remain banking and insurance. But rapid expansion is also expected in other areas, like legal services.

The law, in fact, illustrates how vast is the untapped potential market. About $250 billion is spent on legal services world-wide, about two-thirds of it in America, and as yet only a tiny proportion goes offshore. Forrester, a research outfit, has estimated that, by last year, 12,000 legal jobs had moved offshore, and forecast that this will increase to 35,000 by 2010. India, with its English-language skills and common-law tradition is well-placed to secure a big share of the business. It is not just a question of “paralegal” hack work such as document-preparation. Sanjay Kamlani, of Pangea3, a small Indian firm, calls it “real lawyering”—drafting contracts and patent applications, research and negotiation. His clients are both big law firms and in-house legal teams.

India's fundamental attraction has not changed since it first drew software developers: fantastic cost savings. With American lawyers costing $300 an hour or more, Indian firms can cut bills by 75%. Across the board, despite climbing rates of pay in IT and BPO, where rapid expansion has brought frantic job-hopping, India remains, say NASSCOM and McKinsey, the lowest-cost of all the main outsourcing destinations. It also has, among these countries, by far, the largest pool of employable people—those with the necessary language and technical skills. On this measure, India, which produces 2.5m graduates a year, 250,000 of whom are engineers, has 28% of the global available workforce, compared with 11% in China.

Yet the supply of talent may be the biggest constraint on the Indian industry's growth. On these latest projections, the number of people working in IT and business-process exports in India will increase from about 700,000 now to 2.3m by 2010. But on today's estimates only 1.05m suitably qualified people will graduate from college between now and then, so there will be a shortfall of nearly 500,000, with business-processing the worst affected. McKinsey's Jayant Sinha believes the education system can be fixed in time to plug the gap. A bigger worry, he says, is India's creaking urban infrastructure. IT firms in Bangalore, for example, are in revolt against the local government for its neglect of basic amenities. Yet India's IT and business-process industries will need about 14m square metres (150m square feet) of office space by 2010: “a new Manhattan”.

Hectic building is under way, and not just in the big IT and business-processing centres (Bangalore, Mumbai and around Delhi) or the “second tier” of cities such as Pune, Hyderabad and Chennai (Madras). The industry's worry over infrastructure, as over education, is that it cannot do everything by itself. Having thrived by keeping government at arm's length, business now needs help.

Workplace discrimination

From a recent Gallup survey, as reported by the EEOC.

The Gallup data indicate that 15% of all workers perceived that they had been subjected to some sort of discriminatory or unfair treatment. When broken down into sub-groups, 31% of Asians surveyed reported incidents of discrimination, the largest percentage of any ethnic group, with African Americans constituting the second largest group at 26%.

This finding is probably only surprising to white Americans. Asian-Americans know very well that they are subject to stereotyping (albeit perhaps of the unconscious type) and often passed over for leadership roles. Thanks to Confucian values, they are also less likely to complain about it. You won't hear them complaining about being shortchanged by affirmative action, either.

Within the context of race filings, 82.5% of charges were brought by African Americans, with Asian/Pacific Islanders filing only 3% -- a sharp contrast with the 30% of Asians employees who responding to the Gallup survey that they perceived discrimination on-the-job.

Wall Street

Wall Street (1987). Directed by Oliver Stone, starring Michael Douglas (Gekko), Martin and Charlie Sheen (father and son Carl and Bud Fox). A classic that fully anticipated our age of hyper-finance. Gekko and company look tame compared to LTCM, Enron and our current hedge fund overlords!

Gekko: Greed, for lack of a better word, is good. Greed is right. Greed works. Greed clarifies and cuts through and captures the essence of evolutionary spirit. Greed in all of its forms, greed for life, for money, for love, knowledge has marked the upward surge of mankind. And greed, you mark my words, will not only save Teldar Paper, but that other malfunctioning corporation called the USA.

Gekko: The richest one percent of this country owns half our country's wealth, five trillion dollars. One third of that comes from hard work, two thirds comes from inheritance, interest on interest accumulating to widows and idiot sons and what I do, stock and real estate speculation. It's bullshit. You got ninety percent of the American public out there with little or no net worth. I create nothing. I own.

Carl Fox: Stop going for the easy buck and start producing something with your life. Create, instead of living off the buying and selling of others.

Personally, I followed the Create path, but I'm not sure I came out ahead :-)

The view from the PBOC

From an interview with Yu Yongding, an Oxford-trained economist who sits on the Monetary Policy Committee of the People's Bank of China. Who can doubt that everyone sees the handwriting on the wall? He clearly understands the inevitability of dollar devaluation, the need for Asian countries to dump those dollars, the prisoner's dilemma problem that central banks of China, Taiwan, Japan and Korea have, etc. Barring a disaster in Asia (China invades Taiwan, N-S Korea conflict, etc.), I predict a 2x devaluation of the dollar against E. Asian currencies over the next decade. Sorry that prediction is not precise enough to trade on :-)

(See here for previous posts on Bretton Woods II and dollar devaluation.)

"...in the first stage we must reduce accumulation, then later we should reduce our reserves....[China and Asian countries] don't need that large an amount- more than $2 trillion- of foreign exchange reserves.... This is a very big problem and I think the Chinese government should take some action to reduce the growth rate of the accumulation of foreign exchange reserves as we're still facing the possibility of a big devaluation of the US dollar, so the capital losses will be huge. If that happens, it will be tremendous hit to the Chinese economy."

This is hardly the statement of a gentleman with a benign view toward the US dollar's valuation. It is instead a gentleman, in a position of authority, with a great deal of concern. He went on: "The trouble is, with such a huge amount of foreign exchange reserves, that there is no way to spend it very quickly and there's no plan to sell it of course-- otherwise that inflicts damage on ourselves. You don't want to dump shares when the stock market has not collapsed yet and you are the biggest shareholder." Then, he said "all east Asian countries have tremendous foreign exchange reserves and they all want to get rid of them, but if you do this then you cause competitive devaluation, not of their own currencies, but of the US dollar. So we should do this in an orderly fashion. If Asian countries moved too fast, everyone would lose... It would be utterly unfortunate if Japan sells a proportion [of their reserves, for] that causes problems. Then China panics and China sells a proportion -- it would be very damaging."

The "nicest possibility" for China, Japan and the US to escape this problem was for further "tightening of US monetary policy so that further dramatic devaluation of the US dollar can be stopped. Then, because of the slowdown in the economy, the US current account deficit would reduce and in this way will create conditions for East Asian countries to get off the hook."

2 percent of the Caltech class

A friend of mine who runs a derivatives desk made job offers to 2 percent of the Caltech class of 2006. That's only 4 people, but still a pretty high percentage! 3 have accepted, the other went to a well-known software company. It makes me wonder how many kids who otherwise would have gone on to grad school (e.g., in engineering, math or physics) are heading right into finance these days. Smart move, if you ask me :-)

Most of my colleagues still don't take the theory of modern finance seriously at all. Even researchers who work on complex systems (modeling traffic, sandpiles, networks, ants, etc.) show surprisingly little interest. I guess the future belongs to the young!

Physics still pulls them in...

There is much concern among physicists that our field is no longer getting the same share of the brightest students as in the glory days (e.g., during the cold war). I was pleased to find here that average GRE-general scores in physics PhD programs were still the highest as of 2002, with math and CS close behind. According to these tables, the average IQ of a physics PhD student is about 130. For comparison, english/literature PhD students average about 120 and sociology PhD students only 115. (115 is the average for college students in general, so the typical sociology TA is no smarter than the students in his or her class ;-) Amazingly, the average grad student in education (109) is below the average for college students!

You may find it odd that I converted the results from GRE score to IQ, widely regarded by the politically correct as a discredited measure of intelligence. For a rebuttal of this often repeated, but scientifically unsupported, point of view, see, e.g., Steve Pinker's book The Blank Slate, or any academic research of the type excerpted below.

g on the Job


Although the evidence of genetic and physiological correlates of g argues powerfully for the existence of global intelligence, it has not quelled the critics of intelligence testing. These skeptics argue that even if such a global entity exists, it has no intrinsic functional value and becomes important only to the extent that people treat it as such: for example, by using IQ scores to sort, label and assign students and employees. Such concerns over the proper use of mental tests have prompted a great deal of research in recent decades. This research shows that although IQ tests can indeed be misused, they measure a capability that does in fact affect many kinds of performance and many life outcomes, independent of the tests' interpretations or applications. Moreover, the research shows that intelligence tests measure the capability equally well for all native-born English-speaking groups in the U.S.

If we consider that intelligence manifests itself in everyday life as the ability to deal with complexity, then it is easy to see why it has great functional or practical importance. Children, for example, are regularly exposed to complex tasks once they begin school. Schooling requires above all that students learn, solve problems and think abstractly. That IQ is quite a good predictor of differences in educational achievement is therefore not surprising. When scores on both IQ and standardized achievement tests in different subjects are averaged over several years, the two averages correlate as highly as different IQ tests from the same individual do. High-ability students also master material at many times the rate of their low-ability peers. Many investigations have helped quantify this discrepancy. For example, a 1969 study done for the U.S. Army by the Human Resources Research Office found that enlistees in the bottom fifth of the ability distribution required two to six times as many teaching trials and prompts as did their higher-ability peers to attain minimal proficiency in rifle assembly, monitoring signals, combat plotting and other basic military tasks. Similarly, in school settings the ratio of learning rates between "fast" and "slow" students is typically five to one.

...The existence of biological correlates of intelligence does not necessarily mean that intelligence is dictated by genes. Decades of genetics research have shown, however, that people are born with different hereditary potentials for intelligence and that these genetic endowments are responsible for much of the variation in mental ability among individuals. Last spring an international team of scientists headed by Robert Plomin of the Institute of Psychiatry in London announced the discovery of the first gene linked to intelligence. Of course, genes have their effects only in interaction with environments, partly by enhancing an individual's exposure or sensitivity to formative experiences. Differences in general intelligence, whether measured as IQ or, more accurately, as g are both genetic and environmental in origin--just as are all other psychological traits and attitudes studied so far, including personality, vocational interests and societal attitudes. This is old news among the experts. The experts have, however, been startled by more recent discoveries.

One is that the heritability of IQ rises with age--that is to say, the extent to which genetics accounts for differences in IQ among individuals increases as people get older. Studies comparing identical and fraternal twins, published in the past decade by a group led by Thomas J. Bouchard, Jr., of the University of Minnesota and other scholars, show that about 40 percent of IQ differences among preschoolers stems from genetic differences but that heritability rises to 60 percent by adolescence and to 80 percent by late adulthood. With age, differences among individuals in their developed intelligence come to mirror more closely their genetic differences. It appears that the effects of environment on intelligence fade rather than grow with time. In hindsight, perhaps this should have come as no surprise. Young children have the circumstances of their lives imposed on them by parents, schools and other agents of society, but as people get older they become more independent and tend to seek out the life niches that are most congenial to their genetic proclivities.

A second big surprise for intelligence experts was the discovery that environments shared by siblings have little to do with IQ. Many people still mistakenly believe that social, psychological and economic differences among families create lasting and marked differences in IQ. Behavioral geneticists refer to such environmental effects as "shared" because they are common to siblings who grow up together. Research has shown that although shared environments do have a modest influence on IQ in childhood, their effects dissipate by adolescence. The IQs of adopted children, for example, lose all resemblance to those of their adoptive family members and become more like the IQs of the biological parents they have never known. Such findings suggest that siblings either do not share influential aspects of the rearing environment or do not experience them in the same way. Much behavioral genetics research currently focuses on the still mysterious processes by which environments make members of a household less alike.

Scientific method at Google

Great podcast of an interview with Google busdev VP Omid Kordestani. Kordestani is credited with coming up with the AdSense business model (although it originally came from Overture, and another person gets the credit in a recent NYT article discussed here). There was also a recent WSJ article in which Kordestani is described as one of the Googlers driving up real estate prices in Atherton -- in his case by buying a 16,000 sq ft house for $17.8 million!

What is interesting about the interview is Kordestani's continued references to the scientific methods applied within Google to tasks as varied as screening job applicants, rooting out click fraud, monitoring (and modeling) AdSense auctions, etc. He uses the phrase "teams of PhDs" several times :-) It is amazing to hear a busdev guy talk this way, even considering that Kordestani has an engineering background.

Expert predictions

How good are "experts" at making accurate predictions? Much worse than you think, says psychology professor Philip Tetlock (Haas School of Business at UC Berkeley) in his new book Expert Political Judgment: How Good Is It? How Can We Know? (See New Yorker review.) In detailed studies, in which "experts" were asked to make forecasts about the future (predicting which of three possible futures would occur), it was found that the "experts" did no better than well-informed non-experts! As Tetlock says, “We reach the point of diminishing marginal predictive returns for knowledge disconcertingly quickly,” he reports. “In this age of academic hyperspecialization, there is no reason for supposing that contributors to top journals—distinguished political scientists, area study specialists, economists, and so on—are any better than journalists or attentive readers of the New York Times in ‘reading’ emerging situations.”

Now, I expect the performance of scientific experts to be somewhat better. Questions like "How hot will that spacecraft get while in orbit around Mercury?" or "How many CPU cycles will it take to compute that integral?" are ones where predictions of real experts will far outperform those of lay people. I guess there is something fundamentally different about scientific versus non-scientific expertise? The last bit below about predicting freshman academic performance is amazing (but not unexpected). Let a simple one or two parameter model pick your freshman class :-)

Finally, what type of "expert" would you trust to run your money (make investment predictions)? As Jim Simons said: "The advantage scientists bring into the game is less their mathematical or computational skills than their ability to think scientifically. They are less likely to accept an apparent winning strategy that might be a mere statistical fluke." In other words, they know when they know something, while others might just be fooling themselves ;-)

New Yorker: Tetlock is a psychologist—he teaches at Berkeley—and his conclusions are based on a long-term study that he began twenty years ago. He picked two hundred and eighty-four people who made their living “commenting or offering advice on political and economic trends,” and he started asking them to assess the probability that various things would or would not come to pass, both in the areas of the world in which they specialized and in areas about which they were not expert. Would there be a nonviolent end to apartheid in South Africa? Would Gorbachev be ousted in a coup? Would the United States go to war in the Persian Gulf? Would Canada disintegrate? (Many experts believed that it would, on the ground that Quebec would succeed in seceding.) And so on. By the end of the study, in 2003, the experts had made 82,361 forecasts.

...Tetlock got a statistical handle on his task by putting most of the forecasting questions into a “three possible futures” form. The respondents were asked to rate the probability of three alternative outcomes: the persistence of the status quo, more of something (political freedom, economic growth), or less of something (repression, recession). And he measured his experts on two dimensions: how good they were at guessing probabilities (did all the things they said had an x per cent chance of happening happen x per cent of the time?), and how accurate they were at predicting specific outcomes. The results were unimpressive. On the first scale, the experts performed worse than they would have if they had simply assigned an equal probability to all three outcomes—if they had given each possible future a thirty-three-per-cent chance of occurring. Human beings who spend their lives studying the state of the world, in other words, are poorer forecasters than dart-throwing monkeys, who would have distributed their picks evenly over the three choices.

Tetlock also found that specialists are not significantly more reliable than non-specialists in guessing what is going to happen in the region they study.

...“Expert Political Judgment” is just one of more than a hundred studies that have pitted experts against statistical or actuarial formulas, and in almost all of those studies the people either do no better than the formulas or do worse. In one study, college counsellors were given information about a group of high-school students and asked to predict their freshman grades in college. The counsellors had access to test scores, grades, the results of personality and vocational tests, and personal statements from the students, whom they were also permitted to interview. Predictions that were produced by a formula using just test scores and grades were more accurate.

Postdoc (no)bargain

This was posted as a comment, but I thought it so good I should share it here.

Note if there is almost one postdoc per permanent researcher (or professor), it means that one 3-5 year cohort of PhDs could replace the entire pool of career researchers. If a career lasts 30 years it means there is a six or ten to one ratio between postdocs that leave the field and those who survive with permanent positions. Even if the postdoc to professor ratio is lower (as in physical science and math -- this number is smaller largely because they don't have much funding for postdocs in math), there may still be a big ratio between PhDs produced and number of professors. Unless the average professor produces no more than one PhD per 30 years, there will continue to be a big oversupply and a tough labor market for scientists.

See related posts here and here.

Thanks for the Great Postdoc Bargain
Richard Freeman
30 August 2002

Postdocs are at the heart of the United States?s extraordinarily successful biological and life scientific research program over the past two-plus decades. In this period, postdocs have produced most of the results in academic laboratories and have come to play an increasing role in industrial and government labs as well. Academic institutions, which engage some 80% of postdocs, aren?t sure whether postdocs are employees, students, or some form of apprentice. With responsibility for hiring and career development resting firmly with the principle investigators who employ the postdocs on their research grants, many universities don?t even know how many postdocs they have or what they are paid, much less how they are progressing toward ? whatever the future holds for them.

Whatever they are, however, postdocs are one of the greatest bargains in the U.S. economy. Where else can one hire Ph.D.s, whose training and smarts put them among the best and brightest in the world, to work 60 hours a week for $30,000 to 40,000 a year, with limited benefits and little power to influence their working conditions and pay? Given the long hours that postdocs work, their hourly pay is on the order of $10 to $13 per hour--on par with the wages paid to custodial and other low-paid workers that have spurred living wage campaigns around the country.


...Huzzah! Huzzah!

Two to three decades ago, the U.S. rewarded postdocs with a reasonably good chance of being hired as a principal investigator. Sorry, but we can no longer carry out that part of the bargain. As Table 1 shows, there are just too many postdocs for us to absorb them as tenured faculty. In 1987, the ratio of postdocs to tenured faculty was already too high at 0.54 for most to obtain faculty jobs at the rate of growth of academic employment. By 1997, the ratio had risen by 43% to 0.77. It has presumably risen further since then. As a result--and as many postdocs have learned to their chagrin--the U.S. does not have a place for them on standard academic tracks.

But don?t get discouraged, postdocs. We need you for our research. How about another postdoctorate--a few more years of long hours at low wages?

Table 1: Ratio of the Number of Postdoctorates in Higher Educational Institutions to the Number of Tenured Faculty, 1987 and 1997

Discipline
1987
1997
% Change

Life Sciences
0.54
0.77
43%

Physical Sciences and Mathematics
0.20
0.23
15%

Engineering
0.11
0.19
73%

Source: The National Academies Committee on Science, Engineering, and Public Policy Enhancing the Postdoctoral Experience for Scientists and Engineers, (NAS, 2000), table B-1 and table B-14.


The forces of supply and demand are unlikely to improve the economic situation of postdocs in any plausible time period. One reason is that the supply of postdocs consists not only of U.S. citizens and permanent residents gaining Ph.D.s but also of U.S.- and foreign-trained Ph.D.s from other countries. Indeed, U.S. scientific research could not proceed at anything like its current pace was it not for the influx of foreign postdocs. Roughly half of postdocs currently come from overseas, many from countries with low personal income rates such as China. Remove foreign postdocs from the nation?s labs and postdoc pay would zoom ... at the cost of short-term chaos and a long-term slower rate of scientific progress. Nevertheless, we should not forget to thank our foreign postdocs for their long hours and hard work on behalf of the rest of our society...
--
Posted by scienceguy11 to Information Processing at 12/05/2005 11:37:13 AM

Is gravity ergodic?

New paper! (Available on arxiv.org.)

The universe is very far from maximizing its entropy. Most egregious is the gravitational state of the universe, as noted years ago by Penrose. Gravity, being a long range, unscreened, attractive force, maximizes its entropy by clumping. This is quite unlike systems with ordinary interactions (generally of limited range, or subject to screening), whose entropy is maximized when matter and energy are uniformly distributed. In the case of gravity the most entropically dense configurations are black holes, with exp(A) microstates (A is the area in Planck units). Why did the big bang begin with a smooth background metric, instead of an agglomeration of black holes, when the latter has much higher entropy and hence occupies an exponentially larger portion of phase space? We explore how a phase transition between the smooth, low entropy spacetimes and the high entropy black hole phase might occur.

Note added: After we posted the paper we discovered calculations using Euclidean path integrals (gravitational instantons) which yield the same results for the black hole free energy and nucleation rate as we obtain from our simple intuitive arguments.


Thermal gravity, black holes and cosmological entropy

Authors: Stephen D. H. Hsu, Brian M. Murray

Taking seriously the interpretation of black hole entropy as the logarithm of the number of microstates, we argue that thermal gravitons may undergo a phase transition to a kind of black hole condensate. The phase transition proceeds via nucleation of black holes at a rate governed by a saddlepoint configuration whose free energy is of order the inverse temperature in Planck units. Whether the universe remains in a low entropy state as opposed to the high entropy black hole condensate depends sensitively on its thermal history. Our results may clarify an old observation of Penrose regarding the very low entropy state of the universe.

Vernor Vinge talk

Nice podcast of a talk given at Accelerating Change 2005. Vinge is a mathematician and sci fi writer who popularized the term Singularity (originally due to Ulam?) to describe the rapid acceleration of technological change that might accompany the development of machine intelligence beyond our own.

Previously I discussed the impending singularity, the relation to the Turing test and AI for kids.

Platform companies

Economist Brad Setser has been discussing platform companies as the future for US industry. The idea is that high value-add activities such as design, engineering and management are performed here, and low value-add manufacturing is done abroad. A good example of this is the iPod, which was designed in Cupertino but is manufactured in Taiwan. Apple sold $1.2 billion worth of iPods in Q1 2005, making iPods almost as big as their computer business! Apple's margin on iPods is healthy - estimated at up to 20% -- while I am sure the manufacturer's margin is very slim. So, the conventional wisdom goes, let those low cost countries have the cutthroat manufacturing business. We clever Americans will simply go upstream, creating new products and markets through innovation.

There is a problem with this, and it is related to the winners and losers issue that arises in the usual story of comparative advantage. Overall, societies may enjoy a net benefit from trade (heck, there are even "theorems" going back to Ricardo "proving" this), but even in the optimistic cases the benefits (and losses) will be unevenly distributed. As autoworkers are displaced by Delphi moving production to China, investment bankers or consultants in the US benefit from demand for globalization strategists. But who ensures that some of the benefits are redistributed? Of course we all benefit from lower prices for manufactured goods, but that is only partial recompense to an unemployed auto worker.

I would go even further, and guess that only a small fraction of the population in advanced countries has the cognitive capabilities to be on the winning end of this process. That is, the Apple iPod designers, investment bankers, CEOs, etc. who win at this game are very able -- in terms of conventional IQ or special abilities such as leadership, design creativity, etc. It is questionable whether the average person will ever be able to contribute on a "high value-add" team! Instead, they'll be caught in a process taking place over a generation in which low-skill wage costs equilibrate worldwide. The theorems might still be satisfied by the huge returns to very able people (whether in China, India, Russia or the US), but the trend for the average American might be negative. Instead of an auto worker in Detroit earning enough for a good retirement and college education for their kids, they'll instead make $10-20 per hour, including benefits. (See earlier post Equilibration can hurt.)

Previous discussion of outsourcing here and here.