Fat tails and the cubic law of returns

I came across this nice review article on power law distributions in economic and other contexts. A particularly interesting one is the following, governing short term stock price fluctuations -- surprise, it's not log normal!

6.1.1. The inverse cubic law distribution of stock price fluctuations:

The tail distribution of short-term (15 s to a few days) returns has been analyzed in a series of studies on data sets, with a few thousands of data points (Jansen & de Vries 1991, Lux 1996, Mandelbrot 1963), then with an ever increasing number of data points: Mantegna & Stanley (1995) used 2 million data points, whereas Gopikrishnan et al. (1999) used over 200 million data points. Gopikrishnan et al. (1999) established a strong case for a inverse cubic PL of stock market returns.

...Such a fat-tail PL yields a large number of tail events. Considering that the typical standard daily deviation of a stock is approximately 2%, a 10–standard deviations event is a day in which the stock price moves by at least 20%. From daily experience, the reader can see that those moves are not rare at all: Essentially every week a 10–standard deviations event occurs for one of the (few thousand) stocks in the market.28 The cubic law quantifies that notion and states that a 10–standard deviations event and a 20–standard deviations event are 5^3 = 125 and 10^3 = 1000 times less likely, respectively, than a 2–standard deviations event.

The figure below shows the probability distribution of 15 minute returns on 1000 large company stocks from data taken in 1994-1995. (Click for larger version.)



Here is a figure showing the famous power law scaling of metabolic rate with body mass in animals (click for larger version):

US Track and Field Championships in Eugene

The USATF championships were held here again last week and over the weekend. I run on the same track as these guys, only much slower :-)

The photo below is of Eugenian Nick Symmonds (Oregon Track Club Elite) winning the 800m. See also this panorama shot, which shows the 800m finish and all of Hayward Field.



If you're a track fan, keep your eye on Ashton Eaton, who placed second in the decathalon despite being only a junior at UO (he won the NCAAs earlier in the spring).



Complete results from USATF web site.

Don't cry for Michael

Yes, by the end he had become a freak, perhaps even a monster, a caricature of his younger self. But in 1983, when I was in high school, he really was the king of pop.

I remember how revolutionary MTV and music videos were for kids everywhere, especially in small towns and in states like Iowa.

I caught Michael's famous Motown 25 live performance (below) on television. My brother and I spent hours goofing around trying to figure out how to moonwalk. I was never a huge fan, but you couldn't deny the power of his music or the electricity he generated.



Considering his trajectory in later life, perhaps it was better to go out at 50. Strangely, a song by another departed pop icon comes to mind. Yes, Michael was an odd one, a brother from another planet, a stranger in a strange land. But give him his due -- he did it His Way!

My Way (audio, Frank Sinatra)

And now, the end is near;
And so I face the final curtain.
My friend, I'll say it clear,
I'll state my case, of which I'm certain.

I've lived a life thats full.
I've traveled each and evry highway;
And more, much more than this,
I did it my way.

Regrets, I've had a few;
But then again, too few to mention.
I did what I had to do
And saw it through without exemption.

I planned each charted course;
Each careful step along the byway,
But more, much more than this,
I did it my way.

Yes, there were times, I'm sure you knew
When I bit off more than I could chew.
But through it all, when there was doubt,
I ate it up and spit it out.
I faced it all and I stood tall;
And did it my way.

I've loved, I've laughed and cried.
I've had my fill; my share of losing.
And now, as tears subside,
I find it all so amusing.

To think I did all that;
And may I say - not in a shy way,
No, oh no not me,
I did it my way.

For what is a man, what has he got?
If not himself, then he has naught.
To say the things he truly feels;
And not the words of one who kneels.
The record shows I took the blows -
And did it my way!

Kids these days

Michael Chabon writes in the NY Review of Books. Parents these days are probably the most risk averse, fear driven and statistically innumerate decision makers of all. (Me included!)

Manhood for Amateurs: The Wilderness of Childhood

...Matt Groening once did a great Life in Hell strip that took the form of a map of Bongo's neighborhood. At one end of a street that wound among yards and houses stood Bongo, the little one-eared rabbit boy. At the other stood his mother, about to blow her stack—Bongo was late for dinner again. Between mother and son lay the hazards —labeled angry dogs, roving gang of hooligans, girl with a crush on bongo—of any journey through the Wilderness: deadly animals, antagonistic humans, lures and snares. It captured perfectly the mental maps of their worlds that children endlessly revise and refine. Childhood is a branch of cartography.

The thing that strikes me now when I think about the Wilderness of Childhood is the incredible degree of freedom my parents gave me to adventure there. A very grave, very significant shift in our idea of childhood has occurred since then. The Wilderness of Childhood is gone; the days of adventure are past. ...

...the kind of door-to-door, all-encompassing escort service that we adults have contrived to provide for our children. We schedule their encounters for them, driving them to and from one another's houses so they never get a chance to discover the unexplored lands between. If they are lucky, we send them out to play in the backyard, where they can be safely fenced in and even, in extreme cases, monitored with security cameras. When my family and I moved onto our street in Berkeley, the family next door included a nine-year-old girl; in the house two doors down the other way, there was a nine-year-old boy, her exact contemporary and, like her, a lifelong resident of the street. They had never met.

...But the primary reason for this curtailing of adventure, this closing off of Wilderness, is the increased anxiety we all feel over the abduction of children by strangers; we fear the wolves in the Wilderness. This is not a rational fear; in 1999, for example, according to the Justice Department, the number of abductions by strangers in the United States was 115. Such crimes have always occurred at about the same rate; being a child is exactly no more and no less dangerous than it ever was. What has changed is that the horror is so much better known. [My brainy wife, upon reading this, asked -- if parents are more vigilant, shouldn't the rate of abductions have gone *down* in recent years? If it stayed constant, isn't the world indeed more dangerous for non-vigilant parents?]

...What is the impact of the closing down of the Wilderness on the development of children's imaginations? This is what I worry about the most. I grew up with a freedom, a liberty that now seems breathtaking and almost impossible.

...Art is a form of exploration, of sailing off into the unknown alone, heading for those unmarked places on the map. If children are not permitted—not taught—to be adventurers and explorers as children, what will become of the world of adventure, of stories, of literature itself?

Genetic clustering: 40 years of progress

Represent each individual human by their DNA sequence. When aggregated, they cluster into readily identifiable groups. This has been known for 40 years now, although the technology and methods of analysis continue to improve. Below are results from 1966, 1978 and 2008.

If this seems counterintuitive to you, it might be because the space of genetic variation is of very high dimension. See here for more discussion and an illustration.

(Click images for larger version.)

Population Structure and Human Evolution
L. L. Cavalli-Sforza

Proceedings of the Royal Society of London. Series B, Biological Sciences, Vol. 164, No.995, A Symposium from Mendel's Factors to the Genetic Code (Mar. 22, 1966), pp. 362-379
http://www.jstor.org/stable/pdfplus/75457.pdf




Measurement of Differentiation: Reply to Lewontin, Powell, and Taylor
Jeffry B. Mitton

The American Naturalist, Vol. 112, No. 988 (Nov. - Dec., 1978), pp. 1142-1144
http://www.jstor.org/stable/2460361?origin=JSTOR-
pdf




Current state of the art, as discussed here. Figure: The three clusters shown below are European (top, green + red), Nigerian (light blue) and E. Asian (purple + blue).



According to the mathematical analysis given in this paper, populations with FST as low as .0001 can be resolved with current technology. (Typical FST between northern and southern Europe is about .006, between Europe and E. Asia about .1 and between Europe and Nigeria about .14 .)

Missing giants of modern science

Below is some (lightly edited) correspondence between Vanderbilt cosmologist Robert Scherrer and myself on topics related to my previous post Why are modern scientists so dull?

From: Bob
Subject: why we are all so boring

I enjoyed your posting on why we are all so boring (and I agree with the bulk of it - it is certainly easier for eccentric, brilliant types to thrive in mathematics or theoretical physics than in any other field).

The issue of genius raises an obvious question. If we are cultivating brilliant people more effectively now than at the turn of the 20th century (and I believe we are), so that everyone is brilliant, as opposed to a few outliers, then why did we get the development of quantum mechanics and relativity (or, going back even earlier, classical mechanics and electricity and magnetism) during an era when the level of effort, and the number of "brilliant" scientists, was exponentially smaller? Where are the equivalent breakthroughs of today? Is it possible that the structure of the laws of physics is such that there are basically only a few breakthroughs waiting to happen and easily accessible to an industrialized society, and we've already used them up?

Reply from Steve:

I agree with you that we may have picked a lot of the low hanging fruit. It happened to be the case that in the various "golden ages" of physics technology was available to test the new theories relatively soon after they were proposed, which is not true today. I suspect if we had table top Planck energy accelerators then progress on quantum gravity might have been much faster during our careers. In fact, some people might have revealed themselves as "geniuses" because they looked at the data stream and proposed the correct models, thereby becoming famous instead of obscure scribblers like me :-)

I have an interest in psychology and psychometrics, and have been carefully watching all the "smartest" people I have come across in our field, cross referencing as best I can between older and newer generations. (I'm sure everybody else has too.) I suspect there are plenty of smart guys around today and the old guys weren't as quite spectacular as the glow from their Nobels might suggest -- at least, not when compared to plenty of smart but relatively obscure people of later generations. Sometimes it is better to be lucky than good (an old Wall St. saying)!

More from Bob:

...With regard to the relative level of "smartness", I have an interest in the history of baseball. It is certainly true that an average modern team would completely devastate an "old-time" famous team like the 1927 Yankees, given improvements in nutrition, training, (and pharmaceuticals :) However, I suspect that even if you equalized these things, the modern team might still win, as we are now much more efficient at scouring talent everywhere (the integration of baseball alone is an obvious example).

...It would be interesting to speculate if there are any areas in which we've become LESS efficient in aggregating talent than we were 50 or 100 years ago. Skilled artisans, perhaps? Marksmen?

I'm sure there is a lot of stuff that was more useful in the past than today, and for that reason we don't filter as hard anymore for those talents. But society has gotten richer and more organized, information technology has gotten cheaper, statistical techniques more widely deployed, and in some fields we now have winner-take-all economies. So we're probably overall much, much better at identifying talent, whether the field is tennis, mathematics or even American Idol crooning. I can't think of any old timers who could hang with Usain Bolt!

Why are modern scientists so dull?

On the subject of personality factors and success in science, here is a provocative essay by UK professor Bruce Charlton. (PDF version.) He claims that the modern system selects for conscientiousness over raw intelligence, with negative consequences.

Question: why are so many leading modern scientists so dull and lacking in scientific ambition?

Answer: because the science selection process ruthlessly weeds-out interesting and imaginative people. At each level in education, training and career progression there is a tendency to exclude smart and creative people by preferring Conscientious and Agreeable people. The progressive lengthening of scientific training and the reduced independence of career scientists have tended to deter vocational ‘revolutionary’ scientists in favour of industrious and socially adept individuals better suited to incremental ‘normal’ science. High general intelligence (IQ) is required for revolutionary science. But educational attainment depends on a combination of intelligence and the personality trait of Conscientiousness; and these attributes do not correlate closely.

...At each level in education, training and career progression there is a tendency to exclude smart and creative people by preferring conscientious and sociable people. As science becomes ever-more dominated by ‘peer review’ mechanisms, pro-social behaviour in scientists has been accorded primacy over the brilliant and inspired – but abrasive and rebellious – type of truth-seekers who used to be common among the best scientists.

A majority of senior professional scientists have been through a rigorous and prolonged process of education, selection and training to become professional researchers. Yet the nature of the rigour and the duration of the process in modern science ensures that those who come out at the end and attain long-term scientific employment are not the kind of people capable of top level, revolutionary science. They will very probably be extremely productive and socially compliant, but of only moderately high intelligence and likely to be lacking in imagination [2].

...Modern science is just too dull an activity to attract, retain or promote many of the most intelligent and creative people. In particular the requirement for around 10, 15, even 20 years of postgraduate ‘training’ before even having a chance at doing some independent research of one’s own choosing, is enough to deter almost anyone with a spark of vitality or self-respect; and utterly exclude anyone with an urgent sense of vocation for creative endeavour. Even after a decade or two of ‘training’ the most likely scientific prospect is that of researching a topic determined by the availability of funding rather than scientific importance, or else functioning as a cog in someone else’s research machine. Either way, the scientist will be working on somebody else’s problem – not his own. Why would any serious intellectual wish to aim for such a career? ...

Shorter Charlton: there are too many hoops, and we end up selecting for Agreeableness and Conscientiousness (hoop jumping abilities) rather than raw brainpower.

I partially agree with Charlton's claims, but the specifics vary from field to field. The area he seems most familiar with is medical science, which most physicists (after teaching premeds and biology students) might concur selects for conscientious rather than brilliant types ;-) In physics it seems we are quite tolerant of odd personalities -- hyper aggressive types, those with Asperger's Syndrome, etc., especially if the person in question displays tremendous ability. I would guess the same is largely true in math and engineering. In biology and medicine it may not be that easy to tell the really talented researchers from the rest (at least at early career stages), which would lead to more emphasis on personality traits. It's also true that in many areas of physics (specifically, but not limited to, the theoretical ones) one can work as a single investigator or small group lead investigator quite early. This may be less true in medicine and biology.

I discussed the current incentive system in science here, as well as the job prospects in theoretical physics. Given the situation I can't blame any students who find that alternative careers might be preferable. As I wrote here (in partial agreement with Charlton), this leads to a different kind of selection than in the past:

...Nowadays, success in science seems to be as much a selection for [certain] character or personality traits as it is a selection for talent.

Related posts: frauds , success vs ability .

Regarding Charlton's deeper question: Where have all the geniuses gone? I offer the following from this earlier post. See also Genius, Gleick's biography of Feynman, especially pp.325-328.

... the exact topic discussed in James Gleick's book Genius. In a field where sampling of talents is sparse [like science in its earlier days] ... you might find one giant ... towering above the others, able to do things others cannot. In a well-developed, highly competitive field like modern mathematics, all the top players are "geniuses" in some sense (rare talents, one in a million), even though they don't stand out very much from each other. In Gleick's book, Feynman, discussing how long it might have taken to develop general relativity had Einstein not done it, says "We are not that much smarter than each other"!

To put it simply, if I sample sparsely from a Gaussian distribution, I might find a super-outlier in the resulting set. If I sample densely and have a high minimum cutoff for acceptable points, I will end up with a set entirely composed of outliers, but who do not stand out much from each other. Every guard in the NBA is an athletic freak of nature [and they would destroy their predecessors from the early era of professional basketball], even though they are evenly matched when playing against each other.

Happiness

In the previous post I mentioned my scores on this Big Five personality test. Someone immediately doubted whether I (or any theoretical physicist) could really have scored at the 99th percentile for Stability (opposite of Neuroticism). Upon further reflection, I find the result a little puzzling as well!

One contributing factor I can point to is that I've been thinking about the problem of happiness and the hedonic treadmill for some time.

It's also true that my father passed away while I was still fairly young, so I had the impetus to consider his life in its entirety and to evaluate which of the things he did really mattered, and which didn't.

If you're interested in optimizing your own life satisfaction, I recommend the Happiness Project blog, written by Gretchen Rubin (she is Robert Rubin's daughter in law; I once worked with her husband who was at the time an investment banker). I especially recommend her short movie The Years Are Short (it's only a minute or so long) to any parent with small kids.

I guess I would describe myself as something of a stoic. My favorite bit of advice for academics comes from...

Marcus Aurelius

"Or does the bubble reputation distract you? Keep before your eyes the swift onset of oblivion, and the abysses of eternity before us and behind; mark how hollow are the echoes of applause, how fickle and undiscerning the judgments of professed admirers, and how puny the arena of human fame. For the entire earth is but a point, and the place of our own habitation but a minute corner in it; and how many are therein who will praise you, and what sort of men are they?"

Spent, Miller and Kanazawa

I was wandering in the bookstore today and looked through a copy of Geoffrey Miller's book Spent: Sex, Evolution and Human Behavior. (I am a cheapskate, and I read pretty fast, so I often skim through entire books at the bookstore.) If you like evolutionary psychology, you will probably enjoy the book, which is well written and covers some novel topics. One point emphasized by Miller throughout his career is that the human brain evolved not just to survive, but to survive in competition against other human brains, and particularly in the context of sexual selection. Working out the implications of this observation seems to be one of his main interests. This book concentrates on marketing, consumer behavior, signaling and related subjects.

If you hate evolutionary psychology, perhaps because you feel it's unrigorous and consists of a collection of just-so stories, then you might not like the book as much. Nevertheless, much of what Miller writes remains interesting, particularly his discussion of what he calls the Central Six personality traits -- g (or IQ), Openness, Extraversion, Stability, Agreeableness, Conscientiousness. The "general factor" g is defined as the largest principal component that arises in analysis of the correlation between performance on cognitive tests. Similarly, the Big Five factors are the largest components that arise in the analysis of personality -- it is claimed that they capture most of the variation in personality (see here for more detailed discussion of Big Five). Miller argues that marketers don't make as much use of these principal components as they could.

You can take a brief Big Five test here. My scores were (percentiles): Openness 88, Conscientiousness 94, Extraversion 89, Agreeableness 74 and Neuroticism (opposite of Stability) 1.

Looking at Spent reminded me of a debate between Miller and LSE researcher Satoshi Kanazawa I read a few years ago. Each part was published separately in the journal Evolutionary Psychology (click title for PDF).

The Asian Future of Evolutionary Psychology, by Geoffrey Miller

Abstract: Asia’s population, wealth, cognitive capital, and scientific influence are growing quickly. Reasonable demographic, economic, and psychometric projections suggest that by the mid-21st century, most of the world’s psychology will be done in Asia, by Asians. Even if evolutionary psychology wins the battles for academic respectability in the United States and European Union, if it ignores the rise of Asian psychology, it will fail to have any serious, long-term, global influence in the behavioral sciences after the current generations of researchers are dead. ...

No, It Ain't Gonna Be Like That, by Satoshi Kanazawa

Abstract: For cultural, social, and institutional reasons, Asians cannot make original contributions to basic science. I therefore doubt Miller's prediction for the Asian future of evolutionary psychology. I believe that its future will continue to be in the United States and Europe.

Asian Creativity: A Response to Satoshi Kanazawa, by Geoffrey Miller

Abstract: This article responds to Satoshi Kanazawa’s thoughtful and entertaining comments about my article concerning the Asian future of evolutionary psychology. Contra Kanazawa’s argument that Asian cultural traditions and/or character inhibit Asian scientific creativity, I review historical evidence of high Asian creativity, and psychometric evidence of high Asian intelligence (a cognitive trait) and openness to experience (a personality trait) – two key components of creativity. ...

I find this debate amusing and thought provoking, although I am not at all convinced by most of the arguments presented. One wonders how careful Miller and Kanazawa are about deriving strongly held beliefs from limited data. To what extent do priors dominate their beliefs?

One interesting thing about Miller's first essay above is that he takes a quantitative stab at estimating the population of high-g (IQ > 130) individuals in different parts of the world. You can't get more un-PC than that :-) Of course, to make this estimate he needs to make assumptions not just about average IQs by population, but standard deviations as well!

Plight of the risk managers

I'd like to recommend this podcast interview with Riccardo Rebonato. Rebonato, the author of the prescient book Plight of the Fortune Tellers, written before the financial crisis, is a former physicist turned quant.

Rebonato does not mince words, pointing out the weaknesses of mathematical models, and noting that most quants, although mathematically sophisticated, often lacked deep knowledge about markets and banking (I assume he does not include himself in this group). In my experience many quants never questioned the basic efficient market assumptions underlying their models, although some certainly did -- in particular, those with trading experience.

Rebonato is polite, even urbane, but disagrees with Econtalk interviewer Russ Roberts on many important issues. The most important question, which Rebonato addresses immediately, is whether enlightened, self-interested managers of financial institutions can be relied on to properly manage risk. Regulators accepted, on faith, the self-regulating abilities and properties of a system managed by such people. Thus, one of the main ingredients in the crisis was the ideological (as opposed to political or financial) capture of regulators by efficient market proponents.

Other interesting topics covered are the divergent risk tolerances and interests of bond holders vs equity holders vs regulators of banks [1], and whether moral hazard (anticipation of a bailout) played a role in the crisis -- Russ, the anti-government libertarian, says yes. Rebonato says no, the story only makes sense if told at the institutional level, whereas individual incentives were different. I think Rebonato's logic is impeccable. It's more persuasive to me that incentive schemes which allowed huge compensation based on short term (ultimately illusory) gains were much more of a factor. (See Clawbacks, fake alpha and tail risk.)

Dr. Riccardo Rebonato

Riccardo is Global Head of Market Risk and Global Head of Quantitative Research and Quantitative Analysis for Royal Bank of Scotland based in London. Prior to joining the Royal Bank of Scotland, he was Head of Complex Derivatives Trading Europe desk and Head of Derivatives Research at Barclays Capital, where he worked for nine years.

Riccardo is a Visiting Lecturer at Oxford University in Mathematical Finance and Adjunct Professor at the Tanaka Business School, Imperial College, London.

Before joining the financial world, Riccardo was a Research Fellow in Physics at Oxford University (Corpus Christi College) and, before that, Visiting Scientist at Brookhaven National Laboratory.

Riccardo is the author of the books Plight of the Fortune Tellers ('07), The Perfect Hedger and the Fox (Wiley ’04), Modern Pricing of Interest-Rate Derivatives (Princeton University Press ’02), Interest-Rate Option Models (Wiley ’96,’98), Volatility and Correlation in Option Pricing (Wiley ’99). He has published several papers on finance (option modelling, computational techniques, risk management) in academic journals. He is a regular speaker at conferences worldwide.

[1] Footnote: see my earlier post on the vacuous Modigliani-Miller theorem. I recently learned from Vernon Smith's memoir (see pages 230-231 and 276) that he has similar opinions. Google books link; also search under "MM".

Matter and Antimatter, Angels and Demons: slides, video, audio

Below are links to slides, video and audio for the public lecture I gave on the movie Angels and Demons and related topics in high energy physics. This was part of a broader outreach effort by the high energy physics community -- about 50 such talks were given in multiple countries to coincide with the release of the film.

If you watch the video or listen to the audio, be sure to check out the last part as there were a lot of good questions from the audience.

Slides: Powerpoint file

Video: Windows Media , MP4 for Macs

Audio only: MP3

Vietnam: Strange Ground

At a dinner tonight I ended up seated next to an interesting older professor, a historian of modern SE Asia, probably in his sixties. A Yalie, he stayed in New Haven for graduate school, managing to avoid Vietnam despite being an Army officer with Ranger training. He had been a friend of mathematician Paul Olum, a highly esteemed president of the University of Oregon during the 1980's. (Olum and Feynman at Los Alamos.)

Talking with him about the Vietnam war reminded me of the oral history Strange Ground, by Harry Maurer, which I highly recommend. The recollections in the book read like the uncensored straight dope -- see, for example, this page.

Below is one of the few reviews of the book I could find online. I agree with the positive comments. As to the complaint that the book could have included interviews with Vietnamese or other participants of the war, well, the book is specifically about the American experience there.

Strange Ground is a collection of sixty seven first-hand accounts of the Vietnam war by Americans who were involved. It starts with an OSS mission to aid Ho Chi Minh against the Japanese near the end of World War II and finishes with the frantic evacuation of Saigon in 1975. The range of people included is immense — from alienated grunts in the infantry to gung-ho generals, from anti-war activists visiting North Vietnam to the wives of State Department officials in Saigon. The result is a broad sweeping view of the United States' involvement with Vietnam over three decades, but at the same time one with the feeling of immediacy that only such personal accounts can give.

My only dissatisfaction with this book was that it could have been so much better. As it stands it is like an album of photos of a tree: close up and wide-angle, black and white and colour, in sunlight and in shadow — but all taken from the same direction. Some of the Americans labelled all Vietnamese gooks and hated them, others felt the allure of Vietnamese culture and fell in love with the country. Nowhere, however, do we get any real idea about how these mysterious Vietnamese felt about the Americans. If Strange Ground had covered all the participants in the war — Viet Cong and ARVN and uncommitted peasants and French and North Vietnamese and Cambodians and even Australians — then it would have been a truly great book instead of just a very good one.

Strange Ground will be compulsory reading for anyone interested in the American experience in Vietnam.

University rankings: research and faculty quality

University rankings are a complicated subject; see this Wikipedia entry for an overview of various methodologies. Two common problems with ranking algorithms are:

1. Reliance on reputational surveys -- reputation is a lagging indicator, and often inaccurate (see below).

2. Failure to normalize performance measures to size of institution (i.e., number of faculty). A bigger institution should, all things being equal, produce more research papers and grant dollars. Normalizing to size yields a better indicator of average quality.

The only analysis I could find which corrects for these problems appears in the book The Rise of American Research Universities, a detailed study by two academics, Hugh Davis Graham and Nancy Diamond. The authors construct 3 research productivity indices for the period 1980-1990. The natural and social science indices are computed by counting publications in top journals, while the humanities index is computed by counting awards from the National Endowment for Humanities, American Council of Learned Societies, Guggenheim Foundation, etc. (humanists tend to publish books rather than articles). Graham and Diamond normalize the scores to the number of faculty, providing a true per capita measure of average quality.

The following figure shows the combined results for public research universities. Note the divergence between reputation (right column, from NRC rankings of graduate programs) and actual per capita scholarly productivity. (Click for larger version.)



The natural science index result is also interesting (natural science includes math, engineering, computer science as well as physics, chemistry, biology, etc.). Caltech's score is 3.36, followed by Stanford 1.21, MIT 1.16, Harvard .93, Berkeley .92, Princeton .83. So far, according to reputation, but note UC San Diego is ranked 4th (just below MIT) at 1.07! UC Irvine (.56), UCSB (.53), UCLA (.51) and Colorado (.55) are all in the top 20, comparable to Ivies like Yale (.65), Cornell (.60) and Columbia (.49) and ahead of Big 10 powers Illinois (.45), Wisconsin (.42) and Michigan (.32). Hmm... according to these results Caltech researchers were 3 times as productive per capita (in these subjects, normalized to total faculty size) as their counterparts at MIT and Harvard, and 10 times as productive as those at Michigan. Part of this must be the relatively smaller social science and humanities departments, but that can't explain the whole effect.

Graham and Diamond's results establish that reputation is often a misleading indicator. They identify a number of "rising" public research universities -- often in the west (California, Colorado, Oregon) -- whose reputation rankings are not commensurate with their research quality. It reminds me of the way that traditional east coast sports media underrates Pac-10 football year after year. Sure, Ohio State and Michigan look great playing against other slow as mud teams, until they get to the Rose Bowl :-)

Princeton Companion to Mathematics

I finally got a copy of The Princeton Companion to Mathematics. PCM is over 1000 pages of extremely well written, self-contained essays on a variety of topics in pure and applied mathematics. The book aims to be comprehensive in its coverage, an amazing ambition that seems to have been achieved, at least at the level of overview for non-specialists. Modern mathematics is such a broad and deep subject that PCM will be of use to students and experienced researchers alike. Every essay I have looked at is a pleasure to read.

Because the essays can be read independently, I think the ideal form for this book would be as an online document. Perhaps the publishers could work out a system for granting online access to people who buy the book (copy control issues notwithstanding)?

For hard core researchers, the alternative is the Japanese Encyclopedic Dictionary of Mathematics (EDM), which was prepared by the Mathematical Society of Japan. This book is highly compressed (even at 2000 pages) and is not the place to look for a cursory overview or readable introduction. One reviewer wrote:

EDM is an astonishing achievement. The result of an extraordinary, decades-long collaboration among literally hundreds of celebrated Japanese mathematicians, it will not only never be equalled but in all probability will never be challenged. In two massive volumes, the EDM surveys the whole of the mathematical sciences, both pure and applied, through a series of pithy articles containing the key definitions, methods, and results of every mathematical subdiscipline sufficiently coherent to have a name. It also tabulates vast amounts of information -- homotopy groups of spheres, symmetries of ordinary differential equations, characters of finite groups, class numbers of algebraic number fields, and so forth, seemingly, ad infinitum -- available, as far as I know, in no other single reference work.

...It is likewise only fair to point out that the EDM is a tool for serious research mathematicians. To keep its component articles brief, it makes full, unapologetic use of a wide variety of notational and expositional economies. The EDM seldom if ever provides a heuristic explanation of anything; although it often gives a bare outline of the historical development of a subject area, it resolutely eschews Toeplitz's "genetic" exposition, in which the crucial problems and examples that engendered a field are placed in the foreground. Only those persons comfortable with a very considerable level of compactness and abstraction in the exposition of mathematical ideas will find the EDM easy reading.

This lack of heuristic background and examples has made the EDM very difficult for me to use. It's OK for looking up results, but not for getting a feel for a completely new area. Perhaps there are people who are sufficiently strong that they can just pick up and read the EDM the way I can read the PCM, but probably not very many!

PCM was edited by Fields Medalist Timothy Gowers, who recently posted the latest errata on his blog.

Below are the contents:

I. What is Mathematics?
II. Ideas of Mathematics
III. Mathematical Objects
IV. Branches of Mathematics
V. Mathematicians
VI. Theorems and Problems
VII. The Influence of Mathematics
VIII. Miscellaneous

The following sample articles can be accessed as described below. Enter Username “Guest” and Password “PCM” at this site, then click “Resources” in the sidebar, then “Sample articles” in the sidebar.

Section Title

I Some Fundamental Mathematical Definitions
I The Language and Grammar of Mathematics
II Geometry
III Braid Groups
III Designs
III Determinants
III Distributions
III The Exponential and Logarithmic Functions
III Function Spaces
III Hilbert Spaces
III Knot Polynomials
III Metric Spaces
III Normed Spaces and Banach Spaces
III Permutation Groups
III Riemannian Metric
III Quaternions
III The Exponential and Logarithmic Functions
III The Euclidean Algorithm and Continued Fractions
III The Simplex Algorithm
III The Spectrum
IV Algebraic Geometry
IV Algebraic Numbers
IV Arithmetic Geometry
IV Differential Topology
IV Dynamics
IV Enumerative and Algebraic Combinatorics
IV High-Dimensional Geometry and Its Probablistic Analogues
IV Moduli Spaces
IV Operator Algebras
IV Probabilistic Models of Critical Phenomena
IV The Fourier Transform
V George Birkhoff
V János Bolyai
V Arthur Cayley
V Pierre Fermat
V Kurt Gödel
V Jacques Hadamard
V David Hilbert
V Sonya Kovalevskaya
V Nicolai Ivanovich Lobachevskii
V Pierre-Simon Laplace
V Isaac Newton
V Emmy Noether
V Jules Henri Poincaré
V Karl Weierstrass
VI Dvoretzky's Theorem
VI Gödel's Theorem
VI Liouville's Theorem and Roth's Theorem
VI The Atiyah–Singer Index Theorem
VI The Banach–Tarski Paradox
VI The Classification of Finite Simple Groups
VI The Fundamental Theorem of Algebra
VI The Fundamental Theorem of Arithmetic
VI The Insolubility of the Quintic
VII Analysis, Mathematical and Philosophical
VII Mathematical Biology
VII Mathematics and Chemistry
VII Mathematics and Economic Reasoning
VII Reliable Transmission of Information
VII Routing in Networks
VII The Mathematics of Algorithm Design
VII The Mathematics of Money
VIII Advice to a Young Mathematician
VIII Mathematics: An Experimental Science
VIII The Art of Problem Solving