Copula in cosmology?

A new paper co-authored by one of my collaborators: http://arxiv.org/abs/0909.5187

From Finance to Cosmology: The Copula of Large-Scale Structure

Robert J. Scherrer, Andreas A. Berlind, Qingqing Mao, Cameron K. McBride (Vanderbilt University)

Any multivariate distribution can be uniquely decomposed into marginal (1-point) distributions, and a function called the copula, which contains all of the information on correlations between the distributions. The copula provides an important new methodology for analyzing the density field in large-scale structure. We derive the empirical 2-point copula for the evolved dark matter density field. We find that this empirical copula is well-approximated by a Gaussian copula. We consider the possibility that the full n-point copula is also Gaussian and describe some of the consequences of this hypothesis. Future directions for investigation are discussed.

I don't understand the gravitational dynamics well enough to guess whether we should expect the dark matter distribution today to be described by Gaussian copula. The authors insert a number of caveats, referring to the use of copula to price mortgage securities :-)

See here for some correspondence with Scherrer on the topic of genius and modern science, and here for some popular coverage of the paper we wrote.

Physicist to lead Williams College

I learned yesterday that a contemporary of mine, theoretical physicist Adam Falk, is the new President of Williams College.

Congratulations Adam! I think the future is bright at Williams :-)

See here for more, including some video interviews. Discussion by Williams alumni at Ephblog.

Starship Troopers

This commercial for the video game Halo 3: ODST (Orbital Drop Shock Troops) comes as close as anything I've seen to capturing the flavor of the Heinlein novel Starship Troopers. Forget about the 1997 movie directed by Paul Verhoeven; it's pure camp. Some might classify the original novel as pulp (see excerpt below), but I think it is much more than that.

The ODST project started as a Halo movie project involving Peter Jackson of Lord of the Rings fame. Perhaps this explains the cinematic qualities. (I'm not a gamer; I just noticed the commercial while watching Iowa beat Penn State last night and got interested in what seemed to be an obvious Starship Trooper influence.)

Wikipedia: Starship Troopers is a novel set in an unspecified time in the future, although probably not in the far-flung future. It chronicles the experiences of Juan "Johnnie" Rico, the story's narrator, during his enlistment and training in the Mobile Infantry, and his participation in an interstellar war between the Terran Federation and the Arachnids (referred to as "the Bugs") of Klendathu. It is narrated as a series of flashbacks — one of only a few Heinlein novels to use that narrative device[11] — and contains large sections of character discussion and introspection, as well as exposition, all meant to detail the political theory and philosophical beliefs underlying the society that Juan Rico lives in.

The novel opens with Rico aboard the corvette Rodger Young, platoon transport for "Rasczak's Roughnecks", about to embark on a raid against the planet of the "Skinnies", allies of the Arachnids. We learn that he is a "cap" trooper (called this because they are dropped in capsules from the ship in orbit toward their drop zones) in the Terran Federation's Mobile Infantry (M.I.). The raid itself, one of the few instances of actual combat in the novel, is relatively brief: the Mobile Infantrymen land in the capital city, destroy their targets while trying to avoid unnecessary Skinny casualties, and withdraw, suffering three injured (one fatal) in the process.

The story then flashes back to Rico's graduation from high school and his decision to sign up for Federal Service, rather than attend Harvard University, over the objections of his wealthy father. This is the only chapter that describes Rico's civilian life, and most of it is spent recording the monologues of two people: retired Lt. Colonel Jean V. Dubois, Rico's school instructor in the subject of History and Moral Philosophy, and Fleet Sergeant Ho, a recruiter for the Armed forces of the Terran Federation.

Many readers have felt that Dubois serves as a stand-in for Heinlein throughout the novel. He delivers what is probably the book's most famous soliloquy, on how violence "has settled more issues in history than has any other factor."[12] Fleet Sergeant Ho offers a separate angle on military service to that of Dubois. (Ho has prostheses for several limbs, but does not wear them on duty at the front door of the federal building. This is calculated to remind applicants of the real risks of service, and to weed out those not willing to take such risks in the service of the Federation).

Interspersed throughout the book are other flashbacks to Rico's high school History and Moral Philosophy course, which describe how, in the Terran Federation, the rights of a full Citizen (to vote, and hold public office) must be earned through voluntary Federal service. However, the franchise cannot be exercised until after honorable discharge from the Service, which means that active members of the Service cannot vote. Those residents who opt not to perform Federal Service retain the other rights generally associated with a modern democracy (e.g. free speech, assembly, etc.), but cannot vote or hold public office. This structure arose ad hoc after the collapse of the 20th century Western democracies, brought on by both social failures at home and military defeat by the Chinese Hegemony overseas (i.e. looking forward into the late 20th century from the time the novel was written in the late 1950s).[13]

After enlisting in the Mobile Infantry, Rico is assigned to boot camp at Camp Arthur Currie. Five chapters are spent exploring Rico's training, under the guidance of his chief instructor, Career Ship's Sergeant Charles Zim. Boot camp is deliberately so rigorous that fewer than ten percent of the recruits complete basic training; the rest either resign (with no penalty, save never being able to vote); are expelled (likewise); are given medical discharges (which may however be refused); or assigned to lesser duties (enabling them to vote after their service is finished); or die in training. ...

Here's the version of the book I've had since I was a kid:



Yes, I enjoyed a misspent youth, during which I read all of Heinlein's books and even played the Avalon Hill boardgame version of Starship Troopers.

Give me a few Sergeant Zim's and I'll conquer the galaxy! :-)

Bootcamp for the Mobile Infantry: ... But exercise will keep you warm and they saw to it that we got plenty of that. The first morning we were there they woke us up before daybreak. I had had trouble adjusting to the change in time zones and it seemed to me that I had just got to sleep; I couldn't believe that anyone seriously intended that I should get up in the middle of the night. But they did mean it. A speaker somewhere was blaring out a military march, fit to wake the dead, and a hairy nuisance who had come charging down the company street yelling, "Everybody out! Show a leg! On the bounce!" came marauding back again just as I had pulled the covers over my head, tipped over my cot and dumped me on the cold hard ground. It was an impersonal attention; he didn't even wait to see if I hit. Ten minutes later, dressed in trousers, undershirt, and shoes, I was lined up with the others in ragged ranks for setting-up exercises just as the Sun looked over the eastern horizon.

Facing us was a big broad-shouldered, mean-looking man, dressed just as we were -- except that while I looked and felt like a poor job of embalming, his chin was shaved blue, his trousers were sharply creased, you could have used his shoes for mirrors, and his manner was alert, wide-awake, relaxed, and rested. You got the impression that he never needed to sleep -- just ten-thousand-mile checkups and dust him off occasionally. He bellowed, "C'pnee! Atten . . . shut! I am Career Ship's Sergeant Zim, your company commander. When you speak to me, you will salute and say, `Sir' -- you will salute and `sir' anyone who carries an instructor's baton -- " He was carrying a swagger cane and now made a quick reverse moulinet with it to show what he meant by an instructor's baton; I had noticed men carrying them when we had arrived the night before and had intended to get one myself -- they looked smart. Now I changed my mind. " -- because we don't have enough officers around here for you to practice on. You'll practice on us. ...

Zim turned back to the rest of us, still shivering at attention. He walked up and down, looked us over, and seemed awfully unhappy. At last he stepped out in front of us, shook his head, and said, apparently to himself but he had a voice that carried: "To think that this had to happen to me!" He looked at us. "You apes -- No, not `apes'; you don't rate that much. You pitiful mob of sickly monkeys . . . you sunken-chested, slack-bellied, drooling refugees from apron strings. In my whole life I never saw such a disgraceful huddle of momma's spoiled little darlings in -- you, there! Suck up the gut! Eyes front! I'm talking to you!" I pulled in my belly, even though I was not sure he had addressed me. He went on and on and I began to forget my goose flesh in hearing him storm. He never once repeated himself and he never used either profanity or obscenity. (I learned later that he saved those for very special occasions, which this wasn't.) But he described our shortcomings, physical, mental, moral, and genetic, in great and insulting detail. But somehow I was not insulted; I became greatly interested in studying his command of language. I wished that we had had him on our debate team.

At last he stopped and seemed about to cry. "I can't stand it," he said bitterly. "I've just got to work some of it off -- I had a better set of wooden soldiers when I was six ALL RIGHT! Is there any one of you jungle lice who thinks he can whip me? Is there a man in the crowd? Speak up !" There was a short silence to which I contributed. I didn't have any doubt at all that he could whip me; I was convinced. I heard a voice far down the line, the tall end. "Ah reckon ah can . . . suh." Zim looked happy. "Good! Step out here where I can see you." The recruit did so and he was impressive, at least three inches taller than Sergeant Zim and broader across the shoulders. "What's your name, soldier?" "Breckinridge, suh -- and ah weigh two hundred and ten pounds an' theah ain't any of it `slack-bellied.' " "Any particular way you'd like to fight?" "Suh, you jus' pick youah own method of dyin'. Ah'm not fussy." "Okay, no rules. Start whenever you like." Zim tossed his baton aside. It started -- and it was over. The big recruit was sitting on the ground, holding his left wrist in his right hand. He didn't say anything. Zim bent over him. "Broken?" "Reckon it might be . . . suh." "I'm sorry. You hurried me a little. Do you know where the dispensary is? Never mind -- Jones! Take Breckinridge over to the dispensary."

As they left Zim slapped him on the right shoulder and said quietly, "Let's try it again in a month or so. I'll show you what happened." I think it was meant to be a private remark but they were standing about six feet in front of where I was slowly freezing solid. Zim stepped back and called out, "Okay, we've got one man in this company, at least. I feel better. Do we have another one? Do we have two more? Any two of you scrofulous toads think you can stand up to me?" He looked back and forth along our ranks. "Chicken-livered, spineless -- oh, oh! Yes? Step out." Two men who had been side by side in ranks stepped out together; I suppose they had arranged it in whispers right there, but they also were far down the tall end, so I didn't hear. Zim smiled at them. "Names, for your next of kin, please." "Heinrich." "Heinrich what?" "Heinrich, sir. Bitte." He spoke rapidly to the other recruit and added politely, "He doesn't speak much Standard English yet, sir." "Meyer, mein Herr," the second man supplied. "That's okay, lots of `em don't speak much of it when they get here -I didn't myself. Tell Meyer not to worry, he'll pick it up. But he understands what we are going to do?" "Jawohl," agreed Meyer. "Certainly, sir. He understands Standard, he just can't speak it fluently." "All right. Where did you two pick up those face scars? Heidelberg?" "Nein -- no, sir. Ko:nigsberg." "Same thing." Zim had picked up his baton after fighting Breckinridge; he twirled it and asked, "Perhaps you would each like to borrow one of these?" "It would not be fair to you, sir," Heinrich answered carefully. "Bare hands, if you please." "Suit yourself. Though I might fool you. Ko:nigsberg, eh? Rules?" "How can there be rules, sir, with three?" "An interesting point. Well, let's agree that if eyes are gouged out they must be handed back when it's over. And tell your Korpsbruder that I'm ready now. Start when you like." Zim tossed his baton away; someone caught it. "You joke, sir. We will not gouge eyes." "No eye gouging, agreed. `Fire when ready, Gridley.' " "Please?" "Come on and fight! Or get back into ranks!" Now I am not sure that I saw it happen this way; I may have learned part of it later, in training. But here is what I think happened: The two moved out on each side of our company commander until they had him completely flanked but well out of contact. From this position there is a choice of four basic moves for the man working alone, moves that take advantage of his own mobility and of the superior co-ordination of one man as compared with two -- Sergeant Zim says (correctly) that any group is weaker than a man alone unless they are perfectly trained to work together. For example, Zim could have feinted at one of them, bounced fast to the other with a disabler, such as a broken kneecap then finished off the first at his leisure. Instead he let them attack. Meyer came at him fast, intending to body check and knock him to the ground, I think, while Heinrich would follow through from above, maybe with his boots. That's the way it appeared to start. And here's what I think I saw. Meyer never reached him with that body check. Sergeant Zim whirled to face him, while kicking out and getting Heinrich in the belly -- and then Meyer was sailing through the air, his lunge helped along with a hearty assist from Zim. But all I am sure of is that the fight started and then there were two German boys sleeping peacefully, almost end to end, one face down and one face up, and Zim was standing over them, not even breathing hard.

"Jones," he said. "No, Jones left, didn't he? Mahmud! Let's have the water bucket, then stick them back into their sockets. Who's got my toothpick?" A few moments later the two were conscious, wet, and back in ranks. Zim looked at us and inquired gently, "Anybody else? Or shall we get on with setting-up exercises?" I didn't expect anybody else and I doubt if he did. But from down on the left flank, where the shorties hung out, a boy stepped out of ranks, came front and center. Zim looked down at him. "Just you? Or do you want to pick a partner?" "Just myself, sir." "As you say. Name?" "Shujumi, sir." Zim's eyes widened. "Any relation to Colonel Shujumi?" "I have the honor to be his son, sir." "Ah so! Well! Black Belt?" "No, sir. Not yet." "I'm glad you qualified that. Well, Shujumi, are we going to use contest rules, or shall I send for the ambulance?" "As you wish, sir. But I think, if I may be permitted an opinion, that contest rules would be more prudent." "I don't know just how you mean that, but I agree." Zim tossed his badge of authority aside, then, so help me, they backed off, faced each other, and bowed. After that they circled around each other in a half crouch, making tentative passes with their hands, and looking like a couple of roosters. Suddenly they touched -- and the little chap was down on the ground and Sergeant Zim was flying through the air over his head. But he didn't land with the dull, breath-paralyzing thud that Meyer had; he lit rolling and was on his feet as fast as Shujumi was and facing him. "Banzai!" Zim yelled and grinned. "Arigato," Shujumi answered and grinned back. They touched again almost without a pause and I thought the Sergeant was going to fly again. He didn't; he slithered straight in, there was a confusion of arms and legs and when the motion slowed down you could see that Zim was tucking Shujumi's left foot in his right ear -- a poor fit. Shujumi slapped the ground with a free hand; Zim let him up at once. They again bowed to each other. "Another fall, sir?" "Sorry. We've got work to do. Some other time, eh? For fun . . . and honor. Perhaps I should have told you; your honorable father trained me." "So I had already surmised, sir. Another time it is." Zim slapped him hard on the shoulder. "Back in ranks, soldier. C'pnee!" Then, for twenty minutes, we went through calisthenics that left me as dripping hot as I had been shivering cold. Zim led it himself, doing it all with us and shouting the count.

Cognitive decline of the university population

Razib at GNXP provides some nice figures relevant to my previous post Decline of the humanities.

The figures show the distribution of vocabulary scores (General Social Survey (GSS) WORDSUM) of college graduates and those without college degrees for the periods 1974-1984 and 1998-2008. It looks like the average score for college graduates has dropped by a significant fraction of a standard deviation over 25 years. This effect is entirely predictable given the larger percentage of Americans attending college in recent times.





It should not be surprising that a shrinking percentage of college students can write well or do basic mathematics, let alone appreciate Proust or quantum mechanics.

Additional years of education have not increased verbal abilities in the general population. This observation supports, at least in part, the signaling and sorting model of higher education (the primary value of credentials is that they reflect more or less invariant qualities such as IQ and Conscientiousness), as opposed to the model that higher education builds human capital.

Decline of the humanities

From an essay by William Chace, professor of English and former president of Wesleyan and Emory.

... Here is how the numbers have changed from 1970/71 to 2003/04 (the last academic year with available figures):

English: from 7.6 percent of the majors to 3.9 percent
Foreign languages and literatures: from 2.5 percent to 1.3 percent
Philosophy and religious studies: from 0.9 percent to 0.7 percent
History: from 18.5 percent to 10.7 percent
Business: from 13.7 percent to 21.9 percent

In one generation, then, the numbers of those majoring in the humanities dropped from a total of 30 percent to a total of less than 16 percent; during that same generation, business majors climbed from 14 percent to 22 percent. Despite last year’s debacle on Wall Street, the humanities have not benefited; students are still wagering that business jobs will be there when the economy recovers.

What are the causes for this decline? There are several, but at the root is the failure of departments of English across the country to champion, with passion, the books they teach and to make a strong case to undergraduates that the knowledge of those books and the tradition in which they exist is a human good in and of itself. What departments have done instead is dismember the curriculum, drift away from the notion that historical chronology is important, and substitute for the books themselves a scattered array of secondary considerations (identity studies, abstruse theory, sexuality, film and popular culture). In so doing, they have distanced themselves from the young people interested in good books.

... Alexander W. Astin’s research tells us that in the mid-1960s, more than 80 percent of entering college freshmen reported that nothing was more important than “developing a meaningful philosophy of life.” Astin, director of the Higher Education Research Institute at UCLA, reports that “being very well off financially” was only an afterthought, one that fewer than 45 percent of those freshmen thought to be an essential goal. As the years went on, however, and as tuition shot up, the two traded places; by 1977, financial goals had surged past philosophical ones, and by the year 2001 more than 70 percent of undergraduate students had their eyes trained on financial realities, while only 40 percent were still wrestling with meaningful philosophies.

Regarding the last paragraph, while there has undoubtedly been a general cultural shift, it is also true that a much larger fraction of the population attends college now, with resulting decline of average cognitive ability. Perhaps the elite of the 1960s had the luxury and cognitive ability to concentrate on their philosophy of life, as opposed to earning a living; students today do not.

For more see here:

Education and Verbal Ability over Time: Evidence from Three Multi-Time Sources

Nie, Golde and Butler

Abstract: During the 20th century, there was an unprecedented expansion in the level of educational attainment in America. Using three separate measures, this paper investigates whether there was a concurrent increase in verbal ability and skills. Changes in verbal ability in the general population as well as changes in the verbal ability of graduates of different levels of education are investigated. An additional investigation of how changes in the differences between males' and females' educational attainment are associated with changes in differences between their respective verbal abilities follows. The main finding is that there is little evidence that the large increase in educational attainment has resulted in an increase in any of the measures of verbal abilities and skills.

The origins of behavioral economics: Kahneman interview

Below is a great interview with Daniel Kahneman: Intuition and Rationality (full transcript).

Some years ago I wrote about Choices, Values, and Frames. See here for a nice discussion of behavioral economics.




Obvious to psychologists, difficult for economists. (See here for discussion by Robert Shiller, whose wife is a psychologist.)

Then when we worked on decision making, which was the second thing we did together, we published the theory paper in Econometrica, which is the prestige theory journal in economics. We were not intending to influence economics at all. It was just the best journal for that kind of article. If we had published exactly the same article in Psychological Review it would not have got the Nobel Prize, it [would have been seen as] a trivial detail. If we'd had a hostile referee in Econometrica we would have gone to Psych Review, but the big event that caused the prize is that some economists were interested, particularly an economist who's now my closest friend, Richard Thaler, who became interested in our work. He is the one who both taught me economics and brought these ideas into economics for their behavioral [value.]

So, it's because of the work of economists that I got the prize. I didn't get the prize for work that Amos and I did just because it was very good work. We got the prize (or I got the prize, because he couldn't share in it) because it had an impact on economics; but the impact was because people in economics saw it was relevant. They were a minority of people, and they did brilliant work.

The complexities of life:

It must have been the fall of '41, when there was a curfew for Jews. We were also supposed to be wearing a yellow star, and there was a curfew which I think was 6:00 PM. I was in first or second grade and I'd gone to play with a friend and I was going home and I missed the curfew, I was late. And so, I turned my sweater inside out and walked home, and as I was coming close I remember the street was deserted and there was this German soldier walking towards me. He was wearing the black uniform and I knew that was not good. That was the uniform of the SS. We were walking towards each other and as we were coming close he sort of beckoned me, and of course I went there, and he picked me up and hugged me. I remember being terrified that he would see the Star of David inside my sweater. Then he put me down and took out his wallet and showed me a picture of a boy and gave me some money. That's a formative memory because of what it meant about the complexity of things. I remember being very fascinated at the time by this and by stories of Hitler liking flowers and kissing babies. The complexity of evil was much on my mind as a seven- or eight-year-old.

[While traveling, missing my kids, I find myself smiling goofily at every 3-5 year old I see!]

Psychology research for the Israeli army:

One of the things I did do that I'm still quite proud of is I set up an interviewing system for the Israeli army to interview recruits for combat units. I learned in 2002 [that] that interview is still in place, fifty years later. ... In 1954 Paul Miel published a very important book in which he showed that clinicians are much less good at what they're doing than they think they are, and that their ability to forecast events and psychological behavior in the future is really quite limited. ... I set up an interview that didn't leave a lot of room for clinical intuition on the part of the interviewer. The interviewer went through a script and found out what the person had done as a civilian, basically. It turns out that when you use that information to produce ratings of various strengths, you get ratings that are predictive and are useful on whether somebody is responsible, or appreciates manliness, or is sociable, things like that. [Personality factors are stable and useful! Expert prediction is weak; algorithms beat experts.] In addition, it turned out (and this was important to me years later) that when people had acquired that information, then their clinical judgment was, in fact, good, whereas if they were trying to form a clinical impression [without a script] they couldn't do it.

These observations that I made at that time became quite essential to my research about fifteen years later and they were central to the work that ultimately led to the Nobel Prize. So, it started quite early, by accident.

[Note added: (September 2010) I met Kahneman at Scifoo this year and asked him briefly about his personnel selection experience. He seemed quite proud that the Israeli military is still using his methodology today.]

Advice on research:

My main advice would be to look at the natural traps that await anybody who does research. One of the real traps is to get trapped by a research program that is fundamentally uninteresting, and also, the reluctance to cut your losses so that when you've done something that is not very interesting, wasting time trying to publish it is almost always a big mistake, at least in my judgment. Look for things that are worth doing and discard things that don't work, and know that if you're reasonably good at what you're doing, and otherwise you shouldn't be in that profession, but if you're reasonably good at it, then ideas are a dime a dozen. You'll get lots of ideas. You need some ideas that work and that keep you excited and that you can do something with. And so, not getting stuck on an idea that doesn't work, but looking for other ideas -- that would be my general advice. That certainly has been how I've operated.

The bubble algorithm for human capital allocation

According the this WSJ article, at the peak of our finance bubble about a third of MIT graduates and almost half of all Harvard grads went into finance! Not so long ago a friend of mine at a derivatives desk made offers to 2 percent of the Caltech graduating class (only 4 people, but still).

To what field should society allocate the (N+1)-th big brain to achieve the highest marginal return? For years I've been told that the answer is finance -- markets are efficient after all, and price signals must be correct! ;-)

WSJ: Like nearly 30% of Massachusetts Institute of Technology graduates in recent years, Ted Fernandez set his sights on finance. Though he majored in materials science and engineering, he was wowed by tales of excitement from friends who went to Wall Street.

But when he stopped by an investment bank's booth at a job fair a year ago, it was eerily empty. The booth belonged to Lehman Brothers Holdings Inc., and the date was Sept. 18, three days after the 158-year-old bank filed for bankruptcy. Now Mr. Fernandez, 22 years old, is getting a master's in engineering at M.I.T. and aiming for a career in solar-power technology. [Another bubble? Let's hope not!]

"Undoubtedly, I would have gone into finance if the financial meltdown hadn't occurred," he says. "Now I won't make as much money, but I can go home at night and feel good about what I do. That's worth more than any amount of money."

Over the past 20 years, finance grew faster than almost any other sector of the U.S. economy, offering rich pay and luring a growing share of bright minds to trade securities, make loans, manage portfolios, engineer mergers and turn mortgages into complex derivatives. Now the finance bubble has deflated, forcing hundreds of thousands of employees to search for other work and sending new graduates looking elsewhere for careers.

... Harvard's 2009 graduating class shows the shift in career directions. Those entering finance and consulting tumbled to 20% of graduates this year from nearly twice that in 2008 and 47% the year before, according to a survey by the university's newspaper, the Crimson. ...

Even a modest of shift of talent could have an effect on society. When smart people become entrepreneurs, "they improve technology in the line of business they pursue, and, as a result, productivity and income grow," said a study by economists Kevin M. Murphy, Robert W. Vishny and Andrei Schleifer in 1990. By contrast, they said, allocation of talent to professions such as finance and law -- where returns come from distribution of wealth from others rather than wealth creation -- leads to lower productivity growth, fewer technological opportunities and slower economic growth. [Yes, I know, this argument is far from complete... better people making capital allocation decisions probably does lead to more efficient outcomes... but at what point do we reach saturation?]

"Some professions are socially more useful than others, even if they are not as well compensated," the economists said.

The figure below is from this earlier post:

Ellsberg, Cowen, Meth and My Lai

Our conference is winding down, so I guess I can get back to blogging without feeling guilty that I should be doing something else :-)

One of the pleasures of travel in the pre-internet era was being disconnected from world events. I fondly recall poring over the Herald Tribune in cafes as the preferred method for getting the news. During this trip I've only scanned the news and blogs in a cursory fashion, but I did listen to a fair number of podcasts while on planes, a ferry, and falling asleep near lapping waves. Below are some recommendations.


Daniel Ellsberg (Pentagon Papers) interview. In the interview Ellsberg compares Afghanistan to Vietnam; I don't know enough to judge whether the analogy is reasonable. I recommend this biography, which also covers his early history as an academic, working on game theory and decision theory. Ellsberg wrote this influential 1957 paper discussing the difference between Knightian (unmeasurable) uncertainty and (measurable) risk while a Junior Fellow at Harvard. These comments by Nobelist Thomas Schelling, who was something of a mentor to Ellsberg, are fascinating.

Interview with Tyler Cowen on Econtalk, about his recent book Create Your Own Economy. Buddhists vs autists?

Interview with Nick Reding, author of Methland: the Death and Life of an American Small Town, about methamphetamines and their impact on a small Iowa town.

The My Lai tapes (BBC documentary). Part 2. Sad but true.

The view from Crete

Click for larger version of photo. The first two photos are facing east in the morning. I've learned that digital cameras (even the crappy one in my iPhone) can process direct shots of the sun (nice smoothing algorithms!). The bottom photo is of the Orthodox Academy of Crete, where our conference is being held.

Black holes and decoherence talk

Slides for my talk in Crete. Previous discussion and link to paper.

Greetings from Athens

Click photos for larger versions

My advice: get to the Parthenon early, before the hordes of tourists arrive. I got there just after the opening and only had to share the place with a few dozen others. By the time I left the entrance area looked like a staging zone for D-Day.

The New Acropolis Museum (discussed in an earlier post) is beautiful, but they have a strict no-photographs policy.

The pictures below are from the National Archeological Museum. The weird gizmo is the famous Antikythera mechanism, which I almost missed -- it's in the Bronze Collection, which is overshadowed by all the statuary. The shiny modern thing is the Price model, built to illustrate the device's operation as an analog computer of astronomical motions. Yes, an analog computer built in 150 BC or so that calculates the positions of the sun, moon and planets -- it's unbelievable how advanced the ancient Greeks were!

Some favorite posts

NEW!  Podcast show Manifold.

I started writing this blog in 2004 (it has had millions of visitors!), and by now the content is a bit unwieldy to navigate, even with labels and search. I thought I'd make a list of some of my favorite posts and topics. Please suggest other posts to add to the list!

Pessimism of the Intellect, Optimism of the Will.

Feynman and me, Memories of Feynman. Labels: Feynman, path integrals.

Richard Feynman and the 19 year old me at my Caltech graduation:

Mama said knock you out (learning how to fight).

Many Worlds and quantum mechanics, a brief guide. Label: Many Worlds. Note, the usual quantum probabilities do not emerge naturally in this interpretation. See my papers On the Origin of Probability in Quantum Mechanics and The measure problem in no-collapse (many worlds) quantum mechanics.

Cognitive limitations: statistics , higher ed. Label: bounded rationality, human capital. Brainpower and globalization.

Expert predictions in soft subjects are unreliable. Intellectual honesty. Frauds! Label: expert predictions.

We can (crudely) measure cognitive ability using simple tests. (It is amazing to me that this is a controversial statement.) Randomly sampled eminent scientists have (very) high IQs, and given the observed stability of adult IQ the causality is clear: psychometrics works. The cult of genius? Income, Wealth, and IQ, One hundred thousand brains. Bezos on the Big Brains. Label: psychometrics.

Historically isolated groups of humans cluster genetically according to geographical ancestry. Explained in pictures , words , more words.

I am skeptical of all but the weakest claims of market efficiency. My talk on the 2008 credit crisis. Venn diagram for economics.

Careers, advice to geeks: A tale of two geeks , success vs ability. Labels: careers , startups , entrepreneurs.

Net worth , life satisfaction , happiness , the gilded age.

What is the likely development path for China in the next decades? Sustainability of China growth , China development: how big is the middle class? , Back to the future , Shanghai from an Indian perspective.

That curious institution, Caltech. How did a 16 year old kid from Iowa end up there? (See memories of Feynman above.)

There are geniuses in the world. The cult of genius.

My lovely kids. Photos. Autobiographical.


Update:

Credentialism and elite careers , Defining merit , elitism , brainpower

Recent videos (talks on genomics): https://www.youtube.com/results?search_query=hsu+genomics

Talks (some with slides + video):

Cold Spring Harbor Laboratory
Berkeley Innovative Genomics Institute and OpenAI
Janelia Research Campus (HHMI)
Allen Institute (Seattle) meeting on Genetics of Complex Traits

Review article: On the genetic architecture of cognitive ability and other quantitative traits (2014)

I work on algorithms for phenotype prediction from genotype, using new methods from high dimensional statistics. My estimate is that prediction of complex traits such as height, cognitive ability, or highly polygenic disease conditions will require data sets of order one million individuals (i.e., to build a model which accounts for most of the genetic variance). Once these models are available, human reproduction (and evolution!) will be revolutionized.

These papers are somewhat technical:
https://arxiv.org/abs/1310.2264
http://arxiv.org/abs/1408.6583

This one is a bit less technical and gives a broader overview:
http://arxiv.org/abs/1408.3421

Cow genomics (an existence proof):
http://infoproc.blogspot.com/2012/08/genomic-prediction-no-bull.html
http://infoproc.blogspot.com/2014/08/its-all-in-gene-cows.html

These are for popular audiences (Nautilus Magazine):
http://nautil.us/issue/18/genius/super_intelligent-humans-are-coming
http://nautil.us/issue/28/2050/dont-worry-smart-machines-will-take-us-with-them

2018: As anticipated, we now have good height predictors thanks to the 500k genome release of UK Biobank data: Scientists of Stature

Genomic predictors for common disease risk, constructed via machine learning on hundreds of thousands of genotypes. The predictors use anywhere from a few tens (e.g., 20 or 50) to thousands of SNPs to compute the risk PGS (Poly-Genic Score) for conditions such as diabetes, breast cancer, heart attack, and more: Genomic Prediction of Complex Disease Risk.

The Economist on polygenic risk scores (2019).

Detailed analysis of genetic architectures of disease risk predictors. Implications for pleiotropy.

Sibling validation of genomic predictors.

Recent papers from my group:

https://www.genetics.org/content/210/2/477
https://www.nature.com/articles/s41598-019-51258-x
https://www.nature.com/articles/s41598-020-68881-8
https://www.nature.com/articles/s41598-020-69927-7


2021 review article, prepared for the book Genomic Prediction of Complex Traits, Springer Nature series Methods in Molecular Biology: 

From Genotype to Phenotype: polygenic prediction of complex human traits

Krugman: the Economists have no clothes

Only Nixon could go to China, and only Paul Krugman could write this mea culpa for the discipline of economics.

There was a bubble in housing -- everybody knows that now. There was a bubble in finance itself -- the financial share of national income reached an all-time high just before the crisis; not too many people know this. There was even an academic bubble in the field of economics: the perceived quality of results in the field, reflected in the salaries and prestige of economics professors (e.g., relative to other social scientists), was as inflated as the price of any McMansion -- surely every university dean must understand this now? (Bayesian / Machine Learning comment: do these guys ever update? Or is it "All priors, all the time"?)

I suggest reading the whole thing. I've only excerpted the last three paragraphs below. (See also related essay by Richard Posner and this interview with Bill Janeway.)

How Did Economists Get It So Wrong?: ... So here’s what I think economists have to do. First, they have to face up to the inconvenient reality that financial markets fall far short of perfection, that they are subject to extraordinary delusions and the madness of crowds. Second, they have to admit — and this will be very hard for the people who giggled and whispered over Keynes — that Keynesian economics remains the best framework we have for making sense of recessions and depressions. Third, they’ll have to do their best to incorporate the realities of finance into macroeconomics.

Many economists will find these changes deeply disturbing. It will be a long time, if ever, before the new, more realistic approaches to finance and macroeconomics offer the same kind of clarity, completeness and sheer beauty that characterizes the full neoclassical approach. To some economists that will be a reason to cling to neoclassicism, despite its utter failure to make sense of the greatest economic crisis in three generations. This seems, however, like a good time to recall the words of H. L. Mencken: “There is always an easy solution to every human problem — neat, plausible and wrong.”

When it comes to the all-too-human problem of recessions and depressions, economists need to abandon the neat but wrong solution of assuming that everyone is rational and markets work perfectly. The vision that emerges as the profession rethinks its foundations may not be all that clear; it certainly won’t be neat; but we can hope that it will have the virtue of being at least partly right.

Related posts on Krugman's article from two thoughtful economists, Brad DeLong: Which economists got it so wrong? , Where does macro go from here? and Arnold Kling: Krugman vs Blanchard.

Athens and Crete

I'm off to Athens and Crete next week. The main purpose of the trip is to attend a meeting on quantum gravity, but I'm also looking forward to visiting this new museum:



See slideshow.

Now that they have an appropriate place for them, will the Greeks get the Elgin Marbles back from the British?

NYTimes: ... For more than 30 years, Greece has been working, through diplomacy and public relations offensives, to regain the Elgin Marbles, sections of a decorative frieze that adorned the Parthenon until Lord Elgin ordered them removed in the early 19th century, during his tenure as British ambassador to the Ottoman Empire. Years later, bankrupt, he sold them to the British Museum, where they have been a major attraction since 1816.

Today, almost two centuries on, many Greeks hope the opening of the museum will focus international attention on their country’s claim to the so-called Elgin Marbles, and put an end to Britain’s longtime argument that it is in a better position to look after those 2,500-year-old panels. Last week the Greek government turned down an offer from the British Museum for a three-month loan of the collection, because it came with the condition that the Greeks formally acknowledge British ownership.

“This is a nonstarter for any discussion,” Mr. Samaras said. “No Greek can sign up to that.”

The new museum, 226,000 square feet of glass and concrete designed by the New York architect Bernard Tschumi, replaces the old Acropolis Museum, a small 1874 building tucked into the rock of the Acropolis next to the Parthenon. The design, introduced in 2001, was meant to be completed in time for the 2004 Olympics, but dozens of legal battles — many having to do with some 25 buildings that were demolished to make room for it — delayed the process for years.

Eventually I suspect the Chinese, Egyptians and many others will recover their stolen national treasures from museums in Europe and the US.

Flowers for Algernon

I read this when I was a kid, and it had a strong effect on me, at least in part because I had a next-door neighbor who was my age and mentally retarded. Now that I'm a parent it strikes me that we all follow the path of Charlie Gordon -- starting from modest intellectual ability, developing to a peak, followed by inevitable decline. It happens more gradually for us than for him, but we are equally aware of the process and where it leads.

Wikipedia: ... The titular Algernon is a laboratory mouse who has undergone surgery to increase his intelligence by artificial means. The story is told as a series of progress reports written by Charlie, the first human test subject for the surgery.

... Another key moment came in 1957, while [author] Keyes was teaching English to students with special needs; one student asked him if it would be possible to be put into a regular class if he worked hard and became smart.

... The procedure is a success and, three months later, Charlie's IQ has reached 185. However, as his intelligence, education and understanding of the world around him increases, his relationships with people deteriorate.

... Charlie discovers a flaw in the theories that led Nemur and Strauss to develop their intelligence-enhancing procedure. Shortly thereafter, Algernon starts behaving erratically, loses his new intelligence, and dies. As Charlie does further research, he determines that he too will inevitably revert to his old condition.

From the Wikipedia entry I learned the novel is based on a short story that appeared years earlier and won the science fiction Hugo award. The full text is here (at least for the moment). I think the novel is stronger than the short story.

Here's an excerpt (Google Books).

June 29 -- ... I phoned Landsdoff at the New Institute for Advanced Study, about the possibility of using the pair-production nuclear photoeffect for exploratory work in biophysics. At first he thought I was a crackpot, but when I pointed out to him the flaws in his article in the New Institute Journal he kept me on the phone nearly an hour. He wants me to come to the Institute to discuss my ideas with his group. ... That's the problem, of course. I don't know how much time I have. A month? A year? The rest of my life? That depends on what I find out about the psychophysical side-effects of the experiment. ...