The illusion of skill

Daniel Kahneman claims that differences in the performance of professional investors are mostly due to luck, whereas compensation is awarded as if differences are due to skill. Most alpha is fake alpha.

This of course raises all sorts of questions about why such people are allowed to become so extravagantly wealthy. The usual argument is that their investment decisions lead to more efficient resource allocation in the economy. (They are a "necessary evil" of a capitalist market system that benefits all of us :-) But if the decisions of the highest paid professionals are no better than those of average professionals, we could replace the services of the highest earners at much lower cost (or cap their salaries or impose high marginal tax rates) without negatively impacting the overall quality of decisions or the efficiency of the economy.








The article is worth reading in its entirety.

NYTimes: ... No one in the firm seemed to be aware of the nature of the game that its stock pickers were playing. The advisers themselves felt they were competent professionals performing a task that was difficult but not impossible, and their superiors agreed. On the evening before the seminar, Richard Thaler and I had dinner with some of the top executives of the firm, the people who decide on the size of bonuses. We asked them to guess the year-to-year correlation in the rankings of individual advisers. They thought they knew what was coming and smiled as they said, “not very high” or “performance certainly fluctuates.” It quickly became clear, however, that no one expected the average correlation to be zero.

What we told the directors of the firm was that, at least when it came to building portfolios, the firm was rewarding luck as if it were skill. This should have been shocking news to them, but it was not. There was no sign that they disbelieved us. How could they? After all, we had analyzed their own results, and they were certainly sophisticated enough to appreciate their implications, which we politely refrained from spelling out. We all went on calmly with our dinner, and I am quite sure that both our findings and their implications were quickly swept under the rug and that life in the firm went on just as before. The illusion of skill is not only an individual aberration; it is deeply ingrained in the culture of the industry. Facts that challenge such basic assumptions — and thereby threaten people’s livelihood and self-esteem — are simply not absorbed. The mind does not digest them. This is particularly true of statistical studies of performance, which provide general facts that people will ignore if they conflict with their personal experience.

The next morning, we reported the findings to the advisers, and their response was equally bland. Their personal experience of exercising careful professional judgment on complex problems was far more compelling to them than an obscure statistical result. When we were done, one executive I dined with the previous evening drove me to the airport. He told me, with a trace of defensiveness, “I have done very well for the firm, and no one can take that away from me.” I smiled and said nothing. But I thought, privately: Well, I took it away from you this morning. If your success was due mostly to chance, how much credit are you entitled to take for it?

From the comments.

If you read the whole article, you see that Kahneman does believe in skill. For example, his studies show that some doctors are better at diagnosis than others. I am also sure that some entrepreneurs or some physicists or some athletes are better than others. (Although in the case of entrepreneurs it would be very hard to demonstrate statistically since outcomes are noisy and the number of attempts per entrepreneur is relatively small.)

But there may be areas where *the differences between high level professionals* (e.g., people who have been hired to run money, have top MBAs or graduate degrees, etc.) are statistically seen to be mostly due to luck. This has already been convincingly demonstrated for pundits or analysts of complex world events by Tetlock's studies of expertise. (You can find several posts on this blog on the topic.) Whether it's true of money managers (or even big company CEOs) is controversial. If you argue the skill side, I'd like to see *your* statistical evidence, not just repetition of your priors (again and again).

Expert predictions

In all areas of human activity, even the skill dominated ones, luck plays a big factor. This is a good argument for redistribution -- almost every successful person owes some of their success to luck.

and

It seems to me that the 20th century trend in democracies is toward greater redistribution: social safety nets, guaranteed minimum income, etc. People have been conditioned to believe these are aspects of a just society.

The question is: what is the optimum level of redistribution? (Given a particular utility function for society.)

One argument is that we have to let the rich get rich in order to have strong economic growth. Too much redistribution means a smaller pie to split. But the Illusion of Skill argument (if correct) suggests that for some activities like finance a high marginal tax rate (say, which kicks in above the income of the *average* finance professional; this would then only affect the top earning financiers who, according to the argument are not adding any real value that the average guys can't also provide) would not negatively affect economic efficiency.

If people irrationally and incorrectly believe that only Harvard MDs are capable of treating pneumonia, and bid up their compensation to exorbitant levels (levels so high that the Harvard MDs begin exerting financial and political control over society as a whole), wouldn't it be better for society to impose a high tax rate on Harvard MDs, which kicks in above the income of other doctors with similar credentials (but who are not beneficiaries of the irrational belief)?