Intellectual honesty: how much do we know?

As I have emphasized many times on this blog, when it comes to complex systems like society or economy (and perhaps even climate), experts have demonstrably little predictive power. In rigorous studies, expert performance is often no better than random. This is not surprising to anyone who has studied nonlinear dynamics and is free from ideological or professional bias (see below).

What is worse, experts are usually wildly overconfident about their capabilities. I'm often quite critical here of work done in fields like economics and social science, not because the results are unimpressive -- I acknowledge that the problems are hard and the systems under study intractable -- but because the researchers themselves often have beliefs whose strength is entirely unsupported by available data. They are usually unaware that biases (strong priors) shape their thinking, not scientific hypothesis testing. These biases come in many forms -- market fundamentalism, extreme belief in nature over nurture or vice versa, unrealistic faith in the powers of government, unrealistic faith in simple models, etc. (More from Robin Hanson and company here.)

It's refreshing to hear experts openly proclaim their lack of knowledge. Below are some comments from economist Arnold Kling, which would have gotten a physicist like me into hot water had I made them:

Economists pretending to have knowledge: ...I am shocked at the behavior of my fellow economists during this crisis. They are claiming to know much more than they do about causes and solutions. Rather than trying to understand and explain what is going on, they are engaged in a fierce battle over narrative.

...My main beef with economists is that standard macroeconomics does such a poor job of describing what is going on. The textbooks models are pretty much useless. Where in the textbooks is "liquidity preference" a demand for Treasury securities? Where in the textbooks does it say that injecting capital into banks is a policy tool?

Graduate macro is even worse. Have the courses that use representative-agent models solving Euler equations been abolished? Have the professors teaching those courses been fired? Why not?

I have always thought that the issue of the relationship between financial markets and the "real economy" was really deep. I thought that it was a critical part of macroeconomic theory that was poorly developed. But the economics profession for the past thirty years instead focused on producing stochastic calculus porn to satisfy young men's urge for mathematical masturbation.

More from Paul Romer of Stanford:

In the current crisis, the astonishing and unexpected consequences of the Lehman Brothers bankruptcy should serve as a similarly decisive data point. On the Thursday and Friday before Lehman filed for protection, I was at a conference on the financial crisis. Everyone there expected them to file on Monday. We repeated for each other all the fundamentalist arguments: "Everyone had been given time to prepare." "The courts handle bankruptcies all the time." None of us expected that putting Lehman through a court managed bankruptcy would be much different from arranging a forced sale of Bear Stearns.

We were all wrong. Within days, AIG was insolvent. Runs were developing on Goldman Sachs, Morgan Stanley, and the entire money market fund industry. Banks had stopped lending to each other in the Fed Fund market. Rates on Treasuries approached zero.

Here is one of the best academic papers I have seen on the credit crisis and the growth of the shadow banking system ("New Financial Architecture"). But note that the author James Crotty of U Mass Amherst is not considered a mainstream researcher in the field -- he's "heterodox"! It is amusing that in economics Crotty is considered a quasi-Marxist, whereas his views correspond quite strongly with those of actual financiers (at least when privately expressed).

This PERI Working Paper argues that the ultimate cause of the current global financial crisis is to be found in the deeply flawed institutions and practices of what is often referred to as the New Financial Architecture (NFA) – a globally integrated system of giant bank conglomerates and the so-called ‘shadow banking system’ of investment banks, hedge funds and bank-created Special Investment Vehicles. These institutions are either lightly and badly regulated or not regulated at all, an arrangement defended by and celebrated in the dominant financial economics theoretical paradigm – the theory of efficient capital markets. The NFA has generated a series of ever-bigger financial crises that have been met by larger and larger government bailouts.

After a brief review of the historical evolution of the NFA, this paper analyses its structural flaws: 1) the theoretical foundation of the NFA – the theory of efficient capital markets – is very weak and the celebratory narrative of the NFA accepted by regulators is seriously misleading; 2) widespread perverse incentives embedded in the NFA generated excessive risk-taking throughout financial markets; 3) mortgage-backed securities central to the boom were so complex and nontransparent that they could not possibly be priced correctly; their prices were bound to collapse once the excessive optimism of the boom faded; 4) contrary to the narrative, excessive risk built up in giant banks during the boom; and 5) the NFA generated high leverage and high systemic risk, with channels of contagion that transmitted problems in the US subprime mortgage market around the world.

Finally, as any trader knows, talk is cheap. Your opinion counts nothing unless you have skin in the game. By that accounting, it's George Soros and John Paulson who emerge as the real experts here -- they are each up billions of dollars after making smart bets on the crisis :-)


Obligatory disclaimer concerning string theory and speculative theoretical physics:

There is an old saying in finance: in the short run, the market is a voting machine, but in the long run it's a weighing machine. ...

You might think science is a weighing machine, with experiments determining which theories survive and which ones perish. Healthy sciences certainly are weighing machines, and the imminence of weighing forces honesty in the voting. However, in particle physics the timescale over which voting is superseded by weighing has become decades -- the length of a person's entire scientific career. We will very likely (barring something amazing at the LHC, like the discovery of mini-black holes) have the first generation of string theorists retiring soon with absolutely no experimental tests of their *lifetime* of work. Nevertheless, some have been lavishly rewarded by the academic market for their contributions.