Thursday, April 02, 2009

The future of innovation

A reader sent me this article from PhysicsWorld, which considers the nature of scientific innovation. I particularly agree with Smolin's comments excerpted below. The reward structure of academic science encourages narrow specialization far too much. Tremendous lip service is paid to interdisciplinary work, but in actuality it is very risky to undertake.

Using Smolin's analogy of hill climbing, the dominant strategy today in science is:

1) self-assess own climbing ability
2) choose suitable hill (perhaps inherited from advisor!)
3) climb to local maximum (write some relevant papers with incremental results)
4) squat on hilltop and defend against all attackers (make sure everyone cites your papers; get embedded in small community of researchers defending that hill)
5) train students and postdocs on your hilltop while secretly wishing you understood what other people were doing on their hilltops -- suppressing the curiosity that originally got you into science.

From personal experience, I can tell you that when you leave your little hill to cross a valley and explore somewhere else, the citations of your previous work will plummet, inhabitants of other hills will try to repel you, and funding agencies will ask why you aren't doing mainstream stuff ("he's not serious -- he keeps jumping around"). Based on this incentive system, it is easy to understand why people behave as they do.

...returns on research investment do not arrive steadily and predictably, but erratically and unpredictably, in a manner akin to intellectual earthquakes. Indeed, this idea seems to be more than merely qualitative. Data on human innovation, whether in basic science or technology or business, show that developments emerge from an erratic process with wild unpredictability. For example, as physicist Didier Sornette of the ETH in Zurich and colleagues showed a few years ago, the statistics describing the gross revenues of Hollywood movies over the past 20 years does not follow normal statistics but a power-law curve — closely resembling the famous Gutenberg— Richter law for earthquakes — with a long tail for high-revenue films. A similar pattern describes the financial returns on new drugs produced by the bio-tech industry, on royalties on patents granted to universities, or stock-market returns from hi-tech start-ups.

What we know of processes with power-law dynamics is that the largest events are hugely disproportionate in their consequences. In the metaphor of Nassim Nicholas Taleb’s 2007 best seller The Black Swan, it is not the normal events, the mundane and expected “white swans” that matter the most, but the outliers, the completely unexpected “black swans”. In the context of history, think 11 September 2001 or the invention of the Web. Similarly, scientific history seems to pivot on the rare seismic shifts that no-one predicts or even has a chance of predicting, and on those utterly profound discoveries that transform worlds. They do not flow out of what the philosopher of science Thomas Kuhn called “normal science” — the paradigm-supporting and largely mechanical working out of established ideas — but from “revolutionary”, disruptive and risky science.

Squeezing life out of innovation

All of which, as Sornette has been arguing for several years, has important implications for how we think about and judge research investments. If the path to discovery is full of surprises, and if most of the gains come in just a handful of rare but exceptional events, then even judging whether a research programme is well conceived is deeply problematic. “Almost any attempt to assess research impact over a finite time”, says Sornette, “will include only a few major discoveries and hence be highly unreliable, even if there is a true long-term positive trend.”

This raises an important question: does today’s scientific culture respect this reality? Are we doing our best to let the most important and most disruptive discoveries emerge? Or are we becoming too conservative and constrained by social pressure and the demands of rapid and easily measured returns? The latter possibility, it seems, is of growing concern to many scientists, who suggest that modern science is in danger of losing its creativity unless we can find a systematic way to build a more risk-embracing culture.

The voices making this argument vary widely. For example, the physicist Geoffrey West, who is currently president of the Santa Fe Institute (SFI) in New Mexico, US, points out that in the years following the Second World War, US industry created a steady stream of paradigm-changing innovations, including the transistor and the laser, and it happened because places such as Bell Labs fostered a culture of enormously free innovation. “They brought together serious scientists — physicists, engineers and mathematicians — from across disciplines”, says West, “and created a culture of free thinking without which it’s hard to imagine how these ideas could have come about.”

Unfortunately, today’s academic and corporate cultures seem to be moving in the opposite direction, with practices that stifle risk-taking mavericks who have a broad view of science. At universities and funding agencies, for example, tenure and grant committees take decisions based on narrow criteria (focusing on publication lists, citations and impact factors) or on specific plans for near-term results, all of which inherently favour those working in established fields with well-accepted paradigms. In recent years, tightening business practices and efforts to improve efficiency have also driven corporations in a similar direction. “That may be fine in the accounting department,” says West, “but it’s squeezing the life out of innovation.”

...But physicist Lee Smolin, currently at the Perimeter Institute, suggests that science overall requires a much broader and more coherent approach to risky science. To see the kinds of policies needed, he suggests, it is useful to note that scientists, at least in some rough approximation, follow working styles of two very different kinds, which mirror Kuhn’s distinction between normal and revolutionary science.

Some scientists, he suggests, are what we might call “hill climbers”. They tend to be highly skilled in technical terms and their work mostly takes established lines of insight that pushes them further; they climb upward into the hills in some abstract space of scientific fitness, always taking small steps to improve the agreement of theory and observation. These scientists do “normal” science. In contrast, other scientists are more radical and adventurous in spirit, and they can be seen as “valley crossers”. They may be less skilled technically, but they tend to have strong scientific intuition — the ability to spot hidden assumptions and to look at familiar topics in totally new ways.

To be most effective, Smolin argues, science needs a mix of hill climbers and valley crossers. Too many hill climbers doing normal science, and you end up sooner or later with lots of them stuck on the tops of local hills, each defending their own territory. Science then suffers from a lack of enough valley crossers able to strike out from those intellectually tidy positions to explore further away and find higher peaks.


Ian Smith said...

The problem with academic science is it's academic.

If the goal is to get things done the best way is teams of scientists all working for the same thing and largely in the same place. This is the way the bomb was made and the way man was sent to the moon. It is the way all large engineerig projects are undertaken.

The current organization of science into professor and students research groups funded by the professor's tireless panhandling for grants is inefficient and EVIL.

The Freed Slave said...

Amazing post! Thank you. Too many traders or investment managers are hill climbers too.

CW said...

Meanwhile, we have claims like the following (new in Science [3 April 2009]):

Distilling Free-Form Natural Laws from Experimental Data
Michael Schmidt and Hod Lipson

For centuries, scientists have attempted to identify and document analytical laws that underlie physical phenomena in nature. Despite the prevalence of computing power, the process of finding natural laws and their corresponding equations has resisted automation. A key challenge to finding analytic relations automatically is defining algorithmically what makes a correlation in observed data important and insightful. We propose a principle for the identification of nontriviality. We demonstrated this approach by automatically searching motion-tracking data captured from various physical systems, ranging from simple harmonic oscillators to chaotic double-pendula. Without any prior knowledge about physics, kinematics, or geometry, the algorithm discovered Hamiltonians, Lagrangians, and other laws of geometric and momentum conservation. The discovery rate accelerated as laws found for simpler systems were used to bootstrap explanations for more complex systems, gradually uncovering the "alphabet" used to describe those systems.

CW said...

PS: Put Cosma Shalizi on the case.

Mark said...

Every word rings true. This, unfortunately, is the nature of the beast. The penalty for valley crossing is very high.

On the other hand, at certain times the dynamic changes. My field (mobile robotics) steadily climbed a hill for much of the last two decades. Practically the whole field was labouring away up the hill, and by now the gains to be found are tiny. So we're going through a Cambrian explosion as people scatter off in all directions. The average quality of papers is way down, but novelty is way up. Pretty soon we'll all be labouring up a few hills again. Punctuated equilibrium in action!

CW said...

See this commentary on the increasing irrelevance of political science—indicative of a very widespread trend:

Scholars on the Sidelinesby Joseph S. Nye Jr.
Washington Post, 4/13/2009

Scholars are paying less attention to questions about how their work relates to the policy world, and in many departments a focus on policy can hurt one's career. Advancement comes faster for those who develop mathematical models, new methodologies or theories expressed in jargon that is unintelligible to policymakers. A survey of articles published over the lifetime of the American Political Science Review found that about one in five dealt with policy prescription or criticism in the first half of the century, while only a handful did so after 1967. Editor Lee Sigelman observed in the journal's centennial issue that "if 'speaking truth to power' and contributing directly to public dialogue about the merits and demerits of various courses of action were still numbered among the functions of the profession, one would not have known it from leafing through its leading journal."

Dr. Pradeep V Desai said...

Acceptable to a point... while one can blame the academic method of evaluation of scientific work, the behavior pattern as described will continue. People learn by experience and settle down to the most comfortable position and environment naturally. The system for measuring the sucess for with regard to degree of risk taking still not available. Creating a new path by treading along for the first time is still not very convincing for many. So called people doing "normal science" will continue, along with few path breakers and hill climbers looking for climbing new hills without fully accessing their capability will co-exist. Truly nothing to worry about, change will take place as time elapses and space collapses.

Blog Archive