Friday, June 19, 2009

Why are modern scientists so dull?

On the subject of personality factors and success in science, here is a provocative essay by UK professor Bruce Charlton. (PDF version.) He claims that the modern system selects for conscientiousness over raw intelligence, with negative consequences.

Question: why are so many leading modern scientists so dull and lacking in scientific ambition?

Answer: because the science selection process ruthlessly weeds-out interesting and imaginative people. At each level in education, training and career progression there is a tendency to exclude smart and creative people by preferring Conscientious and Agreeable people. The progressive lengthening of scientific training and the reduced independence of career scientists have tended to deter vocational ‘revolutionary’ scientists in favour of industrious and socially adept individuals better suited to incremental ‘normal’ science. High general intelligence (IQ) is required for revolutionary science. But educational attainment depends on a combination of intelligence and the personality trait of Conscientiousness; and these attributes do not correlate closely.

...At each level in education, training and career progression there is a tendency to exclude smart and creative people by preferring conscientious and sociable people. As science becomes ever-more dominated by ‘peer review’ mechanisms, pro-social behaviour in scientists has been accorded primacy over the brilliant and inspired – but abrasive and rebellious – type of truth-seekers who used to be common among the best scientists.

A majority of senior professional scientists have been through a rigorous and prolonged process of education, selection and training to become professional researchers. Yet the nature of the rigour and the duration of the process in modern science ensures that those who come out at the end and attain long-term scientific employment are not the kind of people capable of top level, revolutionary science. They will very probably be extremely productive and socially compliant, but of only moderately high intelligence and likely to be lacking in imagination [2].

...Modern science is just too dull an activity to attract, retain or promote many of the most intelligent and creative people. In particular the requirement for around 10, 15, even 20 years of postgraduate ‘training’ before even having a chance at doing some independent research of one’s own choosing, is enough to deter almost anyone with a spark of vitality or self-respect; and utterly exclude anyone with an urgent sense of vocation for creative endeavour. Even after a decade or two of ‘training’ the most likely scientific prospect is that of researching a topic determined by the availability of funding rather than scientific importance, or else functioning as a cog in someone else’s research machine. Either way, the scientist will be working on somebody else’s problem – not his own. Why would any serious intellectual wish to aim for such a career? ...

Shorter Charlton: there are too many hoops, and we end up selecting for Agreeableness and Conscientiousness (hoop jumping abilities) rather than raw brainpower.

I partially agree with Charlton's claims, but the specifics vary from field to field. The area he seems most familiar with is medical science, which most physicists (after teaching premeds and biology students) might concur selects for conscientious rather than brilliant types ;-) In physics it seems we are quite tolerant of odd personalities -- hyper aggressive types, those with Asperger's Syndrome, etc., especially if the person in question displays tremendous ability. I would guess the same is largely true in math and engineering. In biology and medicine it may not be that easy to tell the really talented researchers from the rest (at least at early career stages), which would lead to more emphasis on personality traits. It's also true that in many areas of physics (specifically, but not limited to, the theoretical ones) one can work as a single investigator or small group lead investigator quite early. This may be less true in medicine and biology.

I discussed the current incentive system in science here, as well as the job prospects in theoretical physics. Given the situation I can't blame any students who find that alternative careers might be preferable. As I wrote here (in partial agreement with Charlton), this leads to a different kind of selection than in the past:

...Nowadays, success in science seems to be as much a selection for [certain] character or personality traits as it is a selection for talent.

Related posts: frauds , success vs ability .

Regarding Charlton's deeper question: Where have all the geniuses gone? I offer the following from this earlier post. See also Genius, Gleick's biography of Feynman, especially pp.325-328.

... the exact topic discussed in James Gleick's book Genius. In a field where sampling of talents is sparse [like science in its earlier days] ... you might find one giant ... towering above the others, able to do things others cannot. In a well-developed, highly competitive field like modern mathematics, all the top players are "geniuses" in some sense (rare talents, one in a million), even though they don't stand out very much from each other. In Gleick's book, Feynman, discussing how long it might have taken to develop general relativity had Einstein not done it, says "We are not that much smarter than each other"!

To put it simply, if I sample sparsely from a Gaussian distribution, I might find a super-outlier in the resulting set. If I sample densely and have a high minimum cutoff for acceptable points, I will end up with a set entirely composed of outliers, but who do not stand out much from each other. Every guard in the NBA is an athletic freak of nature [and they would destroy their predecessors from the early era of professional basketball], even though they are evenly matched when playing against each other.


Unknown said...

I think it is generally true that as human activities become institutionalized, agreeableness is selected over intelligence. Hence the modern organization's emphasis on "fit".

Seth said...

This little essay is much to the point here.

Bruce Charlton said...

This is the author of the paper (Bruce G Charlton) - thanks for covering it.

I agree that selection for Conscientiousness is likely to be stronger in medical science than in physics, and of course physics selects more strongly for IQ than does medical science.

I have, however, been told by a very famous physicist that he sees no sign of real revolutionary science in the subject at present. I suspect that physics (like medicine) is filtering-out creativity, and also corrupting potentially creative people into careerism.

At any rate medical science is currently by far the largest branch of science in the world, and funding has been doubling every decade for several decades - so in this sense medical science it seems reasonable to regard med sci as most representative science at present.

My main worry is that a lot of extremely bright and creative people are nowadays being filtered out of science; probably including some potential geniuses. This especially applies to men, I suspect.

It is hard to imagine Paul Dirac as a modern professor, for example.

I am not sure what happens to these very smart and creative men nowadays. I suspect they are scattered thinly across many occupations. But it would not be particulary difficult to find out, given a sufficintly large survey including measures of intelligence and 'psychoticism' or some other measure correlated with creativity.

LBJ said...

The revolutionary scientist described in the passage reminds me of Howard Roark from the Fountainhead. Perhaps that Ayn Rand was on to something.

Sabine Hossenfelder said...

Sounds awful! While there is a lot of truth in what he says it's not quite as bad as he makes it sound. There's still places where truth-seeking is valued over agreeableness and socializing skills. Not to mention that in some countries it's more your mafia-connections that get you jobs. The trend is worrisome though.

Makes one wonder how much worse it gets when one states 'smart and creative' people are progressively excluded and 'Modern science is just too dull an activity to attract, retain or promote many of the most intelligent and creative people.' Tell that young students and we might end up in a situation where people are proud of not succeeding. I'm only half joking - just think of all the hobby scientists who proclaim themselves rebels that are not accepted by a too conservative academic system.

I would wish overall people would be somewhat more balanced when they complain about the academic system. While it has a lot of weaknesses, there are many ways to make it even worse.



Michael Salem said...

When reading about these topics I often feel the author has some motive other than presenting an honest assessment of the reality.

In theoretical physics, the first stage of selection is graduate school. This is based on grades and recommendations. These reward people who do well on tests, and those who impress faculty with their insight. In the latter case, the biggest rewards go to those who've achieved some measure of success on a research project.

The second stage of selection is the first postdoc. This is based on one's papers and recommendations. Both of these are heavily influenced by the choice of adviser. Thus top physicists are in high demand as advisers, and from what I've seen, they select based on testing students for problem-solving skills and independence by observing the student response to an initial research project.

There are a series of postdocs. Postdocs are selected for their creative output (papers).

Finally, one applies for faculty positions. In many cases this will be the first time a theoretical physicist actually conducts formal interviews. It seems to me, only at this stage of selection does personality -- such as "conscientiousness" -- enter the picture. Granted, I've heard that personality is important in faculty selection, since it's such a long-term commitment. But, by this time, the scientists has had on order of a decade to establish his/her merit as a scientist. I would think, at least in the extreme cases, merit trumps personality.

Thus, I don't see any impediment to the greatest thinkers rising through the system unencumbered. Granted, for "average" physicists, much will depend on chance details such as who was the adviser, how one presents, etc. But the average physicists are not the trend setters.

As why there are no more "Einsteins" and "Feynmans" -- for one I suspect perhaps there are, but it will take time to recognize. But I also suspect that science has become extremely specialized, and success in such specialized disciplines selects against "big thinkers" who might not have the focus for the more mundane corners that we have pushed ourselves into. Or, maybe we've just solved all the easy problems, and all that's left is just too hard (which is to say if Einstein or Feynman were around today, they would not be revolutionizing the field either).

Anonymous said...

It's like this in the defense industry, too.

Unknown said...

Aside from superstars however, there is a nonzero value for being able to collaborate well with others, particularly since that is how most (non-superstar) science gets done. We online tend to have a bias towards raw ability but tend to overlook the difficulties caused by genuinely "raw" geniuses. It is questionable, for example, whether Dr. House does more good than bad, given his drain on other hospital resources and especially since he does not seem to publish.

Bill Mill said...

> Every guard in the NBA is an athletic freak of nature [and they would destroy their predecessors from the early era of professional basketball], even though they are evenly matched when playing against each other.

What!?! Kobe Bryant is evenly matched against anyone else in the NBA? I can't speak to your physics example, but I can say that it's silly to claim that Kobe Bryant (or Dwayne Wade, if you prefer) aren't geniuses relative to other NBA guards.

Steve Hsu said...

You're taking "evenly matched" too literally. Of course you have some players that are better than others (there is still a distribution), but they're competitive.

On the other hand, most guards from the 50s would have no chance of making an NBA team today. And most NFL players from that era wouldn't be able to start for a division I team (way too small and slow).

Unknown said...

I don't agree that today's scientists are more "dull" than in the past. First of all, the talent pool is much larger, the social mechanisms that identify ability and sort people are more pervasive and far-reaching than before - more smart kids will get channeled into college these days than into say a blue-collar job.

Second, I doubt great scientists lack in Conscentiousness - they may be absent-minded or disorganized in other areas of their life, but when it comes to serious work, I doubt anyone can get by without being organized and detail-oriented - there's simply too much information to deal with otherwise. In my field, economics, some of the most brilliant, "maverick" economists are also famous for being neat freaks when it comes to keeping track of ideas, e.g. Fischer Black typed down every conversation he had into alphabetized text files. I can't imagine it being different in other fields.

Third, one way to raise the general IQ requirements in a field is to make it more mathematical. Economics, of course, has gotten tremendously more mathematical in recent decades, but has it resulted in more "revolutionary" science? On the contrary, a common complaint is that the math has sucked all the common sense out of it. (To be fair, it's also cut down on a lot of squishy verbal hand-waving arguments).

Finally, I think that many fields are simply harder, take more time to learn than before - there's simply more cumulative stuff to know before you can get to the research frontier. Again, speaking of economics, the amount of coursework (mostly math) that you have to know has expanded tremendously, with the result that students who aim to get into good schools have to plan their courses starting in sophomore year, or take a couple more years getting a master's degree. I don't know how it is in physics or math - can students get a deep understanding of current knowledge any faster than say 30 or 40 years ago?

Paul Yarbles said...

Perhaps that Ayn Rand was on to something.

The only thing Ayn Rand was on to was how to tell somewhat intelligent but insecure people that, by following her program, they can be really really special. Oh, that and coming up with a very funny excuse for satiating her animal lust for a much younger man.

gaddeswarup said...

While thinking about Indian scientists before Independence and after, I came to conclusions similar to Bruce Charlton's. However, the global picture seems a bit different. For one thing, I think that there is some tolerance for those who do not fit in, since the rest know that they need good ideas (think of Perelman). Secondly, there is a sort of cumulative intelligence. Interactions and collaborations make heighten people's abilities and make them do better work than they are capable of individually. Problems like Fermat's conjecture or Poincare's conjecture were thought about by great minds before but were only solved in the recent past. The situation may be different in areas where money, organization, external pressures from corporations are involved.
It is an interesting topic and would like to know more.

Ian Smith said...

This true of science and every other field in the US. The US is run by pushy people who prefer pushy people.

At my PhD school those who had research fellowships coming in dropped out at a much higher rate than those who didn't.

Formal education is a waste of time and money. It's disgusting. Standardized tests and curricula is enough. The Society of Actuaries does it. If you're smart enough you know this and becoming a professor is morally unacceptable, as morally unacceptable as becoming a fast food or tobacco executive.

Ian Smith said...

I didn't read the whole post before commenting.

Steve. If you believe there is actually such a thing as "Asperger's Syndrome", then you are one of the hoop jumpers.

donna said...

Um, I think that is true in every field, and every workplace...
That's why I am now a professional slacker...

Anonymous said...

At any rate, I liked some of the vadlo researcher cartoons!

Unknown said...

Just how does peer review differ from mob rule?

Blog Archive