Let the machines do it

Via an intrepid correspondent... the Times covers numerous areas where human "experts" are outperformed by machine intelligence. Of course, we're not quite ready to let the machines take over just yet. I think IA = Intelligence Amplification is much more promising in the short term than AI = Artifical Intelligence.

NYTimes: Do you think your high-paid managers really know best? A Dutch sociology professor has doubts.

The professor, Chris Snijders of the Eindhoven University of Technology, has been studying the routine decisions that managers make, and is convinced that computer models, by and large, can do a better job of it. He even issued a challenge late last year to any company willing to pit its humans against his algorithms.

“As long as you have some history and some quantifiable data from past experiences,” Mr. Snijders claims, a simple formula will soon outperform a professional’s decision-making skills. “It’s not just pie in the sky,” he said. “I have the data to support this.”

Some of Mr. Snijders’s experiments from the last two years have looked at the results that purchasing managers at more than 300 organizations got when they placed orders for computer equipment and software. Computer models given the same tasks achieved better results in categories like timeliness of delivery, adherence to the budget and accuracy of specifications.

No company has directly taken Mr. Snijders up on his challenge. But a Dutch insurer, Interpolis, whose legal aid department has been expanding rapidly in recent years, called in Mr. Snijders to evaluate a computer model it had designed to automate the routing of new cases — a job previously handled manually by the department’s in-house legal staff.

The manager in charge of the project, Ludo Smulders, said the model was much faster and more accurate than the old system. “We’re very satisfied about the results it’s given our organization,” he said. “That doesn’t mean there are no daily problems, but the problems are much smaller than when the humans did it by hand. And it lets them concentrate more on giving legal advice, which is what their job is.”

Mr. Snijders’s work builds on something researchers have known for decades: that mathematical models generally make more accurate predictions than humans do. Studies have shown that models can better predict, for example, the success or failure of a business start-up, the likelihood of recidivism and parole violation, and future performance in graduate school.

They also trump humans at making various medical diagnoses, picking the winning dogs at the racetrack and competing in online auctions. Computer-based decision-making has also grown increasingly popular in credit scoring, the insurance industry and some corners of Wall Street.

The main reason for computers’ edge is their consistency — or rather humans’ inconsistency — in applying their knowledge.

“People have a misplaced faith in the power of judgment and expertise,” said Greg Forsythe, a senior vice president at Schwab Equity Ratings, which uses computer models to evaluate stocks.

The algorithms behind so-called quant funds, he said, act with “much greater depth of data than the human mind can. They can encapsulate experience that managers may not have.” And critically, models don’t get emotional. “Unemotional is very important in the financial world,” he said. “When money is involved, people get emotional.” Many putative managerial qualities, like experience and intuition, may in fact be largely illusory. In Mr. Snijders’s experiments, for example, not only do the machines generally do better than the managers, but some managers perform worse over time, as they develop bad habits that go uncorrected from lack of feedback.

Other cherished decision aids, like meeting in person and poring over dossiers, are of equally dubious value when it comes to making more accurate choices, some studies have found, with face-to-face interviews actually degrading the quality of an eventual decision.

“People’s overconfidence in their ability to read someone in a half-an-hour interview is quite astounding,” said Michael A. Bishop, an associate professor of philosophy at Northern Illinois University who studies the social implications of these models.

And the effects can be serious. “Models will do much better in predicting violence than will parole officers, and in that case, not using them leads to a more dangerous society,” he said. “But people really don’t believe that the models are as accurate as they are.”

Models have other advantages beyond their accuracy and consistency. They allow an organization to codify and centralize its hard-won knowledge in a concrete and easily transferable form, so it stays put when the experts move on. Models also can teach newcomers, in part by explaining the individual steps that lead to a given choice. They are also faster than people, are immune to fatigue and give the human experts more time to work on other tasks beyond the current scope of machines.

So if they’re so good, why aren’t they already used everywhere?

Not everyone is convinced that managers are incorrigibly myopic. “I’ve never seen any evidence that there is a pattern of decline at all, and it just doesn’t fit with the way management literature is going, which is all around the emotional intelligence angle,” said Laura Empson, the director of the Clifford Chance Center of the Said Business School at Oxford University.

“I think there are a lot of people who have a strong technological orientation who would agree life would be a lot simpler if it weren’t for the humans,” she said. “But the reality is, organizations do have a lot of very intense and complicated human issues within them.”

Max H. Bazerman, a professor at Harvard Business School, wonders how many managerial decisions can actually be modeled. “The vast majority of decisions that we make in professional life don’t have this quality,” he said.

He agrees that models can make better decisions about credit card applications and college admissions, he said, “but there are many decisions that are much more unique, where that database doesn’t exist. I’m as skeptical about human intuition as these folks, but it’s not only a computer model that we replace it with. Sometimes it’s thinking more clearly.”

Many in the field of computer-assisted decision-making still refer to the debacle of Long Term Capital Management, a highflying hedge fund that counted several Nobel laureates among its founders. Its algorithms initially mastered the obscure worlds of arbitrage and derivatives with remarkable skill, until the devaluation of the Russian ruble in 1998 sent the fund into a tailspin.

“As long as the underlying conditions were in order, the computer model was almost like a money machine,” said Roger A. Pielke Jr., a professor of environmental studies at the University of Colorado whose work focuses on the relation between science and decision-making. “But when the assumptions that went into the creation of those models were violated, it led to a huge loss of money, and the potential collapse of the global financial system.”

In such situations, “you can never hope to capture all of the contingencies or variables inside of a computer model,” he said. “Humans can make big mistakes also, but humans, unlike computer models, have the ability to recognize when something isn’t quite right.”

Another problem with the models is the issue of accountability. Mr. Forsythe of Schwab pointed out that “there’s no such thing as a 100 percent quantitative fund,” in part because someone has to be in charge if the unexpected happens. “If I’m making decisions,” he said, “I don’t want to give up control and say, ‘Sorry, the model told me.’ The client wants to know that somebody is behind the wheel.”

Still, some consider the continuing ascendance of models as inevitable, and recommend that people start figuring out the best way to adapt to the role reversal. Mark E. Nissen, a professor at the Naval Postgraduate School in Monterey, Calif., who has been studying computer-vs.-human procurement, sees a fundamental shift under way, with humans becoming increasingly peripheral in making routine decisions, concentrating instead on designing ever-better models.

“The newest space, and the one that’s most exciting, is where machines are actually in charge, but they have enough awareness to seek out people to help them when they get stuck,” he said — for example, when making “particularly complex, novel, or risky” decisions.

The ideal future, then, may lie in letting computers and people each do what they do best. One way to facilitate this development is to train people to identify the typical cognitive foibles that lead to bad choices. “I’ve now worked with these models for so long,” Mr. Snijders said, “that my instincts have changed along the way.”

As Mr. Bishop of Northern Illinois University puts it, by making smart use of computer models’ advantages, “you’ll become like the crafty A student who doesn’t work that hard but gets mostly right answers, rather than the really hard-working student who gets lots of wrong answers and as a result gets C’s.”