Do flu vaccines work?

The more I learn about the quality standards of medical research, the less I believe what my doctor tells me! Notice how hard it is, even in a "scientific" context to oppose the conventional wisdom.

Related posts: bounded cognition, Bouchard against group think, the future of innovation.

More from Gary Taubes on diet and nutrition issues, and WIRED on placebo mysteries.

As a postdoc I briefly dated a woman who was doing graduate work at Harvard's school of public health. I was a bit surprised at her low opinion of the scientific and statistical prowess of medical doctors, at least until I thought back to all the pre-med students I had taught while in grad school ;-) She pointed out to me that most MD's (as opposed to MD/PhD's) aren't scientists at all -- they simply apply what they're taught in medical school.

Atlantic Monthly: ... When Lisa Jackson, a physician and senior investigator with the Group Health Research Center, in Seattle, began wondering aloud to colleagues if maybe something was amiss with the estimate of 50 percent mortality reduction for people who get flu vaccine, the response she got sounded more like doctrine than science. “People told me, ‘No good can come of [asking] this,’” she says. “‘Potentially a lot of bad could happen’ for me professionally by raising any criticism that might dissuade people from getting vaccinated, because of course, ‘We know that vaccine works.’ This was the prevailing wisdom.”

Nonetheless, in 2004, Jackson and three colleagues set out to determine whether the mortality difference between the vaccinated and the unvaccinated might be caused by a phenomenon known as the “healthy user effect.” They hypothesized that on average, people who get vaccinated are simply healthier than those who don’t, and thus less liable to die over the short term. People who don’t get vaccinated may be bedridden or otherwise too sick to go get a shot. They may also be more likely to succumb to flu or any other illness, because they are generally older and sicker. To test their thesis, Jackson and her colleagues combed through eight years of medical data on more than 72,000 people 65 and older. They looked at who got flu shots and who didn’t. Then they examined which group’s members were more likely to die of any cause when it was not flu season.

Jackson’s findings showed that outside of flu season, the baseline risk of death among people who did not get vaccinated was approximately 60 percent higher than among those who did, lending support to the hypothesis that on average, healthy people chose to get the vaccine, while the “frail elderly” didn’t or couldn’t. In fact, the healthy-user effect explained the entire benefit that other researchers were attributing to flu vaccine, suggesting that the vaccine itself might not reduce mortality at all. Jackson’s papers “are beautiful,” says Lone Simonsen, who is a professor of global health at George Washington University, in Washington, D.C., and an internationally recognized expert in influenza and vaccine epidemiology. “They are classic studies in epidemiology, they are so carefully done.”

The results were also so unexpected that many experts simply refused to believe them. Jackson’s papers were turned down for publication in the top-ranked medical journals. One flu expert who reviewed her studies for the Journal of the American Medical Association wrote, “To accept these results would be to say that the earth is flat!” When the papers were finally published in 2006, in the less prominent International Journal of Epidemiology, they were largely ignored by doctors and public-health officials. “The answer I got,” says Jackson, “was not the right answer.”

... THE MOST vocal—and undoubtedly most vexing—critic of the gospel of flu vaccine is the Cochrane Collaboration’s Jefferson, who’s also an epidemiologist trained at the famed London School of Tropical Hygiene, and who, in Lisa Jackson’s view, makes other skeptics seem “moderate by comparison.” Among his fellow flu researchers, Jefferson’s outspokenness has made him something of a pariah. At a 2007 meeting on pandemic preparedness at a hotel in Bethesda, Maryland, Jefferson, who’d been invited to speak at the conference, was not greeted by any of the colleagues milling about the lobby. He ate his meals in the hotel restaurant alone, surrounded by scientists chatting amiably at other tables. He shrugs off such treatment. As a medical officer working for the United Nations in 1992, during the siege of Sarajevo, he and other peacekeepers were captured and held for more than a month by militiamen brandishing AK-47s and reeking of alcohol. Professional shunning seems trivial by comparison, he says.

“Tom Jefferson has taken a lot of heat just for saying, ‘Here’s the evidence: it’s not very good,’” says Majumdar. “The reaction has been so dogmatic and even hysterical that you’d think he was advocating stealing babies.” Yet while other flu researchers may not like what Jefferson has to say, they cannot ignore the fact that he knows the flu-vaccine literature better than anyone else on the planet. He leads an international team of researchers who have combed through hundreds of flu-vaccine studies. The vast majority of the studies were deeply flawed, says Jefferson. “Rubbish is not a scientific term, but I think it’s the term that applies.” Only four studies were properly designed to pin down the effectiveness of flu vaccine, he says, and two of those showed that it might be effective in certain groups of patients, such as school-age children with no underlying health issues like asthma. The other two showed equivocal results or no benefit.

... In the flu-vaccine world, Jefferson’s call for placebo-controlled studies is considered so radical that even some of his fellow skeptics oppose it.