Tuesday, October 27, 2009

Do flu vaccines work?

The more I learn about the quality standards of medical research, the less I believe what my doctor tells me! Notice how hard it is, even in a "scientific" context to oppose the conventional wisdom.

Related posts: bounded cognition, Bouchard against group think, the future of innovation.

More from Gary Taubes on diet and nutrition issues, and WIRED on placebo mysteries.

As a postdoc I briefly dated a woman who was doing graduate work at Harvard's school of public health. I was a bit surprised at her low opinion of the scientific and statistical prowess of medical doctors, at least until I thought back to all the pre-med students I had taught while in grad school ;-) She pointed out to me that most MD's (as opposed to MD/PhD's) aren't scientists at all -- they simply apply what they're taught in medical school.

Atlantic Monthly: ... When Lisa Jackson, a physician and senior investigator with the Group Health Research Center, in Seattle, began wondering aloud to colleagues if maybe something was amiss with the estimate of 50 percent mortality reduction for people who get flu vaccine, the response she got sounded more like doctrine than science. “People told me, ‘No good can come of [asking] this,’” she says. “‘Potentially a lot of bad could happen’ for me professionally by raising any criticism that might dissuade people from getting vaccinated, because of course, ‘We know that vaccine works.’ This was the prevailing wisdom.”

Nonetheless, in 2004, Jackson and three colleagues set out to determine whether the mortality difference between the vaccinated and the unvaccinated might be caused by a phenomenon known as the “healthy user effect.” They hypothesized that on average, people who get vaccinated are simply healthier than those who don’t, and thus less liable to die over the short term. People who don’t get vaccinated may be bedridden or otherwise too sick to go get a shot. They may also be more likely to succumb to flu or any other illness, because they are generally older and sicker. To test their thesis, Jackson and her colleagues combed through eight years of medical data on more than 72,000 people 65 and older. They looked at who got flu shots and who didn’t. Then they examined which group’s members were more likely to die of any cause when it was not flu season.

Jackson’s findings showed that outside of flu season, the baseline risk of death among people who did not get vaccinated was approximately 60 percent higher than among those who did, lending support to the hypothesis that on average, healthy people chose to get the vaccine, while the “frail elderly” didn’t or couldn’t. In fact, the healthy-user effect explained the entire benefit that other researchers were attributing to flu vaccine, suggesting that the vaccine itself might not reduce mortality at all. Jackson’s papers “are beautiful,” says Lone Simonsen, who is a professor of global health at George Washington University, in Washington, D.C., and an internationally recognized expert in influenza and vaccine epidemiology. “They are classic studies in epidemiology, they are so carefully done.”

The results were also so unexpected that many experts simply refused to believe them. Jackson’s papers were turned down for publication in the top-ranked medical journals. One flu expert who reviewed her studies for the Journal of the American Medical Association wrote, “To accept these results would be to say that the earth is flat!” When the papers were finally published in 2006, in the less prominent International Journal of Epidemiology, they were largely ignored by doctors and public-health officials. “The answer I got,” says Jackson, “was not the right answer.”

... THE MOST vocal—and undoubtedly most vexing—critic of the gospel of flu vaccine is the Cochrane Collaboration’s Jefferson, who’s also an epidemiologist trained at the famed London School of Tropical Hygiene, and who, in Lisa Jackson’s view, makes other skeptics seem “moderate by comparison.” Among his fellow flu researchers, Jefferson’s outspokenness has made him something of a pariah. At a 2007 meeting on pandemic preparedness at a hotel in Bethesda, Maryland, Jefferson, who’d been invited to speak at the conference, was not greeted by any of the colleagues milling about the lobby. He ate his meals in the hotel restaurant alone, surrounded by scientists chatting amiably at other tables. He shrugs off such treatment. As a medical officer working for the United Nations in 1992, during the siege of Sarajevo, he and other peacekeepers were captured and held for more than a month by militiamen brandishing AK-47s and reeking of alcohol. Professional shunning seems trivial by comparison, he says.

“Tom Jefferson has taken a lot of heat just for saying, ‘Here’s the evidence: it’s not very good,’” says Majumdar. “The reaction has been so dogmatic and even hysterical that you’d think he was advocating stealing babies.” Yet while other flu researchers may not like what Jefferson has to say, they cannot ignore the fact that he knows the flu-vaccine literature better than anyone else on the planet. He leads an international team of researchers who have combed through hundreds of flu-vaccine studies. The vast majority of the studies were deeply flawed, says Jefferson. “Rubbish is not a scientific term, but I think it’s the term that applies.” Only four studies were properly designed to pin down the effectiveness of flu vaccine, he says, and two of those showed that it might be effective in certain groups of patients, such as school-age children with no underlying health issues like asthma. The other two showed equivocal results or no benefit.

... In the flu-vaccine world, Jefferson’s call for placebo-controlled studies is considered so radical that even some of his fellow skeptics oppose it.


Jorge Pullin said...

It is amazing. This has raised a firestorm in otherwise very level headed blogs. See for instance,



Anonymous said...

The placebo controlled study sounds interesting. I normally don't get a flu shot (not sure what to do this year!), but I'd participate in a study where placebo is a possibility. But maybe this somehow means I'm in the healthy-user group and therefore it doesn't matter if I get the real vaccine or not...

Curious Panda said...

The level of "firestorm" - in my opinion - is frustration from clinicians and researchers due to the new and rising trend of anti-vaccination. The anti-vaxers are mostly uneducated people who cling to bits of pseudoscience to justify their fear of modern medicine. They are championed by Jenny McCarthy with her "Google PhD" and Suzanne Sommers, who claims she had "whole body cancer". The effect of this movement has been:

Whopping Cough outbreak in Boulder - where hippy parents don't vaccinate their kids.

MMR vaccination rates in UK drops to 82%, first measles death in 14 years.

By the way, their latest "scientific" theory on autism is that vaccines stimulate testosterone production. Since testosterone is a steroid, too much could bind to crucial gene sites and prevent their activation - hence causing autism. Do you know what they are doing to "treat" it? Injecting autistic kids with hormone blockers, chemically castrating them in the process.


I read through the article on the Atlantic, and came away with three thoughts.
1) Ineffectiveness of vaccines on the elderly is nothing new. Everyone knows about it.

2) What a great story they tell! Some rebel scientist sacrifices his own career for "the truth" This will definitely grab eyeballs.

3) The authors are morons. We aren't dealing with the seasonal flu here, we are dealing with a possible pandemic strain. What's the difference? Take a look at who's dying from the flu.


We are seeing pediatric deaths counts when its not even flu season yet.

Swing flu kills kids (and although its not in this graph, there is lot of data to back this up) and adults between 18-60 at a much higher rate than the regular seasonal flu. That is what so dangerous about it. Vaccines work in the population under 60, not even Dr. Jefferson (the guy the article talks about) would contest the reams of studies.

So stop talking about how vaccines in seniors may not be effective, it works on everyone else. All the Atlantic managed to do was scare more parents and adults into not vaccinating.

Yes, We should examine whether vaccines do help the elderly. But to generalize it to the entire population at a time when the vaccine may be the only protection we have against an epidemic is totally irresponsible.

Ian Smith said...

For years I was treated for elevated blood pressure. I saw a dozen doctors. Nothing worked until I stopped three other medications. I had to look into the side effects of these myself. None of the doctors I saw had a clue.

This could not happen in a country other than the US. The US is the only country where pushiness is valued over intelligence.

Medicine is much more often than not great expense for nothing. What MDs can do for you is very limited. The prestige and authority of medicine has increased much more than its effectiveness.

That drug companies find it profitable to spend more selling to doctors than to patients proves MDs were the dull, diligent, obedient, pushy kids who sat at the front of the class. They're scum.

Seth said...

A lot of medical researchers are practicing statistics without a license. ;)

We live in a country full of people who regard science as "just a bunch of dogma" and inferior to religiously derived 'truth'. Why is that view so prevalent? I suspect one reason is the public is so much more aware of debates about 'social science' or 'educational studies' or 'nutrition standards', etc than they are of issues in harder sciences. If the only 'science' you ever hear about is some study about the role of nature vs. nurture which shows 43.712% of your 'personality' (whatever that is) comes from your genes, or some study that purports to establish that you should 'never eat red meat' (or whatever the scare-of-the-day is) you naturally become a bit jaded about scientific claims generally.

Science gets a tremendous amount of embarrassing publicity because of silly work that purports to be scientific. And this happens because the 'science' label is such a handy source of prestige for ideas that may or may not have any value.

Ian Smith said...

Whatever the theory until it is useful it is just dogma.

So far evolution has no applicaton. At best it is an explanation of why cancers return, why a cocktail is required to keep HIV from developing into AIDS.

All technology and medicine today will still stand without the assisstance of evolution. It will also stand without relativity and black holes and the rest of modern physics's mental masturbation.

It can't stand without classical physics and chemistry however.

Sherwood Smith said...

Steve thanks for taking the time to post your keen insight.

Sherwood Smith

Sean OBrien said...

There are many unanswered questions from this Atlantic Monthly article:

1 10 years of flu shots gives you exposure (and immunity) to as many as 30 different strains of influenza. Will this reduce your long term flu death rate?

2 Will today's H1N1 flu shot stop pandemic spread of a deadly mutation of H1N1?

3 What is the flu death rate for healthy people?

4 Is the flu death rate amongst vulnerable people lowered by flu vaccines?

5 I've known several people proven to have had H1N1. In healthy people it is the mildest flu I have ever seen. Are they now immune from a deadly mutation of H1N1?

6 Why are double blind placebo tests of flu vaccine unethical? Can't this be said of all such testing?

Curious Panda said...

@Sean -
5) Probably not. In the 1918 pandemic there was a first wave of the flu which hit people like any other flu season, followed by the mutant strain. Mortality rates didn't vary between pre-exposed and unexposed populations.

Andrew Foland said...

A reminder: the tau lifetime can be much more accurately calculated, than it can be measured.

I'm a physicist who's had a chance to publish articles on the use of statistics in medical practice. Medical studies are fiendishly difficult to get good samples for, due to both lack of volunteers and (often) ethical issues.

If there were good a priori solid reasons to believe something might work, you might very rationally not want to subject acceptance of that practice to the vagaries of small sample sizes.

I don't know enough about the specifics in these cases to know if a similar dynamic is at work here, just a general comment.

And: I taught the premed physics course for several years, so yes, I have an extremely keen appreciation for the limits of doctors' understanding of science.

Ian Smith said...

Both Taubes and mainstream medicine are right. Refined carbs make fat and calories are all that matters. Low carb dieters don't eat as many calories.

Taubes knows this but he's got books to sell. He's a douche.

Blog Archive