Home » , , » When Studies Collide (Revisited)

When Studies Collide (Revisited)

 

 

 

# 4983

 

 

Long time readers of this blog know I try to back up practically every `statement of fact’ with a cite from a published study, a quote from a respected researcher, or - at the very least – a link to mainstream news report.

 

You’ll find thousands of these links scattered throughout these blog posts, and I hope you find those useful.

 

But constant readers are also aware that legitimate, published and usually peer-reviewed studies don’t always agree and that sometimes they can come up with diametrically opposed conclusions.

 

Alas, science isn’t always neat, tidy, orderly and clear.  

 

We often see conflicting data, analysis, and opinion published in respected journals.  Sorting out what is true . . . or at least . . .  what is likely to be true . . . is a continual challenge for the reader.

 

This built-in ambiguity in the system can be vexing for those looking for a simple, definitive answer, but can be a boon to those who are simply looking for back up for their personal point of view.

 

I see this sort of `cherry-picking’ of studies done often online, particularly among anti-vaccine activists and proponents of certain natural or alternative’ therapies.

 

In medicine, Randomized controlled trials (RCTs) – are considered the `gold standard’  –  but can be impossible to conduct ethically when attempting to evaluate a potentially life saving drug or procedure.

 

And even peer-reviewed RCTs  published in prestigious journals deserve a dash of skepticism on the part of the reader.

 

Last July we saw a cautionary note from Johns Hopkins Medicine on RCTs. Here  is the `money quote’, but follow the link to read the whole thing (emphasis mine).

Overall, 41 percent of the 146 trials in the review had improper or poorly described randomization techniques. Industry-funded trials were six times more likely to have high risk for biased randomization than government-funded trials or those funded by nonprofit organizations.

 

Since RCTs are so difficult to mount, we are often left with less rigorous research methods such as observational studies or retrospective analysis - which may produce valid results - but may also be more prone to bias or error.

 

While we would all like medicine to be based on treatments, drugs, and procedures proved – beyond a shadow of a doubt – to be safe and effective  . . . sometimes we must accept a lower burden of proof -the preponderance of evidence – instead.

 

Which is why I try to be cautious when reading new studies, and try not to invest myself too heavily into any single result. I know from experience that only time, peer-review, and further study will reveal whether their conclusions are valid.

 

Which brings us to a new study, and its ensuing controversy, brought to us via a terrific detailed report by Robert Roos – News Editor of CIDRAP news.  

 

You’ll want to follow the link and read it in its entirety.  When you return, I’ll have more.

 

 

Flu + pneumococcal vaccines in elderly: compound benefits?

Robert Roos * News Editor

Oct 14, 2010 (CIDRAP News) – A large prospective study from Hong Kong raises the possibility that elderly people with chronic illnesses can significantly reduce their risk of stroke and heart attack by getting vaccinated against influenza and pneumococcal disease at the same time, but other researchers are greeting the findings with a mix of caution and skepticism.

 

(Continue . . . )

 

 

As an advocate of both flu shots and the pneumococcal vaccines, I obviously hope that this research is valid. But for now their results are just another data point to be considered . . .  along side other results. 

 

Since we are traipsing through the scientific research minefield this morning, you might wish to revisit a few of these other controversies.

 

`The  Canadian Problem’

News of a controversial, yet unpublished, Canadian study began to emerge in September 2009 suggesting those who had received a seasonal flu shot in the previous year were more susceptible to the pandemic H1N1 virus.

 

News of this `link’ temporarily disrupted seasonal vaccination plans in Canada(see Ontario Adjusts Vaccination Plan), while  CDC and the World Health Organization stated they saw no evidence in their data to suggest a link.

 

A study published in the BMJ in October of 2009 suggested exactly the opposite - that the seasonal flu vaccination may be slightly protective against the swine flu  (see When Studies Collide).

 

Since then, we’ve seen the study published (see New Canadian studies suggest seasonal flu shot increased H1N1 risk) while other research again claimed no link (Study: Eurosurveillance On `The Canadian Problem’).

 

 

Tamiflu Efficacy

 

Nearly a year ago, the BMJ published a cluster of articles which questioned the lack of supportive scientific evidence for the use of Tamiflu against seasonal flu (see BMJ: A Review Of Tamiflu’s Efficacy Against Seasonal Influenza).

 

The bottom line was there was insufficient evidence, according to the authors, to conclude either for or against Tamiflu for use in healthy adults with seasonal influenza.

 

Despite the fact that this study was based on seasonal flu, not pandemic influenza, these results were used by some to criticize the stockpiling of Tamiflu by governments around the world. 

 

The `Tamiflu is useless’ meme became very popular on some websites.

 

Of course, other studies (alas, not RCTs) continued to show significant benefit – particularly among those with severe symptoms, avian flu, or those with risk factors for complications.

 

Swine flu patients benefited from taking Tamiflu, says study
 

Earlier this year, another observational study appeared in JAMA  (see Study: Antivirals Saved Lives Of Pregnant Women) that strongly suggested that Tamiflu was life saving for some patients with pandemic flu.

 

In research that appeared in the IDSA’s  Journal of Infectious Diseases last month, a study that looked at the course of treatment of H5N1 patients found that of 308 cases studied, the overall survival rate was a dismal 43.5%.

 

But . . . among  those who received at least one dose of Tamiflu . . .  60% survived . . .  as opposed to only 24% who received no antivirals (see Study: Antiviral Therapy For H5N1).

 

 

Surgical Masks vs. N95 Respirators

 

For decades, the assumption was that only properly fitted N95 masks protected the wearer, and that surgical masks were worn by HCWs to protect the patient during invasive procedures.

 

image image

N-95 Respirator         Surgical Facemask

 

But over the past year we’ve seen dueling studies that alternately show surgical masks to be an effective barrier against respiratory viruses  . . . or pretty much useless.


Take your pick.

 

In October the NEJM published a perspective article (see NEJM Perspective: Respiratory Protection For HCWs) based on the recent IOM evaluation of surgical masks vs. respirators, and came out in favor of the N95.

 

A few days later JAMA (Journal of the American Medical Association) published a study which reported that HCWs using surgical masks experienced `noninferior rates of laboratory-confirmed influenza’.

 

In March of this year, we saw the following study (see Study: Efficacy of Facemasks Vs. Respirators).

 

To complicate matters, after these studies suggested that surgical masks are reasonably protective against influenza - we recently saw a new study that questions whether surgical masks or N95 respirators provide any substantial protection to the wearer at all.

 

No . . .  I don’t make these things up.  I just report on them.

 

The study appears in the September issue of the AJIC (American Journal of Infection Control), and is titled:

Quantifying exposure risk: Surgical masks and respirators

Keith T. Diaz, MD, Gerald C. Smaldone, MD, PhD

 

 

If you aren’t confused yet, well . . . you must not be paying attention.

 

None of this is intended to suggest that scientific research is useless, or even badly flawed. 

 

Only that gaining scientific knowledge is a process . . . one that evolves over time . . . and scientific certainty is an elusive, often unobtainable goal.

 

So the next time you see a study cited here, or any place else for that matter, keep in mind that while it may be useful . . .  it may not be the last word on the subject.  

 

In fact, I can practically guarantee it.

Related Post:

Research
Vaccine
CIDRAP

Widget by [ Iptek-4u ]
Labels: , ,