Tag: abstracts

Hierarchy of Evidence and Vaccine Papers

Evidence is evidence, right?

Nope.

There is a hierarchy of evidence, from weakest to strongest, that help folks make decisions about science and medicine.

That’s why you can’t just search Google or PubMed, read abstracts, and say that you have done your research.

Hierarchy of Evidence

For any study, you have to review and judge the quality of the evidence it provides.

A meta-analysis with over 1.2 million kids found that vaccines were not associated with autism, while Wakefield's retracted case series included only 12 children.
A meta-analysis with over 1.2 million kids found that vaccines were not associated with autism, while Wakefield’s retracted case series included only 12 children.

Is it a case report (a glorified anecdote), case series, or animal study (lowest quality evidence)?

Or a systemic review or meta-analyses (highest quality evidence)?

“The first and earliest principle of evidence-based medicine indicated that a hierarchy of evidence exists. Not all evidence is the same. This principle became well known in the early 1990s as practising physicians learnt basic clinical epidemiology skills and started to appraise and apply evidence to their practice. Since evidence was described as a hierarchy, a compelling rationale for a pyramid was made.”

Murad et al. on the New Evidence Pyramid

What about case control studies, cohort studies, and randomized controlled trials?

They lie somewhere in between on the hierarchy of evidence scale or pyramid.

And there are other factors to consider when judging the reliability of a study.

“Ultimately, the interpretation of the medical literature requires not only the understanding of the strengths and limitations of different study designs but also an appreciation for the circumstances in which the traditional hierarchy does not apply and integration of complementary information derived from various study designs is needed.”

Ho et al. on Evaluating the Evidence

For example, you might also have to take into account the sample size of the study.

A study can be underpowered if it doesn’t have enough subjects. Unfortunately, even an underpowered study will give you results. They likely won’t be statistically significant results, but folks don’t always realize that.

Even a meta-analysis, usually considered to be at the top of the hierarchy of evidence pyramid, can have problems that make their results less useful, such as not using appropriate inclusion criteria when selecting studies and leaving out important studies.

All in all, there are many factors to look at when reading a medical paper and considering if the results are valid and should influence what you do and how you think. This is especially true when looking at low quality vaccine papers, many of which the anti-vaccine movement uses to scare people, even though they are often poorly designed, and several of which have been retracted.

What to Know About the Hierarchy of Evidence

Learning about the hierarchy of evidence can help you better evaluate medical studies and vaccine papers and understand that there is more to doing your research about vaccines than searching PubMed and reading abstracts.

More on the Hierarchy of Evidence

 

Using Pubmed to Do Research About Vaccines

A lot of the vaccine research that folks do is on PubMed.

Using PubMed to Do Research About Vaccines

And that’s not necessarily a bad thing.

“PubMed comprises more than 27 million citations for biomedical literature from MEDLINE, life science journals, and online books. Citations may include links to full-text content from PubMed Central and publisher web sites.”

All of the studies that say that vaccines are safe, that vaccines work, and that vaccines are necessary are in PubMed.

So are the studies that show that vaccines are not associated with autism, SIDS, and other so-called vaccine induced diseases, like ASIA.

Unfortunately, there are also poorly done studies in PubMed that do purport that vaccines are associated with autism and that ASIA is a real thing.

Can You Use PubMed to Do Research About Vaccines?

Kelly Brogan didn't make history in getting a case report published in a low impact journal who's editorial board includes a Reiki Master, chiropractors, and naturopaths.
Kelly Brogan didn’t make history in getting a case report published in a low impact journal who’s editorial board includes a Reiki Master, chiropractors, and naturopaths.

Just like anyone can put up a website or Facebook page and say whatever they want, almost anyone can get a study or article published in a journal and get it indexed in PubMed.

While PubMed is an index with over 27 million citations, it doesn’t do anything to evaluate those citations to see if they include studies with design flaws, conflicts of interest, or are simply fraudulent.

That means that you need to know that a study does not get a badge of legitimacy for simply being in PubMed!

And it does not automatically mean that the evidence and conclusions from the article are of high quality just because it is listed in PubMed.

So use PubMed to find articles to help you do research about vaccines, but then read the article from beginning to end, not just the abstract, and make sure it is an article you can trust:

  • Was it published in a legitimate journal, like Vaccine or Pediatrics, and some of these high-impact journals? (good)
  • Was it published in a predatory journals?  (bad)
  • Does it involve simply looking at VAERS data?  (usually bad)
  • Is it written by folks with a conflict of interest that makes the article biased?  (bad)
  • Has it already been refuted by other people because it wasn’t designed properly or had other major flaws?  (bad)
  • Is it written by people who have expertise on the topic they are writing about? (good)
  • Has it been retracted?  (very bad)
  • Is it a case report (a glorified anecdote), case series, or animal study (lowest quality evidence) or a systemic review or meta analyses (highest quality evidence)?
  • Is it a case control study, cohort study, and randomized controlled trial, which lie somewhere in between case reports and reviews on the hierarchy of evidence scale?

Are you ready to get educated about vaccines?

That’s great, but PubMed shouldn’t be your first stop, or your only stop.

As you do your research or get bombarded with a list of links or abstracts from PubMed, remember that there is a hierarchy of evidence to consider before deciding if a paper or study is really evidence of anything. And finding a case report, study on rats, or an invitro study won’t win you an argument about vaccines when there are randomized control trials and systemic reviews on the other side.

What to Know About Using PubMed to Do Research About Vaccines

PubMed is a giant index of journal articles, but simply being in PubMed doesn’t mean that an article or study is reliable or of high quality, whether it is about vaccines, a vaccine-preventable disease, or any other medical topic.

More on Using PubMed to Do Research About Vaccines