Evidence is evidence, right?
There is a hierarchy of evidence, from weakest to strongest, that help folks make decisions about science and medicine.
That’s why you can’t just search PubMed, read abstracts, and say that you have done your research. For any study, you have to review and judge the quality of the evidence it provides.
Is it a case report (a glorified anecdote), case series, or animal study (lowest quality evidence) or a systemic review or meta analyses (highest quality evidence)?
What about case control studies, cohort studies, and randomized controlled trials? They lie somewhere in between on the hierarchy of evidence scale.
Anecdotes as Evidence
And where do anecdotes fit in?
“Anecdotes are uncontrolled subjective observations. I have often criticized reliance on anecdotes, which is especially problematic in medicine. The problem with anecdotes is that they are subject to a host of biases, such as confirmation bias. They are easily cherry picked, even unintentionally, and therefore can be used to support just about any position. For every anecdote, there is an equal and opposite anecdote.”
Steven Novella on The Context of Anecdotes and Anomalies
Anecdotes are not scientific evidence.
“An anecdote is a story – in the context of medicine it often relates to an individual’s experience with their disease or symptoms and their efforts to treat it. People generally find anecdotes highly compelling, while scientists are deeply suspicious of anecdotes. We are fond of saying that the plural of anecdote is anecdotes, not data.”
Steven Novella on The Role of Anecdotes in Science-Based Medicine
Believing that anecdotes are important scientific evidence is also what often drives some pediatricians to pander to fears that parents may have about vaccines, helping them create non-standard, parent-selected, delayed protection vaccine schedules.
What else do you need to know about anecdotal evidence?
Anecdotes are basically the fuel of the anti-vaccine movement.
“With little or no evidence-based information to back up claims of vaccine danger, anti-vaccine activists have relied on the power of storytelling to infect an entire generation of parents with fear of and doubt about vaccines. These parent accounts of perceived vaccine injury, coupled with Andrew Wakefield’s fraudulent research study linking the MMR vaccine to autism, created a substantial amount of vaccine hesitancy in new parents, which manifests in both vaccine refusal and the adoption of delayed vaccine schedules.”
Shelby et al on Story and science
What to Know About Anecdotes as Evidence
Anecdotes, although they are easy to believe, are not scientific evidence, and certainly shouldn’t persuade you that vaccines aren’t safe, that vaccines aren’t necessary, or that vaccines are associated with autism.
More on Anecdotes as Evidence
- The hierarchy of evidence: Is the study’s design robust?
- The hierarchy of evidence: Levels and grades of recommendation
- The Role of Anecdotes in Science-Based Medicine
- Vaccine Injury Stories: the Sacred Cows of the Internet?
- 5 reasons why anecdotes are totally worthless
- Anecdotal Evidence
- Study – What makes anti-vaccine websites persuasive? A content analysis of techniques used by anti-vaccine websites to engender anti-vaccine sentiment
- Study – Story and science
- No Love for Anecdotal Evidence
- The Context of Anecdotes and Anomalies
- If anecdotes are evidence, why aren’t you drinking paint thinner?
- Not all evidence is created equally
- Hierarchy of scientific evidence – keys to scientific skepticism
- Evidence-Based Practice
- HPV vaccine anecdotes – not the basis of real science
- [Abstracts] vs. anecdotes: What we have here is a failure to communicate
- Anecdotal Amish-don’t-vaccinate claims disproved by fact-based study
- I was Duped by the Anti-Vaccine Movement
- Watch John Oliver Explore What Fuels the Anti-Vaccine Movement
- GRADE working group
- GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
- Study – Hierarchy of evidence: a framework for ranking evidence evaluating healthcare interventions.
- Study – When should we change our clinical practice based on the results of a clinical study? The hierarchy of evidence.