For any study, you have to review and judge the quality of the evidence it provides.
Is it a case report (a glorified anecdote), case series, or animal study (lowest quality evidence)?
Or a systemic review or meta-analyses (highest quality evidence)?
“The first and earliest principle of evidence-based medicine indicated that a hierarchy of evidence exists. Not all evidence is the same. This principle became well known in the early 1990s as practising physicians learnt basic clinical epidemiology skills and started to appraise and apply evidence to their practice. Since evidence was described as a hierarchy, a compelling rationale for a pyramid was made.”
Murad et al. on the New Evidence Pyramid
What about case control studies, cohort studies, and randomized controlled trials?
They lie somewhere in between on the hierarchy of evidence scale or pyramid.
And there are other factors to consider when judging the reliability of a study.
“Ultimately, the interpretation of the medical literature requires not only the understanding of the strengths and limitations of different study designs but also an appreciation for the circumstances in which the traditional hierarchy does not apply and integration of complementary information derived from various study designs is needed.”
Ho et al. on Evaluating the Evidence
For example, you might also have to take into account the sample size of the study.
A study can be underpowered if it doesn’t have enough subjects. Unfortunately, even an underpowered study will give you results. They likely won’t be statistically significant results, but folks don’t always realize that.
Even a meta-analysis, usually considered to be at the top of the hierarchy of evidence pyramid, can have problems that make their results less useful, such as not using appropriate inclusion criteria when selecting studies and leaving out important studies.
All in all, there are many factors to look at when reading a medical paper and considering if the results are valid and should influence what you do and how you think. This is especially true when looking at low quality vaccine papers, many of which the anti-vaccine movement uses to scare people, even though they are often poorly designed, and several of which have been retracted.
What to Know About the Hierarchy of Evidence
Learning about the hierarchy of evidence can help you better evaluate medical studies and vaccine papers and understand that there is more to doing your research about vaccines than searching PubMed and reading abstracts.
They instead turn to alternative medicine when their kids get sick and for their preventative care.
Those Times Alternative Medicine Got It Wrong
While it is true that science gets it wrong sometimes, these people seem to fail to consider that alternative medicine does too.
“…there’s no such thing as conventional or alternative or complementary or integrative or holistic medicine. There’s only medicine that works and medicine that doesn’t. And the best way to sort it out is by carefully evaluating scientific studies – not by visiting Internet chat rooms, reading magazine articles, or talking to friends.”
Paul Offit, MD on Do You Believe in Magic
More than that, they hardly ever get it right.
Need some examples?
Ayurvedic treatments can be contaminated with toxic metals
biomed treatments for autism – from restrictive diets and chelation to coffee and bleach enemas, these “cures” and treatments have not been shown to be safe, are sometimes known to be dangerous, and don’t even work
Cannabis Oil for kids with cancer – while marijuana-derived products might help some medical conditions, it doesn’t cure cancer
chiropractic neck manipulation of newborns and infants has no benefits and has caused deaths
chronic Lyme disease is not a recognized condition in modern medicine, but that doesn’t keep some ‘Lyme literate’ practitioners from recommending and charging patients for all sorts of unnecessary and sometimes harmful “treatments”
faith healing is still allowing children to die of very treatable conditions, from diabetes and appendicitis to common infections and premature babies
Gerson protocol – often discussed with other forms of cancer quackery this “radical nutritional program combined with purges (particularly coffee enemas)” is believed by some to cure cancer – it doesn’t
HIV denialism – yes, this is a thing, and tragically took the life of Christine Maggiore, her daughter, and many others who eventually died of AIDS
homemade baby formula – notorious for leaving out important nutrients, from iron vitamin D to enough calories for a growing baby
Hoxsey treatment – a natural treatment for cancer that has been around since the 1950s and has never been shown to work, except in people who never actually had cancer
laetrile for cancer – in the late-1970s, kids with treatable forms of cancer had parents who were convinced that this latest fad cure was better. It wasn’t.
naturopathy – although mostly looked at as a holistic alternative to other providers, some of these treatments include vitamin injections, hydrogen peroxide injections, and alternative cancer therapies
shark cartilage – this was the fad cancer cure in the 1990s that was killing kids who’s parents sought alternative cancer treatments. It didn’t work.
What’s the harm with these treatments?
Many, like Acupuncture, Homeopathy, Reiki, and Reflexology, etc., aren’t necessarily dangerous on their own. In fact, most don’t do anything at all, but they can lead people away from real treatments. And that essentially leaves people untreated.
Getting a fake treatment might not be a big deal when it is a condition that can go away on its own, like when Angelina Jolie talked about getting acupuncture when she had Bell’s Palsy, but it often leads to disastrous consequences when a life-threatening condition goes untreated.
Many people who push these alternative “treatments” often also recommend against standard treatments, like vitamin K shots for newborns, RhoGAM shots for their moms, and vaccines.
Those Times Anti-Vaccine Experts Got It Wrong
It shouldn’t be surprising that many of the folks who think that vaccines are dangerous, aren’t necessary, or that they don’t even work also believe in holistic or alternative treatments.
It also shouldn’t be surprising that they are also wrong a lot:
Meryl Dorey – equates vaccination with rape, something many of her followers aren’t even comfortable with
Mark and David Geier – this father and son pair are infamous for pushing a chemical castration treatment (Lupron) for autistic children, a treatment that led to Mark Geier losing his medical license (he’s a geneticist) in several states.
Jay Gordon, MD – once made the comment that “Heaven help us if we have a generation of kids who get a hepatitis B vaccine and a HPV vaccine and they think that now unprotected sex is okay…” Not surprisingly, studies have found that this doesn’t happen. In fact, teen pregnancy rates are at their lowest levels ever.
Suzanne Humphries, MD – a nephrologist who became a homeopath and now pushes anti-vaccine talking points, believes that vaccines don’t work and that polio never really disappeared, and that we don’t “see it anymore” because we changed its name to acute flaccid paralysis.
Neil Z. Miller – a psychologist who has written many anti-vax books, gives lectures at chiropractic associations, and published his daughter’s book, Ambassadors Between Worlds, Intergalactic Gateway to a New Earth, which describes how they are both able to talk to intergalatic beings because she has been doing it for multiple lifetimes. No word yet if folks from the Pleiadians vaccinate their kids…
Tetyana Obukhanych, MD – the Harvard trained immunologist who believes that Immunology has no theoretical or evidence-based explanation for immunity.
Viera Scheibner – the micropaleontologist who thinks that getting a vaccine-preventable disease is good for kids, that vaccines are contaminated with amoebas, and that they cause SIDS and shaken baby syndrome
Stephanie Seneff – the MIT doctor (she has a doctorate in electrical engineering) who thinks that half of kids will have autism in eight years and that glyphosate causes everything from autism to school shootings and terrorist bombings.
Yehuda Shoenfeld, MD – an immunologist who heads the Zabludowicz Center for Autoimmune Diseases and is on the scientific advisory board for the Children’s Medical Safety Research Institute. He is the latest to blame adjuvants for causing disease – his Autoimmune Syndrome Induced by Adjuvants (ASIA), which is often described as being a “basically a made-up syndrome that isn’t generally accepted.”
Sherri Tenpenny, DO – described as an anti-vax “expert” whose advise is “chock full of vaccine pseudoscience.” Once board certified in emergency medicine, Dr. Tenpenny now sells DVDs and supplements on her website, speaks at chiropractic health events, and provides holistic medical care. In a rant about freedom of choice in vaccination, she talks about General Robert E. Lee, Southern war hero and postwar icon of the “Lost Cause of the Confederacy,” the extinction of humanity, and about slavery and eight veils that must be pierced if you want to see what is really going on in the world – that the Illuninati and other secret organizations control us and that they are being controlled by time traveling dragons, lizards, and aliens.
Tim O’Shea, DC – a chiropractor, he speaks at anti-vax conferences and wrote an anti-vaccination book called The Sanctity of Human Blood: Vaccination is not Immunization. Dr. O’Shea does not believe that germs make us sick (germ theory denialism), thinks that vaccines cause peanut allergies, and he sells supplements and seminars.
Kelly Brogan, MD – a holistic psychiatric who recommends that patients wean off their prescribed medications and has talked about HIV denialism.
Erin Elizabeth – is pushing the idea that holistic practitioners are being murdered
Tragically, the pseudo-scientific arguments on many anti-vaccine websites can sometimes be persuasive, especially if you don’t understand that they are mostly the same old arguments that the anti-vaccine movement has been using for over 200 years to scare parents away from vaccinating and protecting their kids.
Sites that are considered anti-vaccine by most people and that push propaganda and myths include:
Age of Autism
Child Health Safety (The facts about vaccine safety your government wont give you)
There is a hierarchy of evidence, from weakest to strongest, that help folks make decisions about science and medicine.
That’s why you can’t just search PubMed, read abstracts, and say that you have done your research. For any study, you have to review and judge the quality of the evidence it provides.
Is it a case report (a glorified anecdote), case series, or animal study (lowest quality evidence) or a systemic review or meta analyses (highest quality evidence)?
What about case control studies, cohort studies, and randomized controlled trials? They lie somewhere in between on the hierarchy of evidence scale.
Anecdotes as Evidence
And where do anecdotes fit in?
“Anecdotes are uncontrolled subjective observations. I have often criticized reliance on anecdotes, which is especially problematic in medicine. The problem with anecdotes is that they are subject to a host of biases, such as confirmation bias. They are easily cherry picked, even unintentionally, and therefore can be used to support just about any position. For every anecdote, there is an equal and opposite anecdote.”
Steven Novella on The Context of Anecdotes and Anomalies
Anecdotes are not scientific evidence.
Unfortunately, some people use anecdotal evidence to make some very serious decisions, including skipping or delaying their children’s vaccines, leaving them unvaccinated and unprotected.
“An anecdote is a story – in the context of medicine it often relates to an individual’s experience with their disease or symptoms and their efforts to treat it. People generally find anecdotes highly compelling, while scientists are deeply suspicious of anecdotes. We are fond of saying that the plural of anecdote is anecdotes, not data.”
Steven Novella on The Role of Anecdotes in Science-Based Medicine
Believing that anecdotes are important scientific evidence is also what often drives some pediatricians to pander to fears that parents may have about vaccines, helping them create non-standard, parent-selected, delayed protection vaccine schedules.
What else do you need to know about anecdotal evidence?
“With little or no evidence-based information to back up claims of vaccine danger, anti-vaccine activists have relied on the power of storytelling to infect an entire generation of parents with fear of and doubt about vaccines. These parent accounts of perceived vaccine injury, coupled with Andrew Wakefield’s fraudulent research study linking the MMR vaccine to autism, created a substantial amount of vaccine hesitancy in new parents, which manifests in both vaccine refusal and the adoption of delayed vaccine schedules.”
Anecdotes, although they are easy to believe, are not scientific evidence, and certainly shouldn’t persuade you that vaccines aren’t safe, that vaccines aren’t necessary, or that vaccines are associated with autism.
“…we have been recently convinced through the promise of technology and corporate prowess that processed food is more reliable, nutritious, and beneficial. We’ve been convinced that Hamburger Helper is better for our families than a homemade Bolognese.”
For some reason, she does not seem to be vegan, as one might expect. Not even vegetarian…
She does seem to believe that people with mental health conditions not should be treated with medication. In fact, she thinks the medications that are routinely used to treat common mental health conditions are behind some of the biggest tragedies happening today.
“The records also listed attention deficit hyperactivity disorder as one of his medical conditions” the proverbial smoking gun of every mass shooting in this country.”
Kelly Brogan on the Sutherland Church Shooting
And she was mentored, not by a psychiatrist, but by a doctor who pushed “a largely dietary treatment for cancer including an individualized organic diet, large amounts of supplements, and pancreatic enzymes,” a regimen that was actually studied and found to be harmful and reduced the quality of life for people with a deadly form of cancer.
Who Is Kelly Brogan?
Kelly Brogan is a holistic psychiatrist.
“Eastern wisdom tells us that when we think we know, we don’t. But when we admit ignorance, we achieve enlightenment. The most profound part of my departure from conventional medicine has been the depths of my surrender to all that we do not, cannot, and must not understand about the body and its experience. Humble awe and wonder are truly the only appropriate states for approaching the complexity of the human condition.”
Do holistic psychiatrists use a different definition for the word humble?
“All suffering is caused by ignorance. People inflict pain on others in the selfish pursuit of their own happiness or satisfaction.”
I’m actually surprised that she hasn’t written about RhoGAM. Maybe she hasn’t gotten around to it. But she doesn’t disappoint. She has shared an article on her Facebook page that claims that the RhoGham shot is just a “Big Pharma Profit Ploy.”
Kelly Brogan is a holistic psychiatrist that wants you to live a medication free life. I guess that could have made her some kind of psychoanalysist, but that doesn’t seem to be the route she went.
“In fact, ‘treatment’ with chemotherapy and radiation not only disrupts a complex process that needs to actually be supported, but also it induces secondary harm, both psychically and physiologically. When we interfere and war with the body, we keep the fight alive – you can’t win the battle against yourself.”
Is she telling people to stop their cancer treatments?
Do folks get a lot of training in treating cancer in their psychiatry residency these days?
And she advises that you “think long and hard about vaccination.”
“As we discover more about the near infinite sophistication of your interconnected bodily systems, and the hyper-individuality of any cause and effect process resulting from a healthcare decision, the one-size-fits-all, indemnified vaccine program may begin to make less and less sense to you. Educate yourself before you make a choice that could change everything for you and your family. Trust your body. Invest in your immunity. And explore a mindset shift that offers you a fear-free way to understand health and wellness.”
Shortly after talking about informed consent, she lets you know how she really feels about vaccines.
“Don’t buy into the lore, don’t make assumptions, and understand that the philosophical underpinnings of the vaccination program are predicated on an antiquated perspective: warring against and attempting to eradicate bad germs. Science has left that childlike notion in the dust, and so should we.”
“One of my favorite medical terms, anosognosia, means lack of awareness of a deficit. I have come to find this useful in description of so many of my colleagues who practice the medicine they were trained to practice without conscious acknowledgement of its gross limitations and even hazards.”
Does anyone else think that it is wildly ironic that anosognosia is Kelly Brogan’s favorite medical term?
Can a psychiatrist be suffering from the Dunning-Kruger effect?
I don’t know, but I’m pretty sure that a holistic psychiatrist can.
A history-making case report? Considering that case reports are the weakest type of scientific evidence, just above YouTube videos and articles on her website, that’s not likely.
Kelly Brogan is a holistic psychiatrist who has flirted with HIV denialism.
You can get educated about vaccines if you are on the fence, but it won’t be from Kelly Brogan, a women’s health holistic psychiatrist.
What to Know About Kelly Brogan
Kelly Brogan is a holistic psychiatrist who seems to charge folks a lot of money in private consultations to help them know that she has faith in their potential to heal themselves naturally – with her help.
The Vaccine-Friendly Plan is the latest book about vaccines that claims to offer a “safe and effective approach to immunity and health.”
What’s the problem with it?
In addition to the fact that there doesn’t seem to be any evidence in the book to support that any of its ideas are indeed safe or effective, the book pushes just about every anti-vaccine talking point out there today.
While Dr. Thomas and Jennifer Margulis talk about providing balanced information, it was right after he stated that “I realized we had poisoned a generation of children with a mercury-derived preservative called thimerosal” and then goes on to talk about how kids are overvaccinated.
So much for balanced information…
But Dr. Thomas isn’t just worried about vaccines. He is also worried about Tylenol, that the chemicals in plastics are endocrine disruptors, GMOs, flame retardants, pesticides, fluoride, artificial sweeteners, chemical dyes, and all of the other toxins that other doctors and the CDC supposedly ignore.
What about the “science” that supports his ideas?
Sure, he is quick to cherry pick studies that support the ideas he likes and label them as “important studies” among “a growing body of evidence,” but if the studies don’t, then they are “a handful of poorly designed, anecdotal studies.”
“Giving a quadruple live-virus vaccine to a toddler is a mistake. When a toddler catches an illness naturally, he does not catch all four at once. I have serious concerns about hitting the immune system of a twelve-month-old baby with four live viruses, even though they are weakened.”
Does Dr. Thomas understand how the immune system works or how many different things our immune systems get “hit” with each and every day? Children are exposed to a lot of live, unweakened viruses and other germs every day and fight them off just fine.
Dr. Thomas also routinely downplays the risks of vaccine-preventable diseases (they all seem to be easily treatable in his world), overstates the risks of vaccines (they all seem to be full of horrible poisons in his world), makes heavy use of anecdotes, repeatedly makes it sound like every other pediatrician is doing something wrong, and again, makes full use of anti-vaccine talking points to scare parents:
You won’t hear this from Dr. Thomas, but unless your ‘kid’ is a baby goat, don’t give him raw goat milk!
So what’s the take home message about Dr. Thomas and his book?
Despite his frequently using the word science in the book, the only “science” in The Vaccine-Friendly Plan is that it is full of pseudoscience.
You can even see this in his ideas about gluten sensitivity. Why do so many of his patients show a sensitivity to gluten? He uses an IgG food sensitivity test that most experts say is basically worthless.
Making a Case for Getting Vaccinated
Perhaps the only good thing about Dr. Paul Thomas’ book is that he makes some very good cases for why you should vaccinate your kids.
His introduction starts off with the story of the death of his three-year-old playmate in Africa. Tragically, the child died of measles.
Like many other pediatricians, he also talks about “the miracle that the Hib vaccine was when it was introduced in 1985.”
“Then in 2012 I had about twenty children in my practice with pertussis: eighteen were school age kids, and two were infants. Interestingly, fifteen of the twenty were fully immunized for pertussis, and the other five were from the small group of families in my practice who refuse all vaccines.”
His story about pertussis in his practice is also very interesting, but not for the reason that Dr. Thomas believes.
Consider that most kids are vaccinated, even in Dr. Thomas’ practice, so the fact that 25% of the kids who got pertussis are unvaccinated means a very high attack rate among unvaccinated children. So even with the problems of waning immunity with the pertussis vaccine, you are still much better off to be vaccinated and protected, even if that protection isn’t perfect.
“Since I opened my practice in 2008, not a single child has received the rotavirus vaccine. I refuse to stock it. Yet only one child in seven years has been hospitalized for severe dehydration. The unvaccinated children in my practice either are not getting rotavirus, or the illness is so mild that it requires no intervention.”
And so much for vaccine choice. How can his patients make a decision to get vaccinated if he doesn’t even have the vaccine?!?
But why don’t they get rotavirus? It is not because the vaccine doesn’t work or isn’t necessary. It is actually called being a free-rider or hiding in the herd.
Like most vaccines, the rotavirus vaccine works and helps create community immunity.
The Most Dangerous Advice in The Vaccine-Friendly Plan
Unfortunately, the dangerous advice in this book extends well beyond repeatedly telling parents to “say no thank you” to multiple vaccines and to delay others.
The advice to “decline vaccines” during pregnancy has to be right up there with the most dangerous advice in his book, but you be the judge…
“It depends on the medication, but the short answer is that it’s best to avoid all over the counter and prescription medications during pregnancy.”
Unbelievably, Dr. Thomas really seems to say that pregnant mothers should try to stop their antidepressants because “women respond differently to pregnancy hormones and some who struggle with mental health issues find the high estrogen and progesterone of pregnancy actually improve mood and mental health.”
“If bilirubin levels remain extremely high – above 20 – for over a week or two, some of the bilirubin can enter the brain, where it can cause permanent brain damage called kernicterus.”
Hopefully most parents are aware that you don’t want to wait “a week or two” to seek treatment if you baby’s jaundice level is above 20.
And hopefully most parents also understand that:
the cutoff for fever in newborns is 100.4°F (38°C) or higher, not 100.6°F (38.4°C)
co-sleeping and letting your baby sleep on your chest are not safe things to do
skipping an evaluation and antibiotics when mom is GBS positive after delivery and she develops a fever (possible chorioamnionitis) is not a safe thing for baby, especially if mom already skipped getting antibiotics during her delivery – it’s called gambling that the baby won’t develop early-onset invasive group B streptococcal disease. Several studies have found very high numbers of newborns in this situation with positive blood cultures, even though they had no symptoms.
if your pediatrician recommends that your child needs treatment for congenital hip dysplasia (which is actually now called Developmental Dysplasia of the Hip), then you should probably listen, instead of hoping it goes away on its own by wearing “your baby on your front or back with his legs splayed.”
you shouldn’t put your baby in direct sunlight without sunscreen for ten to fifteen minutes every day
there is no need to routinely check your baby’s vitamin D level – just give a supplement if you are exclusively breastfeeding
until polio is eradicated, the risk of getting polio is higher than zero and that all of his unvaccinated kids are at risk even if they don’t travel outside the US, like the outbreak among the Amish in 2005
children die from meningococcal disease because it is a severe and terrible disease that progresses very quickly, not because “we pediatricians – so quick to intervene in other, unnecessary ways – fail to listen to a worried mother, dismiss her concerns as “hysterical,” and send a sick child home…” In one study, “Most children had only non-specific symptoms in the first 4-6 h, but were close to death by 24 h.”
preschoolers do not need to routinely take 2,000IU of vitamin D each day – the current recommendation is 600IU if they are not getting enough from the foods they are eating and drinking and 2,000IU only if they have been diagnosed with a vitamin D deficiency
about 4,200 women die of cervical cancer in the United States each year, something Dr. Thomas fails to mention when he says that “some strains of HPV can lead to slow-growing, highly treatable cancers.”
Although the whole book is dangerous, these are serious errors that can harm kids, and it is unbelievable that Penguin Random House would publish this book as a Medical/Parenting book. After all, this is the same company that published NeuroTribes!
“It took me years to realize something I still wish were not true but which you cannot ignore if you want to have a healthy baby in America today: Our government officials and a handful of well-positioned M.D.’s who advise them have ignored some of the most important peer-reviewed studies and most relevant scientific information about immunity and health, both during pregnancy and throughout infancy.”
Dr. Paul Thomas
What else has he got?
the polio vaccines didn’t eliminate polio in the US, instead, it could have been “chlorinating water in public swimming pools” – it wasn’t
rotavirus deaths in the pre-vaccine era are “inaccurate and misleading” because they are estimated numbers “based on a retrospective study that looked at morbidity associated with diarrheal disease between 1968 and 1991,” – except that Dr. Thomas looked at the wrong study. The estimates for hospitalizations and deaths in the pre-vaccine era come from a different study that looked at “Hospitalizations and Deaths from Diarrhea and Rotavirus among Children <5 Years of Age in the United States, 1993–2003.” Oops.
most flu-like illnesses are not really the flu, which “makes it impossible to distinguish influenza infections from other viruses,” unless you go to his office, where he tests kids for the flu – except that using the influenza-like illness (ILI) case definition has a high positive predictive value during flu season and many doctors and hospitals also do rapid flu testing
The Vaccine-Friendly Plan is a dangerous book that not only panders to parent’s fears about vaccines, it goes out of its way to increase those fears by pushing misinformation, telling parents to skip and delay vaccines, and giving other unsafe pediatric and parenting advice.
The only reason to pick it up is because you are looking for some confirmation bias to make you feel better about a decision to not vaccinate your child. If you read it because you were on the fence about vaccines, please consider doing a little more research.
If you are talking about vaccines, this probably isn’t that kind of cherry picking though.
What Is Cherry Picking?
Have you ever been to a cherry orchard?
“Pick only the cherries that are fully red (or whatever color they are supposed to be when ripe!). Part the leaves with your hands to look for hidden cherries ready for harvest.”
Cherry Picking Tips and Facts
Cherries don’t continue to ripen off the tree, so you want to pick them at exactly the right time. Not too early and not too late.
So when you go cherry picking, you are looking for the perfect cherries.
That’s what folks do when they go cherry picking for just the right information to fit their beliefs, but ignore any and all other information that might prove them wrong.
“Whatever one might think about Andrew Wakefield, he was just one man: the MMR autism scare has been driven for a decade now by a media that over-emphasises marginal views, misrepresenting and cherry picking research data to suit its cause. As the Observer scandal makes clear, there is no sign that this will stop.”