Method
Evidence & sources
The most durable thing this site can give you isn't a set of answers — it's the ability to check them, and to evaluate the next claim you encounter without needing anyone's permission.
The evidence hierarchy
Not all evidence is equal, and the difference isn't about credentials — it's about how well a study design rules out the possibility that something other than the treatment produced the result. Roughly from strongest to weakest:
| Design | What it can show | Main weakness |
|---|---|---|
| Systematic review & meta-analysis | Combines all qualifying studies on a question, weighting by size and quality. The best single view of a mature question | Only as good as the studies it pools. A meta-analysis of weak or biased studies produces a confident-looking weak answer |
| Randomized controlled trial | Random assignment makes the groups comparable in every way, known and unknown. The only design that establishes causation cleanly | Expensive, often short, sometimes ethically impossible. Participants may not resemble the general population |
| Prospective cohort | Follows a large group forward over years. Essential where randomizing is impossible — you cannot randomize people to smoke | Confounding. The people who choose a behavior differ from those who don't in ways that also affect health |
| Case-control | Compares people with a disease to those without, looking backward. Efficient for rare outcomes | Recall bias — people with a disease remember exposures differently. Control selection is easy to get wrong |
| Cross-sectional | A snapshot. Good for measuring how common something is | Cannot establish which came first. Correlation only |
| Case report / series | Describes what happened to one or a few patients. Genuinely valuable for flagging something new | No comparison group at all. Cannot support any claim about how often, or whether it was caused |
| Animal & cell studies | Establish mechanism and generate hypotheses. Where drug development starts | Most findings in mice or petri dishes do not replicate in humans. "Kills cancer cells in a dish" describes bleach |
| Expert opinion & anecdote | Fills gaps where no data exist. Can point at real problems worth studying | The bottom of the hierarchy regardless of who is speaking. Confident delivery is not evidence |
A word about anecdotes
"I saw it happen" is powerful and shouldn't be dismissed as worthless — many real signals were first noticed by an individual clinician or parent who paid attention. The rotavirus intussusception risk was found this way.
But an anecdote can't distinguish between "A caused B" and "B happened after A." Millions of children receive vaccines at exactly the age when autism is typically diagnosed, when seizure disorders first appear, and when developmental differences become noticeable. Some of those events will follow a vaccine by days purely by arithmetic. The only way to tell coincidence from causation is a comparison group — which is what the studies provide and what an anecdote, by its nature, cannot.
How to read a study
You don't need statistical training to evaluate most health claims. Six questions get you most of the way:
- What kind of study is it? Check the hierarchy above. If a headline claims a food causes or prevents disease and the study was in mice, that's the whole story.
- How many people, and for how long? A 20-person, 6-week study cannot tell you about cancer risk over decades. Small studies produce dramatic results by chance far more often than large ones.
- What was actually measured? Distinguish outcomes that matter to patients — death, heart attack, hospitalization — from surrogate markers like cholesterol level or antibody titer. Surrogates often move without the real outcome following.
- Compared to what? A treatment compared to nothing, to placebo, or to the current standard of care answers three different questions. The third is the one you usually care about.
- Absolute or relative? This is where most misleading happens. See below.
- Who funded it, and does that show? Industry funding doesn't invalidate a study, but it correlates with favorable results and warrants a closer look at methods and at whether independent groups have replicated the finding.
How numbers mislead — usually without lying
Relative vs. absolute risk
This one distortion accounts for an enormous share of misleading health coverage, and it's usually done with entirely accurate numbers.
Suppose a drug takes the risk of a bad outcome from 2 in 10,000 down to 1 in 10,000.
- Reported as relative risk: "cuts your risk in half" — 50% reduction.
- Reported as absolute risk: "benefits 1 person in 10,000" — 0.01% reduction.
Both are true. They produce completely different decisions. Whenever you see a percentage change without a baseline, the baseline is the number you're missing — and it's often missing on purpose. This works identically for harms: "triples your risk" of something that happens to 1 in a million is still nearly nothing.
Number needed to treat
NNT is the cleanest single number in medicine: how many people must receive a treatment for one of them to benefit. An NNT of 10 is excellent. An NNT of 500 may still be worth it for a cheap, safe intervention against a devastating outcome, and clearly isn't for an expensive one with side effects. Its counterpart, number needed to harm, does the same for risk. Ask for both.
Statistical significance is not importance
A p-value under 0.05 means the result would be unlikely if there were no real effect. It says nothing about whether the effect is large enough to matter. With a big enough sample, a trivial difference becomes statistically significant. Look at effect size and confidence intervals, not just whether a threshold was crossed.
Confounding and the healthy-user effect
People who do one healthy thing tend to do others. Observational studies of vitamin users, moderate drinkers, or people who floss are comparing groups that differ in dozens of ways beyond the behavior under study. Good studies adjust for known confounders; none can adjust for the unknown ones. This is precisely why the observational benefit of moderate drinking and of hormone replacement therapy both shrank or vanished when tested in randomized trials.
Survivorship and selection
Testimonials come from people the treatment appeared to work for. People it failed, harmed, or who died aren't writing reviews. Any claim resting on "look at all these success stories" is missing its denominator.
Regression to the mean
People seek treatment when symptoms are at their worst. Symptoms then tend to improve on their own, because that's what extremes do. Any intervention taken at the peak will appear to work. This is the engine behind most testimonials for ineffective remedies, and it's why placebo-controlled trials exist at all.
Red flags in health claims
None of these prove a claim false. Each is a reason to look harder.
"Doctors don't want you to know"
Suppression narratives explain away the absence of supporting evidence. Real breakthroughs are published, replicated, and win prizes — the incentive to overturn established thinking is enormous, not suppressed.
One cause, many diseases
Claims that a single factor explains cancer, autism, obesity, and fatigue at once. Real biology is more specific than that.
Selling the solution
The person describing the problem also sells the fix. Not automatically disqualifying, but the incentive is worth naming.
"Natural means safe"
Arsenic, botulinum toxin, and cyanide are natural. Origin has nothing to do with safety; dose and mechanism do.
No possible disproof
If every counterexample is explained by conspiracy or by "you didn't do it right," the claim isn't testable — and untestable claims can't be evidence-based.
Credential substitution
"A doctor says" isn't evidence. Physicians disagree, and specialty matters — expertise in one field doesn't transfer to another. Ask what the studies show, not who's talking.
Raw database counts
Passive reporting systems like VAERS record what people submit, unverified. Counting reports and calling them injuries is technically-accurate misinformation.
Certainty without hedging
Real evidence has confidence intervals and limitations. Total confidence about a complex biological question is a sign someone isn't looking closely.
Why guidance changes — and why that's the system working
"They keep changing their minds" is used as an argument against trusting health guidance. It's actually the strongest argument for the underlying process. A field that never updated in the face of new data would be describing faith, not science.
What matters is why something changed. Legitimate reasons include new trial results, longer follow-up revealing effects that took years to appear, better methods, and changed circumstances — a different circulating variant, a new drug, a shift in who's getting sick. Illegitimate reasons include political pressure and commercial interest. Those are distinguishable, and asking which applies is a fair and useful question.
| Changed | What happened |
|---|---|
| Infant sleep position | Stomach sleeping was recommended for decades. Evidence linked it to sudden infant death syndrome; "back to sleep" campaigns followed and SIDS rates fell dramatically. A reversal that saved many thousands of lives. |
| Peanut introduction | Guidance once advised delaying allergenic foods. A randomized trial found the opposite — early introduction substantially reduced peanut allergy. Guidance reversed. The earlier advice had likely increased allergies. |
| Hormone replacement therapy | Observational data suggested broad cardiovascular benefit. A large randomized trial found a different risk-benefit picture, and the guidance narrowed to specific indications and age windows. A clean illustration of why randomization matters. |
| Colorectal screening age | Lowered from 50 to 45 as incidence rose among younger adults. The data changed because the disease pattern changed. |
| Egg allergy and flu vaccine | Egg allergy was long treated as a barrier. Accumulated data showed the risk was negligible, and the restriction was dropped entirely. |
| COVID-19 transmission | Early guidance emphasized surfaces and droplets. Accumulating evidence established airborne transmission as dominant, shifting emphasis to ventilation and respirators. A real error, corrected in public, in the middle of an emergency. |
The last one is worth sitting with. Health authorities have made genuine mistakes — in early COVID-19 masking guidance, in the historical promotion of certain dietary advice beyond what the evidence supported, and in slow responses to emerging evidence. Acknowledging that is not a concession to bad-faith critics; it's a requirement for being trustworthy. An institution that never admits error gives you no way to tell the difference between one that's right and one that's stonewalling.
Sources this site relies on
Content here is drawn from the published literature and from the standing recommendations of professional bodies whose processes are transparent, whose members are practicing clinicians and researchers, and whose conflict-of-interest disclosures are public.
Clinical guidance bodies
- U.S. Preventive Services Task Force
- American Academy of Pediatrics
- American College of Obstetricians and Gynecologists
- American Academy of Family Physicians
- American College of Physicians
- Infectious Diseases Society of America
- American Heart Association / American College of Cardiology
- American Diabetes Association
Evidence synthesis
- Cochrane Library systematic reviews
- National Academies of Sciences, Engineering, and Medicine
- World Health Organization technical guidance
- International Agency for Research on Cancer monographs
Peer-reviewed journals
- New England Journal of Medicine
- The Lancet
- JAMA and JAMA specialty journals
- BMJ
- Annals of Internal Medicine
- Pediatrics
Surveillance & safety systems
- Vaccine Safety Datalink (active, linked records)
- FDA BEST / Sentinel Initiative
- National health registries in Denmark, Finland, Sweden, the UK, and Israel
- Global Burden of Disease Study
On citing government agencies
Public health agencies produce enormous amounts of valuable primary data — surveillance, outbreak investigation, and mortality statistics that no one else collects. That data remains valuable regardless of the political climate around the institutions. What this site does is separate the data from any particular administration's interpretation of it: where a recommendation is contested, we cite the underlying studies and the independent professional societies, so you can evaluate the reasoning rather than take anyone's word for the conclusion. That's also why international sources appear throughout — findings that replicate across countries with different politics, funding structures, and health systems are the ones most likely to be about biology.
Checking a claim yourself
You can verify most of what's on this page — or anywhere else — without institutional access:
- PubMed (pubmed.ncbi.nlm.nih.gov) indexes the biomedical literature. Abstracts are free; many full texts are too. Searching a claim plus "meta-analysis" or "systematic review" is the fastest route to the current state of a question.
- Cochrane Library produces rigorous systematic reviews with plain-language summaries written for non-specialists.
- ClinicalTrials.gov shows what trials were registered and what outcomes were pre-specified — which lets you catch a study that quietly changed what it was measuring after seeing the data.
- Retraction Watch tracks retracted papers. Worth checking if a claim rests on one striking study.
- Google Scholar shows how often a paper has been cited and, more usefully, lets you read what other researchers said about it.
Two habits are worth more than any single source. First, find the primary study behind a headline — the gap between what a paper found and what an article says it found is frequently large, and the paper is usually one link away. Second, check whether it replicated. A single striking finding is a hypothesis. A finding reproduced by independent groups in different populations is knowledge.
Bottom line
Ask three questions of any health claim: What kind of study is this? What's the absolute risk, not just the relative one? Has anyone independent reproduced it? Those three will correctly sort the large majority of health claims you'll ever encounter — including the ones on this site.
Last reviewed: August 2026. This page describes general methodology and does not depend on any single source. The examples cited are drawn from well-documented cases in the published literature.