Polite Dissent

I wrote this post a little over five years ago, but I think it is just as important now as it was then. It gives non-physicians a few quick guidelines on how to tell legitimate studies from nonsense studies when they are reported in the press. I originally wrote in about newspapers, but it replies equally well to television news programs and internet news sites.

Almost every day it seems that a new groundbreaking medical reports is mentioned in a front page newspaper article or on the cover of a weekly magazine. The claims are bold: eating red meat leads to colon cancer, drinking soda leads to diabetes, green tea extract cures strokes and so on. But are any of these claims legitimate?

Medicine can be a confusing field, and statistics even more so. T-scores? Z-scores? Power? P value? How is a non-physician (or non-statistician) supposed to find out which reports are reasonable and which are unfounded?

It’s not that difficult if you remember to be skeptical and follow these two simple rules.

Skepticism is Your Friend
Approach all medical articles with a great deal of skepticism. These articles and reports are trying to convince you to do something different, such as eat less of this or that or take this vitamin or medicine. Don’t just take their word at it. Make them prove it to you.

Rule #1
Where was the study published? To be believable, it should have been published in a well-known, well-respected medical or scientific journal*. These journals include the Journal of the American Medical Association (JAMA), the New England Journal of Medicine and Lancet.

Be very wary if the report is from a presentation at a conference and has not been published. Published articles are closely examined and reviewed by experts. The same doesn’t necessarily hold true for a presentation.

Don’t trust a news release or report put out that is not published in a legitimate journal, or at least presented at a legitimate conference. Most suspicious advice, sloppy science and bad medicine comes from these “reports” (and the most eye catching headlines too).

(I would also add that readers should beware medicine by press report. A legitimate finding should be available in a journal that is currently available or one that will be shortly available. If no publication or pending publication is mentioned, then it is likely medicine by PR and less likely to be legitimate)

Rule #2
Look at the number of participants in the study. If it is for a well-known condition (such as heart disease, stroke or cancer) or addresses a common situation (diet, exercise) then there should be several hundred, if not thousands, of participants. A study that addresses a common condition or makes sweeping statements yet only has a hundred — or fewer — participants should be viewed extremely skeptically.

Following these two rules will allow you to efficiently separate the wheat from the chaff and discover which newspaper medical reports you really need to pay attention to, and which can be dropped at the bottom of the birdcage.

---

* That is not to say that smaller medical journals don’t produce quality groundbreaking articles; they do, but it is rare. Big name journals also publish poor papers from time to time. Still, if it is published in a journal even a non-physician has heard of, then it’s more likely to be believable and legitimate.