What is Evidence Based Medicine?

With the growth of the Internet, we are flooded with information about attention deficit hyperactivity disorder from many sources, most of which aim to provide useful and compelling “facts” about the disorder. But, for the cautious reader, separating fact from opinion can be difficult when writers have not spelled out how they have come to decide that the information they present is factual.

My blogs several guidelines to reassure readers that the information they read about ADHD is up-to-date and dependable. They are as follows:
Nearly all the information presented is based on peer-reviewed publications in the scientific literature about ADHD. “Peer-reviewed” means that other scientists read the article and made suggestions for changes and approved that it was of sufficient quality for publication. I say “nearly all” because in some cases I’ve used books or other information published by colleagues who have a reputation for high quality science.

When expressing certainty about putative facts, I am guided by the principles of evidenced based medicine, which recognizes that the degree to which we can be certain about the truth of scientific statements depends on several features of the scientific papers used to justify the statements such as the number of studies available and the quality of the individual studies. For example, compare these two types of studies. One study gives drug X to 10 ADHD patients and reports that 7 improved. Another gave drug Y to 100 patients and a placebo to 100 other patients and used statistics to show that the rate of improvement was significantly greater in the drug treated group. The second study is much better and much larger, so we should be more confident in its conclusions. The rules of evidence are fairly complex and can be viewed at the Oxford Center for Evidenced Based Medicine (OCEBM; http://www.cebm.net/).

The evidenced-based approach incorporates two types of information: a) the quality of the evidence and b) the magnitude of the treatment effect. The OCEBM levels of evidence quality are defined as follows (higher numbers are better:

1. Mechanism based reasoning.   For example, some data suggest that oxidative stress leads to ADHD and we know that omega-3 fatty acids reduce oxidative stress.  So there is a reasonable mechanism whereby omega-3 therapy might help ADHD people.
2. Studies of one or a few people without a control group or studies that compare treated patients to those that were not treated in the past.
3. Non-randomized, controlled studies.    In these studies the treatment group is compared to a group that receives a placebo treatment, which is a fake treatment not expected to work.   Non-randomized means that the comparison might be confounded by having placed different types of patients in the treatment and control groups.
4. Single randomized trial.   This type of study is not confounded.
5. Systematic review and meta-analysis of randomized trials.  This means that many randomized trials have been completed and someone has combined them to reach a more accurate conclusion.

It is possible to have high quality evidence proving that a treatment ‘works’ but the treatment might not work very well.  So it is important to consider the magnitude of the treatment effect, also called the “effect size” by statisticians.  For ADHD, it is easiest to think about ranking treatments on a ten point scale.   The stimulant medications have a quality rating of 5 and also have the strongest magnitude of effect, about 9 or 10.  Omega-3 fatty acid supplementation ‘works’ with a quality rating of 5, but the score for magnitude of effect is only 2 so it doesn’t work very well.  We have to take into account patient or parent preferences, comorbid conditions, prior response to treatment and other issues when choosing a treatment for a specific patient, but we can only use an evidenced-based approach when deciding which treatments are well supported as helpful for a disorder.