Evidence

Letter to the Editor 

To the editor: Thank you for the opportunity to express my humble disdain for the pre-eminent role of evidence in your Journal. I have practiced medicine successfully for 50 years without a shred of it, never having to endure a single lawsuit, dissatisfied patient, nor any of those nasty entangling alliances with third party payers or managed care companies. My office is always full and I consistently rank among the “best doctors” in my region.

Do we really need randomized, controlled trials to illuminate the obvious worthlessness of most remedies, or the occasional value of others? Something either works, or it does not, and common sense plus keen observation make the distinction obvious (at least to me). Statistics and evidence cloud the rational mind with endless jargon and perennial uncertainty. Moreover, as Voltaire1 notes, “The art of medicine consists in amusing the patient while nature cures the disease.” I therefore suggest an “amusing the patient” section for your Journal, and would be honored to serve as the first associate editor (having much experience as an entertainer).

My sentiments may astonish your erudite readers, but rest assured, many colleagues are like-minded. Although your editorship is new it will likely be brief, as I confidently predict the demise of your Journal – and many others – as the time-tested values of experience, common sense, and good intentions return to the forefront of health policy. Perhaps you can postpone this demise by publishing more case reports and case series instead of increasingly obtuse original research. Ditto for a return to those quaint narrative reviews by self-anointed experts, free of the pesky statistics in systematic reviews and meta-analyses. My colleagues and I stand ready to assist with endless anecdotes, should you desire, and may even be able to offer part-time employment should your editorship be eliminated as predicted.

Your concerned colleague,

Anthony Anecdote, General Otolaryngologist

Somewhere in the USA (location purposefully withheld)

Editor’s Response 

The mission of our Journal is “to publish contemporary, ethical, clinically relevant information in otolaryngology head and neck surgery that can be used by otolaryngologists, scientists, and related specialists to improve patient care and public health.” The most important information we publish is original research, which supplies the primary ingredient for EBM. Clinician experience and patient preference are also important, but must be balanced by the quality and strength of existing evidence (or lack thereof).2

The evolution of EBM spans more than 2,000 years. Hippocrates made observation and experience cornerstones of clinical practice, but Galen perfected eminence-based medicine with over 22 volumes of medical text (2.5 million words) that spanned the knowledge abyss of the Middle Ages. Keen observation resurfaced as Vesalius challenged Galenic dogma, and the power of experiment became apparent when Harvey proved the circulation of the blood. Clinical research blossomed rapidly, moving in less than 200 years from Louis’ numerical method to Osler’s critical thinking and clinical trials based on randomization concepts from Fisher and Hill. A critical mass of randomized trials mandated techniques for systematic review and guideline development, culminating in the current EBM paradigm.

Why not return to the halcyon days of observation and experience as foundations for clinical practice? Because a myopic focus on outcomes precludes distinguishing real treatment effects from natural history, spontaneous resolution, and the placebo response (caring effects).3 Research and critical thinking are antidotes to the delusory certainty that stems from equating good results to effective treatment. Moreover, every intervention – including observation – has potential for harm, which can only be justified if true benefits are known. Improvement after therapy differs from improvement because of therapy, and distinguishing the two is essential to establishing an evidence base for meaningful health policy.

Well-crafted evidence transforms observations into generalizations.4 This process, called inference, is fundamental to scientific progress. I am delighted to learn of Dr. Anecdote’s proclivity for patient satisfaction, but will simple reporting of his experience allow others to do the same? The answer is a resounding “no” unless precautions are taken to ensure validity. Internal validity defines credibility of findings for the study sample, and results from proper study design, unbiased measurements, and sound statistical analysis. For inference to occur, however, results must apply beyond the study sample to a broader target population. External validity, also called generalizability, considers the sampling scheme, subject selection criteria, and descriptive characteristics of the resulting sample.

To maximize the validity of evidence we publish, reviewers of original research must answer five questions (no, maybe, or yes) before their comments can be submitted:

Although not a guarantee of quality, the above list serves as a constant reminder of the factors associated with evidence likely to enhance inference and thereby promote EBM. If the answer to any question is not “yes,” the study will be returned to the authors for revision or rejected if one or more fatal flaws are present. Reviewers are also asked to consider and comment on individual sections of the manuscript.5

Our goal is not simply to publish high levels of evidence, but rather the best and most appropriate level given what has already been published on a topic.6 A case series describing a rare disorder or novel technique improves our knowledge beyond case reports, but is of less value when controlled or comparative studies already exist. Similarly, when an intervention has already been assessed by one or more randomized trials, publishing observational or uncontrolled research is of marginal benefit. We recognize, however, that randomized trials may be unethical, impractical, or inappropriate for some topics,7 and that bench research (often the lowest level of evidence for clinical decisions) is an essential component of EBM.

Although most studies in otolaryngology journals describe therapy, we are keenly interested in articles about harm, prevention, etiology, symptom prevalence, differential diagnosis, prognosis, and diagnostic tests. All of these study types can benefit from an improved level of evidence.8 Lastly, we relish properly performed systematic reviews that synthesize evidence with minimal bias,9 but will often ask that old-fashioned narrative reviews be condensed and rewritten as commentaries.

I thank Dr. Anecdote for sharing his concerns, and sincerely hope that my remarks above have clarified the role of evidence in the Journal. I envision a bright future and invite all readers to participate by submitting manuscripts and sharing wisdom with letters and commentaries. Although I agree with Dr. Anecdote that Voltaire’s comment is insightful, I do not anticipate a new section on “amusing patients” to be a part of the Journal in the foreseeable future.