The Plural of Anecdote is Not Ambien

A few years ago Princeton health economist Uwe Reinhardt, Dartmouth geriatrician Joann Lynn, and I filmed a documentary on the effect of advancing biomedical technology on affordable healthcare in the United States. Five hours of interviews were reduced to 10-minute bookends for a set of short, emotional stories that obliterated the complexities of the issues. As Reinhardt quipped in our cab, repeating the oft-cited quote, the plural of anecdote is not data. Except on television. We've all had this experience: You bring work home and talk about concepts central to biomedical research such as evidence-based medicine, controlled trials, equipoise, peer-review, or impact factor. Friends' eyes roll up into their skulls in boredom. Yet the same topics come up in everyday conversation all the time, just framed in a different way: "I know a person who lost his house to the cost of drugs," and "you know a guy who is alive because of Lipitor." Heated arguments ensue about real problems in science, but driven by someone's single story. Stories are not the enemy of good science and evidence-based medicine. Physicians make crucial but subtle changes in their practices based on individual experiences. Scientists all use intuition and inductive reasoning in the nascent period of an investigation. But anecdotes cannot substitute for either ethnography or controlled study. When Terri Schiavo became the world's test case for diagnosing persistent vegetative state (PVS), the emotional intonations about Ms. Schiavo waking up began to sound like Intelligent Design. A paradox of biomedical research is that huge controlled trials, meta-analyses, and reviews of the literature are ubiquitous, but the number of "case reports" - and journals comprised entirely of incidental "findings" - is growing. The media has no idea how to deal with case reports. The worst example of this in recent times was a case study of Zolpidem, the nonbenzodiazepine-branded 'Ambien' and approved by the US Food & Drug Administration for the treatment of insomnia. Physicians Ralf Clauss and Wally Nel have published, a few cases at a time, their very different use of the medication. The Guardian carried a breathless report of Clauss et al's August report in the journal NeuroRehabilitation of three cases involving patients who have been in PVS, they report, for more than three years. Claus and Nel grabbed the front page with the Guardian's report that they used Ambien to wake up these patients. The 'investigators' had administered Zolpidem for between three and six years and saw each of the three 'treated' patients wake up each day as a result of the medication; one even "caught a baseball." When the medication wore off, the patients dropped back into PVS each evening. Stunning science? It seemed so, too, back in 2000, when Claus and a different set of South African colleagues published in the South African Medical Journal on a single case with essentially the same outcome. In 2001 they made the same claim in a letter to the same journal. At no point did the investigators conduct an actual study of the phenomenon, with an IRB-approved research protocol or informed consent. Again and again they "wrote up" their "cases," describing their work as innovative medical management rather than research. Journal editors, asleep at the switch, have been derelict in publishing bad research disguised as cases - in this instance a case with the impact of finding a life-extending potion or the presence of extraterrestrial life. Investigators who jumpstart their programs with case reports are often in search of research support, as was Claus - who as a result of the case report is now funded. In this respect they, and the journals who publish nothing but these case reports, are like the television producers I worked with: They aim at using the power of stories to make the claims and reap the rewards that come from research, but without doing the research. The victims are journalists, readers, and in the Claus case the patients and their families, who are exposed to uncontrolled experiments framed as good medicine. It is time for editors, journalists, the FDA, and the US Health & Human Services Office for Human Research Protections (OHRP) to clamp down on the "case study." Glenn McGee is the director of the Alden March Bioethics Institute at Albany Medical College, where he holds the John A. Balint Endowed Chair in Medical Ethics. gmcgee@the-scientist.com

The Plural of Anecdote is Not Ambien

Aided and abetted by the press, researchers are relying more on case reports.

Interested in reading more?

Become a Member of