The Plural of Anecdote is Not Ambien

Aided and abetted by the press, researchers are relying more on case reports.

By | October 1, 2006

A few years ago Princeton health economist Uwe Reinhardt, Dartmouth geriatrician Joann Lynn, and I filmed a documentary on the effect of advancing biomedical technology on affordable healthcare in the United States. Five hours of interviews were reduced to 10-minute bookends for a set of short, emotional stories that obliterated the complexities of the issues. As Reinhardt quipped in our cab, repeating the oft-cited quote, the plural of anecdote is not data. Except on television.

We've all had this experience: You bring work home and talk about concepts central to biomedical research such as evidence-based medicine, controlled trials, equipoise, peer-review, or impact factor. Friends' eyes roll up into their skulls in boredom. Yet the same topics come up in everyday conversation all the time, just framed in a different way: "I know a person who lost his house to the cost of drugs," and "you know a guy who is alive because of Lipitor." Heated arguments ensue about real problems in science, but driven by someone's single story.

Stories are not the enemy of good science and evidence-based medicine. Physicians make crucial but subtle changes in their practices based on individual experiences. Scientists all use intuition and inductive reasoning in the nascent period of an investigation. But anecdotes cannot substitute for either ethnography or controlled study. When Terri Schiavo became the world's test case for diagnosing persistent vegetative state (PVS), the emotional intonations about Ms. Schiavo waking up began to sound like Intelligent Design.

A paradox of biomedical research is that huge controlled trials, meta-analyses, and reviews of the literature are ubiquitous, but the number of "case reports" - and journals comprised entirely of incidental "findings" - is growing.

The media has no idea how to deal with case reports. The worst example of this in recent times was a case study of Zolpidem, the nonbenzodiazepine-branded 'Ambien' and approved by the US Food & Drug Administration for the treatment of insomnia. Physicians Ralf Clauss and Wally Nel have published, a few cases at a time, their very different use of the medication. The Guardian carried a breathless report of Clauss et al's August report in the journal NeuroRehabilitation of three cases involving patients who have been in PVS, they report, for more than three years. Claus and Nel grabbed the front page with the Guardian's report that they used Ambien to wake up these patients.

The 'investigators' had administered Zolpidem for between three and six years and saw each of the three 'treated' patients wake up each day as a result of the medication; one even "caught a baseball." When the medication wore off, the patients dropped back into PVS each evening.

Stunning science? It seemed so, too, back in 2000, when Claus and a different set of South African colleagues published in the South African Medical Journal on a single case with essentially the same outcome. In 2001 they made the same claim in a letter to the same journal. At no point did the investigators conduct an actual study of the phenomenon, with an IRB-approved research protocol or informed consent. Again and again they "wrote up" their "cases," describing their work as innovative medical management rather than research. Journal editors, asleep at the switch, have been derelict in publishing bad research disguised as cases - in this instance a case with the impact of finding a life-extending potion or the presence of extraterrestrial life.

Investigators who jumpstart their programs with case reports are often in search of research support, as was Claus - who as a result of the case report is now funded. In this respect they, and the journals who publish nothing but these case reports, are like the television producers I worked with: They aim at using the power of stories to make the claims and reap the rewards that come from research, but without doing the research. The victims are journalists, readers, and in the Claus case the patients and their families, who are exposed to uncontrolled experiments framed as good medicine. It is time for editors, journalists, the FDA, and the US Health & Human Services Office for Human Research Protections (OHRP) to clamp down on the "case study."

Glenn McGee is the director of the Alden March Bioethics Institute at Albany Medical College, where he holds the John A. Balint Endowed Chair in Medical Ethics. gmcgee@the-scientist.com

Comments

Avatar of: Dr Andrew Sutton

Dr Andrew Sutton

Posts: 1

October 27, 2006

Dear colleagues,\nThe editorial of October the 5th was worse than a saddening blast of blanket bureaucratic thinking, it was a travesty of the real history of the discovery of Ambien's effect. Clauss and Nel had made a most extraordinary discovery that could transform the lives and care of many severely injured patients so it was their clear ethical duty to report it. \nSo ignorant were the editorial comments that one is bound to wonder if the writer had actually read their reports and the several others that exist in the literature from quite separate sources. For example the statement "At no point did the investigators conduct an actual study of the phenomenon" is obviously untrue as they reported the astounding fact that they had seen in SPECT scans that areas of the brain started to function while zolpidem was present. This unexpected effect of a sedative could hardly have been more conclusive evidence in an N of 1, so it was an absolute disgrace that the editorial denied its existence. Clauss has even continued to study the phenomenon in monkeys had the author bothered to check the literature for even a few minutes..\n\nThe editorial's writer should also have realised that Clauss and Nel had no funds to conduct a clinical trial so the comment on having no referral to an ethics committee is again just ignorance. They were using zolpidem off-label, an ethically sound procedure used by the countless thousands in the USA every day in the best interests of patients, particularly in chemotherapy and paediatrics. Where they are available SPECT scans are commmonly done for brain injured patients to locate lesions, so its use in these patients is well within the clinical remit of the investigators. Ambien is a known safe medication and was given to the first patient as a sedative not an experimental new compound.\n\nHence it is rich indeed that this article is really accusing Clauss and Nel of a lack of objectivity, whilst actually committing that particular scientific sin itself, even in condemnatory language. Such thoughtless condemnation is wounding to honest people and may well frighten others from reporting findings that could be helpful to research. \n\nIt is not as though anecdotal findings have a totally bad history. Jenner reported vaccination in one milkmaid and Fleming did not need a clinical trial to detect the effect of penicillin. \n\nI write this because I think it a serious issue when a director of an ethics faculty no less, writes such a thoughtless piece. His high status brings the responsibility to be more balanced and objective, considering that he is regarded an opinion leader by his peers.\n\nLastly let me declare a financial interest and my own position. My comments are intended to be objective, and I hope others will think so, but as I am now working for and have shares in the company that is trying to investigate this discovery through all the proper ethical channels I feel I should explain my own position. \n\nThe discovery has produced some profound improvements for a few patients but I agree with the writer's underlying premise that we must not expect it to be a panacaea so I whole-heartedly agree that sound and ethical trials must be done. Dr Nel has found that many patients are too sedated for a fully active life so we are trying to develop and assess properly some less sedative formulations. We are doing this under patents in the normal fashion of the Pharmaceutical industry since otherwise no company would have the incentive. I agree the existence of financial incentives endangers objectivity but in this case such was the benefit that I saw for my own eyes in Nel's patients that sorting out the value of this discovery to patients and making bit available to them and their carers, is far more important than any money could be. \n\n\nAndrew Sutton MBBS, MD(London), FFA\n\n
Avatar of: Chris Idzikowski

Chris Idzikowski

Posts: 1

November 1, 2006

This has to be the silliest article I've read for a long time. If media have a problem handling case studies then we need to educate the media? To imply that case studies are conducted to raise research money is a remarkable slur. Case studies are often written up when a therapeutic intervention has a surprising result and this leads to hypothesis-driven, controlled research.\nThe author also seems to have an idealistic view that neurological/psychiatric interventions can be invented de novo and can therefore be subjected to experimental methods immediately; unfortunately the reverse is the case, most are discovered accidentally (case studies?).\nJust stumbled on this article but sufficiently exasperated to leave a comment.

Popular Now

  1. Major German Universities Cancel Elsevier Contracts
  2. Grass Routes
    Features Grass Routes

    Researchers are discovering a suite of new locations and functions of endocannabinoid receptors that play roles in sickness and in health.

  3. Studies Retracted After UCLA Investigation
  4. Trump Nominates Sam Clovis to Lead USDA Research
AAAS