When the results of the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) were published in 2005, researchers presented clinicians with a surprise: Doctors had thought that second-generation, or atypical, antipsychotic medications were more effective, and produced fewer extrapyramidal side effects, such as tremors, rigidity, restlessness, and tardive dyskinesia, than their predecessors. However, the National Institute of Mental Health study found no significant difference between the generations in terms of efficacy, and only small differences in terms of side-effect profiles (NEJM, 353:1209-23, 2005).

For example, patients discontinued olanzapine, an atypical antipsychotic, and perphenazine, a typical antipsychotic, at the same rate due to intolerable side effects. For olanzapine, the reason was weight gain or metabolic effects, whereas for perphenazine it was extrapyramidal effects. "CATIE provided rigorous data that pulled the curtain back - the-emperor's-new-clothes effect," says the study's principle investigator, Jeffrey Lieberman. "The superior effectiveness of the new drugs was not what had been believed. That wasn't necessarily a pleasant, popular, or welcome development."

The reaction to CATIE included several papers that questioned the study's methods. One controversial aspect was CATIE's use of all-cause discontinuation (ACD) as the primary outcome measure. "CATIE used a new method that had not been used widely in other psychiatric clinical trials: time to discontinuation," says Peter Weiden, a professor of psychiatry at the University of Illinois, Chicago, who was also a CATIE investigator. "The problem with transposing that into psychiatry is that medication change is not always a bad outcome. It can be a gray area."

In a recent paper (J Clin Psych, 68 Suppl:12-9, 2007), Weiden noted that researchers counted a patient's discontinuation of a drug only if a doctor confirmed the cause. So, for example, if a patient stopped using olanzapine due to side effects, but a doctor disagreed with the patient's opinion, then that particular discontinuation would not be tallied. This effectively lowered a drug's discontinuation rate, since that patient was still included in the total number of volunteers in the trial. When Weiden reanalyzed the data, correcting for the discontinuation bias, he saw a significantly greater rate of discontinuation for the first-generation drug (perphenazine) over the second-generation ones (olanzapine, quetiapine, risperidone and ziprasidone).

Still, Weiden emphasizes that CATIE was a landmark study, and that ambiguity will always occur with a large change in methodology. "It's not surprising," he says. "I was surprised how it was interpreted one way without mention of this alternate interpretation."

The controversy over methodology may have contributed to the fact that CATIE does not seem to have changed how doctors treat schizophrenia, which includes $8.5 billion spent every year on atypical antipsychotics. The study "hasn't necessarily impacted practice patterns nearly as much as the debate," says Lieberman, who is also chair of Columbia University's psychiatry department.

Lieberman emphasizes that the atypicals are not inferior to their predecessors - simply that they are not better, or only incrementally better. Yet the atypicals have existed for nearly 20 years, starting with clozapine in 1989, and they have been established as a significant advance over the typicals. "Once it's a belief that's strongly and long held, it doesn't change overnight," Lieberman says. "The truth doesn't necessarily change people's thinking."