Advertisement

Opinion: Misleading Drug Trials

Amgen’s incomplete report on an early major trial of epoetin misled the medical community about the anemia drug’s risks and benefits—and helped make Amgen rich.

By | May 14, 2012

image: Opinion: Misleading Drug Trials stock.xchng, gerard79

STOCK.XCHNG, GERARD79

The Normal Hematocrit Trial, conducted in the mid-1990s, was the largest study ever to compare the use of epoetin, a drug that stimulates blood production, to treat dialysis patients, who suffer from anemia, or a below-normal red blood cell count, called hematocrit. In healthy individuals, 39 to 45 percent of their blood is comprised of red blood cells. Severe anemia, a hematocrit below 25 percent, can stress heart function, cause marked fatigue, and require blood transfusions.

Most dialysis patients need some epoetin or a similar drug to avoid severe anemia, and the higher the desired hematocrit, the higher the dose required. The study aimed to compare the effects of standard epoetin doses, which maintain patients’ hematocrit around 30 percent, and much higher doses, to raise hematocrit to normal (42 percent). The trial was stopped early in May 1996, just 29 months after it began, because of a trend toward increased deaths and heart attacks in the patients given the higher doses of epoetin. This was the first outcomes trial to show epoetin might be harmful. An editorial accompanying the August 1998 publication of the study in the New England Journal of Medicine, described how, “disappointingly,” there was a trend toward more deaths in the higher dosage group, but argued that the study still supported the recommendations of the National Kidney Foundation’s guidelines, published in the year before, that sufficient epoetin be used to maintain hematocrit between 33 and 36 percent—the upper half of the US Food and Drug Administration’s recommendation at that time—to reduce transfusions, improve quality of life, and possibly reduce deaths.

In March 2012, 14 years later, I published my own analysis of the trial based on Amgen’s clinical trial report, which I obtained through a Freedom of Information Act request. The results were dramatically different.

In 1998, although the risk of death and heart attack was significantly greater among patients receiving the higher doses of epoetin, the NEJM editors reportedly accepted Amgen’s position that the statistics should be adjusted because the company and the trials’ leaders terminated the trial early based on the recommendation of the data safety monitoring board. Therefore the trial results were reported as showing only a trend toward—and not solid evidence for—increased harm. The 1998 paper did not report the unadjusted statistics, or state that the statistical rules employed required the p value to reach 0.00088 to be considered significant, a much high bar than the traditional 0.05. This was also not appreciated by most experts.

According to the 1998 NEJM publication, the only definitive risk from higher epoetin doses was an increased risk of clotting the fistula from which blood is taken to perform hemodialysis, a common and less severe problem. And the higher dose group benefited by receiving fewer transfusions and enjoying higher measures of the “physical function” component of quality of life.

Thus, while the 1998 NEJM publication discouraged high epoetin doses to target hematocrit to 42 percent because of a trend toward increased risks, it also identified clear benefits of exceeding a hematocrit target of 30 percent—fewer transfusions and better quality of life. Epoetin use had already started to increase following the release of National Kidney Foundation’s 1997 KDOQI anemia guidelines, which incidentally, were funded by Amgen. After the NEJM paper was published, epoetin use in dialysis patients exploded, becoming a $2.5 billion a year market in the US alone.

By 2006, new KDOQI guidelines, again supported by Amgen, recommended all dialysis and kidney disease patients should receive doses of epoetin to maintain a hematocrit of 33 to 39 percent based on quality of life improvements—citing as evidence the Normal Hematocrit Trial and some smaller trials.

The results of my own analysis, on the other hand, published this March in Kidney International, showed that the quality of life scores had not improved in the higher dosing arm. The results I found in the Amgen report, filed with the FDA in 1996, showed that bigger epoetin doses to target higher hematocrit did not improve the physical function quality of life component at all, and had significantly increased the risk of death, heart attack, other thrombotic events, and hospitalizations.

The only benefit from higher epoetin doses was reduced transfusions, though the benefit was minor: one needed to treat 10 patients for 14 months at an additional epoetin cost of about $200,000 to avoid one person being transfused. Increased hospitalizations would further increase the total cost of higher epoetin doses.

The strikingly different results were because the 1998 NEJM publication had replaced the predefined outcomes and analyses with statistical adjustments and post hoc assessments, including replacing the total lack of effect of higher doses on physical function scores with the observation that patients with higher physical function scores had higher hematocrit, presumably because healthier patients respond better to epoetin. The wording was sufficiently unclear that even the KDOQI anemia guidelines misread the quality of life results as indicating that higher doses of epoetin to target higher hematocrit had caused an improvement in quality of life, when in fact, the trial results showed no improvement.

The academic authors of the 1998 publication state there was no intent to mislead, claiming the NEJM editors removed from drafts all the adverse results that I reported in 2012. They also state the 1998 publication clearly discouraged targeting hematocrit to 42 percent. What the 1998 publication did not do, however, was make clear, as my report does, that higher epoetin doses carry great risks, while the only benefit was a meager reduction in transfusion risk at great monetary cost.

Of course, even if there was no intentional deception, the effect was to force experts to say targeting hematocrit to about 42 percent using higher epoetin doses improved quality of life and reduced transfusions, and prevented them from saying such management significantly increased deaths, cardiac events, thrombotic events, and hospitalizations. Amgen controlled the debate, and by 2012 had made $37 billion from epoetin sales in the United States alone.

And if the intent was not to mislead, why not just publish subsequent articles clarifying the results, especially the quality of life results? The anemia workgroups that developed the KDOQI guidelines included one of the 1998 NEJM authors. Why not notify the KDOQI organization and other workgroup members that they were misreading the 1998 NEJM results? I cannot imagine what Amgen would have done if they had intended to mislead.

I waited 1,260 days to receive the trial report from the FDA. Two weeks before receiving the report, on June 24, 2011, the FDA released a safety warning and new label advice for use of epoetin in chronic kidney disease. It withdrew the previous recommended dose and hematocrit target of 30-36 percent. The new label states that there is no known safe dose of epoetin, no proven safe hematocrit target, and when using epoetin in dialysis patients, decrease or stop the epoetin when hematocrit exceeds 33 percent. The label now reports the Normal Hematocrit Trial’s hazard ratio and confidence intervals for death and heart attack and all-cause death as significantly higher in the higher hematocrit arm, just as I reported them earlier this year.

Finally, as strange as it seems, I am now the sole author of the publication on the predefined primary and secondary results of the largest outcomes trial of epoetin in dialysis patients, and I didn’t even participate in the trial. Perhaps the FDA will make the epoetin label cite my paper.

Daniel Coyne is a Professor of Medicine at Washington University School of Medicine in St. Louis, Missouri.

Add a Comment

Avatar of: You

You

Processing...
Processing...

Sign In with your LabX Media Group Passport to leave a comment

Not a member? Register Now!

LabX Media Group Passport Logo

Comments

Avatar of: MarcWathelet

MarcWathelet

Posts: 2

May 15, 2012

Fantastic work, Daniel, kudos to you!

Are you forwarding your work to the district attorney of the relevant jurisdictions?

What are NEJM editors going to do about this, bury their heads in the sand?

And this is probably the tip of the proverbial iceberg, what a world...

Avatar of: Daniel Coyne

Daniel Coyne

Posts: 1457

May 15, 2012

The purpose of academic research should be to inform and educate physicians. Amgen and the NEJM editors can provide their own explanations. However, this was an open-label trial, and every weekly hematocrit result was sent to Amgen directly which would then decide how the EPO dose should change. Amgen would also know about every hospitalization and every death within 2 days. So Amgen was undoubtedly tracking the primary endpoint and deaths and knew the p value was hovering around 0.01 when they stopped th trial. Amgen, not the safety board, stopped the trial. The safety board only made recommendations. Amgen stopping the trial was in their interest, and the editors at NEJM knew that, and effectively rewarded Amgen by adjusting all the outcomes to insignificance.

May 15, 2012

Any incomplete work or findings shouldn't b tried practically at mass level. Patience is required in scientific conclusion.

Avatar of: smaloj

smaloj

Posts: 1

May 15, 2012

It is high time for NEJM to say something about this! 

Avatar of: Michel

Michel

Posts: 1457

May 15, 2012

Very facisnating.
You may be aware of the book Blood Feud that details the court cases of a former sales rep from Ortho Biotech (J&J division) about exactly the same argument as you present (“Blood Feud: the man who blew the whistle on one of the
deadliest prescription drugs everâ€쳌 (NY: Dutton, 2011)  by Kathleen
Sharp). Unfortunately, the sales rep died before the case was ever heard. Perhaps it is time for a new look at it.

Avatar of: JustAskAlice

JustAskAlice

Posts: 5

May 15, 2012

Thank you for your investigative work and for sharing your findings. You've done a great service to the community.

May 15, 2012

You're the sole author of a paper you knew nothing about?  Methinks that is scientific misconduct on the part of the people who submitted the article under your name.

Avatar of: Daniel Coyne

Daniel Coyne

Posts: 1457

May 16, 2012

If you change all the words in this sentence, you could be the author!
"I am now the sole author of the publication on the predefined primary and secondary results of the largest outcomes trial of epoetin in dialysis patients, and I didn’t even participate in the trial."

Avatar of: Rama

Rama

Posts: 1457

May 16, 2012

Dear Dr Coyne,
USA scientists are the best in the world but they are also the worst in the world because they are champion in making a cat to look like a lion.
In India, poverty did not allow the patients of chronic renal failure to take more than one injection weekly or once in 15 days, which was good for them.
How can I invite you to have a drink with us Dr Daniel Coyne?
RBSingh
www.rbsingh.bizpa.com

Avatar of: jhnycmltly

jhnycmltly

Posts: 65

May 16, 2012

I caught these guys quite awhile ago {Apr 22 2006] when they began to study WHY for-profit dialysis centers had such a high use of erythropoietin.
At that time I wondered why RATHER than to test WHY for-profit dialysis centers 
were using too much erythropoietin , they SHOULD be using their time telling everyone about the dangers OF using to much erythropoietin as opposed to study 'why' they WERE using too much epo. It was quite awhile before they decided to place epo black box warnings. I still question smart guys about where THEY were during this increased deaths using erythropoietin. Never get too many answers.

Avatar of: jhnycmltly

jhnycmltly

Posts: 65

May 16, 2012

On a side note here , cup half full , these epo studies tell us  , the hemoglobin we accept as normal may in fact be too high as has been proposed in children ?
"The diagnostic criteria for iron deficiency in infants should be reevaluated."
The investigative studies of epo found a person on dialysis begins to die when the hemoglobin rises just up to "low normal". One MUST conclude hemoglobins at low normal or above DO have evidence of increased death , at least in some. Investigators should now try to PROVE , rather than accept blindly , what we consider to BE healthy hemoglobin ?
Especially NOW since even going as high as low normal has been shown to have adverse effects , at least in some ?
Study populations with "low normal" versus "normal" in regards to increased or decreased health , as opposed to the customary "how do you feel" , approach ? Science wise as opposed to subjective wise ?
Such as this ?
"Iron Deficiency Protects Against Severe Plasmodium falciparum Malaria and Death in Young Children"

Avatar of: Daniel Coyne

Daniel Coyne

Posts: 1457

May 16, 2012

I read the book, and it was indeed fascinating.

Avatar of: Daniel Coyne

Daniel Coyne

Posts: 1457

May 16, 2012

Scientifically valid conclusions require examination of all the evidence, and exclusion of those predisposed to misjudge the evidence. My article highlights the necessity of getting the results of ALL clinical trial data into the public domain in a timely fashion. As a wise person recently said to me "Human data belongs in the human domain, not locked in the vaults of the sponsor or the FDA."

Avatar of: Rama

Rama

Posts: 1457

May 16, 2012

Dear Dr Coyne,
USA scientists are the best in the world but they are also the worst in the world because they are champion in making a cat to look like a lion.
In India, poverty did not allow the patients of chronic renal failure to take more than one injection weekly or once in 15 days, which was good for them.
How can I invite you to have a drink with us Dr Daniel Coyne?
RBSingh
www.rbsingh.bizpa.com

Avatar of: Navdeep Tangri

Navdeep Tangri

Posts: 1457

May 18, 2012

Dear Dr. Coyne
Really excellent and illuminating work. Many of our colleagues will share that high Hb in our patient population really don't impact patient's physical well being or vitality, but rather act as a placebo for the nephrologist who thinks he is having an impact on HRQOL. Its time we shatter the myth of ESAs.

Congratulations again, Sincerely
Navdeep Tangri

Avatar of: Gary Williams

Gary Williams

Posts: 1457

May 18, 2012

And this happens despite the presence of regulatory bodies and the peer-review standard for publishing. Can you imagine what would happen if those "free-market" promoters/propagandists (eg. CATO, Manhattan Institutes) actually got their way? To even suggest things like industry and private enterprise *always* knows best. Or that the market alone should decide matters of whether a drug's efficacy (or any commercial product really) outweighs it's lethality..... How anyone could take them seriously, nevertheless the many millions who do, seems to me to be a field of research with huge importance for humanity as a whole. It's no coincidence whatsoever that the same people denying the existence of AGW also happen to come largely from the Ayn Rand, libertarian camp of social Darwinism proponents.

Avatar of: Echohawk

Echohawk

Posts: 4

May 19, 2012

Great read , Daniel Coyne , thank you :)
I realize that no system of approval is perfect but what suggestions can we give governing bodies like the FDA to improve their system ?
Are there any perceived gaps ?
Are the standards of approval of new drug therapies insufficient ?

Avatar of: LeoK711

LeoK711

Posts: 1

May 30, 2012

I found this article via a reference on Medpage Today.

Immediately above the headline for this was another one:

Drug Firms, Journals Tackle Credibility Gap 

"Eight big pharmaceutical firms and representatives of major medical journals
have agreed on 10 steps for increasing the transparency and credibility
of industry-sponsored clinical studies."

Not a moment too soon, apparently! Hopefully it's more than window-dressing.

Avatar of: Arthur De Vany

Arthur De Vany

Posts: 2

May 30, 2012

Your point, if there is one, has nothing to do with the debate. It does show that government is easily manipulated and that drug studies are too. Any Manhatten Institute or CATO scholar would say that Amgen and the deceptive scientists (and even the journal) ought to pay for this fraud and the damage it has wrought. Including the government watchdogs who were not even smart enough or cared enough to prevent the fraud.

My wife died and I saw the quality of her life decline for many reasons. Epoetin was just one of the many aspects of that. 

It is time for that study to be withdrawn by the journal. And shame on the editors and referees. As a long-time referee in another field I see this sort of thing all the time.

Avatar of: Arthur De Vany

Arthur De Vany

Posts: 2

May 30, 2012

How have so many journals become complicit in these sorts of frauds? Is it the funding? 

Avatar of: DCoyne

DCoyne

Posts: 3

October 12, 2012

Sorry for a late reply. You are correct that those who did much less than experts recommended actually protected their patients.

Avatar of: DCoyne

DCoyne

Posts: 3

October 12, 2012

Thank you Dr. Tangri for your comments. You are precisely correct that ESAs are now being sold as nothing more than an expensive placebo with dangerous side effects. Regretably, the new guideline group - KDIGO also promotes this nonsense, recommending Hgb goals as high as 11-12.9 g/dL, which is totally unnecessary to prevent transusion, and two trials have shown no imrpovement in HRQOL in that range vs lower targets. Rather than state their guidelien is despite trial evidence, KDIGO lists the recommendations as 'not (evidence) graded'. It should be graded "false".

Avatar of: DCoyne

DCoyne

Posts: 3

October 12, 2012

A bleated thanks to Gary Williams, Echohawk, LeoK and Arthur De Vany for the comments. My condolences to Mr. De Vany on the death of his wife. It is a tragedy of the EPO story that many patients were hurt by unnecessary treatment and overtreatment, and how many doctors did so in the honest belief they were helping their patients. Please read Amgen's Sean Harper's rebuttal on this site which refutes none of my facts. Amgen and Sean harper are effectively saying they would do exactly the same thing now, and indeed I am certain they would.

Follow The Scientist

icon-facebook icon-linkedin icon-twitter icon-vimeo icon-youtube
Advertisement

Stay Connected with The Scientist

  • icon-facebook The Scientist Magazine
  • icon-facebook The Scientist Careers
  • icon-facebook Neuroscience Research Techniques
  • icon-facebook Genetic Research Techniques
  • icon-facebook Cell Culture Techniques
  • icon-facebook Microbiology and Immunology
  • icon-facebook Cancer Research and Technology
  • icon-facebook Stem Cell and Regenerative Science
Advertisement
Thermo Scientific
Thermo Scientific
Advertisement
NeuroScientistNews
NeuroScientistNews
Life Technologies