NIH Women's Health Researchers Rebut Criticisms Of Their Study

Project scientists reject Institute of Medicine claims that their ambitious initiative is scientifically and financially deficient One of the main questions that Women's Health Initiative (WHI) investigators hope to answer, and one specifically challenged by the Institute of Medicine (IOM), is whether a low-fat diet will reduce breast cancer risk. This hypothesis arises from comparisons of epidemiological data from countries with different fat-consumption patterns and of data tracking the change in breast cancer occurrence among migrants moving between some of these countries. Japanese women, for example, have much lower breast cancer rates than do women in the United States, and the traditional Japanese diet contains much less fat than is the norm in the U.S. Women whose parents moved from Japan to the U.S. have breast cancer rates comparable with those of other women in the U.S. The IOM report, however, suggested that these comparisons are based on the amount of fat produced and sold, not necessarily on the amount actually consumed by women. It also noted that many epidemiologists believe the amount of fat consumed by women during adolescence, when breast tissues are developing, may be more important to overall breast cancer risk than that consumed as adults. Hence, lowering dietary fat levels in the WHI study population of women, all over 50 years of age, may have little effect on their cancer rates. Several studies also have failed to show a link between dietary fat and breast cancer rates, according to Walter C. Willett, a professor of epidemiology and nutrition at the Harvard School of Public Health, Boston. Willett is a coinvestigator in the 120,000-participant, 17-year-long Nurses Health Study at Harvard-affiliated Brigham and Women's Hospital, also in Boston. In 1989, 110,000 younger women were added, with Willett as principal investigator for that part of the study. "When investigators like ourselves started to do more detailed studies," Willett says, "where you look at fat and control for other factors, the relationship between fat intake and breast cancer hasn't shown up. As the evidence has come in, the hypothesis, which was a reasonable one-- that's why we spent a lot of time looking at it--just hasn't been supported." William Harlan, codirector of the WHI and associate director of NIH's Office of Disease Prevention, counters that these studies, many conducted within single countries, are not a fair test of the low-fat hypothesis. "The range of fat intake within the U.S. and within most industrialized countries tends to be quite high and quite narrow," Harlan says. "It does not encompass the low fat levels of some of the countries where the breast cancer rates are low--for example, in Japan." He adds: "You won't find many people [in the U.S.] down at the 20 percent level, which is a level that some people feel is the important level for total fat intake." The suggestion by IOM that the WHI switch its primary focus to the link between dietary fat and coronary heart disease is unwarranted, WHI researchers say. That link already has been demonstrated by other studies, they say, and is a less compelling goal for this landmark women's health research effort than breast cancer. "On that panel, there were assembled a group of very capable people, some of whom have experience in this particular area, some of whom do not," says Richard A. Carleton, co- principal investigator of the WHI Vanguard Clinical Center at Memorial Hospital, Pawtucket, R.I. "They looked at this and said that, in their view, the dietary modification was more likely to change coronary disease. That is not scintillating new information." Several clinical studies already provide strong evidence that dietary changes can affect heart disease outcomes, says Carleton. "You can argue a lot of those data come from men," Carleton says, "but, in this area, I don't have terribly good reasons to hypothesize that women are going to be radically different from men. If you ask yourself where is the biggest public health issue for which there are imperfect--very imperfect--data, it is breast cancer." Some critics, however, go beyond the IOM suggestion that the WHI emphasize different questions. Willett, for example, says that the WHI not only is asking the wrong questions, but also will not, in any case, be able to answer them as currently designed. "There's a fundamental problem with the design of the women's health study," Willett says, "and that's that they're changing many factors simultaneously. They're not just lowering fat intake, they're increasing fruit and vegetable and vitamin intake. So, even if you do see something, you can't say that it's due to fat. If that's an interesting question, they're doing the wrong study." Willett says the WHI may be important politically, but not scientifically. He adds that the high cost of the study amplifies the significance of the problems. "The scientific world largely believes it's a disaster," Willett says. "It's a huge investment of money--it's going to be a billion dollars or more in the end--and it can't answer the questions that it was set out to answer. No one would be disturbed about it if it were a few tens of millions of dollars, but it's definitely taking money out of lots of other laboratories." WHI investigators reject such statements and say the study questions are not only important, but that the WHI, as designed, can answer them. "The important thing about the WHI is that it will test in experimental fashion some hypotheses that have not been tested in humans any other way," says Robert Wallace, principal investigator of the Vanguard Clinical Center at the University of Iowa, Iowa City, "the fundamental one being the effect of low-fat diet on a variety of cancers. That's never been done." Echoing other WHI researchers, Wallace characterizes challenges to the dietary fat-breast cancer link hypothesis as legitimate differences of epidemiological data interpretation. "It's an honest scientific disagreement," Wallace says. "Various investigators--as we all do--lean more heavily on their own data than on others. Some studies show an effect and others don't. But low-fat diet has the potential for improving health in so many areas that it is very much worth testing." "To most of us who believe that one needs to test this, the clinical trial seems the only way to do it," Harlan says. "It's a matter of deciding whether you believe that the hypothesis merits a test." Carleton suggests that the debate over the WHI's scientific goals, in itself, shows the need for the study. "Where there are differences of opinion, the facts, usually, are not known," he says. An Institute of Medicine (IOM) report released late last year said that one of the study's primary hypotheses--that a low-fat diet will reduce the occurrence of breast cancer in women over 50 years of age--is scientifically weak, given the existing data. The IOM group suggested instead that the investigation of a secondary hypothesis, that a low-fat dietary pattern reduces the risk of coronary heart disease, be emphasized. The panel also found that the WHI underestimated the per-participant, per-year costs at about $1,000--far lower than the $2,000 to $3,000 required in other NIH studies--a fact that might endanger the study's successful completion. The women's health project ranked as one of former NIH director Bernadine Healy's highest priorities. As early as her 1991 confirmation hearings, she announced her intent to launch such a program as a corrective device for years of health research that excluded women as subjects. Healy pressed the project forward aggressively throughout her directorship, which ended in June 1993. The resulting WHI is the largest research study ever conducted by NIH, involving more than 160,000 women over the age of 50 at 45 clinical centers across the United States. The women are to be tracked for 14 years at a cost of more than $625 million. The study has been widely viewed as one of the foremost achievements of Healy's tenure at NIH. Partly because of the history of women's health research, however, or perhaps because of Healy's confrontational style as director, the WHI has attracted more political interest than do most NIH research efforts. Some women's health advocates suspect that, even now, the criticisms of the WHI may be, at least in part, politically based. "I believe, with all my heart, that some of this is political," says Nancy Brinker, founding director of the Susan G. Komen Breast Cancer Foundation in Dallas and chairwoman of the President's Cancer Panel Special Commission on Breast Cancer. "Politics does enter medicine and science." Brinker is among those who praise Healy's efforts to make the WHI a reality. "It was a major accomplishment," says Brinker. "Maybe the study was designed, in some people's eyes, a little too rapidly; maybe it wasn't perfectly designed. But what people forget is that Bernadine Healy had the guts to stand up and talk about [women's health] issues." WHI officials answer the IOM critique by saying that WHI hypotheses and the report's suggestions simply represent differing assessments of the existing epidemiological data. In fact, they say, this is why the WHI's clinical trial is so important--it will provide hard data against which to check these differences (see accompanying story). "The disagreement really came about as an honest scientific disagreement among people collecting data and interpreting them in various ways," says William Harlan, codirector of the WHI and associate director of NIH's Office of Disease Prevention. "It's not resolvable by further [epidemiological] studies of the same type." Estimates of study costs also were carefully done, WHI investigators say, and the study can be completed as budgeted. "We budgeted it tightly, but, we think, realistically," says Robert Wallace, principal investigator of the Vanguard Clinical Center at the University of Iowa, Iowa City. "But it's very hard to know. Obviously, budgeting for that long a period is an exercise in guesswork." So far, 16 of an eventual total of 45 WHI clinical centers have been funded; these first 16 are known as the Vanguard Clinical Centers. One coordinating center, at the Fred Hutchinson Cancer Research Center in Seattle, has also been funded. "We looked at our budget six ways from Sunday to be sure that we could do it," says Richard A. Carleton, co-principal investigator of the WHI Vanguard Clinical Center at Memorial Hospital, Pawtucket, R.I. "We can do what we have said we will do for the amount that they told us we could have," says Carleton, who is also physician-in-chief at Memorial Hospital. In addition to challenging the WHI design and budget, the IOM panel was openly angry about delays at NIH that resulted in the panel's reviewing a project that was already awarding multiyear contracts and scheduled to begin participant recruitment during the review period. The House Appropriations Committee asked for the IOM review of the WHI in July of 1992; NIH did not approach IOM about the review until February 1993, and a contract was not signed until April 1993. There were limits, as a result, to what the committee felt it could responsibly recommend. "There is a tension evident in this report," wrote committee chairwoman Marion J. Finkel, a vice president at Sandoz Pharmaceuticals Corp., East Hanover, N.J. "Had the committee been asked to design a plan for women's health research, it would not have designed this WHI." Meanwhile, advocates for improved women's health research and care say the time has come to move beyond arguments over which scientific questions should be asked by the WHI. The project--as designed--is important, they say, and should go forward. "I'm so frustrated at the length of time that this has taken, at the exclusion of women from important trials and so on, so that, at this point in 1994, we're still asking questions that should have been answered--or at least asked- -20 years ago," says Brinker. "I would hate to see this trial abandoned. I've always thought it was an ambitious, appropriate thing to do." In its report, the IOM panel said that the per-participant, per-year costs might be twice or even three times the $1,000 estimated by NIH and that the total costs for the WHI clinical trial probably would be greater than the $625 million allotted. This assessment, they said, was based on comparisons to other recent, similar NIH studies of women's health. The additional costs, they said, would have to be borne by the 45 clinical centers, a fact they felt might threaten the overall WHI. "NIH and Vanguard Clinical Center representatives have indicated that the additional funds necessary for successful completion of the trial will be covered by the institutions in which the Clinical Centers are based," the report said. "This reliance on institutional support may be reasonable in the case of the [16] Vanguard Clinical Centers, but the committee felt it is unlikely that an additional 29 institutions can be identified that have both the experience to carry out the tasks of high-quality research and the ability to provide additional resources." WHI officials take strong exception to the IOM cost assessment, including disagreements on some points of fact. They say, for instance, that individual clinical centers will not be asked to take on additional expenses at any time. "Our contracts are very clear on that," says Carleton. "The IOM had an erroneous understanding, somehow, on that one." Carleton adds that the costs were carefully and conservatively analyzed, including adjustments for factors that have skewed the results of several previous studies. Such factors include, for instance, the "healthy volunteer effect." People who volunteer for clinical health studies such as the WHI tend to be in better health than the average person, it has been shown. WHI director Harlan defends the per-participant calculations that support the overall project estimate of $625 million and says IOM was not comparing "apples with apples" in its critique. "The comparisons that were made are really inappropriate," says WHI codirector Harlan. "The comparisons they made were with relatively small studies of 1,000 or fewer participants, lasting two to three years. A good part of the cost in a large clinical trial is recruitment and the early initiation of people into the treatment and measuring of the adverse effects. If you take those costs, which may be a third or 40 percent of the total cost, and amortize them over the 14-year period, the costs per participant, per year become much smaller." Harlan adds: "They didn't carry out a very careful cost analysis--in fact, they didn't provide an independent cost analysis to buttress their argument." Panel chairwoman Marion Finkel acknowledges that the panel had difficulty assessing WHI costs, but with good reason. "We didn't get sufficient detailed information from NIH to be able to make those calculations," Finkel says. "Our calculations were based upon some of the information that was supplied by the Vanguard Centers, and we just costed it up." One IOM recommendation--based on its assessment that the costs would exceed the $625 million budgeted--was that the study end sooner, in 2002 instead of 2005. This suggestion was partly a reflection of their hoped-for changes in hypotheses, also. The shortened project would allow time to test the heart disease hypothesis, for instance, but not enough time to test the effects of dietary modification on breast cancer risk. But, most important, according to Finkel, the shorter study would begin to address concerns over the budget. "It would save money," Finkel says. "The committee recommended that the NIH look at the study several years before its scheduled completion, to see if any trends were emerging that would require that the study continue, and, if there were no such trends emerging, to terminate it." In another area, the panel said that women in the study should be better informed about the potentially higher risk of breast cancer that may result from hormone replacement therapy, important in one part of the study. Here, WHI officials agree with the recommendation and plan changes in consent forms and other materials so that participants in the huge project will be more fully informed about these health risks. The report noted that the House Appropriations Committee directed NIH in July 1992 to contract with IOM "to complete a review of the design and estimated costs of the WHI by February 1, 1993, in time to make adjustments, if necessary, before major study activities began." Yet NIH did not formally discuss the project with IOM until February 1993, the report says. A contract was not agreed upon until late April 1993, well after the due date requested by Congress. "Whether the delay was due to a slow-moving bureaucracy or to other causes is unknown to the IOM committee," they wrote. "As a result, however, the committee was faced with the task of critiquing a study in progress. In the interim, NIH had requested, received, and evaluated proposals, and entered into contracts with 17 research institutions to serve as the Clinical Coordinating Center and Vanguard Clinical Centers." Harlan says that the delay can be explained, at least in part, by a closer look at the necessary legislative process. While the House Appropriations Committee may have asked for the review in July 1992, he says, the appropriations bill containing the request did not become law until October. "We didn't have a bill and a mandate to carry out a study until October of 1992," Harlan says. He acknowledges, however, that NIH's delay between October 1992 and February 1993 in engaging IOM for the then congressionally mandated study cannot be entirely blamed on bureaucratic process. "Dr. Healy, who was then the director, didn't move quickly on it," Harlan says. "I don't know how to say it. I think that probably led to some of the irate response that you sense in the report." In response to the IOM panel's complaints about being asked to review an ongoing project, Harlan says that there was never a time when the panel could have looked at a project that was still under development. "Had they started in October, which was the original presumption on the part of the House Appropriations Committee," Harlan says, "the project would still have been under way at that point, in that we had, at that juncture, funded a coordinating center and were negotiating the funding of the first 16 Vanguard Clinical Centers." Harlan does say that, with the help of new NIH director Harold Varmus, there is a "movement toward resolution" under way. WHI officials, IOM panel members, and, perhaps, others will be invited to discuss relevant scientific and other issues and to make a recommendation to the director, Harlan says. Despite the sharp nature of the IOM critique, there is general agreement on the value of the WHI and reason to believe that an accommodation can be reached. "If the committee really felt very strongly that the study had serious problems, we could have recommended that it be terminated," says Finkel. "But we didn't. We thought there were a lot of strengths in this study."

NIH Women's Health Researchers Rebut Criticisms Of Their Study

Interested in reading more?

Become a Member of