NIH's Raub on Misconduct

Author: Tabitha M. Powledge Date: December 15, 1986 In August, William F Raub, a 20-year veteran of the National Institutes of Health, was named its deputy director. Raub received an A.B. from Wilkes College in 1961 and his Ph.D. in physiology from the University of Pennsylvania in 1965. He directed the development of PROPHET—an integrated computer system for studying chemical/biological interrelationships. From 1983-1986, he headed the agency's extramural program, including all research funded by grants, cooperative agreements and contracts. He was also NIH's point man on issues such as research misconduct and the care and use of laboratory animals. He was interviewed November 25 by Tabitha M. Powledge, editor of THE SCIENTIST; this is an edited, shortened transcript of their talk. Q:You've spent the last several years here at NIH dealing with some of the most important policy issues facing American research. I'd like to begin with one that's very much in the news at the moment: misconduct, particularly fraud, in science. What's your estimate about how widespread it really is? RAUB: I don't think anybody, including me, is in a position to give any kind of verifiable estimate as to how widespread it is. It's my belief that it's extremely infrequent. The few cases we see are certainly not everything but are, I believe, a large part of the fundamental wrongdoing in the scientific community. I believe the kinds of incidents we've seen lately probably have always occurred. In the past, they either went undetected, or were detected and dealt with in some quiet and summary fashion that didn't generate much publicity and didn't unjustly involve individuals. There's a difference between now and, say, 10 years ago. There's a greater willingness today for people to deal with it in the open, not only to report to higher-level executives in the case of universities, or from universities to government sponsors such as NIH, but also to follow the whole process through the investigations and the follow-up. A byproduct is what I call the improvement of our institutional reflexes. In the early 1980s when a case occurred, there was considerable uncertainty both in the scientific community—especially the universities—and here at NIH as to what to do about it. As we gained more experience we also got much more predictable in our own behavior, and developed some rules. The latter have now been published as a way of making clear to everyone what it is we're concerned about and how we go about looking into matters. We also specify what kind of sanctions or penalties we impose when we determine that wrongdoing has occurred. A cornerstone of our policy is that the grantee institution has the first responsibility to promote proper behavior and to deal with real or apparent misbehavior. NIH re serves the right to follow up on what they do or even to start over if their investigation doesn't produce an outcome that we feel is fully satisfactory. As we gain more experience we're closing that gap. Not only is it more and more likely that a university or other grantee institution will pick up promptly on a problem, but, generally speaking, they now do it in such a way that NIH often can affirm what they've done. In earlier cases we found it necessary to do an additional inquiry. In a few cases we had to satisfy ourselves that the institution was as complete as was realistic, and in a few very early cases, we started over. In one of those cases, at least, the institution asked us to do so. GUIDELINES ON MISCONDUCT A document entitled, “Policy and Procedures for Dealing with Possible Misconduct in Science” was approved last spring by the Assistant Secretary for Health. It was then published as a special issue of a document called “The NIH Guide for Grants and Contracts,” which is a weekly notification that we send to approximately 30,000-35,000 individuals or institutions. That document covers not just NIH, but all of the U.S. Public Health Service. It covers real or apparent misconduct associated with awards—that is, grants and contracts—for NIH and the other agencies, such as the Alcohol, Drug Abuse, and Mental Health Administration (ADAMHA), the Food and Drug Administration (FDA), the Centers for Disease Control (CDC), and so on. It also covers possible misconduct by scientists employed in our intramural program. For PHS, the lion's share of that is at NIH, but it also includes the National Institute of Mental Health, its labs and clinics, and the laboratories for other parts of ADAMHA, the FDA, and the CDC in Atlanta. Finally, there is research that is not funded by any part of PHS, but that is regulated by it—mostly by FDA, to a lesser ex tent by the National Institute of Occupational Safety and Health, which is part of the CDC. That's a different kind of relationship: the PHS neither employs nor funds the grantee, but as a regulator has some responsibility. For example, if someone is found to have submitted false information to the FDA in support of a new drug application, it raises questions about his or her fitness to work for us as an intramural scientist, to be on our advisory committees, or to get a grant from us. Also, in the NIH reauthorization bill passed last year, there is a one-sentence provision that says that institutions who get grants and contracts must have a misconduct policy in process. There is now a pro posed amendment to our regulations saying that assurance must be given on the grant application. Under our current regulations, there is no requirement that apparent misconduct be reported to us. So, now we have it explicitly in the new authorization bill, and are in the rule-making process as well. Q:Are institutions really interested in getting help with the process? Are they uncertain about how to investigate these cases? RAUB: Not as uncertain as in the past. Typically, they would come to us and say “Here's something we've put together. Would you comment on it?” That was a good two-way street. We learned as much as we gave in those exchanges. Then there was the initiative of the Association of American Medical Colleges and the American Association of Universities, who put together a statement that they hoped their member institutions could embrace. In large part, we not only supported those documents, but we encouraged people to follow them. We really think they're a first-class effort. RESPONSIBILITIES OF CO-AUTHORS Q: There's been a lot of attention to the responsibilities of co-authors in the last few months as a result of the Harvard case, the Slutsky case, and the Darsee case. RAUB: I think there is renewed attention to that problem. There was a flurry of attention at the time of the Darsee case, but it was eclipsed by some of the other issues of fabricated data and other celebrated aspects of that case. I saw very little organized attention to it as an issue to the scientific community until the Slutsky case and some of the more recent ones were publicized. That's a very healthy renewal of concern. I'm hopeful that the science community in general, and especially the journals, will in some organized way deal with that question. Q:You see that as a journal responsibility? RAUB: Primarily. In putting together our procedures, we were very careful to define misconduct as the big sins: fabrication of results, falsification of research records, plagiarism, deliberate false reporting, things that any of us would agree violate every canon of science: they are fraudulent and dishonest. We added to the definition of misconduct the failure to abide by the terms of the awards, such as the care and use of laboratory animals, the protection of human subjects of research, and so forth. We deliberately omitted behaviors that may or may not be wrong depending on the view of the individual or the extent to which the behavior is indulged in. Co-authorship is a good example of that. There is wide disagreement within the scientific community as to what constitutes proper and appropriate co-authorship. Almost everybody would agree that cases of adding names to papers for purely honorific purposes or without people knowing their names are to be added are clearly wrong. But while most people I know would call the last practice fraudulent, they would not call the honorific case fraudulent. We've taken the position that those kinds of things are more the practices of the profession than a proper subject for government regulations. We're more than willing to be a catalyst and a player in that game, but we don't feel it's primarily the responsibility of the federal government, nor should we even be the initiator of regulation. I think it's time, perhaps overdue, that some of the major journals stepped out and said “We in the profession and the community have a problem and we should deal with it.” Q:Do you think it's accidental that the fraud cases that have surfaced so far have all been in biomedical research as opposed to, say, high-energy physics? RAUB: I don't think it's accidental; many explanations have been offered for it. One is the nature of biological and biomedical investigation. It has a greater inherent variability in the subject matter itself and a much heavier reliance on an observational or experimental paradigm with little theoretical base to drive it. As a result, both the variety and the experience in the laboratory and the difficulty of detecting false information are greater in biology and medicine. I would add that in several of the more celebrated cases like Slutsky and Darsee, the individuals concerned had publication rates that are prodigious by any standard. That was true even though they worked in communities of scientists where the norm was a dozen or more publications pen year, in contrast to some other fields where a good publication or two a year is considered substantial. PEER REVIEW IN FUNDING Q:How satisfied are you with the present peer-review process as a way of deciding what research gets funded? RAUB: No one would claim it's perfect. But I think it is on the whole a first-rate and very reliable system and that there is a considerable commitment both outside and inside NIH to making it as good as it can possibly be. I never cease to be amazed at the energy and the discipline and the integrity of peer reviewers and what they commit to the process. We try as hard as we can to elicit and sustain that kind of behavior from people by recognizing their worth and the quality of their contributions and by reinforcing it through an elaborate set of procedures governing how we deal with real or apparent conflicts of interest and so on to keep the system as free of criticism as we can. There are two chronic problems with peer review. One is that as the volume of applications grows, the likelihood of a flat-out mistake grows. With the workloads we now have, the probability of our making an error, either in substance or in process, is greater. Twenty years ago when a recommendation of approval from a peer-review group was almost a guarantee of being funded, the sys tern had a tolerance for errors, even though I believe they were made at a lower rate. These days the higher volume of applications and the stringent competition mean that something that is seen to be only very good and not truly superb may go unfunded even though our dollar resources have grown each year. So the tolerance for such an error in the whole system is less, especially considering its effect on the career and the general frame of mind of the scientist out there. The other is the timeless problem of the institutionalization of scientific orthodoxy. The very act of bringing together contemporary experts tends to reinforce the contemporary dogma, and therefore, in some instances, the inability to distinguish between something that is truly silly and something that is the product of genius working a different paradigm. Q:So you think it is biased against new ideas? RAUB: It has the capacity to be biased, but I think it's unfair and wrong to label the sys tem as inherently biased against new ideas. In most cases we've looked at, a second closer examination has shown them to have been silly ideas—they were not well supported either by theory or by preliminary evidence. But there have been occasions when our secondary examinations have led us to conclude that the review was a bit too locked into some established way of seeing it, and the second examination gave us an opportunity to loosen up the system. Q:Can you give me an example? RAUB: There is some work involving the mechanisms of carcinogenesis. Over a succession of years an individual had brought in a stream of grant applications. Almost a! ways those grant applications were recommended for approval, but most of the time with a score that was just outside the range to be funded. On one occasion one was funded, and he received just enough funding for the laboratory to sustain itself. So the individual continued what he viewed as his battle with the establishment. Because we know that we and our reviewers can make mistakes, we set up a formal appeals process where someone can challenge the review and get a hearing of one kind or another. The appeal has two levels to it. The first level is very efficient—a letter of rebuttal in which you explain what you think are the fallacies of the review. We then ask the institute to which the application was assigned to make some substantive response to that rebuttal. There's a variety of possible outcomes. At one extreme, that rebuttal goes back for reconsideration by either the full study section or some equivalent. That's still quicker than revising and resubmitting the application. The individual gets an effective protracted review the second time around. At the other extreme, our institute staff have both the authority and the responsibility, if they are persuaded by that rebuttal letter, to do the appropriate thing then and there. Q:How many times a year does that happen? RAUB: Only a handful, I'm sure. Q:How long does the average appeals procedure take? RAUB: On the order of three to four months. The really long ones can go a whole year, and those are the ones that lead legitimately to concern about people being put in limbo. Nothing stops them from submitting an un related application if their scientific interest is so broad that they have another field of research, as many people do. We try very hand for their sake as well as ours not to get locked into that kind of prolonged dispute. In the case of the work on carcinogenesis, the individual submitted an appeal. The grant application had been recommended for approval, but just out of reach of the funding. In the course of one afternoon, I read 10 years' worth of this individual's encounters with our peer-review system. What I found fascinating was the shifting positions of the scientific establishment. This individual proposed a particular mechanism of carcinogenesis and the first critique said it couldn't possibly work that way. Three or four years later, he was farther along and he proposed some variation on this theme. The new critique acknowledged that the fundamental phenomenon existed in nature, but said it couldn't really be quantitatively very important. About—a little hyperbole now—seven years later, he was onto some further variant, and they were now acknowledging it as a fundamental mechanism. So he was wearing them down. In that case we went through our whole appeals process and reached the stage of collective disagreement. The head of the review operation, who had not been involved in any of the conduct of the review, felt that the review had been proper. The director of the relevant institute felt that there could have been more flexibility in the review procedures even though there was nothing wrong. He was leaning in favor of the investigator, and I broke the tie in his favor. THE FUTURE OF NIH Q:You've been at NIH essentially your entire career. Are there new directions you'd like to see NIH move in? RAUB: No, I don't see a need for a fundamental change. To me the most important thing is the recognition that the environment in which we work is changing constantly. Some of that evolution comes out of science itself. A preoccupation with molecular biology 20 years ago would have been fruitless. If we're not prepared to exploit it today, we're missing the point. The world around us has changed, too. Twenty years ago, not only did we view ourselves as a pure research entity, but the rest of the world viewed us a pure research entity. Increasingly, we've been expected to re late outside the narrow research arena. No one is demanding that we deliver health care, but there are legitimate expectations that we will have some influence on the health care that's delivered.

NIH's Raub on Misconduct

Interested in reading more?

Become a Member of