Developing Damage Control for Bad Data

Hippocrates wrote "I look upon it as being a great part of the art to be able to judge properly of that which has been written." The aphorism is as true today as it was in his time. One major difference, of course, is scale; today's medical publishers turn out some 15 million pages a year. For a scientist or practitioner to "judge properly" of all that is written within a major discipline is clearly impossible. There are quality control checks, however. Colleagues, peer reviewers, editorial boards and editors apply their professional judgment to ensure that meritorious manuscripts are accepted and published. Copy editors, proofreaders and even computerized spelling checkers are indispensable in ensuring that what appears in print is what was intended. Librarians, indexers and database producers take pains to see that bibliographic data are accurately recorded; a reversed digit in a year or volume number can make it difficult to locate an article. Nevertheless, even with all these controls, errors do occur. Sometimes the errors are embarrassing but inconsequential. On the cover of the November 10, 1978 issue of that prestigious journal, the title SCIENC appeared in 72-point type. However, a small error, typographically speaking, can be serious. A dosage figure cited as 1.0 that should have been 0.1 could have fatal consequences. For six months in 1986, the National Library of Medicine (NLM) kept track of all errata notices in the literature it indexes for Index Medicus. The frequency of error was found to be 1 in 1,000 articles. Many errors were not life-threatening; misspelled names are injurious only to the author's vanity. In about 1 in 2,500 articles, however, the error was serious—a misstatement of dosage, measurement or formula data. Equally disturbing, and detrimental not to individual patients but to the entire enterprise of science, is the occasional instance of acknowledged and reported fraudulent data. The publication of research results is the foundation of all subsequent investigation. Although prudent scientists expect to encounter error in the literature—insufficient data, erroneous inference, wrong conclusions—they presume the good faith of the author. When this trust is breached, the edifice of science shows evidence of crumbling. What We Can Do We must do what we can to limit the damage of errors in the literature. Editors should publish errata notices and letters to the editor to correct important data. Authors should include a correction in reprints they distribute. Librarians should insert published errata slips with the original article and, when feasible, actually correct the original text. Database producers should correct errors in computerized records. The NLM, for example, includes a reference to an erratum notice in the citation of the original article. We also alter the computerized records to correct names, titles, acknowledged incorrect dosages and other crucial figures, as well as minor misspellings. When a journal issues a statement retracting an article because of fraudulent data, the Library of Medicine appends to the original citation the phrase "retracted in …," followed by a journal reference. The published Cumulated Index Medicus now also carries retractions under the heading Retraction of Publication. Thirty-seven such retractions have been identified since the policy was begun in the summer of 1984. The NLM follows the policy of using as sources only citable errata notices in an indexed citation. This means that the error notices must be clearly labeled and printed on a regularly numbered page in the journal in which the original article appeared. Loose notes, unpublished letters, and telephone calls cannot be used as sources, although we welcome calls and letters from editors with the understanding that the correction will be printed in a forthcoming edition. The ability to correct the online file quickly is a great advantage of electronic information retrieval. MEDLINE, the NLM's primary online database, is updated monthly, both with new references and with any retractions or ordinary errata that have come to our attention. Wrong dosage or other key data in an abstract that could have adverse effects if followed are corrected immediately. In more than one case, NLM indexers have identified a discrepancy between a drug dosage recommendation in an abstract and that carried in the text of an article. In such instances the editors have given permission by telephone to correct the MEDLINE record, and this has been done within hours. Under these circumstances, the users of online services are better served than are users of print publications or CD-ROMs. Our current errata policy, circumspect as it is, would seem to offer users of the NLM's products some protection against misinformation, deliberate or accidental. A more aggressive approach might place us in the unwelcome role of censor. We welcome the advice of scientists on how we can improve our procedures, and the assistance of editors in sending us timely notices of retractions and errata. The NLM's first director, John Shaw Billings, once said "the proportion of what is both new and true is not much greater in medicine than it is in theology." With the help of scientists and editors, perhaps we can improve the ratio. Lindberg is director of the National Library of Medicine, 8600 Rockuille Pike, Bethesda, MD 20894.

Developing Damage Control for Bad Data

Interested in reading more?

Become a Member of