The Scientist

Last year's discredited Science article on cloned stem cell lines presented now-obvious signs of fraud, such as claims that images of the same cells came from different patients -- raising many questions about what journals can do to find fraud before it's published. Although editors maintain that no practical procedures will find all instances of scientific fraud, many journals are nevertheless investigating ways to screen submissions for signs of misconduct. There is little doubt who will win this "arms race," said computer scientist Hany Farid, of Dartmouth College in Hanover, New Hampshire, who has helped journals detect image tampering. "It's much easier to manipulate technology than to detect it." Still, with appropriate screening, "we can take it out of the hands of the novice," he told The Scientist. One practical step journals are taking involves looking for modification of individual images. The Journal of Cell Biology, for one, has pioneered the use of simple, routine checks since September 2002. "We check every image of every accepted manuscript for signs of manipulation," said managing editor Mike Rossner - a step that has uncovered some alterations that caused editors to withdraw acceptance of papers and in some cases to notify relevant institutions. The Office of Research Integrity of the U.S. Department of Health and Human Services recommends such notification whenever fraud is suspected in a manuscript. Several journals are following JCB's lead. For instance, Nature executive editor Linda Miller and Cell editor Emilie Marcus said the journals are meeting with JCB about checking images. "We've been looking at software that various companies make," Miller told The Scientist. Proceedings of the National Academy of Sciences (PNAS) editor Nick Cozzarelli, meanwhile, said that the journal plans to do "some image screening," but nothing too extensive. "We don't have plans to make a big deal out of it," he said. But it's relatively easy to find tampering in images, since alterations such as changing size or contrast, or masking with pixels from another region, leave telltale fingerprints. In contrast, the diversity of non-image data makes it difficult to devise standardized procedures for checking for fraud, experts say. A possible exception concerns the statistical properties of large data sets such as those used in clinical trials. In the now-discredited Lancet paper headed by Jon Sudbø at Oslo's Norwegian Radium Hospital, nearly one third of the study participants were listed in the underlying database as sharing the same birthday. The data that results from fraud often includes red flags that reviewers can spot. For example, last summer BMJ published a statistical analysis of a study submitted to the journal in 1993 but never published, for which reviewers had questioned anomalies such as the strong effect of dietary intervention in cardiovascular disease. The data revealed other problems, including large differences in the variability between two randomly selected groups. Because no change in the underlying science could explain this difference, the report concluded that "the data from the ? trial were either fabricated or falsified." The senior author of the analysis, Stephen Evans of the London School of Hygiene and Tropical Medicine, said that journals have to ask for raw data to do such detailed analyses. "You can often find something [suspicious] very quickly," but proving misconduct takes much more effort, he cautioned. Evans noted that routine statistical analyses of raw data are not practical, but journals could consider doing "random checks." Indeed, former BMJ editor Richard Smith who requested the analysis by Evans and his colleagues, said that journals are "not well set up" to do routine analyses, and looking at data is "very expensive, difficult, complicated." Editors at top-tier journals said they typically request raw supporting data only in response to specific criticisms. The journals also require authors to place large, standard data types, such as protein structures or expression data, in public repositories, but they do not generally inspect them. The Lancet, which published Sudbø's fraudulent study, declined to comment for this story. Editor's Note: Is prosecution the answer? See a related story. Don Monroe freelance@donmonroe.info Links within this article I. Oransky, "All Hwang human cloning work fraudulent," The Scientist, January 10, 2006 http://www.the-scientist.com/news/display/22933/ G. Vogel, "Landmark paper has image problem," Science Now, December 6, 2005. http://sciencenow.sciencemag.org/cgi/content/full/2005/1206/1 Hany Farid http://www.cs.dartmouth.edu/~farid/ Mike Rossner and Kenneth M. Yamada, "What's in a picture? The temptation of image manipulation," Journal of Cell Biology, July 5, 2004 PM_ID: 15240566. Office of Research Integrity, "Managing Allegations of Scientific Misconduct: A Guidance Document for Editros," January, 2000 http://ori.dhhs.gov/documents/masm_2000.pdf S. Pincock, "Lancet study faked," The Scientist, January 16, 2006. http://www.the-scientist.com/news/display/22952/ S. Al-Marzouki, S. Evans, T. Marshall, and I.Roberts, "Are these data real? Statistical methods for the detection of data fabrication in clinical trials," BMJ July 30, 2005. PM_ID: 16052019. Stephen Evans http://www.lshtm.ac.uk/msu/staff/sevans.html A.McCook, "Scientific fraud: Is prosecution the answer?" The Scientist, February 10, 2006. http://www.the-scientist.com/news/display/23105/