DATA: Past Due

Ned Shaw Open your file drawer all the way, and force your fingers to pry through the folders wedged in the back. Or take down that black binder from a decade ago, labeled with the name of a student you can no longer picture. Perhaps you saved that dataset for the sparkling nugget of an unexpected finding, hoping to determine later whether it might be fool's gold or the real thing. But, in relinquishing it to the vault, it's become like mystery meat, wrapped tightly in aluminum foil inside a Ziploc bag and engulfed in the growing wall of ice in the back of your freezer. Data get old. While not subject to freezer burn, data lose their relevance as a field advances. You may have a valid reason, however, for keeping unpublished data beyond its expiration date. That reason could be the new National Institutes of Health requirement that funded scientists archive and share their research. Why do scientists leave data languishing in file drawers rather than submitting them for publication? Some blame funding pressures. "You tend to write up data for grants that need to be renewed, because you have to show progress," says Linda Porrino, professor of physiology and pharmacology at Wake Forest University, Winston-Salem, NC. "So you put the other stuff on the back burner." Chris-Ellyn Johanson, professor of psychiatry at Wayne State University, Detroit, says she may not write something up because she's "very ambivalent about the robustness of the finding." Only people who are advanced in their careers can afford to sit on equivocal data these days, however. "One of the sad things is the pressure to publish no matter what." NEW DATA SHARING POLICY Starting in October 2003 investigators submitting an NIH application must include a plan for sharing data. Publishing data satisfies the data sharing commitment, but unpublished data must be shared as well. Alternate methods include archiving data and providing access, perhaps via a Web site. Examples of data sharing plans include protein and DNA sequences made available to researchers through data archives, such as GenBank. Candidates for data sharing include final research data of all kinds. The NIH defines final research data as "recorded factual material commonly accepted in the scientific community as necessary to validate research findings." This does not include laboratory notebooks, partial datasets, preliminary analyses, drafts of scientific papers, plans for future research, peer review reports, communications with colleagues, or physical objects such as gels or laboratory specimens. Grants for archiving are available. Strike a balance between the value of final research data and the costs associated with archiving. The NIH policy acknowledges that data of limited usefulness may not be worth the expense and effort of archiving them. NIH Data Sharing Policy grants.nih.gov/grants/policy/data_sharing FAQs on Data Sharing grants.nih.gov/grants/policy/data_sharing/data_sharing_faqs.htm Final NIH Statement on Sharing Research Data grants.nih.gov/grants/guide/notice-files/NOT-OD-03-032.html Relying on students and postdoctoral fellows to write up studies often results in data pileups. "Sometimes they drop the ball," says Bob Balster, professor of pharmacology and toxicology at the Medical College of Virginia in Richmond. "Typically what happens is students leave." SAVE IT OR THROW IT OUT? Balster recently had to move his office and sort through his files, including old data. The primary investigators (students and postdocs) never got around to writing up the carefully stored data. Balster threw them away. "Once I realized the world had gotten along without these data, I began to feel less concerned about it, he says. If this sounds radical, it's because most scientists are trained to save data. "You don't throw out data," says Eric Simon, professor of psychiatry and pharmacology at New York University School of Medicine. Johanson says she feels obliged to keep data for seven years. "I know it sounds like the income tax law," she says. "If it's good enough for the IRS [Internal Revenue Service], it better be good enough for NIH." The National Institutes of Health recently published a policy statement on data sharing (see sidebar), which says that research funded by the federal government should be accessible to the public. Investigators applying for funding must now include a plan for sharing data, even unpublished data, by archiving them and making them accessible. Sometimes old data retain their usefulness. Porrino images monkey brains and catalogs the data. Her initial hypotheses may focus on one or two brain areas, yet her images contain data from a whole slice. Other portions can be stored to answer new questions. Pack rats are driven by the belief that saved items might come in handy someday. It is no different with data; you might publish them or build on them. "You can't help wondering if you might get back to it, but the probability is--in fact I know the reality is--that I won't," Cox acknowledges. Most researchers have a story about revisiting old data in a constructive way. "We have huge numbers of notebooks sitting on top shelves," says Simon, who sometimes pulls down old notebooks to check if they have done a particular experiment before. With a laugh, he says they occasionally find it. If she is still excited about old data, Johanson says it can spur a new study "either by doing it differently or doing it better." Old data can become "pertinent to a question now" and written up, Porrino says, "but that doesn't happen so often." This caveat is frequently the coda to such stories: It doesn't happen very often. On the other hand, the costs of saving data are tangible and need to be weighed against admittedly limited benefits. Time is a key factor. "When you have several things going on, it doesn't leave much time for going back," says Simon. The only way it happens "is if you are starved for papers." Space is also an issue: boxes in the basement, notebooks lining shelves, file cabinets blocking corridors. Electronic storage can reduce space problems, but still, some scientists worry about permanence. Many researchers have experienced the irretrievable crashing of a hard drive. And, Cox mentions the constant changes in preferred format--3.5-inch floppies, compact discs, zip disks--as a reason that he still keeps a paper copy. But what about the mental energy expended on old data? "I do think about it. And I have a list of papers that I'm supposed to write up next to my computer," Porrino says. "So yes, I do take my dose of guilt everyday." ENVISION EMPTY SHELVES How can researchers prevent old data from accumulating? Organization and a vision of the end product are the keys. "Efficient experimental design and efficient data management," says Porrino, who also will sometimes stop doing experiments (and collecting more data) to write. "It's much harder to write than to collect," she says. Balster agrees. "When designing experiments, I try to think about the way it's going to look in a publication and to imagine what weaknesses it might have from the point of view of a reviewer," he says. Publishing your data is the most straightforward way to satisfy the new data sharing requirement for NIH-funded research. "One of the things I keep saying is the old adage: It doesn't exist until it's published," Johanson says. Think about the life cycle of data: Collect, analyze, put into context, draw conclusions, and write. If you allow a dataset to fall off the track, it risks passing its expiration date. So defrost your freezer every once in awhile and unwrap those foiled packages. Is there a viable "what if"? Or is it time for the disposal? Jill U. Adams (jilluadams@aol.com) is a freelance writer in Delmar, NY. function sendData() { document.frm.pathName.value = location.pathname; result = false if (document.frm.score[0].checked) result = true; if (document.frm.score[1].checked) result = true; if (document.frm.score[2].checked) result = true; if (document.frm.score[3].checked) result = true; if (document.frm.score[4].checked) result = true; if (!result) alert("Please select") return result; } .myradio{background-color : #94bee5} Please indicate on a 1 - 5 scale how strongly you would recommend this article to your colleagues? Not recommended 1 2 3 4 5 Highly recommended Please register your vote

DATA: Past Due

Interested in reading more?

Become a Member of