Technological And Cultural Impediments Slow Electronic NIH Grant Submission

Computer compatibility and investigator reluctance blamed for the long delay in agency's move into the digital age. The Internal Revenue Service began accepting electronic transmission of federal income tax forms in 1986. This time-saving option has become quite popular, and by 1994 was used by some 14 million taxpayers. Yet at the National Institutes of Health--which funds some of the most scientifically sophisticated researchers in the world--grant proposals are, for the most part, still being submitted and processed on paper. Why is NIH taking so long to move into the electronic age? The answer, report researchers and NIH officials, is, in part, that electronic submission of a complete government grant application is fraught with difficulties. In addition to the technical problems that have been encountered in attempts to produce a paperless grant application submission process--such as incompatibilities among computer systems--there are human factors. For example, some investigators are too busy to want to take time away from research to learn new ways to submit proposals and some--especially older researchers--are still not very comfortable with computers. The NIH instructions and forms (PHS 398) for grant applications are generally revised every three years. The forms now in use are dated 9/91. New forms have recently been approved by the Office of Management and Budget (OMB) for use through Sept. 30, 1997. The new packet will have a May 1995 revision date and should be available to the extramural community by June, according to Barbara Wassell of the NIH Division of Research Grants (DRG) Referral and Review Branch. The PHS 398 is revised by a committee with broad representation from NIH, the Public Health Service (PHS), and the grantee client community. Wassell then shepherds the application packets through proofreading, getting OMB approval, typesetting, and printing. She says there are no major surprises in the new packet and summarizes the changes as follows: The application booklet has been reorganized into three main sections: An expanded foreword with information about how to get the NIH Guide for Grants and Contracts, how to access the NIH GrantLine (an electronic NIH information service), and so forth. The main part of the packet, consisting of three sections: (a) Instructions about how to prepare the application and a sample set of forms. Gender and minority reporting for clinical trials has been added. (b) Information about submission, receipt, review, and award cycles, as well as the peer-review process. (c) Other information, including definitions of terms such as "Collaborator," "Cooperative Agreement," and "AIDS-Related Research," as well as assurances and certifications. Instructions for other types of applications: (a) The former "RCDA" (Research Career Development Award) section has become the "RCA" (Research Career Award) section and consolidates instructions for the RCDA and the K series of grant awards (NIH Physician Scientist Award, Research Career Re-entry Program, and other awards). (b) The last section is for National Research Service Awards (NRSA), the so-called Institutional Training Grants. The space for "Performance Site" has been moved from the face page to the abstract page. This has resulted in a smaller space for the abstract. The personnel entries on the abstract page have been simplified and the definition of "Key Personnel" has been changed. The "Other Support" page has been changed to a "format" page with more detailed explanation and examples. The Checklist Page has reverted to one page. The "boxes" formerly used to check off compliance with various assurances and certifications have been deleted. The signature of the authorized representative of the applicant organization on the face page will henceforth certify compliance with assurances. The Personnel Report section from the PHS 2590 (NIH noncompetitive renewal packet) has been added to the PHS 398 but is required only for competing continuation applications. The new PHS 398 is intended to be used as a reference document and kept for multiple grant submissions. Principal investigators (PIs) can obtain separate forms-only packets from NIH. This will avoid wasting a whole PHS 398 packet every time a PI needs additional forms. Wassell points out that many people are already generating some of the forms on computer via software programs, and "as electronic grants administration becomes a reality, much of the basic information, such as assurances, will be stored and reviewed annually. PIs will then have to deal with only those parts of the application that change from one project period to another, such as the research plan and the budget." --L.R.-L. Some researchers contend that an ongoing NIH project to bring about electronic applications is missing the mark on both counts. About five years ago, the agency tried to develop a DOS-based prototype software program, Automated Grant Application System (AGAS), to transmit a complete grant application. By 1993-94 NIH was testing AGAS at seven sites, including the Harvard-affiliated Beth Israel Hospital (BIH) in Boston. Charles Safran, director of research at the Center for Clinical Computing at BIH, whose laboratory was the first to submit a grant application to NIH electronically, was a special consultant to the NIH Division of Research Grants (DRG) for review of the AGAS software. Safran maintains that problems encountered in this attempt at electronic submission were as much in the approach taken by NIH as in the technology: "NIH, in trying to satisfy the perceived internal needs of the agency, lost control of the needs and problems of those who generate the applications. "For example, AGAS failed to take advantage of existing word- processing programs used by real PIs [principal investigators]; did not consider the best computer interface for very busy people, many of whom already have much of the 'boilerplate' data for grant applications on their computers [which, however, could not easily be imported into the field- oriented forms of the AGAS software]; and did not appreciate the desire of PIs to have their applications visually look good to the reviewer." Testing of a modified system called AGAS-LESS, which addresses transmission of only the administrative data, is now under way. But NIH officials say that total electronic submission and processing of applications is not likely to happen for some time. Scientists' experiences with the AGAS system have been varied. Theodore Bale, research grants coordinator of Beth Israel Hospital's Office of Science and Technology (OST) when the hospital was a test site, found the process "challenging but rewarding." He notes that the "successful completion of a given [computer] application depended on a variety of factors, such as the number of illustrations/figures in the research plan and whether or not information about human subjects was required." Nan Clark, assistant director of OST, found it challenging but frustrating: "There were lots of bugs in the system. For example, the program had an error checker that reported many non-errors. "Also, the system was geared to the expectation that a full proposal would fit on one disk, but only two proposals brought to our office fit on single disks; the rest each had three to 25 diskettes. The disks had to be copied onto the AGAS system before the BIH Grants Office could sign off on the application--and then recopied to disks for mailing. Copying sometimes took hours! Moreover, copying multiple disk files required use of several DOS commands. As the person responsible for this aspect of the process, I ended up writing the commands for 'restore' and 'backup' on the back of my hand with indelible ink. "However, we were, after all, a test site, and the [NIH AGAS help hotline] people were very good and very patient--and NIH was very lenient about our submission deadline." Bale says everyone in his section of the hospital has now reverted to paper applications--with the help of computer forms made by Envisage Inc. (see accompanying story). Safran-- who points out that the AGAS software did not even calculate budgets the way the PIs' own spreadsheets did--says he is happy using the Envisage templates. About three dozen applications were submitted via AGAS from the seven test sites before the trials were discontinued in mid-1994 and NIH/DRG shifted to testing "AGAS-LESS," or "LESS." LESS, which like AGAS is only for DOS-based computers, provides for input of the Face Page, PI Personal Data Sheet, Abstract, Key Personnel, and bottom-line total direct costs for each budget year. Meanwhile, in March 1994, NIH decided to concentrate on development of internal standards and to invite third-party developers to create software for electronic submission of applications (NIH Guide for Grants and Contracts, 23[11]:11, March 18, 1994). Jim Lowrie, chief of the Systems Analysis Section of DRG, reports that most of the responses to the solicitation have been from PIs rather than commercial companies. "NIH hopes that a number of user-friendly interfaces will be developed that are suited to the hardware and software requirements found in the diverse applicant institutions," comments Lowrie. Diana Jaeger, assistant grants policy officer in the Office of Policy for Extramural Research Administration at NIH, reports that "in the past year, NIH has intensified its commitment to electronic research administration. Much of this intensified effort has focused on a coordinated venture among the major research funding agencies [the departments of Defense and Energy, as well as the National Science Foundation and NIH] to develop an Electronic Data Interchange [EDI] standard for submission and receipt of federal grant applications." The EDI standard is awaiting approval from the American National Standards Institute (ANSI), a nongovernment agency, in 1995. Lowrie says if all goes well, the standard will be published in January 1996. But he cautions that "this standard does not yet address the issue of 'rich formatted text,'"-- that is, text containing photos, figures, tables, and equations, such as the research plan in a grant application. "Although such documents may be transmissible, the EDI standard does not specify how 'rich text' documents should be created and viewed." Jaeger points out that "the EDI standards are only for computer-to-computer communication of data. Once the specs are published, commercial firms and institutions will be able to develop form-independent systems for proposal submission with the help of commercially available software and EDI translators. NIH is also developing technological options, such as allowing applicants/grantees to submit grant-related information [electronically, using the Internet or software such as] Mosaic or similar World Wide Web browsers." Jaeger says the AGAS experience has "caused NIH to rethink how it operates electronically internally." She says, "Plans are well under way for electronic exchange of certain grant- related information such as the application shell [the forms, as opposed to'rich text'], trainee appointments, and award notices." Another innovation that NIH "has already developed and is testing," according to Jaeger, "is a system for electronic reporting of inventions and patents. There is also an intense effort by DRG to upgrade its [internal] database," which contains statistical information about all NIH/Public Health Service extramural programs, grant applications, awards, and related items. Lowrie cites additional problems with electronic submission, "such as the need to authenticate and protect the privacy of transmitted content. This requires encryption of some of the transmitted material." Moreover, he maintains, "before the electronic grant application can become a practical reality, NIH needs to establish a sound capability to handle the huge volume of information that will eventually need to be transmitted, stored, and retrieved. Meanwhile, many reviewers are already sending their reports to scientific review administrators [SRAs] on diskettes or via E-mail." STEP BY STEP: NIH's Wendy Baldwin says the agency should approach the issue of electronic submission incrementally. Jaeger predicts that "when all is in place at NIH, the agency will encourage--but probably not mandate--electronic submission of grant applications." Wendy Baldwin, NIH deputy director for extramural research, contends that "NIH should approach electronic submission of grant applications incrementally, by separating the scientific plan from the many other parts of the grant application," as it is now trying to do. She speculates that completely paperless electronic transmission of grant applications is still far off. Baldwin poses the question, "Realistically, how many reviewers do you know who would want to read a hundred applications on a computer screen?" She also notes that "applications need to look good, and at the moment that's hard to achieve with electronic transmission of the so-called rich text." From her perspective, the most beneficial, least costly thing for NIH to concentrate on now is electronic transmission of the administrative data. The LESS program is currently being tested at the last two sites added to the AGAS testing: the University of California, Los Angeles, and the University of Southern California. Pamela Webb, assistant director of the Office of Research Administration at UCLA, was somewhat frustrated by the original AGAS software. She says, "PIs, many of whom already had much of the administrative information in electronic format, had to re-input this information because AGAS required entries into specific fields. SCHOOL OF HARD KNOCKS: UCLA's Pamela Webb calls her institution's test of electronic NIH grant submissions "a valuable learning experience." "Transmission time was also a problem; one of [UCLA's] R01 [investigator-initiated research grant] applications took three and a half hours to transmit via ftp [file transfer protocol]." Webb relates that UCLA submitted 19 applications (including R01s, R29s, RFAs, and Program Projects) using LESS for the October/November 1994 deadlines. She reports, "These limited transmissions were much more successful than the original AGAS transmissions, but the LESS software was new, not derived from AGAS, and, although both were written in FoxPro, they were very different conceptually. "LESS was 'form-based' and you could see the complete form on the screen--a definite plus--but the software was at a very early stage of development. On October 1, it couldn't even print. But by November 1, NIH had fixed the problem." Webb has been working with NIH's Lowrie on testing the LESS software. She says, "It's been a valuable learning experience." Webb was pleased that NIH "bent over backwards to be helpful and to make sure that no PI was at a disadvantage for having participated in the experiment. Nonetheless, a lot more work would need to be done to make the LESS software a reliable tool for widespread use." But Webb guesses that "LESS, too, will be abandoned or significantly changed because the LESS technology will become obsolete before the program can be perfected. Moreover, LESS is likely to merge with a three-year experimental computerization effort at NSF called 'FastLane' [see accompanying story], for which UCLA is also a test site." Webb adds that "any eventually successful program will have to be available for DOS, Windows, and Macintosh clients." Given the hurdles involved, it is not surprising to NIH's Baldwin that IRS beat her agency to electronic submission: "It's easy to transmit a string of numbers [as in a tax return] electronically, but the complexities of the components of an average NIH grant application present a far greater challenge." Liane Reif-Lehrer is president of Erimon Associates, a consulting firm based in the Boston area. She is the author of Grant Application Writer's Handbook (Boston, Jones and Bartlett Publishers, 1995). One painstaking aspect of preparing a grant application is filling out the various administrative forms--a task that is often still done on a typewriter. But computer templates for the PHS 398 application forms and other documents are now available from several sources. Some provide spreadsheet calculation capabilities. The sources listed here furnish software that allows applicants to enter data on-screen and then print it out on a simulated form that is acceptable to NIH. John Livesey, of the department of physiology and pharmacology at the University of the Pacific, provides forms for Windows and Macintosh users. He considers his forms--which allow users to simulate them, fill them out on a computer, then print them out--a stop-gap measure between the typewriter and electronic submission. The forms can be downloaded from the Internet and require the use of both Microsoft Excel and Microsoft Word. Livesey began to develop the forms in 1990. Since spring 1992, when he first offered a complete set of NIH forms, he has received some 2,000 inquiries/requests for the forms, but he suspects that many more people are getting them from derivative sources. For information about getting the forms--and instructions for novices--send E-mail to jlivesey@uop.edu. Others in the research community have created forms for NIH/NRSA fellowship applications and for applications to the National Science Foundation (for the Grant Proposal Guide [GPG]) and the Department of Defense. These are available via ftp (file transfer protocol), and the procedure for accessing these forms is described in the file provided by Livesey. Rice University manages a TRAM server that university administrators developed for the Texas Research Administrators Group. This server provides access to a set of grant application forms from various sources and to information about funding opportunities. To access TRAM, you need an Internet access such as ftp or gopher, or a World Wide Web (WWW) browser. The URL (uniform resource locator) for the server using a WWW browser is: http://tram.rice.edu/TRAM/ Within gopher, access the server tram.rice.edu and enter the directory "TRAM Research Administration gopher" TRAM can also be reached through anonymous ftp to the same host. Address questions about TRAM to sdc@rice.edu Envisage Inc. sells forms software (GrantFormPak S398) for Word Perfect (WP) for DOS and Windows. The forms for any version of WP 5 have been deemed very satisfactory, but there are problems with WP 6.0 that are unlikely to be resolved until the next version of WP is released. For information, contact Envisage Inc., 5111-6 Baymeadows, Suite 315, Jacksonville, Fla. 32217-4860; (904) 739-0060. Fax: (904) 731- 1993. GrantFormPak sells for about $190 for the non- spreadsheet version and $250 for the spreadsheet version. Lisa Swindell, vice president for software sales at Envisage, says, "If the NIH LESS software were successful and adopted NIH-wide, the GrantFormPak product made by our company would probably become obsolete. However, at the rate things have been moving at NIH in past years, we are not worried." Envisage began selling GrantFormPak about four years ago. About 600 universities and hospitals are now using the software. Site licenses are available. NSF is testing several aspects of a project called "FastLane," to help streamline the grants process at NSF. One part of the project involves an NSF forms server that will permit electronic submission of administrative information related to proposals. For information, E-mail to Fastlane@nsf.gov or call (703) 306-1160. --L.R.-L.

Technological And Cultural Impediments Slow Electronic NIH Grant Submission

Interested in reading more?

Become a Member of