Rockland Immunochemicals CSO Testifies on Scientific Reproducibility Issues at National Academies Session

Ascoli went on to explain that Rockland and other leading antibody manufacturers and resellers, along with funding agencies and some journals, are embracing change that will make a difference. For instance: (1) better use of standards and controls can be made when collecting experimental data; (2) the ‘knowledge gap” that has resulted from outsourcing antibody technology can be addressed; (3) funding agencies can require precise description of reagents and methods to foster reproducibility; (4) journals can publish all necessary details of reagents and protocols critical for reproducibility; and (5) antibody manufacturers and resellers can adopt immunoassay specific validation guidelines and provide greater transparency when describing antibody production, screening methods and release criteria.

Opinion leaders testifying at this session challenged whether a “crisis” of reproducibility exists, as some have claimed in the literature, especially when considering that the clear majority of research performed results in the collection of high quality data sufficient to support and reproduce scientific claims. For instance, the pace at which new treatments have accelerated from the bench to the clinic has resulted in an explosion of breakthroughs in medicine, and antibodies are central to the discovery, diagnosis and cure of many diseases that just a decade ago were considered to have poor patient outcomes.

The root causes of data irreproducibility identified at this meeting were biases in experimental design, biases in statistical analysis, the resultant inaccuracies of mathematical standards and constants, and the inherent variability that exists when biological reagents are used. Focus on these causative factors is imperative as the stakes are high. One economist speculated that the potential economic impact of data irreproducibility, especially for pre-clinical and clinical applications, ranges from 190 to 380 billion US dollars, cumulatively.

The committee was given a timeline to assess issues pertaining to reproducibility and replication, hear testimony by subject matter experts, deliberate and complete their investigations within approximately 18 months of the start of activities in October 2017. At the conclusion, a written report will be issued to Congress that includes an assessment of current activities to improve reproducibility and replication highlighting examples of good practices and examine factors that adversely affect reproducibility and replication.