Science communication under scrutiny

The Royal Society is to launch a wide-ranging consultation among scientists, the media, and the public next month, into the best way to communicate the results of original research. In the spotlight will be an issue central to the practice of science—the peer review process.Under the chairmanship of Patrick Bateson, the society's biological secretary, a working group will produce guidance on best practice, to be published sometime in the fall. It will be sent to anyone receiving funding from the Royal Society and to the fellows, and it will be disseminated to the wider scientific community both within and outside industry. A separate brief is to be produced for the public.The reports will identify ways in which peer review can be improved to increase public confidence in research. They will also consider alternatives to peer review for assessing the quality of research results released to the public.In an interview with The Scientist, Bateson said there is mistrust of science as evidenced, for example, by some responses to the Royal Society's work on genetically modified organisms (GMOs). Given the current centrality of peer review to scientific claims and the importance of claims from science in such controversial areas as GMOs and the measles, mumps, and rubella vaccine, the working party will examine this process closely.Some of the better known and more widely discussed concerns about peer review are: how journals and grant-giving bodies select reviewers; whether reviewers should remain anonymous; whether reviewers ever hold up the publication of their rivals' work or purloin data; whether papers submitted by big names in their field are as carefully scrutinized as is the work of lesser known researchers. "Much of this is paranoid," said Bateson, "but not all. The issues need to be examined openly. "Some have even said the system of peer review is so flawed, why not simply do away with it," he added. Yet alternative methods of ensuring the quality of research findings also have drawbacks. An example is preprint publication, in which unpublished findings are openly subjected to the wider criticism of peers. This currently happens in some fields of physics, in artificial intelligence, and in larger, specialized institutions.In branches of the biomedical sciences, however, such an approach could be counterproductive. For example, simply making findings public could lead readers to falsely conclude that a cure for a particular disease is imminent when it is not.How well such a system of unregulated peer review can work is also dependent on the culture in a discipline. Larger, more competitive fields can be brutal, said Bateson, his apparent implication being that the results of such an approach would not be pretty to watch.It is not only the practice of peer review, however, that will come under scrutiny. Bateson himself is interested in ways in which personal prejudice—often unconscious—can influence data selection and interpretation, often at an early stage. This is an important consideration if scientists release findings early and before the evidence is weighed. "Sydney Brenner," recounted Bateson, "has a slide showing a straight line with a large black mark on it. The black mark is a thumbprint." It transpires that under the thumbprint are the less decorously arranged data points. "These outliers," said Bateson, "tell you exactly what you need to know, and this is something about which we also need to educate scientists."For the public, his concern is that it is not easy for them to assess uncertainty. "If a journalist asks you whether GMOs are safe and you say 'I don't know,' then it is reported as, 'scientists do not know whether GMOs are safe.' But this implies something more certain than is meant and that there is nothing useful to be said about GMOs when there is."The working group assessing all these issues comprises academic and nonacademic scientists, scientific publishers, and the representatives of the media.

Science communication under scrutiny

Peer review process to be examined in Royal Society consultation

Interested in reading more?

Become a Member of