Advertisement

Bring On the Transparency Index

Grading journals on how well they share information with readers will help deliver accountability to an industry that often lacks it.

By | August 1, 2012

image: Bring On the Transparency Index Flickr, moonlightbulb

Scientists are universally familiar with the Impact Factor, even if they’re often frustrated with how it can be manipulated and misused. More recently, Ferric Fang and Arturo Casadevall have introduced the idea of the Retraction Index, a measure of how many papers journals retract for every 1,000 they publish. As science journalists who have spent the last 2 years closely monitoring retractions, we think this is a great idea.

Last year, in a post on our blog Retraction Watch, we recommended that journals publicize their Retraction Indices just as they trumpet their Impact Factors. It’s unlikely many will take us up on the suggestion, but we’ll go once more into the breach anyway and suggest another metric of journal performance: the Transparency Index.

Regardless of what metric scientists use to rank journals, one of the reasons they read the top-ranked journals is their sense that the information is reliable. We believe, and we’re not alone here, that journals become more trustworthy when they are open about not only their successes, but also their failures.

We understand—in theory, at least—why some journals and editors might be reluctant to share the details of a retraction with their readers. Sometimes the problems involve shoddy reviews, failure to check a manuscript for evidence of plagiarism or duplicate publication, or other avoidable mistakes.

But lack of transparency serves only to reinforce a sense of incompetence. Journals and editors willing to pull aside the curtain to show readers what went wrong with a particular article or group of articles send the messages that 1) they care about conveying truth to their audiences; 2) they are committed to producing a high-quality publication; and 3) potential fraudsters are not welcome in their pages.

Our hope is to turn the above criteria into a numerical metric that can give authors and readers a sense of a journal’s transparency. How much can they trust what’s in its pages? Help us refine the Transparency Index at retractionwatch.wordpress.com/transparencyindex. The number, however, will just be an indicator. Scientists’ judgment will still be the most important factor.

 

Add a Comment

Avatar of: You

You

Processing...
Processing...

Sign In with your LabX Media Group Passport to leave a comment

Not a member? Register Now!

LabX Media Group Passport Logo

Comments

Avatar of: YouKnowBestOfAll

YouKnowBestOfAll

Posts: 16

August 17, 2012

Transparency Index (TI) is a great idea which should promote and encourage best practice, while exposing publication malpractice. In order to be useful TI should reflect predominantly one thing which is:
“Did the journal (editor and publisher) do the right thing?â€쳌

Retraction can happen due to variety of different reasons.
Many retractions might indicate poor reviewers’ and/or editor’s practice. However, in other cases a retraction might indicate the opposite – that the editor did the right thing after discovering irregularities about which s/he was unaware at the time of publication.

So, TI based only on No. of retractions from the total No. of publications will blur the real picture.

On the other hand, a journal (editor and publisher) might have brilliant policies/frameworks/procedures to deal with misconduct, which, however, are never (or seldom) used in practice. Such duplicity would indicate malpractice which intends to cover up previous malpractice. Thus such state of affairs would be even more dangerous as it intends to further deceive the readers.

Therefore, I think that TI should have as one of its main components:
Do-the-Right-Thing (DRT), which can be measured by:
No. of retractions from No. of cases (with evidence about misconduct) brought to the attention of the journal.

In ideal world DRT should be 1.
Example 1:
8 cases about irregularities, all backed up with evidence were reported to the attention of the journal, and the journal retracted all 8 articles (8/8=1).

Example 2:
8 cases about irregularities, all backed up with evidence were reported to the attention of the journal, and the journal retracted only 2 articles (2/8=1/4=0.25).

Provided that both journals have in place nice Frameworks to deal with misconduct, the later example would present the journal in a better light, while in reality the second journal is much worse, since it refuses (for whatever reason) to do the right thing.

Avatar of: FJScientist

FJScientist

Posts: 52

August 22, 2012

We absolutely need retractions to clear the record and must avoid doing anything to inhibit retractions. For example, it can be difficult to publish a result different than what has been previously published. Not only must you be ultra-sure of your result (which should be true for any publication), you then need to establish why it is different than the precedent result.
There are many retractions that are a result of sloppy science and/or statistical anomalies. One can argue that the report should never have been published. But it was. And the authors and journals are doing the right thing by retracting it. One wants to devise a retraction index that does not hinder the authors and/or journals from publishing those retractions.
Then there are the 'forced' retractions resulting from an ethical irregularity. Unfortunately, those retractions rely to a certain extent not just on the honesty of the original authors and the journals, but on the authors' home institution as well. If a typical journal receives a report of an irregularity, the procedure for most is to bring this report to the authors' attention. If not answer to the journal's satisfaction, most journals do not have the resources to conduct an indepth investigation. The original report and the authors' response instead will be forwarded by the editor to the institution. There are large disparities in the extent to which those institutions follow-up. Perhaps the 'transparency index' most appropriately should be targeted towards the institution's responses to enquiries???
Take home message. Make sure your index doesn't hinder what is necessary and good. For targeting that which is genuinely bad, make sure you target all of the right contributors if your goal is to improve behavior.

Avatar of: Guest

Anonymous

August 25, 2012

YES! Totally agree that
“the 'transparency index' most appropriately should be targeted towards the institution's responses to enquiriesâ€쳌

See my comment from 19 August here http://retractionwatch.wordpre...

TI can be very useful for purifying the Academia and reducing the misuse of scares public resources (obtained by deception based on publication irregularities).

This will be beneficial for everybody except the fraudsters.

Avatar of: Guest

Anonymous

August 25, 2012

YES! Totally agree with FJScientist that
“the 'transparency index' most appropriately should be targeted towards the institution's responses to enquiriesâ€쳌

See my comment here http://retractionwatch.wordpre...

TI can be very useful for purifying the Academia and reducing the misuse of scares public resources (obtained by deception based on publication irregularities).

This will be beneficial for everybody except the fraudsters.

Avatar of: Guest

Anonymous

August 25, 2012

YES! Totally agree that
“the 'transparency index' most appropriately should be targeted towards the institution's responses to enquiriesâ€쳌

See my comment here http://retractionwatch.wordpre...

TI can be very useful for purifying the Academia and reducing the misuse of scares public resources (obtained by deception based on publication irregularities).

This will be beneficial for everybody except the fraudsters.

Avatar of: YouKnowBestOfAll

YouKnowBestOfAll

Posts: 16

August 26, 2012

Transparency Index for institutions has been suggested by me at Retraction Watch on August 19, 2012 at 11:02 PM EDT.

See my comment here http://retractionwatch.wordpre...
"YouKnowBestOfAll, August 19, 2012 at 11:02 pm"

Transparency Index can be very useful for purifying the Academia and reducing the misuse of scares public resources (obtained by deception based on publication irregularities).

This will be beneficial for everybody except the fraudsters.

Follow The Scientist

icon-facebook icon-linkedin icon-twitter icon-vimeo icon-youtube
Advertisement

Stay Connected with The Scientist

  • icon-facebook The Scientist Magazine
  • icon-facebook The Scientist Careers
  • icon-facebook Neuroscience Research Techniques
  • icon-facebook Genetic Research Techniques
  • icon-facebook Cell Culture Techniques
  • icon-facebook Microbiology and Immunology
  • icon-facebook Cancer Research and Technology
  • icon-facebook Stem Cell and Regenerative Science
Advertisement
BioTek
BioTek
Advertisement
The Scientist
The Scientist