Complaints About Government Contest

Contestants criticize the organization and scoring of a Pentagon competition challenging scientists to detect bioterrorism threats by analyzing DNA sequences.

By | July 24, 2013

FLICKR, SHAURYParticipants in a Department of Defense competition aimed at improving detection of bioterrorism threats using DNA sequencing analysis say that the contest was poorly organized and scored, ScienceInsider reported.

Researchers were offered a $1 million prize for identifying the genetic code of individual species within masses of raw DNA sequence data, with the ultimate goal of creating better tools for detecting dangerous organisms. In the end, 103 groups or individuals submitted work. Of those, three teams have made the final round. However, participants complain that the rules of the contest and its scoring encouraged researchers to manipulate programs to fit opaque scoring rubrics.

Participants were scored on a 0-to-100 scale and were allowed to submit their programs multiple times to check their scores. “We spent the last month trying to get the [scoring] algorithm to tell us that we’ve done well, instead of actually doing the proper science to produce a good result,” David Ainsworth, a bioinformatics graduate student at Imperial College London a member of one of the finalist teams, told ScienceInsider.

Another participant, Derrick Wood of the University of Maryland, College Park, who did not make the final round, complained that instructions were unclear. He was penalized for failing to mention the strains of some species he identified, while for other species, naming the strain did not appear to garner him points.

The Department of Defense pushed back the deadline multiple time and changed contest rules because no contestants had yet qualified to move on to the next round. Then, in the final week of the contest, participants were encouraged to start collaborating and mixing and matching their algorithms, leading to a mad rush for alliances.

“In terms of the goal of the contest as I understood it, I think it’s unlikely that the best algorithm will win,” Steven Salzberg of Johns Hopkins University, whose lab members participated in the contest, told ScienceInsider.

Add a Comment

Avatar of: You



Sign In with your LabX Media Group Passport to leave a comment

Not a member? Register Now!

LabX Media Group Passport Logo

Popular Now

  1. How Gaining and Losing Weight Affects the Body
    Daily News How Gaining and Losing Weight Affects the Body

    Millions of measurements from 23 people who consumed extra calories every day for a month reveal changes in proteins, metabolites, and gut microbiota that accompany shifts in body mass.

  2. That Other CRISPR Patent Dispute
    Daily News That Other CRISPR Patent Dispute

    The Broad Institute and Rockefeller University disagree over which scientists should be named as inventors on certain patents involving the gene-editing technology.

  3. Neurons Use Virus-Like Proteins to Transmit Information
  4. EPO Revokes Broad’s CRISPR Patent
    The Nutshell EPO Revokes Broad’s CRISPR Patent

    Shortly after ruling out the earliest priority dates on a foundational patent for CRISPR gene-editing technology, the European Patent Office rescinded the patent entirely—and more are likely to follow.