The Data Analysis Grand Prix

See related Hot Paper, "The Rise of Free, Global Gene Expression Data Sets". The rapid-fire advances in molecular biology, genetics, automation, and microarray analysis are a constant boon to drug discovery and basic biology, but that influx of data is also creating a serious quandary: How does one analyze it all? There is no shortage of approaches. As data piles up, computer scientists and statisticians step in to develop new methodologies. The problem is, there are too many options: "Biologists all of a sudden got totally lost in the choices of different data analysis methods," says Simon Lin, manager of Duke University's Bioinformatics Shared Resource in Durham, NC. To combat that problem, Lin helped found an annual competition called Critical Assessment of Microarray Data Analysis (CAMDA; www.camda.duke.edu). The idea is to present statisticians with published data sets and turn them loose to analyze the data in any way they choose, asking any biological question they choose. The Challenge In 2001, the competition featured a choice of two data sets, one of which was produced by groups headed by John Weinstein, a senior investigator at the National Cancer Institute (NCI), and Patrick Brown, biochemistry professor and Howard Hughes Medical Institute investigator at Stanford University. The data features gene expression profiles of 60 cancer cell lines that have been subjected to over 70,000 drug candidates since 1990 as part of the NCI's cancer screening program. The CAMDA organizing team chose the papers because the data sets are large enough to be a computational challenge, and because "we were looking for some important biology behind it. This is not a pure computational game here—we are looking forward to getting some useful biological insight by re-analyzing the data," says Lin. Judging the competition is tricky, Lin admits, because the techniques used vary widely and there's no "right answer" to shoot for. The judging has two components—an audience vote and a panel of judges. The criteria, he says, are innovation and the biological relevance of the method to the question being asked. And the Winner Is... Courtesy of Kevin CoombesKevin Coombes The winner of the 2001 competition was Kevin Coombes, associate professor and head of the bioinformatics section at the MD Anderson Cancer Center in Houston, Texas, for his entry entitled, "Biology-Driven Clustering of Microarray Data: Applications to the NCI60 Data Set." Coombes and his team asked two basic questions: Could they find a set of genes that were already known to be related and that correctly clustered the experimental samples, and could they use their data analysis methods to tell if some kinds of cancer are more heterogeneous than others? The answers to both these questions, says Coombes, turned out to be yes. "We really did want to try to use the biology some place early in the analysis. A lot of techniques just manipulate the numbers without using this biological information." In fact, the statistical methods the team used were fairly basic, noted Coombes. Because cancer is often associated with chromosomal abnormalities, one of the team's first steps was to examine the microarray data based on which chromosome each gene resides on. They performed separate analyses for each chromosome—just the genes on chromosome one, then two, and so on—and asked how well the genes on each chromosome distinguish the cancers under investigation. "One of the interesting results that came out was that there was a big difference," says Coombes. "Some chromosomes could easily distinguish the kinds of cancer, while others clusters were no better at predicting than random [associations]." The team also grouped genes by biological processes, first finding the updated annotations of the arrayed genes in GenBank, UniGene, LocusLink, and other databases. With the list of known functions in hand, they began to look at the data in these smaller groupings. One of the more interesting biological functions turned out to be apoptosis. "The surprising result was that we couldn't classify any cancers reliably just by looking at apoptosis genes," says Coombes. He suggests two possible interpretations. Either all cancers have similar malfunctions in the apoptosis pathway, or each cancer adopts a unique strategy to avoid apoptosis. Yet both interpretations will affect pharmaceutical companies that are investigating drugs to target the programmed cell death pathways. "It may make it harder to run clinical trials because... you're going to need an additional test that asks, 'Does this person express the particular gene in the pathway that this treatment is going after?' You may have to screen across different kinds of cancers." Those kinds of insights are what have biologists drooling over microarray data, but they may remain elusive until effective analytical techniques become widespread. It's too early to recommend specific methods, says Lin, but he hopes that CAMDA and similar competitions can hasten the adoption of standards. Coombes is optimistic: "Having a chance to put these methods out there and try them on a collection of data sets... I think that will eventually lead to a consensus on which methods are appropriate for arrays and which ones aren't." Jim Kling (jkling67@nasw.org) is a freelance writer in Washington, DC.

The Data Analysis Grand Prix

Interested in reading more?

Become a Member of