FLICKR, LENDINGMEMOConcerns about widespread misunderstanding and misuse of p-values in science have prompted the American Statistical Association (ASA) to issue its first-ever policy statement about the proper use of the statistical tool. On March 7, the organization released a set of six principles on the power and limitations of the p-value.

For instance, determining policy or making scientific conclusions should not be based on a p-value alone. “Practices that reduce data analysis or scientific inference to mechanical ‘bright-line’ rules (such as ‘p < 0.05’) for justifying scientific claims or conclusions can lead to erroneous beliefs and poor decision-making,” according to the ASA’s statement. “A conclusion does not immediately become ‘true’ on one side of the divide and ‘false’ on the other.”

Rather, complementing p-values with other statistics, such as confidence intervals, may better address the validity of a hypothesis.

In a commentary on the statement, Stanford University’s John Ioannidis...

The ASA’s statement also points out what is arguably the biggest misconception about p-values. As FiveThirtyEight explained: “A common misconception among nonstatisticians is that p-values can tell you the probability that a result occurred by chance. . . . The p-value only tells you something about the probability of seeing your results given a particular hypothetical explanation—it cannot tell you the probability that the results are true or whether they’re due to random chance.”

Giovanni Parmigiani, a biostatistician at the Dana Farber Cancer Institute, told Nature News that guidance on proper p-value use has been needed. “Surely if this happened twenty years ago, biomedical research could be in a better place now.”FLICKR, LENDINGMEMO

Interested in reading more?

The Scientist ARCHIVES

Become a Member of

Receive full access to more than 35 years of archives, as well as TS Digest, digital editions of The Scientist, feature stories, and much more!