How to Use P-Values Correctly

The American Statistical Association offers guidance on best practices for the oft-misused tool.

Kerry Grens
Kerry Grens
Mar 9, 2016

FLICKR, LENDINGMEMOConcerns about widespread misunderstanding and misuse of p-values in science have prompted the American Statistical Association (ASA) to issue its first-ever policy statement about the proper use of the statistical tool. On March 7, the organization released a set of six principles on the power and limitations of the p-value.

For instance, determining policy or making scientific conclusions should not be based on a p-value alone. “Practices that reduce data analysis or scientific inference to mechanical ‘bright-line’ rules (such as ‘p < 0.05’) for justifying scientific claims or conclusions can lead to erroneous beliefs and poor decision-making,” according to the ASA’s statement. “A conclusion does not immediately become ‘true’ on one side of the divide and ‘false’ on the other.” 

Rather, complementing p-values with other statistics, such as confidence intervals, may better address the validity of a hypothesis.

In a commentary on the statement, Stanford University’s John Ioannidis...

The ASA’s statement also points out what is arguably the biggest misconception about p-values. As FiveThirtyEight explained: “A common misconception among nonstatisticians is that p-values can tell you the probability that a result occurred by chance. . . . The p-value only tells you something about the probability of seeing your results given a particular hypothetical explanation—it cannot tell you the probability that the results are true or whether they’re due to random chance.”

Giovanni Parmigiani, a biostatistician at the Dana Farber Cancer Institute, told Nature News that guidance on proper p-value use has been needed. “Surely if this happened twenty years ago, biomedical research could be in a better place now.”FLICKR, LENDINGMEMO