FLICKR, LENDINGMEMOConcerns about widespread misunderstanding and misuse of p-values in science have prompted the American Statistical Association (ASA) to issue its first-ever policy statement about the proper use of the statistical tool. On March 7, the organization released a set of six principles on the power and limitations of the p-value.
For instance, determining policy or making scientific conclusions should not be based on a p-value alone. “Practices that reduce data analysis or scientific inference to mechanical ‘bright-line’ rules (such as ‘p < 0.05’) for justifying scientific claims or conclusions can lead to erroneous beliefs and poor decision-making,” according to the ASA’s statement. “A conclusion does not immediately become ‘true’ on one side of the divide and ‘false’ on the other.”
Rather, complementing p-values with other statistics, such as confidence intervals, may better address the validity of a hypothesis.
In a commentary on the statement, Stanford University’s John Ioannidis wrote that adding more statistical layers does not solve the problems of “hidden multiplicity and selective reporting biases.” Transparency—another of the ASA’s principles—is essential. “Efforts to promote ...
