How to Use P-Values Correctly

The American Statistical Association offers guidance on best practices for the oft-misused tool.

By | March 9, 2016

FLICKR, LENDINGMEMOConcerns about widespread misunderstanding and misuse of p-values in science have prompted the American Statistical Association (ASA) to issue its first-ever policy statement about the proper use of the statistical tool. On March 7, the organization released a set of six principles on the power and limitations of the p-value.

For instance, determining policy or making scientific conclusions should not be based on a p-value alone. “Practices that reduce data analysis or scientific inference to mechanical ‘bright-line’ rules (such as ‘p < 0.05’) for justifying scientific claims or conclusions can lead to erroneous beliefs and poor decision-making,” according to the ASA’s statement. “A conclusion does not immediately become ‘true’ on one side of the divide and ‘false’ on the other.”

Rather, complementing p-values with other statistics, such as confidence intervals, may better address the validity of a hypothesis.

In a commentary on the statement, Stanford University’s John Ioannidis wrote that adding more statistical layers does not solve the problems of “hidden multiplicity and selective reporting biases.” Transparency—another of the ASA’s principles—is essential. “Efforts to promote transparency in study design, conduct and reporting may have more to offer in this setting than blaming p-values,” Ioannidis wrote.

The ASA’s statement also points out what is arguably the biggest misconception about p-values. As FiveThirtyEight explained: “A common misconception among nonstatisticians is that p-values can tell you the probability that a result occurred by chance. . . . The p-value only tells you something about the probability of seeing your results given a particular hypothetical explanation—it cannot tell you the probability that the results are true or whether they’re due to random chance.”

Giovanni Parmigiani, a biostatistician at the Dana Farber Cancer Institute, told Nature News that guidance on proper p-value use has been needed. “Surely if this happened twenty years ago, biomedical research could be in a better place now.”FLICKR, LENDINGMEMO

You

Processing...

Not a member? Register Now!

JonRichfield

Posts: 125

March 10, 2016

I shall be linking to this item on my Facebook page, as well as the link to the policy statement. And I shall link to them from a few other pages frequented by research workers.

I am no statistician myself, but have long been of the opinion that applied statistics is a GIGO discipline; garbage in, garbage out. And the trickiest garbage comes in the form of the question asked; if you don't ask the right question, the precision and and labour of your statistical work are irrelevant; you could do better to waste your resources partying on the beach than trying to design a programme of research.

Such incompetence is a major cause of wasted research; in science there most decidedly are such things as stupid questions --  and there are many times more dangerously stupid questions than constructively meaningful questions -- and to put statistical tools in the hands of a naive worker who cannot recognise the implications of a stupid question is disastrous. No matter how high the incompetent's classroom and project marks, that would be like leaving an untrained teenager behind the controls of a high-performance jet. He might fly it all right, but where and how expensively he lands, would be in the lap of happenstance.

So: more strength to the American Statistical Association, say I.

Popular Now

1. Features Your Body Is Teeming with Weed Receptors

And the same endocannabinoid system that translates marijuana's buzz-inducing compounds into a high plays crucial roles in health and disease outside the brain.