In "On Downsizing the Status of Science," Dave Britton writes "... Truth is only what scientists can get each other to agree is true. And even then, on average one out of a hundred statistically significant findings of p < 0.01 is really just a coincidence."1
Britton misinterprets the long-term average type 1 error rate (alpha, erroneously rejecting a true null hypothesis) for the frequency of statistical errors that occur in the published literature. On average, 1 of every 100 "significant findings" will be "just coincidence" only if one always tests only true null hypotheses with alpha = 0.01 established a priori. But in "real life," many of those results for which P < 0.01 will have occurred because the null hypothesis is false.
So, a result of "P < 0.01" that leads to a rejection of the null hypothesis may be a type 1 error, but it also may ...
