The insignificance of the significant pvalue

filedrawer1

p<0.05

We’re told early and often that this means that your data is significant. But statisticians and biologists that are statistically inclined, while tell you (and have been telling us for awhile) that this is a completely arbitrary figure. Like most tools, in statistics if you use the pvalue incorrectly you’re doing yourself and your science a disservice.

And the American Statistics Association agreed, and disagreed. Last week they released an AWESOME statement on p-values.

Read the original, or the equally excellent synopsis over at the Molecular Ecologist (I can’t give that blog enough love…)

xkcd comics

xkcd comics

Advertisements

3 comments on “The insignificance of the significant pvalue

  1. Hmm. I’m glad you like the ASA statement (which was, indeed awesome). But that statement emphatically did NOT say that a significant p-value was “insignificant”, per your post’s title! And no matter how awesome xkcd is (and it’s pretty awesome), that particular cartoon actually makes fun of something (continuous interpretation of p-values) that the ASA endorses (and about which I posted here: http://wp.me/p5x2kS-cR).

    So – I’m glad to see your post and referring people to the excellent ASA statement is a Good Thing. But I wonder if you could expand a bit on what argument you’re making here? Because I don’t think it’s actually what a reader might think from the title or the graphics… But correct me if I’m wrong!

    • cej9f says:

      Sorry about the title. I’m in the middle of writing my dissertation and my ability to come up with clever titles is currently limited as my brain power is being expended elsewhere.

      I like your post! In reference to it (and to my general argument) is that I’m a continualist, and I think the value of a p value is that it tells you something important about your data, but is not a line in the sand by which results should be significant or not. There are too many other factors that should be considered, and I think that misusing analyses, or not clearly understanding what a pvalue is telling you is the reason some significant results are not significant or vice versa.

      If I gave the wrong impression from the title or graphics I do apologize. I once had one of my undergraduates berated at a conference because he said p=0.052 was slightly significant. I think that demonstrates a lack of understanding of the significance of a pvalue. May just be me…

      • Thanks for replying – I think we agree entirely, as you explain it! Your undergraduate ran afoul of someone who *thinks* they understand statistics, but actually doesn’t… they are quite common. One can be an absolutist for good reasons, mind you; but absolutists who really understand why they are so are hugely outnumbered by absolutists who are parroting what they’ve been told! And now I bet I’ve ticked off many of your readers :-)
        By the way, don’t worry, my post titles get criticized too… Good luck with the dissertation!

Comments are closed.