On the use and misuse of p-values

Abstract

This talk consists of two parts. In the first part of the talk, we examine and quantify p-value uncertainty and replication power and show that over-interpretation of significant p-values is an important factor contributing to the unexpectedly high incidence of non-replication in research. In the second part of the talk, we discuss why p-values are just “the tip of the iceberg” when it comes to non-replication of research findings and how common and accepted statistical practices, such as data-dependent analyses, as well as the researcher’s desire to find novel and statistically significant results make false positives vastly more likely than the nominal 5% significance level (i.e. a reported p-value < 0.05). This is a joint work with Drs. Lazzeroni and Belitskaya-Levy.