Null Hypothesis Significance Test, p-Value, A Priori Procedure and Gain-Probability Diagram

Abstract

The null hypothesis significance test (NHST) has traditionally served as the default statistical technique used for deciding between two contradictory hypotheses, especially concerning population means. However, this technique, along with its associated p-value and confidence intervals, come with many shortcomings and limitations. This talk explores alternative techniques, namely the a priori procedure (APP) and the gain-probability (G-P) analysis. Before data collection, we will consider two important questions: (1) “How closely does the researchers desire the sample mean to approximate the population mean?” and (2) “With what probability does the researcher aim for the sample mean to fall within the specific distance of the population mean?”. By specifying the precision for (1) and the confidence for (2), we can determine the necessary sample size across various distributional settings. Numerically examples will be provided for illustration.