A More Meaningful Statistical Inference Using Minimum-Effect Significance Testing
Presented at SPSP 2022 Annual Convention
Psychology relies heavily on statistical significance tests. Yet, they only provide evidence against the possibility of absolutely no true relationship; trivially weak relationships may be considered "significant." Furthermore, they are incapable of distinguishing between results that reflect truly weak relationships from those that lack sufficient data. Reporting effect sizes and confidence intervals has been an insufficient remedy, because the over-reliance on NHST has not shown any sign of decline. In this workshop, we show how these problems can be solved by integrating Minimum-Effect Significance Testing with Equivalence Testing, making it possible to test if a relationship is strong enough to matter, too small to be of consequence, or if more data are required to infer with confidence. The framework we propose is both easy to implement and versatile: MEST and EqT can be conducted by simply calculating confidence intervals and comparing them to an alternative null hypothesis. As a result, our framework can be utilized for any statistic in which confidence intervals can be calculated.
Speakers: Adam Smiley, University of Washington; Jessica Glazier, University of Washington; Yuichi Shoda, University of Washington