Decisions under uncertainty: A statistical framework for evaluating practical relevance in interval-based hypothesis testing.

Abstract

Psychological researchers are increasingly encouraged to move beyond a narrow focus on detecting statistically significant effects that differ from zero. To interpret findings more meaningfully, best practices emphasize evaluating effect magnitude to determine whether it is practically meaningful or negligible. This requires specifying a smallest effect size of interest and conducting interval-based hypothesis tests. Although these approaches improve current practice by explicitly accounting for effect sizes, interval-based hypothesis tests can yield inconclusive results, leaving uncertainty about whether an effect is practically relevant or negligible. In this article, we first explain what interval-based hypothesis testing is and highlight challenges posed by inconclusive outcomes. We then propose a set of complementary tools implemented in an accompanying Web application ( https://paulriesthuis.github.io/SESOI-Decisions/ ) to support more informed decision-making under such uncertainty and help assess the practical relevance of results. These tools aim to help researchers draw more nuanced statistical and practical decisions and improve the interpretation of results obtained from interval-based hypothesis testing.