On a generalizable approach for sample size determination in Bayesian t tests.

Abstract

The Bayes factor is often proposed as a superior replacement to p values in testing null hypotheses for various reasons, with the availability of many user-friendly and easily accessible statistical software tools facilitating the use of Bayesian tests. Meanwhile, Bayes factor design analysis (BFDA), the counterpart of power analysis, is also proposed to ensure the maximum efficiency and informativeness of a study. Despite tools for conducting BFDA being limited and mostly relying on Monte Carlo methodology, methods based on root-finding algorithms have been recently developed (e.g., Pawel and Held, 2025), overcoming weaknesses of simulation approaches. This paper builds on these advancements by presenting a method generalizing the existing approach for conducting BFDA for sample size determination in t tests. The major advantage of the current method is that it does not assume normality of the effect size estimate, allowing more flexibility in the specification of the design and analysis priors. We developed and showcase a user-friendly Shiny app for facilitating the use of BFDA, illustrated with an empirical example. Furthermore, using our method, we explore the operating characteristics of the Bayes factors using various priors.