Consistency of Bayes factor estimates in Bayesian analysis of variance.

Abstract

Factorial designs lend themselves to a variety of analyses with Bayesian methodology. The de facto standard is Bayesian analysis of variance (ANOVA) with Monte Carlo integration. Alternative, and readily available methods, are Bayesian ANOVA with Laplace approximation as well as Bayesian t tests for individual effects. This simulation study compared the three approaches regarding ordinal and metric agreement of the resulting Bayes factors for a 2 × 2 mixed design. Simulation results indicate remarkable disagreement of the three methods in certain cases, particularly when effect sizes are small and studies include small sample sizes. Findings further replicate and extend previous observations of substantial variability of ANOVAs with Monte Carlo integration across different runs of one and the same analysis. These observations showcase important limitations of current implementations of Bayesian ANOVA. Researchers should be mindful of these limitations when interpreting corresponding analyses, ideally applying multiple approaches to establish converging results. (PsycInfo Database Record (c) 2024 APA, all rights reserved).