Reply to Mac Giolla and Ly (2019): On the reporting of Bayes factors in deception research

Abstract

Bayes factors provide a continuous measure of evidence for one hypothesis (e.g., the null, H0) relative to another (e.g., the alternative, H1). Warmelink, Subramanian, Tkacheva and McLatchie (2019) reported Bayes factors alongside p-values to draw inferences about whether the order of expected versus unexpected questions influenced the amount of details interviewees provided during an interview. Mac Giolla & Ly (2019) provided several recommendations to improve the reporting of Bayesian analyses, and used Warmelink et al (2019) as a concrete example. These included (I) not to over-rely on cut-offs when interpreting Bayes factors; (II) to rely less on Bayes factors, and switch to “nominal support”; and (III) to report the posterior distribution. This paper elaborates on their recommendations and provides two further suggestions for improvement. First, we recommend deception researchers report Robustness Regions to demonstrate the sensitivity of their conclusions. Second, we encourage deception researchers to estimate a priori the sample size likely to be required to produce conclusive results.