Psychology Faculty Overestimate the Magnitude of Cohen’s d Effect Sizes by Half a Standard Deviation

IF 3.1 3区 心理学 Q1 PSYCHOLOGY, MULTIDISCIPLINARY
Brendan A. Schuetze, Veronica X. Yan
{"title":"Psychology Faculty Overestimate the Magnitude of Cohen’s d Effect Sizes by Half a Standard Deviation","authors":"Brendan A. Schuetze, Veronica X. Yan","doi":"10.1525/collabra.74020","DOIUrl":null,"url":null,"abstract":"In this experiment, we recruited 261 psychology faculty to determine the extent to which they were able to visually estimate the overlap of two distributions given a Cohen’s d effect size; and vice-versa estimate d given two distributions of varying overlap. In a pre-test, participants in both conditions over-estimated effect sizes by half a standard deviation on average. No significant differences in estimation accuracy by psychology sub-field were found, but having taught statistics coursework was a significant predictor of better performance. After a short training session, participants improved substantially on both tasks on the post-test, with 63% reduction in absolute error and negligible overall bias (98% bias reduction). Furthermore, post-test performance indicated that learning transferred across answering modes. Teachers of statistics might find it beneficial to include a short exercise (less than 10 minutes) requiring the visual estimation of effect sizes in statistics coursework to better train future psychology researchers.","PeriodicalId":45791,"journal":{"name":"Collabra-Psychology","volume":null,"pages":null},"PeriodicalIF":3.1000,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Collabra-Psychology","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1525/collabra.74020","RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PSYCHOLOGY, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0

Abstract

In this experiment, we recruited 261 psychology faculty to determine the extent to which they were able to visually estimate the overlap of two distributions given a Cohen’s d effect size; and vice-versa estimate d given two distributions of varying overlap. In a pre-test, participants in both conditions over-estimated effect sizes by half a standard deviation on average. No significant differences in estimation accuracy by psychology sub-field were found, but having taught statistics coursework was a significant predictor of better performance. After a short training session, participants improved substantially on both tasks on the post-test, with 63% reduction in absolute error and negligible overall bias (98% bias reduction). Furthermore, post-test performance indicated that learning transferred across answering modes. Teachers of statistics might find it beneficial to include a short exercise (less than 10 minutes) requiring the visual estimation of effect sizes in statistics coursework to better train future psychology researchers.
心理学教授高估了科恩效应大小的一半标准差
在这个实验中,我们招募了261名心理学教师,以确定他们能够在多大程度上直观地估计两个分布的重叠,给定科恩效应大小;反之亦然,在两个不同重叠的分布下估计d。在预测试中,两种情况下的参与者平均高估了效应大小的一半标准差。心理学子领域在估计准确性上没有发现显著差异,但教过统计学课程是更好的表现的显著预测因子。经过短暂的训练,参与者在后测的两个任务上都有了很大的提高,绝对误差减少了63%,总体偏差可以忽略不计(偏差减少了98%)。此外,测试后的表现表明学习跨回答模式转移。统计学教师可能会发现,在统计学课程中加入一个需要对效应大小进行视觉估计的简短练习(不到10分钟),以更好地培养未来的心理学研究人员,这是有益的。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Collabra-Psychology
Collabra-Psychology PSYCHOLOGY, MULTIDISCIPLINARY-
CiteScore
3.60
自引率
4.00%
发文量
47
审稿时长
16 weeks
期刊介绍: Collabra: Psychology has 7 sections representing the broad field of psychology, and a highlighted focus area of “Methodology and Research Practice.” Are: Cognitive Psychology Social Psychology Personality Psychology Clinical Psychology Developmental Psychology Organizational Behavior Methodology and Research Practice.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信