How to communicate evidence to teachers: Comparing the effects of verbal and visual effect size representation [Registered Report - Stage I]

IF 5.7 1区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH

Learning and Instruction Pub Date : 2025-12-01 Epub Date: 2025-09-08 DOI:10.1016/j.learninstruc.2025.102179

Florian Kühlwein , Samuel Merk , Jürgen Schneider , Kirstin Schmidt

{"title":"How to communicate evidence to teachers: Comparing the effects of verbal and visual effect size representation [Registered Report - Stage I]","authors":"Florian Kühlwein , Samuel Merk , Jürgen Schneider , Kirstin Schmidt","doi":"10.1016/j.learninstruc.2025.102179","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>Teachers are encouraged to consider relevant evidence from educational research to improve the quality of their instruction. This approach requires them to have a reasonably accurate idea of the effectiveness of educational interventions. Currently prevalent ways of communicating effect sizes, however, often lead to imprecise effect magnitude estimates.</div></div><div><h3>Aims</h3><div>We aim to investigate how teachers estimate and perceive verbally and visually communicated effect sizes.</div></div><div><h3>Sample</h3><div>For the pilot study, <em>N</em> = 20 English-speaking teachers were recruited on Prolific, and for the main study, <em>N</em> = 120 will be recruited.</div></div><div><h3>Planned methods</h3><div>We employ a randomized 4 × 6 × 2 between–within-within-person experiment with the outcomes of interest being teachers' effect magnitude estimates and perceived relevance, informativeness, and difficulty.</div></div><div><h3>Pilot results</h3><div>Across the conditions, teachers' effect magnitude estimates correlated strongly (<em>τ</em> = 0.69) with true effect sizes and personified relevance correlated moderately (<em>τ</em> = 0.41) with objective relevance. However, teachers were less accurate in the verbal than in the visual condition (Cohen's <em>d</em> = −0.52, 95 % <em>CI</em> = [−0.81, −0.23]). Conversely, they perceived verbally communicated Cohen's U<sub>3</sub> expressions as more informative (r<sub>rb</sub> = −0.27, 95 % <em>CI</em> = [−0.40, −0.13]), and less difficult (r<sub>rb</sub> = 0.31, 95 % <em>CI</em> = [0.17, 0.44]) to understand than visually communicated effect sizes.</div></div><div><h3>Preliminary conclusion</h3><div>Communicating effect sizes can work verbally and visually.</div></div>","PeriodicalId":48357,"journal":{"name":"Learning and Instruction","volume":"100 ","pages":"Article 102179"},"PeriodicalIF":5.7000,"publicationDate":"2025-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Learning and Instruction","FirstCategoryId":"95","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0959475225001033","RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2025/9/8 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

Background

Teachers are encouraged to consider relevant evidence from educational research to improve the quality of their instruction. This approach requires them to have a reasonably accurate idea of the effectiveness of educational interventions. Currently prevalent ways of communicating effect sizes, however, often lead to imprecise effect magnitude estimates.

Aims

We aim to investigate how teachers estimate and perceive verbally and visually communicated effect sizes.

Sample

For the pilot study, N = 20 English-speaking teachers were recruited on Prolific, and for the main study, N = 120 will be recruited.

Planned methods

We employ a randomized 4 × 6 × 2 between–within-within-person experiment with the outcomes of interest being teachers' effect magnitude estimates and perceived relevance, informativeness, and difficulty.

Pilot results

Across the conditions, teachers' effect magnitude estimates correlated strongly (τ = 0.69) with true effect sizes and personified relevance correlated moderately (τ = 0.41) with objective relevance. However, teachers were less accurate in the verbal than in the visual condition (Cohen's d = −0.52, 95 % CI = [−0.81, −0.23]). Conversely, they perceived verbally communicated Cohen's U₃ expressions as more informative (r_rb = −0.27, 95 % CI = [−0.40, −0.13]), and less difficult (r_rb = 0.31, 95 % CI = [0.17, 0.44]) to understand than visually communicated effect sizes.

Preliminary conclusion

Communicating effect sizes can work verbally and visually.

查看原文本刊更多论文

如何向教师传达证据：比较语言和视觉效果大小表示的效果[注册报告-第一阶段]

背景鼓励教师考虑来自教育研究的相关证据，以提高教学质量。这种方法要求他们对教育干预的有效性有一个相当准确的认识。然而，目前流行的传播效应大小的方法往往导致不精确的效应大小估计。目的我们的目的是调查教师如何估计和感知口头和视觉传达的效应大小。作为试点研究，在多产网站上招募了N = 20名英语教师，作为主要研究，将招募N = 120名教师。计划方法我们采用了随机的4 × 6 × 2的人与人之间的实验，感兴趣的结果是教师的效应大小估计和感知的相关性、信息量和难度。在所有条件下，教师的效应量估计值与真实效应量强烈相关（τ = 0.69），人格化相关性与客观相关性适度相关（τ = 0.41）。然而，教师在语言方面的准确性低于视觉方面（Cohen’s d = - 0.52, 95% CI =[- 0.81, - 0.23]）。相反，他们认为口头传达的Cohen的U3表达比视觉传达的效果更有信息量（rrb = - 0.27, 95% CI =[- 0.40, - 0.13]），并且更容易理解（rrb = 0.31, 95% CI =[0.17, 0.44]）。初步结论沟通效应量可以通过口头和视觉两种方式进行。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Learning and Instruction Multiple-

CiteScore

11.30

自引率

4.80%

发文量

109

期刊介绍： As an international, multi-disciplinary, peer-refereed journal, Learning and Instruction provides a platform for the publication of the most advanced scientific research in the areas of learning, development, instruction and teaching. The journal welcomes original empirical investigations. The papers may represent a variety of theoretical perspectives and different methodological approaches. They may refer to any age level, from infants to adults and to a diversity of learning and instructional settings, from laboratory experiments to field studies. The major criteria in the review and the selection process concern the significance of the contribution to the area of learning and instruction, and the rigor of the study.