Janneke van de Pol, Eleanor Rowan, Eva Janssen, Tamara van Gog
{"title":"Effects of availability of diagnostic and non-diagnostic cues on the accuracy of teachers’ judgments of students’ text comprehension","authors":"Janneke van de Pol, Eleanor Rowan, Eva Janssen, Tamara van Gog","doi":"10.1007/s11409-024-09383-0","DOIUrl":null,"url":null,"abstract":"<p>Accurately judging students’ comprehension is a key professional competence for teachers. It is crucial for adapting instruction to students’ needs and thereby promoting student learning. According to the cue-utilization framework, the accuracy of teachers’ judgments depends on how predictive (or diagnostic) the information (or cues) that teachers use to make judgments is of student performance. It is, however, unclear from prior studies if merely providing access to diagnostic cues aids accuracy, or whether this only helps if non-diagnostic cues are unavailable or ignored. Therefore, we investigated, using a within-subjects experimental design, the accuracy of secondary school teachers’ (<i>N</i> = 33) judgments of anonymous students’ text comprehension under four cue availability conditions: 1) non-diagnostic cues only; 2) diagnostic cues only; 3) a mix of diagnostic and non-diagnostic cues; and, 4) after an intervention informing them of the diagnosticity of cues, again a mix of diagnostic and non-diagnostic cues. Access to diagnostic cues enhanced teachers’ judgment accuracy, while access to non-diagnostic cues hindered it. While teachers’ judgment accuracy was not enhanced by the intervention (presumably because it was already relatively high), their diagnostic cue utilization increased, and non-diagnostic cue utilization decreased. In addition, teachers’ calibration increased after the intervention: They knew better when their judgments were (in)accurate. Furthermore, teachers were quite aware that diagnostic cues are diagnostic, but their awareness that non-diagnostic cues (especially students’ interest) are not, could be improved. These results could be useful in designing effective interventions to further foster teachers’ judgment accuracy.</p>","PeriodicalId":47385,"journal":{"name":"Metacognition and Learning","volume":null,"pages":null},"PeriodicalIF":3.9000,"publicationDate":"2024-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Metacognition and Learning","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1007/s11409-024-09383-0","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0
Abstract
Accurately judging students’ comprehension is a key professional competence for teachers. It is crucial for adapting instruction to students’ needs and thereby promoting student learning. According to the cue-utilization framework, the accuracy of teachers’ judgments depends on how predictive (or diagnostic) the information (or cues) that teachers use to make judgments is of student performance. It is, however, unclear from prior studies if merely providing access to diagnostic cues aids accuracy, or whether this only helps if non-diagnostic cues are unavailable or ignored. Therefore, we investigated, using a within-subjects experimental design, the accuracy of secondary school teachers’ (N = 33) judgments of anonymous students’ text comprehension under four cue availability conditions: 1) non-diagnostic cues only; 2) diagnostic cues only; 3) a mix of diagnostic and non-diagnostic cues; and, 4) after an intervention informing them of the diagnosticity of cues, again a mix of diagnostic and non-diagnostic cues. Access to diagnostic cues enhanced teachers’ judgment accuracy, while access to non-diagnostic cues hindered it. While teachers’ judgment accuracy was not enhanced by the intervention (presumably because it was already relatively high), their diagnostic cue utilization increased, and non-diagnostic cue utilization decreased. In addition, teachers’ calibration increased after the intervention: They knew better when their judgments were (in)accurate. Furthermore, teachers were quite aware that diagnostic cues are diagnostic, but their awareness that non-diagnostic cues (especially students’ interest) are not, could be improved. These results could be useful in designing effective interventions to further foster teachers’ judgment accuracy.
期刊介绍:
The journal "Metacognition and Learning" addresses various components of metacognition, such as metacognitive awareness, experiences, knowledge, and executive skills.
Both general metacognition as well as domain-specific metacognitions in various task domains (mathematics, physics, reading, writing etc.) are considered. Papers may address fundamental theoretical issues, measurement issues regarding both quantitative and qualitative methods, as well as empirical studies about individual differences in metacognition, relations with other learner characteristics and learning strategies, developmental issues, the training of metacognition components in learning, and the teacher’s role in metacognition training. Studies highlighting the role of metacognition in self- or co-regulated learning as well as its relations with motivation and affect are also welcomed.
Submitted papers are judged on theoretical relevance, methodological thoroughness, and appeal to an international audience. The journal aims for a high academic standard with relevance to the field of educational practices.
One restriction is that papers should pertain to the role of metacognition in learning situations. Self-regulation in clinical settings, such as coping with phobia or anxiety outside learning situations, is beyond the scope of the journal.