The predictive value of gaze behavior and mouse-clicking in testing listening proficiency: A sensor technology study

IF 4.9 1区文学 Q1 EDUCATION & EDUCATIONAL RESEARCH

System Pub Date : 2024-09-19 DOI:10.1016/j.system.2024.103440

Yue Qiu, Vahid Aryadoust

{"title":"The predictive value of gaze behavior and mouse-clicking in testing listening proficiency: A sensor technology study","authors":"Yue Qiu, Vahid Aryadoust","doi":"10.1016/j.system.2024.103440","DOIUrl":null,"url":null,"abstract":"<div><p>This study employed eye-tracking and mouse click frequency analysis to investigate the predictive power of gaze behaviors, mouse-clicking, and their interactive effects with linguistic backgrounds on the IELTS (International English Language Testing System) listening test scores. A total of 77 test takers (45 with English as their first language (E-L1) and 32 with English as their second language (E-L2)) participated in this study. Their eye movements and mouse click frequencies were recorded as they took a computer-based IELTS listening test. The subsequent data analysis, utilizing linear mixed models, showed that gaze patterns, mouse actions, and language background significantly predicted listening test outcomes across four listening test sections and between E-L1 and E-L2 candidates, accounting for 33.2% of the variance observed in test scores. These results indicate the effect of potential sources of construct-irrelevant variance on test scores, which are not predicted in the available construct definitions of the test used in the study. Implications for the listening construct and test validity are discussed.</p></div>","PeriodicalId":48185,"journal":{"name":"System","volume":"126 ","pages":"Article 103440"},"PeriodicalIF":4.9000,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"System","FirstCategoryId":"98","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0346251X24002227","RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

This study employed eye-tracking and mouse click frequency analysis to investigate the predictive power of gaze behaviors, mouse-clicking, and their interactive effects with linguistic backgrounds on the IELTS (International English Language Testing System) listening test scores. A total of 77 test takers (45 with English as their first language (E-L1) and 32 with English as their second language (E-L2)) participated in this study. Their eye movements and mouse click frequencies were recorded as they took a computer-based IELTS listening test. The subsequent data analysis, utilizing linear mixed models, showed that gaze patterns, mouse actions, and language background significantly predicted listening test outcomes across four listening test sections and between E-L1 and E-L2 candidates, accounting for 33.2% of the variance observed in test scores. These results indicate the effect of potential sources of construct-irrelevant variance on test scores, which are not predicted in the available construct definitions of the test used in the study. Implications for the listening construct and test validity are discussed.

查看原文本刊更多论文

凝视行为和鼠标点击在听力水平测试中的预测价值：传感器技术研究

本研究采用眼动跟踪和鼠标点击频率分析法，研究了注视行为、鼠标点击及其与语言背景的交互作用对雅思（国际英语语言测试系统）听力测试成绩的预测能力。共有 77 名考生（45 名以英语为第一语言（E-L1），32 名以英语为第二语言（E-L2））参加了本研究。他们在参加电脑雅思听力测试时的眼球运动和鼠标点击频率都被记录了下来。随后利用线性混合模型进行的数据分析显示，目光模式、鼠标操作和语言背景对四个听力测试部分以及 E-L1 和 E-L2 考生之间的听力测试结果有显著的预测作用，占测试分数差异的 33.2%。这些结果表明，与建构无关的潜在变异源对测试成绩产生了影响，而这些变异源在本研究所用测试的现有建构定义中并未预测到。本研究讨论了听力建构和测试效度的影响。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

System Multiple-

CiteScore

8.80

自引率

8.30%

发文量

202

审稿时长

64 days

期刊介绍： This international journal is devoted to the applications of educational technology and applied linguistics to problems of foreign language teaching and learning. Attention is paid to all languages and to problems associated with the study and teaching of English as a second or foreign language. The journal serves as a vehicle of expression for colleagues in developing countries. System prefers its contributors to provide articles which have a sound theoretical base with a visible practical application which can be generalized. The review section may take up works of a more theoretical nature to broaden the background.