Measuring scientific inquiry ability related to hands-on practice: An automated approach based on multimodal data analysis

IF 4.8 2区 教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH
Yishen Song, Liming Guo, Qinhua Zheng
{"title":"Measuring scientific inquiry ability related to hands-on practice: An automated approach based on multimodal data analysis","authors":"Yishen Song, Liming Guo, Qinhua Zheng","doi":"10.1007/s10639-024-12991-7","DOIUrl":null,"url":null,"abstract":"<p>Scientific inquiry ability is closely related to the process of hands-on inquiry practice. However, its assessment is often separated from this practice due to the limitation of technical basis and labor cost. The development of multimodal data analysis provides a new opportunity to realize automated assessment based on hands-on practice. Therefore, this study aims to explore whether and how we can use automated multimodal data analysis approaches to measure the scientific inquiry ability of students during the hands-on inquiry practice. In a scientific inquiry activity called \"Explore the Moon,\" designed for 472 fourth graders, we collected textual, tabular, and video data. Aiming to analyze and evaluate the data, we first designed a modal conversion method based on the multimodal pre-trained model LLaVA-7B and a text scoring method integrating keyword matching, one-way nearness, and Jaccard similarity. Then, to bridge the computing ability with the scoring criteria from science teachers, we constructed a structured representation language and verified the human–machine consistency of automated scoring. Finally, we used a multidimensional item response theory (IRT) model to validate the assessment's overall quality and analyze the participants' scientific inquiry ability. The proposed data analysis method has high man–machine consistency, and the results of IRT analysis present reasonable item characteristics. In summary, we constructed a low-cost and scalable multimodal assessment approach based on scientific inquiry activities, providing methodological support for science teachers to carry out formative evaluation of students' scientific inquiry activities in the daily inquiry-based learning environment.</p>","PeriodicalId":51494,"journal":{"name":"Education and Information Technologies","volume":null,"pages":null},"PeriodicalIF":4.8000,"publicationDate":"2024-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Education and Information Technologies","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1007/s10639-024-12991-7","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0

Abstract

Scientific inquiry ability is closely related to the process of hands-on inquiry practice. However, its assessment is often separated from this practice due to the limitation of technical basis and labor cost. The development of multimodal data analysis provides a new opportunity to realize automated assessment based on hands-on practice. Therefore, this study aims to explore whether and how we can use automated multimodal data analysis approaches to measure the scientific inquiry ability of students during the hands-on inquiry practice. In a scientific inquiry activity called "Explore the Moon," designed for 472 fourth graders, we collected textual, tabular, and video data. Aiming to analyze and evaluate the data, we first designed a modal conversion method based on the multimodal pre-trained model LLaVA-7B and a text scoring method integrating keyword matching, one-way nearness, and Jaccard similarity. Then, to bridge the computing ability with the scoring criteria from science teachers, we constructed a structured representation language and verified the human–machine consistency of automated scoring. Finally, we used a multidimensional item response theory (IRT) model to validate the assessment's overall quality and analyze the participants' scientific inquiry ability. The proposed data analysis method has high man–machine consistency, and the results of IRT analysis present reasonable item characteristics. In summary, we constructed a low-cost and scalable multimodal assessment approach based on scientific inquiry activities, providing methodological support for science teachers to carry out formative evaluation of students' scientific inquiry activities in the daily inquiry-based learning environment.

Abstract Image

测量与动手实践相关的科学探究能力:基于多模态数据分析的自动化方法
科学探究能力与动手探究实践过程密切相关。然而,由于技术基础和人力成本的限制,其评价往往与实践相分离。多模态数据分析的发展为实现基于动手实践的自动化测评提供了新的契机。因此,本研究旨在探索能否以及如何使用自动化的多模态数据分析方法来测量学生在动手探究实践中的科学探究能力。在为 472 名四年级学生设计的名为 "探索月球 "的科学探究活动中,我们收集了文本、表格和视频数据。为了分析和评估这些数据,我们首先设计了一种基于多模态预训练模型 LLaVA-7B 的模态转换方法,以及一种整合了关键词匹配、单向接近度和 Jaccard 相似度的文本评分方法。然后,为了将计算能力与科学教师的评分标准结合起来,我们构建了一种结构化表示语言,并验证了自动评分的人机一致性。最后,我们使用多维项目反应理论(IRT)模型验证了测评的整体质量,并分析了参与者的科学探究能力。所提出的数据分析方法具有较高的人机一致性,IRT 分析结果呈现出合理的项目特征。总之,我们构建了一种基于科学探究活动的低成本、可扩展的多模态评价方法,为科学教师在日常探究式学习环境中对学生的科学探究活动进行形成性评价提供了方法支持。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Education and Information Technologies
Education and Information Technologies EDUCATION & EDUCATIONAL RESEARCH-
CiteScore
10.00
自引率
12.70%
发文量
610
期刊介绍: The Journal of Education and Information Technologies (EAIT) is a platform for the range of debates and issues in the field of Computing Education as well as the many uses of information and communication technology (ICT) across many educational subjects and sectors. It probes the use of computing to improve education and learning in a variety of settings, platforms and environments. The journal aims to provide perspectives at all levels, from the micro level of specific pedagogical approaches in Computing Education and applications or instances of use in classrooms, to macro concerns of national policies and major projects; from pre-school classes to adults in tertiary institutions; from teachers and administrators to researchers and designers; from institutions to online and lifelong learning. The journal is embedded in the research and practice of professionals within the contemporary global context and its breadth and scope encourage debate on fundamental issues at all levels and from different research paradigms and learning theories. The journal does not proselytize on behalf of the technologies (whether they be mobile, desktop, interactive, virtual, games-based or learning management systems) but rather provokes debate on all the complex relationships within and between computing and education, whether they are in informal or formal settings. It probes state of the art technologies in Computing Education and it also considers the design and evaluation of digital educational artefacts.  The journal aims to maintain and expand its international standing by careful selection on merit of the papers submitted, thus providing a credible ongoing forum for debate and scholarly discourse. Special Issues are occasionally published to cover particular issues in depth. EAIT invites readers to submit papers that draw inferences, probe theory and create new knowledge that informs practice, policy and scholarship. Readers are also invited to comment and reflect upon the argument and opinions published. EAIT is the official journal of the Technical Committee on Education of the International Federation for Information Processing (IFIP) in partnership with UNESCO.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信