{"title":"Measuring scientific inquiry ability related to hands-on practice: An automated approach based on multimodal data analysis","authors":"Yishen Song, Liming Guo, Qinhua Zheng","doi":"10.1007/s10639-024-12991-7","DOIUrl":null,"url":null,"abstract":"<p>Scientific inquiry ability is closely related to the process of hands-on inquiry practice. However, its assessment is often separated from this practice due to the limitation of technical basis and labor cost. The development of multimodal data analysis provides a new opportunity to realize automated assessment based on hands-on practice. Therefore, this study aims to explore whether and how we can use automated multimodal data analysis approaches to measure the scientific inquiry ability of students during the hands-on inquiry practice. In a scientific inquiry activity called \"Explore the Moon,\" designed for 472 fourth graders, we collected textual, tabular, and video data. Aiming to analyze and evaluate the data, we first designed a modal conversion method based on the multimodal pre-trained model LLaVA-7B and a text scoring method integrating keyword matching, one-way nearness, and Jaccard similarity. Then, to bridge the computing ability with the scoring criteria from science teachers, we constructed a structured representation language and verified the human–machine consistency of automated scoring. Finally, we used a multidimensional item response theory (IRT) model to validate the assessment's overall quality and analyze the participants' scientific inquiry ability. The proposed data analysis method has high man–machine consistency, and the results of IRT analysis present reasonable item characteristics. In summary, we constructed a low-cost and scalable multimodal assessment approach based on scientific inquiry activities, providing methodological support for science teachers to carry out formative evaluation of students' scientific inquiry activities in the daily inquiry-based learning environment.</p>","PeriodicalId":51494,"journal":{"name":"Education and Information Technologies","volume":null,"pages":null},"PeriodicalIF":4.8000,"publicationDate":"2024-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Education and Information Technologies","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1007/s10639-024-12991-7","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0
Abstract
Scientific inquiry ability is closely related to the process of hands-on inquiry practice. However, its assessment is often separated from this practice due to the limitation of technical basis and labor cost. The development of multimodal data analysis provides a new opportunity to realize automated assessment based on hands-on practice. Therefore, this study aims to explore whether and how we can use automated multimodal data analysis approaches to measure the scientific inquiry ability of students during the hands-on inquiry practice. In a scientific inquiry activity called "Explore the Moon," designed for 472 fourth graders, we collected textual, tabular, and video data. Aiming to analyze and evaluate the data, we first designed a modal conversion method based on the multimodal pre-trained model LLaVA-7B and a text scoring method integrating keyword matching, one-way nearness, and Jaccard similarity. Then, to bridge the computing ability with the scoring criteria from science teachers, we constructed a structured representation language and verified the human–machine consistency of automated scoring. Finally, we used a multidimensional item response theory (IRT) model to validate the assessment's overall quality and analyze the participants' scientific inquiry ability. The proposed data analysis method has high man–machine consistency, and the results of IRT analysis present reasonable item characteristics. In summary, we constructed a low-cost and scalable multimodal assessment approach based on scientific inquiry activities, providing methodological support for science teachers to carry out formative evaluation of students' scientific inquiry activities in the daily inquiry-based learning environment.
期刊介绍:
The Journal of Education and Information Technologies (EAIT) is a platform for the range of debates and issues in the field of Computing Education as well as the many uses of information and communication technology (ICT) across many educational subjects and sectors. It probes the use of computing to improve education and learning in a variety of settings, platforms and environments.
The journal aims to provide perspectives at all levels, from the micro level of specific pedagogical approaches in Computing Education and applications or instances of use in classrooms, to macro concerns of national policies and major projects; from pre-school classes to adults in tertiary institutions; from teachers and administrators to researchers and designers; from institutions to online and lifelong learning. The journal is embedded in the research and practice of professionals within the contemporary global context and its breadth and scope encourage debate on fundamental issues at all levels and from different research paradigms and learning theories. The journal does not proselytize on behalf of the technologies (whether they be mobile, desktop, interactive, virtual, games-based or learning management systems) but rather provokes debate on all the complex relationships within and between computing and education, whether they are in informal or formal settings. It probes state of the art technologies in Computing Education and it also considers the design and evaluation of digital educational artefacts. The journal aims to maintain and expand its international standing by careful selection on merit of the papers submitted, thus providing a credible ongoing forum for debate and scholarly discourse. Special Issues are occasionally published to cover particular issues in depth. EAIT invites readers to submit papers that draw inferences, probe theory and create new knowledge that informs practice, policy and scholarship. Readers are also invited to comment and reflect upon the argument and opinions published. EAIT is the official journal of the Technical Committee on Education of the International Federation for Information Processing (IFIP) in partnership with UNESCO.