AI-Assisted Assessment of Inquiry Skills in Socioscientific Issue Contexts

IF 5.1 2区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH

Journal of Computer Assisted Learning Pub Date : 2024-12-25 DOI:10.1111/jcal.13102

Wen Xin Zhang, John J. H. Lin, Ying-Shao Hsu

{"title":"AI-Assisted Assessment of Inquiry Skills in Socioscientific Issue Contexts","authors":"Wen Xin Zhang, John J. H. Lin, Ying-Shao Hsu","doi":"10.1111/jcal.13102","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background Study</h3>\n \n <p>Assessing learners' inquiry-based skills is challenging as social, political, and technological dimensions must be considered. The advanced development of artificial intelligence (AI) makes it possible to address these challenges and shape the next generation of science education.</p>\n </section>\n \n <section>\n \n <h3> Objectives</h3>\n \n <p>The present study evaluated the SSI inquiry skills of students in an AI-enabled scoring environment. An AI model for socioscientific issues that can assess students' inquiry skills was developed. Responses to a learning module were collected from 1250 participants, and the open-ended responses were rated by humans in accordance with a designed rubric. The collected data were then preprocessed and used to train an AI rater that can process natural language. The effects of two hyperparameters, the dropout rate and complexity of the AI neural network, were evaluated.</p>\n </section>\n \n <section>\n \n <h3> Results and Conclusion</h3>\n \n <p>The results suggested neither of the two hyperparameters was found to strongly affect the accuracy of the AI rater. In general, the human and AI raters exhibited certain levels of agreement; however, agreement varied among rubric categories. Discrepancies were identified and are discussed both quantitatively and qualitatively.</p>\n </section>\n </div>","PeriodicalId":48071,"journal":{"name":"Journal of Computer Assisted Learning","volume":"41 1","pages":""},"PeriodicalIF":5.1000,"publicationDate":"2024-12-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Computer Assisted Learning","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/jcal.13102","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

Background Study

Assessing learners' inquiry-based skills is challenging as social, political, and technological dimensions must be considered. The advanced development of artificial intelligence (AI) makes it possible to address these challenges and shape the next generation of science education.

Objectives

The present study evaluated the SSI inquiry skills of students in an AI-enabled scoring environment. An AI model for socioscientific issues that can assess students' inquiry skills was developed. Responses to a learning module were collected from 1250 participants, and the open-ended responses were rated by humans in accordance with a designed rubric. The collected data were then preprocessed and used to train an AI rater that can process natural language. The effects of two hyperparameters, the dropout rate and complexity of the AI neural network, were evaluated.

Results and Conclusion

The results suggested neither of the two hyperparameters was found to strongly affect the accuracy of the AI rater. In general, the human and AI raters exhibited certain levels of agreement; however, agreement varied among rubric categories. Discrepancies were identified and are discussed both quantitatively and qualitatively.

查看原文本刊更多论文

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Computer Assisted Learning EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

9.70

自引率

6.00%

发文量

116

期刊介绍： The Journal of Computer Assisted Learning is an international peer-reviewed journal which covers the whole range of uses of information and communication technology to support learning and knowledge exchange. It aims to provide a medium for communication among researchers as well as a channel linking researchers, practitioners, and policy makers. JCAL is also a rich source of material for master and PhD students in areas such as educational psychology, the learning sciences, instructional technology, instructional design, collaborative learning, intelligent learning systems, learning analytics, open, distance and networked learning, and educational evaluation and assessment. This is the case for formal (e.g., schools), non-formal (e.g., workplace learning) and informal learning (e.g., museums and libraries) situations and environments. Volumes often include one Special Issue which these provides readers with a broad and in-depth perspective on a specific topic. First published in 1985, JCAL continues to have the aim of making the outcomes of contemporary research and experience accessible. During this period there have been major technological advances offering new opportunities and approaches in the use of a wide range of technologies to support learning and knowledge transfer more generally. There is currently much emphasis on the use of network functionality and the challenges its appropriate uses pose to teachers/tutors working with students locally and at a distance. JCAL welcomes: -Empirical reports, single studies or programmatic series of studies on the use of computers and information technologies in learning and assessment -Critical and original meta-reviews of literature on the use of computers for learning -Empirical studies on the design and development of innovative technology-based systems for learning -Conceptual articles on issues relating to the Aims and Scope