Design and validation of the AI literacy questionnaire: The affective, behavioural, cognitive and ethical approach

IF 6.7 1区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH

British Journal of Educational Technology Pub Date : 2023-12-13 DOI:10.1111/bjet.13411

Davy Tsz Kit Ng, Wenjie Wu, Jac Ka Lok Leung, Thomas Kin Fung Chiu, Samuel Kai Wah Chu

{"title":"Design and validation of the AI literacy questionnaire: The affective, behavioural, cognitive and ethical approach","authors":"Davy Tsz Kit Ng, Wenjie Wu, Jac Ka Lok Leung, Thomas Kin Fung Chiu, Samuel Kai Wah Chu","doi":"10.1111/bjet.13411","DOIUrl":null,"url":null,"abstract":"<div>\n \n <section>\n \n <p>Artificial intelligence (AI) literacy is at the top of the agenda for education today in developing learners' AI knowledge, skills, attitudes and values in the 21st century. However, there are few validated research instruments for educators to examine how secondary students develop and perceive their learning outcomes. After reviewing the literature on AI literacy questionnaires, we categorized the identified competencies in four dimensions: (1) affective learning (intrinsic motivation and self-efficacy/confidence), (2) behavioural learning (behavioural commitment and collaboration), (3) cognitive learning (know and understand; apply, evaluate and create) and (4) ethical learning. Then, a 32-item self-reported questionnaire on AI literacy (AILQ) was developed and validated to measure students' literacy development in the four dimensions. The design and validation of AILQ were examined through theoretical review, expert judgement, interview, pilot study and first- and second-order confirmatory factor analysis. This article reports the findings of a pilot study using a preliminary version of the AILQ among 363 secondary school students in Hong Kong to analyse the psychometric properties of the instrument. Results indicated a four-factor structure of the AILQ and revealed good reliability and validity. The AILQ is recommended as a reliable measurement scale for assessing how secondary students foster their AI literacy and inform better instructional design based on the proposed affective, behavioural, cognitive and ethical (ABCE) learning framework.</p>\n </section>\n \n <section>\n \n <div>\n \n <div>\n \n <h3>Practitioner notes</h3>\n <p>What is already known about this topic\n\n </p><ul>\n \n <li>AI literacy has drawn increasing attention in recent years and has been identified as an important digital literacy.</li>\n \n <li>Schools and universities around the world started to incorporate AI into their curriculum to foster young learners' AI literacy.</li>\n \n <li>Some studies have worked to design suitable measurement tools, especially questionnaires, to examine students' learning outcomes in AI learning programmes.</li>\n </ul>\n <p>What this paper adds\n\n </p><ul>\n \n <li>Develops an AI literacy questionnaire (AILQ) to evaluate students' literacy development in terms of affective, behavioural, cognitive and ethical (ABCE) dimensions.</li>\n \n <li>Proposes a parsimonious model based on the ABCE framework and addresses a skill set of AI literacy.</li>\n </ul>\n <p>Implications for practice and/or policy\n\n </p><ul>\n \n <li>Researchers are able to use the AILQ as a guide to measure students' AI literacy.</li>\n \n <li>Practitioners are able to use the AILQ to assess students' AI literacy development.</li>\n </ul>\n </div>\n </div>\n </section>\n </div>","PeriodicalId":48315,"journal":{"name":"British Journal of Educational Technology","volume":"55 3","pages":"1082-1104"},"PeriodicalIF":6.7000,"publicationDate":"2023-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/bjet.13411","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"British Journal of Educational Technology","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/bjet.13411","RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

Artificial intelligence (AI) literacy is at the top of the agenda for education today in developing learners' AI knowledge, skills, attitudes and values in the 21st century. However, there are few validated research instruments for educators to examine how secondary students develop and perceive their learning outcomes. After reviewing the literature on AI literacy questionnaires, we categorized the identified competencies in four dimensions: (1) affective learning (intrinsic motivation and self-efficacy/confidence), (2) behavioural learning (behavioural commitment and collaboration), (3) cognitive learning (know and understand; apply, evaluate and create) and (4) ethical learning. Then, a 32-item self-reported questionnaire on AI literacy (AILQ) was developed and validated to measure students' literacy development in the four dimensions. The design and validation of AILQ were examined through theoretical review, expert judgement, interview, pilot study and first- and second-order confirmatory factor analysis. This article reports the findings of a pilot study using a preliminary version of the AILQ among 363 secondary school students in Hong Kong to analyse the psychometric properties of the instrument. Results indicated a four-factor structure of the AILQ and revealed good reliability and validity. The AILQ is recommended as a reliable measurement scale for assessing how secondary students foster their AI literacy and inform better instructional design based on the proposed affective, behavioural, cognitive and ethical (ABCE) learning framework.

Practitioner notes

What is already known about this topic

AI literacy has drawn increasing attention in recent years and has been identified as an important digital literacy.
Schools and universities around the world started to incorporate AI into their curriculum to foster young learners' AI literacy.
Some studies have worked to design suitable measurement tools, especially questionnaires, to examine students' learning outcomes in AI learning programmes.

What this paper adds

Develops an AI literacy questionnaire (AILQ) to evaluate students' literacy development in terms of affective, behavioural, cognitive and ethical (ABCE) dimensions.
Proposes a parsimonious model based on the ABCE framework and addresses a skill set of AI literacy.

Implications for practice and/or policy

Researchers are able to use the AILQ as a guide to measure students' AI literacy.
Practitioners are able to use the AILQ to assess students' AI literacy development.

Abstract Image

查看原文本刊更多论文

设计和验证人工智能素养问卷：情感、行为、认知和伦理方法

人工智能（AI）素养是当今教育议程的重中之重，旨在培养学习者在 21 世纪的人工智能知识、技能、态度和价值观。然而，供教育工作者研究中学生如何发展和感知其学习成果的有效研究工具却很少。在回顾了有关人工智能素养问卷的文献后，我们将已确定的能力分为四个维度：(1) 情感学习（内在动机和自我效能/自信），(2) 行为学习（行为承诺和协作），(3) 认知学习（了解和理解；应用、评价和创造）和 (4) 道德学习。然后，开发并验证了 32 个项目的人工智能素养自我报告问卷（AILQ），以测量学生在这四个维度上的素养发展情况。通过理论回顾、专家判断、访谈、试点研究以及一阶和二阶确认性因子分析，对 AILQ 的设计和验证进行了研究。本文报告了一项试验性研究的结果，该研究使用了 AILQ 的初步版本，对香港的 363 名中学生进行了测试，以分析该工具的心理测量特性。结果表明，AILQ 具有四因素结构，并显示出良好的信度和效度。建议将 AILQ 作为可靠的测量量表，用于评估中学生如何培养自己的人工智能素养，并根据建议的情感、行为、认知和道德（ABCE）学习框架进行更好的教学设计。一些研究致力于设计合适的测量工具，特别是调查问卷，以检查学生在人工智能学习课程中的学习成果。本文的补充内容开发了人工智能素养调查问卷（AILQ），从情感、行为、认知和道德（ABCE）维度评估学生的素养发展。对实践和/或政策的启示研究人员可以使用 AILQ 作为衡量学生人工智能素养的指南，从业人员可以使用 AILQ 评估学生的人工智能素养发展。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

British Journal of Educational Technology EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

15.60

自引率

4.50%

发文量

111

期刊介绍： BJET is a primary source for academics and professionals in the fields of digital educational and training technology throughout the world. The Journal is published by Wiley on behalf of The British Educational Research Association (BERA). It publishes theoretical perspectives, methodological developments and high quality empirical research that demonstrate whether and how applications of instructional/educational technology systems, networks, tools and resources lead to improvements in formal and non-formal education at all levels, from early years through to higher, technical and vocational education, professional development and corporate training.