Use of technology-based assessments: a systematic review covering over 30 countries

IF 2.7 3区 教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH
Dandan Chen, Amos Jeng, Shiyu Sun, Bradley Kaptur
{"title":"Use of technology-based assessments: a systematic review covering over 30 countries","authors":"Dandan Chen, Amos Jeng, Shiyu Sun, Bradley Kaptur","doi":"10.1080/0969594x.2023.2270181","DOIUrl":null,"url":null,"abstract":"ABSTRACTThe widespread adoption of technology-based assessments during the COVID-19 pandemic has exacerbated concerns about the digital divide, given global disparities in digital access, connectivity, and coping strategies. This systematic review was intended to assess how the use of technology-based assessments has affected the education system’s functioning in pre-college education, compared to traditional assessments. It covered 34 countries from 34 full-text English sources in 2018–2022. A total of 12 assumptions were tested, corresponding to six hypotheses about the learning, educating, and management facets associated with the use of technology-based assessments. Our findings revealed mixed evidence about technology-based assessments in reducing cheating, enhancing learning, supporting monitoring, improving instruction, and reducing non-teaching workload. However, strong evidence supported the assumptions that technology-based assessments improve measurement precision, interpretability, engagement, interaction, and teacher-parent communication. Limited but positive evidence supported the assumptions that technology-based assessments may reduce the cost and time of test administration.KEYWORDS: Educational technologysystematic reviewtechnology-based assessmentinternational context AcknowledgmentsThe authors express sincere appreciation and gratitude to Dr. Priyadarshani Joshi (Senior Research Analyst, GEM Report, UNESCO) and Dr. Patrick Montjouridès (Senior Program Specialist, UNESCO), who offered considerable feedback on the first draft of this paper. Also, the authors thank Dr. Jinming Zhang (Professor), Ms. Jing Zhang, and Mr. Yingbin Zhang (doctoral students) at the University of Illinois Urbana-Champaign for their support, and the journal editor and anonymous reviewers whose comments led to a significant improvement of this article. Any remaining errors or omissions are the authors’ own.Disclosure statementNo potential conflict of interest was reported by the authors.GeolocationArgentina, Australia, Canada, Chile, China, Croatia, Cyprus, Czech Republic, Denmark, France, Germany, India, Indonesia, Italy, Kazakhstan, Lithuania, Malaysia, Netherlands, Norway, Pakistan, Philippines, Poland, Russia, Saudi Arabia, Singapore, Slovakia, Slovenia, South Korea, Spain, Switzerland, Thailand, Turkey, United Kingdom, Vietnam.Data availability statementThe data that support the findings of this study are available, but restrictions apply to the availability of these data, which were used under licence for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request.Supplementary materialSupplemental data for this article can be accessed online at https://doi.org/10.1080/0969594X.2023.2270181.Additional informationFundingThis work was supported by the Global Education Monitoring (GEM) Report, UNESCO in Paris, France (Contract No. 4500461800). A previous version of this paper was prepared as background research for Global Education Monitoring Report 2023 on Technology in Education : A tool on whose terms. The comments and conceptualisations presented here represent those of the authors and should not be attributed to any individual or organisation.Notes on contributorsDandan ChenDandan Chen works as an Associate Research Scientist at Pearson in the United States. She has a Ph.D. in Educational Psychology from the University of Illinois Urbana-Champaign, specialising in psychometrics. She has received funding from UNESCO (2022, 2017), National Science Foundation (2018-2020), and Rockefeller Philanthropy Advisors (2016), and has authored systematic reviews and participated in reports for the World Bank, the Education Commission, UNESCO, and UNFPA.Amos JengAmos Jeng is a doctoral student in the Department of Educational Psychology at the University of Illinois Urbana-Champaign. His work primarily focuses on learning-oriented interactions in educational settings. He is especially interested in understanding and improving college students’ experiences with help-seeking and help-giving in introductory STEM course settings, as well as placing these interactive behaviours in a cultural context.Shiyu SunShiyu Sun is a doctoral student in the Department of Educational Psychology at the University of Illinois Urbana-Champaign. Her work investigates college students’ sense of belonging in different contextual settings. More specifically, she is interested in exploring international students’ sense of belonging in university settings and understanding how different cultural and social factors affect students’ belonging experiences.Bradley KapturBradley Kaptur is a resident physician in radiology at HSHS Saint John’s Hospital and was a medical student at the Carle Illinois College of Medicine at the University of Illinois Urbana-Champaign. He has previously published on healthcare utilisation patterns and geospatial disparities in healthcare access. His research interests include the utilisation of technology in medical education and assessment methods used in medical school.","PeriodicalId":51515,"journal":{"name":"Assessment in Education-Principles Policy & Practice","volume":null,"pages":null},"PeriodicalIF":2.7000,"publicationDate":"2023-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Assessment in Education-Principles Policy & Practice","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1080/0969594x.2023.2270181","RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0

Abstract

ABSTRACTThe widespread adoption of technology-based assessments during the COVID-19 pandemic has exacerbated concerns about the digital divide, given global disparities in digital access, connectivity, and coping strategies. This systematic review was intended to assess how the use of technology-based assessments has affected the education system’s functioning in pre-college education, compared to traditional assessments. It covered 34 countries from 34 full-text English sources in 2018–2022. A total of 12 assumptions were tested, corresponding to six hypotheses about the learning, educating, and management facets associated with the use of technology-based assessments. Our findings revealed mixed evidence about technology-based assessments in reducing cheating, enhancing learning, supporting monitoring, improving instruction, and reducing non-teaching workload. However, strong evidence supported the assumptions that technology-based assessments improve measurement precision, interpretability, engagement, interaction, and teacher-parent communication. Limited but positive evidence supported the assumptions that technology-based assessments may reduce the cost and time of test administration.KEYWORDS: Educational technologysystematic reviewtechnology-based assessmentinternational context AcknowledgmentsThe authors express sincere appreciation and gratitude to Dr. Priyadarshani Joshi (Senior Research Analyst, GEM Report, UNESCO) and Dr. Patrick Montjouridès (Senior Program Specialist, UNESCO), who offered considerable feedback on the first draft of this paper. Also, the authors thank Dr. Jinming Zhang (Professor), Ms. Jing Zhang, and Mr. Yingbin Zhang (doctoral students) at the University of Illinois Urbana-Champaign for their support, and the journal editor and anonymous reviewers whose comments led to a significant improvement of this article. Any remaining errors or omissions are the authors’ own.Disclosure statementNo potential conflict of interest was reported by the authors.GeolocationArgentina, Australia, Canada, Chile, China, Croatia, Cyprus, Czech Republic, Denmark, France, Germany, India, Indonesia, Italy, Kazakhstan, Lithuania, Malaysia, Netherlands, Norway, Pakistan, Philippines, Poland, Russia, Saudi Arabia, Singapore, Slovakia, Slovenia, South Korea, Spain, Switzerland, Thailand, Turkey, United Kingdom, Vietnam.Data availability statementThe data that support the findings of this study are available, but restrictions apply to the availability of these data, which were used under licence for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request.Supplementary materialSupplemental data for this article can be accessed online at https://doi.org/10.1080/0969594X.2023.2270181.Additional informationFundingThis work was supported by the Global Education Monitoring (GEM) Report, UNESCO in Paris, France (Contract No. 4500461800). A previous version of this paper was prepared as background research for Global Education Monitoring Report 2023 on Technology in Education : A tool on whose terms. The comments and conceptualisations presented here represent those of the authors and should not be attributed to any individual or organisation.Notes on contributorsDandan ChenDandan Chen works as an Associate Research Scientist at Pearson in the United States. She has a Ph.D. in Educational Psychology from the University of Illinois Urbana-Champaign, specialising in psychometrics. She has received funding from UNESCO (2022, 2017), National Science Foundation (2018-2020), and Rockefeller Philanthropy Advisors (2016), and has authored systematic reviews and participated in reports for the World Bank, the Education Commission, UNESCO, and UNFPA.Amos JengAmos Jeng is a doctoral student in the Department of Educational Psychology at the University of Illinois Urbana-Champaign. His work primarily focuses on learning-oriented interactions in educational settings. He is especially interested in understanding and improving college students’ experiences with help-seeking and help-giving in introductory STEM course settings, as well as placing these interactive behaviours in a cultural context.Shiyu SunShiyu Sun is a doctoral student in the Department of Educational Psychology at the University of Illinois Urbana-Champaign. Her work investigates college students’ sense of belonging in different contextual settings. More specifically, she is interested in exploring international students’ sense of belonging in university settings and understanding how different cultural and social factors affect students’ belonging experiences.Bradley KapturBradley Kaptur is a resident physician in radiology at HSHS Saint John’s Hospital and was a medical student at the Carle Illinois College of Medicine at the University of Illinois Urbana-Champaign. He has previously published on healthcare utilisation patterns and geospatial disparities in healthcare access. His research interests include the utilisation of technology in medical education and assessment methods used in medical school.
使用基于技术的评估:覆盖30多个国家的系统审查
摘要2019冠状病毒病大流行期间广泛采用基于技术的评估,加剧了人们对数字鸿沟的担忧,因为全球在数字访问、连通性和应对策略方面存在差异。这项系统审查旨在评估与传统评估相比,使用基于技术的评估如何影响教育系统在大学前教育中的运作。在2018-2022年期间,它涵盖了34个国家的34个全文英文来源。总共测试了12个假设,对应于与使用基于技术的评估相关的学习、教育和管理方面的6个假设。我们的研究结果揭示了基于技术的评估在减少作弊、加强学习、支持监控、改善教学和减少非教学工作量方面的混合证据。然而,强有力的证据支持基于技术的评估提高测量精度、可解释性、参与、互动和教师与家长沟通的假设。有限但积极的证据支持基于技术的评估可能减少考试管理的成本和时间的假设。作者对Priyadarshani Joshi博士(联合国教科文组织GEM报告高级研究分析师)和Patrick montjourid<e:1>博士(联合国教科文组织高级项目专家)在本文初稿中提供的大量反馈意见表示诚挚的感谢。同时,作者感谢伊利诺伊大学香槟分校张金明博士(教授)、张静女士和张颖斌先生(博士生)的支持,以及期刊编辑和匿名审稿人的意见,他们的意见使本文得到了重大改进。任何剩余的错误或遗漏都是作者自己的。披露声明作者未报告潜在的利益冲突。地理位置阿根廷、澳大利亚、加拿大、智利、中国、克罗地亚、塞浦路斯、捷克共和国、丹麦、法国、德国、印度、印度尼西亚、意大利、哈萨克斯坦、立陶宛、马来西亚、荷兰、挪威、巴基斯坦、菲律宾、波兰、俄罗斯、沙特阿拉伯、新加坡、斯洛伐克、斯洛文尼亚、韩国、西班牙、瑞士、泰国、土耳其、英国、越南。数据可用性声明支持本研究结果的数据是可用的,但这些数据的可用性受到限制,这些数据是在当前研究的许可下使用的,因此不能公开获取。但如有合理要求,可向作者索取资料。本文的补充资料可从以下网址获取:https://doi.org/10.1080/0969594X.2023.2270181.Additional information资助本工作得到法国巴黎联合国教科文组织《全球教育监测报告》(合同编号:4500461800)的支持。本文的前一个版本是作为《2023年全球教育监测报告:教育技术:一个工具》的背景研究而编写的。这里的评论和概念代表了作者的观点,不应归因于任何个人或组织。陈丹丹,美国皮尔逊大学副研究员。她拥有伊利诺伊大学厄巴纳-香槟分校教育心理学博士学位,专攻心理测量学。她获得了联合国教科文组织(2022年,2017年)、美国国家科学基金会(2018-2020年)和洛克菲勒慈善顾问(2016年)的资助,并为世界银行、教育委员会、联合国教科文组织和联合国人口基金撰写了系统评论并参与了报告。Amos Jeng是伊利诺伊大学厄巴纳-香槟分校教育心理学系的博士生。他的工作主要集中在教育环境中以学习为导向的互动。他特别感兴趣的是理解和改善大学生在STEM入门课程设置中寻求帮助和给予帮助的体验,以及将这些互动行为置于文化背景中。孙世宇,伊利诺伊大学香槟分校教育心理学系博士生。她的工作是调查大学生在不同情境下的归属感。更具体地说,她对探索国际学生在大学环境中的归属感感兴趣,并了解不同的文化和社会因素如何影响学生的归属感体验。Bradley KapturBradley kapturr是HSHS圣约翰医院放射学的住院医师,曾是伊利诺伊大学厄巴纳-香槟分校卡尔伊利诺伊医学院的医学生。他曾发表过关于医疗保健利用模式和医疗保健获取中的地理空间差异的文章。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Assessment in Education-Principles Policy & Practice
Assessment in Education-Principles Policy & Practice EDUCATION & EDUCATIONAL RESEARCH-
CiteScore
5.70
自引率
3.10%
发文量
29
期刊介绍: Recent decades have witnessed significant developments in the field of educational assessment. New approaches to the assessment of student achievement have been complemented by the increasing prominence of educational assessment as a policy issue. In particular, there has been a growth of interest in modes of assessment that promote, as well as measure, standards and quality. These have profound implications for individual learners, institutions and the educational system itself. Assessment in Education provides a focus for scholarly output in the field of assessment. The journal is explicitly international in focus and encourages contributions from a wide range of assessment systems and cultures. The journal''s intention is to explore both commonalities and differences in policy and practice.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信