A Grade for Artificial Intelligence: A Study on School Teachers' Ability to Identify Assignments Written by Generative Artificial Intelligence.

Abstract

Artificial intelligence (AI) is rapidly advancing across various sectors, including education. However, AI in education raises ethical concerns, for example, regarding the originality of students' homework, which could affect both learning outcomes and student-teacher's trust. Despite AI's potential benefits, many teachers feel skeptical about its use, fearing that students may use it unfairly. This study aims to explore teachers' ability to assess the originality of student assignments and identify AI-generated content, taking into consideration teachers' expertise, self-efficacy, and personality. A sample of 67 middle and high-school teachers evaluated six short assignments, half written by real students and half by AI (ChatGPT 3.5). t Tests and analysis of variance were conducted to compare the identification accuracy of assignments and the relationship with teachers' expertise, and regressions were performed to examine the relationships between identification accuracy, personality traits, and self-efficacy in detecting originality. Teachers were able to identify AI-generated assignments but struggled with student-generated ones. Furthermore, teachers with more expertise exhibited a potential bias against students, mistakenly identifying their work as AI-generated. While teachers were able to evaluate assignments objectively, openness and conscientiousness predicted their self-efficacy in assessing originality. We discuss how educators may learn new opportunities to use generative AI to promote learning and engagement. Although students may leverage AI to minimize their workload, AI represents a way to support them during the learning process, if it is developed taking into account students' and teachers' needs and characteristics.