From Imitation Games to Robot-Teachers: A Review and Discussion of the Role of LLMs in Computing Education

Abstract

Background

The recent advent of powerful, exam-passing large language models (LLMs) in public awareness has led to concerns over students cheating, but has also given rise to calls for including or even focusing education on LLMs. There is a perceived urgency to react immediately, as well as claims that AI-based reforms of education will lead to a broadening of accessibility to high-quality education.

Objectives

We review and discuss three major themes that appear in the research literature on LLMs and computing education, namely that (i) LLMs exhibit human-like performance and can pass exams, (ii) LLMs are freely available and intuitive to use, and (iii) students use LLMs to cheat or accept the results without critical evaluation. Moreover, we highlight the importance of a more human-centric view on the topic.

Methods

The discussion is based on a review of the (research) literature in the fields related to computing education, picks up claims and statements from the literature, and compares them with research findings from the area. By making some of the rather tacit premises more explicit and putting them into context, we aim to base the discourse about AI in education on more solid grounds.

Results and Conclusion

We find that claims such as the broadening of accessibility to high-quality education or calls for urgent educational reforms are not supported by evidence. Furthermore, we argue that there is a central human element in education that cannot be automated or replaced by AI tools.