ChatGPT and the course vulnerability index

Abstract

AbstractThis article develops an index to measure the level of susceptibility of courses to cheating using ChatGPT (Chat Generative Pre-trained Transformer), an advanced text-based artificial intelligence (AI) language model. It demonstrates the application of the index to a sample of business courses in a mid-sized university. The study finds that the vulnerability index varies across disciplines and teaching modalities. As advanced language models become more common in academic settings and create new educational challenges, the study provides an intuitive and practical mechanism for instructors and academic units to measure and assess the vulnerability of their courses to various language-based predictive models.Keywords: ChatGPTcheating using AIclassroom cheatingcourse vulnerability index AcknowledgmentsWe thank the participants of the Midwest Business Association Administration (MBAA) 2023 conference for their helpful suggestions and comments. We also thank Marc Lafuente for his assistance.Author contributionsConceptualization: N. Adilov.Analysis: N. Adilov, J. W. Cline, H. Hanke, K. Kauffman, L. Meneau, E. Resendez, S. Singh, M. Slaubaugh, N. Suntornpithug.Writing and editing: N. Adilov, J. W. Cline, H. Hanke, K. Kauffman, L. Meneau, E. Resendez, S. Singh, M. Slaubaugh, N. Suntornpithug.Disclosure statementNo potential conflict of interest was reported by the author(s).Notes1 ChatGPT uses multiple layers of “transformers” to understand text, to recognize the patterns, and to predict the next word in the text. ChatGPT has been trained on vast amounts of text data and then “fine-tuned” to be more user-friendly when responding to queries.2 To the best of authors’ knowledge, no other researchers have utilized a similar course vulnerability index. Consequently, we took it upon ourselves to develop the index. However, Adilov and Cline (Citation2023) presented an economic theoretical model and suggested that the value of our course vulnerability index affects the amount of effort a student invests in a course.3 The VI could theoretically increase beyond 100 if an instructor offers extra credit points.4 Hybrid courses are approximately 50% face-to-face and 50% online.5 As an exception to this pattern, ChatGPT’s accuracy in business statistics was at 90%.6 Our recommendations are consistent with the theoretical findings of Adilov and Cline (Citation2023) that the percentage of course requirements where students are able to use AI to cheat should be minimized.