Using GenAI to Assess Design Patterns in Student-Written Code

Abstract

As programming education scales, evaluating student code becomes increasingly challenging. In object-oriented programming (OOP) courses, design patterns are crucial for teaching maintainable reusable solutions that meet industry standards. While traditional automated assessment tools have successfully addressed code correctness and quality, automating the detection of design patterns presents a distinct challenge. Current methods, such as static code analysis combined with graph similarity, have proven effective for library code but often struggle with the variability of student submissions. This article investigates the application of generative artificial intelligence (GenAI) models to improve accuracy in detecting design patterns in student-written code. Our research addresses two key questions: 1) Which of the current GenAI models offer optimal performance, accuracy, and reasoning capabilities for design pattern assessment? and 2) How does a cloud-based Software as a Service solution (such as ChatGPT) compare to a cloudlet solution (local model deployed on the University’s cluster) in terms of reliability and scalability? We assess the effectiveness of these approaches using a representative sample of student assignments specifically crafted to require design pattern implementation. Our findings discuss the educational utility of GenAI in reducing instructors’ grading burdens, enhancing students’ self-assessment opportunities, and its potential to guide industry practitioners in design pattern evaluation. We highlight human-in-the-loop supervision to ensure that GenAI recommendations align with educational standards and advocate for open-source cloudlets as a scalable cost-effective infrastructure for universities. Our study provides guidelines for integrating GenAI into OOP coursework and highlights the implications for teaching, industry applicability, and future research directions.