Generative AI in Curriculum Design: Empirical Insights Into Model Performance and Educational Constraints

Abstract

This study verifies the ability of large language models (LLMs) to generate a curriculum and develop syllabi for specific courses. We prompted four models to generate two sets of curricula for a bachelor’s degree in Economics and Management. We also generated syllabi for the courses included in the curriculum. We chose five Polish public economics universities offering those degree programs for comparison. Four LLMs were used in this experiment: ChatGPT-3.5, ChatGPT-4, Google Bard, and Gemini. Two of them are multimodal models. The study used an iterative approach, increasing the detail of the prompt in each iteration. The results show that the more specific prompt is given to the LLM, the less accurate the results are. Moreover, the experiment shows that none of the LLMs developed a complete curriculum at a level comparable to that generated by humans. However, LLMs can significantly help create a curriculum and develop syllabi by humans, provided that there is close human–artificial intelligence (AI) collaboration. The results obtained from the AI-assisted curriculum design differ depending on the model. By analyzing the differences between the tools and the real degree programs and syllabi, we determined that multimodal models are better suited for this task than older models.