Capturing the Process of Students' AI Interactions When Creating and Learning Complex Network Structures

Abstract

Despite the growing use of large language models (LLMs) in educational contexts, there is no evidence on how these can be operationalized by students to generate custom datasets suitable for teaching and learning. Moreover, in the context of network science, little is known about whether LLMs can replicate real-life network properties. This study addresses these gaps by evaluating the use of generative artificial intelligence (AI), specifically LLMs, to create synthetic network datasets for educational use. The analyzed data include students’ AI-generated network datasets, their interactions with the LLMs, and their perceptions and evaluations of the task's value. The results indicate that the LLM-generated networks had properties closer to real-life networks, such as higher transitivity, network density, and smaller mean distances compared to randomly generated networks. Thus, our findings show that students can use LLMs to produce synthetic networks with realistic structures while tailoring to the individual preferences of each student. The analysis of students’ interactions (prompts) with the LLMs revealed a predominant use of direct instructions and output specifications, with less emphasis on providing contextual details or iterative refinement of the LLM's responses, which highlights the need for AI literacy training to optimize students’ use of generative AI. Students’ perceptions of the use of AI were overall positive; they found using LLMs time saving and beneficial, although opinions on output relevance and quality varied, especially for assignments requiring replication of specific networks.