The Process of Undergraduates' Collaboration With a Generative Artificial Intelligence Chatbot: Insights From Conversation Content and Epistemic Network Analysis

Abstract

Background

Generative artificial intelligence (GenAI) chatbots extend transformative impact in higher education. Current research requires more comprehensive evaluations of the collaborative learning fostered by students and GenAI chatbots. However, existing articles have rarely explored the dynamic process of student–AI collaboration in higher education.

Objectives

This study aims to analyse and visualise the changes in the process of undergraduates' collaboration with a GenAI chatbot. The interaction patterns of the collaboration were explored under the perspective of social constructivist learning theory. The differences between student-AI interaction patterns at 5 time points (after 5 lessons) were further compared to show the dynamic collaboration process.

Method

A 9-week course was implemented for 40 Chinese undergraduates, who completed 5 rounds of collaboration with a GenAI chatbot named ERNIE Bot. Employing a designed coding scheme, a total of 6180 codes was collected from the conversation content of each round. Based on the interval data, content analysis and epistemic network analysis (ENA) were conducted.

Results

First, undergraduates gradually became more active and targeted in their collaboration with the GenAI chatbot. Second, the focal points of their collaboration changed from “Comprehension” (the first–third lessons) to “Generation” (the third–fifth lessons), along with different interaction patterns. Notably, the interaction patterns changed more rapidly and prominently during the “Comprehension” phase than the “Generation” phase.

Implications

The findings contribute to understanding the social constructivist learning process within student-AI collaboration in higher education. Practical recommendations for students and educators were offered as well.