Machine learning–driven analysis of student evaluation comments: Advancing beyond manual coding through a combined approach

Abstract

Introduction

This study examines pharmacy students' qualitative faculty and course evaluation (FCE) feedback through an integrated machine learning and human coding approach to uncover insights on faculty teaching, course quality, and areas for improvements, informing instructional enhancement.

Methods

Between 2019 and 2023, text data from 1267 FCEs were compiled and analyzed using WordStat, a text mining software. The content analysis primarily relied on machine learning techniques, including word clustering, word co-occurrence mapping, phrase extraction, and topic modeling, to uncover patterns in the student feedback data. To enhance interpretive depth and ensure contextual accuracy, a supplemental manual thematic analysis was conducted using both deductive and inductive coding approaches. Descriptive statistics were applied to quantify and interpret the frequency of identified codes and themes.

Results

Word cluster analysis identified commonly cited words and their co-occurrences, including professor, class, students, teaching, great, materials, and lectures. The frequently occurring phrases included excellent professor, great professor, excellent teaching style, knowledgeable professors, caring professors, flexible with students, and goes extra miles. The topics with high coherence values included understanding the materials, great professors, real-life experience, knowledgeable professor, excellent content, waste of time, and reading the slides. The manual coding analysis identified 1088 codes grouped under 38 subthemes constituting three major themes including faculty personal attributes (45.86 % of codes), faculty teaching effectiveness (28.92 %), and course quality (23.24 %).

Conclusions

This study highlights the value of analyzing open-ended FCE comments by utilizing machine learning to gain meaningful insights that deepen understanding of the student learning experience. Educators and curriculum planners in health professions education can make data-informed decisions, improve curriculum design, and enhance teaching effectiveness by thoughtfully integrating student feedback into program-level reviews.