Evaluation of Students’ Learning Through Reflection on Doing Based on Sentiment Analysis

Abstract

In this paper, we address the following question: How can instructors leverage assessment instruments used to process information gleaned from design, build, and test courses to simultaneously improve student outcomes and assess student learning well enough to improve the course in the future? A Take-away is an unstructured text written by students in AME4163: Principles of Engineering Design offered at the University of Oklahoma, Norman, USA, to record what they understand by reflecting on authentic and immersive experiences throughout the semester. The immersive experiences include lectures, assignments, reviews, building, testing, and a post-analysis for the design of an electro-mechanical system to address a given customer need. In the context of a Take-away, a student then writes a Learning Statement that is a structured sentence written as a triple, i.e., [Experience - Learning - Value]. Between 2019–2021, we collected about 10,000 Take-aways and Learning Statements from almost 400 students. In this paper, we address the question from the perspective of students’ feelings and use dictionary-based sentiment analysis to evaluate students’ subjective feelings toward what they are learning. Through quantitative results, we provide evidence-based guidance to instructors on improving the delivery of the course in the future. Our focus in this paper is on explaining the method using data from AME4163. The proposed method is general and can be extended for other types of courses.