Evaluating classroom response systems in engineering education: Which metrics better reflect student performance?

Abstract

Classroom response systems (CRS) represent an innovative educational technology that can be used to promote active learning and student engagement. This study explores the effectiveness of CRS in enhancing student learning and performance across various engineering courses related to heat engines. During five academic years, CRS have been used weekly in six subjects that belong to two bachelor’s degrees and one master’s degree to promote continuous learning. The performance of the students in the CRS benefits the student by up to + 10% in the final mark. In this period, data from more than 1,000 students has been collected, including key performance indicators (KPIs) derived from Kahoot!, such as the number and correctness of responses and time spent answering, and other evaluations such as theory and numerical problem exams results. The objective of the study is to find the KPI that best identifies the knowledge acquisition by the students. In order to do so, the correlation of the KPI with the final exams is analysed by means of the correlation of determination and the root mean square error. The findings reveal that KPIs based on the number of questions asked exhibit stronger correlations with exam results compared to those based on responses or correctness. The study also highlights the positive impact of CRS on student engagement, classroom dynamics, and exam preparation, supported by student feedback. In essence, this research provides insights into the relationship between CRS metrics and academic outcomes, offering valuable considerations for the integration of technology in engineering education.