Multistakeholder Assessment of Project-Based Service-Learning in Medical Education: A Comparative Evaluation.

Abstract

Purpose: Traditional single-assessment models in service-learning courses do not facilitate comprehensive assessments of learning outcomes. Effective assessments should incorporate perspectives from multiple stakeholders. The present study developed a service-learning course assessment model that incorporates assessments from multiple stakeholders, compared assessments between stakeholder types, and explored the effects of evaluator-student relationship.

Participants and methods: The study recruited 126 students from a service-learning course at China Medical University in 2024. Six different groups of stakeholders, namely peers, teaching assistants, service institutions, primary instructors, group instructors, and final report evaluators, evaluated student performance and learning outcomes. Experts ensured that assessment criteria were relevant and comprehensive. Confirmatory factor and principal component analyses were performed to assess the construct validity. The study used descriptive statistics and performed interrater reliability and correlation analyses.

Results: The six groups of evaluators were mostly consistent in their assessments, which clustered into two distinct factors: individual performance (Factor 1) and team/service performance (Factor 2). Factor 1 comprised evaluations from peers, teaching assistants, primary instructors, and group instructors, emphasizing individual students' attendance, participation, and contribution throughout the course. Factor 2 comprised evaluations from service institutions and final report evaluators, focusing on group-level service outcomes and teamwork effectiveness. These two factors explained a cumulative variance of 77.94%. The study identified 15 correlation coefficients: 8 were significantly positive-indicating agreement within or across factors; 2 were significantly negative-highlighting potential divergences in perspective; and 5 were nonsignificant. The relationship between evaluator and student significantly affected assessment outcomes. For instance, peer assessments were the most variable due to subjective influences such as interpersonal dynamics and collaboration history, whereas group instructor assessments showed the least variability, possibly due to a more outcome-focused evaluation approach.

Conclusion: Assessments by different types of evaluators are relatively consistent, and the evaluator-student relationship influences assessment outcomes.