Surgical Professionalism: Development and Educational Suitability Assessment of a Combined Situational Judgment Test and Guided Reflection among General Surgery Residents in Indonesia.

Abstract

Introduction: Assessing professionalism in a valid, objective, and cost-effective manner remains a persistent challenge in clinical education. In Indonesia, this issue is particularly relevant in surgical training, where residents must integrate technical competence with professional behaviour in high-stakes, resource-limited settings. This study aimed to develop and evaluate the educational suitability of a Situational Judgement Test (SJT) combined with guided written reflection to assess and promote professionalism among general surgery residents.

Methods: A mixed-methods sequential design was used. An SJT was developed through blueprinting, item writing, and expert validation, resulting in 26 scenarios across six professionalism domains. Residents (n = 64 at baseline; n = 44 completers) completed a proctored pre-test SJT, participated in an 8-week guided reflection program (four submissions), and completed an asynchronous post-test SJT. Educational suitability evidence was examined through content validity (CVI) and internal consistency (Kendall's W). No construct or criterion validity testing was performed. Reflections were scored using Kember's four-level reflective framework by three independent assessors with Fleiss' Kappa for inter-rater agreement. Changes in SJT and reflection depth were analysed using paired tests and correlation analysis. Data were analysed using SPSS version 23.

Results: The mean CVI across items was 0.97 (range 0.90-1.00). The mean SJT score increased significantly from 386.59 ± 19.31 to 420.45 ± 12.56 among completers, with a mean difference of 33.86 ± 16.03 (p < 0.001). Reliability improved from Kendall's W = 0.38 to 0.52. Sensitivity analysis comparing baseline pre-test scores between completers (386.6 ± 19.3) and dropouts (391.0 ± 15.5) showed no significant difference (Mann-Whitney U, p = 0.258), indicating limited attrition bias. Reflective depth progressed from median level 1 to 4 across four submissions (median Δ = +3, p < 0.001), with substantial inter-rater agreement (κ = 0.68). There was no significant correlation between Δ reflection and Δ SJT (r = 0.059, p = 0.703).

Conclusions: Integrating SJT with guided reflection demonstrated strong content validity, internal consistency, and educational benefit, supporting its suitability for formative educational use in surgical professionalism training.