Quality assessment of OSCE-style examinations in a chiropractic program: A follow-up report.

Abstract

Objective: This study aimed to evaluate the quality of 3 objective structured clinical examinations (OSCEs) administered at a chiropractic college in 2024. The focus was on applying postexamination quality assurance methods suitable for small cohorts to assess examiner consistency, scoring patterns, and how well the exams distinguished between different levels of student performance.

Methods: Data were collected from 3 summative OSCEs: Clinic Entrance, Chiropractic Management, and Introduction to Chiropractic Practice. Borderline regression was used to calculate cut-scores. Additional QA metrics included number of failures, coefficient of determination (R2), intergrade discrimination, and between-group variation.

Results: Cut-scores for most OSCE stations were above the standard 70% threshold, indicating generally strong student performance. The exception was the neurological station in the Clinic Entrance exam, which had a cut-score of 66.7% and a failure rate of 53.1 percent. R2 values ranged from 0.40-0.79, with several stations showing moderate to strong alignment between checklist scores and overall examiner judgments. Between-group variation ranged from 40.9-79.5, suggesting examiner consistency varied across stations. Intergrade discrimination was low overall, indicating limited discrimination between performance levels and some examiner inconsistency.

Conclusion: Evidence-informed analysis of examinations is an important aspect of quality control in teaching. These findings suggest that OSCE quality is constrained more by examiner inconsistency and weak intergrade discrimination than by student performance per se. Targeted calibration of examiners and program-level redesign of station checklists and global rating scales are priorities for quality improvement.