Structured oral clinical assessment for pharmacotherapy competencies in medical education: a study of validity and reliability analyses of seven domains.

Purpose: To develop and validate a structured oral clinical assessment (SOCA) tool to evaluate pharmacotherapy competencies among medical students during clinical clerkships.

Methods: The instrument was designed around seven core pharmacotherapy domains. A total of 30 pharmacology experts participated in the face and content validity assessments. The cognitive complexity was evaluated using Bloom's taxonomy. Nine trained examiners assessed 77 clerkship students using the SOCA tool. Construct validity was tested using Spearman correlation, whereas inter-rater reliability was analyzed using Cohen's kappa and Krippendorff's alpha. Internal consistency was assessed using Cronbach's alpha.

Results: All items showed strong content validity (item-level content validity index and face validity index=1.00). Most questions reflected moderate to high cognitive complexity (Bloom's C2-C5). Construct validity was supported by significant domain-total score correlations (r=0.406-0.750; p<0.05). Inter-rater reliability was substantial (kappa=0.651-0.830; Krippendorff's alpha=0.639-0.834), and internal consistency was acceptable (Cronbach's alpha=0.759).

Conclusion: The SOCA tool has strong validity and reliability for evaluating pharmacotherapy competence through oral clinical examination. It offers a structured, feasible alternative to existing formats and has the potential for broader use following external validation.