Knowledge Self-Assessment Engagement and Family Medicine Board Examination Outcomes.

Abstract

Background and objectives: Evidence on the relationship between formative assessment engagement and summative assessment outcomes in practicing physicians is sparse. We evaluated the relationship between engagement in the American Board of Family Medicine (ABFM) formative Continuous Knowledge Self-Assessment (CKSA) and performance on high-stakes summative assessments.

Methods: This retrospective cohort study included 24,926 ABFM diplomates who completed CKSA modules and summative assessments between 2017 and 2023. We analyzed CKSA engagement metrics-such as the number of quarters completed, time of completion, and self-reported confidence-against performance on summative assessments, measured by z scores. Multivariable regression models controlled for demographic factors and prior assessment performance.

Results: The overall cohort summative assessment pass rate during the study period was 90.3%. Greater CKSA engagement was strongly associated with higher summative assessment performance. Diplomates who completed all four CKSA quarters had significantly higher summative assessment z scores than those completing fewer quarters (P<.001). Early CKSA completion and spending more time on low-confidence questions were also positively correlated with both CKSA and summative assessment scores (P<.001). These effects were observed across different levels of prior exam performance.

Conclusions: Engagement in formative assessments like CKSA, particularly early and consistent participation and reviewing incorrect or low-confidence questions, is linked to better outcomes on high-stakes assessments. Future research should explore the mechanisms underlying these associations and consider developing an index of engagement to identify physicians at risk of poor performance. Incorporating structured, longitudinal self-assessments like CKSA into certification requirements could enhance continuous learning and improve summative exam readiness.