Ophthalmic Knowledge Assessment Program Scores and Written Qualifying Examination Performance.

Abstract

Importance The Ophthalmic Knowledge Assessment Program (OKAP) has been widely used to determine readiness to take the high-stakes Written Qualifying Examination (WQE). However, it is unclear how well OKAP performance can predict WQE outcomes. Objective To examine the association between the OKAP and WQE nationally. Design, Setting, and Participants This was a retrospective cohort study conducted from 2017 to 2022. This national data analysis included residents from ophthalmology residency programs in the US. Study data were analyzed June 2023 to December 2024. Exposures OKAP and WQE data. Main Outcomes and Measures The national OKAP examination data and the WQE data were matched. Candidates who had complete data for the OKAP during postgraduate year (PGY) 2, 3, and 4 and completed the WQE were included in the study. Two-sample t tests, logistic regression analysis, receiver operating characteristic curves, and confusion matrix were used to examine the association of various test scores (PGY-2 OKAP, PGY-3 OKAP, PGY-4 OKAP, minimum OKAP, maximum OKAP, average OKAP) as predictors for WQE outcomes (pass or fail). Multiple logistic regression analyses were also used to assess the association of gender, time since graduation, program size, and WQE pass/fail outcomes. Results A total of 1597 residents (907 male [56.8%]) from 117 residency programs were included in this study. First-time pass rate of the WQE was 89% (1418 of 1597 residents). OKAP scores, particularly average OKAP, maximum OKAP, and PGY-4 OKAP scores (all had area under the curve = 0.88), were the most useful predictor of WQE outcomes. PGY-4 OKAP scores were used for the rest of the analysis for simplicity. A PGY-4 OKAP scaled score of 550 corresponded to 94% of candidates passing the WQE. Using PGY-4 OKAP scores to predict passing/failing, the WQE had 90.7% accuracy. Gender (odds ratio [OR], 1.50; 95% CI, 1.01-2.26; P = .048), program size (OR, 1.55; 95% CI, 0.98-2.41; P = .06), and time since residency graduation (OR, 1.00; 95% CI, 1.00-1.00; P = .14) were not included in the logistic regression model investigating the association of PGY-4 OKAP scores with the likelihood of passing the WQE. Conclusions and Relevance Results suggest that OKAP examination scores were associated with passing the WQE after adjusting for program size, gender, and time since residency graduation. Further analyses may be warranted to predict failing the WQE so that potential interventions can be attempted.