Chanon Thanitcul,Sarah Schnabel,George B Bartley,Andrew G Lee,Hally Crump,Dale E Fajardo,Kathryn Peters,Beth Wilson,Divya Srikumaran
{"title":"Ophthalmic Knowledge Assessment Program Scores and Written Qualifying Examination Performance.","authors":"Chanon Thanitcul,Sarah Schnabel,George B Bartley,Andrew G Lee,Hally Crump,Dale E Fajardo,Kathryn Peters,Beth Wilson,Divya Srikumaran","doi":"10.1001/jamaophthalmol.2025.1933","DOIUrl":null,"url":null,"abstract":"Importance\r\nThe Ophthalmic Knowledge Assessment Program (OKAP) has been widely used to determine readiness to take the high-stakes Written Qualifying Examination (WQE). However, it is unclear how well OKAP performance can predict WQE outcomes.\r\n\r\nObjective\r\nTo examine the association between the OKAP and WQE nationally.\r\n\r\nDesign, Setting, and Participants\r\nThis was a retrospective cohort study conducted from 2017 to 2022. This national data analysis included residents from ophthalmology residency programs in the US. Study data were analyzed June 2023 to December 2024.\r\n\r\nExposures\r\nOKAP and WQE data.\r\n\r\nMain Outcomes and Measures\r\nThe national OKAP examination data and the WQE data were matched. Candidates who had complete data for the OKAP during postgraduate year (PGY) 2, 3, and 4 and completed the WQE were included in the study. Two-sample t tests, logistic regression analysis, receiver operating characteristic curves, and confusion matrix were used to examine the association of various test scores (PGY-2 OKAP, PGY-3 OKAP, PGY-4 OKAP, minimum OKAP, maximum OKAP, average OKAP) as predictors for WQE outcomes (pass or fail). Multiple logistic regression analyses were also used to assess the association of gender, time since graduation, program size, and WQE pass/fail outcomes.\r\n\r\nResults\r\nA total of 1597 residents (907 male [56.8%]) from 117 residency programs were included in this study. First-time pass rate of the WQE was 89% (1418 of 1597 residents). OKAP scores, particularly average OKAP, maximum OKAP, and PGY-4 OKAP scores (all had area under the curve = 0.88), were the most useful predictor of WQE outcomes. PGY-4 OKAP scores were used for the rest of the analysis for simplicity. A PGY-4 OKAP scaled score of 550 corresponded to 94% of candidates passing the WQE. Using PGY-4 OKAP scores to predict passing/failing, the WQE had 90.7% accuracy. Gender (odds ratio [OR], 1.50; 95% CI, 1.01-2.26; P = .048), program size (OR, 1.55; 95% CI, 0.98-2.41; P = .06), and time since residency graduation (OR, 1.00; 95% CI, 1.00-1.00; P = .14) were not included in the logistic regression model investigating the association of PGY-4 OKAP scores with the likelihood of passing the WQE.\r\n\r\nConclusions and Relevance\r\nResults suggest that OKAP examination scores were associated with passing the WQE after adjusting for program size, gender, and time since residency graduation. Further analyses may be warranted to predict failing the WQE so that potential interventions can be attempted.","PeriodicalId":14518,"journal":{"name":"JAMA ophthalmology","volume":"13 1","pages":""},"PeriodicalIF":7.8000,"publicationDate":"2025-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"JAMA ophthalmology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1001/jamaophthalmol.2025.1933","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"OPHTHALMOLOGY","Score":null,"Total":0}
引用次数: 0
Abstract
Importance
The Ophthalmic Knowledge Assessment Program (OKAP) has been widely used to determine readiness to take the high-stakes Written Qualifying Examination (WQE). However, it is unclear how well OKAP performance can predict WQE outcomes.
Objective
To examine the association between the OKAP and WQE nationally.
Design, Setting, and Participants
This was a retrospective cohort study conducted from 2017 to 2022. This national data analysis included residents from ophthalmology residency programs in the US. Study data were analyzed June 2023 to December 2024.
Exposures
OKAP and WQE data.
Main Outcomes and Measures
The national OKAP examination data and the WQE data were matched. Candidates who had complete data for the OKAP during postgraduate year (PGY) 2, 3, and 4 and completed the WQE were included in the study. Two-sample t tests, logistic regression analysis, receiver operating characteristic curves, and confusion matrix were used to examine the association of various test scores (PGY-2 OKAP, PGY-3 OKAP, PGY-4 OKAP, minimum OKAP, maximum OKAP, average OKAP) as predictors for WQE outcomes (pass or fail). Multiple logistic regression analyses were also used to assess the association of gender, time since graduation, program size, and WQE pass/fail outcomes.
Results
A total of 1597 residents (907 male [56.8%]) from 117 residency programs were included in this study. First-time pass rate of the WQE was 89% (1418 of 1597 residents). OKAP scores, particularly average OKAP, maximum OKAP, and PGY-4 OKAP scores (all had area under the curve = 0.88), were the most useful predictor of WQE outcomes. PGY-4 OKAP scores were used for the rest of the analysis for simplicity. A PGY-4 OKAP scaled score of 550 corresponded to 94% of candidates passing the WQE. Using PGY-4 OKAP scores to predict passing/failing, the WQE had 90.7% accuracy. Gender (odds ratio [OR], 1.50; 95% CI, 1.01-2.26; P = .048), program size (OR, 1.55; 95% CI, 0.98-2.41; P = .06), and time since residency graduation (OR, 1.00; 95% CI, 1.00-1.00; P = .14) were not included in the logistic regression model investigating the association of PGY-4 OKAP scores with the likelihood of passing the WQE.
Conclusions and Relevance
Results suggest that OKAP examination scores were associated with passing the WQE after adjusting for program size, gender, and time since residency graduation. Further analyses may be warranted to predict failing the WQE so that potential interventions can be attempted.
期刊介绍:
JAMA Ophthalmology, with a rich history of continuous publication since 1869, stands as a distinguished international, peer-reviewed journal dedicated to ophthalmology and visual science. In 2019, the journal proudly commemorated 150 years of uninterrupted service to the field. As a member of the esteemed JAMA Network, a consortium renowned for its peer-reviewed general medical and specialty publications, JAMA Ophthalmology upholds the highest standards of excellence in disseminating cutting-edge research and insights. Join us in celebrating our legacy and advancing the frontiers of ophthalmology and visual science.