Journal of surgical education最新文献

{"title":"Artificial Intelligence-Assisted Narratives: Analysis of Surgical Residency Personal Statements.","authors":"Oswaldo Subillaga, Aixa Pérez Coulter, David Tashjian, Neal Seymour, Daniel Hubbs","doi":"10.1016/j.jsurg.2025.103566","DOIUrl":"10.1016/j.jsurg.2025.103566","url":null,"abstract":"<p><strong>Objective: </strong>Personal statements (PSs) express applicants' personal characteristics and motivations informing pursuit of a surgical career. Generative artificial intelligence (AI) is a revolutionary technology. There are currently no data to suggest how and to what extent AI is used in surgical residency applications. We examined the prevalence of AI use and applicant pool characteristics in PSs submitted to a surgical residency.</p><p><strong>Design: </strong>PSs from US MD and DO applicants to an academic general surgery program were collected for both the 2022-23 and 2023-24 NRMP Match cycles. PSs were analyzed using 2 AI-detection tools: GPTZero and Copyleaks. Data were analyzed using T-test and Fisher's Exact Test.</p><p><strong>Setting: </strong>UMass Chan Medical School-Baystate general surgery residency program in Springfield, Massachusetts.</p><p><strong>Participants: </strong>There were 1332 applications during 2022-23 NRMP Match cycle and 1221 for 2023-24. After excluding international medical graduates and incomplete applications, 1490 PSs were analyzed.</p><p><strong>Results: </strong>1490 PS were included (758 [50.9%] for 2022-23; 732 [49.1%] for 2023-24). Demographic characteristics did not differ between the 2 cycles. GPTZero identified AI use in 77 (10.2%) PSs in 2022-23 and 268 (36.6%) in 2023-24 (p < 0.001). Copyleaks identified AI use in 20 (2.6%) PSs in 2022-23 and 165 (22.5%) in 2023-24 (p < 0.001). Concordance in AI detection with both tools was observed in 13 (1.7% of total PSs) for 2022-23 and 155 (21.2%) for 2023-24 (p < 0.001). Subgroup analysis of concordance in 2023-24 showed increased non-English native language characteristics (38.7% vs 19.6%; p < 0.001), a lower average personal statement word count (597.3 vs 645.9; p < 0.001) and shorter average sentence (10.0 vs 10.4 words; p < 0.001) in the AI group.</p><p><strong>Conclusions: </strong>Although AI-detection tools are imperfect, demonstration of increased AI use in personal statement preparation is compelling. Implications of AI use in residency applications are unknown, and programs must develop policies anticipating ongoing and potentially increased use of AI in the upcoming application cycles.</p>","PeriodicalId":94109,"journal":{"name":"Journal of surgical education","volume":" ","pages":"103566"},"PeriodicalIF":0.0,"publicationDate":"2025-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144334690","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Exploring the Value of Peer Letters of Recommendation in the Holistic Review of Surgery Residency Applications: A Qualitative Study.","authors":"Caitlin Silvestri, Viemma Nwigwe, Subhash Krishnamoorthy, Cary B Aarons","doi":"10.1016/j.jsurg.2025.103580","DOIUrl":"https://doi.org/10.1016/j.jsurg.2025.103580","url":null,"abstract":"<p><strong>Objective: </strong>Residency applications rely on traditional letters of recommendation (tLORs) from faculty or mentors to evaluate applicants. However, interpretation of tLORs can be limited by potential biases, overuse of hyperbolic language, and a lack of longitudinal contact. We aimed to assess whether incorporating peer letters of recommendation (pLORs) would add a complementary perspective to the holistic review of an applicant's attributes and potential.</p><p><strong>Design: </strong>All applicants to a single, university-based general surgery residency program were invited to submit an optional pLOR in the 2023 to 2024 recruitment cycle. Thematic analysis of applicants' pLORs and tLORs was performed to identify patterns and sentiments.</p><p><strong>Setting: </strong>Large general surgery residency program at a single, tertiary academic center.</p><p><strong>Participants: </strong>Applicants selected for an interview for a general surgery residency program who submitted a pLOR in addition to their tLORs (n = 95).</p><p><strong>Results: </strong>Ninety-five applicants (78%) selected for interview submitted a pLOR along with their standard application to the categorical (n = 77) and preliminary (n = 18) tracks. Peer letter writers knew applicants for an average of 6.14 years (SD 4.7). Thematic analysis identified notable differences in pLORs: (1) peer letter writers more often evaluated applicants across diverse settings (professional and personal) over longer time periods, (2) pLORs placed greater emphasis on the applicants' impact on others (peers, individuals, patients), and (3) provided more specific, tangible examples of each positive attribute. Lastly, pLORs summative assessments often included personal language while tLORs tended to stratify applicants using percentiles or coded language.</p><p><strong>Conclusion: </strong>Peer letters of recommendation offer a unique, complementary perspective in the holistic residency application review process. Compared with traditional letters, pLORs provide a richer context of an applicant's impact in a community of their peers, more often providing tangible examples. This perspective is crucial for evaluating applicants as we build diverse and collaborative learning communities each year.</p>","PeriodicalId":94109,"journal":{"name":"Journal of surgical education","volume":" ","pages":"103580"},"PeriodicalIF":0.0,"publicationDate":"2025-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144319042","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Redesign of a Resident Evaluation Tool Using Exploratory Factor Analysis.","authors":"Carly Chappell, Stephen Markowiak, Gang Ren, Laura Wharry, Stephen Stanek, Joseph Sferra","doi":"10.1016/j.jsurg.2025.103477","DOIUrl":"https://doi.org/10.1016/j.jsurg.2025.103477","url":null,"abstract":"<p><strong>Purpose: </strong>Residents at our training program identified timeliness of faculty feedback as an area for improvement, while faculty felt the evaluation tool was time-consuming and redundant. We sought to resolve these issues through stakeholder input and modern data analysis.</p><p><strong>Methods: </strong>Core faculty and senior residents met to revamp the \"Faculty Evaluation of Resident\" end of rotation tool. The results of the most recent 5-years of evaluations were analyzed using Exploratory Factor Analysis, a dimension reduction tool provided in the IBM SPSS statistical package, to identify questions which were highly correlated. A new condensed tool was then generated by combining highly correlated questions with committee approval. Spearman's rank order correlation test was used to evaluate each new question versus the eliminated redundant questions. Time to survey completion and frequency of written feedback were compared using t-test. One-way ANOVA was then used to compare scores for each new question versus the eliminated questions that had been grouped together.</p><p><strong>Results: </strong>3,268 surveys were completed by 73 attendings regarding 55 resident subjects. Data were blinded by the program coordinator before analysis. Exploratory Factor Analysis indicated the initial 30-question instrument could be reduced to 12-questions, while retaining 96% of the variability in performance. The component matrix indicates that 4 areas accounted for the most variability in resident performance: Overall Performance, Communication, Operative Skill, and Systems Based Practice. Following implementation of the new evaluation form, attending surgeons completed resident evaluations at a median of 36 days after the rotation (IQR 18-59). Faculty left written feedback more frequently (53.4% vs 40.8%, p < 0.0001). For some new questions, the resident performances were statistically different. For example, new question 9 had an average rating of 3.55 out of 5.00, while the questions it replaced averaged 3.76-3.89 (p < 0.001). For other questions, no statistically significant difference was found. For example, new question 12 and the questions it replaced all averaged 3.75 out of 5.00 (p = 0.985). Survey response rates also improved from 35% to 76% at 2 months and 89.3% to 93.5% at 6 months.</p><p><strong>Conclusions: </strong>Faculty input and advanced statistical analysis shortened a 30-question resident evaluation tool to 12-questions while retaining 96% of the variability in resident performance. The new instrument resulted in improved response rate and increased number of written comments from attendings. Application of Exploratory Factor Analysis to resident education represents novel use of this tool in surgical education.</p>","PeriodicalId":94109,"journal":{"name":"Journal of surgical education","volume":" ","pages":"103477"},"PeriodicalIF":0.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143538271","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"To Blind or Not to Blind: Evaluating the Impact of Withholding Scores and Grades From Interviewers in General Surgery Resident Recruitment.","authors":"Nicole E Brooks, Judith C French, Jeremy M Lipman, Ajita S Prabhu","doi":"10.1016/j.jsurg.2025.103463","DOIUrl":"https://doi.org/10.1016/j.jsurg.2025.103463","url":null,"abstract":"<p><strong>Objective: </strong>Compare scoring outcomes between interviewers blinded to scores/grades/MSPE and those with the full applicant file to evaluate the effect of blinding on interview scores and ensure applicants can be confidently evaluated when blinding is used.</p><p><strong>Design, setting and participants: </strong>Nineteen interviewers were purposively randomized to receive a complete application or file with all information except applicant grades/MSPE/USMLE score(s) of 90 applicants prior to 218 interviews during 2022 to 2023 general surgery recruitment. Blinding was randomly assigned while ensuring blinded and nonblinded interviews for both interviewers and applicants. Two program leaders involved in study implementation were excluded from blinding. All other aspects of the selection process remained unchanged from historic methods. Each applicant had 3 to 4 interviews. Each interview was scored prior to discussion with other faculty using a 10-point scale. Descriptive and univariate statistics analyzed scoring patterns. Qualitative data regarding the experiences of blinded interviewers was analyzed to generate themes.</p><p><strong>Results: </strong>There were no differences in interview scores or difference from the applicants' mean scores between blinding groups. This remained true for within-applicant analyses and for all but 1 interviewer (95%) for within-interviewer analyses. Between-interviewer score differences were seen for interview scores across all interviewers and when comparing nonblinded vs. nonblinded scores across interviewers, but not when comparing blinded vs. blinded scores across interviewers. Qualitative data support the ability to confidently evaluate interview performance when blinded, frequent practice of \"self-blinding\" to limit bias even when given scores/grades/MSPE, and belief that scores/grades/MSPE are relevant for screening, but the interview has separate priorities.</p><p><strong>Conclusions: </strong>Blinding of interviewers to scores/grades/MSPE did not significantly change interview scoring outcomes. Interviewer experiences support the ability to confidently evaluate interview performance when blinded. Given that negative effects of blinding were not found and prior work supports that bias may be mitigated by blinded interviews, we support its use in residency recruitment.</p>","PeriodicalId":94109,"journal":{"name":"Journal of surgical education","volume":" ","pages":"103463"},"PeriodicalIF":0.0,"publicationDate":"2025-02-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143434757","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Perspectives on Application and Interview Capping in Residency Selection of Surgical Subspecialties.","authors":"Shwetha Mudalegundi, Marisa Clifton, Scott Lifchez, Dawn LaPorte, Saras Ramanathan, Ahmed H Sabit, Fasika Woreta","doi":"10.1016/j.jsurg.2024.04.007","DOIUrl":"10.1016/j.jsurg.2024.04.007","url":null,"abstract":"<p><strong>Objective: </strong>With the advent of virtual interviews, the potential for interview hoarding by applicants became of greater concern due to lack of financial constraints associated with in-person interviewing. Simultaneously, the average number of applications submitted each year is rising. Currently there is no cap to the number of applications or interviews an applicant may complete when applying to residency, with the exception of ophthalmology with a cap of 15 interviews. No studies have assessed the applicants' perspectives on an application or interview cap. We assessed the attitudes of surgical subspecialty applicants towards capping, which may be useful when considering innovations in residency selection.</p><p><strong>Design/setting/participants: </strong>About 1841 applicants to the Johns Hopkins' ophthalmology, urology, plastic surgery, and orthopedic surgery residency programs from the 2022-2023 cycle were invited to respond to a 22-item questionnaire. Statistical analyses of aggregate data were conducted using R.</p><p><strong>Results: </strong>Of the 776/1841 (42%) responses, 288 (40%) were in support of an application cap, while 455 (63%) were in support of an interview cap. Specialty (p < 0.001), gender (p < 0.001), taking a gap year (p = 0.02), medical school region (p = 0.04), and number of interviews accepted off of a waitlist (p = 0.01) were all significantly associated with a difference in opinion regarding an application cap. Specialty (p < 0.001), USMLE Step 1 score (p = 0.004), number of interviews (p < 0.001), and number of programs ranked (p < 0.001) were all significantly associated with a difference in opinion regarding an interview cap. Of those applicants who were in support of the respective caps they believed that on average a cap should consist of 48.1 (16.1) applications and 16.0 (8.0) interviews.</p><p><strong>Conclusions: </strong>Our findings highlight the desire for interview caps among the majority of applicants to surgical subspecialties and thus this innovation may be considered by other specialties in the era of virtual interviews.</p>","PeriodicalId":94109,"journal":{"name":"Journal of surgical education","volume":" ","pages":"1013-1023"},"PeriodicalIF":0.0,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140961225","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Development of a Competency Framework Defining Effective Surgical Educators.","authors":"Neha Sharma, Emily Steinhagen, Jeffrey M Marks, J. Ammori","doi":"10.1016/j.jsurg.2023.11.017","DOIUrl":"https://doi.org/10.1016/j.jsurg.2023.11.017","url":null,"abstract":"","PeriodicalId":94109,"journal":{"name":"Journal of surgical education","volume":"443 ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139020166","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Exploring Medical Students' Perceptions of Peer-to-Peer Interactions Related to Applying to a Surgical Residency.","authors":"Michael Ho-Yan Lee, Yajur Iyengar, Dan Budiansky, P. Veinot, M. Law","doi":"10.1016/j.jsurg.2023.11.007","DOIUrl":"https://doi.org/10.1016/j.jsurg.2023.11.007","url":null,"abstract":"","PeriodicalId":94109,"journal":{"name":"Journal of surgical education","volume":"333 ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139014520","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Self-Assessment Versus Peer-Assessment in Microsurgery Learning: A Comparative Retrospective Study in a Surgery Residents Cohort.","authors":"Eva Deveze, A. Traoré, Nicolas Ribault, D. Estoppey, Benoît Latelise, H. Fournier, N. Bigorre","doi":"10.2139/ssrn.4170638","DOIUrl":"https://doi.org/10.2139/ssrn.4170638","url":null,"abstract":"INTRODUCTION\u0000In surgical learning, self-assessment allows the physician to identify and improve his strong and weak points. However, its scientific validity has yet to be demonstrated. The aim of this study was to analyze if there is a link between self-assessment accuracy and improvement in surgical skills. We make the hypothesis that an accurate self-assessment allows a greater improvement MATERIAL AND METHOD: We set up a retrospective cohort study at the tertiary University Hospital of Angers. Between 2019 and 2021, twenty-eight surgery residents took part into a microsurgery program and were included in the study. For two weeks, they performed anastomosis training on inert material and living anesthetized rats under microscope. Each resident was evaluated during the workshop by senior surgeons on 10 items: movement stability and fluidity, instrument manipulation, needles, dissection, clamp setting, vessel manipulation, suture, checking before clamp removal, checking after clamp removal, watertighness. Self-assessment was performed by the residents with the same grid, at the end of the workshop. Residents' and senior's evaluations were double-blind. We retrospectively analyzed the concordance between senior objective assessment and self-assessment, and the effect of an accurate self-assessment on technical improvement.\u0000\u0000\u0000RESULTS\u0000Data for twenty-five residents were analyzed, 14 were female (56%). The mean age was 29 years. Surgical specialties were orthopedics (44%), maxillofacial surgery (45.4%), neurosurgery (12%), gynecology (4%) and vascular surgery (4%). According to Cohen's kappa coefficient, 14 residents (56%) underestimated themselves, 7 (28%) were concordant with peer-assessment and 4 (16%) overestimated themselves. The concordance between self and peer assessment during sessions was positive for the most objective items, and negative for the most subjective items. Technical skills improvement in term of peer-assessment averages was positive for each item in each group, without statistical differences between groups.\u0000\u0000\u0000CONCLUSION\u0000We found that the ability to self-assess in a fast-track microsurgery module for surgery residents varied according to analyzed gestures. We demonstrated an improvement in term of self-assessment for objective items, and a decrease for subjective items. However, we didn't find any relation between improvement curve and the accuracy of self-assessment.","PeriodicalId":94109,"journal":{"name":"Journal of surgical education","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45978712","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"New Heuristics to Stratify Applicants: Predictors of General Surgery Residency Applicant Step 1 Scores.","authors":"Sarah Lund, Jonathan D. D’Angelo, Anne-Lise D. D’Angelo, Stephanie F. Heller, John Stulak, Mariela Rivera","doi":"10.1016/j.jamcollsurg.2021.07.472","DOIUrl":"https://doi.org/10.1016/j.jamcollsurg.2021.07.472","url":null,"abstract":"","PeriodicalId":94109,"journal":{"name":"Journal of surgical education","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47196197","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Rigorous Curricular Innovation: Development, Integration, and Evaluation of Anatomic Clinical Correlations Module.","authors":"Julia D. Nedimyer, Atsusi Hirumi, J. Cendan","doi":"10.1016/J.JAMCOLLSURG.2020.07.392","DOIUrl":"https://doi.org/10.1016/J.JAMCOLLSURG.2020.07.392","url":null,"abstract":"","PeriodicalId":94109,"journal":{"name":"Journal of surgical education","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1016/J.JAMCOLLSURG.2020.07.392","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41426779","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}