Journal of Medical Education and Curricular Development最新文献

{"title":"Evaluating Medical Student Engagement in Flipped Classrooms: Insights on Motivation and Peer Learning.","authors":"Khalid Khadawardi, Dahlia Mirdad, Hisham Nasief, Amber Hassan, Humaira Waseem, Ammara Butt, Njoud E Aldardeir, Wed Salah, Abeer F Zakariyah, Abdulrahman Alboog","doi":"10.1177/23821205251320756","DOIUrl":"10.1177/23821205251320756","url":null,"abstract":"<p><strong>Background: </strong>Flipped classrooms have gained popularity for their ability to improve educational outcomes by flipping traditional teaching approaches and fostering interactive learning settings. Our study aims to assess the impact of these classrooms on medical students' motivation, critical thinking skills, and facilitation of collaborative learning.</p><p><strong>Methods: </strong>A quasi-experimental study was conducted to involve quantitative surveys administered to students both before and after the teaching session. A predefined questionnaire was used to assess 5 key outcomes: Motivation, Critical Thinking, Student-centered learning, Learning benefits, and Peer-Assisted Learning. Data were collected at 2 points to capture changes throughout the intervention.</p><p><strong>Results: </strong>The introduction of the flipped classroom approach led to a significant improvement in medical student performance, with average test scores rising from 13.25 ± 2.36 to 16.08 ± 1.5 (<i>P</i> < .05). Students' perceptions of the flipped classroom also improved, with the impact scale score increasing from 41.91 ± 4.9 to 69.71 ± 9.72 (<i>P</i> < .05). There was a statistically significant difference before and after the engagement, suggesting that the flipped classroom approach successfully improved students' motivation, engagement, collaborative learning, and critical thinking skills.</p><p><strong>Conclusion: </strong>Our findings highlight the positive efficacy of flipped classrooms in creating a more interactive and supportive learning environment. The study concludes with recommendations for educators on how to implement best practices to enhance student engagement and improve learning outcomes in flipped classroom settings.</p>","PeriodicalId":45121,"journal":{"name":"Journal of Medical Education and Curricular Development","volume":"12 ","pages":"23821205251320756"},"PeriodicalIF":2.0,"publicationDate":"2025-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11866390/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143524957","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Complex Adaptive Systems Approach to Health Professions Education Transformation: The Case of the University of Missouri-Columbia School of Medicine and an Integrated Quality Improvement-Interprofessional Collaborative Practice Curriculum.","authors":"Katherine Stevenson, Marcel D'Eon, Linda Headrick, Boel Andersson Gäre","doi":"10.1177/23821205251315624","DOIUrl":"https://doi.org/10.1177/23821205251315624","url":null,"abstract":"<p><strong>Objectives: </strong>In 2003, the University of Missouri-Columbia School of Medicine (MU SOM) initiated an integrated quality improvement-interprofessional collaborative practice (QI-ICP) curriculum as part of a larger curriculum renewal process. While exploring a different case study focused on the content of the curriculum, investigators became intrigued by MU SOM's approach to change, specifically, how complex adaptive systems (CAS) thinking may have supported sustained transformational change in curriculum across multiple health professions programs. The primary aim of this study was to elucidate the aspects of MU's experience with transformation that were grounded in CAS. A secondary aim was to explore the usefulness of a CAS-based management framework for organizational transformation using the case of curricular transformation in health professions education at MU.</p><p><strong>Methods: </strong>Data collection involved interviews, with twelve faculty from a variety of programs, and document analysis, including previously published papers and gray literature (websites, organizational charts and planning documents, and faculty development materials).</p><p><strong>Results: </strong>Using abductive analysis, we found that leadership in the health professions programs at MU, informed by earlier learning about organizational sensemaking and CAS theory, addressed all 9 of the characteristics of CAS presented in the initial framework. Additionally, systematic combining revealed the need to adjust the framework to ensure applicability to health professions education. The analysis of MU's experience also offered key insights into how that transformation happened in practice. The CAS framework adjustments make explicit the importance of common purpose and the concept of leadership as an emergent event and make it easier to apply the framework to a broader set of organizational contexts, including health professions education.</p><p><strong>Conclusion: </strong>The use of the adjusted framework, informed by insights from this specific case, may help health professions education programs evaluate past change efforts or plan for future change.</p>","PeriodicalId":45121,"journal":{"name":"Journal of Medical Education and Curricular Development","volume":"12 ","pages":"23821205251315624"},"PeriodicalIF":2.0,"publicationDate":"2025-02-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11869308/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143543990","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"\"YouTube\" for Surgical Training and Education in Donor Nephrectomy: Friend or Foe?","authors":"Mohamed El-Mahrouk, Derar Jaradat, Tim Eichler, Robert Sucher, Christian Margreiter, Andri Lederer, Robert Karitnig, Antonia Geisler, Nora Jahn, Hans Michael Hau","doi":"10.1177/23821205241301552","DOIUrl":"10.1177/23821205241301552","url":null,"abstract":"<p><strong>Background: </strong>The COVID-19 pandemic has accelerated the shift toward e-learning and online education in surgical training. With the increasing prevalence of end-stage chronic kidney disease, kidney transplantation is in high demand. Donor safety is crucial in nephrectomy procedures, highlighting the importance of effective training. This study evaluates the quality and effectiveness of YouTube videos focusing on laparoscopic and robotic donor nephrectomy for surgical education.</p><p><strong>Methods: </strong>On October 24, 2023, searches on YouTube for \"laparoscopic live donor nephrectomy\" and \"robotic live donor nephrectomy\" returned 121 videos, with 63 included in the study. Popularity was evaluated using the Video Power Index (VPI), while reliability and quality were assessed using the LAP-VEGaS Video Assessment Tool and Journal of the American Medical Association (JAMA) benchmark criteria. Additionally, a structured descriptive tool called the \"Live Donor Nephrectomy Completeness (LDNC)\" was created to evaluate the completeness and educational value of procedural technical steps.</p><p><strong>Results: </strong>Out of 63 videos reviewed, laparoscopic surgical procedures were depicted in 71.4% of them, while robotic approaches were shown in 28.6%. Academic backgrounds were associated with 54% of the videos, and individual physician backgrounds with 46%. Mean scores were LAP-VEGaS 9.79 ± 3.87, VPI 6.32 ± 3.31, and LDNC 9.68 ± 1.97. JAMA scores varied, with 34.9% receiving 1 point, 34.9% receiving 2 points, 17.5% receiving 3 points, and 12.7% receiving 4 points. Academic videos scored significantly higher in LAP-VEGaS and LDNC (all <i>p</i> < .01). While LAP-VEGaS, VPI, and LDNC scores correlated significantly (all <i>p</i> < .05), no correlation was found between JAMA score and other scoring systems. Videos with more clicks and likes showed significantly better scores across all measures (all <i>p</i> < .05).</p><p><strong>Conclusion: </strong>Amidst the challenges posed by the pandemic on surgical education, YouTube has emerged as a valuable resource for learning about laparoscopic and robotic donor nephrectomy for living kidney donation. However, the quality and reliability of these videos vary greatly, and many lack thorough reviews, leading to incomplete information. To enhance their educational value, it's proposed that videos undergo professional evaluation before publication and adhere to standardized, structured, and validated scoring systems, ensuring logical structure and improved quality.</p>","PeriodicalId":45121,"journal":{"name":"Journal of Medical Education and Curricular Development","volume":"12 ","pages":"23821205241301552"},"PeriodicalIF":2.0,"publicationDate":"2025-02-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11851798/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143504739","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Shifting Towards Teaching How to Communicate Statistics in Medical Education.","authors":"Lathan Liou, Murray A Mittleman","doi":"10.1177/23821205241279751","DOIUrl":"10.1177/23821205241279751","url":null,"abstract":"<p><p>Despite the widespread inclusion of statistics in medical school curricula as per the Liaison Committee on Medical Education requirements, the statistical competency among medical students and clinicians remains low. A 2007 study of 277 medical residents revealed only 41.1% scored correctly on a statistical knowledge survey, with minimal understanding of key concepts such as confidence intervals and adjusted odds ratios. A more recent 2023 study of 898 clinicians showed similar deficiencies in understanding efficacy, <i>p</i>-values, and discrimination metrics despite high confidence. This perspective argues for a paradigm shift from teaching statistical applications to focusing on statistical communication. We believe current statistics instruction lacks emphasis on communicating statistical results to patients. Teaching statistical concepts as tools for patient communication, rather than extensions of mathematics, can enhance understanding and ensure patients make informed decisions. Reframing statistical education to focus on communication could potentially address traditionally perceived learning barriers, improve understanding, and foster confidence. In this article, we outline several example reframings of teaching classical statistical concepts emphasizing interpretation and communication. Future strategies such as aligning statistics education closer to residency, revising exam content, updating accreditation requirements, and developing standardized communication primers can help ensure future clinicians are well-equipped to practice evidence-based medicine and effectively communicate statistical information in our increasingly data-driven world.</p>","PeriodicalId":45121,"journal":{"name":"Journal of Medical Education and Curricular Development","volume":"12 ","pages":"23821205241279751"},"PeriodicalIF":2.0,"publicationDate":"2025-02-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11848876/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143494086","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Enhancing Professionalism Online (Netiquette) in Medical Schools: A Systematic Scoping Review.","authors":"Donovan Kai Wei Ng, Jonathan Zhen Liang, Ruth Si Man Wong, Vijayprasanth Raveendran, Gillian Li Gek Phua, Warren Fong, Crystal Lim, Jamie Xuelian Zhou, Lalit Kumar Radha Krishna","doi":"10.1177/23821205241255268","DOIUrl":"10.1177/23821205241255268","url":null,"abstract":"<p><strong>Background: </strong>The relaxing of COVID-19 pandemic restrictions has not seen the return to previous in-person teaching formats. As blended training continues to be used, there is emphasis on the need to better appreciate the expectations, etiquette, and professional code of conduct (\"netiquette\") surrounding online learning, especially in light of evidence that poor online professionalism compromises learning and clinical practice.</p><p><strong>Objectives: </strong>This review seeks to map regnant netiquette guidelines in medical schools that will inform and provide preliminary recommendations for a clinically relevant framework.</p><p><strong>Design: </strong>This study is a systematic scoping review (SSR).</p><p><strong>Methods: </strong>Krishna's Systematic Evidence-Based Approach (SEBA)'s Constructivist ontological and Relativist epistemological lens was used to guide this SSR. The SEBA process involves 6 stages, including the <i>Systematic Approach, Split Approach, Jigsaw Perspective, Funneling, Analysis of evidence-based and non-data-driven literature</i>, and <i>Synthesis of the SSR in SEBA</i>.</p><p><strong>Results: </strong>In total, 7941 abstracts were reviewed, 198 full text articles were evaluated, and 83 articles were included. The analysis of the results revealed 4 key domains: (1) current guidelines, (2) manifestations, (3) contributing factors, and (4) implications. This SSR in SEBA highlights variability and gaps in current guidelines and reveals the impact of sociocultural factors on breaches in netiquette. Unsurprisingly, contextual and clinical considerations shape the contributory factors impacting lapses in netiquette and their implications.</p><p><strong>Conclusions: </strong>Based on the data accrued, this article proposes basic guidelines on netiquette and measures to support their effective employment. This includes curricular adaptations, methods of teaching and enhancing engagement with the students and faculty training. Drawing on prevailing studies, it also recommends methods of assessing netiquette, online professionalism, and the learning environment. Suggestions are also made for future areas of study.</p>","PeriodicalId":45121,"journal":{"name":"Journal of Medical Education and Curricular Development","volume":"12 ","pages":"23821205241255268"},"PeriodicalIF":2.0,"publicationDate":"2025-02-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11851755/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143504742","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evaluating the Use of Artificial Intelligence as a Study Tool for Preclinical Medical School Exams.","authors":"Peyton G Sakelaris, Kaitlyn V Novotny, Miriam S Borvick, Gemma G Lagasca, Edward G Simanton","doi":"10.1177/23821205251320150","DOIUrl":"10.1177/23821205251320150","url":null,"abstract":"<p><strong>Background: </strong>The purpose of this 2024 study was to determine if there is an association between the usage of artificial intelligence (AI) to study and exam scores of medical students in the preclinical phase of their schooling.</p><p><strong>Methods: </strong>We created and distributed a survey via an unbiased third-party to students in the class of 2027 at the Kirk Kerkorian School of Medicine at UNLV to evaluate students AI use to study for their preclinical system-based exams. Students were categorized into two groups, those that use AI to study and those who do not. Two-sample t-tests were run to compare the mean exam scores of both groups on six different organ system exams as well as the cumulative final exam score for each group. The group that did use AI was further asked about which AI tools they use and how exactly they use these tools to study for preclinical examinations.</p><p><strong>Results: </strong>The results of the study showed that there is no statistically significant difference in exam scores between students who use AI for study purposes and students who do not. It was also found that most AI users studied with ChatGPT. The most common way users studied was by using AI to simplify and clarify topics they did not understand.</p><p><strong>Conclusions: </strong>Based on the results of this study, we concluded that usage of AI programs for students for medical examinations did not yield a positive or negative effect on students' organ system-based exam scores.</p>","PeriodicalId":45121,"journal":{"name":"Journal of Medical Education and Curricular Development","volume":"12 ","pages":"23821205251320150"},"PeriodicalIF":2.0,"publicationDate":"2025-02-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11851803/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143504744","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Schwartz Rounds in Higher Education Settings: A Systematic Review of the Research with Recommendations.","authors":"Amy Zile, Joel Owen, Hugh Gorick, Amy Orford, Georgia Panagiotaki","doi":"10.1177/23821205251320152","DOIUrl":"10.1177/23821205251320152","url":null,"abstract":"<p><strong>Background: </strong>Higher education institutions (HEIs) have recently integrated Schwartz Rounds (Rounds) as innovative ways of supporting health professional students' reflective practice, interprofessional learning and compassionate care. Emerging evidence suggests that Rounds benefit students' shared sense of belonging and insight into emotional aspects of training and care, providing safe places to share experiences and reflect. Our review examines the current literature regarding health students' evaluation of Rounds in HEIs, exploring experiences and perceived benefits of participating. We consider methodological issues with the existing literature and propose ways of addressing these in future research.</p><p><strong>Methods: </strong>Systematic searches were conducted on EBSCO, PsycINFO, PubMed and ASSIA to identify available literature relating to Schwartz Rounds in higher education. Eight studies - six with a mixed method design and two qualitative - were included in the review. Study quality and risk of bias were assessed. Evaluation survey data and themes generated from qualitative responses were considered together and a narrative synthesis was constructed.</p><p><strong>Results: </strong>Despite challenges with differing measurements and scale usage, five themes were identified reflecting the experience and perceived benefits for students attending Rounds. Students highlighted the value of peer connection and communication; normalisation of emotions; collaborative reflection and self-awareness; and the relevance of Rounds in their daily work and patient care. Students also commented on barriers to participation including Rounds' multidisciplinary nature, their group size, and the presence of staff. The majority of students rated Rounds as excellent or exceptional and agreed they should be integrated into the curriculum.</p><p><strong>Conclusions: </strong>Our findings on Schwartz Rounds in higher education suggest they support student reflective practice and confidence in providing compassionate, high-quality patient care. However, the quality of research in this area is mixed, reflecting the relatively recent integration of Rounds in Higher education, and the challenges of implementing and evaluating new interventions in educational settings.</p>","PeriodicalId":45121,"journal":{"name":"Journal of Medical Education and Curricular Development","volume":"12 ","pages":"23821205251320152"},"PeriodicalIF":2.0,"publicationDate":"2025-02-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11851745/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143504746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Impact of a USMLE Step 2 Prediction Model on Medical Student Motivations.","authors":"Anthony Shanks, Ben Steckler, Sarah Smith, Debra Rusk, Emily Walvoord, Erin Dafoe, Paul Wallach","doi":"10.1177/23821205251321812","DOIUrl":"10.1177/23821205251321812","url":null,"abstract":"<p><strong>Purpose: </strong>With the transition of USMLE Step 1 to Pass/Fail, Step 2 CK carries added weight in the residency selection process. Our goal was to develop a Step 2 predicted score to provide to students earlier in medical school to assist with career mentoring. We also sought to understand how the predicted scores affected student's plans.</p><p><strong>Method: </strong>Traditional statistical models and machine learning algorithms to identify predictors of Step 2 CK performance were utilized. Predicted scores were provided to all students in the Class of 2024 at a large allopathic medical school. A cross-sectional survey was conducted to assess if the estimated score influenced career or study plans.</p><p><strong>Results: </strong>The independent variables that resulted in the most predictive model included CBSE score, Organ System course exam scores and Phase 2 (Third Year Clinical Clerkships) NBME percentile scores (Step2CK = 191.984 + 0.42 (CBSE score) + 0.294 (Organ Systems) + 0.409 (Average NBME). The standard error of the prediction model was 7.6 with better accuracy for predicted scores greater than 230 (SE 8.1) as compared to less than 230 (SE 12.8). Nineteen percent of respondents changed their study plan based on the predicted score result. Themes identified from the predicted score included reassurance for career planning and the creation of anxiety and stress.</p><p><strong>Conclusion: </strong>A Step 2 Predicted Score, created from pre-existing metrics, was a good estimator of Step 2 CK performance. Given the timing of Step 2 CK, a predicted score would be a useful tool to counsel students during the specialty and residency selection process.</p>","PeriodicalId":45121,"journal":{"name":"Journal of Medical Education and Curricular Development","volume":"12 ","pages":"23821205251321812"},"PeriodicalIF":2.0,"publicationDate":"2025-02-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11837067/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143459893","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Community Preceptors' Views on the Training Needs of Medical Students and Factors Affecting Medical Education in the Community Setting: An Exploratory Study.","authors":"Megumi Yasuda, Takuya Saiki, Chihiro Kawakami, Rintaro Imafuku","doi":"10.1177/23821205251317152","DOIUrl":"10.1177/23821205251317152","url":null,"abstract":"<p><strong>Objectives: </strong>To identify community preceptors' views on the qualities and abilities required of physicians in community medicine, the education to acquire those qualities and abilities, and the factors affecting preceptors' involvement in community-based teaching.</p><p><strong>Methods: </strong>We conducted semi-structured interviews with 13 male community preceptors in northern Japan who had at least 5 years of experience in community medicine and prior teaching experience with preclinical students. Participants were selected using purposive sampling to ensure diversity in clinical roles and institutional affiliations. A thematic analysis was conducted on verbatim transcripts to identify recurring themes.</p><p><strong>Results: </strong>Participants emphasized \"communication skills,\" \"understanding the perspectives and backgrounds of patients and their families,\" and \"love for the community and its people\" among the essential qualities and abilities for community physicians. They cited deepening relationships with and fostering an attachment to the community as necessary to cultivate these skills. The factors identified as facilitating community-based teaching included \"positive emotions felt throughout the teaching experience,\" \"supporting hospitals and the wider community,\" and \"devotion to students.\" Factors that hindered involvement included \"perceived difficulty of teaching preclinical students\" and \"education-related issues at universities assigning students on placement,\" such as the lack of opportunities for community-based teaching within the curriculum.</p><p><strong>Conclusions: </strong>In line with the \"social axis\" in Worley's four Rs model, deep community involvement is essential to developing communication skills and patient-centered medicine. This study identified the qualities and abilities required for community physicians, as well as the educational activities needed to cultivate them from the early years of medical school. Strengthening collaboration between universities and community preceptors is vital to advancing community-engaged medical education (CEME). Cross-cultural studies could further explore how CEME operates in diverse contexts, contributing to socially accountable and community-responsive medical education.</p>","PeriodicalId":45121,"journal":{"name":"Journal of Medical Education and Curricular Development","volume":"12 ","pages":"23821205251317152"},"PeriodicalIF":2.0,"publicationDate":"2025-02-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11837060/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143459622","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Program Evaluation in Competence by Design: A Mixed-Methods Study.","authors":"Jenna Milosek, Kaylee Eady, Katherine A Moreau","doi":"10.1177/23821205251321791","DOIUrl":"10.1177/23821205251321791","url":null,"abstract":"<p><strong>Background: </strong>The evaluation of Competence by Design (CBD) residency programs is crucial for enhancing program effectiveness. However, literature on evaluating CBD programs is limited. We conducted a 2-phase mixed-methods study to (a) assess the extent of program evaluation activities in CBD residency programs in Canada, (b) explore reasons for engaging or not engaging in these activities, (c) examine how CBD programs are conducting program evaluations, and (d) identify ways to build capacity for program evaluation.</p><p><strong>Methods: </strong>Phase 1 involved surveying 149 program directors from specialty/subspecialty programs that transitioned to CBD between 2017 and 2020. We calculated descriptive statistics for 22 closed-ended survey items. Phase 2 comprised interviews with a subset of program directors from Phase 1. Data analysis followed a 3-step iterative process: data condensation, data display, and drawing and verifying conclusions.</p><p><strong>Results: </strong>In Phase 1, we received 149 responses, with a 33.5% response rate. Of these, 127 (85.2%) indicated their programs engage in evaluation, while 22 (14.8%) do not. Among the 127 programs that engage in evaluation, 29 (22.8%) frequently or always develop evaluation questions, and 23 (18.1%) design evaluation proposals/plans. Reasons for engaging in evaluation included decision-making and stimulating changes in educational practices. Conversely, reasons for not engaging included lack of knowledge, personnel, and funding. In Phase 2, 15 program directors were interviewed. They reported that CBD programs face challenges such as limited resources and buy-in, rely on ad hoc evaluation methods, and use a team-based evaluation format. To enhance evaluation capacities, interviewees suggested (a) developing expertise in program evaluation, (b) acquiring evaluation resources, and (c) advocating for clear evaluation expectations.</p><p><strong>Conclusions: </strong>Most CBD residency programs are engaged in program evaluations, but the quality is often questionable. To fully realize the potential of program evaluation, CBD programs need additional resources and support to improve evaluation practices and outcomes.</p>","PeriodicalId":45121,"journal":{"name":"Journal of Medical Education and Curricular Development","volume":"12 ","pages":"23821205251321791"},"PeriodicalIF":2.0,"publicationDate":"2025-02-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11833823/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143449658","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}