Moving Beyond Simplistic Research Design in Health Professions Education: What a One-Group Pretest-Posttest Design Will Not Prove.

Introduction: Educational research presents unique study design challenges. Novice researchers in health professions education (HPE) frequently misuse the one-group pretest-posttest design, highlighting the need for improved training in research design. This workshop aimed to enhance understanding of research design among novice HPE researchers, specifically addressing the inherent limitations of the one-group pretest-posttest design and offering alternative approaches.

Methods: Experienced HPE researchers developed this workshop to address common misunderstandings of research design. Leaders from the AAMC Medical Education, Scholarship, Research, and Evaluation section facilitated 60-75-minute workshops conducted at the four 2024 regional meetings hosted by the AAMC Group on Educational Affairs (GEA). Workshop activities included large-group discussion, small-group case-based discussion, and critiques of research designs. Participants discussed internal validity threats and alternative research designs and scholarly approaches.

Results: Approximately 120 GEA regional meeting registrants attended, with 74 (61%) completing a feedback questionnaire immediately after the workshop. Most respondents reported achieving the workshop's educational objectives, such as being better able to identify internal validity threats associated with the one-group pretest-posttest design (100%) and to discuss alternative approaches to evaluate educational innovations (100%). Additionally, >95% of respondents agreed that the workshop was well organized, interactive, and valuable in providing content they could apply to their educational scholarship.

Discussion: The workshop successfully clarified misconceptions surrounding the one-group pretest-posttest design while introducing participants to more rigorous research approaches. Facilitator expertise is essential. Future iterations should consider participants' experiences to tailor content further and expand offerings about research methodologies.