SCALES-AI: A Supervision- and Context-Aligned Entrustment Framework for Integrating Artificial Intelligence Into Emergency Medicine Education.

Abstract

Introduction: The rapid entry of artificial intelligence (AI) into emergency medicine (EM) education demands a practical way to decide how much autonomy to grant specific tools for specific educational tasks. Existing frameworks are often too technical or too abstract to inform day-to-day decisions, leaving programs at risk of piecemeal adoption that could undermine core training principles and amplify bias.

Methods: We developed SCALES-AI (Supervision- and Context-Aligned Levels of Entrustment for AI in Education) through a three-phase process: a targeted literature synthesis extending recent scoping reviews on AI in medical education; three structured 90-minute virtual workshops with a nine-member multidisciplinary working group spanning six institutions and four U.S. geographic regions; and iterative refinement with national stakeholder groups, including an AI-focused subgroup of the SAEM Education Committee, the SAEM Informatics, Data Science & AI Interest Group, and the ACEP AI Task Force.

Analysis: SCALES-AI rates AI tools-rather than trainees or faculty-on an entrustment scale for educational (not clinical) use. It links four Foundational Principles (human-centered augmentation, evidence-based scalability, context-specific adaptation, and an ethical foundation) to four Dimensions of Trustworthiness (Ability, Integrity, Benevolence, and Equity). These dimensions inform a five-level entrustment scale (0-5), ranging from "not appropriate" to "full autonomy for low-stakes tasks," with supervision intensity and audit cadence matched to level. Re-evaluation is triggered by model, prompt, corpus, or guideline changes and by observed drift. We illustrate application through concrete EM scenarios, including documentation feedback, practice-question generation, simulation debrief support, and portfolio analytics.

Recommendations: Programs should adopt graduated entrustment for AI tools, pairing each tool-task decision with explicit scope controls, supervision requirements, and pre-specified re-evaluation triggers. To support implementation and reproducibility, we provide a SCALES-AI Checklist to document tool-task ratings, supervision and scope controls, and monitoring plans, along with a SCALES-AI Inter-Rater Calibration Template to enable local reliability assessment. Although motivated by the high-stakes, time-pressured EM environment, SCALES-AI is generalizable across health professions education. Treating AI-tool trustworthiness as dynamic and context-dependent enables safe, equitable, and auditable adoption while preserving essential human elements of medical training.