Fit for purpose: Ensuring robust, contextually appropriate global accreditation practices

Abstract

Medical school accreditation has become an increasingly popular tool for ensuring high quality training of medical doctors across the globe, with global systems of accreditation being strongly encouraged by the World Health Assembly, World Medical Association and the World Federation for Medical Education (WFME).¹ Despite this growing trend towards standardisation, medical school accreditation has only recently become a topic of research inquiry.² The article by You et al.³ in this issue adds important data to the accreditation evidence base. After localising the WFME standards to the Chinese context, You et al.³ conclude that first-round accreditation improves pass rates on China's Medical Licensing Examination. We commend You et al.³ for the breadth and scope of their work, which included 105 medical programmes across three ‘tiers’ of schools in China, and for being cautious in assuming that an improvement in licensing exam pass rates will continue beyond the first round of accreditation. You et al.³ provide an example of accreditation research that accounts for the unique sociopolitical context of medical education in China. We applaud accreditation research that accounts for historical, social, political and economic contexts. We further propose that as accreditation research expands, both within North America and in geographies with newly acquired accreditation systems, researchers make use of methodologies, tools and metrics that are meaningful and valid; account for political and power dynamics within accreditation systems; and explore the unintended consequences of accreditation.

You et al.³ make comparisons between countries, like their own, that are newly adopting systems of accreditation, and North America, which they describe as having a ‘mature’ accreditation system. The development of accreditation in the United States is described as ‘one of the greatest chapters in the history of the profession’. While we agree that North American accreditation is mature in the sense that it has a longer history than in most countries, accreditation and accreditation research in North America are not without their own challenges. Within North America, there has historically been minimal medical education accreditation scholarship. A 2019 scoping review identified only 203 articles from inception until 2019 on accreditation, with only 36 of these representing rigorous scholarship.² BMC Medical Education published a supplement in 2020 on ‘Current themes and challenges facing Health Professions Education (HPE) accreditation in the 21st century’.⁴ In those six articles, only three countries were represented on the authorship teams (all from the Global North), and only one paper was empirical. This small body of literature on accreditation (even within Global North contexts) suggests that systems of accreditation may not be sufficiently grounded in evidence. You et al.³ call for additional quantitative research on accreditation. We agree that additional research is necessary for accreditation to be truly evidence-informed, including studies using diverse research methodologies (including qualitative research), theoretical perspectives and outcome measures.

You et al.'s³ study relies on licensing exam scores as a measure of accreditation impact, a metric that has also been used by other researchers.^5-8 However, a recent critique proposes that licensing exam scores may be too far removed from the accreditation process itself to be a valid measure of accreditation impact.⁹ Without ensuring that a metric has rigorous construct validity,¹⁰ it is impossible to determine whether it is truly measuring the phenomenon of interest. It is also impossible to determine from You et al.'s³ study what aspects of the implemented accreditation process were responsible for the improvement in licensing exam scores. It is not clear whether the localised WFME standards themselves had an impact on pass rates or whether any newly introduced system of accreditation or quality improvement would also have led to improvement in student outcomes. As accreditation research expands, it will be important to identify valid metrics for measuring the impacts of accreditation and to tease apart specific components that have impact.

Accreditation is not a politically neutral enterprise, and accreditation research should explore the political implications of global systems of accreditation. You et al.'s³ research is attentive to the sociopolitical context of Chinese medical education. They found that localising the WFME standards had different effects within different Chinese medical school contexts: Tier 3 medical schools experienced greater improvements in licensing exam pass rates than Tier 2 and Tier 1 medical schools. In a recent study¹¹ of accreditation of undergraduate medical education programmes in Canada, two authors of this commentary similarly found that standardised alterations to the accreditation system had different effects at two schools with different structural and sociopolitical contexts: a longstanding medical school with a centralised structure within an urban setting and a newer rural medical school with a distributed structure. We found that power flowed differently in response to identical accreditation changes, with the effects experienced differently at each school.

Rationales for the adoption of medical education accreditation include improving education quality, providing optimal learning environments, holding education institutions accountable to meeting societal needs and ensuring that physicians can move seamlessly between contexts.^{12, 13} While research should aim to evaluate whether these goals are being met, it should also seek to provide an understanding of the unintended consequences of accreditation and accreditation change. Scholars have noted the complexity of developing accreditation standards that meet both local and global needs in non-EuroAmerican contexts, including structural, regulatory, developmental and aspirational complexities.¹⁴ Others have identified key barriers that low- and middle-income countries (LMICs) face when attempting to design systems of accreditation.¹⁵ Countries that adopt/adapt universal standards face the double burden of developing accreditation programmes that meet global expectations (or those of the Global North/United States)¹⁶ and meeting the needs of their local populations. Accreditation is a resource-intensive process, and for LMICs, this may mean that resources are being diverted from other important aspects of health care and education.¹⁶

Another critique of adopting WFME accreditation standards across diverse global contexts relates to risks of imposing values from dominant countries when standardising medical education across the globe.¹⁷ As countries adopt accreditation systems that are recognised by the WFME, this allows their students to meet the requirements of the Educational Commission for Foreign Medical Graduates (ECFMG) for entry to US residency programmes.¹⁸ Will this encourage a migration of medical trainees from LMICs to the United States, thereby increasing brain-drain and worsening inequities in the Global South?¹⁶ Research should explore these unintended consequences to safeguard against greater harm accruing in LMICs. Taber et al.¹³ argue that a ‘fit for purpose’ approach to accreditation design is necessary for ensuring that different accreditation systems are optimally adapted to their political, social and economic context. We agree and would add that accreditation research must also be creatively designed to move beyond measuring pass rates on licensing exams, while considering the larger and unintended impacts of accreditation on local populations.

All authors contributed to the conceptualisation and drafting of the original manuscript draft.