Medical Education and artificial intelligence: Responsible and effective practice requires human oversight

Abstract

I have a confession to make. I have been slow to generate an official policy statement for Medical Education about artificial intelligence (AI) because I find the discussion terribly boring. Don't confuse that statement with lack of interest—I consider the technology exhilarating, use it routinely, and marvel at its potential.¹ Don't confuse it either with being dismissive—I recognise, appreciate, and wish to help guard against the ethical harms that could be done from, among other things, loss of intellectual property and reinforcement of systemic bias.² However, I find most discussion about the use of AI in publishing (be it about writing, enabling better and faster peer review, or the need to guard against unscrupulous practices) to boil down to the same basic sentiment: Responsible and effective practice requires human oversight.

With over 14 500 seemingly viable AI resources readily available,³ there is great risk of overgeneralization and I will not profess to having deep knowledge of the means through which each has been generated. I do, however, believe this class of technologies, as a whole, to best be conceived of as tools (that happen to be proliferating at unprecedented speed and with little empirical testing).^{4, 5} Some of the panic the rate of development creates amounts to worry that we ourselves will become tools, used by the computers, but that is not the reality we are dealing with at the moment and there are very good reasons to not believe the futurists in that regard.⁶ As such, we must focus on what all tools require for responsible and effective practice: Human, or at least biological,⁷ oversight. So let's consider the role each group involved in journal publication has to play in that regard.

We encourage authors to use AI if and when it helps strengthen their capacity to improve awareness of pre-existing literature,⁸ to formulate stronger research questions or to bolster research designs and analyses (i.e. any time it helps to make their scholarship better). We are not going to force disclosure of every way in which AI influenced their submissions because it would be impossible to craft a sufficiently detailed guideline (especially given that people are often unaware of how AI has been embedded in common software packages). Further, a dominant theme in our International Editorial Advisory Board's debate about this issue was that requiring such disclosure is likely to be increasingly nonsensical, tantamount to needing to disclose the use of Google, spell-check, a keyboard, or any other tool that is similarly omnipresent in academic work. If using AI was of fundamental importance to your project, then what made it so should be disclosed in the body of your paper. That standard, however, is the same as has always been applied to disclosing use of tools like nVivo or SPSS for primary analyses or any of countless databases for literature searches: authors are responsible for clearly describing aspects of their efforts that readers need to know to understand the rigour and replicability of the study. In doing so, it is of course important to keep in mind, just as we do routinely with other technologies, that any tool can be misused, requiring caution and investment in learning about the tool's strengths and limitations.^{1, 9, 10} Responsible and effective practice requires human oversight.

Optimistically, we hope this position will improve equity in the field by reducing barriers for those who wish to publish in our pages despite their first language not being English. We understand the risk of other barriers being created by virtue of privileging those who can afford the technology, but remain hopeful that is a lesser challenge given the truism that computer technology gets cheaper with time.¹¹ There can be no doubt that AI hallucinates,¹⁰ requiring individuals to double check its claims about the world while recognising that the author, not the computer, is accountable for the final text.¹² For those reasons, I would never myself dream of submitting a paper in a language other than one in which I was fluent without careful and triangulated effort to confirm that the translation said exactly what I intended it to. Responsible and effective practice requires human oversight.

Whether authors have used AI or not, peer review remains the best tool we have at our disposal for improving the work we are collectively undertaking as a field of study.¹³ Given that AI is currently built on a corpus of knowledge that is predominantly English,² we need reviewers to raise questions about whether a project responsibly represents the state of knowledge in the world. That standard, however, is the same as has always been applied in attempts to judge the adequacy of a paper's framing. Similarly, while it would be inappropriate to submit a manuscript one received for peer review to an AI device without permission to do so, that aligns with the same confidentiality standard that has existed for decades. Asking a question of AI to clarify one's thinking or to contemplate clearer (or more courteous) ways of conveying one's concerns is encouraged if and when it improves the reviewer's capacity to offer feedback to authors or professional development for reviewers themselves.¹⁴ Out of curiosity, I once submitted some of my own writing to ChatGPT with a request to ‘write a rejection letter’ (to see if it could predict the objections peer reviewers might raise). After the first response largely parroted back the claims I had made in the abstract, I instructed the computer to try again, stressing that I wanted a rejection letter. Its response was informative: ‘I am not programmed for critical appraisal.’ Even the AI ‘knew’ that responsible and effective practice requires human oversight.

As curators and stewards of the journal, we too pledge to use AI if and when it helps to improve reader, author, and reviewer experience. For example, AI has enabled implementation of a ‘free format’ submission system at Medical Education, so our authors no longer have to go through the tedium of formatting references in a specific way; authors will also note that most of the effort involved in making a submission now amounts to uploading a manuscript and confirming that the software has accurately identified author names, the title, abstract and so on. AI has been used for years to try to detect unethical publication practices such as duplicate submission and plagiarism. Similarly, the software we use to manage peer review has AI embedded that suggests reviewers it thinks to be particularly well suited to the content of the manuscript under consideration. While these systems will undoubtedly continue to improve, each is far from perfect. Truly fraudulent behaviour cannot be caught by any existing software while far more common lesser transgressions are generally over-called. Further, our editors must remain cognizant of the value of hearing diverse voices to inform our peer reviews if we are to facilitate a truly inclusive academic community.¹⁵ As a result, we will not automate decision-making or empower AI to conduct it. Instead, these resources will continue to be used as tools, taking advantage of their greater capacity to flag potential issues and opportunities while continuing to investigate them with the care and thoughtfulness required to yield the best outcomes we can achieve. Responsible and effective practice requires human oversight.

While less of a policy than a perspective, I offer this editorial as proof that I did (eventually) conclude it necessary to share these views for the sake of transparency, however boring (i.e. non-reactive or reinforcing of the status quo) they happen to be. As technology changes, our policies will continue to evolve, but for now we encourage everyone involved in academic publishing to use whatever tools they have available for improving the field and its capacity to improve health through education. That, to my mind, defines responsible and effective oversight.