Artificial Intelligence in Health Care: A Rallying Cry for Critical Clinical Research and Ethical Thinking

Artificial intelligence (AI) will impact a large proportion of jobs in the short to medium term, especially in the developed countries. The consequences will be felt across many sectors including health care, a critical sector for implementation of AI tools because glitches in algorithms or biases in training datasets may lead to suboptimal treatment that may negatively affect the health of an individual. The stakes are obviously higher in case of potentially life-threatening diseases such as cancer and therapies with a potential for causing severe or even fatal adverse events.

Over the last two decades, much of the research on AI in health care has focussed on diagnostic radiology and digital pathology, but a solid body of research is emerging on AI tools in the radiation oncology workflow. Many of these applications are relatively uncontroversial, although there is still a lack of evidence regarding effectiveness rather than efficiency, and—the ultimate bar—evidence of clinical utility. Proponents of AI will argue that these algorithms should be implemented with robust human supervision. One challenge here is the deskilling effect associated with new technologies. We will become increasingly dependent on the AI tools over time, and we will become less capable of assessing the quality of the AI output.

Much of this research appears almost old-fashioned in view of the rapid advances in Generative artificial intelligence (GenAI). GenAI can draw from multiple types of data and produce output that is personalised and appears relevant in the given context. Especially the rapid progress in large language models (LLMs) has opened a wide field of potential applications that were out of bounds just a few years ago. One LLM, Generative Pre-trained Transformer 4 (GPT-4), has been made widely accessible to end-users as ChatGPT-4, which passed a rigorous Turing test in a recent study. In this viewpoint, I argue for the necessity of independent academic research to establish evidence-based applications of AI in medicine. Algorithmic medicine is an intervention similar to a new drug or a new medical device. We should be especially concerned about under-represented minorities and rare/atypical clinical cases that may drown in the petabyte-sized training sets. A huge educational push is needed to ensure that the end-users of AI in health care understand the strengths and weaknesses of algorithmic medicine. Finally, we need to address the ethical boundaries for where and when GenAI can replace humans in the relation between patients and healthcare providers.