Leveraging artificial intelligence to detect ethical concerns in medical research: a case study.

Abstract

Background: Institutional review boards (IRBs) have been criticised for delays in approvals for research proposals due to inadequate or inexperienced IRB staff. Artificial intelligence (AI), particularly large language models (LLMs), has significant potential to assist IRB members in a prompt and efficient reviewing process.

Methods: Four LLMs were evaluated on whether they could identify potential ethical issues in seven validated case studies. The LLMs were prompted with queries related to the proposed eligibility criteria of the study participants, vulnerability issues, information to be disclosed in the informed consent document (ICD), risk-benefit assessment and justification of the use of a placebo. Another query was issued to the LLMs to generate ICDs for these case scenarios.

Results: All four LLMs were able to provide answers to the queries related to all seven cases. In general, the responses were homogeneous with respect to most elements. LLMs performed suboptimally in identifying the suitability of the placebo arm, risk mitigation strategies and potential risks to study participants in certain case studies with a single prompt. However, multiple prompts led to better outputs in all of these domains. Each of the LLMs included all of the fundamental elements of the ICD for all case scenarios. Use of jargon, understatement of benefits and failure to state potential risks were the key observations in the AI-generated ICD.

Conclusion: It is likely that LLMs can enhance the identification of potential ethical issues in clinical research, and they can be used as an adjunct tool to prescreen research proposals and enhance the efficiency of an IRB.