Leveraging FDA Labeling Documents and Large Language Model to Enhance Annotation, Profiling, and Classification of Drug Adverse Events with AskFDALabel.

Abstract

Background: Drug adverse events (AEs) represent a significant public health concern. US Food and Drug Administration (FDA) drug labeling documents are an essential resource for studying drug safety such as assessing a drug's likelihood to cause certain organ toxicities; however, the manual extraction of AEs is labor-intensive, requires specialized expertise, and is challenging to maintain, due to frequent updates of the labeling documents.

Objective: To automate the extraction of AE data from FDA drug labeling documents, we developed a workflow based on AskFDALabel, a large language model (LLM)-powered framework, and its demonstration in drug safety studies.

Methods: This framework incorporates a retrieval-augmented generation (RAG) component based on FDALabel to enhance standard LLM inference. Key steps include (1) selection of a task-specific template, (2) FDALabel database querying, and (3) content preparation for LLM processing. We evaluated the performance of the framework in three benchmark experiments, including drug-induced liver injury (DILI) classification, drug-induced cardiotoxicity (DICT) classification, and AE term recognition.

Results: AskFDALabel achieved F1-scores of 0.978 for DILI, 0.931 for DICT, and 0.911 for AE annotation, outperforming other traditional methods. It also provided cited labeling content and detailed explanations, facilitating manual verification.

Conclusion: AskFDALabel exhibited high consistency with human AE annotation, particularly in classifying and profiling DILI and DICT. Thus, it can significantly enhance the efficiency and accuracy of AE annotation, with promising potential for advanced AE surveillance and drug safety research.