Use of Open-Source Large Language Models for Automatic Synthesis of the Entire Imaging Medical Records of Patients: A Feasibility Study.

Background/objectives: Reviewing the entire history of imaging exams of a single patient's records is an essential step in clinical practice, but it is time and resource consuming, with potential negative effects on workflow and on the quality of medical decisions. The main objective of this study was to evaluate the applicability of three open-source large language models (LLMs) for the automatic generation of concise summaries of patient's imaging records. Secondary objectives were to assess correlations among the LLMs and to evaluate the length reduction provided by each model.

Methods: Three state-of-the-art open-source large language models were selected: Llama 3.2 11B, Mistral 7B, and Falcon 7B. Each model was given a set of radiology reports. The summaries produced by the models were evaluated by two experienced radiologists and one experienced clinical physician using standardized metrics.

Results: A variable number of radiological reports (n = 12-56) from four patients were selected and evaluated. The summaries generated by the three LLM showed a good level of accuracy compared with the information contained in the original reports, with positive ratings on both clinical relevance and ease of reference. According to the experts' evaluations, the use of the summaries generated by LLMs could help to reduce the time spent on reviewing the previous imaging examinations performed, preserving the quality of clinical data.

Conclusions: Our results suggest that LLMs are able to generate summaries of the imaging history of patients, and these summaries could improve radiology workflow making it easier to manage large volumes of reports.