Prediction of tumor board procedural recommendations using large language models

Abstract

Introduction

Multidisciplinary tumor boards are meetings where a team of medical specialists, including medical oncologists, radiation oncologists, radiologists, surgeons, and pathologists, collaborate to determine the best treatment plan for cancer patients. While decision-making in this context is logistically and cost-intensive, it has a significant positive effect on overall cancer survival.

Methods 

We evaluated the quality and accuracy of predictions by several large language models for recommending procedures by a Head and Neck Oncology tumor board, which we adapted for the task using parameter-efficient fine-tuning or in-context learning. Records were divided into two sets: n=229 used for training and n=100 records for validation of our approaches. Randomized, blinded, manual human expert classification was used to evaluate the different models.

Results 

Treatment line congruence varied depending on the model, reaching up to 86%, with medically justifiable recommendations up to 98%. Parameter-efficient fine-tuning yielded better outcomes than in-context learning, and larger/commercial models tend to perform better.

Conclusion

Providing precise, medically justifiable procedural recommendations for complex oncology patients is feasible. Extending the data corpus to a larger patient cohort and incorporating the latest guidelines, assuming the model can handle sufficient context length, could result in more factual and guideline-aligned responses and is anticipated to enhance model performance. We, therefore, encourage further research in this direction to improve the efficacy and reliability of large language models as support in medical decision-making processes.