Marc Aubreville, Jonathan Ganz, Jonas Ammeling, Emely Rosbach, Thomas Gehrke, Agmal Scherzad, Stephan Hackenberg, Miguel Goncalves
{"title":"利用大型语言模型预测肿瘤委员会的程序建议","authors":"Marc Aubreville, Jonathan Ganz, Jonas Ammeling, Emely Rosbach, Thomas Gehrke, Agmal Scherzad, Stephan Hackenberg, Miguel Goncalves","doi":"10.1007/s00405-024-08947-9","DOIUrl":null,"url":null,"abstract":"<h3 data-test=\"abstract-sub-heading\">Introduction</h3><p>Multidisciplinary tumor boards are meetings where a team of medical specialists, including medical oncologists, radiation oncologists, radiologists, surgeons, and pathologists, collaborate to determine the best treatment plan for cancer patients. While decision-making in this context is logistically and cost-intensive, it has a significant positive effect on overall cancer survival.</p><h3 data-test=\"abstract-sub-heading\">Methods </h3><p>We evaluated the quality and accuracy of predictions by several large language models for recommending procedures by a Head and Neck Oncology tumor board, which we adapted for the task using parameter-efficient fine-tuning or in-context learning. Records were divided into two sets: n=229 used for training and n=100 records for validation of our approaches. Randomized, blinded, manual human expert classification was used to evaluate the different models.</p><h3 data-test=\"abstract-sub-heading\">Results </h3><p>Treatment line congruence varied depending on the model, reaching up to 86%, with medically justifiable recommendations up to 98%. Parameter-efficient fine-tuning yielded better outcomes than in-context learning, and larger/commercial models tend to perform better.</p><h3 data-test=\"abstract-sub-heading\">Conclusion</h3><p>Providing precise, medically justifiable procedural recommendations for complex oncology patients is feasible. Extending the data corpus to a larger patient cohort and incorporating the latest guidelines, assuming the model can handle sufficient context length, could result in more factual and guideline-aligned responses and is anticipated to enhance model performance. We, therefore, encourage further research in this direction to improve the efficacy and reliability of large language models as support in medical decision-making processes.</p>","PeriodicalId":11952,"journal":{"name":"European Archives of Oto-Rhino-Laryngology","volume":"20 1","pages":""},"PeriodicalIF":1.9000,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Prediction of tumor board procedural recommendations using large language models\",\"authors\":\"Marc Aubreville, Jonathan Ganz, Jonas Ammeling, Emely Rosbach, Thomas Gehrke, Agmal Scherzad, Stephan Hackenberg, Miguel Goncalves\",\"doi\":\"10.1007/s00405-024-08947-9\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<h3 data-test=\\\"abstract-sub-heading\\\">Introduction</h3><p>Multidisciplinary tumor boards are meetings where a team of medical specialists, including medical oncologists, radiation oncologists, radiologists, surgeons, and pathologists, collaborate to determine the best treatment plan for cancer patients. While decision-making in this context is logistically and cost-intensive, it has a significant positive effect on overall cancer survival.</p><h3 data-test=\\\"abstract-sub-heading\\\">Methods </h3><p>We evaluated the quality and accuracy of predictions by several large language models for recommending procedures by a Head and Neck Oncology tumor board, which we adapted for the task using parameter-efficient fine-tuning or in-context learning. Records were divided into two sets: n=229 used for training and n=100 records for validation of our approaches. Randomized, blinded, manual human expert classification was used to evaluate the different models.</p><h3 data-test=\\\"abstract-sub-heading\\\">Results </h3><p>Treatment line congruence varied depending on the model, reaching up to 86%, with medically justifiable recommendations up to 98%. Parameter-efficient fine-tuning yielded better outcomes than in-context learning, and larger/commercial models tend to perform better.</p><h3 data-test=\\\"abstract-sub-heading\\\">Conclusion</h3><p>Providing precise, medically justifiable procedural recommendations for complex oncology patients is feasible. Extending the data corpus to a larger patient cohort and incorporating the latest guidelines, assuming the model can handle sufficient context length, could result in more factual and guideline-aligned responses and is anticipated to enhance model performance. We, therefore, encourage further research in this direction to improve the efficacy and reliability of large language models as support in medical decision-making processes.</p>\",\"PeriodicalId\":11952,\"journal\":{\"name\":\"European Archives of Oto-Rhino-Laryngology\",\"volume\":\"20 1\",\"pages\":\"\"},\"PeriodicalIF\":1.9000,\"publicationDate\":\"2024-09-13\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"European Archives of Oto-Rhino-Laryngology\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1007/s00405-024-08947-9\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"OTORHINOLARYNGOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"European Archives of Oto-Rhino-Laryngology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s00405-024-08947-9","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"OTORHINOLARYNGOLOGY","Score":null,"Total":0}
Prediction of tumor board procedural recommendations using large language models
Introduction
Multidisciplinary tumor boards are meetings where a team of medical specialists, including medical oncologists, radiation oncologists, radiologists, surgeons, and pathologists, collaborate to determine the best treatment plan for cancer patients. While decision-making in this context is logistically and cost-intensive, it has a significant positive effect on overall cancer survival.
Methods
We evaluated the quality and accuracy of predictions by several large language models for recommending procedures by a Head and Neck Oncology tumor board, which we adapted for the task using parameter-efficient fine-tuning or in-context learning. Records were divided into two sets: n=229 used for training and n=100 records for validation of our approaches. Randomized, blinded, manual human expert classification was used to evaluate the different models.
Results
Treatment line congruence varied depending on the model, reaching up to 86%, with medically justifiable recommendations up to 98%. Parameter-efficient fine-tuning yielded better outcomes than in-context learning, and larger/commercial models tend to perform better.
Conclusion
Providing precise, medically justifiable procedural recommendations for complex oncology patients is feasible. Extending the data corpus to a larger patient cohort and incorporating the latest guidelines, assuming the model can handle sufficient context length, could result in more factual and guideline-aligned responses and is anticipated to enhance model performance. We, therefore, encourage further research in this direction to improve the efficacy and reliability of large language models as support in medical decision-making processes.
期刊介绍:
Official Journal of
European Union of Medical Specialists – ORL Section and Board
Official Journal of Confederation of European Oto-Rhino-Laryngology Head and Neck Surgery
"European Archives of Oto-Rhino-Laryngology" publishes original clinical reports and clinically relevant experimental studies, as well as short communications presenting new results of special interest. With peer review by a respected international editorial board and prompt English-language publication, the journal provides rapid dissemination of information by authors from around the world. This particular feature makes it the journal of choice for readers who want to be informed about the continuing state of the art concerning basic sciences and the diagnosis and management of diseases of the head and neck on an international level.
European Archives of Oto-Rhino-Laryngology was founded in 1864 as "Archiv für Ohrenheilkunde" by A. von Tröltsch, A. Politzer and H. Schwartze.