Jennifer Gotta, Quang Anh Le Hong, Vitali Koch, Leon D Gruenewald, Tobias Geyer, Simon S Martin, Jan-Erik Scholtz, Christian Booz, Daniel Pinto Dos Santos, Scherwin Mahmoudi, Katrin Eichler, Tatjana Gruber-Rouh, Renate Hammerstingl, Teodora Biciusca, Lisa Joy Juergens, Elena Höhne, Christoph Mader, Thomas J Vogl, Philipp Reschke
{"title":"Large language models (LLMs) in radiology exams for medical students: Performance and consequences.","authors":"Jennifer Gotta, Quang Anh Le Hong, Vitali Koch, Leon D Gruenewald, Tobias Geyer, Simon S Martin, Jan-Erik Scholtz, Christian Booz, Daniel Pinto Dos Santos, Scherwin Mahmoudi, Katrin Eichler, Tatjana Gruber-Rouh, Renate Hammerstingl, Teodora Biciusca, Lisa Joy Juergens, Elena Höhne, Christoph Mader, Thomas J Vogl, Philipp Reschke","doi":"10.1055/a-2437-2067","DOIUrl":"https://doi.org/10.1055/a-2437-2067","url":null,"abstract":"<p><p>The evolving field of medical education is being shaped by technological advancements, including the integration of Large Language Models (LLMs) like ChatGPT. These models could be invaluable resources for medical students, by simplifying complex concepts and enhancing interactive learning by providing personalized support. LLMs have shown impressive performance in professional examinations, even without specific domain training, making them particularly relevant in the medical field. This study aims to assess the performance of LLMs in radiology examinations for medical students, thereby shedding light on their current capabilities and implications.This study was conducted using 151 multiple-choice questions, which were used for radiology exams for medical students. The questions were categorized by type and topic and were then processed using OpenAI's GPT-3.5 and GPT- 4 via their API, or manually put into Perplexity AI with GPT-3.5 and Bing. LLM performance was evaluated overall, by question type and by topic.GPT-3.5 achieved a 67.6% overall accuracy on all 151 questions, while GPT-4 outperformed it significantly with an 88.1% overall accuracy (p<0.001). GPT-4 demonstrated superior performance in both lower-order and higher-order questions compared to GPT-3.5, Perplexity AI, and medical students, with GPT-4 particularly excelling in higher-order questions. All GPT models would have successfully passed the radiology exam for medical students at our university.In conclusion, our study highlights the potential of LLMs as accessible knowledge resources for medical students. GPT-4 performed well on lower-order as well as higher-order questions, making ChatGPT-4 a potentially very useful tool for reviewing radiology exam questions. Radiologists should be aware of ChatGPT's limitations, including its tendency to confidently provide incorrect responses. · ChatGPT demonstrated remarkable performance, achieving a passing grade on a radiology examination for medical students that did not include image questions.. · GPT-4 exhibits significantly improved performance compared to its predecessors GPT-3.5 and Perplexity AI with 88% of questions answered correctly.. · Radiologists as well as medical students should be aware of ChatGPT's limitations, including its tendency to confidently provide incorrect responses.. · Gotta J, Le Hong QA, Koch V et al. Large language models (LLMs) in radiology exams for medical students: Performance and consequences. Fortschr Röntgenstr 2024; DOI 10.1055/a-2437-2067.</p>","PeriodicalId":21490,"journal":{"name":"Rofo-fortschritte Auf Dem Gebiet Der Rontgenstrahlen Und Der Bildgebenden Verfahren","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142576852","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Peter Isfort, Christof M Sommer, Philipp Bruners, Bettina Maiwald, Jens-Peter Kühn, Christoph Georg Radosa, Roman Kloeckner, Patrick Freyhardt, Mareike Franke, Michael Moche, Ralf-Thorsten Hoffmann, Konstantin Nikolaou, Andreas H Mahnken, Marcus Katoh
{"title":"Position Paper of the German Society for Interventional Radiology and Minimally Invasive Therapy (DeGIR) and the German Roentgen Society (DRG) on Structural and Professional Requirements in Interventional Oncology.","authors":"Peter Isfort, Christof M Sommer, Philipp Bruners, Bettina Maiwald, Jens-Peter Kühn, Christoph Georg Radosa, Roman Kloeckner, Patrick Freyhardt, Mareike Franke, Michael Moche, Ralf-Thorsten Hoffmann, Konstantin Nikolaou, Andreas H Mahnken, Marcus Katoh","doi":"10.1055/a-2373-1013","DOIUrl":"https://doi.org/10.1055/a-2373-1013","url":null,"abstract":"<p><p>Interventional oncology (IO) employs various techniques to enable minimally invasive, image-guided treatment of tumor diseases with both curative and palliative goals. Additionally, it significantly contributes to managing tumor-related and perioperative complications, offering diverse supportive procedures for patients at all stages of their diseases. The execution of IO procedures places unique demands on the equipment, personnel, and structural organization of radiological clinics, necessitating specific expertise from interventional radiologists.This position paper aims to comprehensively outline the multifaceted aspects of IO and discuss the requisite criteria for hospitals, radiological clinics, and interventional radiologists (IRs). Furthermore, it underscores overarching considerations of quality assurance that clinics and professional societies should prioritize.The requirements for hospitals, radiological clinics, and IRs are varied and demand not only a high level of proficiency in performing IO procedures but also in-depth knowledge of the differential therapy for various tumor diseases. This expertise is essential for effectively serving as clinical partners in the interdisciplinary treatment of oncologic patients. Additionally, a thorough understanding and safe handling of ionizing radiation technologies, along with proficiency in radiation protection methods, which are fundamental aspects of radiological specialist training, is crucial for ensuring the safety of IO procedures for both patients and staff. The Deutsche Gesellschaft für Interventionelle Radiologie und minimal-invasive Therapie (DeGIR) and the Cardiovascular and Interventional Radiological Society of Europe (CIRSE) have long-established dedicated quality management programs, accrediting radiology clinics and certifying IRs. These initiatives aim to uphold the highest standards of care and meet the quality expectations set by politics in healthcare system, particularly in the realm of interventional radiology. · The various procedures in the field of interventional oncology (IO) are complex medical interventions that require not only the most advanced technical equipment but also adequate human resources, particularly specialized expertise in interventional radiology, diagnostic imaging, oncology, and radiation protection.. · This expertise is an integral part of the specialized medical training in radiology and is certified by professional societies such as the German Society for Interventional Radiology (DeGIR) and the Cardiovascular and Interventional Radiological Society of Europe (CIRSE).. · Professional societies like DeGIR, CIRSE, and the American Society of Interventional Radiology (SIR) establish the necessary quality assurance framework for comprehensive, high-quality IO therapy through quality assurance (QA) registries, standard operating procedure (SOP) documents, and participation in guideline development.. · Currently, radiology is the only disciplin","PeriodicalId":21490,"journal":{"name":"Rofo-fortschritte Auf Dem Gebiet Der Rontgenstrahlen Und Der Bildgebenden Verfahren","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142576853","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"[Thoracic penetrating aortic ulcer: conventional radiography as a valuable diagnostic tool].","authors":"Fiona Mankertz, Eiko Rathmann, Alexandra Busemann","doi":"10.1055/a-2441-5303","DOIUrl":"https://doi.org/10.1055/a-2441-5303","url":null,"abstract":"","PeriodicalId":21490,"journal":{"name":"Rofo-fortschritte Auf Dem Gebiet Der Rontgenstrahlen Und Der Bildgebenden Verfahren","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142576851","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Maximilian Frederik Russe, Marco Reisert, Fabian Bamberg, Alexander Rau
{"title":"Improving the use of LLMs in radiology through prompt engineering: from precision prompts to zero-shot learning.","authors":"Maximilian Frederik Russe, Marco Reisert, Fabian Bamberg, Alexander Rau","doi":"10.1055/a-2264-5631","DOIUrl":"10.1055/a-2264-5631","url":null,"abstract":"<p><strong>Purpose: </strong> Large language models (LLMs) such as ChatGPT have shown significant potential in radiology. Their effectiveness often depends on prompt engineering, which optimizes the interaction with the chatbot for accurate results. Here, we highlight the critical role of prompt engineering in tailoring the LLMs' responses to specific medical tasks.</p><p><strong>Materials and methods: </strong> Using a clinical case, we elucidate different prompting strategies to adapt the LLM ChatGPT using GPT4 to new tasks without additional training of the base model. These approaches range from precision prompts to advanced in-context methods such as few-shot and zero-shot learning. Additionally, the significance of embeddings, which serve as a data representation technique, is discussed.</p><p><strong>Results: </strong> Prompt engineering substantially improved and focused the chatbot's output. Moreover, embedding of specialized knowledge allows for more transparent insight into the model's decision-making and thus enhances trust.</p><p><strong>Conclusion: </strong> Despite certain challenges, prompt engineering plays a pivotal role in harnessing the potential of LLMs for specialized tasks in the medical domain, particularly radiology. As LLMs continue to evolve, techniques like few-shot learning, zero-shot learning, and embedding-based retrieval mechanisms will become indispensable in delivering tailored outputs.</p><p><strong>Key points: </strong> · Large language models might impact radiological practice and decision-masking.. · However, implementation and performance are dependent on the assigned task.. · Optimization of prompting strategies can substantially improve model performance.. · Strategies for prompt engineering range from precision prompts to zero-shot learning..</p><p><strong>Citation format: </strong>· Russe MF, Reisert M, Bamberg F et al. Improving the use of LLMs in radiology through prompt engineering: from precision prompts to zero-shot learning . Fortschr Röntgenstr 2024; 196: 1166 - 1170.</p>","PeriodicalId":21490,"journal":{"name":"Rofo-fortschritte Auf Dem Gebiet Der Rontgenstrahlen Und Der Bildgebenden Verfahren","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139973344","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Strahlenschutzkurse der Röntgen Akademie.","authors":"","doi":"10.1055/a-2406-1148","DOIUrl":"https://doi.org/10.1055/a-2406-1148","url":null,"abstract":"","PeriodicalId":21490,"journal":{"name":"Rofo-fortschritte Auf Dem Gebiet Der Rontgenstrahlen Und Der Bildgebenden Verfahren","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142507058","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Die NIS-2-Richtlinie der EU und ihre Umsetzung in nationales Recht – Neue Vorgaben zur Cybersicherheit in der Arztpraxis ab 2025.","authors":"","doi":"10.1055/a-2406-1023","DOIUrl":"https://doi.org/10.1055/a-2406-1023","url":null,"abstract":"","PeriodicalId":21490,"journal":{"name":"Rofo-fortschritte Auf Dem Gebiet Der Rontgenstrahlen Und Der Bildgebenden Verfahren","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142507049","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Last Call: Noch bis zum 4. November 2024 Abstracts zum RÖKO 2025 einreichen!","authors":"","doi":"10.1055/a-2406-0900","DOIUrl":"https://doi.org/10.1055/a-2406-0900","url":null,"abstract":"","PeriodicalId":21490,"journal":{"name":"Rofo-fortschritte Auf Dem Gebiet Der Rontgenstrahlen Und Der Bildgebenden Verfahren","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142507055","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Weg von der Bordsteinkante.","authors":"","doi":"10.1055/a-2406-1189","DOIUrl":"https://doi.org/10.1055/a-2406-1189","url":null,"abstract":"","PeriodicalId":21490,"journal":{"name":"Rofo-fortschritte Auf Dem Gebiet Der Rontgenstrahlen Und Der Bildgebenden Verfahren","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142507060","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Two rare vascular findings in one patient: Retro-psoas Iliac Artery & Common Pulmonary Venous Confluence.","authors":"Bastian Schulz, André Euler","doi":"10.1055/a-2311-0043","DOIUrl":"10.1055/a-2311-0043","url":null,"abstract":"","PeriodicalId":21490,"journal":{"name":"Rofo-fortschritte Auf Dem Gebiet Der Rontgenstrahlen Und Der Bildgebenden Verfahren","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140916879","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Johannes Beat Fingerhut, Charlotte Kulka, Michael Doppler, Katharina Vogt, Wibke Uller, Niklas Verloh
{"title":"[Stent-PTA of tumor-related venous obstructions].","authors":"Johannes Beat Fingerhut, Charlotte Kulka, Michael Doppler, Katharina Vogt, Wibke Uller, Niklas Verloh","doi":"10.1055/a-2284-5587","DOIUrl":"10.1055/a-2284-5587","url":null,"abstract":"","PeriodicalId":21490,"journal":{"name":"Rofo-fortschritte Auf Dem Gebiet Der Rontgenstrahlen Und Der Bildgebenden Verfahren","volume":null,"pages":null},"PeriodicalIF":1.3,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140850459","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}