用于解释3D医学图像和视频的多模态生成AI。

IF 12.4 1区医学 Q1 HEALTH CARE SCIENCES & SERVICES

NPJ Digital Medicine Pub Date : 2025-05-13 DOI:10.1038/s41746-025-01649-4

Jung-Oh Lee,Hong-Yu Zhou,Tyler M Berzin,Daniel K Sodickson,Pranav Rajpurkar

{"title":"用于解释3D医学图像和视频的多模态生成AI。","authors":"Jung-Oh Lee,Hong-Yu Zhou,Tyler M Berzin,Daniel K Sodickson,Pranav Rajpurkar","doi":"10.1038/s41746-025-01649-4","DOIUrl":null,"url":null,"abstract":"This perspective proposes adapting video-text generative AI to 3D medical imaging (CT/MRI) and medical videos (endoscopy/laparoscopy) by treating 3D images as videos. The approach leverages modern video models to analyze multiple sequences simultaneously and provide real-time AI assistance during procedures. The paper examines medical imaging's unique characteristics (synergistic information, metadata, and world model), outlines applications in automated reporting, case retrieval, and education, and addresses challenges of limited datasets, benchmarks, and specialized training.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":"25 1","pages":"273"},"PeriodicalIF":12.4000,"publicationDate":"2025-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Multimodal generative AI for interpreting 3D medical images and videos.\",\"authors\":\"Jung-Oh Lee,Hong-Yu Zhou,Tyler M Berzin,Daniel K Sodickson,Pranav Rajpurkar\",\"doi\":\"10.1038/s41746-025-01649-4\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"This perspective proposes adapting video-text generative AI to 3D medical imaging (CT/MRI) and medical videos (endoscopy/laparoscopy) by treating 3D images as videos. The approach leverages modern video models to analyze multiple sequences simultaneously and provide real-time AI assistance during procedures. The paper examines medical imaging's unique characteristics (synergistic information, metadata, and world model), outlines applications in automated reporting, case retrieval, and education, and addresses challenges of limited datasets, benchmarks, and specialized training.\",\"PeriodicalId\":19349,\"journal\":{\"name\":\"NPJ Digital Medicine\",\"volume\":\"25 1\",\"pages\":\"273\"},\"PeriodicalIF\":12.4000,\"publicationDate\":\"2025-05-13\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"NPJ Digital Medicine\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1038/s41746-025-01649-4\",\"RegionNum\":1,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"HEALTH CARE SCIENCES & SERVICES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"NPJ Digital Medicine","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1038/s41746-025-01649-4","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}

引用次数: 0

摘要

该观点提出通过将3D图像作为视频处理，将视频文本生成AI应用于3D医学成像（CT/MRI）和医学视频（内窥镜/腹腔镜）。该方法利用现代视频模型同时分析多个序列，并在过程中提供实时人工智能辅助。本文探讨了医学成像的独特特征（协同信息、元数据和世界模型），概述了在自动报告、病例检索和教育中的应用，并解决了有限数据集、基准和专业培训的挑战。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Multimodal generative AI for interpreting 3D medical images and videos.

This perspective proposes adapting video-text generative AI to 3D medical imaging (CT/MRI) and medical videos (endoscopy/laparoscopy) by treating 3D images as videos. The approach leverages modern video models to analyze multiple sequences simultaneously and provide real-time AI assistance during procedures. The paper examines medical imaging's unique characteristics (synergistic information, metadata, and world model), outlines applications in automated reporting, case retrieval, and education, and addresses challenges of limited datasets, benchmarks, and specialized training.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

NPJ Digital Medicine Multiple-

CiteScore

25.10

自引率

3.30%

发文量

170

审稿时长

15 weeks

期刊介绍： npj Digital Medicine is an online open-access journal that focuses on publishing peer-reviewed research in the field of digital medicine. The journal covers various aspects of digital medicine, including the application and implementation of digital and mobile technologies in clinical settings, virtual healthcare, and the use of artificial intelligence and informatics. The primary goal of the journal is to support innovation and the advancement of healthcare through the integration of new digital and mobile technologies. When determining if a manuscript is suitable for publication, the journal considers four important criteria: novelty, clinical relevance, scientific rigor, and digital innovation.