Tom Julius Blöcker, Elia Lombardo, Sebastian Marschner, Claus Belka, Stefanie Corradini, Miguel A Palacios, Marco Riboldi, Christopher Kurz, Guillaume Landry
{"title":"MRgRT real-time target localization using foundation models for contour point tracking and promptable mask refinement.","authors":"Tom Julius Blöcker, Elia Lombardo, Sebastian Marschner, Claus Belka, Stefanie Corradini, Miguel A Palacios, Marco Riboldi, Christopher Kurz, Guillaume Landry","doi":"10.1088/1361-6560/ad9dad","DOIUrl":null,"url":null,"abstract":"<p><strong>Objective: </strong>This study aimed to evaluate two real-time target tracking approaches for magnetic resonance imaging (MRI) guided radiotherapy (MRgRT) based on foundation artificial intelligence (AI) models.

Approach: The first approach used a point-tracking model that propagates points from a reference contour. The second approach used a video-object-segmentation model, based on Segment Anything Model 2 (SAM2). Both approaches were evaluated and compared against each other, inter-observer variability, and a transformer-based image registration model, TransMorph, with and without patient-specific (PS) fine-tuning. The evaluation was carried out on 2D cine MRI datasets from two institutions, containing scans from 33 patients with 8060 labeled frames, with annotations from 2 to 5 observers per frame, totaling 29179 ground truth segmentations. The segmentations produced were assessed using the Dice similarity coefficient (DSC), 50% and 95% Hausdorff distances (HD50 / HD95), and the Euclidean center distance (ECD).

Main results: The results showed that the contour tracking (median DSC 0.92 ± 0.04 and ECD 1.9 ± 1.0 mm) and SAM2-based (median DSC 0.93 ± 0.03 and ECD 1.6 ± 1.1 mm) approaches produced target segmentations comparable or superior to TransMorph without PS fine-tuning (median DSC 0.91 ± 0.07 and ECD 2.6 ± 1.4 mm) and slightly inferior to TransMorph with PS fine-tuning (median DSC 0.94 ± 0.03 and ECD 1.4 ± 0.8 mm). Between the two novel approaches, the one based on SAM2 performed marginally better at a higher computational cost (inference times 92 ms for contour tracking and 109 ms for SAM2). Both approaches and TransMorph with PS fine-tuning exceeded inter-observer variability (median DSC 0.90 ± 0.06 and ECD 1.7 ± 0.7 mm).

Significance: This study demonstrates the potential of foundation models to achieve high-quality real-time target tracking in MRgRT, offering performance that matches state-of-the-art methods without requiring PS fine-tuning.</p>","PeriodicalId":20185,"journal":{"name":"Physics in medicine and biology","volume":" ","pages":""},"PeriodicalIF":3.3000,"publicationDate":"2024-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Physics in medicine and biology","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1088/1361-6560/ad9dad","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, BIOMEDICAL","Score":null,"Total":0}
引用次数: 0
Abstract
Objective: This study aimed to evaluate two real-time target tracking approaches for magnetic resonance imaging (MRI) guided radiotherapy (MRgRT) based on foundation artificial intelligence (AI) models.
Approach: The first approach used a point-tracking model that propagates points from a reference contour. The second approach used a video-object-segmentation model, based on Segment Anything Model 2 (SAM2). Both approaches were evaluated and compared against each other, inter-observer variability, and a transformer-based image registration model, TransMorph, with and without patient-specific (PS) fine-tuning. The evaluation was carried out on 2D cine MRI datasets from two institutions, containing scans from 33 patients with 8060 labeled frames, with annotations from 2 to 5 observers per frame, totaling 29179 ground truth segmentations. The segmentations produced were assessed using the Dice similarity coefficient (DSC), 50% and 95% Hausdorff distances (HD50 / HD95), and the Euclidean center distance (ECD).
Main results: The results showed that the contour tracking (median DSC 0.92 ± 0.04 and ECD 1.9 ± 1.0 mm) and SAM2-based (median DSC 0.93 ± 0.03 and ECD 1.6 ± 1.1 mm) approaches produced target segmentations comparable or superior to TransMorph without PS fine-tuning (median DSC 0.91 ± 0.07 and ECD 2.6 ± 1.4 mm) and slightly inferior to TransMorph with PS fine-tuning (median DSC 0.94 ± 0.03 and ECD 1.4 ± 0.8 mm). Between the two novel approaches, the one based on SAM2 performed marginally better at a higher computational cost (inference times 92 ms for contour tracking and 109 ms for SAM2). Both approaches and TransMorph with PS fine-tuning exceeded inter-observer variability (median DSC 0.90 ± 0.06 and ECD 1.7 ± 0.7 mm).
Significance: This study demonstrates the potential of foundation models to achieve high-quality real-time target tracking in MRgRT, offering performance that matches state-of-the-art methods without requiring PS fine-tuning.
期刊介绍:
The development and application of theoretical, computational and experimental physics to medicine, physiology and biology. Topics covered are: therapy physics (including ionizing and non-ionizing radiation); biomedical imaging (e.g. x-ray, magnetic resonance, ultrasound, optical and nuclear imaging); image-guided interventions; image reconstruction and analysis (including kinetic modelling); artificial intelligence in biomedical physics and analysis; nanoparticles in imaging and therapy; radiobiology; radiation protection and patient dose monitoring; radiation dosimetry