Design and evaluation of an AR-based thermal imaging system for planning reconstructive surgeries.

IF 2.3 3区 医学 Q3 ENGINEERING, BIOMEDICAL
Michael Unger, Annika Hänel, Claire Chalopin, Dirk Halama
{"title":"Design and evaluation of an AR-based thermal imaging system for planning reconstructive surgeries.","authors":"Michael Unger, Annika Hänel, Claire Chalopin, Dirk Halama","doi":"10.1007/s11548-024-03184-1","DOIUrl":null,"url":null,"abstract":"<p><strong>Introduction: </strong>Thermal imaging can be used for the non-invasive detection of blood vessels of the skin. However, mapping the results to the patient currently lacks user-friendliness. Augmented reality may provide a useful tool to superimpose thermal information on the patient.</p><p><strong>Methods: </strong>A system to support planning in reconstructive surgery using a thermal camera was designed. The obtained information was superimposed on the physical object using a Microsoft HoloLens. An RGB, depth, and thermal camera were combined to capture a scene of different modalities and reconstruct a virtual scene in real time. To register the different cameras and the AR device, an active calibration target was developed and evaluated. A Vuforia marker was used to register the hologram in the virtual space. The accuracy of the projected hologram was evaluated in a laboratory setting with participants by measuring the error between the physical object and the hologram.</p><p><strong>Results: </strong>The AR-based system was evaluated by 21 participants in a laboratory setting. The mean projection error is 10.3 ± 9.4 mm. The system is able to stream a three-dimensional scene with augmented thermal information in real time at 5 frames per second. The active calibration target can be used independently of the environment.</p><p><strong>Conclusion: </strong>The calibration target provides an easy-to-use method for the registration of cameras capturing the visible to long-infrared spectral range. The inside-out tracking of the HoloLens in combination with a Vuforia marker is not accurate enough for the intended clinical use.</p>","PeriodicalId":51251,"journal":{"name":"International Journal of Computer Assisted Radiology and Surgery","volume":" ","pages":"1659-1666"},"PeriodicalIF":2.3000,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Computer Assisted Radiology and Surgery","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1007/s11548-024-03184-1","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/5/24 0:00:00","PubModel":"Epub","JCR":"Q3","JCRName":"ENGINEERING, BIOMEDICAL","Score":null,"Total":0}
引用次数: 0

Abstract

Introduction: Thermal imaging can be used for the non-invasive detection of blood vessels of the skin. However, mapping the results to the patient currently lacks user-friendliness. Augmented reality may provide a useful tool to superimpose thermal information on the patient.

Methods: A system to support planning in reconstructive surgery using a thermal camera was designed. The obtained information was superimposed on the physical object using a Microsoft HoloLens. An RGB, depth, and thermal camera were combined to capture a scene of different modalities and reconstruct a virtual scene in real time. To register the different cameras and the AR device, an active calibration target was developed and evaluated. A Vuforia marker was used to register the hologram in the virtual space. The accuracy of the projected hologram was evaluated in a laboratory setting with participants by measuring the error between the physical object and the hologram.

Results: The AR-based system was evaluated by 21 participants in a laboratory setting. The mean projection error is 10.3 ± 9.4 mm. The system is able to stream a three-dimensional scene with augmented thermal information in real time at 5 frames per second. The active calibration target can be used independently of the environment.

Conclusion: The calibration target provides an easy-to-use method for the registration of cameras capturing the visible to long-infrared spectral range. The inside-out tracking of the HoloLens in combination with a Vuforia marker is not accurate enough for the intended clinical use.

Abstract Image

设计和评估基于 AR 的热成像系统,用于规划重建手术。
引言热成像技术可用于无创皮肤血管检测。然而,将结果映射到病人身上目前还缺乏用户友好性。增强现实技术可以提供一种有用的工具,将热信息叠加到病人身上:方法:设计了一个使用热像仪支持重建手术规划的系统。方法:设计了一套利用热像仪支持重建手术规划的系统,利用微软 HoloLens 将获得的信息叠加到实物上。RGB、深度和红外热像仪结合使用,可捕捉不同模式的场景并实时重建虚拟场景。为了注册不同的摄像头和 AR 设备,开发并评估了一个主动校准目标。Vuforia 标记用于在虚拟空间中注册全息图。通过测量实物与全息图之间的误差,在实验室环境中与参与者一起评估了投射全息图的准确性:21 名参与者在实验室环境中对基于 AR 的系统进行了评估。平均投影误差为 10.3 ± 9.4 毫米。该系统能够以每秒 5 帧的速度实时传输带有增强热信息的三维场景。主动校准目标的使用不受环境影响:校准目标为捕捉可见光至长红外光谱范围的摄像机提供了一种易于使用的注册方法。HoloLens 结合 Vuforia 标记的内向外跟踪对于预期的临床用途来说不够精确。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
International Journal of Computer Assisted Radiology and Surgery
International Journal of Computer Assisted Radiology and Surgery ENGINEERING, BIOMEDICAL-RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING
CiteScore
5.90
自引率
6.70%
发文量
243
审稿时长
6-12 weeks
期刊介绍: The International Journal for Computer Assisted Radiology and Surgery (IJCARS) is a peer-reviewed journal that provides a platform for closing the gap between medical and technical disciplines, and encourages interdisciplinary research and development activities in an international environment.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信