Michal K. Grzeszczyk, Przemysław Korzeniowski, Samer Alabed, Andrew J. Swift, Tomasz Trzciński, Arkadiusz Sitek
{"title":"TabMixer: Noninvasive Estimation of the Mean Pulmonary Artery Pressure via Imaging and Tabular Data Mixing","authors":"Michal K. Grzeszczyk, Przemysław Korzeniowski, Samer Alabed, Andrew J. Swift, Tomasz Trzciński, Arkadiusz Sitek","doi":"arxiv-2409.07564","DOIUrl":null,"url":null,"abstract":"Right Heart Catheterization is a gold standard procedure for diagnosing\nPulmonary Hypertension by measuring mean Pulmonary Artery Pressure (mPAP). It\nis invasive, costly, time-consuming and carries risks. In this paper, for the\nfirst time, we explore the estimation of mPAP from videos of noninvasive\nCardiac Magnetic Resonance Imaging. To enhance the predictive capabilities of\nDeep Learning models used for this task, we introduce an additional modality in\nthe form of demographic features and clinical measurements. Inspired by\nall-Multilayer Perceptron architectures, we present TabMixer, a novel module\nenabling the integration of imaging and tabular data through spatial, temporal\nand channel mixing. Specifically, we present the first approach that utilizes\nMultilayer Perceptrons to interchange tabular information with imaging features\nin vision models. We test TabMixer for mPAP estimation and show that it\nenhances the performance of Convolutional Neural Networks, 3D-MLP and Vision\nTransformers while being competitive with previous modules for imaging and\ntabular data. Our approach has the potential to improve clinical processes\ninvolving both modalities, particularly in noninvasive mPAP estimation, thus,\nsignificantly enhancing the quality of life for individuals affected by\nPulmonary Hypertension. We provide a source code for using TabMixer at\nhttps://github.com/SanoScience/TabMixer.","PeriodicalId":501289,"journal":{"name":"arXiv - EE - Image and Video Processing","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - EE - Image and Video Processing","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.07564","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
Abstract
Right Heart Catheterization is a gold standard procedure for diagnosing
Pulmonary Hypertension by measuring mean Pulmonary Artery Pressure (mPAP). It
is invasive, costly, time-consuming and carries risks. In this paper, for the
first time, we explore the estimation of mPAP from videos of noninvasive
Cardiac Magnetic Resonance Imaging. To enhance the predictive capabilities of
Deep Learning models used for this task, we introduce an additional modality in
the form of demographic features and clinical measurements. Inspired by
all-Multilayer Perceptron architectures, we present TabMixer, a novel module
enabling the integration of imaging and tabular data through spatial, temporal
and channel mixing. Specifically, we present the first approach that utilizes
Multilayer Perceptrons to interchange tabular information with imaging features
in vision models. We test TabMixer for mPAP estimation and show that it
enhances the performance of Convolutional Neural Networks, 3D-MLP and Vision
Transformers while being competitive with previous modules for imaging and
tabular data. Our approach has the potential to improve clinical processes
involving both modalities, particularly in noninvasive mPAP estimation, thus,
significantly enhancing the quality of life for individuals affected by
Pulmonary Hypertension. We provide a source code for using TabMixer at
https://github.com/SanoScience/TabMixer.