{"title":"Multi-View and Multi-Scale Alignment for Contrastive Language-Image Pre-training in Mammography.","authors":"Yuexi Du, John A Onofrey, Nicha C Dvornek","doi":"10.1007/978-3-031-96625-5_17","DOIUrl":"10.1007/978-3-031-96625-5_17","url":null,"abstract":"<p><p>Contrastive Language-Image Pre-training (CLIP) demonstrates strong potential in medical image analysis but requires substantial data and computational resources. Due to these restrictions, existing CLIP applications in medical imaging focus mainly on modalities like chest X-rays that have abundant image-report data available, leaving many other important modalities under-explored. Here, we propose one of the first adaptations of the full CLIP model to mammography, which presents significant challenges due to labeled data scarcity, high-resolution images with small regions of interest, and class-wise imbalance. We first develop a specialized supervision framework for mammography that leverages its multi-view nature. Furthermore, we design a symmetric local alignment module to better focus on detailed features in high-resolution images. Lastly, we incorporate a parameter-efficient fine-tuning approach for large language models pre-trained with medical knowledge to address data limitations. Our multi-view and multi-scale alignment (MaMA) method outperforms state-of-the-art baselines for three different tasks on two large real-world mammography datasets, EMBED and RSNA-Mammo, with only 52% model size compared with the largest baseline.</p>","PeriodicalId":73379,"journal":{"name":"Information processing in medical imaging : proceedings of the ... conference","volume":"15830 ","pages":"247-262"},"PeriodicalIF":0.0,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12456755/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145139326","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Y. Huang, Wanqing Xie, Mingzhen Li, Mingmei Cheng, Jinzhou Wu, Weixiao Wang, Jane You, Xiaofeng Liu
{"title":"Vicinal Feature Statistics Augmentation for Federated 3D Medical Volume Segmentation","authors":"Y. Huang, Wanqing Xie, Mingzhen Li, Mingmei Cheng, Jinzhou Wu, Weixiao Wang, Jane You, Xiaofeng Liu","doi":"10.1007/978-3-031-34048-2_28","DOIUrl":"https://doi.org/10.1007/978-3-031-34048-2_28","url":null,"abstract":"","PeriodicalId":73379,"journal":{"name":"Information processing in medical imaging : proceedings of the ... conference","volume":"5 1","pages":"360-371"},"PeriodicalIF":0.0,"publicationDate":"2023-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"78747201","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Better Generalization of White Matter Tract Segmentation to Arbitrary Datasets with Scaled Residual Bootstrap","authors":"Wan Liu, Chuyang Ye","doi":"10.1007/978-3-031-34048-2_48","DOIUrl":"https://doi.org/10.1007/978-3-031-34048-2_48","url":null,"abstract":"","PeriodicalId":73379,"journal":{"name":"Information processing in medical imaging : proceedings of the ... conference","volume":"39 1","pages":"629-640"},"PeriodicalIF":0.0,"publicationDate":"2023-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79474426","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Unsupervised Adaptation of Polyp Segmentation Models via Coarse-to-Fine Self-Supervision","authors":"Jiexiang Wang, Chaoqi Chen","doi":"10.1007/978-3-031-34048-2_20","DOIUrl":"https://doi.org/10.1007/978-3-031-34048-2_20","url":null,"abstract":"","PeriodicalId":73379,"journal":{"name":"Information processing in medical imaging : proceedings of the ... conference","volume":"6 1","pages":"250-262"},"PeriodicalIF":0.0,"publicationDate":"2023-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82819634","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Ouyang, Li Chen, Gary Y. Li, Naveen Balaraju, Shubham Patil, C. Mehanian, Sourabh Kulhare, R. Millin, K. Gregory, Cynthia Gregory, Meihua Zhu, David O. Kessler, L. Malia, Almaz S. Dessie, J. Rabiner, D. Coneybeare, B. Shopsin, A. Hersh, C. Madar, J. Shupp, L. Johnson, Jacob Avila, K. Dwyer, P. Weimersheimer, B. Raju, J. Kruecker, A. Chen
{"title":"Weakly Semi-supervised Detection in Lung Ultrasound Videos","authors":"J. Ouyang, Li Chen, Gary Y. Li, Naveen Balaraju, Shubham Patil, C. Mehanian, Sourabh Kulhare, R. Millin, K. Gregory, Cynthia Gregory, Meihua Zhu, David O. Kessler, L. Malia, Almaz S. Dessie, J. Rabiner, D. Coneybeare, B. Shopsin, A. Hersh, C. Madar, J. Shupp, L. Johnson, Jacob Avila, K. Dwyer, P. Weimersheimer, B. Raju, J. Kruecker, A. Chen","doi":"10.1007/978-3-031-34048-2_16","DOIUrl":"https://doi.org/10.1007/978-3-031-34048-2_16","url":null,"abstract":"","PeriodicalId":73379,"journal":{"name":"Information processing in medical imaging : proceedings of the ... conference","volume":"39 1","pages":"195-207"},"PeriodicalIF":0.0,"publicationDate":"2023-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76189144","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"mSPD-NN: A Geometrically Aware Neural Framework for Biomarker Discovery from Functional Connectomics Manifolds.","authors":"Niharika S D'Souza, Archana Venkataraman","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Connectomics has emerged as a powerful tool in neuroimaging and has spurred recent advancements in statistical and machine learning methods for connectivity data. Despite connectomes inhabiting a matrix manifold, most analytical frameworks ignore the underlying data geometry. This is largely because simple operations, such as mean estimation, do not have easily computable closed-form solutions. We propose a geometrically aware neural framework for connectomes, i.e., the mSPD-NN, designed to estimate the geodesic mean of a collections of symmetric positive definite (SPD) matrices. The mSPD-NN is comprised of bilinear fully connected layers with tied weights and utilizes a novel loss function to optimize the matrix-normal equation arising from Fréchet mean estimation. Via experiments on synthetic data, we demonstrate the efficacy of our mSPD-NN against common alternatives for SPD mean estimation, providing competitive performance in terms of scalability and robustness to noise. We illustrate the real-world flexibility of the mSPD-NN in multiple experiments on rs-fMRI data and demonstrate that it uncovers stable biomarkers associated with subtle network differences among patients with ADHD-ASD comorbidities and healthy controls.</p>","PeriodicalId":73379,"journal":{"name":"Information processing in medical imaging : proceedings of the ... conference","volume":"2023 ","pages":"53-65"},"PeriodicalIF":0.0,"publicationDate":"2023-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11513587/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142523786","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Peiqi Wang, William M Wells, Seth Berkowitz, Steven Horng, Polina Golland
{"title":"Using Multiple Instance Learning to Build Multimodal Representations.","authors":"Peiqi Wang, William M Wells, Seth Berkowitz, Steven Horng, Polina Golland","doi":"10.1007/978-3-031-34048-2_35","DOIUrl":"10.1007/978-3-031-34048-2_35","url":null,"abstract":"<p><p>Image-text multimodal representation learning aligns data across modalities and enables important medical applications, e.g., image classification, visual grounding, and cross-modal retrieval. In this work, we establish a connection between multimodal representation learning and multiple instance learning. Based on this connection, we propose a generic framework for constructing permutation-invariant score functions with many existing multimodal representation learning approaches as special cases. Furthermore, we use the framework to derive a novel contrastive learning approach and demonstrate that our method achieves state-of-the-art results in several downstream tasks.</p>","PeriodicalId":73379,"journal":{"name":"Information processing in medical imaging : proceedings of the ... conference","volume":"397 1","pages":"457-470"},"PeriodicalIF":0.0,"publicationDate":"2023-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12456966/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88972802","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ye Han, Jared Vicory, Guido Gerig, Patricia Sabin, Hannah Dewey, Silvani Amin, Ana Sulentic, Christian Hertz, Matthew Jolley, Beatriz Paniagua, James Fishbaugh
{"title":"Hierarchical Geodesic Polynomial Model for Multilevel Analysis of Longitudinal Shape.","authors":"Ye Han, Jared Vicory, Guido Gerig, Patricia Sabin, Hannah Dewey, Silvani Amin, Ana Sulentic, Christian Hertz, Matthew Jolley, Beatriz Paniagua, James Fishbaugh","doi":"10.1007/978-3-031-34048-2_62","DOIUrl":"https://doi.org/10.1007/978-3-031-34048-2_62","url":null,"abstract":"<p><p>Longitudinal analysis is a core aspect of many medical applications for understanding the relationship between an anatomical subject's function and its trajectory of shape change over time. Whereas mixed-effects (or hierarchical) modeling is the statistical method of choice for analysis of longitudinal data, we here propose its extension as hierarchical geodesic polynomial model (HGPM) for multilevel analyses of longitudinal shape data. 3D shapes are transformed to a non-Euclidean shape space for regression analysis using geodesics on a high dimensional Riemannian manifold. At the subject-wise level, each individual trajectory of shape change is represented by a univariate geodesic polynomial model on timestamps. At the population level, multivariate polynomial expansion is applied to uni/multivariate geodesic polynomial models for both anchor points and tangent vectors. As such, the trajectory of an individual subject's shape changes over time can be modeled accurately with a reduced number of parameters, and population-level effects from multiple covariates on trajectories can be well captured. The implemented HGPM is validated on synthetic examples of points on a unit 3D sphere. Further tests on clinical 4D right ventricular data show that HGPM is capable of capturing observable effects on shapes attributed to changes in covariates, which are consistent with qualitative clinical evaluations. HGPM demonstrates its effectiveness in modeling shape changes at both subject-wise and population levels, which is promising for future studies of the relationship between shape changes over time and the level of dysfunction severity on anatomical objects associated with disease.</p>","PeriodicalId":73379,"journal":{"name":"Information processing in medical imaging : proceedings of the ... conference","volume":"13939 ","pages":"810-821"},"PeriodicalIF":0.0,"publicationDate":"2023-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10323213/pdf/nihms-1912654.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9807388","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mohammad Farazi, Zhangsihao Yang, Wenhui Zhu, Peijie Qiu, Yalin Wang
{"title":"TetCNN: Convolutional Neural Networks on Tetrahedral Meshes.","authors":"Mohammad Farazi, Zhangsihao Yang, Wenhui Zhu, Peijie Qiu, Yalin Wang","doi":"10.1007/978-3-031-34048-2_24","DOIUrl":"10.1007/978-3-031-34048-2_24","url":null,"abstract":"<p><p>Convolutional neural networks (CNN) have been broadly studied on images, videos, graphs, and triangular meshes. However, it has seldom been studied on tetrahedral meshes. Given the merits of using volumetric meshes in applications like brain image analysis, we introduce a novel interpretable graph CNN framework for the tetrahedral mesh structure. Inspired by ChebyNet, our model exploits the volumetric Laplace-Beltrami Operator (LBO) to define filters over commonly used graph Laplacian which lacks the Riemannian metric information of 3D manifolds. For pooling adaptation, we introduce new objective functions for localized minimum cuts in the Graclus algorithm based on the LBO. We employ a piece-wise constant approximation scheme that uses the clustering assignment matrix to estimate the LBO on sampled meshes after each pooling. Finally, adapting the Gradient-weighted Class Activation Mapping algorithm for tetrahedral meshes, we use the obtained heatmaps to visualize discovered regions-of-interest as biomarkers. We demonstrate the effectiveness of our model on cortical tetrahedral meshes from patients with Alzheimer's disease, as there is scientific evidence showing the correlation of cortical thickness to neurodegenerative disease progression. Our results show the superiority of our LBO-based convolution layer and adapted pooling over the conventionally used unitary cortical thickness, graph Laplacian, and point cloud representation.</p>","PeriodicalId":73379,"journal":{"name":"Information processing in medical imaging : proceedings of the ... conference","volume":"13939 ","pages":"303-315"},"PeriodicalIF":0.0,"publicationDate":"2023-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10765307/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139099273","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Learning Probabilistic Piecewise Rigid Atlases of Model Organisms via Generative Deep Networks.","authors":"Amin Nejatbakhsh, Neel Dey, Vivek Venkatachalam, Eviatar Yemini, Liam Paninski, Erdem Varol","doi":"10.1007/978-3-031-34048-2_26","DOIUrl":"10.1007/978-3-031-34048-2_26","url":null,"abstract":"<p><p>Atlases are crucial to imaging statistics as they enable the standardization of inter-subject and inter-population analyses. While existing atlas estimation methods based on fluid/elastic/diffusion registration yield high-quality results for the human brain, these deformation models do not extend to a variety of other challenging areas of neuroscience such as the anatomy of <i>C. elegans</i> worms and fruit flies. To this end, this work presents a general probabilistic deep network-based framework for atlas estimation and registration which can flexibly incorporate various deformation models and levels of keypoint supervision that can be applied to a wide class of model organisms. Of particular relevance, it also develops a deformable piecewise rigid atlas model which is regularized to preserve inter-observation distances between neighbors. These modeling considerations are shown to improve atlas construction and key-point alignment across a diversity of datasets with small sample sizes including neuron positions in <i>C. elegans</i> hermaphrodites, fluorescence microscopy of male <i>C. elegans</i>, and images of fruit fly wings. Code is accessible at https://github.com/amin-nejat/Deformable-Atlas.</p>","PeriodicalId":73379,"journal":{"name":"Information processing in medical imaging : proceedings of the ... conference","volume":"13939 ","pages":"332-343"},"PeriodicalIF":0.0,"publicationDate":"2023-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10358289/pdf/nihms-1910173.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10239564","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}