{"title":"Reflection-equivariant convolutional neural networks improve segmentation over reflection augmentation","authors":"Shuo Han, Jerry L Prince, A. Carass","doi":"10.1117/12.2549399","DOIUrl":"https://doi.org/10.1117/12.2549399","url":null,"abstract":"Convolutional neural networks (CNNs) have been successfully applied to human brain segmentation. To in- corporate the left and right symmetry property of the brain into a network architecture, we propose a 3D left-right-reflection equivariant network to segment the anatomical structures of the brain. We extended previous group convolutions to account for left-right paired labels in the delineation. The proposed networks were compared with conventional networks trained with left-right reflection data augmentation in several tasks, showing improved performance. This is also the first work to extend reflection-equivariant CNNs to left-right paired labels in the human brain.","PeriodicalId":300887,"journal":{"name":"Medical Imaging: Image Processing","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116498788","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Cardiac cine MRI left ventricle segmentation combining deep learning and graphical models","authors":"F. Guo, Matthew Ng, G. Wright","doi":"10.1117/12.2549321","DOIUrl":"https://doi.org/10.1117/12.2549321","url":null,"abstract":"We developed an approach for cardiac cine magnetic resonance image (MRI) left ventricle segmentation using small training dataset with noisy manual annotations. Our approach combined the strengths of deep neural networks and normalized cut with continuous regularization. We entered U-net coarse segmentation into the regularized normalized cut module that evaluates the partitioning quality within and between segmentation regions. The resulting challenging optimization problem was efficiently solved through upper bound relaxation in an iterative manner with guaranteed convergence. Within each iteration, we derived an upper bound of the high-order normalized cut term, which was combined with image-grid continuous regularization and solved using a continuous min-cut/max- ow framework. Using 5 and 10 subjects with noisy manual labels for network training, we observed much improved segmentation accuracy and minimized effects due to the size of training dataset and the quality of training annotations.","PeriodicalId":300887,"journal":{"name":"Medical Imaging: Image Processing","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133687271","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yabo Fu, Y. Lei, Yingzi Liu, Tonghe Wang, W. Curran, Tian Liu, P. Patel, Xiaofeng Yang
{"title":"Cone-beam Computed Tomography (CBCT) and CT image registration aided by CBCT-based synthetic CT","authors":"Yabo Fu, Y. Lei, Yingzi Liu, Tonghe Wang, W. Curran, Tian Liu, P. Patel, Xiaofeng Yang","doi":"10.1117/12.2549095","DOIUrl":"https://doi.org/10.1117/12.2549095","url":null,"abstract":"CBCT has been widely integrated into modern linear accelerators in radiation therapy for image guidance purpose due to its cost effectiveness and low dose to patients. Planning CT images are registered with CBCT images for patient setup, contour propagations and dose calculations. However, it is challenging to accurately register the two since CBCT images often contain much image artifacts and noise, and intensity between CT and CBCT is not consistent. Therefore, traditional DIRs with intensity-based image similarity measures such as sum of squared differences, mean absolute differences are not applicable to CT-CBCT image registration. To address this issue, we propose to synthesize a high quality CT from CBCT to reduce image artifacts and perform intensity correction prior to image registration. Traditional Demons registration method was used to register the CT images with the CBCT-based synthetic CT (sCT). CT-sCT registration was tested on 5 patients’ datasets. On average, the mean absolute error between the fixed and deformed images were reduced from 102.2 to 92.4 HU by replacing CBCT with sCT.","PeriodicalId":300887,"journal":{"name":"Medical Imaging: Image Processing","volume":"190 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131865056","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
K. Hoebel, V. Andrearczyk, Andrew L Beers, J. Patel, Ken Chang, A. Depeursinge, Henning Mueller, Jayashree Kalpathy-Cramer
{"title":"An exploration of uncertainty information for segmentation quality assessment","authors":"K. Hoebel, V. Andrearczyk, Andrew L Beers, J. Patel, Ken Chang, A. Depeursinge, Henning Mueller, Jayashree Kalpathy-Cramer","doi":"10.1117/12.2548722","DOIUrl":"https://doi.org/10.1117/12.2548722","url":null,"abstract":"Including uncertainty information in the assessment of a segmentation of pathologic structures on medical images, offers the potential to increase trust into deep learning algorithms for the analysis of medical imaging. Here, we examine options to extract uncertainty information from deep learning segmentation models and the influence of the choice of cost functions on these uncertainty measures. To this end we train conventional UNets without dropout, deep UNet ensembles, and Monte-Carlo (MC) dropout UNets to segment lung nodules on low dose CT using either soft Dice or weighted categorical cross-entropy (wcc) as loss functions. We extract voxel-wise uncertainty information from UNet models based on softmax maximum probability and from deep ensembles and MC dropout UNets using mean voxel-wise entropy. Upon visual assessment, areas of high uncertainty are localized in the periphery of segmentations and are in good agreement with incorrectly labelled voxels. Furthermore, we evaluate how well uncertainty measures correlate with segmentation quality (Dice score). Mean uncertainty over the segmented region (Ulabelled) derived from conventional UNet models does not show a strong quantitative relationship with the Dice score (Spearman correlation coefficient of -0.45 for the soft Dice vs -0.64 for the wcc model respectively). By comparison, image-level uncertainty measures derived from soft Dice as well as wcc MC UNet and deep UNet ensemble models correlate well with the Dice score. In conclusion, using uncertainty information offers ways to assess segmentation quality fully automatically without access to ground truth. Models trained using weighted categorical cross-entropy offer more meaningful uncertainty information on a voxel-level.","PeriodicalId":300887,"journal":{"name":"Medical Imaging: Image Processing","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114712656","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yabo Fu, Y. Lei, Jun Zhou, Tonghe Wang, A. Jani, P. Patel, H. Mao, W. Curran, Tian Liu, Xiaofeng Yang
{"title":"Non-rigid MRI-CT image registration with unsupervised deepdlearning-based deformation prediction","authors":"Yabo Fu, Y. Lei, Jun Zhou, Tonghe Wang, A. Jani, P. Patel, H. Mao, W. Curran, Tian Liu, Xiaofeng Yang","doi":"10.1117/12.2549317","DOIUrl":"https://doi.org/10.1117/12.2549317","url":null,"abstract":"In this study, we propose to use an unsupervised deep learning-based method to directly register the MRI to CT with the help of synthetic MRI (sMRI). Our synthesis results showed that CT-generated sMRI could partially restore soft-tissue contrast in the pelvic region, which helps to improve the image registration accuracy near the prostate, rectum and bladder. Additionally, sMRI has similar intensity value to the true MRI which make it easier to register the images. The registration network was trained in an unsupervised manner, meaning ground truth DVF is not required. The trained network can predict the deformation vector field (DVF) in a single shot. After registration, the structural similarity index metric (SSIM) and peak signal to noise ratio (PSNR) between the deformed sMRI and the fixed MRI were on average 0.79 and 25.15 dB respectively.","PeriodicalId":300887,"journal":{"name":"Medical Imaging: Image Processing","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130651824","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Influence of decoder size for binary segmentation tasks in medical imaging","authors":"J. V. D. Putten, F. V. D. Sommen, P. D. With","doi":"10.1117/12.2542199","DOIUrl":"https://doi.org/10.1117/12.2542199","url":null,"abstract":"Symmetric design of encoder-decoder networks is common in deep learning. For almost all segmentation problems, the output segmentation is vastly less complex compared to the input image. However, the effect of the size of the decoder on segmentation performance has not been investigated in literature. This work investigates the effect of reducing decoder size on binary segmentation performance in a medical imaging application. To this end, we propose a methodology to reduce the size of the decoder in encoder-decoder networks, where residual skip connections are employed in combination with a 1x1 convolution instead of concatenations (as employed by U-Net) to achieve models with asymmetric design. The results on the ISIC2017 data set show that the amount of trainable parameters in the decoder can be reduced by up to a factor 100 compared to standard U-Net, while retaining segmentation performance. Additionally, the reduced amount of trainable decoder parameters in the proposed models leads to inference times up to 3 times faster compared to standard U-Net.","PeriodicalId":300887,"journal":{"name":"Medical Imaging: Image Processing","volume":"196 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132199100","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Christian Neumann, Klaus D. Tönnies, R. Pohle-Fröhlich
{"title":"Deep similarity learning using a Siamese ResNet trained on similarity labels from disparity maps of cerebral MRA MIP pairs","authors":"Christian Neumann, Klaus D. Tönnies, R. Pohle-Fröhlich","doi":"10.1117/12.2548576","DOIUrl":"https://doi.org/10.1117/12.2548576","url":null,"abstract":"Similarity measures are a critical component for iterative registration tasks. In intrinsic 2D/3D registration an initialization close to the solution is usually needed for convergence. Traditional similarity measures are limited in their applicability to perspective projection data. Deep Learning allows us to create similarity measures that encode almost arbitrary non-linear relationships like perspective projection. We apply a siamese network and a 2-channel network to the problem of comparing two perspective maximum intensity projections of medical data. We propose the use of residual units as the building block. A major challenge is to define similarity under perspective projection so that it can be used for training. Extrinsic registration gives us a gold standard of registered examples but we need a way to quantify the dissimilarity of unregistered image pairs for training. We propose the use of disparity as the basis for construction of similarity labels, i.e. labels are based on distances between projected points when viewing the same 3D point through different cameras. The final similarity labels are independent of the choice of camera parametrization and describe the visual similarity of two given views of a common 3D model. We compare our deep similarity metric to traditional similarity measures for a comparative evaluation. Our best configuration has a Pearson correlation coefficient of 0.792 and a Spearman’s rank correlation coefficient of 0.480. The best traditional method is normalized cross correlation with a 5.4% lower Spearman’s rank correlation coefficient.","PeriodicalId":300887,"journal":{"name":"Medical Imaging: Image Processing","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125713476","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Attention-guided channel to pixel convolution network for retinal layer segmentation with choroidal neovascularization","authors":"Xiaoling Yang, Xinjian Chen, Dehui Xiang","doi":"10.1117/12.2548940","DOIUrl":"https://doi.org/10.1117/12.2548940","url":null,"abstract":"This paper introduces an attention-guided channel to pixel convolution network for a fully automatic segmentation of retinal layers with choroidal neovascularization from optical coherence tomography (OCT) images. The proposed framework, consists of two new strategies for retinal layers segmentation: Channel to Pixel Block and Attention block. To deal with the contrast reduction of adjacent retinal layers caused by choroidal neovascularization, we firstly design a Channel to Pixel Block to convert particular channels into pixels in one bigger feature map, followed by a convolution layer optimized by a novel edge loss. Faced with large morphological changes of retinal layers, the attention mechanism is then introduced to extract more context information. The proposed method was trained on augmented 1280 OCT images and tested on 384 OCT images with choroidal neovascularization. The experimental results showed that the proposed method outperformed the state-of-art methods for retinal OCT image segmentation.","PeriodicalId":300887,"journal":{"name":"Medical Imaging: Image Processing","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133576872","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Michael MacDonald, T. R. Fennel, A. Singanamalli, N. Cruz, Mohammad YousefHussein, Yousef Al-Kofahi, Benjamin S. Freedman
{"title":"Improved automated segmentation of human kidney organoids using deep convolutional neural networks","authors":"Michael MacDonald, T. R. Fennel, A. Singanamalli, N. Cruz, Mohammad YousefHussein, Yousef Al-Kofahi, Benjamin S. Freedman","doi":"10.1117/12.2549830","DOIUrl":"https://doi.org/10.1117/12.2549830","url":null,"abstract":"Organoids are multicellular structures grown in the lab that resemble tissues or organs of the body. We recently generated human kidney organoids compatible with high throughput screening for developmental and disease phenotypes. Accurately segmenting large-scale image collections of organoids remains a challenge. We investigated automated segmentation of these structures using both conventional image processing algorithms and two different deep convolutional neural network architectures. Our dataset consisted of multi-channel images of organoids in 384-well plates, labeling distal tubules, proximal tubules, and podocytes as distinct segments. These images were used either for training and validation, or for testing. Each image was initially subjected to automated segmentation using a customized CellProfiler workflow. Separately, we performed semantic organoid segmentation using a Residual UNet (ResUNet) architecture, and instance organoid segmentation using a Mask R-CNN (MRCNN) architecture. For the latter, we compared model performance after initializing network weights in three different ways: randomly, using ResNet-50 weights pre-trained on the COCO dataset, and using ResUNet weights pre-trained on organoid images. Using ResUNet or randomly initializing MRCNN backbone weights provided improved semantic segmentation compared to using precomputed weights from COCO or ResUNet, or to using the CellProfiler workflow. Conversely, using precomputed weights to initialize MRCNN provided better instance segmentation accuracy and sensitivity than random initialization. Our findings provide a basis for automated segmentation of organoids with convolutional neural networks, to aid in high throughput screening for compounds relevant to renal phenotypes.","PeriodicalId":300887,"journal":{"name":"Medical Imaging: Image Processing","volume":"78 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115146376","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Fully convolutional network with sparse feature-maps composition for automatic lung tumor segmentation from PET images","authors":"Haihong Tian, Dehui Xiang, Weifang Zhu, Fei Shi, Xinjian Chen","doi":"10.1117/12.2548670","DOIUrl":"https://doi.org/10.1117/12.2548670","url":null,"abstract":"Accurate lung tumor delineation plays an important role in radiotherapy treatment planning. Since the lung tumor has poor boundary in positron emission tomography (PET) images, it is a challenging task to accurately segment lung tumor. In addition, the heart, liver, bones and other tissues generally have the similar gray value as the lung tumor, therefore the segmentation results usually have high false positive. In this paper, we propose a novel and efficient fully convolutional network with a trainable compressed sensing module and deep supervision mechanism with sparse constraints to comprehensively address these challenges; and we call it fully convolutional network with sparse feature-maps composition (SFC-FCN). Our SFC-FCN is able to conduct end-to-end learning and inference, compress redundant features within channels and extract key uncorrelated features. In addition, we use deep a supervision mechanism with sparse constraints to guide the features extraction by a compressed sensing module. The mechanism is developed by driving an objective function that directly guides the training of both lower and upper layers in the network. We have achieved more accurate segmentation results than that of state-of-the-art approaches with a much faster speed and much fewer parameters.","PeriodicalId":300887,"journal":{"name":"Medical Imaging: Image Processing","volume":"56 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132732461","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}