A 2.5d Yolo-Based Fusion Algorithm for 3d Localization Of Cells

Advances in microscopy techniques such as lattice-light-sheet, confocal, two-photon, and electron microscopy have enabled the visualization of 3D image volumes of tightly packed cells, extracellular structures in tissues, organelles, and subcellular components inside cells. These images sampled by 2D projections are often not accurately interpreted even by human experts. As a use case we focus on 3D image volumes of tightly packed nuclei in brain tissue. Due to out-of-plane excitation and low resolution in the z-axis, non-overlapping cells appear as overlapping 3D volumes and make detecting individual cells challenging. On the other hand, running 3D detection algorithms is computationally expensive and infeasible for large datasets. In addition, most existing 3D algorithms are designed to extract 3D objects by identifying the depth in the 2D images. In this work, we propose a YOLO-based 2.5D fusion algorithm for 3D localization of individual cells in densely packed volumes of nuclei. The proposed method fuses 2D detection of nuclei in sagittal, coronal, and axial planes and predicts six coordinates of the 3D bounding cubes around the detected 3D cells. Promising results were obtained on multiple examples of synthetic dense volumes of nuclei imitating confocal microscopy experimental datasets.