Graph-based cell pattern recognition for merging the multi-modal optical microscopic image of neurons

A deep understanding of neuron structure and function is crucial for elucidating brain mechanisms, diagnosing and treating diseases. Optical microscopy, pivotal in neuroscience, illuminates neuronal shapes, projections, and electrical activities. To explore the projection of specific functional neurons, scientists have been developing optical-based multimodal imaging strategies to simultaneously capture dynamic in vivo signals and static ex vivo structures from the same neuron. However, the original position of neurons is highly susceptible to displacement during ex vivo imaging, presenting a significant challenge for integrating multimodal information at the single-neuron level. This study introduces a graph-model-based approach for cell image matching, facilitating precise and automated pairing of sparsely labeled neurons across different optical microscopic images. It has been shown that utilizing neuron distribution as a matching feature can mitigate modal differences, the high-order graph model can address scale inconsistency, and the nonlinear iteration can resolve discrepancies in neuron density. This strategy was applied to the connectivity study of the mouse visual cortex, performing cell matching between the two-photon calcium image and the HD-fMOST brain-wide anatomical image sets. Experimental results demonstrate 96.67% precision, 85.29% recall rate, and 90.63% F1 Score, comparable to expert technicians. This study builds a bridge between functional and structural imaging, offering crucial technical support for neuron classification and circuitry analysis.