When Multi-Focus Image Fusion Meets Nonlinear Spiking Neural P Systems

Abstract

Multi-focus image fusion (MFIF) aims to combine multiple images captured in the same scene by imaging devices with varying focal lengths into one complete clear image. While artificial neural network-based methods have achieved remarkable results in MFIF tasks, their black-box working mechanisms easily leadto information losses, limiting further fusion performance improvements. To solve this issue, we introduce an interpretable neural computation model called the nonlinear spiking neural P (NSNP) system. The NSNP model effectively mitigates the information losses induced by neurons during the information transmission process by controlling the internal spike values of neurons. According to the NSNP model, we propose a novel fusion method, NSNPFuse, that effectively avoids information losses during the challenging MFIF task. On the one hand, NSNPFuse uses nonlinear spiking neurons to construct the network backbone, which yields improved feature extraction performance and reduces the induced feature loss. On the other hand, NSNPFuse embeds a feature fusion module (FFM) based on nonlinear spiking neurons to selectively retain meaningful information and reduce distortions. We conduct experiments on multiple multi-focus image datasets, including Lytro, MFFW, MFI-WHU, and Road-MF, and the subjective and objective performances of the proposed approach surpass those of 15 state-of-the-art MFIF methods. The results demonstrate that our NSNPFuse method offers more competitive performance. Furthermore, we show that NSNPFuse enhances the downstream performance achieved in salient detection and object detection tasks.