CDME: Convolutional Dictionary Iterative Model for Pansharpening With Mixture of Experts

Abstract

In this letter, we propose a convolutional dictionary iterative model for pansharpening with a mixture of experts. First, we define an observation model to model the common and unique feature information between multispectral (MS) and panchromatic (PAN) images. During this process, a proximal gradient algorithm is used to iteratively update the network parameters. The adaptive expert module (AEM) is designed to handle the unique and common features separately by using PAN mixture of experts (PMOE), multispectral mixture-of-experts (MMOE), and common mixture-of-experts (CMOE) modules, to achieve effective information reconstruction. Finally, the expert mixture fusion module (EMFM) adaptively integrates the information from the three mixture-of-experts (MOE) components by dynamically adjusting their respective weights, resulting in the final fused image. We conducted full-resolution and reduce-resolution experiments on GF2 and WV3 datasets with current state-of-the-art methods, and the experimental results show that our method performs best. The code is released on https://github.com/who15/CDME.