OptNet: Optimization-inspired network beyond deep unfolding for structural artifact reduction

Abstract

Structural artifact reduction (SAR), such as metal artifact reduction (MAR) in computed tomography (CT) images and single image deraining (SID), aims to remove the artifacts with repeated structural patterns from the corrupted images. Recently, deep unfolding networks, also known as model-driven networks, have achieved remarkable performance, but they typically require multiple proximity sub-networks to replace the corresponding proximal operators for multivariable updates, increasing the number of learnable parameters. Moreover, existing SAR methods ignore advanced priors, such as textual priors, leaving room for further recovery performance improvement. To address these limitations, we rethink the deep unfolding framework and propose a universal optimization-inspired network architecture, termed OptNet, which introduces a novel multi-channel network design to reduce learnable parameter count while enhancing performance via incorporating textual priors. Concretely, we design the so-called OptNet with contrastive loss to perform multivariable updates, replacing multiple proximity sub-networks typically in iterative optimization algorithms with a multi-channel sub-network, thus reducing the learnable parameter count. OptNet is flexible and can incorporate any advanced priors. Specially, we integrate a pre-trained contrastive language-image pretraining (CLIP) model into an elaborated information fusion module (IFM) to incorporate textual priors, enabling multimodal information interaction that guides more accurate structural artifact reduction, enhancing generalizability across various degradation levels. Extensive experiments demonstrate that OptNet outperforms existing methods, achieving improvements of up to 0.25 dB on MAR and 0.6 dB on SID tasks, while surpassing its deep unfolding variant with a 1.33 dB gain on MAR and reducing parameters by approximately 50%.