TripleS: Mitigating multi-task learning conflicts for semantic change detection in high-resolution remote sensing imagery

Abstract

Periodical earth observation from multi-temporal high spatial resolution remote sensing imagery (RSI) offers valuable insights into the complex dynamics of land surface changes. Semantic change detection (SCD), cooperating with deep learning (DL) architectures, has evolved from binary change detection (BCD) into an effective technique capable of not only identifying change locations but also specifying land-cover and land-use (LCLU) categories. Recent advancements suggest that SCD can be modeled as a multi-task learning (MTL) framework, involving multiple branches for individual subtasks to process dual RSI inputs, and optimized through joint training. However, limitations remain in the inadequate interactions between bi-temporal branches and semantic-change branches, as well as the pervasive gradient conflicts among subtasks within MTL frameworks, which can lead to counterbalanced performances. To address the above limitations, we propose an MTL-oriented SCD model (MOSCD), which mutually enhances bi-temporal features, while ensuring that representations across the subtask branches are coherently correlated. Furthermore, the TripleS framework is designed to enhance the optimization of the MTL framework through counteracting the conflicting subtask objectives, which incorporates three novel schemes: Stepwise multi-task optimization, Selective parameter binding, and Scheduling for dynamically training MTL bindings. Extensive experiments conducted on three full-coverage land-cover SCD datasets, including one public dataset (HRSCD) and two self-constructed datasets (SC-SCD7 and CC-SCD5), demonstrate that the MOSCD enhanced with TripleS outperforms eleven existing SCD methods and three MTL methods by up to 21.17% on SeK metrics. The robust performances over diverse landscapes and transferability on other componentized benchmarks validate that the MOSCD trained with TripleS is a practicable tool for detecting subtle land-cover changes from high spatial resolution RSI data. Codes and the two constructed datasets will be available at https://github.com/StephenApX/MTL-TripleS.