Joint optimization of AI large and small models for surface temperature and emissivity retrieval using knowledge distillation

Abstract

The rapid advancement of artificial intelligence in domains such as natural language processing has catalyzed AI research across various fields. This study introduces a novel strategy, the AutoKeras-Knowledge Distillation (AK-KD), which integrates knowledge distillation technology for joint optimization of large and small models in the retrieval of surface temperature and emissivity using thermal infrared remote sensing. The approach addresses the challenges of limited accuracy in surface temperature retrieval by employing a high-performance large model developed through AutoKeras as the teacher model, which subsequently enhances a less accurate small model through knowledge distillation. The resultant student model is interactively integrated with the large model to further improve specificity and generalization capabilities. Theoretical derivations and practical applications validate that the AK-KD strategy significantly enhances the accuracy of temperature and emissivity retrieval. For instance, a large model trained with simulated ASTER data achieved a Pearson Correlation Coefficient (PCC) of 0.999 and a Mean Absolute Error (MAE) of 0.348 K in surface temperature retrieval. In practical applications, this model demonstrated a PCC of 0.967 and an MAE of 0.685 K. Although the large model exhibits high average accuracy, its precision in complex terrains is comparatively lower. To ameliorate this, the large model, serving as a teacher, enhances the small model's local accuracy. Specifically, in surface temperature retrieval, the small model's PCC improved from an average of 0.978 to 0.979, and the MAE decreased from 1.065 K to 0.724 K. In emissivity retrieval, the PCC rose from an average of 0.827 to 0.898, and the MAE reduced from 0.0076 to 0.0054. This research not only provides robust technological support for further development of thermal infrared remote sensing in temperature and emissivity retrieval but also offers important references and key technological insights for the universal model construction of other geophysical parameter retrievals.