A new machine-learning based cloud mask using harmonized data of two Meteosat generations shows a general decrease in cloudiness over Europe in recent decades

Abstract

Mid-latitude stratus clouds with large spatial extent are important cooling engines in a warming world, while other types of clouds may accelerate warming. However, our understanding of cloud feedback in a changing climate remains incomplete in both space and time. A key factor contributing to this knowledge gap is the lack of long-term observations with spatio-temporally continuous information over large areas. Satellite data from the geostationary orbit could help in this regard, but they were never originally intended for climatological studies and, as a result, provide inconsistent data between individual satellites of different satellite generations. However, for investigations on a time scale of 30 years and more, a homogeneous dataset of gross cloud occurrence is essential to assess changes in cloud cover over the last decades. In addition, such a dataset is the basis for further analyzing long-term changes in other cloud types such as fog and low stratus (FLS). The generation of temporally homogeneous cloud information over Europe requires a dataset that is consistent in space and time. The current study develops a new cloud detection scheme based on harmonized radiances obtained by cross-calibrating Meteosat First (MFG) MVIRI (Meteosat Visible Infra-Red Imager) and Second Generation (MSG) SEVIRI (Spinning Enhanced Visible and Infra-Red Imager) data. The harmonized data set consists of two MFG bands (thermal infrared IR and water vapour WV), which guarantee long-term (1991–2020) availability over the full diurnal cycle (24 h). The new cloud classification scheme is based on eXtreme Gradient Boosting (XGBoost) and uses the two MFG channels as primary predictors or features. While cloud detection using only two MFG channels is a challenging task, additional features such as temporal trends in brightness temperature (BT), its spatial heterogeneity, clear sky reference BTs, topographic variables, and solar and satellite angles are also considered in the XGBoost model. The EUMETSAT CM SAF SEVIRI cloud mask based on MSG SEVIRI is used in part as the binary target variable to train the XGBoost model (cloudy/clear-sky) and as a benchmark to test the performance of the newly developed cloud detection scheme. Test results show very good agreement with the benchmark CM SAF SEVIRI cloud mask, with an average Heidke Skill Score (HSS) of 0.83 for day-time and 0.8 for night-time cloud occurrence. Further testing shows that the new cloud mask clearly outperforms the existing EUMETSAT Optimal Cloud Analysis (OCA) dataset based on MSG visible and IR 10.8 μm channels. In particular, the FLS detection in our cloud mask was found to be superior to the OCA during night and boreal winter. Based on the trend analysis of the generated time series of cloud frequencies, we found a general decrease in cloudiness over the last 30 years in many parts of Europe.