Accurate initial field estimation for weather forecasting with a variational constrained neural network

Weather forecasting is crucial for scientific research and society. Recently, deep learning (DL) methods have achieved significant advancements in medium-range weather forecasting. However, they generally depend on the initial fields generated by the computationally expensive four-dimensional variational (4DVar) data assimilation (DA) technique, which limits their real-time applicability in multivariate three-dimensional (3D) weather forecasting. Here we propose 4DVarFormer by exploring the potential of integrating the 4DVar constraint into an attention-based neural network. 4DVarFormer eliminates the need for background error covariance statistics and the complex adjoint model development. It can generate multivariate 3D weather states within 0.37 s. Moreover, 4DVarFormer can capture inter-variable relationships, allowing the assimilation of observed variables to correct unobserved variables. Hence, medium-range forecasts initiated by 4DVarFormer outperform those of DL-based DA methods and achieve performance comparable to the forecasts initiated by ERA5 reanalyses. These promising findings contribute to future advancements in integrated end-to-end DL weather forecasting systems.