The Grain for Green Project is a significant environmental protection initiative in China designed to maintain ecological benefits through large-scale vegetation restoration. Such projects primarily affect vegetation cover, which in turn influences soil moisture dynamics. This study investigates the changes in surface soil moisture and total soil moisture in the Yellow River Basin before and after the implementation of the Grain for Green Project, thereby assessing its impact on soil moisture conditions. By calculating the trends of soil moisture and NDVI for the periods 1982–1998 and 1999–2014, the effects of the Grain for Green Project on soil moisture were evaluated. We employed partial correlation analysis to obtain the relationship between soil moisture and NDVI. Additionally, an Long Short-Term Memory (LSTM) network model and the SHapley Additive exPlanations (SHAP) values were used to identify the key factors influencing soil moisture. The results indicated that the areas with a significant increase in vegetation are mainly concentrated in the middle reaches of the Yellow River Basin. Moreover, the Grain for Green Project has resulted in a decreasing trend in surface soil moisture and total soil moisture across more than 60% of the Yellow River Basin, with an average reduction of 0.016 m3·m−3·decade−1 in the trend of surface soil moisture and 0.021 m3·m−3·decade−1 in the trend of total soil moisture. Furthermore, precipitation was found to have the greatest impact on surface soil moisture, while temperature had the most significant influence on total soil moisture. This study provides valuable insights into the effectiveness of the Grain for Green Project in promoting vegetation growth and soil moisture conservation and encourages sustainable management of land and water resources in the Yellow River Basin and beyond.