Traditional deep learning methods for rolling bearing fault diagnosis require a lot of computational time and resources. At the same time, the accuracy of fault diagnosis is affected by missing data collected due to the instability of sensors or data acquisition systems. In this paper, we propose a fault diagnosis method based on Gated Recurrent Unit with Decays and Maskless Update—Depthwise Separable Convolution Neural Network (GRUDMU-DSCNN). First, we use the trainable attenuation mechanism in GRUDMU for effective imputation of missingness and change the position of mask vectors to deal with missing data and solve the problem of missing data affecting the accuracy of fault diagnosis. In addition, we combine GRUDMU with DSCNN and deploy the model to edge devices. This improves the effectiveness of real-time fault diagnosis in edge computing scenarios. Furthermore, to verify whether the proposed method is effective in improving the accuracy of fault diagnosis in two missing patterns, namely Interval Missing and Missing Completely At Random (MCAR), we used a customized experimental equipment dataset and open experiments. The NVIDIA Jetson Xavier NX suite served as the edge computing platform to verify the effectiveness and superiority of the proposed model. The results indicate an average improvement in classification accuracy of 8.07% and 9.65% on both datasets when compared to existing methods.