Neural Sign Language Translation with SF-Transformer

The popular methods are based on the combination of CNNs and RNNs in the sign language translation. Recently, Transformer has also attracted the attention of researchers and achieved success in this subject. However, researchers usually only focus on the accuracy of their model, while ignoring the practical application value. In this paper, we propose the SF-Transformer, a lightweight model based on Encoder-Decoder architecture for sign language translation, which achieves new state-of-the-art performance on Chinese Sign Language (CSL) dataset. We used 2D/3D convolution blocks of SF-Net and Transformer's Decoders to build our network. Benefiting from fewer parameters and a high level of parallelization, the training and inference speed of our model is faster. We hope that our method can contribute to the practical application of sign language translation on low-computing devices such as mobile phones.