GSAP: A Hybrid GRU and Self-Attention Based Model for Dual Medical NLP Tasks

Abstract

This paper proposes a hybrid Gated Recurrent Unit (GRU) and Self-Attention based model, named GSAP, for dual medical related NLP tasks. GSAP stacks three famous neural network units: GRU, self-attention and pooling of Convolutional Neural Network (CNN) to improve the accuracy. In the GSAP, GRU is first adopted to comprehend sentences. Second, the Self-Attention layer helps the model to focus on key points of inputs. Finally, the pooling layer eases the outfitting and upgrades the system accuracy. The proposed GSAP is applied to two different medical NLP tasks: medical QA matching and smoking status classification and demonstrates outstanding results. In the smoking prediction, GSAP obtains an accuracy around 80%. Regarding to the medical QA matching task, GSAP upgrades the accuracy up to around 90%.