Enhancing SPARQL query generation for question answering with a hybrid encoder–decoder and cross-attention model

Abstract

A question-answering (QA) system is essential for helping users retrieve relevant and accurate answers based on their queries. The precision of SPARQL query syntax generation is directly linked to the accuracy of the answers provided. Recently, many studies on knowledge graph-based natural language question-answering (KGQA) systems have leveraged the Neural Machine Translation (NMT) framework to translate input questions into SPARQL query syntax, a process known as Text-to-SPARQL. In NMT, cross-attention-based Transformers, ConvS2S, and BiLSTM models are commonly used for training. However, comparing the translation performance of these models is challenging due to their significant architectural differences. To address this issue, this paper integrates various encoder and cross-attention methods with a fixed LSTM decoder to form hybrid models, which are then trained and evaluated on QA systems. Beyond the hybrid models discussed, this study introduces an improved ConvS2S architecture featuring a Multi-Head Convolutional (MHC) encoder, designated as QAWizer_MHC. The MHC encoder incorporates the Transformer’s multi-head attention mechanism to compute dependencies within the input sequence. Additionally, the enhanced ConvS2S model captures local hidden features across different receptive fields within the input sequence. Experimental results demonstrate that QAWizer_MHC outperforms other models, achieving BLEU-1 scores of 76.52% and 83.37% on the QALD-9 and LC-QuAD-1.0 datasets, respectively. Furthermore, in end-to-end system evaluations on the same datasets, the model attained Macro F1 scores of 52% and 66%, respectively, surpassing other KGQA systems. The experimental findings indicate that even with limited computational resources and general embeddings, a well-designed encoder–decoder architecture that integrates cross-attention can achieve performance comparable to large pre-trained models.