A Deep Learning Model of Multiple Knowledge Sources Integration for Community Question Answering

Abstract

The community Question Answering (cQA) problem requires the task that given a question it aims at selecting the most related question-answer tuples (a question and its answers) from the stored question-answer tuples data set. The core mission of this task is to measure the similarity (or relationship) between an input question and questions from the given question-answer data set. Under our observation, there are either various information sources as well as di erent measurement models which can provide complementary knowledge for indicating the relationship between questions and question-answer tuples. In this paper we address the problem of modeling and combining multiple knowledge sources for determining and ranking the most related question-answer tuples given an input question for cQA problem. Our proposed model will generate di erent features based on di erent representations of the data as well as on di erent methods and then integrate this information into the BERT model for similarity measurement in cQA problem. We evaluate our proposed model on the SemEval 2016 data set and achieve the state-of-the-art result. Keywords Community question answering, Multi knowledge sources, Deep learning, The BERT model References [1] C. Alberto, D. Bonadiman, G. D. S. Martino, Answer and Question Selection for Question Answering on Arabic and English Fora, in Proceedings of SemEval-2016, 2016, pp. 896-903. [2] Filice, D. Croce, A. Moschitti, R. Basili, Learning Semantic Relations between Questions and Answers, in Proceedings of SemEval-2016, 2016, pp. 1116-1123. [3] Wang, Z. Ming, T. S. Chua, A Syntactic Tree Matching Approach to Finding Similar Questions in Community-based qa Services, in SIGIR, 2009, pp. 187-194. [4] Pengfei, Q. Xipeng, C. Jifan, H. Xuanjing, Deep Fusion lstms for Text Semantic Matching, in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Vol. 1, 2016, pp. 1034-1043, https://doi.org/ 10.18653/v1/P16-1098. [5] Jonas, T. Aditya, Siamese Recurrent Architectures for Learning Sentence Similarity, in Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16), 2016, pp. 2786-2792. [6] Jacob, C. M. Wei, L. Kenton, T. Kristina, Bert: Pre-training of Deep Bidirectional Transformers for Language Understanding, in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019, pp. 4171-4186. [7] Wissam, B. Fady, H. Hazem, Arabert: Transformer-based Model for Arabic Language Understanding, in Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on O ensive Language Detection, 2020, pp. 9-15. [8] Lukovnikov, A. Fischer, J. Lehmann, Pretrained Transformers for Simple Question Answering Over Knowledge Graphs, ArXiv, abs/2001.11985, 2019. [9] V. Aken, B. Winter, A. Loser, F. Gers, How Does BERT Answer Questions?: A Layer-Wise Analysis of Transformer Representations, in Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019. [10] Chan, Y. Fan, A Recurrent BERT-based Model for Question Generation, in Proceedings of the Second Workshop on Machine Reading for Question Answering, 2019, pp. 154-162. [11] Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, R. Soricut, ALBERT: A Lite BERT for Self-supervised Learning of Language Representations, ArXiv, abs/1909.11942, 2020. [12] Ngai, Y. Park, J. Chen, M. Parsapoor, Transformer-Based Models for Question Answering on COVID19, ArXiv, abs/2101.11432, 2021. [13] Yu, L. Wu, Y. Deng, R. Mahindru, Q. Zeng, S. Guven, M. Jiang, A Technical Question Answering System with Transfer Learning, in Proceedings of the 2020 EMNLP (Systems Demonstrations), 2020, pp. 92-99. [14] S. McCarley, R. Chakravarti, A. Sil, Structured Pruning of a BERT-based Question Answering Model, arXiv: Computation and Language, 2019. [15] Almiman, N. Osman, M. Torki, Deep Neural Network Approach for Arabic Community Question Answering, Alexandria Engineering Journal, Vol. 59, 2020, pp. 4427-4434. [16] Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, I. Polosukhin, Attention is all you Need, in Advances in Neural Information Processing Systems, 2017, pp. 5998-6008. [17] T. Nguyen, A. C. Le, H. N. Nguyen, A Model of Convolutional Neural Network Combined with External Knowledge to Measure the Question Similarity for Community Question Answering Systems, International Journal of Machine Learning and Computing, Vol. 11, No. 3, 2021, pp. 194-201, https://doi.org/ 10.18178/ijmlc.2021.11.3.1035.