GlintLM: Graph-Layered Integration with Nodal Topology with Language Models — A Bipartite Approach to Question Answering

Abstract

In modern Question Answering (QA) systems, Language Models (LMs) are often combined with Knowledge Graphs (KGs) to better handle challenges like word ambiguity and complex sentence structures. This combination helps LMs gain a deeper understanding by grounding them in structured knowledge. However, existing approaches often fall short in two areas: (1) they do not fully use the features of Knowledge Graphs and Graph Neural Networks (GNNs) during reasoning, and (2) they miss opportunities to better rank and filter information using the outputs of LMs and GNNs. To address this, we propose GlintLM, a system with two key innovations. First, the Enhanced Topological Node Representation (ETNR) module, which uses graph structure and a custom node feature method to improve reasoning. Second, the Multiplex Contextual Scorer (MCS) module, which combines pre-trained LM outputs with GNN attention to better score and filter relevant nodes. Together, these components create a more effective and adaptable system for QA. GlintLM demonstrates improved performance on common-sense (CommonsenseQA, OpenBookQA) and biomedical (MedQA-USMLE) QA benchmarks, showing improved performance across commonsense and medical domains.²