SciConNav: Knowledge navigation through contextual learning of extensive scientific research trajectories

Abstract

New knowledge builds upon existing foundations, which means an interdependent relationship exists between knowledge, manifested in the historical records of the scientific system for hundreds of years. By leveraging natural language processing techniques, this study introduces the Scientific Concept Navigator, an embedding-based navigation model to infer the “knowledge pathway” from the research trajectories of millions of scholars. We validate that the learned representations effectively delineate disciplinary boundaries and capture the intricate relationships between diverse concepts. Utility of the navigation space is showcased through multiple applications. Firstly, we demonstrate the multi-step analogy inferences between concepts from various disciplines. Secondly, we formulate the cross-domain conceptual dimensions of knowledge, observing the distributional shifts of 19 disciplines along these conceptual dimensions, including “Theoretical” to “Applied,” and “Societal” to “Economic,” highlighting the evolution of functional attributes across diverse domains. Lastly, by analyzing the knowledge network structure, we find that knowledge connects with shorter global pathways, and interdisciplinary concepts play a critical role in enhancing accessibility. Our framework offers a novel approach to mining knowledge inheritance pathways from extensive scientific literature, which is of great significance for understanding scientific progression patterns, tailoring scientific learning trajectories, and accelerating scientific progress.