Text Mining

Abstract

Part of the abstract comes from a submitted paper sent 2022, in the Electronic theses and dissertations (ETDs) contain valuable knowledge that can be useful in a wide range of research areas. Accordingly, we are building electronic infrastructure leveraging advanced work on digital libraries, for discovering and accessing the knowledge buried in ETDs. We focus on our work to incorporate topic modeling into digital libraries for ETDs. We present ETD-Topics, a framework that extracts topics from a large text corpus in an unsupervised way. The representations learnt from topic models can be useful for downstream tasks such as searching and/or browsing documents by topic, document recommendation, topic recommendation, and describing temporal topic trends (e.g., from the perspective of disciplines or universities). We provide four modes (LDA, NeuralLDA, ProdLDA, and CTM) to serve user groups with different browsing and searching requirements. Our job was to import the preprocessed database and to apply the four trained models, and then to accurately display key information (such as topics, document title, abstract, etc.) on web pages. We chose Python as the main language to implement the back-end, while using Flask as a bridge connecting the back-end and front-end. While using HTML for displaying data, we were able to employ JavaScript and CSS to make web pages that include graphic bars, buttons (like “Submit”, “Show more”, etc.), and tables.