A New Framework for Textual Information Mining over Parse Trees

Abstract

This paper introduces a new text mining framework using a tree-based Linguistic Query Language, called LQL. The framework generates more than one parse tree for each sentence using a probabilistic parser, and annotates each node of these parse trees with \textit{main-parts} information which is set of key terms from the node's branch based on the branch's linguistic structure. Using main-parts-annotated parse trees, the system can efficiently answer individual queries as well as mine the text for a given set of queries. The framework can also support grammatical ambiguity through probabilistic rules and linguistic exceptions.