FIF: A NLP-based Feature Identification Framework for Data Warehouses

Abstract

In a data warehouse, selecting the relevant features is an iterative process that is laborious, time-consuming, and error-prone due to selection bias introduced by either the data expert or the data-analyst. In order to address this challenge, this paper introduces FIF, a Feature Identification Framework that uses Natural Language Processing (NLP) to analyze the hypotheses, identify the relevant feature space and predict the appropriate data mining task and model. The FIF is designed on the principles of microservices architecture pattern, comprising of five core groups of microservices: (a) NLP Pre-processor, (b) Attribute Identifier, (c) Feature Identifier, (d) Topic Modeller, and (e) Data Mining Task Evaluator. Finally, FIF is evaluated with five hypotheses against our data warehouse. CCS CONCEPTS • Information systems → Data warehouses; Wrappers (data mining); Document topic models; Similarity measures; • Computing methodologies → Feature selection; Natural language processing.