Developing a goal-driven data integration framework for effective data analytics

Abstract

Data integration plays a crucial role in business intelligence, aiding decision-makers by consolidating data from heterogeneous sources to provide deep insights into business operations and performance. In the big data era, automated data integration solutions need to process high volumes of disparate data robustly and seamlessly for various analytical needs or operational actions. Existing data integration solutions exhibit limited capabilities for capturing and modeling users' needs to execute on-demand data integration. This study, underpinned by affordance theory and the goal definition principles from the Goal-Question-Metric approach, designs and instantiates a goal-driven data integration framework for data analytics. The proposed innovative design automates data integration for non-technical data users. Specifically, it demonstrates how to elicit and ontologize users' data-analytic goals and addresses semantic heterogeneity, thereby recognizing goal-relevant datasets. In a structured evaluation using the context of counter-terrorism analytics, our design artifact shows promising performance in capturing diverse and dynamic user goals for data analytics and in generating integrated data tailored to these goals. Our research establishes a theoretical framework to guide future scholars and practitioners in building smart, goal-driven data integration.