Eliciting big data requirement from big data itself: A task-directed approach

Abstract

The characteristics of big data not only challenge the processing methods of large volume of data, but also the way we make use of such semantic-rich resources, among which how users plan to manipulate the intermediate or final results requires to be well considered. This is especially challenging in building analytics systems as big data is richer in semantics and the heterogeneous data modalities also impose burdens on semantic fusion and visualization. While most of current research focuses on how to mine semantic information from big data, we emphasize the importance of the role of users in terms of requirement acquisition while building practical system functions in data analytics or management. This paper proposes a task-directed approach of requirement analysis for big data analytics which enhances requirement elicitation with modern data mining approaches. Based on the entities and semantic relationships learned from big data, a user-in-loop semi-automated requirement elicitation is carried out to generate a requirement repository which is affordable for further maintenance and modification. State-of-the-art user modeling methods can be incorporated aiming at refining the requirement iteratively according to the task interaction of users with system functions. Taking the case study of lifelogging analytics, we further discuss the advantages and limitations of our perspective.