Cloudets: Cloud-based cognition for large streaming data

Abstract

Big data cognition has become a dominant problem in interactive visual analytics for event detection and response, metereology, cosmology, and large smart city applications including traffic monitoring and management, search and rescue operations, crowd management and logistics. The main problems are mainly due to big data volume and velocity and, in some cases, variety in both dimension and type. A practical approach to understanding and viewing big data features is through streaming operations. Streaming allows for both volume and velocity characteristics of big data, and often, for variety as well. However, performing analytics at interactive rates is currently an open challenge in most big data applications. Cloud computing platforms provide practical support and leverage to solving some of the big data and visual analytics problems, especially when dealing with the volume and velocity characteristics of current data generation. In order to interact with streaming data patterns in an elastic cloud environment, we present a new elastic framework for big data visual analytics in the cloud, the Cloudet. The Cloudet is a self-adaptive cloud-based platform that treats both data and compute nodes as elastic objects. The main objective is to readily achieve the scalability and elasticity of cloud computing platforms in order to process large streaming data and adapt to potential interactions between data stream features. Our main contributions include a robust cloud-based framework, the Cloudet, which can flexibly process the streaming data and applications to illustrate the setup and operations of this framework. The framework includes a cloud profile manager that attempts to optimize the cloudet parameters in order to achieve expressivity, scalability, reliability, and the proper aggregation of the data streams into several density maps for the purpose of dynamic visualization of data features.