NIST Big Data Reference Architecture for Analytics and Beyond

Big Data is the term used to describe the deluge of data in our networked, digitized, sensor-laden, information driven world. There is a broad agreement among commercial, academic, and government leaders about the remarkable potential of "Big Data" to spark innovation, fuel commerce, and drive progress. The availability of vast data resources carries the potential to answer questions previously out of reach. However, there is also broad agreement on the ability of Big Data to overwhelm traditional approaches. Big Data architectures come in many shapes and forms ranging from academic research settings to product-oriented workflows. With massive-scale dynamic data being generate from social media, Internet of Things, Smart Cities, and others, it is critical to analyze these data in real-time and provide proactive decision. With the advancement of computer architecture in multi-cores and GPUs, and fast communication between CPUs and GPUs, parallel processing utilizes these platforms could optimize resources at a reduced time. This presentation will provide the past, current, and future activities of the NIST Big Data Public Working Group (NBD-PWG) and how the NIST Reference Architecture may address the rate at which data volumes, speeds, and complexity are growing requires new forms of computing infrastructure to enable Big Data analytics interoperability such that analytics tools can be re-usable, deployable, and operational. The focus of NBD-PWG is to form a community of interest from industry, academia, and government, with the goal of developing consensus definitions, taxonomies, secure reference architectures, and standards roadmap which would create vendor-neutral, technology and infrastructure agnostic framework. The aim is to enable Big Data stakeholders to pick-and-choose best analytics tools for their processing under the most suitable computing platforms and clusters while allowing value-additions from Big Data service providers and flow of data between the stakeholders in a cohesive and secure manner.