Interactive data analytics: the new frontier

Abstract

Data analytics often involves data exploration, where a data set is repeatedly analyzed to understand root causes, find patterns, or extract insights. Such analysis is frequently bottlenecked by the underlying data processing system, as analysts wait for their queries to complete against a complex multilayered software stack. In this talk, I'll describe some exploratory analytics applications we've build in the MIT database group over the past few years, and will then describe some of the challenges and opportunities that arise when building more efficient data exploration systems that will allow these applications to become truly interactive, even when processing billions of data points.