Big data analytics on flash storage with accelerators

Abstract

Complex analytics of the vast amount of data collected via social media, cell phones, ubiquitous smart sensors, and satellites is likely to be the biggest economic driver for the IT industry over the next decade. For many “Big Data” applications, the limiting factor in performance is often the transportation of large amount of data from hard disks to where it can be processed, i.e. DRAM. We will present BlueDBM, an architecture for a scalable distributed flash store which overcomes this limitation by providing a high-performance, high-capacity, scalable random-access flash storage, and by allowing computation near the data via a FPGA-based programmable flash controller. We will present the preliminary results for two applications, (1) key-value store (KVS) and (2) sparse-matrix accelerator for graph processing, on BlueDBM consisting of 20 nodes and 20TB of flash.