Minimizing Response Time for MapReduce Applications on In-Storage Processing Architecture

Abstract

Integrating In-Storage Processing architecture with the Hadoop model offers a potential solution for accelerating applications with large-scale data requirements. However, the data movement overhead among nodes and the task interference delay among multiple applications are getting serious with the increased data size of applications. To minimize the response time of applications while maintaining the throughput, we propose workload-aware scheduling and waiting-time-aware dispatcher to manage multiple MapReduce applications under the In-Storage Processing architecture.