A Dynamic Message-Aware Communication Scheduler for Ceph Storage System

With the proliferation of cloud computing technologies, the Ceph, a distributed object-based storage system has been an attractive alternative to building a storage backend due to its excellent performance, reliability, and scalability. As the storage system processes huge amount data and the network traffic generated from the cloud increases rapidly, designing a high-performance messenger in the storage system has created a lot of challenging issues. Although the async messenger, one of the Ceph's messengers, is known to be efficient and flexible, it contains several performance problems due to its simple round-robin based scheduling scheme that assigns a connection to a worker thread without any consideration for the amount of workloads transferred through the connections. This causes the imbalance of worker threads and adversely affects the performance of the Ceph storage system. This paper proposes a dynamic message-aware communication scheduler for Ceph storage system that balances the workloads of worker threads based on the types of incoming messages, while avoiding unnecessary connection movements among worker threads. We use genetic algorithm (GA) to solve this problem and implement the proposed scheduling algorithm using the async messenger. The benchmarking results show that the proposed approach outperforms the original async messenger by as much as 12.5% under the same workload from clients and 24% under the random workloads from clients.