Scalable Huge Directories through OSD+ Devices

Abstract

Management of directories with millions of files, accessed by thousands of clients at the same time, is a problem recently identified in HPC environments. This paper introduces an OSD+-based technique to deal with those directories. We use directory objects in OSD+ devices for dynamically distributing a huge directory among several servers. Directory objects work independently, achieving good performance and scalability. Experiments show that, by using just 8 OSD+s and Ext4, FPFS is able to create, stat and delete more than 70,000, 120,000 and 37,000 files per second, respectively. With ReiserFS, these numbers are 118,000, 97,000 and 67,000. Experiments, however, have produced unforeseen results too. While distribution is beneficial when a huge directory is accessed by many clients, it can also downgrade the performance when several huge directories are concurrently accessed by a few clients.