EDGESTORE: A Single Namespace and Resource-Aware Federation File System for Edge Servers

Abstract

With the increasing adoption of edge computing, the capacity requirements of the edge servers are also growing. Especially the data volume generated from a large number of edge clients and/or edge devices demand more capacity to be able to store them for processing. The growing gap between the data volume and current storage capacity is motivating the need towards building aggregated storage spaces. Aggregated storage can be an effective way to extend edge servers' overall storage capacity by combining storage resources of other nodes under the agreement to share. Several Federation file systems exist to meet this aggregate storage needs but are not without limitations. Dependency to the specific software stack makes it unfit for general-purpose use and they often neglect important features critical for the performance. In this paper, we address the important challenges of building the Federation on top of edge servers with the heterogeneous file system and resource configurations. We prototyped EDGESTORE, a Federation File System for Edge Servers. EDGESTORE equips the users with an aggregate storage namespace and federates resources of edge servers, to enable high resource-sharing in Federation. We propose, Job and Resource-Aware Request Placement algorithm (JRAP) to take advantage of edge server resource heterogeneity. To evaluate the usefulness of EDGESTORE, we consider two federation scenarios i) with same resource configurations and ii) with different resource configurations. We evaluate the efficacy of various big data applications from data storage to analysis using EDGESTORE on a real testbed.