Building an extensible file system via policy-based data management

Abstract

In the practice of data management, different user communities often use different forms of data and metadata, support different operations, and implement different policies. For example, forms of data include blocks, tuples, streams, time series, and so forth, while forms of metadata include different vocabularies, schemas, and namespaces. In addition, the forms of data and metadata often change over time. The diverse and emergent nature of these requirements pose to data management systems a challenge which traditional file systems with fixed functionality have become inadequate to address. Extensible file systems can be built via policy-based data management. We describe the practical and theoretical aspects of policy-based data management and provide a wide range of examples of prototypical and production applications of the integrated Rule-Oriented Data System (iRODS) in data grids that apply this type of file system.