Database managed external file update

Abstract

Relational DBMSs (RDBMSs) have evolved to an extent that they are used to manage almost all traditional business data in a robust fashion. Nevertheless, a large fraction of unstructured and semi-structured data continues to be managed by file systems. As companies increasingly depend on non-traditional data for their daily business operations, it becomes more and more important to provide higher degree of integrity, security and reliability to the data stored in file systems. DataLinks technology, developed at IBM Almaden Research Center, achieves this by providing a vital integration between a RDBMS and a file system. It enables the DBMS to manage files residing in file systems as though they are logically within the database. Current DataLinks technology supports only read access to external files that are being managed by the DBMS. This severely restricts the applicability of DataLinks technology in transaction-oriented and/or e-business applications. Traditional database systems enforce ACID properties for database updates. Extending these properties to cover both external files stored outside of a DBMS and metadata stored in the DBMS is a hard problem. This is because files are updated through a standard file-system API while metadata, which references the files, is updated through a database API. This paper describes our experiences in the design and prototyping of an advanced DataLinks technology that supports database-managed external file updates. This enhanced capability makes DataLinks technology an even more attractive solution for managing the world's data.