Osprey: peer-to-peer enabled content distribution

Abstract

As the size of data and files increases, digital repositories face a growing problem in content distribution. High quality multimedia and research data sets can range from 100's of megabytes to over a terabyte. Web-based digital repositories may experience a substantial increase in operating and bandwidth costs when providing materials to the public. Peer-to-peer networks are sometimes suggested as an alternative to traditional centralized repositories. However, critical issues such as data integrity, access control, and content availability exist when using peer-to-peer technologies. Osprey (http://osprey.ibiblio.org) addresses these problems by combining a flexible metadata management system with the BitTorrent peer-to-peer protocol. A Web database application provides searching and browsing of collection objects, and the peer-to-peer component lowers the bandwidth costs by employing distributed downloading. The Permaseed application supplies reliable, persistent peer-to-peer access to files in the digital repository