Distributed computation on dynamo-style distributed storage: riak pipe

Abstract

The Dynamo model, as described by Amazon in 2007, has become a popular concept in the development of distributed storage systems. The model accounts for only CRUD operations, however. This paper describes a system called Riak Pipe that enables the use of generic functions in place of CRUD operations. This allows Dynamo-model users to exploit other resources, such as CPU time, available in their cluster, as well as to gain the efficiencies offered by data-local processing.