Vision: information-centric dataflow: re-imagining reactive distributed computing

Abstract

This paper describes an Information-Centric Dataflow system that is based on name-based access to computation results, NDN PSync dataset synchronization for enabling consuming compute functions to learn about updates and for coordinating the set of compute functions in a distributed Dataflow pipeline. We describe how relevant Dataflow concepts can be mapped to ICN and how data-sharing, data availability and scalability can be improved compared to state-of-the-art systems. We also provide a specification of an application-independent namespace design and report on our experience with a first prototype implementation.