Dynamic Workflow Planning on Programmable Infrastructure

Abstract

The Network Service Interface (NSI) has been created as a result of collaborative development of network and application engineers primarily associated with the Research and Education (R&E) community. The NSI allows workflow systems not only to check available service points for a workflow engine to schedule executions, but also to reserve and provide network connections among those service points. The Open Flow technology provides programmability on the network Flow and allows software to define dynamically behaviour of the network. These new features offer data intensive applications new opportunities to optimize the mapping between data Flow patterns and the infrastructure yielding better system level quality. However, they also require the computing support systems effectively capture not only the characteristics of the application workflow but also the controllability of the underlying network. In this paper we discussed the extension of our previous system called Network QoS Planner (NEWQoSPlanner) and investigated how reservation based connection services can be enhanced by dynamic network Flow control. We also discusse how NEWQoSPlanner invokes network services to achieve connection reservation and provisioning, and includes Open Flow to realize dynamic Flow optimization for data intensive workflows.