Pipeline Manager: A Flexible Semi-automatic Dataflow Analysis Framework

Abstract

Industrial big data analysis has received a bunch of attentions in recent decades. There are several famous machine learning or deep learning frameworks used in different scenarios. However, we lack a stable and easy-to-operate pipeline framework. In this paper, the purpose is to propose an algorithm pipeline integration framework to help industrial AI systems deal with loads, scheduling and automatic operations.