TaskRouter: A newly designed online data processing framework

Abstract

TaskRouter is a newly designed framework for distributed computing. It can be used to develop online processing system for High Energy Physics experiments. The framework takes the responsibility of data transmission. Users can determine how data being processed and routed on each node by implementing a single callback interface. One or more backup slaves can be configured for critical nodes in TaskRouter system. The processing procedure can recover from any single-point failure without data loss. TaskRouter is flexible, easy to use and with high availability. This paper presents the core design and implementation of TaskRouter. For function and performance test, a dummy online processing software based on TaskRouter is developed.