Cluster-based implementation of resource brokering strategy for parallel training of neural networks

Abstract

The implementation issues of a cluster-based resource brokering strategy intended for efficient parallelization of neural networks training are presented in this paper. We describe a strategy of resource brokering based on the prediction of execution time and parallelization efficiency of algorithms using a BSP computation model and Pareto optimality with a weight coefficients approach for choosing optimal solution. Our results show a reasonable adaptation of the resource brokering strategy to the environment of a real computational cluster providing the minimal total time to delivery of the parallel application.