MTrain: Enable Efficient CNN Training on Heterogeneous FPGA-Based Edge Servers

Abstract

FPGA-based edge servers are used in many applications in smart cities, hospitals, retail, etc. Equipped with heterogeneous FPGA-based accelerator cards, the servers can be implemented with multiple tasks, including efficient video prepossessing, machine learning algorithm acceleration, etc. These servers are required to implement inference during the daytime while retraining the model during the night to adapt to new environments, domains, or new users. During the retraining, conventionally, the incoming data are transmitted to the cloud, and then the updated machine learning models will be transferred back to the edge server. Such a process is inefficient and cannot protect users’ privacy, so it is desirable for the models to be directly trained on the edge servers. Deploying convolutional neural network (CNN) training on heterogeneous resource-constrained FPGAs is challenging since it needs to consider both the complex data dependency of the training process and the communication bottleneck among different FPGAs. Previous multiaccelerator training algorithms select optimal scheduling strategies for data parallelism (DP), tensor parallelism (TP), and pipeline parallelism (PP). However, PP cannot deal with batch normalization (BN) which is an essential CNN operator, while purely applying DP and TP suffers from resource under-utilization and intensive communication costs. In this work, we propose MTrain, a novel multiaccelerator training scheduling strategy that transfers the training process into a multibranch workflow, thus independent suboperations of different branches are executed on different training accelerators in parallelism for better utilization and reduced communication overhead. Experimental results show that we can achieve efficient CNN training on heterogeneous FPGA-based edge servers with $1.07\times $ – $2.21\times $ speedup under 15-GB/s peer-to-peer bandwidth compared to the state-of-the-art work.