Applicability of Deep Learning Model Trainings on Embedded GPU Devices: An Empirical Study

Abstract

The wide applications of deep learning techniques have motivated the inclusion of both embedded GPU devices and workstation GPU cards into contemporary Industrial Internet-of-Things (IIoT) systems. Due to substantial differences between the two types of GPUs, deep-learning model training in its current practice is run on GPU cards, and embedded GPU devices are used for inferences or partial model training at best. To supply with empirical evidence and aid the decision of deep learning workload placement, this paper reports a set of experiments on the timeliness and energy efficiency of each GPU type, running both Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) model training. The results suggest that embedded GPUs did save the total energy cost despite the longer response time, but the amount of energy saving might not be significant in a practical sense. Further in this paper we report a case study for prognostics applications using LSTM. The results suggest that, by comparison, an embedded GPU may save about 90 percent of energy consumption at the cost of doubling the application response time. But neither the save in energy cost nor the increase in response time is significant enough to impact the application. These findings suggest that it may be feasible to place model training workload on either workstation GPU or embedded GPU.