PerfNet

The technology of deep learning has grown rapidly and been widely used in the industry. In addition to the accuracy of the deep learning (DL) models, system developers are also interested in comprehending their performance aspects to make sure that the hardware design and the systems deployed to meet the application demands. However, developing a performance model to serve the aforementioned purpose needs to take many issues into account, e.g. the DL model, the runtime software, and the system architecture, which is quite complex. In this work, we propose a multi-layer regression network, called PerfNet, to predict the performance of DL models on heterogeneous systems. To train the PerfNet, we develop a tool to collect the performance features and characteristics of DL models on a set of heterogeneous systems, including key hyper-parameters such as loss functions, network shapes, and dataset size, as well as the hardware specifications. Our experiments show that the results of our approach are more accurate than previously published methods. In the case of VGG16 on GTX1080Ti, PerfNet yields a mean absolute percentage error of 20%, while the referenced work constantly overestimates with errors larger than 200%.