Hierarchical Training for Distributed Deep Learning Based on Multimedia Data over Band-Limited Networks

Abstract

Distributed deep learning (DL) plays a critical role in many wireless Internet of Things (IoT) applications including remote camera deployment. This work addresses three practical challenges in cyber-deployment of distributed DL over band-limited channels. Specifically, many IoT systems consist of sensor nodes for raw data collection and encoding, and servers for learning and inference tasks. Adaptation of DL over band-limited network data links has only been scantly addressed. A second challenge is the need for pre-deployed encoders being compatible with flexible decoders that can be upgraded or retrained. The third challenge is the robustness against erroneous training labels. Addressing these three challenges, we develop a hierarchical learning strategy to improve image classification accuracy over band-limited links between sensor nodes and servers. Experimental results show that our hierarchically-trained models can improve link spectrum efficiency without performance loss, reduce storage and computational complexity, and achieve robustness against training label corruption.