A Service Management Method for Distributed Deep Learning

Abstract

With the advance of deep learning technologies, many applications and/or services that rely on them can be easily found these days. Applications relying on deep learning varies from video, audio, text and time-series data, and they provide high-accuracy services that are built with the software platforms such as TensorFlow or PyTorch. Usually, a deep learning service requires rich resources such as GPU and large memory. For instance, GPT-3 requires memories up to a few hundred gigabytes, and for the video processing it needs accelerators such as GPU. The cost will be increased if all the resources are on the cloud, and there are many works on offloading these workloads of deep learning onto distributed infrastructure. One of the focuses of these works are distribution of deep learning workloads onto various resource and providing an end-to-end service by the combination of them. Edge computing or Fog computing is one of the architectures providing workload distribution method from cloud to edge resources. This paper proposes a method that enables autonomous configuration between distributed services. In the proposed method, the composition of distributed services is described in systematic way so to configure the connections between them more intuitively. Further, the proposed method includes binding of a resource to a service which enables management of multiple service distributions, and how it can work with existing standards.