Survey of scaling platforms for Deep Neural Networks

Deep Neural Networks have become a state of the art approach in perception processing like speech recognition, image processing and natural language processing. Many state of the art benchmarks for these algorithms are using deep learning techniques. The deep neural networks in today's applications need to process very large amount of data. Different approaches have been proposed to solve scaling these algorithms. Few approach look for providing a solution over existing big data processing platform which usually runs over a large scale commodity cpu cluster. As training deep learning workload require many small computations to be done and large communication to pass the data between layers, General Purpose GPUs seems to the best platforms to train these networks. Different approaches have been proposed to scale processing on cluster of GPU servers. We have summarized various approaches used in this regard.