Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices

• Published in: Proceedings of the International Conference on Distributed Computing Systems pp. 328 - 339
• Main Authors: Teerapittayanon, Surat, McDanel, Bradley, Kung, H. T.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2017
• Subjects: Artificial neural networks; Cloud computing; Computational modeling; ddnn; deep neural networks; distributed computing hierarchies; distributed deep neural networks; dnn; edge computing; embedded dnn; Performance evaluation; sensor fusion; Training
• ISSN: 1063-6927
• DOI: 10.1109/ICDCS.2017.226

We propose distributed deep neural networks (DDNNs) over distributed computing hierarchies, consisting of the cloud, the edge (fog) and end devices. While being able to accommodate inference of a deep neural network (DNN) in the cloud, a DDNN also allows fast and localized inference using shallow portions of the neural network at the edge and end devices. When supported by a scalable distributed computing hierarchy, a DDNN can scale up in neural network size and scale out in geographical span. Due to its distributed nature, DDNNs enhance sensor fusion, system fault tolerance and data privacy for DNN applications. In implementing a DDNN, we map sections of a DNN onto a distributed computing hierarchy. By jointly training these sections, we minimize communication and resource usage for devices and maximize usefulness of extracted features which are utilized in the cloud. The resulting system has built-in support for automatic sensor fusion and fault tolerance. As a proof of concept, we show a DDNN can exploit geographical diversity of sensors to improve object recognition accuracy and reduce communication cost. In our experiment, compared with the traditional method of offloading raw sensor data to be processed in the cloud, DDNN locally processes most sensor data on end devices while achieving high accuracy and is able to reduce the communication cost by a factor of over 20x.