7.7 LNPU: A 25.3TFLOPS/W Sparse Deep-Neural-Network Learning Processor with Fine-Grained Mixed Precision of FP8-FP16

• Published in: Digest of technical papers - IEEE International Solid-State Circuits Conference pp. 142 - 144
• Main Authors: Lee, Jinsu, Lee, Juhyoung, Han, Donghyeon, Lee, Jinmook, Park, Gwangtae, Yoo, Hoi-Jun
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.02.2019
• Subjects: Acceleration; Deep learning; Neural networks; Registers; Throughput; Training
• ISSN: 2376-8606
• DOI: 10.1109/ISSCC.2019.8662302

Recently, deep neural network (DNN) hardware accelerators have been reported for energy-efficient deep learning (DL) acceleration [1-6]. Most prior DNN inference accelerators are trained in the cloud using public datasets; parameters are then downloaded to implement AI [1-5]. However, local DNN learning with domain-specific and private data is required meet various user preferences on edge or mobile devices. Since edge and mobile devices contain only limited computation capability with battery power, an energy-efficient DNN learning processor is necessary. Only [6] supported on-chip DNN learning, but it was not energy-efficient, as it did not utilize sparsity which represents 37%-61% of the inputs for various CNNs, such as VGG16, AlexNet and ResNet-18, as shown in Fig. 7.7.1. Although [3-5] utilized the sparsity, they only considered the inference phase with inter-channel accumulation in Fig. 7.7.1, and did not support intra-channel accumulation for the weight-gradient generation (WG) step of the learning phase. Also, [6] adopted FP16, but it was not energy optimal because FP8 is enough for many input operands with 4× less energy than FP16.