eWB: Event-Based Weight Binarization Algorithm for Spiking Neural Networks

• Published in: IEEE access Vol. 9; pp. 38097 - 38106
• Main Authors: Kim, Dohun, Kim, Guhyun, Hwang, Cheol Seong, Jeong, Doo Seok
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Algorithms; Approximation algorithms; Back propagation; Back propagation networks; Classification algorithms; Event-based weight binarization; event-driven learning algorithm; Lagrange multiplier; Lagrange multiplier method; Linear programming; Machine learning; Neural networks; Neurons; Optimization; Spiking; spiking neural networks; Synapses; System-on-chip; Training; Weight
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2021.3062405

Learning binary weights to minimize the difference between target and actual outputs can be considered as a parameter optimization task within the given constraints, and thus, it belongs to the application domain of the Lagrange multiplier method (LMM). Based on the LMM, we propose a novel event-based weight binarization (eWB) algorithm for spiking neural networks (SNNs) with binary synaptic weights (−1, 1). The algorithm features (i) event-based asymptotic weight binarization using local data only, (ii) full compatibility with event-based end-to-end learning algorithms (e.g., event-driven random backpropagation (eRBP) algorithm), and (iii) the capability to address various constraints (including the binary weight constraint). As a proof of concept, we combine eWB with eRBP (eWB-eRBP) to obtain a single algorithm for learning binary weights to generate correct classifications. Fully connected SNNs were trained using eWB-eRBP and achieved an accuracy of 95.35% on MNIST. To the best of our knowledge, this is the first report on completely binary SNNs trained using an event-based learning algorithm. Given that eRBP with full-precision (32-bit) weights exhibited 97.20% accuracy, the binarization comes at the cost of an accuracy reduction of approximately 1.85%. The python code is available online: https://github.com/galactico7/eWB .