A Novel Fractional Gradient-Based Learning Algorithm for Recurrent Neural Networks

• Published in: Circuits, systems, and signal processing Vol. 37; no. 2; pp. 593 - 612
• Main Authors: Khan, Shujaat, Ahmad, Jawwad, Naseem, Imran, Moinuddin, Muhammad
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.02.2018; Springer Nature B.V
• Subjects: Algorithms; Back propagation; Back propagation networks; Chaos theory; Circuits and Systems; Electrical Engineering; Electronics and Microelectronics; Engineering; Fractional calculus; Instrumentation; Machine learning; Neural networks; Nonlinear systems; Recurrent neural networks; Signal,Image and Speech Processing; System identification; Gradient descent; Minimum redundancy and maximum relevance (mRMR); Back-propagation through time (BPTT); Mackey–Glass chaotic time series; Fractional calculus; Recurrent neural network (RNN)
• ISSN: 0278-081X; 1531-5878
• DOI: 10.1007/s00034-017-0572-z

In this research, we propose a novel algorithm for learning of the recurrent neural networks called as the fractional back-propagation through time (FBPTT). Considering the potential of the fractional calculus, we propose to use the fractional calculus-based gradient descent method to derive the FBPTT algorithm. The proposed FBPTT method is shown to outperform the conventional back-propagation through time algorithm on three major problems of estimation namely nonlinear system identification, pattern classification and Mackey–Glass chaotic time series prediction.