A new solution to the hyperbolic tangent implementation in hardware: polynomial modeling of the fractional exponential part

The most difficult part of an artificial neural network to implement in hardware is the nonlinear activation function. For most implementations, the function used is the hyperbolic tangent. This function has received much attention in relation to hardware implementation. Nevertheless, there is no co...

Full description

Saved in:
Bibliographic Details
Published inNeural computing & applications Vol. 23; no. 2; pp. 363 - 369
Main Authors Nascimento, Ivo, Jardim, Ricardo, Morgado-Dias, Fernando
Format Journal Article
LanguageEnglish
Published London Springer London 01.08.2013
Springer
Subjects
Applied sciences
Artificial Intelligence
Computational Biology/Bioinformatics
Computational Science and Engineering
Computer Science
Computer science; control theory; systems
Data Mining and Knowledge Discovery
Exact sciences and technology
Image Processing and Computer Vision
Inference from stochastic processes; time series analysis
Learning and adaptive systems
Mathematics
Original Article
Probability and statistics
Probability and Statistics in Computer Science
Sciences and techniques of general use
Statistics
Hardware description languages
Gate arrays
CORDIC
Algorithms implemented in hardware
Neural networks
Polynomial
Neural computation
Description language
Error estimation
Memory
Attention
Non linear function
Neural network
Algorithm
Modeling
Implementation
Array
Activation function
Online AccessGet full text
ISSN0941-0643
1433-3058
DOI10.1007/s00521-012-0919-0

Cover

More Information
Summary:The most difficult part of an artificial neural network to implement in hardware is the nonlinear activation function. For most implementations, the function used is the hyperbolic tangent. This function has received much attention in relation to hardware implementation. Nevertheless, there is no consensus regarding the best solution. In this paper, we propose a new approach by implementing the hyperbolic tangent in hardware with a polynomial modeling of the fractional exponential part. The results in the paper then demonstrate, through the use of an example, that this solution is faster than the CORDIC algorithm, but slower than the piecewise linear solution with the same error. The advantage over the piecewise linear approach is that it uses less memory.
ISSN:0941-0643
1433-3058
DOI:10.1007/s00521-012-0919-0