Electronic neural net algorithm for maximum entropy solutions to ill-posed problems. II. Multiply connected electronic circuit implementation

• Published in: IEEE transactions on circuits and systems Vol. 37; no. 1; pp. 110 - 113
• Main Authors: Marrian, C.R.K., Mack, I.A., Banks, C., Peckerar, M.C.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.01.1990; Institute of Electrical and Electronics Engineers
• Subjects: Analog circuits; Applied sciences; Circuit properties; Circuit stability; Cost function; Electric, optical and optoelectronic circuits; Electronic circuits; Electronic components; Electronics; Entropy; Exact sciences and technology; Integrated circuit interconnections; Minimization; Miscellaneous; Neural networks; Operational amplifiers; Voltage; Method of maximum entropy; Neural network; Implementation; Lyapunov function; Information theory
• ISSN: 0098-4094
• DOI: 10.1109/31.45695

For pt.I see ibid., vol.36, no.2, p.288-94 (1989). Using standard electronic components, the authors have constructed a multiply connected analog circuit that minimizes a specific cost function of its external inputs, interconnects, and node characteristics. The circuit is designed so that the cost function contains an informational entropy regularizer to give maximum-entropy solutions to ill-posed problems. The circuit performs a gradient search to achieve the minimization in a time defined by a circuit RC time constant rather than the complexity of the problem. The circuit reaches a stable equilibrium condition irrespective of the initial values of its outputs. For the circuit considered, once a partial compensation was made for the input offset voltages of the operational amplifiers used, the solution generated by the circuit was close to the 0.1% estimated for an ideal circuit. Settling times of about 15 ms were observed. The nature of the particular problem chosen required extremely high gains on the constraint nodes in the circuit. The resulting stability problems were overcome by slowing the response of the signal nodes in the circuit.< >