Making the Error-Controlling Algorithm of Observable Operator Models Constructive

• Published in: Neural computation Vol. 21; no. 12; pp. 3460 - 3486
• Main Authors: Zhao, Ming-Jie, Jaeger, Herbert, Thon, Michael
• Format: Journal Article
• Language: English
• Published: One Rogers Street, Cambridge, MA 02142-1209, USA MIT Press 01.12.2009; MIT Press Journals, The
• Subjects: Accuracy; Algorithms; Applied sciences; Artificial intelligence; Asymptotic methods; Biological and medical sciences; Computer science; control theory; systems; Exact sciences and technology; Fundamental and applied biological sciences. Psychology; General aspects; Humans; Learning; Learning and adaptive systems; Letters; Markov analysis; Markov Chains; Mathematics; Mathematics in biology. Statistical analysis. Models. Metrology. Data processing in biology (general aspects); Miscellaneous; Models, Statistical; Neural Networks (Computer); Parametric inference; Probability and statistics; Sciences and techniques of general use; Statistics; Stochastic models; Symbolism; Stochastic model; Neural computation; Error estimation; Iterative method; Iteration; Markov model; Neural network; Stochastic process; Modeling; Learning; Upper bound; Hidden Markov models; Hidden Markov model; EM algorithm; Learning algorithm; Gain
• ISSN: 0899-7667; 1530-888X
• DOI: 10.1162/neco.2009.10-08-878

Observable operator models (OOMs) are a class of models for stochastic processes that properly subsumes the class that can be modeled by finite-dimensional hidden Markov models (HMMs). One of the main advantages of OOMs over HMMs is that they admit asymptotically correct learning algorithms. A series of learning algorithms has been developed, with increasing computational and statistical efficiency, whose recent culmination was the error-controlling (EC) algorithm developed by the first author. The EC algorithm is an iterative, asymptotically correct algorithm that yields (and minimizes) an assured upper bound on the modeling error. The run time is faster by at least one order of magnitude than EM-based HMM learning algorithms and yields significantly more accurate models than the latter. Here we present a significant improvement of the EC algorithm: the constructive error-controlling (CEC) algorithm. CEC inherits from EC the main idea of minimizing an upper bound on the modeling error but is constructive where EC needs iterations. As a consequence, we obtain further gains in learning speed without loss in modeling accuracy.