Metric Entropy and the Optimal Prediction of Chaotic Signals

• Published in: SIAM journal on applied dynamical systems Vol. 12; no. 2; pp. 1085 - 1113
• Main Authors: Viswanath, Divakar, Liang, Xuan, Serkh, Kirill
• Format: Journal Article
• Language: English
• Published: Philadelphia Society for Industrial and Applied Mathematics 01.01.2013
• Subjects: Applied mathematics; Automotive components; Chaos theory; Derivation; Dynamical systems; Entropy; Information theory; Optimization; Texts; Time series; Tolerances
• ISSN: 1536-0040; 1536-0040
• DOI: 10.1137/110824772

Suppose we are given a time series or a signal $x(t)$ for $0\leq t\leq T$. We consider the problem of predicting the signal in the interval $T<t\leq T+t_{f}$ based on a knowledge of its history and nothing more. We ask the following question: what is the largest value of $t_{f}$ for which a prediction can be made? We show that the answer to this question is contained in a fundamental result of information theory due to Wyner, Ziv, Ornstein, and Weiss. In particular, for the class of chaotic signals, the upper bound is $t_{f}\leq\log_{2}T/H$ in the limit $T\rightarrow\infty$, with $H$ being entropy in a sense that is explained in the text. If $\bigl|x(T-s)-x(t^{\ast}-s)\bigr|$ is small for $0\leq s\leq\tau$, where $\tau$ is of the order of a characteristic time scale, the pattern of events leading up to $t=T$ is similar to the pattern of events leading up to $t=t^{\ast}$. It is reasonable to expect $x(t^{\ast}+t_{f})$ to be a good predictor of $x(T+t_{f})$. All existing methods for prediction use this idea in one way or another. Unfortunately, this intuitively reasonable idea is fundamentally deficient, and all existing methods fall well short of the Wyner--Ziv entropy bound on $t_{f}$. An optimal predictor should decompose the distance between the pattern of events leading up to $t=T$ and the pattern leading up to $t=t^{\ast}$ into stable and unstable components. A good match should have suitably small unstable components but will in general allow stable components which are as large as the tolerance for correct prediction. For the special case of toral automorphisms, we use Pade approximants and derive a predictor which has these properties and which seems to point the way to the derivation of a more general optimal predictor. [PUBLICATION ABSTRACT]