The stability of variable step-size LMS algorithms

• Published in: IEEE transactions on signal processing Vol. 47; no. 12; pp. 3277 - 3288
• Main Authors: Gelfand, S.B., Yongbin Wei, Krogmeier, J.V.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.12.1999; Institute of Electrical and Electronics Engineers
• Subjects: Adaptive filters; Algorithms; Applied sciences; Computer simulation; Covariance; Covariance matrix; Dealing; Detection, estimation, filtering, equalization, prediction; Difference equations; Exact sciences and technology; Filtering algorithms; Gaussian; Information, signal and communications theory; Least squares approximation; Mathematical analysis; Nonlinear equations; Robustness; Signal and communications theory; Signal, noise; Stability; Stability analysis; Symmetric matrices; Telecommunications and information theory; Adaptive filtering; Stability; Performance analysis; Adaptive algorithm; Least squares method; Variable step method
• ISSN: 1053-587X
• DOI: 10.1109/78.806072

Variable step-site LMS (VSLMS) algorithms are a popular approach to adaptive filtering, which can provide improved performance while maintaining the simplicity and robustness of conventional fixed step-size LMS. Here, we examine the stability of VSLMS with uncorrelated stationary Gaussian data. Most VSLMS described in the literature use a data-dependent step-size, where the step-size either depends on the data before the current time (prior step-size rule) or through the current time (posterior step-size rule). It has often been assumed that VSLMS algorithms are stable (in the sense of mean-square bounded weights), provided that the step-size is constrained to lie within the corresponding stability region for the LMS algorithm. For a single tap fitter, we find exact expressions for the stability region of VSLMS over the classes of prior and posterior step-sizes and show that the stability region for prior step size coincides with that of fixed step-size, but the region for posterior step-size is strictly smaller than for fixed step-size. For the multiple tap case, we obtain bounds on the stability regions with similar properties. The approach taken here is a generalization of the classical method of analyzing, the exponential stability of the weight covariance equation for LMS. Although it is not possible to derive a weight covariance equation for general data-dependent VSLMS, the weight variances can be upper bounded by the solution of a linear time-invariant difference equation, after appropriately dealing with certain nonlinear terms. For prior step-size (like fixed step-size), the state matrix is symmetric, whereas for posterior step-size, the symmetry is lost, requiring a more detailed analysis. The results are verified by computer simulations.