Optimization algorithms on matrix manifolds

Many problems in the sciences and engineering can be rephrased as optimization problems on matrix search spaces endowed with a so-called manifold structure. This book shows how to exploit the special structure of such problems to develop efficient numerical algorithms. It places careful emphasis on...

Full description

Saved in:

Bibliographic Details
Main Authors	Absil, P.A, Mahony,R, Sepulchre,R
Format	eBook Book
Language	English
Published	Princeton Princeton University Press 2008
Edition	1
Subjects	Affine space Algorithm Algorithms Analytic function Complex projective space Computation Computational problem COMPUTERS / Computer Science Conjugate gradient method Constrained optimization Directional derivative Eigenvalue algorithm Eigenvalues and eigenvectors Equation Euclidean vector Finsler manifold Fixed-point theorem Gauss–Newton algorithm Geometry Global optimization Gradient descent Inclusion map Invariant subspace Invariant subspace problem Invertible matrix Iteration Iterative method Levenberg–Marquardt algorithm Line search Linear algebra Linear map Linear space (geometry) Linear subspace Local convergence Manifold Mathematical optimization MATHEMATICS MATHEMATICS / Applied Matrices Matrix (mathematics) Matrix decomposition Newton's method Normed vector space Numerical analysis Numerical integration Numerical linear algebra Optimization problem Principal component analysis Product topology Projection (linear algebra) Quadratic equation Quasi-Newton method Quotient space (topology) Rate of convergence Rayleigh quotient Rayleigh quotient iteration Real projective space Riemannian geometry Riemannian manifold Riemannian submanifold Row and column spaces Scale invariance Simultaneous equations Stiefel manifold Submanifold Submersion (mathematics) Subset Subspace topology Tangent space Tangent vector Taylor's theorem Technology Technology & Engineering / Engineering (General) Theorem Topological space Topology Vector space Subset Euclidean vector Iterative method Projection (linear algebra) Matrix (mathematics) Tangent vector Gradient descent Vector space Eigenvalue algorithm Eigenvalues and eigenvectors Invertible matrix Tangent space Finsler manifold Real projective space Numerical linear algebra Invariant subspace problem Topology Subspace topology Riemannian manifold Matrix decomposition Algorithm Complex projective space Gauss–Newton algorithm Line search Directional derivative Numerical analysis Quasi-Newton method Computation Topological space Submersion (mathematics) Linear algebra Inclusion map Normed vector space Mathematical optimization Rayleigh quotient Submanifold Invariant subspace Product topology Fixed-point theorem Iteration Quotient space (topology) Linear subspace Computational problem Riemannian geometry Taylor's theorem Manifold Linear map Rayleigh quotient iteration Stiefel manifold Theorem Row and column spaces Rate of convergence Local convergence Conjugate gradient method Scale invariance Numerical integration Linear space (geometry) Levenberg–Marquardt algorithm Equation Analytic function Affine space Quadratic equation Constrained optimization Geometry Simultaneous equations Global optimization Newton's method Optimization problem Riemannian submanifold Principal component analysis
Online Access	Get full text
ISBN	9780691132983 0691132984 1400830249 9781400830244
DOI	10.1515/9781400830244

Cover

Table of Contents:

Optimization algorithms on matrix manifolds -- Contents -- List of Algorithms -- Foreword -- Notation Conventions -- Chapter One: Introduction -- Chapter Two: Motivation and Applications -- Chapter Three: Matrix Manifolds: First-Order Geometry -- Chapter Four: Line-Search Algorithms on Manifolds -- Chapter Five: Matrix Manifolds: Second-Order Geometry -- Chapter Six: Newton's Method -- Chapter Seven: Trust-Region Methods -- Chapter Eight: A Constellation of Superlinear Algorithms -- Appendix A: Elements of Linear Algebra, Topology, and Calculus -- Bibliography -- Index
Front Matter Table of Contents List of Algorithms Foreword Notation Conventions Chapter One: Introduction Chapter Two: Motivation and Applications Chapter Three: Matrix Manifolds: Chapter Four: Line-Search Algorithms on Manifolds Chapter Five: Matrix Manifolds: Chapter Six: Newton’s Method Chapter Seven: Trust-Region Methods Chapter Eight: A Constellation of Superlinear Algorithms Appendix A Bibliography Index
Intro -- Contents -- List of Algorithms -- Foreword -- Notation Conventions -- 1. Introduction -- 2. Motivation and Applications -- 2.1 A case study: the eigenvalue problem -- 2.1.1 The eigenvalue problem as an optimization problem -- 2.1.2 Some benefits of an optimization framework -- 2.2 Research problems -- 2.2.1 Singular value problem -- 2.2.2 Matrix approximations -- 2.2.3 Independent component analysis -- 2.2.4 Pose estimation and motion recovery -- 2.3 Notes and references -- 3. Matrix Manifolds: First-Order Geometry -- 3.1 Manifolds -- 3.1.1 Definitions: charts, atlases, manifolds -- 3.1.2 The topology of a manifold* -- 3.1.3 How to recognize a manifold -- 3.1.4 Vector spaces as manifolds -- 3.1.5 The manifolds Rnxp and R*nxp -- 3.1.6 Product manifolds -- 3.2 Differentiable functions -- 3.2.1 Immersions and submersions -- 3.3 Embedded submanifolds -- 3.3.1 General theory -- 3.3.2 The Stiefel manifold -- 3.4 Quotient manifolds -- 3.4.1 Theory of quotient manifolds -- 3.4.2 Functions on quotient manifolds -- 3.4.3 The real projective space RPn-1 -- 3.4.4 The Grassmann manifold Grass(p, n) -- 3.5 Tangent vectors and differential maps -- 3.5.1 Tangent vectors -- 3.5.2 Tangent vectors to a vector space -- 3.5.3 Tangent bundle -- 3.5.4 Vector fields -- 3.5.5 Tangent vectors as derivations* -- 3.5.6 Differential of a mapping -- 3.5.7 Tangent vectors to embedded submanifolds -- 3.5.8 Tangent vectors to quotient manifolds -- 3.6 Riemannian metric, distance, and gradients -- 3.6.1 Riemannian submanifolds -- 3.6.2 Riemannian quotient manifolds -- 3.7 Notes and references -- 4. Line-Search Algorithms on Manifolds -- 4.1 Retractions -- 4.1.1 Retractions on embedded submanifolds -- 4.1.2 Retractions on quotient manifolds -- 4.1.3 Retractions and local coordinates* -- 4.2 Line-search methods -- 4.3 Convergence analysis -- 4.3.1 Convergence on manifolds
6.5 Analysis of Rayleigh quotient algorithms -- 6.5.1 Convergence analysis -- 6.5.2 Numerical implementation -- 6.6 Notes and references -- 7. Trust-Region Methods -- 7.1 Models -- 7.1.1 Models in Rn -- 7.1.2 Models in general Euclidean spaces -- 7.1.3 Models on Riemannian manifolds -- 7.2 Trust-region methods -- 7.2.1 Trust-region methods in Rn -- 7.2.2 Trust-region methods on Riemannian manifolds -- 7.3 Computing a trust-region step -- 7.3.1 Computing a nearly exact solution -- 7.3.2 Improving on the Cauchy point -- 7.4 Convergence analysis -- 7.4.1 Global convergence -- 7.4.2 Local convergence -- 7.4.3 Discussion -- 7.5 Applications -- 7.5.1 Checklist -- 7.5.2 Symmetric eigenvalue decomposition -- 7.5.3 Computing an extreme eigenspace -- 7.6 Notes and references -- 8. A Constellation of Superlinear Algorithms -- 8.1 Vector transport -- 8.1.1 Vector transport and affine connections -- 8.1.2 Vector transport by differentiated retraction -- 8.1.3 Vector transport on Riemannian submanifolds -- 8.1.4 Vector transport on quotient manifolds -- 8.2 Approximate Newton methods -- 8.2.1 Finite difference approximations -- 8.2.2 Secant methods -- 8.3 Conjugate gradients -- 8.3.1 Application: Rayleigh quotient minimization -- 8.4 Least-square methods -- 8.4.1 Gauss-Newton methods -- 8.4.2 Levenberg-Marquardt methods -- 8.5 Notes and references -- A. Elements of Linear Algebra, Topology, and Calculus -- A.1 Linear algebra -- A.2 Topology -- A.3 Functions -- A.4 Asymptotic notation -- A.5 Derivatives -- A.6 Taylor's formula -- Bibliography -- Index -- A -- B -- C -- D -- E -- F -- G -- H -- I -- J -- K -- L -- M -- N -- O -- P -- Q -- R -- S -- T -- U -- V -- W -- Z
4.3.2 A topological curiosity* -- 4.3.3 Convergence of line-search methods -- 4.4 Stability of fixed points -- 4.5 Speed of convergence -- 4.5.1 Order of convergence -- 4.5.2 Rate of convergence of line-search methods* -- 4.6 Rayleigh quotient minimization on the sphere -- 4.6.1 Cost function and gradient calculation -- 4.6.2 Critical points of the Rayleigh quotient -- 4.6.3 Armijo line search -- 4.6.4 Exact line search -- 4.6.5 Accelerated line search: locally optimal conjugate gradient -- 4.6.6 Links with the power method and inverse iteration -- 4.7 Refining eigenvector estimates -- 4.8 Brockett cost function on the Stiefel manifold -- 4.8.1 Cost function and search direction -- 4.8.2 Critical points -- 4.9 Rayleigh quotient minimization on the Grassmann manifold -- 4.9.1 Cost function and gradient calculation -- 4.9.2 Line-search algorithm -- 4.10 Notes and references -- 5. Matrix Manifolds: Second-Order Geometry -- 5.1 Newton's method in Rn -- 5.2 Affine connections -- 5.3 Riemannian connection -- 5.3.1 Symmetric connections -- 5.3.2 Definition of the Riemannian connection -- 5.3.3 Riemannian connection on Riemannian submanifolds -- 5.3.4 Riemannian connection on quotient manifolds -- 5.4 Geodesics, exponential mapping, and parallel translation -- 5.5 Riemannian Hessian operator -- 5.6 Second covariant derivative* -- 5.7 Notes and references -- 6. Newton's Method -- 6.1 Newton's method on manifolds -- 6.2 Riemannian Newton method for real-valued functions -- 6.3 Local convergence -- 6.3.1 Calculus approach to local convergence analysis -- 6.4 Rayleigh quotient algorithms -- 6.4.1 Rayleigh quotient on the sphere -- 6.4.2 Rayleigh quotient on the Grassmann manifold -- 6.4.3 Generalized eigenvalue problem -- 6.4.4 The nonsymmetric eigenvalue problem -- 6.4.5 Newton with subspace acceleration: Jacobi-Davidson
Chapter Six. Newton’s Method
List of Algorithms
Notation Conventions
Index
-
/
Chapter Four. Line-Search Algorithms On Manifolds
Contents
Foreword
Chapter One. Introduction
Chapter Three. Matrix Manifolds: First-Order Geometry
Chapter Eight. A Constellation Of Superlinear Algorithms
A. Elements Of Linear Algebra, Topology, And Calculus
Chapter Two. Motivation and Applications
Frontmatter --
Chapter Seven. Trust-Region Methods
Chapter Five. Matrix Manifolds: Second-Order Geometry
Bibliography