Gauss Newton Method for Solving Variational Problems of PDEs with Neural Network Discretizaitons

• Published in: Journal of scientific computing Vol. 100; no. 1; p. 17
• Main Authors: Hao, Wenrui, Hong, Qingguo, Jin, Xianlin
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.07.2024; Springer Nature B.V
• Subjects: Algorithms; Approximation; Boundary conditions; Computational mathematics; Computational Mathematics and Numerical Analysis; Differential equations; Energy; Greedy algorithms; Machine learning; Mathematical analysis; Mathematical and Computational Engineering; Mathematical and Computational Physics; Mathematics; Mathematics and Statistics; Methods; Neural networks; Newton methods; Nonlinear equations; Partial differential equations; Ritz method; Theoretical; Gauss–Newton method; Variational form; Partial differential equations; Neural network discretization; Convergence analysis; Gauss-Newton method
• ISSN: 0885-7474; 1573-7691; 1573-7691
• DOI: 10.1007/s10915-024-02535-z

The numerical solution of differential equations using machine learning-based approaches has gained significant popularity. Neural network-based discretization has emerged as a powerful tool for solving differential equations by parameterizing a set of functions. Various approaches, such as the deep Ritz method and physics-informed neural networks, have been developed for numerical solutions. Training algorithms, including gradient descent and greedy algorithms, have been proposed to solve the resulting optimization problems. In this paper, we focus on the variational formulation of the problem and propose a Gauss–Newton method for computing the numerical solution. We provide a comprehensive analysis of the superlinear convergence properties of this method, along with a discussion on semi-regular zeros of the vanishing gradient. Numerical examples are presented to demonstrate the efficiency of the proposed Gauss–Newton method.