Results 1 to 10 of about 12,502,488 (286)

An Adaptive Heavy Ball Method for Ill-Posed Inverse Problems

SIAM Journal on Imaging Sciences
In this paper we consider ill-posed inverse problems, both linear and nonlinear, by a heavy ball method in which a strongly convex regularization function is incorporated to detect the feature of the sought solution. We develop ideas on how to adaptively
Qinian Jin, Qin Huang
exaly   +6 more sources

Averaged heavy-ball method [PDF]

Computer Research and Modeling, 2021
Heavy-Ball method (HB) is known for its simplicity in implementation and practical efficiency. However, as with other momentum methods, it has non-monotone behavior, and for optimal parameters, the method suffers from the so-called peak effect.
Marina Danilova, G. Malinovsky
semanticscholar   +5 more sources

Convergence rates of the Heavy-Ball method under the Łojasiewicz property

Mathematical Programming, 2022
zbMATH Open Web Interface contents unavailable due to conflicting licenses.
Aude Rondepierre
exaly   +5 more sources

A robust control approach to asymptotic optimality of the heavy ball method for optimization of quadratic functions [PDF]

Automatica, 2023
Among first order optimization methods, Polyak's heavy ball method has long been known to guarantee the asymptotic rate of convergence matching Nesterov's lower bound for functions defined in an infinite-dimensional space.
Iman Shames, Ian Petersen
exaly   +7 more sources

Heavy Ball Restarted CMRH Methods for Linear Systems [PDF]

Mathematical and Computational Applications, 2018
The restarted CMRH method (changing minimal residual method based on the Hessenberg process) using fewer operations and storage is an alternative method to the restarted generalized minimal residual method (GMRES) method for linear systems.
Zhongming Teng, Xuansheng Wang
doaj   +3 more sources

An adaptive polyak heavy-ball method

Machine Learning, 2022
zbMATH Open Web Interface contents unavailable due to conflicting licenses.
Samer Said Jr Saab, Minghui Zhu, Asok Ray   +2 more
exaly   +4 more sources

Convergence analysis of a stochastic heavy-ball method for linear ill-posed problems

Journal of Computational and Applied Mathematics
In this paper we consider a stochastic heavy-ball method for solving linear ill-posed inverse problems. With suitable choices of the step-sizes and the momentum coefficients, we establish the regularization property of the method under {\it a priori ...
Qinian Jin, Yanjun Liu
exaly   +5 more sources

Provable non-accelerations of the heavy-ball method [PDF]

Mathematical Programming, 2023
In this work, we show that the heavy-ball ($\HB$) method provably does not reach an accelerated convergence rate on smooth strongly convex problems. More specifically, we show that for any condition number and any choice of algorithmic parameters, either
Baptiste Goujaud, Adrien B. Taylor, Aymeric Dieuleveut   +2 more
semanticscholar   +3 more sources

Universal heavy-ball method for nonconvex optimization under Hölder continuous Hessians [PDF]

Mathematical Programming, 2023
We propose a new first-order method for minimizing nonconvex functions with Lipschitz continuous gradients and Hölder continuous Hessians. The proposed algorithm is a heavy-ball method equipped with two particular restart mechanisms.
Naoki Marumo, Akiko Takeda
semanticscholar   +3 more sources

Global Convergence and Asymptotic Optimality of the Heavy Ball Method for a Class of Nonconvex Optimization Problems [PDF]

IEEE Control Systems Letters, 2022
In this letter we revisit the famous heavy ball method and study its global convergence for a class of non-convex problems with sector-bounded gradient.
V. Ugrinovskii, I. Petersen, Iman Shames
semanticscholar   +5 more sources