Results 11 to 20 of about 12,502,488 (286)
Accelerated Gossip via Stochastic Heavy Ball Method [PDF]
2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2018 In this paper we show how the stochastic heavy ball method (SHB)–a popular method for solving stochastic convex and non-convex optimization problems–operates as a randomized gossip algorithm.
Nicolas Loizou, Peter Richtárik
semanticscholar +8 more sources
On growth error bound conditions with an application to heavy ball method
Journal of Optimization Theory and Applications, 2023 In this paper, we investigate the growth error bound condition. By using the proximal point algorithm, we first provide a more accessible and elementary proof of the fact that Kurdyka-Łojasiewicz conditions imply growth error bound conditions for convex ...
Qi-Nian Jin
semanticscholar +4 more sources
FedUHB: Accelerating Federated Unlearning via Polyak Heavy Ball Method [PDF]
2024 IEEE Information Theory Workshop (ITW)Federated learning facilitates collaborative machine learning, enabling multiple participants to collectively develop a shared model while preserving the privacy of individual data.
Yu Jiang, Chee-Wei Tan, Kwok-Yan Lam
semanticscholar +5 more sources
Convergence Rates of the Heavy Ball Method for Quasi-strongly Convex Optimization
SIAM Journal on Optimization, 2020 In this paper, we study the behavior of solutions of the ODE associated to the Heavy Ball method. Since the pioneering work of B.T. Polyak [25], it is well known that such a scheme is very efficient for C2 strongly convex functions with Lipschitz ...
Jean-François Aujol +2 more
semanticscholar +5 more sources
Non-monotone Behavior of the Heavy Ball Method [PDF]
Difference Equations and Discrete Dynamical Systems with Applications, 2018 We focus on the solutions of second-order stable linear difference equations and demonstrate that their behavior can be non-monotone and exhibit peak effects depending on initial conditions.
Marina Danilova +2 more
semanticscholar +4 more sources
Local Convergence of the Heavy-Ball Method and iPiano for Non-convex Optimization [PDF]
Journal of Optimization Theory and Applications, 2016 A local convergence result for an abstract descent method is proved. The sequence of iterates is attracted by a local (or global) minimum, stays in its neighborhood, and converges within this neighborhood.
Peter Ochs
semanticscholar +6 more sources
Accelerated Convergence of Stochastic Heavy Ball Method under Anisotropic Gradient Noise
CoRR, 2023 Heavy-ball momentum with decaying learning rates is widely used with SGD for optimizing deep learning models. In contrast to its empirical popularity, the understanding of its theoretical property is still quite limited, especially under the standard ...
Rui Pan +3 more
semanticscholar +5 more sources
Global convergence of the Heavy-ball method for convex optimization [PDF]
2015 European Control Conference (ECC), 2015 This paper establishes global convergence and provides global bounds of the convergence rate of the Heavy-ball method for convex optimization problems. When the objective function has Lipschitz-continuous gradient, we show that the Cesaro average of the iterates converges to the optimum at a rate of $O(1/k)$ where k is the number of iterations.
Mikael Johansson +2 more
exaly +4 more sources
On the Global Convergence of Continuous–Time Stochastic Heavy–Ball Method for Nonconvex Optimization [PDF]
2019 IEEE International Conference on Big Data (Big Data), 2017 We study the convergence behavior of a stochastic heavy-ball method with a small stepsize. Under a change of time scale, we approximate the discrete scheme by a stochastic differential equation that models small random perturbations of a coupled system ...
Wenqing Hu, C. Li, Xiang Zhou
semanticscholar +3 more sources
Evolution Equations and Control Theory, 2022 Let \begin{document}$ f: {\mathcal H} \rightarrow \mathbb{R} $\end{document} be a convex differentiable function whose solution set \begin{document}$ {{\rm{argmin}}}\; f $\end{document} is nonempty.
Akram Chahid Bagy, Z. Chbani, H. Riahi
semanticscholar +3 more sources