Results 91 to 100 of about 1,273 (111)

Runtime Analysis of Stochastic Gradient Descent

Proceedings of the 4th International Conference on Computer Science and Application Engineering, 2020
Stochastic gradient descent (SGD) is one of the most famous methods for minimization. However, there are few results about the runtime analysis of SGD because of its randomness. In this paper, we explain how to approximate SGD by stochastic differential equations (SDE).
Guanqiang Hu, Yushan Zhang
openaire   +1 more source

Conjugate Directions for Stochastic Gradient Descent

2002
The method of conjugate gradients provides a very effective way to optimize large, deterministic systems by gradient descent. In its standard form, however, it is not amenable to stochastic approximation of the gradient. Here we explore ideas from conjugate gradient in the stochastic (online) setting, using fast Hessian-gradient products to set up low ...
Nicol N. Schraudolph, Thore Graepel
openaire   +2 more sources

Stochastic Gradient Descent

2017
This chapter gives a broad overview and a historical context around the subject of deep learning. It also gives the reader a roadmap for navigating the book, the prerequisites, and further reading to dive deeper into the subject matter.
openaire   +1 more source

Online Covariance Matrix Estimation in Stochastic Gradient Descent

Journal of the American Statistical Association, 2023

exaly  

Sign Stochastic Gradient Descents without bounded gradient assumption for the finite sum minimization

Neural Networks, 2022
Sun Tao, Dongsheng Li
exaly  

Resampling Stochastic Gradient Descent Cheaply

2023 Winter Simulation Conference (WSC), 2023
Henry Lam, Zitong Wang 0005
openaire   +2 more sources

Robust Stochastic Gradient Descent With Student-t Distribution Based First-Order Momentum

IEEE Transactions on Neural Networks and Learning Systems, 2022
Wendyam Eric Lionel Ilboudo, Taisuke Kobayashi, Kenji Sugimoto   +2 more
exaly  

A Sharp Estimate on the Transient Time of Distributed Stochastic Gradient Descent

IEEE Transactions on Automatic Control, 2022
Ioannis Ch. Paschalidis, Shi Pu, Alex Olshevsky   +2 more
exaly  

Novel Convergence Results of Adaptive Stochastic Gradient Descents

IEEE Transactions on Image Processing, 2021
Sun Tao, Dongsheng Li, Linbo Qiao
exaly  

Efficient and High-quality Recommendations via Momentum-incorporated Parallel Stochastic Gradient Descent-Based Learning

IEEE/CAA Journal of Automatica Sinica, 2021
Xin Luo, Mengchu Zhou, Khaled Sedraoui
exaly