Results 21 to 30 of about 1,338,397 (280)

Conjugate directions for stochastic gradient descent

, 2002
. The method of conjugate gradients provides a very effective way to optimize large, deterministic systems by gradient descent. In its standard form, however, it is not amenable to stochastic approximation of the gradient.
Thore Graepel, Nicol N. Schraudolph
core   +3 more sources

A Sharp Estimate on the Transient Time of Distributed Stochastic Gradient Descent [PDF]

IEEE Transactions on Automatic Control, 2022
Ioannis Ch. Paschalidis, Shi Pu, Alex Olshevsky   +2 more
exaly   +2 more sources

Dual Stochastic Natural Gradient Descent

CoRR, 2020
[EN] Although theoretically appealing, Stochastic Natural Gradient Descent (SNGD) is computationally expensive, it has been shown to be highly sensitive to the learning rate, and it is not guaranteed to be convergent.
Sánchez-López, Borja, Cerquides, Jesús   +1 more
core   +3 more sources

Stochastic Modified Flows for Riemannian Stochastic Gradient Descent

SIAM Journal on Control and Optimization
Gess B, Kassing S, Rana N. Stochastic Modified Flows for Riemannian Stochastic Gradient Descent. SIAM Journal on Control and Optimization. 2024;62(6):3288-3314.We give quantitative estimates for the rate of convergence of Riemannian stochastic gradient ...
Kassing, Sebastian, Gess, Benjamin, Rana, Nimit   +2 more
core   +6 more sources

Communication-efficient stochastic gradient descent, with applications to neural networks [PDF]

, 2017
Parallel implementations of stochastic gradient descent (SGD) have received significant research attention, thanks to its excellent scalability properties.
Vojnovic, Milan, Grubic, Demjan, Tomioka, Ryota, Alistarh, Dan, Liu, Jerry   +4 more
core   +5 more sources

Stochastic gradient descent for wind farm optimization [PDF]

Wind Energy Science, 2023
It is important to optimize wind turbine positions to mitigate potential wake losses. To perform this optimization, atmospheric conditions, such as the inflow speed and direction, are assigned probability distributions according to measured data, which ...
J. Quick, P.-E. Rethore, M. Mølgaard Pedersen, R. V. Rodrigues, M. Friis-Møller   +4 more
doaj   +1 more source

AG-SGD: Angle-Based Stochastic Gradient Descent

IEEE Access, 2021
In the field of neural network, stochastic gradient descent is often employed as an effective method of accelerating the result's convergence. Generating the new gradient from the past gradient is a common method adopted by many existing optimization ...
Chongya Song, Alexander Pons, Kang Yen
doaj   +1 more source

Stochastic Reweighted Gradient Descent

CoRR, 2021
Despite the strong theoretical guarantees that variance-reduced finite-sum optimization algorithms enjoy, their applicability remains limited to cases where the memory overhead they introduce (SAG/SAGA), or the periodic full gradient computation they require (SVRG/SARAH) are manageable.
Ayoub El Hanchi, David A. Stephens
openaire   +2 more sources

Almost sure convergence of randomised‐difference descent algorithm for stochastic convex optimisation

IET Control Theory & Applications, 2021
Stochastic gradient descent algorithm is a classical and useful method for stochastic optimisation. While stochastic gradient descent has been theoretically investigated for decades and successfully applied in machine learning such as training of deep ...
Xiaoxue Geng, Gao Huang, Wenxiao Zhao
doaj   +1 more source

Training a Two-Layer ReLU Network Analytically

Sensors, 2023
Neural networks are usually trained with different variants of gradient descent-based optimization algorithms such as the stochastic gradient descent or the Adam optimizer.
Adrian Barbu
doaj   +1 more source