Results 11 to 20 of about 1,338,246 (278)
The inverse variance–flatness relation in stochastic gradient descent is critical for finding flat minima [PDF]
Proceedings of the National Academy of Sciences of the United States of America, 2021 Yuhai Tu, Yu Feng
exaly +2 more sources
Dual Stochastic Natural Gradient Descent
CoRR, 2020 [EN] Although theoretically appealing, Stochastic Natural Gradient Descent (SNGD) is computationally expensive, it has been shown to be highly sensitive to the learning rate, and it is not guaranteed to be convergent.
Sánchez-López, Borja +1 more
core +3 more sources
Stochastic Modified Flows for Riemannian Stochastic Gradient Descent
SIAM Journal on Control and OptimizationGess B, Kassing S, Rana N. Stochastic Modified Flows for Riemannian Stochastic Gradient Descent. SIAM Journal on Control and Optimization. 2024;62(6):3288-3314.We give quantitative estimates for the rate of convergence of Riemannian stochastic gradient ...
Kassing, Sebastian +2 more
core +6 more sources
Communication-efficient stochastic gradient descent, with applications to neural networks [PDF]
, 2017 Parallel implementations of stochastic gradient descent (SGD) have received significant research attention, thanks to its excellent scalability properties.
Vojnovic, Milan +4 more
core +5 more sources
On the different regimes of stochastic gradient descent. [PDF]
Proc Natl Acad Sci U S AModern deep networks are trained with stochastic gradient descent (SGD) whose key hyperparameters are the number of data considered at each step or batch size B , and the step size or learning rate
Sclocchi A, Wyart M.
europepmc +6 more sources
Stochastic gradient descent for wind farm optimization [PDF]
Wind Energy Science, 2023 It is important to optimize wind turbine positions to mitigate potential wake losses. To perform this optimization, atmospheric conditions, such as the inflow speed and direction, are assigned probability distributions according to measured data, which ...
J. Quick +4 more
doaj +1 more source
Efficient high-resolution refinement in cryo-EM with stochastic gradient descent [PDF]
Acta Crystallographica Section D: Structural BiologyRoy R Lederman, Marcus A Brubaker
exaly +2 more sources
AG-SGD: Angle-Based Stochastic Gradient Descent
IEEE Access, 2021 In the field of neural network, stochastic gradient descent is often employed as an effective method of accelerating the result's convergence. Generating the new gradient from the past gradient is a common method adopted by many existing optimization ...
Chongya Song, Alexander Pons, Kang Yen
doaj +1 more source
Stochastic Reweighted Gradient Descent
CoRR, 2021 Despite the strong theoretical guarantees that variance-reduced finite-sum optimization algorithms enjoy, their applicability remains limited to cases where the memory overhead they introduce (SAG/SAGA), or the periodic full gradient computation they require (SVRG/SARAH) are manageable.
Ayoub El Hanchi, David A. Stephens
openaire +2 more sources
IET Control Theory & Applications, 2021 Stochastic gradient descent algorithm is a classical and useful method for stochastic optimisation. While stochastic gradient descent has been theoretically investigated for decades and successfully applied in machine learning such as training of deep ...
Xiaoxue Geng, Gao Huang, Wenxiao Zhao
doaj +1 more source