Results 91 to 100 of about 22,588 (154)

Stochastic Modified Flows for Riemannian Stochastic Gradient Descent

SIAM Journal on Control and Optimization
We give quantitative estimates for the rate of convergence of Riemannian stochastic gradient descent (RSGD) to Riemannian gradient flow and to a diffusion process, the so-called Riemannian stochastic modified flow (RSMF). Using tools from stochastic differential geometry we show that, in the small learning rate regime, RSGD can be approximated by the ...
Benjamin Gess, Sebastian Kassing, Nimit Rana   +2 more
openaire   +4 more sources

An Improved Reacceleration Optimization Algorithm Based on the Momentum Method for Image Recognition

Mathematics
The optimization algorithm plays a crucial role in image recognition by neural networks. However, it is challenging to accelerate the model’s convergence and maintain high precision.
Haijing Sun, Ying Cai, Ran Tao, Yichuan Shao, Lei Xing, Can Zhang, Qian Zhao   +6 more
doaj   +1 more source

The inverse variance-flatness relation in stochastic gradient descent is critical for finding flat minima. [PDF]

Proc Natl Acad Sci U S A, 2021
Feng Y, Tu Y.
europepmc   +1 more source

Dual Stochastic Natural Gradient Descent

CoRR, 2020
[EN] Although theoretically appealing, Stochastic Natural Gradient Descent (SNGD) is computationally expensive, it has been shown to be highly sensitive to the learning rate, and it is not guaranteed to be convergent. Convergent Stochastic Natural Gradient Descent (CSNGD) aims at solving the last two problems.
Sánchez-López, Borja, Cerquides, Jesús   +1 more
openaire   +2 more sources

Non-Iterative Phase-Only Hologram Generation via Stochastic Gradient Descent Optimization

Photonics
In this work, we explored, for the first time, to the best of our knowledge, the potential of stochastic gradient descent (SGD) to optimize random phase functions for application in non-iterative phase-only hologram generation.
Alejandro Velez-Zea, John Fredy Barrera-Ramírez   +1 more
doaj   +1 more source

Sentiment classification for employees reviews using regression vector- stochastic gradient descent classifier (RV-SGDC). [PDF]

PeerJ Comput Sci, 2021
Gaye B, Zhang D, Wulamu A.
europepmc   +1 more source

A Geometric Interpretation of Stochastic Gradient Descent Using Diffusion Metrics. [PDF]

Entropy (Basel), 2020
Fioresi R, Chaudhari P, Soatto S.
europepmc   +1 more source

Tighter privacy auditing of differentially private stochastic gradient descent in the hidden state threat model. [PDF]

Sci Rep
Bhuekar A.
europepmc   +1 more source

SSGD: SPARSITY-PROMOTING STOCHASTIC GRADIENT DESCENT ALGORITHM FOR UNBIASED DNN PRUNING. [PDF]

Proc IEEE Int Conf Acoust Speech Signal Process, 2020
Lee CH, Fedorov I, Rao BD, Garudadri H.
europepmc   +1 more source

Mutual Information Based Learning Rate Decay for Stochastic Gradient Descent Training of Deep Neural Networks. [PDF]

Entropy (Basel), 2020
Vasudevan S.
europepmc   +1 more source