Results 11 to 20 of about 1,338,397 (280)

On the different regimes of stochastic gradient descent [PDF]

Proceedings of the National Academy of Sciences of the United States of America
Modern deep networks are trained with stochastic gradient descent (SGD) whose key hyperparameters are the number of data considered at each step or batch size B , and the step size or learning rate
Antonio Sclocchi, Matthieu Wyart
exaly   +7 more sources

Stochastic gradient descent for optimization for nuclear systems [PDF]

Scientific Reports, 2023
The use of gradient descent methods for optimizing k-eigenvalue nuclear systems has been shown to be useful in the past, but the use of k-eigenvalue gradients have proved computationally challenging due to their stochastic nature.
Austin Williams, Noah Walton, Austin Maryanski, Sandra Bogetic, Wes Hines, Vladimir Sobes   +5 more
doaj   +2 more sources

Stabilizing updates in differentially private stochastic gradient descent with buffered rejection [PDF]

Scientific Reports
Differentially private stochastic gradient descent is a standard algorithm for training deep models on sensitive data, but under tight privacy budgets it must add large noise to every step, which slows convergence and reduces accuracy.
Sifan Deng, Kai Zhang, Weilin Zhang, Huiqin Jiang, Pei-Wei Tsai   +4 more
doaj   +2 more sources

Byzantine stochastic gradient descent

CoRR, 2018
This paper studies the problem of distributed stochastic optimization in an adversarial setting where, out of m machines which allegedly compute stochastic gradients every iteration, an α-fraction are Byzantine, and may behave adversarially.
Li, Jerry, Allen-Zhu, Zeyuan, Alistarh, Dan-Adrian ; https://orcid.org/   +2 more
core   +7 more sources

Adaptive Stochastic Gradient Descent Method for Convex and Non-Convex Optimization

Fractal and Fractional, 2022
Stochastic gradient descent is the method of choice for solving large-scale optimization problems in machine learning. However, the question of how to effectively select the step-sizes in stochastic gradient descent methods is challenging, and can ...
Ruijuan Chen, Xiaoquan Tang, Xiuting Li
doaj   +3 more sources

Stochastic Gradient Descent for Kernel-Based Maximum Correntropy Criterion [PDF]

Entropy
Maximum correntropy criterion (MCC) has been an important method in machine learning and signal processing communities since it was successfully applied in various non-Gaussian noise scenarios.
Tiankai Li, Baobin Wang, Chaoquan Peng, Hong Yin   +3 more
doaj   +2 more sources

Semi-Stochastic Gradient Descent Methods [PDF]

Frontiers in Applied Mathematics and Statistics, 2017
In this paper we study the problem of minimizing the average of a large number of smooth convex loss functions. We propose a new method, S2GD (Semi-Stochastic Gradient Descent), which runs for one or several epochs in each of which a single full gradient
Jakub Konečný, Peter Richtárik
doaj   +3 more sources

Recent Advances in Stochastic Gradient Descent in Deep Learning

Mathematics, 2023
In the age of artificial intelligence, the best approach to handling huge amounts of data is a tremendously motivating and hard problem. Among machine learning models, stochastic gradient descent (SGD) is not only simple but also very effective.
Yingjie Tian, Yuqi Zhang, Haibin Zhang
doaj   +3 more sources

Implicit Stochastic Gradient Descent Method for Cross-Domain Recommendation System [PDF]

Sensors, 2020
The previous recommendation system applied the matrix factorization collaborative filtering (MFCF) technique to only single domains. Due to data sparsity, this approach has a limitation in overcoming the cold-start problem.
Nam D. Vo, Minsung Hong, Jason J. Jung
doaj   +2 more sources

Analysis of stochastic gradient descent in continuous time [PDF]

Statistics and Computing, 2021
: Stochastic gradient descent is an optimisation method that combines classical gradient descent with random subsampling within the target functional.

core   +12 more sources