Results 11 to 20 of about 22,588 (154)
On the different regimes of stochastic gradient descent [PDF]
Proceedings of the National Academy of Sciences of the United States of AmericaModern deep networks are trained with stochastic gradient descent (SGD) whose key hyperparameters are the number of data considered at each step or batch size B , and the step size or learning rate
Antonio Sclocchi +2 more
exaly +6 more sources
Stochastic gradient descent for optimization for nuclear systems [PDF]
Scientific Reports, 2023 The use of gradient descent methods for optimizing k-eigenvalue nuclear systems has been shown to be useful in the past, but the use of k-eigenvalue gradients have proved computationally challenging due to their stochastic nature.
Austin Williams +5 more
doaj +2 more sources
Stabilizing updates in differentially private stochastic gradient descent with buffered rejection [PDF]
Scientific ReportsDifferentially private stochastic gradient descent is a standard algorithm for training deep models on sensitive data, but under tight privacy budgets it must add large noise to every step, which slows convergence and reduces accuracy.
Sifan Deng +4 more
doaj +2 more sources
Stochastic Gradient Descent on Riemannian Manifolds [PDF]
IEEE Transactions on Automatic Control, 2013 Stochastic gradient descent is a simple approach to find the local minima of a cost function whose evaluations are corrupted by noise. In this paper, we develop a procedure extending stochastic gradient descent algorithms to the case where the function is defined on a Riemannian manifold.
Silvere Bonnabel
exaly +3 more sources
Preconditioned Stochastic Gradient Descent [PDF]
IEEE Transactions on Neural Networks and Learning Systems, 2018 13 pages, 9 figures. To appear in IEEE Transactions on Neural Networks and Learning Systems.
Xi-Lin Li
exaly +4 more sources
Semi-Stochastic Gradient Descent Methods [PDF]
Frontiers in Applied Mathematics and Statistics, 2017 In this paper we study the problem of minimizing the average of a large number of smooth convex loss functions. We propose a new method, S2GD (Semi-Stochastic Gradient Descent), which runs for one or several epochs in each of which a single full gradient
Jakub Konečný, Peter Richtárik
doaj +3 more sources
A stochastic multiple gradient descent algorithm [PDF]
European Journal of Operational Research, 2018 zbMATH Open Web Interface contents unavailable due to conflicting licenses.
Fabrice Poirion +2 more
exaly +4 more sources
Stochastic Gradient Descent for Kernel-Based Maximum Correntropy Criterion [PDF]
EntropyMaximum correntropy criterion (MCC) has been an important method in machine learning and signal processing communities since it was successfully applied in various non-Gaussian noise scenarios.
Tiankai Li +3 more
doaj +2 more sources
Implicit Stochastic Gradient Descent Method for Cross-Domain Recommendation System [PDF]
Sensors, 2020 The previous recommendation system applied the matrix factorization collaborative filtering (MFCF) technique to only single domains. Due to data sparsity, this approach has a limitation in overcoming the cold-start problem.
Nam D. Vo, Minsung Hong, Jason J. Jung
doaj +2 more sources
Recent Advances in Stochastic Gradient Descent in Deep Learning
Mathematics, 2023 In the age of artificial intelligence, the best approach to handling huge amounts of data is a tremendously motivating and hard problem. Among machine learning models, stochastic gradient descent (SGD) is not only simple but also very effective.
Yingjie Tian, Yuqi Zhang, Haibin Zhang
doaj +3 more sources