Stochastic gradient descent for optimization for nuclear systems [PDF]
Scientific Reports, 2023 The use of gradient descent methods for optimizing k-eigenvalue nuclear systems has been shown to be useful in the past, but the use of k-eigenvalue gradients have proved computationally challenging due to their stochastic nature.
Austin Williams +5 more
doaj +2 more sources
Stabilizing updates in differentially private stochastic gradient descent with buffered rejection [PDF]
Scientific ReportsDifferentially private stochastic gradient descent is a standard algorithm for training deep models on sensitive data, but under tight privacy budgets it must add large noise to every step, which slows convergence and reduces accuracy.
Sifan Deng +4 more
doaj +2 more sources
Semi-Stochastic Gradient Descent Methods [PDF]
Frontiers in Applied Mathematics and Statistics, 2017 In this paper we study the problem of minimizing the average of a large number of smooth convex loss functions. We propose a new method, S2GD (Semi-Stochastic Gradient Descent), which runs for one or several epochs in each of which a single full gradient
Jakub Konečný, Peter Richtárik
doaj +3 more sources
Stochastic Gradient Descent for Kernel-Based Maximum Correntropy Criterion [PDF]
EntropyMaximum correntropy criterion (MCC) has been an important method in machine learning and signal processing communities since it was successfully applied in various non-Gaussian noise scenarios.
Tiankai Li +3 more
doaj +2 more sources
Recent Advances in Stochastic Gradient Descent in Deep Learning
Mathematics, 2023 In the age of artificial intelligence, the best approach to handling huge amounts of data is a tremendously motivating and hard problem. Among machine learning models, stochastic gradient descent (SGD) is not only simple but also very effective.
Yingjie Tian, Yuqi Zhang, Haibin Zhang
doaj +3 more sources
Implicit Stochastic Gradient Descent Method for Cross-Domain Recommendation System [PDF]
Sensors, 2020 The previous recommendation system applied the matrix factorization collaborative filtering (MFCF) technique to only single domains. Due to data sparsity, this approach has a limitation in overcoming the cold-start problem.
Nam D. Vo, Minsung Hong, Jason J. Jung
doaj +2 more sources
Damped Newton Stochastic Gradient Descent Method for Neural Networks Training
Mathematics, 2021 First-order methods such as stochastic gradient descent (SGD) have recently become popular optimization methods to train deep neural networks (DNNs) for good generalization; however, they need a long training time.
Jingcheng Zhou +3 more
doaj +3 more sources
Emergent universal long-range structure in random-organizing systems [PDF]
Nature CommunicationsSelf-organization through noisy interactions is ubiquitous across physics, mathematics, and machine learning, yet how long-range structure emerges from local noisy dynamics remains poorly understood.
Satyam Anand +2 more
doaj +2 more sources
A Sharp Estimate on the Transient Time of Distributed Stochastic Gradient Descent [PDF]
IEEE Transactions on Automatic Control, 2022 Ioannis Paschalidis +2 more
exaly +2 more sources
The inverse variance–flatness relation in stochastic gradient descent is critical for finding flat minima [PDF]
Proceedings of the National Academy of Sciences of the United States of America, 2021 Yuhai Tu, Yu Feng
exaly +2 more sources