The convergence of the Stochastic Gradient Descent (SGD) : a self-contained proof [PDF]
, 2021 We give here a proof of the convergence of the Stochastic Gradient Descent (SGD) in a self-contained ...
Turinici, Gabrel, Gabriel Turinici
core +1 more source
Adaptive Optical Closed-Loop Control Based on the Single-Dimensional Perturbation Descent Algorithm
Sensors, 2023 Modal-free optimization algorithms do not require specific mathematical models, and they, along with their other benefits, have great application potential in adaptive optics.
Bo Chen +4 more
doaj +1 more source
Carathéodory sampling for stochastic gradient descent [PDF]
, 2023 Many problems require to optimize empirical risk functions over large data sets. Gradient descent methods that calculate the full gradient in every descent step do not scale to such datasets.
core +1 more source
An efficient algorithm for data parallelism based on stochastic optimization
Alexandria Engineering Journal, 2022 Deep neural network models can achieve greater performance in numerous machine learning tasks by raising the depth of the model and the amount of training data samples.
Khalid Abdulaziz Alnowibet +3 more
doaj +1 more source
Distributed stochastic gradient descent for link prediction in signed social networks
EURASIP Journal on Advances in Signal Processing, 2019 This paper considers the link prediction problem defined over a signed social network, where the relationship between any two network users can be either positive (friends) or negative (foes).
Han Zhang, Gang Wu, Qing Ling
doaj +1 more source
SSRGD: Simple stochastic recursive gradient descent for escaping saddle points [PDF]
, 2019 We analyze stochastic gradient algorithms for optimizing nonconvex problems. In particular, our goal is to find local minima (second-order stationary points) instead of just finding first-order stationary points which may be some bad unstable saddle ...
Li, Zhize
core
Adaptive gradient descent for convex and non-convex stochastic optimization [PDF]
, 2019 In this paper we propose several adaptive gradient methods for stochastic optimization. Our methods are based on Armijo-type line search and they simultaneously adapt to the unknown Lipschitz constant of the gradient and variance of the stochastic ...
Dvurechensky, Pavel +4 more
core +1 more source
Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework
Tongxin xuebao, 2018 Aiming at the contradiction between the efficiency and privacy of stochastic gradient descent algorithm in distributed computing environment,a stochastic gradient descent algorithm preserving differential privacy based on MapReduce was proposed.Based on ...
Yihan YU, Yu FU, Xiaoping WU
doaj +2 more sources
One-step corrected projected stochastic gradient descent for statistical estimation [PDF]
A generic, fast and asymptotically efficient method for parametric estimation is described. It is based on the projected stochastic gradient descent on the log-likelihood function corrected by a single step of the Fisher scoring algorithm.
Esstafa, Youssef, Brouste, Alexandre
core +3 more sources
MBGDT:Robust Mini-Batch Gradient Descent [PDF]
, 2022 In high dimensions, most machine learning method perform fragile even there are a little outliers. To address this, we hope to introduce a new method with the base learner, such as Bayesian regression or stochastic gradient descent to solve the problem ...
Wang, Yue, Luo, Haozheng, Wang, Hanming
core +1 more source