Results 71 to 80 of about 22,588 (154)
On the regularizing property of stochastic gradient descent
Inverse Problems, 2018 Stochastic gradient descent is one of the most successful approaches for solving large-scale problems, especially in machine learning and statistics. At each iteration, it employs an unbiased estimator of the full gradient computed from one single randomly selected data point.
Bangti Jin, Xiliang Lu
openaire +3 more sources
Adam Algorithm with Step Adaptation
AlgorithmsAdam (Adaptive Moment Estimation) is a well-known algorithm for the first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments.
Vladimir Krutikov +2 more
doaj +1 more source
A Bootstrap Perspective on Stochastic Gradient Descent
CoRRMachine learning models trained with \emph{stochastic} gradient descent (SGD) can generalize better than those trained with deterministic gradient descent (GD). In this work, we study SGD's impact on generalization through the lens of the statistical bootstrap: SGD uses gradient variability under batch sampling as a proxy for solution variability under
Hongjian Lan +2 more
openaire +2 more sources
Adaptive Natural Gradient Method for Learning of Stochastic Neural Networks in Mini-Batch Mode
Applied Sciences, 2019 Gradient descent method is an essential algorithm for learning of neural networks. Among diverse variations of gradient descent method that have been developed for accelerating learning speed, the natural gradient learning is based on the theory of ...
Hyeyoung Park, Kwanyong Lee
doaj +1 more source
Convergence of Stochastic Gradient Descent for PCA
CoRR, 2015 We consider the problem of principal component analysis (PCA) in a streaming stochastic setting, where our goal is to find a direction of approximate maximal variance, based on a stream of i.i.d. data points in $\reals^d$. A simple and computationally cheap algorithm for this is stochastic gradient descent (SGD), which incrementally updates its ...
openaire +3 more sources
Mixing of Stochastic Accelerated Gradient Descent
CoRR, 2019 We study the mixing properties for stochastic accelerated gradient descent (SAGD) on least-squares regression. First, we show that stochastic gradient descent (SGD) and SAGD are simulating the same invariant distribution. Motivated by this, we then establish mixing rate for SAGD-iterates and compare it with those of SGD-iterates.
Peiyuan Zhang +2 more
openaire +2 more sources
Stochastic Gradient Descent with Adaptive Data
Operations ResearchStochastic Gradient Descent with Adaptive Data Stochastic gradient descent (SGD) is a central tool in modern optimization, but its classical theory relies on the assumption that data are independent of the decisions being optimized. In many operations research settings, this assumption fails: policies influence system dynamics, and ...
Ethan Che, Jing Dong, Xin T. Tong
openaire +2 more sources
Scaling of hardware-compatible perturbative training algorithms
APL Machine LearningIn this work, we explore the capabilities of multiplexed gradient descent (MGD), a scalable and efficient perturbative zeroth-order training method for estimating the gradient of a loss function in hardware and training it via stochastic gradient descent.
B. G. Oripov +3 more
doaj +1 more source
A novel deep learning technique for medical image analysis using improved optimizer
Health Informatics JournalApplication of Convolutional neural network in spectrum of Medical image analysis are providing benchmark outputs which converges the interest of many researchers to explore it in depth.
Vertika Agarwal +2 more
doaj +1 more source
Stochastic gradient descent optimisation for convolutional neural network for medical image segmentation. [PDF]
Open Life Sci, 2023 Nagendram S +7 more
europepmc +1 more source