Results 21 to 30 of about 28,952 (292)
Adaptive Stochastic Gradient Descent Method for Convex and Non-Convex Optimization
Fractal and Fractional, 2022 Stochastic gradient descent is the method of choice for solving large-scale optimization problems in machine learning. However, the question of how to effectively select the step-sizes in stochastic gradient descent methods is challenging, and can ...
Ruijuan Chen, Xiaoquan Tang, Xiuting Li
doaj +1 more source
Stochastic gradient descent for hybrid quantum-classical optimization [PDF]
Quantum, 2020 Within the context of hybrid quantum-classical optimization, gradient descent based optimizers typically require the evaluation of expectation values with respect to the outcome of parameterized quantum circuits. In this work, we explore the consequences
Ryan Sweke +6 more
doaj +1 more source
Understanding and Optimizing Asynchronous Low-Precision Stochastic Gradient Descent [PDF]
Computer Architecture News, 2017 Christopher De Sá +2 more
exaly +2 more sources
The effective noise of stochastic gradient descent
Journal of Statistical Mechanics: Theory and Experiment, 2022 Abstract Stochastic gradient descent (SGD) is the workhorse algorithm of deep learning technology. At each step of the training phase, a mini batch of samples is drawn from the training dataset and the weights of the neural network are adjusted according to the performance on this specific subset of examples.
Mignacco, Francesca +1 more
openaire +2 more sources
Pipelined Stochastic Gradient Descent with Taylor Expansion
Applied Sciences, 2023 Stochastic gradient descent (SGD) is an optimization method typically used in deep learning to train deep neural network (DNN) models. In recent studies for DNN training, pipeline parallelism, a type of model parallelism, is proposed to accelerate SGD ...
Bongwon Jang, Inchul Yoo, Dongsuk Yook
doaj +1 more source
Stochastic Gradient Descent in Continuous Time [PDF]
SSRN Electronic Journal, 2017 Stochastic gradient descent in continuous time (SGDCT) provides a computationally efficient method for the statistical learning of continuous-time models, which are widely used in science, engineering, and finance. The SGDCT algorithm follows a (noisy) descent direction along a continuous stream of data.
Justin A. Sirignano +1 more
openaire +2 more sources
Scaling transition from momentum stochastic gradient descent to plain stochastic gradient descent
CoRR, 2021 The plain stochastic gradient descent and momentum stochastic gradient descent have extremely wide applications in deep learning due to their simple settings and low computational complexity. The momentum stochastic gradient descent uses the accumulated gradient as the updated direction of the current parameters, which has a faster training speed ...
Kun Zeng +3 more
openaire +2 more sources
Natural Evolutionary Gradient Descent Strategy for Variational Quantum Algorithms
Intelligent Computing, 2023 Recent research has demonstrated that parametric quantum circuits (PQCs) are affected by gradients that progressively vanish to zero as a function of the number of qubits.
Jianshe Xie +4 more
doaj +1 more source
Randomized Stochastic Gradient Descent Ascent
CoRR, 2021 An increasing number of machine learning problems, such as robust or adversarial variants of existing algorithms, require minimizing a loss function that is itself defined as a maximum. Carrying a loop of stochastic gradient ascent (SGA) steps on the (inner) maximization problem, followed by an SGD step on the (outer) minimization, is known as Epoch ...
Sebbouh, Othmane +2 more
openaire +4 more sources
Stochastic gradient-free descents
CoRR, 2019 In this paper we propose stochastic gradient-free methods and accelerated methods with momentum for solving stochastic optimization problems. All these methods rely on stochastic directions rather than stochastic gradients. We analyze the convergence behavior of these methods under the mean-variance framework, and also provide a theoretical analysis ...
Xiaopeng Luo, Xin Xu 0006
openaire +2 more sources