Results 21 to 30 of about 22,674 (253)
Stochastic Gradient Descent in Continuous Time [PDF]
SSRN Electronic Journal, 2017 Stochastic gradient descent in continuous time (SGDCT) provides a computationally efficient method for the statistical learning of continuous-time models, which are widely used in science, engineering, and finance. The SGDCT algorithm follows a (noisy) descent direction along a continuous stream of data.
Justin A. Sirignano +1 more
openaire +2 more sources
Byzantine Stochastic Gradient Descent
CoRR, 2018 This paper studies the problem of distributed stochastic optimization in an adversarial setting where, out of the $m$ machines which allegedly compute stochastic gradients every iteration, an $α$-fraction are Byzantine, and can behave arbitrarily and adversarially.
Alistarh, Dan-Adrian +2 more
openaire +4 more sources
Scaling transition from momentum stochastic gradient descent to plain stochastic gradient descent
CoRR, 2021 The plain stochastic gradient descent and momentum stochastic gradient descent have extremely wide applications in deep learning due to their simple settings and low computational complexity. The momentum stochastic gradient descent uses the accumulated gradient as the updated direction of the current parameters, which has a faster training speed ...
Kun Zeng +3 more
openaire +2 more sources
Natural Evolutionary Gradient Descent Strategy for Variational Quantum Algorithms
Intelligent Computing, 2023 Recent research has demonstrated that parametric quantum circuits (PQCs) are affected by gradients that progressively vanish to zero as a function of the number of qubits.
Jianshe Xie +4 more
doaj +1 more source
Randomized Stochastic Gradient Descent Ascent
CoRR, 2021 An increasing number of machine learning problems, such as robust or adversarial variants of existing algorithms, require minimizing a loss function that is itself defined as a maximum. Carrying a loop of stochastic gradient ascent (SGA) steps on the (inner) maximization problem, followed by an SGD step on the (outer) minimization, is known as Epoch ...
Sebbouh, Othmane +2 more
openaire +4 more sources
Stochastic gradient-free descents
CoRR, 2019 In this paper we propose stochastic gradient-free methods and accelerated methods with momentum for solving stochastic optimization problems. All these methods rely on stochastic directions rather than stochastic gradients. We analyze the convergence behavior of these methods under the mean-variance framework, and also provide a theoretical analysis ...
Xiaopeng Luo, Xin Xu 0006
openaire +2 more sources
On the discrepancy principle for stochastic gradient descent
Inverse Problems, 2020 Abstract Stochastic gradient descent (SGD) is a promising numerical method for solving large-scale inverse problems. However, its theoretical properties remain largely underexplored in the lens of classical regularization theory. In this note, we study the classical discrepancy principle, one of the most popular a posteriori choice rules,
Tim Jahn, Bangti Jin
openaire +4 more sources
Featured Hybrid Recommendation System Using Stochastic Gradient Descent
International Journal of Networked and Distributed Computing (IJNDC), 2021 Beside cold-start and sparsity, developing incremental algorithms emerge as interesting research to recommendation system in real-data environment. While hybrid system research is insufficient due to the complexity in combining various source of each ...
Si Thin Nguyen +3 more
doaj +1 more source
Granular Elastic Network Regression with Stochastic Gradient Descent
Mathematics, 2022 Linear regression is the use of linear functions to model the relationship between a dependent variable and one or more independent variables. Linear regression models have been widely used in various fields such as finance, industry, and medicine.
Linjie He +3 more
doaj +1 more source
Adaptive Gradient Estimation Stochastic Parallel Gradient Descent Algorithm for Laser Beam Cleanup
Photonics, 2021 For a high-power slab solid-state laser, obtaining high output power and high output beam quality are the most important indicators. Adaptive optics systems can significantly improve beam qualities by compensating for the phase distortions of the laser ...
Shiqing Ma +8 more
doaj +1 more source