Results 11 to 20 of about 142,362 (315)
Efficient high-resolution refinement in cryo-EM with stochastic gradient descent. [PDF]
Acta Crystallogr D Struct BiolToader B, Brubaker MA, Lederman RR.
europepmc +3 more sources
The inverse variance-flatness relation in stochastic gradient descent is critical for finding flat minima. [PDF]
Proc Natl Acad Sci U S A, 2021 Feng Y, Tu Y.
europepmc +3 more sources
AG-SGD: Angle-Based Stochastic Gradient Descent
IEEE Access, 2021 In the field of neural network, stochastic gradient descent is often employed as an effective method of accelerating the result's convergence. Generating the new gradient from the past gradient is a common method adopted by many existing optimization ...
Chongya Song, Alexander Pons, Kang Yen
doaj +1 more source
Asynchronous Decentralized Accelerated Stochastic Gradient Descent [PDF]
IEEE Journal on Selected Areas in Information Theory, 2021 In this work, we introduce an asynchronous decentralized accelerated stochastic gradient descent type of method for decentralized stochastic optimization, considering communication and synchronization are the major bottlenecks. We establish $\mathcal{O}(1/ )$ (resp., $\mathcal{O}(1/\sqrt )$) communication complexity and $\mathcal{O}(1/ ^2)$ (resp., $
Guanghui Lan, Yi Zhou
openaire +2 more sources
IET Control Theory & Applications, 2021 Stochastic gradient descent algorithm is a classical and useful method for stochastic optimisation. While stochastic gradient descent has been theoretically investigated for decades and successfully applied in machine learning such as training of deep ...
Xiaoxue Geng, Gao Huang, Wenxiao Zhao
doaj +1 more source
Training a Two-Layer ReLU Network Analytically
Sensors, 2023 Neural networks are usually trained with different variants of gradient descent-based optimization algorithms such as the stochastic gradient descent or the Adam optimizer.
Adrian Barbu
doaj +1 more source
Stochastic Gradient Descent with Polyak’s Learning Rate [PDF]
Journal of Scientific Computing, 2021 Stochastic gradient descent (SGD) for strongly convex functions converges at the rate $\bO(1/k)$. However, achieving good results in practice requires tuning the parameters (for example the learning rate) of the algorithm. In this paper we propose a generalization of the Polyak step size, used for subgradient methods, to Stochastic gradient descent. We
Mariana Prazeres, Adam M. Oberman
openaire +2 more sources
Adaptive Stochastic Gradient Descent Method for Convex and Non-Convex Optimization
Fractal and Fractional, 2022 Stochastic gradient descent is the method of choice for solving large-scale optimization problems in machine learning. However, the question of how to effectively select the step-sizes in stochastic gradient descent methods is challenging, and can ...
Ruijuan Chen, Xiaoquan Tang, Xiuting Li
doaj +1 more source
Stochastic gradient descent for hybrid quantum-classical optimization [PDF]
Quantum, 2020 Within the context of hybrid quantum-classical optimization, gradient descent based optimizers typically require the evaluation of expectation values with respect to the outcome of parameterized quantum circuits. In this work, we explore the consequences
Ryan Sweke +6 more
doaj +1 more source
Pipelined Stochastic Gradient Descent with Taylor Expansion
Applied Sciences, 2023 Stochastic gradient descent (SGD) is an optimization method typically used in deep learning to train deep neural network (DNN) models. In recent studies for DNN training, pipeline parallelism, a type of model parallelism, is proposed to accelerate SGD ...
Bongwon Jang, Inchul Yoo, Dongsuk Yook
doaj +1 more source