Results 61 to 70 of about 22,588 (154)

Analysis of stochastic gradient descent in continuous time [PDF]

Statistics and Computing, 2021
AbstractStochastic gradient descent is an optimisation method that combines classical gradient descent with random subsampling within the target functional. In this work, we introduce the stochastic gradient process as a continuous-time representation of stochastic gradient descent.
openaire   +5 more sources

On the Generalization of Stochastic Gradient Descent with Momentum

J. Mach. Learn. Res., 2018
While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that there exists a convex loss function for which the stability gap for multiple epochs of SGD with ...
Ramezani-Kebrya, Ali, Antonakopoulos, Kimon, Cevher, Volkan, Khisti, Ashish, Liang, Ben   +4 more
openaire   +4 more sources

A new approach to training neural networks using natural gradient descent with momentum based on Dirichlet distributions

Компьютерная оптика, 2023
In this paper, we propose a natural gradient descent algorithm with momentum based on Dirichlet distributions to speed up the training of neural networks. This approach takes into account not only the direction of the gradients, but also the convexity of
R.I. Abdulkadirov, P.A. Lyakhov
doaj   +1 more source

Balancing Privacy and Utility in Artificial Intelligence-Based Clinical Decision Support: Empirical Evaluation Using De-Identified Electronic Health Record Data

Applied Sciences
The secondary use of electronic health records is essential for developing artificial intelligence-based clinical decision support systems. However, even after direct identifiers are removed, de-identified electronic health records remain vulnerable to ...
Jungwoo Lee, Kyu Hee Lee
doaj   +1 more source

Revisiting Stochastic Approximation and Stochastic Gradient Descent

CoRR
31 ...
Rajeeva Laxman Karandikar, Bhamidi Visweswara Rao, Mathukumalli Vidyasagar   +2 more
openaire   +2 more sources

Federated Accelerated Stochastic Gradient Descent

CoRR, 2020
Accepted to NeurIPS 2020. Best paper in International Workshop on Federated Learning for User Privacy and Data Confidentiality in Conjunction with ICML 2020 (FL-ICML'20).
Honglin Yuan, Tengyu Ma 0001
openaire   +3 more sources

A Static Security Region Analysis of New Power Systems Based on Improved Stochastic–Batch Gradient Pile Descent

Applied Sciences
The uncertainty in the new power system has increased, leading to limitations in traditional stability analysis methods. Therefore, considering the perspective of the three-dimensional static security region (SSR), we propose a novel approach for system ...
Jiahui Wu, Yide Zhou, Haiyun Wang, Weiqing Wang   +3 more
doaj   +1 more source

Semi-Cyclic Stochastic Gradient Descent

CoRR, 2019
We consider convex SGD updates with a block-cyclic structure, i.e. where each cycle consists of a small number of blocks, each with many samples from a possibly different, block-specific, distribution. This situation arises, e.g., in Federated Learning where the mobile devices available for updates at different times during the day have different ...
Hubert Eichner, Tomer Koren, Brendan McMahan, Nathan Srebro, Kunal Talwar   +4 more
openaire   +3 more sources

On the Hyperparameters in Stochastic Gradient Descent with Momentum

CoRR, 2021
34 pages, 4 figures.
openaire   +3 more sources

Unforgeability in Stochastic Gradient Descent

Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, 2023
Teodora Baluta, Ivica Nikolic, Racchit Jain, Divesh Aggarwal, Prateek Saxena   +4 more
openaire   +1 more source