Results 11 to 20 of about 36,144 (280)
Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delays [PDF]
Advances in Neural Information Processing Systems 35, 2022 The existing analysis of asynchronous stochastic gradient descent (SGD) degrades dramatically when any delay is large, giving the impression that performance depends primarily on the delay. On the contrary, we prove much better guarantees for the same asynchronous SGD algorithm regardless of the delays in the gradients, depending instead just on the ...
Mishchenko, Konstantin +3 more
core +10 more sources
STL-SGD: Speeding Up Local SGD with Stagewise Communication Period
Proceedings of the AAAI Conference on Artificial Intelligence, 2021 Distributed parallel stochastic gradient descent algorithms are workhorses for large scale machine learning tasks. Among them, local stochastic gradient descent (Local SGD) has attracted significant attention due to its low communication complexity.
Shuheng Shen +3 more
core +5 more sources
SGD: Saccharomyces Genome Database [PDF]
Nucleic Acids Research, 1998 The Saccharomyces Genome Database (SGD) provides Internet access to the complete Saccharomyces cerevisiae genomic sequence, its genes and their products, the phenotypes of its mutants, and the literature supporting these data. The amount of information and the number of features provided by SGD have increased greatly following the release of the S ...
J. Michael Cherry +11 more
openaire +3 more sources
PCDP-SGD: Improving the Convergence of Differentially Private SGD via Projection in Advance [PDF]
CoRR, 2023 We have made major revisions to the content and it is not suitable for ...
Sha, Haichao +3 more
core +5 more sources
Shift-curvature, SGD, and generalization
Machine Learning: Science and Technology, 2022 Abstract A longstanding debate surrounds the related hypotheses that low-curvature minima generalize better, and that stochastic gradient descent (SGD) discourages curvature. We offer a more complete and nuanced view in support of both hypotheses.
Arwen V. Bradley +2 more
openaire +4 more sources
Communication-efficient SGD: From Local SGD to One-Shot Averaging [PDF]
CoRR, 2021 We consider speeding up stochastic gradient descent (SGD) by parallelizing it across multiple workers. We assume the same data set is shared among $N$ workers, who can take SGD steps and coordinate with a central server. While it is possible to obtain a linear reduction in the variance by averaging all the stochastic gradients at every step, this ...
Artin Spiridonoff +2 more
core +5 more sources
Statistical Inference Using SGD
Proceedings of the AAAI Conference on Artificial Intelligence, 2018 We present a novel method for frequentist statistical inference in M-estimation problems, based on stochastic gradient descent (SGD) with a fixed step size: we demonstrate that the average of such SGD sequences can be used for statistical inference, after proper scaling.
Tianyang Li +3 more
openaire +4 more sources
RCD-SGD: Resource-Constrained Distributed SGD in Heterogeneous Environment Via Submodular Partitioning [PDF]
2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2023 9 pages and 5 ...
Haoze He, Parijat Dube
core +6 more sources
Bootstrap SGD: Algorithmic stability and robustness [PDF]
Analysis and ApplicationsIn this paper, some methods to use the empirical bootstrap approach for stochastic gradient descent (SGD) to minimize the empirical risk over a separable Hilbert space are investigated from the view point of algorithmic stability and statistical robustness. The first two types of approaches are based on averages and are investigated from a theoretical
Andreas Christmann, Yunwen Lei
core +6 more sources
Have it your way: Individualized Privacy Assignment for DP-SGD
, 2023 When training a machine learning model with differential privacy, one sets a privacy budget. This budget represents a maximal privacy violation that any user is willing to face by contributing their data to the training set.
Adam Dziedzic (17978635) +4 more
core +2 more sources