Results 11 to 20 of about 4,832,495 (330)
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time [PDF]
International Conference on Machine Learning, 2022 The conventional recipe for maximizing model accuracy is to (1) train multiple models with various hyperparameters and (2) pick the individual model which performs best on a held-out validation set, discarding the remainder. In this paper, we revisit the
Mitchell Wortsman +10 more
semanticscholar +1 more source
Stochastic Controlled Averaging for Federated Learning with Communication Compression [PDF]
International Conference on Learning Representations, 2023 Communication compression, a technique aiming to reduce the information volume to be transmitted over the air, has gained great interests in Federated Learning (FL) for the potential of alleviating its communication overhead.
Xinmeng Huang, Ping Li, Xiaoyun Li
semanticscholar +1 more source
Decentralized Federated Averaging [PDF]
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021 Federated averaging (FedAvg) is a communication-efficient algorithm for distributed training with an enormous number of clients. In FedAvg, clients keep their data locally for privacy protection; a central parameter server is used to communicate between ...
Tao Sun, Dongsheng Li, Bao Wang
semanticscholar +1 more source
Diverse Weight Averaging for Out-of-Distribution Generalization [PDF]
Neural Information Processing Systems, 2022 Standard neural networks struggle to generalize under distribution shifts in computer vision. Fortunately, combining multiple networks can consistently improve out-of-distribution generalization. In particular, weight averaging (WA) strategies were shown
Alexandre Ramé +5 more
semanticscholar +1 more source
Federated Learning with Fair Averaging [PDF]
International Joint Conference on Artificial Intelligence, 2021 Fairness has emerged as a critical problem in federated learning (FL). In this work, we identify a cause of unfairness in FL -- conflicting gradients with large differences in the magnitudes. To address this issue, we propose the federated fair averaging
Z. Wang +5 more
semanticscholar +1 more source
Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning [PDF]
AAAI Conference on Artificial Intelligence, 2018 In distributed training of deep neural networks, parallel minibatch SGD is widely used to speed up the training process by using multiple workers. It uses multiple workers to sample local stochastic gradients in parallel, aggregates all gradients in a ...
Hao Yu, Sen Yang, Shenghuo Zhu
semanticscholar +1 more source
Averaging over Narain moduli space [PDF]
Journal of High Energy Physics, 2020 Recent developments involving JT gravity in two dimensions indicate that under some conditions, a gravitational path integral is dual to an average over an ensemble of boundary theories, rather than to a specific boundary theory.
A. Maloney, E. Witten
semanticscholar +1 more source
Genome Biology, 2002 BackgroundGene-expression analysis is increasingly important in biological research, with real-time reverse transcription PCR (RT-PCR) becoming the method of choice for high-throughput and accurate expression profiling of selected genes.
J. Vandesompele +6 more
semanticscholar +1 more source
Revisiting the learning curve (once again)
Frontiers in Psychology, 2013 The vast majority of published work in the field of associative learning seeks to test the adequacy of various theoretical accounts of the learning process using average data.
Steven eGlautier
doaj +1 more source
Discrete-Time Semi-Markov Random Evolutions in Asymptotic Reduced Random Media with Applications
Mathematics, 2020 This paper deals with discrete-time semi-Markov random evolutions (DTSMRE) in reduced random media. The reduction can be done for ergodic and non ergodic media.
Nikolaos Limnios, Anatoliy Swishchuk
doaj +1 more source