Results 111 to 120 of about 1,338,397 (280)
Stability and Generalization of Decentralized Stochastic Gradient Descent
, 2021 The stability and generalization of stochastic gradient-based methods provide valuable insights into understanding the algorithmic performance of machine learning models.
Li, Dongsheng, Wang, Bao, Sun, Tao
core +1 more source
Bioinspired Adaptive Sensors: A Review on Current Developments in Theory and Application
Advanced Materials, EarlyView.This review comprehensively summarizes the recent progress in the design and fabrication of sensory‐adaptation‐inspired devices and highlights their valuable applications in electronic skin, wearable electronics, and machine vision. The existing challenges and future directions are addressed in aspects such as device performance optimization ...
Guodong Gong +12 more
wiley +1 more source
Asymptotic Analysis of Conditioned Stochastic Gradient Descent [PDF]
, 2023 In this paper, we investigate a general class of stochastic gradient descent (SGD) algorithms, called Conditioned SGD, based on a preconditioning of the gradient direction.
Leluc, Rémi, Portier, François
core +1 more source
Adaptive gradient descent for convex and non-convex stochastic optimization [PDF]
, 2019 In this paper we propose several adaptive gradient methods for stochastic optimization. Our methods are based on Armijo-type line search and they simultaneously adapt to the unknown Lipschitz constant of the gradient and variance of the stochastic ...
Dvurechensky, Pavel +4 more
core +1 more source
A Bootstrap Perspective on Stochastic Gradient Descent
CoRRMachine learning models trained with \emph{stochastic} gradient descent (SGD) can generalize better than those trained with deterministic gradient descent (GD). In this work, we study SGD's impact on generalization through the lens of the statistical bootstrap: SGD uses gradient variability under batch sampling as a proxy for solution variability under
Hongjian Lan +2 more
openaire +2 more sources
Advanced Materials, EarlyView.A Nb‐proximitized Josephson junction based on a WTe2/α‐Fe2O3 heterostructure exhibits a robust superconducting diode effect with programmable polarity. The diode direction can be trained by magnetic fields and switched by temperature cycling, revealing tunable finite‐momentum pairing states and competing superconducting states in symmetry‐broken ...
Enze Zhang +9 more
wiley +1 more source
Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks [PDF]
Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent. The picture can be radically different for narrow net-works, which tend to get stuck in badly-generalizing ...
Stephan, Ludovic +4 more
core
Convergence of Stochastic Gradient Descent for PCA
CoRR, 2015 We consider the problem of principal component analysis (PCA) in a streaming stochastic setting, where our goal is to find a direction of approximate maximal variance, based on a stream of i.i.d. data points in $\reals^d$. A simple and computationally cheap algorithm for this is stochastic gradient descent (SGD), which incrementally updates its ...
openaire +3 more sources
Advanced Materials, EarlyView.ABSTRACT The accelerating expansion of data‐centric technologies is sharply increasing the energy burden of information storage, placing unprecedented pressure on the efficiency of magnetic switching. Conventional field‐driven reversal, once the foundation of magnetic memory, has become impractical in modern architectures due to its high energy cost ...
Mohammad H. Badarneh +2 more
wiley +1 more source
Stochastic gradient descent with finite samples sizes
, 2016 The minimization of empirical risks over finite sample sizes is an important problem in large-scale machine learning. A variety of algorithms has been proposed in the literature to alleviate the computational burden per iteration at the expense of ...
Ali H. Sayed +7 more
core +2 more sources