Saddle-to-saddle dynamics in diagonal linear networks* [PDF]
Journal of Statistical Mechanics: Theory and Experiment, 2023 Abstract In this paper we fully describe the trajectory of gradient flow over two-layer diagonal linear networks for the regression setting in the limit of vanishing initialisation. We show that the limiting flow successively jumps from a saddle of the training loss to another until reaching the minimum
Pesme, Scott, Flammarion, Nicolas
openaire +4 more sources
SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics [PDF]
Annual Conference Computational Learning Theory, 2023 We investigate the time complexity of SGD learning on fully-connected neural networks with isotropic data. We put forward a complexity measure -- the leap -- which measures how"hierarchical"target functions are.
E. Abbe +2 more
semanticscholar +1 more source
Electronic instabilities of kagome metals: Saddle points and Landau theory [PDF]
Physical review B, 2021 We study electronic instabilities of a kagomé metal with a Fermi energy close to saddle points at the hexagonal Brillouin zone face centers.
Ta-Ryeong Park, Mengxing Ye, L. Balents
semanticscholar +1 more source
Spread complexity in saddle-dominated scrambling [PDF]
Journal of High Energy Physics, 2023 Recently, the concept of spread complexity, Krylov complexity for states, has been introduced as a measure of the complexity and chaoticity of quantum systems.
Kyoung-Bum Huh +2 more
semanticscholar +1 more source
Escaping Saddle Points for Effective Generalization on Class-Imbalanced Data [PDF]
Neural Information Processing Systems, 2022 Real-world datasets exhibit imbalances of varying types and degrees. Several techniques based on re-weighting and margin adjustment of loss are often used to enhance the performance of neural networks, particularly on minority classes.
Harsh Rangwani +3 more
semanticscholar +1 more source
Searching the solution landscape by generalized high-index saddle dynamics [PDF]
Science China Mathematics, 2020 We introduce a generalized numerical algorithm to construct the solution landscape, which is a pathway map consisting of all stationary points and their connections.
Jianyuan Yin, Bing Yu, Lei Zhang
semanticscholar +1 more source
Leather and Sewing Workshops on the Trade and Craft District of Medieval Bolgar
Археология евразийских степей, 2023 The authors consider results of generalization and analysis of materials related to the treatment of raw materials and manufacturing of leatherware, originating from the territory of the Bolgar fortified settlement.
Denis Yu. Badeev, Renat R. Valiev
doaj +1 more source
Distributed saddle point problems: lower bounds, near-optimal and robust algorithms [PDF]
Optim. Methods Softw., 2020 This paper focuses on the distributed optimization of stochastic saddle point problems. The first part of the paper is devoted to lower bounds for the centralized and decentralized distributed methods for smooth (strongly) convex-(strongly) concave ...
A. Beznosikov, V. Samokhin, A. Gasnikov
semanticscholar +1 more source
Constrained High-Index Saddle Dynamics for the Solution Landscape with Equality Constraints [PDF]
Journal of Scientific Computing, 2020 We propose a constrained high-index saddle dynamics (CHiSD) method to search for index-k saddle points of an energy functional subject to equality constraints.
Jianyuan Yin, Zhen Huang, Lei Zhang
semanticscholar +1 more source
Dirac cone, flat band and saddle point in kagome magnet YMn6Sn6
Nature Communications, 2020 Kagome-lattices of 3d-transition metals hosting Weyl/Dirac fermions and topological flat bands exhibit non-trivial topological characters and novel quantum phases, such as the anomalous Hall effect and fractional quantum Hall effect.
Man Li +11 more
semanticscholar +1 more source