Results 11 to 20 of about 9,520,135 (306)
Non-convex Optimization for Machine Learning [PDF]
Foundations and Trends® in Machine Learning, 2017 A vast majority of machine learning algorithms train their models and perform inference by solving optimization problems. In order to capture the learning and prediction problems accurately, structural constraints such as sparsity or low rank are frequently imposed or else the objective itself is designed to be a non-convex function. This is especially
Prateek Jain 0002, Purushottam Kar
openaire +5 more sources
Barrier Algorithms for Constrained Non-Convex Optimization [PDF]
arXiv admin note: text overlap with arXiv:2111 ...
Pavel E. Dvurechensky, Mathias Staudigl
openaire +4 more sources
Recent Theoretical Advances in Non-Convex Optimization
, 2022 Motivated by recent increased interest in optimization algorithms for non-convex optimization in application to training deep neural networks and other optimization problems in data analysis, we give an overview of recent theoretical results on global performance guarantees of optimization algorithms for non-convex optimization. We start with classical
Danilova, Marina +6 more
openaire +4 more sources
Non-convex multi-objective optimization
Optimization Methods and Software, 2017 During several decades, multi-objective optimization is a very active research area. The actuality of this subject stems from real-life applications as well as from its high theoretical importance....
Žilinskas, Antanas +5 more
openaire +2 more sources
Non-Convex Distributed Optimization [PDF]
IEEE Transactions on Automatic Control, 2017 We study distributed non-convex optimization on a time-varying multi-agent network. Each node has access to its own smooth local cost function, and the collective goal is to minimize the sum of these functions. We generalize the results obtained previously to the case of non-convex functions. Under some additional technical assumptions on the gradients
Tatiana Tatarenko, Behrouz Touri
openaire +3 more sources
Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion
CoRR, 2023 We present new algorithms for optimizing non-smooth, non-convex stochastic objectives based on a novel analysis technique. This improves the current best-known complexity for finding a $(δ,ε)$-stationary point from $O(ε^{-4}δ^{-1})$ stochastic gradient queries to $O(ε^{-3}δ^{-1})$, which we also show to be optimal.
Ashok Cutkosky +2 more
openaire +3 more sources
Online Optimization with Predictions and Non-convex Losses [PDF]
2021 55th Annual Conference on Information Sciences and Systems (CISS), 2021 Smoothed online optimization considers an online learning setting where the learner, besides a per-round hitting cost, also incurs a switching cost when changing its decision between rounds. It has received significant attention recently due to its close connections with applications in control and resource allocation. A line of work in this area seeks
Lin, Yiheng, Goel, Gautam, Wierman, Adam
openaire +10 more sources
Replica Exchange for Non-Convex Optimization
J. Mach. Learn. Res., 2020 Gradient descent (GD) is known to converge quickly for convex objective functions, but it can be trapped at local minima. On the other hand, Langevin dynamics (LD) can explore the state space and find global minima, but in order to give accurate estimates, LD needs to run with a small discretization step size and weak stochastic force, which in general
Dong, J, Tong, XT
openaire +4 more sources
Lower bounds for non-convex stochastic optimization
Mathematical Programming, 2022 We lower bound the complexity of finding $ε$-stationary points (with gradient norm at most $ε$) using stochastic first-order methods. In a well-studied model where algorithms access smooth, potentially non-convex functions through queries to an unbiased stochastic gradient oracle with bounded variance, we prove that (in the worst case) any algorithm ...
Yossi Arjevani +5 more
openaire +5 more sources
Evolutionary Gradient Descent for Non-convex Optimization [PDF]
Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, 2021 Non-convex optimization is often involved in artificial intelligence tasks, which may have many saddle points, and is NP-hard to solve. Evolutionary algorithms (EAs) are general-purpose derivative-free optimization algorithms with a good ability to find the global optimum, which can be naturally applied to non-convex optimization. Their performance is,
Ke Xue 0001 +3 more
openaire +1 more source