Results 11 to 20 of about 26,185 (264)
Non-Convex Distributed Optimization [PDF]
IEEE Transactions on Automatic Control, 2017 We study distributed non-convex optimization on a time-varying multi-agent network. Each node has access to its own smooth local cost function, and the collective goal is to minimize the sum of these functions. We generalize the results obtained previously to the case of non-convex functions. Under some additional technical assumptions on the gradients
Tatiana Tatarenko, Behrouz Touri
openaire +3 more sources
Online Optimization with Predictions and Non-convex Losses [PDF]
2021 55th Annual Conference on Information Sciences and Systems (CISS), 2021 Smoothed online optimization considers an online learning setting where the learner, besides a per-round hitting cost, also incurs a switching cost when changing its decision between rounds. It has received significant attention recently due to its close connections with applications in control and resource allocation. A line of work in this area seeks
Lin, Yiheng, Goel, Gautam, Wierman, Adam
openaire +10 more sources
Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion
CoRR, 2023 We present new algorithms for optimizing non-smooth, non-convex stochastic objectives based on a novel analysis technique. This improves the current best-known complexity for finding a $(δ,ε)$-stationary point from $O(ε^{-4}δ^{-1})$ stochastic gradient queries to $O(ε^{-3}δ^{-1})$, which we also show to be optimal.
Ashok Cutkosky +2 more
openaire +3 more sources
Replica Exchange for Non-Convex Optimization
J. Mach. Learn. Res., 2020 Gradient descent (GD) is known to converge quickly for convex objective functions, but it can be trapped at local minima. On the other hand, Langevin dynamics (LD) can explore the state space and find global minima, but in order to give accurate estimates, LD needs to run with a small discretization step size and weak stochastic force, which in general
Dong, J, Tong, XT
openaire +4 more sources
Non-convex Optimization for Machine Learning [PDF]
Foundations and Trends® in Machine Learning, 2017 A vast majority of machine learning algorithms train their models and perform inference by solving optimization problems. In order to capture the learning and prediction problems accurately, structural constraints such as sparsity or low rank are frequently imposed or else the objective itself is designed to be a non-convex function. This is especially
Prateek Jain 0002, Purushottam Kar
openaire +4 more sources
Lower bounds for non-convex stochastic optimization
Mathematical Programming, 2022 We lower bound the complexity of finding $ε$-stationary points (with gradient norm at most $ε$) using stochastic first-order methods. In a well-studied model where algorithms access smooth, potentially non-convex functions through queries to an unbiased stochastic gradient oracle with bounded variance, we prove that (in the worst case) any algorithm ...
Yossi Arjevani +5 more
openaire +5 more sources
Localization and Approximations for Distributed Non-convex Optimization
Journal of Optimization Theory and Applications, 2023 Distributed optimization has many applications, in communication networks, sensor networks, signal processing, machine learning, and artificial intelligence. Methods for distributed convex optimization are widely investigated, while those for non-convex objectives are not well understood.
Hsu Kao, Vijay G. Subramanian
openaire +2 more sources
Evolutionary Gradient Descent for Non-convex Optimization [PDF]
Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, 2021 Non-convex optimization is often involved in artificial intelligence tasks, which may have many saddle points, and is NP-hard to solve. Evolutionary algorithms (EAs) are general-purpose derivative-free optimization algorithms with a good ability to find the global optimum, which can be naturally applied to non-convex optimization. Their performance is,
Ke Xue 0001 +3 more
openaire +1 more source
Adaptive Strategies in Non-convex Optimization
CoRR, 2023 An algorithm is said to be adaptive to a certain parameter (of the problem) if it does not need a priori knowledge of such a parameter but performs competitively to those that know it. This dissertation presents our work on adaptive algorithms in following scenarios: 1.
openaire +3 more sources
Modified projected Newton scheme for non-convex function with simple constraints [PDF]
Yugoslav Journal of Operations Research, 2021 In this paper, a descent line search scheme is proposed to find a local minimum point of a non-convex optimization problem with simple constraints. The idea ensures that the scheme escapes the saddle points and finally settles for a local minimum point ...
Chakraborty Suvra Kanti +1 more
doaj +1 more source