Results 11 to 20 of about 24,619 (260)
Online Optimization with Predictions and Non-convex Losses [PDF]
2021 55th Annual Conference on Information Sciences and Systems (CISS), 2021 Smoothed online optimization considers an online learning setting where the learner, besides a per-round hitting cost, also incurs a switching cost when changing its decision between rounds. It has received significant attention recently due to its close connections with applications in control and resource allocation. A line of work in this area seeks
Lin, Yiheng, Goel, Gautam, Wierman, Adam
openaire +8 more sources
Evolutionary Gradient Descent for Non-convex Optimization [PDF]
Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, 2021 Non-convex optimization is often involved in artificial intelligence tasks, which may have many saddle points, and is NP-hard to solve. Evolutionary algorithms (EAs) are general-purpose derivative-free optimization algorithms with a good ability to find the global optimum, which can be naturally applied to non-convex optimization. Their performance is,
Ke Xue 0001 +3 more
openaire +1 more source
Replica Exchange for Non-Convex Optimization
J. Mach. Learn. Res., 2020 Gradient descent (GD) is known to converge quickly for convex objective functions, but it can be trapped at local minima. On the other hand, Langevin dynamics (LD) can explore the state space and find global minima, but in order to give accurate estimates, LD needs to run with a small discretization step size and weak stochastic force, which in general
Dong, J, Tong, XT
openaire +4 more sources
Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion
CoRR, 2023 We present new algorithms for optimizing non-smooth, non-convex stochastic objectives based on a novel analysis technique. This improves the current best-known complexity for finding a $(δ,ε)$-stationary point from $O(ε^{-4}δ^{-1})$ stochastic gradient queries to $O(ε^{-3}δ^{-1})$, which we also show to be optimal.
Ashok Cutkosky +2 more
openaire +3 more sources
Lower bounds for non-convex stochastic optimization
Mathematical Programming, 2022 We lower bound the complexity of finding $ε$-stationary points (with gradient norm at most $ε$) using stochastic first-order methods. In a well-studied model where algorithms access smooth, potentially non-convex functions through queries to an unbiased stochastic gradient oracle with bounded variance, we prove that (in the worst case) any algorithm ...
Yossi Arjevani +5 more
openaire +3 more sources
Non-convex Optimization for Machine Learning [PDF]
Foundations and Trends® in Machine Learning, 2017 A vast majority of machine learning algorithms train their models and perform inference by solving optimization problems. In order to capture the learning and prediction problems accurately, structural constraints such as sparsity or low rank are frequently imposed or else the objective itself is designed to be a non-convex function. This is especially
Prateek Jain 0002, Purushottam Kar
openaire +2 more sources
Localization and Approximations for Distributed Non-convex Optimization
Journal of Optimization Theory and Applications, 2023 Distributed optimization has many applications, in communication networks, sensor networks, signal processing, machine learning, and artificial intelligence. Methods for distributed convex optimization are widely investigated, while those for non-convex objectives are not well understood.
Hsu Kao, Vijay G. Subramanian
openaire +2 more sources
Adaptive Strategies in Non-convex Optimization
CoRR, 2023 An algorithm is said to be adaptive to a certain parameter (of the problem) if it does not need a priori knowledge of such a parameter but performs competitively to those that know it. This dissertation presents our work on adaptive algorithms in following scenarios: 1.
openaire +2 more sources
Online non-convex optimization with imperfect feedback
CoRR, 2020 We consider the problem of online learning with non-convex losses. In terms of feedback, we assume that the learner observes - or otherwise constructs - an inexact model for the loss function encountered at each stage, and we propose a mixed-strategy learning policy based on dual averaging.
Héliou, Amélie +3 more
openaire +4 more sources
ON OPTIMUM DESIGN OF FRAME STRUCTURES
Acta Polytechnica CTU Proceedings, 2020 Optimization of frame structures is formulated as a non-convex optimization problem, which is currently solved to local optimality. In this contribution, we investigate four optimization approaches: (i) general non-linear optimization, (ii) optimality ...
Marek Tyburec +3 more
doaj +1 more source