Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers [PDF]
International Conference on Machine Learning, 2022 Vision transformers using self-attention or its proposed alternatives have demonstrated promising results in many image related tasks. However, the underpinning inductive bias of attention is not well understood.
Arda Sahiner +5 more
semanticscholar +1 more source
Sparse Regularization via Convex Analysis [PDF]
IEEE Transactions on Signal Processing, 2017 Sparse approximate solutions to linear equations are classically obtained via L1 norm regularized least squares, but this method often underestimates the true solution.
I. Selesnick
semanticscholar +1 more source
A Unified and Refined Convergence Analysis for Non-Convex Decentralized Learning [PDF]
IEEE Transactions on Signal Processing, 2021 We study the consensus decentralized optimization problem where the objective function is the average of $n$ agents private non-convex cost functions; moreover, the agents can only communicate to their neighbors on a given network topology.
Sulaiman A. Alghunaim, Kun Yuan
semanticscholar +1 more source
On the sufficiency of K-positivity for truncated compactly supported generalized moment problems
Examples and Counterexamples, 2021 Given a compact set K and a finite set of continuous basis functions, the truncated generalized K-moment problem asks for a characterization of all sequences that can be obtained as moments, with respect to the basis functions, of some nonnegative ...
Axel Ringh
doaj +1 more source
Compositions and Averages of Two Resolvents: Relative Geometry of Fixed Points Sets and a Partial Answer to a Question by C. Byrne [PDF]
, 2010 We show that the set of fixed points of the average of two resolvents can be found from the set of fixed points for compositions of two resolvents associated with scaled monotone operators.
Bauschke, Heinz H., Wang, Xianfu
core +1 more source
Variational methods in the presence of symmetry
Advances in Nonlinear Analysis, 2013 The purpose of this paper is to survey and to provide a unified framework to connect a diverse group of results, currently scattered in the literature, that can be usefully viewed as consequences of applying variational methods to problems involving ...
Borwein Jonathan M., Zhu Qiji J.
doaj +1 more source
General optimal euclidean Sobolev and Gagliardo-Nirenberg inequalities
Anais da Academia Brasileira de Ciências, 2005 We prove general optimal euclidean Sobolev and Gagliardo-Nirenberg inequalities by using mass transportation and convex analysis results. Explicit extremals and the computation of some optimal constants are also provided.
Jurandir Ceccon, Marcos Montenegro
doaj +1 more source
Quantitative Stability of Linear Infinite Inequality Systems under Block Perturbations with Applications to Convex Systems [PDF]
, 2011 The original motivation for this paper was to provide an efficient quantitative analysis of convex infinite (or semi-infinite) inequality systems whose decision variables run over general infinite-dimensional (resp.
AD Ioffe +22 more
core +5 more sources
AN EFFICIENT ALGORITHM FOR THE CONVEX HULL OF PLANAR SCATTERED POINT SET [PDF]
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2012 Computing the convex hull of a point set is requirement in the GIS applications. This paper studies on the problem of minimum convex hull and presents an improved algorithm for the minimum convex hull of planar scattered point set.
Z. Fu, Y. Lu
doaj +1 more source
Sensitivity Analysis for Mirror-Stratifiable Convex Functions [PDF]
, 2018 This paper provides a set of sensitivity analysis and activity identification results for a class of convex functions with a strong geometric structure, that we coined "mirror-stratifiable".
Fadili, Jalal +2 more
core +5 more sources