Equivalence of Informations Characterizes Bregman Divergences [PDF]
EntropyBregman divergences form a class of distance-like comparison functions which plays fundamental roles in optimization, statistics, and information theory.
Philip S Chodrow
exaly +4 more sources
Block-Active ADMM to Minimize NMF with Bregman Divergences [PDF]
Sensors, 2023 Over the last ten years, there has been a significant interest in employing nonnegative matrix factorization (NMF) to reduce dimensionality to enable a more efficient clustering analysis in machine learning.
Xinyao Li, Akhilesh Tyagi
doaj +2 more sources
Statistical Divergences between Densities of Truncated Exponential Families with Nested Supports: Duo Bregman and Duo Jensen Divergences [PDF]
Entropy, 2022 By calculating the Kullback–Leibler divergence between two probability measures belonging to different exponential families dominated by the same measure, we obtain a formula that generalizes the ordinary Fenchel–Young divergence.
Frank Nielsen
doaj +2 more sources
Symplectic Bregman Divergences [PDF]
EntropyWe present a generalization of Bregman divergences in finite-dimensional symplectic vector spaces that we term symplectic Bregman divergences. Symplectic Bregman divergences are derived from a symplectic generalization of the Fenchel–Young inequality ...
Frank Nielsen
doaj +2 more sources
On Voronoi Diagrams on the Information-Geometric Cauchy Manifolds [PDF]
Entropy, 2020 We study the Voronoi diagrams of a finite set of Cauchy distributions and their dual complexes from the viewpoint of information geometry by considering the Fisher-Rao distance, the Kullback-Leibler divergence, the chi square divergence, and a flat ...
Frank Nielsen
doaj +2 more sources
Divergences Induced by the Cumulant and Partition Functions of Exponential Families and Their Deformations Induced by Comparative Convexity [PDF]
EntropyExponential families are statistical models which are the workhorses in statistics, information theory, and machine learning, among others. An exponential family can either be normalized subtractively by its cumulant or free energy function, or ...
Frank Nielsen
doaj +2 more sources
On a Generalization of the Jensen–Shannon Divergence and the Jensen–Shannon Centroid [PDF]
Entropy, 2020 The Jensen−Shannon divergence is a renown bounded symmetrization of the Kullback−Leibler divergence which does not require probability densities to have matching supports. In this paper, we introduce a vector-skew generalization of the scalar
Frank Nielsen
doaj +2 more sources
Understanding Higher-Order Interactions in Information Space [PDF]
EntropyMethods used in topological data analysis naturally capture higher-order interactions in point cloud data embedded in a metric space. This methodology was recently extended to data living in an information space, by which we mean a space measured with an
Herbert Edelsbrunner +2 more
doaj +2 more sources
Mirror Descent and Exponentiated Gradient Algorithms Using Trace-Form Entropies [PDF]
EntropyThis paper introduces a broad class of Mirror Descent (MD) and Generalized Exponentiated Gradient (GEG) algorithms derived from trace-form entropies defined via deformed logarithms. Leveraging these generalized entropies yields MD and GEG algorithms with
Andrzej Cichocki +3 more
doaj +2 more sources
Generalized Legendre Transforms Have Roots in Information Geometry [PDF]
EntropyArtstein-Avidan and Milman [Annals of mathematics (2009), (169):661–674] characterized invertible reverse-ordering transforms in the space of lower, semi-continuous, extended, real-valued convex functions as affine deformations of the ordinary Legendre ...
Frank Nielsen
doaj +2 more sources