Results 21 to 30 of about 255,284 (162)
Decoupled Kullback-Leibler Divergence Loss
Advances in Neural Information Processing Systems 37In this paper, we delve deeper into the Kullback-Leibler (KL) Divergence loss and mathematically prove that it is equivalent to the Decoupled Kullback-Leibler (DKL) Divergence loss that consists of 1) a weighted Mean Square Error (wMSE) loss and 2) a Cross-Entropy loss incorporating soft labels. Thanks to the decomposed formulation of DKL loss, we have
Jiequan Cui +5 more
openaire +4 more sources
Kullback-Leibler divergence and the Pareto-Exponential approximation. [PDF]
Springerplus, 2016 Recent radar research interests in the Pareto distribution as a model for X-band maritime surveillance radar clutter returns have resulted in analysis of the asymptotic behaviour of this clutter model. In particular, it is of interest to understand when the Pareto distribution is well approximated by an Exponential distribution.
Weinberg GV.
europepmc +4 more sources
Bounds for Kullback-Leibler divergence
Electronic Journal of Differential Equations, 2016 Entropy, conditional entropy and mutual information for discrete-valued random variables play important roles in the information theory. The purpose of this paper is to present new bounds for relative entropy $D(p||q)$ of two probability distributions
Pantelimon G. Popescu +3 more
doaj +2 more sources
Generalization of the Kullback–Leibler divergence in the Tsallis statistics
Journal of Mathematical Analysis and Applications, 2016 zbMATH Open Web Interface contents unavailable due to conflicting licenses.
Liu Hong, Juntao Huang, Wen-An Yong
exaly +3 more sources
Entropy, 2022 By calculating the Kullback–Leibler divergence between two probability measures belonging to different exponential families dominated by the same measure, we obtain a formula that generalizes the ordinary Fenchel–Young divergence.
Frank Nielsen
doaj +1 more source
On the Interventional Kullback-Leibler Divergence
CoRR, 2023 Modern machine learning approaches excel in static settings where a large amount of i.i.d. training data are available for a given task. In a dynamic environment, though, an intelligent agent needs to be able to transfer knowledge and re-use learned components across domains. It has been argued that this may be possible through causal models, aiming to
Wildberger, J. ; https://orcid.org/0000-0002-3433-5920 +3 more
openaire +4 more sources
Chained Kullback-Leibler divergences [PDF]
2016 IEEE International Symposium on Information Theory (ISIT), 2016 We define and characterize the "chained" Kullback-Leibler divergence min w D(p‖w) + D(w‖q) minimized over all intermediate distributions w and the analogous k-fold chained K-L divergence min D(p‖wk-1) + … + D(w2‖w1) + D(w1‖q) minimized over the entire path (w1,…,wk-1).
Dmitri S. Pavlichin, Tsachy Weissman
openaire +2 more sources
Statistical and Geometrical Properties of Regularized Kernel Kullback-Leibler Divergence
International audienceIn this paper, we study the statistical and geometrical properties of the Kullback-Leibler divergence with kernel covariance operators (KKL) introduced by Bach [2022].
Bach, Francis +2 more
core +10 more sources
Parameter Estimation Based on Cumulative Kullback–Leibler Divergence
Revstat Statistical Journal, 2021 In this paper, we propose some estimators for the parameters of a statistical model based on Kullback–Leibler divergence of the survival function in continuous setting and apply it to type I censored data.
Yaser Mehrali , Majid Asadi
doaj +1 more source
On Voronoi Diagrams on the Information-Geometric Cauchy Manifolds
Entropy, 2020 We study the Voronoi diagrams of a finite set of Cauchy distributions and their dual complexes from the viewpoint of information geometry by considering the Fisher-Rao distance, the Kullback-Leibler divergence, the chi square divergence, and a flat ...
Frank Nielsen
doaj +1 more source