Results 191 to 200 of about 389,136 (225)

The Kullback–Leibler Divergence Between Lattice Gaussian Distributions

Journal of the Indian Institute of Science, 2021
A lattice Gaussian distribution of given mean and covariance matrix is a discrete distribution supported on a lattice maximizing Shannon’s entropy under these mean and covariance constraints.
F. Nielsen
semanticscholar   +1 more source

Kullback–Leibler Divergence-Based Optimal Stealthy Sensor Attack Against Networked Linear Quadratic Gaussian Systems

IEEE Transactions on Cybernetics, 2021
This article concentrates on designing optimal stealthy attack strategies for cyber-physical systems (CPSs) modeled by the linear quadratic Gaussian (LQG) dynamics, where the attacker aims to increase the quadratic cost maximally and keeping a certain ...
Xiu-Xiu Ren, Guang‐Hong Yang
semanticscholar   +1 more source

KLDet: Detecting Tiny Objects in Remote Sensing Images via Kullback–Leibler Divergence

IEEE Transactions on Geoscience and Remote Sensing
Remote sensing images (RSIs) frequently contain quite a few tiny objects with a finite number of pixels to study. The limited spatial information poses a challenge for extracting discriminative features for representing the characteristics of tiny ...
Zhuangzhuang Zhou, Yingying Zhu
semanticscholar   +1 more source

Wasserstein Distance Rivals Kullback-Leibler Divergence for Knowledge Distillation

Neural Information Processing Systems
Since pioneering work of Hinton et al., knowledge distillation based on Kullback-Leibler Divergence (KL-Div) has been predominant, and recently its variants have achieved compelling performance.
Jiaming Lv, Haoyuan Yang, Peihua Li
semanticscholar   +1 more source

Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models

International Conference on Computational Linguistics
Kullback-Leiber divergence has been widely used in Knowledge Distillation (KD) to compress Large Language Models (LLMs). Contrary to prior assertions that reverse Kullback-Leibler (RKL) divergence is mode-seeking and thus preferable over the mean-seeking
Taiqiang Wu, Chaofan Tao, Jiahao Wang, Zhe Zhao, Ngai Wong   +4 more
semanticscholar   +1 more source

Generalized Kullback-Leibler Divergence Loss

arXiv.org
In this paper, we delve deeper into the Kullback-Leibler (KL) Divergence loss and mathematically prove that it is equivalent to the Decoupled Kullback-Leibler (DKL) Divergence loss that consists of (1) a weighted Mean Square Error (wMSE) loss and (2) a ...
Jiequan Cui, Beier Zhu, Qingshan Xu, Zhuotao Tian, Xiaojuan Qi, Bei Yu, H. Zhang, Richang Hong   +7 more
semanticscholar   +1 more source

Algorithms for Nonnegative Matrix Factorization with the Kullback–Leibler Divergence

Journal of Scientific Computing, 2020
Nonnegative matrix factorization (NMF) is a standard linear dimensionality reduction technique for nonnegative data sets. In order to measure the discrepancy between the input data and the low-rank approximation, the Kullback–Leibler (KL) divergence is ...
L. Hien, Nicolas Gillis
semanticscholar   +1 more source

Bivariate Kullback–Leibler divergence

Communications in Statistics - Theory and Methods
. Kullback and Leibler (1951) introduced Kullback–Leibler divergence to measure the disparity between two random variables. In this article, we introduce a bivariate extension of Kullback–Leibler divergence and study its various properties.
Mary Rafflesia Chackochan, P. Sankaran, N. Unnikrishnan Nair   +2 more
semanticscholar   +1 more source

Kullback–Leibler Divergence-Based Fuzzy C-Means Clustering Incorporating Morphological Reconstruction and Wavelet Frames for Image Segmentation

IEEE Transactions on Cybernetics, 2020
In this article, we elaborate on a Kullback–Leibler (KL) divergence-based Fuzzy ${C}$ -Means (FCM) algorithm by incorporating a tight wavelet frame transform and morphological reconstruction (MR).
Cong Wang, W. Pedrycz, Zhiwu Li, Mengchu Zhou   +3 more
semanticscholar   +1 more source

A Distributionally Robust Optimization Model for Unit Commitment Based on Kullback–Leibler Divergence

IEEE Transactions on Power Systems, 2018
This paper proposes a new distance-based distributionally robust unit commitment (DB-DRUC) model via Kullback–Leibler (KL) divergence, considering volatile wind power generation.
Yuwei Chen, Qinglai Guo, Hongbin Sun, Zhengshuo Li, Wenchuan Wu, Zihao Li   +5 more
semanticscholar   +1 more source