Results 21 to 30 of about 255,317 (213)

Model Averaging Estimation Method by Kullback–Leibler Divergence for Multiplicative Error Model

Complexity, 2022
In this paper, we propose the model averaging estimation method for multiplicative error model and construct the corresponding weight choosing criterion based on the Kullback–Leibler divergence with a hyperparameter to avoid the problem of overfitting ...
Wanbo Lu, Wenhui Shi
doaj   +1 more source

On accuracy of PDF divergence estimators and their applicability to representative data sampling [PDF]

, 2011
Generalisation error estimation is an important issue in machine learning. Cross-validation traditionally used for this purpose requires building multiple models and repeating the whole procedure many times in order to produce reliable error estimates ...
Musial, Katarzyna, Gabrys, Bogdan, Katarzyna Musial, Bogdan Gabrys, Musial-Gabrys, Katarzyna, Budka, Marcin, Marcin Budka   +6 more
core   +1 more source

Rényi Relative Entropy from Homogeneous Kullback-Leibler Divergence Lagrangian [PDF]

, 2021
We study the homogeneous extension of the Kullback-Leibler divergence associated to a covariant variational problem on the statistical bundle. We assume a finite sample space.
Goffredo Chirco, Chirco G.
core   +1 more source

Guaranteed Bounds on Information-Theoretic Measures of Univariate Mixtures Using Piecewise Log-Sum-Exp Inequalities

Entropy, 2016
Information-theoretic measures, such as the entropy, the cross-entropy and the Kullback–Leibler divergence between two mixture models, are core primitives in many signal processing tasks.
Frank Nielsen, Ke Sun
doaj   +1 more source

Some bounds for skewed α-Jensen-Shannon divergence

Results in Applied Mathematics, 2019
Based on the skewed Kullback-Leibler divergence introduced in the natural language processing, we derive the upper and lower bounds on the skewed version of the Jensen-Shannon divergence and investigate properties of them.
Takuya Yamano
doaj   +1 more source

Interpreting Kullback-Leibler divergence with the Neyman-Pearson lemma [PDF]

, 2006
Kullback-Leibler divergence and the Neyman-Pearson lemma are two fundamental concepts in statistics. Both are about likelihood ratios: Kullback-Leibler divergence is the expected log-likelihood ratio, and the Neyman-Pearson lemma is about error rates of ...
Copas, John B., Copas, John, Eguchi, Shinto   +2 more
core   +1 more source

Dynamic fine‐tuning layer selection using Kullback–Leibler divergence

Engineering Reports, 2023
The selection of layers in the transfer learning fine‐tuning process ensures a pre‐trained model's accuracy and adaptation in a new target domain. However, the selection process is still manual and without clearly defined criteria. If the wrong layers in
Raphael Ngigi Wanjiku, Lawrence Nderu, Michael Kimwele   +2 more
doaj   +1 more source

Generalizing the Alpha-Divergences and the Oriented Kullback–Leibler Divergences with Quasi-Arithmetic Means

Algorithms, 2022
The family of α-divergences including the oriented forward and reverse Kullback–Leibler divergences is often used in signal processing, pattern recognition, and machine learning, among others.
Frank Nielsen
doaj   +1 more source

On the Jensen–Shannon Symmetrization of Distances Relying on Abstract Means

Entropy, 2019
The Jensen–Shannon divergence is a renowned bounded symmetrization of the unbounded Kullback–Leibler divergence which measures the total Kullback–Leibler divergence to the average mixture distribution.
Frank Nielsen
doaj   +1 more source

Rényi Divergence and Kullback-Leibler Divergence

, 2013
Accepted by IEEE Transactions on Information Theory. To appear.Rényi divergence is related to Rényi entropy much like Kullback-Leibler divergence is related to Shannon's entropy, and comes up in many settings.
Harremoës, Peter, van Erven, Tim
core   +2 more sources