The Conditional Entropy Bottleneck [PDF]
Entropy, 2020 Much of the field of Machine Learning exhibits a prominent set of failure modes, including vulnerability to adversarial examples, poor out-of-distribution (OoD) detection, miscalibration, and willingness to memorize random labelings of datasets.
Ian Fischer
doaj +6 more sources
Belavkin–Staszewski Relative Entropy, Conditional Entropy, and Mutual Information [PDF]
Entropy, 2022 Belavkin–Staszewski relative entropy can naturally characterize the effects of the possible noncommutativity of quantum states. In this paper, two new conditional entropy terms and four new mutual information terms are first defined by replacing quantum ...
Yuan Zhai, Bo Yang, Zhengjun Xi
doaj +4 more sources
On Conditional Tsallis Entropy [PDF]
Entropy, 2021 There is no generally accepted definition for conditional Tsallis entropy. The standard definition of (unconditional) Tsallis entropy depends on a parameter α that converges to the Shannon entropy as α approaches 1.
Andreia Teixeira +2 more
doaj +4 more sources
Feature Selection Using Approximate Conditional Entropy Based on Fuzzy Information Granule for Gene Expression Data Classification [PDF]
Frontiers in Genetics, 2021 Classification is widely used in gene expression data analysis. Feature selection is usually performed before classification because of the large number of genes and the small sample size in gene expression data.
Hengyi Zhang
doaj +2 more sources
Conditional Entropy: A Potential Digital Marker for Stress [PDF]
Entropy, 2021 Recent decades have witnessed a substantial progress in the utilization of brain activity for the identification of stress digital markers. In particular, the success of entropic measures for this purpose is very appealing, considering (1) their ...
Soheil Keshmiri
doaj +2 more sources
Conditional Entropy and Location Error in Indoor Localization Using Probabilistic Wi-Fi Fingerprinting [PDF]
Sensors, 2016 Localization systems are increasingly valuable, but their location estimates are only useful when the uncertainty of the estimate is known. This uncertainty is currently calculated as the location error given a ground truth, which is then used as a ...
Rafael Berkvens +2 more
doaj +2 more sources
An Extended Result on the Optimal Estimation Under the Minimum Error Entropy Criterion
Entropy, 2014 The minimum error entropy (MEE) criterion has been successfully used in fields such as parameter estimation, system identification and the supervised machine learning.
Badong Chen +3 more
doaj +4 more sources
Information–Entropy Analysis of Stellar Evolutionary Stages with Application to FS CMa Objects [PDF]
EntropyTheoretical foundations are presented for the application of information–entropy methods from statistical physics to the determination of stellar evolutionary stages. A balance equation involving normalized conditional information and entropy is proposed.
Zeinulla Zhanabaev +2 more
doaj +2 more sources
Negative Conditional Entropy of Post-Selected States [PDF]
Physical Review A, 2014 We define a quantum entropy conditioned on post-selection which has the von Neumann entropy of pure states as a special case. This conditional entropy can take negative values which is consistent with part of a quantum system containing less information ...
Salek, Sina +2 more
core +3 more sources
On the implementation of maximum entropy sampling with unequal probabilities and without replacement [PDF]
MethodsXSampling with maximum entropy offers robustness to statistical inference based on randomization theory. However, there were no comprehensive, practical guides explaining how to implement maximum entropy sampling for finite populations with unequal ...
Philippe Aubry
doaj +2 more sources