Results 11 to 20 of about 15,770,233 (254)
Generalized Mutual Information [PDF]
Stats, 2020 Mutual information is one of the essential building blocks of information theory. It is however only finitely defined for distributions in a subclass of the general class of all distributions on a joint alphabet.
Zhiyi Zhang
doaj +3 more sources
An Axiomatic Characterization of Mutual Information
Entropy, 2023 We characterize mutual information as the unique map on ordered pairs of discrete random variables satisfying a set of axioms similar to those of Faddeev’s characterization of the Shannon entropy.
James Fullwood
doaj +5 more sources
Mutual information bounded by Fisher information
Physical Review ResearchWe derive a general upper bound to mutual information in terms of the Fisher information. The bound may be further used to derive a lower bound for the Bayesian quadratic cost. These two provide alternatives to other inequalities in the literature (e.g.,
Wojciech Górecki +3 more
doaj +3 more sources
Mutual information rate and bounds for it. [PDF]
PLoS ONE, 2012 The amount of information exchanged per unit of time between two nodes in a dynamical network or between two data sets is a powerful concept for analysing complex systems.
Murilo S Baptista +5 more
doaj +8 more sources
Error Exponents and α-Mutual Information
Entropy, 2021 Over the last six decades, the representation of error exponent functions for data transmission through noisy channels at rates below capacity has seen three distinct approaches: (1) Through Gallager’s E0 functions (with and without cost constraints); (2)
Sergio Verdú
doaj +1 more source
Finite Sample Based Mutual Information
IEEE Access, 2021 Mutual information is a popular metric in machine learning. In case of a discrete target variable and a continuous feature variable the mutual information can be calculated as a sum-integral of weighted log likelihood ratio of joint and marginal density ...
Khairan Rajab, Firuz Kamalov
doaj +1 more source
The kernel mutual information [PDF]
2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)., 2004 We introduce a new contrast function, the kernel mutual information (KMI), to measure the degree of independence of continuous random variables. This contrast function provides an approximate upper bound on the mutual information, as measured near independence, and is based on a kernel density estimate of the mutual information between a discretised ...
Gretton, A., Herbrich, R., Smola, A.
openaire +4 more sources
Estimating mutual information [PDF]
Physical Review E, 2004 We present two classes of improved estimators for mutual information $M(X,Y)$, from samples of random points distributed according to some joint probability density $μ(x,y)$. In contrast to conventional estimators based on binnings, they are based on entropy estimates from $k$-nearest neighbour distances.
Kraskov, A. +2 more
openaire +4 more sources
Mutual Information and Multi-Agent Systems
Entropy, 2022 We consider the use of Shannon information theory, and its various entropic terms to aid in reaching optimal decisions that should be made in a multi-agent/Team scenario.
Ira S. Moskowitz +2 more
doaj +1 more source
Mutual information for fermionic systems
Physical Review Research, 2022 We study the behavior of the mutual information (MI) in various quadratic fermionic chains, with and without pairing terms and both with short- and long-range hoppings.
Luca Lepori +3 more
doaj +1 more source