Results 1 to 10 of about 90,245 (141)
The Supervised Information Bottleneck [PDF]
EntropyThe Information Bottleneck (IB) framework offers a theoretically optimal approach to data modeling, although it is often intractable. Recent efforts have optimized supervised deep neural networks (DNNs) using a variational upper bound on the IB objective,
Nir Z. Weingarten +3 more
doaj +4 more sources
On the Difference between the Information Bottleneck and the Deep Information Bottleneck [PDF]
Entropy, 2020 Combining the information bottleneck model with deep learning by replacing mutual information terms with deep neural nets has proven successful in areas ranging from generative modelling to interpreting deep neural networks. In this paper, we revisit the
Aleksander Wieczorek, Volker Roth
doaj +8 more sources
The Double-Sided Information Bottleneck Function [PDF]
Entropy, 2022 A double-sided variant of the information bottleneck method is considered. Let (X,Y) be a bivariate source characterized by a joint pmf PXY. The problem is to find two independent channels PU|X and PV|Y (setting the Markovian structure U→X→Y→V), that ...
Michael Dikshtein +2 more
doaj +7 more sources
Partial Information Decomposition: Redundancy as Information Bottleneck [PDF]
EntropyThe partial information decomposition (PID) aims to quantify the amount of redundant information that a set of sources provides about a target. Here, we show that this goal can be formulated as a type of information bottleneck (IB) problem, termed the ...
Artemy Kolchinsky
doaj +5 more sources
Nonlinear Information Bottleneck [PDF]
Entropy, 2019 Information bottleneck (IB) is a technique for extracting information in one random variable X that is relevant for predicting another random variable Y. IB works by encoding X in a compressed “bottleneck” random variable M from which Y can be accurately decoded. However, finding the optimal bottleneck variable involves a difficult optimization problem,
Artemy Kolchinsky +2 more
exaly +5 more sources
Elastic Information Bottleneck
Mathematics, 2022 Information bottleneck is an information-theoretic principle of representation learning that aims to learn a maximally compressed representation that preserves as much information about labels as possible. Under this principle, two different methods have
Yuyan Ni, Yanyan Lan, Ao Liu, Zhiming Ma
doaj +3 more sources
Learnability for the Information Bottleneck [PDF]
Entropy, 2019 The Information Bottleneck (IB) method provides an insightful and principled approach for balancing compression and prediction for representation learning. The IB objective I ( X ; Z ) - β I ( Y ; Z ) employs a Lagrange multiplier β
Tailin Wu +3 more
doaj +5 more sources
Multivariate Time Series Information Bottleneck [PDF]
Entropy, 2023 Time series (TS) and multiple time series (MTS) predictions have historically paved the way for distinct families of deep learning models. The temporal dimension, distinguished by its evolutionary sequential aspect, is usually modeled by decomposition ...
Denis Ullmann +2 more
doaj +2 more sources
The Convex Information Bottleneck Lagrangian [PDF]
Entropy, 2020 The information bottleneck (IB) problem tackles the issue of obtaining relevant compressed representations T of some random variable X for the task of predicting Y.
Borja Rodríguez Gálvez +2 more
doaj +5 more sources
Counterfactual Supervision-Based Information Bottleneck for Out-of-Distribution Generalization [PDF]
Entropy, 2023 Learning invariant (causal) features for out-of-distribution (OOD) generalization have attracted extensive attention recently, and among the proposals, invariant risk minimization (IRM) is a notable solution.
Bin Deng, Kui Jia
doaj +2 more sources