Results 1 to 10 of about 333,210 (136)
The Double-Sided Information Bottleneck Function [PDF]
Entropy, 2022 A double-sided variant of the information bottleneck method is considered. Let (X,Y) be a bivariate source characterized by a joint pmf PXY. The problem is to find two independent channels PU|X and PV|Y (setting the Markovian structure U→X→Y→V), that ...
Michael Dikshtein +2 more
doaj +5 more sources
Multivariate Time Series Information Bottleneck [PDF]
Entropy, 2023 Time series (TS) and multiple time series (MTS) predictions have historically paved the way for distinct families of deep learning models. The temporal dimension, distinguished by its evolutionary sequential aspect, is usually modeled by decomposition ...
Denis Ullmann +2 more
doaj +4 more sources
Partial Information Decomposition: Redundancy as Information Bottleneck [PDF]
EntropyThe partial information decomposition (PID) aims to quantify the amount of redundant information that a set of sources provides about a target. Here, we show that this goal can be formulated as a type of information bottleneck (IB) problem, termed the ...
Artemy Kolchinsky
doaj +5 more sources
On the Difference between the Information Bottleneck and the Deep Information Bottleneck [PDF]
Entropy, 2020 Combining the information bottleneck model with deep learning by replacing mutual information terms with deep neural nets has proven successful in areas ranging from generative modelling to interpreting deep neural networks. In this paper, we revisit the
Aleksander Wieczorek, Volker Roth
doaj +7 more sources
The Supervised Information Bottleneck [PDF]
EntropyThe Information Bottleneck (IB) framework offers a theoretically optimal approach to data modeling, although it is often intractable. Recent efforts have optimized supervised deep neural networks (DNNs) using a variational upper bound on the IB objective,
Nir Z. Weingarten +3 more
doaj +3 more sources
Nonlinear Information Bottleneck [PDF]
Entropy (Basel), 2019 Information bottleneck (IB) is a technique for extracting information in one random variable $X$ that is relevant for predicting another random variable $Y$.
Kolchinsky A, Tracey B, Wolpert D.
europepmc +4 more sources
Elastic Information Bottleneck
Mathematics, 2022 Information bottleneck is an information-theoretic principle of representation learning that aims to learn a maximally compressed representation that preserves as much information about labels as possible. Under this principle, two different methods have
Yuyan Ni, Yanyan Lan, Ao Liu, Zhiming Ma
doaj +3 more sources
The Convex Information Bottleneck Lagrangian [PDF]
Entropy, 2020 The information bottleneck (IB) problem tackles the issue of obtaining relevant compressed representations T of some random variable X for the task of predicting Y.
Borja Rodríguez Gálvez +2 more
doaj +5 more sources
Counterfactual Supervision-Based Information Bottleneck for Out-of-Distribution Generalization [PDF]
Entropy, 2023 Learning invariant (causal) features for out-of-distribution (OOD) generalization have attracted extensive attention recently, and among the proposals, invariant risk minimization (IRM) is a notable solution.
Bin Deng, Kui Jia
doaj +2 more sources
Information Bottleneck Driven Deep Video Compression—IBOpenDVCW [PDF]
EntropyVideo compression remains a challenging task despite significant advancements in end-to-end optimized deep networks for video coding. This study, inspired by information bottleneck (IB) theory, introduces a novel approach that combines IB theory with ...
Timor Leiderman, Yosef Ben Ezra
doaj +2 more sources