The Double-Sided Information Bottleneck Function [PDF]
Entropy, 2022 A double-sided variant of the information bottleneck method is considered. Let (X,Y) be a bivariate source characterized by a joint pmf PXY. The problem is to find two independent channels PU|X and PV|Y (setting the Markovian structure U→X→Y→V), that ...
Michael Dikshtein +2 more
doaj +5 more sources
Multivariate Time Series Information Bottleneck [PDF]
Entropy, 2023 Time series (TS) and multiple time series (MTS) predictions have historically paved the way for distinct families of deep learning models. The temporal dimension, distinguished by its evolutionary sequential aspect, is usually modeled by decomposition ...
Denis Ullmann +2 more
doaj +4 more sources
Partial Information Decomposition: Redundancy as Information Bottleneck [PDF]
EntropyThe partial information decomposition (PID) aims to quantify the amount of redundant information that a set of sources provides about a target. Here, we show that this goal can be formulated as a type of information bottleneck (IB) problem, termed the ...
Artemy Kolchinsky
doaj +5 more sources
On the Difference between the Information Bottleneck and the Deep Information Bottleneck [PDF]
Entropy, 2020 Combining the information bottleneck model with deep learning by replacing mutual information terms with deep neural nets has proven successful in areas ranging from generative modelling to interpreting deep neural networks. In this paper, we revisit the
Aleksander Wieczorek, Volker Roth
doaj +7 more sources
The Supervised Information Bottleneck [PDF]
EntropyThe Information Bottleneck (IB) framework offers a theoretically optimal approach to data modeling, although it is often intractable. Recent efforts have optimized supervised deep neural networks (DNNs) using a variational upper bound on the IB objective,
Nir Z. Weingarten +3 more
doaj +3 more sources
Counterfactual Supervision-Based Information Bottleneck for Out-of-Distribution Generalization [PDF]
Entropy, 2023 Learning invariant (causal) features for out-of-distribution (OOD) generalization have attracted extensive attention recently, and among the proposals, invariant risk minimization (IRM) is a notable solution.
Bin Deng, Kui Jia
doaj +2 more sources
Information Bottleneck Driven Deep Video Compression—IBOpenDVCW [PDF]
EntropyVideo compression remains a challenging task despite significant advancements in end-to-end optimized deep networks for video coding. This study, inspired by information bottleneck (IB) theory, introduces a novel approach that combines IB theory with ...
Timor Leiderman, Yosef Ben Ezra
doaj +2 more sources
Information Bottleneck Analysis by a Conditional Mutual Information Bound [PDF]
Entropy, 2021 Task-nuisance decomposition describes why the information bottleneck loss I(z;x)−βI(z;y) is a suitable objective for supervised learning. The true category y is predicted for input x using latent variables z.
Taro Tezuka, Shizuma Namekawa
doaj +2 more sources
Revisiting Sequential Information Bottleneck: New Implementation and Evaluation [PDF]
Entropy, 2022 We introduce a modern, optimized, and publicly available implementation of the sequential Information Bottleneck clustering algorithm, which strikes a highly competitive balance between clustering quality and speed.
Assaf Toledo, Elad Venezian, Noam Slonim
doaj +2 more sources
Information Bottleneck Signal Processing and Learning to Maximize Relevant Information for Communication Receivers [PDF]
Entropy, 2022 Digital communication receivers extract information about the transmitted data from the received signal in subsequent processing steps, such as synchronization, demodulation and channel decoding.
Jan Lewandowsky +2 more
doaj +2 more sources