The Supervised Information Bottleneck [PDF]
EntropyThe Information Bottleneck (IB) framework offers a theoretically optimal approach to data modeling, although it is often intractable. Recent efforts have optimized supervised deep neural networks (DNNs) using a variational upper bound on the IB objective,
Nir Z. Weingarten +3 more
doaj +5 more sources
Partial Information Decomposition: Redundancy as Information Bottleneck [PDF]
EntropyThe partial information decomposition (PID) aims to quantify the amount of redundant information that a set of sources provides about a target. Here, we show that this goal can be formulated as a type of information bottleneck (IB) problem, termed the ...
Artemy Kolchinsky
doaj +8 more sources
Multivariate Time Series Information Bottleneck [PDF]
Entropy, 2023 Time series (TS) and multiple time series (MTS) predictions have historically paved the way for distinct families of deep learning models. The temporal dimension, distinguished by its evolutionary sequential aspect, is usually modeled by decomposition ...
Denis Ullmann +2 more
doaj +3 more sources
The Double-Sided Information Bottleneck Function [PDF]
Entropy, 2022 A double-sided variant of the information bottleneck method is considered. Let (X,Y) be a bivariate source characterized by a joint pmf PXY. The problem is to find two independent channels PU|X and PV|Y (setting the Markovian structure U→X→Y→V), that ...
Michael Dikshtein +2 more
doaj +9 more sources
Counterfactual Supervision-Based Information Bottleneck for Out-of-Distribution Generalization [PDF]
Entropy, 2023 Learning invariant (causal) features for out-of-distribution (OOD) generalization have attracted extensive attention recently, and among the proposals, invariant risk minimization (IRM) is a notable solution.
Bin Deng, Kui Jia
doaj +2 more sources
Information Bottleneck Driven Deep Video Compression—IBOpenDVCW [PDF]
EntropyVideo compression remains a challenging task despite significant advancements in end-to-end optimized deep networks for video coding. This study, inspired by information bottleneck (IB) theory, introduces a novel approach that combines IB theory with ...
Timor Leiderman, Yosef Ben Ezra
doaj +2 more sources
Curiosity-Driven Exploration with Information Bottleneck Representations and Matrix-Based Mutual Information [PDF]
EntropyCuriosity empowers humans to ask questions about the world and explore it without relying on extrinsic, encouraging rewards such as money. To investigate how this mechanism drives exploration, we implement a curiosity-based approach and test it in a ...
Zhaoxu Meng, Yong Cui
doaj +2 more sources
Revisiting Sequential Information Bottleneck: New Implementation and Evaluation [PDF]
Entropy, 2022 We introduce a modern, optimized, and publicly available implementation of the sequential Information Bottleneck clustering algorithm, which strikes a highly competitive balance between clustering quality and speed.
Assaf Toledo, Elad Venezian, Noam Slonim
doaj +2 more sources
Exact and Soft Successive Refinement of the Information Bottleneck [PDF]
Entropy, 2023 The information bottleneck (IB) framework formalises the essential requirement for efficient information processing systems to achieve an optimal balance between the complexity of their representation and the amount of information extracted about ...
Hippolyte Charvin +2 more
doaj +2 more sources
Image-Based Ship Detection Using Deep Variational Information Bottleneck [PDF]
Sensors, 2023 Image-based ship detection is a critical function in maritime security. However, lacking high-quality training datasets makes it challenging to train a robust supervision deep learning model.
Duc-Dat Ngo +4 more
doaj +2 more sources