Revisiting Sequential Information Bottleneck: New Implementation and Evaluation [PDF]
Entropy, 2022 We introduce a modern, optimized, and publicly available implementation of the sequential Information Bottleneck clustering algorithm, which strikes a highly competitive balance between clustering quality and speed.
Assaf Toledo, Elad Venezian, Noam Slonim
doaj +2 more sources
Information Bottleneck Analysis by a Conditional Mutual Information Bound [PDF]
Entropy, 2021 Task-nuisance decomposition describes why the information bottleneck loss I(z;x)−βI(z;y) is a suitable objective for supervised learning. The true category y is predicted for input x using latent variables z.
Taro Tezuka, Shizuma Namekawa
doaj +2 more sources
Information Bottleneck Signal Processing and Learning to Maximize Relevant Information for Communication Receivers [PDF]
Entropy, 2022 Digital communication receivers extract information about the transmitted data from the received signal in subsequent processing steps, such as synchronization, demodulation and channel decoding.
Jan Lewandowsky +2 more
doaj +2 more sources
Image-Based Ship Detection Using Deep Variational Information Bottleneck [PDF]
Sensors, 2023 Image-based ship detection is a critical function in maritime security. However, lacking high-quality training datasets makes it challenging to train a robust supervision deep learning model.
Duc-Dat Ngo +4 more
doaj +2 more sources
The information bottleneck method [PDF]
, 1999 We define the relevant information in a signal $x\in X$ as being the information that this signal provides about another signal $y\in \Y$. Examples include the information that face images provide about the names of the people portrayed, or the ...
Bialek, William +2 more
core +7 more sources
Optimal Kullback-Leibler Aggregation via Information Bottleneck [PDF]
IEEE Transactions on Automatic Control, 2015 In this paper, we present a method for reducing a regular, discrete-time Markov chain (DTMC) to another DTMC with a given, typically much smaller number of states.
Geiger, Bernhard C. +3 more
core +4 more sources
The Information Bottleneck's Ordinary Differential Equation: First-Order Root Tracking for the Information Bottleneck. [PDF]
Entropy (Basel), 2023 The Information Bottleneck (IB) is a method of lossy compression of relevant information. Its rate-distortion (RD) curve describes the fundamental tradeoff between input compression and the preservation of relevant information embedded in the input. However, it conceals the underlying dynamics of optimal input encodings.
Agmon S.
europepmc +4 more sources
Pareto-Optimal Clustering with the Primal Deterministic Information Bottleneck [PDF]
Entropy, 2022 At the heart of both lossy compression and clustering is a trade-off between the fidelity and size of the learned representation. Our goal is to map out and study the Pareto frontier that quantifies this trade-off.
Andrew K. Tan +2 more
doaj +2 more sources
Information Bottleneck Theory Based Exploration of Cascade Learning [PDF]
Entropy, 2021 In solving challenging pattern recognition problems, deep neural networks have shown excellent performance by forming powerful mappings between inputs and targets, learning representations (features) and making subsequent predictions.
Xin Du +2 more
doaj +2 more sources
On Neural Networks Fitting, Compression, and Generalization Behavior via Information-Bottleneck-like Approaches [PDF]
Entropy, 2023 It is well-known that a neural network learning process—along with its connections to fitting, compression, and generalization—is not yet well understood.
Zhaoyan Lyu +2 more
doaj +2 more sources