Results 11 to 20 of about 15,576,316 (273)
On the Difference between the Information Bottleneck and the Deep Information Bottleneck [PDF]
Entropy, 2020 Combining the information bottleneck model with deep learning by replacing mutual information terms with deep neural nets has proven successful in areas ranging from generative modelling to interpreting deep neural networks. In this paper, we revisit the
Aleksander Wieczorek, Volker Roth
doaj +7 more sources
Elastic Information Bottleneck
Mathematics, 2022 Information bottleneck is an information-theoretic principle of representation learning that aims to learn a maximally compressed representation that preserves as much information about labels as possible. Under this principle, two different methods have
Yuyan Ni, Yanyan Lan, Ao Liu, Zhiming Ma
doaj +4 more sources
The Conditional Entropy Bottleneck [PDF]
Entropy, 2020 Much of the field of Machine Learning exhibits a prominent set of failure modes, including vulnerability to adversarial examples, poor out-of-distribution (OoD) detection, miscalibration, and willingness to memorize random labelings of datasets.
Ian Fischer
exaly +4 more sources
Learnability for the Information Bottleneck [PDF]
Entropy, 2019 The Information Bottleneck (IB) method provides an insightful and principled approach for balancing compression and prediction for representation learning. The IB objective I ( X ; Z ) - β I ( Y ; Z ) employs a Lagrange multiplier β
Tailin Wu +3 more
doaj +6 more sources
The Convex Information Bottleneck Lagrangian [PDF]
Entropy, 2020 The information bottleneck (IB) problem tackles the issue of obtaining relevant compressed representations T of some random variable X for the task of predicting Y.
Borja Rodríguez Gálvez +2 more
doaj +6 more sources
Bottleneck Problems: An Information and Estimation-Theoretic View
Entropy, 2020 Information bottleneck (IB) and privacy funnel (PF) are two closely related optimization problems which have found applications in machine learning, design of privacy algorithms, capacity problems (e.g., Mrs.
Shahab Asoodeh, Flavio P. Calmon
doaj +2 more sources
Entropy, 2022 Digital communication receivers extract information about the transmitted data from the received signal in subsequent processing steps, such as synchronization, demodulation and channel decoding.
Jan Lewandowsky +2 more
doaj +2 more sources
Information Bottleneck: Theory and Applications in Deep Learning
Entropy, 2020 The information bottleneck (IB) framework, proposed in [...]
Bernhard C. Geiger, Gernot Kubin
doaj +2 more sources
, 2021 The celebrated information bottleneck (IB) principle of Tishby et al. has recently enjoyed renewed attention due to its application in the area of deep learning. This collection investigates the IB principle in this new context.
core +2 more sources
Faithful Explanation Regeneration via Cauchy–Schwarz Mixture Information Bottleneck [PDF]
EntropyLarge pretrained language models can generate fluent free-text explanations for natural language reasoning tasks, but these explanations may contain redundant, irrelevant, or unsupported information.
Ziyang Wang, Junliang Du
doaj +2 more sources