Results 311 to 320 of about 1,866,673 (365)

A neural vector quantizer

[Proceedings] 1992 IEEE International Symposium on Circuits and Systems, 2003
A technique of vector quantization based on neural networks called neural vector quantization is investigated. A neural vector quantizer has been developed for data compression and is capable of faster parallel quantization than conventional vector quantizers. The architecture, dynamics, and training strategies are presented.
J. Hanson, Zhongde Wang
openaire   +2 more sources

Extension of two-stage vector quantization-lattice vector quantization

IEEE Transactions on Communications, 1997
This paper is the extension of two-stage vector quantization-(spherical) lattice vector quantization (VQ-(S)LVQ) recently introduced by Pan and Fischer (see IEEE Trans. Inform. Theory, vol.41, p.155, 1995). First, according to high resolution quantization theory, generalized vector quantization-lattice vector quantization (G-VQ-LVQ) is formulated in ...
openaire   +2 more sources

QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks

International Conference on Machine Learning
Post-training quantization (PTQ) reduces the memory footprint of LLMs by quantizing their weights to low-precision. In this work, we introduce QuIP#, a weight-only PTQ method that achieves state-of-the-art results in extreme compression regimes ($\le$ 4 ...
Albert Tseng, Jerry Chee, Qingyao Sun, Volodymyr Kuleshov, Christopher De Sa   +4 more
semanticscholar   +1 more source

Vector Quantization

2006
Publisher Summary A quantization strategy that works with sequences or blocks of output provides some improvement in performance over scalar quantization. This chapter provides an overview of vector quantization. The vector of source outputs forms the input to the vector quantizer. At both the encoder and decoder of the vector quantizer, one has a set
openaire   +3 more sources

KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

Neural Information Processing Systems
LLMs are seeing growing use for applications which require large context windows, and with these large context windows KV cache activations surface as the dominant contributor to memory consumption during inference.
Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Michael W. Mahoney, Y. Shao, Kurt Keutzer, A. Gholami   +6 more
semanticscholar   +1 more source

Vector quantization of neural networks

IEEE Transactions on Neural Networks, 1998
The problem of vector quantizing the parameters of a neural network is addressed, followed by a discussion of different algorithms applicable for quantizer design. Optimal, as well as several suboptimal quantization schemes are described. Simulations involving nonlinear prediction of speech signals are presented to compare the performance of different ...
W.C. Chu, N.K. Bose
openaire   +3 more sources

Unrestricted multistage vector quantizers

IEEE Transactions on Information Theory, 1992
A low-complexity, high-performance vector quantization technique for medium-to-high-bit-rate encoding based on multistage (residual) VQ is introduced. The design, a variable-rate version of multistage VQ, is the vector extension of the piecewise uniform scalar quantizers of P.F. Swaszek and J.B.
openaire   +4 more sources

Trellis coded vector quantization

IEEE Transactions on Communications, 1992
A vector generalization of trellis coded quantization (TCQ), called trellis coded vector quantization (TCVQ), and experimental results showing its performance for an i.i.d. Gaussian source are presented. For a given rate, TCVQ yields a lower distortion that TCQ at the cost of an increase in implementation complexity. In addition, TCVQ allows fractional
H.S. Wang, Nader Moayeri
openaire   +2 more sources

Spectrum vector quantization

2008 7th World Congress on Intelligent Control and Automation, 2008
This paper proposes a new spectrum vector quantization algorithm (SVQ). SVQ conducts vector quantization in spectrum space. It is characterized by some novels. The first is the informed initialization of prototypes, which is achieved by a modified support vector clustering procedure. The second is the SVD-based spectrum analysis. This technique employs
Ping Ling, Ming Xu, Xiangsheng Rong, Dajin Gao, Xiangyang You   +4 more
openaire   +2 more sources

Constrained Vector Quantization

1992
There is no better way to quantize a single vector than to use VQ with a codebook that is optimal for the probability distribution describing the random vector. However, direct use of VQ suffers from a serious complexity barrier that greatly limits its practical use as a complete and self-contained coding ...
Robert M. Gray, Allen Gersho
openaire   +2 more sources