Results 271 to 280 of about 14,259,947 (309)

Low-Rank Preserving Projections

IEEE Transactions on Cybernetics, 2016
As one of the most popular dimensionality reduction techniques, locality preserving projections (LPP) has been widely used in computer vision and pattern recognition. However, in practical applications, data is always corrupted by noises. For the corrupted data, samples from the same class may not be distributed in the nearest area, thus LPP may lose ...
Yuwu Lu, Zhihui Lai 0001, Yong Xu 0001, Xuelong Li 0001, David Zhang 0001, Chun Yuan 0003   +5 more
openaire   +3 more sources

LoRA-Pro: Are Low-Rank Adapters Properly Optimized?

International Conference on Learning Representations
Low-rank adaptation, also known as LoRA, has emerged as a prominent method for parameter-efficient fine-tuning of foundation models. Despite its computational efficiency, LoRA still yields inferior performance compared to full fine-tuning. In this paper,
Zhengbo Wang, Jian Liang
semanticscholar   +1 more source

Compressing Large Language Models using Low Rank and Low Precision Decomposition

Neural Information Processing Systems
The prohibitive sizes of Large Language Models (LLMs) today make it difficult to deploy them on memory-constrained edge devices. This work introduces $\rm CALDERA$ -- a new post-training LLM compression algorithm that harnesses the inherent low-rank ...
R. Saha, Naomi Sagan, Varun Srivastava, Andrea J. Goldsmith, Mert Pilanci   +4 more
semanticscholar   +1 more source

Mixture-of-Subspaces in Low-Rank Adaptation

Conference on Empirical Methods in Natural Language Processing
In this paper, we introduce a subspace-inspired Low-Rank Adaptation (LoRA) method, which is computationally efficient, easy to implement, and readily applicable to large language, multimodal, and diffusion models. Initially, we equivalently decompose the
Taiqiang Wu, Jiahao Wang, Zhe Zhao, Ngai Wong   +3 more
semanticscholar   +1 more source

Discriminative Low-Rank Tracking

2015 IEEE International Conference on Computer Vision (ICCV), 2015
Good tracking performance is in general attributed to accurate representation over previously obtained targets or reliable discrimination between the target and the surrounding background. In this work, we exploit the advantages of the both approaches to achieve a robust tracker.
Yao Sui, Yafei Tang, Li Zhang 0023
openaire   +1 more source

Low-Rank Adaptation for Foundation Models: A Comprehensive Review

arXiv.org
The rapid advancement of foundation modelslarge-scale neural networks trained on diverse, extensive datasetshas revolutionized artificial intelligence, enabling unprecedented advancements across domains such as natural language processing, computer ...
Menglin Yang, Jialin Chen, Yifei Zhang, Jiahong Liu, Jiasheng Zhang, Qiyao Ma, Harshit Verma, Qianru Zhang, Min Zhou, Irwin King, Rex Ying   +10 more
semanticscholar   +1 more source

Low-Rank Tensor Tracking

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), 2019
Visual object tracking is an important step for many computer vision applications. Visual tracking becomes more challenging when the target object observes severe occlusion, lighting variations, background clutter, and deformation difficulties to name a few.
Sajid Javed, Jorge Dias 0001, Naoufel Werghi   +2 more
openaire   +1 more source

Low-Rank Quantization-Aware Training for LLMs

arXiv.org
Large language models (LLMs) are omnipresent, however their practical deployment is challenging due to their ever increasing computational and memory demands.
Yelysei Bondarenko, Riccardo Del Chiaro, Markus Nagel   +2 more
semanticscholar   +1 more source

Loki: Low-Rank Keys for Efficient Sparse Attention

Neural Information Processing Systems
Inference on large language models (LLMs) can be expensive in terms of the compute and memory costs involved, especially when long sequence lengths are used.
Prajwal Singhania, Siddharth Singh, Shwai He, S. Feizi, A. Bhatele   +4 more
semanticscholar   +1 more source

Low-rank preserving embedding

Pattern Recognition, 2017
Abstract In this paper, we consider the problem of linear dimensionality reduction with the novel technique of low-rank representation, which is a promising tool of discovering subspace structures of given data. Existing approaches based on graph embedding usually capture structure of data via stacking the local structure of each datum, such as ...
Yupei Zhang, Ming Xiang, Bo Yang 0041
openaire   +1 more source