Results 261 to 270 of about 14,259,947 (309)

LoRA-GA: Low-Rank Adaptation with Gradient Approximation

Neural Information Processing Systems
Fine-tuning large-scale pretrained models is prohibitively expensive in terms of computational and memory costs. LoRA, as one of the most popular Parameter-Efficient Fine-Tuning (PEFT) methods, offers a cost-effective alternative by fine-tuning an ...
Shaowen Wang, Linxi Yu, Jian Li
semanticscholar   +1 more source

Learning Tensor Low-Rank Representation for Hyperspectral Anomaly Detection

IEEE Transactions on Cybernetics, 2022
Recently, low-rank representation (LRR) methods have been widely applied for hyperspectral anomaly detection, due to their potentials in separating the backgrounds and anomalies.
Minghua Wang, Qiang Wang, D. Hong, S. K. Roy, J. Chanussot   +4 more
semanticscholar   +1 more source

SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models

arXiv.org
Diffusion models can effectively generate high-quality images. However, as they scale, rising memory demands and higher latency pose substantial deployment challenges.
Muyang Li, Yujun Lin, Zhekai Zhang, Tianle Cai, Xiuyu Li, Junxian Guo, Enze Xie, Chenlin Meng, Jun-Yan Zhu, Song Han   +9 more
semanticscholar   +1 more source

Low-Rank Multilinear Filtering

Digital Signal Processing
Published by Elsevier Digital Signal Processing. ; International audience ; Linear filtering methods are well-known and have been successfully applied to system identification and equalization problems. However, when high-dimensional systems are modeled, these methods often perform unsatisfactorily due to their slow convergence and to the high number ...
Maryam Dehghan, José Henrique de Morais Goulart, André L. F. de Almeida   +2 more
openaire   +2 more sources

Low-Rank and Sparse Representation for Hyperspectral Image Processing: A review

IEEE Geoscience and Remote Sensing Magazine, 2022
Combining rich spectral and spatial information, a hyperspectral image (HSI) can provide a more comprehensive characterization of the Earth’s surface. To better exploit HSIs, a large number of algorithms have been developed during the past few decades ...
Jiangtao Peng, Weiwei Sun, Hengchao Li, Wei Li, Xiangchao Meng, Chiru Ge, Qian Du   +6 more
semanticscholar   +1 more source

Flora: Low-Rank Adapters Are Secretly Gradient Compressors

International Conference on Machine Learning
Despite large neural networks demonstrating remarkable abilities to complete different tasks, they require excessive memory usage to store the optimization states for training.
Yongchang Hao, Yanshuai Cao, Lili Mou
semanticscholar   +1 more source

Low Rank Fourier Ptychography

2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018
In this paper, we introduce a principled algorithmic approach for Fourier ptychographic imaging of dynamic, time-varying targets. To the best of our knowledge, this setting has not been explicitly addressed in the ptychography literature. We argue that such a setting is very natural, and that our methods provide an important first step towards helping ...
Zhengyu Chen 0003, Gauri Jagatap, Seyedehsara Nayer, Chinmay Hegde, Namrata Vaswani   +4 more
openaire   +1 more source

Asymmetry in Low-Rank Adapters of Foundation Models

International Conference on Machine Learning
Parameter-efficient fine-tuning optimizes large, pre-trained foundation models by updating a subset of parameters; in this class, Low-Rank Adaptation (LoRA) is particularly effective.
Jiacheng Zhu, K. Greenewald, Kimia Nadjahi, H. Borde, Rickard Brüel Gabrielsson, Leshem Choshen, Marzyeh Ghassemi, M. Yurochkin, Justin Solomon   +8 more
semanticscholar   +1 more source

Selective Aggregation for Low-Rank Adaptation in Federated Learning

International Conference on Learning Representations
We investigate LoRA in federated learning through the lens of the asymmetry analysis of the learned $A$ and $B$ matrices. In doing so, we uncover that $A$ matrices are responsible for learning general knowledge, while $B$ matrices focus on capturing ...
Pengxin Guo, Shuang Zeng, Yanran Wang, Huijie Fan, Feifei Wang, Liangqiong Qu   +5 more
semanticscholar   +1 more source

Structural Low-Rank Tracking

2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2019
Visual object tracking is an important step for many computer vision applications. The task becomes very challenging when the target undergoes heavy occlusion, background clutters, and sudden illumination variations. Methods that incorporate sparse representation and low-rank assumptions on the target particles have achieved promising results. However,
Sajid Javed, Arif Mahmood, Jorge Dias 0001, Naoufel Werghi   +3 more
openaire   +1 more source