Results 1 to 10 of about 173,284 (257)
The Expressive Power of Low-Rank Adaptation
CoRR, 2023 40 pages, 5 ...
Yuchen Zeng, Kangwook Lee 0001
openaire +4 more sources
Low-rank adaptation for edge AI. [PDF]
Sci RepThe rapid advancement of edge artificial intelligence (AI) has unlocked transformative applications across various domains. However, it also poses significant challenges in efficiently updating models on edge devices, which are often constrained by limited computational and communication resources.
Wang Z, Ma H, Zhai J.
europepmc +4 more sources
Continual Learning with Low Rank Adaptation
CoRR, 2023 Recent work using pretrained transformers has shown impressive performance when fine-tuned with data from the downstream problem of interest. However, they struggle to retain that performance when the data characteristics changes. In this paper, we focus on continual learning, where a pre-trained transformer is updated to perform well on new data ...
Martin Wistuba +3 more
openaire +3 more sources
Batched Low-Rank Adaptation of Foundation Models
CoRR, 2023 16 pages, 3 ...
Yeming Wen, Swarat Chaudhuri
openaire +4 more sources
Lightweight Low-Rank Adaptation Vision Transformer Framework for Cervical Cancer Detection and Cervix Type Classification [PDF]
BioengineeringZhenchen Hong, Yu K Mo
exaly +2 more sources
Adaptive quantile low-rank matrix factorization [PDF]
Pattern Recognition, 2020 Low-rank matrix factorization (LRMF) has received much popularity owing to its successful applications in both computer vision and data mining. By assuming noise to come from a Gaussian, Laplace or mixture of Gaussian distributions, significant efforts have been made on optimizing the (weighted) $L_1$ or $L_2$-norm loss between an observed matrix and ...
Shuang Xu, Chunxia Zhang, Jiangshe Zhang
openaire +2 more sources
Adaptive Low-Rank Approximation of Collocation Matrices [PDF]
Computing, 2003 In this paper there is dealt with the solution of integral equations using collocation methods with almost linear complexity. There are used fast multipole, panel clustering and \(H\)-matrix methods which gain their efficiency from approximating the kernel function. The proposed \(H\)-matrix algorithm is purely algebraic.
Mario Bebendorf, Sergej Rjasanow
openaire +2 more sources
FouRA: Fourier Low-Rank Adaptation
Advances in Neural Information Processing Systems 37While Low-Rank Adaptation (LoRA) has proven beneficial for efficiently fine-tuning large models, LoRA fine-tuned text-to-image diffusion models lack diversity in the generated images, as the model tends to copy data from the observed training samples. This effect becomes more pronounced at higher values of adapter strength and for adapters with higher ...
Shubhankar Borse +9 more
openaire +3 more sources
A Bayesian Interpretation of Adaptive Low-Rank Adaptation
ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)Motivated by the sensitivity-based importance score of the adaptive low-rank adaptation (AdaLoRA), we utilize more theoretically supported metrics, including the signal-to-noise ratio (SNR), along with the Improved Variational Online Newton (IVON) optimizer, for adaptive parameter budget allocation.
Haolin Chen, Philip N. Garner
openaire +2 more sources
WaRA: Wavelet Low Rank Adaptation
CoRRAdapting large pretrained vision models to medical image classification is often limited by memory, computation, and task-specific specializations. Parameter-efficient fine-tuning (PEFT) methods like LoRA reduce this cost by learning low-rank updates, but operating directly in feature space can struggle to capture the localized, multi-scale features ...
Heidari, Moein +5 more
openaire +2 more sources