Results 41 to 50 of about 9,649,421 (248)

Can LoRA Teach? Cross A-Matrix in Single Space for Low-Rank Knowledge Distillation

IEEE Access
Low-Rank Adaptation (LoRA) reduces fine-tuning costs by freezing the pre-trained parameters and updating only lightweight low-rank parameters. However, because LoRA still relies on frozen pre-trained parameters, additional Knowledge Distillation (KD ...
Hyeon-Ki Jo, Yuri Seo, Eui-Nam Huh
doaj   +1 more source

Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models

CoRR
Large Language Models usually put more emphasis on accuracy and therefore, will guess even when not certain about the prediction, which is especially severe when fine-tuned on small datasets due to the inherent tendency toward miscalibration. In this work, we introduce Bayesian-LoRA, which reformulates the deterministic LoRA update as a probabilistic ...
Moule Lin, Shuhao Guan, Andrea Patane, David Gregg, Goetz Botterweck   +4 more
openaire   +2 more sources

Potential contribution of biomass gasification‐based technology in energy transition: a technical review coupled with bibliometric studies

Biofuels, Bioproducts and Biorefining, EarlyView.
Abstract Biomass gasification technology has been extensively researched around the world; however, there is a need to evaluate the current research landscape and evolutionary direction of research in the broader context of energy transition. A systematic bibliometric analysis of the Web of Science database was performed for articles that fall within ...
Olasunkanmi Opeoluwa Adeoye, Electo Eduardo Silva Lora, Rubenildo Vieira Andrade, Leonardo Peña Pupo, René Lesme Jaén, Osvaldo José Venturini   +5 more
wiley   +1 more source

M3LoRA: Flexible Task Adaptation via Multiple Low‐Rank Matrices With Mixture‐of‐Subspaces and Minor Singular Components Initialization

CAAI Transactions on Intelligence Technology
Parameter‐efficient fine‐tuning (PEFT) has become a crucial paradigm for domain adaptation, achieving strong performance by updating only a small fraction of model parameters.
Xu Luo, Yongbin Liu, Chunping Ouyang, Ying Yu, Yang Yang   +4 more
doaj   +1 more source

LoRA Fusion: Enhancing Image Generation

Mathematics
Recent advancements in low-rank adaptation (LoRA) have shown its effectiveness in fine-tuning diffusion models for generating images tailored to new downstream tasks.
Dooho Choi, Jeonghyeon Im, Yunsick Sung
doaj   +1 more source

ST-LoRA: Low-Rank Adaptation for Spatio-Temporal Forecasting

Published at ECML-PKDD ...
Weilin Ruan, Wei Chen 0070, Xilin Dang, Jianxiang Zhou, Weichuang Li, Xu Liu 0014, Yuxuan Liang 0002   +6 more
openaire   +3 more sources

D2-LoRA: A Synergistic Approach to Differential and Directional Low-Rank Adaptation

CoRR
19 pages, 3 ...
Nozomu Fujisawa, Masaaki Kondo
openaire   +3 more sources

Video‐based diagnostics supported by artificial intelligence as an opportunity to address the epilepsy diagnostic gap: A narrative review

Epilepsia, EarlyView.
Abstract Despite advancements in epilepsy care, a substantial diagnostic gap persists, particularly in resource‐limited settings. This narrative review explores the potential of video‐based diagnostics augmented by artificial intelligence (AI) to address this gap by enabling earlier and more accessible seizure detection and classification.
Gadi Miron, Robert Terziev, Maximilian Schöls, John McLaren, Tobias Loddenkemper, Jörg Wellmer, Sigrid Mues, Christian Meisel   +7 more
wiley   +1 more source

SQ-LoRA: Memory-Efficient Language Model Compression Through Stable-Rank-Guided Quantization for Edge Computing Applications

Applied Sciences
The deployment of transformer-based language models on resource-constrained edge devices presents fundamental challenges in computational efficiency and memory utilization. We introduce SQ-LoRA (Stable-rank Quantized Low-Rank Adaptation), a theoretically
Seda Bayat Toksöz, Gültekin Işik
doaj   +1 more source

LoRAE: Low-Rank Adaptation for Edge AI

Abstract The rapid advancement of edge artificial intelligence (AI) has unlocked transformative applications across various domains. However, it also poses significant challenges in efficiently updating models on edge devices, which are often constrained by limited computational and communication resources.
Zhixue Wang, Hongyao Ma, Jiahui Zhai
openaire   +1 more source