Democratizing protein language models with parameter-efficient fine-tuning [PDF]
Proceedings of the National Academy of Sciences of the United States of America, 2023 Abstract Proteomics has been revolutionized by large pre-trained protein language models, which learn unsupervised representations from large corpora of sequences. The parameters of these models are then fine-tuned in a supervised setting to tailor the model to a specific downstream task.
Samuel Sledzieski +2 more
exaly +5 more sources
Structure-Aware Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
Mathematics, 2023 With the growing scale of pre-trained language models (PLMs), full parameter fine-tuning becomes prohibitively expensive and practically infeasible.
Yahao Hu +4 more
doaj +4 more sources
Frozen Weights as Prior for Parameter-Efficient Fine-Tuning
IEEE AccessIn the fields of natural language processing and computer vision, the emergence of large pre-trained models has led to the adoption of fine-tuning them for downstream tasks as an important paradigm. However, the full fine-tuning approach often comes with
Xiaolong Ma +7 more
doaj +3 more sources
Parameter-Efficient Fine-Tuning Method for Task-Oriented Dialogue Systems
Mathematics, 2023 The use of Transformer-based pre-trained language models has become prevalent in enhancing the performance of task-oriented dialogue systems. These models, which are pre-trained on large text data to grasp the language syntax and semantics, fine-tune the
Yunho Mo, Joon Yoo, Sangwoo Kang
doaj +2 more sources
Parameter-efficient fine-tuning of large language models using semantic knowledge tuning [PDF]
Scientific ReportsLarge Language Models (LLMs) are gaining significant popularity in recent years for specialized tasks using prompts due to their low computational cost. Standard methods like prefix tuning utilize special, modifiable tokens that lack semantic meaning and
Nusrat Jahan Prottasha +6 more
doaj +2 more sources
U-SplitDoRA: an improved privacy-preserved U-shaped split parameter-efficient fine-tuning framework through weight decomposition for large language models [PDF]
Frontiers in Artificial IntelligenceAs large language models (LLMs) are getting bigger with respect to the parameter count, ranging from a few million to billions, methods like parameter-efficient fine-tuning (PEFT) have emerged as a crucial approach for adapting these LLMs, such as GPT ...
Samar Singh +3 more
doaj +2 more sources
InfoMSD: an information-maximization self-distillation framework for parameter-efficient fine-tuning on artwork images [PDF]
Frontiers in Artificial IntelligenceIn recent years, despite the remarkable performance of large-scale vision language models across various visual classification tasks, their substantial parameter counts and high fine-tuning costs have hindered deployment in resource-constrained cultural ...
Feng Guan +3 more
doaj +2 more sources
The BSRA framework for dual sparse parameter efficient fine tuning with block structured gating and rank adaptation [PDF]
Scientific ReportsCurrent large-scale language models face severe resource constraints when applied to downstream tasks. While full-fledged fine-tuning enhances model performance, it incurs substantial computational costs. Parameter-efficient fine-tuning (PEFT) techniques
Wang Jian +5 more
doaj +2 more sources
Lottery Rank-Pruning Adaptation Parameter Efficient Fine-Tuning
MathematicsRecent studies on parameter-efficient fine-tuning (PEFT) have introduced effective and efficient methods for fine-tuning large language models (LLMs) on downstream tasks using fewer parameters than required by full fine-tuning.
Juhyeong Kim, Gyunyeop Kim, Sangwoo Kang
doaj +2 more sources
Parameter-Efficient Fine-Tuning without Introducing New Latency [PDF]
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023 ACL2023 camera-ready ...
Baohao Liao, Yan Meng, Christof Monz
openaire +4 more sources