Frozen Weights as Prior for Parameter-Efficient Fine-Tuning
IEEE AccessIn the fields of natural language processing and computer vision, the emergence of large pre-trained models has led to the adoption of fine-tuning them for downstream tasks as an important paradigm. However, the full fine-tuning approach often comes with
Xiaolong Ma +7 more
doaj +2 more sources
Lottery Rank-Pruning Adaptation Parameter Efficient Fine-Tuning
MathematicsRecent studies on parameter-efficient fine-tuning (PEFT) have introduced effective and efficient methods for fine-tuning large language models (LLMs) on downstream tasks using fewer parameters than required by full fine-tuning.
Juhyeong Kim, Gyunyeop Kim, Sangwoo Kang
doaj +2 more sources
Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning [PDF]
, 2023 Visual Parameter-Efficient Fine-Tuning (PEFT) has become a powerful alternative for full fine-tuning so as to adapt pre-trained vision models to downstream tasks, which only tunes a small number of parameters while freezing the vast majority ones to ease
Zhang, Jing +4 more
core +1 more source
Efficient Abstractive Text Summarization with Large Language Models: A Focused Survey [PDF]
International Journal of Intelligent Computing and Information SciencesRecent advances in Large Language Models (LLMs) have significantly improved the quality of abstractive text summarization. However, the high computational cost and memory requirements of these models limit their practical adoption, particularly in ...
Mahmoud Fayez +3 more
doaj +1 more source
A semi-automated design of instance-based fuzzy parameter tuning for metaheuristics based on decision tree induction [PDF]
, 2015 Two main concepts are established in the literature for the Parameter Setting Problem (PSP) of metaheuristics: Parameter Tuning Strategies (PTS) and Parameter Control Strategies (PCS).
Ries, Jana, Beullens, Patrick
core +1 more source
Gradient-based Parameter Selection for Efficient Fine-Tuning [PDF]
With the growing size of pre-trained models, full fine-tuning and storing all the parameters for various down-stream tasks is costly and infeasible.
Zhang, S. +6 more
core +3 more sources
Deepfake Detection Method Integrating Multiple Parameter-Efficient Fine-Tuning Techniques [PDF]
Jisuanji kexue yu tansuoIn recent years, as deepfake technology matures, face-swapping software and synthesized videos have become widespread. While these techniques offer entertainment, they also provide opportunities for misuse by malicious actors.
ZHANG Yiwen, CAI Manchun, CHEN Yonghao, ZHU Yi, YAO Lifeng
doaj +1 more source
CE-Prompt: enhance prompt expression stability by multiple understanding [PDF]
PeerJ Computer ScienceIn this article, we propose CE-Prompt, an enhanced version of Prompt-Tuning designed to address issues such as the instability of random initialization and inefficiencies caused by long text in pre-trained large language models (LLMs).
Wujian Yang +3 more
doaj +2 more sources
Exploring The Principles and Prospects for Efficient Fine-Tuning of Transformer-Based Pre-Trained Large Language Models [PDF]
ITM Web of ConferencesIn recent years, large language models (LLMs) have made breakthroughs in natural language processing and multimodal tasks. However, the growing model size and the high cost of full parameter fine-tuning pose challenges to their efficient adaptation. This
He Ruiqi
doaj +1 more source
Parameter-Efficient Fine-Tuning Design Spaces
, 2023 Parameter-efficient fine-tuning aims to achieve performance comparable to fine-tuning, using fewer trainable parameters. Several strategies (e.g., Adapters, prefix tuning, BitFit, and LoRA) have been proposed.
Li, Mu +5 more
core