Results 21 to 30 of about 124,133 (290)

LayerNorm: A key component in parameter-efficient fine-tuning

CoRR
Fine-tuning a pre-trained model, such as Bidirectional Encoder Representations from Transformers (BERT), has been proven to be an effective method for solving many natural language processing (NLP) tasks. However, due to the large number of parameters in many state-of-the-art NLP models, including BERT, the process of fine-tuning is computationally ...
Taha ValizadehAslani, Hualou Liang
openaire   +3 more sources

SLoRA: Federated Parameter Efficient Fine-Tuning of Language Models

CoRR, 2023
Transfer learning via fine-tuning pre-trained transformer models has gained significant success in delivering state-of-the-art results across various NLP tasks. In the absence of centralized data, Federated Learning (FL) can benefit from distributed and private data of the FL edge clients for fine-tuning.
Sara Babakniya, Ahmed Roushdy Elkordy, Yahya H. Ezzeldin, Qingfeng Liu, Kee-Bong Song, Mostafa El-Khamy, Salman Avestimehr   +6 more
openaire   +3 more sources

CPMI-ChatGLM: parameter-efficient fine-tuning ChatGLM with Chinese patent medicine instructions [PDF]

Scientific Reports
Chinese patent medicine (CPM) is a typical type of traditional Chinese medicine (TCM) preparation that uses Chinese herbs as raw materials and is an important means of treating diseases in TCM. Chinese patent medicine instructions (CPMI) serve as a guide
Can Liu, Kaijie Sun, Qingqing Zhou, Yuchen Duan, Jianhua Shu, Hongxing Kan, Zongyun Gu, Jili Hu   +7 more
doaj   +2 more sources

PeFoMed: Parameter efficient fine-tuning of multimodal large language models for medical CXR [PDF]

Scientific Reports
Multimodal large language models (MLLMs) represent an evolutionary expansion in the capabilities of traditional large language models, enabling them to tackle challenges that surpass the scope of purely text-based applications.
Gang Liu, Xiaotian Tang, Jinlong He, Pengfei Li, Zhaolin Chen, Shenjun Zhong   +5 more
doaj   +2 more sources

Quantum-PEFT: Ultra parameter-efficient fine-tuning

CoRR
ICLR ...
Toshiaki Koike-Akino, Francesco Tonin, Yongtao Wu, Frank Zhengqing Wu, Leyla Naz Candogan, Volkan Cevher   +5 more
openaire   +4 more sources

Multiscale feature fusion for few-shot medical image learning with fisher information-driven layer selection [PDF]

Visual Computing for Industry, Biomedicine, and Art
Few-shot medical image classification is a highly challenging problem in computer-aided diagnosis, with the central difficulty being enabling deep models to learn discriminative features conducive to classification from limited labeled samples.
Kai Zhang, Yanjun Peng, Bo Pang, Xue Chen   +3 more
doaj   +2 more sources

Strong Baselines for Parameter-Efficient Few-Shot Fine-Tuning

Proceedings of the AAAI Conference on Artificial Intelligence
Few-shot classification (FSC) entails learning novel classes given only a few examples per class after a pre-training (or meta-training) phase on a set of base classes. Recent works have shown that simply fine-tuning a pre-trained Vision Transformer (ViT) on new test classes is a strong approach for FSC. Fine-tuning ViTs, however, is expensive in time,
Samyadeep Basu, Daniela Massiceti, Shell Xu Hu, Soheil Feizi   +3 more
openaire   +3 more sources

SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors

Advances in Neural Information Processing Systems 37
Popular parameter-efficient fine-tuning (PEFT) methods, such as LoRA and its variants, freeze pre-trained model weights \(W\) and inject learnable matrices \(ΔW\). These \(ΔW\) matrices are structured for efficient parameterization, often using techniques like low-rank approximations or scaling vectors.
Vijay Lingam, Atula Tejaswi, Aditya Vavre, Aneesh Shetty, Gautham Krishna Gudur, Joydeep Ghosh, Alex Dimakis, Eunsol Choi, Aleksandar Bojchevski, Sujay Sanghavi   +9 more
openaire   +4 more sources

Parameter-efficient fine-tuning in reading comprehension

, 2023
Question Answering is an important task in Natural Language Processing. There are different approaches to answering questions, such as using the knowledge learned during pre-training or extracting an answer from a given context, which is commonly known as reading comprehension.
Abdumalikov, Rustam
openaire   +2 more sources

Bayesian Parameter-Efficient Fine-Tuning for Overcoming Catastrophic Forgetting

IEEE/ACM Transactions on Audio, Speech, and Language Processing
We are motivated primarily by the adaptation of text-to-speech synthesis models; however we argue that more generic parameter-efficient fine-tuning (PEFT) is an appropriate framework to do such adaptation. Nevertheless, catastrophic forgetting remains an issue with PEFT, damaging the pre-trained model's inherent capabilities.
Haolin Chen, Philip N. Garner
openaire   +3 more sources