Results 1 to 10 of about 2,331,313 (323)
Visual Instruction Tuning [PDF]
Neural Information Processing Systems, 2023 Instruction tuning large language models (LLMs) using machine-generated instruction-following data has improved zero-shot capabilities on new tasks, but the idea is less explored in the multimodal field. In this paper, we present the first attempt to use
Haotian Liu +3 more
semanticscholar +1 more source
Improved Baselines with Visual Instruction Tuning [PDF]
Computer Vision and Pattern Recognition, 2023 Large multimodal models (LMM) have recently shown encouraging progress with visual instruction tuning. In this paper, we present the first systematic study to investigate the design choices of LMMs in a controlled setting under the LLaVA framework.
Haotian Liu +3 more
semanticscholar +1 more source
Prefix-Tuning: Optimizing Continuous Prompts for Generation [PDF]
Annual Meeting of the Association for Computational Linguistics, 2021 Fine-tuning is the de facto way of leveraging large pretrained language models for downstream tasks. However, fine-tuning modifies all the language model parameters and therefore necessitates storing a full copy for each task.
Xiang Lisa Li, Percy Liang
semanticscholar +1 more source
The Power of Scale for Parameter-Efficient Prompt Tuning [PDF]
Conference on Empirical Methods in Natural Language Processing, 2021 In this work, we explore “prompt tuning,” a simple yet effective mechanism for learning “soft prompts” to condition frozen language models to perform specific downstream tasks.
Brian Lester +2 more
semanticscholar +1 more source
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning [PDF]
Neural Information Processing Systems, 2023 Large-scale pre-training and instruction tuning have been successful at creating general-purpose language models with broad competence. However, building general-purpose vision-language models is challenging due to the rich input distributions and task ...
Wenliang Dai +8 more
semanticscholar +1 more source
DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation [PDF]
Computer Vision and Pattern Recognition, 2022 Large text-to-image models achieved a remarkable leap in the evolution of AI, enabling high-quality and diverse synthesis of images from a given text prompt.
Nataniel Ruiz +5 more
semanticscholar +1 more source
European Conference on Computer Vision, 2022 The current modus operandi in adapting pre-trained models involves updating all the backbone parameters, ie, full fine-tuning. This paper introduces Visual Prompt Tuning (VPT) as an efficient and effective alternative to full fine-tuning for large-scale ...
Menglin Jia +6 more
semanticscholar +1 more source
Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! [PDF]
International Conference on Learning Representations, 2023 Optimizing large language models (LLMs) for downstream use cases often involves the customization of pre-trained LLMs through further fine-tuning. Meta's open release of Llama models and OpenAI's APIs for fine-tuning GPT-3.5 Turbo on custom datasets also
Xiangyu Qi +6 more
semanticscholar +1 more source
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning [PDF]
Neural Information Processing Systems, 2022 Few-shot in-context learning (ICL) enables pre-trained language models to perform a previously-unseen task without any gradient-based training by feeding a small number of training examples as part of the input.
Haokun Liu +6 more
semanticscholar +1 more source
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models [PDF]
Annual Meeting of the Association for Computational Linguistics, 2021 We introduce BitFit, a sparse-finetuning method where only the bias-terms of the model (or a subset of them) are being modified. We show that with small-to-medium training data, applying BitFit on pre-trained BERT models is competitive with (and ...
Elad Ben-Zaken +2 more
semanticscholar +1 more source