Results 231 to 240 of about 188,127 (258)
Some of the next articles are maybe not open access.
A Comprehensive Survey on Knowledge Distillation
Trans. Mach. Learn. Res.Deep Neural Networks (DNNs) have achieved notable performance in the fields of computer vision and natural language processing with various applications in both academia and industry.
Amir M. Mansourian +10 more
semanticscholar +1 more source
International Conference on Machine Learning
We introduce Score identity Distillation (SiD), an innovative data-free method that distills the generative capabilities of pretrained diffusion models into a single-step generator.
Mingyuan Zhou +4 more
semanticscholar +1 more source
We introduce Score identity Distillation (SiD), an innovative data-free method that distills the generative capabilities of pretrained diffusion models into a single-step generator.
Mingyuan Zhou +4 more
semanticscholar +1 more source
A Survey on Knowledge Distillation of Large Language Models
arXiv.orgIn the era of Large Language Models (LLMs), Knowledge Distillation (KD) emerges as a pivotal methodology for transferring advanced capabilities from leading proprietary LLMs, such as GPT-4, to their open-source counterparts like LLaMA and Mistral ...
Xiaohan Xu +8 more
semanticscholar +1 more source
Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
arXiv.orgKnowledge distillation improves large language model (LLM) reasoning by compressing the knowledge of a teacher LLM to train smaller LLMs. On-policy distillation advances this approach by having the student sample its own trajectories while a teacher LLM ...
Siyan Zhao +6 more
semanticscholar +1 more source
Experimental demonstration of logical magic state distillation
NatureRealizing universal fault-tolerant quantum computation is a key goal in quantum information science1, 2, 3–4. By encoding quantum information into logical qubits using quantum error correcting codes, physical errors can be detected and corrected ...
Pedro Sales Rodriguez +72 more
semanticscholar +1 more source
D4M: Dataset Distillation via Disentangled Diffusion Model
Computer Vision and Pattern RecognitionDataset distillation offers a lightweight synthetic dataset for fast network training with promising test accuracy. To imitate the performance of the original dataset, most approaches employ bi-level optimization and the distillation space relies on the ...
Duo Su +4 more
semanticscholar +1 more source
Reciprocal Teacher-Student Learning via Forward and Feedback Knowledge Distillation
IEEE transactions on multimediaKnowledge distillation (KD) is a prevalent model compression technique in deep learning, aiming to leverage knowledge from a large teacher model to enhance the training of a smaller student model.
Jianping Gou +6 more
semanticscholar +1 more source
Reinforcement Learning via Self-Distillation
arXiv.orgLarge language models are increasingly post-trained with reinforcement learning in verifiable domains such as code and math. Yet, current methods for reinforcement learning with verifiable rewards (RLVR) learn only from a scalar outcome reward per ...
Jonas Hubotter +10 more
semanticscholar +1 more source
Spot-Adaptive Knowledge Distillation
IEEE Transactions on Image Processing, 2022Jie Song, Jingwen Ye, Mingli Song
exaly
Pore wetting in membrane distillation: A comprehensive review
Progress in Materials Science, 2021Hooman Chamani +2 more
exaly