Results 11 to 20 of about 21,063 (212)
Knowledge distillation in deep learning and its applications [PDF]
PeerJ Computer Science, 2021 Deep learning based models are relatively large, and it is hard to deploy such models on resource-limited devices such as mobile phones and embedded devices.
Abdolmaged Alkhulaifi +2 more
doaj +2 more sources
Reverse Self-Distillation Overcoming the Self-Distillation Barrier
IEEE Open Journal of the Computer Society, 2023 Deep neural networks generally cannot gather more helpful information with limited data in image classification, resulting in poor performance. Self-distillation, as a novel knowledge distillation technique, integrates the roles of teacher and student ...
Shuiping Ni +4 more
doaj +1 more source
Multi-assistant Dynamic Setting Method for Knowledge Distillation [PDF]
Jisuanji kexueKnowledge distillation is increasingly gaining attention in key areas such as model compression for object recognition.Through in-depth research into the efficiency of knowledge distillation and an analysis of the characteristics of knowledge transfer ...
SI Yuehang, CHENG Qing, HUANG Jincai
doaj +1 more source
A novel model compression method based on joint distillation for deepfake video detection
Journal of King Saud University: Computer and Information Sciences, 2023 In recent years, deepfake videos have been abused to create fake news, which threaten the integrity of digital videos. Although existing detection methods leveraged cumbersome neural networks to achieve promising detection performance, they cannot be ...
Xiong Xu +5 more
doaj +1 more source
Forest Fire Object Detection Analysis Based on Knowledge Distillation
Fire, 2023 This paper investigates the application of the YOLOv7 object detection model combined with knowledge distillation techniques in forest fire detection.
Jinzhou Xie, Hongmin Zhao
doaj +1 more source
Discriminator-Enhanced Knowledge-Distillation Networks
Applied Sciences, 2023 Query auto-completion (QAC) serves as a critical functionality in contemporary textual search systems by generating real-time query completion suggestions based on a user’s input prefix. Despite the prevalent use of language models (LMs) in QAC candidate
Zhenping Li +4 more
doaj +1 more source
A non-negative feedback self-distillation method for salient object detection [PDF]
PeerJ Computer Science, 2023 Self-distillation methods utilize Kullback-Leibler divergence (KL) loss to transfer the knowledge from the network itself, which can improve the model performance without increasing computational resources and complexity. However, when applied to salient
Lei Chen +6 more
doaj +2 more sources
IEEE AccessThe time-dimensional self-distillation seeks to transfer knowledge from earlier historical models to subsequent ones with minimal computational overhead.
Yingchao Wang, Wenqi Niu, Hanpo Hou
doaj +1 more source
International Journal of Applied Earth Observations and Geoinformation, 2023 Knowledge distillation is one effective approach to compress deep learning models. However, the current distillation methods are relatively monotonous. There are still rare studies about the combination of distillation strategies using multiple types of ...
Ziyi Chen +5 more
doaj +1 more source
MKD: Mixup-Based Knowledge Distillation for Mandarin End-to-End Speech Recognition
Algorithms, 2022 Large-scale automatic speech recognition model has achieved impressive performance. However, huge computational resources and massive amount of data are required to train an ASR model.
Xing Wu +4 more
doaj +1 more source