Results 31 to 40 of about 21,351 (259)
A novel model compression method based on joint distillation for deepfake video detection
Journal of King Saud University: Computer and Information Sciences, 2023 In recent years, deepfake videos have been abused to create fake news, which threaten the integrity of digital videos. Although existing detection methods leveraged cumbersome neural networks to achieve promising detection performance, they cannot be ...
Xiong Xu +5 more
doaj +1 more source
Explaining Knowledge Distillation by Quantifying the Knowledge [PDF]
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020 This paper presents a method to interpret the success of knowledge distillation by quantifying and analyzing task-relevant and task-irrelevant visual concepts that are encoded in intermediate layers of a deep neural network (DNN). More specifically, three hypotheses are proposed as follows. 1.
Xu Cheng 0005 +3 more
openaire +2 more sources
Federated Knowledge Distillation
CoRR, 2020 30 pages, 12 figures, 2 tables; This chapter is written for the forthcoming book, Machine Learning and Wireless Communications (Cambridge University Press), edited by H. V. Poor, D. Gunduz, A.
Hyowoon Seo +4 more
openaire +2 more sources
Hint-Dynamic Knowledge Distillation
ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023 Knowledge Distillation (KD) transfers the knowledge from a high-capacity teacher model to promote a smaller student model. Existing efforts guide the distillation by matching their prediction logits, feature embedding, etc., while leaving how to efficiently utilize them in junction less explored.
Yiyang Liu +4 more
openaire +2 more sources
Forest Fire Object Detection Analysis Based on Knowledge Distillation
Fire, 2023 This paper investigates the application of the YOLOv7 object detection model combined with knowledge distillation techniques in forest fire detection.
Jinzhou Xie, Hongmin Zhao
doaj +1 more source
A non-negative feedback self-distillation method for salient object detection [PDF]
PeerJ Computer Science, 2023 Self-distillation methods utilize Kullback-Leibler divergence (KL) loss to transfer the knowledge from the network itself, which can improve the model performance without increasing computational resources and complexity. However, when applied to salient
Lei Chen +6 more
doaj +2 more sources
Discriminator-Enhanced Knowledge-Distillation Networks
Applied Sciences, 2023 Query auto-completion (QAC) serves as a critical functionality in contemporary textual search systems by generating real-time query completion suggestions based on a user’s input prefix. Despite the prevalent use of language models (LMs) in QAC candidate
Zhenping Li +4 more
doaj +1 more source
Knowledge Condensation Distillation
, 2022 Knowledge Distillation (KD) transfers the knowledge from a high-capacity teacher network to strengthen a smaller student. Existing methods focus on excavating the knowledge hints and transferring the whole knowledge to the student. However, the knowledge redundancy arises since the knowledge shows different values to the student at different learning ...
Chenxin Li +7 more
openaire +2 more sources
To Distill or Not to Distill? On the Robustness of Robust Knowledge Distillation
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)Arabic is known to present unique challenges for Automatic Speech Recognition (ASR). On one hand, its rich linguistic diversity and wide range of dialects complicate the development of robust, inclusive models. On the other, current multilingual ASR models are compute-intensive and lack proper comprehensive evaluations. In light of these challenges, we
Abdul Waheed +2 more
openaire +2 more sources
IEEE AccessThe time-dimensional self-distillation seeks to transfer knowledge from earlier historical models to subsequent ones with minimal computational overhead.
Yingchao Wang, Wenqi Niu, Hanpo Hou
doaj +1 more source