Results 1 to 10 of about 21,351 (259)

Knowledge Distillation: A Survey [PDF]

International Journal of Computer Vision, 2021
In recent years, deep neural networks have been successful in both industry and academia, especially for computer vision tasks. The great success of deep learning is mainly due to its scalability to encode large-scale data and to maneuver billions of model parameters.
Dacheng Tao, Baosheng Yu, Jianping Gou   +2 more
exaly   +4 more sources

Spot-Adaptive Knowledge Distillation

IEEE Transactions on Image Processing, 2022
12 pages, 8 ...
Jingwen Ye, Mingli Song, Jie Song
exaly   +4 more sources

Decoupled Classifier Knowledge Distillation. [PDF]

PLoS ONE
Mainstream knowledge distillation methods primarily include self-distillation, offline distillation, online distillation, output-based distillation, and feature-based distillation.
Hairui Wang, Mengjie Dong, Guifu Zhu, Ya Li   +3 more
doaj   +2 more sources

A survey on knowledge distillation: Recent advancements

Machine Learning with Applications
Deep learning has achieved notable success across academia, medicine, and industry. Its ability to identify complex patterns in large-scale data and to manage millions of parameters has made it highly advantageous. However, deploying deep learning models
Amir Moslemi, Anna Briskina, Zubeka Dang, Jason Li   +3 more
doaj   +3 more sources

MTAKD: multi-teacher agreement knowledge distillation for edge AI skin disease diagnosis [PDF]

Scientific Reports
Skin disease diagnosis remains challenging in remote areas due to limited access to dermatology specialists and unreliable internet connectivity. Edge AI offers a potential solution by offloading the inference process from cloud servers to mobile devices.
Andreas Winata, Nur Afny Catur Andryani, Alexander Agung Santoso Gunawan, Ford Lumban Gaol, Tokuro Matsuo   +4 more
doaj   +2 more sources

Stepwise self-knowledge distillation for skin lesion image classification [PDF]

Scientific Reports
Self-knowledge distillation, which involves using the same network structure for both the teacher and student models, has gained considerable attention in the field of medical image classification.
Jian Zheng, Kewei Xie, Dingwen Zhang, Zhiming Lv, Xiangchun Yu   +4 more
doaj   +2 more sources

Mutual Learning Knowledge Distillation Based on Multi-stage Multi-generative Adversarial Network [PDF]

Jisuanji kexue, 2022
Aiming at the problems of insufficient knowledge distillation efficiency,single stage training methods,complex training processes and difficult convergence of traditional knowledge distillation methods in image classification tasks,this paper designs a ...
HUANG Zhong-hao, YANG Xing-yao, YU Jiong, GUO Liang, LI Xiang
doaj   +1 more source

Multiple-Stage Knowledge Distillation

Applied Sciences, 2022
Knowledge distillation (KD) is a method in which a teacher network guides the learning of a student network, thereby resulting in an improvement in the performance of the student network.
Chuanyun Xu, Nanlan Bai, Wenjian Gao, Tian Li, Mengwei Li, Gang Li, Yang Zhang   +6 more
doaj   +1 more source

Memory-Replay Knowledge Distillation

Sensors, 2021
Knowledge Distillation (KD), which transfers the knowledge from a teacher to a student network by penalizing their Kullback–Leibler (KL) divergence, is a widely used tool for Deep Neural Network (DNN) compression in intelligent sensor systems ...
Jiyue Wang, Pei Zhang, Yanxiong Li
doaj   +1 more source

Review of Recent Distillation Studies [PDF]

MATEC Web of Conferences, 2023
Knowledge distillation has gained a lot of interest in recent years because it allows for compressing a large deep neural network (teacher DNN) into a smaller DNN (student DNN), while maintaining its accuracy.
Gao Minghong
doaj   +1 more source