Results 251 to 260 of about 14,848 (289)
Self-Distillation: Towards Efficient and Compact Neural Networks [PDF]
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021 Remarkable achievements have been obtained by deep neural networks in the last several years. However, the breakthrough in neural networks accuracy is always accompanied by explosive growth of computation and parameters, which leads to a severe limitation of model deployment. In this paper, we propose a novel knowledge distillation technique named self-
Linfeng Zhang, Kaisheng
exaly +3 more sources
Some of the next articles are maybe not open access.
Related searches:
Related searches:
Tolerant Self-Distillation for image classification
Neural NetworksDeep neural networks tend to suffer from the overfitting issue when the training data are not enough. In this paper, we introduce two metrics from the intra-class distribution of correct-predicted and incorrect-predicted samples to provide a new perspective on the overfitting issue.
Mushui Liu +4 more
openaire +2 more sources
Self-Distillation for Few-Shot Image Captioning
2021 IEEE Winter Conference on Applications of Computer Vision (WACV), 2021The development of large-scale image-captioning datasets is expensive, while the abundance of unpaired images and text corpus can potentially help reduce the efforts of manual annotation. In this paper, we study the few-shot image captioning problem that only requires a small amount of annotated image-caption pairs.
Xianyu Chen, Ming Jiang, Qi Zhao
openaire +1 more source
Self-Distillation for Gaussian Process Models [PDF]
, 2023 We propose two approaches to extend the notion of knowledge distillation to Gaussian Process Regression (GPR) and Gaussian Process Classification (GPC); data-centric and distribution-centric. The data-centric approach resembles most current distillation techniques for machine learning, and refits a model on deterministic predictions from the teacher ...
Borup, Kenneth, Andersen, Lars Nørvang
openaire +1 more source
Probabilistic online self-distillation
Neurocomputing, 2022Tzelepi, Maria +2 more
openaire +1 more source
Intra-class progressive and adaptive self-distillation
Neural NetworksIn recent years, knowledge distillation (KD) has become widely used in compressing models, training compact and efficient students to reduce computational load and training time due to the increasing parameters in deep neural networks. To minimize training costs, self-distillation has been proposed, with methods like offline-KD and online-KD requiring ...
Jianping Gou +5 more
openaire +2 more sources
SSSD: Self-Supervised Self Distillation
2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023Wei-Chi Chen, Wei-Ta Chu
openaire +1 more source
Restructuring the Teacher and Student in Self-Distillation
IEEE Transactions on Image ProcessingKnowledge distillation aims to achieve model compression by transferring knowledge from complex teacher models to lightweight student models. To reduce reliance on pre-trained teacher models, self-distillation methods utilize knowledge from the model itself as additional supervision.
Yujie Zheng +5 more
openaire +2 more sources
Image classification based on self-distillation
Applied Intelligence, 2022Yuting Li +4 more
openaire +1 more source