Results 11 to 20 of about 3,080,648 (304)
Self-Distillation for Randomized Neural Networks [PDF]
IEEE Transactions on Neural Networks and Learning Systems, 2023 Knowledge distillation (KD) is a conventional method in the field of deep learning that enables the transfer of dark knowledge from a teacher model to a student model, consequently improving the performance of the student model. In randomized neural networks, due to the simple topology of network architecture and the insignificant relationship between ...
Minghui Hu 0001 +2 more
openaire +5 more sources
SAF-SD: Self-Distillation Object Segmentation Method Based on Sequential Three-Way Mask and Attention Fusion [PDF]
SensorsTransformer models have achieved powerful performance in various computer vision tasks. However, their black-box nature severely limits model interpretability and the reliability of real-world applications.
Biao Wang +3 more
doaj +2 more sources
Similarity and Consistency by Self-distillation Method [PDF]
Jisuanji kexue, 2023 Due to high data pre-processing costs and missing local features detection in self-distillation methods for models compression,a similarity and consistency by self-distillation(SCD) method is proposed to improve model classification accuracy.Firstly ...
WAN Xu, MAO Yingchi, WANG Zibo, LIU Yi, PING Ping
doaj +1 more source
On Self-Distilling Graph Neural Network [PDF]
Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, 2021 Recently, the teacher-student knowledge distillation framework has demonstrated its potential in training Graph Neural Networks (GNNs). However, due to the difficulty of training over-parameterized GNN models, one may not easily obtain a satisfactory teacher model for distillation.
Yuzhao Chen +5 more
openaire +2 more sources
Knowledge Distillation With Feature Self Attention
IEEE Access, 2023 With the rapid development of deep learning technology, the size and performance of the network continuously grow, making network compression essential for commercial applications.
Sin-Gu Park, Dong-Joong Kang
doaj +1 more source
CoRR, 2022 Knowledge distillation is the procedure of transferring "knowledge" from a large model (the teacher) to a more compact one (the student), often being used in the context of model compression. When both models have the same architecture, this procedure is called self-distillation.
Minh Pham 0005 +3 more
openaire +2 more sources
A self‐distillation object segmentation method via frequency domain knowledge augmentation
IET Computer Vision, 2023 Most self‐distillation methods need complex auxiliary teacher structures and require lots of training samples in object segmentation task. To solve this challenging, a self‐distillation object segmentation method via frequency domain knowledge ...
Lei Chen +3 more
doaj +1 more source
Frontiers in Earth Science, 2023 Salt structures are crucial targets in oil and gas seismic exploitation so that one fast, automatic and accurate method is necessary for accelerating salt structure identification in the exploitation process.
Keran Li +6 more
doaj +1 more source
Cervical Cell Image Classification-Based Knowledge Distillation
Biomimetics, 2022 Current deep-learning-based cervical cell classification methods suffer from parameter redundancy and poor model generalization performance, which creates challenges for the intelligent classification of cervical cytology smear images.
Wenjian Gao +5 more
doaj +1 more source
Frontiers in Ecology and Evolution, 2022 Masked image modeling (MIM) is a learning method in which the unmasked components of the input are utilized to learn and predict the masked signal, enabling learning from large amounts of unannotated data.
Xuying Wang +4 more
doaj +1 more source