Results 21 to 30 of about 1,057,004 (267)
Self-attention in vision transformers performs perceptual grouping, not attention
Frontiers in Computer Science, 2023 Recently, a considerable number of studies in computer vision involve deep neural architectures called vision transformers. Visual processing in these models incorporates computational models that are claimed to implement attention mechanisms. Despite an
Paria Mehrani, John K. Tsotsos
doaj +1 more source
Self-Attentive Associative Memory
CoRR, 2020 Heretofore, neural networks with external memory are restricted to single memory with lossy representations of memory interactions. A rich representation of relationships between memory pieces urges a high-order and segregated relational memory.
Hung Le 0002 +2 more
openaire +3 more sources
The Lipschitz Constant of Self-Attention
CoRR, 2020 Lipschitz constants of neural networks have been explored in various contexts in deep learning, such as provable adversarial robustness, estimating Wasserstein distance, stabilising training of GANs, and formulating invertible neural networks. Such works have focused on bounding the Lipschitz constant of fully connected or convolutional networks ...
Hyunjik Kim +2 more
openaire +3 more sources
Denoising Self-Attentive Sequential Recommendation
Proceedings of the 16th ACM Conference on Recommender Systems, 2022 Transformer-based sequential recommenders are very powerful for capturing both short-term and long-term sequential item dependencies. This is mainly attributed to their unique self-attention networks to exploit pairwise item-item interactions within the sequence.
Huiyuan Chen +8 more
openaire +2 more sources
Self and hetero-perception and discrimination in Attention Deficit Hyperactivity Disorder
Heliyon, 2020 This study intends to show the external perception that Primary Education students have of their schoolmates with Attention Deficit Hyperactivity Disorder (ADHD) and the perception of the student who has being diagnosed with ADHD himself/herself in order
David Pérez-Jorge +4 more
doaj +1 more source
Introduce the Result Into Self-Attention
CoRR, 2021 Traditional self-attention mechanisms in convolutional networks tend to use only the output of the previous layer as input to the attention network, such as SENet, CBAM, etc. In this paper, we propose a new attention modification method that tries to get the output of the classification network in advance and use it as a part of the input of the ...
openaire +2 more sources
CoRR, 2019 Advanced methods of applying deep learning to structured data such as graphs have been proposed in recent years. In particular, studies have focused on generalizing convolutional neural networks to graph data, which includes redefining the convolution and the downsampling (pooling) operations for graphs.
Junhyun Lee, Inyeop Lee, Jaewoo Kang
openaire +3 more sources
Self Residual Attention Network for Deep Face Recognition
IEEE Access, 2019 Discriminative feature embedding is of essential importance in the field of large scale face recognition. In this paper, we propose a self residual attention-based convolutional neural network (SRANet) for discriminative face feature embedding, which ...
Hefei Ling +5 more
doaj +1 more source
Acoustic Impedance Inversion from Seismic Imaging Profiles Using Self Attention U-Net
Remote Sensing, 2023 Seismic impedance inversion is a vital way of geological interpretation and reservoir investigation from a geophysical perspective. However, it is inevitably an ill-posed problem due to the noise or the band-limited characteristic of seismic data ...
Liurong Tao, Haoran Ren, Zhiwei Gu
doaj +1 more source
Formula Graph Self‐Attention Network for Representation‐Domain Independent Materials Discovery
Advanced Science, 2022 The success of machine learning (ML) in materials property prediction depends heavily on how the materials are represented for learning. Two dominant families of material descriptors exist, one that encodes crystal structure in the representation and the
Achintha Ihalage, Yang Hao
doaj +1 more source