Results 11 to 20 of about 5,906,664 (269)
Convolutional Self-Attention Networks [PDF]
Proceedings of the 2019 Conference of the North, 2019 Self-attention network (SAN) has recently attracted increasing interest due to its fully parallelized computation and flexibility in modeling dependencies. It can be further enhanced with multi-headed attention mechanism by allowing the model to jointly attend to information from different representation subspaces at different positions (Vaswani et al.,
Baosong Yang +4 more
openaire +3 more sources
Self-Attentional Acoustic Models [PDF]
Interspeech 2018, 2018 Published at Interspeech ...
Matthias Sperber +4 more
openaire +3 more sources
Lightweight Self-Attentive Sequential Recommendation [PDF]
Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021 Modern deep neural networks (DNNs) have greatly facilitated the development of sequential recommender systems by achieving state-of-the-art recommendation performance on various sequential recommendation tasks. Given a sequence of interacted items, existing DNN-based sequential recommenders commonly embed each item into a unique vector to support ...
Yang Li 0140 +3 more
openaire +4 more sources
On the Robustness of Self-Attentive Models [PDF]
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019 This work examines the robustness of self-attentive neural networks against adversarial input perturbations. Specifically, we investigate the attention and feature extraction mechanisms of state-of-the-art recurrent neural networks and self-attentive architectures for sentiment analysis, entailment and machine translation under adversarial attacks.
Yu-Lun Hsieh +5 more
openaire +1 more source
On The Computational Complexity of Self-Attention
CoRR, 2022 Transformer architectures have led to remarkable progress in many state-of-art applications. However, despite their successes, modern transformers rely on the self-attention mechanism, whose time- and space-complexity is quadratic in the length of the input.
Feyza Duman Keles +2 more
openaire +3 more sources
Self-Attentive Sequential Recommendation [PDF]
2018 IEEE International Conference on Data Mining (ICDM), 2018 Accepted by ICDM'18 as a long ...
Wang-Cheng Kang, Julian J. McAuley
openaire +2 more sources
Switchable Self-attention Module
CoRR, 2022 Attention mechanism has gained great success in vision recognition. Many works are devoted to improving the effectiveness of attention mechanism, which finely design the structure of the attention operator. These works need lots of experiments to pick out the optimal settings when scenarios change, which consumes a lot of time and computational ...
ShanShan Zhong, Wushao Wen, Jinghui Qin
openaire +2 more sources
Self-Attentive Associative Memory
CoRR, 2020 Heretofore, neural networks with external memory are restricted to single memory with lossy representations of memory interactions. A rich representation of relationships between memory pieces urges a high-order and segregated relational memory.
Hung Le 0002 +2 more
openaire +3 more sources
The Lipschitz Constant of Self-Attention
CoRR, 2020 Lipschitz constants of neural networks have been explored in various contexts in deep learning, such as provable adversarial robustness, estimating Wasserstein distance, stabilising training of GANs, and formulating invertible neural networks. Such works have focused on bounding the Lipschitz constant of fully connected or convolutional networks ...
Hyunjik Kim +2 more
openaire +3 more sources
On the Integration of Self-Attention and Convolution
2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022 Accepted to ...
Xuran Pan +6 more
openaire +2 more sources