Results 61 to 70 of about 1,111,935 (267)

Centered Self-attention Layers

The self-attention mechanism in transformers and the message-passing mechanism in graph neural networks are repeatedly applied within deep learning architectures. We show that this application inevitably leads to oversmoothing, i.e., to similar representations at the deeper layers for different tokens in transformers and different nodes in graph neural
Ameen Ali, Tomer Galanti, Lior Wolf
openaire   +2 more sources

Self-Attention for Incomplete Utterance Rewriting

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022
Incomplete utterance rewriting (IUR) has recently become an essential task in NLP, aiming to complement the incomplete utterance with sufficient context information for comprehension. In this paper, we propose a novel method by directly extracting the coreference and omission relationship from the self-attention weight matrix of the transformer instead
Yong Zhang 0058, Zhitao Li 0002, Jianzong Wang, Ning Cheng 0001, Jing Xiao 0006   +4 more
openaire   +3 more sources

“We Cannot Talk as a Family”: Complexities of Survivorship Communication Among Hispanic/Latino Young Adult Survivors of Childhood Cancer, Parents, and Clinicians

Pediatric Blood &Cancer, EarlyView.
ABSTRACT Background Young adult childhood cancer survivors (YA‐CCS) have an increased risk of treatment‐related health complications throughout their lives. Lifelong survivorship care is recommended, yet many YA‐CCS experience disruptions in care during pediatric‐to‐adult transitions and do not receive recommended survivorship care.
Kevin Shanchen Pu, Nicole Fernández‐Viña, Briana C. Martin‐Villa, Daniela Ramirez, Rebecca M. Lewinsohn, Mary Smith, Heidi Boynton, Lidia Schapira, Stephanie M. Smith   +8 more
wiley   +1 more source

Self-Attention Neural Bag-of-Features

2022 IEEE 32nd International Workshop on Machine Learning for Signal Processing (MLSP), 2022
In this work, we propose several attention formulations for multivariate sequence data. We build on top of the recently introduced 2D-Attention and reformulate the attention learning methodology by quantifying the relevance of feature/temporal dimensions through latent spaces based on self-attention rather than learning them directly.
Kateryna Chumachenko, Alexandros Iosifidis, Moncef Gabbouj   +2 more
openaire   +3 more sources

Effectiveness of Resistance Intradialytic Exercise Compared to Aerobic Intradialytic Exercise for Patients With Chronic Kidney Disease: A Randomized Controlled Clinical Trial

Therapeutic Apheresis and Dialysis, EarlyView.
ABSTRACT Background Patients with chronic kidney disease undergoing hemodialysis commonly experience reduced physical function, fatigue, poor sleep quality, and impaired health‐related quality of life. Intradialytic exercise has been proposed as a non‐pharmacological strategy to improve these outcomes.
Klebson da Silva Almeida, Gisela Cristiane Miyamoto, Daniel da Costa Torres, Ian Setubal Reis Chaves, Jonas do Carmo Paschoalin, Braúlio Lima Nascimento, Luciana Dias Chiavegato   +6 more
wiley   +1 more source

Self-reflection Orients Visual Attention Downward

Frontiers in Psychology, 2017
Previous research has demonstrated abstract concepts associated with spatial location (e.g., God in the Heavens) could direct visual attention upward or downward, because thinking about the abstract concepts activates the corresponding vertical ...
Yi Liu, Yi Liu, Yu Tong, Hong Li
doaj   +1 more source

Comparative Evaluation of Hemodiafiltration, Hemoperfusion, and Standard Hemodialysis on Efficacy, Inflammatory Control, Dialysis Adequacy, and Safety in End‐Stage Renal Disease: A Prospective Observational Study

Therapeutic Apheresis and Dialysis, EarlyView.
ABSTRACT Background Chronic micro‐inflammation in patients with end‐stage renal disease (ESRD) is a significant driver of cardiovascular complications and diminished quality of life. While standard hemodialysis (SHD) effectively manages small‐molecule clearance, its ability to remove medium‐to‐large uremic toxins—the primary catalysts of systemic ...
Hongwei Zuo, Weisi Dai, Jie Mao, Mei Ye, Xingshan Lu, Xiaocui Zhang   +5 more
wiley   +1 more source

Exclusive Self Attention

CoRR
We introduce exclusive self attention (XSA), a simple modification of self attention (SA) that improves Transformer's sequence modeling performance. The key idea is to constrain attention to capture only information orthogonal to the token's own value vector (thus excluding information of self position), encouraging better context modeling.
openaire   +3 more sources

A Self-Attentive model for Knowledge Tracing

CoRR, 2019
International Conference on Education Data ...
Shalini Pandey, George Karypis
openaire   +4 more sources

A lightweight transformer with linear self‐attention for defect recognition

Electronics Letters
Visual defect recognition techniques based on deep learning models are crucial for modern industrial quality inspection. The backbone, serving as the primary feature extraction component of the defect recognition model, has not been thoroughly exploited.
Yuwen Zhai, Xinyu Li, Liang Gao, Yiping Gao   +3 more
doaj   +1 more source