Results 11 to 20 of about 43,272 (165)
Multimodal transformer augmented fusion for speech emotion recognition
Frontiers in Neurorobotics, 2023 Speech emotion recognition is challenging due to the subjectivity and ambiguity of emotion. In recent years, multimodal methods for speech emotion recognition have achieved promising results.
Yuanyuan Wang +7 more
doaj +1 more source
Attention Bottlenecks for Multimodal Fusion
CoRR, 2021 Published at NeurIPS 2021. Note this version updates numbers due to a bug in the AudioSet mAP calculation in Table 1 (last row)
Arsha Nagrani +5 more
openaire +3 more sources
Contrastive Multimodal Fusion with TupleInfoNCE [PDF]
2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021 This paper proposes a method for representation learning of multimodal data using contrastive losses. A traditional approach is to contrast different modalities to learn the information shared between them. However, that approach could fail to learn the complementary synergies between modalities that might be useful for downstream tasks.
Yunze Liu +5 more
openaire +2 more sources
IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023 Hand gesture recognition (HGR) based on surface electromyogram (sEMG) and Accelerometer (ACC) signals is increasingly attractive where fusion strategies are crucial for performance and remain challenging.
Shengcai Duan +3 more
doaj +1 more source
Survey of Multimodal Data Fusion Research [PDF]
Jisuanji kexue yu tansuoAlthough the powerful learning ability of deep learning has achieved excellent results in the field of single-modal applications, it has been found that the feature representation of a single modality is difficult to fully contain the complete ...
ZHANG Hucheng, LI Leixiao, LIU Dongjiang
doaj +1 more source
Research on a Microexpression Recognition Technology Based on Multimodal Fusion
Complexity, 2021 Microexpressions have extremely high due value in national security, public safety, medical, and other fields. However, microexpressions have characteristics that are obviously different from macroexpressions, such as short duration and weak changes ...
Jie Kang +5 more
doaj +1 more source
Developments in the Built Environment, 2023 Construction equipment operations that require high levels of attention can cause mental fatigue, which can lead to inefficiencies and accidents. Previous studies classified mental fatigue using single-modal data with acceptable accuracy. However, mental
Imran Mehmood +7 more
doaj +1 more source
Data fusion methods in multimodal human computer dialog
Virtual Reality & Intelligent Hardware, 2019 In multimodal human computer dialog, non-verbal channels, such as facial expression, posture, gesture, etc, combined with spoken information, are also important in the procedure of dialogue. Nowadays, in spite of high performance of users’ single channel
Ming-Hao YANG, Jian-Hua TAO
doaj +1 more source
Deep Equilibrium Multimodal Fusion
CoRR, 2023 Multimodal fusion integrates the complementary information present in multiple modalities and has gained much attention recently. Most existing fusion approaches either learn a fixed fusion strategy during training and inference, or are only capable of fusing the information to a certain extent.
Jinhong Ni +4 more
openaire +2 more sources
On Consistent Fusion of Multimodal Biometrics [PDF]
2006 IEEE International Conference on Acoustics Speed and Signal Processing Proceedings, 2006 Audio-visual (AV) biometrics offer complementary information sources, and the use of both voice and facial images for biometric authentication has recently become economically feasible. Therefore, multi-modality adaptive fusion, combining audio and visual information, offers an efficient tool for substantially improving the classification performance ...
Sun-Yuan Kung, Man-Wai Mak
openaire +1 more source