Results 11 to 20 of about 153,605 (299)
Do Neural Network Cross-Modal Mappings Really Bridge Modalities? [PDF]
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2018 Feed-forward networks are widely used in cross-modal applications to bridge modalities by mapping distributed vectors of one modality to the other, or to a shared space. The predicted vectors are then used to perform e.g., retrieval or labeling.
Collell Talleda, Guillem +1 more
openaire +5 more sources
What Sound Does That Taste? Cross-Modal Mappings across Gustation and Audition [PDF]
Perception, 2010 All people share implicit mappings across the senses, which give us preferences for certain sensory combinations over others (eg light colours are preferentially paired with higher-pitch sounds; Ward et al, 2006 Cortex42 264–280). Although previous work has tended to focus on the cross-modality of vision with other senses, here we present evidence of ...
Simner, Julia +2 more
openaire +4 more sources
Spatial metaphor in language can promote the development of cross‐modal mappings in children [PDF]
Developmental Science, 2014 AbstractPitch is often described metaphorically: for example, Farsi and Turkish speakers use a ‘thickness’ metaphor (low sounds are ‘thick’ and high sounds are ‘thin’), while German and English speakers use a height metaphor (‘low’, ‘high’). This study examines how child and adult speakers of Farsi, Turkish, and German map pitch and thickness using a ...
Shayan, S. +3 more
openaire +5 more sources
Cross-Modal Sound Mapping Using Deep Learning
New Interfaces for Musical Expression, 2013 We present a method for automatic feature extraction and cross-modal mappingusing deep learning. Our system uses stacked autoencoders to learn a layeredfeature representation of the data. Feature vectors from two (or more)different domains are mapped to each other, effectively creating a cross-modalmapping. Our system can either run fully unsupervised,
Fried, Ohad, Fiebrink, Rebecca
openaire +2 more sources
Cross-modal Memory Networks for Radiology Report Generation [PDF]
Annual Meeting of the Association for Computational Linguistics, 2022 Medical imaging plays a significant role in clinical practice of medical diagnosis, where the text reports of the images are essential in understanding them and facilitating later treatments.
Zhihong Chen +3 more
semanticscholar +1 more source
Is Cross-Modal Information Retrieval Possible Without Training? [PDF]
European Conference on Information Retrieval, 2023 Encoded representations from a pretrained deep learning model (e.g., BERT text embeddings, penultimate CNN layer activations of an image) convey a rich set of features beneficial for information retrieval.
Hyunjin Choi +3 more
semanticscholar +1 more source
Pix2Map: Cross-Modal Retrieval for Inferring Street Maps from Images
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023 12 pages, 8 ...
Wu, Xindi +4 more
openaire +2 more sources
Space oddity: musical syntax is mapped onto visual space
Scientific Reports, 2021 Increasing evidence has uncovered associations between the cognition of abstract schemas and spatial perception. Here we examine such associations for Western musical syntax, tonality.
Neta B. Maimon +2 more
doaj +1 more source
Cross-modal Map Learning for Vision and Language Navigation
2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022 We consider the problem of Vision-and-Language Navigation (VLN). The majority of current methods for VLN are trained end-to-end using either unstructured memory such as LSTM, or using cross-modal attention over the egocentric observations of the agent.
Georgakis, Georgios +6 more
openaire +2 more sources
Deep Perceptual Mapping for Cross-Modal Face Recognition [PDF]
International Journal of Computer Vision, 2016 This is the extended version (invited IJCV submission) with new results of our previous submission (arXiv:1507.02879)
M. Saquib Sarfraz, Rainer Stiefelhagen
openaire +3 more sources