Shared computational principles for language processing in humans and deep language models
Departing from traditional linguistic models, advances in deep learning have resulted in a new type of predictive (autoregressive) deep language models (DLMs).
Ariel Goldstein +31 more
semanticscholar +1 more source
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing [PDF]
Motivated by the success of T5 (Text-To-Text Transfer Transformer) in pre-trained natural language processing models, we propose a unified-modal SpeechT5 framework that explores the encoder-decoder pre-training for self-supervised speech/text ...
Junyi Ao +12 more
semanticscholar +1 more source
Linearity in process languages
The meaning and mathematical consequences of linearity (managing without a presumed ability to copy) are studied for a path-based model of processes which is also a model of affine-linear logic. This connection yields an affine-linear language for processes, automatically respecting open-map bisimulation, in which a range of process operations can be ...
Nygaard, Mikkel, Winskel, Glynn
openaire +3 more sources
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing [PDF]
This paper describes SentencePiece, a language-independent subword tokenizer and detokenizer designed for Neural-based text processing, including Neural Machine Translation. It provides open-source C++ and Python implementations for subword units.
Taku Kudo, John Richardson
semanticscholar +1 more source
Application of Tensor Train Decomposition in S2VT Model for Sign Language Recognition
Sign language recognition is a conversion of sign language into text or speech, bridging the communication between the hearing and society. Recently, sequence-to-sequence video to text (S2VT) models has been employed in the field of sign language ...
Biao Xu, Shiliang Huang, Zhongfu Ye
doaj +1 more source
The Stanford CoreNLP Natural Language Processing Toolkit
We describe the design and use of the Stanford CoreNLP toolkit, an extensible pipeline that provides core natural language analysis. This toolkit is quite widely used, both in the research NLP community and also among commercial and government users of ...
Christopher D. Manning +5 more
semanticscholar +1 more source
Natural language processing applied to mental illness detection: a narrative review
Mental illness is highly prevalent nowadays, constituting a major cause of distress in people’s life with impact on society’s health and well-being. Mental illness is a complex multi-factorial disease associated with individual risk factors and a variety
Tianlin Zhang +3 more
semanticscholar +1 more source
A Survey of Active Learning for Natural Language Processing [PDF]
In this work, we provide a literature review of active learning (AL) for its applications in natural language processing (NLP). In addition to a fine-grained categorization of query strategies, we also investigate several other important aspects of ...
Zhisong Zhang, Emma Strubell, E. Hovy
semanticscholar +1 more source
Fine-tuning large neural language models for biomedical natural language processing [PDF]
Summary Large neural language models have transformed modern natural language processing (NLP) applications. However, fine-tuning such models for specific tasks remains challenging as model size increases, especially with small labeled datasets, which ...
Robert Tinn +7 more
semanticscholar +1 more source
Morphological paradigms in language processing and language disorders [PDF]
We present results from two cross-modal morphological priming experiments investigating regular person and number inflection on finite verbs in German. We found asymmetries in the priming patterns between different affixes that can be predicted from the ...
Harald Clahsen +4 more
core +4 more sources

