Results 1 to 10 of about 248,393 (200)

FedMed: A Federated Learning Framework for Language Modeling [PDF]

Sensors, 2020
Federated learning (FL) is a privacy-preserving technique for training a vast amount of decentralized data and making inferences on mobile devices. As a typical language modeling problem, mobile keyboard prediction aims at suggesting a probable next word
Xing Wu, Zhaowang Liang, Jianjia Wang
doaj   +2 more sources

The sociolinguistic foundations of language modeling [PDF]

Frontiers in Artificial Intelligence
In this article, we introduce a sociolinguistic perspective on language modeling. We claim that language models in general are inherently modeling varieties of language, and we consider how this insight can inform the development and deployment of ...
Jack Grieve, Sara Bartl, Matteo Fuoli, Jason Grafmiller, Weihang Huang, Alejandro Jawerbaum, Akira Murakami, Marcus Perlman, Dana Roemling, Bodo Winter   +9 more
doaj   +2 more sources

A Sequential and Intensive Weighted Language Modeling Scheme for Multi-Task Learning-Based Natural Language Understanding

Applied Sciences, 2021
Multi-task learning (MTL) approaches are actively used for various natural language processing (NLP) tasks. The Multi-Task Deep Neural Network (MT-DNN) has contributed significantly to improving the performance of natural language understanding (NLU ...
Suhyune Son, Seonjeong Hwang, Sohyeun Bae, Soo Jun Park, Jang-Hwan Choi   +4 more
doaj   +1 more source

Next-Generation Neural Networks: Capsule Networks With Routing-by-Agreement for Text Classification

IEEE Access, 2021
These days, neural networks constantly prove their high capacity for nearly every application case and are considered as key technology for learning systems.
Nikolai A. K. Steur, Friedhelm Schwenker
doaj   +1 more source

Contemporary Approaches in Evolving Language Models

Applied Sciences, 2023
This article provides a comprehensive survey of contemporary language modeling approaches within the realm of natural language processing (NLP) tasks.
Dina Oralbekova, Orken Mamyrbayev, Mohamed Othman, Dinara Kassymova, Kuralai Mukhsina   +4 more
doaj   +1 more source

Plain Template Insertion: Korean-Prompt-Based Engineering for Few-Shot Learners

IEEE Access, 2022
Prompt-based learning is a method used for language models to interpret natural language by remembering the prior knowledge acquired and the training objective.
Jaehyung Seo, Hyeonseok Moon, Chanhee Lee, Sugyeong Eo, Chanjun Park, Jihoon Kim, Changwoo Chun, Heuiseok Lim   +7 more
doaj   +1 more source

Automatic story and item generation for reading comprehension assessments with transformers

International Journal of Assessment Tools in Education, 2022
Reading comprehension is one of the essential skills for students as they make a transition from learning to read to reading to learn. Over the last decade, the increased use of digital learning materials for promoting literacy skills (e.g., oral fluency
Seyma Nur Yildirim-erbasli, Okan Bulut
doaj   +1 more source

Red Teaming Language Models with Language Models

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022
Language Models (LMs) often cannot be deployed because of their potential to harm users in hard-to-predict ways. Prior work identifies harmful behaviors before deployment by using human annotators to hand-write test cases. However, human annotation is expensive, limiting the number and diversity of test cases.
Ethan Perez, Saffron Huang, H. Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, Geoffrey Irving   +8 more
openaire   +2 more sources

New RNN Activation Technique for Deeper Networks: LSTCM Cells

IEEE Access, 2020
Long short-term memory (LSTM) has shown good performance when used with sequential data, but gradient vanishing or exploding problem can arise, especially when using deeper layers to solve complex problems. Thus, in this paper, we propose a new LSTM cell
Soo-Han Kang, Ji-Hyeong Han
doaj   +1 more source

Language Models are not Models of Language.

CoRR, 2021
Natural Language Processing (NLP) has become one of the leading application areas in the current Artificial Intelligence boom. Transfer learning has enabled large deep learning neural networks trained on the language modeling task to vastly improve performance in almost all downstream language tasks.
openaire   +7 more sources