Results 11 to 20 of about 6,804,949 (307)
The Impact of Positional Encoding on Length Generalization in Transformers [PDF]
Neural Information Processing Systems, 2023 Length generalization, the ability to generalize from small training context sizes to larger ones, is a critical challenge in the development of Transformer-based language models. Positional encoding (PE) has been identified as a major factor influencing
Amirhossein Kazemnejad +4 more
semanticscholar +1 more source
A Long Way to Go: Investigating Length Correlations in RLHF [PDF]
arXiv.org, 2023 Great success has been reported using Reinforcement Learning from Human Feedback (RLHF) to align large language models, with open preference datasets enabling wider experimentation, particularly for"helpfulness"in tasks like dialogue and web question ...
Prasann Singhal +3 more
semanticscholar +1 more source
Phenaki: Variable Length Video Generation From Open Domain Textual Description [PDF]
International Conference on Learning Representations, 2022 We present Phenaki, a model capable of realistic video synthesis, given a sequence of textual prompts. Generating videos from text is particularly challenging due to the computational cost, limited quantities of high quality text-video data and variable ...
Ruben Villegas +8 more
semanticscholar +1 more source
Transformer-XL: Attentive Language Models beyond a Fixed-Length Context [PDF]
Annual Meeting of the Association for Computational Linguistics, 2019 Transformers have a potential of learning longer-term dependency, but are limited by a fixed-length context in the setting of language modeling. We propose a novel neural architecture Transformer-XL that enables learning dependency beyond a fixed length ...
Zihang Dai +5 more
semanticscholar +1 more source
Exploring Length Generalization in Large Language Models [PDF]
Neural Information Processing Systems, 2022 The ability to extrapolate from short problem instances to longer ones is an important form of out-of-distribution generalization in reasoning tasks, and is crucial when learning from datasets where longer problem instances are rare.
Cem Anil +9 more
semanticscholar +1 more source
Structural basis for the recognition of SARS-CoV-2 by full-length human ACE2
Science, 2020 How SARS-CoV-2 binds to human cells Scientists are racing to learn the secrets of severe acute respiratory syndrome–coronavirus 2 (SARS-CoV-2), which is the cause of the pandemic disease COVID-19. The first step in viral entry is the binding of the viral
R. Yan +5 more
semanticscholar +1 more source
A Length-Extrapolatable Transformer [PDF]
Annual Meeting of the Association for Computational Linguistics, 2022 Position modeling plays a critical role in Transformers. In this paper, we focus on length extrapolation, i.e., training on short texts while evaluating longer sequences. We define attention resolution as an indicator of extrapolation.
Yutao Sun +8 more
semanticscholar +1 more source
De novo assembly of the Aedes aegypti genome using Hi-C yields chromosome-length scaffolds [PDF]
Science, 2016 Hi-C for mosquito genomes Most genomes sequenced today are determined through the generation of short sequenced bits of DNA that are computationally pieced together like a jigsaw puzzle. This has resulted in the need for funds and additional data to fill
O. Dudchenko +10 more
semanticscholar +1 more source
Polygenic basis and biomedical consequences of telomere length variation
Nature Genetics, 2021 Telomeres, the end fragments of chromosomes, play key roles in cellular proliferation and senescence. Here we characterize the genetic architecture of naturally occurring variation in leukocyte telomere length (LTL) and identify causal links between LTL ...
V. Codd +27 more
semanticscholar +1 more source
Depth in Coxeter groups of type $B$ [PDF]
Discrete Mathematics & Theoretical Computer Science, 2015 The depth statistic was defined for every Coxeter group in terms of factorizations of its elements into product of reflections. Essentially, the depth gives the minimal path cost in the Bruaht graph, where the edges have prescribed weights. We present an
Eli Bagno +2 more
doaj +1 more source