Efficient Memory Management for Large Language Model Serving with PagedAttention [PDF]
Symposium on Operating Systems Principles, 2023 High throughput serving of large language models (LLMs) requires batching sufficiently many requests at a time. However, existing systems struggle because the key-value cache (KV cache) memory for each request is huge and grows and shrinks dynamically ...
Woosuk Kwon +8 more
semanticscholar +1 more source
Immunological memory to SARS-CoV-2 assessed for up to 8 months after infection
Science, 2021 Variable memory Immune memory against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) helps to determine protection against reinfection, disease risk, and vaccine efficacy.
J. Dan +20 more
semanticscholar +1 more source
Ohm-GPU: Integrating New Optical Network and Heterogeneous Memory into GPU Multi-Processors [PDF]
, 2021 Traditional graphics processing units (GPUs) suffer from the low memory capacity and demand for high memory bandwidth. To address these challenges, we propose Ohm-GPU, a new optical network based heterogeneous memory design for GPUs. Specifically, Ohm-GPU can expand the memory capacity by combing a set of high-density 3D XPoint and DRAM modules as ...
arxiv +1 more source
Design and Evaluation of a Rack-Scale Disaggregated Memory Architecture For Data Centers [PDF]
, 2023 Memory disaggregation is being considered as a strong alternative to traditional architecture to deal with the memory under-utilization in data centers. Disaggregated memory can adapt to dynamically changing memory requirements for the data center applications like data analytics, big data, etc., that require in-memory processing. However, such systems
arxiv +1 more source
Ultrafast and memory-efficient alignment of short DNA sequences to the human genome
Genome Biology, 2009 Bowtie is an ultrafast, memory-efficient alignment program for aligning short DNA sequence reads to large genomes. For the human genome, Burrows-Wheeler indexing allows Bowtie to align more than 25 million reads per CPU hour with a memory footprint of ...
Ben Langmead +3 more
semanticscholar +1 more source
SOAPdenovo2: an empirically improved memory-efficient short-read de novo assembler
GigaScience, 2012 BackgroundThere is a rapidly increasing amount of de novo genome assembly using next-generation sequencing (NGS) short reads; however, several big challenges remain to be overcome in order for this to be efficient and accurate.
Ruibang Luo +29 more
semanticscholar +1 more source
The magical number 4 in short-term memory: A reconsideration of mental storage capacity
Behavioral and Brain Sciences, 2001 Miller (1956) summarized evidence that people can remember about seven chunks in short-term memory (STM) tasks. However, that number was meant more as a rough estimate and a rhetorical device than as a real capacity limit.
N. Cowan
semanticscholar +1 more source
A Limited Memory Algorithm for Bound Constrained Optimization
SIAM Journal on Scientific Computing, 1995 An algorithm for solving large nonlinear optimization problems with simple bounds is described. It is based on the gradient projection method and uses a limited memory BFGS matrix to approximate the Hessian of the objective function.
R. Byrd, P. Lu, J. Nocedal, C. Zhu
semanticscholar +1 more source
LOSS OF RECENT MEMORY AFTER BILATERAL HIPPOCAMPAL LESIONS
Journal of Neurology Neurosurgery & Psychiatry, 1957 Bilateral medial temporal lobe resection in man results in a persistent impairment of recent memory whenever the removal is carried far enough posteriorly to damage portions of the anterior hippocampus and hippocampal gyrus.
W. Scoville, B. Milner
semanticscholar +1 more source
Memorizing Normality to Detect Anomaly: Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection [PDF]
IEEE International Conference on Computer Vision, 2019 Deep autoencoder has been extensively used for anomaly detection. Training on the normal data, the autoencoder is expected to produce higher reconstruction error for the abnormal inputs than the normal ones, which is adopted as a criterion for ...
Dong Gong +6 more
semanticscholar +1 more source