Results 121 to 130 of about 126,488 (157)

Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark

International Conference on Machine Learning
In the evolving landscape of natural language processing (NLP), fine-tuning pre-trained Large Language Models (LLMs) with first-order (FO) optimizers like SGD and Adam has become standard. Yet, as LLMs grow {in size}, the substantial memory overhead from
Yihua Zhang, Pingzhi Li, Junyuan Hong, Jiaxiang Li, Yimeng Zhang, Wenqing Zheng, Pin-Yu Chen, Jason D. Lee, Wotao Yin, Mingyi Hong, Zhangyang Wang, Sijia Liu, Tianlong Chen   +12 more
semanticscholar   +1 more source

A resonant metamaterial clock distribution network for superconducting logic

Nature Electronics, 2022
J. Strong, V. Talanov, Max E. Nielsen, Andrew Brownfield, N. Bailey, Q. Herr, A. Herr   +6 more
semanticscholar   +1 more source

Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning

arXiv.org
While fine-tuning large language models (LLMs) for specific tasks often yields impressive results, it comes at the cost of memory inefficiency due to back-propagation in gradient-based training.
Yong Liu, Zirui Zhu, Chaoyu Gong, Minhao Cheng, Cho-Jui Hsieh, Yang You   +5 more
semanticscholar   +1 more source

Private Fine-tuning of Large Language Models with Zeroth-order Optimization

Trans. Mach. Learn. Res.
Differentially private stochastic gradient descent (DP-SGD) allows models to be trained in a privacy-preserving manner, but has proven difficult to scale to the era of foundation models.
Xinyu Tang, Ashwinee Panda, Milad Nasr, Saeed Mahloujifar, Prateek Mittal   +4 more
semanticscholar   +1 more source

Second-Order Fine-Tuning without Pain for LLMs: A Hessian Informed Zeroth-Order Optimizer

International Conference on Learning Representations
Fine-tuning large language models (LLMs) with classic first-order optimizers entails prohibitive GPU memory due to the backpropagation process. Recent works have turned to zeroth-order optimizers for fine-tuning, which save substantial memory by using ...
Yanjun Zhao, Sizhe Dang, Haishan Ye, Guang Dai, Yi Qian, I. Tsang   +5 more
semanticscholar   +1 more source

The monotonicity and convexity of unnormalized interval type-2 TSK Fuzzy Logic Systems

International Conference on Fuzzy Systems, 2010
Tiechao Wang, J. Yi, Chengdong Li
semanticscholar   +1 more source

Bipolar Sets and YinYang Bipolar Dynamic Logic ( BDL )

, 2016

semanticscholar   +1 more source

The Effect of Spatial Variations in Voltage and Temperature in Emitter Coupled Logic Systems

IEEE Transactions on Electronic Computers, 1966
H. Josephs
semanticscholar   +1 more source

A Model of Tolerance

Studia Logica: An International Journal for Symbolic Logic, 2008
E. Zardini
semanticscholar   +1 more source

Boolean Algebras

, 1998
Asher M. Kach
semanticscholar   +1 more source