COSMOS: A Hybrid Adaptive Optimizer for Memory-Efficient Training of LLMs

Liming Liu, Zhenghao Xu, Zixuan Zhang, Hao Kang, Zichong Li, Chen Liang, Weizhu Chen, Tuo Zhao
February 2026
Arxiv Github
Type
Preprint
Publication
The 14th International Conference on Learning Representations (ICLR), 2026