About Me

Welcome to Chen Liang’s (Chinese: 梁辰) homepage! I am a senior researcher at Microsoft. Prior to this, I completed my Ph.D. degree in Machine Learning from the Georgia Institute of Technology (Georgia Tech), where I was very fortunate to be advised by Prof. Tuo Zhao. Before that, I obtained my B.S. degree in Electrical Engineering from the University of Southern California (USC).

My research interests are generally centered on machine learning and natural language processing, with a primary focus on improving the efficiency and generalizability of neural language models and vision-language models.

Education

Ph.D in Machine Learning, Georgia Tech, School of Industrial&System Engineering, 2023

M.S in Computational Science&Engineering, Georgia Tech, School of Computational Science&Engineering, 2020

B.S in Electrical Engineering, USC, Department of Electrical&Computer Engineering, 2018

Publications (check full list here)

LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models

Yixiao Li*, Yifan Yu*, Chen Liang, Pengcheng He, Nikos Karampatziakis, Weizhu Chen, Tuo Zhao

The 12th International Conference on Learning Representations (ICLR), 2024
Arxiv Blog Github Ckpt
Module-wise Adaptive Distillation for Multimodality Foundation Models

Chen Liang, Jiahui Yu, Ming-Hsuan Yang, Matthew Brown, Yin Cui, Tuo Zhao, Boqing Gong, Tianyi Zhou

The 37th Conference on Neural Information Processing Systems (NeurIPS), 2023
Arxiv
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation

Yixiao Li*, Yifan Yu*, Qingru Zhang, Chen Liang, Pengcheng He, Weizhu Chen, Tuo Zhao

The 40th International Conference on Machine Learning (ICML), 2023
Arxiv Github
Less is More: Task-aware Layer-wise Distillation for Language Model Compression

Chen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He, Weizhu Chen, Tuo Zhao

The 40th International Conference on Machine Learning (ICML), 2023
Arxiv Github Ckpt
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers

Chen Liang, Haoming Jiang, Zheng Li, Xianfeng Tang, Bing Yin, Tuo Zhao

The 11th International Conference on Learning Representations (ICLR), 2023
Arxiv
PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance

Qingru Zhang, Simiao Zuo, Chen Liang, Alexander Bukharin, Pengcheng He, Weizhu Chen, Tuo Zhao

The 39th International Conference on Machine Learning (ICML), 2022
Arxiv Github
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation

Simiao Zuo, Qingru Zhang, Chen Liang, Pengcheng He, Tuo Zhao and Weizhu Chen

The 2022 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2022
Arxiv Github
CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing

Chen Liang, Pengcheng He, Yelong Shen, Weizhu Chen, Tuo Zhao

The 60th Annual Meeting of the Association for Computational Linguistics (ACL), 2022
Arxiv Github
No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models

Chen Liang, Haoming Jiang, Simiao Zuo, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao

The 10th International Conference on Learning Representations (ICLR), 2022
Arxiv Github
Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach

Simiao Zuo, Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Jianfeng Gao, Weizhu Chen, Tuo Zhao

The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021
Arxiv Github
Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization

Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, Weizhu Chen

The 59th Annual Conference of the Association for Computational Linguistics (ACL), 2021
Arxiv Github

Experience

Senior Researcher, Microsoft, January 2024 – Present
Research Intern, Microsoft, February 2023 – May 2023
Research Intern, Google Research May 2022 – August 2022
Applied Scientist Intern, Amazon, September 2021 – December 2021
Research Intern, Microsoft, May 2021 – July 2021
Deep Learning Software Intern, NVIDIA, May 2018 – August 2018

Teaching & Services

Teaching Assistant, ISyE 3030 Basic Statistics Method,  Georgia Tech,  2020 Fall, 2021 Spring
Teaching Assistant, ISyE 3770 Statistics & Applications,  Georgia Tech,  2020 Summer
Teaching Assistant, CSE 6140 Algorithms,  Georgia Tech,  2019 Fall
Course Producer, EE 364 Introduction to Probability & Statistics for EECS,  USC,  2017 Fall
Reviewer: NeurIPS (2021-Present), ICML (2021-Present), ICLR (2021-Present), EMNLP (2021-2022), ACL (2021-2022), NAACL (2021-2022), EACL (2021), COLING (2021).  

Talks

2023.12. LoftQ: LoRA-Fine-Tuning-Aware Quantization @ NeurIPS Third Workshop on Efficient Natural Language and Speech Processing   
2023.09. On Parameter Efficiency of Neural Language Models @ Allen Institute for AI