About Me

Welcome to Chen Liang’s (Chinese: 梁辰) homepage! I am a senior researcher at Microsoft. Prior to this, I completed my Ph.D. degree in Machine Learning from the Georgia Institute of Technology (Georgia Tech), advised by Prof. Tuo Zhao. I obtained my B.S. degree in Electrical Engineering from the University of Southern California (USC).

My research interests are generally centered on machine learning and natural language processing, with a primary focus on improving the efficiency and generalizability of neural language models and vision-language models.

Education

Ph.D in Machine Learning, Georgia Tech, School of Industrial&System Engineering, 2023

M.S in Computational Science&Engineering, Georgia Tech, School of Computational Science&Engineering, 2020

B.S in Electrical Engineering, USC, Department of Electrical&Computer Engineering, 2018

Publications

LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models

Yixiao Li*, Yifan Yu*, Chen Liang, Pengcheng He, Nikos Karampatziakis, Weizhu Chen, Tuo Zhao

The 12th International Conference on Learning Representations (ICLR), 2024 (Oral)
Arxiv Github
Module-wise Adaptive Distillation for Multimodality Foundation Models

Chen Liang, Jiahui Yu, Ming-Hsuan Yang, Matthew Brown, Yin Cui, Tuo Zhao, Boqing Gong, Tianyi Zhou

The 37th Conference on Neural Information Processing Systems (NeurIPS), 2023
Arxiv
Less is More: Task-aware Layer-wise Distillation for Language Model Compression

Chen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He, Weizhu Chen, Tuo Zhao

The 40th International Conference on Machine Learning (ICML), 2023
Arxiv Github
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation

Yixiao Li*, Yifan Yu*, Qingru Zhang, Chen Liang, Pengcheng He, Weizhu Chen, Tuo Zhao

The 40th International Conference on Machine Learning (ICML), 2023
Arxiv Github
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers

Chen Liang, Haoming Jiang, Zheng Li, Xianfeng Tang, Bing Yin, Tuo Zhao

The 11th International Conference on Learning Representations (ICLR), 2023
Arxiv
PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance

Qingru Zhang, Simiao Zuo, Chen Liang, Alexander Bukharin, Pengcheng He, Weizhu Chen, Tuo Zhao

The 39th International Conference on Machine Learning (ICML), 2022
Arxiv Github
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation

Simiao Zuo, Qingru Zhang, Chen Liang, Pengcheng He, Tuo Zhao and Weizhu Chen

The 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2022
Arxiv Github
Self-Training with Differentiable Teacher

Simiao Zuo, Yue Yu, Chen Liang, Haoming Jiang, Siawpeng Er, Chao Zhang, Tuo Zhao, Hongyuan Zha

The 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL Findings), 2022
Arxiv
CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing

Chen Liang, Pengcheng He, Yelong Shen, Weizhu Chen, Tuo Zhao

The 60th Annual Meeting of the Association for Computational Linguistics (ACL), 2022
Arxiv Github
No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models

Chen Liang, Haoming Jiang, Simiao Zuo, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao

The Tenth International Conference on Learning Representations (ICLR), 2022
Arxiv Github
Adversarial Training as Stackelberg Game: An Unrolled Optimization Approach

Simiao Zuo, Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Jianfeng Gao, Weizhu Chen, Tuo Zhao

The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021
Arxiv Github
ARCH: Efficient Adversarial Regularized Training with Caching

Simiao Zuo, Chen Liang, Haoming Jiang, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao

The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP Findings), 2021
Arxiv Github
Token-wise Curriculum Learning for Neural Machine Translation

Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao, Tuo Zhao

The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP Findings), 2021
Arxiv
Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization

Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, Weizhu Chen

The 59th Annual Conference of the Association for Computational Linguistics (ACL), 2021
Arxiv Github
BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision

Chen Liang*, Yue Yu*, Haoming Jiang*, Siawpeng Er, Ruijia Wang, Tuo Zhao, Chao Zhang

The 26th SIGKDD Conference on Knowledge Discovery and Pattern Mining (KDD), 2020
Arxiv Github
Multi-Domain Neural Machine Translation with Word-Level Adaptive Layer-wise Domain Mixing

Haoming Jiang, Chen Liang, Chong Wang, Tuo Zhao

The 58th Annual Conference of the Association for Computational Linguistics (ACL), 2020
Arxiv Github
A Fully Convolutional Tri-branch Network (FCTN) for Domain Adaptation

Juning Zhang, Chen Liang, C.-C Jay Kuo

International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018
Arxiv

Experience

Senior Researcher, Microsoft, January 2024 – Present
Research Intern, Microsoft, February 2023 – May 2023
Research Intern, Google Research May 2022 – August 2022
Applied Scientist Intern, Amazon, September 2021 – December 2021
Research Intern, Microsoft, May 2021 – July 2021
Deep Learning Software Intern, NVIDIA, May 2018 – August 2018

Teaching & Services

Teaching Assistant, ISyE 3770 Statistics & Applications,  Georgia Tech,  2020 Summer
Teaching Assistant, CSE 6140 Algorithms,  Georgia Tech,  2019 Fall
Course Producer, EE 364 Introduction to Probability & Statistics for EECS,  USC,  2017 Fall
Reviewer: NeurIPS (2021-Present), ICML (2021-Present), ICLR (2021-Present), EMNLP (2021-2022), ACL (2021-2022), NAACL (2021-2022), EACL (2021), COLING (2021).  

Talks

2023.12. LoftQ: LoRA-Fine-Tuning-Aware Quantization @ NeurIPS Third Workshop on Efficient Natural Language and Speech Processing   
2023.09. On Parameter Efficiency of Neural Language Models @ Allen Institute for AI