About Me

Welcome to Chen Liang’s (Chinese: 梁辰) homepage! I am a Senior Researcher at Microsoft, working on LLM training and adaptation (e.g., Github Copilot, Phi-MoE). Prior to this, I completed my Ph.D. degree in Machine Learning from the Georgia Institute of Technology (Georgia Tech), where I was very fortunate to be advised by Prof. Tuo Zhao. Before that, I obtained my B.S. degree in Electrical Engineering from the University of Southern California (USC).

My research interests are generally centered on machine learning and natural language processing, with a primary focus on improving the efficiency and generalizability of neural language models. Feel free to shoot me an email if you’d like to chat about training methods (e.g., pre-training, distillation, PEFT, RL), data strategies (e.g., data generation, data selection), or seek part-time intern opportunities.

Education

Ph.D in Machine Learning, Georgia Tech, School of Industrial&System Engineering, December 2023

M.S in Computational Science&Engineering, Georgia Tech, School of Computational Science&Engineering, May 2020

B.S in Electrical Engineering, USC, Department of Electrical&Computer Engineering, May 2018

Publications (check full list here)

LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models

Yixiao Li*, Yifan Yu*, Chen Liang, Pengcheng He, Nikos Karampatziakis, Weizhu Chen, Tuo Zhao

The 12th International Conference on Learning Representations (ICLR), 2024

Arxiv Microsoft Blog Ckpt Github

Module-wise Adaptive Distillation for Multimodality Foundation Models

Chen Liang, Jiahui Yu, Ming-Hsuan Yang, Matthew Brown, Yin Cui, Tuo Zhao, Boqing Gong, Tianyi Zhou

The 37th Conference on Neural Information Processing Systems (NeurIPS), 2023

Arxiv

LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation

Yixiao Li*, Yifan Yu*, Qingru Zhang, Chen Liang, Pengcheng He, Weizhu Chen, Tuo Zhao

The 40th International Conference on Machine Learning (ICML), 2023

Arxiv Github

Less is More: Task-aware Layer-wise Distillation for Language Model Compression

Chen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He, Weizhu Chen, Tuo Zhao

The 40th International Conference on Machine Learning (ICML), 2023

Arxiv Github Ckpt

HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers

Chen Liang, Haoming Jiang, Zheng Li, Xianfeng Tang, Bing Yin, Tuo Zhao

The 11th International Conference on Learning Representations (ICLR), 2023

Arxiv

PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance

Qingru Zhang, Simiao Zuo, Chen Liang, Alexander Bukharin, Pengcheng He, Weizhu Chen, Tuo Zhao

The 39th International Conference on Machine Learning (ICML), 2022

Arxiv Github

No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models

Chen Liang, Haoming Jiang, Simiao Zuo, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao

The 10th International Conference on Learning Representations (ICLR), 2022

Arxiv Github

CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing

Chen Liang, Pengcheng He, Yelong Shen, Weizhu Chen, Tuo Zhao

The 60th Annual Meeting of the Association for Computational Linguistics (ACL), 2022

Arxiv Github

Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach

Simiao Zuo, Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Jianfeng Gao, Weizhu Chen, Tuo Zhao

The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021

Arxiv Github

Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization

Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, Weizhu Chen

The 59th Annual Conference of the Association for Computational Linguistics (ACL), 2021

Arxiv Github

About Me

Education

Publications (check full list here)

Experience

Teaching & Services

Talks