About Me

Welcome to Chen Liang (Chinese: 梁辰)‘s homepage! I am a third-year student in the Machine Learning Ph.D Program at Georgia Institute of Technology (Georgia Tech). I am very fortunate to work with Prof. Tuo Zhao in the FLASH (Foundations of LeArning Systems for alcHemy) research group. I received my M.S degree in Computational Science & Engineering from Georgia Tech, and received my B.S degree in Electrical Engineering from University of Southern California (USC). My undergrad advisor is Prof. C.-C Jay Kuo.

I am generally interested in machine learning for natural language processing. My research mainly focuses on developing methodologies and algorithms to improve parameter efficiency and model generalization of neural language models. My interests also include transfer learning and representation learning (e.g., multi-domain and multi-task learning).

Education

Ph.D in Machine Learning, Georgia Tech, School of Industrial&System Engineering, 2023 (Expected)

M.S in Computational Science&Engineering, Georgia Tech, School of Computational Science&Engineering, 2020

B.S in Electrical Engineering, USC, Department of Electrical&Computer Engineering, 2018

Publications

Less is More: Task-aware Layer-wise Distillation for Language Model Compression

Chen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He, Weizhu Chen, Tuo Zhao

The 40th International Conference on Machine Learning (ICML), 2023
Arxiv Github
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation

Yixiao Li*, Yifan Yu*, Qingru Zhang, Chen Liang, Pengcheng He, Weizhu Chen, Tuo Zhao

The 40th International Conference on Machine Learning (ICML), 2023
PDF
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers

Chen Liang, Haoming Jiang, Zheng Li, Xianfeng Tang, Bing Yin, Tuo Zhao

The 11th International Conference on Learning Representations (ICLR), 2023
Arxiv
PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance

Qingru Zhang, Simiao Zuo, Chen Liang, Alexander Bukharin, Pengcheng He, Weizhu Chen, Tuo Zhao

The 39th International Conference on Machine Learning (ICML), 2022
Arxiv Github
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation

Simiao Zuo, Qingru Zhang, Chen Liang, Pengcheng He, Tuo Zhao and Weizhu Chen

The 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2022
Arxiv Github
Self-Training with Differentiable Teacher

Simiao Zuo, Yue Yu, Chen Liang, Haoming Jiang, Siawpeng Er, Chao Zhang, Tuo Zhao, Hongyuan Zha

The 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL Findings), 2022
Arxiv
CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing

Chen Liang, Pengcheng He, Yelong Shen, Weizhu Chen, Tuo Zhao

The 60th Annual Meeting of the Association for Computational Linguistics (ACL), 2022
Arxiv Github
No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models

Chen Liang, Haoming Jiang, Simiao Zuo, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao

The Tenth International Conference on Learning Representations (ICLR), 2022
Arxiv Github
Adversarial Training as Stackelberg Game: An Unrolled Optimization Approach

Simiao Zuo, Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Jianfeng Gao, Weizhu Chen, Tuo Zhao

The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021
Arxiv Github
ARCH: Efficient Adversarial Regularized Training with Caching

Simiao Zuo, Chen Liang, Haoming Jiang, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao

The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP Findings), 2021
Arxiv Github
Token-wise Curriculum Learning for Neural Machine Translation

Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao, Tuo Zhao

The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP Findings), 2021
Arxiv
Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization

Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, Weizhu Chen

The 59th Annual Conference of the Association for Computational Linguistics (ACL), 2021
Arxiv Github
BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision

Chen Liang*, Yue Yu*, Haoming Jiang*, Siawpeng Er, Ruijia Wang, Tuo Zhao, Chao Zhang

The 26th SIGKDD Conference on Knowledge Discovery and Pattern Mining (KDD), 2020
Arxiv Github
Multi-Domain Neural Machine Translation with Word-Level Adaptive Layer-wise Domain Mixing

Haoming Jiang, Chen Liang, Chong Wang, Tuo Zhao

The 58th Annual Conference of the Association for Computational Linguistics (ACL), 2020
Arxiv Github
A Fully Convolutional Tri-branch Network (FCTN) for Domain Adaptation

Juning Zhang, Chen Liang, C.-C Jay Kuo

International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018
Arxiv

Experience

Research Intern, Microsoft Azure AI, February 2023 – May 2023
Research Intern, Google Research, May 2022 – August 2022
Applied Scientist Intern, Amazon Search, September 2021 – December 2021
Research Intern, Microsoft Azure AI, May 2021 – July 2021
Software Development Intern, Amazon, May 2019 – July 2019
Deep Learning Software Intern, NVIDIA, May 2018 – August 2018

Teaching & Services

Teaching Assistant, ISyE 3770 Statistics & Applications,  Georgia Tech,  2020 Summer
Teaching Assistant, CSE 6140 Algorithms,  Georgia Tech,  2019 Fall
Course Producer, EE 364 Introduction to Probability & Statistics for EECS,  USC,  2017 Fall
Reviewers: NeurIPS, ICLR, ICML, ACL, EMNLP, NAACL, COLING, EACL, WACV