MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation

Simiao Zuo, Qingru Zhang, Chen Liang, Pengcheng He, Tuo Zhao and Weizhu Chen
April 2022
Arxiv Github
Type
Conference paper
Publication
The 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2022