Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models - Tang, Anke; Shen, Li; Luo, Yong; Xie, Shuai; Hu, Han; Zhang, Lefei; Du, Bo; Tao, Dacheng (IEEE transactions on pattern analysis and machine intelligence, 2025)
Keywords: Adaptation models; Computational modeling; Electronic mail; Interference; Large Language Model; Matrix decomposition; Mixture of Experts; Model Fusion; Optimization; Predictive models; Subspace Decomposition; Training; Tuning; Vectors