ByteTransformer: A High-Performance Transformer Boosted for Variable-Length Inputs - Zhai, Yujia; Jiang, Chengquan; Wang, Leyuan; Jia, Xiaoying; Zhang, Shang; Chen, Zizhong; Liu, Xin; Zhu, Yibo (Proceedings - IEEE International Parallel and Distributed Processing Symposium, 2023)
Keywords: BERT; Bit error rate; CUTLASS; Deep learning; Distributed processing; Graphics processing units; Large Language Models; Multi-head Attention; Natural Language Processing; NVIDIA GPU; Optimization methods; Technological innovation; Training; Transformer