Ma, Shuming

Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers

Dai, D., Sun, Y., Dong, L., Hao, Y., Ma, S., Sui, Z., & Wei, F. (2023) · Findings of the Association for Computational Linguistics: ACL 2023

Topic ModelingDomain Adaptation and Few-Shot Learning

Cited by 154

Retentive Network: A Successor to Transformer for Large Language Models

Sun, Y., Dong, L., Huang, S., Ma, S., Xia, Y., Xue, J., Wang, J., & Wei, F. (2023) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 142

DeepNet: Scaling Transformers to 1,000 Layers

Wang, H., Ma, S., Dong, L., Huang, S., Zhang, D., & Wei, F. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence

Topic ModelingNatural Language Processing Techniques

Cited by 113

A Length-Extrapolatable Transformer

Sun, Y., Dong, L., Patra, B., Ma, S., Huang, S., Benhaim, A., Chaudhary, V., Song, X., & Wei, F. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Topic ModelingNatural Language Processing Techniques

Cited by 59

LongNet: Scaling Transformers to 1, 000, 000, 000 Tokens

Ding, J., Ma, S., Dong, L., Zhang, X., Huang, S., Wang, W., Zheng, N., & Wei, F. (2023) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 34

BitNet: Scaling 1-bit Transformers for Large Language Models

Wang, H., Ma, S., Dong, L., Huang, S., Wang, H., Ma, L., Yang, F., Wang, R., Wu, Y., & Wei, F. (2023) · arXiv

Advanced Neural Network ApplicationsTopic Modeling

Cited by 26