Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers

Dai, Damai, Sun, Yutao, Dong, Li · arXiv · 2022