Transformers learn to implement preconditioned gradient descent for in-context learning

Ahn, Kwangjun, Cheng, Xiang, Daneshmand, Hadi · arXiv · 2023