Mixture-of-Depths: Dynamically allocating compute in transformer-based language models

Raposo, David, Ritter, Sam, Richards, Blake · arXiv · 2024