Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned

Voita, Elena, Talbot, David, Moiseev, Fedor · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · 2019