When Transformers Multiply Their Heads: What Increasing Multi-Head Attention Really Does

Transformers have become the backbone of many AI breakthroughs, in NLP, vision, speech, etc. A central component is multi-head…