The Graph Inside Attention

Full attention is a simple, expensive idea: every token in a sequence is allowed to look at every other token. For a sequence of length n…