Attention Is O(n²): FlashAttention vs Linear Attention

Standard attention on a 32K-token sequence allocates 2,199 GB of score-matrix memory across a 32-layer model — 27× the capacity of an A100…