Drawbacks of Low Rank Methods for Attention

Low-rank approximation methods are widely used to reduce the computational complexity and memory requirements of large-scale problems…