Every Transformer Running Today Is a MacLaurin Series in Disguise.

The attention mechanism, the softmax function, the positional encoding, the GELU activation — all of them are polynomial approximations to…