Decoding Positional Encoding: How the Transformer’s Sin/Cos Formula Was Actually Thought Up

Every Transformer tutorial shows you the positional encoding formula, pastes the heatmap, says “sin and cos encode position,” and moves on…