Positional Encodings in LLMs. The full story (from integer counting to RoPE)

I am writing this article b/c positional encoding is one of those topics which looks very scary if you directly open the “Attention is All…