Implementing ALiBi in C++ from the Ground Up

ALiBi is a positional encoding method in the same way RoPE or the standard sinusoidal positional encoding functions are; the difference…