When RLVR Starts Cheating: Lessons from Long-Horizon GRPO Training on GSM8K

I kept training an RLVR model long enough … and discovered a new language.