The Context Window Lie

Everyone flexes about context windows now. “128K tokens.” “1 million tokens.” Sounds impressive. Sounds like the model can hold an entire…