Can We Actually Interpret What a Language Model Is “Thinking”? A Tour of NLP Interpretability

We can measure what a model outputs. We still can’t reliably explain why. That gap might be the most consequential unsolved problem in AI…