There is a column in every lab review meeting called ‘not blocking’

It is where the known weaknesses of a benchmark chart go to die quietly. The METR errors are exactly the shape of things that end up in…