The Wall You Hit After You Ship: Inference Optimisation in Production GenAI Systems

Every team that has shipped a GenAI system in production has lived through the same arc. The demo worked. The prototype was fast enough…