Can KV Cache be chopped by half by removing V cache

Can KV Cache be chopped by half by removing V?

This blog details my minimal experiments to see if V cache can be removed altogether and be derived by the kernel from K cache instead?

This in theory would remove the KV cache reservation space by half!! Allowing for more sequence length or sequence batch processing

In vanilla attention q,k,v projection matrices are often square ones

With eqns Q = Wq x Latent K = Wk x Latent V = Wv x Latent

and square matrices are invertible

so you can also formulate V eqn as

V= Wv x Wkinv x K (you can cache the Wv x Wkinv as a projection matrix here)

With an efficient flash attention rewrite the compute overhead too will be negligible so yay we can reduce kv cache !!

Since this was a low hanging fruit asked grok to search if any papers already did this.. and .. SlimAttention already pulled it off …. in 2025…

But the paper also mentioned something interesting that this can’t be done with GQA (Grouped Query Attention), because the projection matrices are rectangular meaning those are not invertible

The projected dimension often was smaller than latent projection (dmodel > dquery = dkey = dvalue).

But then i thought why not learn a dkey x dkey transformation matrix from K -> V since both K and V itself are nothing but linear projections of the latent

Made opencode — gpt5.5 spinup an experiment on gemma3 1b since its based on gqa, to see the results in reconstruction error between V and the linear transformation approximation

V = KB (where B is the learned tranformation) (easily achieved with torch.linalg)

In most layers the error was high … but why?

Latent -> K (by Wk) Latent -> V (by Wv)

are compressive transformations, and Wk could be capturing a different subset of latent when compared to Wv

So what do we do???

What if we compute a compressed latent which dcompressed such that dlatent > dcompressed > dkey = dvalue = dquery and cache the compressed latent instead of Key and Values??

Welcome son, you just discovered multi latent attention of deepseek from first principles !!

If you need the code base just ping me .. or may be i’ll add it later to the blog

Hope you liked it