Worth reading | LLMs corrupt your documents

Just recently Microsoft published a study regarding the ability of LLMs to provide correct document handling in terms of working with them, but not altering them, e.g. adding wrong or unwanted content.

And, surprise, surprise:

they failed.

Quote from their conclusion:

“We find that current LLMs are unreliable delegates: even frontier models corrupt an average of 25% of document content over long workflows, with sparse but severe errors that silently compound over time. Our analysis shows that degradation worsens with document length, interaction horizon, and distractor context, and is not mitigated by agentic tool use. These results highlight a fundamental gapin reliability that undermines trust in delegation.”

So, what does this tell us?

Maybe that, although is praised as the new technology that — eventually — solves all our problems, especially in the digital field, AI is not entirely reliable with regards to forensic science or digital investigations.

It can’t be stressed enough: AI output needs verification in order to be a valuable piece of evidence in any investigation.

Please see the original ressource here: https://arxiv.org/pdf/2604.15597

Further reading: https://github.com/microsoft/DELEGATE52