Lost in the Middle: The Hidden Blind Spot Sabotaging Your AI Prompts

Lost in Middle problem in LLM

Well, I was using Codex for my code automation purposes, and I gave it a long prompt, and I observed that it was picking up my instructions from the last sentence. I experimented with that, and I came across a common pattern that LLM is turning blind towards the middle part. I started to research this in my free time and came to know that this is called “Lost in Middle “.

Modern LLM can technically process extremely long context from 4K to 100K+ tokens. Just because a model can accept long context does NOT mean it can effectively use all parts of it.

Researchers discover that LLMs often remember the information at start (primacy bias) and information at the end (recency bias), but struggle to use the information in middle.

In real-world systems, prompts are rarely short.

Modern AI systems often process:

• Long coding instructions
• Large research papers
• Multi-document RAG pipelines
• Legal documents
• Medical reports
• Agent memory histories
• Long chat conversations

If important instructions or facts are buried in the middle, the model may:

• ignore them,
• partially follow them,
• hallucinate around them,
• or produce inconsistent outputs.

Why does this happen?

This problem occurs because of how transformer-based language models process the information internally.

Transformers use a mechanism called the Self-Attention Mechanism. This decides the score or importance that each token gets, how each token is related to other tokens, and how much influence each token should have. In theory, every token attends to every other token. But in reality, the attention is not distributed uniformly. Some positions naturally become stronger than others.

Imagine a sentence: “The scientist won the Nobel Prize in Physics.” when the model process the word “Physics” , it tries to determine which earlier words matter most and how strongly they are connected to physics. So the model strongly attends to “scientist”, ”Nobel”, ”Prize”. This selective focus is called Attention.

In long context, not all tokens receive the same amount of reinforcement.

U-shaped attention behavior

This creates Primacy bias, which is that earlier tokens become influential. Recency bias, where recent tokens remain fresh and highly accessible. Middle Weakness: where middle tokens receive less attention. This creates a DEAD ZONE in long contexts.

Another major reason comes from Casual Masking. Transformer LLMs generate text sequentially. A token can mainly attend to itself and previous tokens. If the sequence is “ A B C E F G”, when processing the token “G”, the model can see tokens from “A” to “F”, but when processing the token “C”, it can see the tokens from “A” and “B”.

Early tokens accumulate influence over time. Middle tokens receive less reinforcement, creating structural imbalances inside the network. The beginning becomes deeply embedded in the model’s internal representations.

Transformers naturally don’t understand the order. Without extra mechanisms, the model can not distinguish between “Dog Bites Man” and “Dog Man Bites”. So, the modern LLMs use RoPE — Rotary Positional Encoding. RoPE helps the model to understand token order, relative distance, sequence structure, and the relation between nearby tokens. It mathematically rotates token embeddings so positional relationships become encoded into attention calculations.

But RoPE has a problem — RoPE works extremely well for nearby tokens. But as distance increases, attention quality weakens.

Middle information often depends on: long-range retrieval, distant references, and cross-context reasoning. But RoPE struggles with extremely long dependencies. So, beginning tokens stay reinforced, recent tokens stay fresh, middle tokens become harder to retrieve accurately. This is another reason the middle becomes a “dead zone.”

Long prompts create enormous competition for attention. The model has a limited effective attention capacity. Even though mathematically it can “see” all tokens, cognitively it cannot focus strongly on everything simultaneously.

Interestingly, this behavior resembles human memory psychology. Humans show something called “Serial Position Effect” . People naturally remember: first items best, last items best, middle items worst.

The “Lost in the Middle” problem is not caused by one single issue.

It emerges from multiple architectural effects working together.

Lost in Middle In short !

How to overcome this issue?