MiA-RAG: Building a “Whole-Book” Brain for Document QA

When people process long texts, they often benefit from an activated global semantic frame before reasoning about details. A holistic semantic representation can guide how people interpret new inputs and integrate dispersed evidence.

This idea can be applied to RAG.

Why Many RAG Systems Miss the “Global Map” for Long Documents

Many current RAG systems primarily operate on local evidence-level signals and lack explicit global context awareness. They focus on small chunks of evidence and often miss the bigger picture. That’s why they tend to struggle with long documents.

This matters especially for personalized, long-context collections where relevant evidence is dispersed. Think personal project docs, full-length novels, or government reports, all sprawling and complex. To be truly useful, a system needs to gather clues while also understanding the larger structure at play.

Standard RAG often lacks an explicit global semantic frame, think of it as having fragments without a document-level map. What’s missing is an explicit global semantic scaffold (a mindscape summary), constructed via hierarchical summarization, which guides both the retrieval and generation processes, rather than just a perfect ‘full map’.

MiA-RAG: Build a Mindscape to Guide Retrieval and Generation

Figure 1: Average model ranks across five long-context benchmarks under 3/5/10-chunk settings. [Source].

The core idea is simple: first build a “Mindscape” — a global summary, then use it to guide both the retrieval of evidence and the generation of answers.

Let’s say you’re working with a detective novel that’s hundreds of pages long. The question is: When did the killer first lie?

Here’s the process:

1. Chunk the Text

Start by splitting the novel into smaller chunks ci (e.g., token-based windows; 1200 tokens for NarrativeQA, 200 tokens for NoCha).

2. Build the Mindscape

Figure 2: Prompt templates used in the two-step hierarchical summarization process. [Source].

For each chunk, generate a chunk-level summary si using a summarizer model (specifically GPT-4o). These become si.

Then, merge all those si into a single high-level summary S of the whole book. Something like: “The murder took place in an old mansion. Suspect A had a tense relationship with the butler. The key timeline spans three nights…”

This summary acts as a mindscape that guides retrieval and reasoning, rather than standalone evidence.

3. Retrieval (MiA-Emb)

Instead of using just the question to look for relevant passages, MiA-Emb encodes the query conditioned on the global mindscape S as a joint input to the embedding model (using a residual integration mechanism to balance original query intent with global guidance), producing a mindscape-aware query representation for retrieval.

This makes a difference. The mindscape can bias retrieval toward the active topic’s conceptual frame, reducing ambiguities from other topics.

4. Generation (MiA-Gen)

Finally, take the summary S, the question, and the retrieved evidence (a few raw text passages) and feed them to the generator.

The generator uses the mindscape as global contextual guidance to interpret retrieved chunks and their relation to the query. It arranges the evidence into a coherent reasoning path, and produces answers by reasoning over retrieved evidence within a coherent global context.

Training

Training the Retriever: Automatically Creating Silver Evidence Labels

Many QA datasets include only questions and answers, but don’t specify where the answer comes from in the text. To work around this, it designed a pipeline that automatically identifies relevant evidence chunks, what they call silver labels.

Figure 3: Silver Evidence Annotation. [Source].

The approach combines three steps:

1. Query augmentation using {q, q+answer, answer}.
2. Ensemble candidate retrieval with voting over retrieved pools (VoteAndSelectTopK).
3. Apply an LLM to filter and finalize the selections

This yields a validated silver supervision signal without manual annotation. The final dataset includes 27,117 questions. On average, each question is paired with around 2.3 silver-labeled chunks and 2.9 supporting evidence nodes.

To train the retriever itself, they build MiA-Emb by applying LoRA on top of Qwen3-Embedding-8B.

The training objective is multi-task contrastive: it learns to pull relevant pairs closer and push irrelevant ones (constructed from a mix of hard and simple negatives) apart. On top of that, a residual integration mechanism is added to balance between the original intent of the question and the global context from the Mindscape.

Training the Generator

In the main experiments, the generator is a fully fine-tuned version of Qwen2.5–14B-Instruct.

The fine-tuning data, called D̃_gen, comes from two sources:

• NarrativeQA, a dataset for long-form question answering
• CLIPPER, which involves verifying narrative statements as true or false, including intermediate reasoning chains

Each training sample is structured like this:

[INST] gen; Mindscape S; Retrieved evidence Ĉret; Question qi;

The output is the target answer yi.

Training follows the standard autoregressive language modeling setup, using cross-entropy loss to help the model predict the next word in the sequence.

Evaluation

The full MiA-RAG system achieved the best overall performance across five long-context benchmarks.

Figure 4: Key experimental results. [Source].

MiA-RAG (14B) reached an average ranking of 1.13, outperforming other setups. For comparison, the vanilla 72B model averaged 3.00, while the vanilla-RAG 14B came in at 5.00.

Thoughts

MiA-RAG introduces an explicit form of global semantic memory called the Mindscape. By generating a high-level summary upfront, it gives the RAG process a sense of direction, helping it better navigate long documents.

My main concern here is that the system’s reliability depends heavily on that initial summary. If the underlying content keeps changing, or if generating a good summary is difficult, the whole pipeline could be constrained. There’s also a risk that commercial LLM–derived supervision may introduce latent biases or hallucinated content into the training signal / mindscape construction.

In practice, this approach may be useful, but the choice of scenario matters.

• For relatively stable, long-form knowledge sources, like documentation or archived reports, the Mindscape can be cached offline. Even with a smaller model, performance gains could be non-trivial.
• But for fast-changing content, without incremental updates or version control, an outdated Mindscape could introduce consistent retrieval or reasoning errors.

Reference: Mindscape-Aware Retrieval Augmented Generation for Improved Long Context Understanding.