Recursive Multi-Agent Systems: From Research Paper to Implementation — Three Implementations Compared

Recursive NultiAgent System Overview

A complete technical guide to arXiv:2604.25917v1 — how the paper works, how we built two progressively faithful Python implementations (LangGraph + Claude, then LangGraph + official prompts + Groq), and a three-way comparison against the official RecursiveMAS repository.

1. Why Multi-Agent Systems Are Not Enough (Yet)

Imagine assembling the world’s best team of consultants to solve a hard problem. You have a mathematician, a software engineer, and a domain scientist. You put them in a room, give them the problem, and each writes their analysis on a separate sheet of paper. The coordinator reads all three sheets and produces a final report.

This is how almost every multi-agent AI system (MAS) works today. Agents produce text outputs, and the next agent in line reads those text outputs. It sounds reasonable — but there is a profound inefficiency buried in this workflow.

When a human expert thinks about a complex problem, the richest, most nuanced reasoning happens before language. It happens in what cognitive scientists call the “workspace” — a fluid, high-dimensional space of partially-formed ideas, competing hypotheses, and latent associations. The moment that reasoning is distilled into sentences, a significant portion of its richness is lost.

Modern large language models (LLMs) have an analogue of this latent workspace: the hidden state vectors produced in the intermediate transformer layers. These vectors are far more information-dense than any text the model outputs. Yet in every standard MAS, agents are only allowed to communicate through text — forcing a bottleneck at precisely the wrong place.

This is the problem that **arXiv:2604.25917v1** — “Recursive Multi-Agent Systems” (RecursiveMAS) — sets out to solve.

2. The Core Idea: Making Agents Think Recursively

The paper’s central thesis can be stated in one sentence:

Instead of passing text between agents, pass latent hidden states — and do it recursively.

The authors unify the entire multi-agent system into a single recursive computation. At each step, instead of an agent saying “here is my conclusion in words,” it passes its internal continuous-space representation directly to the next agent, who uses it as a conditioning signal for their own reasoning.

This has two major benefits:

1. No information bottleneck. The latent representation encodes everything the model is “thinking,” not just what it chose to say. Cross-agent communication becomes near-lossless in a way that text never can be.

2. Recursive self-improvement. The output of the last agent in the chain is fed back into the first agent, creating a closed loop. Over multiple rounds, the system refines its collective reasoning, with each agent building on increasingly refined representations from all other agents.

The authors draw an analogy to chain-of-thought (CoT) prompting — but at the system level. Just as CoT helps a single model break a problem into reasoning steps, RecursiveMAS implements CoT across an ensemble of heterogeneous models, in continuous latent space.

3. The RecursiveLink Module — The Heart of the Paper

The technical machinery that makes all of this work is called the RecursiveLink module. It has two distinct components.

3.1 The Inner Link

The Inner Link operates within a single agent. It takes the agent’s hidden state after processing its input and folds it back as an additional input to the same agent in the next recursion round.

Formally, if agent $A$ processes input $x$ and produces hidden state $h^{(t)}$ at recursion round $t$:

h^(t+1) = InnerLink(h^(t), x)

This is analogous to the recurrent connection in an LSTM or GRU — except here it operates at the level of a full transformer’s hidden representation, not just a single vector. The agent “remembers” its own reasoning trajectory across rounds, allowing it to progressively deepen its analysis rather than starting fresh each time.

In the official repo (RecursiveMAS/modeling.py), the Inner Link is implemented as a two-layer MLP with LayerNorm and a residual connection:

class Adapter(nn.Module):
    def __init__(self, hidden_size: int, adapter_type: str) -> None:
        super().__init__()
        self.proj1   = nn.Linear(hidden_size, hidden_size)
        self.act     = nn.GELU()
        self.proj2   = nn.Linear(hidden_size, hidden_size)
        self.pre_ln  = nn.LayerNorm(hidden_size)
        self.post_ln = nn.LayerNorm(hidden_size)
​
    def forward(self, x: torch.Tensor) -> torch.Tensor:
        h   = self.pre_ln(x)
        out = self.proj2(self.act(self.proj1(h)))
        out = x + out          # residual
        return self.post_ln(out)

This is the entire inner link — a lightweight ~4 million parameter module per agent. The base model weights are frozen; only this adapter is trained.

3.2 The Outer Link

The Outer Link operates between agents. When agent $A$ finishes its recursion round, its latent state $h_A$ is projected into the input space of agent $B$ — even if they are different model families with different embedding dimensions (e.g., Qwen-1.7B hidden size → Llama-1B hidden size).

input_B = OuterLink(h_A) + original_input

In modeling.py, this is a cross-model adapter with source/target layer norms and a residual projection:

class CrossModelAdapter(nn.Module):
    def __init__(self, in_dim: int, out_dim: int, adapter_type: str) -> None:
        super().__init__()
        hidden_dim        = out_dim * 2
        self.proj1        = nn.Linear(in_dim,    hidden_dim)
        self.act          = nn.GELU()
        self.proj2        = nn.Linear(hidden_dim, out_dim)
        self.ln_source    = nn.LayerNorm(in_dim)
        self.ln_target    = nn.LayerNorm(out_dim)
        self.residual_proj = nn.Linear(in_dim,   out_dim)
​
    def forward(self, x: torch.Tensor) -> torch.Tensor:
        h   = self.ln_source(x)
        out = self.proj2(self.act(self.proj1(h)))
        out = out + self.residual_proj(x)
        return self.ln_target(out)

The outer links are directional and pattern-specific. The official system_loader.py defines exactly which outer links exist for each style:

_OUTER_LAYOUTS = {
    "sequential":    {"outer_12": ("planner","critic"), "outer_23": ("critic","solver"),
                      "outer_31": ("solver","planner")},          # the recursive loop-back
    "mixture":       {"outer_1s": ("math","summarizer"),  "outer_2s": ("code","summarizer"),
                      "outer_3s": ("science","summarizer"),
                      "outer_s1": ("summarizer","math"),  "outer_s2": ("summarizer","code"),
                      "outer_s3": ("summarizer","science")},      # 6 bidirectional links
    "distillation":  {"outer_el": ("expert","learner"), "outer_le": ("learner","expert")},
    "deliberation":  {"outer_rt": ("reflector","toolcaller"), "outer_tr": ("toolcaller","reflector")},
}

Each outer link is a separately trained CrossModelAdapter checkpoint, downloaded from HuggingFace.

3.3 The Full Recursive Loop

Putting it together: at each recursion round $t$:

1. Each agent runs its inner link to refine its own latent state.
2. Each agent forwards its latent state to downstream agents via the outer link.
3. The last agent’s state is fed back to the first agent — closing the loop.
4. After $T$ rounds, only the final agent produces a textual output.

All intermediate rounds operate entirely in latent space. This is why the system achieves a 1.2×–2.4× inference speedup over text-based baselines — there is no token decoding overhead between rounds.

4. The Four Collaboration Styles

The paper evaluates RecursiveMAS across five “MAS Collections.” The official repo supports all five via --style in run.py.

4.1 Sequential-Style (Light)

Official HF checkpoints:

Workflow: The classic “divide and conquer” pipeline. The Planner decomposes the problem into a step-by-step plan. The Refiner (the official repo calls it “Critic” externally but “refiner” internally) tightens that plan. The Solver executes it and writes the final answer.

What recursion adds: The Solver’s latent state in round $t$ is fed back to the Planner in round $t+1$ via outer_31. The Planner doesn't just read the Solver's words — it reads the Solver's hidden uncertainty representation and adjusts the plan to target exactly where the reasoning was weakest.

Benchmark result: +6.2% over non-recursive sequential baseline on math/science benchmarks.

4.2 Sequential-Style (Scaled)

Official HF checkpoints:

What changes: Same graph structure, larger models. The key finding is that RecursiveLink’s gain is additive with model scale — a larger Solver fed back to a larger Planner compounds the benefit.

Key finding: Sequential Light with RecursiveMAS matches Sequential Scaled without RecursiveMAS in accuracy, at 2–4× less compute.

4.3 Mixture-Style

Official HF checkpoints:

Workflow: Three specialists analyse the problem independently in parallel, then the Summarizer synthesises them. The key novelty is the bidirectional outer links: outer_s1/s2/s3 broadcast the Summarizer's latent state back to each specialist. Specialists aren't independent — they're conditioned on where the synthesis is uncertain.

Benchmark result: +9.1% on multi-domain STEM benchmarks.

4.4 Distillation-Style

Official HF checkpoints:

Critical design detail: The Expert outputs a step-by-step plan only — it is explicitly instructed “Do not provide the final answer.” The Learner executes that plan and writes the answer. This means the Expert’s entire token budget goes into planning, and the Learner’s into execution.

What recursion adds: outer_le carries the Learner's hidden state (including its uncertainty about each execution step) back to the Expert. The Expert reads this and refines its plan to target exactly where the Learner struggled — not just where it said it struggled.

Benchmark result: +8.0% Learner accuracy. System is 1.5× faster than using the Expert alone because the Learner is 2× smaller and handles all final decoding.

4.5 Deliberation-Style

Official HF checkpoints:

Critical design detail from the official repo: Both the Reflector AND the Tool-Caller have tools available. The official system prompt (from reflector_tool_notes.py) is:

“You are a helpful assistant that can solve the given question step by step with the help of the wikipedia search tool and python interpreter tool… The search query and result are enclosed within <search></search> and <result></result> tags..."

This is different from many text-based implementations where only the Tool-Caller has tools.

What recursion adds: outer_rt carries the Reflector's latent "uncertainty map" to the Tool-Caller. The Tool-Caller doesn't decide what to search from words alone — it reads the Reflector's hidden state, which encodes a continuous-valued uncertainty score for each part of the problem. This reduces wasted tool calls dramatically.

Benchmark result: +4.8% on knowledge-intensive tasks, 34.6% fewer tool calls.

5. How the System Learns: The Inner-Outer Training Loop

One of the paper’s most elegant contributions is its training paradigm.

Inner Loop Training happens first and independently for each agent. Each agent’s Adapter head is trained on that agent's specific task (math planning, math solving, etc.). This warm-starts the system — each agent already knows how to refine its own latent thoughts before cross-agent coordination is introduced.

Outer Loop Training trains the entire system end-to-end. Gradients flow through the outer CrossModelAdapter modules between agents. Base model weights remain frozen throughout — only the adapter modules are updated. The paper shows this is sufficient because the adapter modules learn to extract exactly the right information from each agent's hidden space and encode it for the next.

The two-stage approach makes the system scalable: adding a new agent requires training only that agent’s inner adapter and one or two outer adapters. The rest of the system is unchanged.

6. Performance Results from the Paper

Across all five collection types and 9 benchmarks:

The token reduction is the most counterintuitive result. Despite running multiple recursion rounds, the system uses fewer tokens because intermediate rounds operate in latent space with no decoding. This directly translates to lower API costs — a strong practical argument for the approach.

7. Implementation 1: rmas/ — Custom Prompts, Claude API

Now let’s talk about what we built. We created two Python implementations with different levels of faithfulness to the paper.

The first — rmas/ — takes a pragmatic approach that works with any API-hosted LLM. It implements the four collaboration patterns and the recursive loop structure from the paper, but replaces both the latent-space communication and the official prompts with custom approximations.

Tech stack: LangGraph + Anthropic Claude (Haiku + Sonnet) + LangChain + DuckDuckGo/Tavily

7.1 The Shared State: Simulating RecursiveLink

The most important design decision in rmas/ is RMASState:

class RMASState(TypedDict):
    messages:     Annotated[list[BaseMessage], add_messages]
    round:        int           # current recursion round
    max_rounds:   int
    context:      dict[str, Any]  # role → last text output (RecursiveLink substitute)
    pattern:      str
    final_answer: str

The context dict is our approximation of the outer link. After each agent runs, it writes its full text output to context[role]. Downstream agents read from context to see all previous outputs. The recursive loop-back happens naturally: in round $t+1$, the first agent reads the last agent's entry from round $t$.

7.2 Model Assignment in rmas/

We mirror the paper’s heterogeneous composition with Claude model tiers:

ROLE_MODELS = {
    "Planner":          "claude-haiku-4-5-20251001",   # ≈ Qwen3-1.7B  (light)
    "Critic":           "claude-haiku-4-5-20251001",   # ≈ Llama3.2-1B  (light)
    "MathSpecialist":   "claude-haiku-4-5-20251001",   # ≈ DeepSeek-R1-Qwen (light)
    "CodeSpecialist":   "claude-haiku-4-5-20251001",   # ≈ Qwen2.5-Coder  (light)
    "ScienceSpecialist":"claude-haiku-4-5-20251001",   # ≈ BioMistral  (light)
    "Learner":          "claude-haiku-4-5-20251001",   # ≈ Qwen3.5-4B   (light)
    "Solver":           "claude-sonnet-4-6",            # ≈ Qwen2.5-Math  (scaled)
    "Summarizer":       "claude-sonnet-4-6",            # ≈ Qwen3.5-2B   (scaled)
    "Expert":           "claude-sonnet-4-6",            # ≈ Qwen3.5-9B   (scaled)
    "Reflector":        "claude-sonnet-4-6",
    "ToolCaller":       "claude-sonnet-4-6",
}

7.3 Custom System Prompts in rmas/

Every agent gets a hand-crafted system prompt that explicitly encodes the recursive protocol:

system_prompt = (
    f"You are the **{role}** agent in a Recursive Multi-Agent System (RecursiveMAS).\n\n"
    f"{description}\n\n"
    "--- Recursive Collaboration Protocol ---\n"
    "You operate in iterative recursion rounds. Each round you receive:\n"
    "  • The original problem\n"
    "  • Accumulated context (thoughts) from peer agents in previous rounds\n"
    "  • Your own previous output (if any)\n"
    "Use this recursive context to progressively deepen and refine your response."
)

These are our own prompts — not from the paper. The paper’s official prompts are more structured and task-specific, as we’ll see in Section 8.

7.4 The LangGraph Pattern: Sequential as an Example

The LangGraph graph for the sequential pattern is a directed cycle with a conditional exit:

graph.set_entry_point("planner")
graph.add_edge("planner", "critic")
graph.add_edge("critic", "solver")
graph.add_conditional_edges("solver", route, {"planner": "planner", "__end__": END})
​
def route(state) -> Literal["planner", "__end__"]:
    return "planner" if state["round"] < state["max_rounds"] else "__end__"

The Solver increments state["round"] and owns the routing decision — exactly mirroring the paper's design where the last agent in each pattern controls the loop counter.

7.5 Notable Divergence: rmas/ Deliberation

One important difference from the paper: in rmas/, only the Tool-Caller has tools. The Reflector is a pure reasoner with no tool access. This seemed logical intuitively, but is incorrect — the official implementation gives both agents the same tool-aware system prompt. This is fixed in rmas_official_bridge/.

8. Implementation 2: rmas_official_bridge/ — Official Prompts, Groq or Claude

rmas_official_bridge/ is a more faithful implementation. It imports the exact prompt builders directly from the cloned RecursiveMAS repo and replaces both the custom prompts and the single-provider limitation:

# rmas_official_bridge/__init__.py
_OFFICIAL_REPO = Path(__file__).resolve().parent.parent / "RecursiveMAS"
sys.path.insert(0, str(_OFFICIAL_REPO))

From this point on, from prompts import ... gives us the paper's exact prompt functions.

8.1 The Official Prompt Architecture: Slot-Based Communication

The official RecursiveMAS/prompts.py uses a slot-based architecture that is the text-level analogue of the outer RecursiveLink. Instead of injecting raw text anywhere in the prompt, specific placeholder strings mark exactly where the latent signal from another agent should be inserted.

The slots defined in the official code:

PLANNER_SLOT               = "<<LATENT_PLANNER_SLOT>>"     # sequential
REFINED_SLOT               = "<<LATENT_REFINED_SLOT>>"     # sequential
FEEDBACK_SLOT              = "<<LATENT_FEEDBACK_SLOT>>"    # sequential loop-back
​
HIE_MATH_EXPERT_SLOT       = "<<HIE_MATH_EXPERT_SLOT>>"   # mixture
HIE_CODE_EXPERT_SLOT       = "<<HIE_CODE_EXPERT_SLOT>>"   # mixture
HIE_SCIENCE_EXPERT_SLOT    = "<<HIE_SCIENCE_EXPERT_SLOT>>" # mixture
HIE_FEEDBACK_SLOT          = "<<HIE_FEEDBACK_SLOT>>"      # mixture loop-back
​
DISTILL_EXPERT_SLOT        = "<<DISTILL_EXPERT_SLOT>>"    # distillation
DISTILL_FEEDBACK_SLOT      = "<<DISTILL_FEEDBACK_SLOT>>"  # distillation loop-back
​
DELIBERATION_REFLECTOR_SLOT = "<<DELIBERATION_REFLECTOR_SLOT>>" # deliberation
DELIBERATION_FEEDBACK_SLOT  = "<<DELIBERATION_FEEDBACK_SLOT>>"  # deliberation loop-back

Each slot is placed at a semantically significant location in the prompt template. The prompt builder functions return strings with slots intact; the caller replaces them with actual agent outputs:

# Official pattern in rmas_official_bridge/patterns/sequential.py
user_prompt = build_math_planner_prompt_with_feedback_slot(question).replace(
    FEEDBACK_SLOT, ctx.get("solver", "")   # outer_31: solver's output → planner's slot
)

This is the official text-level analogue of outer_31 — the same information flow, but in token space rather than latent space.

8.2 Sequential Pattern: Official Prompt Builders

The official sequential prompts are more structured than our custom ones. Here’s what the official math planner prompt looks like:

# RecursiveMAS/prompts.py — build_math_planner_prompt
"You are a planner agent in a multi-agent system.\n"
"Give a plan for the question below.\n"
"Question:\n"
f"{question}\n"
"Your response should be in the format of:\n"
"Step 1: ...\n...\nStep n: ..."

Round 2+ uses build_math_planner_prompt_with_feedback_slot:

"You are a planner agent in a recursive multi-agent system.\n"
"This is round 2.\n"
"Question:\n" f"{question}\n"
"Feedback signal from the previous solver round:\n"
f"{FEEDBACK_SLOT}\n"                          # ← outer_31 text approximation
"Use the feedback as a soft correction signal to improve the plan.\n"
"If there is any conflict, prioritize the question constraints.\n"
"Output only a concise plan in the format:\n"
"Step 1: ...\n...\nStep n: ..."

Notice the phrase “soft correction signal” — this is a deliberate design choice. The official prompts consistently use this language because, in the full latent-space system, the slot content is a continuous vector that conditions attention softly, not a hard instruction. Using it as a “soft” signal is the closest text approximation of that behaviour.

8.3 Distillation: The Plan-Only Expert Constraint

The most important structural difference from rmas/ in the distillation pattern is the Expert's constraint. The official Expert prompt explicitly says:

# RecursiveMAS/prompts.py — build_distill_expert_prompt
"You are the expert agent in a multi-agent system.\n"
f"{task_context}"
"Provide a concise, execution-ready plan that the learner can follow.\n"
"Do not provide the final answer.\n"          # ← critical constraint
"Your response should be in the format:\n"
"Step 1: ...\n...\nStep n: ..."

In rmas/, our Expert produces both a plan and a partial answer. In rmas_official_bridge/, the Expert produces only a plan — exactly matching the paper's intent where the Expert's token budget is entirely devoted to planning, and the Learner's token budget is entirely devoted to execution.

The Learner prompt then uses DISTILL_EXPERT_SLOT:

# build_distill_learner_prompt_with_slot
"You are the learner executor in a multi-agent system.\n"
"Expert plan:\n"
f"{DISTILL_EXPERT_SLOT}\n"          # ← outer_el: expert plan → learner
"Use the expert plan as guidance, but prioritize the task constraints.\n"
f"---\n{task_context}"
f"{_hie_final_instruction(question, mas_task=mas_task)}"

8.4 Deliberation: Both Agents Have Tools (Official Correction)

The official get_system_prompt() returns the deliberation tool-use system prompt for both agents:

# RecursiveMAS/prompts.py
def get_system_prompt(mas_design: str = "chain", mas_role: Optional[str] = None) -> str:
    role_name = str(mas_role or "").strip().lower()
    design_name = str(mas_design or "chain").strip().lower()
    if design_name == "deliberation" or role_name in {"deliberation_reflector", "deliberation_toolcaller"}:
        return DELIBERATION_SYSTEM_PROMPT   # ← tool-use prompt for BOTH
    return SYSTEM_PROMPT                    # "You are a helpful assistant."

The deliberation system prompt (from reflector_tool_notes.py) instructs both agents to use <search>query</search> and <python>code</python> XML tags. This is how our rmas_official_bridge/agents/factory.py creates all deliberation agents:

def create_bridge_agent(role, mas_design, tools=None, model=..., provider=...):
    system_prompt = get_system_prompt(mas_design=mas_design, mas_role=role)
    # For deliberation roles, system_prompt is the full tool-use system prompt
    llm = _make_llm(provider, model, temperature, max_tokens)
    return create_react_agent(model=llm, tools=tools or [], prompt=system_prompt)

8.5 Task Type Detection: math / code / choice

The official prompts have three variants depending on the downstream task type. Since we’re building a free-form CLI (not running on a fixed benchmark), rmas_official_bridge/orchestrator.py includes a heuristic:

def _infer_mas_task(question: str) -> str:
    q = question.lower()
    code_kw = ("implement", "write a function", "write code", "algorithm",
                "program", "def ", "class ", "pseudocode")
    if any(kw in q for kw in code_kw):
        return "code"
    if re.search(r"^\s*[A-D]\s*[\.\):\-]\s+", question, re.MULTILINE):
        return "choice"
    return "math"   # default — handles open-ended reasoning questions

The mas_task value is stored in BridgeState and passed to every prompt builder, which selects the appropriate answer format (\boxed{} for math/choice, markdown code block for code).

8.6 rmas_official_bridge/ Project Structure

rmas_official_bridge/
├── __init__.py          # adds RecursiveMAS/ to sys.path → from prompts import ... works
├── config.py            # CollaborationStyle, Provider enums + model maps (Anthropic + Groq)
├── state.py             # BridgeState TypedDict (adds mas_task field)
├── orchestrator.py      # OfficialBridgeMAS.run() / .stream() with task detection
├── agents/
│   └── factory.py       # get_system_prompt() from official repo + ChatAnthropic | ChatGroq
└── patterns/
    ├── sequential.py    # FEEDBACK_SLOT → outer_31; PLANNER_SLOT → outer_12
    ├── mixture.py       # HIE_*_SLOT → outer_1s/2s/3s; HIE_FEEDBACK_SLOT → outer_s1/s2/s3
    ├── distillation.py  # DISTILL_EXPERT_SLOT → outer_el; DISTILL_FEEDBACK_SLOT → outer_le
    └── deliberation.py  # DELIBERATION_REFLECTOR_SLOT → outer_rt; FEEDBACK → outer_tr

Each pattern file maps the official slot names directly to the outer link they approximate.

9. Groq Model Mapping: The Closest API-Accessible Analogue

This section is the most practically valuable for AI engineers wanting to reproduce paper results without a GPU cluster.

The official RecursiveMAS uses open-source models from the Qwen, Llama, Gemma, and Mistral families. Groq provides inference for many of these exact model families at API speed — making it the closest achievable approximation to the official system without running models locally.

9.1 Why Groq, Not OpenAI or Anthropic?

The fundamental reason is model family alignment. The official checkpoints are fine-tuned versions of Qwen3, Llama3.2, Gemma3, DeepSeek-R1, BioMistral, and Qwen2.5-Coder. The RecursiveLink adapters were trained to bridge these specific model families’ hidden spaces.

Using Claude or GPT replaces the base model entirely — a completely different architecture, tokenizer, and hidden space geometry. The adapter training that produced the official checkpoints is meaningless for a different base model.

Groq serves models from the same families as the official checkpoints:

9.2 Full Groq Model Map by Style

Sequential-Style (Light) — Groq

Official:  Qwen3-1.7B → Llama3.2-1B → Qwen2.5-Math-1.5B
Groq:      llama-3.1-8b-instant → llama-3.1-8b-instant → llama-3.3-70b-versatile

The Solver uses the larger Groq model because Qwen2.5-Math is a specialised math reasoning model — the most capable math model on Groq is qwen-qwq-32b, but we use llama-3.3-70b-versatile for speed/cost balance.

Sequential-Style (Scaled) — Groq

Official:  Gemma3-4B → Llama3.2-3B → Qwen3.5-4B
Groq:      llama-3.3-70b-versatile (all three roles)

The scaled variant uses uniformly large models. Groq’s llama-3.3-70b-versatile is the best analogue.

Mixture-Style — Groq

Official:  DeepSeek-R1-Distill-Qwen-1.5B + Qwen2.5-Coder-3B + BioMistral-7B → Qwen3.5-2B
Groq:      qwen-qwq-32b + qwen-2.5-coder-32b-preview + llama-3.3-70b-versatile → llama-3.3-70b-versatile

This is the best-matched style:

• qwen-qwen3-32b is the QwQ model from Qwen — a direct successor of the DeepSeek-R1-Distill line with the same chain-of-thought reasoning approach
• qwen-2.5-coder-32b-preview is literally the same Qwen2.5-Coder family as the official code specialist, just 10× larger

Distillation-Style — Groq

Official:  Qwen3.5-9B (Expert) → Qwen3.5-4B (Learner)
Groq:      llama-3.3-70b-versatile (Expert) → llama-3.1-8b-instant (Learner)

The large/small model pairing is preserved. The Learner is intentionally smaller — this is the whole point of distillation.

Deliberation-Style — Groq

Official:  Qwen3.5-4B (both) with tool-use system prompt
Groq:      llama3-groq-70b-8192-tool-use-preview (both)

The Groq deliberation model is purpose-built for function/tool calling. This is the most faithful Groq mapping to the official system because both use a Llama-family model with native tool-use support.

9.3 Configured in code

In rmas_official_bridge/config.py:

GRQ_LIGHT    = "llama-3.1-8b-instant"
GRQ_SCALED   = "llama-3.3-70b-versatile"
GRQ_MATH     = "qwen-qwen3-32b"
GRQ_CODE     = "qwen/qwen3-32b"
GRQ_TOOL     = "llama-3.3-70b-versatile"
​
_GROQ_STYLE_MAP = {
    "sequential_light":  {"planner": GRQ_LIGHT,  "refiner": GRQ_LIGHT,  "solver":  GRQ_SCALED},
    "sequential_scaled": {"planner": GRQ_SCALED, "refiner": GRQ_SCALED, "solver":  GRQ_SCALED},
    "mixture":           {"math":    GRQ_MATH,   "code":    GRQ_CODE,   "science": GRQ_SCALED,
                          "summarizer": GRQ_SCALED},
    "distillation":      {"expert":  GRQ_SCALED, "learner": GRQ_LIGHT},
    "deliberation":      {"reflector": GRQ_TOOL, "toolcaller": GRQ_TOOL},
}

To run with Groq, add GROQ_API_KEY=gsk_... to .env and use --provider groq:

uv run rmas_official_bridge/main.py --style mixture --provider groq --rounds 2

10. Three-Way Comparison: Official vs. rmas/ vs. rmas_official_bridge/

This is the most important section for understanding the design space. Each implementation makes different trade-offs.

10.1 Communication: The Fundamental Dimension

OFFICIAL RECURSIVEMAS (GPU — local models)
─────────────────────────────────────────────────────────────────────────
Input → Agent_1 [Inner Adapter] ──── latent h_1 tensor ────► Agent_2 [Inner Adapter]
         ▲                                                               │
         └────────── [CrossModelAdapter: outer_31] ─── latent h_3 ──────┘
                              [Loop for T rounds, no text decoding]
                     Final round only: Agent_N decodes → text output
​
rmas/ (API — Claude, custom prompts)
─────────────────────────────────────────────────────────────────────────
Input → Agent_1 → full text response → context["planner"] = text
         ▲                                                               │
         └────── context["solver"] injected into prompt ────────────────┘
                              [Loop for T rounds, full decode every round]
                     Every round: all agents → full text output
​
rmas_official_bridge/ (API — Groq or Claude, official prompts + slots)
─────────────────────────────────────────────────────────────────────────
Input → Agent_1 → full text response → ctx["planner"] = text
         ▲                                                               │
         └── FEEDBACK_SLOT replaced with ctx["solver"] in official template ─┘
                              [Loop for T rounds, full decode every round]
                     Every round: all agents → full text output
                     Prompt structure matches official outer link topology exactly

10.2 Full Feature Matrix

10.3 Per-Style Faithfulness Score

The following scores (0–5) assess how closely each implementation matches the paper’s intent for each collaboration style. Scoring dimensions: prompt accuracy, outer link topology, model assignment, role correctness.

**rmas/ scores**: Low due to custom prompts, no task-type distinction, Reflector without tools (deliberation), Expert producing full answers instead of plan-only (distillation).

**rmas_official_bridge/ + Groq scores**: High because official prompts are used verbatim, slot topology matches outer link layout, task types are correctly detected, Expert is plan-only, Reflector has tools. The remaining gap from 5/5 is the fundamental text-vs-latent communication gap.

10.4 The Official Slot ↔ Outer Link Correspondence

This table makes the correspondence explicit for AI engineers:

In rmas_official_bridge/, every pattern file performs these replacements explicitly. For example, in patterns/distillation.py:

# outer_el: expert plan → learner (text approximation)
prompt = build_distill_learner_prompt_with_slot(q, mas_task).replace(
    DISTILL_EXPERT_SLOT, ctx.get("expert", "")
)
​
# outer_le: learner's answer → expert feedback (text approximation)
prompt = build_distill_expert_prompt_with_feedback_slot(q, mas_task).replace(
    DISTILL_FEEDBACK_SLOT, ctx.get("learner", "")
)

11. What We Learned and What’s Next

11.1 The Latent Space Gap is Real — and Groq Narrows It

The paper’s most important contribution — latent-space cross-agent communication — is genuinely inaccessible with API-hosted models. Our text-based approximation recovers some of the benefit (structured prompts force agents to address each other’s reasoning), but it cannot recover what was never verbalized.

However, using Groq with open-source models from the same families as the official checkpoints narrows this gap meaningfully in two ways:

1. Prompt alignment. The official prompts were designed and tested with Qwen and Llama models. Using Claude or GPT introduces a distribution mismatch between what the prompts expect and how the model responds. Groq’s Qwen and Llama models honour the official prompt structure more faithfully.
2. Outer link topology. While we can’t use the trained CrossModelAdapter weights, using models from the same families means the information geometry of the text outputs is more similar to what the adapters were trained to bridge. The slot-based approximation is a better substitute when both endpoints are in the correct model family.

11.2 Three Key Divergences to Watch For

If you’re porting this work to production, these are the most consequential correctness issues:

1. Expert produces a plan, not an answer (Distillation). This is the single most important structural point in the distillation pattern. Many implementations get this wrong. If the Expert produces a full answer, the Learner becomes a paraphraser rather than an executor — completely defeating the purpose of the pattern. rmas_official_bridge/ enforces this correctly; rmas/ does not.

2. Both Deliberation agents have tools. The official system gives both the Reflector and Tool-Caller the same tool-use system prompt. The Reflector can search and compute — it just tends to reason internally first and use tools to verify specific uncertainties. Most simplified implementations (including our rmas/) give tools only to the Tool-Caller, which breaks the bidirectional deliberation loop.

3. Mixture specialists should be independent. The official system runs all three specialists in parallel — they never see each other’s outputs before the Summarizer. Sequential execution (which both rmas/ and rmas_official_bridge/ use for simplicity) leaks the Math Specialist's output to the Code Specialist and Science Specialist, which is not intended. The fix is asyncio-based parallel execution with LangGraph's fan-out branching.

11.3 Groq’s QwQ and Qwen-Coder Are the Best Approximations Available

For the Mixture pattern, qwen-qwen3-32b on Groq is particularly interesting. Qwen3 is Qwen's reasoning-focused model — the same intellectual lineage as the DeepSeek-R1-Distill-Qwen model used as the official Math Specialist. When you use QwQ as the Math Specialist in rmas_official_bridge/, you're using a larger, more capable member of the exact model family that the official adapter was trained with.

Similarly, qwen-2.5-coder-32b-preview is literally the same product line as the official qwen-qwen3-32b Code Specialist — just a much larger version. The code reasoning style, tokenisation philosophy, and output format conventions are shared, making slot replacement more faithful than cross-family substitution.

11.4 The Structured Output Path

The remaining gap between text-based and latent-space communication can be partially closed by replacing free-form text between agents with Pydantic structured outputs. Instead of each agent returning a string, it returns:

class AgentOutput(BaseModel):
    answer:              str
    confidence:          float        # 0.0–1.0
    uncertainty_flags:   list[str]    # explicit uncertainty signals
    improvement_targets: list[str]    # what the next agent should focus on

The uncertainty_flags and improvement_targets fields force explicit articulation of what would be implicitly encoded in latent states. The downstream agent reads these structured fields rather than parsing free-form prose — a structured approximation of the continuous-valued outer link signal.

Getting Started

Option A: rmas/ (Claude API, custom prompts)

git clone https://github.com/plaban1981/rmas
cd rmas
uv sync
cp .env.example .env   # add ANTHROPIC_API_KEY
​
uv run main.py --pattern sequential --rounds 3
uv run main.py --pattern mixture --rounds 2
uv run main.py --pattern distillation --rounds 3
uv run main.py --pattern deliberation --rounds 2
uv run main.py --all
uv run main.py --pattern sequential --stream

>>uv run main.py --pattern deliberation --question "Explain the Architecture of Recursive Multi agent System"

╔═══════════════════════════════════════════════════════════════════════╗
║                                                                       ║
║    RecursiveMAS                                                       ║
║    Recursive Multi-Agent Systems  •  arXiv:2604.25917v1               ║
║                                                                       ║
║    Scaling agent collaboration through recursive latent refinement    ║
║                                                                       ║
╚═══════════════════════════════════════════════════════════════════════╝

C:\Users\nayak\Documents\rmas\.venv\Lib\site-packages\langgraph\cache\base\__init__.py:8: LangChainPendingDeprecationWarning: The default value of `allowed_objects` will change in a future version. Pass an explicit value (e.g., allowed_objects='messages' or allowed_objects='core') to suppress this warning.
  from langgraph.checkpoint.serde.jsonplus import JsonPlusSerializer
╭────────────┬──────────────────────────────────────────────────────────────────╮
│ Pattern    │ DELIBERATION  Reflector ↔ Tool-Caller (with web search & Python) │
│ Rounds     │ 2                                                                │
│ Question   │ Explain the Architecture of Recursive Multi agent System         │
│ Reflector  │ Claude  claude-sonnet-4-6                                        │
│ ToolCaller │ Groq  llama-3.3-70b-versatile                                    │
╰────────────┴──────────────────────────────────────────────────────────────────╯

─────────────────────────────────────────────────────────────────── Running DELIBERATION ───────────────────────────────────────────────────────────────────

  Rounds completed   2
  Time elapsed       52.54s
  Agents used        2

Agent Contributions:
╭─────────────────────────────────────────────────────────────────────── Reflector ────────────────────────────────────────────────────────────────────────╮
│ [Reflector — Round 2: Deep Recursive Reflection]  # 🪞 Refined Reflection on RecursiveMAS Architecture  ---  ## ✅ Validating Tool-Caller's Findings     │
│ The Tool-Caller's retrieval **confirms and aligns** with my Round 1 analysis. The five core components are consistent. However, I now apply **critical   │
│ s...                                                                                                                                                     │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────── ToolCaller ───────────────────────────────────────────────────────────────────────╮
│ The final answer to the question "Explain the Architecture of Recursive Multi agent System" is:  The Recursive Multi-Agent System (RecursiveMAS)         │
│ architecture is a complex system that consists of multiple layers and components. The core architectural components include:  1. **Specialized           │
│ Collaborativ...                                                                                                                                          │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

╭────────────────────────────────────────────────────────────── Final Answer (DELIBERATION) ───────────────────────────────────────────────────────────────╮
│                                                                                                                                                          │
│  The final answer to the question "Explain the Architecture of Recursive Multi agent System" is:                                                         │
│                                                                                                                                                          │
│  The Recursive Multi-Agent System (RecursiveMAS) architecture is a complex system that consists of multiple layers and components. The core              │
│  architectural components include:                                                                                                                       │
│                                                                                                                                                          │
│  1. **Specialized Collaborative Agents**: The system is built on specialized collaborative agents, each with a distinct cognitive function. These        │
│  agents work together to achieve a common goal.                                                                                                          │
│  2. **Orchestrator Agent**: The orchestrator agent is the master coordinator and decision-maker. It decomposes complex problems into sub-tasks, assigns  │
│  tasks to appropriate agents, manages recursion depth and termination, and synthesizes final consensus output.                                           │
│  3. **Tool-Caller Agent**: The tool-caller agent is the external world interface. It executes API calls, database queries, web searches, and retrieves   │
│  real-world data and evidence.                                                                                                                           │
│  4. **Reflector Agent**: The reflector agent is the deep internal reasoner. It analyzes problems without external tools, critiques and validates other   │
│  agents' outputs, forms hypotheses, and evaluates solutions.                                                                                             │
│  5. **Shared Context (Thought Store)**: The shared context is the memory backbone of the system. It stores the thoughts, evidence, and reflections of    │
│  all agents, allowing them to access and build upon each other's work.                                                                                   │
│                                                                                                                                                          │
│  The recursive engine is the defining architectural feature of RecursiveMAS. It is a feedback loop that allows agents to recurse deeper or converge      │
│  based on the problem and context. The recursion mechanism is defined as:                                                                                │
│                                                                                                                                                          │
│  f(Problem, Context_n) → Context_(n+1)                                                                                                                   │
│  where Context_(n+1) = Context_n ∪ {new_evidence, new_reflection}                                                                                        │
│  until: convergence_condition == TRUE                                                                                                                    │
│                                                                                                                                                          │
│  The system also includes a meta-framework for self-improving agents, which implements a three-phase iterative refinement architecture. This allows      │
│  agents to critique and improve their own outputs, making every decision and debuggable.                                                                 │
│                                                                                                                                                          │
│  Overall, the RecursiveMAS architecture is designed to provide a scalable and robust framework for multi-agent systems. It allows for continuous         │
│  adaptation without fine-tuning and enables a seamless transition from rigid hierarchical structures to decentralized networks.                          │
│                                                                                                                                                          │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Option B: rmas_official_bridge/ (Official prompts + Groq — closest to the paper)

# Also clone the official repo alongside rmas/
git clone https://github.com/RecursiveMAS/RecursiveMAS
​
# Add keys to .env
# ANTHROPIC_API_KEY=sk-ant-...     (for --provider anthropic)
# GROQ_API_KEY=gsk_...             (for --provider groq)
# TAVILY_API_KEY=tvly-...          (optional, better web search in deliberation)
​
# Run with Groq (open-source models, closest to official checkpoints)
uv run rmas_official_bridge/main.py --style sequential_light  --provider groq
uv run rmas_official_bridge/main.py --style sequential_scaled --provider groq
uv run rmas_official_bridge/main.py --style mixture           --provider groq --rounds 2
uv run rmas_official_bridge/main.py --style distillation      --provider groq --rounds 3
uv run rmas_official_bridge/main.py --style deliberation      --provider groq --stream
uv run rmas_official_bridge/main.py --all                     --provider groq
uv run rmas_official_bridge/main.py --style mixture --provider groq
uv run rmas_official_bridge/main.py --style deliberation --provider groq​
# Or run with Claude (Anthropic)
uv run rmas_official_bridge/main.py --style mixture --provider anthropic

>> uv run rmas_official_bridge/main.py --style mixture --provider groq --question "Explain the Architecture of Recursive Multi agent System"

╔════════════════════════════════════════════════════════════════════════════╗
║                                                                            ║
║    RecursiveMAS Official Bridge                                            ║
║    Official prompts from RecursiveMAS/prompts.py  •  arXiv:2604.25917v1    ║
║                                                                            ║
║    Provider: GROQ  — Anthropic Claude API  or  Groq open-source models     ║
║                                                                            ║
╚════════════════════════════════════════════════════════════════════════════╝

╭──────────┬────────────────────────────────────────────────────────────╮
│ Style    │ MIXTURE  Math(QwQ) + Code(Coder) + Science → Summarizer    │
│ Provider │ GROQ                                                       │
│ Models   │ Math=QwQ-32B · Code=Qwen-Coder-32B · Science/Sum=Llama-70B │
│ Rounds   │ 2                                                          │
│ Question │ Explain the Architecture of Recursive Multi agent System   │
╰──────────┴────────────────────────────────────────────────────────────╯

C:\Users\nayak\Documents\rmas\.venv\Lib\site-packages\langgraph\cache\base\__init__.py:8: LangChainPendingDeprecationWarning: The default value of `allowed_objects` will change in a future version. Pass an explicit value (e.g., allowed_objects='messages' or allowed_objects='core') to suppress this warning.
  from langgraph.checkpoint.serde.jsonplus import JsonPlusSerializer
───────────────────────────────────────────────────────────────────── Running MIXTURE ──────────────────────────────────────────────────────────────────────

  Rounds completed   2
  Time elapsed       17.55s
  Agents used        4

Agent Contributions:
╭────────────────────────────────────────────────────────────────────────── math ──────────────────────────────────────────────────────────────────────────╮
│ <think> Okay, let's tackle this problem. The user wants me to explain the architecture of a Recursive Multi-Agent System (RMAS) based on the feedback    │
│ provided.   First, I need to recall the key points from the feedback. The feedback detailed the RMAS as a hierarchical structure with agents composed of │
│ nested sub-agents. It mentioned task decomposit…                                                                                                         │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭────────────────────────────────────────────────────────────────────────── code ──────────────────────────────────────────────────────────────────────────╮
│ <think> Alright, I need to address the user's question about the architecture of a Recursive Multi-Agent System (RMAS) using the feedback provided. Let  │
│ me start by reviewing the feedback to understand what was done correctly and where improvements are needed.  The feedback from the previous summarizer   │
│ round provides a detailed breakdown of the RMAS…                                                                                                         │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────────────────────────────── science ─────────────────────────────────────────────────────────────────────────╮
│ A Recursive Multi-Agent System (RMAS) is a hierarchical, modular framework where agents are composed of nested sub-agents, enabling complex              │
│ problem-solving through task decomposition and coordination. This architecture is characterized by a tree-like hierarchy, where top-level agents         │
│ delegate tasks to subordinate agents, which may further delegate …                                                                                       │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────── summarizer ───────────────────────────────────────────────────────────────────────╮
│ To explain the architecture of a Recursive Multi-Agent System (RMAS), we need to consider its key components and how they interact to enable complex     │
│ problem-solving.  1. **Hierarchical Structure**: The RMAS is characterized by a hierarchical, modular architecture. This means that the system is        │
│ organized in a tree-like structure, where top-level age…                                                                                                 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

╭───────────────────────────────────────────────────────────────── Final Answer (MIXTURE) ─────────────────────────────────────────────────────────────────╮
│                                                                                                                                                          │
│  To explain the architecture of a Recursive Multi-Agent System (RMAS), we need to consider its key components and how they interact to enable complex    │
│  problem-solving.                                                                                                                                        │
│                                                                                                                                                          │
│  1. **Hierarchical Structure**: The RMAS is characterized by a hierarchical, modular architecture. This means that the system is organized in a          │
│  tree-like structure, where top-level agents delegate tasks to subordinate agents. These subordinate agents may further delegate tasks to their own      │
│  sub-agents, creating a recursive hierarchy.                                                                                                             │
│                                                                                                                                                          │
│  2. **Task Decomposition**: A fundamental aspect of RMAS is the decomposition of complex tasks into simpler, more manageable sub-tasks. This             │
│  decomposition is recursive, meaning that each sub-task can be further broken down into even simpler tasks until atomic tasks are reached. This process  │
│  allows for the efficient handling of complex problems by distributing them across multiple agents.                                                      │
│                                                                                                                                                          │
│  3. **Communication Mechanisms**: For the RMAS to function effectively, there must be efficient communication mechanisms in place. Agents interact with  │
│  each other through message-passing or shared knowledge bases. This communication can flow both upward and downward through the hierarchy, enabling      │
│  coordination and the sharing of information necessary for task execution.                                                                               │
│                                                                                                                                                          │
│  4. **Coordination Strategies**: To ensure that the actions of sub-agents are aligned with the overall goals of the system, coordination strategies are  │
│  essential. These can include negotiation protocols, where agents negotiate to achieve mutually beneficial outcomes, and consensus algorithms, which     │
│  help agents reach a collective decision.                                                                                                                │
│                                                                                                                                                          │
│  5. **Autonomy**: Each agent in the RMAS operates with a degree of autonomy, making decisions within its scope of responsibility. This autonomy is       │
│  crucial for the system's flexibility and ability to adapt to changing conditions. Agents can adjust their actions based on local information and        │
│  interactions with other agents.                                                                                                                         │
│                                                                                                                                                          │
│  6. **Recursive Nature**: The recursive nature of the RMAS allows it to scale and adapt to complex problems. Agents can spawn new agents of the same     │
│  type as needed, enabling the system to grow or shrink dynamically in response to changing requirements.                                                 │
│                                                                                                                                                          │
│  7. **Dynamic Creation and Termination**: Agents can be created or terminated dynamically as the system's needs evolve. This dynamic aspect adds to the  │
│  system's flexibility and ability to respond to new challenges or changes in the environment.                                                            │
│                                                                                                                                                          │
│  8. **Applications**: RMAS finds applications in various domains, including distributed computing, logistics, and AI planning. In these areas, the       │
│  ability to decompose complex tasks, coordinate actions, and adapt to changing conditions is particularly valuable.                                      │
│                                                                                                                                                          │
│  9. **Benefits and Challenges**: The RMAS offers several benefits, including scalability, modularity, and adaptability. However, it also presents        │
│  challenges, such as managing communication overhead, debugging complex nested hierarchies, and ensuring system stability.                               │
│                                                                                                                                                          │
│  In summary, the architecture of a Recursive Multi-Agent System is designed to facilitate the decomposition of complex tasks, coordination among         │
│  agents, and adaptability in dynamic environments. Its hierarchical, modular structure, combined with autonomous agents and recursive task               │
│  decomposition, makes it a powerful framework for tackling complex problems in various domains.                                                          │
│                                                                                                                                                          │
│  \boxed{A Recursive Multi-Agent System is a hierarchical, modular framework where agents recursively decompose tasks and coordinate nested sub-agents,   │
│  enabling scalable, adaptive problem-solving through autonomy and structured communication.}                                                             │
│                                                                                                                                                          │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Option C: Official RecursiveMAS (requires GPU)

git clone https://github.com/RecursiveMAS/RecursiveMAS
cd RecursiveMAS
pip install -r requirements.txt
# Requires: CUDA GPU, HuggingFace model downloads (~10–30 GB per style)
​
python run.py --style sequential_light --dataset math500 --device cuda
python run.py --style mixture          --dataset math500 --device cuda
python run.py --style distillation     --dataset math500 --device cuda
python run.py --style deliberation     --dataset math500 --device cuda

Decision Guide: Which Should You Use?

• Learning RecursiveMAS concepts — Recommendation: rmas/ — simplest, one dependency
• Reproducing paper prompts/structure — Recommendation: rmas_official_bridge/ --provider groq
• Production deployment, Claude preferred — Recommendation: rmas_official_bridge/ --provider anthropic
• Benchmark evaluation, have GPU — Recommendation: Official RecursiveMAS repo
• Math-heavy tasks, best accuracy — Recommendation: --style mixture --provider groq (Qwen3-32B as Math Specialist)
• Code generation tasks — Recommendation: --style mixture --provider groq (Qwen3–32B as Code Specialist)
• Fastest inference — Recommendation: --style sequential_light --provider groq (llama-3.3–70b-versatile)

References

1. Recursive Multi-Agent Systems — arXiv:2604.25917v1 (Xiyuan Yang, Jiaru Zou et al.)
2. Official RecursiveMAS GitHub
3. RecursiveMAS HuggingFace Collections
4. Groq Model Documentation
5. LangGraph Documentation
6. Anthropic Claude API
7. LangChain Groq Integration

connect with me