Never Hand Over the Keys, Make the Context Visible — Building “moeca”, a Desktop App for Securely Managing Multi-Agent Systems

~Designing AI Guardrails for Multi-Agent Systems~

Now that “boosting productivity with AI” has become the norm, you’ve probably run into some of these:

• The output isn’t what I asked for…
• There’s irrelevant data mixed into the output…
• It confidently returns results that are factually wrong…

And once a team starts using agents, a different kind of anxiety appears:

• Where is this agent sending our internal code and data?
• We’re handing it API keys — is that actually okay?

I’ve personally hit the “maybe AI isn’t as almighty as advertised” wall many times. And when I traced the causes, they rarely came down to raw model capability. Far more often, the root problem was that I couldn’t see what went into the context, or where the agent was connected to.

So I’ve been building moeca as a personal project: a single desktop app that covers multi-agent task management, execution, auditing, and context optimization end-to-end. This post walks through the overall picture and the two design areas I've invested in most: security and context optimization.

The repository with the relevant assets is public here: https://github.com/cancan007/moeca

The app is under active solo development. The major features all work end-to-end, but I’m continuously refining the details.

1. What is moeca?

1.1 Overview

moeca is a desktop app — currently in active solo development — for securely managing, running, and reviewing multiple AI agents.

moeca is a desktop app — currently in active solo development — for securely managing, running, and reviewing multiple AI agents.

• Form factor — Desktop app (Tauri + React + TypeScript)
• Backend — Go sidecars (gateway / hostagent / sandbox / registry proxy / RAG indexer)
• Agent runtime — Docker sandboxes (disposable, one per run)
• Screens — Delivery / Daily / Terminal / Knowledge / Audit / Settings

The concept in one sentence:

Don’t trust the agent. Instead, fully decide — on our side — where the agent can operate and what context it can see.

Before any more words, here’s a 27-second tour through the six screens.

moeca intro gif

The tour goes Delivery (review) → Daily (scheduled runs) → Audit (logs) → Knowledge (knowledge graph) → Settings. This is real data: 90 documents from four fictional services are indexed, shown right after running one agent task.

1.2 Main features

(1) Manage everything from task creation to execution in one place

There are two task tracks.

Delivery — tasks that produce code artifacts. A board of inbox → working (awaiting decision) → done, organized per repository.

• Each task gets its own git worktree, and the agent works only inside it
• A review drawer shows the diff / original (editable) / artifacts / evidence
• CI gate: self-review approval stays locked until CI passes
• Approval triggers a merge on the host side (merging is a host-privilege operation the agent can never perform)

Delivery page

One card = one task. The assigned agent, diff line counts, and CI status are shown on the card.

Delivery card detail

The CI gate is at the top; approval stays locked until it passes. The A2A log at the bottom shows which agent-to-agent exchange produced this diff.

Daily — recurring tasks. A calendar/gallery view listing generated artifacts (text, images, video, audio).

Daily calendar schedule view

• Schedules fire only while the app is running on your local machine
• Scheduled runs can be managed on a calendar

Daily gallery view

The artifact gallery. Text, images, audio, and video are color-coded by type and can be played or viewed in place.

(2) Execution agents are flexibly configurable

You create agent templates in Settings → Agents. There are three forms, and all of them are saved and directly runnable.

Agent templates settings view

• Solo — Standalone. Pick provider + model; optionally enable HTTP tools / RAG
• Static Multi-Agent — supervisor + graph. The graph compiles to a Stage DAG; the supervisor runs as plan → workers → integrate
• Dynamic Orchestration — Routes to templates at runtime

Each stage of the Stage DAG runs in its own independent sandbox, handing work off through a shared worktree. Stages launch in parallel — up to maxParallel — as soon as their dependsOn conditions are met.

Command template settings view

The key point: a stage specifies a “policy name” (base / poly / media), not an image reference. The actual image, network configuration, and resource limits are supplied by the controller. That means the hardening flags are identical for every image.

• **base — distroless, agent binary only (no shell, no toolchain**). Network: egress
• **poly** — Node 22 / Python 3 / Go 1.25 + shell (command stages for build/test). Network: egress
• **media — ffmpeg / ImageMagick / libvips (for Daily image/video generation). Network: none**

Keeping media out of the common image is deliberate. Media parsers consume untrusted binary input, and I don't want to widen the attack surface of every agent. As an independent stage it doesn't even need a network — it ends up more locked down than a normal sandbox.

Command template settings view

A template only decides “which agents run, in what order.” One node = one stage.

What each agent is allowed to use is decided in the Solo agent editor — the contents of each node.

Solo Agent template settings view

RAG, web search, image generation, audio generation, and video generation are granted per agent. I didn’t collapse the three media generators into one flag because their costs differ by an order of magnitude. And tool types that aren’t granted simply do not exist — not disabled, but absent. The model can’t see them, so it can’t even ask for them.

The media policy and the "video generation" tool are different things. The former is which box the code runs in (a sandbox image with ffmpeg etc., selected for "command" stages). The latter is which tool the agent is handed (permission to call generation APIs via the gateway). The names look similar, but one is the encoding side and the other is the generating side.

API Key settings views

API keys are write-only, stored in the OS keychain, and injected only by the gateway. They never appear to the agent — nor in localStorage.

(3) Agent-run logs at every granularity

The Audit tab lets you walk A2A-format logs with their hierarchy intact:

Context
└ Task
  ├ Artifact
  └ Message
    └ Part
Extensions / Metadata

You can filter by granularity, type, and time; token-count and session-count metrics live on the same screen.

The important part: agents cannot bypass this log. Agent traffic physically has no route except through the gateway (more on that below), so a “leave without leaving a log entry” path does not exist. Records are stored in an append-only SQLite with a hash chain (hash = SHA256(prevHash + record)), and /_gateway/audit/verify reports the first broken row. The consolidated security boundary doubles as a tamper-evident audit log that detects simple in-place modification.

A2A audit logs view

The tree on the left is Context → Task → Message → Part; the body is on the right. From a run-level row you can jump straight into the trace described in the next section.

(4) Optimize context while tracing from logs

This is moeca’s centerpiece. Hitting the trace button on an Audit run takes you to the Knowledge tab, where the knowledge that run actually reached is overlaid on the graph.

• Select a stage, and only the nodes that stage reached are focused
• Step through stages and watch the references shift
• Nodes that were never reached are visible → which directly supports the call: “this task didn’t need this group”
• In the right drawer, you can edit and save that stage’s system prompt on the spot

Trace — Audit run → reached Knowledge nodes → prompt fix

Audit run → Knowledge trace → in-place prompt fix, all on a single path.

One real example. For the task “The retry in src/retry.ts is also firing on 4xx. Investigate the rationale, then fix it," the agent searched 7 times and reached 26 of 90 nodes.

Reached (top hits):
  ×3  docs/payments/adr-001.md             ledger is append-only
  ×3  docs/payments/incident-2026-05-20.md incident from retry stampede
  ×2  media/payments/walkthrough.ja.vtt    ← video subtitles are hitting
  …(26 nodes total. The remaining 64 nodes were never reached)

The fact that 64 nodes stayed dark is the value of this screen. For any group that was never reached, you can say — with evidence — that this task didn’t need it.

Knowledge graph with reached nodes lit

Lit nodes are what this run reached. Dark nodes never came back in any search.

Prompt editing drawer over the trace

You can fix the system prompt in the right drawer while looking at the trace. The “impact scope” indicator shown on save is what the next paragraph is about.

What I was careful about on this screen: the UI must be honest.

We can only know “reached,” not “referenced.” What can be recorded is only that a search result was delivered to the agent. Whether the agent actually read it and used it in its judgment is unknowable. So the display is always an upper bound, and the UI copy consistently says reached / returned rather than referenced. Because log-size limits can truncate the tail, the UI also explicitly flags "partially unrecorded."

1.3 Current development status

Where things stand — all of the following are implemented:

• Security boundary (L3/L4/L7, key injection, audit log) — verified end-to-end from sandbox → agent execution
• Delivery (worktree / real diffs / CI gate / merge)
• Daily (schedules, artifacts, pulling external tickets)
• Agent templates (Solo / Graph / Supervisor / Dynamic) — saved and runnable
• Four artifact output types (text / image / audio / video)
• Knowledge tab (graph / region view / scope assignment)
• Trace-from-logs → RAG optimization loop — verified on real task runs

Beyond “task management through execution,” the review-and-fix loop now works end-to-end. From here it’s a matter of refining details through actual use.

2. The two problems I focused on, and how I designed for them

The problems I framed as central were, broadly:

• Guaranteeing security
• Transparency of context optimization

2.1 Security — don’t “deny”; remove the route entirely

As we’ve come to wire everything together — MCP and beyond — the number of surfaces where data can leak has multiplied. So I set the app’s policy as:

Secrets needed for external services live only where a security boundary can guarantee them wholesale. They are never handed to the agent, not even once.

And for the mechanism, instead of “denying” with firewall rules, I chose to make sure the route to reach anything simply doesn’t exist.

security architecture diagram

Three networks, and the gateway as the only entity that straddles them. What follows is a walkthrough of this diagram.

L3 (network layer) — the topology itself is the primary defense

The network is split into three:

• host — the local PC with env vars and assets you don’t want exposed
• orchestra-egress — the sandbox network where agents run and work. **--internal (fully private)**
• orchestra-upstream — the public network (NAT to the internet)

The linchpin is Docker’s **--internal. An --internal network has no route whatsoever to the host or the internet. Containers attached to it become an "island": for anything outside the island, a packet destination simply does not exist.**

Agent sandboxes are attached only to this egress island. As a result:

• They can’t reach the host (e.g., the management process on :8788) — because there is no route
• They can’t reach other sandboxes
• They can’t reach the internet

There is no firewall, so there are no firewall misconfigurations.

The single exception is the gateway, which alone belongs to two networks (i.e., dual-homed, with two NICs):

gateway interfaces (RFC1918 auto-assigned by Docker)
  eth0  172.20.0.2   <- orchestra-egress   (sandboxes reach the gateway here)
  eth1  172.21.0.2   <- orchestra-upstream (NAT to the internet)

“To speak on a network, you must have a foot (NIC) on that network” — this requirement is the isolation. Sandboxes only have a foot on egress, so they can reach the gateway’s egress-side IP, but have no foot on the upstream side or the internet beyond it. This is classic multi-homing — what routers and bastion hosts do.

A trap for implementers: published ports only take effect when the container starts with a non-internal network as its primary. If you start the gateway with the internal network as primary, -p 127.0.0.1:8787 gets recorded but never actually binds (docker port comes back empty). The correct order is: start with upstream as primary → publish with -p → then docker network connect egress.

L4 (transport layer) — “can’t raw TCP bypass L7?” → no

A common objection goes:

Inspection above HTTP (L7) can be defeated by hammering the TCP (L4) underneath, right?

The answer is no.

• TCP is a pipe (a byte stream to a port), and HTTP is one language you can speak through that pipe. True, TCP doesn’t force HTTP
• But what’s listening on :8787 is a Go HTTP server. Feed it non-HTTP bytes and you get **400 Bad Request or an immediate disconnect. There is no SOCKS/tunnel mode that forwards raw TCP through**
• In other words, “the only language spoken at the far end of that pipe is HTTP”

More fundamental still: L3 --internal isolation is protocol-agnostic. TCP, UDP, ICMP — without a route to the destination subnet, not one byte leaves.

What happens when a sandbox aims raw TCP at each destination:

• gateway (same egress subnet) → TCP connects, but anything non-HTTP is rejected
• host / internet → no route — the destination doesn’t exist
• another container on upstream → no NIC on that subnet — can’t even address it

Note that the host UI’s only path to the gateway is the loopback bind on 127.0.0.1:8787. Inside a sandbox, 127.0.0.1 is "the sandbox's own loopback," not the host's.

L7 (application layer) — the gateway governs where requests actually get forwarded

L3 funnels every request into the gateway; the gateway then decides destinations at the application layer. Cross-network traversal is an L7 proxy hop, not transparent IP forwarding. The sandbox sends the gateway an explicit HTTP request, and the gateway originates a fresh request out of its upstream-side NIC. That indirection is what makes inspection possible.

Every request passes through these gates:

1. Session auth (a short-lived session per run)
2. Routing (/anthropic/*, /github/*, /rag/* …)
3. Body-size limits / rate limits / token & cost budgets
4. Allowlist (positive list) — only configured upstreams
5. SSRF / host-reach deny (negative list) — docker host aliases like host.docker.internal, and loopback / RFC1918 / link-local IP literals are always blocked (dynamic targets are resolved before checking, as a DNS-rebinding countermeasure)
6. Write authorization (deny-by-default) — GitHub mutations are denied by default. Only branch creation / PR creation / comments pass; PR merges, pushes to protected branches, and direct commits are denied even with a valid token
7. Key injection — the agent sends nothing but Content-Type. The x-api-key or bearer header is injected by the gateway from its own memory

Dependency fetching (npm install / pip install / go mod download) follows the same philosophy via a separate registry proxy: GET/HEAD only (agents can fetch packages but can never publish), fixed upstreams, no credentials. That's how a sandbox with "no route to the internet" can still resolve dependencies.

Why keys never go in the sandbox — what L3 cannot protect

This is the part I most wanted to write. Two common misconceptions:

• “It’s isolated at L3, so putting keys in env vars is fine, right?”
• “Key injection is there to prevent host access?”

Both are wrong. If you separate the properties being protected, these two mechanisms are orthogonal:

• Can’t reach the wrong destination — handled by L3 + SSRF-deny. In one line: “can’t connect to weird places”
• The secret itself doesn’t leak through legitimate exits — handled by gateway key injection. In one line: “secrets don’t leave even through the correct exit”

Here’s the crux: L3 stops reaching arbitrary hosts — but the upstreams the gateway allows are still reachable. If the key lived in the sandbox’s environment variables, a compromised (or prompt-injected) agent could exfiltrate it through those permitted surfaces:

• Write the key into a worktree file → it lands in the repository at review/merge
• Embed the key in an allowed PR body or comment
• Smuggle the key inside a model-API request body (api.anthropic.com is allowlisted)

So “key in the sandbox + L3” is not equivalent to “key never enters in the first place.” Because the secret’s value never enters the trust boundary at all, there is simply nothing to carry out.

And “never hand over the keys” is a necessary condition, not a sufficient one. Even without the key, the agent is a deputy that can exercise the key’s capability through the gateway.

Effective privilege = min(scope of the injected key itself, gateway’s authorization policy)

So the design principle is least-privilege + attribution + revocability: minimize at the source (fine-grained tokens, budget caps), issue short-lived sessions per run, and unlock only the upstreams that run needs.

Let me also state the boundary explicitly. Gateway injection covers only “HTTP traffic that passes through the gateway.” Header injection can’t reach native, non-HTTP auth like an SDK that reads AWS_ACCESS_KEY_ID and does SigV4 signing, or SSH. There the options are (a) route that traffic through the gateway as a signing proxy too, or (b) if you absolutely must put it in env, treat short-lived, least-privilege as the precondition.

Summary (defense in depth)

• L3 network — --internal egress island + dual-homed gateway → sandboxes have no route to host / internet / other containers
• L4 port — gateway is published only on 127.0.0.1:8787 → host↔gateway is a single loopback door
• L7 app — auth → allowlist → SSRF-deny → write authorization → governs destinations; also blocks gateway-as-pivot host access
• Key injection — secrets live only in gateway memory, injected only at the exit → the secret’s value never enters the boundary = can’t leak even via legitimate exits
• Audit — single chokepoint + hash chain → unbypassable, tamper-evident record of all I/O

If you take away one thing, make it these two orthogonal properties:

1. Unreachability (L3) — the agent cannot reach the wrong destination. Not because rules deny it, but because no route exists.
2. Non-exfiltration (key injection) — the secret cannot leave even through the correct destination. Because the value never enters the boundary in the first place.

2.2 Context optimization — see what entered the context, then fix it

Premise: context has granularity

• System prompt (owned by the agent template)
• RAG (documents inside the repository) — folders you designate, e.g. under docs/
• RAG (fetched via the RAG server) — a vector index plus a human-drawn relation graph

Accidents almost always come from the third. If you can’t tell why something entered the context, you can’t fix it.

Sorting out index / entities / scope

I abstracted the design like this:

• reference — the reference data itself (local file / external HTTPS document). The entity that gets vectorized
• index — a way of pulling a reference. One reference can carry several
• group — a user-declared “region.” Also serves as a relation endpoint
• relation — a relationship between references. A line drawn explicitly by a human
• scope — restricts the retrievable range per task before RAG runs. global / project / organization

The design’s linchpin was this:

Even with vectors in place, what ultimately gets read is the original file itself. The index is a “means of arrival,” not the substance of the data.

And groups are treated as regions, not tags. They overlap, they nest, and a pure concept with no physical substance can also be a group. That’s why the UI draws them as circles (regions).

The hierarchy is organization ──1:N──▶ project ──1:N──▶ group. A task belongs to a project and carries an array of group tags it may reference.

Relations boost retrieval too — but must never cross permissions

Relations aren’t just lines on a graph; they’re used so that a search also pulls in the “connected range.”

1. Run a normal vector search; take the top hits
2. Collect the groups the hit chunks belong to
3. Collect the groups connected via relations
4. Filter down to permitted groups only          ← this order is non-negotiable
5. Re-search "the same query" within that range; add the top m chunks

Step 4’s position is the single most important design decision. If following a relation could return a non-permitted group, then drawing one line would be enough to bypass permissions. A relation is a hint about “what’s worth fetching” — never about “what you’re allowed to see.”

The re-search in step 5 exists because naively stuffing in chunks from connected groups is just noise. Relations only widen the candidate pool; ranking is left to similarity all the way through.

Implementation-wise, the gateway declares the session's groups in a header, and ragindex filters by it. Agents cannot spoof their own permissions (because it's the gateway, not the agent, doing the declaring).

One more thing the UI is strict about: “unset” and “empty” are opposites.

• Unset — no policy = everything is referenceable
• Empty — nothing is referenceable

Confuse these two and permissions invert, so the UI draws a hard distinction.

“How far to widen” lives on the template, not the task

• Group tags (what may be seen = permission) live on the task — because they change per task
• Widening amount (how to fetch = strategy) lives on the agent template — because “is this a broad-context reasoner?” is a working-style setting

And the knob is not “depth (hop count)” but a cap on the number of chunks added via relations. Depth depends too much on graph shape — one hop in a dense graph can exceed three hops in a sparse one. A count cap keeps behavior predictable as the shape changes. Depth survives only as a runaway-prevention ceiling.

The RAG settings UI splits into three modes

They’re different views of the same data, not different data.

• Node view — dots = references; lines = vector proximity (machine-computed closeness). For exploring and verifying
• Semantic view — groups drawn as circles with nodes inside; relations as arrows (human-declared meaning). For creating/editing groups & relations
• Assignment view — assignment to organization / project / group; scope operations. For permission assignment

Node view is "closeness the machine computed"; Semantic view is "meaning a human declared." The goal is for that contrast to land visually.

Knowledge Nodes view

Dots are references; lines are vector proximity. Threshold and line count are slider-adjustable.

Knowledge semantic view

Human-declared groups (regions) and relations (arrows) overlaid on the same nodes. Cross-cutting groups straddling per-service groups is exactly why these are regions, not tags.

Knowledge Assignment View

Nodes selected here become the search’s reference range as-is.

Operating cost of the vectors

Some real-world estimates:

• Data volume: vector payload : source text ≈ 1.7 : 1 (text-embedding-3-small = 1536 dims, default chunk size 1200 runes)
• Read time: on the order of tens of ms at 50k chunks (estimated for the Go implementation)

To prevent re-embedding storms, the cache key is:

key = sha256(chunk text + embedModel + maxChunkRune)

With this, re-embedding fires only when you change the embedding model or the chunk size.

The optimization loop

What it all adds up to:

1. Run the agent
2. The artifact plus the A2A session log that led to it are recorded
3. Trace from Audit → Knowledge and see which knowledge was reached
4. Fix it on the spot

• Edit the system prompt at the template’s granularity
• Fix the RAG (edit the source markdown / add it to the vector index / redraw groups & relations)

5. The next run gets better

If “the screen where you read logs” and “the screen where you fix things” are separate, you forget what you were looking at. That’s why I insist on fixing from the same screen.

3. Reflections

The strongest impression from building this: most quality problems with AI agents were problems of boundary design, not of the model.

• Weird output → you can’t see what entered the context
• Too scary to use → you can’t see what it’s connected to

Both reduce to “can’t see” — and making things visible requires changing the application’s structure. No amount of prompt cleverness changed that.

The technically delightful part was that security, auditing, and context optimization all converged on the same single chokepoint. The moment I chose “every ingress and egress goes through the gateway,” the place to inject keys, the place to record all I/O, and the place to record “which knowledge was returned” all became the same place. A structure built for security turned out to be, unchanged, the data source for optimization. I didn’t foresee this when I started designing, and it was an honestly pleasant discovery.

What was genuinely hard, on the other hand, was not lying in the UI. Rewriting “referenced” to “reached.” Drawing “unset” and “empty” as clearly distinct. Showing the impact scope on save. Each is unglamorous, but leave any of them ambiguous and users will misread permissions or impact — so these ate more time than the features themselves.

With the “trace from logs → optimize” loop now working end-to-end, the next phase is refining details through daily use. I’ll write again as things progress.

(The design described in this article is that of moeca, a secure multi-agent platform under personal development.)

Repository: https://github.com/cancan007/moeca