One MCP Gateway, N Services, Zero Integration Code

Harbor

How to scale an MCP gateway with your API estate — not your engineering team.

[Update — June 2026]

Harbor has now been open-sourced and is available on

GitHub: *https://github.com/vijaydeepsinha/harbor*

Feel free to explore, use, and contribute to the project.

Harbor is an MCP (Model Context Protocol) gateway: a single entry point where AI clients connect once, then call your backends through a small, fixed set of tools instead of a separate tool for every HTTP route.

The wall: “one tool per endpoint” does not scale

If you’ve built anything on top of the Model Context Protocol (MCP), you almost certainly started where most of us did: one MCP tool per backend endpoint.

It works beautifully for the first ten endpoints.

At twenty, the tool manifest starts feeling heavy. Every session begins with the AI reading lot of tool definitions before the user has typed a word.

At fifty, the model spends more tokens scanning the manifest than it spends answering — and tool-selection accuracy quietly collapses. When three near-identical tools compete for the same request, the wrong one wins more often than you’d like to admit.

At a hundred, you’ve rebuilt your OpenAPI spec by hand inside MCP. Every new endpoint ships as a gateway pull request. The team that owns the gateway becomes a bottleneck for every team that owns a service. You’ve built exactly the kind of coupling you were trying to avoid.

And before any of those gradual failures bite, there’s a harder ceiling waiting: most AI clients cap the number of tools they will register per server. Tool-per-endpoint crashes into that ceiling while your API estate is still small — long before the selection-accuracy problem even gets a chance to matter.

That’s the scaling wall. It isn’t a performance wall. It’s a design wall. And once you’ve hit it, the shape of the problem turns out to be wrong:

You should not be exposing endpoints. You should be exposing capabilities.

Code Mode: fewer tools, more room for the model to think

MCP Code Mode (described well by Anthropic and **Cloudflare) gives the model a sandbox and a small, stable tool surface. The model writes short code** that reads your API contract (for example, OpenAPI) and calls the endpoints it needs — instead of choosing among dozens of route-shaped tools every turn.

In plain terms: what to do next happens inside that small program. The client talks to the MCP server about running and updating code, not about stepping through a giant tool menu each time. That cuts manifest bloat and brittle “which tool?” mistakes at the AI boundary.

Code Mode mainly solves the AI-side surface. It does not by itself solve the gateway-side onboarding story.

Without a deliberate gateway design, each new backend still tends to mean custom auth, HTTP wiring, resilience, registration in the tool layer, and a new gateway release — a repeating week of work per service. Teams hit that wall, fall back to one-tool-per-endpoint because it is easy to add once, and the cycle repeats.

Code Mode helps the model scale. This article is about helping the gateway scale. Harbor is one way to do that.

A detour: why five tools, not two

Before we get to the onboarding story, a design decision worth naming — because it shapes everything that follows.

Most public Code Mode writing collapses the pattern to two tools: read the spec, call the API. That framing is clean, pedagogically useful, and — in production — under-delivers.

Harbor exposes five tools —

Harbor tools

The count is not the point. The design principle behind the split is:

*Each tool is a capability boundary, not a function.* Tools divide where their sandboxes need different injected capabilities.**

Three consequences:

Routing gets its own step. discover_services helps the model match user intent to the right service before any service-scoped work. That reduces “wrong backend” mistakes at scale.

Reading and doing must live in separate isolates. search_code Spec search runs without network access. The api_execute runs with network, without handing the whole spec into the same sandbox as untrusted prompt content. That split supports a security boundary you would blur if you merged everything into one mega-tool.

Skills need two tools, not one. discover_skills returns metadata the AI filters over; get_skill_details returns full Markdown; Listing metadata separately from loading full Markdown saves tokens when only one skill matters.

If your world is small enough that two tools are truly enough, use two. When routing, isolation, and token cost differ, extra tools are paying for boundaries, not for ceremony.

The primitive: folder-as-service

Services dock into Harbor by adding a folder — no TypeScript in the gateway, no per-endpoint tool registration, no framework PR for each API.

services/
  campaigns/
    spec.yaml          # OpenAPI (YAML or JSON)
    config.json        # host, auth, resilience, refresh
    skills/            # optional — team playbooks 
      launch-campaign.md
      pause-and-resume.md

On startup, Harbor scans services/, reads each config.json, wires the strategies you named, and the service is available through the same five tools as everything else.

A minimal config.json is mostly data—a type field picks a built-in implementation for auth, spec loading, circuit breaking, idempotency, and so on:

{
  "description": "Campaigns service — create, schedule, launch, pause, and archive marketing campaigns across channels.",
  "api": {
    "host": "campaigns.internal",
    "port": 8090,
    "basePath": "/v1"
  },
  "auth": {
    "type": "oauth-introspection",
    "host": "auth.internal",
    "port": 8083,
    "introspectionPath": "/oauth/auth",
    "refreshPath": "/oauth/token"
  },
  "circuitBreaker": {
    "type": "count-based",
    "failureThreshold": 5,
    "recoveryTimeMs": 30000
  },
  "idempotency": {
    "type": "memcache",
    "idempotencyKeyTtlMs": 600000
  },
  "spec": {
    "source": "url-with-fallback",
    "url": "http://campaigns.internal:8090/openapi.json"
  },
  "serviceRefreshIntervalMs": 300000
}

Two fields do outsized work:

• description — Plain language returned when the AI asks which service fits the user’s request. Write it like you would explain the service to a new teammate; good text improves first-shot routing without extra retrieval plumbing.
• serviceRefreshIntervalMs — On each tick, Harbor can reload the OpenAPI spec and rescan skills/ and swap them in atomically. Teams can ship API and guidance together without restarting the gateway or updating every AI client by hand.

One pattern: every cross-cutting concern is a strategy

Inside Harbor, auth, spec loading, circuit breaking, idempotency, token cache, and permissions are interfaces with multiple implementations. Each service picks implementations by name in config.json.

Cross Cutting Concerns

The tool layer talks to interfaces, not to one-off integrations. A new auth style is a new strategy implementation plus wiring — not a rewrite of the five tools. That is what makes “zero integration code” stay true as the catalog grows. Harbor’s strategy interfaces are narrow enough that new implementations slot in without any tool-layer awareness at all. Services published a year from now will onboard the same way as services published today.

Runtime topology — the shape of one process

Harbor Architecture

What matters:

1. One process, one port, many services — Operations stay simple: one place to expose, monitor, and secure.
2. Per-session tool bindings — The bearer token is bound to the session’s tool surface so a leaked session id alone does not grant someone else’s authorization context.
3. Per-service bundles — Each service gets its own spec store, skills, auth, HTTP client, breaker, and idempotency path. Failure in one service does not have to take neighbors with it.
4. Shared cache backend, isolated keys — One pool to Redis, Memcache, or Couchbase-style storage; keys are scoped so services do not read each other’s cache entries.
5. Sandbox per call — Each run that executes model-authored code uses a fresh V8 isolate, then disposes it. Short-lived sandboxes keep blast radius small.

Why the sandbox is safe

Handing the AI a JavaScript runtime is the part that usually makes security teams nervous. It shouldn’t — if the runtime is built correctly. Be specific about what “correctly” means.

The sandbox is a V8 isolate (via isolated-vm), not a Node VM. The distinction matters:

• It has its own V8 heap. The gateway’s Node heap is unreachable from inside.
• It has no fetch, no require, no process, no filesystem, no sockets. The only capability that exists inside the isolate is whatever the gateway explicitly injects at call time.
• It has a hard memory cap (64 MB by default, per-service overridable).
• It has a dual timeout — an inner CPU interrupt from V8 for runaway synchronous loops, and an outer wall-clock timer that disposes the isolate for runaway await chains.

Crucially, capabilities are scoped per tool, not per sandbox:

• The search sandbox sees the OpenAPI spec. It does not see the network.
• The skills sandbox sees the skill metadata. It does not see the spec or the network.
• The execute sandbox sees a single bridged api.request() function, routed to one specific service's HTTP client. It does not see the spec. It does not see other services.

These are separate isolates. The blast radius of a prompt-injection attack against the “reading” side cannot trivially cross into the “doing” side, because the doing side runs in a different isolate that never saw the attacker’s input.

The service owner doesn’t configure any of this. Sandbox policy is the gateway’s job. A service team writes an OpenAPI spec and a config file, and gets the security properties for free.

Horizontal scale: replicas, not rewrites

Configurable is not the same as horizontally scalable. Harbor aims for three operational ideas:

• State is either per-pod and cheap, or global and pluggable. Circuit-breaker state and in-flight refresh tracking are per-pod. That’s acceptable because an unhealthy pod trips its own breaker locally and takes itself out of rotation — a distributed breaker would add coordination cost for marginal benefit. The shared state — token cache, idempotency cache — is pluggable. In development you use the in-memory backend. In production you switch one environment variable to redis, memcache or couchbase etc and every replica starts sharing state. No application code changes.
• Cache-down never means gateway-down. Every cache path is written with one invariant: the cache may fail, but the request must still be serviceable. Read failures return “miss” — the gateway falls through to the upstream source. Write failures are logged at warn-level and the current result is returned anyway. You can lose the cache cluster and the gateway keeps serving. You pay in latency, not availability.
• Errors stay service-scoped — Because each service has its own auth middleware, its own circuit breaker, and its own HTTP client instance, a 5xx storm on the campaigns service cannot drag down tickets. The tool layer catches the service-scoped error, surfaces it to the AI with a structured machine-readable code (CIRCUIT_OPEN, API_ERROR, INTROSPECTION_FAILED, SESSION_EXPIRED, and so on), and the AI decides whether to retry, pivot, or explain the failure to the user.

Put these together and a single Harbor image can N stateless replicas behind a load balancer, each holding whatever subset of sessions sticky-routing sent its way, all sharing a token and idempotency cache, all addressing the same service catalog. Scaling out is a replica-count change. The framework itself does not care how many replicas exist.

Live refresh — bridge to Part 2

On each refresh interval, Harbor reloads the OpenAPI source, rescans skills/, and swaps both atomically so readers never see half-updated spec paired with stale skills (or the reverse).

That matters because APIs and “how we actually use them” often change together. Most documentation-plus-code pipelines do not have this property. They treat docs and the API as separate refresh streams, and the gap between them is exactly where bad AI behaviour lives. Harbor closes that gap by construction.

Part 2 goes deeper on skills: institutional rules next to the spec, how that changes behavior compared with prompt-only guidance, and how privacy can follow from the architecture — not from bolt-on policy alone.

When this pattern is not the fit

Straight talk: folder-based, zero-code gateway onboarding is not always the answer.

• Few endpoints, stable surface — Explicit per-route tools can be simpler; gateway overhead may not pay back.
• Clients that cannot run sandboxed code — Small models or strict “no code execution” policies need a deterministic tool layer instead.
• No trustworthy OpenAPI — The model only knows what the spec says; fix or generate truth there first.
• Truly one-shot, single-call flows — A sandbox may be more moving parts than you need.

For many internal services, different team cadences, and clients that can run short, audited code, Harbor’s shape — MCP gateway + Code Mode + folder onboarding + strategies — tends to earn its keep.

Closing thought — and where Part 2 picks up

The gateway is a substrate, not a product.

A substrate does not grow a new integration layer every time a team ships an API. When adding a service is drop a folder, write a clear description, tune config.json, restart (or rely on refresh where applicable)—and the framework binary stays stable while the catalog grows—you are treating the gateway as infrastructure.

The hard design work is drawing the line between what is fixed (tool surface, sandbox rules, session model) and what is pluggable (everything cross-cutting). Get that split right, and “scale the AI boundary” shifts from endless gateway projects to operations: replicas, caches, identity uptime.

**Part 2** opens the other half of the folder: skills/—the playbooks a new hire learns in month one.

See you there.