How we build a Self-Defending Edge Proxy: Brownout, Admission Control, and Circuit Breakers

Most proxies gives you options like set connection limit, watch it trip, get 503s. Whereas Spooky gives you mulitple admission layers so a spike on one route or one upstream does not take down everything else.

Wondering What is Spooky? Spooky is an open-source HTTP/3 (QUIC) edge proxy written in Rust that terminates QUIC connections and forwards to HTTP/2 backends.

This post walks through each layer from the outside in, with the exact config fields and the metrics you can watch in production.

Overload Pipeline

Request that fails overload admission are rejected immediately with 503 Serice Unavailable ; overload response include a Retry-After header.

Layer 1 — Brownout Route Shedding

Brownout is the most blunt overload control; when pressure becames too high, non-critical routes are shed entirely so critical traffic can continue flowing.

How it works?

The Brownout control watches adaptive_admission.inflight_percent — the percent/fraction of the adaptive limit currently in use. When the number crosses the triggered threshold, brownout activates. Any incoming request whose upstream is not in the core_routes list get a 503 immediately. This way you can keep your critical routes running.

The hysteresis gap between trigger (90%) and recover (60%) prevents flapping — the system won’t toggle on and off every few milliseconds under sustained load.

Config

resilience:
    brownout:
      enabled: true
      trigger_inflight_percent: 90   # activate when adaptive inflight percent ≥ 90
      recover_inflight_percent: 60   # deactivate when adaptive inflight percent ≤ 60
      core_routes:
        - payments
        - auth
        - health

Metrics to watch

spooky_overload_shed_by_reason_total{reason="brownout"}

A increase in above counter means your system is regularly hitting brownout trigger pressure.

Layer 2 — Adaptive Admission Gate

Adaptive admission is a dynamic traffic gate that accepts or rejects requests based on real-time overload and latency, reducing effective concurrency when the system slows down.

How it works

A more better way to understand Adaptive Admission via graph:

Config

resilience:
    adaptive_admission:
      enabled: true
      min_limit: 512
      max_limit: 4096
      decrease_step: 64
      increase_step: 16
      high_latency_ms: 500

Metric to watch

spooky_overload_shed_by_reason_total{reason="adaptive_admission"}

Layer 3 — Route Queue Cap

After adaptive admission, route queue caps provide per-route and global queue admission bounds.

How it works?

Config

resilience:
    route_queue:
      default_cap: 256
      global_cap: 1024
      caps:
        payments: 512
      shed_retry_after_seconds: 1

Metrics to watch

spooky_overload_shed_by_reason_total{reason="route_cap"}
spooky_overload_shed_by_reason_total{reason="route_global_cap"}

Layer 4 — Global Inflight Gate

Every request must acquire a permit from the global semaphore before forwarding. This is the system-wide hard ceiling.

Config

performance:
  global_inflight_limit: 4096

Metrics to watch

spooky_overload_shed_by_reason_total{reason="global_inflight"}

Layer 5 — Per-upstream inflight

Each upstream has its own semaphore. A thundering herd on one upstream cannot consume all global slots because the per-upstream gate can shed first.

Config

performance:
  per_upstream_inflight_limit: 1024

Metrics to watch

spooky_overload_shed_by_reason_total{reason="upstream_inflight"}

Layer 6 — Per-backend inflight

The finest-grained inflight gate is at HTTP/2 connection pool, one semaphore per backend. Default is 64 conncurrent requests per backend.

How it works

The permit is held until response headers are received from the backend, then released — so the cap limits how many concurrent requests are awaiting a response, not how many are still draining the body.

Config

performance:
  per_backend_inflight_limit: 64

Metric to watch

spooky_overload_shed_by_reason_total{reason="backend_inflight"}

Circuit Breaker

The circuit breaker operates orthogonally to the inflight gates. Instead of capping concurrency it stops routing to a backend that is actively failing, giving it time to recover.

How it works?

• CLOSED — normal operation; consecutive failures are counted.
• OPEN — all requests to this backend get an immediate error; no upstream connection is attempted.
• HALF-OPEN — after open_ms, a single probe request ( half_open_max_probes:1 ) is allowed through. Success closes the circuit; failure re-opens it.

Failures and successes are tracked per backend.

Config

resilience:
  circuit_breaker:
    enabled: true
    failure_threshold: 3         # consecutive failures before opening
    open_ms: 30000               # how long to stay open (30s)
    half_open_max_probes: 1      # concurrent probe requests in half-open

Observing everything at once

The /metrics endpoint (default 127.0.0.1:9901/metrics) exposes the full picture in Prometheus format.

Key Counters

# How many requests were shed, and why
spooky_overload_shed_by_reason_total{reason="brownout"}
spooky_overload_shed_by_reason_total{reason="route_cap"}
spooky_overload_shed_by_reason_total{reason="route_global_cap"}
spooky_overload_shed_by_reason_total{reason="global_inflight"}
spooky_overload_shed_by_reason_total{reason="upstream_inflight"}
spooky_overload_shed_by_reason_total{reason="backend_inflight"}
spooky_overload_shed_by_reason_total{reason="circuit_open"}
spooky_overload_shed_by_reason_total{reason="adaptive_admission"}
spooky_overload_shed_by_reason_total{reason="request_buffer_cap"}
spooky_overload_shed_by_reason_total{reason="response_prebuffer_cap"}
spooky_overload_shed_by_reason_total{reason="connection_cap"}

Putting it all together

Here is a realistic config that shows the overload layers wired up:

performance:
  global_inflight_limit: 4096
  per_upstream_inflight_limit: 1024
  per_backend_inflight_limit: 64

resilience:
  adaptive_admission:
    enabled: true
    min_limit: 512
    max_limit: 4096
    decrease_step: 64
    increase_step: 16
    high_latency_ms: 500

  route_queue:
    default_cap: 256
    global_cap: 1024
    caps:
      payments: 512
    shed_retry_after_seconds: 1

  circuit_breaker:
    enabled: true
    failure_threshold: 3
    open_ms: 30000
    half_open_max_probes: 1

  brownout:
    enabled: true
    trigger_inflight_percent: 90
    recover_inflight_percent: 60
    core_routes:
      - payments
      - auth
      - health

Under normal load the gates are mostly invisible. Under a spike:

1. Brownout may shed non-core routes when adaptive inflight pressure crosses trigger.
2. Adaptive admission can deny new requests dynamically.
3. Route queue caps can shed by route or globally.
4. Global and per-upstream inflight semaphores enforce fixed hard limits.
5. Per-backend inflight caps isolate overloaded backends.

At no point does a problem in one route, upstream, or backend propagate to the others.

You can check out spooky at -> https://github.com/Supernova-Labs-Org/spooky