Building a Lightweight API Gateway with Envoy (Without the Complexities)

Static routes, rate limiting, and authentication filters via Lua scripting — no service mesh, no control plane, just a config file and a plan.

A few months back, our squad hit a familiar wall. We had six backend services — a mix of Go microservices handling payments, KYC, and merchant onboarding — and every single one of them was reinventing the same three things: routing, rate limiting, and token validation. Copy-pasted middleware everywhere. One service had a rate limiter bug that let a partner integration hammer our KYC endpoint until it fell over. That was the last straw.

The obvious answer was “put a gateway in front of everything.” The less obvious part was picking one that wouldn’t turn into a second job to operate.

We looked at Kong, Istio, and a couple of managed API gateway products first. Kong needed a database (or a not-quite-mature DB-less mode at the time) and its own admin API to babysit. Istio meant buying into an entire service mesh — sidecars, mTLS everywhere, a control plane, and a learning curve steep enough that half the team would’ve spent a sprint just reading docs. For six services and a few hundred requests per second, that felt like bringing a container ship to move a couch.

Envoy, on the other hand, could run as a single static binary with one YAML file. No control plane required if you don’t want one. That’s what pulled us in.

Why static config instead of xDS

Envoy is famous for its dynamic configuration story — xDS, service discovery, hot reloads pushed from a control plane like Istio’s Pilot or Consul. That’s genuinely powerful if you’re running hundreds of services across a mesh. We are not. We have a known, fairly stable set of upstream services, and our deploy pipeline can push a new Envoy config the same way it pushes anything else.

So we went with static bootstrap.yaml. Routes are declared upfront, listeners are fixed, and if we add a new backend, it's a config change plus a rolling restart — which for us happens in under ten seconds thanks to Envoy's fast startup and our existing Kubernetes rollout strategy.

Here’s a trimmed version of what our gateway config actually looks like:

static_resources:
  listeners:
    - name: listener_http
      address:
        socket_address: { address: 0.0.0.0, port_value: 8080 }
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: ingress_http
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: backend_services
                      domains: ["*"]
                      routes:
                        - match: { prefix: "/api/payments" }
                          route: { cluster: payments_service }
                        - match: { prefix: "/api/kyc" }
                          route: { cluster: kyc_service }
                        - match: { prefix: "/api/merchants" }
                          route: { cluster: merchant_service }
                http_filters:
                  - name: envoy.filters.http.lua
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
                      default_source_code:
                        inline_string: |
                          -- auth + rate-limit logic lives here
                  - name: envoy.filters.http.router

  clusters:
    - name: payments_service
      connect_timeout: 1s
      type: STRICT_DNS
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: payments_service
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address: { address: payments.internal, port_value: 9090 }

Nothing fancy. Three routes, three clusters, one HTTP connection manager. This is the entirety of what most small-to-mid teams actually need from a gateway layer.

Authentication, done in Lua instead of a sidecar

This was the part I was most skeptical about going in. Envoy’s Lua filter has a reputation for being a bit of an escape hatch — “if the built-in filters don’t do what you want, here’s a scripting language, good luck.” But for JWT validation and light request shaping, it turned out to be exactly the right amount of power.

Instead of standing up a separate auth microservice that every request has to hop through, we validate the token right in the filter chain:

function envoy_on_request(request_handle)
  local auth_header = request_handle:headers():get("authorization")

  if auth_header == nil then
    request_handle:respond(
      {[":status"] = "401"},
      "missing authorization header"
    )
    return
  end

  local token = string.match(auth_header, "Bearer%s+(.+)")
  if token == nil then
    request_handle:respond(
      {[":status"] = "401"},
      "malformed authorization header"
    )
    return
  end

  -- delegate signature + claims check to a lightweight internal validator
  local headers, body = request_handle:httpCall(
    "auth_validator_cluster",
    {
      [":method"] = "POST",
      [":path"] = "/validate",
      [":authority"] = "auth-validator.internal",
      ["content-type"] = "application/json"
    },
    '{"token":"' .. token .. '"}',
    200,
    false
  )

  if headers[":status"] ~= "200" then
    request_handle:respond({[":status"] = "403"}, "invalid token")
    return
  end

  request_handle:headers():add("x-user-verified", "true")
end

We kept the actual JWT signature verification in a small Go service — Lua isn’t the place to be parsing cryptographic claims — but the orchestration of “check the header exists, call the validator, reject or forward” lives entirely in Envoy. That single decision removed an entire hop from our request path for every rejected request, since invalid tokens never even reach a backend service now.

Rate limiting without standing up Redis

We initially assumed rate limiting meant deploying Envoy’s global rate limit service, which wants its own gRPC backend and usually a Redis instance behind it. For our traffic volume, that’s overkill. We went with Envoy’s local rate limit filter instead — token bucket, per-listener, no external dependency:

- name: envoy.filters.http.local_ratelimit
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
    stat_prefix: http_local_rate_limiter
    token_bucket:
      max_tokens: 100
      tokens_per_fill: 100
      fill_interval: 60s
    filter_enabled:
      runtime_key: local_rate_limit_enabled
      default_value:
        numerator: 100
        denominator: HUNDRED
    filter_enforced:
      runtime_key: local_rate_limit_enforced
      default_value:
        numerator: 100
        denominator: HUNDRED
    response_headers_to_add:
      - append: false
        header:
          key: x-local-rate-limit
          value: "true"

100 requests per minute per Envoy instance, enforced locally with no round trip to a shared store. It’s not perfectly accurate across a fleet of gateway replicas — each instance tracks its own bucket — but for our scale, that imprecision is a fair trade for zero added infrastructure. If we ever need cluster-wide accuracy, the global rate limit service is a drop-in upgrade path, not a rewrite.

A tiny Go sidecar for the auth validator

Since I mentioned it above, here’s the stripped-down version of the validator service the Lua filter calls out to. Nothing exotic — just fast JWT parsing and a clean HTTP surface:

package main

import (
 "encoding/json"
 "net/http"

 "github.com/golang-jwt/jwt/v5"
)

var secretKey = []byte("replace-with-real-secret-from-vault")

type validateRequest struct {
 Token string `json:"token"`
}

func validateHandler(w http.ResponseWriter, r *http.Request) {
 var req validateRequest
 if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
  http.Error(w, "bad request", http.StatusBadRequest)
  return
 }

 token, err := jwt.Parse(req.Token, func(t *jwt.Token) (interface{}, error) {
  return secretKey, nil
 })

 if err != nil || !token.Valid {
  http.Error(w, "invalid token", http.StatusForbidden)
  return
 }

 w.WriteHeader(http.StatusOK)
 w.Write([]byte(`{"status":"valid"}`))
}

func main() {
 http.HandleFunc("/validate", validateHandler)
 http.ListenAndServe(":9091", nil)
}

This service does one thing. It doesn’t know about routing, it doesn’t know about rate limits, and it barely knows about HTTP beyond parsing a JSON body. That narrowness is the point — it’s easy to test, easy to reason about, and easy to hand to someone new on the team.

Where the requests actually go

Here’s roughly how a request flows through the setup, start to finish:

Client
  │
  ▼
┌─────────────────────────┐
│   Envoy (port 8080)      │
│                           │
│  ┌─────────────────────┐ │
│  │ Lua filter            │ │
│  │  - check auth header  │──────► auth-validator (Go, :9091)
│  │  - reject or tag       │◄──────  200 / 403
│  └─────────────────────┘ │
│              │             │
│  ┌─────────────────────┐ │
│  │ Local rate limit       │ │
│  │  - 100 req/min bucket  │ │
│  └─────────────────────┘ │
│              │             │
│  ┌─────────────────────┐ │
│  │ Router                 │ │
│  │  /api/payments  ──────┼──────► payments_service
│  │  /api/kyc       ──────┼──────► kyc_service
│  │  /api/merchants ──────┼──────► merchant_service
│  └─────────────────────┘ │
└─────────────────────────┘

And the latency breakdown, once we put it under load with a modest synthetic traffic generator, looked like this:

p50 latency added by gateway layer
────────────────────────────────────
No gateway (direct)   │ 4ms
Envoy, static routing │██ 6ms
Envoy + Lua auth       │████ 9ms
Envoy + Lua + ratelim  │█████ 11ms
                        0    5    10   15

Eleven milliseconds of added latency at p50 for auth, routing, and rate limiting combined was a number the whole team was comfortable signing off on. Compare that to the round trip we used to pay for a dedicated auth microservice call from within each backend, and we actually came out faster.

What I’d tell someone starting from scratch

If you’re a small team drowning in duplicated middleware across services, you don’t need a service mesh to fix it. You need a request to have one place to get authenticated, one place to get rate limited, and one place to get routed. Envoy running as a static, stateless proxy in front of your services gets you there without asking you to adopt an entirely new operational model.

The Lua filter isn’t elegant. It won’t win any awards for readability compared to a proper Go or Rust extension. But it’s fast enough, it’s contained to the gateway layer, and it means your actual business logic never has to think about tokens or throttling again. Sometimes the “hacky” option is the one that ships and keeps shipping.

We’ve been running this setup in production for a while now, across the payments and KYC paths specifically because those were the ones that got burned first. No Redis, no control plane, no sidecars. Just a YAML file, a bit of Lua, and a Go service small enough to read in one sitting.