Day 143 — Cilium Ingress: A Smarter Entry Gate for AI Workloads and AI Governance

22nd May 2026, Netherlands — When we talk about AI workloads on Kubernetes, we often focus on GPUs, model training, inference latency, vector databases, and MLOps pipelines.

But one important question remains:

How does external traffic enter your AI platform safely, reliably, and with governance?

In Kubernetes, Ingress is the entry point for HTTP and HTTPS traffic coming from outside the cluster. For example, when a user accesses an AI chatbot, an internal ML API, a model inference endpoint, or a document analysis portal, that request needs to enter the Kubernetes cluster and reach the correct service.

Traditionally, this is handled by an Ingress controller such as NGINX, Traefik, HAProxy, or cloud-native load balancers. Cilium also provides Ingress support, but with a different architecture. Cilium Ingress is closely integrated with the Cilium networking stack and uses eBPF and Envoy to manage traffic entering the cluster. Cilium documentation explains that, unlike many traditional Ingress controllers deployed as separate workloads, Cilium’s Ingress traffic is intercepted by eBPF and transparently forwarded to Envoy.

For AI governance, this matters because the entry point is not just a routing layer. It becomes a control point for security, visibility, policy enforcement, and operational reliability.

What Is Kubernetes Ingress?

In simple words, Ingress is a Kubernetes object that defines how external HTTP or HTTPS traffic should reach internal services.

Example:

A user opens:

https://ai.example.com/chat

The request may need to go to:

ai-chatbot-service

Another request:

https://ai.example.com/embed

may need to go to:

embedding-api-service

Ingress helps define these routing rules.

A basic Kubernetes Ingress can manage:

Host-based routing
Path-based routing
TLS termination
Traffic forwarding to Kubernetes services

For AI platforms, this becomes very useful because one cluster may run many services:

Model inference API
RAG application
Vector database API
Feature store
MLflow tracking server
Prompt evaluation service
Internal admin dashboard

Without a proper Ingress strategy, traffic management becomes messy, insecure, and difficult to govern.

What Is Cilium Ingress?

Cilium Ingress is Cilium’s implementation of Kubernetes Ingress.

Cilium is already used as a CNI for Kubernetes networking, security, and observability. With Cilium Ingress, the same networking layer can also manage incoming HTTP and HTTPS traffic.

Cilium Ingress uses:

eBPF for high-performance networking
Envoy for Layer 7 HTTP traffic management
CiliumNetworkPolicy for traffic control
Hubble for network visibility
Kubernetes Ingress resources for routing definitions

Cilium’s official documentation highlights that Ingress and Gateway API traffic bound to backend services passes through a per-node Envoy proxy. This allows Cilium to apply Layer 7-aware traffic handling and policy controls. (Cilium Documentation)

This makes Cilium Ingress different from a traditional standalone Ingress controller.

In a traditional model, you usually have:

External Load Balancer
        ↓
Ingress Controller Pod
        ↓
Kubernetes Service
        ↓
Application Pod

With Cilium Ingress, the flow is more integrated with the networking layer:

External Traffic
        ↓
Cilium eBPF datapath
        ↓
Envoy proxy
        ↓
Kubernetes Service
        ↓
Application Pod

This tight integration is one of the key reasons Cilium Ingress is interesting for modern platform engineering and AI workloads.

Why AI Workloads Need Strong Ingress Governance

AI workloads are different from normal web applications in several ways.

They often handle:

Sensitive user prompts
Private documents
Embeddings
Model responses
Business data
Customer metadata
Training or fine-tuning datasets
Internal APIs
GPU-backed inference endpoints

This means AI traffic needs stronger governance.

Governance is not only about compliance documents. In platform engineering, governance means:

Who can access the service?
Which endpoint is exposed?
Is traffic encrypted?
Can we observe requests?
Can we enforce policy?
Can we restrict risky access?
Can we audit communication?
Can we prevent accidental public exposure?

Ingress is the first layer where many of these questions can be controlled.

If an AI inference API is accidentally exposed without proper routing, authentication, TLS, or network policy, it may lead to serious risk.

Examples:

A private LLM endpoint becomes publicly reachable.
A model serving API accepts traffic from unknown sources.
A development RAG app exposes internal documents.
An admin dashboard is available from the internet.
A vector database API is exposed without proper access control.

Cilium Ingress helps reduce these risks by bringing networking, security, and visibility closer together.

How Cilium Ingress Helps in AI Governance

1. Controlled Entry Point for AI Services

In an AI platform, not every service should be exposed publicly.

For example:

Public:
- AI chatbot frontend
- Public inference API

Private:
- Vector database
- Embedding service
- MLflow
- Internal admin dashboard
- Training pipeline APIs

Cilium Ingress allows platform teams to clearly define which services are exposed through Ingress.

Example routing design:

ai.example.com/chat        → chatbot-service
ai.example.com/inference   → model-inference-service
ai.example.com/docs        → rag-frontend-service

But internal services remain unexposed:

vector-db-service
embedding-worker-service
training-job-service
gpu-scheduler-service

This supports the basic governance principle:

Expose only what must be exposed. Keep everything else private by default.

2. Better Network Policy Integration

One powerful part of Cilium is its policy model.

Cilium supports Kubernetes NetworkPolicy and CiliumNetworkPolicy. With Cilium, traffic control can go beyond simple IP and port rules and include Layer 7 awareness for supported protocols.

Cilium documentation specifically discusses how Cilium Ingress configuration works with CiliumNetworkPolicy, because Ingress and Gateway API traffic bound to backend services passes through Envoy.

This is important for AI governance.

For example, you may want to allow:

Only Ingress traffic to reach the model inference API.
Only specific namespaces to call the embedding API.
Only approved services to access the vector database.
Only internal platform tools to access MLflow.

A governance-friendly model looks like this:

Internet
   ↓
Cilium Ingress
   ↓
Approved AI API Service
   ↓
Internal AI Services protected by Cilium policies

This creates a layered defense.

Ingress controls external entry. Network policies control internal movement.

Together, they reduce the blast radius.

3. TLS Termination and Secure Access

AI traffic may include sensitive information.

For example:

User prompts
Business documents
Customer queries
Internal knowledge base content
Generated answers
API keys in headers
Metadata about users or tenants

This traffic should not move as plain HTTP from the outside world.

Ingress commonly handles TLS termination, where HTTPS traffic is decrypted at the Ingress layer and then forwarded internally.

With Cilium Ingress, TLS can be configured through Kubernetes Ingress resources, allowing secure HTTPS access to AI services.

This helps enforce:

Encrypted user access
Centralized certificate handling
Standard HTTPS endpoints
Clear separation between external and internal traffic

For AI governance, this is not optional. It is a baseline requirement.

4. Path-Based Routing for Different AI Components

Modern AI applications are rarely one service.

A RAG-based document analysis platform may have:

/frontend
/upload
/chat
/embed
/search
/admin
/metrics

Each path may need to reach a different backend service.

Example:

/api/chat      → llm-orchestrator-service
/api/embed     → embedding-service
/api/search    → vector-search-service
/admin         → admin-dashboard-service

Cilium Ingress can route traffic based on host and path rules.

This helps platform teams organize AI applications cleanly.

Instead of exposing many LoadBalancer services, you can use one controlled entry point:

https://ai-platform.example.com

and route traffic internally based on path.

This gives better governance because entry is centralized.

5. Visibility into North-South AI Traffic

In Kubernetes networking, we usually talk about two traffic types:

North-South traffic: traffic entering or leaving the cluster
East-West traffic: traffic between services inside the cluster

Ingress mainly handles north-south traffic.

For AI governance, north-south visibility is critical.

You need to know:

Which clients are calling the AI API?
Which endpoint is receiving high traffic?
Are there unusual spikes?
Are failed requests increasing?
Is one tenant overusing the system?
Is the inference endpoint under pressure?

Cilium, together with Hubble, can provide visibility into service communication. This helps SREs and platform engineers understand traffic flow and troubleshoot faster.

For AI workloads, visibility supports:

Auditability
Security investigation
Capacity planning
Incident response
Cost control
SLO tracking

If an AI inference endpoint suddenly receives abnormal traffic, the platform team needs to detect it quickly.

Cilium Ingress and Gateway API

While this blog focuses on Cilium Ingress, it is also important to understand the future direction: Gateway API.

Gateway API is a Kubernetes SIG-Network project designed as a more expressive and role-oriented successor to the older Ingress API. The official Gateway API documentation describes it as a set of resources for service networking in Kubernetes, designed to be portable, expressive, extensible, and role-oriented.

Cilium supports both Kubernetes Ingress and Gateway API. Cilium’s Gateway API support also uses Cilium’s networking stack and Envoy.

In simple terms:

Ingress = simple and widely used
Gateway API = newer, more flexible, better separation of roles

For AI platforms, Gateway API may become more attractive when teams need advanced routing, ownership separation, and more structured traffic management.

But Ingress is still useful, simple, and widely understood.

Real-World AI Governance Scenario

Let us imagine an organization running an internal AI document analysis platform.

The platform has these components:

1. Frontend UI
2. RAG API
3. Embedding service
4. Vector database
5. LLM inference service
6. Admin dashboard
7. Observability dashboard

Without good governance, someone may expose too many services:

frontend-service           Public
rag-api-service            Public
embedding-service          Public
vector-db-service          Public
admin-dashboard-service    Public

This is risky.

A better design using Cilium Ingress:

Public through Ingress:
- frontend-service
- rag-api-service

Private inside cluster:
- embedding-service
- vector-db-service
- llm-inference-service
- admin-dashboard-service

Traffic flow:

User
 ↓
Cilium Ingress
 ↓
RAG API
 ↓
Embedding Service
 ↓
Vector Database
 ↓
LLM Inference Service

Governance rules:

Only RAG API can call embedding service.
Only embedding service can write embeddings.
Only RAG API can query vector database.
Only platform admins can access admin dashboard.
Only approved namespaces can call inference service.

This is where Cilium Ingress plus CiliumNetworkPolicy becomes powerful.

Ingress handles controlled external access. Network policies handle internal authorization boundaries. Observability helps audit and troubleshoot traffic.

Example: Simple Cilium Ingress Resource

Below is a simple example of a Kubernetes Ingress using Cilium as the Ingress class.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ai-platform-ingress
  namespace: ai-platform
spec:
  ingressClassName: cilium
  rules:
    - host: ai.example.com
      http:
        paths:
          - path: /chat
            pathType: Prefix
            backend:
              service:
                name: ai-chat-service
                port:
                  number: 80
          - path: /inference
            pathType: Prefix
            backend:
              service:
                name: model-inference-service
                port:
                  number: 80

This configuration means:

https://ai.example.com/chat
goes to ai-chat-service

https://ai.example.com/inference
goes to model-inference-service

This looks simple, but from a governance point of view it is powerful because it clearly defines what is reachable from outside the cluster.

Example: Governance Thinking Around This Ingress

Once the Ingress is created, the platform team should ask:

Should /chat be public?
Should /inference be public?
Should /admin be exposed?
Should only corporate VPN users access this?
Should we require authentication before traffic reaches the service?
Should rate limits be added?
Should request logs be monitored?
Should backend services accept traffic only from Ingress?

This is how Ingress becomes a governance discussion, not just a YAML file.

Important Governance Controls Around Cilium Ingress

1. Use a Dedicated Namespace

Keep AI workloads in a dedicated namespace:

ai-platform
ml-serving
rag-apps
model-inference

This makes it easier to apply policies, RBAC, and resource quotas.

2. Avoid Exposing Internal Services

Do not expose every service through Ingress.

Expose only user-facing or API-facing components.

Bad pattern:

Expose vector database through public Ingress

Better pattern:

Allow only approved internal services to access vector database

3. Use TLS for External Access

AI APIs should be exposed through HTTPS.

This protects sensitive prompts, responses, and metadata during transmission.

4. Combine Ingress with Network Policies

Ingress alone is not enough.

Use CiliumNetworkPolicy to control what happens after traffic enters the cluster.

Example principle:

Ingress can reach API service.
API service can reach inference service.
Inference service cannot directly access unrelated namespaces.

5. Observe Traffic Continuously

Use Cilium and Hubble to observe service communication.

Look for:

Unexpected external access
Rejected connections
Traffic spikes
Unusual service-to-service calls
High request failures
Latency changes

This is important for SREs managing AI workloads because AI systems can be expensive and sensitive.

Why SREs and Platform Engineers Should Care

For SREs and platform engineers, Cilium Ingress is not just another traffic routing tool.

It helps answer operational questions:

Is the AI API reachable?
Is traffic going to the correct service?
Is the model endpoint overloaded?
Are users seeing errors?
Is traffic encrypted?
Can we trace request flow?
Can we restrict access?
Can we prove which service talked to which service?

In AI platforms, these questions are directly linked to reliability and governance.

A poorly managed Ingress can cause:

Security exposure
Unexpected cloud cost
Model abuse
Data leakage
Uncontrolled API access
Difficult troubleshooting
SLO violations

A well-managed Ingress helps create:

Controlled access
Better visibility
Secure routing
Cleaner platform boundaries
More reliable AI services
Better governance posture

Cilium Ingress vs Traditional Ingress Controllers

Here is a simple comparison.

AreaTraditional Ingress ControllerCilium IngressDeployment modelUsually separate controller podsIntegrated with Cilium networkingData pathController receives and forwards trafficeBPF intercepts and forwards to EnvoyL7 proxyDepends on controllerEnvoyPolicy integrationOften separate from CNIWorks closely with Cilium policiesObservabilityDepends on controller toolingCan integrate with Cilium/Hubble visibilityGovernance valueRouting and TLSRouting, policy, visibility, and network control together

Cilium’s documentation specifically notes that its Ingress and Gateway API implementation is closely tied to the CNI, unlike many other controllers that run as separate Deployments or DaemonSets.

This close integration is what makes it attractive for platform teams that want networking, security, and observability together.

Best Practices for Cilium Ingress in AI Workloads

1. Start with a Clear Exposure Model

Before writing YAML, define:

Public endpoints
Internal endpoints
Admin endpoints
Restricted endpoints

Do not expose services by accident.

2. Use Separate Hostnames for Different Purposes

Example:

chat.ai.example.com
api.ai.example.com
admin.ai.example.com
mlflow.internal.example.com

This makes routing and access control easier.

3. Protect Admin and Internal Tools

Tools like MLflow, Grafana, Argo Workflows, model registries, and vector DB dashboards should not be publicly exposed unless there is a strong reason.

4. Use Network Policies After Ingress

After traffic enters the cluster, restrict lateral movement.

For example:

frontend → rag-api
rag-api → embedding-service
rag-api → vector-db
rag-api → inference-service

Do not allow:

frontend → vector-db
internet → vector-db
random namespace → inference-service

5. Monitor Ingress Traffic as Part of SLOs

For AI APIs, define SLOs such as:

99.9% successful responses for inference API
P95 latency under target threshold
Error rate below defined limit
Availability for chatbot frontend

Ingress traffic metrics and service visibility help detect when these SLOs are at risk.

The Bigger Picture: Cilium Ingress as Part of AI Platform Governance

AI governance is often discussed at a high level:

Responsible AI
Data privacy
Model risk
Access control
Auditability
Compliance

But in real platforms, governance becomes practical through infrastructure controls.

Cilium Ingress contributes to AI governance by helping platform teams control:

How users enter the AI platform
Which services are reachable
How traffic is routed
How access boundaries are enforced
How communication is observed
How risks are reduced

In other words:

AI governance is not only a policy document. It is also how your platform behaves at runtime.

Cilium Ingress helps convert governance principles into enforceable network behavior.

Conclusion

Cilium Ingress is more than a Kubernetes traffic routing feature.

For AI workloads, it becomes an important governance layer.

It helps platform engineers and SREs build controlled, observable, and secure entry points for AI applications. With its integration into Cilium’s eBPF-based networking stack and Envoy-based Layer 7 traffic handling, it provides a strong foundation for managing external access to AI services.

In simple terms:

Cilium Ingress controls how traffic enters.
Cilium policies control where traffic can go.
Cilium observability helps understand what is happening.
Together, they support better AI governance.

As AI workloads become more common in Kubernetes, platform teams need stronger control over traffic, security, and visibility.

Cilium Ingress is one practical step in that direction.