DNS Resolution in K8s :From Service Discovery to Pod Routing- Complete Flow

When I first started working with Kubernetes, one thing puzzled me: containers could magically find each other by name, regardless of where they were running or how many times they restarted. Kubernetes seemed to handle everything automatically.

But how does a frontend Pod running on Node 1 find a database Pod on Node 3 just by querying “postgres”? How does traffic actually reach the right Pod when there are dozens of replicas? And what’s really happening when we create a Service?

In this deep dive, we’ll explore:

• How CoreDNS provides cluster-wide service discovery
• The role of label selectors in routing traffic to Pods
• How Services bridge the gap between names and IPs
• What changes (and what doesn’t) with different Service types

Part 1: The Foundation — Understanding K8s Networking

The Pod Networking Challenge

In Kubernetes, Pods are ephemeral. They get created, destroyed, and rescheduled constantly:

Time T0:
┌─────────────────────────────────────────────────┐
│ api-pod-1: 10.244.1.5  (Node 1)                │
│ api-pod-2: 10.244.2.8  (Node 2)                │
│ api-pod-3: 10.244.3.12 (Node 3)                │
└─────────────────────────────────────────────────┘

Time T1: api-pod-2 crashes, gets replaced
┌─────────────────────────────────────────────────┐
│ api-pod-1: 10.244.1.5  (Node 1)                │
│ api-pod-4: 10.244.1.20 (Node 1) ← New IP!      │
│ api-pod-3: 10.244.3.12 (Node 3)                │
└─────────────────────────────────────────────────┘

The Problem: If your frontend is configured to connect to 10.244.2.8, it breaks when that Pod is replaced. You can't hardcode Pod IPs!

The Solution: Kubernetes uses a two-layer approach:

1. Services — Provide stable virtual IPs (ClusterIPs)
2. DNS — Maps service names to those stable IPs

This is the foundation of service discovery in Kubernetes.

Part 2: CoreDNS — The Cluster DNS Server

What is CoreDNS?

CoreDNS is a flexible, plugin-based DNS server that runs as a Deployment inside your cluster. Think of it as your cluster’s internal phone book.

┌────────────────────────────────────────────────────────┐
│  kube-system Namespace                                  │
│                                                          │
│  ┌─────────────┐      ┌─────────────┐                 │
│  │  CoreDNS    │      │  CoreDNS    │                 │
│  │  Pod #1     │      │  Pod #2     │                 │
│  └─────────────┘      └─────────────┘                 │
│         ↑                     ↑                         │
│         └──────────┬──────────┘                         │
│                    │                                    │
│         ┌──────────▼────────────┐                      │
│         │  kube-dns Service     │                      │
│         │  ClusterIP: 10.96.0.10│                      │
│         └──────────┬────────────┘                      │
└────────────────────┼───────────────────────────────────┘
                     │
      All Pods query this IP for DNS

Checking CoreDNS in Your Cluster

# View CoreDNS pods
kubectl get pods -n kube-system -l k8s-app=kube-dns

# Output:
NAME                       READY   STATUS    RESTARTS   AGE
coredns-74ff55c5b-7mqxj   1/1     Running   0          45d
coredns-74ff55c5b-n8lkw   1/1     Running   0          45d

# View the kube-dns Service
kubectl get svc -n kube-system kube-dns

# Output:
NAME       TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE
kube-dns   ClusterIP   10.96.0.10   <none>        53/UDP,53/TCP   90d

Every Pod in your cluster is automatically configured to use 10.96.0.10 for DNS queries!

Inside a Pod’s DNS Configuration

Let’s see what a Pod actually sees:

# Create a test pod
kubectl run test-pod --image=busybox --restart=Never -- sleep 3600

# Check its DNS config
kubectl exec test-pod -- cat /etc/resolv.conf

Output:

nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5

Breaking this down:

1. nameserver 10.96.0.10 — All DNS queries go to CoreDNS
2. search domains — Automatic completion for short names

• default.svc.cluster.local - Services in same namespace
• svc.cluster.local - Services in any namespace
• cluster.local - Cluster domain

3. ndots:5 — Query behavior control (we’ll explore this later)

Part 3: Services — The Stable Endpoint Layer

Before DNS can work, we need something stable to resolve to. Enter: Services.

Creating Pods with Labels

First, let’s create some Pods. The key here is labels — they’re how Services find Pods:

yaml

# api-pods.yaml
apiVersion: v1
kind: Pod
metadata:
  name: api-pod-1
  labels:
    app: api           # Label 1
    version: v1        # Label 2
    tier: backend      # Label 3
spec:
  containers:
  - name: api
    image: hashicorp/http-echo
    args:
    - "-text=Hello from Pod 1"
    - "-listen=:8080"
    ports:
    - containerPort: 8080
---
apiVersion: v1
kind: Pod
metadata:
  name: api-pod-2
  labels:
    app: api
    version: v1
    tier: backend
spec:
  containers:
  - name: api
    image: hashicorp/http-echo
    args:
    - "-text=Hello from Pod 2"
    - "-listen=:8080"
    ports:
    - containerPort: 8080
---
apiVersion: v1
kind: Pod
metadata:
  name: api-pod-3
  labels:
    app: api
    version: v1
    tier: backend
spec:
  containers:
  - name: api
    image: hashicorp/http-echo
    args:
    - "-text=Hello from Pod 3"
    - "-listen=:8080"
    ports:
    - containerPort: 8080

Apply it:

kubectl apply -f api-pods.yaml

# Check the pods
kubectl get pods -o wide

Output:

NAME        READY   STATUS    RESTARTS   AGE   IP            NODE
api-pod-1   1/1     Running   0          10s   10.244.1.5    node-1
api-pod-2   1/1     Running   0          10s   10.244.2.8    node-2
api-pod-3   1/1     Running   0          10s   10.244.3.12   node-3

Creating a Service with Label Selectors

Now let’s create a Service that uses label selectors to find these Pods:

# api-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: api-service
spec:
  selector:
    app: api       # Match Pods with app=api
    version: v1    # AND version=v1
  ports:
  - protocol: TCP
    port: 80         # Service port (what clients connect to)
    targetPort: 8080 # Pod port (where traffic goes)
  type: ClusterIP

Apply it:

kubectl apply -f api-service.yaml

# Check the service
kubectl get svc api-service

Output:

NAME          TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
api-service   ClusterIP   10.100.50.10   <none>        80/TCP    5s

Part 4: The Magic — How Label Selectors Route to Pods

The Endpoints Object

When you create a Service with a selector, Kubernetes automatically creates an Endpoints object with the same name:

kubectl get endpoints api-service

Output:

NAME          ENDPOINTS                                      AGE
api-service   10.244.1.5:8080,10.244.2.8:8080,10.244.3.12:8080   30s

Let’s look at the full object:

kubectl get endpoints api-service -o yaml

apiVersion: v1
kind: Endpoints
metadata:
  name: api-service  # Same name as the Service!
subsets:
- addresses:
  - ip: 10.244.1.5
    targetRef:
      kind: Pod
      name: api-pod-1
  - ip: 10.244.2.8
    targetRef:
      kind: Pod
      name: api-pod-2
  - ip: 10.244.3.12
    targetRef:
      kind: Pod
      name: api-pod-3
  ports:
  - port: 8080
    protocol: TCP

this is the bridge between the service and the pod

The Endpoint Controller (running in kube-controller-manager) continuously:

1. Watch all Services with selectors
2. For each Service:

• Query: “Find all Pods matching these labels”
• Filter: Only include READY Pods
• Extract: Get their IP addresses
• Update: Write to Endpoints object

1. **kube-proxy watches Endpoints**
2. **kube-proxy updates iptables/IPVS** rules

Tracing the process of endpoint creation

# Watch the endpoints in real-time

kubectl get endpoints api-service -w

In another terminal:

# Delete a pod
kubectl delete pod api-pod-2

Back in the watch terminal, you'll see:

NAME          ENDPOINTS                                AGE
api-service   10.244.1.5:8080,10.244.3.12:8080        1m
               ↑
               └── Pod 2 removed immediately!

The Endpoint Controller detected that api-pod-2 is gone and updated the Endpoints object instantly.

Testing Label Selector Matching

# See which Pods match the Service selector
kubectl get pods -l app=api,version=v1

# Output:
NAME        READY   STATUS    RESTARTS   AGE
api-pod-1   1/1     Running   0          5m
api-pod-2   1/1     Running   0          5m
api-pod-3   1/1     Running   0          5m

apiVersion: v1
kind: Pod
metadata:
  name: api-pod-v2
  labels:
    app: api
    version: v2      # Different version!
    tier: backend
spec:
  containers:
  - name: api
    image: hashicorp/http-echo
    args:
    - "-text=Hello from v2"
    - "-listen=:8080"
    ports:
    - containerPort: 8080

kubectl apply -f api-pod-v2.yaml

# Check if it's in the endpoints
kubectl get endpoints api-service

Output:

NAME          ENDPOINTS                                      AGE
api-service   10.244.1.5:8080,10.244.2.8:8080,10.244.3.12:8080   10m

Notice: The v2 Pod is NOT included! The selector requires version: v1.

# Part 5: DNS Resolution — Putting It All Together

Now let's see how DNS resolves service names to ClusterIPs.

The Complete DNS Resolution Flow

┌──────────────────────────────────────────────────────────┐
│  STEP 1: Application Makes Request                       │
│  ┌────────────────────────────────┐                      │
│  │  Client Pod                     │                      │
│  │  curl http://api-service        │                      │
│  └────────────┬───────────────────┘                      │
└───────────────┼──────────────────────────────────────────┘
                │
                ▼
┌──────────────────────────────────────────────────────────┐
│  STEP 2: DNS Lookup Triggered                            │
│  ┌────────────────────────────────┐                      │
│  │  /etc/resolv.conf says:        │                      │
│  │  nameserver 10.96.0.10         │                      │
│  │  search default.svc.cluster... │                      │
│  └────────────┬───────────────────┘                      │
└───────────────┼──────────────────────────────────────────┘
                │
                ▼
┌──────────────────────────────────────────────────────────┐
│  STEP 3: Apply Search Domains                            │
│  'api-service' has 1 dot (< 5)                           │
│  Try: api-service.default.svc.cluster.local ✓            │
└───────────────┼──────────────────────────────────────────┘
                │
                ▼
┌──────────────────────────────────────────────────────────┐
│  STEP 4: Query CoreDNS                                   │
│  DNS query to 10.96.0.10:53                              │
│  Question: IP of api-service.default.svc.cluster.local?  │
└───────────────┼──────────────────────────────────────────┘
                │
                ▼
┌──────────────────────────────────────────────────────────┐
│  STEP 5: CoreDNS Processes Request                       │
│  ┌────────────────────────────────┐                      │
│  │  1. Parse query                │                      │
│  │  2. Query Kubernetes API:      │                      │
│  │     "Get Service 'api-service' │                      │
│  │      in namespace 'default'"   │                      │
│  │  3. K8s returns ClusterIP      │                      │
│  │  4. Build DNS response         │                      │
│  └────────────┬───────────────────┘                      │
└───────────────┼──────────────────────────────────────────┘
                │
                ▼
┌──────────────────────────────────────────────────────────┐
│  STEP 6: DNS Response                                    │
│  Answer: api-service = 10.100.50.10                      │
│  TTL: 30 seconds                                         │
└───────────────┼──────────────────────────────────────────┘
                │
                ▼
┌──────────────────────────────────────────────────────────┐
│  STEP 7: Connect to Service ClusterIP                   │
│  Client connects to 10.100.50.10:80                      │
└───────────────┼──────────────────────────────────────────┘
                │
                ▼
┌──────────────────────────────────────────────────────────┐
│  STEP 8: kube-proxy Routes to Pod                       │
│  iptables rules route to one of:                         │
│  - 10.244.1.5:8080  (Pod 1)                             │
│  - 10.244.2.8:8080  (Pod 2)                             │
│  - 10.244.3.12:8080 (Pod 3)                             │
│  ✅ Request succeeds!                                    │
└──────────────────────────────────────────────────────────┘

Testing DNS Resolution

# Test DNS from our test pod
kubectl exec test-pod -- nslookup api-service

Output:

Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      api-service
Address 1: 10.100.50.10 api-service.default.svc.cluster.local

Perfect! DNS returned the Service's ClusterIP.

Making a Request

# Make an HTTP request using the service name
kubectl exec test-pod -- wget -qO- http://api-service

Output:

Hello from Pod 2

Run it again:

kubectl exec test-pod -- wget -qO- http://api-service

Output:

Hello from Pod 1

Load balancing in action! Each request hits a different Pod.

Part 6: How Traffic Actually Reaches Pods (kube-proxy)

The Missing Link

We know DNS gives us the ClusterIP (10.100.50.10), but ClusterIP is just a virtual IP - it doesn't actually exist on any network interface! So how does traffic reach the real Pods?

Answer: kube-proxy creates iptables rules that intercept traffic destined for ClusterIPs and redirects it to actual Pod IPs.

┌─────────────────────────────────────────────────────────────────┐
│                                                                 │
│  CLIENT POD                                                     │
│  "I want to connect to 10.100.50.10:80"                        │
│                                                                 │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼

         Packet leaves pod with destination:
                  10.100.50.10:80
                             │
                             ▼

╔═════════════════════════════════════════════════════════════════╗
║                                                                 ║
║  NODE'S NETWORK STACK                                          ║
║  (iptables rules created by kube-proxy)                        ║
║                                                                 ║
║  ┌───────────────────────────────────────────────────────────┐ ║
║  │ Rule: If destination = 10.100.50.10:80                    │ ║
║  │       Then randomly pick a Pod:                           │ ║
║  │                                                            │ ║
║  │       • 33% → Change to 10.244.1.5:8080                  │ ║
║  │       • 33% → Change to 10.244.2.8:8080                  │ ║
║  │       • 33% → Change to 10.244.3.12:8080                 │ ║
║  └───────────────────────────────────────────────────────────┘ ║
║                                                                 ║
╚═════════════════════════════════════════════════════════════════╝
                             │
                             ▼

         Packet now has destination:
                  10.244.2.8:8080
                             │
                             ▼

┌─────────────────────────────────────────────────────────────────┐
│                                                                 │
│  DESTINATION POD                                                │
│  Routed to actual Pod on Node 2                                │
│  ✅ Connection established!                                     │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

Conclusion

Understanding Kubernetes DNS reveals the elegant simplicity behind what initially seems like magic. When you type curl http://api-service, a precisely orchestrated chain unfolds: CoreDNS resolves the name to a stable ClusterIP, label selectors connect the Service to matching Pods through Endpoints, and kube-proxy's iptables rules route traffic to a healthy Pod — all in milliseconds.

The beauty lies in the abstraction. Pods crash, scale, and move between nodes, yet your application code never changes. DNS always returns the same ClusterIP, while the Endpoint Controller quietly updates which Pods receive traffic behind the scenes.

This knowledge transforms how you debug: slow DNS? Check ndots. No endpoints? Verify label selectors. Connection refused? Inspect Pod readiness probes. You're no longer guessing — you understand the system.