Quality of Service (QoS) in Kubernetes: Managing Resources for Application Stability

Why Quality of Service Matters in Kubernetes:

Kubernetes clusters often run many applications at the same time. Each application wants CPU and memory to do its work. Without proper controls, this creates a resource free-for-all that can bring down your entire system.

Here’s what happens without QoS controls:

• Non-critical applications could consume all the available resources
• Critical applications get starved for resources
• Important apps may run very slowly or even crash
• System stability becomes unpredictable

To prevent this chaos, Kubernetes uses Quality of Service (QoS). QoS helps manage resources fairly and ensures stability in the cluster. Here’s what it accomplishes:

• Critical apps always run: They get their reserved resources guaranteed
• Important apps get flexibility: They can use extra resources when available
• Non-critical apps use leftovers: They run only when resources are available and get evicted first during resource shortages

Let me give you a practical example. Imagine you run three applications:

• A Payment Service (critical for business)
• An Analytics Service (important but can be flexible)
• A Log Collector (helpful but non-critical)

With QoS, Kubernetes ensures the Payment Service always gets its reserved CPU and memory, the Analytics Service can burst and use more resources when available, and the Log Collector runs only when free resources exist. If the node runs out of resources, the Log Collector gets stopped first, protecting your critical payment processing.

Quality of Service: Balancing performance and resource efficiency

Understanding Quality of Service in Kubernetes:

Quality of Service (QoS) in Kubernetes is a way to control how CPU and memory are shared among applications running in the cluster. It is basically a priority system that makes sure critical applications receive the resources they need and do not make slow down or crash, especially during the resource contention.

The Three QoS Class Types:

When you run applications in Kubernetes, each Pod needs CPU and memory. But not all Pods are equally important — some are mission-critical, some are important but flexible, and others are just background tasks. To handle this fairly, Kubernetes automatically assigns one of three QoS classes based on whether you define resource requests (minimum guaranteed) and limits (maximum allowed) for CPU and memory:

1. Guaranteed — Highest priority
2. Burstable — Medium priority
3. BestEffort — Lowest priority

Let’s explore each class in detail.

1. Guaranteed QoS Class:

When you set the request equal to the limit for both CPU and memory, Kubernetes classifies the Pod as Guaranteed. These Pods receive the highest priority treatment because they always get exactly the resources you reserved for them.

Key characteristics:

• Highest priority in the cluster
• Always receive their requested resources
• Last to be evicted during resource pressure
• Perfect for mission-critical applications

When to use: Payment services, databases, authentication services, or any application where downtime costs money.

apiVersion: v1
kind: Pod
metadata:
  name: payment-app
spec:
  containers:
  - name: payment-container
    image: nginx
    resources:
      requests:
        cpu: "100m"
        memory: "500Mi"
      limits:
        cpu: "100m"
        memory: "500Mi"
    # Requests = Limits → Guaranteed QoS

2. Burstable QoS Class:

If you set a request lower than the limit, the Pod gets classified as Burstable. This means the Pod gets a guaranteed minimum amount of resources but can burst and use more when extra resources are available.

Key characteristics:

• Medium priority in resource allocation
• Guaranteed minimum resources (request values)
• Can use additional resources when available (up to limit values)
• Evicted after BestEffort but before Guaranteed Pods
• Great balance of efficiency and reliability

When to use: Analytics services, batch processing jobs, web applications that can handle occasional restarts.

apiVersion: v1
kind: Pod
metadata:
  name: analytics-service
spec:
  containers:
  - name: analytics-app
    image: python:3.9
    resources:
      requests:
        cpu: "50m"
        memory: "128Mi"
      limits:
        cpu: "100m"
        memory: "256Mi"
    command: ["python", "-c", "while True: print('running analytics')"]
    # Requests < Limits → Burstable QoS

3. BestEffort QoS Class:

When you don’t specify any requests or limits for CPU and memory, the Pod automatically gets the BestEffort class. These are the lowest priority Pods that can only run when there are free resources available.

Key characteristics:

• Lowest priority in the cluster
• No guaranteed resources
• Can use any available resources when the system is idle
• First to be evicted during resource pressure
• Most efficient use of cluster resources

When to use: Log collectors, monitoring agents, backup processes, development and testing workloads.

apiVersion: v1
kind: Pod
metadata:
  name: log-collector-app
spec:
  containers:
  - name: log-container
    image: busybox
    command: ["sh", "-c", "while true; do echo 'collecting logs'; sleep 5; done"]
    # No requests or limits → BestEffort QoS

At 100% resource usage, Kubernetes deletes pods based on QoS Class: BestEffort first, then Burstable, and finally Guaranteed.

Checking QoS Classes:

# Check a single pod's QoS class
kubectl get pod <pod-name> -o jsonpath='{.status.qosClass}'

# List all pods in a namespace with their QoS class
kubectl get pods -o custom-columns=NAME:.metadata.name,QOS:.status.qosClass

Practical Hands-On Example:

Let’s create a real-world scenario with three different applications, each demonstrating a different QoS class. We’ll build:

• Payment-app → Guaranteed QoS Class
• Analytics-app → Burstable QoS Class
• Log-collector-app → BestEffort QoS Class

Step 1: Create the Payment Service Pod (Guaranteed QoS)

Save the following manifest as payment-app.yaml:

# vi pyament-app.yaml

apiVersion: v1
kind: Pod
metadata:
  name: payment-app
spec:
  containers:
  - name: payment-container
    image: nginx
    resources:
      requests:
        cpu: "100m"
        memory: "500Mi"
      limits:
        cpu: "100m"
        memory: "500Mi"
    # Requests = Limits → Guaranteed QoS

Step 2: Create the Analytics Service Pod (Burstable QoS)

Save the following manifest as analytics-app.yaml:

# vi analytics-app.yaml

apiVersion: v1
kind: Pod
metadata:
  name: analytics-app
spec:
  containers:
  - name: analytics-app
    image: python:3.9
    resources:
      requests:
        cpu: "50m"
        memory: "128Mi"
      limits:
        cpu: "100m"
        memory: "256Mi"
    command: ["python", "-c", "while True: print('running analytics')"]
    # Requests < Limits → Burstable QoS

Step 3: Create the Log Collector Pod (BestEffort QoS)

Save the following manifest as log-collector-app.yaml:

# vi log-collector-app.yaml

apiVersion: v1
kind: Pod
metadata:
  name: log-collector-app
spec:
  containers:
  - name: log-container
    image: busybox
    command: ["sh", "-c", "while true; do echo 'collecting logs'; sleep 5; done"]
    # No requests or limits → BestEffort QoS

Step 4: Deploy All Applications

Run these commands to create all three Pods:

kubectl apply -f pyament-app.yaml
kubectl apply -f analytics-app.yaml
kubectl apply -f log-collector-app.yaml

Step 5: Verify Pod Status

Check that all Pods are running properly:

kubectl get pods

Step 6: Confirm QoS Classifications

Verify that Kubernetes assigned the correct QoS classes:

kubectl get pod payment-app -o jsonpath='{.status.qosClass}'
kubectl get pod analytics-app -o jsonpath='{.status.qosClass}'
kubectl get pod log-collector-app -o jsonpath='{.status.qosClass}'

Or use this single command to see all QoS classes at once:

kubectl get pods -o custom-columns=NAME:.metadata.name,QOS:.status.qosClass

You should see:

• payment-app: Guaranteed
• analytics-app: Burstable
• log-collector-app: BestEffort

7. Test eviction order of QOS Class on Kubernetes.

To do, we put pressure on Kubernetes nodes intentionally to see the pod eviction order.

login into the node and install stress package on the linux machine.

ssh <node-name>

apt install -y stress-ng   # on the node OS, not in the pod

Run following command to put pressure on Memory.

stress-ng --vm 1 --vm-bytes 150% --vm-keep --timeout 120s

Step 9: Monitor Pod Evictions

Watch the Pod status in real-time to see the eviction order:

kubectl get pods -w

Expected behavior during resource pressure:

• BestEffort Pod (log-collector-app) → Gets evicted first
• Burstable Pod (analytics-app) → Gets evicted next if needed
• Guaranteed Pod (payment-app) → Protected until absolutely no other choice

This demonstration shows exactly how QoS classes protect your critical applications during resource crises.

Step 10: Clean Up

Remove all test resources:

kubectl delete -f payment-app.yaml
kubectl delete -f analytics-app.yaml
kubectl delete -f log-collector-app.yaml
kubectl delete -f stress-test.yaml

1. What happens when a Guaranteed QoS pod exceeds its defined CPU or memory limits?

When a Guaranteed Pod exceeds its resource limits, Kubernetes enforces those limits very strictly — being “Guaranteed” doesn’t mean it can use more than what you defined. Here’s what happens:

Behavior of Guaranteed Pods:

CPU Limit Exceeded

• Kubernetes throttles the Pod’s CPU usage.
• The Pod won’t be killed, but its performance slows down because it cannot consume more CPU than the limit.
• Example: If you set cpu: 500m and the app tries to use 1 core, Kubernetes will cap it at 0.5 cores.

Memory Limit Exceeded

• Memory is different: if a Pod tries to use more memory than its limit, the container runtime (like Docker or containerd) will terminate the container with an OOMKilled (Out Of Memory) error.
• The Pod may restart depending on its restart policy.
• Example: If you set memory 1Gi and the app tries to use 1.5Gi, it will be killed immediately.

2. How does Kubernetes decide which specific pod to evict when multiple pods share the same QoS class?

Imagine you have two critical Pods:

• payment-app (Guaranteed QoS)
• login-app (Guaranteed QoS)

Both are in the Guaranteed QoS class, which means they are the highest priority and the last to be evicted when the node runs out of resources.

But if the node is under extreme pressure and Kubernetes must evict one of them, it doesn’t choose based on QoS anymore (since both are equal). Instead, it looks at other factors like:

Resource usage: The Pod consuming more memory/CPU may be targeted first.

Eviction signals: If one Pod exceeds its memory limit (OOMKilled), it will be killed regardless of QoS.

Pod priority/preemption: If you have defined Pod Priority and PriorityClasses, the Pod with lower priority is evicted first.

Without explicit PriorityClass, eviction order between two Guaranteed Pods is not deterministic — it depends on runtime conditions like which Pod is using more memory or hitting limits.

QoS ensures Guaranteed Pods are protected, but when multiple Guaranteed Pods compete under extreme node pressure, Kubernetes falls back to runtime conditions and priority policies. To control eviction order between critical apps, always define PriorityClasses.

3. Is it possible to change the QoS class of a Pod (for example, from Burstable to Guaranteed) while the Pod is already running in Kubernetes?

You cannot directly modify the QoS class of a running Pod. QoS class is not a field you set manually — it is automatically assigned by Kubernetes based on the Pod’s resource requests and limits at the time of creation.

QoS is immutable, which means that once the Pod is created, you cannot change or update its QoS class type.

If you want to change the QoS class, you must:

· Update the Pod manifest with new resource requests and limits.

· Delete the existing Pod (or let a Deployment/ReplicaSet replace it).

· Recreate the Pod with the updated configuration.