Kubernetes Autoscaling and Scheduling Deep Dive: HPA, VPA, Cluster Autoscaler, and Affinity

A complete practical guide to making your Kubernetes cluster self-managing — with real YAML, real errors, and real fixes from a running TaskManager app

One of the most powerful things about Kubernetes is that it can manage itself. Traffic spikes at 9 am — pods scale up automatically. Traffic drops at midnight — pods and nodes scale back down to save cost. Your app always gets the right resources — not your initial guess, but actual measured values.

This article covers the four features that make this possible:

• HPA — automatically scales pod count based on CPU and memory
• Cluster Autoscaler — automatically scales node count
• VPA — automatically right-sizes pod resource requests
• Node and Pod Affinity — controls where pods are placed

Part 1: HorizontalPodAutoscaler (HPA)

What problem HPA solves

Without HPA:
  9am  — traffic spikes — 2 pods overwhelmed — users see 500 errors
  You notice alert at 9:15am — run kubectl scale — too late
With HPA:
  9am  - traffic spikes - CPU hits 70%
  9:00:15am - HPA already adding pod-3
  9:01am  - pod-3 Running - traffic distributed - users unaffected

The scaling formula

HPA uses this formula every 15 seconds:

desiredReplicas = ceil(currentReplicas × currentMetric / targetMetric)
Scale up example:
  current replicas: 2
  current CPU:      90%
  target CPU:       70%
  desired = ceil(2 × 90/70) = ceil(2.57) = 3  →  scale to 3 pods
Scale down example:
  current replicas: 5
  current CPU:      20%
  target CPU:       70%
  desired = ceil(5 × 20/70) = ceil(1.43) = 2  →  scale to 2 pods

Full HPA YAML for your TaskManager backend

# hpa-backend.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: backend-hpa
  namespace: taskmanager
spec:
  # Which Deployment to scale
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: backend
# Never go below 2 or above 10 pods
  minReplicas: 2
  maxReplicas: 10
  metrics:
    # Scale on CPU - safe and predictable
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70    # keep average CPU at 70%
  # Prevent flapping - scale fast up, slow down
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 15     # scale up fast
      policies:
        - type: Pods
          value: 2                       # add max 2 pods at a time
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300    # scale down slow - 5 min window
      policies:
        - type: Pods
          value: 1                       # remove max 1 pod at a time
          periodSeconds: 60

Why HPA on memory is risky

A common question is whether HPA can scale on memory. Technically, yes — but it has a critical problem:

CPU is compressible:
  High CPU → add pod → CPU spreads across pods → usage drops → HPA scales down ✅
Memory is incompressible:
  High memory → add pod → OLD pod STILL holds its memory
  Memory doesn't drop → HPA sees still high → adds MORE pods
  Infinite scale up loop until maxReplicas! ❌

The rule:

HPA  →  CPU utilisation   (safe — compressible resource)
VPA  →  memory right-size (safe — handles incompressible resource)

Install Metrics Server (required for HPA)

# On Minikube
minikube addons enable metrics-server

# Verify metrics are flowing
kubectl top pods -n taskmanager
# NAME               CPU(cores)   MEMORY
# backend-xxx        45m          380Mi

Apply and watch HPA work

# Apply HPA
kubectl apply -f hpa-backend.yaml

# Check HPA status
kubectl get hpa -n taskmanager
# NAME          REFERENCE             TARGETS    MINPODS   MAXPODS   REPLICAS
# backend-hpa   Deployment/backend    45%/70%    2         10        2

# Simulate traffic spike — watch HPA respond
kubectl run load-test \
  --image=busybox:1.36 \
  --restart=Never \
  -n taskmanager \
  -- sh -c "while true; do wget -q -O- http://backend-service:8080/api/tasks; done"

# Watch in real time
kubectl get hpa -n taskmanager -w
# TARGETS goes 45%/70% → 90%/70% → scaling!
# REPLICAS goes 2 → 3 → 4

# Stop load test
kubectl delete pod load-test -n taskmanager

# Watch scale down (takes 5 minutes — stabilization window)
kubectl get hpa -n taskmanager -w

Common HPA issues

TARGETS shows unknown/70%
  → Metrics Server not installed
  → Resource requests not set on Deployment (required for % calculation)

HPA never scales down
  → stabilizationWindowSeconds too high (default 300s)
  → CPU never drops below target

Pods crash after scale up
  → Memory limit too low → OOMKilled under load
  → Fix: increase memory limit or tune JVM heap

Part 2: Cluster Autoscaler

What problem Cluster Autoscaler solves

HPA adds pods. But what if the cluster runs out of nodes to place them on?

HPA creates 5 new backend pods
    ↓
Scheduler: no node has enough CPU for these pods
    ↓
Pods stuck in Pending state — not serving traffic
    ↓
Cluster Autoscaler sees Pending pods
    ↓
Calls cloud API: "add 2 more EC2 nodes"
    ↓
New nodes ready in ~2 minutes
    ↓
Scheduler places pending pods — all Running ✅

And in reverse:

Traffic drops → HPA removes pods
    ↓
Node-3 is now 80% empty — wasting $0.04/hr
    ↓
Cluster Autoscaler: can those remaining pods fit elsewhere?
    YES → evict pods from Node-3
    ↓
Pods rescheduled on Node-1 and Node-2
    ↓
Node-3 terminated — cost saved ✅

HPA + Cluster Autoscaler working together

9am — traffic spike
  HPA:  2 pods → 8 pods (CPU too high)
  CA:   2 nodes → 4 nodes (pods can't schedule)

12am — traffic drops
  HPA:  8 pods → 2 pods (CPU too low)
  CA:   4 nodes → 2 nodes (nodes now idle)

This is the complete self-managing cluster — no human intervention needed for scaling.

Scale up flow — step by step

1. HPA creates new pods
2. Scheduler tries to place them — no capacity on existing nodes
3. Pods go to Pending state
4. CA checks every 10 seconds for Pending pods
5. CA simulates: "which node group could fit these pods?"
6. CA calls cloud API to add nodes
7. New EC2 instances boot (~2 minutes)
8. Scheduler places pending pods on new nodes
9. Pods: Pending → Running

Scale down flow — step by step

1. CA checks nodes every 10 seconds
2. Node utilisation below 50% for 10 continuous minutes
3. CA checks: can all pods on this node fit elsewhere?
   YES → safe to remove
   NO  → keep the node (protects against downtime)
4. CA evicts pods gracefully
5. Pods rescheduled on remaining nodes
6. CA terminates the empty node

Pods CA will NOT evict

1. Pods with annotation safe-to-evict: false
2. Pods with local storage (emptyDir, hostPath)
3. Pods protected by PodDisruptionBudget (minAvailable)
4. Pods in kube-system namespace (if configured)
5. Standalone pods not managed by a controller

Install CA on EKS

# Tag your Auto Scaling Group (CA uses these tags to find node groups)
aws autoscaling create-or-update-tags \
  --tags \
    ResourceId=<your-asg-name> \
    ResourceType=auto-scaling-group \
    Key=k8s.io/cluster-autoscaler/enabled,Value=true,PropagateAtLaunch=true \
    ResourceId=<your-asg-name> \
    ResourceType=auto-scaling-group \
    Key=k8s.io/cluster-autoscaler/demo-eks,Value=owned,PropagateAtLaunch=true

# Deploy Cluster Autoscaler
kubectl apply -f https://raw.githubusercontent.com/kubernetes/autoscaler/master/cluster-autoscaler/cloudprovider/aws/examples/cluster-autoscaler-autodiscover.yaml

# Update cluster name
kubectl set env deployment cluster-autoscaler \
  -n kube-system \
  CLUSTER_NAME=demo-eks

Full CA configuration

containers:
  - name: cluster-autoscaler
    image: registry.k8s.io/autoscaling/cluster-autoscaler:v1.31.0
    command:
      - ./cluster-autoscaler
      - --cloud-provider=aws
      - --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/demo-eks
      # Scale down config
      - --scale-down-utilization-threshold=0.5   # remove if below 50%
      - --scale-down-unneeded-time=10m           # idle for 10 minutes
      - --scale-down-delay-after-add=10m         # wait after scaling up
      # Safety
      - --skip-nodes-with-local-storage=true
      - --skip-nodes-with-system-pods=true

Protect critical pods from eviction

# MySQL pod — never evict the database!
metadata:
  annotations:
    cluster-autoscaler.kubernetes.io/safe-to-evict: "false"
# Backend pods - CA can evict freely
metadata:
  annotations:
    cluster-autoscaler.kubernetes.io/safe-to-evict: "true"

PodDisruptionBudget — always keep a minimum number of pods running

# pdb-backend.yaml
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: backend-pdb
  namespace: taskmanager
spec:
  minAvailable: 1       # always keep at least 1 pod running
  selector:
    matchLabels:
      app: backend

Watch CA in action

# See current nodes
kubectl get nodes

# Exhaust capacity — force CA to add nodes
kubectl scale deployment backend --replicas=20 -n taskmanager

# Watch pods go Pending
kubectl get pods -n taskmanager -w
# backend-xxx   0/1   Pending   0   ← no room!

# Watch CA add nodes
kubectl get nodes -w
# ip-10-0-1-102   NotReady   <none>   10s  ← CA just added this
# ip-10-0-1-102   Ready      <none>   90s  ← now ready

# Scale back down
kubectl scale deployment backend --replicas=2 -n taskmanager

# Wait 10 min — watch CA remove the empty node
kubectl get nodes -w

# See CA decisions in logs
kubectl logs -n kube-system \
  -l app=cluster-autoscaler \
  --follow
# "Scale up triggered"
# "Removing node ip-xxx"

Part 3: VerticalPodAutoscaler (VPA)

What problem VPA solves

When you write Kubernetes YAML, you guess resource values:

resources:
  requests:
    cpu: "250m"      # ← your guess
    memory: "512Mi"  # ← your guess
  limits:
    cpu: "1000m"
    memory: "1Gi"

Your guess is almost always wrong. Over-provisioned pods waste money. Under-provisioned pods get OOMKilled. VPA watches your actual usage and tells you — or automatically sets — the correct values.

VPA vs HPA — completely different

HPA:  traffic spikes  → add MORE pods (same size)
VPA:  pod uses 120m   → reduce request FROM 250m TO 120m (right size)
      pod uses 380Mi  → reduce request FROM 512Mi TO 380Mi (right size)

Three VPA components

Recommender — watches metrics server, analyzes historical usage, calculates optimal resource values. Updates status.recommendation in the VPA object.

Updater — checks if running pods match the recommendation. If too far off, it evicts the pod so it restarts with correct resources.

Admission Controller — intercepts pod creation and injects recommended values before the pod starts.

VPA YAML

# vpa-backend.yaml
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: backend-vpa
  namespace: taskmanager
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: backend
updatePolicy:
    updateMode: "Off"     # start here - recommend only, no changes
  resourcePolicy:
    containerPolicies:
      - containerName: backend
        minAllowed:
          cpu: "100m"
          memory: "256Mi"
        maxAllowed:
          cpu: "2000m"
          memory: "2Gi"
        controlledResources:
          - cpu
          - memory

Four update modes

Off       →  show recommendations only
             YOU apply the values manually — zero downtime
             best for: learning, production first step

Initial   →  set resources at pod creation only
             no eviction of running pods
             best for: stateful apps that can't be restarted

Recreate  →  evict pods when recommendation differs significantly
             at least one pod stays running (with enough replicas)
             best for: Deployments with 2+ replicas

Auto      →  same as Recreate today
             future: in-place resize without restart
             best for: stateless apps comfortable with rolling restarts

Does VPA cause downtime?

Mode Off      →  zero downtime (no pod changes)
Mode Initial  →  zero downtime (only new pods)
Mode Recreate →  brief gap per pod (30-60 seconds)
               replicas > 1 + PDB = minimal impact

Future K8s 1.27+ in-place resize:
  VPA can update resources WITHOUT pod restart
  zero downtime — coming to Auto mode when stable

VPA in practice — step by step

# Step 1 — install VPA
git clone https://github.com/kubernetes/autoscaler.git
cd autoscaler/vertical-pod-autoscaler
./hack/vpa-up.sh

# Verify
kubectl get pods -n kube-system | grep vpa
# vpa-admission-controller   1/1   Running
# vpa-recommender            1/1   Running
# vpa-updater                1/1   Running

# Step 2 — apply VPA in Off mode
kubectl apply -f vpa-backend.yaml

# Step 3 — generate traffic for history
kubectl run load-test \
  --image=busybox:1.36 \
  --restart=Never \
  -n taskmanager \
  -- sh -c "for i in $(seq 1 200); do wget -q -O- http://backend-service:8080/api/tasks; done"

# Step 4 — check recommendation after 10 minutes
kubectl describe vpa backend-vpa -n taskmanager
# Status:
#   Recommendation:
#     Container Recommendations:
#       Container Name: backend
#       Lower Bound:
#         Cpu:     80m
#         Memory:  300Mi
#       Target:            ← apply these values
#         Cpu:     120m
#         Memory:  380Mi
#       Upper Bound:
#         Cpu:     600m
#         Memory:  800Mi

# Step 5 — apply recommendation via rolling update (zero downtime)
kubectl set resources deployment backend \
  -n taskmanager \
  --requests=cpu=120m,memory=380Mi \
  --limits=cpu=600m,memory=700Mi

kubectl rollout status deployment/backend -n taskmanager

Combining HPA and VPA safely

WRONG — conflict:
  HPA scaling on CPU + VPA adjusting CPU requests
  VPA changes the baseline HPA uses → unstable scaling

RIGHT — no conflict:
  HPA on CPU          → controls pod count
  VPA on memory only  → right-sizes memory

RIGHT — no conflict:
  HPA on requests/sec (custom metric) → controls pod count
  VPA on CPU + memory                 → right-sizes everything

# Safe combination in VPA
resourcePolicy:
  containerPolicies:
    - containerName: backend
      controlledResources:
        - memory          # VPA only manages memory
      controlledValues: RequestsOnly
      # CPU left entirely to HPA

Part 4: Node Affinity and Pod Affinity

Why do you need affinity rules

Without affinity, all backend replicas might land on the same node. If that node dies, all replicas die together. Zero high availability.

Without affinity, your database might land on a slow HDD node instead of an SSD. Performance suffers.

Affinity lets you control where pods run.

Three types

Node affinity      →  pod chooses WHICH NODE to run on
                      "schedule me only on disk=ssd nodes"

Pod affinity       →  pod runs NEAR other pods
                      "schedule me on same node as frontend pod"

Pod anti-affinity  →  pod runs FAR from other pods
                      "never put two backend replicas on same node"

Two strictness levels

required   →  hard rule — pod stays Pending if no match found
              use when: hardware requirement is non-negotiable (MySQL needs SSD)

preferred  →  soft rule — try to match, schedule anywhere if no match
              use when: preference, not requirement (prefer same zone)

Label your nodes first

# Label nodes with their hardware and zone properties
kubectl label node node-1 disk=ssd zone=us-east-1a
kubectl label node node-2 disk=hdd zone=us-east-1b
kubectl label node node-3 disk=ssd gpu=true

# Verify
kubectl get nodes --show-labels

Node affinity YAML

# MySQL must run on SSD nodes
spec:
  affinity:
    nodeAffinity:
# HARD RULE - Pending if no SSD node available
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: disk
                operator: In
                values:
                  - ssd
      # SOFT RULE - prefer us-east-1a but schedule anywhere
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 80              # 0-100, higher = stronger preference
          preference:
            matchExpressions:
              - key: zone
                operator: In
                values:
                  - us-east-1a

Node affinity operators

operator: In          # label value is in this list
operator: NotIn       # label value is NOT in this list
operator: Exists      # label exists (any value)
operator: DoesNotExist # label does not exist
operator: Gt          # numeric greater than
operator: Lt          # numeric less than

Pod affinity — co-locate for low latency

Frontend and backend on the same node = no network hop between them = lower latency:

# backend — prefer same node as frontend
spec:
  affinity:
    podAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          podAffinityTerm:
            labelSelector:
              matchExpressions:
                - key: app
                  operator: In
                  values:
                    - frontend
            topologyKey: "kubernetes.io/hostname"  # same node

Pod anti-affinity — spread for high availability

Never put two backend replicas on the same node — if a node dies you still have pods running:

# backend — spread replicas across nodes (HA)
spec:
  affinity:
    podAntiAffinity:
      # HARD RULE — one backend pod per node maximum
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchExpressions:
              - key: app
                operator: In
                values:
                  - backend
          topologyKey: "kubernetes.io/hostname"

Result:

3 nodes:
  Node 1 → backend-pod-1   ✅
  Node 2 → backend-pod-2   ✅
  Node 3 → backend-pod-3   ✅

Try to add 4th backend pod with only 3 nodes:
  All nodes blocked → pod stays Pending
  Add a 4th node → Cluster Autoscaler handles this!

topologyKey — what near and spread mean

topologyKey: "kubernetes.io/hostname"         # same physical node
topologyKey: "topology.kubernetes.io/zone"    # same availability zone
topologyKey: "topology.kubernetes.io/region"  # same region

Full production YAML — all three affinities together

apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend
  namespace: taskmanager
spec:
  replicas: 3
  selector:
    matchLabels:
      app: backend
  template:
    metadata:
      labels:
        app: backend
    spec:
      affinity:
# 1 - prefer SSD nodes for better IO
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 80
              preference:
                matchExpressions:
                  - key: disk
                    operator: In
                    values:
                      - ssd
        # 2 - prefer same node as frontend (low latency)
        podAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 50
              podAffinityTerm:
                labelSelector:
                  matchLabels:
                    app: frontend
                topologyKey: "kubernetes.io/hostname"
        # 3 - never two backends on same node (HA)
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app: backend
              topologyKey: "kubernetes.io/hostname"
      containers:
        - name: backend
          image: task-app:latest
          imagePullPolicy: Never
          resources:
            requests:
              cpu: "250m"
              memory: "512Mi"
            limits:
              cpu: "1000m"
              memory: "1Gi"

Taints and Tolerations — the opposite of affinity

Affinity is the pod saying “I want to go there.” Taints are the node saying “you can’t come here unless you tolerate me.”

# Reserve node-3 for GPU workloads only
kubectl taint node node-3 gpu=true:NoSchedule

# Normal pods → blocked from node-3
# GPU pods → allowed if they have this toleration:

spec:
  tolerations:
    - key: "gpu"
      operator: "Equal"
      value: "true"
      effect: "NoSchedule"

Debug scheduling failures

# Pod stuck in Pending? Find out why
kubectl describe pod <pod-name> -n taskmanager
# Events section shows:
# Warning  FailedScheduling  0/3 nodes available:
#   1 node(s) didn't match pod affinity rules
#   2 node(s) had untolerated taint

# Add missing label to fix
kubectl label node minikube disk=ssd

Quick decision guide

Need pod on specific hardware (SSD, GPU)?
  → nodeAffinity required

Prefer a zone but OK anywhere?
  → nodeAffinity preferred

Frontend and backend close for speed?
  → podAffinity preferred, topologyKey: hostname

Backend replicas never share a node?
  → podAntiAffinity required, topologyKey: hostname

Reserve nodes for specific workloads?
  → Taints + Tolerations

Everything working together — your TaskManager

9am — traffic spike hits

HPA:  backend 2 pods → 6 pods (CPU hit 85%)
CA:   2 nodes → 3 nodes (new pods can't schedule)
Affinity: each backend pod lands on different node (anti-affinity)
          pods prefer SSD nodes (node affinity)

12pm — traffic normalizes

HPA:  backend 6 pods → 2 pods (CPU back to 30%)
CA:   3 nodes → 2 nodes (node-3 idle for 10 min)

Nightly — VPA checks recommendations
VPA:  sees backend only used 120m CPU not 250m
      recommends: reduce cpu request from 250m to 120m
      you apply during next maintenance window
      each node now fits MORE pods → CA needs fewer nodes → cost savings

Full comparison table

Feature HPA VPA Cluster Autoscaler Affinity What it changes Pod count Pod size Node count Pod placement Trigger CPU/memory/custom Historical usage Pending/idle Your rules Speed Seconds Minutes 2–5 minutes At scheduling Downtime None Brief (if Auto mode) None None Cost impact Medium High savings High savings None Use for Traffic spikes Right-sizing Node capacity HA/hardware

Conclusion

These four features together make Kubernetes truly self-managing:

HPA responds to demand in seconds — adding pods when traffic spikes and removing them when it drops. Always use CPU as the primary metric, not memory.

Cluster Autoscaler responds to capacity in minutes — adding nodes when pods can’t schedule and removing them when they sit idle. Pair it with PodDisruptionBudgets to protect against data loss during scale down.

VPA responds to data over time — replacing your guessed resource values with measured ones. Use it in Off mode first to learn, then apply recommendations manually until you’re comfortable with Auto mode.

Affinity rules respond to your architecture requirements — spreading replicas for high availability, co-locating services for low latency, and reserving specific hardware for specific workloads.

None of this requires changing your application code. Spring Boot keeps calling mysql-service:3306. React keeps calling /api/tasks. Kubernetes handles the scaling, placement, and right-sizing automatically underneath.

Diagrams to add in Medium

Add screenshots of these diagrams from the conversation at these points:

1. After HPA introduction, the HPA scaling flow diagram showing metrics server, HPA controller, and Deployment
2. After the HPA formula, the scale-up and scale-down scenario boxes
3. After Cluster Autoscaler introduction — the CA scale-up and scale-down flow diagram
4. After VPA introduction — the VPA three components diagram (Recommender, Updater, Admission Controller)
5. After VPA modes — the before/after resource comparison
6. After Affinity introduction — the node affinity, pod affinity, and anti-affinity diagram
7. After full production YAML — the required vs preferred diagram

All YAML and commands in this article come from a real TaskManager app (React + Spring Boot + MySQL) deployed on Minikube on Windows. Every scaling scenario was tested against a live cluster.

If this helped you understand K8s autoscaling, leave a clap.