KEDA 2026: Event-Driven Autoscaling Patterns That Shrank Our AWS Bill by 40%

We were spending $47,000 per month running a Kubernetes cluster that processed SQS messages, handled scheduled batch jobs, and served a handful of internal APIs. Fourteen months ago, a full audit of our workload behaviour revealed something embarrassing: 60% of our compute was running 24 hours a day to handle work that arrived for at most 8 hours per day. Workers that processed overnight batch jobs were running all afternoon. Message consumers with zero queue depth were consuming full CPU allocations. API services handling 3 requests per hour were keeping 3 replicas alive around the clock.

KEDA — the Kubernetes Event-Driven Autoscaler — was the tool that changed this. KEDA v2, now at version 2.17 and CNCF Graduated, extends Kubernetes’ Horizontal Pod Autoscaler to scale on any external signal: SQS queue depth, Kafka consumer lag, Prometheus query results, Azure Service Bus message count, scheduled Cron expressions, HTTP request rate, and 50+ other scalers. Crucially, it can scale to zero replicas when there is no work — and scale back up when work arrives, in seconds.

This is the complete story of how we reduced our AWS compute bill by 40% using five KEDA patterns. It covers real numbers, real configuration, the mistakes we made, and the patterns that produced the biggest savings. Everything here is production-tested and ready to adapt.

OUR BASELINE

$47,000/month AWS compute (EC2 + ECS/EKS). Workload types: SQS-based message processors (6 services), Kafka consumers (3 services), batch/ETL jobs (8 services), HTTP API services (4 services), ML inference endpoints (2 services). 14-month optimisation journey. Final result: $28,200/month — a $18,800/month ($225,600/year) reduction.

Why Standard HPA Was Not Enough

Kubernetes’ built-in Horizontal Pod Autoscaler scales on CPU and memory. For services that do meaningful compute work in proportion to request volume, CPU-based scaling works. For our actual workloads, it did not:

• SQS consumers: CPU stays near zero when the queue is empty — they sit idle waiting for messages. When a message batch arrives, CPU spikes briefly but the queue drains faster than HPA’s 2-minute evaluation window. Result: workers always running at minimum replicas, often unnecessary.
• Scheduled batch jobs: jobs run between midnight and 6am. CPU-based scaling keeps them at minimum replicas during the day. These workers were consuming $3,200/month to sit idle for 18 hours per day.
• ML inference endpoints: inference is bursty — long idle periods, then intense GPU utilisation. GPU utilisation is not a native HPA metric without custom metrics setup, and the scale-down lag left expensive GPU instances running through quiet periods.

The core problem: CPU and memory metrics measure what is happening now. Event-driven workloads need to scale based on what is waiting to happen — the queue depth, the consumer lag, the number of pending jobs. KEDA provides this by pulling scaling decisions from the event sources themselves rather than from pod-level resource metrics.

Pattern 1: SQS Queue-Based Scaling (Saved $4,200/month)

Our order processing workers read from an SQS queue. Before KEDA: 5 workers running 24/7. After KEDA: 0 workers at night and weekends, 12 workers at peak, average 2 workers during business hours.

# KEDA ScaledObject: Scale order processors based on SQS queue depth
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-processor-scaler
  namespace: production
spec:
  scaleTargetRef:
    name: order-processor
  minReplicaCount: 0      # Scale to ZERO when queue is empty
  maxReplicaCount: 20     # Maximum 20 workers during peak
  pollingInterval: 15     # Check queue depth every 15 seconds
  cooldownPeriod: 300     # Wait 5 minutes before scaling down
  triggers:
  - type: aws-sqs-queue
    authenticationRef:
      name: aws-credentials
    metadata:
      queueURL: https://sqs.us-east-1.amazonaws.com/123456/order-queue
      queueLength: '10'   # Target: 10 messages per worker replica
      awsRegion: us-east-1
      scaleOnInFlight: 'true'  # Count in-flight messages too
---
# TriggerAuthentication for AWS credentials (use IRSA for production)
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: aws-credentials
  namespace: production
spec:
  podIdentity:
    provider: aws  # Uses EKS IRSA — no stored credentials

The scale-to-zero mechanics: when the SQS queue reaches 0 messages, KEDA waits for the cooldownPeriod (300 seconds) and then scales the deployment to 0 replicas. When a new message arrives, KEDA detects it in the next polling interval (15 seconds) and triggers the HPA to scale from 0 to 1 immediately. The first worker starts in approximately 20–30 seconds (pod scheduling + container startup). For workloads where a 30-second cold start is acceptable, this saves hours of idle compute per day.

Pattern 2: Kafka Consumer Lag Scaling (Saved $3,100/month)

Our event stream consumers processed product catalogue updates from a Kafka topic. Consumer lag is the perfect scaling signal: when lag grows, add consumers; when lag is zero, scale down. KEDA’s Kafka scaler reads consumer group lag directly from Kafka’s consumer group coordinator.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: catalogue-consumer-scaler
spec:
  scaleTargetRef:
    name: catalogue-consumer
  minReplicaCount: 0
  maxReplicaCount: 12   # Max = number of partitions (Kafka constraint)
  triggers:
  - type: kafka
    metadata:
      bootstrapServers: kafka.kafka.svc:9092
      consumerGroup: catalogue-consumer-group
      topic: product-catalogue-updates
      lagThreshold: '100'    # 1 replica per 100 messages of lag
      offsetResetPolicy: latest
      allowIdleConsumers: 'false'
      scaleToZeroOnInvalidOffset: 'true'

# Critical Kafka KEDA constraint:
# maxReplicaCount MUST be <= number of topic partitions
# More replicas than partitions = idle consumers that never receive messages
# Always set maxReplicaCount = partition count for Kafka scalers

Pattern 3: Cron-Based Scale-to-Zero for Batch Jobs (Saved $5,400/month)

Our ETL batch jobs ran every night between midnight and 6am. During the remaining 18 hours, they consumed compute doing nothing. The Cron scaler in KEDA is the simplest and highest-ROI pattern for predictable scheduled workloads.

# KEDA Cron scaler: scale up at midnight, scale to zero at 6am
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: etl-batch-scaler
spec:
  scaleTargetRef:
    name: etl-batch-worker
  minReplicaCount: 0      # Zero replicas when not running
  maxReplicaCount: 10
  triggers:
  - type: cron
    metadata:
      timezone: America/New_York
      start: '0 0 * * *'   # Scale UP at midnight
      end:   '0 6 * * *'   # Scale to ZERO at 6am
      desiredReplicas: '10' # Number of workers during the window

# Combined: SQS scaler + Cron scaler on the same ScaledObject
# (KEDA picks the maximum from all active scalers)
# This ensures: batch window gets full replicas, ad-hoc messages get
# appropriate scaling, zero replicas outside both windows
spec:
  triggers:
  - type: cron
    metadata:
      timezone: America/New_York
      start: '0 0 * * *'
      end: '0 6 * * *'
      desiredReplicas: '10'
  - type: aws-sqs-queue
    metadata:
      queueURL: https://sqs.us-east-1.amazonaws.com/123456/etl-priority
      queueLength: '5'
      awsRegion: us-east-1

Pattern 4: Prometheus Query Scaling (Saved $2,800/month)

Our recommendation engine scaled on business logic metrics: active user sessions and recommendation request rate. Neither of these maps to CPU, but both are available as Prometheus metrics. KEDA’s Prometheus scaler runs a PromQL query on a configurable schedule and uses the result as the scaling signal.

# KEDA Prometheus scaler: scale on active user sessions
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: recommendation-scaler
spec:
  scaleTargetRef:
    name: recommendation-engine
  minReplicaCount: 1    # Always keep 1 replica for availability
  maxReplicaCount: 20
  pollingInterval: 30
  cooldownPeriod: 600   # 10-minute cooldown (inference is expensive to restart)
  triggers:
  - type: prometheus
    metadata:
      serverAddress: http://prometheus.monitoring.svc:9090
      metricName: active_user_sessions
      # Target: 1 replica per 500 concurrent active sessions
      query: sum(active_user_sessions_total{app='web'})
      threshold: '500'

# Advanced: scale on a RATE metric (requests per second)
  - type: prometheus
    metadata:
      serverAddress: http://prometheus.monitoring.svc:9090
      metricName: recommendation_rps
      # Target: 1 replica per 50 req/s
      query: |
        sum(rate(recommendation_requests_total[2m]))
      threshold: '50'
      ignoreNullValues: 'true'

Pattern 5: HTTP Request Count Scaling with Scale-to-Zero (Saved $3,300/month)

KEDA’s http-add-on allows HTTP services to scale to zero and back based on request count. For internal APIs that receive traffic only during business hours, this is the highest-ROI scale-to-zero pattern — requiring no changes to the service itself.

# Step 1: Install KEDA HTTP Add-on
helm install keda-add-ons-http kedacore/keda-add-ons-http \
  --namespace keda

# Step 2: Create HTTPScaledObject (replaces standard HPA for HTTP services)
apiVersion: http.keda.sh/v1alpha1
kind: HTTPScaledObject
metadata:
  name: internal-api-scaler
spec:
  hosts:
  - internal-api.yourorg.com
  targetPendingRequests: 100  # Scale up when 100 requests are queued
  scaledownPeriod: 300
  scaleTargetRef:
    name: internal-api
    apiVersion: apps/v1
    kind: Deployment
    service: internal-api
    port: 8080
  replicas:
    min: 0  # Scale to zero
    max: 10

# The HTTP add-on proxy intercepts requests and holds them
# while KEDA scales up from zero — transparent to clients
# First request after scale-to-zero: 15-30s cold start included

The Complete Bill Reduction Breakdown

Mistakes We Made and How to Avoid Them

Mistake 1: Cooldown period too short

We initially set cooldownPeriod to 60 seconds on our SQS consumers. The result: rapid scale-up/scale-down oscillations when message bursts arrived in waves. KEDA would scale up to 10 workers, the burst would clear, cooldown would expire, KEDA would scale to 0, then the next burst would arrive and the workers would cold-start again. We lost messages to processing delays. Fix: set cooldownPeriod to at least 3x the expected burst inter-arrival time.

Mistake 2: Scale to zero without a warm replica

Our recommendation engine scaled to zero at night. The first morning recommendation request would trigger a 45-second cold start (container pull + model load). Users got a slow experience. Fix: set minReplicaCount to 1 for user-facing services. Only pure background workers should go to zero.

Mistake 3: Kafka maxReplicaCount exceeding partition count

We set maxReplicaCount to 20 for a Kafka topic with 12 partitions. The extra 8 workers started but received no messages (Kafka only assigns one consumer per partition per consumer group). We were paying for idle consumers. Fix: always set maxReplicaCount to exactly the number of partitions for Kafka scalers.

Mistake 4: Not accounting for scale-up latency in SLOs

Scale-to-zero means the first request after a quiet period takes longer (cold start time). We did not update our SLOs to account for this, which triggered false alert fires on cold-start latency spikes. Fix: either keep minReplicaCount at 1 for latency-sensitive services, or update SLOs to have a separate threshold for cold-start scenarios.

IMPLEMENTATION SEQUENCE

Week 1: Install KEDA, deploy Cron scaler on your most obvious batch-only workers (guaranteed savings, zero risk). Week 2: Add SQS or Kafka scalers for message consumers. Week 3: Add Prometheus scalers for application-metric-driven services. Week 4: Evaluate HTTP scale-to-zero for internal APIs with acceptable cold-start tolerance. Measure cost impact at each step.