← Back to list

How to Build an Automatic Kill Switch for Amazon Bedrock Costs

Your AI bill doesn’t have a speed limit

James Wynne · 2026-08-04 22:16 · 0 claps · 3.9 min read
#aws #aws-lambda #amazon-bedrock #aws-cloudwatch
Open on Medium ↗
Wiki topics: ☁️ · DevOps & Cloud

How to Build an Automatic Kill Switch for Amazon Bedrock Costs

Your AI bill doesn’t have a speed limit

Here’s a scenario I’ve seen play out more than once: a team enables Amazon Bedrock, builds something clever with an AI agent, and three weeks later gets a bill that makes someone’s eye twitch.

The problem is straightforward. Bedrock charges per token on its on-demand tier. A simple chatbot request? A few hundred tokens. An agentic workflow that plans, executes, reflects, and iterates? Easily ten to twenty times that per interaction. There’s no built-in “stop at $500” switch. If something loops unexpectedly or traffic spikes, the meter keeps running until a human intervenes.

I’m going to show you how to build a circuit breaker that catches this automatically using services you probably already have running.

The idea in 30 seconds

Amazon Bedrock publishes token-usage metrics to CloudWatch in near real-time. We create a CloudWatch alarm that watches total token consumption over a rolling window. When it breaches your threshold, an SNS notification triggers a Lambda function that attaches an IAM deny policy to the consuming role. Bedrock calls stop immediately. When usage drops back below the threshold, the deny policy is automatically removed.

No application code changes. No API gateway to build. Just native AWS services wired together.

Why AWS Budgets isn’t enough

You might be thinking doesn’t AWS Budgets handle this? It does alert you, but there’s a lag. Budgets operates on billing data that can be hours behind actual usage. By the time the email lands in your inbox, your agentic workflow has already consumed another few hundred thousand tokens.

CloudWatch metrics, on the other hand, update in near real-time. That’s the difference between catching a runaway process in minutes versus catching it tomorrow morning.

What Bedrock gives you for free

Every time your application calls the bedrock-runtime endpoint, AWS publishes metrics to CloudWatch under the AWS/Bedrock namespace. No setup required. The ones we care about:

  • InputTokenCount: tokens sent to the model
  • OutputTokenCount: tokens the model generates
  • Invocations: number of API calls

These support a ModelId dimension, so you can monitor specific models or aggregate across all of them.

Let’s build it

  1. Create the alarm

This alarm fires when total tokens (input + output) exceed 500,000 in one hour. Adjust the threshold to your budget.

aws cloudwatch put-metric-alarm \
  --alarm-name "bedrock-token-budget-exceeded" \
  --alarm-description "Fires when Bedrock token usage exceeds hourly budget" \
  --metrics '[
    {
      "Id": "input",
      "MetricStat": {
        "Metric": {
          "Namespace": "AWS/Bedrock",
          "MetricName": "InputTokenCount",
          "Dimensions": []
        },
        "Period": 3600,
        "Stat": "Sum"
      },
      "ReturnData": false
    },
    {
      "Id": "output",
      "MetricStat": {
        "Metric": {
          "Namespace": "AWS/Bedrock",
          "MetricName": "OutputTokenCount",
          "Dimensions": []
        },
        "Period": 3600,
        "Stat": "Sum"
      },
      "ReturnData": false
    },
    {
      "Id": "total_tokens",
      "Expression": "input + output",
      "Label": "TotalTokens",
      "ReturnData": true
    }
  ]' \
  --evaluation-periods 1 \
  --threshold 500000 \
  --comparison-operator GreaterThanThreshold \
  --treat-missing-data notBreaching \
  --alarm-actions "arn:aws:sns:REGION:ACCOUNT_ID:bedrock-cost-alerts" \
  --ok-actions "arn:aws:sns:REGION:ACCOUNT_ID:bedrock-cost-alerts"

The — ok-actions line is crucial it’s what triggers the automatic recovery when usage drops back down.

  1. Wire up SNS
aws sns create-topic --name bedrock-cost-alerts

aws sns subscribe \
  --topic-arn "arn:aws:sns:<REGION>:<ACCOUNT_ID>:bedrock-cost-alerts" \
  --protocol lambda \
  --notification-endpoint "arn:aws:lambda:<REGION>:<ACCOUNT_ID>:function:bedrock-cost-guardrail"
  1. The Lambda function

Here’s the core logic. When the alarm fires, it attaches a deny policy. When the alarm clears, it removes it.

import json
import boto3
import os
import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

iam = boto3.client("iam")

ROLE_NAME = os.environ["TARGET_ROLE_NAME"]
POLICY_NAME = "BedrockCostGuardrailDeny"

DENY_POLICY = json.dumps({
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DenyBedrockInvocations",
            "Effect": "Deny",
            "Action": [
                "bedrock:InvokeModel",
                "bedrock:InvokeModelWithResponseStream"
            ],
            "Resource": "*"
        }
    ]
})

def handler(event, context):
    message = json.loads(event["Records"][0]["Sns"]["Message"])
    new_state = message.get("NewStateValue")

    logger.info(f"Alarm transitioned to: {new_state}")

    if new_state == "ALARM":
        logger.info(f"Attaching deny policy to role: {ROLE_NAME}")
        iam.put_role_policy(
            RoleName=ROLE_NAME,
            PolicyName=POLICY_NAME,
            PolicyDocument=DENY_POLICY,
        )
    elif new_state == "OK":
        logger.info(f"Removing deny policy from role: {ROLE_NAME}")
        try:
            iam.delete_role_policy(
                RoleName=ROLE_NAME,
                PolicyName=POLICY_NAME,
            )
        except iam.exceptions.NoSuchEntityException:
            logger.info("Deny policy already removed.")

    return {"statusCode": 200}
  1. Lock down the Lambda’s permissions

The Lambda execution role only needs two IAM actions, scoped to the single role you’re protecting:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["iam:PutRolePolicy", "iam:DeleteRolePolicy"],
      "Resource": "arn:aws:iam::ACCOUNT_ID:role/TARGET_ROLE_NAME"
    }
  ]
}

Plus the usual CloudWatch Logs permissions for the function itself.

  1. Allow SNS to invoke Lambda
aws lambda add-permission \
  --function-name bedrock-cost-guardrail \
  --statement-id sns-invoke \
  --action lambda:InvokeFunction \
  --principal sns.amazonaws.com \
  --source-arn "arn:aws:sns:REGION:ACCOUNT_ID:bedrock-cost-alerts"

Things that will trip you up

  • Threshold too aggressive. Start generous. Look at your highest observed hourly consumption, add 50%, and use that. Tighten later once you have a baseline.
  • No OK action. Without it, a single spike permanently locks out your application until a human intervenes. Always wire up the recovery path.
  • Scoping too broadly. If you deny bedrock:InvokeModel on a role shared by multiple services, you’ll break things you didn’t intend to. Use separate roles or IAM conditions.
  • Missing data panic. If nobody uses Bedrock during an evaluation period, the alarm enters INSUFFICIENT_DATA. That’s why we set treat-missing-data to notBreaching — silence means no spend, not a problem.
  • Not testing recovery. Deploy this and then test it. Simulate the alarm, confirm the deny policy appears, wait for the OK transition, confirm it’s removed. A stuck deny policy is an incident.

Making it smarter

This is the minimum viable guardrail. You can extend it:

  • Per-model alarms: add a ModelId dimension so expensive models have tighter thresholds
  • Tiered response: first threshold sends a Slack alert, second threshold applies the deny policy
  • Per-team control: multiple alarms targeting different IAM roles
  • Dashboard: create a CloudWatch dashboard showing token burn rate alongside your threshold line
  • Budget correlation: combine this with AWS Cost Anomaly Detection for defence in depth

Wrapping up

Amazon Bedrock doesn’t have a spending cap. That’s unlikely to change soon, pay-per-token is how the economics work. But you don’t have to accept unbounded risk.

CloudWatch already has the data. Lambda already has the IAM access. You’re just wiring them together into something that watches your back while your team builds.

Ship this on a Friday afternoon. You’ll sleep better over the weekend.


메타데이터
post_id
b4e2d85e7839
slug
how-to-build-an-automatic-kill-switch-for-amazon-bedrock-costs-b4e2d85e7839
url
https://medium.com/@james_28793/how-to-build-an-automatic-kill-switch-for-amazon-bedrock-costs-b4e2d85e7839
canonical_url
https://medium.com/@james_28793/how-to-build-an-automatic-kill-switch-for-amazon-bedrock-costs-b4e2d85e7839
author_url
https://medium.com/@james_28793
status
ok
fetched_at
2026-08-09 03:56:23