Four AI Agents That Keep Your dbt Project Honest

Description drift, stale tests, hidden blast radius, missing contracts — four agents that catch what code review misses

⬅️ Previous: Scaffolding a Production dbt Project in 60 Seconds with AI

There’s a particular kind of failure that’s worse than a broken pipeline.

Your dbt project builds green. Tests pass. Dashboards load. Everything looks fine — and yet the project is quietly lying to you. Descriptions that describe nothing. Tests that don’t reflect real data. A model that 8 dashboards depend on with no contract protecting it. Nobody knows, because nothing is broken yet.

This post is about fixing that. Not with discipline or process or code review checklists — with four AI agents that run automatically and make honesty the default.

This is Post 2 in the series “AI-Augmented dbt Best Practices at the Data Platform Level”.

→ github.com/TechPopsicles/dbt-mesh-platform

The four ways your dbt project lies

Before we build the agents, let’s name the dishonesty precisely.

Lie 1 — Descriptions that describe nothing

"Monetary value for totalprice." That's not a description. That's the column name in different words. A business analyst reading it learns nothing. An engineer onboarding learns nothing. The dbt docs catalog becomes a graveyard of name-echo descriptions that waste everyone's time.

Lie 2 — Tests that don’t reflect reality

In Post 1, the boilerplate agent generated accepted_values: [PLACEHOLDER_A, PLACEHOLDER_B] — tests that always fail because nobody replaced the placeholders with real values. But even hand-written tests go stale. A status field that had 3 values in January has 5 in June. Nobody updated the test. It passes — because it was never checking the right thing.

Lie 3 — Lineage that hides blast radius

The DAG looks clean in dbt docs. But which models have zero tests? Which sources are defined but never referenced? Which model, if it breaks, takes down 8 downstream dependents? Nobody knows until it happens — because nobody checked the manifest.

Lie 4 — Contracts that don’t exist

A downstream team is SELECT *-ing your mart and assuming the schema is stable. It isn't. You renamed a column last sprint and their pipeline broke silently at 2am. A dbt contract would have caught this at compile time, before the PR merged.

The four agents — one for each lie

All four agents live in agents/ in the repo. They run independently — or in sequence as a pipeline after every dbt build. Each one reads what dbt already produces (YAML files, SQL files, manifest.json) and writes back to those same files. No new infrastructure.

Agent 1 — Description Agent

File: agents/description_agent.py

What it fixes: Lie 1 — thin, name-echo column descriptions

How it works: One API call per model (not per column — cheaper and more coherent). The agent reads the staging SQL, all column names and types, and sibling column context, then sends everything to claude-sonnet-4-6 with a structured prompt. The response is a JSON object mapping column names to business-context descriptions. The agent parses it and overwrites the YAML.

Run it:

python agents/description_agent.py \
  --project-dir dbt_platform \
  --models-dir  dbt_platform/models/staging/tpch \
  --source      tpch

Results: 8 models · 60 columns enriched · 8 API calls

The before/after that says it all:

Before (boilerplate agent):

orderstatus  →  "Categorical field — orderstatus."
totalprice   →  "Monetary value for totalprice."
orderpriority → "Orderpriority."

After (description agent):

orderstatus   → "Single-character code indicating the fulfillment status of
                  the order (e.g., 'O' for open, 'F' for fulfilled, 'P' for
                  partially fulfilled)."

totalprice    → "Total monetary value of the order in USD, summing all line
                  item prices after discounts and before taxes."

orderpriority → "Categorical ranking of the order's business urgency
                  (e.g., '1-URGENT', '2-HIGH', '3-MEDIUM', '4-NOT SPECIFIED',
                  '5-LOW')."

The agent knew the actual values because it read the SQL context. It saw the accepted_values tests we'd written and understood what the values meant.

The design decision that matters: one call per model, not per column. The API sees all sibling columns in one prompt — so when it describes discount_rate, it knows net_price and gross_price exist, and the description reflects that relationship. Isolated column-by-column calls miss this entirely.

the git diff showing before/after descriptions and the description_agent run stats

Agent 2 — Test Agent

File: agents/test_agent.py

What it fixes: Lie 2 — placeholder tests, stale accepted_values

How it works: No API call. Pure Snowflake. The agent connects to your staging views (not the raw source tables — the post-transformation views) and profiles every column: null rate, distinct count, and for low-cardinality columns (≤10 distinct values), all actual values. It generates data_tests: in dbt 1.9+ format from real data.

Four test types generated:

**not_null** — column has zero nulls in the profile

**unique** — distinct count equals row count, sole primary key only

**accepted_values** — ≤10 distinct values, real values from Snowflake, dbt 1.9+ arguments: nesting

**relationships** — FK columns auto-detected from name patterns, target model resolved from a FK map built by scanning sibling YAMLs

Run it:

python agents/test_agent.py \
  --project-dir  dbt_platform \
  --models-dir   dbt_platform/models/staging/tpch \
  --source       tpch \
  --database     PLATFORM_DEV \
  --db-schema    KIRAN_STAGING

Results: 8 models · 61 columns profiled · 74 data_tests generated · 0 placeholders

The profiling output that makes the story:

stg_tpch__orders   Row count: 1,500,000
  orderstatus      nulls=0.0%  3 distinct  → ['not_null', 'accepted_values']
  orderpriority    nulls=0.0%  5 distinct  → ['not_null', 'accepted_values']
  custkey          nulls=0.0%  99,996 distinct → ['not_null', 'relationships']

stg_tpch__lineitem Row count: 6,001,215
  returnflag       nulls=0.0%  3 distinct  → ['not_null', 'accepted_values']
  linestatus       nulls=0.0%  2 distinct  → ['not_null', 'accepted_values']
  shipmode         nulls=0.0%  7 distinct  → ['not_null', 'accepted_values']

What the YAML looks like after:

- name: orderstatus
  data_tests:
    - not_null
    - accepted_values:
        arguments:
          values: ['F', 'O', 'P']    ← real values from Snowflake
        config:
          severity: warn             ← AI guardrail, not a blocker

- name: custkey
  data_tests:
    - not_null
    - relationships:
        arguments:
          to: ref('stg_tpch__customer')
          field: custkey
        config:
          severity: warn

The severity decision: AI-generated tests use severity: warn. They surface problems without blocking builds. Human-verified tests (the not_null + unique on primary keys you wrote yourself) stay at the default severity: error. That distinction is what makes this safe to deploy — AI as guardrail, not gatekeeper.

the git diff showing before/after data_tests and the test_agent run stats

Agent 3 — Lineage Agent

File: agents/lineage_agent.py

What it fixes: Lie 3 — hidden blast radius, orphaned sources, untested models

How it works: No Snowflake connection. No API call. Pure Python reading manifest.json — the compiled graph dbt generates on every dbt parse or dbt build. The manifest contains every node, every edge, every test, every source definition. The lineage agent walks that graph and surfaces four risk categories.

Run it:

cd dbt_platform && dbt parse && cd ..

python agents/lineage_agent.py \
  --manifest   dbt_platform/target/manifest.json \
  --project    dbt_platform \
  --output-dir agents/reports

The four risks it surfaces:

Risk 1 — Orphaned sources: sources defined in YAML but never referenced by any model

Risk 2 — Untested models: models with zero data_tests — no assertions on output quality

Risk 3 — High blast radius: models with 3+ downstream dependents — breakage cascades

Risk 4 — Thin model descriptions: model-level descriptions under 10 words

Our result after running all four agents:

Overall health: 🟢 HEALTHY
Models:         8 total · 8 tested (100% coverage)
Data tests:     80 total
Errors:         0 · Warnings: 0

Risk 1 — Orphaned Sources (0)     ✅ No orphaned sources found
Risk 2 — Untested Models (0)      ✅ All models have at least one data_test
Risk 3 — High Blast Radius (0)    ✅ No models exceed threshold of 3
Risk 4 — Thin Descriptions (0)    ✅ All models have adequate descriptions

Two output files committed to the repo:

agents/reports/lineage_report.md — the living health document. Every run overwrites it. The git history becomes a timeline of your project's health improving over time.

agents/reports/lineage_report.json — machine-readable. The scorecard in Post 5 reads this file as one of its scoring inputs.

Lineage Health Check Report

Agent 4 — Constraint Agent

File: agents/constraint_agent.py

What it fixes: Lie 4 — missing contracts, undocumented freshness SLAs

How it works: No Snowflake, no API. Reads YAML, writes YAML. Adds contract: enforced: blocks to model configs and writes SLA-based freshness thresholds to source definitions.

Run it:

python agents/constraint_agent.py \
  --models-dir dbt_platform/models/staging/tpch \
  --source     tpch \
  --layer      staging

Two things it writes:

Contract blocks on every model — staging gets enforced: false (private layer, documented not enforced), mart models get enforced: true (public contracts, schema changes fail at compile time):

config:
  contract:
    enforced: false    # staging — internal, not enforced

Freshness thresholds based on meta.sla tag — three tiers:

realtime → warn: 1h    error: 3h    (event streams, CDC)
daily    → warn: 25h   error: 49h   (nightly batch ETL)
static   → null        null          (reference data — TPC-H)

Set the SLA once on your source table, rerun the agent, the right thresholds appear automatically. No threshold arithmetic, no copy-paste between tables.

The agent also cleans up after itself — removes deprecated top-level freshness:, loaded_at_field:, and meta: properties from source definitions (dbt 1.9+ moved these inside config:). Zero manual fixes needed.

Models Contract & Freshness

The final build — proof that all four agents worked

dbt build --select staging.tpch --no-partial-parse

Found 8 models, 1 operation, 74 data tests, 8 sources

Done. PASS=83 WARN=0 ERROR=0 SKIP=0 NO-OP=0 TOTAL=83

No deprecation warnings. No compilation errors. No placeholder test failures.

Before the four agents (Post 1 state):

42 tests  ·  PLACEHOLDER values  ·  thin descriptions
no contracts  ·  no freshness SLAs  ·  no lineage report

After the four agents:

74 data_tests  ·  real Snowflake values  ·  business-context descriptions
contracts documented  ·  SLA freshness thresholds  ·  🟢 HEALTHY lineage report

Same project. Same SQL. Completely different level of honesty.

The pipeline — running all four in sequence

These agents aren’t meant to be one-off scripts. The right pattern is to run them after every significant schema change — or wire them into your CI pipeline as a post-build step:

# After dbt build completes

# 1. Enrich any new or changed column descriptions
python agents/description_agent.py \
  --project-dir dbt_platform \
  --models-dir  dbt_platform/models/staging/tpch \
  --source      tpch

# 2. Re-profile columns that may have new distinct values
python agents/test_agent.py \
  --project-dir dbt_platform \
  --models-dir  dbt_platform/models/staging/tpch \
  --source      tpch \
  --database    PLATFORM_DEV \
  --db-schema   KIRAN_STAGING

# 3. Check lineage health after any schema changes
python agents/lineage_agent.py \
  --manifest   dbt_platform/target/manifest.json \
  --project    dbt_platform \
  --output-dir agents/reports

# 4. Add contracts and freshness to any new models
python agents/constraint_agent.py \
  --models-dir dbt_platform/models/staging/tpch \
  --source     tpch \
  --layer      staging

The agents compose. The description agent enriches descriptions. The test agent validates data. The lineage agent reads the manifest that includes both. The constraint agent locks down the contracts. Together they maintain a level of project honesty that no code review process can reliably achieve at scale.

What’s coming in Post 3

The four agents keep the platform project honest. But the real test of dbt governance comes when you have five teams building on top of each other — each with their own staging layer, their own marts, their own version of “what does revenue mean.”

In Post 3 — Scaling dbt Across Teams: Mesh, Contracts and Versioning, we build the multi-project topology:

• dbt_commercial — sales, CRM, partnerships
• dbt_finance — ARR, billing, FP&A
• dbt_product — events, features, growth
• dbt_marketing — SEO, SEM, content, lifecycle
• dbt_analytics — cross-domain marts and metric registry

Each project publishes public contracts. Other projects ref() those contracts. When a contract changes, versioning kicks in. When a version is deprecated, consumers get compile-time warnings. And the constraint agent we built today generates those contracts automatically.

Follow along at **github.com/TechPopsicles/dbt-mesh-platform** — the repo grows with each post. ⬅️ Previous: Scaffolding a Production dbt Project in 60 Seconds with AI

Kiran Pothina is a data platform engineer applying AI to modernize data engineering practices — from automated boilerplate generation to intelligent governance at scale. This series documents building a production dbt Mesh reference implementation from scratch on Snowflake.

Tags: dbt · Snowflake · Analytics Engineering · Data Engineering · AI · Python · Anthropic