How I Built a Data Contract Enforcer for Reliable Data and AI Pipelines

Modern data systems rarely fail dramatically. More often, they fail quietly — and that is the harder problem to solve.

A job still runs. A dashboard still refreshes. An AI workflow still returns output. But somewhere upstream, a field changed scale, a timestamp became invalid, or a model-facing input drifted just enough to make the downstream result unreliable.

That is the problem I wanted to solve with Data Contract Enforcer: a project that turns data assumptions into machine-checkable contracts, validates new snapshots against those contracts, traces failures through lineage, adds AI-specific checks, and generates a final report for humans.

“A contract is not just a schema. It is an operational promise.”

The Problem

In a multi-stage pipeline, every system depends on assumptions made by another system — a confidence score expected to stay between 0.0 and 1.0, a document ID expected to be present and valid, an enum expected to stay within an allowed set, prompt inputs expected to follow a schema, LLM outputs expected to remain structurally valid.

When those assumptions are not explicitly enforced, breakages spread silently.

System Architecture

The project is built around six core components that form a pipeline — from raw JSONL data all the way to a human-readable enforcer report.

Step 1 — Generate a contract from real data

The first part of the system reads structured JSONL data and converts it into a contract. For my Week 3 extraction dataset, the generator loaded the file, flattened nested facts and entities, profiled columns, inferred types and required fields, derived numeric ranges, and wrote a timestamped snapshot for later comparison.

One of the most important rules it created was this:

That clause matters because it captures a real business expectation. A confidence score should remain normalized between 0 and 1. If it changes to a 0–100 scale, downstream logic may still run — but the meaning is broken.

Step 2 — Validate clean data and establish a baseline

The runner checks each clause and produces a structured report. On the clean Week 3 dataset:

The runner also established numeric baselines for drift detection — fact_confidence.mean = 0.8637, stddev = 0.07555. This gives the system a definition of "normal behavior," not just "valid structure."

Step 3 — Inject a realistic breaking change

To prove the system works, I introduced an intentional failure. The script create_violation.py transforms confidence values from 0.0–1.0 into 0–100. A valid value like 0.92 becomes 92.0.

This is a perfect demo scenario because it is realistic — it is the kind of upstream change that can silently break downstream systems without causing an obvious crash.

Step 4 — Catch the breakage

The validation flow for every run looks like this:

When I ran validation against the violated dataset, the system caught two important failures:

That means the system did not just notice a range violation. It also detected that the statistical behavior of the field had moved massively away from its baseline. This is the difference between simple schema validation and actual data reliability engineering.

Step 5 — Attribute the failure

Detection is useful, but in real systems the next question is always: where did this come from, and who else is affected?

For my failed run, the attribution output showed 2 violation records with 4 blast radius nodes, including file::src/week4/cartographer.py and pipeline::week5-event-pipeline. This turns a simple validation error into an investigation path.

Step 6 — Extend contracts into AI reliability

Traditional data contracts are not enough for AI systems. I added three AI-specific extensions:

AI systems can degrade even when the normal tabular schema still looks valid. That is exactly what the embedding drift check caught here.

Step 7 — The final report

The last stage reads all validation reports, violation logs, schema evolution data, and AI extension results — then produces a single enforcer report with a real computed health score.

Top violation in plain English:

What I learned

Key takeaways

• Silent data failures are more dangerous than obvious crashes
• A contract is not just a schema — it is an operational promise
• Baselines matter because “valid” does not always mean “normal”
• Attribution and blast radius are what make validation actionable
• AI systems need extra reliability checks beyond normal data validation

The most valuable part of this project is not just that it catches bad data. It explains what broke, how severe it is, where it likely came from, what downstream systems may be affected, and what to do next.

That shift — from silent breakage to explicit enforcement — is the real value of the project.