← Back to list

Data Vault Explained: The Data Modeling Approach Built for Messy, Inconsistent, and Multi-Source…

A side-by-side breakdown of a real e-commerce ID conflict problem

Henry in Towards Data Engineering · 2026-05-31 07:57 · 50 claps · 7.8 min read paywalled
#data-engineering #data-modeling #data-warehouse #data-engineer #analytics-engineering
Open on Medium ↗
Wiki topics: GRW · Growth & Analytics 🔧 · Data Engineering

Data Vault Explained: The Data Modeling Approach Built for Messy, Inconsistent, and Multi-Source Warehouses

A side-by-side breakdown of a real e-commerce ID conflict problem

Not a Medium member? You can read from here.

Imagine you’re building an e-commerce data warehouse. Sales data comes from your online store. Customer profiles live in your CRM. In-store transaction records flow in from your point-of-sale system. Each source assigns its own customer ID — cust_web_001, CRM-4892, POS_00213— and none of them match. Every time an analyst asks, “How much did our top 500 customers spend across all channels last quarter?”, the team has to manually merge three ID systems before any real analysis can start.

The obvious fix is to build a mapping table — something that translates cust_web_001 to CRM-4892 and treats them as the same person. That works for three sources. But when a fourth channel arrives, such as a mobile loyalty app with its own ID format, we need to redesign the mapping table to avoid breaking existing logic.

Then we adjust again when a fifth source arrives.

Data Vault was built to solve this problem: a modeling approach that accepts every source’s ID scheme as-is and handles the reconciliation in a separate, controlled layer — so adding a new source never forces us to rebuild what already works.

In this article, we will discuss what a data vault is, the modelling concept behind it, and a real-world example of the decision-making process to implement a data vault.

What Is Data Vault?

Data Vault is a data warehouse modelling approach with one core rule: store raw data exactly as it arrives and keep all business logic — including decisions like “these two IDs belong to the same customer” — in a separate layer on top.

Unlike a standard ETL pipeline, where we transform and unify data as it lands, Data Vault delays that unification intentionally. The raw data stays untouched. The rules that interpret it live elsewhere, and we can update those rules without touching the data underneath.

The structure that makes this possible looks a lot like a social network — not the feed, but the graph underneath it. A social network tracks people (nodes), the connections between them (edges), and profile information that changes over time.

Data Vault

Data Vault

Data Vault uses exactly the same three-part structure.

This might sound similar to a graph database, but Data Vault is not one. A graph database like Neo4j stores and queries relationships natively — it is optimised to traverse connections between nodes at speed.

Data Vault uses relational tables and SQL. It borrows the conceptual shape of a graph to organise those tables, but it runs in a standard data warehouse like Snowflake, BigQuery, or Redshift. We don’t need new infrastructure to use Data Vault, just a different way of designing our tables.

The Three Building Blocks

The three building blocks are Hubs, Links, and Satellites. Each has one job and one job only — none of them overlap with the others — and that strict separation is what makes the model resilient to change.

Here is what each one does.

Hub

A Hub stores the unique business keys for a single entity — nothing else. A Customer Hub holds customer IDs. A Product Hub holds product IDs. A Store Hub holds store IDs. No names, no addresses, no prices.

Just the key and a record of which source system it came from. In the social network analogy, a Hub is a person node: it says “this person exists” and gives them a permanent identifier in our warehouse.

Link

A Link records a relationship between two or more Hubs. A sale link might connect the Customer Hub, the Product Hub, and the Store Hub — capturing the fact that a specific customer bought a specific product at a specific store.

Links are append-only: we never delete a relationship, we just stop adding new ones. In the social network analogy, a Link is the connection between two people — it records that the relationship happened, without storing any details about it.

Satellite

A satellite holds all the descriptive attributes, with full history built in. A customer satellite stores name, email address, shipping address, and loyalty tier — with a timestamp on every row.

When a customer’s email changes, we don’t overwrite the old record. We add a new row. The old one stays. In the social network analogy, a satellite is the profile page: all the information about a person, preserved across every update.

Data Vault ER diagram

Data Vault ER diagram

Where Does Data Vault Sit in Your Stack?

Data Vault is not a replacement for a data warehouse — it’s a way of organising one. It occupies a specific layer in the stack, and understanding that layer makes the whole architecture clear.

Most implementations follow a three-layer structure.

The Raw Vault is where source data lands, modelled as Hubs, Links, and Satellites, with no business logic applied.

The Business Vault sits on top and is where derived fields, business rules, and calculated metrics live — things like “unified customer ID” or “customer lifetime value".

The presentation layer is what analysts and BI tools actually query: star schemas or flat tables built from the business vault, organised around specific business questions.

Data Vault is not meant to be queried directly. It’s the foundation that makes the Presentation Layer trustworthy.

Workflow with Data Vault

Workflow with Data Vault

When to Use It — and When Not To

Data Vault isn’t the right call for every team. Here’s an honest split:

Use Data Vault when:

  • Multiple source systems have conflicting or overlapping keys – like the multi-channel retailer problem above
  • Auditability is a hard requirement: finance, healthcare, and government teams often need to trace every data change back to its source
  • The business is actively changing — acquisitions, new product lines, new channels — and the warehouse needs to absorb new sources without a redesign each time

Think twice before using it when:

  • The team is small, and the sources are clean and stable — a well-built Kimball star schema will be faster to build and easier for analysts to query
  • Time to first insight matters more than long-term flexibility
  • Analysts query the warehouse directly — the Raw Vault is not analyst-friendly; without a mature Presentation Layer, it creates more confusion than it solves

A Real Decision: ShopFlow’s Multi-Channel Problem

ShopFlow started as an online-only retailer and grew to include physical stores and a mobile loyalty app over three years. By the time the data team sat down to design their warehouse, they were sitting with the exact problem from the opening: three customer ID systems, no common key, and a fourth on the way.

The web platform used sequential integers. The CRM generated UUIDs. The in-store POS used a store-prefixed code like NYC-0042. Analysts were spending roughly 30% of their time on ID reconciliation before any real analysis could start.

What’s worse, when a customer returned an online purchase in-store, the transaction appeared as two separate, unrelated events — a web refund and a store credit — with nothing linking them.

Two options for the team to design their data warehouse.

Kimball Star Schema

The first was a Kimball star schema with a unified customer dimension, resolving all three ID formats during ETL.

That would have worked for the current sources, but the mobile loyalty app was six months from launch, bringing a fourth ID version. Implementing Kimball meant writing the ETL unification logic now for three formats, then rewriting it again for the fourth one, and restructuring the customer dimension table in the process.

Data Vault

The second option was Data Vault:

  1. load each source’s customer IDs into the Customer Hub as separate business keys tagged by source
  2. Record cross-channel behavior through Links using Hub references instead of raw IDs
  3. Push all ID unification logic into the Business Vault, where it could be updated without touching the raw data.

Inconsistant key problem

Inconsistant key problem

Final Decision

They chose Data Vault — not because it was simpler, but because the loyalty app launch made the Kimball option feel like building something they already knew they’d have to rebuild.

Here is how the Hub-Link-Satellite structure handled the problem.

Each source’s customer IDs flowed into the same Customer Hub, each carrying a source tag: source = web, source = crm, source = pos. The Hub accepted all three without forcing a decision about which ID was the “real” one.

The Sale Links recorded transactions using Hub references — not raw source IDs — so a purchase was stored as a relationship between Hub records, not between conflicting identifier strings.

The satellites captured each source’s customer attributes separately, preserving exactly what each system knew about that person.

The decision on whether cust_web_001 and CRM-4892 were the same human being was made in the Business Vault, in a dedicated reconciliation table using business-defined matching logic.

When the loyalty app launched, integrating it meant adding a new source tag to the Customer Hub and a new satellite for loyalty-specific attributes — no restructuring of existing tables, no disruption to existing reports.

How It Compares to Inmon and Kimball

Lastly, let’s have a quick comparison of the other two popular data modelling approaches.

All three approaches have their own advantages.

Inmon

Inmon bets that we can normalise sources into a single coherent model before serving data.

Kimball

Kimball bets that we can reshape sources into dimensions and facts that analysts can query directly.

Data Vault

Data Vault bets that we can’t — or shouldn’t — make those structural decisions upfront and that deferring them into a separate Business Vault layer is worth the added complexity.

In practice, most Data Vault teams don’t abandon Kimball entirely: they use the Raw and Business Vault for ingestion and integration, then build Kimball-style star schemas in the Presentation Layer for their analysts. The approaches play different positions in the same architecture.

Summary

Data Vault earns its complexity when the sources cannot be consolidated, the business is changing, and the cost of redesigning the warehouse every time a new system comes in exceeds the cost of building it right the first time.

If you like this article and want to show some love:

  • Clap 60 times — each one helps more than you think! 👏
  • **Follow me**, so you won’t miss it when a new article is published
  • You can buy m**e a Coffee** to support me further.
  • Let’s connect with me at **LinkedIn or lhungen@gmail.com to chat more about data!**

[embed]Data Modeling for Data Engineers: OLTP, OLAP, Inmon, and Kimball Explained Understanding OLTP, OLAP, and the two roads your data can take after the transaction is recordedblog.dataengineerthings.org

[embed]DuckDB and Cloud Data Warehouses: How to Choose For Your Project Using DuckDB for High-Performance Micro-ETL and Data Lake Queryingblog.dataengineerthings.org

[embed]From Normalization to Denormalization: A 101 Data Engineer’s Guide to Star Schema Design An E-commerce example to guide you on how to denormalize your transactional datablog.dataengineerthings.org


메타데이터
post_id
addcc790ebcc
slug
data-vault-explained-the-data-modeling-approach-built-for-messy-inconsistent-and-multi-source-addcc790ebcc
url
https://medium.com/towards-data-engineering/data-vault-explained-the-data-modeling-approach-built-for-messy-inconsistent-and-multi-source-addcc790ebcc
canonical_url
https://medium.com/towards-data-engineering/data-vault-explained-the-data-modeling-approach-built-for-messy-inconsistent-and-multi-source-addcc790ebcc
author_url
https://medium.com/@lhungen
status
ok
fetched_at
2026-06-09 15:37:30