Day 10: Versioning Data with DVC

Welcome to Day 10 of the 100 Days of MLOps challenge!

Up until now, we have focused heavily on versioning our code, standardizing our environments, and automating our tests. But Machine Learning introduces a massive challenge that traditional software engineering doesn’t face: Data and Models.

Git is phenomenal for tracking code, but it completely falls apart when you try to commit a 50GB CSV file or a 2GB neural network weight file. Repositories bloat, clone times skyrocket, and GitHub will flat-out reject your push.

Today, we solve this for xFusionCorp Industries by introducing DVC (Data Version Control) into our ML repository.

The Scenario

The xFusionCorp Industries ML team is adopting DVC so that datasets and model files are versioned separately from the underlying Python code.

We have an existing Git repository located at /root/code/fraud-detection/ with an initial commit already in place. Our mission is to initialize DVC inside this repository and record the initialization files into Git.

Our Objectives:

1. Initialize DVC in the fraud-detection repository.
2. Ensure the standard .dvc/ control directory and .dvcignore file are created.
3. Stage the files DVC produces and record them with the commit message: "Initialize DVC".

Let’s get started.

Step-by-Step Solution

Step 1: Navigate to the Project

First, ensure you are inside the correct Git repository.

cd /root/code/fraud-detection/

Before doing anything, you can run git status to verify that you are on the master or main branch and that your working tree is clean.

Step 2: Initialize DVC

Just like initializing Git (git init), initializing DVC is a one-time setup command for the project.

Run the following command:

dvc init

What just happened? When you run this command, DVC creates a .dvc/ directory. This folder contains internal configuration files (like config) that dictate how DVC operates. It also creates a .dvcignore file (similar to .gitignore) to ensure that heavy data caches aren't accidentally tracked by Git.

Crucially, DVC is smart enough to know that its own configuration files should be tracked by Git. It automatically stages these internal files for you.

Step 3: Commit the DVC Configuration to Git

DVC manages your data, but Git manages DVC. To save this configuration, we need to commit the newly generated DVC tracking files to our Git repository.

Because DVC already staged the files (.dvc/config, .dvcignore, etc.), you simply need to commit them with the requested message:

git commit -m "Initialize DVC"

Output should look similar to this:

[master (root-commit) abc1234] Initialize DVC
 3 files changed, 6 insertions(+)
 create mode 100644 .dvc/.gitignore
 create mode 100644 .dvc/config
 create mode 100644 .dvcignore

Step 4: Verification

If you are using an IDE like VS Code with the DVC extension installed (as configured in the KodeKloud lab environment), you will immediately notice a change in your UI.

Once initialization is complete, the DVC extension detects the new .dvc/ directory. You will see a new DVC TRACKED section appear in your Explorer panel, and a DVC indicator will light up in the bottom status bar.

This confirms that your repository is now DVC-aware and ready to start tracking heavy datasets.

Why This Matters for MLOps

Initializing DVC is the first step toward true Machine Learning reproducibility.

Think of DVC as a bridge. It replaces your heavy files (like data.csv or model.pkl) with tiny, lightweight pointer files (like data.csv.dvc).

• Git tracks these tiny pointer files.
• DVC reads the pointers and downloads the actual heavy data from a remote storage bucket (like AWS S3, Google Cloud Storage, or an Azure Blob).

When a new engineer joins xFusionCorp, they can simply run git clone to get the code, and then dvc pull to fetch the exact datasets and models associated with that specific git commit. No more "I trained this on the old dataset" errors!

Stay tuned for Day 11, where we will start actively tracking data and configuring remote storage!

Tags: #MLOps #DVC #Git #DataScience #DevOps #KodeKloud