MLOps V3: Infrastructure as Code (IaC)

Ditching manual configs

As a continuation of the MLOps guides I’ve been writing, Infrastructure as Code deserves a blog of its own. The concept of IaC extends beyond ML, it’s actually more of a DevOps solution, relevant across numerous domains. This blog covers what IaC is, and takes a practical look at how I incorporated it into my AWS architecture.

Photo by Michael Dziedzic on Unsplash

Why any Ops pipeline needs IaC?

To keep a production system healthy, your infrastructure needs to be reproducible, consistent, and scalable. This is simple enough in theory. In practice, you’re not managing one setup, you’re managing several.

You need multiple sets of machines for each stage: development, testing, production. Each with its own compute, isolated network, and firewall rules.

Rebuilding all of that manually every time is error prone. When you add new features, your environments will drift from each other. Bugs will appear in production that never showed up in testing. That’s not acceptable at scale.

Comparison between Infrastructure as Code (IaC) and Manual Configuration

IaC solves the problem of uncontrolled drift by making your infrastructure reproducible by definition.

So what actually is IaC?

Infrastructure as Code means your infrastructure is defined in code (services, configurations, dependencies, all of it). Instead of clicking through a console, you declare what your environment should look like and you let a tool(like Terraform) handle the rest.

The key shift is declarative vs. imperative thinking, and IaC can be written either way!

Imperative (step by step) vs. Declarative (end state)

Imperative says how to do something: “create this VM, install this package.” Declarative says what you want: “this VM should exist, with this package installed.” You describe the end state. The framework figures out how to get there.

That distinction matters more than it sounds. If you accidentally delete a resource, a declarative tool can detect the drift and fix it. An imperative script on the other hand, would just try to re run steps that might partially fail.

Terraform, for example, uses a declarative syntax, it’s great for ensuring an environment ends up in a desired state regardless of whatever half broken state it’s currently in.

Why choose OpenTofu specifically?

Cloud providers like AWS and Azure expose APIs for everything, for VMs, containers, storage, networks, databases. IaC tools sit on top of those APIs. The problem is that most provider native tools (like AWS CDK) lock you in. If you decide to switch providers, or mix components from multiple clouds, you will be rewriting everything.

Terraform solves this issue by being provider agnostic! I went with OpenTofu specifically, which is a fork of Terraform, created after HashiCorp moved it to a non open-source license. Opentofu is also accessible in regions where Terraform isn’t available, which was relevant in my case.

Practically speaking, the syntax and concepts are identical. The only real difference is the CLI: you will use tofu instead of terraform.

Caveat: An initial confusion was, should I change the lockfile from .terraform.lock.hcl to .opentofu.lock.hcl? the answer is no.

**.terraform.lock.hcl** should remain as is.

My setup before IaC

Before writing a single line of OpenTofu, I built everything manually in AWS. CloudFront, S3, API Gateway, Lambda, all configured by hand. That experience was actually valuable. I wouldn’t have appreciated IaC without first feeling what it’s like to do without it.

Here’s what my architecture looked like:

My project’s AWS architecture diagram

As you can see, the architecture needs a lot of components to be set up. Doing this once is fine. But trying to reproduce or modify it across environments will make it fall apart.

Moving to OpenTofu

I translated the entire set up above into OpenTofu scripts, so every resource, every configuration, was declared in code. Deployment through tedious steps became just:

tofu init
tofu apply

The commands above are all you need to have a thing which is reproducible and version controlled, it’s something you can hand off without having to remember what you set up six months ago.

Why IaC Hits Different in MLOps

Generic software deployments are relatively stable. You deploy an API, it serves traffic, you scale it occasionally. ML systems don’t work like that.

A single ML pipeline touches a surprising number of moving parts: data ingestion jobs, feature stores, training instances (often GPU), model registries, inference endpoints, and monitoring hooks. Each of these has its own compute requirements, permissions, and configuration. And unlike a typical backend service, these components don’t all run at the same time.

This creates a specific problem: your environments will diverge fast. The dev environment where you’re experimenting with a new model architecture looks nothing like production after a few weeks. Without IaC, that drift is invisible until something breaks.

There’s also the cost angle. GPU instances are expensive! In a manual setup, it’s easy to forget to tear something down after a training run.

With OpenTofu, your infrastructure is defined explicitly, nothing exists unless it’s declared, and tearing down an environment is one command.

In my setup specifically, having Lambda, API Gateway, and S3 declared in code meant that when I needed to test a new model version in an isolated environment, I wasn’t clicking through the AWS console for an hour. I ran tofu apply with a different variable set and had a clean replica ready.

That’s the MLOps specific value of IaC, reproducibility on demand, across environments that are genuinely complex and genuinely expensive to get wrong.

Final Notes

In this blog, we defined what Iac is, we compared imperative vs declarative thinking. We mentioned Opentofu as an available alternative of Terraform in restricted regions. And through my brief experience with AWS architecture, I demonstrated how fast it is to do everything on OpenTofu.

To wrap things up, we can consider that traditional code tells a computer how to do something. IaC on the other hand, tells the computer what you want to exist. You only have define the infrastructure, not the process.

IaC is simple as a concept, but it changes everything about how you manage systems at scale.

References

• https://aws.amazon.com/what-is/iac/