Helm for Data Engineers: A Beginner’s Guide Using kind and PostgreSQL

From local Kubernetes to installing a real database with Helm

Data engineers do not only write SQL, build DAGs, and move data between systems anymore.

In many teams, data engineers also touch infrastructure: Airflow, PostgreSQL, Kafka, Trino, Superset, dbt jobs, monitoring tools, metadata platforms, and internal services. A lot of these tools are deployed on Kubernetes. And in Kubernetes-based environments, one of the most common ways to install and manage these tools is Helm.

Helm is often described as the package manager for Kubernetes. More practically, Helm helps us define, install, configure, upgrade, and roll back Kubernetes applications. The official Helm quickstart lists a Kubernetes cluster and a configured Helm installation as prerequisites, which is why we will use kind to create a small local Kubernetes cluster for learning. (Helm)

In this guide, we will go from zero to installing PostgreSQL on a local Kubernetes cluster using the Bitnami PostgreSQL Helm chart. Bitnami’s PostgreSQL chart is a good beginner example because it creates real Kubernetes resources: Pods, Services, Secrets, StatefulSets, and PersistentVolumeClaims. (Artifact Hub)

By the end, you should understand not just which commands to run, but also what Helm is doing behind the scenes.

What we will cover

In this guide, we will learn:

1. Why Helm matters for data engineers
2. What kind is and why we use it
3. How Helm works conceptually
4. How to install PostgreSQL with a Helm chart
5. How to customize a chart using values.yaml
6. How to inspect the Kubernetes resources Helm creates
7. How to upgrade and roll back a Helm release
8. Common debugging commands
9. Production lessons for data engineers

Why should data engineers care about Helm?

A lot of data tools are not just Python packages or SQL scripts. They are full services.

For example:

PostgreSQL     -> database
Airflow        -> workflow orchestration
Kafka          -> event streaming
Trino          -> distributed SQL query engine
Superset       -> BI/dashboarding
Prometheus     -> monitoring
Grafana        -> visualization
OpenMetadata   -> metadata management

In a real data platform, these services usually need:

- configuration
- secrets
- persistent storage
- networking
- resource limits
- upgrades
- rollbacks
- monitoring
- environment-specific values

Managing all of that using raw Kubernetes YAML becomes painful very quickly.

That is where Helm helps.

Instead of writing dozens of YAML files manually, Helm gives us a reusable package called a chart. We install that chart into Kubernetes as a release, and we customize it using a values file.

A simple way to think about it:

Chart + Values = Kubernetes YAML
Kubernetes YAML = Actual running resources

Helm does not run PostgreSQL by itself.

Kubernetes runs PostgreSQL.

Helm prepares and manages the Kubernetes resources needed to run PostgreSQL.

That distinction is very important.

The basic Helm mental model

Before touching commands, let’s define the core concepts.

For example:

helm upgrade --install lab-postgres bitnami/postgresql -f values.yaml

This means:

Take the bitnami/postgresql chart,
combine it with my values.yaml file,
render Kubernetes manifests,
and apply them to the cluster as a release called lab-postgres.

Step 1: Create a local Kubernetes cluster with kind

For this guide, we do not need a cloud Kubernetes cluster.

We can use kind, which creates a local Kubernetes cluster using containers as Kubernetes nodes. The kind documentation describes creating a cluster as simply running kind create cluster, and it also supports deleting the cluster easily when we are done. (Kind)

You need:

- Docker
- kind
- kubectl
- Helm

Create a cluster:

kind create cluster --name helm-lab

Check that the cluster exists:

kubectl cluster-info --context kind-helm-lab
kubectl get nodes

You should see one Kubernetes node.

Example:

NAME                     STATUS   ROLES           AGE   VERSION
helm-lab-control-plane   Ready    control-plane   1m    v1.xx.x

Check your tools:

kind version
kubectl version --client
helm version

At this point, we have a working local Kubernetes cluster.

Step 2: Add the Bitnami Helm repository

Helm charts can be stored in repositories.

For this guide, we will use the Bitnami PostgreSQL chart.

Add the Bitnami chart repository:

helm repo add bitnami https://charts.bitnami.com/bitnami

Update your local chart index:

helm repo update

Search for PostgreSQL:

helm search repo bitnami/postgresql

You can inspect basic chart metadata:

helm show chart bitnami/postgresql

You can also inspect the default configuration:

helm show values bitnami/postgresql > postgres-default-values.yaml

This command is extremely useful.

When you use a new Helm chart, do not guess the configuration fields. Always inspect the default values first.

Step 3: Create a namespace

A Kubernetes namespace helps us isolate resources.

Create a namespace for this lab:

kubectl create namespace data-lab

Check it:

kubectl get namespaces

Now all our PostgreSQL resources will live inside data-lab.

Step 4: Create a values file

A Helm chart usually has many configurable options.

We do not want to pass all values through long CLI flags. Instead, we create a file called pg-values.yaml.

auth:
  postgresPassword: postgres-admin-password
  username: de_user
  password: de_password
  database: warehouse

primary:
  persistence:
    enabled: true
    size: 1Gi

This tells the chart:

- Create a PostgreSQL database called warehouse
- Create a user called de_user
- Set the user's password
- Set the postgres admin password
- Enable persistence
- Create a 1Gi persistent volume claim

For a local workshop, hardcoded passwords are fine.

For production, this is not acceptable. Secrets should not be committed to Git as plaintext.

Step 5: Install PostgreSQL using Helm

Now we install PostgreSQL.

helm upgrade --install lab-postgres bitnami/postgresql \
  --namespace data-lab \
  --values pg-values.yaml

I prefer helm upgrade --install even for beginner examples because this is closer to real-world usage.

It means:

If the release does not exist, install it.
If it already exists, upgrade it.

Check the release:

helm list -n data-lab

Check release status:

helm status lab-postgres -n data-lab

Now inspect the Kubernetes resources:

kubectl get all -n data-lab

Also check storage and secrets:

kubectl get pvc -n data-lab
kubectl get secret -n data-lab
kubectl get statefulset -n data-lab

This is the moment where Helm starts to make sense.

You did not manually create a StatefulSet, Service, Secret, or PVC.

The Helm chart generated them for you.

Step 6: Understand what Helm created

For PostgreSQL, you will usually see resources like:

Pod              -> the running PostgreSQL container
StatefulSet      -> manages the database pod with stable identity
Service          -> gives PostgreSQL a stable network address
Secret           -> stores credentials
PVC              -> stores database data
ConfigMap        -> may store configuration

This is important for data engineers.

PostgreSQL is not a stateless web app.

It needs persistent storage. If the pod is deleted and recreated, the data should survive. That is why the chart creates a PersistentVolumeClaim.

A good beginner distinction is:

DeploymentStatefulSetUsually for stateless appsUsually for stateful appsPods are interchangeablePods have stable identityOften no persistent diskUsually has persistent storageEasier to replaceMore careful upgrades needed

Databases usually run as StatefulSets.

Step 7: Connect to PostgreSQL

First, find the service name:

kubectl get svc -n data-lab

You will likely see a service similar to:

lab-postgres-postgresql

Port-forward it to your local machine:

kubectl port-forward -n data-lab svc/lab-postgres-postgresql 5432:5432

In another terminal, connect with psql:

PGPASSWORD=de_password psql \
  -h 127.0.0.1 \
  -U de_user \
  -d warehouse \
  -c "select current_database(), current_user, version();"

If you do not have psql installed locally, use a temporary PostgreSQL client pod:

kubectl run pg-client \
  --rm -it \
  --namespace data-lab \
  --image=postgres:16 \
  --env PGPASSWORD=de_password \
  -- psql -h lab-postgres-postgresql -U de_user -d warehouse

Now we have PostgreSQL running inside Kubernetes, installed through Helm.

Step 8: Render the chart before applying it

One of the most useful Helm commands is:

helm template

This renders the chart locally and shows the final Kubernetes YAML.

helm template lab-postgres bitnami/postgresql \
  --namespace data-lab \
  --values pg-values.yaml

This command does not install anything.

It only shows what Helm would generate.

You can also do a dry run:

helm upgrade --install lab-postgres bitnami/postgresql \
  --namespace data-lab \
  --values pg-values.yaml \
  --dry-run

This is a habit worth building early.

Before applying a Helm chart, ask:

What YAML will this generate?
Which resources will it create?
Which values am I overriding?
What will change in the cluster?

Step 9: Upgrade the release

Let’s say we want to increase the persistent volume size from 1Gi to 2Gi.

Update pg-values.yaml:

auth:
  postgresPassword: postgres-admin-password
  username: de_user
  password: de_password
  database: warehouse

primary:
  persistence:
    enabled: true
    size: 2Gi

Apply the change:

helm upgrade lab-postgres bitnami/postgresql \
  --namespace data-lab \
  --values pg-values.yaml

Check the release history:

helm history lab-postgres -n data-lab

Check the PVC:

kubectl get pvc -n data-lab

This is a good place to discuss an important production lesson:

Increasing a volume may be possible depending on the StorageClass. Shrinking a volume is generally not allowed.

That matters a lot for data systems.

A Helm upgrade can change Kubernetes manifests, but Kubernetes and the underlying storage system decide what is actually possible.

Step 10: Roll back a release

Helm keeps release history.

Check the history:

helm history lab-postgres -n data-lab

Rollback to a previous revision:

helm rollback lab-postgres 1 -n data-lab

Check status:

helm status lab-postgres -n data-lab

But here is the critical warning:

Helm rollback reverts Kubernetes resources. It does not magically roll back database data.

For example, if an application migration changed your database schema, Helm rollback will not automatically undo that schema migration.

This is one of the biggest differences between rolling back a stateless app and rolling back a data service.

For data engineers, this is not a small detail. This is the whole game.

Step 11: Debugging Helm releases

When something fails, beginners often only look at Helm.

But Helm is only one layer.

You need both Helm and Kubernetes commands.

Start with Helm:

helm list -n data-lab
helm status lab-postgres -n data-lab
helm history lab-postgres -n data-lab
helm get values lab-postgres -n data-lab
helm get manifest lab-postgres -n data-lab

Then inspect Kubernetes:

kubectl get pods -n data-lab
kubectl describe pod <pod-name> -n data-lab
kubectl logs <pod-name> -n data-lab
kubectl get events -n data-lab --sort-by=.lastTimestamp
kubectl get pvc -n data-lab
kubectl describe pvc <pvc-name> -n data-lab

A simple debugging flow:

1. Is the Helm release deployed?
2. Did Helm generate the resources I expected?
3. Are the Pods running?
4. If not, what does kubectl describe say?
5. Are there image pull errors?
6. Are there scheduling errors?
7. Are there PVC/storage errors?
8. Are the credentials correct?
9. Are the logs showing application-level errors?

In real work, this flow is more valuable than memorizing commands.

The most important Helm commands for beginners

You do not need to learn every Helm command on day one.

Start with these:

helm repo add
helm repo update
helm search repo
helm show chart
helm show values
helm install
helm upgrade --install
helm list
helm status
helm get values
helm get manifest
helm history
helm rollback
helm uninstall

And always pair them with:

kubectl get
kubectl describe
kubectl logs
kubectl get events

Helm tells you what was installed.

Kubernetes tells you what is actually happening.

Production lessons for data engineers

Installing PostgreSQL locally is easy.

Running data services in production is not.

Before installing or upgrading a Helm chart in production, ask:

- Did I pin the chart version?
- Did I pin the image tag?
- Where are secrets stored?
- Is persistence enabled?
- Which StorageClass is used?
- Do I have backups?
- What happens during upgrade?
- Are resource requests and limits configured?
- Is monitoring enabled?
- Are metrics exposed?
- Are there readiness and liveness probes?
- Can I roll back safely?
- What does rollback not cover?

This checklist matters for tools like:

- PostgreSQL
- Kafka
- Airflow
- Trino
- Redis
- Elasticsearch
- ClickHouse
- Superset
- OpenMetadata

The mistake is thinking Helm makes everything safe.

It does not.

Helm makes deployment manageable.

You still need to understand the application, storage, networking, secrets, and upgrade behavior.

Clean up

When you are done with the lab, remove PostgreSQL:

helm uninstall lab-postgres -n data-lab

Delete the namespace:

kubectl delete namespace data-lab

Delete the kind cluster:

kind delete cluster --name helm-lab

Final thoughts

Helm is one of those tools that looks simple at first:

helm install something

But the real value is not the install command.

The real value is understanding:

- how charts are configured
- how values change the generated Kubernetes YAML
- how releases are upgraded
- how rollbacks work
- how to inspect the resources Helm created
- how to debug failures using both Helm and kubectl

For data engineers, Helm is especially important because many data tools are stateful. A bad upgrade is not just a broken pod. It can mean broken storage, broken credentials, failed migrations, or downtime for pipelines and dashboards.

So the goal is not just to “learn Helm.”

The goal is to become comfortable managing real data infrastructure on Kubernetes.

And PostgreSQL is a great place to start.