10 years of werf: The Cloud Native story we made together

werf brief stats

werf’s first commit was on January 22, 2016, and the project is now celebrating its 10th anniversary. To honor its anniversary, we decided to take a look at its key moments, milestones, wins, and future plans.

Here’s a brief timeline:

Below, you can find a more detailed story of how werf evolved.

werf v0 (dapp): January 2016 — December 2018

The project was originally known as dapp, and it started as a tool created in a DevOps agency for building container images within CI/CD pipelines. The focus on efficiency made it special: the incremental approach allowed reusing the results from the previous builds.

The first version was written in Ruby. We used a Vagrantfile-like Dappfile for defining build configurations, which featured an imperative Ruby-based DSL.

dimg 'symfony-demo-app' do
  docker.from 'ubuntu:16.04'

  git do
    add '/' do
      to '/demo'
      stage_dependencies.before_setup 'composer.json', 'composer.lock'
    end
  end

  shell do
    before_install do
      run 'apt-get update',
          'apt-get install -y curl php7.0',
          # add the phpapp user
          'groupadd -g 242 phpapp',
          'useradd -m  -d /home/phpapp -g 242 -u 242 phpapp'
    end
    install do
      run 'apt-get install -y php7.0-sqlite3 php7.0-xml php7.0-zip',
          # install composer
          'curl -LsS https://getcomposer.org/download/1.4.1/composer.phar -o /usr/local/bin/composer',
          'chmod a+x /usr/local/bin/composer'
    end
    before_setup do
      # modify source code permissions and run composer install
      run 'chown phpapp:phpapp -R /demo && cd /demo',
          "su -c 'composer install' phpapp"
    end
    setup do
      # use the current date as the application version
      run 'echo `date` > /demo/version.txt',
          'chown phpapp:phpapp /demo/version.txt'
    end
  end

  # the port must match the port specified in start.sh
  docker.expose 8000
end

The standout feature was its stage-based build model. Every single instruction created a new Docker image (a “stage”). This whole approach became the basis for build orchestration and efficient caching. Even in its first version, werf/dapp supported building any number of images in parallel using a shared configuration.

June 2016: Advanced build debugging tools

Pretty early in the game, dapp has got some build debugging tools. The user could hop into a container interactively at any build stage — before instructions were executed, after they completed, or upon an error. This greatly simplified the development and debugging of complex build scenarios and became one of dapp’s key features at the time.

July 2016: Chef support for describing assembly instructions

As builds grew more complex, it became clear that defining modular logic in a plain shell was a pain. To address this, we introduced support for Chef, a tool widely used in our company at the time.

This way, we could use Chef recipes during the image build process while ensuring no Chef artifacts or its cookbooks cluttered up the final image. So dapp has got a powerful mechanism for modularity and build logic reuse:

dimg do
  docker.from 'ubuntu:16.04'

  git.add('/').to('/app')
  docker.workdir '/app'

  docker.cmd ['/bin/bash', '-lec', 'bundle exec ruby app.rb']
  docker.expose 4567

  chef do
    cookbook 'apt'
    cookbook 'rvm'

    recipe 'ruby'
    recipe 'bundle_gems'
    recipe 'app_config'
  end
end

March 2017: Deploying to Kubernetes via Helm

Next year, dapp gained the capability to deploy applications to Kubernetes. It worked by calling the system’s Helm, and you just had to have a chart in the .helm directory.

Presenting the dapp deploy to deploy Helm charts in Kubernetes (2017)

But dapp did more than just calling Helm. It introduced linking the build and deploy steps right in the templates, using special Helm functions to handle images. We also added a basic way to track resource statuses during a deployment.

December 2017: Smart cleanup of the container registry

With more and more builds and tags piling up, cleaning up the container registry became a priority. So, at the end of 2017, we added a cleanup feature to dapp. It was a garbage collection system that worked with different policies (like for branches, commits, or tags) and was smart enough to check which images were being used in Kubernetes.

What are the challenges of cleaning up the container registry? Read more about our approach here.

March 2018: Support for YAML configuration and Ansible for describing assembly instructions (Chef gets replaced)

At this point, dapp shifted to a more user-friendly and familiar approach by adding YAML support. Moving from the imperative Ruby DSL to a declarative syntax made configurations easier to read, more predictable, and simpler to manage (particularly in complex projects).

At the same time, we moved away from Chef in favor of Ansible for build instructions. Chef turned out to be too cumbersome and didn’t live up to our expectations. Ansible, on the other hand, let us keep the modular and declarative style we wanted, while making it easier to get started with and more straightforward to run. Here’s how it looked:

dimg: ~
from: alpine:latest
git:
- add: /
  to: /app
  owner: app
  group: app
  excludePaths:
  - public/assets
  - vendor
  - .helm
  stageDependencies:
    install:
    - package.json
    - Bowerfile
    - Gemfile.lock
    - "app/assets/*"
- url: https://github.com/kr/beanstalkd.git
  add: /
  to: /build
ansible:
  beforeInstall:
  - name: "Create non-root main application user"
    user:
      name: app
      comment: "Non-root main application user"
      uid: 7000
      shell: /bin/bash
      home: /app
  - name: "Disable docs and man files installation in dpkg"
    copy:
      content: |
        path-exclude=/usr/share/man/*
        path-exclude=/usr/share/doc/*
      dest: /etc/dpkg/dpkg.cfg.d/01_nodoc
  install:
  - name: "Precompile assets"
    shell: |
      set -e
      export RAILS_ENV=production
      source /etc/profile.d/rvm.sh
      cd /app
      bundle exec rake assets:precompile
    args:
      executable: /bin/bash

October 2018: Spinning off deployment tracking into kubedog

The logic for tracking deployments (including statuses, events, logs, and waiting for resources to become ready) was moved out of dapp and into its own project: kubedog.

Separating kubedog into a standalone library allowed us to isolate this feature, make it reusable for other tools, and evolve its monitoring capabilities independently of the main product. Over time, kubedog became a key component of the werf ecosystem and was also adopted by the community for other projects.

November 2018: Support for secret values and files in deployment configs

dapp has got support for secret values and secret files for deployment configurations. That addressed the challenge of securely storing and using sensitive data like tokens, access keys, TLS certificates, and private keys — without having them stored in plain text in Git (or your Helm charts). Secrets were decrypted only at deployment time for use in Kubernetes templates and manifests

December 2018: Seamless migration from Ruby to Go

With the project’s growth, expanding use cases, and feedback from our users, it became obvious that the Ruby implementation was holding us back. So, we commenced the migration to Go.

The migration was done gradually to ensure a smooth experience for users, allowing us to keep developing the product while preserving its stability. In the end, we got rid of the old limitations and made it easier to integrate with the Kubernetes ecosystem.

Lines of code in Ruby vs. Go in dapp/werf on the way to the v1 release

werf v1: December 2018 — March 2020

January 2019: A new name: werf

This is when the project got its new name: werf. We allowed everyone in the company and the wider community participate in discussions and voting for the name. We ended up with around 100 suggestions, from sea and pirate themes to more abstract and technical concepts.

After the vote, three front-runners were identified:

• grog — 32 %
• flimb — 29,7 %
• werf — 27 %

Although werf did not end up on the first place in the vote, the team ultimately went with it. It just “clicked” — best reflected the project’s vibe, being associated with a place of assembly and creation — a “shipyard” (“werf” in Dutch) — which organically fit into the brand’s vision and future plans. (By the way, note that we prefer to use a lowercase initial when spelling “werf.”)

werf: new naming and logo of the project since January 2019

January 2019: The werf update manager (multiwerf)

multiwerf implemented our approach to handling versions through update channels. Its job was to auto-update werf and, crucially, to isolate a specific version for the active shell session.

The . $(multiwerf use 1.1 stable --as-file) shell command updated werf automatically in the background from a channel set by the user (stable for v1.1) and “pinned” that version for the active shell session. So you could use multiple werf versions on the same machine without any conflicts.

January 2019: Availability on all major operating systems

This is when werf was made available for all major operating systems, with testing and distribution now covering Linux, macOS, and Windows.

April 2019: Switching to our own fork of Helm

A major architectural choice for v1 was to build Helm right into the werf binary. Both the Helm client and Tiller ran inside the same werf process during deployment, so you didn’t need to install anything extra in your Kubernetes cluster. That approach ensured werf was fully compatible with Helm 2 while getting rid of most of its operational headaches and made things more secure.

Fun fact: Helm didn’t adopt this Tiller-less approach until Helm 3 came out in November 2019.

Having Helm built-in allowed us to level-up our deployment process. Instead of treating Helm like a black box with the --wait flag, werf started keeping a close eye on resources. It could track statuses, print events and logs, and stop the deployment process instantly on an error instead of waiting for a timeout. All the monitoring logic for this came from our kubedog project, which we had already spun off into its own solid library.

August 2019: 3-way merge is implemented in our Helm 2 fork

By this point, werf had begun contributing to the Helm upstream while at the same time developing its own Helm fork to deliver value to users more quickly and to experiment with features the official Helm lacked. One such extension was the 3-way merge for updating Kubernetes resources.

Having 3-way merge in our Helm 2 fork meant we could apply changes to existing resources way more accurately, because we considered their actual state in the cluster. This feature was introduced in werf before the Helm 3 showed up. It significantly improved the predictability and safety of updates.

To enhance reliability, werf also introduced locks to prevent parallel deployments of the same release. This helped prevent race conditions and inconsistent states when multiple deployments happened at once.

August 2019: First 1,000 stars on GitHub

The project surpassed the 1,000-star milestone on GitHub, a clear signal of growing interest and recognition from the Open Source community.

September 2019: Dockerfile support

Adding Dockerfile support was a major step in welcoming more users and making migration easier. werf learned to work with both its own Stapel syntax and classic Dockerfiles.

December 2019: Distributed locking put into a separate project

We pulled out the Go library for distributed locking from werf and made it into a separate tool: lockgate. It supports both local file locks and distributed locking via Kubernetes or an HTTP lock server, making it suitable for different infrastructure scenarios.

The library was well-received by the community, garnering external contributions and seeing adoption in other projects (over 250 ⭐ on GitHub), which proved it was a genuinely useful tool — again, not just for werf!

werf v1.1: March 2020 — November 2020

March 2020: Content-based tagging

Support for content-based tags was introduced. This laid out the basis for the subsequent move away from contextual tagging strategies (based on branches, commits, or CI) and toward a unified approach to image handling — at the configuration, cleanup, and storage levels — and made our use of the container registry way more efficient.

April 2020: Distributed layer storage based on container registry

From this moment, werf took on the task of synchronizing parallel builders that use the same container registry. The mechanism was modeled after how Docker handles its local storage when saving, selecting, and ensuring layer immutability.

The key difference was the scale: Docker synchronized processes that use a single host’s local storage, whereas werf applied this principle to a distributed environment. This way, multiple builders could work with the same container registry at the same time — no conflicts.

werf v1.2: November 2020 — April 2024

December 2020: Bundle support is added

Bundling was a new way to ship a Helm chart and all its container images as a single artifact package. Bundles contain everything you need to deploy an application, and you can distribute them around and use them without relying on the original Git repo.

Published bundles can be deployed with werf or any tool that supports Helm charts in OCI registries (like Helm, Argo CD, Flux). You can copy them between container registries, export them as tar files, use offline, or in air-gapped environments. When creating a bundle, werf automatically includes image details, tags, passed values, and global annotations and labels in the chart. This enables reproducible and standalone deployments — all without being tied to the project’s Git repository.

werf bundles simplified shipping a Helm chart with all related container images

May 2021: Secure update manager to replace multiwerf (trdl)

multiwerf was replaced by trdl (“true delivery”) — an update manager focused on the secure delivery of binaries from a Git repository to the user’s host. trdl was designed as a secure update channel that eliminates a whole class of risks associated with artifact substitution, compromise, or uncontrolled distribution.

trdl’s security is based on the combination of Git, a TUF repository, and HashiCorp Vault. Such a combination prevents supply chain attacks, verifies the integrity and authenticity of updates, and minimizes potential damage even if individual components are compromised.

trdl is not limited to werf and can be used for the secure release and distribution of any software. The CLI supports a wide range of scenarios, but the core update and usage approach known from multiwerf has been kept intact: . "$(trdl use werf 1.2 stable)" will pull the correct version of werf, runs a cryptographic check on it, and activates it in your current shell. That way, you can safely use different versions of tools on the same machine without any conflicts or global installs.

Releasing a new software version (v1.0.1) with trdl

December 2021: Online tutorial for developers dedicated to Kubernetes and deployment with werf

An online tutorial was launched, aimed at developers and DevOps engineers seeking to master Kubernetes and practical application delivery with werf. It integrated theory with step-by-step practical guides, covering a spectrum from basic concepts to advanced CI/CD scenarios.

The tutorial was tailored to popular languages and frameworks, featuring examples of applications and infrastructure (IaC). It allowed users to choose a familiar technology stack to learn Kubernetes and practice with werf on real-world use cases.

February 2022: Secure image builds; no privileged daemon required

werf got experimental support for Buildah, which enabled secure image builds in a rootless mode without using the Docker daemon. The old Docker-based build workflow was also preserved for situations where it was needed.

At the same time, the werf release process was updated to include ready-to-use images for running builds with Docker as well as right inside a Kubernetes cluster. This streamlined werf’s integration into diverse CI/CD environments and broadened its potential applications.

August 2022: Telemetry was introduced

werf has got telemetry to help analyze how the tool is being used. It collects data on versions, update channels, project activity, runtime environments, CLI command usage, and build metrics.

Telemetry became the basis for understanding how werf is used in the real world, and helped us to assess the stability of releases and identify bottlenecks in the delivery process.

The number of active projects that have been using werf throughout 2025

December 2022: werf joined CNCF!

The werf project was accepted into the CNCF (Cloud Native Computing Foundation), marking its official recognition in the Cloud Native space. This confirmed the project’s maturity and openness, signaling its readiness for wider adoption while encouraging greater community involvement in its development and integration with other CNCF tools.

May 2023: Abandoning the Helm fork and launching Nelm

The first commit to Nelm marked a new stage in the evolution of werf’s deployment mechanism. By this point, it had become clear that further development of the custom Helm fork was constrained by Helm’s own architecture. So we decided to abandon the fork and rewrite the deployment subsystem from scratch while maintaining backward compatibility.

2023–2024: Community involvement

During these years, werf was showcased at several offline events:

• at KCD Czech & Slovak 2023 in Bratislava;
• at the CNCF Project Pavilion during KubeCon + CloudNativeCon Europe 2023 in Amsterdam;
• at the CNCF Project Pavilion during KubeCon + CloudNativeCon Europe 2024 in Paris.

[embed]

werf v2: April 2024 — …

March 2025: Nelm 1.0 released

Nelm 1.0 was released. It is a stable, backward-compatible alternative to Helm 3, designed to work with Helm charts. Nelm comes as a standalone CLI tool and can also be used as a library in other tools.

November 2025: Ongoing Nelm development and how it compares with Helm 4

Nelm adoption is growing, and so does the list of features it provides. After the long-awaited Helm 4 release — its first major update in years, — the community asked whether Nelm was still relevant. Thus, we published a detailed overview of how Nelm continues to evolve and remains a decent alternative with a broader feature set and a dedicated user base.

2024–2025: Community involvement

werf was featured at:

• the CNCF YouTube channel, featuring the “From improving Helm to developing Nelm: the evolution of deployments in werf” webinar;
• the “Specialized Templating” episode of the “You Choose!” YouTube show (Ch. 05, Ep. 05), alongside a diverse set of CNCF tools: Porter, Radius, Score, and PipeCD;
• FOSSASIA Summit 2025 in Bangkok.

[embed]

January 2026: werf turns 10!

The werf project is celebrating its 10th anniversary. Over this period, it has evolved from a simple experiment in incremental Docker image building to a mature Open Source ecosystem for application delivery in Kubernetes. This ecosystem includes its own tools for building, deploying, and distributing software, as well as a number of standalone projects.

The werf ecosystem in numbers

werf (website + GitHub):

• 4600+ GitHub ⭐
• 1300+ releases
• 18,000+ active projects using werf
• 15,000+ commits
• 60+ contributors
• 6000+ merged pull requests

Other projects include:

• Nelm: 1000+ ⭐, 45 releases, 800+ commits
• trdl (website): 290+ ⭐, 45 releases, 900+ commits
• kubedog: 700+ ⭐, 38 releases, 650+ commits
• lockgate: 250+ ⭐, 1 release, ~100 commits

GitHub star history for werf’s repositories

What’s next for werf

But we’re not stopping here — we have big plans ahead:

• A new build architecture featuring deep integration with Docker BuildKit.
• Enhanced supply chain security, including image signing, verification, SBOMs, and vulnerability scanning.
• A Nelm operator to integrate with tools like Argo CD and Flux, or to be used on its own (Issue #494).
• The ability to patch Helm chart resources (Issue #115).
• An alternative to Helm templates (but not a replacement!): our TypeScript experiment is almost ready for you to try (PR #502, Issue #54).
• …and a whole lot more. Stay tuned!