Controller-gen: Building a Custom Kubernetes CRD Without kubebuilder

Controller-gen Standalone: Building a Custom Kubernetes CRD Without kubebuilder

A few months back I built a Kubernetes operator from scratch in Go. It watches HTTPRoute resources and automatically syncs them to Cloudflare Tunnel. Every time you create an HTTPRoute, the operator pushes the rule to Cloudflare and creates the DNS CNAME record. I wrote about building that operator here: https://medium.com/@rk90229/building-a-kubernetes-operator-from-scratch-automate-cloudflare-tunnel-with-httproute-421d6642dc11?sk=d15ed60a0df8e7f144edcc2703dc39af

The full operator code, including the CRD types and reconciler changes covered here, is at https://github.com/rajeshkio/cf-tunnel-operator.

On a free medium plan? Read here for free.

It worked. But after running it for a while I kept hitting the same problem. I had no way to see what the operator had actually done. kubectl get httproute shows what I declared. It says nothing about what the operator pushed to Cloudflare, whether the sync succeeded, or when it last ran. My options were to open the Cloudflare dashboard or grep through logs. Neither felt right for something I wanted to use day-to-day.

I wanted one command that showed me everything.

kubectl -n cf-tunnel-operator-system get tunnelstatuses
NAME                                          AGE
cattle-neuvector-system-neuvector-httproute   105m
cattle-system-rancher-httproute               105m
mlops-mlflow                                  105m

And inspecting any one of them.

kubectl -n cf-tunnel-operator-system get tunnelstatuses mlops-mlflow -o yaml
apiVersion: cf-tunnel-operator.rajesh-kumar.in/v1alpha1
kind: TunnelStatus
metadata:
  name: mlops-mlflow
  namespace: cf-tunnel-operator-system
spec:
  httpRouteName: mlflow
  httpRouteNamespace: mlops
status:
  backendService: http://mlflow.mlops.svc.cluster.local:5000
  hostname: mlflow.rajesh-kumar.in
  lastSyncTime: "2026-06-03T15:15:16Z"
  message: ""
  notlsverify: false
  scheme: http
  syncStatus: Success

That resource did not exist. I built it. This article is about how, what broke along the way, and what I learned.

Why Not kubebuilder

The first question anyone asks when they hear “custom CRD” is whether I used kubebuilder. Kubebuilder is a framework that scaffolds an entire operator project: directory structure, main entry point, controller boilerplate, CRD generation, webhooks. For a greenfield project it saves a lot of setup.

My operator already existed and was running in production. It uses controller-runtime directly, which is the same library kubebuilder wraps. Adopting kubebuilder at this point means reorganising the project to fit its conventions, which touches files that are already working. I only wanted to add one new type.

The tool that actually generates CRD manifests from Go structs is controller-gen. It works independently of kubebuilder. You install it standalone and point it at your api directory.

go install sigs.k8s.io/controller-tools/cmd/controller-gen@latest
export PATH=$PATH:$(go env GOPATH)/bin
echo 'export PATH=$PATH:$(go env GOPATH)/bin' >> ~/.zshrc
source ~/.zshrc

That is the approach here.

Creating the Directory Structure

My project had no api directory. I created it following the same convention Kubernetes uses to organise its own APIs. Kubernetes groups built-in resources by group and version: apps/v1 for Deployments, networking.k8s.io/v1 for Ingress, gateway.networking.k8s.io/v1 for HTTPRoute. Custom resources follow the same pattern. Your types live in api/version/.

The version I picked is v1alpha1. The Kubernetes API reference defines exactly what alpha means: “the API may change in incompatible ways in a later software release without notice.” That is the honest version label for a type I just defined and have not proven stable. If the fields need to change later, v1alpha1 signals to anyone using it that they should expect that. The CRD versioning docs follow the same convention: version strings start with v, a number, and an optional alpha or beta designation.

mkdir -p api/v1alpha1
touch api/v1alpha1/groupversion_info.go
touch api/v1alpha1/tunnelstatus_types.go

Two files. Here is what each one does and why it exists.

groupversion_info.go: Registering the API Group

Every Kubernetes resource belongs to an API group and version. Together with the kind, these three form what Kubernetes calls a GroupVersionKind, or GVK. When you write apiVersion: apps/v1 in a manifest, apps is the group and v1 is the version. For my custom resource, the apiVersion will be cf-tunnel-operator.rajesh-kumar.in/v1alpha1.

This file declares that combination and provides the AddToScheme function that main.go will call later.

// +groupName=cf-tunnel-operator.rajesh-kumar.in
package v1alpha1
import (
    "k8s.io/apimachinery/pkg/runtime/schema"
    "sigs.k8s.io/controller-runtime/pkg/scheme"
)
var (
    GroupVersion  = schema.GroupVersion{Group: "cf-tunnel-operator.rajesh-kumar.in", Version: "v1alpha1"}
    SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion}
    AddToScheme   = SchemeBuilder.AddToScheme
)

The comment at the top, // +groupName=cf-tunnel-operator.rajesh-kumar.in, is not a comment for humans. It is a marker that controller-gen reads when it generates the CRD manifest later.

The group uses my domain as the prefix, which is the convention for operator APIs. This becomes the apiVersion prefix in any manifest: cf-tunnel-operator.rajesh-kumar.in/v1alpha1.

tunnelstatus_types.go: Defining the Type

This is where the actual struct lives. I thought for a bit about what fields are actually useful. The minimum: hostname, the backend service URL the operator pushed, whether TLS verification is disabled, the scheme, when the last sync happened, and whether it succeeded or failed with a message if not.

package v1alpha1
import metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
// +kubebuilder:object:root=true
// +kubebuilder:subresource:status
type TunnelStatus struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`
    Spec              TunnelStatusSpec   `json:"spec,omitempty"`
    Status            TunnelStatusStatus `json:"status,omitempty"`
}
type TunnelStatusSpec struct {
    HTTPRouteNamespace string `json:"httpRouteNamespace"`
    HTTPRouteName      string `json:"httpRouteName"`
}
// +kubebuilder:object:generate=true
type TunnelStatusStatus struct {
    Hostname       string      `json:"hostname,omitempty"`
    BackendService string      `json:"backendService"`
    LastSyncTime   metav1.Time `json:"lastSyncTime,omitempty"`
    SyncStatus     string      `json:"syncStatus"`
    Scheme         string      `json:"scheme"`
    NoTLSVerify    bool        `json:"notlsverify"`
    Message        string      `json:"message"`
}
// +kubebuilder:object:root=true
type TunnelStatusList struct {
    metav1.TypeMeta `json:",inline"`
    metav1.ListMeta `json:"metadata,omitempty"`
    Items           []TunnelStatus `json:"tunnelStatus"`
}
func init() {
    SchemeBuilder.Register(&TunnelStatus{}, &TunnelStatusList{})
}

The comments with // +kubebuilder: are markers for controller-gen, not human readers. +kubebuilder:object:root=true marks this as a top-level resource. +kubebuilder:subresource:status is something I will explain when we get to the hardest debugging section.

+kubebuilder:object:generate=true on TunnelStatusStatus tells controller-gen to generate a DeepCopy method for this nested struct specifically. Without it, the generated zz_generated.deepcopy.go will have a DeepCopyObject for TunnelStatus but nothing for TunnelStatusStatus, and the DeepCopy for the parent struct will be incomplete.

Spec only holds a reference to the HTTPRoute this resource belongs to. The operator manages TunnelStatus entirely so there is nothing for a user to configure. Kubernetes requires every custom resource to have a spec field. The HTTPRoute name and namespace there make the relationship explicit.

TunnelStatusList is required because the Kubernetes API has two endpoints for every resource: one returning a single object and one returning a list. The list endpoint returns a different JSON structure with a metadata.resourceVersion and an items array. controller-runtime needs a separate Go type to deserialise that response. The init function registers both types with the SchemeBuilder.

What Is the Scheme and Why Does It Matter

Before this code can work, it needs to be registered in what controller-runtime calls the scheme. The controller-runtime package docs describe it as: “Clients, Caches, and many other things in Kubernetes use Schemes to associate Go types to Kubernetes API Kinds.”

In practice: when your reconciler calls r.Get() with a TunnelStatus pointer, the client needs to know which API endpoint to call. It cannot infer this from the Go type name. The scheme is the registry that maps the Go struct TunnelStatus to the GVK cf-tunnel-operator.rajesh-kumar.in/v1alpha1/TunnelStatus, which tells the client to call /apis/cf-tunnel-operator.rajesh-kumar.in/v1alpha1/namespaces/foo/tunnelstatuses.

The SchemeBuilder in groupversion_info.go and the SchemeBuilder.Register call in init() are setting up that registry. main.go will activate it. Without registration, the client has no idea what your struct is and returns an unknown type error at runtime.

What Is DeepCopy and Why the Compiler Requires It

At this point the code does not compile.

cannot use &TunnelStatus{} (value of type *TunnelStatus) as runtime.Object value:
*TunnelStatus does not implement runtime.Object (missing method DeepCopyObject)

The scheme’s Register function expects objects that implement the runtime.Object interface. That interface requires DeepCopyObject(). The reason Kubernetes requires this connects to how the informer cache works. When your reconciler reads an object via r.Get(), it gets a pointer into a shared in-memory cache. If you modify that object without copying it first, you corrupt the cache for every other goroutine that holds a reference to it. DeepCopyObject gives you an independent copy that is safe to modify. The controller-runtime maintainers are direct about this: modifying the underlying cache object directly “is terrible.”

Writing DeepCopy methods by hand for every nested struct is error prone. controller-gen generates them from the marker annotations.

controller-gen object:headerFile="" paths="./api/..."

This creates api/v1alpha1/zz_generated.deepcopy.go. The zz_ prefix means generated, do not edit by hand.

Generating the CRD Manifest and Applying It

controller-gen crd paths="./api/..." output:crd:artifacts:config=deploy/crd

This produces a YAML file you apply to the cluster.

One thing to watch: without the +groupName marker in groupversion_info.go, controller-gen produces a file named _tunnelstatuses.yaml with group: “” in the spec. The marker is what gives it the correct filename and group field. I hit this on the first run, empty group, useless filename, and adding the marker and regenerating fixed it.

kubectl apply -f deploy/crd/cf-tunnel-operator.rajesh-kumar.in_tunnelstatuses.yaml

kubectl get crd tunnelstatuses.cf-tunnel-operator.rajesh-kumar.in
NAME                                                CREATED AT
tunnelstatuses.cf-tunnel-operator.rajesh-kumar.in   2026-06-03T11:53:54Z

Wiring the Scheme Into main.go

The SchemeBuilder is set up but not active yet. It gets activated in main.go when you add TunnelStatus to the manager’s scheme.

scheme := runtime.NewScheme()
clientgoscheme.AddToScheme(scheme)   // Pod, ConfigMap, Secret etc.
gatewayv1.Install(scheme)            // HTTPRoute
v1alpha1.AddToScheme(scheme)         // TunnelStatus and TunnelStatusList

That last line is AddToScheme from groupversion_info.go. Calling it adds TunnelStatus and TunnelStatusList to the runtime scheme. Its the same scheme the manager passes to every client it creates. Without this line, the operator starts fine and crashes the moment it tries to create a TunnelStatus resource with an unknown type error.

RBAC: Sort This Out Before Anything Else

The operator service account needs explicit permission to manage TunnelStatus resources. Add this to the ClusterRole before testing anything else.

- apiGroups: ["cf-tunnel-operator.rajesh-kumar.in"]
  resources: ["tunnelstatuses", "tunnelstatuses/status"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

tunnelstatuses and tunnelstatuses/status are separate entries. The /status subresource is a distinct permission in Kubernetes RBAC. Without it you will hit the below error. If you are reading this wondering why status writes are forbidden even though your RBAC looks complete, this is exactly why.

User "system:serviceaccount:cf-tunnel-operator-system:cf-tunnel-operator"
cannot update resource "tunnelstatuses/status" in API group
"cf-tunnel-operator.rajesh-kumar.in"

Making the Operator Namespace Configurable

The upsertTunnelStatus function needs to know which namespace to create TunnelStatus resources in.

The Kubernetes Downward API solves this. It lets a pod read its own metadata and expose it as an environment variable. The kubelet injects the actual value at runtime from the pod spec with nothing to configure manually.

In the Helm chart deployment template, one entry in the env block alongside the Cloudflare credentials.

- name: POD_NAMESPACE
  valueFrom:
    fieldRef:
      fieldPath: metadata.namespace

When running locally with go run, the Downward API does not exist. You set it in your shell.

export POD_NAMESPACE=cf-tunnel-operator-system
go run main.go

The reconciler struct gained a new field and main.go passes the value through.

type HTTPRouteReconciler struct {
    client.Client
    CF                *cf.Client
    OperatorNamespace string
}

The Status Subresource: What It Is Before You Write Code That Uses It

The marker +kubebuilder:subresource:status in the type definition causes the generated CRD to include this.

subresources:
  status: {}

This tells the API server to treat spec and status as two separate API endpoints. A normal write to the resource only touches spec. To write status you must call the /status endpoint explicitly via r.Status().Update(). This exists so that a user editing spec cannot accidentally overwrite status the operator wrote, and so the operator updating status cannot accidentally wipe spec changes.

The Debugging: Why Status Was Always Empty

My first version of upsertTunnelStatus set status fields inside the CreateOrUpdate mutate function alongside the spec fields.

_, err := controllerutil.CreateOrUpdate(ctx, r.Client, tsResource, func() error {
    tsResource.Spec.HTTPRouteName = route.Name
    tsResource.Spec.HTTPRouteNamespace = route.Namespace
    tsResource.Status.Hostname = tunnelStatusInput.Hostname
    tsResource.Status.LastSyncTime = metav1.Now()
    tsResource.Status.SyncStatus = tunnelStatusInput.SyncStatus
    return nil
})
if err != nil {
    return err
}
if err := r.Status().Update(ctx, tsResource); err != nil {
    return err
}

No errors. But kubectl showed this.

spec:
  httpRouteName: neuvector-httproute
  httpRouteNamespace: cattle-neuvector-system

No status block. I added a log line after CreateOrUpdate.

log.Info("Status before update", "lastSyncTime", tsResource.Status.LastSyncTime, "syncStatus", tsResource.Status.SyncStatus)

Output.

{"msg":"Status before update","lastSyncTime":"0001-01-01 00:00:00 +0000 UTC","syncStatus":""}

Zero values. The fields I assigned inside the mutate function were gone by the time Status().Update() ran.

The diagram shows the actual field values inside tsResource at each step. Step 3 is where you set Hostname and SyncStatus in memory inside the mutate function, they are there. Step 5 is where they disappear. CreateOrUpdate loads the API server response back into tsResource after writing spec. The server has empty status because the /status endpoint was never called. So your in-memory values get replaced with empty strings.

Step 6, Status().Update(), then faithfully writes those empty strings to the cluster. No error, wrong result.

The fix is in the bottom box: assign status fields after step 5 returns, then call Status().Update(). By that point CreateOrUpdate is done touching tsResource.

_, err := controllerutil.CreateOrUpdate(ctx, r.Client, tsResource, func() error {
    tsResource.Spec.HTTPRouteName = route.Name
    tsResource.Spec.HTTPRouteNamespace = route.Namespace
    return nil
})
if err != nil {
    log.Error(err, "Failed to upsert TunnelStatus")
    return err
}

tsResource.Status.Hostname = tunnelStatusInput.Hostname
tsResource.Status.BackendService = tunnelStatusInput.Service
tsResource.Status.LastSyncTime = tunnelStatusInput.LastSyncTime
tsResource.Status.NoTLSVerify = tunnelStatusInput.NoTLSVerify
tsResource.Status.Scheme = tunnelStatusInput.Scheme
tsResource.Status.SyncStatus = tunnelStatusInput.SyncStatus
tsResource.Status.Message = tunnelStatusInput.Message
if err := r.Status().Update(ctx, tsResource); err != nil {
    log.Error(err, "Failed to update TunnelStatus status")
    return err
}

Mutate function handles spec only. Status is assigned after CreateOrUpdate finishes, then Status().Update() sends it to the /status endpoint with the values intact.

The Validation Error With lastSyncTime

Even after fixing the CreateOrUpdate issue there was another error.

TunnelStatus.cf-tunnel-operator.rajesh-kumar.in "mlops-mlflow" is invalid:
status.lastSyncTime: Required value

The generated CRD schema had lastSyncTime in the required list because the struct tag had no omitempty.

LastSyncTime metav1.Time `json:"lastSyncTime"`

When Status().Update() ran on a newly created resource where status had never been written, the API server received a zero time value for a required field and rejected it. Adding omitempty fixes it.

LastSyncTime metav1.Time `json:"lastSyncTime,omitempty"`

Regenerate the CRD and apply it.

controller-gen crd paths="./api/..." output:crd:artifacts:config=deploy/crd
kubectl apply -f deploy/crd/cf-tunnel-operator.rajesh-kumar.in_tunnelstatuses.yaml

omitempty tells the API server not to require this field when the value is a zero. At runtime the field always has a value because you always pass metav1.Now(), so it is never actually empty.

The Working Output

After all of it, the NeuVector route which uses HTTPS with a self-signed certificate and has TLS verification disabled.

kubectl -n cf-tunnel-operator-system get tunnelstatuses cattle-neuvector-system-neuvector-httproute -o yaml
status:
  backendService: https://neuvector-service-webui.cattle-neuvector-system.svc.cluster.local:8443
  hostname: nv.rajesh-kumar.in
  lastSyncTime: "2026-06-03T15:15:13Z"
  message: ""
  notlsverify: true
  scheme: https
  syncStatus: Success

and the MLFlow route:

status:
  backendService: http://mlflow.mlops.svc.cluster.local:5000
  hostname: mlflow.rajesh-kumar.in
  lastSyncTime: "2026-06-03T15:15:16Z"
  message: ""
  notlsverify: false
  scheme: http
  syncStatus: Success

This is what I wanted from the start. One command, all three routes, correct state for each one.

What surprised me was how much I enjoyed building it. Every piece had a reason. And all of it was designed by engineers who had to make these decisions work at a scale most of us will never operate at. Clusters with thousands of nodes. Hundreds of controllers running at the same time. APIs that millions of people depend on daily. They built guardrails into the foundation so that people like me can add a new resource type on a home lab cluster over a weekend and have it just work.

I find that genuinely impressive. The complexity is real but it is hidden in the right places. What we write is a Go struct and a few marker comments. What runs underneath is something much larger. This project made me appreciate that gap a little more than I did before.

If you found this useful, let us connect on LinkedIn. I write about infrastructure engineering, AI systems, and building things from scratch.