🌱 Securely Bootstrapping Secrets in a ClusterAPI Cluster with ExternalSecrets & PushSecret

When I started automating Kubernetes cluster creation using ClusterAPI, one problem kept coming back:

How can I securely inject secrets into a brand new cluster right at bootstrap — without storing them in Git or relying on fragile post-install hacks?

After a lot of searching, I found a hidden gem in External Secrets Operator (ESO): the powerful but lesser-known PushSecret feature.

🌍 The Problem: Secure Secret Bootstrap

In a typical ClusterAPI setup, you have a management cluster that provisions workload clusters. That works great, but raises a tricky issue:

How can you get critical secrets (certs, tokens, credentials) into a cluster at creation time — securely, and without manual steps?

Most solutions I found fell short:

• Putting secrets in Git (security nightmare),
• Writing brittle post-creation scripts,
• Waiting for the cluster to be ready before installing ESO/Vault (too late for early-stage secrets).

🔑 The Solution: ExternalSecrets + PushSecret

If you’ve used External Secrets Operator before, you likely know it for syncing secrets from Vault, AWS Secrets Manager, GCP Secret Manager, etc. to Kubernetes.

But what many people don’t know is that ESO has a very useful feature called PushSecret.

🧪 What Is PushSecret?

PushSecret lets you sync a Kubernetes secret to another cluster — without installing ESO on the target cluster.

That means you can:

• Define a secret source (Vault, AWS, or even a local Secret)
• Push it from the management cluster to a remote child cluster
• Do this without Git, without ESO installed on the target, and without manual work

⚙️ How Does It Work?

Here’s the basic idea:

1. Install ESO only on the management cluster.
2. Define a ClusterSecretStore or SecretStore (with kubeconfig) to point to the child cluster.
3. Create a PushSecret that selects a local secret and sends it to the target cluster.
4. ESO does the sync as soon as the child cluster API is available.

Example: PushSecret

Define a ClusterSecretStore:

apiVersion: external-secrets.io/v1
kind: ClusterSecretStore
metadata:
  name: mycapicluster-secretstore
  namespace: default
spec:
  provider:
    kubernetes:
      remoteNamespace: default
      authRef:
        name: mycapicluster-kubeconfig
        key: value
        namespace: default

💡 These kubeconfigs are automatically generated by ClusterAPI or its bootstrap provider (CAPBK).

Create a PushSecret:

apiVersion: external-secrets.io/v1alpha1
kind: PushSecret
metadata:
  name: pushsecret-mysecret
spec:
  refreshInterval: 60s
  selector:
    secret:
      name: mysecret
  secretStoreRefs:
  - kind: ClusterSecretStore
    name: mycapicluster-secretstore
  data:
    - match:
        remoteRef:
          remoteKey: mysecret

🎯 Real-World Use Cases

• 🔐 Injecting CSI driver credentials (AWS EBS, AzureDisk, Vault CSI)
• 🔑 Sharing a global TLS certificate across clusters
• 📦 Distributing container registry tokens (e.g., GitHub Container Registry)
• ⚙️ Bootstrapping cert-manager or external-dns with initial secrets

❤️ Why ClusterAPI + ExternalSecrets Works So Well

ClusterAPI + ExternalSecrets is a powerful combo because:

• The management cluster generates kubeconfig and CA for new clusters automatically
• Cluster names are predictable (based on the cluster object)
• You can predefine PushSecrets pointing to future clusters → ESO pushes secrets as soon as the API is live — no manual steps

🔍 Concrete Example

Provisioning a cluster named devcluster creates a secret named devcluster-kubeconfig in the namespace. You can reference it like this:

apiVersion: external-secrets.io/v1
kind: ClusterSecretStore
metadata:
  name: devcluster-secretstore
  namespace: kube-system
spec:
  provider:
    kubernetes:
      remoteNamespace: kube-system
      authRef:
        name: devcluster-kubeconfig
        key: value
        namespace: devcluster
---
apiVersion: external-secrets.io/v1alpha1
kind: PushSecret
metadata:
  name: registry-token-push
  namespace: kube-system
spec:
  refreshInterval: 60s
  selector:
    secret:
      name: registry-token
      namespace: kube-system
  secretStoreRefs:
  - kind: ClusterSecretStore
    name: devcluster-secretstore
  data:
    - match:
        remoteRef:
          remoteKey: registry-token

Check status:

$ kubectl get pushsecrets -n  kube-system
NAME                    STATUS      AGE
registry-token-push     Synchronized   2m

✅ Benefits

• 🔐 Secure: No secrets in Git
• ⚙️ Automated: Push when API is ready
• 🧼 Minimal: ESO only on management cluster
• 🌐 Flexible: Works with Vault, AWS, GCP, Azure, and native secrets

🧵 Summary

If you use ClusterAPI and want a secure, automated way to inject secrets from day one — try PushSecret in ExternalSecrets Operator.

It solved a long-standing pain point for me, and it might save you hours of scripting too.

📘 Docs:

• External Secrets PushSecret
• ClusterAPI
• Devops-notes.com