Kubernetes. Runtime Security with Falco

In the previous two articles, we set up identity management for Kubernetes using OIDC and Keycloak, and network isolation between tenant namespaces using Cilium. Both layers are important, but they only guard the perimeter. RBAC controls what users can do through the Kubernetes API. Cilium controls which pods can reach which other pods over the network. Neither of them sees what happens inside a running container — which processes are spawning, which files are being read or written, which system calls are being made. This is the gap that Falco fills.

Falco is a CNCF-graduated runtime security project originally created by Sysdig. It hooks into the Linux kernel using eBPF and watches system calls in real time. When a running process matches one of Falco’s rules — a shell opened inside a container, a sensitive file read, a package manager launched at runtime — Falco fires an alert. It does not block (that is the job of admission controllers), but it observes and reports, which is exactly what you need to detect unexpected or malicious behaviour at runtime.

In this article, we will install Falco on a local Kind cluster, trigger a few built-in rules to see it in action, and write a custom rule of our own.

As in the previous articles in this series, I use Kind (Kubernetes-IN-Docker) to run the cluster locally. Unlike the Cilium setup, we do not need to disable the default CNI here — Falco operates at the syscall level and is independent of the network plugin. Pretty standard kind-configuration:

[embed]

#1: a plain Kind cluster with no extra configuration;

#5..6: one control-plane and one worker are enough for this demo.

[embed]Start kind Cluster

Falco is installed via its official Helm chart. The key decision is which kernel driver to use. Since version 0.40, Falco ships with a modern eBPF driver, that is compiled into the binary itself — no kernel headers, no module compilation, no extra setup on Kind nodes:

[embed]

#1: the official Falco chart repository;

#6: pass values file rather than inline — set flags, to keep things readable.

Falco configuration is humble:

[embed]

#1: keeps alert output readable in pod logs;

#3: explicitly selects the modern eBPF driver; works on Kind without any extra node configuration.

Let’s wait for the DaemonSet to be ready before moving on:

[embed]

That is the end of simple Falco installation.

Falco ships with a large set of built-in rules covering the most common attack patterns. The most recognisable one is *Terminal shell in container *— it fires whenever a shell process is opened interactively inside a running container.

To test it, let’s start a simple test pod and exec into it:

[embed]

#1: plain Alpine pod that just sleeps (nothing special about it);

#2: this exec is the action Falco will catch.

Before entering the last command, open a different Terminal section for Falco’s logs:

[embed]

and observe momentarily tailed new message:

[embed]

The alert includes the user, process name, pod name, and namespace -enough context to know exactly what happened and where.

Another useful built-in rule catches reads of sensitive system files. Even when a container should never touch ‘/etc/shadow’, a compromised or misconfigured process might try. Let’s emulate it with the command ‘cat /etc/shadow‘, entered directly into Pod’s Terminal session:

[embed]

Falco reacts immediately with a new log entry:

[embed]

In a short demo, we saw two alerts, zero configuration changes — Falco was watching the whole time. Still, Falco has a set of building blocks for custom alert construction.

Built-in rules cover well-known attack patterns, but real workloads need custom detection. A practical and concrete example could be: detecting any package manager launched inside a running container. Installing packages at runtime is a strong signal that something unexpected is happening — packages belong in the image build, not in a live container.

Falco rules are written in YAML and follow a consistent structure built from three primitives: *lists, [macros](https://falco.org/docs/concepts/rules/basic-elements/#macros), and [rules](https://falco.org/docs/concepts/rules/basic-elements/#rules)*. Leveraging all three to compose own custom alert:

[embed]

#1..2: named list of binaries we consider package managers;

#4..5: a reusable macro that matches any spawned process whose name is in list;

#7..17: the rule itself, which is fired when a new process spawns inside any container and matches the macro. The output template uses Falco field variables to capture useful context automatically.

To load custom rules via the Helm chart, add them under the ‘customRules’ key in the values file. Helm will mount them as an additional rules file inside the Falco pod:

[embed]Updated version of Falco’s deployment time values

#5: the key is the filename Falco will use when mounting the rules andthe value is the YAML content inline.

Now refresh Falco via Helm with the upgraded configuration:

[embed]

and trigger the rule with the artificial package install command in Pod ‘kubectl exec test-pod — -apk add curl’ and immediately find in Falco’s logs the trace:

[embed]

Similar approach could be used to any workload-specific behaviour you want to detect — unexpected outbound connections, writes to config directories, or privilege escalation attempts. Refer to the official documentation for more examples.

By default Falco writes alerts to its own pod logs, which is fine for local exploration. In a real Cluster, they should be routed somewhere actionable — Slack, PagerDuty, a SIEM, or a custom webhook. That is the job of Falco Sidekick, a companion service that receives Falco events over HTTP and forwards them to more than 60 integrations.

Enabling it is a single extra flag on the Helm install:

[embed]

#4: deploys Falco Sidekick as a sidecar;

#5: enables the Sidekick web UI for a visual real-time feed of alerts.

With Falco in place, the Kubernetes multitenancy picture built across this series is now complete:

• *OIDC with Keycloak* - controls identity: who can authenticate and what roles they carry.
• *Cilium network policies* - controls connectivity: which pods can reach each other.
• [Falco](http://this article) - controls visibility: what processes are doing inside containers at runtime.

None of these layers replaces the others. A compromise that bypasses one is still caught by the next. Together, they give defence-in-depth with practical tooling that can be run locally on Kind and promoted to production with the same Helm values.

The Kind config, Helm values, and custom rules from this article are available in my GitHub repository.