What Happens When You Run 'kubectl apply -f deployment.yaml'

Every time you run ‘kubectl apply -f deployment.yaml’, a surprisingly complex chain of events happens behind the scenes. What feels like a single command actually triggers a coordinated sequence across multiple independent components the API server, controllers, scheduler, and kubelet each performing one specific job and passing responsibility to the next.

STEP 1: kubectl → API Server

1. When you run ‘kubectl apply’, it serializes your YAML manifest and sends it to the Kubernetes API Server as an HTTP POST for creating new resources or PUT for updates request.
2. API Server authenticates you, verifying your identity using certificates, tokens or other configured identity providers. Next it authorizes you, checking against RBAC policies to confirm you have permission to perform the requested action on the specified resource and namespace.
3. API Server then passes the manifest through admission controllers, which can mutate or validate the request for example, injecting default values or enforcing organizational policies. It validates the schema, ensuring all required fields are present, values are of the correct type, and the resource’s API specification. After all these checks passed, API server saves the object to etcd, a distributed key-value store that serves as the cluster’s brain/single source of truth.

STEP 2: Deployment Controller Notices

The Deployment Controller is a reconciliation loop which running inside the controller manager that constantly watches for any new or changed deployment objects inside etcd via API server.

1. Observe: It constantly watches for any new or changed deployment objects inside etcd via API server.
2. Compare: When it detects your newly created deployment, it compares the desired state for example, replicas: 1 against the current state, which at this point is zero running pods.
3. Act: To bridge that gap, it takes action by creating a ReplicaSet object.

The ReplicaSet encodes the pod template and the desired replica count from the deployment.

Why does the Deployment Controller create a ReplicaSet instead of Pods directly?

So that Deployment Controller can manage rolling updates, rollbacks, rollout history and creating new rs and scaling the old rs down.

On the other hand, ReplicaSet is solely responsible for maintaining the correct number of pod replicas.

STEP 3: ReplicaSet Controller notices

The ReplicaSet controller is another reconciliation loop running inside the controller manager that continuously watches for new or changed ReplicaSet objects.

1. When it picks up the ReplicaSet just created by the deployment controller, it compares desired state versus actual state, this ReplicaSet wants 1 Pod, but 0 exist.
2. Its action is straightforward: it creates a Pod object by sending a request to the API Server, which stores it in etcd.
3. At this point the Pod is nothing more than data in etcd, no container is running anywhere in the cluster yet. The Pod object simply a set of specification of which container image to use, what resources it needs, which volumes to mount, and so on. It sits in a Pending state with no node assigned.

STEP 4: Scheduler picks a Node

The Scheduler (kube-scheduler) is yet another watch loop on the control plane. It constantly monitors the etcd for Pod objects that exists but have no node assigned by checking their spec.nodeName field is empty.

1. When it spots the newly created Pod object sitting in a Pending state, it kicks off a two-phase process: filtering and scoring.

2. During filtering, the Scheduler eliminates nodes that cannot run the Pod at all. It checks a series of constraints: -> Does the node have enough CPU available ? -> Does it have enough free memory ? -> Does the node have access volume ? -> Does it satisfy node selectors, affinities, taints and tolerations ?

3. During scoring, the Scheduler ranks the remaining nodes using priority functions. For example, nodes that already have the container image cached locally or nodes that the most balanced resource utilization across the cluster.

4. Once a winner node is chosen, the scheduler binds the Pod to the selected node by updating the Pod’s spec.nodeName field in etcd.

STEP 5: Kubelet on that Node takes over

The kubelet is an agent process running inside every worker node in the cluster. It continuously watches the API Server for Pods whose spec.nodeName matches its own node. Mainly it converts the declarative data in etcd into a running container.

1. imagePullSecrets: kubelet checks if the pod spec has any imagePullSecrets. If so, it fetches the secret objects from the API Server so that it can authenticate against private registries.
2. volumes: The kubelet prepares all volumes declared in the Pod spec. These mount points are then made available inside the containers at the paths specified in volumeMounts.
3. ConfigMaps and Secrets: The kubelet retrieves every ConfigMap and Secret referenced by the Pod’s containers.

STEP 6: Container Starts

Init Container Runs: If the Pod defines init containers, the kubelet processes them first, sequentially and in order. Each init container must run to completion and exit successfully before the next one starts. No application container will start until every init container has succeeded.

Main Container starts: After all init containers have succeeded, the kubelet moves on to launching the main application containers.

1. Image pull: The kubelet instructs the container runtime to pull the image specified in the container spec.

2. Create the container: The kubelet calls the container runtime to create the container with the Pod specification.

• Resource constraints are applied.
• Fetched env variables are injected into the container’s process.
• Volumes are mounted into the container’s filesystem at the mountPath.

3. Start the application: The container runtime executes the image’s entrypoint defined by ENTRYPOINT and CMD in the Dockerfile. For example, the entrypoint might run python main.py.

4. Container begins listening: The application initializes and binds to the container port. At this point the container is running.

STEP 7: Pod is Running

With the container is running, kubelet updates the pod status as running in etcd via API Server. The ReplicaSet controller sees healthy pod matching its selector, and the Deployment controller confirms that 1 replica requested, 1 replica running and desired state matches actual state.