Kubernetes becomes less strange when you stop treating it as a big remote server. It is not one machine with a nicer SSH prompt. It is a system that accepts descriptions of how workloads should look, then keeps working to make reality match those descriptions.

That sentence sounds simple. It took me a while to feel it.

At the beginning, Kubernetes can look like a pile of nouns: Pod, Deployment, Service, Ingress, ConfigMap, Secret, Node, Namespace. The temptation is to memorise definitions and hope the picture appears later. Definitions help, but they are not enough. A better way in is to build a few mental models that make the definitions connect.

This post is for that stage: some technical background, maybe Docker basics, maybe a first cluster running in kind, minikube, or a cloud provider, but not yet the instinct for what Kubernetes is doing under the hood.

Model 1: desired state, not remote control

On a traditional server, you often perform actions:

  • Install a package.
  • Start a process.
  • Edit a config file.
  • Restart a service.

Kubernetes asks for a different habit. You describe desired state:

  • I want three replicas of this application.
  • I want this container image.
  • I want this port exposed inside the cluster.
  • I want this configuration mounted into the Pod.

Then Kubernetes tries to converge the actual state toward that desired state.

That word, converge, matters. Kubernetes is not a magic instant executor. It is a reconciliation system. You submit an object to the API server. Controllers notice it. The scheduler places Pods. Kubelets on nodes start containers. Status gets reported back. If something breaks, controllers usually try again.

A useful beginner question is:

What did I ask Kubernetes for, and what does Kubernetes currently report back?

Those are not always the same.

kubectl get deployment <name> -n <namespace>
kubectl describe deployment <name> -n <namespace>
kubectl get pods -n <namespace> -l app=<label>

kubectl get gives the short cockpit view. kubectl describe gives the story: events, selectors, replica counts, rollout status, and often the first clue when something is stuck.

Common misconception: “I applied the YAML, so it is running.” Applying a manifest means the API accepted your desired state. It does not mean the image pulled, the Pod scheduled, the app started, or the readiness probe passed.

Model 2: Kubernetes is a control loop machine

Many Kubernetes components behave like control loops:

  1. Observe the current state.
  2. Compare it with desired state.
  3. Take action to reduce the difference.
  4. Repeat.

That pattern explains a lot.

A Deployment controller notices that a Deployment wants three replicas. It creates or updates a ReplicaSet. The ReplicaSet controller notices that three Pods should exist. If one Pod disappears, it creates another. The scheduler notices unscheduled Pods and chooses nodes. The kubelet notices Pods assigned to its node and tries to run their containers.

This also explains why manual changes often disappear.

If you delete a Pod owned by a ReplicaSet, Kubernetes usually creates a replacement. That is not stubbornness. That is the control loop doing exactly what you asked at the higher level.

kubectl get pod <pod-name> -n <namespace> -o yaml | grep -A5 ownerReferences
kubectl get replicaset -n <namespace>
kubectl get deployment -n <namespace>

When a beginner says “Kubernetes keeps recreating my Pod”, the next question is: who owns it? The answer is usually in ownerReferences, and the fix is usually at the owner level, not by fighting the Pod.

Common misconception: “A Pod is my application.” A Pod may contain your application process, but in production you usually manage the Deployment that manages the ReplicaSet that manages the Pods. The Pod is often the replaceable unit, not the source of truth.

Model 3: Pods are cattle, but not in a careless way

The old “pets versus cattle” phrase is overused, but the basic idea is useful. A Pod is meant to be replaceable. Kubernetes can restart it, reschedule it, or create a new one with a different name.

That does not mean Pods are disposable in the sense that evidence does not matter. Logs, events, exit codes, and probe failures are valuable. But your application design should not depend on one specific Pod name living forever.

This affects how you think about storage, configuration, and debugging.

If a Pod writes important data only to its container filesystem, that data can disappear with the Pod. If another service calls a Pod by its direct IP, that IP can change. If you manually patch a running container, the change will vanish on the next restart.

The practical beginner habit:

  • Put repeatable configuration in manifests, Helm values, or GitOps.
  • Use Services for stable network access.
  • Use persistent volumes for data that must survive Pod replacement.
  • Treat manual Pod changes as temporary debugging, not real fixes.

Useful checks:

kubectl get pods -n <namespace> -o wide
kubectl describe pod <pod-name> -n <namespace>
kubectl logs <pod-name> -n <namespace>
kubectl logs <pod-name> -n <namespace> --previous

--previous is easy to forget. It shows logs from the previous container instance in a Pod after a restart, which is often exactly where the crash clue lives.

Model 4: labels are the wiring

Labels look like metadata until something breaks. Then you realise they are the wiring.

A Deployment uses selectors to know which Pods belong to it. A Service uses selectors to know which Pods should receive traffic. Many kubectl commands use label selectors to filter a noisy cluster into one readable slice.

If labels are wrong, Kubernetes may be healthy while your app is unreachable.

Imagine a Service selecting app: web, but the Pods are labelled app: frontend. The Service exists. The Pods exist. DNS might even resolve. But the Service has no endpoints, so traffic goes nowhere useful.

Check that path directly:

kubectl get service <service-name> -n <namespace> -o wide
kubectl get endpoints <service-name> -n <namespace>
kubectl get pods -n <namespace> --show-labels
kubectl get pods -n <namespace> -l app=<label>

Common misconception: “The Service points to my Deployment.” Usually it does not. A Service points to Pods by label selector. The Deployment also creates Pods with labels. The relationship is indirect, which is powerful but easy to misread.

This is one reason I like drawing a small box diagram for a new workload:

  • Deployment creates Pods with labels.
  • Service selects Pods with matching labels.
  • Ingress or Gateway routes external traffic to the Service.
  • ConfigMaps and Secrets feed configuration into the Pods.

If the drawing cannot explain the labels, the YAML probably deserves another look.

Model 5: namespaces are rooms, not clusters

Namespaces divide one cluster into named spaces. They are useful for separating teams, environments, or system components. They are not full security boundaries by themselves, and they are not separate clusters.

For beginners, the most important namespace lesson is boring and practical: many mistakes are just commands run in the wrong namespace.

kubectl config current-context
kubectl get namespaces
kubectl get pods -n <namespace>
kubectl config set-context --current --namespace=<namespace>

I prefer being explicit with -n <namespace> when learning or debugging. Changing the default namespace can be convenient, but it can also hide where commands are really going.

Common misconception: “It worked in one namespace, so the cluster is fine.” Namespaces can have different resource quotas, network policies, secrets, service accounts, and image pull permissions. Same YAML, different namespace, different result.

Model 6: scheduling is a constraint problem

When a Pod stays Pending, Kubernetes is often not “slow”. It may have no valid place to put the Pod.

The scheduler must consider CPU and memory requests, node taints, tolerations, node selectors, affinity rules, volume constraints, and sometimes cluster autoscaler behaviour. Beginners do not need every detail on day one, but they should know that scheduling is a decision with constraints.

Start with events:

kubectl describe pod <pod-name> -n <namespace>
kubectl get events -n <namespace> --sort-by='.lastTimestamp'
kubectl get nodes -o wide
kubectl describe node <node-name>

If the event says Insufficient cpu, the useful question is not “why does Kubernetes hate my Pod?” It is “what resource request did I make, and does any node have room for it?”

Requests are not decoration. They tell Kubernetes what the workload needs for scheduling. Limits are different: they cap usage at runtime. Mixing those up causes many confusing beginner problems.

Model 7: health checks are part of the contract

A container process can be running while the application is not useful. Maybe it has not loaded configuration yet. Maybe it cannot reach the database. Maybe it is alive but wedged.

Kubernetes gives you probes to describe this more honestly:

  • A startup probe gives slow-starting apps time to boot.
  • A readiness probe says whether the Pod should receive traffic.
  • A liveness probe says whether Kubernetes should restart the container.

The common beginner mistake is making liveness too aggressive. If liveness fails during a slow dependency or temporary overload, Kubernetes may restart a process that would have recovered. Readiness is often the safer first tool: stop sending traffic until the app is ready again.

Check probe failures in the Pod description:

kubectl describe pod <pod-name> -n <namespace>
kubectl get pod <pod-name> -n <namespace> -o jsonpath='{.status.containerStatuses[*].restartCount}'
kubectl logs <pod-name> -n <namespace> --previous

A small debugging sequence

When something does not work, I try to avoid jumping straight into random fixes. A calm first pass looks like this:

kubectl config current-context
kubectl get pods -n <namespace> -o wide
kubectl get deploy,rs,svc -n <namespace>
kubectl describe pod <pod-name> -n <namespace>
kubectl get events -n <namespace> --sort-by='.lastTimestamp'
kubectl logs <pod-name> -n <namespace>

That sequence asks:

  • Am I looking at the right cluster?
  • Am I looking at the right namespace?
  • Are the expected resources present?
  • Is Kubernetes reporting scheduling, image, probe, or permission problems?
  • Does the application log agree with the Kubernetes view?

It is not advanced. That is the point. Most early Kubernetes confusion improves when the first five minutes are structured.

What to memorise, and what not to

Do memorise a few basic commands. Muscle memory helps when the cluster is noisy.

Do not try to memorise every field in every manifest. Use the tools.

kubectl explain deployment.spec
kubectl explain pod.spec.containers
kubectl explain service.spec

kubectl explain is not glamorous, but it keeps you close to the API. That matters because Kubernetes is not mainly a command-line tool. It is an API with controllers around it. kubectl is one client.

The deeper beginner skill is learning to ask better questions:

  • Which object is the source of truth here?
  • Which controller owns the object I am looking at?
  • What labels connect these resources?
  • What does desired state say?
  • What does current status say?
  • What event was recorded most recently?

Those questions travel well. They help in minikube, in a managed cloud cluster, and in OpenShift with extra platform rules around the same core concepts.

Final thought

Kubernetes is complex, but it is not random. It has a strong internal logic: desired state, reconciliation, replaceable Pods, label-based wiring, constrained scheduling, and explicit health signals.

Beginners often feel they are failing because they cannot hold all the nouns in their head. I think the better goal is smaller: understand the control loops, follow ownership, read events, and check labels. Once those mental models settle, the nouns become less like vocabulary cards and more like parts of one operating system for distributed applications.