By the end you'll know
- What a pod is, and why Kubernetes doesn't just run containers directly
- How to create a pod from a file, look inside it, and read its logs
- What the two most common pod failures look like, and how to diagnose them
- Why you almost never create pods by hand
What a pod is
In Lesson 2 you saw pods everywhere, even the control plane runs as pods. A pod is the smallest thing Kubernetes runs. It's a wrapper around one or more containers that:
- share one network address. The pod gets a single IP address, and containers inside it reach each other on
localhost. That also means they share ports: two containers in one pod can't both listen on port 80. - can share storage. A folder (called a volume) can be mounted into more than one container in the pod.
- always run together, on the same node. The scheduler places the whole pod, never half of it.
When would you put two containers in one pod? Only when they truly can't live apart, like a helper that reads the main app's log files. If two things could run on different machines, they belong in different pods.
Try it: your first pod
You need a cluster again. Either one from Lesson 2 works; the single-node version is plenty:
kind create cluster --name learn
Save this as pod.yaml:
apiVersion: v1 # which version of the Kubernetes API this uses
kind: Pod # what you're creating
metadata:
name: hello # the pod's name, used in kubectl commands
labels:
app: hello # a tag; other objects use labels to find pods (Lesson 8)
spec: # what should be inside
containers:
- name: web # the container's name inside the pod
image: nginx:1.27 # the image to download and run
ports:
- containerPort: 80 # the port the app listens on
You'll read files like this line by line in Lesson 4.
Send it to the cluster and check on it:
kubectl apply -f pod.yaml
kubectl get pods
NAME READY STATUS RESTARTS AGE
hello 1/1 Running 0 15s
READY 1/1 means one of one containers is ready. If you're quick you may catch it in ContainerCreating while the image downloads.
Look inside
Where is it, and what's its address?
kubectl get pod hello -o wide
The IP column is the pod's own address inside the cluster, and NODE is where the scheduler put it.
What happened, step by step?
kubectl describe pod hello
Scroll to the bottom, to Events:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 40s default-scheduler Successfully assigned default/hello to learn-control-plane
Normal Pulling 39s kubelet Pulling image "nginx:1.27"
Normal Pulled 33s kubelet Successfully pulled image "nginx:1.27"
Normal Created 33s kubelet Created container: web
Normal Started 33s kubelet Started container web
That's Lesson 2 happening in front of you: the scheduler picked a node, then that node's kubelet pulled the image and started the container. When a pod misbehaves, describe and its events are the first place to look.
Is it actually working? Forward a port from your laptop to the pod:
kubectl port-forward pod/hello 8080:80
Open http://localhost:8080 in your browser and you'll see "Welcome to nginx!". Press Ctrl+C to stop forwarding.
What has it been doing?
kubectl logs hello
You'll see a line for the page you just loaded. Logs are whatever the container prints, which is why apps in containers write their logs to the screen instead of to files.
Run a command inside it:
kubectl exec hello -- nginx -v
Break it on purpose
Pods fail in a handful of recognizable ways. Two of them cover most of what you'll see.
1. A bad image name. Start a pod whose image doesn't exist:
kubectl run typo --image=nginx:not-a-real-tag
kubectl get pods
NAME READY STATUS RESTARTS AGE
typo 0/1 ImagePullBackOff 0 30s
ErrImagePull, then ImagePullBackOff: the kubelet can't download the image, so it keeps retrying, waiting longer each time. kubectl describe pod typo tells you exactly which image failed and why. The fix is almost always a typo in the name or tag, or a private image the cluster isn't allowed to pull.
2. An app that keeps crashing. Save this as crashy.yaml. The container prints two lines and exits with an error:
apiVersion: v1
kind: Pod
metadata:
name: crashy
spec:
containers:
- name: app
image: busybox:1.36
command: ["sh", "-c", "echo starting up; sleep 3; echo something went wrong; exit 1"]
kubectl apply -f crashy.yaml
kubectl get pods --watch
Watch for a minute (Ctrl+C to stop). The status flips between Error and CrashLoopBackOff, and RESTARTS keeps climbing. The kubelet restarts a crashed container automatically, waiting longer between tries, up to five minutes. To see why it died, read the logs from the attempt that crashed:
kubectl logs crashy --previous
CrashLoopBackOff is not the problem itself. It means "this keeps dying, and I'm pausing before the next try". The real reason is in the logs, or under Last State in kubectl describe.
A third failure is worth knowing by sight. When a container uses more memory than it's allowed, kubectl describe pod shows this under the container's Last State:
Last State: Terminated
Reason: OOMKilled
Exit Code: 137
OOMKilled means "out of memory, killed": the container went over its memory limit (Lesson 9) and was stopped. The kubelet restarts it, it runs out of memory again, and you're in CrashLoopBackOff.
In my real cluster
This is exactly what happened to my Prometheus monitoring in October. It kept running out of memory and sat in CrashLoopBackOff for 9 days, with 2,012 restarts. A restart count in the thousands is one of the loudest signals Kubernetes gives you, if something is watching for it.
Why you rarely create pods yourself
Delete the first pod:
kubectl delete pod hello
kubectl get pods
It's gone, and nothing brings it back. In Lesson 2, when you deleted a pod, a new one appeared. The difference: that pod belonged to a Deployment, whose controller kept "two copies" true. This pod was on its own.
A bare pod gets its crashed containers restarted by the kubelet, but if the pod itself is deleted, or its node dies, it's gone for good. That's why real apps are almost always run through a Deployment, which creates and replaces pods for you. That's Lesson 5, which also covers the other ways to run pods, such as one copy on every node, or a task on a schedule.
Clean up
kind delete cluster --name learn
Check yourself
Two containers in the same pod both try to listen on port 8080. What happens?
A new pod shows ImagePullBackOff. What should you do first?
A pod is in CrashLoopBackOff. Which command shows why the last attempt died?
A container shows Reason: OOMKilled and Exit Code: 137. What happened?
You delete a pod you created from pod.yaml. Does it come back?
When should two containers share a pod?
Key terms
- Pod: one or more containers that share a network address and storage, and always run together on one node.
- Sidecar: a helper container in the same pod as the main app, such as a log shipper.
- Events: the step-by-step history at the bottom of
kubectl describe. Your first stop when something's wrong. - ImagePullBackOff: the node can't download the container image and keeps retrying.
- CrashLoopBackOff: the container keeps crashing, and Kubernetes is waiting before restarting it again.
- OOMKilled: the container used more memory than its limit allows and was stopped.
Go deeper
- Pods (opens in a new tab) in the official docs.
- Pod lifecycle (opens in a new tab) explains every status and restart rule.