Kubernetes Troubleshooting Hub: Common Errors, Causes, and Fixes

Kubernetes troubleshooting starts with reading the right signal. A pod stuck in CrashLoopBackOff tells a different story than a node showing NotReady. Each error carries a specific exit code, a specific cause class, and a specific first command. This hub organizes the most common Kubernetes errors by category so you can move from symptom to resolution without guessing. Use the symptom table below as your entry point, then follow the guide links for full diagnostic workflows.

Key Takeaways

• Start with the right signal. Pod status, exit codes, and kubectl events together tell a complete story. Each error type has a specific first command – jumping to fixes without reading the signal wastes time.
• OOMKilled (exit 137) means the container exceeded its memory limit. The fix starts with memory requests and limits, not the application.
• CrashLoopBackOff is a restart loop, not a single error. Use kubectl logs --previous to read the log from the most recent crashed container before Kubernetes replaces it.
• ImagePullBackOff means the kubelet cannot pull the container image. Check the image name and tag first, then registry authentication, then network – in that order.
• Node NotReady means the kubelet stopped reporting. Check kubectl describe node for conditions and kubelet logs for the root cause.
• Exit codes map to causes: 0 = clean stop, 1 = app error, 137 = OOM kill (SIGKILL), 143 = graceful termination (SIGTERM).

How to Use This Hub

Each section covers one error category: the symptom, the likely cause, and the first kubectl command to run. For brevity, each section links out to a deep-dive guide rather than reproducing the full diagnostic loop here. Start with the table below to find your symptom fastest. Then read the section that applies and follow the guide link for step-by-step resolution.

General-purpose debug toolkit: These commands give you immediate situational awareness in any namespace before you drill into a specific error section:

# Sort all recent events in a namespace by timestamp
kubectl get events -n <namespace> --sort-by='.lastTimestamp'

# Check current resource consumption
kubectl top pod -n <namespace>
kubectl top node

Note: kubectl top requires metrics-server. Verify it is running with: kubectl get deployment metrics-server -n kube-system. If not installed, see the metrics-server GitHub release page for installation instructions.

Memory and CPU Errors: Kubernetes Troubleshooting for OOM Failures

OOMKilled (Exit Code 137)

OOMKilled means the Linux kernel sent SIGKILL (signal 9) to a container that exceeded its memory limit. Exit code 137 confirms this. First, run the describe command and look for the specific string in Last State:

kubectl describe pod <pod-name> -n <namespace>

Look for: Last State: Terminated, Reason: OOMKilled. Exit code 137 appears for all SIGKILL events, not only OOM kills. Check the Reason: OOMKilled field explicitly to confirm the Linux OOM killer was responsible rather than an external kill signal.

Causes: Memory limit set below the application’s peak working set. Additionally, a memory leak grows usage until it hits the ceiling. Node-level MemoryPressure from other pods can also trigger kills before a container hits its own limit. Finally, BestEffort pods (no requests or limits defined) have no protection at all.

QoS and eviction order: Kubernetes evicts pods by Quality of Service class. BestEffort pods (no requests or limits) die first. Burstable pods die second. Guaranteed pods (requests equal limits) die last. Therefore, setting explicit requests and limits moves your pods out of the most vulnerable QoS class.

Fix direction: Set the memory limit at the p95 working set plus a headroom buffer. Run VPA in Off mode first to collect data-driven recommendations before applying any changes. For the full diagnostic loop and sizing approach, see the OOMKilled deep-dive.

Production context: The Cast AI 2026 State of Kubernetes Optimization Report (from direct measurement across tens of thousands of production clusters, not survey estimates) found 79% memory overprovisioning across production clusters. Yet OOM kills remain common. More padding does not equal fewer crashes. Correct sizing from real working set data solves both overprovisioning and instability simultaneously.

Pod Lifecycle Errors

CrashLoopBackOff

CrashLoopBackOff means a container crashes repeatedly and kubelet restarts it with exponential backoff: 10 seconds initially, doubling up to a 300-second cap. The RESTARTS counter in kubectl get pods climbs with each cycle. However, the status string itself does not explain why the container crashes. The logs do.

Causes: Application error (exit code 1 or 2). OOMKilled (exit code 137) triggering the loop. Failing liveness probe causing kubelet to restart a healthy container. Wrong image or entrypoint. Missing ConfigMap or Secret the application requires. Init container failure blocking the main container from starting.

First command:

kubectl logs <pod> -n <namespace> --previous

The --previous flag reads from the last terminated container, not the currently running (or crashing) one. This is the most common mistake in CrashLoopBackOff diagnosis: running kubectl logs without --previous returns empty or irrelevant output.

Note: The --previous flag only works if the container’s previous instance is still on the same node. If the pod was rescheduled to a different node, the previous logs are not available. In that case, check events instead:

kubectl get events -n <namespace> --sort-by='.lastTimestamp'

Kubernetes events expire after approximately one hour by default. If the pod restarted more than an hour ago, check your log aggregator (Loki, CloudWatch, Stackdriver) instead.

For the full 4-command diagnostic sequence, see the CrashLoopBackOff guide.

Pod Exit Codes

Exit codes appear in kubectl describe pod under the Last State field. Each code maps to a specific failure class. Reading the exit code before investigating saves significant time because it immediately narrows the cause category.

Exit code 143 signals any graceful termination: rolling updates, scale-down events, kubectl delete pod, node drain, and eviction all produce 143. It is not inherently a problem. Exit code 127 points to a wrong image or a misconfigured entrypoint. Read the exit code first and let it direct your investigation. For the complete exit code reference and diagnostic approach, see the exit codes guide.

ImagePullBackOff

ImagePullBackOff (or ErrImagePull on the first attempt) means kubelet cannot pull the container image. The pod never enters Running state. However, the fix is straightforward: the Events section of kubectl describe pod contains the exact error string, and that string maps directly to the root cause.

Causes: Wrong image name or tag (including tags that were deleted or never pushed). Missing imagePullSecret for a private registry. Expired or rotated registry credentials. Docker Hub rate limit on anonymous pulls. Unreachable private registry due to network or DNS failure.

First command:

kubectl describe pod <pod> -n <namespace>

Scroll to the Events section. The error string is specific: “unauthorized” means credentials, “not found” means wrong name or tag, “network timeout” means registry connectivity. Match the string to the cause and apply the fix directly. For the complete workflow, see the ImagePullBackOff guide.

Node Errors

Node NotReady

A node in NotReady (or Unknown) status means the control plane lost contact with the kubelet on that node. The kubelet renews its lease with the API server every 10 seconds. After 40 seconds without a renewal, the node transitions to Unknown. After 300 seconds (5 minutes), Kubernetes evicts all pods from that node, generating exit code 143 across every workload running there.

Causes: Kubelet process crashed or stopped responding. MemoryPressure from pods consuming all node RAM. DiskPressure from log or image accumulation filling the disk. CNI misconfiguration breaking the network plugin. Underlying cloud instance failure or preemption.

Diagnosis-first sequence: Diagnose before you remediate. Cordoning before you understand the cause prevents scheduling but does not fix anything.

# Step 1: Diagnose: read Conditions for MemoryPressure, DiskPressure, NotReady reasons
kubectl describe node <node>

# Step 2: Check kubelet logs directly on the node (if you have SSH access)
journalctl -u kubelet --lines=200

# Step 3: After understanding root cause, cordon to stop new pods scheduling here
kubectl cordon <node>

# Step 4: If the node is unrecoverable, drain it before replacing
kubectl drain <node> --ignore-daemonsets --delete-emptydir-data
# --ignore-daemonsets and --delete-emptydir-data are required on most clusters

Read the Conditions section in kubectl describe node first. MemoryPressure, DiskPressure, and PIDPressure flags each point to a different root cause. On the node itself, journalctl -u kubelet --lines=200 surfaces kubelet-level failures that do not appear in Kubernetes events. On managed clusters (GKE, EKS, AKS) without node SSH access, check node problem logs via your cloud provider: Cloud Logging on GKE, CloudWatch Logs on EKS, or Azure Monitor on AKS. Only after understanding the root cause should you cordon, and proceed to drain and replace if the node is unrecoverable. For the full recovery procedure and cordon-drain-replace workflow, see the Node NotReady guide.

The Common Root Cause: Resource Misconfiguration

Most Kubernetes errors share a single root cause: resources are not sized correctly. The Cast AI 2026 State of Kubernetes Optimization Report (from direct measurement across tens of thousands of production clusters, not survey estimates) found average CPU utilization at just 8% across production clusters. Additionally, 69% of requested CPU goes unused, up from 40% year over year. Memory overprovisioning sits at 79%.

These are not just efficiency numbers. They are stability numbers.

Here is how the failure chain works in practice:

• Pods without memory limits fill a node until MemoryPressure triggers
• MemoryPressure causes node-level OOM kills, with BestEffort pods dying first
• OOM kills produce exit code 137, which drives CrashLoopBackOff
• Sustained MemoryPressure eventually takes the node to NotReady
• NotReady triggers evictions across all pods on that node, producing exit code 143 cluster-wide

Adding headroom does not break this chain. One production cluster with generous padding averaged 40 to 50 OOM kills per monitoring interval (typically measured per hour). After automated rightsizing using p95 working set data, OOM kills dropped to near zero. The fix was not more memory. The fix was correct memory.

Correct requests and limits based on actual working set data is the fix at the root. Manual sizing from guesswork or copy-pasted examples creates the exact conditions that drive every error in this hub.

Teams can approach rightsizing manually using Kubernetes VPA in Off mode to gather recommendations, or through custom Prometheus alerts on working set vs. limit ratios. For teams managing dozens or hundreds of workloads, automated rightsizing with the Cast AI Workload Autoscaler handles this continuously at scale, applying p95 data across your entire fleet without anyone adjusting YAML by hand. The cluster with 40 to 50 OOM kills per hour reached near zero after automated rightsizing took over.

Conclusion

Kubernetes troubleshooting follows repeatable patterns. OOMKilled, CrashLoopBackOff, ImagePullBackOff, and node NotReady each have a specific first command, a specific cause category, and a specific fix path. The symptom table at the top of this hub maps every error to its starting point.

For deeper diagnosis on any error, follow the guide links in each section above. For the root cause driving most of these errors at scale, start with resource misconfiguration and work from real p95 data. Manual guesswork on limits is where the chain starts. Correct sizing is where it ends.

Frequently Asked Questions