
Skill
pod-failure-diagnosis
diagnose OpenShift pod failure states
Description
Troubleshoot CrashLoopBackOff, ImagePullBackOff, Pending, Error, or OOMKilled status. Use when a workload keeps restarting, fails to start, or is crash-looping.
SKILL.md
Pod Failure Diagnosis
When a user reports a pod that is not running, follow this structured triage to identify root cause and provide remediation.
1. Classify the Failure Mode
Get the pod status and recent events to determine which failure category applies:
- CrashLoopBackOff — container starts but exits repeatedly
- ImagePullBackOff — container image cannot be pulled
- Pending — pod is not scheduled to any node
- Error / Init:Error — init container or main container failed on startup
- Terminating (stuck) — pod is not cleaning up
Ask the user for namespace and pod/deployment name if not provided. If they describe symptoms instead ("my app keeps restarting"), map to the correct category before proceeding.
2. CrashLoopBackOff Triage
If the pod is crash-looping:
- Retrieve container logs for the current attempt to see the latest error.
- Retrieve container logs for the previous attempt (
--previous) — the crash reason is often clearer there. - Check pod events for OOMKilled signals — this means the container exceeded its memory limit.
- If OOMKilled: compare the container's memory limit against actual usage and recommend increasing it or fixing the memory leak.
- If application error: cite the specific log line and recommend the fix (config error, missing dependency, failed health check, etc.).
Do not suggest "just increase resources" unless OOMKilled is confirmed.
3. ImagePullBackOff Triage
If the image cannot be pulled:
- Check the exact image reference in the pod spec — look for typos, wrong tags, or missing registry prefix.
- Check pod events for the specific pull error message (authentication required, not found, timeout).
- For authentication errors: verify the imagePullSecrets on the pod and the referenced Secret's content.
- For "not found" errors: verify the image exists in the registry with the specified tag.
- For registry connectivity: check if the node can reach the registry (relevant for air-gapped or proxy environments).
Report the exact error message from events — do not guess which sub-case applies.
4. Pending Pod Triage
If the pod is stuck in Pending:
- Check pod events for scheduling failure reasons.
- Insufficient resources: compare the pod's resource requests against available node capacity. Show the arithmetic.
- Taints/tolerations: identify which nodes have taints the pod does not tolerate.
- Node selectors / affinity: verify the pod's constraints match at least one available node.
- PVC binding: if the pod mounts a PersistentVolumeClaim, check if the PVC is Bound. If Pending, diagnose the PVC (missing StorageClass, insufficient capacity, zone mismatch).
For each sub-case, provide the specific fix — do not list all possibilities when the events already tell you which one applies.
5. Provide Remediation
Once root cause is identified:
- State the root cause in one sentence with the supporting evidence (event message, log line, or metric).
- Provide one or two actionable commands or manifest changes the user can apply.
- If the fix involves deleting or force-replacing a resource, warn explicitly before suggesting it.
- If the issue is an application bug (not a platform issue), say so clearly and recommend the user check their application code/config.
Quality Standards
- Always cite the specific event message or log line that reveals the cause — do not provide generic troubleshooting checklists.
- Distinguish between application errors and platform/infrastructure issues. The remediation path is different.
- If multiple pods are failing, prioritize the earliest failure — cascading failures often share a root cause.
- Do not suggest destructive actions (delete pod, force delete, remove finalizers) without explicit warning about consequences.
- If evidence is insufficient to determine root cause, ask one focused clarification question rather than speculating.
More skills from the lightspeed-service repository
View all 5 skillsdegraded-operator-recovery
recover degraded OpenShift cluster operators
Apr 6DebuggingOpenShiftOperationsSREnamespace-troubleshooting
troubleshoot OpenShift namespace and RBAC issues
Apr 6Access ControlDebuggingOpenShiftSREnode-not-ready
troubleshoot OpenShift node status issues
Apr 6DebuggingEngineeringOpenShiftOperationsroute-ingress-troubleshooting
troubleshoot OpenShift Route and Ingress connectivity
Apr 6DebuggingNetworkingOpenShiftSRE
More from OpenShift (Red Hat)
View publishercluster-update-advisor
assess OpenShift cluster update readiness
agentic-skills
May 12DeploymentOpenShiftOperationsRisk Assessmentkubernetes-docs
search and read Kubernetes documentation
agentic-skills
May 12DocumentationKubernetesopenshift-docs
search and read OpenShift documentation
agentic-skills
May 12DocumentationKubernetesOpenShiftproduct-lifecycle
query Red Hat product lifecycle data
agentic-skills
Jul 31OpenShiftOperationsReporting