[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-openshift-node-not-ready":3,"mdc-a3pq43-key":35,"related-org-openshift-node-not-ready":374,"related-repo-openshift-node-not-ready":486},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":24,"repoUrl":25,"updatedAt":26,"license":27,"forks":28,"topics":29,"repo":30,"sourceUrl":33,"mdContent":34},"node-not-ready","troubleshoot OpenShift node status issues","Troubleshoot NotReady or SchedulingDisabled node status. Use when a node is down, unschedulable, or needs to be drained and restored.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"openshift","OpenShift (Red Hat)","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fopenshift.png",[12,16,19,21],{"name":13,"slug":14,"type":15},"Operations","operations","tag",{"name":17,"slug":18,"type":15},"Engineering","engineering",{"name":20,"slug":8,"type":15},"OpenShift",{"name":22,"slug":23,"type":15},"Debugging","debugging",70,"https:\u002F\u002Fgithub.com\u002Fopenshift\u002Flightspeed-service","2026-04-06T18:43:26.607462",null,88,[],{"repoUrl":25,"stars":24,"forks":28,"topics":31,"description":32},[],"Core repository for an AI-powered OCP assistant service","https:\u002F\u002Fgithub.com\u002Fopenshift\u002Flightspeed-service\u002Ftree\u002FHEAD\u002Fskills\u002Fnode-not-ready","---\nname: node-not-ready\ndescription: Troubleshoot NotReady or SchedulingDisabled node status. Use when a node is down, unschedulable, or needs to be drained and restored.\n---\n\n# Node Not Ready Diagnosis\n\nWhen a user reports a node that is NotReady, flapping, or has evicted workloads unexpectedly, follow this structured approach to identify the cause and restore the node.\n\n## 1. Identify Affected Nodes\n\nList all nodes and their statuses:\n\n1. Note which nodes are NotReady and how long they have been in that state. Recent (minutes) vs. prolonged (hours\u002Fdays) requires different urgency.\n2. Determine if the affected node is a **control plane node** or a **worker**. Control plane nodes are higher priority — losing quorum affects the entire cluster.\n3. Check if the node is `SchedulingDisabled` (cordoned) — this may be intentional (maintenance) or a side effect of a MachineHealthCheck remediation.\n\n## 2. Read Node Conditions\n\nDescribe the affected node and examine its conditions:\n\n1. **MemoryPressure=True**: the node is running low on memory. Identify the top memory-consuming pods and recommend eviction or resource limit adjustments.\n2. **DiskPressure=True**: the node is running low on disk. Check for large container logs, unused images, or full persistent volumes. Recommend cleanup actions.\n3. **PIDPressure=True**: the node is exhausting process IDs. Identify pods with excessive process creation (fork bombs, misconfigured worker pools).\n4. **NetworkUnavailable=True**: the node's network plugin is not ready. Check SDN\u002FOVN-Kubernetes pod health on that specific node.\n5. **Ready=False with no other pressure conditions**: kubelet is not posting status. This usually means the kubelet process is down or the node is unreachable.\n\nThe specific condition tells you exactly where to look next — do not run through all possibilities if the condition is clear.\n\n## 3. Check Infrastructure Layer\n\nIf the node conditions suggest the node itself is unreachable or the kubelet is down:\n\n1. Check the Machine and MachineSet objects — is the machine reported as running by the infrastructure provider?\n2. Check if a MachineHealthCheck has already detected the issue and is remediating (creating a replacement machine). If so, advise waiting for the remediation to complete before manual intervention.\n3. For bare-metal or user-provisioned infrastructure: the user needs to check the host directly (SSH, console, BMC). OLS cannot diagnose host-level issues without cluster-side evidence.\n\n## 4. Check Certificate Issues\n\nNode certificate problems are a common cause of NotReady, especially after extended downtime:\n\n1. Check for pending Certificate Signing Requests (CSRs) — nodes that were offline during certificate rotation may need their CSRs manually approved.\n2. If CSRs are pending: approve them and verify the node returns to Ready state.\n3. If no CSRs are pending but the node was offline for a long time: the kubelet certificates may have expired entirely and the node may need to be re-joined.\n\n## 5. Check Network Plugin Health\n\nIf the condition is NetworkUnavailable:\n\n1. Identify which network plugin the cluster uses (OpenShift SDN or OVN-Kubernetes).\n2. Check the network plugin pods running on the affected node — are they in CrashLoopBackOff or not scheduled?\n3. Check the node's network plugin logs for specific errors (failed to configure interface, VXLAN\u002FGeneve tunnel issues, OVS database corruption).\n\n## 6. Provide Recovery Steps\n\nOnce the cause is identified:\n\n1. State the specific condition and evidence.\n2. Provide the targeted fix:\n   - **Resource pressure**: identify consumers, suggest eviction or resource adjustments.\n   - **Kubelet down**: check machine status, recommend restart or machine replacement.\n   - **Pending CSRs**: approve them.\n   - **Network plugin failure**: restart the network plugin pod on the affected node or investigate the specific error.\n3. After remediation, verify the node transitions back to Ready.\n\n## Quality Standards\n\n- Always report how long the node has been NotReady — this affects triage urgency and likely causes.\n- Check whether a MachineHealthCheck is already remediating before suggesting manual action — do not compete with automated remediation.\n- Warn before suggesting cordon\u002Fdrain if the cluster is already capacity-constrained — draining a node when others are also unhealthy can cause cascading evictions.\n- Distinguish control plane nodes from workers in all recommendations. Restarting a control plane node has different risk than restarting a worker.\n- If the node is flapping (oscillating between Ready and NotReady), focus on the transition events rather than the current state.\n",{"data":36,"body":37},{"name":4,"description":6},{"type":38,"children":39},"root",[40,49,55,62,67,111,117,122,175,180,186,191,209,215,220,238,244,249,267,273,278,340,346],{"type":41,"tag":42,"props":43,"children":45},"element","h1",{"id":44},"node-not-ready-diagnosis",[46],{"type":47,"value":48},"text","Node Not Ready Diagnosis",{"type":41,"tag":50,"props":51,"children":52},"p",{},[53],{"type":47,"value":54},"When a user reports a node that is NotReady, flapping, or has evicted workloads unexpectedly, follow this structured approach to identify the cause and restore the node.",{"type":41,"tag":56,"props":57,"children":59},"h2",{"id":58},"_1-identify-affected-nodes",[60],{"type":47,"value":61},"1. Identify Affected Nodes",{"type":41,"tag":50,"props":63,"children":64},{},[65],{"type":47,"value":66},"List all nodes and their statuses:",{"type":41,"tag":68,"props":69,"children":70},"ol",{},[71,77,97],{"type":41,"tag":72,"props":73,"children":74},"li",{},[75],{"type":47,"value":76},"Note which nodes are NotReady and how long they have been in that state. Recent (minutes) vs. prolonged (hours\u002Fdays) requires different urgency.",{"type":41,"tag":72,"props":78,"children":79},{},[80,82,88,90,95],{"type":47,"value":81},"Determine if the affected node is a ",{"type":41,"tag":83,"props":84,"children":85},"strong",{},[86],{"type":47,"value":87},"control plane node",{"type":47,"value":89}," or a ",{"type":41,"tag":83,"props":91,"children":92},{},[93],{"type":47,"value":94},"worker",{"type":47,"value":96},". Control plane nodes are higher priority — losing quorum affects the entire cluster.",{"type":41,"tag":72,"props":98,"children":99},{},[100,102,109],{"type":47,"value":101},"Check if the node is ",{"type":41,"tag":103,"props":104,"children":106},"code",{"className":105},[],[107],{"type":47,"value":108},"SchedulingDisabled",{"type":47,"value":110}," (cordoned) — this may be intentional (maintenance) or a side effect of a MachineHealthCheck remediation.",{"type":41,"tag":56,"props":112,"children":114},{"id":113},"_2-read-node-conditions",[115],{"type":47,"value":116},"2. Read Node Conditions",{"type":41,"tag":50,"props":118,"children":119},{},[120],{"type":47,"value":121},"Describe the affected node and examine its conditions:",{"type":41,"tag":68,"props":123,"children":124},{},[125,135,145,155,165],{"type":41,"tag":72,"props":126,"children":127},{},[128,133],{"type":41,"tag":83,"props":129,"children":130},{},[131],{"type":47,"value":132},"MemoryPressure=True",{"type":47,"value":134},": the node is running low on memory. Identify the top memory-consuming pods and recommend eviction or resource limit adjustments.",{"type":41,"tag":72,"props":136,"children":137},{},[138,143],{"type":41,"tag":83,"props":139,"children":140},{},[141],{"type":47,"value":142},"DiskPressure=True",{"type":47,"value":144},": the node is running low on disk. Check for large container logs, unused images, or full persistent volumes. Recommend cleanup actions.",{"type":41,"tag":72,"props":146,"children":147},{},[148,153],{"type":41,"tag":83,"props":149,"children":150},{},[151],{"type":47,"value":152},"PIDPressure=True",{"type":47,"value":154},": the node is exhausting process IDs. Identify pods with excessive process creation (fork bombs, misconfigured worker pools).",{"type":41,"tag":72,"props":156,"children":157},{},[158,163],{"type":41,"tag":83,"props":159,"children":160},{},[161],{"type":47,"value":162},"NetworkUnavailable=True",{"type":47,"value":164},": the node's network plugin is not ready. Check SDN\u002FOVN-Kubernetes pod health on that specific node.",{"type":41,"tag":72,"props":166,"children":167},{},[168,173],{"type":41,"tag":83,"props":169,"children":170},{},[171],{"type":47,"value":172},"Ready=False with no other pressure conditions",{"type":47,"value":174},": kubelet is not posting status. This usually means the kubelet process is down or the node is unreachable.",{"type":41,"tag":50,"props":176,"children":177},{},[178],{"type":47,"value":179},"The specific condition tells you exactly where to look next — do not run through all possibilities if the condition is clear.",{"type":41,"tag":56,"props":181,"children":183},{"id":182},"_3-check-infrastructure-layer",[184],{"type":47,"value":185},"3. Check Infrastructure Layer",{"type":41,"tag":50,"props":187,"children":188},{},[189],{"type":47,"value":190},"If the node conditions suggest the node itself is unreachable or the kubelet is down:",{"type":41,"tag":68,"props":192,"children":193},{},[194,199,204],{"type":41,"tag":72,"props":195,"children":196},{},[197],{"type":47,"value":198},"Check the Machine and MachineSet objects — is the machine reported as running by the infrastructure provider?",{"type":41,"tag":72,"props":200,"children":201},{},[202],{"type":47,"value":203},"Check if a MachineHealthCheck has already detected the issue and is remediating (creating a replacement machine). If so, advise waiting for the remediation to complete before manual intervention.",{"type":41,"tag":72,"props":205,"children":206},{},[207],{"type":47,"value":208},"For bare-metal or user-provisioned infrastructure: the user needs to check the host directly (SSH, console, BMC). OLS cannot diagnose host-level issues without cluster-side evidence.",{"type":41,"tag":56,"props":210,"children":212},{"id":211},"_4-check-certificate-issues",[213],{"type":47,"value":214},"4. Check Certificate Issues",{"type":41,"tag":50,"props":216,"children":217},{},[218],{"type":47,"value":219},"Node certificate problems are a common cause of NotReady, especially after extended downtime:",{"type":41,"tag":68,"props":221,"children":222},{},[223,228,233],{"type":41,"tag":72,"props":224,"children":225},{},[226],{"type":47,"value":227},"Check for pending Certificate Signing Requests (CSRs) — nodes that were offline during certificate rotation may need their CSRs manually approved.",{"type":41,"tag":72,"props":229,"children":230},{},[231],{"type":47,"value":232},"If CSRs are pending: approve them and verify the node returns to Ready state.",{"type":41,"tag":72,"props":234,"children":235},{},[236],{"type":47,"value":237},"If no CSRs are pending but the node was offline for a long time: the kubelet certificates may have expired entirely and the node may need to be re-joined.",{"type":41,"tag":56,"props":239,"children":241},{"id":240},"_5-check-network-plugin-health",[242],{"type":47,"value":243},"5. Check Network Plugin Health",{"type":41,"tag":50,"props":245,"children":246},{},[247],{"type":47,"value":248},"If the condition is NetworkUnavailable:",{"type":41,"tag":68,"props":250,"children":251},{},[252,257,262],{"type":41,"tag":72,"props":253,"children":254},{},[255],{"type":47,"value":256},"Identify which network plugin the cluster uses (OpenShift SDN or OVN-Kubernetes).",{"type":41,"tag":72,"props":258,"children":259},{},[260],{"type":47,"value":261},"Check the network plugin pods running on the affected node — are they in CrashLoopBackOff or not scheduled?",{"type":41,"tag":72,"props":263,"children":264},{},[265],{"type":47,"value":266},"Check the node's network plugin logs for specific errors (failed to configure interface, VXLAN\u002FGeneve tunnel issues, OVS database corruption).",{"type":41,"tag":56,"props":268,"children":270},{"id":269},"_6-provide-recovery-steps",[271],{"type":47,"value":272},"6. Provide Recovery Steps",{"type":41,"tag":50,"props":274,"children":275},{},[276],{"type":47,"value":277},"Once the cause is identified:",{"type":41,"tag":68,"props":279,"children":280},{},[281,286,335],{"type":41,"tag":72,"props":282,"children":283},{},[284],{"type":47,"value":285},"State the specific condition and evidence.",{"type":41,"tag":72,"props":287,"children":288},{},[289,291],{"type":47,"value":290},"Provide the targeted fix:\n",{"type":41,"tag":292,"props":293,"children":294},"ul",{},[295,305,315,325],{"type":41,"tag":72,"props":296,"children":297},{},[298,303],{"type":41,"tag":83,"props":299,"children":300},{},[301],{"type":47,"value":302},"Resource pressure",{"type":47,"value":304},": identify consumers, suggest eviction or resource adjustments.",{"type":41,"tag":72,"props":306,"children":307},{},[308,313],{"type":41,"tag":83,"props":309,"children":310},{},[311],{"type":47,"value":312},"Kubelet down",{"type":47,"value":314},": check machine status, recommend restart or machine replacement.",{"type":41,"tag":72,"props":316,"children":317},{},[318,323],{"type":41,"tag":83,"props":319,"children":320},{},[321],{"type":47,"value":322},"Pending CSRs",{"type":47,"value":324},": approve them.",{"type":41,"tag":72,"props":326,"children":327},{},[328,333],{"type":41,"tag":83,"props":329,"children":330},{},[331],{"type":47,"value":332},"Network plugin failure",{"type":47,"value":334},": restart the network plugin pod on the affected node or investigate the specific error.",{"type":41,"tag":72,"props":336,"children":337},{},[338],{"type":47,"value":339},"After remediation, verify the node transitions back to Ready.",{"type":41,"tag":56,"props":341,"children":343},{"id":342},"quality-standards",[344],{"type":47,"value":345},"Quality Standards",{"type":41,"tag":292,"props":347,"children":348},{},[349,354,359,364,369],{"type":41,"tag":72,"props":350,"children":351},{},[352],{"type":47,"value":353},"Always report how long the node has been NotReady — this affects triage urgency and likely causes.",{"type":41,"tag":72,"props":355,"children":356},{},[357],{"type":47,"value":358},"Check whether a MachineHealthCheck is already remediating before suggesting manual action — do not compete with automated remediation.",{"type":41,"tag":72,"props":360,"children":361},{},[362],{"type":47,"value":363},"Warn before suggesting cordon\u002Fdrain if the cluster is already capacity-constrained — draining a node when others are also unhealthy can cause cascading evictions.",{"type":41,"tag":72,"props":365,"children":366},{},[367],{"type":47,"value":368},"Distinguish control plane nodes from workers in all recommendations. Restarting a control plane node has different risk than restarting a worker.",{"type":41,"tag":72,"props":370,"children":371},{},[372],{"type":47,"value":373},"If the node is flapping (oscillating between Ready and NotReady), focus on the transition events rather than the current state.",{"items":375,"total":485},[376,389,402,409,422,435,452,463,473],{"slug":377,"name":377,"fn":378,"description":379,"org":380,"tags":381,"stars":24,"repoUrl":25,"updatedAt":388},"degraded-operator-recovery","recover degraded OpenShift cluster operators","Troubleshoot ClusterOperator in Degraded, Unavailable, or not Progressing state. Use when operator status shows error conditions, reconciliation failures, or degraded health checks.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[382,383,384,385],{"name":22,"slug":23,"type":15},{"name":20,"slug":8,"type":15},{"name":13,"slug":14,"type":15},{"name":386,"slug":387,"type":15},"SRE","sre","2026-04-06T18:43:30.354728",{"slug":390,"name":390,"fn":391,"description":392,"org":393,"tags":394,"stars":24,"repoUrl":25,"updatedAt":401},"namespace-troubleshooting","troubleshoot OpenShift namespace and RBAC issues","Troubleshoot namespace stuck in Terminating state, ResourceQuota exhaustion, or RBAC permission denied errors. Use when resources cannot be created or forbidden errors occur.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[395,398,399,400],{"name":396,"slug":397,"type":15},"Access Control","access-control",{"name":22,"slug":23,"type":15},{"name":20,"slug":8,"type":15},{"name":386,"slug":387,"type":15},"2026-04-06T18:43:29.107881",{"slug":4,"name":4,"fn":5,"description":6,"org":403,"tags":404,"stars":24,"repoUrl":25,"updatedAt":26},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[405,406,407,408],{"name":22,"slug":23,"type":15},{"name":17,"slug":18,"type":15},{"name":20,"slug":8,"type":15},{"name":13,"slug":14,"type":15},{"slug":410,"name":410,"fn":411,"description":412,"org":413,"tags":414,"stars":24,"repoUrl":25,"updatedAt":421},"pod-failure-diagnosis","diagnose OpenShift pod failure states","Troubleshoot CrashLoopBackOff, ImagePullBackOff, Pending, Error, or OOMKilled status. Use when a workload keeps restarting, fails to start, or is crash-looping.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[415,416,419,420],{"name":22,"slug":23,"type":15},{"name":417,"slug":418,"type":15},"Kubernetes","kubernetes",{"name":20,"slug":8,"type":15},{"name":386,"slug":387,"type":15},"2026-04-06T18:43:27.866868",{"slug":423,"name":423,"fn":424,"description":425,"org":426,"tags":427,"stars":24,"repoUrl":25,"updatedAt":434},"route-ingress-troubleshooting","troubleshoot OpenShift Route and Ingress connectivity","Troubleshoot Route or Ingress connectivity failures. Use when traffic returns 502, 503, connection refused, or the endpoint is not reachable externally.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[428,429,432,433],{"name":22,"slug":23,"type":15},{"name":430,"slug":431,"type":15},"Networking","networking",{"name":20,"slug":8,"type":15},{"name":386,"slug":387,"type":15},"2026-04-06T18:43:31.593028",{"slug":436,"name":436,"fn":437,"description":438,"org":439,"tags":440,"stars":449,"repoUrl":450,"updatedAt":451},"cluster-update-advisor","assess OpenShift cluster update readiness","Assess OpenShift cluster update (upgrade) readiness and risk. Use when evaluating whether a cluster is safe to update, when an update is available, or when the user asks about update risks, prerequisites, blockers, or best practices.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[441,444,445,446],{"name":442,"slug":443,"type":15},"Deployment","deployment",{"name":20,"slug":8,"type":15},{"name":13,"slug":14,"type":15},{"name":447,"slug":448,"type":15},"Risk Assessment","risk-assessment",5,"https:\u002F\u002Fgithub.com\u002Fopenshift\u002Fagentic-skills","2026-05-12T06:07:16.759342",{"slug":453,"name":453,"fn":454,"description":455,"org":456,"tags":457,"stars":449,"repoUrl":450,"updatedAt":462},"kubernetes-docs","search and read Kubernetes documentation","Search and read Kubernetes documentation in markdown format. Use when the user asks about Kubernetes concepts, tasks, API reference, kubectl, or any upstream k8s topic — including pods, deployments, services, RBAC, networking, storage, or scheduling.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[458,461],{"name":459,"slug":460,"type":15},"Documentation","documentation",{"name":417,"slug":418,"type":15},"2026-05-12T06:07:15.393562",{"slug":464,"name":464,"fn":465,"description":466,"org":467,"tags":468,"stars":449,"repoUrl":450,"updatedAt":472},"openshift-docs","search and read OpenShift documentation","Search and read OpenShift Container Platform documentation in markdown format. Use when the user asks about OpenShift features, configuration, installation, troubleshooting, or any OCP-specific topic — including operators, routes, services, oc, RBAC, networking, storage, or cluster administration.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[469,470,471],{"name":459,"slug":460,"type":15},{"name":417,"slug":418,"type":15},{"name":20,"slug":8,"type":15},"2026-05-12T06:07:14.03809",{"slug":474,"name":474,"fn":475,"description":476,"org":477,"tags":478,"stars":449,"repoUrl":450,"updatedAt":484},"product-lifecycle","query Red Hat product lifecycle data","Query Red Hat Product Life Cycle data for support phases, end-of-life dates, and OpenShift version compatibility. Use when evaluating whether installed operators or layered products are supported on a given OCP version, approaching end of life, or need upgrading before a cluster upgrade. Also use when the user asks about product support status, EOL dates, or lifecycle phases for any Red Hat product.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[479,480,481],{"name":20,"slug":8,"type":15},{"name":13,"slug":14,"type":15},{"name":482,"slug":483,"type":15},"Reporting","reporting","2026-07-31T05:55:39.981651",9,{"items":487,"total":449},[488,495,502,509,516],{"slug":377,"name":377,"fn":378,"description":379,"org":489,"tags":490,"stars":24,"repoUrl":25,"updatedAt":388},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[491,492,493,494],{"name":22,"slug":23,"type":15},{"name":20,"slug":8,"type":15},{"name":13,"slug":14,"type":15},{"name":386,"slug":387,"type":15},{"slug":390,"name":390,"fn":391,"description":392,"org":496,"tags":497,"stars":24,"repoUrl":25,"updatedAt":401},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[498,499,500,501],{"name":396,"slug":397,"type":15},{"name":22,"slug":23,"type":15},{"name":20,"slug":8,"type":15},{"name":386,"slug":387,"type":15},{"slug":4,"name":4,"fn":5,"description":6,"org":503,"tags":504,"stars":24,"repoUrl":25,"updatedAt":26},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[505,506,507,508],{"name":22,"slug":23,"type":15},{"name":17,"slug":18,"type":15},{"name":20,"slug":8,"type":15},{"name":13,"slug":14,"type":15},{"slug":410,"name":410,"fn":411,"description":412,"org":510,"tags":511,"stars":24,"repoUrl":25,"updatedAt":421},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[512,513,514,515],{"name":22,"slug":23,"type":15},{"name":417,"slug":418,"type":15},{"name":20,"slug":8,"type":15},{"name":386,"slug":387,"type":15},{"slug":423,"name":423,"fn":424,"description":425,"org":517,"tags":518,"stars":24,"repoUrl":25,"updatedAt":434},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[519,520,521,522],{"name":22,"slug":23,"type":15},{"name":430,"slug":431,"type":15},{"name":20,"slug":8,"type":15},{"name":386,"slug":387,"type":15}]