[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-cockroachdb-diagnosing-cockroachdb-helm-deployments":3,"mdc-e0i85p-key":36,"related-org-cockroachdb-diagnosing-cockroachdb-helm-deployments":5941,"related-repo-cockroachdb-diagnosing-cockroachdb-helm-deployments":6092},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":25,"repoUrl":26,"updatedAt":27,"license":28,"forks":29,"topics":30,"repo":31,"sourceUrl":34,"mdContent":35},"diagnosing-cockroachdb-helm-deployments","diagnose CockroachDB Helm chart deployments","Diagnoses failed or unhealthy CockroachDB Helm chart deployments by checking Helm release state, operator health, CrdbCluster and CrdbNode status, pod readiness, RBAC, webhooks, TLS, upgrades, scaling, PVCs, DNS, and multi-region assumptions. Use when Helm install or upgrade fails, pods are not Ready, or the operator is not reconciling.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"cockroachdb","CockroachDB","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fcockroachdb.png",[12,16,19,22],{"name":13,"slug":14,"type":15},"Database","database","tag",{"name":17,"slug":18,"type":15},"Deployment","deployment",{"name":20,"slug":21,"type":15},"Kubernetes","kubernetes",{"name":23,"slug":24,"type":15},"Debugging","debugging",105,"https:\u002F\u002Fgithub.com\u002Fcockroachdb\u002Fhelm-charts","2026-07-12T07:57:24.018818",null,153,[],{"repoUrl":26,"stars":25,"forks":29,"topics":32,"description":33},[],"Helm charts for cockroachdb","https:\u002F\u002Fgithub.com\u002Fcockroachdb\u002Fhelm-charts\u002Ftree\u002FHEAD\u002Fskills\u002Fcockroachdb-observability-and-diagnostics\u002Fdiagnosing-cockroachdb-helm-deployments","---\nname: diagnosing-cockroachdb-helm-deployments\ndescription: Diagnoses failed or unhealthy CockroachDB Helm chart deployments by checking Helm release state, operator health, CrdbCluster and CrdbNode status, pod readiness, RBAC, webhooks, TLS, upgrades, scaling, PVCs, DNS, and multi-region assumptions. Use when Helm install or upgrade fails, pods are not Ready, or the operator is not reconciling.\ncompatibility: CockroachDB Helm v2 charts and operator-managed crdb.cockroachlabs.com\u002Fv1beta1 resources. Requires Kubernetes read access to the operator namespace and CockroachDB namespace; some remediation requires cluster-admin or platform-team action.\nmetadata:\n  author: cockroachdb\n  version: \"1.1\"\n---\n\n# Diagnosing CockroachDB Helm Deployments\n\nDiagnoses CockroachDB Helm install, upgrade, and readiness failures for operator-managed clusters. Keep the flow customer-facing: collect read-only evidence, classify the failure, and propose the smallest safe remediation. If the issue needs TSC escalation or deep operator forensics, use [collecting-cockroachdb-operator-escalation-packet](..\u002Fcollecting-cockroachdb-operator-escalation-packet\u002FSKILL.md).\n\n## When to Use This Skill\n\n- `helm install` or `helm upgrade` fails\n- `CrdbCluster.status.observedGeneration` is behind `metadata.generation`\n- `CrdbCluster.status.reconciled` is false or missing\n- CockroachDB pods are Pending, Init, CrashLoopBackOff, Running but not Ready, or stuck on an old image\n- The operator Deployment is unavailable, silent, or not reconciling\n- Errors mention RBAC, CRDs, TLS, certificates, webhooks, node locality, PVCs, DNS, upgrades, scale operations, decommissioning, or multi-region networking\n\n## Related Skills\n\n- Use [collecting-cockroachdb-operator-escalation-packet](..\u002Fcollecting-cockroachdb-operator-escalation-packet\u002FSKILL.md) when the customer needs to escalate or when TSC asks for a complete data bundle.\n- Use [debugging-cockroachdb-operator-migrations](..\u002F..\u002Fcockroachdb-onboarding-and-migrations\u002Fdebugging-cockroachdb-operator-migrations\u002FSKILL.md) for Helm StatefulSet or public operator migration problems.\n- Use [configuring-cockroachdb-helm-tls](..\u002F..\u002Fcockroachdb-operations-and-lifecycle\u002Fconfiguring-cockroachdb-helm-tls\u002FSKILL.md) for TLS provider selection and certificate validation.\n- Use [validating-cockroachdb-helm-multiregion](..\u002F..\u002Fcockroachdb-onboarding-and-migrations\u002Fvalidating-cockroachdb-helm-multiregion\u002FSKILL.md) for cross-region DNS, networking, and certificate checks.\n\n## Inputs\n\n- Exact failed command and stderr\n- Operator namespace, operator Helm release name, and operator chart version\n- CockroachDB namespace, CockroachDB Helm release name, and CockroachDB chart version\n- Values file or `--set` values used\n- Kubernetes context and Kubernetes version\n- CockroachDB image\u002Fversion and operator image\u002Fversion\n- Whether this is install, upgrade, scale up\u002Fdown, certificate rotation, migration, maintenance, or recovery\n- What the customer already tried, in order\n\n## Safety Considerations\n\n- Collect evidence before retrying. Repeated Helm or kubectl attempts can obscure the original failure.\n- Do not delete PVCs, Secrets, `CrdbCluster`, or `CrdbNode` resources unless the user explicitly asks for teardown and understands data impact.\n- Do not manually edit `CrdbCluster.status`; the operator owns status.\n- Do not run `cockroach init` on an existing cluster.\n- Do not change service settings such as `publishNotReadyAddresses` without operator-team guidance.\n- Do not delete version checker jobs or pods during an upgrade until their status and logs are collected.\n- Do not restart or scale down the operator during an active rollout, decommission, or migration unless basic evidence has already been collected.\n- Do not uninstall the operator before checking whether it manages other namespaces or clusters.\n\n## Execution Discipline\n\n- Execute one step at a time and inspect the output before moving on. Do not run whole sections, unrelated command groups, or later diagnostic branches in parallel; earlier output determines which later checks are relevant.\n- Treat commands as templates. Substitute namespaces, release names, chart names, and pod names deliberately before running anything.\n- Do not run any mutating command unless the user explicitly approves it for the target cluster. This includes `kubectl patch`, `kubectl annotate`, `kubectl delete`, `kubectl scale`, `kubectl rollout restart`, `helm upgrade`, drain\u002Fdecommission commands, and interactive `kubectl exec` or `kubectl debug` shells.\n- In production or whenever the impact is unclear, stop and escalate to TSE or the operator team before pprof\u002Fmetrics collection, debug containers, timestamp-based rolling restarts, mode changes, operator restarts, scale changes, or decommission actions.\n\n## Step 1: Collect Baseline State\n\n```bash\n# Helm release state\nhelm -n \u003Coperator-namespace> status \u003Coperator-release> || true\nhelm -n \u003Ccockroachdb-namespace> status \u003Ccockroachdb-release> || true\nhelm -n \u003Coperator-namespace> history \u003Coperator-release> || true\nhelm -n \u003Ccockroachdb-namespace> history \u003Ccockroachdb-release> || true\n\n# Operator state\nkubectl -n \u003Coperator-namespace> get deploy,pod,svc -o wide | grep -E 'cockroach-operator|NAME'\nkubectl -n \u003Coperator-namespace> logs -l app=cockroach-operator --tail=200 || true\n\n# CRD and CockroachDB resources\nkubectl get crd crdbclusters.crdb.cockroachlabs.com crdbnodes.crdb.cockroachlabs.com\nkubectl -n \u003Ccockroachdb-namespace> get crdbcluster,crdbnode,pod,svc,endpoints,pvc,pdb -o wide\nkubectl -n \u003Ccockroachdb-namespace> describe crdbcluster \u003Ccockroachdb-release> || true\nkubectl -n \u003Ccockroachdb-namespace> get events --sort-by=.lastTimestamp | tail -50\n```\n\nFor a stuck pod or node:\n\n```bash\nkubectl -n \u003Ccockroachdb-namespace> describe pod \u003Ccrdb-pod>\nkubectl -n \u003Ccockroachdb-namespace> logs \u003Ccrdb-pod> -c cockroachdb --tail=200\nkubectl -n \u003Ccockroachdb-namespace> logs \u003Ccrdb-pod> -c cockroachdb --previous\nkubectl -n \u003Ccockroachdb-namespace> describe crdbnode \u003Ccrdbnode-name>\n```\n\n## Step 2: Classify the Failure\n\n| Symptom | Likely Class | Next Check |\n|---|---|---|\n| `no matches for kind \"CrdbCluster\"` | CRDs\u002Foperator not installed or not ready | [CRD and operator readiness](#crd-and-operator-readiness) |\n| `attempt to grant extra privileges` | Helm RBAC restriction | [RBAC and node-reader failures](#rbac-and-node-reader-failures) |\n| TLS values validation error | TLS provider conflict | [TLS and certificate failures](#tls-and-certificate-failures) |\n| Operator pod CrashLoopBackOff or OOMKilled | Operator crash or resource limit | [Operator health](#operator-health) |\n| Operator running but no reconcile logs | Watch namespace mismatch or blocked worker | [Operator health](#operator-health) |\n| `observedGeneration` behind `generation` | Reconcile is stuck or skipped | [Reconciliation not progressing](#reconciliation-not-progressing) |\n| Pods Pending | Scheduling, storage, topology, image pull, or node labels | [Pod scheduling and storage failures](#pod-scheduling-and-storage-failures) |\n| Pods Running but not Ready | CRDB readiness, TLS, join, DNS, network, or recovery | [Pod readiness and CRDB issues](#pod-readiness-and-crdb-issues) |\n| Upgrade stuck with mixed pod images | Version validation, rejected image, rollout dependency, scheduling | [Upgrade and version validation](#upgrade-and-version-validation) |\n| Multi-region pods cannot join | DNS, network, region list, CA mismatch | [DNS, service, and network issues](#dns-service-and-network-issues) |\n| Scale-down stuck | Decommission\u002Fdrain blocked or multiple nodes decommissioning | [Scale down and decommission](#scale-down-and-decommission) |\n| Migration labels\u002Fstatus stuck | Migration controller issue | [debugging-cockroachdb-operator-migrations](..\u002F..\u002Fcockroachdb-onboarding-and-migrations\u002Fdebugging-cockroachdb-operator-migrations\u002FSKILL.md) |\n\n## CRD and Operator Readiness\n\n```bash\nkubectl -n \u003Coperator-namespace> rollout status deploy\u002Fcockroach-operator --timeout=5m\nkubectl get crd crdbclusters.crdb.cockroachlabs.com -o jsonpath='{.spec.versions[*].name}{\"\\n\"}'\nkubectl get crd crdbnodes.crdb.cockroachlabs.com -o jsonpath='{.spec.versions[*].name}{\"\\n\"}'\nkubectl -n \u003Coperator-namespace> get deploy cockroach-operator -o jsonpath='{.spec.template.spec.containers[0].image}{\"\\n\"}'\n```\n\nRemediation:\n\n- Install or upgrade the operator chart first.\n- Wait for the operator Deployment before installing the CockroachDB chart.\n- For split charts, upgrade the operator chart before the CockroachDB chart.\n- Check the Helm chart changelog for version-specific operator fixes before deep debugging older releases.\n\n## Operator Health\n\n```bash\nkubectl -n \u003Coperator-namespace> get pods -l app=cockroach-operator -o wide\nkubectl -n \u003Coperator-namespace> describe pod \u003Coperator-pod>\nkubectl -n \u003Coperator-namespace> logs -l app=cockroach-operator --tail=100\nkubectl -n \u003Coperator-namespace> get deploy cockroach-operator -o jsonpath='{.spec.template.spec.containers[0].env}{\"\\n\"}'\n```\n\nInterpretation:\n\n- CrashLoopBackOff: collect previous logs and check for panics.\n- OOMKilled: inspect limits and consider increasing operator memory.\n- Empty or unset `WATCH_NAMESPACE`: global mode.\n- Non-empty `WATCH_NAMESPACE`: the operator watches only listed namespaces.\n- If the CockroachDB namespace is not watched, deploy an operator for it or add it to the watch scope.\n\nCheck recent logs to see whether reconciliation is active:\n\n```bash\nkubectl -n \u003Coperator-namespace> logs -l app=cockroach-operator --tail=100 | grep -i reconcil || true\n```\n\nDo not add ad hoc annotations to trigger reconciliation. If a user-approved reconcile-triggering change is required, use the chart-supported timestamp path through `helm upgrade --reuse-values`; this updates `helm.sh\u002FrestartedAt` and may roll CockroachDB pods, so treat it as a mutating operation:\n\n```bash\nhelm -n \u003Ccockroachdb-namespace> upgrade \u003Ccockroachdb-release> \u003Ccockroachdb-chart> \\\n  --reuse-values \\\n  --set-string cockroachdb.crdbCluster.timestamp=\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\"\n```\n\nIf the operator is healthy but silent and no user-approved mutation is appropriate, use [collecting-cockroachdb-operator-escalation-packet](..\u002Fcollecting-cockroachdb-operator-escalation-packet\u002FSKILL.md) to gather pprof and metrics before restarting it.\n\n## RBAC and Node-Reader Failures\n\nCommon error:\n\n```text\nattempt to grant extra privileges\n```\n\nCause:\n\n- The installing principal cannot create cluster-scoped RBAC.\n- CockroachDB pods need node read access to derive locality.\n\nChecks:\n\n```bash\nkubectl auth can-i create clusterroles.rbac.authorization.k8s.io\nkubectl auth can-i create clusterrolebindings.rbac.authorization.k8s.io\nkubectl auth can-i get nodes\n```\n\nRemediation options:\n\n- Platform team installs the operator chart with `nodeReader.enabled=true` and subjects matching the CockroachDB ServiceAccount.\n- Tenant chart sets `cockroachdb.crdbCluster.rbac.nodeReader.create=false` only after the platform-owned binding exists.\n- If the customer accepts cluster-admin install privileges, rerun the Helm operation with an identity that can create the required ClusterRole and ClusterRoleBinding.\n\nDo not set `nodeReader.create=false` before replacement RBAC exists.\n\n## Webhook Checks\n\n```bash\nkubectl -n \u003Coperator-namespace> get svc cockroach-webhook-service\nkubectl -n \u003Coperator-namespace> get endpoints cockroach-webhook-service\nkubectl get validatingwebhookconfigurations | grep cockroach\n```\n\nIf webhook validation fails, verify the CA bundle:\n\n```bash\nkubectl get validatingwebhookconfiguration cockroach-webhook-config \\\n  -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | base64 -d | openssl x509 -noout -dates -subject -issuer\n```\n\nFor scoped operators, webhook configurations may be namespace-suffixed, such as `cockroach-webhook-config-\u003Cnamespace>`.\n\n## Reconciliation Not Progressing\n\n```bash\nkubectl -n \u003Ccockroachdb-namespace> get crdbcluster \u003Ccockroachdb-release> -o json | jq '{\n  mode: .spec.mode,\n  image: .spec.image,\n  generation: .metadata.generation,\n  observedGeneration: .status.observedGeneration,\n  statusImage: .status.image,\n  actions: .status.actions,\n  conditions: .status.conditions\n}'\n\nkubectl -n \u003Ccockroachdb-namespace> get crdbnodes \\\n  -o custom-columns=NAME:.metadata.name,GENERATION:.metadata.generation,OBSERVED:.status.observedGeneration,PHASE:.status.phase,HASH:.metadata.annotations[\"crdb.cockroachlabs.com\u002Fhash-revision\"],NODE_ID:.status.nodeID\n```\n\nChecklist:\n\n1. Confirm the operator is running and watching the CockroachDB namespace.\n2. Confirm `spec.mode` is not `Disabled`.\n3. Check whether initialization conditions look correct for an existing cluster.\n4. Check operator logs for reconcile start\u002Fend pairs and errors.\n5. If no progress is visible, collect the escalation packet before restarting the operator.\n\n## Pod Readiness and CRDB Issues\n\n```bash\nkubectl -n \u003Ccockroachdb-namespace> get pods -l app.kubernetes.io\u002Fname=cockroachdb -o wide\nkubectl -n \u003Ccockroachdb-namespace> describe pod \u003Ccrdb-pod>\nkubectl -n \u003Ccockroachdb-namespace> logs \u003Ccrdb-pod> -c cockroachdb --tail=200\nkubectl -n \u003Ccockroachdb-namespace> logs \u003Ccrdb-pod> -c cockroachdb --previous\nkubectl -n \u003Ccockroachdb-namespace> get pod \u003Ccrdb-pod> -o jsonpath='{.spec.containers[0].readinessProbe}{\"\\n\"}'\nkubectl -n \u003Ccockroachdb-namespace> get pods -l app.kubernetes.io\u002Fname=cockroachdb \\\n  -o custom-columns=NAME:.metadata.name,IMAGE:.spec.containers[0].image,PHASE:.status.phase,READY:.status.containerStatuses[0].ready,NODE:.spec.nodeName\n```\n\nCommon pod issues:\n\n- Pending during upgrade: old pods may still carry pre-upgrade scheduling or affinity constraints.\n- CrashLoopBackOff: inspect storage errors, TLS errors, join address failures, and previous logs.\n- Running but not Ready: check the readiness probe, CRDB health endpoint, certificate trust, join service, and whether the node is recovering.\n\n## Upgrade and Version Validation\n\n```bash\nkubectl -n \u003Ccockroachdb-namespace> get crdbcluster \u003Ccockroachdb-release> -o json | jq '{\n  specImage: .spec.image,\n  statusImage: .status.image,\n  actions: .status.actions,\n  conditions: [.status.conditions[]? | select(.type | test(\"Upgrade|Version|Validate\"))]\n}'\n\nkubectl -n \u003Ccockroachdb-namespace> get crdbcluster \u003Ccockroachdb-release> -o jsonpath='{.metadata.annotations}{\"\\n\"}' | jq .\nkubectl -n \u003Ccockroachdb-namespace> get jobs\nkubectl -n \u003Ccockroachdb-namespace> describe job \u003Cversion-checker-job>\nkubectl -n \u003Ccockroachdb-namespace> logs -l job-name=\u003Cversion-checker-job>\nkubectl -n \u003Ccockroachdb-namespace> get pods -l app.kubernetes.io\u002Fname=cockroachdb \\\n  -o custom-columns=NAME:.metadata.name,IMAGE:.spec.containers[0].image,REVISION:.metadata.annotations[\"crdb\\.cockroachlabs\\.com\u002Fhash-revision\"],PHASE:.status.phase\n```\n\nInterpretation:\n\n- If `spec.image` differs from `status.image`, an upgrade is in progress or stuck.\n- If a rejected-image annotation exists, inspect its value; the operator rejected the target version.\n- If the version checker job exists but the pod is gone, use job status and operator logs for validation messages.\n- Do not delete version checker jobs or pods until their status and logs are captured.\n\n## DNS, Service, and Network Issues\n\nThe operator creates separate service paths for pod DNS and join traffic. Do not change service settings without operator-team guidance.\n\n```bash\nkubectl -n \u003Ccockroachdb-namespace> get service \u003Ccockroachdb-release> -o yaml\nkubectl -n \u003Ccockroachdb-namespace> get service \u003Ccockroachdb-release>-join -o yaml\nkubectl -n \u003Ccockroachdb-namespace> get endpoints \u003Ccockroachdb-release>\nkubectl -n \u003Ccockroachdb-namespace> get endpoints \u003Ccockroachdb-release>-join\n\nkubectl -n \u003Ccockroachdb-namespace> exec \u003Ccrdb-pod> -c cockroachdb -- \\\n  nslookup \u003Ccockroachdb-release>.\u003Ccockroachdb-namespace>.svc.cluster.local 2>&1 || true\n\nkubectl -n \u003Ccockroachdb-namespace> exec \u003Ccrdb-pod> -c cockroachdb -- \\\n  nslookup \u003Ccockroachdb-release>-join.\u003Ccockroachdb-namespace>.svc.cluster.local 2>&1 || true\n```\n\nFor multi-region checks, use [validating-cockroachdb-helm-multiregion](..\u002F..\u002Fcockroachdb-onboarding-and-migrations\u002Fvalidating-cockroachdb-helm-multiregion\u002FSKILL.md).\n\n## TLS and Certificate Failures\n\nUse [configuring-cockroachdb-helm-tls](..\u002F..\u002Fcockroachdb-operations-and-lifecycle\u002Fconfiguring-cockroachdb-helm-tls\u002FSKILL.md) for TLS mode selection and detailed certificate checks.\n\nQuick checks:\n\n```bash\nhelm template \u003Crelease> \u003Cchart> -n \u003Cnamespace> -f values.yaml >\u002Ftmp\u002Frendered.yaml\nkubectl -n \u003Cnamespace> get secret,configmap | grep -E 'cockroach|crdb|cert|ca|tls'\nkubectl -n \u003Cnamespace> get crdbcluster \u003Crelease> -o yaml | grep -A20 certificates\nkubectl -n \u003Cnamespace> get pod \u003Ccrdb-pod> -o jsonpath='{.spec.containers[*].name}{\"\\n\"}'\nkubectl -n \u003Cnamespace> logs \u003Ccrdb-pod> -c cert-reloader --tail=100\n```\n\nRemediation:\n\n- Ensure exactly one TLS provider is enabled.\n- Ensure self-signer `caProvided=true` has a valid `caSecret`.\n- Ensure cert-manager `Issuer` or `ClusterIssuer` exists and `Certificate` resources become Ready.\n- Ensure external certificate Secrets and CA ConfigMap have expected keys and share a trust root.\n- Collect expiry, subject, issuer, and SANs, but do not print private key contents.\n\n## Pod Scheduling and Storage Failures\n\n```bash\nkubectl -n \u003Cnamespace> describe pod \u003Cpod-name>\nkubectl -n \u003Cnamespace> get pvc -o wide\nkubectl get storageclass\nkubectl get nodes -L topology.kubernetes.io\u002Fregion,topology.kubernetes.io\u002Fzone\nkubectl -n \u003Cnamespace> exec \u003Ccrdb-pod> -c cockroachdb -- df -h \u002Fcockroach\u002Fcockroach-data\n```\n\nCommon causes:\n\n- PVCs cannot bind because no default StorageClass exists or `storageClassName` is wrong.\n- Topology spread constraints cannot be satisfied because node zone labels are missing or insufficient.\n- Node resources are too small for requested CPU\u002Fmemory.\n- Image pull failures from registry policy or air-gapped environments.\n- Migrated PVCs may be missing ownerReferences; use the migration debugging skill before deleting anything.\n\n## Scale Down and Decommission\n\n```bash\nkubectl -n \u003Ccockroachdb-namespace> exec \u003Cready-crdb-pod> -c cockroachdb -- \\\n  \u002Fcockroach\u002Fcockroach node status --decommission\n\nkubectl -n \u003Ccockroachdb-namespace> get crdbnodes -o json | jq '[.items[] | select(.status.phase==\"Decommissioning\")] | {count: length, nodes: [.[].metadata.name]}'\n\nkubectl -n \u003Coperator-namespace> logs -l app=cockroach-operator --tail=300 | grep -Ei 'decommission|drain|scale|blocking_ranges' || true\n```\n\nQuestions to answer:\n\n- What was the original and target node count?\n- Were multiple nodes scaled down at the same time?\n- Were manual decommission or drain commands issued?\n- Did any pods or PVCs get deleted manually?\n\n## Temporary Mitigations\n\nOnly use these after collecting evidence and confirming the risk with the customer or operator team.\n\nDisable reconciliation for one cluster:\n\n```bash\nkubectl -n \u003Ccockroachdb-namespace> patch crdbcluster \u003Ccockroachdb-release> --type=merge -p '{\"spec\":{\"mode\":\"Disabled\"}}'\n\n# Resume reconciliation:\nkubectl -n \u003Ccockroachdb-namespace> patch crdbcluster \u003Ccockroachdb-release> --type=merge -p '{\"spec\":{\"mode\":\"MutableOnly\"}}'\n```\n\nRestart the operator after evidence is collected:\n\n```bash\nkubectl -n \u003Coperator-namespace> rollout restart deploy\u002Fcockroach-operator\n```\n\nUser-approved timestamp rolling restart:\n\n```bash\nhelm -n \u003Ccockroachdb-namespace> upgrade \u003Ccockroachdb-release> \u003Ccockroachdb-chart> \\\n  --reuse-values \\\n  --set-string cockroachdb.crdbCluster.timestamp=\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\"\n```\n\n## Output Format\n\nReturn findings in this order:\n\n1. Failure class\n2. Evidence: exact command output or Kubernetes status field\n3. Root cause or most likely cause\n4. Minimal remediation\n5. Verification command\n6. Data\u002Favailability risk, if any\n7. Whether escalation packet collection is needed\n\n## References\n\n- [CockroachDB Helm Chart Versioning](..\u002F..\u002F..\u002Fcockroachdb-parent\u002Fdocs\u002FVERSIONING.md)\n- [Operator Helm Chart README](..\u002F..\u002F..\u002Fcockroachdb-parent\u002Fcharts\u002Foperator\u002FREADME.md)\n- [CockroachDB Helm Chart README](..\u002F..\u002F..\u002Fcockroachdb-parent\u002Fcharts\u002Fcockroachdb\u002FREADME.md)\n- [v1alpha1 to v1beta1 Migration Guide](..\u002F..\u002F..\u002Fcockroachdb-parent\u002FMIGRATION_v1alpha1_to_v1beta1.md)\n- [CockroachDB Docs: Kubernetes troubleshooting](https:\u002F\u002Fwww.cockroachlabs.com\u002Fdocs\u002Fstable\u002Forchestrate-cockroachdb-with-kubernetes)\n",{"data":37,"body":41},{"name":4,"description":6,"compatibility":38,"metadata":39},"CockroachDB Helm v2 charts and operator-managed crdb.cockroachlabs.com\u002Fv1beta1 resources. Requires Kubernetes read access to the operator namespace and CockroachDB namespace; some remediation requires cluster-admin or platform-team action.",{"author":8,"version":40},"1.1",{"type":42,"children":43},"root",[44,52,67,74,142,148,198,204,255,261,344,350,431,437,1064,1069,1306,1312,1625,1631,1828,1833,1856,1862,2077,2082,2125,2130,2207,2228,2351,2362,2368,2373,2383,2388,2401,2406,2488,2493,2527,2540,2546,2663,2668,2773,2785,2791,3022,3027,3071,3077,3436,3441,3459,3465,3904,3908,3947,3953,3958,4461,4471,4477,4487,4492,4857,4861,4927,4933,5152,5157,5193,5199,5448,5453,5476,5482,5487,5492,5661,5666,5714,5719,5830,5836,5841,5879,5885,5935],{"type":45,"tag":46,"props":47,"children":48},"element","h1",{"id":4},[49],{"type":50,"value":51},"text","Diagnosing CockroachDB Helm Deployments",{"type":45,"tag":53,"props":54,"children":55},"p",{},[56,58,65],{"type":50,"value":57},"Diagnoses CockroachDB Helm install, upgrade, and readiness failures for operator-managed clusters. Keep the flow customer-facing: collect read-only evidence, classify the failure, and propose the smallest safe remediation. If the issue needs TSC escalation or deep operator forensics, use ",{"type":45,"tag":59,"props":60,"children":62},"a",{"href":61},"..\u002Fcollecting-cockroachdb-operator-escalation-packet\u002FSKILL.md",[63],{"type":50,"value":64},"collecting-cockroachdb-operator-escalation-packet",{"type":50,"value":66},".",{"type":45,"tag":68,"props":69,"children":71},"h2",{"id":70},"when-to-use-this-skill",[72],{"type":50,"value":73},"When to Use This Skill",{"type":45,"tag":75,"props":76,"children":77},"ul",{},[78,99,116,127,132,137],{"type":45,"tag":79,"props":80,"children":81},"li",{},[82,89,91,97],{"type":45,"tag":83,"props":84,"children":86},"code",{"className":85},[],[87],{"type":50,"value":88},"helm install",{"type":50,"value":90}," or ",{"type":45,"tag":83,"props":92,"children":94},{"className":93},[],[95],{"type":50,"value":96},"helm upgrade",{"type":50,"value":98}," fails",{"type":45,"tag":79,"props":100,"children":101},{},[102,108,110],{"type":45,"tag":83,"props":103,"children":105},{"className":104},[],[106],{"type":50,"value":107},"CrdbCluster.status.observedGeneration",{"type":50,"value":109}," is behind ",{"type":45,"tag":83,"props":111,"children":113},{"className":112},[],[114],{"type":50,"value":115},"metadata.generation",{"type":45,"tag":79,"props":117,"children":118},{},[119,125],{"type":45,"tag":83,"props":120,"children":122},{"className":121},[],[123],{"type":50,"value":124},"CrdbCluster.status.reconciled",{"type":50,"value":126}," is false or missing",{"type":45,"tag":79,"props":128,"children":129},{},[130],{"type":50,"value":131},"CockroachDB pods are Pending, Init, CrashLoopBackOff, Running but not Ready, or stuck on an old image",{"type":45,"tag":79,"props":133,"children":134},{},[135],{"type":50,"value":136},"The operator Deployment is unavailable, silent, or not reconciling",{"type":45,"tag":79,"props":138,"children":139},{},[140],{"type":50,"value":141},"Errors mention RBAC, CRDs, TLS, certificates, webhooks, node locality, PVCs, DNS, upgrades, scale operations, decommissioning, or multi-region networking",{"type":45,"tag":68,"props":143,"children":145},{"id":144},"related-skills",[146],{"type":50,"value":147},"Related Skills",{"type":45,"tag":75,"props":149,"children":150},{},[151,162,174,186],{"type":45,"tag":79,"props":152,"children":153},{},[154,156,160],{"type":50,"value":155},"Use ",{"type":45,"tag":59,"props":157,"children":158},{"href":61},[159],{"type":50,"value":64},{"type":50,"value":161}," when the customer needs to escalate or when TSC asks for a complete data bundle.",{"type":45,"tag":79,"props":163,"children":164},{},[165,166,172],{"type":50,"value":155},{"type":45,"tag":59,"props":167,"children":169},{"href":168},"..\u002F..\u002Fcockroachdb-onboarding-and-migrations\u002Fdebugging-cockroachdb-operator-migrations\u002FSKILL.md",[170],{"type":50,"value":171},"debugging-cockroachdb-operator-migrations",{"type":50,"value":173}," for Helm StatefulSet or public operator migration problems.",{"type":45,"tag":79,"props":175,"children":176},{},[177,178,184],{"type":50,"value":155},{"type":45,"tag":59,"props":179,"children":181},{"href":180},"..\u002F..\u002Fcockroachdb-operations-and-lifecycle\u002Fconfiguring-cockroachdb-helm-tls\u002FSKILL.md",[182],{"type":50,"value":183},"configuring-cockroachdb-helm-tls",{"type":50,"value":185}," for TLS provider selection and certificate validation.",{"type":45,"tag":79,"props":187,"children":188},{},[189,190,196],{"type":50,"value":155},{"type":45,"tag":59,"props":191,"children":193},{"href":192},"..\u002F..\u002Fcockroachdb-onboarding-and-migrations\u002Fvalidating-cockroachdb-helm-multiregion\u002FSKILL.md",[194],{"type":50,"value":195},"validating-cockroachdb-helm-multiregion",{"type":50,"value":197}," for cross-region DNS, networking, and certificate checks.",{"type":45,"tag":68,"props":199,"children":201},{"id":200},"inputs",[202],{"type":50,"value":203},"Inputs",{"type":45,"tag":75,"props":205,"children":206},{},[207,212,217,222,235,240,245,250],{"type":45,"tag":79,"props":208,"children":209},{},[210],{"type":50,"value":211},"Exact failed command and stderr",{"type":45,"tag":79,"props":213,"children":214},{},[215],{"type":50,"value":216},"Operator namespace, operator Helm release name, and operator chart version",{"type":45,"tag":79,"props":218,"children":219},{},[220],{"type":50,"value":221},"CockroachDB namespace, CockroachDB Helm release name, and CockroachDB chart version",{"type":45,"tag":79,"props":223,"children":224},{},[225,227,233],{"type":50,"value":226},"Values file or ",{"type":45,"tag":83,"props":228,"children":230},{"className":229},[],[231],{"type":50,"value":232},"--set",{"type":50,"value":234}," values used",{"type":45,"tag":79,"props":236,"children":237},{},[238],{"type":50,"value":239},"Kubernetes context and Kubernetes version",{"type":45,"tag":79,"props":241,"children":242},{},[243],{"type":50,"value":244},"CockroachDB image\u002Fversion and operator image\u002Fversion",{"type":45,"tag":79,"props":246,"children":247},{},[248],{"type":50,"value":249},"Whether this is install, upgrade, scale up\u002Fdown, certificate rotation, migration, maintenance, or recovery",{"type":45,"tag":79,"props":251,"children":252},{},[253],{"type":50,"value":254},"What the customer already tried, in order",{"type":45,"tag":68,"props":256,"children":258},{"id":257},"safety-considerations",[259],{"type":50,"value":260},"Safety Considerations",{"type":45,"tag":75,"props":262,"children":263},{},[264,269,290,303,316,329,334,339],{"type":45,"tag":79,"props":265,"children":266},{},[267],{"type":50,"value":268},"Collect evidence before retrying. Repeated Helm or kubectl attempts can obscure the original failure.",{"type":45,"tag":79,"props":270,"children":271},{},[272,274,280,282,288],{"type":50,"value":273},"Do not delete PVCs, Secrets, ",{"type":45,"tag":83,"props":275,"children":277},{"className":276},[],[278],{"type":50,"value":279},"CrdbCluster",{"type":50,"value":281},", or ",{"type":45,"tag":83,"props":283,"children":285},{"className":284},[],[286],{"type":50,"value":287},"CrdbNode",{"type":50,"value":289}," resources unless the user explicitly asks for teardown and understands data impact.",{"type":45,"tag":79,"props":291,"children":292},{},[293,295,301],{"type":50,"value":294},"Do not manually edit ",{"type":45,"tag":83,"props":296,"children":298},{"className":297},[],[299],{"type":50,"value":300},"CrdbCluster.status",{"type":50,"value":302},"; the operator owns status.",{"type":45,"tag":79,"props":304,"children":305},{},[306,308,314],{"type":50,"value":307},"Do not run ",{"type":45,"tag":83,"props":309,"children":311},{"className":310},[],[312],{"type":50,"value":313},"cockroach init",{"type":50,"value":315}," on an existing cluster.",{"type":45,"tag":79,"props":317,"children":318},{},[319,321,327],{"type":50,"value":320},"Do not change service settings such as ",{"type":45,"tag":83,"props":322,"children":324},{"className":323},[],[325],{"type":50,"value":326},"publishNotReadyAddresses",{"type":50,"value":328}," without operator-team guidance.",{"type":45,"tag":79,"props":330,"children":331},{},[332],{"type":50,"value":333},"Do not delete version checker jobs or pods during an upgrade until their status and logs are collected.",{"type":45,"tag":79,"props":335,"children":336},{},[337],{"type":50,"value":338},"Do not restart or scale down the operator during an active rollout, decommission, or migration unless basic evidence has already been collected.",{"type":45,"tag":79,"props":340,"children":341},{},[342],{"type":50,"value":343},"Do not uninstall the operator before checking whether it manages other namespaces or clusters.",{"type":45,"tag":68,"props":345,"children":347},{"id":346},"execution-discipline",[348],{"type":50,"value":349},"Execution Discipline",{"type":45,"tag":75,"props":351,"children":352},{},[353,358,363,426],{"type":45,"tag":79,"props":354,"children":355},{},[356],{"type":50,"value":357},"Execute one step at a time and inspect the output before moving on. Do not run whole sections, unrelated command groups, or later diagnostic branches in parallel; earlier output determines which later checks are relevant.",{"type":45,"tag":79,"props":359,"children":360},{},[361],{"type":50,"value":362},"Treat commands as templates. Substitute namespaces, release names, chart names, and pod names deliberately before running anything.",{"type":45,"tag":79,"props":364,"children":365},{},[366,368,374,376,382,383,389,390,396,397,403,404,409,411,417,418,424],{"type":50,"value":367},"Do not run any mutating command unless the user explicitly approves it for the target cluster. This includes ",{"type":45,"tag":83,"props":369,"children":371},{"className":370},[],[372],{"type":50,"value":373},"kubectl patch",{"type":50,"value":375},", ",{"type":45,"tag":83,"props":377,"children":379},{"className":378},[],[380],{"type":50,"value":381},"kubectl annotate",{"type":50,"value":375},{"type":45,"tag":83,"props":384,"children":386},{"className":385},[],[387],{"type":50,"value":388},"kubectl delete",{"type":50,"value":375},{"type":45,"tag":83,"props":391,"children":393},{"className":392},[],[394],{"type":50,"value":395},"kubectl scale",{"type":50,"value":375},{"type":45,"tag":83,"props":398,"children":400},{"className":399},[],[401],{"type":50,"value":402},"kubectl rollout restart",{"type":50,"value":375},{"type":45,"tag":83,"props":405,"children":407},{"className":406},[],[408],{"type":50,"value":96},{"type":50,"value":410},", drain\u002Fdecommission commands, and interactive ",{"type":45,"tag":83,"props":412,"children":414},{"className":413},[],[415],{"type":50,"value":416},"kubectl exec",{"type":50,"value":90},{"type":45,"tag":83,"props":419,"children":421},{"className":420},[],[422],{"type":50,"value":423},"kubectl debug",{"type":50,"value":425}," shells.",{"type":45,"tag":79,"props":427,"children":428},{},[429],{"type":50,"value":430},"In production or whenever the impact is unclear, stop and escalate to TSE or the operator team before pprof\u002Fmetrics collection, debug containers, timestamp-based rolling restarts, mode changes, operator restarts, scale changes, or decommission actions.",{"type":45,"tag":68,"props":432,"children":434},{"id":433},"step-1-collect-baseline-state",[435],{"type":50,"value":436},"Step 1: Collect Baseline State",{"type":45,"tag":438,"props":439,"children":444},"pre",{"className":440,"code":441,"language":442,"meta":443,"style":443},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","# Helm release state\nhelm -n \u003Coperator-namespace> status \u003Coperator-release> || true\nhelm -n \u003Ccockroachdb-namespace> status \u003Ccockroachdb-release> || true\nhelm -n \u003Coperator-namespace> history \u003Coperator-release> || true\nhelm -n \u003Ccockroachdb-namespace> history \u003Ccockroachdb-release> || true\n\n# Operator state\nkubectl -n \u003Coperator-namespace> get deploy,pod,svc -o wide | grep -E 'cockroach-operator|NAME'\nkubectl -n \u003Coperator-namespace> logs -l app=cockroach-operator --tail=200 || true\n\n# CRD and CockroachDB resources\nkubectl get crd crdbclusters.crdb.cockroachlabs.com crdbnodes.crdb.cockroachlabs.com\nkubectl -n \u003Ccockroachdb-namespace> get crdbcluster,crdbnode,pod,svc,endpoints,pvc,pdb -o wide\nkubectl -n \u003Ccockroachdb-namespace> describe crdbcluster \u003Ccockroachdb-release> || true\nkubectl -n \u003Ccockroachdb-namespace> get events --sort-by=.lastTimestamp | tail -50\n","bash","",[445],{"type":45,"tag":83,"props":446,"children":447},{"__ignoreMap":443},[448,460,531,589,646,702,712,721,800,856,864,873,900,946,1008],{"type":45,"tag":449,"props":450,"children":453},"span",{"class":451,"line":452},"line",1,[454],{"type":45,"tag":449,"props":455,"children":457},{"style":456},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[458],{"type":50,"value":459},"# Helm release state\n",{"type":45,"tag":449,"props":461,"children":463},{"class":451,"line":462},2,[464,470,476,482,487,493,498,503,507,512,516,520,525],{"type":45,"tag":449,"props":465,"children":467},{"style":466},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[468],{"type":50,"value":469},"helm",{"type":45,"tag":449,"props":471,"children":473},{"style":472},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[474],{"type":50,"value":475}," -n",{"type":45,"tag":449,"props":477,"children":479},{"style":478},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[480],{"type":50,"value":481}," \u003C",{"type":45,"tag":449,"props":483,"children":484},{"style":472},[485],{"type":50,"value":486},"operator-namespac",{"type":45,"tag":449,"props":488,"children":490},{"style":489},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[491],{"type":50,"value":492},"e",{"type":45,"tag":449,"props":494,"children":495},{"style":478},[496],{"type":50,"value":497},">",{"type":45,"tag":449,"props":499,"children":500},{"style":472},[501],{"type":50,"value":502}," status",{"type":45,"tag":449,"props":504,"children":505},{"style":478},[506],{"type":50,"value":481},{"type":45,"tag":449,"props":508,"children":509},{"style":472},[510],{"type":50,"value":511},"operator-releas",{"type":45,"tag":449,"props":513,"children":514},{"style":489},[515],{"type":50,"value":492},{"type":45,"tag":449,"props":517,"children":518},{"style":478},[519],{"type":50,"value":497},{"type":45,"tag":449,"props":521,"children":522},{"style":478},[523],{"type":50,"value":524}," ||",{"type":45,"tag":449,"props":526,"children":528},{"style":527},"--shiki-light:#6182B8;--shiki-default:#82AAFF;--shiki-dark:#82AAFF",[529],{"type":50,"value":530}," true\n",{"type":45,"tag":449,"props":532,"children":534},{"class":451,"line":533},3,[535,539,543,547,552,556,560,564,568,573,577,581,585],{"type":45,"tag":449,"props":536,"children":537},{"style":466},[538],{"type":50,"value":469},{"type":45,"tag":449,"props":540,"children":541},{"style":472},[542],{"type":50,"value":475},{"type":45,"tag":449,"props":544,"children":545},{"style":478},[546],{"type":50,"value":481},{"type":45,"tag":449,"props":548,"children":549},{"style":472},[550],{"type":50,"value":551},"cockroachdb-namespac",{"type":45,"tag":449,"props":553,"children":554},{"style":489},[555],{"type":50,"value":492},{"type":45,"tag":449,"props":557,"children":558},{"style":478},[559],{"type":50,"value":497},{"type":45,"tag":449,"props":561,"children":562},{"style":472},[563],{"type":50,"value":502},{"type":45,"tag":449,"props":565,"children":566},{"style":478},[567],{"type":50,"value":481},{"type":45,"tag":449,"props":569,"children":570},{"style":472},[571],{"type":50,"value":572},"cockroachdb-releas",{"type":45,"tag":449,"props":574,"children":575},{"style":489},[576],{"type":50,"value":492},{"type":45,"tag":449,"props":578,"children":579},{"style":478},[580],{"type":50,"value":497},{"type":45,"tag":449,"props":582,"children":583},{"style":478},[584],{"type":50,"value":524},{"type":45,"tag":449,"props":586,"children":587},{"style":527},[588],{"type":50,"value":530},{"type":45,"tag":449,"props":590,"children":592},{"class":451,"line":591},4,[593,597,601,605,609,613,617,622,626,630,634,638,642],{"type":45,"tag":449,"props":594,"children":595},{"style":466},[596],{"type":50,"value":469},{"type":45,"tag":449,"props":598,"children":599},{"style":472},[600],{"type":50,"value":475},{"type":45,"tag":449,"props":602,"children":603},{"style":478},[604],{"type":50,"value":481},{"type":45,"tag":449,"props":606,"children":607},{"style":472},[608],{"type":50,"value":486},{"type":45,"tag":449,"props":610,"children":611},{"style":489},[612],{"type":50,"value":492},{"type":45,"tag":449,"props":614,"children":615},{"style":478},[616],{"type":50,"value":497},{"type":45,"tag":449,"props":618,"children":619},{"style":472},[620],{"type":50,"value":621}," history",{"type":45,"tag":449,"props":623,"children":624},{"style":478},[625],{"type":50,"value":481},{"type":45,"tag":449,"props":627,"children":628},{"style":472},[629],{"type":50,"value":511},{"type":45,"tag":449,"props":631,"children":632},{"style":489},[633],{"type":50,"value":492},{"type":45,"tag":449,"props":635,"children":636},{"style":478},[637],{"type":50,"value":497},{"type":45,"tag":449,"props":639,"children":640},{"style":478},[641],{"type":50,"value":524},{"type":45,"tag":449,"props":643,"children":644},{"style":527},[645],{"type":50,"value":530},{"type":45,"tag":449,"props":647,"children":649},{"class":451,"line":648},5,[650,654,658,662,666,670,674,678,682,686,690,694,698],{"type":45,"tag":449,"props":651,"children":652},{"style":466},[653],{"type":50,"value":469},{"type":45,"tag":449,"props":655,"children":656},{"style":472},[657],{"type":50,"value":475},{"type":45,"tag":449,"props":659,"children":660},{"style":478},[661],{"type":50,"value":481},{"type":45,"tag":449,"props":663,"children":664},{"style":472},[665],{"type":50,"value":551},{"type":45,"tag":449,"props":667,"children":668},{"style":489},[669],{"type":50,"value":492},{"type":45,"tag":449,"props":671,"children":672},{"style":478},[673],{"type":50,"value":497},{"type":45,"tag":449,"props":675,"children":676},{"style":472},[677],{"type":50,"value":621},{"type":45,"tag":449,"props":679,"children":680},{"style":478},[681],{"type":50,"value":481},{"type":45,"tag":449,"props":683,"children":684},{"style":472},[685],{"type":50,"value":572},{"type":45,"tag":449,"props":687,"children":688},{"style":489},[689],{"type":50,"value":492},{"type":45,"tag":449,"props":691,"children":692},{"style":478},[693],{"type":50,"value":497},{"type":45,"tag":449,"props":695,"children":696},{"style":478},[697],{"type":50,"value":524},{"type":45,"tag":449,"props":699,"children":700},{"style":527},[701],{"type":50,"value":530},{"type":45,"tag":449,"props":703,"children":705},{"class":451,"line":704},6,[706],{"type":45,"tag":449,"props":707,"children":709},{"emptyLinePlaceholder":708},true,[710],{"type":50,"value":711},"\n",{"type":45,"tag":449,"props":713,"children":715},{"class":451,"line":714},7,[716],{"type":45,"tag":449,"props":717,"children":718},{"style":456},[719],{"type":50,"value":720},"# Operator state\n",{"type":45,"tag":449,"props":722,"children":724},{"class":451,"line":723},8,[725,730,734,738,742,746,750,755,760,765,770,775,780,785,790,795],{"type":45,"tag":449,"props":726,"children":727},{"style":466},[728],{"type":50,"value":729},"kubectl",{"type":45,"tag":449,"props":731,"children":732},{"style":472},[733],{"type":50,"value":475},{"type":45,"tag":449,"props":735,"children":736},{"style":478},[737],{"type":50,"value":481},{"type":45,"tag":449,"props":739,"children":740},{"style":472},[741],{"type":50,"value":486},{"type":45,"tag":449,"props":743,"children":744},{"style":489},[745],{"type":50,"value":492},{"type":45,"tag":449,"props":747,"children":748},{"style":478},[749],{"type":50,"value":497},{"type":45,"tag":449,"props":751,"children":752},{"style":472},[753],{"type":50,"value":754}," get",{"type":45,"tag":449,"props":756,"children":757},{"style":472},[758],{"type":50,"value":759}," deploy,pod,svc",{"type":45,"tag":449,"props":761,"children":762},{"style":472},[763],{"type":50,"value":764}," -o",{"type":45,"tag":449,"props":766,"children":767},{"style":472},[768],{"type":50,"value":769}," wide",{"type":45,"tag":449,"props":771,"children":772},{"style":478},[773],{"type":50,"value":774}," |",{"type":45,"tag":449,"props":776,"children":777},{"style":466},[778],{"type":50,"value":779}," grep",{"type":45,"tag":449,"props":781,"children":782},{"style":472},[783],{"type":50,"value":784}," -E",{"type":45,"tag":449,"props":786,"children":787},{"style":478},[788],{"type":50,"value":789}," '",{"type":45,"tag":449,"props":791,"children":792},{"style":472},[793],{"type":50,"value":794},"cockroach-operator|NAME",{"type":45,"tag":449,"props":796,"children":797},{"style":478},[798],{"type":50,"value":799},"'\n",{"type":45,"tag":449,"props":801,"children":803},{"class":451,"line":802},9,[804,808,812,816,820,824,828,833,838,843,848,852],{"type":45,"tag":449,"props":805,"children":806},{"style":466},[807],{"type":50,"value":729},{"type":45,"tag":449,"props":809,"children":810},{"style":472},[811],{"type":50,"value":475},{"type":45,"tag":449,"props":813,"children":814},{"style":478},[815],{"type":50,"value":481},{"type":45,"tag":449,"props":817,"children":818},{"style":472},[819],{"type":50,"value":486},{"type":45,"tag":449,"props":821,"children":822},{"style":489},[823],{"type":50,"value":492},{"type":45,"tag":449,"props":825,"children":826},{"style":478},[827],{"type":50,"value":497},{"type":45,"tag":449,"props":829,"children":830},{"style":472},[831],{"type":50,"value":832}," logs",{"type":45,"tag":449,"props":834,"children":835},{"style":472},[836],{"type":50,"value":837}," -l",{"type":45,"tag":449,"props":839,"children":840},{"style":472},[841],{"type":50,"value":842}," app=cockroach-operator",{"type":45,"tag":449,"props":844,"children":845},{"style":472},[846],{"type":50,"value":847}," --tail=200",{"type":45,"tag":449,"props":849,"children":850},{"style":478},[851],{"type":50,"value":524},{"type":45,"tag":449,"props":853,"children":854},{"style":527},[855],{"type":50,"value":530},{"type":45,"tag":449,"props":857,"children":859},{"class":451,"line":858},10,[860],{"type":45,"tag":449,"props":861,"children":862},{"emptyLinePlaceholder":708},[863],{"type":50,"value":711},{"type":45,"tag":449,"props":865,"children":867},{"class":451,"line":866},11,[868],{"type":45,"tag":449,"props":869,"children":870},{"style":456},[871],{"type":50,"value":872},"# CRD and CockroachDB resources\n",{"type":45,"tag":449,"props":874,"children":876},{"class":451,"line":875},12,[877,881,885,890,895],{"type":45,"tag":449,"props":878,"children":879},{"style":466},[880],{"type":50,"value":729},{"type":45,"tag":449,"props":882,"children":883},{"style":472},[884],{"type":50,"value":754},{"type":45,"tag":449,"props":886,"children":887},{"style":472},[888],{"type":50,"value":889}," crd",{"type":45,"tag":449,"props":891,"children":892},{"style":472},[893],{"type":50,"value":894}," crdbclusters.crdb.cockroachlabs.com",{"type":45,"tag":449,"props":896,"children":897},{"style":472},[898],{"type":50,"value":899}," crdbnodes.crdb.cockroachlabs.com\n",{"type":45,"tag":449,"props":901,"children":903},{"class":451,"line":902},13,[904,908,912,916,920,924,928,932,937,941],{"type":45,"tag":449,"props":905,"children":906},{"style":466},[907],{"type":50,"value":729},{"type":45,"tag":449,"props":909,"children":910},{"style":472},[911],{"type":50,"value":475},{"type":45,"tag":449,"props":913,"children":914},{"style":478},[915],{"type":50,"value":481},{"type":45,"tag":449,"props":917,"children":918},{"style":472},[919],{"type":50,"value":551},{"type":45,"tag":449,"props":921,"children":922},{"style":489},[923],{"type":50,"value":492},{"type":45,"tag":449,"props":925,"children":926},{"style":478},[927],{"type":50,"value":497},{"type":45,"tag":449,"props":929,"children":930},{"style":472},[931],{"type":50,"value":754},{"type":45,"tag":449,"props":933,"children":934},{"style":472},[935],{"type":50,"value":936}," crdbcluster,crdbnode,pod,svc,endpoints,pvc,pdb",{"type":45,"tag":449,"props":938,"children":939},{"style":472},[940],{"type":50,"value":764},{"type":45,"tag":449,"props":942,"children":943},{"style":472},[944],{"type":50,"value":945}," wide\n",{"type":45,"tag":449,"props":947,"children":949},{"class":451,"line":948},14,[950,954,958,962,966,970,974,979,984,988,992,996,1000,1004],{"type":45,"tag":449,"props":951,"children":952},{"style":466},[953],{"type":50,"value":729},{"type":45,"tag":449,"props":955,"children":956},{"style":472},[957],{"type":50,"value":475},{"type":45,"tag":449,"props":959,"children":960},{"style":478},[961],{"type":50,"value":481},{"type":45,"tag":449,"props":963,"children":964},{"style":472},[965],{"type":50,"value":551},{"type":45,"tag":449,"props":967,"children":968},{"style":489},[969],{"type":50,"value":492},{"type":45,"tag":449,"props":971,"children":972},{"style":478},[973],{"type":50,"value":497},{"type":45,"tag":449,"props":975,"children":976},{"style":472},[977],{"type":50,"value":978}," describe",{"type":45,"tag":449,"props":980,"children":981},{"style":472},[982],{"type":50,"value":983}," crdbcluster",{"type":45,"tag":449,"props":985,"children":986},{"style":478},[987],{"type":50,"value":481},{"type":45,"tag":449,"props":989,"children":990},{"style":472},[991],{"type":50,"value":572},{"type":45,"tag":449,"props":993,"children":994},{"style":489},[995],{"type":50,"value":492},{"type":45,"tag":449,"props":997,"children":998},{"style":478},[999],{"type":50,"value":497},{"type":45,"tag":449,"props":1001,"children":1002},{"style":478},[1003],{"type":50,"value":524},{"type":45,"tag":449,"props":1005,"children":1006},{"style":527},[1007],{"type":50,"value":530},{"type":45,"tag":449,"props":1009,"children":1011},{"class":451,"line":1010},15,[1012,1016,1020,1024,1028,1032,1036,1040,1045,1050,1054,1059],{"type":45,"tag":449,"props":1013,"children":1014},{"style":466},[1015],{"type":50,"value":729},{"type":45,"tag":449,"props":1017,"children":1018},{"style":472},[1019],{"type":50,"value":475},{"type":45,"tag":449,"props":1021,"children":1022},{"style":478},[1023],{"type":50,"value":481},{"type":45,"tag":449,"props":1025,"children":1026},{"style":472},[1027],{"type":50,"value":551},{"type":45,"tag":449,"props":1029,"children":1030},{"style":489},[1031],{"type":50,"value":492},{"type":45,"tag":449,"props":1033,"children":1034},{"style":478},[1035],{"type":50,"value":497},{"type":45,"tag":449,"props":1037,"children":1038},{"style":472},[1039],{"type":50,"value":754},{"type":45,"tag":449,"props":1041,"children":1042},{"style":472},[1043],{"type":50,"value":1044}," events",{"type":45,"tag":449,"props":1046,"children":1047},{"style":472},[1048],{"type":50,"value":1049}," --sort-by=.lastTimestamp",{"type":45,"tag":449,"props":1051,"children":1052},{"style":478},[1053],{"type":50,"value":774},{"type":45,"tag":449,"props":1055,"children":1056},{"style":466},[1057],{"type":50,"value":1058}," tail",{"type":45,"tag":449,"props":1060,"children":1061},{"style":472},[1062],{"type":50,"value":1063}," -50\n",{"type":45,"tag":53,"props":1065,"children":1066},{},[1067],{"type":50,"value":1068},"For a stuck pod or node:",{"type":45,"tag":438,"props":1070,"children":1072},{"className":440,"code":1071,"language":442,"meta":443,"style":443},"kubectl -n \u003Ccockroachdb-namespace> describe pod \u003Ccrdb-pod>\nkubectl -n \u003Ccockroachdb-namespace> logs \u003Ccrdb-pod> -c cockroachdb --tail=200\nkubectl -n \u003Ccockroachdb-namespace> logs \u003Ccrdb-pod> -c cockroachdb --previous\nkubectl -n \u003Ccockroachdb-namespace> describe crdbnode \u003Ccrdbnode-name>\n",[1073],{"type":45,"tag":83,"props":1074,"children":1075},{"__ignoreMap":443},[1076,1131,1193,1253],{"type":45,"tag":449,"props":1077,"children":1078},{"class":451,"line":452},[1079,1083,1087,1091,1095,1099,1103,1107,1112,1116,1121,1126],{"type":45,"tag":449,"props":1080,"children":1081},{"style":466},[1082],{"type":50,"value":729},{"type":45,"tag":449,"props":1084,"children":1085},{"style":472},[1086],{"type":50,"value":475},{"type":45,"tag":449,"props":1088,"children":1089},{"style":478},[1090],{"type":50,"value":481},{"type":45,"tag":449,"props":1092,"children":1093},{"style":472},[1094],{"type":50,"value":551},{"type":45,"tag":449,"props":1096,"children":1097},{"style":489},[1098],{"type":50,"value":492},{"type":45,"tag":449,"props":1100,"children":1101},{"style":478},[1102],{"type":50,"value":497},{"type":45,"tag":449,"props":1104,"children":1105},{"style":472},[1106],{"type":50,"value":978},{"type":45,"tag":449,"props":1108,"children":1109},{"style":472},[1110],{"type":50,"value":1111}," pod",{"type":45,"tag":449,"props":1113,"children":1114},{"style":478},[1115],{"type":50,"value":481},{"type":45,"tag":449,"props":1117,"children":1118},{"style":472},[1119],{"type":50,"value":1120},"crdb-po",{"type":45,"tag":449,"props":1122,"children":1123},{"style":489},[1124],{"type":50,"value":1125},"d",{"type":45,"tag":449,"props":1127,"children":1128},{"style":478},[1129],{"type":50,"value":1130},">\n",{"type":45,"tag":449,"props":1132,"children":1133},{"class":451,"line":462},[1134,1138,1142,1146,1150,1154,1158,1162,1166,1170,1174,1178,1183,1188],{"type":45,"tag":449,"props":1135,"children":1136},{"style":466},[1137],{"type":50,"value":729},{"type":45,"tag":449,"props":1139,"children":1140},{"style":472},[1141],{"type":50,"value":475},{"type":45,"tag":449,"props":1143,"children":1144},{"style":478},[1145],{"type":50,"value":481},{"type":45,"tag":449,"props":1147,"children":1148},{"style":472},[1149],{"type":50,"value":551},{"type":45,"tag":449,"props":1151,"children":1152},{"style":489},[1153],{"type":50,"value":492},{"type":45,"tag":449,"props":1155,"children":1156},{"style":478},[1157],{"type":50,"value":497},{"type":45,"tag":449,"props":1159,"children":1160},{"style":472},[1161],{"type":50,"value":832},{"type":45,"tag":449,"props":1163,"children":1164},{"style":478},[1165],{"type":50,"value":481},{"type":45,"tag":449,"props":1167,"children":1168},{"style":472},[1169],{"type":50,"value":1120},{"type":45,"tag":449,"props":1171,"children":1172},{"style":489},[1173],{"type":50,"value":1125},{"type":45,"tag":449,"props":1175,"children":1176},{"style":478},[1177],{"type":50,"value":497},{"type":45,"tag":449,"props":1179,"children":1180},{"style":472},[1181],{"type":50,"value":1182}," -c",{"type":45,"tag":449,"props":1184,"children":1185},{"style":472},[1186],{"type":50,"value":1187}," cockroachdb",{"type":45,"tag":449,"props":1189,"children":1190},{"style":472},[1191],{"type":50,"value":1192}," --tail=200\n",{"type":45,"tag":449,"props":1194,"children":1195},{"class":451,"line":533},[1196,1200,1204,1208,1212,1216,1220,1224,1228,1232,1236,1240,1244,1248],{"type":45,"tag":449,"props":1197,"children":1198},{"style":466},[1199],{"type":50,"value":729},{"type":45,"tag":449,"props":1201,"children":1202},{"style":472},[1203],{"type":50,"value":475},{"type":45,"tag":449,"props":1205,"children":1206},{"style":478},[1207],{"type":50,"value":481},{"type":45,"tag":449,"props":1209,"children":1210},{"style":472},[1211],{"type":50,"value":551},{"type":45,"tag":449,"props":1213,"children":1214},{"style":489},[1215],{"type":50,"value":492},{"type":45,"tag":449,"props":1217,"children":1218},{"style":478},[1219],{"type":50,"value":497},{"type":45,"tag":449,"props":1221,"children":1222},{"style":472},[1223],{"type":50,"value":832},{"type":45,"tag":449,"props":1225,"children":1226},{"style":478},[1227],{"type":50,"value":481},{"type":45,"tag":449,"props":1229,"children":1230},{"style":472},[1231],{"type":50,"value":1120},{"type":45,"tag":449,"props":1233,"children":1234},{"style":489},[1235],{"type":50,"value":1125},{"type":45,"tag":449,"props":1237,"children":1238},{"style":478},[1239],{"type":50,"value":497},{"type":45,"tag":449,"props":1241,"children":1242},{"style":472},[1243],{"type":50,"value":1182},{"type":45,"tag":449,"props":1245,"children":1246},{"style":472},[1247],{"type":50,"value":1187},{"type":45,"tag":449,"props":1249,"children":1250},{"style":472},[1251],{"type":50,"value":1252}," --previous\n",{"type":45,"tag":449,"props":1254,"children":1255},{"class":451,"line":591},[1256,1260,1264,1268,1272,1276,1280,1284,1289,1293,1298,1302],{"type":45,"tag":449,"props":1257,"children":1258},{"style":466},[1259],{"type":50,"value":729},{"type":45,"tag":449,"props":1261,"children":1262},{"style":472},[1263],{"type":50,"value":475},{"type":45,"tag":449,"props":1265,"children":1266},{"style":478},[1267],{"type":50,"value":481},{"type":45,"tag":449,"props":1269,"children":1270},{"style":472},[1271],{"type":50,"value":551},{"type":45,"tag":449,"props":1273,"children":1274},{"style":489},[1275],{"type":50,"value":492},{"type":45,"tag":449,"props":1277,"children":1278},{"style":478},[1279],{"type":50,"value":497},{"type":45,"tag":449,"props":1281,"children":1282},{"style":472},[1283],{"type":50,"value":978},{"type":45,"tag":449,"props":1285,"children":1286},{"style":472},[1287],{"type":50,"value":1288}," crdbnode",{"type":45,"tag":449,"props":1290,"children":1291},{"style":478},[1292],{"type":50,"value":481},{"type":45,"tag":449,"props":1294,"children":1295},{"style":472},[1296],{"type":50,"value":1297},"crdbnode-nam",{"type":45,"tag":449,"props":1299,"children":1300},{"style":489},[1301],{"type":50,"value":492},{"type":45,"tag":449,"props":1303,"children":1304},{"style":478},[1305],{"type":50,"value":1130},{"type":45,"tag":68,"props":1307,"children":1309},{"id":1308},"step-2-classify-the-failure",[1310],{"type":50,"value":1311},"Step 2: Classify the Failure",{"type":45,"tag":1313,"props":1314,"children":1315},"table",{},[1316,1340],{"type":45,"tag":1317,"props":1318,"children":1319},"thead",{},[1320],{"type":45,"tag":1321,"props":1322,"children":1323},"tr",{},[1324,1330,1335],{"type":45,"tag":1325,"props":1326,"children":1327},"th",{},[1328],{"type":50,"value":1329},"Symptom",{"type":45,"tag":1325,"props":1331,"children":1332},{},[1333],{"type":50,"value":1334},"Likely Class",{"type":45,"tag":1325,"props":1336,"children":1337},{},[1338],{"type":50,"value":1339},"Next Check",{"type":45,"tag":1341,"props":1342,"children":1343},"tbody",{},[1344,1371,1397,1419,1441,1461,1495,1517,1539,1561,1583,1605],{"type":45,"tag":1321,"props":1345,"children":1346},{},[1347,1357,1362],{"type":45,"tag":1348,"props":1349,"children":1350},"td",{},[1351],{"type":45,"tag":83,"props":1352,"children":1354},{"className":1353},[],[1355],{"type":50,"value":1356},"no matches for kind \"CrdbCluster\"",{"type":45,"tag":1348,"props":1358,"children":1359},{},[1360],{"type":50,"value":1361},"CRDs\u002Foperator not installed or not ready",{"type":45,"tag":1348,"props":1363,"children":1364},{},[1365],{"type":45,"tag":59,"props":1366,"children":1368},{"href":1367},"#crd-and-operator-readiness",[1369],{"type":50,"value":1370},"CRD and operator readiness",{"type":45,"tag":1321,"props":1372,"children":1373},{},[1374,1383,1388],{"type":45,"tag":1348,"props":1375,"children":1376},{},[1377],{"type":45,"tag":83,"props":1378,"children":1380},{"className":1379},[],[1381],{"type":50,"value":1382},"attempt to grant extra privileges",{"type":45,"tag":1348,"props":1384,"children":1385},{},[1386],{"type":50,"value":1387},"Helm RBAC restriction",{"type":45,"tag":1348,"props":1389,"children":1390},{},[1391],{"type":45,"tag":59,"props":1392,"children":1394},{"href":1393},"#rbac-and-node-reader-failures",[1395],{"type":50,"value":1396},"RBAC and node-reader failures",{"type":45,"tag":1321,"props":1398,"children":1399},{},[1400,1405,1410],{"type":45,"tag":1348,"props":1401,"children":1402},{},[1403],{"type":50,"value":1404},"TLS values validation error",{"type":45,"tag":1348,"props":1406,"children":1407},{},[1408],{"type":50,"value":1409},"TLS provider conflict",{"type":45,"tag":1348,"props":1411,"children":1412},{},[1413],{"type":45,"tag":59,"props":1414,"children":1416},{"href":1415},"#tls-and-certificate-failures",[1417],{"type":50,"value":1418},"TLS and certificate failures",{"type":45,"tag":1321,"props":1420,"children":1421},{},[1422,1427,1432],{"type":45,"tag":1348,"props":1423,"children":1424},{},[1425],{"type":50,"value":1426},"Operator pod CrashLoopBackOff or OOMKilled",{"type":45,"tag":1348,"props":1428,"children":1429},{},[1430],{"type":50,"value":1431},"Operator crash or resource limit",{"type":45,"tag":1348,"props":1433,"children":1434},{},[1435],{"type":45,"tag":59,"props":1436,"children":1438},{"href":1437},"#operator-health",[1439],{"type":50,"value":1440},"Operator health",{"type":45,"tag":1321,"props":1442,"children":1443},{},[1444,1449,1454],{"type":45,"tag":1348,"props":1445,"children":1446},{},[1447],{"type":50,"value":1448},"Operator running but no reconcile logs",{"type":45,"tag":1348,"props":1450,"children":1451},{},[1452],{"type":50,"value":1453},"Watch namespace mismatch or blocked worker",{"type":45,"tag":1348,"props":1455,"children":1456},{},[1457],{"type":45,"tag":59,"props":1458,"children":1459},{"href":1437},[1460],{"type":50,"value":1440},{"type":45,"tag":1321,"props":1462,"children":1463},{},[1464,1481,1486],{"type":45,"tag":1348,"props":1465,"children":1466},{},[1467,1473,1475],{"type":45,"tag":83,"props":1468,"children":1470},{"className":1469},[],[1471],{"type":50,"value":1472},"observedGeneration",{"type":50,"value":1474}," behind ",{"type":45,"tag":83,"props":1476,"children":1478},{"className":1477},[],[1479],{"type":50,"value":1480},"generation",{"type":45,"tag":1348,"props":1482,"children":1483},{},[1484],{"type":50,"value":1485},"Reconcile is stuck or skipped",{"type":45,"tag":1348,"props":1487,"children":1488},{},[1489],{"type":45,"tag":59,"props":1490,"children":1492},{"href":1491},"#reconciliation-not-progressing",[1493],{"type":50,"value":1494},"Reconciliation not progressing",{"type":45,"tag":1321,"props":1496,"children":1497},{},[1498,1503,1508],{"type":45,"tag":1348,"props":1499,"children":1500},{},[1501],{"type":50,"value":1502},"Pods Pending",{"type":45,"tag":1348,"props":1504,"children":1505},{},[1506],{"type":50,"value":1507},"Scheduling, storage, topology, image pull, or node labels",{"type":45,"tag":1348,"props":1509,"children":1510},{},[1511],{"type":45,"tag":59,"props":1512,"children":1514},{"href":1513},"#pod-scheduling-and-storage-failures",[1515],{"type":50,"value":1516},"Pod scheduling and storage failures",{"type":45,"tag":1321,"props":1518,"children":1519},{},[1520,1525,1530],{"type":45,"tag":1348,"props":1521,"children":1522},{},[1523],{"type":50,"value":1524},"Pods Running but not Ready",{"type":45,"tag":1348,"props":1526,"children":1527},{},[1528],{"type":50,"value":1529},"CRDB readiness, TLS, join, DNS, network, or recovery",{"type":45,"tag":1348,"props":1531,"children":1532},{},[1533],{"type":45,"tag":59,"props":1534,"children":1536},{"href":1535},"#pod-readiness-and-crdb-issues",[1537],{"type":50,"value":1538},"Pod readiness and CRDB issues",{"type":45,"tag":1321,"props":1540,"children":1541},{},[1542,1547,1552],{"type":45,"tag":1348,"props":1543,"children":1544},{},[1545],{"type":50,"value":1546},"Upgrade stuck with mixed pod images",{"type":45,"tag":1348,"props":1548,"children":1549},{},[1550],{"type":50,"value":1551},"Version validation, rejected image, rollout dependency, scheduling",{"type":45,"tag":1348,"props":1553,"children":1554},{},[1555],{"type":45,"tag":59,"props":1556,"children":1558},{"href":1557},"#upgrade-and-version-validation",[1559],{"type":50,"value":1560},"Upgrade and version validation",{"type":45,"tag":1321,"props":1562,"children":1563},{},[1564,1569,1574],{"type":45,"tag":1348,"props":1565,"children":1566},{},[1567],{"type":50,"value":1568},"Multi-region pods cannot join",{"type":45,"tag":1348,"props":1570,"children":1571},{},[1572],{"type":50,"value":1573},"DNS, network, region list, CA mismatch",{"type":45,"tag":1348,"props":1575,"children":1576},{},[1577],{"type":45,"tag":59,"props":1578,"children":1580},{"href":1579},"#dns-service-and-network-issues",[1581],{"type":50,"value":1582},"DNS, service, and network issues",{"type":45,"tag":1321,"props":1584,"children":1585},{},[1586,1591,1596],{"type":45,"tag":1348,"props":1587,"children":1588},{},[1589],{"type":50,"value":1590},"Scale-down stuck",{"type":45,"tag":1348,"props":1592,"children":1593},{},[1594],{"type":50,"value":1595},"Decommission\u002Fdrain blocked or multiple nodes decommissioning",{"type":45,"tag":1348,"props":1597,"children":1598},{},[1599],{"type":45,"tag":59,"props":1600,"children":1602},{"href":1601},"#scale-down-and-decommission",[1603],{"type":50,"value":1604},"Scale down and decommission",{"type":45,"tag":1321,"props":1606,"children":1607},{},[1608,1613,1618],{"type":45,"tag":1348,"props":1609,"children":1610},{},[1611],{"type":50,"value":1612},"Migration labels\u002Fstatus stuck",{"type":45,"tag":1348,"props":1614,"children":1615},{},[1616],{"type":50,"value":1617},"Migration controller issue",{"type":45,"tag":1348,"props":1619,"children":1620},{},[1621],{"type":45,"tag":59,"props":1622,"children":1623},{"href":168},[1624],{"type":50,"value":171},{"type":45,"tag":68,"props":1626,"children":1628},{"id":1627},"crd-and-operator-readiness",[1629],{"type":50,"value":1630},"CRD and Operator Readiness",{"type":45,"tag":438,"props":1632,"children":1634},{"className":440,"code":1633,"language":442,"meta":443,"style":443},"kubectl -n \u003Coperator-namespace> rollout status deploy\u002Fcockroach-operator --timeout=5m\nkubectl get crd crdbclusters.crdb.cockroachlabs.com -o jsonpath='{.spec.versions[*].name}{\"\\n\"}'\nkubectl get crd crdbnodes.crdb.cockroachlabs.com -o jsonpath='{.spec.versions[*].name}{\"\\n\"}'\nkubectl -n \u003Coperator-namespace> get deploy cockroach-operator -o jsonpath='{.spec.template.spec.containers[0].image}{\"\\n\"}'\n",[1635],{"type":45,"tag":83,"props":1636,"children":1637},{"__ignoreMap":443},[1638,1684,1726,1766],{"type":45,"tag":449,"props":1639,"children":1640},{"class":451,"line":452},[1641,1645,1649,1653,1657,1661,1665,1670,1674,1679],{"type":45,"tag":449,"props":1642,"children":1643},{"style":466},[1644],{"type":50,"value":729},{"type":45,"tag":449,"props":1646,"children":1647},{"style":472},[1648],{"type":50,"value":475},{"type":45,"tag":449,"props":1650,"children":1651},{"style":478},[1652],{"type":50,"value":481},{"type":45,"tag":449,"props":1654,"children":1655},{"style":472},[1656],{"type":50,"value":486},{"type":45,"tag":449,"props":1658,"children":1659},{"style":489},[1660],{"type":50,"value":492},{"type":45,"tag":449,"props":1662,"children":1663},{"style":478},[1664],{"type":50,"value":497},{"type":45,"tag":449,"props":1666,"children":1667},{"style":472},[1668],{"type":50,"value":1669}," rollout",{"type":45,"tag":449,"props":1671,"children":1672},{"style":472},[1673],{"type":50,"value":502},{"type":45,"tag":449,"props":1675,"children":1676},{"style":472},[1677],{"type":50,"value":1678}," deploy\u002Fcockroach-operator",{"type":45,"tag":449,"props":1680,"children":1681},{"style":472},[1682],{"type":50,"value":1683}," --timeout=5m\n",{"type":45,"tag":449,"props":1685,"children":1686},{"class":451,"line":462},[1687,1691,1695,1699,1703,1707,1712,1717,1722],{"type":45,"tag":449,"props":1688,"children":1689},{"style":466},[1690],{"type":50,"value":729},{"type":45,"tag":449,"props":1692,"children":1693},{"style":472},[1694],{"type":50,"value":754},{"type":45,"tag":449,"props":1696,"children":1697},{"style":472},[1698],{"type":50,"value":889},{"type":45,"tag":449,"props":1700,"children":1701},{"style":472},[1702],{"type":50,"value":894},{"type":45,"tag":449,"props":1704,"children":1705},{"style":472},[1706],{"type":50,"value":764},{"type":45,"tag":449,"props":1708,"children":1709},{"style":472},[1710],{"type":50,"value":1711}," jsonpath=",{"type":45,"tag":449,"props":1713,"children":1714},{"style":478},[1715],{"type":50,"value":1716},"'",{"type":45,"tag":449,"props":1718,"children":1719},{"style":472},[1720],{"type":50,"value":1721},"{.spec.versions[*].name}{\"\\n\"}",{"type":45,"tag":449,"props":1723,"children":1724},{"style":478},[1725],{"type":50,"value":799},{"type":45,"tag":449,"props":1727,"children":1728},{"class":451,"line":533},[1729,1733,1737,1741,1746,1750,1754,1758,1762],{"type":45,"tag":449,"props":1730,"children":1731},{"style":466},[1732],{"type":50,"value":729},{"type":45,"tag":449,"props":1734,"children":1735},{"style":472},[1736],{"type":50,"value":754},{"type":45,"tag":449,"props":1738,"children":1739},{"style":472},[1740],{"type":50,"value":889},{"type":45,"tag":449,"props":1742,"children":1743},{"style":472},[1744],{"type":50,"value":1745}," crdbnodes.crdb.cockroachlabs.com",{"type":45,"tag":449,"props":1747,"children":1748},{"style":472},[1749],{"type":50,"value":764},{"type":45,"tag":449,"props":1751,"children":1752},{"style":472},[1753],{"type":50,"value":1711},{"type":45,"tag":449,"props":1755,"children":1756},{"style":478},[1757],{"type":50,"value":1716},{"type":45,"tag":449,"props":1759,"children":1760},{"style":472},[1761],{"type":50,"value":1721},{"type":45,"tag":449,"props":1763,"children":1764},{"style":478},[1765],{"type":50,"value":799},{"type":45,"tag":449,"props":1767,"children":1768},{"class":451,"line":591},[1769,1773,1777,1781,1785,1789,1793,1797,1802,1807,1811,1815,1819,1824],{"type":45,"tag":449,"props":1770,"children":1771},{"style":466},[1772],{"type":50,"value":729},{"type":45,"tag":449,"props":1774,"children":1775},{"style":472},[1776],{"type":50,"value":475},{"type":45,"tag":449,"props":1778,"children":1779},{"style":478},[1780],{"type":50,"value":481},{"type":45,"tag":449,"props":1782,"children":1783},{"style":472},[1784],{"type":50,"value":486},{"type":45,"tag":449,"props":1786,"children":1787},{"style":489},[1788],{"type":50,"value":492},{"type":45,"tag":449,"props":1790,"children":1791},{"style":478},[1792],{"type":50,"value":497},{"type":45,"tag":449,"props":1794,"children":1795},{"style":472},[1796],{"type":50,"value":754},{"type":45,"tag":449,"props":1798,"children":1799},{"style":472},[1800],{"type":50,"value":1801}," deploy",{"type":45,"tag":449,"props":1803,"children":1804},{"style":472},[1805],{"type":50,"value":1806}," cockroach-operator",{"type":45,"tag":449,"props":1808,"children":1809},{"style":472},[1810],{"type":50,"value":764},{"type":45,"tag":449,"props":1812,"children":1813},{"style":472},[1814],{"type":50,"value":1711},{"type":45,"tag":449,"props":1816,"children":1817},{"style":478},[1818],{"type":50,"value":1716},{"type":45,"tag":449,"props":1820,"children":1821},{"style":472},[1822],{"type":50,"value":1823},"{.spec.template.spec.containers[0].image}{\"\\n\"}",{"type":45,"tag":449,"props":1825,"children":1826},{"style":478},[1827],{"type":50,"value":799},{"type":45,"tag":53,"props":1829,"children":1830},{},[1831],{"type":50,"value":1832},"Remediation:",{"type":45,"tag":75,"props":1834,"children":1835},{},[1836,1841,1846,1851],{"type":45,"tag":79,"props":1837,"children":1838},{},[1839],{"type":50,"value":1840},"Install or upgrade the operator chart first.",{"type":45,"tag":79,"props":1842,"children":1843},{},[1844],{"type":50,"value":1845},"Wait for the operator Deployment before installing the CockroachDB chart.",{"type":45,"tag":79,"props":1847,"children":1848},{},[1849],{"type":50,"value":1850},"For split charts, upgrade the operator chart before the CockroachDB chart.",{"type":45,"tag":79,"props":1852,"children":1853},{},[1854],{"type":50,"value":1855},"Check the Helm chart changelog for version-specific operator fixes before deep debugging older releases.",{"type":45,"tag":68,"props":1857,"children":1859},{"id":1858},"operator-health",[1860],{"type":50,"value":1861},"Operator Health",{"type":45,"tag":438,"props":1863,"children":1865},{"className":440,"code":1864,"language":442,"meta":443,"style":443},"kubectl -n \u003Coperator-namespace> get pods -l app=cockroach-operator -o wide\nkubectl -n \u003Coperator-namespace> describe pod \u003Coperator-pod>\nkubectl -n \u003Coperator-namespace> logs -l app=cockroach-operator --tail=100\nkubectl -n \u003Coperator-namespace> get deploy cockroach-operator -o jsonpath='{.spec.template.spec.containers[0].env}{\"\\n\"}'\n",[1866],{"type":45,"tag":83,"props":1867,"children":1868},{"__ignoreMap":443},[1869,1921,1973,2017],{"type":45,"tag":449,"props":1870,"children":1871},{"class":451,"line":452},[1872,1876,1880,1884,1888,1892,1896,1900,1905,1909,1913,1917],{"type":45,"tag":449,"props":1873,"children":1874},{"style":466},[1875],{"type":50,"value":729},{"type":45,"tag":449,"props":1877,"children":1878},{"style":472},[1879],{"type":50,"value":475},{"type":45,"tag":449,"props":1881,"children":1882},{"style":478},[1883],{"type":50,"value":481},{"type":45,"tag":449,"props":1885,"children":1886},{"style":472},[1887],{"type":50,"value":486},{"type":45,"tag":449,"props":1889,"children":1890},{"style":489},[1891],{"type":50,"value":492},{"type":45,"tag":449,"props":1893,"children":1894},{"style":478},[1895],{"type":50,"value":497},{"type":45,"tag":449,"props":1897,"children":1898},{"style":472},[1899],{"type":50,"value":754},{"type":45,"tag":449,"props":1901,"children":1902},{"style":472},[1903],{"type":50,"value":1904}," pods",{"type":45,"tag":449,"props":1906,"children":1907},{"style":472},[1908],{"type":50,"value":837},{"type":45,"tag":449,"props":1910,"children":1911},{"style":472},[1912],{"type":50,"value":842},{"type":45,"tag":449,"props":1914,"children":1915},{"style":472},[1916],{"type":50,"value":764},{"type":45,"tag":449,"props":1918,"children":1919},{"style":472},[1920],{"type":50,"value":945},{"type":45,"tag":449,"props":1922,"children":1923},{"class":451,"line":462},[1924,1928,1932,1936,1940,1944,1948,1952,1956,1960,1965,1969],{"type":45,"tag":449,"props":1925,"children":1926},{"style":466},[1927],{"type":50,"value":729},{"type":45,"tag":449,"props":1929,"children":1930},{"style":472},[1931],{"type":50,"value":475},{"type":45,"tag":449,"props":1933,"children":1934},{"style":478},[1935],{"type":50,"value":481},{"type":45,"tag":449,"props":1937,"children":1938},{"style":472},[1939],{"type":50,"value":486},{"type":45,"tag":449,"props":1941,"children":1942},{"style":489},[1943],{"type":50,"value":492},{"type":45,"tag":449,"props":1945,"children":1946},{"style":478},[1947],{"type":50,"value":497},{"type":45,"tag":449,"props":1949,"children":1950},{"style":472},[1951],{"type":50,"value":978},{"type":45,"tag":449,"props":1953,"children":1954},{"style":472},[1955],{"type":50,"value":1111},{"type":45,"tag":449,"props":1957,"children":1958},{"style":478},[1959],{"type":50,"value":481},{"type":45,"tag":449,"props":1961,"children":1962},{"style":472},[1963],{"type":50,"value":1964},"operator-po",{"type":45,"tag":449,"props":1966,"children":1967},{"style":489},[1968],{"type":50,"value":1125},{"type":45,"tag":449,"props":1970,"children":1971},{"style":478},[1972],{"type":50,"value":1130},{"type":45,"tag":449,"props":1974,"children":1975},{"class":451,"line":533},[1976,1980,1984,1988,1992,1996,2000,2004,2008,2012],{"type":45,"tag":449,"props":1977,"children":1978},{"style":466},[1979],{"type":50,"value":729},{"type":45,"tag":449,"props":1981,"children":1982},{"style":472},[1983],{"type":50,"value":475},{"type":45,"tag":449,"props":1985,"children":1986},{"style":478},[1987],{"type":50,"value":481},{"type":45,"tag":449,"props":1989,"children":1990},{"style":472},[1991],{"type":50,"value":486},{"type":45,"tag":449,"props":1993,"children":1994},{"style":489},[1995],{"type":50,"value":492},{"type":45,"tag":449,"props":1997,"children":1998},{"style":478},[1999],{"type":50,"value":497},{"type":45,"tag":449,"props":2001,"children":2002},{"style":472},[2003],{"type":50,"value":832},{"type":45,"tag":449,"props":2005,"children":2006},{"style":472},[2007],{"type":50,"value":837},{"type":45,"tag":449,"props":2009,"children":2010},{"style":472},[2011],{"type":50,"value":842},{"type":45,"tag":449,"props":2013,"children":2014},{"style":472},[2015],{"type":50,"value":2016}," --tail=100\n",{"type":45,"tag":449,"props":2018,"children":2019},{"class":451,"line":591},[2020,2024,2028,2032,2036,2040,2044,2048,2052,2056,2060,2064,2068,2073],{"type":45,"tag":449,"props":2021,"children":2022},{"style":466},[2023],{"type":50,"value":729},{"type":45,"tag":449,"props":2025,"children":2026},{"style":472},[2027],{"type":50,"value":475},{"type":45,"tag":449,"props":2029,"children":2030},{"style":478},[2031],{"type":50,"value":481},{"type":45,"tag":449,"props":2033,"children":2034},{"style":472},[2035],{"type":50,"value":486},{"type":45,"tag":449,"props":2037,"children":2038},{"style":489},[2039],{"type":50,"value":492},{"type":45,"tag":449,"props":2041,"children":2042},{"style":478},[2043],{"type":50,"value":497},{"type":45,"tag":449,"props":2045,"children":2046},{"style":472},[2047],{"type":50,"value":754},{"type":45,"tag":449,"props":2049,"children":2050},{"style":472},[2051],{"type":50,"value":1801},{"type":45,"tag":449,"props":2053,"children":2054},{"style":472},[2055],{"type":50,"value":1806},{"type":45,"tag":449,"props":2057,"children":2058},{"style":472},[2059],{"type":50,"value":764},{"type":45,"tag":449,"props":2061,"children":2062},{"style":472},[2063],{"type":50,"value":1711},{"type":45,"tag":449,"props":2065,"children":2066},{"style":478},[2067],{"type":50,"value":1716},{"type":45,"tag":449,"props":2069,"children":2070},{"style":472},[2071],{"type":50,"value":2072},"{.spec.template.spec.containers[0].env}{\"\\n\"}",{"type":45,"tag":449,"props":2074,"children":2075},{"style":478},[2076],{"type":50,"value":799},{"type":45,"tag":53,"props":2078,"children":2079},{},[2080],{"type":50,"value":2081},"Interpretation:",{"type":45,"tag":75,"props":2083,"children":2084},{},[2085,2090,2095,2108,2120],{"type":45,"tag":79,"props":2086,"children":2087},{},[2088],{"type":50,"value":2089},"CrashLoopBackOff: collect previous logs and check for panics.",{"type":45,"tag":79,"props":2091,"children":2092},{},[2093],{"type":50,"value":2094},"OOMKilled: inspect limits and consider increasing operator memory.",{"type":45,"tag":79,"props":2096,"children":2097},{},[2098,2100,2106],{"type":50,"value":2099},"Empty or unset ",{"type":45,"tag":83,"props":2101,"children":2103},{"className":2102},[],[2104],{"type":50,"value":2105},"WATCH_NAMESPACE",{"type":50,"value":2107},": global mode.",{"type":45,"tag":79,"props":2109,"children":2110},{},[2111,2113,2118],{"type":50,"value":2112},"Non-empty ",{"type":45,"tag":83,"props":2114,"children":2116},{"className":2115},[],[2117],{"type":50,"value":2105},{"type":50,"value":2119},": the operator watches only listed namespaces.",{"type":45,"tag":79,"props":2121,"children":2122},{},[2123],{"type":50,"value":2124},"If the CockroachDB namespace is not watched, deploy an operator for it or add it to the watch scope.",{"type":45,"tag":53,"props":2126,"children":2127},{},[2128],{"type":50,"value":2129},"Check recent logs to see whether reconciliation is active:",{"type":45,"tag":438,"props":2131,"children":2133},{"className":440,"code":2132,"language":442,"meta":443,"style":443},"kubectl -n \u003Coperator-namespace> logs -l app=cockroach-operator --tail=100 | grep -i reconcil || true\n",[2134],{"type":45,"tag":83,"props":2135,"children":2136},{"__ignoreMap":443},[2137],{"type":45,"tag":449,"props":2138,"children":2139},{"class":451,"line":452},[2140,2144,2148,2152,2156,2160,2164,2168,2172,2176,2181,2185,2189,2194,2199,2203],{"type":45,"tag":449,"props":2141,"children":2142},{"style":466},[2143],{"type":50,"value":729},{"type":45,"tag":449,"props":2145,"children":2146},{"style":472},[2147],{"type":50,"value":475},{"type":45,"tag":449,"props":2149,"children":2150},{"style":478},[2151],{"type":50,"value":481},{"type":45,"tag":449,"props":2153,"children":2154},{"style":472},[2155],{"type":50,"value":486},{"type":45,"tag":449,"props":2157,"children":2158},{"style":489},[2159],{"type":50,"value":492},{"type":45,"tag":449,"props":2161,"children":2162},{"style":478},[2163],{"type":50,"value":497},{"type":45,"tag":449,"props":2165,"children":2166},{"style":472},[2167],{"type":50,"value":832},{"type":45,"tag":449,"props":2169,"children":2170},{"style":472},[2171],{"type":50,"value":837},{"type":45,"tag":449,"props":2173,"children":2174},{"style":472},[2175],{"type":50,"value":842},{"type":45,"tag":449,"props":2177,"children":2178},{"style":472},[2179],{"type":50,"value":2180}," --tail=100",{"type":45,"tag":449,"props":2182,"children":2183},{"style":478},[2184],{"type":50,"value":774},{"type":45,"tag":449,"props":2186,"children":2187},{"style":466},[2188],{"type":50,"value":779},{"type":45,"tag":449,"props":2190,"children":2191},{"style":472},[2192],{"type":50,"value":2193}," -i",{"type":45,"tag":449,"props":2195,"children":2196},{"style":472},[2197],{"type":50,"value":2198}," reconcil",{"type":45,"tag":449,"props":2200,"children":2201},{"style":478},[2202],{"type":50,"value":524},{"type":45,"tag":449,"props":2204,"children":2205},{"style":527},[2206],{"type":50,"value":530},{"type":45,"tag":53,"props":2208,"children":2209},{},[2210,2212,2218,2220,2226],{"type":50,"value":2211},"Do not add ad hoc annotations to trigger reconciliation. If a user-approved reconcile-triggering change is required, use the chart-supported timestamp path through ",{"type":45,"tag":83,"props":2213,"children":2215},{"className":2214},[],[2216],{"type":50,"value":2217},"helm upgrade --reuse-values",{"type":50,"value":2219},"; this updates ",{"type":45,"tag":83,"props":2221,"children":2223},{"className":2222},[],[2224],{"type":50,"value":2225},"helm.sh\u002FrestartedAt",{"type":50,"value":2227}," and may roll CockroachDB pods, so treat it as a mutating operation:",{"type":45,"tag":438,"props":2229,"children":2231},{"className":440,"code":2230,"language":442,"meta":443,"style":443},"helm -n \u003Ccockroachdb-namespace> upgrade \u003Ccockroachdb-release> \u003Ccockroachdb-chart> \\\n  --reuse-values \\\n  --set-string cockroachdb.crdbCluster.timestamp=\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\"\n",[2232],{"type":45,"tag":83,"props":2233,"children":2234},{"__ignoreMap":443},[2235,2306,2318],{"type":45,"tag":449,"props":2236,"children":2237},{"class":451,"line":452},[2238,2242,2246,2250,2254,2258,2262,2267,2271,2275,2279,2283,2287,2292,2297,2301],{"type":45,"tag":449,"props":2239,"children":2240},{"style":466},[2241],{"type":50,"value":469},{"type":45,"tag":449,"props":2243,"children":2244},{"style":472},[2245],{"type":50,"value":475},{"type":45,"tag":449,"props":2247,"children":2248},{"style":478},[2249],{"type":50,"value":481},{"type":45,"tag":449,"props":2251,"children":2252},{"style":472},[2253],{"type":50,"value":551},{"type":45,"tag":449,"props":2255,"children":2256},{"style":489},[2257],{"type":50,"value":492},{"type":45,"tag":449,"props":2259,"children":2260},{"style":478},[2261],{"type":50,"value":497},{"type":45,"tag":449,"props":2263,"children":2264},{"style":472},[2265],{"type":50,"value":2266}," upgrade",{"type":45,"tag":449,"props":2268,"children":2269},{"style":478},[2270],{"type":50,"value":481},{"type":45,"tag":449,"props":2272,"children":2273},{"style":472},[2274],{"type":50,"value":572},{"type":45,"tag":449,"props":2276,"children":2277},{"style":489},[2278],{"type":50,"value":492},{"type":45,"tag":449,"props":2280,"children":2281},{"style":478},[2282],{"type":50,"value":497},{"type":45,"tag":449,"props":2284,"children":2285},{"style":478},[2286],{"type":50,"value":481},{"type":45,"tag":449,"props":2288,"children":2289},{"style":472},[2290],{"type":50,"value":2291},"cockroachdb-char",{"type":45,"tag":449,"props":2293,"children":2294},{"style":489},[2295],{"type":50,"value":2296},"t",{"type":45,"tag":449,"props":2298,"children":2299},{"style":478},[2300],{"type":50,"value":497},{"type":45,"tag":449,"props":2302,"children":2303},{"style":489},[2304],{"type":50,"value":2305}," \\\n",{"type":45,"tag":449,"props":2307,"children":2308},{"class":451,"line":462},[2309,2314],{"type":45,"tag":449,"props":2310,"children":2311},{"style":472},[2312],{"type":50,"value":2313},"  --reuse-values",{"type":45,"tag":449,"props":2315,"children":2316},{"style":489},[2317],{"type":50,"value":2305},{"type":45,"tag":449,"props":2319,"children":2320},{"class":451,"line":533},[2321,2326,2331,2336,2341,2346],{"type":45,"tag":449,"props":2322,"children":2323},{"style":472},[2324],{"type":50,"value":2325},"  --set-string",{"type":45,"tag":449,"props":2327,"children":2328},{"style":472},[2329],{"type":50,"value":2330}," cockroachdb.crdbCluster.timestamp=",{"type":45,"tag":449,"props":2332,"children":2333},{"style":478},[2334],{"type":50,"value":2335},"\"$(",{"type":45,"tag":449,"props":2337,"children":2338},{"style":466},[2339],{"type":50,"value":2340},"date",{"type":45,"tag":449,"props":2342,"children":2343},{"style":472},[2344],{"type":50,"value":2345}," -u +%Y-%m-%dT%H:%M:%SZ",{"type":45,"tag":449,"props":2347,"children":2348},{"style":478},[2349],{"type":50,"value":2350},")\"\n",{"type":45,"tag":53,"props":2352,"children":2353},{},[2354,2356,2360],{"type":50,"value":2355},"If the operator is healthy but silent and no user-approved mutation is appropriate, use ",{"type":45,"tag":59,"props":2357,"children":2358},{"href":61},[2359],{"type":50,"value":64},{"type":50,"value":2361}," to gather pprof and metrics before restarting it.",{"type":45,"tag":68,"props":2363,"children":2365},{"id":2364},"rbac-and-node-reader-failures",[2366],{"type":50,"value":2367},"RBAC and Node-Reader Failures",{"type":45,"tag":53,"props":2369,"children":2370},{},[2371],{"type":50,"value":2372},"Common error:",{"type":45,"tag":438,"props":2374,"children":2378},{"className":2375,"code":2377,"language":50,"meta":443},[2376],"language-text","attempt to grant extra privileges\n",[2379],{"type":45,"tag":83,"props":2380,"children":2381},{"__ignoreMap":443},[2382],{"type":50,"value":2377},{"type":45,"tag":53,"props":2384,"children":2385},{},[2386],{"type":50,"value":2387},"Cause:",{"type":45,"tag":75,"props":2389,"children":2390},{},[2391,2396],{"type":45,"tag":79,"props":2392,"children":2393},{},[2394],{"type":50,"value":2395},"The installing principal cannot create cluster-scoped RBAC.",{"type":45,"tag":79,"props":2397,"children":2398},{},[2399],{"type":50,"value":2400},"CockroachDB pods need node read access to derive locality.",{"type":45,"tag":53,"props":2402,"children":2403},{},[2404],{"type":50,"value":2405},"Checks:",{"type":45,"tag":438,"props":2407,"children":2409},{"className":440,"code":2408,"language":442,"meta":443,"style":443},"kubectl auth can-i create clusterroles.rbac.authorization.k8s.io\nkubectl auth can-i create clusterrolebindings.rbac.authorization.k8s.io\nkubectl auth can-i get nodes\n",[2410],{"type":45,"tag":83,"props":2411,"children":2412},{"__ignoreMap":443},[2413,2440,2464],{"type":45,"tag":449,"props":2414,"children":2415},{"class":451,"line":452},[2416,2420,2425,2430,2435],{"type":45,"tag":449,"props":2417,"children":2418},{"style":466},[2419],{"type":50,"value":729},{"type":45,"tag":449,"props":2421,"children":2422},{"style":472},[2423],{"type":50,"value":2424}," auth",{"type":45,"tag":449,"props":2426,"children":2427},{"style":472},[2428],{"type":50,"value":2429}," can-i",{"type":45,"tag":449,"props":2431,"children":2432},{"style":472},[2433],{"type":50,"value":2434}," create",{"type":45,"tag":449,"props":2436,"children":2437},{"style":472},[2438],{"type":50,"value":2439}," clusterroles.rbac.authorization.k8s.io\n",{"type":45,"tag":449,"props":2441,"children":2442},{"class":451,"line":462},[2443,2447,2451,2455,2459],{"type":45,"tag":449,"props":2444,"children":2445},{"style":466},[2446],{"type":50,"value":729},{"type":45,"tag":449,"props":2448,"children":2449},{"style":472},[2450],{"type":50,"value":2424},{"type":45,"tag":449,"props":2452,"children":2453},{"style":472},[2454],{"type":50,"value":2429},{"type":45,"tag":449,"props":2456,"children":2457},{"style":472},[2458],{"type":50,"value":2434},{"type":45,"tag":449,"props":2460,"children":2461},{"style":472},[2462],{"type":50,"value":2463}," clusterrolebindings.rbac.authorization.k8s.io\n",{"type":45,"tag":449,"props":2465,"children":2466},{"class":451,"line":533},[2467,2471,2475,2479,2483],{"type":45,"tag":449,"props":2468,"children":2469},{"style":466},[2470],{"type":50,"value":729},{"type":45,"tag":449,"props":2472,"children":2473},{"style":472},[2474],{"type":50,"value":2424},{"type":45,"tag":449,"props":2476,"children":2477},{"style":472},[2478],{"type":50,"value":2429},{"type":45,"tag":449,"props":2480,"children":2481},{"style":472},[2482],{"type":50,"value":754},{"type":45,"tag":449,"props":2484,"children":2485},{"style":472},[2486],{"type":50,"value":2487}," nodes\n",{"type":45,"tag":53,"props":2489,"children":2490},{},[2491],{"type":50,"value":2492},"Remediation options:",{"type":45,"tag":75,"props":2494,"children":2495},{},[2496,2509,2522],{"type":45,"tag":79,"props":2497,"children":2498},{},[2499,2501,2507],{"type":50,"value":2500},"Platform team installs the operator chart with ",{"type":45,"tag":83,"props":2502,"children":2504},{"className":2503},[],[2505],{"type":50,"value":2506},"nodeReader.enabled=true",{"type":50,"value":2508}," and subjects matching the CockroachDB ServiceAccount.",{"type":45,"tag":79,"props":2510,"children":2511},{},[2512,2514,2520],{"type":50,"value":2513},"Tenant chart sets ",{"type":45,"tag":83,"props":2515,"children":2517},{"className":2516},[],[2518],{"type":50,"value":2519},"cockroachdb.crdbCluster.rbac.nodeReader.create=false",{"type":50,"value":2521}," only after the platform-owned binding exists.",{"type":45,"tag":79,"props":2523,"children":2524},{},[2525],{"type":50,"value":2526},"If the customer accepts cluster-admin install privileges, rerun the Helm operation with an identity that can create the required ClusterRole and ClusterRoleBinding.",{"type":45,"tag":53,"props":2528,"children":2529},{},[2530,2532,2538],{"type":50,"value":2531},"Do not set ",{"type":45,"tag":83,"props":2533,"children":2535},{"className":2534},[],[2536],{"type":50,"value":2537},"nodeReader.create=false",{"type":50,"value":2539}," before replacement RBAC exists.",{"type":45,"tag":68,"props":2541,"children":2543},{"id":2542},"webhook-checks",[2544],{"type":50,"value":2545},"Webhook Checks",{"type":45,"tag":438,"props":2547,"children":2549},{"className":440,"code":2548,"language":442,"meta":443,"style":443},"kubectl -n \u003Coperator-namespace> get svc cockroach-webhook-service\nkubectl -n \u003Coperator-namespace> get endpoints cockroach-webhook-service\nkubectl get validatingwebhookconfigurations | grep cockroach\n",[2550],{"type":45,"tag":83,"props":2551,"children":2552},{"__ignoreMap":443},[2553,2594,2634],{"type":45,"tag":449,"props":2554,"children":2555},{"class":451,"line":452},[2556,2560,2564,2568,2572,2576,2580,2584,2589],{"type":45,"tag":449,"props":2557,"children":2558},{"style":466},[2559],{"type":50,"value":729},{"type":45,"tag":449,"props":2561,"children":2562},{"style":472},[2563],{"type":50,"value":475},{"type":45,"tag":449,"props":2565,"children":2566},{"style":478},[2567],{"type":50,"value":481},{"type":45,"tag":449,"props":2569,"children":2570},{"style":472},[2571],{"type":50,"value":486},{"type":45,"tag":449,"props":2573,"children":2574},{"style":489},[2575],{"type":50,"value":492},{"type":45,"tag":449,"props":2577,"children":2578},{"style":478},[2579],{"type":50,"value":497},{"type":45,"tag":449,"props":2581,"children":2582},{"style":472},[2583],{"type":50,"value":754},{"type":45,"tag":449,"props":2585,"children":2586},{"style":472},[2587],{"type":50,"value":2588}," svc",{"type":45,"tag":449,"props":2590,"children":2591},{"style":472},[2592],{"type":50,"value":2593}," cockroach-webhook-service\n",{"type":45,"tag":449,"props":2595,"children":2596},{"class":451,"line":462},[2597,2601,2605,2609,2613,2617,2621,2625,2630],{"type":45,"tag":449,"props":2598,"children":2599},{"style":466},[2600],{"type":50,"value":729},{"type":45,"tag":449,"props":2602,"children":2603},{"style":472},[2604],{"type":50,"value":475},{"type":45,"tag":449,"props":2606,"children":2607},{"style":478},[2608],{"type":50,"value":481},{"type":45,"tag":449,"props":2610,"children":2611},{"style":472},[2612],{"type":50,"value":486},{"type":45,"tag":449,"props":2614,"children":2615},{"style":489},[2616],{"type":50,"value":492},{"type":45,"tag":449,"props":2618,"children":2619},{"style":478},[2620],{"type":50,"value":497},{"type":45,"tag":449,"props":2622,"children":2623},{"style":472},[2624],{"type":50,"value":754},{"type":45,"tag":449,"props":2626,"children":2627},{"style":472},[2628],{"type":50,"value":2629}," endpoints",{"type":45,"tag":449,"props":2631,"children":2632},{"style":472},[2633],{"type":50,"value":2593},{"type":45,"tag":449,"props":2635,"children":2636},{"class":451,"line":533},[2637,2641,2645,2650,2654,2658],{"type":45,"tag":449,"props":2638,"children":2639},{"style":466},[2640],{"type":50,"value":729},{"type":45,"tag":449,"props":2642,"children":2643},{"style":472},[2644],{"type":50,"value":754},{"type":45,"tag":449,"props":2646,"children":2647},{"style":472},[2648],{"type":50,"value":2649}," validatingwebhookconfigurations",{"type":45,"tag":449,"props":2651,"children":2652},{"style":478},[2653],{"type":50,"value":774},{"type":45,"tag":449,"props":2655,"children":2656},{"style":466},[2657],{"type":50,"value":779},{"type":45,"tag":449,"props":2659,"children":2660},{"style":472},[2661],{"type":50,"value":2662}," cockroach\n",{"type":45,"tag":53,"props":2664,"children":2665},{},[2666],{"type":50,"value":2667},"If webhook validation fails, verify the CA bundle:",{"type":45,"tag":438,"props":2669,"children":2671},{"className":440,"code":2670,"language":442,"meta":443,"style":443},"kubectl get validatingwebhookconfiguration cockroach-webhook-config \\\n  -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | base64 -d | openssl x509 -noout -dates -subject -issuer\n",[2672],{"type":45,"tag":83,"props":2673,"children":2674},{"__ignoreMap":443},[2675,2700],{"type":45,"tag":449,"props":2676,"children":2677},{"class":451,"line":452},[2678,2682,2686,2691,2696],{"type":45,"tag":449,"props":2679,"children":2680},{"style":466},[2681],{"type":50,"value":729},{"type":45,"tag":449,"props":2683,"children":2684},{"style":472},[2685],{"type":50,"value":754},{"type":45,"tag":449,"props":2687,"children":2688},{"style":472},[2689],{"type":50,"value":2690}," validatingwebhookconfiguration",{"type":45,"tag":449,"props":2692,"children":2693},{"style":472},[2694],{"type":50,"value":2695}," cockroach-webhook-config",{"type":45,"tag":449,"props":2697,"children":2698},{"style":489},[2699],{"type":50,"value":2305},{"type":45,"tag":449,"props":2701,"children":2702},{"class":451,"line":462},[2703,2708,2712,2716,2721,2725,2729,2734,2739,2743,2748,2753,2758,2763,2768],{"type":45,"tag":449,"props":2704,"children":2705},{"style":472},[2706],{"type":50,"value":2707},"  -o",{"type":45,"tag":449,"props":2709,"children":2710},{"style":472},[2711],{"type":50,"value":1711},{"type":45,"tag":449,"props":2713,"children":2714},{"style":478},[2715],{"type":50,"value":1716},{"type":45,"tag":449,"props":2717,"children":2718},{"style":472},[2719],{"type":50,"value":2720},"{.webhooks[0].clientConfig.caBundle}",{"type":45,"tag":449,"props":2722,"children":2723},{"style":478},[2724],{"type":50,"value":1716},{"type":45,"tag":449,"props":2726,"children":2727},{"style":478},[2728],{"type":50,"value":774},{"type":45,"tag":449,"props":2730,"children":2731},{"style":466},[2732],{"type":50,"value":2733}," base64",{"type":45,"tag":449,"props":2735,"children":2736},{"style":472},[2737],{"type":50,"value":2738}," -d",{"type":45,"tag":449,"props":2740,"children":2741},{"style":478},[2742],{"type":50,"value":774},{"type":45,"tag":449,"props":2744,"children":2745},{"style":466},[2746],{"type":50,"value":2747}," openssl",{"type":45,"tag":449,"props":2749,"children":2750},{"style":472},[2751],{"type":50,"value":2752}," x509",{"type":45,"tag":449,"props":2754,"children":2755},{"style":472},[2756],{"type":50,"value":2757}," -noout",{"type":45,"tag":449,"props":2759,"children":2760},{"style":472},[2761],{"type":50,"value":2762}," -dates",{"type":45,"tag":449,"props":2764,"children":2765},{"style":472},[2766],{"type":50,"value":2767}," -subject",{"type":45,"tag":449,"props":2769,"children":2770},{"style":472},[2771],{"type":50,"value":2772}," -issuer\n",{"type":45,"tag":53,"props":2774,"children":2775},{},[2776,2778,2784],{"type":50,"value":2777},"For scoped operators, webhook configurations may be namespace-suffixed, such as ",{"type":45,"tag":83,"props":2779,"children":2781},{"className":2780},[],[2782],{"type":50,"value":2783},"cockroach-webhook-config-\u003Cnamespace>",{"type":50,"value":66},{"type":45,"tag":68,"props":2786,"children":2788},{"id":2787},"reconciliation-not-progressing",[2789],{"type":50,"value":2790},"Reconciliation Not Progressing",{"type":45,"tag":438,"props":2792,"children":2794},{"className":440,"code":2793,"language":442,"meta":443,"style":443},"kubectl -n \u003Ccockroachdb-namespace> get crdbcluster \u003Ccockroachdb-release> -o json | jq '{\n  mode: .spec.mode,\n  image: .spec.image,\n  generation: .metadata.generation,\n  observedGeneration: .status.observedGeneration,\n  statusImage: .status.image,\n  actions: .status.actions,\n  conditions: .status.conditions\n}'\n\nkubectl -n \u003Ccockroachdb-namespace> get crdbnodes \\\n  -o custom-columns=NAME:.metadata.name,GENERATION:.metadata.generation,OBSERVED:.status.observedGeneration,PHASE:.status.phase,HASH:.metadata.annotations[\"crdb.cockroachlabs.com\u002Fhash-revision\"],NODE_ID:.status.nodeID\n",[2795],{"type":45,"tag":83,"props":2796,"children":2797},{"__ignoreMap":443},[2798,2876,2884,2892,2900,2908,2916,2924,2932,2944,2951,2991],{"type":45,"tag":449,"props":2799,"children":2800},{"class":451,"line":452},[2801,2805,2809,2813,2817,2821,2825,2829,2833,2837,2841,2845,2849,2853,2858,2862,2867,2871],{"type":45,"tag":449,"props":2802,"children":2803},{"style":466},[2804],{"type":50,"value":729},{"type":45,"tag":449,"props":2806,"children":2807},{"style":472},[2808],{"type":50,"value":475},{"type":45,"tag":449,"props":2810,"children":2811},{"style":478},[2812],{"type":50,"value":481},{"type":45,"tag":449,"props":2814,"children":2815},{"style":472},[2816],{"type":50,"value":551},{"type":45,"tag":449,"props":2818,"children":2819},{"style":489},[2820],{"type":50,"value":492},{"type":45,"tag":449,"props":2822,"children":2823},{"style":478},[2824],{"type":50,"value":497},{"type":45,"tag":449,"props":2826,"children":2827},{"style":472},[2828],{"type":50,"value":754},{"type":45,"tag":449,"props":2830,"children":2831},{"style":472},[2832],{"type":50,"value":983},{"type":45,"tag":449,"props":2834,"children":2835},{"style":478},[2836],{"type":50,"value":481},{"type":45,"tag":449,"props":2838,"children":2839},{"style":472},[2840],{"type":50,"value":572},{"type":45,"tag":449,"props":2842,"children":2843},{"style":489},[2844],{"type":50,"value":492},{"type":45,"tag":449,"props":2846,"children":2847},{"style":478},[2848],{"type":50,"value":497},{"type":45,"tag":449,"props":2850,"children":2851},{"style":472},[2852],{"type":50,"value":764},{"type":45,"tag":449,"props":2854,"children":2855},{"style":472},[2856],{"type":50,"value":2857}," json",{"type":45,"tag":449,"props":2859,"children":2860},{"style":478},[2861],{"type":50,"value":774},{"type":45,"tag":449,"props":2863,"children":2864},{"style":466},[2865],{"type":50,"value":2866}," jq",{"type":45,"tag":449,"props":2868,"children":2869},{"style":478},[2870],{"type":50,"value":789},{"type":45,"tag":449,"props":2872,"children":2873},{"style":472},[2874],{"type":50,"value":2875},"{\n",{"type":45,"tag":449,"props":2877,"children":2878},{"class":451,"line":462},[2879],{"type":45,"tag":449,"props":2880,"children":2881},{"style":472},[2882],{"type":50,"value":2883},"  mode: .spec.mode,\n",{"type":45,"tag":449,"props":2885,"children":2886},{"class":451,"line":533},[2887],{"type":45,"tag":449,"props":2888,"children":2889},{"style":472},[2890],{"type":50,"value":2891},"  image: .spec.image,\n",{"type":45,"tag":449,"props":2893,"children":2894},{"class":451,"line":591},[2895],{"type":45,"tag":449,"props":2896,"children":2897},{"style":472},[2898],{"type":50,"value":2899},"  generation: .metadata.generation,\n",{"type":45,"tag":449,"props":2901,"children":2902},{"class":451,"line":648},[2903],{"type":45,"tag":449,"props":2904,"children":2905},{"style":472},[2906],{"type":50,"value":2907},"  observedGeneration: .status.observedGeneration,\n",{"type":45,"tag":449,"props":2909,"children":2910},{"class":451,"line":704},[2911],{"type":45,"tag":449,"props":2912,"children":2913},{"style":472},[2914],{"type":50,"value":2915},"  statusImage: .status.image,\n",{"type":45,"tag":449,"props":2917,"children":2918},{"class":451,"line":714},[2919],{"type":45,"tag":449,"props":2920,"children":2921},{"style":472},[2922],{"type":50,"value":2923},"  actions: .status.actions,\n",{"type":45,"tag":449,"props":2925,"children":2926},{"class":451,"line":723},[2927],{"type":45,"tag":449,"props":2928,"children":2929},{"style":472},[2930],{"type":50,"value":2931},"  conditions: .status.conditions\n",{"type":45,"tag":449,"props":2933,"children":2934},{"class":451,"line":802},[2935,2940],{"type":45,"tag":449,"props":2936,"children":2937},{"style":472},[2938],{"type":50,"value":2939},"}",{"type":45,"tag":449,"props":2941,"children":2942},{"style":478},[2943],{"type":50,"value":799},{"type":45,"tag":449,"props":2945,"children":2946},{"class":451,"line":858},[2947],{"type":45,"tag":449,"props":2948,"children":2949},{"emptyLinePlaceholder":708},[2950],{"type":50,"value":711},{"type":45,"tag":449,"props":2952,"children":2953},{"class":451,"line":866},[2954,2958,2962,2966,2970,2974,2978,2982,2987],{"type":45,"tag":449,"props":2955,"children":2956},{"style":466},[2957],{"type":50,"value":729},{"type":45,"tag":449,"props":2959,"children":2960},{"style":472},[2961],{"type":50,"value":475},{"type":45,"tag":449,"props":2963,"children":2964},{"style":478},[2965],{"type":50,"value":481},{"type":45,"tag":449,"props":2967,"children":2968},{"style":472},[2969],{"type":50,"value":551},{"type":45,"tag":449,"props":2971,"children":2972},{"style":489},[2973],{"type":50,"value":492},{"type":45,"tag":449,"props":2975,"children":2976},{"style":478},[2977],{"type":50,"value":497},{"type":45,"tag":449,"props":2979,"children":2980},{"style":472},[2981],{"type":50,"value":754},{"type":45,"tag":449,"props":2983,"children":2984},{"style":472},[2985],{"type":50,"value":2986}," crdbnodes",{"type":45,"tag":449,"props":2988,"children":2989},{"style":489},[2990],{"type":50,"value":2305},{"type":45,"tag":449,"props":2992,"children":2993},{"class":451,"line":875},[2994,2998,3003,3008,3013,3017],{"type":45,"tag":449,"props":2995,"children":2996},{"style":472},[2997],{"type":50,"value":2707},{"type":45,"tag":449,"props":2999,"children":3000},{"style":472},[3001],{"type":50,"value":3002}," custom-columns=NAME:.metadata.name,GENERATION:.metadata.generation,OBSERVED:.status.observedGeneration,PHASE:.status.phase,HASH:.metadata.annotations[",{"type":45,"tag":449,"props":3004,"children":3005},{"style":478},[3006],{"type":50,"value":3007},"\"",{"type":45,"tag":449,"props":3009,"children":3010},{"style":472},[3011],{"type":50,"value":3012},"crdb.cockroachlabs.com\u002Fhash-revision",{"type":45,"tag":449,"props":3014,"children":3015},{"style":478},[3016],{"type":50,"value":3007},{"type":45,"tag":449,"props":3018,"children":3019},{"style":472},[3020],{"type":50,"value":3021},"],NODE_ID:.status.nodeID\n",{"type":45,"tag":53,"props":3023,"children":3024},{},[3025],{"type":50,"value":3026},"Checklist:",{"type":45,"tag":3028,"props":3029,"children":3030},"ol",{},[3031,3036,3056,3061,3066],{"type":45,"tag":79,"props":3032,"children":3033},{},[3034],{"type":50,"value":3035},"Confirm the operator is running and watching the CockroachDB namespace.",{"type":45,"tag":79,"props":3037,"children":3038},{},[3039,3041,3047,3049,3055],{"type":50,"value":3040},"Confirm ",{"type":45,"tag":83,"props":3042,"children":3044},{"className":3043},[],[3045],{"type":50,"value":3046},"spec.mode",{"type":50,"value":3048}," is not ",{"type":45,"tag":83,"props":3050,"children":3052},{"className":3051},[],[3053],{"type":50,"value":3054},"Disabled",{"type":50,"value":66},{"type":45,"tag":79,"props":3057,"children":3058},{},[3059],{"type":50,"value":3060},"Check whether initialization conditions look correct for an existing cluster.",{"type":45,"tag":79,"props":3062,"children":3063},{},[3064],{"type":50,"value":3065},"Check operator logs for reconcile start\u002Fend pairs and errors.",{"type":45,"tag":79,"props":3067,"children":3068},{},[3069],{"type":50,"value":3070},"If no progress is visible, collect the escalation packet before restarting the operator.",{"type":45,"tag":68,"props":3072,"children":3074},{"id":3073},"pod-readiness-and-crdb-issues",[3075],{"type":50,"value":3076},"Pod Readiness and CRDB Issues",{"type":45,"tag":438,"props":3078,"children":3080},{"className":440,"code":3079,"language":442,"meta":443,"style":443},"kubectl -n \u003Ccockroachdb-namespace> get pods -l app.kubernetes.io\u002Fname=cockroachdb -o wide\nkubectl -n \u003Ccockroachdb-namespace> describe pod \u003Ccrdb-pod>\nkubectl -n \u003Ccockroachdb-namespace> logs \u003Ccrdb-pod> -c cockroachdb --tail=200\nkubectl -n \u003Ccockroachdb-namespace> logs \u003Ccrdb-pod> -c cockroachdb --previous\nkubectl -n \u003Ccockroachdb-namespace> get pod \u003Ccrdb-pod> -o jsonpath='{.spec.containers[0].readinessProbe}{\"\\n\"}'\nkubectl -n \u003Ccockroachdb-namespace> get pods -l app.kubernetes.io\u002Fname=cockroachdb \\\n  -o custom-columns=NAME:.metadata.name,IMAGE:.spec.containers[0].image,PHASE:.status.phase,READY:.status.containerStatuses[0].ready,NODE:.spec.nodeName\n",[3081],{"type":45,"tag":83,"props":3082,"children":3083},{"__ignoreMap":443},[3084,3136,3187,3246,3305,3377,3424],{"type":45,"tag":449,"props":3085,"children":3086},{"class":451,"line":452},[3087,3091,3095,3099,3103,3107,3111,3115,3119,3123,3128,3132],{"type":45,"tag":449,"props":3088,"children":3089},{"style":466},[3090],{"type":50,"value":729},{"type":45,"tag":449,"props":3092,"children":3093},{"style":472},[3094],{"type":50,"value":475},{"type":45,"tag":449,"props":3096,"children":3097},{"style":478},[3098],{"type":50,"value":481},{"type":45,"tag":449,"props":3100,"children":3101},{"style":472},[3102],{"type":50,"value":551},{"type":45,"tag":449,"props":3104,"children":3105},{"style":489},[3106],{"type":50,"value":492},{"type":45,"tag":449,"props":3108,"children":3109},{"style":478},[3110],{"type":50,"value":497},{"type":45,"tag":449,"props":3112,"children":3113},{"style":472},[3114],{"type":50,"value":754},{"type":45,"tag":449,"props":3116,"children":3117},{"style":472},[3118],{"type":50,"value":1904},{"type":45,"tag":449,"props":3120,"children":3121},{"style":472},[3122],{"type":50,"value":837},{"type":45,"tag":449,"props":3124,"children":3125},{"style":472},[3126],{"type":50,"value":3127}," app.kubernetes.io\u002Fname=cockroachdb",{"type":45,"tag":449,"props":3129,"children":3130},{"style":472},[3131],{"type":50,"value":764},{"type":45,"tag":449,"props":3133,"children":3134},{"style":472},[3135],{"type":50,"value":945},{"type":45,"tag":449,"props":3137,"children":3138},{"class":451,"line":462},[3139,3143,3147,3151,3155,3159,3163,3167,3171,3175,3179,3183],{"type":45,"tag":449,"props":3140,"children":3141},{"style":466},[3142],{"type":50,"value":729},{"type":45,"tag":449,"props":3144,"children":3145},{"style":472},[3146],{"type":50,"value":475},{"type":45,"tag":449,"props":3148,"children":3149},{"style":478},[3150],{"type":50,"value":481},{"type":45,"tag":449,"props":3152,"children":3153},{"style":472},[3154],{"type":50,"value":551},{"type":45,"tag":449,"props":3156,"children":3157},{"style":489},[3158],{"type":50,"value":492},{"type":45,"tag":449,"props":3160,"children":3161},{"style":478},[3162],{"type":50,"value":497},{"type":45,"tag":449,"props":3164,"children":3165},{"style":472},[3166],{"type":50,"value":978},{"type":45,"tag":449,"props":3168,"children":3169},{"style":472},[3170],{"type":50,"value":1111},{"type":45,"tag":449,"props":3172,"children":3173},{"style":478},[3174],{"type":50,"value":481},{"type":45,"tag":449,"props":3176,"children":3177},{"style":472},[3178],{"type":50,"value":1120},{"type":45,"tag":449,"props":3180,"children":3181},{"style":489},[3182],{"type":50,"value":1125},{"type":45,"tag":449,"props":3184,"children":3185},{"style":478},[3186],{"type":50,"value":1130},{"type":45,"tag":449,"props":3188,"children":3189},{"class":451,"line":533},[3190,3194,3198,3202,3206,3210,3214,3218,3222,3226,3230,3234,3238,3242],{"type":45,"tag":449,"props":3191,"children":3192},{"style":466},[3193],{"type":50,"value":729},{"type":45,"tag":449,"props":3195,"children":3196},{"style":472},[3197],{"type":50,"value":475},{"type":45,"tag":449,"props":3199,"children":3200},{"style":478},[3201],{"type":50,"value":481},{"type":45,"tag":449,"props":3203,"children":3204},{"style":472},[3205],{"type":50,"value":551},{"type":45,"tag":449,"props":3207,"children":3208},{"style":489},[3209],{"type":50,"value":492},{"type":45,"tag":449,"props":3211,"children":3212},{"style":478},[3213],{"type":50,"value":497},{"type":45,"tag":449,"props":3215,"children":3216},{"style":472},[3217],{"type":50,"value":832},{"type":45,"tag":449,"props":3219,"children":3220},{"style":478},[3221],{"type":50,"value":481},{"type":45,"tag":449,"props":3223,"children":3224},{"style":472},[3225],{"type":50,"value":1120},{"type":45,"tag":449,"props":3227,"children":3228},{"style":489},[3229],{"type":50,"value":1125},{"type":45,"tag":449,"props":3231,"children":3232},{"style":478},[3233],{"type":50,"value":497},{"type":45,"tag":449,"props":3235,"children":3236},{"style":472},[3237],{"type":50,"value":1182},{"type":45,"tag":449,"props":3239,"children":3240},{"style":472},[3241],{"type":50,"value":1187},{"type":45,"tag":449,"props":3243,"children":3244},{"style":472},[3245],{"type":50,"value":1192},{"type":45,"tag":449,"props":3247,"children":3248},{"class":451,"line":591},[3249,3253,3257,3261,3265,3269,3273,3277,3281,3285,3289,3293,3297,3301],{"type":45,"tag":449,"props":3250,"children":3251},{"style":466},[3252],{"type":50,"value":729},{"type":45,"tag":449,"props":3254,"children":3255},{"style":472},[3256],{"type":50,"value":475},{"type":45,"tag":449,"props":3258,"children":3259},{"style":478},[3260],{"type":50,"value":481},{"type":45,"tag":449,"props":3262,"children":3263},{"style":472},[3264],{"type":50,"value":551},{"type":45,"tag":449,"props":3266,"children":3267},{"style":489},[3268],{"type":50,"value":492},{"type":45,"tag":449,"props":3270,"children":3271},{"style":478},[3272],{"type":50,"value":497},{"type":45,"tag":449,"props":3274,"children":3275},{"style":472},[3276],{"type":50,"value":832},{"type":45,"tag":449,"props":3278,"children":3279},{"style":478},[3280],{"type":50,"value":481},{"type":45,"tag":449,"props":3282,"children":3283},{"style":472},[3284],{"type":50,"value":1120},{"type":45,"tag":449,"props":3286,"children":3287},{"style":489},[3288],{"type":50,"value":1125},{"type":45,"tag":449,"props":3290,"children":3291},{"style":478},[3292],{"type":50,"value":497},{"type":45,"tag":449,"props":3294,"children":3295},{"style":472},[3296],{"type":50,"value":1182},{"type":45,"tag":449,"props":3298,"children":3299},{"style":472},[3300],{"type":50,"value":1187},{"type":45,"tag":449,"props":3302,"children":3303},{"style":472},[3304],{"type":50,"value":1252},{"type":45,"tag":449,"props":3306,"children":3307},{"class":451,"line":648},[3308,3312,3316,3320,3324,3328,3332,3336,3340,3344,3348,3352,3356,3360,3364,3368,3373],{"type":45,"tag":449,"props":3309,"children":3310},{"style":466},[3311],{"type":50,"value":729},{"type":45,"tag":449,"props":3313,"children":3314},{"style":472},[3315],{"type":50,"value":475},{"type":45,"tag":449,"props":3317,"children":3318},{"style":478},[3319],{"type":50,"value":481},{"type":45,"tag":449,"props":3321,"children":3322},{"style":472},[3323],{"type":50,"value":551},{"type":45,"tag":449,"props":3325,"children":3326},{"style":489},[3327],{"type":50,"value":492},{"type":45,"tag":449,"props":3329,"children":3330},{"style":478},[3331],{"type":50,"value":497},{"type":45,"tag":449,"props":3333,"children":3334},{"style":472},[3335],{"type":50,"value":754},{"type":45,"tag":449,"props":3337,"children":3338},{"style":472},[3339],{"type":50,"value":1111},{"type":45,"tag":449,"props":3341,"children":3342},{"style":478},[3343],{"type":50,"value":481},{"type":45,"tag":449,"props":3345,"children":3346},{"style":472},[3347],{"type":50,"value":1120},{"type":45,"tag":449,"props":3349,"children":3350},{"style":489},[3351],{"type":50,"value":1125},{"type":45,"tag":449,"props":3353,"children":3354},{"style":478},[3355],{"type":50,"value":497},{"type":45,"tag":449,"props":3357,"children":3358},{"style":472},[3359],{"type":50,"value":764},{"type":45,"tag":449,"props":3361,"children":3362},{"style":472},[3363],{"type":50,"value":1711},{"type":45,"tag":449,"props":3365,"children":3366},{"style":478},[3367],{"type":50,"value":1716},{"type":45,"tag":449,"props":3369,"children":3370},{"style":472},[3371],{"type":50,"value":3372},"{.spec.containers[0].readinessProbe}{\"\\n\"}",{"type":45,"tag":449,"props":3374,"children":3375},{"style":478},[3376],{"type":50,"value":799},{"type":45,"tag":449,"props":3378,"children":3379},{"class":451,"line":704},[3380,3384,3388,3392,3396,3400,3404,3408,3412,3416,3420],{"type":45,"tag":449,"props":3381,"children":3382},{"style":466},[3383],{"type":50,"value":729},{"type":45,"tag":449,"props":3385,"children":3386},{"style":472},[3387],{"type":50,"value":475},{"type":45,"tag":449,"props":3389,"children":3390},{"style":478},[3391],{"type":50,"value":481},{"type":45,"tag":449,"props":3393,"children":3394},{"style":472},[3395],{"type":50,"value":551},{"type":45,"tag":449,"props":3397,"children":3398},{"style":489},[3399],{"type":50,"value":492},{"type":45,"tag":449,"props":3401,"children":3402},{"style":478},[3403],{"type":50,"value":497},{"type":45,"tag":449,"props":3405,"children":3406},{"style":472},[3407],{"type":50,"value":754},{"type":45,"tag":449,"props":3409,"children":3410},{"style":472},[3411],{"type":50,"value":1904},{"type":45,"tag":449,"props":3413,"children":3414},{"style":472},[3415],{"type":50,"value":837},{"type":45,"tag":449,"props":3417,"children":3418},{"style":472},[3419],{"type":50,"value":3127},{"type":45,"tag":449,"props":3421,"children":3422},{"style":489},[3423],{"type":50,"value":2305},{"type":45,"tag":449,"props":3425,"children":3426},{"class":451,"line":714},[3427,3431],{"type":45,"tag":449,"props":3428,"children":3429},{"style":472},[3430],{"type":50,"value":2707},{"type":45,"tag":449,"props":3432,"children":3433},{"style":472},[3434],{"type":50,"value":3435}," custom-columns=NAME:.metadata.name,IMAGE:.spec.containers[0].image,PHASE:.status.phase,READY:.status.containerStatuses[0].ready,NODE:.spec.nodeName\n",{"type":45,"tag":53,"props":3437,"children":3438},{},[3439],{"type":50,"value":3440},"Common pod issues:",{"type":45,"tag":75,"props":3442,"children":3443},{},[3444,3449,3454],{"type":45,"tag":79,"props":3445,"children":3446},{},[3447],{"type":50,"value":3448},"Pending during upgrade: old pods may still carry pre-upgrade scheduling or affinity constraints.",{"type":45,"tag":79,"props":3450,"children":3451},{},[3452],{"type":50,"value":3453},"CrashLoopBackOff: inspect storage errors, TLS errors, join address failures, and previous logs.",{"type":45,"tag":79,"props":3455,"children":3456},{},[3457],{"type":50,"value":3458},"Running but not Ready: check the readiness probe, CRDB health endpoint, certificate trust, join service, and whether the node is recovering.",{"type":45,"tag":68,"props":3460,"children":3462},{"id":3461},"upgrade-and-version-validation",[3463],{"type":50,"value":3464},"Upgrade and Version Validation",{"type":45,"tag":438,"props":3466,"children":3468},{"className":440,"code":3467,"language":442,"meta":443,"style":443},"kubectl -n \u003Ccockroachdb-namespace> get crdbcluster \u003Ccockroachdb-release> -o json | jq '{\n  specImage: .spec.image,\n  statusImage: .status.image,\n  actions: .status.actions,\n  conditions: [.status.conditions[]? | select(.type | test(\"Upgrade|Version|Validate\"))]\n}'\n\nkubectl -n \u003Ccockroachdb-namespace> get crdbcluster \u003Ccockroachdb-release> -o jsonpath='{.metadata.annotations}{\"\\n\"}' | jq .\nkubectl -n \u003Ccockroachdb-namespace> get jobs\nkubectl -n \u003Ccockroachdb-namespace> describe job \u003Cversion-checker-job>\nkubectl -n \u003Ccockroachdb-namespace> logs -l job-name=\u003Cversion-checker-job>\nkubectl -n \u003Ccockroachdb-namespace> get pods -l app.kubernetes.io\u002Fname=cockroachdb \\\n  -o custom-columns=NAME:.metadata.name,IMAGE:.spec.containers[0].image,REVISION:.metadata.annotations[\"crdb\\.cockroachlabs\\.com\u002Fhash-revision\"],PHASE:.status.phase\n",[3469],{"type":45,"tag":83,"props":3470,"children":3471},{"__ignoreMap":443},[3472,3547,3555,3562,3569,3577,3588,3595,3680,3716,3770,3827,3874],{"type":45,"tag":449,"props":3473,"children":3474},{"class":451,"line":452},[3475,3479,3483,3487,3491,3495,3499,3503,3507,3511,3515,3519,3523,3527,3531,3535,3539,3543],{"type":45,"tag":449,"props":3476,"children":3477},{"style":466},[3478],{"type":50,"value":729},{"type":45,"tag":449,"props":3480,"children":3481},{"style":472},[3482],{"type":50,"value":475},{"type":45,"tag":449,"props":3484,"children":3485},{"style":478},[3486],{"type":50,"value":481},{"type":45,"tag":449,"props":3488,"children":3489},{"style":472},[3490],{"type":50,"value":551},{"type":45,"tag":449,"props":3492,"children":3493},{"style":489},[3494],{"type":50,"value":492},{"type":45,"tag":449,"props":3496,"children":3497},{"style":478},[3498],{"type":50,"value":497},{"type":45,"tag":449,"props":3500,"children":3501},{"style":472},[3502],{"type":50,"value":754},{"type":45,"tag":449,"props":3504,"children":3505},{"style":472},[3506],{"type":50,"value":983},{"type":45,"tag":449,"props":3508,"children":3509},{"style":478},[3510],{"type":50,"value":481},{"type":45,"tag":449,"props":3512,"children":3513},{"style":472},[3514],{"type":50,"value":572},{"type":45,"tag":449,"props":3516,"children":3517},{"style":489},[3518],{"type":50,"value":492},{"type":45,"tag":449,"props":3520,"children":3521},{"style":478},[3522],{"type":50,"value":497},{"type":45,"tag":449,"props":3524,"children":3525},{"style":472},[3526],{"type":50,"value":764},{"type":45,"tag":449,"props":3528,"children":3529},{"style":472},[3530],{"type":50,"value":2857},{"type":45,"tag":449,"props":3532,"children":3533},{"style":478},[3534],{"type":50,"value":774},{"type":45,"tag":449,"props":3536,"children":3537},{"style":466},[3538],{"type":50,"value":2866},{"type":45,"tag":449,"props":3540,"children":3541},{"style":478},[3542],{"type":50,"value":789},{"type":45,"tag":449,"props":3544,"children":3545},{"style":472},[3546],{"type":50,"value":2875},{"type":45,"tag":449,"props":3548,"children":3549},{"class":451,"line":462},[3550],{"type":45,"tag":449,"props":3551,"children":3552},{"style":472},[3553],{"type":50,"value":3554},"  specImage: .spec.image,\n",{"type":45,"tag":449,"props":3556,"children":3557},{"class":451,"line":533},[3558],{"type":45,"tag":449,"props":3559,"children":3560},{"style":472},[3561],{"type":50,"value":2915},{"type":45,"tag":449,"props":3563,"children":3564},{"class":451,"line":591},[3565],{"type":45,"tag":449,"props":3566,"children":3567},{"style":472},[3568],{"type":50,"value":2923},{"type":45,"tag":449,"props":3570,"children":3571},{"class":451,"line":648},[3572],{"type":45,"tag":449,"props":3573,"children":3574},{"style":472},[3575],{"type":50,"value":3576},"  conditions: [.status.conditions[]? | select(.type | test(\"Upgrade|Version|Validate\"))]\n",{"type":45,"tag":449,"props":3578,"children":3579},{"class":451,"line":704},[3580,3584],{"type":45,"tag":449,"props":3581,"children":3582},{"style":472},[3583],{"type":50,"value":2939},{"type":45,"tag":449,"props":3585,"children":3586},{"style":478},[3587],{"type":50,"value":799},{"type":45,"tag":449,"props":3589,"children":3590},{"class":451,"line":714},[3591],{"type":45,"tag":449,"props":3592,"children":3593},{"emptyLinePlaceholder":708},[3594],{"type":50,"value":711},{"type":45,"tag":449,"props":3596,"children":3597},{"class":451,"line":723},[3598,3602,3606,3610,3614,3618,3622,3626,3630,3634,3638,3642,3646,3650,3654,3658,3663,3667,3671,3675],{"type":45,"tag":449,"props":3599,"children":3600},{"style":466},[3601],{"type":50,"value":729},{"type":45,"tag":449,"props":3603,"children":3604},{"style":472},[3605],{"type":50,"value":475},{"type":45,"tag":449,"props":3607,"children":3608},{"style":478},[3609],{"type":50,"value":481},{"type":45,"tag":449,"props":3611,"children":3612},{"style":472},[3613],{"type":50,"value":551},{"type":45,"tag":449,"props":3615,"children":3616},{"style":489},[3617],{"type":50,"value":492},{"type":45,"tag":449,"props":3619,"children":3620},{"style":478},[3621],{"type":50,"value":497},{"type":45,"tag":449,"props":3623,"children":3624},{"style":472},[3625],{"type":50,"value":754},{"type":45,"tag":449,"props":3627,"children":3628},{"style":472},[3629],{"type":50,"value":983},{"type":45,"tag":449,"props":3631,"children":3632},{"style":478},[3633],{"type":50,"value":481},{"type":45,"tag":449,"props":3635,"children":3636},{"style":472},[3637],{"type":50,"value":572},{"type":45,"tag":449,"props":3639,"children":3640},{"style":489},[3641],{"type":50,"value":492},{"type":45,"tag":449,"props":3643,"children":3644},{"style":478},[3645],{"type":50,"value":497},{"type":45,"tag":449,"props":3647,"children":3648},{"style":472},[3649],{"type":50,"value":764},{"type":45,"tag":449,"props":3651,"children":3652},{"style":472},[3653],{"type":50,"value":1711},{"type":45,"tag":449,"props":3655,"children":3656},{"style":478},[3657],{"type":50,"value":1716},{"type":45,"tag":449,"props":3659,"children":3660},{"style":472},[3661],{"type":50,"value":3662},"{.metadata.annotations}{\"\\n\"}",{"type":45,"tag":449,"props":3664,"children":3665},{"style":478},[3666],{"type":50,"value":1716},{"type":45,"tag":449,"props":3668,"children":3669},{"style":478},[3670],{"type":50,"value":774},{"type":45,"tag":449,"props":3672,"children":3673},{"style":466},[3674],{"type":50,"value":2866},{"type":45,"tag":449,"props":3676,"children":3677},{"style":472},[3678],{"type":50,"value":3679}," .\n",{"type":45,"tag":449,"props":3681,"children":3682},{"class":451,"line":802},[3683,3687,3691,3695,3699,3703,3707,3711],{"type":45,"tag":449,"props":3684,"children":3685},{"style":466},[3686],{"type":50,"value":729},{"type":45,"tag":449,"props":3688,"children":3689},{"style":472},[3690],{"type":50,"value":475},{"type":45,"tag":449,"props":3692,"children":3693},{"style":478},[3694],{"type":50,"value":481},{"type":45,"tag":449,"props":3696,"children":3697},{"style":472},[3698],{"type":50,"value":551},{"type":45,"tag":449,"props":3700,"children":3701},{"style":489},[3702],{"type":50,"value":492},{"type":45,"tag":449,"props":3704,"children":3705},{"style":478},[3706],{"type":50,"value":497},{"type":45,"tag":449,"props":3708,"children":3709},{"style":472},[3710],{"type":50,"value":754},{"type":45,"tag":449,"props":3712,"children":3713},{"style":472},[3714],{"type":50,"value":3715}," jobs\n",{"type":45,"tag":449,"props":3717,"children":3718},{"class":451,"line":858},[3719,3723,3727,3731,3735,3739,3743,3747,3752,3756,3761,3766],{"type":45,"tag":449,"props":3720,"children":3721},{"style":466},[3722],{"type":50,"value":729},{"type":45,"tag":449,"props":3724,"children":3725},{"style":472},[3726],{"type":50,"value":475},{"type":45,"tag":449,"props":3728,"children":3729},{"style":478},[3730],{"type":50,"value":481},{"type":45,"tag":449,"props":3732,"children":3733},{"style":472},[3734],{"type":50,"value":551},{"type":45,"tag":449,"props":3736,"children":3737},{"style":489},[3738],{"type":50,"value":492},{"type":45,"tag":449,"props":3740,"children":3741},{"style":478},[3742],{"type":50,"value":497},{"type":45,"tag":449,"props":3744,"children":3745},{"style":472},[3746],{"type":50,"value":978},{"type":45,"tag":449,"props":3748,"children":3749},{"style":472},[3750],{"type":50,"value":3751}," job",{"type":45,"tag":449,"props":3753,"children":3754},{"style":478},[3755],{"type":50,"value":481},{"type":45,"tag":449,"props":3757,"children":3758},{"style":472},[3759],{"type":50,"value":3760},"version-checker-jo",{"type":45,"tag":449,"props":3762,"children":3763},{"style":489},[3764],{"type":50,"value":3765},"b",{"type":45,"tag":449,"props":3767,"children":3768},{"style":478},[3769],{"type":50,"value":1130},{"type":45,"tag":449,"props":3771,"children":3772},{"class":451,"line":866},[3773,3777,3781,3785,3789,3793,3797,3801,3805,3810,3815,3819,3823],{"type":45,"tag":449,"props":3774,"children":3775},{"style":466},[3776],{"type":50,"value":729},{"type":45,"tag":449,"props":3778,"children":3779},{"style":472},[3780],{"type":50,"value":475},{"type":45,"tag":449,"props":3782,"children":3783},{"style":478},[3784],{"type":50,"value":481},{"type":45,"tag":449,"props":3786,"children":3787},{"style":472},[3788],{"type":50,"value":551},{"type":45,"tag":449,"props":3790,"children":3791},{"style":489},[3792],{"type":50,"value":492},{"type":45,"tag":449,"props":3794,"children":3795},{"style":478},[3796],{"type":50,"value":497},{"type":45,"tag":449,"props":3798,"children":3799},{"style":472},[3800],{"type":50,"value":832},{"type":45,"tag":449,"props":3802,"children":3803},{"style":472},[3804],{"type":50,"value":837},{"type":45,"tag":449,"props":3806,"children":3807},{"style":472},[3808],{"type":50,"value":3809}," job-name=",{"type":45,"tag":449,"props":3811,"children":3812},{"style":478},[3813],{"type":50,"value":3814},"\u003C",{"type":45,"tag":449,"props":3816,"children":3817},{"style":472},[3818],{"type":50,"value":3760},{"type":45,"tag":449,"props":3820,"children":3821},{"style":489},[3822],{"type":50,"value":3765},{"type":45,"tag":449,"props":3824,"children":3825},{"style":478},[3826],{"type":50,"value":1130},{"type":45,"tag":449,"props":3828,"children":3829},{"class":451,"line":875},[3830,3834,3838,3842,3846,3850,3854,3858,3862,3866,3870],{"type":45,"tag":449,"props":3831,"children":3832},{"style":466},[3833],{"type":50,"value":729},{"type":45,"tag":449,"props":3835,"children":3836},{"style":472},[3837],{"type":50,"value":475},{"type":45,"tag":449,"props":3839,"children":3840},{"style":478},[3841],{"type":50,"value":481},{"type":45,"tag":449,"props":3843,"children":3844},{"style":472},[3845],{"type":50,"value":551},{"type":45,"tag":449,"props":3847,"children":3848},{"style":489},[3849],{"type":50,"value":492},{"type":45,"tag":449,"props":3851,"children":3852},{"style":478},[3853],{"type":50,"value":497},{"type":45,"tag":449,"props":3855,"children":3856},{"style":472},[3857],{"type":50,"value":754},{"type":45,"tag":449,"props":3859,"children":3860},{"style":472},[3861],{"type":50,"value":1904},{"type":45,"tag":449,"props":3863,"children":3864},{"style":472},[3865],{"type":50,"value":837},{"type":45,"tag":449,"props":3867,"children":3868},{"style":472},[3869],{"type":50,"value":3127},{"type":45,"tag":449,"props":3871,"children":3872},{"style":489},[3873],{"type":50,"value":2305},{"type":45,"tag":449,"props":3875,"children":3876},{"class":451,"line":902},[3877,3881,3886,3890,3895,3899],{"type":45,"tag":449,"props":3878,"children":3879},{"style":472},[3880],{"type":50,"value":2707},{"type":45,"tag":449,"props":3882,"children":3883},{"style":472},[3884],{"type":50,"value":3885}," custom-columns=NAME:.metadata.name,IMAGE:.spec.containers[0].image,REVISION:.metadata.annotations[",{"type":45,"tag":449,"props":3887,"children":3888},{"style":478},[3889],{"type":50,"value":3007},{"type":45,"tag":449,"props":3891,"children":3892},{"style":472},[3893],{"type":50,"value":3894},"crdb\\.cockroachlabs\\.com\u002Fhash-revision",{"type":45,"tag":449,"props":3896,"children":3897},{"style":478},[3898],{"type":50,"value":3007},{"type":45,"tag":449,"props":3900,"children":3901},{"style":472},[3902],{"type":50,"value":3903},"],PHASE:.status.phase\n",{"type":45,"tag":53,"props":3905,"children":3906},{},[3907],{"type":50,"value":2081},{"type":45,"tag":75,"props":3909,"children":3910},{},[3911,3932,3937,3942],{"type":45,"tag":79,"props":3912,"children":3913},{},[3914,3916,3922,3924,3930],{"type":50,"value":3915},"If ",{"type":45,"tag":83,"props":3917,"children":3919},{"className":3918},[],[3920],{"type":50,"value":3921},"spec.image",{"type":50,"value":3923}," differs from ",{"type":45,"tag":83,"props":3925,"children":3927},{"className":3926},[],[3928],{"type":50,"value":3929},"status.image",{"type":50,"value":3931},", an upgrade is in progress or stuck.",{"type":45,"tag":79,"props":3933,"children":3934},{},[3935],{"type":50,"value":3936},"If a rejected-image annotation exists, inspect its value; the operator rejected the target version.",{"type":45,"tag":79,"props":3938,"children":3939},{},[3940],{"type":50,"value":3941},"If the version checker job exists but the pod is gone, use job status and operator logs for validation messages.",{"type":45,"tag":79,"props":3943,"children":3944},{},[3945],{"type":50,"value":3946},"Do not delete version checker jobs or pods until their status and logs are captured.",{"type":45,"tag":68,"props":3948,"children":3950},{"id":3949},"dns-service-and-network-issues",[3951],{"type":50,"value":3952},"DNS, Service, and Network Issues",{"type":45,"tag":53,"props":3954,"children":3955},{},[3956],{"type":50,"value":3957},"The operator creates separate service paths for pod DNS and join traffic. Do not change service settings without operator-team guidance.",{"type":45,"tag":438,"props":3959,"children":3961},{"className":440,"code":3960,"language":442,"meta":443,"style":443},"kubectl -n \u003Ccockroachdb-namespace> get service \u003Ccockroachdb-release> -o yaml\nkubectl -n \u003Ccockroachdb-namespace> get service \u003Ccockroachdb-release>-join -o yaml\nkubectl -n \u003Ccockroachdb-namespace> get endpoints \u003Ccockroachdb-release>\nkubectl -n \u003Ccockroachdb-namespace> get endpoints \u003Ccockroachdb-release>-join\n\nkubectl -n \u003Ccockroachdb-namespace> exec \u003Ccrdb-pod> -c cockroachdb -- \\\n  nslookup \u003Ccockroachdb-release>.\u003Ccockroachdb-namespace>.svc.cluster.local 2>&1 || true\n\nkubectl -n \u003Ccockroachdb-namespace> exec \u003Ccrdb-pod> -c cockroachdb -- \\\n  nslookup \u003Ccockroachdb-release>-join.\u003Ccockroachdb-namespace>.svc.cluster.local 2>&1 || true\n",[3962],{"type":45,"tag":83,"props":3963,"children":3964},{"__ignoreMap":443},[3965,4026,4090,4141,4197,4204,4269,4331,4338,4401],{"type":45,"tag":449,"props":3966,"children":3967},{"class":451,"line":452},[3968,3972,3976,3980,3984,3988,3992,3996,4001,4005,4009,4013,4017,4021],{"type":45,"tag":449,"props":3969,"children":3970},{"style":466},[3971],{"type":50,"value":729},{"type":45,"tag":449,"props":3973,"children":3974},{"style":472},[3975],{"type":50,"value":475},{"type":45,"tag":449,"props":3977,"children":3978},{"style":478},[3979],{"type":50,"value":481},{"type":45,"tag":449,"props":3981,"children":3982},{"style":472},[3983],{"type":50,"value":551},{"type":45,"tag":449,"props":3985,"children":3986},{"style":489},[3987],{"type":50,"value":492},{"type":45,"tag":449,"props":3989,"children":3990},{"style":478},[3991],{"type":50,"value":497},{"type":45,"tag":449,"props":3993,"children":3994},{"style":472},[3995],{"type":50,"value":754},{"type":45,"tag":449,"props":3997,"children":3998},{"style":472},[3999],{"type":50,"value":4000}," service",{"type":45,"tag":449,"props":4002,"children":4003},{"style":478},[4004],{"type":50,"value":481},{"type":45,"tag":449,"props":4006,"children":4007},{"style":472},[4008],{"type":50,"value":572},{"type":45,"tag":449,"props":4010,"children":4011},{"style":489},[4012],{"type":50,"value":492},{"type":45,"tag":449,"props":4014,"children":4015},{"style":478},[4016],{"type":50,"value":497},{"type":45,"tag":449,"props":4018,"children":4019},{"style":472},[4020],{"type":50,"value":764},{"type":45,"tag":449,"props":4022,"children":4023},{"style":472},[4024],{"type":50,"value":4025}," yaml\n",{"type":45,"tag":449,"props":4027,"children":4028},{"class":451,"line":462},[4029,4033,4037,4041,4045,4049,4053,4057,4061,4065,4069,4073,4077,4082,4086],{"type":45,"tag":449,"props":4030,"children":4031},{"style":466},[4032],{"type":50,"value":729},{"type":45,"tag":449,"props":4034,"children":4035},{"style":472},[4036],{"type":50,"value":475},{"type":45,"tag":449,"props":4038,"children":4039},{"style":478},[4040],{"type":50,"value":481},{"type":45,"tag":449,"props":4042,"children":4043},{"style":472},[4044],{"type":50,"value":551},{"type":45,"tag":449,"props":4046,"children":4047},{"style":489},[4048],{"type":50,"value":492},{"type":45,"tag":449,"props":4050,"children":4051},{"style":478},[4052],{"type":50,"value":497},{"type":45,"tag":449,"props":4054,"children":4055},{"style":472},[4056],{"type":50,"value":754},{"type":45,"tag":449,"props":4058,"children":4059},{"style":472},[4060],{"type":50,"value":4000},{"type":45,"tag":449,"props":4062,"children":4063},{"style":478},[4064],{"type":50,"value":481},{"type":45,"tag":449,"props":4066,"children":4067},{"style":472},[4068],{"type":50,"value":572},{"type":45,"tag":449,"props":4070,"children":4071},{"style":489},[4072],{"type":50,"value":492},{"type":45,"tag":449,"props":4074,"children":4075},{"style":478},[4076],{"type":50,"value":497},{"type":45,"tag":449,"props":4078,"children":4079},{"style":472},[4080],{"type":50,"value":4081},"-join",{"type":45,"tag":449,"props":4083,"children":4084},{"style":472},[4085],{"type":50,"value":764},{"type":45,"tag":449,"props":4087,"children":4088},{"style":472},[4089],{"type":50,"value":4025},{"type":45,"tag":449,"props":4091,"children":4092},{"class":451,"line":533},[4093,4097,4101,4105,4109,4113,4117,4121,4125,4129,4133,4137],{"type":45,"tag":449,"props":4094,"children":4095},{"style":466},[4096],{"type":50,"value":729},{"type":45,"tag":449,"props":4098,"children":4099},{"style":472},[4100],{"type":50,"value":475},{"type":45,"tag":449,"props":4102,"children":4103},{"style":478},[4104],{"type":50,"value":481},{"type":45,"tag":449,"props":4106,"children":4107},{"style":472},[4108],{"type":50,"value":551},{"type":45,"tag":449,"props":4110,"children":4111},{"style":489},[4112],{"type":50,"value":492},{"type":45,"tag":449,"props":4114,"children":4115},{"style":478},[4116],{"type":50,"value":497},{"type":45,"tag":449,"props":4118,"children":4119},{"style":472},[4120],{"type":50,"value":754},{"type":45,"tag":449,"props":4122,"children":4123},{"style":472},[4124],{"type":50,"value":2629},{"type":45,"tag":449,"props":4126,"children":4127},{"style":478},[4128],{"type":50,"value":481},{"type":45,"tag":449,"props":4130,"children":4131},{"style":472},[4132],{"type":50,"value":572},{"type":45,"tag":449,"props":4134,"children":4135},{"style":489},[4136],{"type":50,"value":492},{"type":45,"tag":449,"props":4138,"children":4139},{"style":478},[4140],{"type":50,"value":1130},{"type":45,"tag":449,"props":4142,"children":4143},{"class":451,"line":591},[4144,4148,4152,4156,4160,4164,4168,4172,4176,4180,4184,4188,4192],{"type":45,"tag":449,"props":4145,"children":4146},{"style":466},[4147],{"type":50,"value":729},{"type":45,"tag":449,"props":4149,"children":4150},{"style":472},[4151],{"type":50,"value":475},{"type":45,"tag":449,"props":4153,"children":4154},{"style":478},[4155],{"type":50,"value":481},{"type":45,"tag":449,"props":4157,"children":4158},{"style":472},[4159],{"type":50,"value":551},{"type":45,"tag":449,"props":4161,"children":4162},{"style":489},[4163],{"type":50,"value":492},{"type":45,"tag":449,"props":4165,"children":4166},{"style":478},[4167],{"type":50,"value":497},{"type":45,"tag":449,"props":4169,"children":4170},{"style":472},[4171],{"type":50,"value":754},{"type":45,"tag":449,"props":4173,"children":4174},{"style":472},[4175],{"type":50,"value":2629},{"type":45,"tag":449,"props":4177,"children":4178},{"style":478},[4179],{"type":50,"value":481},{"type":45,"tag":449,"props":4181,"children":4182},{"style":472},[4183],{"type":50,"value":572},{"type":45,"tag":449,"props":4185,"children":4186},{"style":489},[4187],{"type":50,"value":492},{"type":45,"tag":449,"props":4189,"children":4190},{"style":478},[4191],{"type":50,"value":497},{"type":45,"tag":449,"props":4193,"children":4194},{"style":472},[4195],{"type":50,"value":4196},"-join\n",{"type":45,"tag":449,"props":4198,"children":4199},{"class":451,"line":648},[4200],{"type":45,"tag":449,"props":4201,"children":4202},{"emptyLinePlaceholder":708},[4203],{"type":50,"value":711},{"type":45,"tag":449,"props":4205,"children":4206},{"class":451,"line":704},[4207,4211,4215,4219,4223,4227,4231,4236,4240,4244,4248,4252,4256,4260,4265],{"type":45,"tag":449,"props":4208,"children":4209},{"style":466},[4210],{"type":50,"value":729},{"type":45,"tag":449,"props":4212,"children":4213},{"style":472},[4214],{"type":50,"value":475},{"type":45,"tag":449,"props":4216,"children":4217},{"style":478},[4218],{"type":50,"value":481},{"type":45,"tag":449,"props":4220,"children":4221},{"style":472},[4222],{"type":50,"value":551},{"type":45,"tag":449,"props":4224,"children":4225},{"style":489},[4226],{"type":50,"value":492},{"type":45,"tag":449,"props":4228,"children":4229},{"style":478},[4230],{"type":50,"value":497},{"type":45,"tag":449,"props":4232,"children":4233},{"style":472},[4234],{"type":50,"value":4235}," exec",{"type":45,"tag":449,"props":4237,"children":4238},{"style":478},[4239],{"type":50,"value":481},{"type":45,"tag":449,"props":4241,"children":4242},{"style":472},[4243],{"type":50,"value":1120},{"type":45,"tag":449,"props":4245,"children":4246},{"style":489},[4247],{"type":50,"value":1125},{"type":45,"tag":449,"props":4249,"children":4250},{"style":478},[4251],{"type":50,"value":497},{"type":45,"tag":449,"props":4253,"children":4254},{"style":472},[4255],{"type":50,"value":1182},{"type":45,"tag":449,"props":4257,"children":4258},{"style":472},[4259],{"type":50,"value":1187},{"type":45,"tag":449,"props":4261,"children":4262},{"style":472},[4263],{"type":50,"value":4264}," --",{"type":45,"tag":449,"props":4266,"children":4267},{"style":489},[4268],{"type":50,"value":2305},{"type":45,"tag":449,"props":4270,"children":4271},{"class":451,"line":714},[4272,4277,4281,4285,4289,4293,4297,4301,4305,4309,4313,4318,4323,4327],{"type":45,"tag":449,"props":4273,"children":4274},{"style":472},[4275],{"type":50,"value":4276},"  nslookup",{"type":45,"tag":449,"props":4278,"children":4279},{"style":478},[4280],{"type":50,"value":481},{"type":45,"tag":449,"props":4282,"children":4283},{"style":472},[4284],{"type":50,"value":572},{"type":45,"tag":449,"props":4286,"children":4287},{"style":489},[4288],{"type":50,"value":492},{"type":45,"tag":449,"props":4290,"children":4291},{"style":478},[4292],{"type":50,"value":497},{"type":45,"tag":449,"props":4294,"children":4295},{"style":472},[4296],{"type":50,"value":66},{"type":45,"tag":449,"props":4298,"children":4299},{"style":478},[4300],{"type":50,"value":3814},{"type":45,"tag":449,"props":4302,"children":4303},{"style":472},[4304],{"type":50,"value":551},{"type":45,"tag":449,"props":4306,"children":4307},{"style":489},[4308],{"type":50,"value":492},{"type":45,"tag":449,"props":4310,"children":4311},{"style":478},[4312],{"type":50,"value":497},{"type":45,"tag":449,"props":4314,"children":4315},{"style":472},[4316],{"type":50,"value":4317},".svc.cluster.local",{"type":45,"tag":449,"props":4319,"children":4320},{"style":478},[4321],{"type":50,"value":4322}," 2>&1",{"type":45,"tag":449,"props":4324,"children":4325},{"style":478},[4326],{"type":50,"value":524},{"type":45,"tag":449,"props":4328,"children":4329},{"style":527},[4330],{"type":50,"value":530},{"type":45,"tag":449,"props":4332,"children":4333},{"class":451,"line":723},[4334],{"type":45,"tag":449,"props":4335,"children":4336},{"emptyLinePlaceholder":708},[4337],{"type":50,"value":711},{"type":45,"tag":449,"props":4339,"children":4340},{"class":451,"line":802},[4341,4345,4349,4353,4357,4361,4365,4369,4373,4377,4381,4385,4389,4393,4397],{"type":45,"tag":449,"props":4342,"children":4343},{"style":466},[4344],{"type":50,"value":729},{"type":45,"tag":449,"props":4346,"children":4347},{"style":472},[4348],{"type":50,"value":475},{"type":45,"tag":449,"props":4350,"children":4351},{"style":478},[4352],{"type":50,"value":481},{"type":45,"tag":449,"props":4354,"children":4355},{"style":472},[4356],{"type":50,"value":551},{"type":45,"tag":449,"props":4358,"children":4359},{"style":489},[4360],{"type":50,"value":492},{"type":45,"tag":449,"props":4362,"children":4363},{"style":478},[4364],{"type":50,"value":497},{"type":45,"tag":449,"props":4366,"children":4367},{"style":472},[4368],{"type":50,"value":4235},{"type":45,"tag":449,"props":4370,"children":4371},{"style":478},[4372],{"type":50,"value":481},{"type":45,"tag":449,"props":4374,"children":4375},{"style":472},[4376],{"type":50,"value":1120},{"type":45,"tag":449,"props":4378,"children":4379},{"style":489},[4380],{"type":50,"value":1125},{"type":45,"tag":449,"props":4382,"children":4383},{"style":478},[4384],{"type":50,"value":497},{"type":45,"tag":449,"props":4386,"children":4387},{"style":472},[4388],{"type":50,"value":1182},{"type":45,"tag":449,"props":4390,"children":4391},{"style":472},[4392],{"type":50,"value":1187},{"type":45,"tag":449,"props":4394,"children":4395},{"style":472},[4396],{"type":50,"value":4264},{"type":45,"tag":449,"props":4398,"children":4399},{"style":489},[4400],{"type":50,"value":2305},{"type":45,"tag":449,"props":4402,"children":4403},{"class":451,"line":858},[4404,4408,4412,4416,4420,4424,4429,4433,4437,4441,4445,4449,4453,4457],{"type":45,"tag":449,"props":4405,"children":4406},{"style":472},[4407],{"type":50,"value":4276},{"type":45,"tag":449,"props":4409,"children":4410},{"style":478},[4411],{"type":50,"value":481},{"type":45,"tag":449,"props":4413,"children":4414},{"style":472},[4415],{"type":50,"value":572},{"type":45,"tag":449,"props":4417,"children":4418},{"style":489},[4419],{"type":50,"value":492},{"type":45,"tag":449,"props":4421,"children":4422},{"style":478},[4423],{"type":50,"value":497},{"type":45,"tag":449,"props":4425,"children":4426},{"style":472},[4427],{"type":50,"value":4428},"-join.",{"type":45,"tag":449,"props":4430,"children":4431},{"style":478},[4432],{"type":50,"value":3814},{"type":45,"tag":449,"props":4434,"children":4435},{"style":472},[4436],{"type":50,"value":551},{"type":45,"tag":449,"props":4438,"children":4439},{"style":489},[4440],{"type":50,"value":492},{"type":45,"tag":449,"props":4442,"children":4443},{"style":478},[4444],{"type":50,"value":497},{"type":45,"tag":449,"props":4446,"children":4447},{"style":472},[4448],{"type":50,"value":4317},{"type":45,"tag":449,"props":4450,"children":4451},{"style":478},[4452],{"type":50,"value":4322},{"type":45,"tag":449,"props":4454,"children":4455},{"style":478},[4456],{"type":50,"value":524},{"type":45,"tag":449,"props":4458,"children":4459},{"style":527},[4460],{"type":50,"value":530},{"type":45,"tag":53,"props":4462,"children":4463},{},[4464,4466,4470],{"type":50,"value":4465},"For multi-region checks, use ",{"type":45,"tag":59,"props":4467,"children":4468},{"href":192},[4469],{"type":50,"value":195},{"type":50,"value":66},{"type":45,"tag":68,"props":4472,"children":4474},{"id":4473},"tls-and-certificate-failures",[4475],{"type":50,"value":4476},"TLS and Certificate Failures",{"type":45,"tag":53,"props":4478,"children":4479},{},[4480,4481,4485],{"type":50,"value":155},{"type":45,"tag":59,"props":4482,"children":4483},{"href":180},[4484],{"type":50,"value":183},{"type":50,"value":4486}," for TLS mode selection and detailed certificate checks.",{"type":45,"tag":53,"props":4488,"children":4489},{},[4490],{"type":50,"value":4491},"Quick checks:",{"type":45,"tag":438,"props":4493,"children":4495},{"className":440,"code":4494,"language":442,"meta":443,"style":443},"helm template \u003Crelease> \u003Cchart> -n \u003Cnamespace> -f values.yaml >\u002Ftmp\u002Frendered.yaml\nkubectl -n \u003Cnamespace> get secret,configmap | grep -E 'cockroach|crdb|cert|ca|tls'\nkubectl -n \u003Cnamespace> get crdbcluster \u003Crelease> -o yaml | grep -A20 certificates\nkubectl -n \u003Cnamespace> get pod \u003Ccrdb-pod> -o jsonpath='{.spec.containers[*].name}{\"\\n\"}'\nkubectl -n \u003Cnamespace> logs \u003Ccrdb-pod> -c cert-reloader --tail=100\n",[4496],{"type":45,"tag":83,"props":4497,"children":4498},{"__ignoreMap":443},[4499,4586,4647,4725,4797],{"type":45,"tag":449,"props":4500,"children":4501},{"class":451,"line":452},[4502,4506,4511,4515,4520,4524,4528,4532,4537,4541,4545,4549,4553,4558,4562,4566,4571,4576,4581],{"type":45,"tag":449,"props":4503,"children":4504},{"style":466},[4505],{"type":50,"value":469},{"type":45,"tag":449,"props":4507,"children":4508},{"style":472},[4509],{"type":50,"value":4510}," template",{"type":45,"tag":449,"props":4512,"children":4513},{"style":478},[4514],{"type":50,"value":481},{"type":45,"tag":449,"props":4516,"children":4517},{"style":472},[4518],{"type":50,"value":4519},"releas",{"type":45,"tag":449,"props":4521,"children":4522},{"style":489},[4523],{"type":50,"value":492},{"type":45,"tag":449,"props":4525,"children":4526},{"style":478},[4527],{"type":50,"value":497},{"type":45,"tag":449,"props":4529,"children":4530},{"style":478},[4531],{"type":50,"value":481},{"type":45,"tag":449,"props":4533,"children":4534},{"style":472},[4535],{"type":50,"value":4536},"char",{"type":45,"tag":449,"props":4538,"children":4539},{"style":489},[4540],{"type":50,"value":2296},{"type":45,"tag":449,"props":4542,"children":4543},{"style":478},[4544],{"type":50,"value":497},{"type":45,"tag":449,"props":4546,"children":4547},{"style":472},[4548],{"type":50,"value":475},{"type":45,"tag":449,"props":4550,"children":4551},{"style":478},[4552],{"type":50,"value":481},{"type":45,"tag":449,"props":4554,"children":4555},{"style":472},[4556],{"type":50,"value":4557},"namespac",{"type":45,"tag":449,"props":4559,"children":4560},{"style":489},[4561],{"type":50,"value":492},{"type":45,"tag":449,"props":4563,"children":4564},{"style":478},[4565],{"type":50,"value":497},{"type":45,"tag":449,"props":4567,"children":4568},{"style":472},[4569],{"type":50,"value":4570}," -f",{"type":45,"tag":449,"props":4572,"children":4573},{"style":472},[4574],{"type":50,"value":4575}," values.yaml",{"type":45,"tag":449,"props":4577,"children":4578},{"style":478},[4579],{"type":50,"value":4580}," >",{"type":45,"tag":449,"props":4582,"children":4583},{"style":472},[4584],{"type":50,"value":4585},"\u002Ftmp\u002Frendered.yaml\n",{"type":45,"tag":449,"props":4587,"children":4588},{"class":451,"line":462},[4589,4593,4597,4601,4605,4609,4613,4617,4622,4626,4630,4634,4638,4643],{"type":45,"tag":449,"props":4590,"children":4591},{"style":466},[4592],{"type":50,"value":729},{"type":45,"tag":449,"props":4594,"children":4595},{"style":472},[4596],{"type":50,"value":475},{"type":45,"tag":449,"props":4598,"children":4599},{"style":478},[4600],{"type":50,"value":481},{"type":45,"tag":449,"props":4602,"children":4603},{"style":472},[4604],{"type":50,"value":4557},{"type":45,"tag":449,"props":4606,"children":4607},{"style":489},[4608],{"type":50,"value":492},{"type":45,"tag":449,"props":4610,"children":4611},{"style":478},[4612],{"type":50,"value":497},{"type":45,"tag":449,"props":4614,"children":4615},{"style":472},[4616],{"type":50,"value":754},{"type":45,"tag":449,"props":4618,"children":4619},{"style":472},[4620],{"type":50,"value":4621}," secret,configmap",{"type":45,"tag":449,"props":4623,"children":4624},{"style":478},[4625],{"type":50,"value":774},{"type":45,"tag":449,"props":4627,"children":4628},{"style":466},[4629],{"type":50,"value":779},{"type":45,"tag":449,"props":4631,"children":4632},{"style":472},[4633],{"type":50,"value":784},{"type":45,"tag":449,"props":4635,"children":4636},{"style":478},[4637],{"type":50,"value":789},{"type":45,"tag":449,"props":4639,"children":4640},{"style":472},[4641],{"type":50,"value":4642},"cockroach|crdb|cert|ca|tls",{"type":45,"tag":449,"props":4644,"children":4645},{"style":478},[4646],{"type":50,"value":799},{"type":45,"tag":449,"props":4648,"children":4649},{"class":451,"line":533},[4650,4654,4658,4662,4666,4670,4674,4678,4682,4686,4690,4694,4698,4702,4707,4711,4715,4720],{"type":45,"tag":449,"props":4651,"children":4652},{"style":466},[4653],{"type":50,"value":729},{"type":45,"tag":449,"props":4655,"children":4656},{"style":472},[4657],{"type":50,"value":475},{"type":45,"tag":449,"props":4659,"children":4660},{"style":478},[4661],{"type":50,"value":481},{"type":45,"tag":449,"props":4663,"children":4664},{"style":472},[4665],{"type":50,"value":4557},{"type":45,"tag":449,"props":4667,"children":4668},{"style":489},[4669],{"type":50,"value":492},{"type":45,"tag":449,"props":4671,"children":4672},{"style":478},[4673],{"type":50,"value":497},{"type":45,"tag":449,"props":4675,"children":4676},{"style":472},[4677],{"type":50,"value":754},{"type":45,"tag":449,"props":4679,"children":4680},{"style":472},[4681],{"type":50,"value":983},{"type":45,"tag":449,"props":4683,"children":4684},{"style":478},[4685],{"type":50,"value":481},{"type":45,"tag":449,"props":4687,"children":4688},{"style":472},[4689],{"type":50,"value":4519},{"type":45,"tag":449,"props":4691,"children":4692},{"style":489},[4693],{"type":50,"value":492},{"type":45,"tag":449,"props":4695,"children":4696},{"style":478},[4697],{"type":50,"value":497},{"type":45,"tag":449,"props":4699,"children":4700},{"style":472},[4701],{"type":50,"value":764},{"type":45,"tag":449,"props":4703,"children":4704},{"style":472},[4705],{"type":50,"value":4706}," yaml",{"type":45,"tag":449,"props":4708,"children":4709},{"style":478},[4710],{"type":50,"value":774},{"type":45,"tag":449,"props":4712,"children":4713},{"style":466},[4714],{"type":50,"value":779},{"type":45,"tag":449,"props":4716,"children":4717},{"style":472},[4718],{"type":50,"value":4719}," -A20",{"type":45,"tag":449,"props":4721,"children":4722},{"style":472},[4723],{"type":50,"value":4724}," certificates\n",{"type":45,"tag":449,"props":4726,"children":4727},{"class":451,"line":591},[4728,4732,4736,4740,4744,4748,4752,4756,4760,4764,4768,4772,4776,4780,4784,4788,4793],{"type":45,"tag":449,"props":4729,"children":4730},{"style":466},[4731],{"type":50,"value":729},{"type":45,"tag":449,"props":4733,"children":4734},{"style":472},[4735],{"type":50,"value":475},{"type":45,"tag":449,"props":4737,"children":4738},{"style":478},[4739],{"type":50,"value":481},{"type":45,"tag":449,"props":4741,"children":4742},{"style":472},[4743],{"type":50,"value":4557},{"type":45,"tag":449,"props":4745,"children":4746},{"style":489},[4747],{"type":50,"value":492},{"type":45,"tag":449,"props":4749,"children":4750},{"style":478},[4751],{"type":50,"value":497},{"type":45,"tag":449,"props":4753,"children":4754},{"style":472},[4755],{"type":50,"value":754},{"type":45,"tag":449,"props":4757,"children":4758},{"style":472},[4759],{"type":50,"value":1111},{"type":45,"tag":449,"props":4761,"children":4762},{"style":478},[4763],{"type":50,"value":481},{"type":45,"tag":449,"props":4765,"children":4766},{"style":472},[4767],{"type":50,"value":1120},{"type":45,"tag":449,"props":4769,"children":4770},{"style":489},[4771],{"type":50,"value":1125},{"type":45,"tag":449,"props":4773,"children":4774},{"style":478},[4775],{"type":50,"value":497},{"type":45,"tag":449,"props":4777,"children":4778},{"style":472},[4779],{"type":50,"value":764},{"type":45,"tag":449,"props":4781,"children":4782},{"style":472},[4783],{"type":50,"value":1711},{"type":45,"tag":449,"props":4785,"children":4786},{"style":478},[4787],{"type":50,"value":1716},{"type":45,"tag":449,"props":4789,"children":4790},{"style":472},[4791],{"type":50,"value":4792},"{.spec.containers[*].name}{\"\\n\"}",{"type":45,"tag":449,"props":4794,"children":4795},{"style":478},[4796],{"type":50,"value":799},{"type":45,"tag":449,"props":4798,"children":4799},{"class":451,"line":648},[4800,4804,4808,4812,4816,4820,4824,4828,4832,4836,4840,4844,4848,4853],{"type":45,"tag":449,"props":4801,"children":4802},{"style":466},[4803],{"type":50,"value":729},{"type":45,"tag":449,"props":4805,"children":4806},{"style":472},[4807],{"type":50,"value":475},{"type":45,"tag":449,"props":4809,"children":4810},{"style":478},[4811],{"type":50,"value":481},{"type":45,"tag":449,"props":4813,"children":4814},{"style":472},[4815],{"type":50,"value":4557},{"type":45,"tag":449,"props":4817,"children":4818},{"style":489},[4819],{"type":50,"value":492},{"type":45,"tag":449,"props":4821,"children":4822},{"style":478},[4823],{"type":50,"value":497},{"type":45,"tag":449,"props":4825,"children":4826},{"style":472},[4827],{"type":50,"value":832},{"type":45,"tag":449,"props":4829,"children":4830},{"style":478},[4831],{"type":50,"value":481},{"type":45,"tag":449,"props":4833,"children":4834},{"style":472},[4835],{"type":50,"value":1120},{"type":45,"tag":449,"props":4837,"children":4838},{"style":489},[4839],{"type":50,"value":1125},{"type":45,"tag":449,"props":4841,"children":4842},{"style":478},[4843],{"type":50,"value":497},{"type":45,"tag":449,"props":4845,"children":4846},{"style":472},[4847],{"type":50,"value":1182},{"type":45,"tag":449,"props":4849,"children":4850},{"style":472},[4851],{"type":50,"value":4852}," cert-reloader",{"type":45,"tag":449,"props":4854,"children":4855},{"style":472},[4856],{"type":50,"value":2016},{"type":45,"tag":53,"props":4858,"children":4859},{},[4860],{"type":50,"value":1832},{"type":45,"tag":75,"props":4862,"children":4863},{},[4864,4869,4889,4917,4922],{"type":45,"tag":79,"props":4865,"children":4866},{},[4867],{"type":50,"value":4868},"Ensure exactly one TLS provider is enabled.",{"type":45,"tag":79,"props":4870,"children":4871},{},[4872,4874,4880,4882,4888],{"type":50,"value":4873},"Ensure self-signer ",{"type":45,"tag":83,"props":4875,"children":4877},{"className":4876},[],[4878],{"type":50,"value":4879},"caProvided=true",{"type":50,"value":4881}," has a valid ",{"type":45,"tag":83,"props":4883,"children":4885},{"className":4884},[],[4886],{"type":50,"value":4887},"caSecret",{"type":50,"value":66},{"type":45,"tag":79,"props":4890,"children":4891},{},[4892,4894,4900,4901,4907,4909,4915],{"type":50,"value":4893},"Ensure cert-manager ",{"type":45,"tag":83,"props":4895,"children":4897},{"className":4896},[],[4898],{"type":50,"value":4899},"Issuer",{"type":50,"value":90},{"type":45,"tag":83,"props":4902,"children":4904},{"className":4903},[],[4905],{"type":50,"value":4906},"ClusterIssuer",{"type":50,"value":4908}," exists and ",{"type":45,"tag":83,"props":4910,"children":4912},{"className":4911},[],[4913],{"type":50,"value":4914},"Certificate",{"type":50,"value":4916}," resources become Ready.",{"type":45,"tag":79,"props":4918,"children":4919},{},[4920],{"type":50,"value":4921},"Ensure external certificate Secrets and CA ConfigMap have expected keys and share a trust root.",{"type":45,"tag":79,"props":4923,"children":4924},{},[4925],{"type":50,"value":4926},"Collect expiry, subject, issuer, and SANs, but do not print private key contents.",{"type":45,"tag":68,"props":4928,"children":4930},{"id":4929},"pod-scheduling-and-storage-failures",[4931],{"type":50,"value":4932},"Pod Scheduling and Storage Failures",{"type":45,"tag":438,"props":4934,"children":4936},{"className":440,"code":4935,"language":442,"meta":443,"style":443},"kubectl -n \u003Cnamespace> describe pod \u003Cpod-name>\nkubectl -n \u003Cnamespace> get pvc -o wide\nkubectl get storageclass\nkubectl get nodes -L topology.kubernetes.io\u002Fregion,topology.kubernetes.io\u002Fzone\nkubectl -n \u003Cnamespace> exec \u003Ccrdb-pod> -c cockroachdb -- df -h \u002Fcockroach\u002Fcockroach-data\n",[4937],{"type":45,"tag":83,"props":4938,"children":4939},{"__ignoreMap":443},[4940,4992,5036,5052,5078],{"type":45,"tag":449,"props":4941,"children":4942},{"class":451,"line":452},[4943,4947,4951,4955,4959,4963,4967,4971,4975,4979,4984,4988],{"type":45,"tag":449,"props":4944,"children":4945},{"style":466},[4946],{"type":50,"value":729},{"type":45,"tag":449,"props":4948,"children":4949},{"style":472},[4950],{"type":50,"value":475},{"type":45,"tag":449,"props":4952,"children":4953},{"style":478},[4954],{"type":50,"value":481},{"type":45,"tag":449,"props":4956,"children":4957},{"style":472},[4958],{"type":50,"value":4557},{"type":45,"tag":449,"props":4960,"children":4961},{"style":489},[4962],{"type":50,"value":492},{"type":45,"tag":449,"props":4964,"children":4965},{"style":478},[4966],{"type":50,"value":497},{"type":45,"tag":449,"props":4968,"children":4969},{"style":472},[4970],{"type":50,"value":978},{"type":45,"tag":449,"props":4972,"children":4973},{"style":472},[4974],{"type":50,"value":1111},{"type":45,"tag":449,"props":4976,"children":4977},{"style":478},[4978],{"type":50,"value":481},{"type":45,"tag":449,"props":4980,"children":4981},{"style":472},[4982],{"type":50,"value":4983},"pod-nam",{"type":45,"tag":449,"props":4985,"children":4986},{"style":489},[4987],{"type":50,"value":492},{"type":45,"tag":449,"props":4989,"children":4990},{"style":478},[4991],{"type":50,"value":1130},{"type":45,"tag":449,"props":4993,"children":4994},{"class":451,"line":462},[4995,4999,5003,5007,5011,5015,5019,5023,5028,5032],{"type":45,"tag":449,"props":4996,"children":4997},{"style":466},[4998],{"type":50,"value":729},{"type":45,"tag":449,"props":5000,"children":5001},{"style":472},[5002],{"type":50,"value":475},{"type":45,"tag":449,"props":5004,"children":5005},{"style":478},[5006],{"type":50,"value":481},{"type":45,"tag":449,"props":5008,"children":5009},{"style":472},[5010],{"type":50,"value":4557},{"type":45,"tag":449,"props":5012,"children":5013},{"style":489},[5014],{"type":50,"value":492},{"type":45,"tag":449,"props":5016,"children":5017},{"style":478},[5018],{"type":50,"value":497},{"type":45,"tag":449,"props":5020,"children":5021},{"style":472},[5022],{"type":50,"value":754},{"type":45,"tag":449,"props":5024,"children":5025},{"style":472},[5026],{"type":50,"value":5027}," pvc",{"type":45,"tag":449,"props":5029,"children":5030},{"style":472},[5031],{"type":50,"value":764},{"type":45,"tag":449,"props":5033,"children":5034},{"style":472},[5035],{"type":50,"value":945},{"type":45,"tag":449,"props":5037,"children":5038},{"class":451,"line":533},[5039,5043,5047],{"type":45,"tag":449,"props":5040,"children":5041},{"style":466},[5042],{"type":50,"value":729},{"type":45,"tag":449,"props":5044,"children":5045},{"style":472},[5046],{"type":50,"value":754},{"type":45,"tag":449,"props":5048,"children":5049},{"style":472},[5050],{"type":50,"value":5051}," storageclass\n",{"type":45,"tag":449,"props":5053,"children":5054},{"class":451,"line":591},[5055,5059,5063,5068,5073],{"type":45,"tag":449,"props":5056,"children":5057},{"style":466},[5058],{"type":50,"value":729},{"type":45,"tag":449,"props":5060,"children":5061},{"style":472},[5062],{"type":50,"value":754},{"type":45,"tag":449,"props":5064,"children":5065},{"style":472},[5066],{"type":50,"value":5067}," nodes",{"type":45,"tag":449,"props":5069,"children":5070},{"style":472},[5071],{"type":50,"value":5072}," -L",{"type":45,"tag":449,"props":5074,"children":5075},{"style":472},[5076],{"type":50,"value":5077}," topology.kubernetes.io\u002Fregion,topology.kubernetes.io\u002Fzone\n",{"type":45,"tag":449,"props":5079,"children":5080},{"class":451,"line":648},[5081,5085,5089,5093,5097,5101,5105,5109,5113,5117,5121,5125,5129,5133,5137,5142,5147],{"type":45,"tag":449,"props":5082,"children":5083},{"style":466},[5084],{"type":50,"value":729},{"type":45,"tag":449,"props":5086,"children":5087},{"style":472},[5088],{"type":50,"value":475},{"type":45,"tag":449,"props":5090,"children":5091},{"style":478},[5092],{"type":50,"value":481},{"type":45,"tag":449,"props":5094,"children":5095},{"style":472},[5096],{"type":50,"value":4557},{"type":45,"tag":449,"props":5098,"children":5099},{"style":489},[5100],{"type":50,"value":492},{"type":45,"tag":449,"props":5102,"children":5103},{"style":478},[5104],{"type":50,"value":497},{"type":45,"tag":449,"props":5106,"children":5107},{"style":472},[5108],{"type":50,"value":4235},{"type":45,"tag":449,"props":5110,"children":5111},{"style":478},[5112],{"type":50,"value":481},{"type":45,"tag":449,"props":5114,"children":5115},{"style":472},[5116],{"type":50,"value":1120},{"type":45,"tag":449,"props":5118,"children":5119},{"style":489},[5120],{"type":50,"value":1125},{"type":45,"tag":449,"props":5122,"children":5123},{"style":478},[5124],{"type":50,"value":497},{"type":45,"tag":449,"props":5126,"children":5127},{"style":472},[5128],{"type":50,"value":1182},{"type":45,"tag":449,"props":5130,"children":5131},{"style":472},[5132],{"type":50,"value":1187},{"type":45,"tag":449,"props":5134,"children":5135},{"style":472},[5136],{"type":50,"value":4264},{"type":45,"tag":449,"props":5138,"children":5139},{"style":472},[5140],{"type":50,"value":5141}," df",{"type":45,"tag":449,"props":5143,"children":5144},{"style":472},[5145],{"type":50,"value":5146}," -h",{"type":45,"tag":449,"props":5148,"children":5149},{"style":472},[5150],{"type":50,"value":5151}," \u002Fcockroach\u002Fcockroach-data\n",{"type":45,"tag":53,"props":5153,"children":5154},{},[5155],{"type":50,"value":5156},"Common causes:",{"type":45,"tag":75,"props":5158,"children":5159},{},[5160,5173,5178,5183,5188],{"type":45,"tag":79,"props":5161,"children":5162},{},[5163,5165,5171],{"type":50,"value":5164},"PVCs cannot bind because no default StorageClass exists or ",{"type":45,"tag":83,"props":5166,"children":5168},{"className":5167},[],[5169],{"type":50,"value":5170},"storageClassName",{"type":50,"value":5172}," is wrong.",{"type":45,"tag":79,"props":5174,"children":5175},{},[5176],{"type":50,"value":5177},"Topology spread constraints cannot be satisfied because node zone labels are missing or insufficient.",{"type":45,"tag":79,"props":5179,"children":5180},{},[5181],{"type":50,"value":5182},"Node resources are too small for requested CPU\u002Fmemory.",{"type":45,"tag":79,"props":5184,"children":5185},{},[5186],{"type":50,"value":5187},"Image pull failures from registry policy or air-gapped environments.",{"type":45,"tag":79,"props":5189,"children":5190},{},[5191],{"type":50,"value":5192},"Migrated PVCs may be missing ownerReferences; use the migration debugging skill before deleting anything.",{"type":45,"tag":68,"props":5194,"children":5196},{"id":5195},"scale-down-and-decommission",[5197],{"type":50,"value":5198},"Scale Down and Decommission",{"type":45,"tag":438,"props":5200,"children":5202},{"className":440,"code":5201,"language":442,"meta":443,"style":443},"kubectl -n \u003Ccockroachdb-namespace> exec \u003Cready-crdb-pod> -c cockroachdb -- \\\n  \u002Fcockroach\u002Fcockroach node status --decommission\n\nkubectl -n \u003Ccockroachdb-namespace> get crdbnodes -o json | jq '[.items[] | select(.status.phase==\"Decommissioning\")] | {count: length, nodes: [.[].metadata.name]}'\n\nkubectl -n \u003Coperator-namespace> logs -l app=cockroach-operator --tail=300 | grep -Ei 'decommission|drain|scale|blocking_ranges' || true\n",[5203],{"type":45,"tag":83,"props":5204,"children":5205},{"__ignoreMap":443},[5206,5270,5292,5299,5363,5370],{"type":45,"tag":449,"props":5207,"children":5208},{"class":451,"line":452},[5209,5213,5217,5221,5225,5229,5233,5237,5241,5246,5250,5254,5258,5262,5266],{"type":45,"tag":449,"props":5210,"children":5211},{"style":466},[5212],{"type":50,"value":729},{"type":45,"tag":449,"props":5214,"children":5215},{"style":472},[5216],{"type":50,"value":475},{"type":45,"tag":449,"props":5218,"children":5219},{"style":478},[5220],{"type":50,"value":481},{"type":45,"tag":449,"props":5222,"children":5223},{"style":472},[5224],{"type":50,"value":551},{"type":45,"tag":449,"props":5226,"children":5227},{"style":489},[5228],{"type":50,"value":492},{"type":45,"tag":449,"props":5230,"children":5231},{"style":478},[5232],{"type":50,"value":497},{"type":45,"tag":449,"props":5234,"children":5235},{"style":472},[5236],{"type":50,"value":4235},{"type":45,"tag":449,"props":5238,"children":5239},{"style":478},[5240],{"type":50,"value":481},{"type":45,"tag":449,"props":5242,"children":5243},{"style":472},[5244],{"type":50,"value":5245},"ready-crdb-po",{"type":45,"tag":449,"props":5247,"children":5248},{"style":489},[5249],{"type":50,"value":1125},{"type":45,"tag":449,"props":5251,"children":5252},{"style":478},[5253],{"type":50,"value":497},{"type":45,"tag":449,"props":5255,"children":5256},{"style":472},[5257],{"type":50,"value":1182},{"type":45,"tag":449,"props":5259,"children":5260},{"style":472},[5261],{"type":50,"value":1187},{"type":45,"tag":449,"props":5263,"children":5264},{"style":472},[5265],{"type":50,"value":4264},{"type":45,"tag":449,"props":5267,"children":5268},{"style":489},[5269],{"type":50,"value":2305},{"type":45,"tag":449,"props":5271,"children":5272},{"class":451,"line":462},[5273,5278,5283,5287],{"type":45,"tag":449,"props":5274,"children":5275},{"style":472},[5276],{"type":50,"value":5277},"  \u002Fcockroach\u002Fcockroach",{"type":45,"tag":449,"props":5279,"children":5280},{"style":472},[5281],{"type":50,"value":5282}," node",{"type":45,"tag":449,"props":5284,"children":5285},{"style":472},[5286],{"type":50,"value":502},{"type":45,"tag":449,"props":5288,"children":5289},{"style":472},[5290],{"type":50,"value":5291}," --decommission\n",{"type":45,"tag":449,"props":5293,"children":5294},{"class":451,"line":533},[5295],{"type":45,"tag":449,"props":5296,"children":5297},{"emptyLinePlaceholder":708},[5298],{"type":50,"value":711},{"type":45,"tag":449,"props":5300,"children":5301},{"class":451,"line":591},[5302,5306,5310,5314,5318,5322,5326,5330,5334,5338,5342,5346,5350,5354,5359],{"type":45,"tag":449,"props":5303,"children":5304},{"style":466},[5305],{"type":50,"value":729},{"type":45,"tag":449,"props":5307,"children":5308},{"style":472},[5309],{"type":50,"value":475},{"type":45,"tag":449,"props":5311,"children":5312},{"style":478},[5313],{"type":50,"value":481},{"type":45,"tag":449,"props":5315,"children":5316},{"style":472},[5317],{"type":50,"value":551},{"type":45,"tag":449,"props":5319,"children":5320},{"style":489},[5321],{"type":50,"value":492},{"type":45,"tag":449,"props":5323,"children":5324},{"style":478},[5325],{"type":50,"value":497},{"type":45,"tag":449,"props":5327,"children":5328},{"style":472},[5329],{"type":50,"value":754},{"type":45,"tag":449,"props":5331,"children":5332},{"style":472},[5333],{"type":50,"value":2986},{"type":45,"tag":449,"props":5335,"children":5336},{"style":472},[5337],{"type":50,"value":764},{"type":45,"tag":449,"props":5339,"children":5340},{"style":472},[5341],{"type":50,"value":2857},{"type":45,"tag":449,"props":5343,"children":5344},{"style":478},[5345],{"type":50,"value":774},{"type":45,"tag":449,"props":5347,"children":5348},{"style":466},[5349],{"type":50,"value":2866},{"type":45,"tag":449,"props":5351,"children":5352},{"style":478},[5353],{"type":50,"value":789},{"type":45,"tag":449,"props":5355,"children":5356},{"style":472},[5357],{"type":50,"value":5358},"[.items[] | select(.status.phase==\"Decommissioning\")] | {count: length, nodes: [.[].metadata.name]}",{"type":45,"tag":449,"props":5360,"children":5361},{"style":478},[5362],{"type":50,"value":799},{"type":45,"tag":449,"props":5364,"children":5365},{"class":451,"line":648},[5366],{"type":45,"tag":449,"props":5367,"children":5368},{"emptyLinePlaceholder":708},[5369],{"type":50,"value":711},{"type":45,"tag":449,"props":5371,"children":5372},{"class":451,"line":704},[5373,5377,5381,5385,5389,5393,5397,5401,5405,5409,5414,5418,5422,5427,5431,5436,5440,5444],{"type":45,"tag":449,"props":5374,"children":5375},{"style":466},[5376],{"type":50,"value":729},{"type":45,"tag":449,"props":5378,"children":5379},{"style":472},[5380],{"type":50,"value":475},{"type":45,"tag":449,"props":5382,"children":5383},{"style":478},[5384],{"type":50,"value":481},{"type":45,"tag":449,"props":5386,"children":5387},{"style":472},[5388],{"type":50,"value":486},{"type":45,"tag":449,"props":5390,"children":5391},{"style":489},[5392],{"type":50,"value":492},{"type":45,"tag":449,"props":5394,"children":5395},{"style":478},[5396],{"type":50,"value":497},{"type":45,"tag":449,"props":5398,"children":5399},{"style":472},[5400],{"type":50,"value":832},{"type":45,"tag":449,"props":5402,"children":5403},{"style":472},[5404],{"type":50,"value":837},{"type":45,"tag":449,"props":5406,"children":5407},{"style":472},[5408],{"type":50,"value":842},{"type":45,"tag":449,"props":5410,"children":5411},{"style":472},[5412],{"type":50,"value":5413}," --tail=300",{"type":45,"tag":449,"props":5415,"children":5416},{"style":478},[5417],{"type":50,"value":774},{"type":45,"tag":449,"props":5419,"children":5420},{"style":466},[5421],{"type":50,"value":779},{"type":45,"tag":449,"props":5423,"children":5424},{"style":472},[5425],{"type":50,"value":5426}," -Ei",{"type":45,"tag":449,"props":5428,"children":5429},{"style":478},[5430],{"type":50,"value":789},{"type":45,"tag":449,"props":5432,"children":5433},{"style":472},[5434],{"type":50,"value":5435},"decommission|drain|scale|blocking_ranges",{"type":45,"tag":449,"props":5437,"children":5438},{"style":478},[5439],{"type":50,"value":1716},{"type":45,"tag":449,"props":5441,"children":5442},{"style":478},[5443],{"type":50,"value":524},{"type":45,"tag":449,"props":5445,"children":5446},{"style":527},[5447],{"type":50,"value":530},{"type":45,"tag":53,"props":5449,"children":5450},{},[5451],{"type":50,"value":5452},"Questions to answer:",{"type":45,"tag":75,"props":5454,"children":5455},{},[5456,5461,5466,5471],{"type":45,"tag":79,"props":5457,"children":5458},{},[5459],{"type":50,"value":5460},"What was the original and target node count?",{"type":45,"tag":79,"props":5462,"children":5463},{},[5464],{"type":50,"value":5465},"Were multiple nodes scaled down at the same time?",{"type":45,"tag":79,"props":5467,"children":5468},{},[5469],{"type":50,"value":5470},"Were manual decommission or drain commands issued?",{"type":45,"tag":79,"props":5472,"children":5473},{},[5474],{"type":50,"value":5475},"Did any pods or PVCs get deleted manually?",{"type":45,"tag":68,"props":5477,"children":5479},{"id":5478},"temporary-mitigations",[5480],{"type":50,"value":5481},"Temporary Mitigations",{"type":45,"tag":53,"props":5483,"children":5484},{},[5485],{"type":50,"value":5486},"Only use these after collecting evidence and confirming the risk with the customer or operator team.",{"type":45,"tag":53,"props":5488,"children":5489},{},[5490],{"type":50,"value":5491},"Disable reconciliation for one cluster:",{"type":45,"tag":438,"props":5493,"children":5495},{"className":440,"code":5494,"language":442,"meta":443,"style":443},"kubectl -n \u003Ccockroachdb-namespace> patch crdbcluster \u003Ccockroachdb-release> --type=merge -p '{\"spec\":{\"mode\":\"Disabled\"}}'\n\n# Resume reconciliation:\nkubectl -n \u003Ccockroachdb-namespace> patch crdbcluster \u003Ccockroachdb-release> --type=merge -p '{\"spec\":{\"mode\":\"MutableOnly\"}}'\n",[5496],{"type":45,"tag":83,"props":5497,"children":5498},{"__ignoreMap":443},[5499,5574,5581,5589],{"type":45,"tag":449,"props":5500,"children":5501},{"class":451,"line":452},[5502,5506,5510,5514,5518,5522,5526,5531,5535,5539,5543,5547,5551,5556,5561,5565,5570],{"type":45,"tag":449,"props":5503,"children":5504},{"style":466},[5505],{"type":50,"value":729},{"type":45,"tag":449,"props":5507,"children":5508},{"style":472},[5509],{"type":50,"value":475},{"type":45,"tag":449,"props":5511,"children":5512},{"style":478},[5513],{"type":50,"value":481},{"type":45,"tag":449,"props":5515,"children":5516},{"style":472},[5517],{"type":50,"value":551},{"type":45,"tag":449,"props":5519,"children":5520},{"style":489},[5521],{"type":50,"value":492},{"type":45,"tag":449,"props":5523,"children":5524},{"style":478},[5525],{"type":50,"value":497},{"type":45,"tag":449,"props":5527,"children":5528},{"style":472},[5529],{"type":50,"value":5530}," patch",{"type":45,"tag":449,"props":5532,"children":5533},{"style":472},[5534],{"type":50,"value":983},{"type":45,"tag":449,"props":5536,"children":5537},{"style":478},[5538],{"type":50,"value":481},{"type":45,"tag":449,"props":5540,"children":5541},{"style":472},[5542],{"type":50,"value":572},{"type":45,"tag":449,"props":5544,"children":5545},{"style":489},[5546],{"type":50,"value":492},{"type":45,"tag":449,"props":5548,"children":5549},{"style":478},[5550],{"type":50,"value":497},{"type":45,"tag":449,"props":5552,"children":5553},{"style":472},[5554],{"type":50,"value":5555}," --type=merge",{"type":45,"tag":449,"props":5557,"children":5558},{"style":472},[5559],{"type":50,"value":5560}," -p",{"type":45,"tag":449,"props":5562,"children":5563},{"style":478},[5564],{"type":50,"value":789},{"type":45,"tag":449,"props":5566,"children":5567},{"style":472},[5568],{"type":50,"value":5569},"{\"spec\":{\"mode\":\"Disabled\"}}",{"type":45,"tag":449,"props":5571,"children":5572},{"style":478},[5573],{"type":50,"value":799},{"type":45,"tag":449,"props":5575,"children":5576},{"class":451,"line":462},[5577],{"type":45,"tag":449,"props":5578,"children":5579},{"emptyLinePlaceholder":708},[5580],{"type":50,"value":711},{"type":45,"tag":449,"props":5582,"children":5583},{"class":451,"line":533},[5584],{"type":45,"tag":449,"props":5585,"children":5586},{"style":456},[5587],{"type":50,"value":5588},"# Resume reconciliation:\n",{"type":45,"tag":449,"props":5590,"children":5591},{"class":451,"line":591},[5592,5596,5600,5604,5608,5612,5616,5620,5624,5628,5632,5636,5640,5644,5648,5652,5657],{"type":45,"tag":449,"props":5593,"children":5594},{"style":466},[5595],{"type":50,"value":729},{"type":45,"tag":449,"props":5597,"children":5598},{"style":472},[5599],{"type":50,"value":475},{"type":45,"tag":449,"props":5601,"children":5602},{"style":478},[5603],{"type":50,"value":481},{"type":45,"tag":449,"props":5605,"children":5606},{"style":472},[5607],{"type":50,"value":551},{"type":45,"tag":449,"props":5609,"children":5610},{"style":489},[5611],{"type":50,"value":492},{"type":45,"tag":449,"props":5613,"children":5614},{"style":478},[5615],{"type":50,"value":497},{"type":45,"tag":449,"props":5617,"children":5618},{"style":472},[5619],{"type":50,"value":5530},{"type":45,"tag":449,"props":5621,"children":5622},{"style":472},[5623],{"type":50,"value":983},{"type":45,"tag":449,"props":5625,"children":5626},{"style":478},[5627],{"type":50,"value":481},{"type":45,"tag":449,"props":5629,"children":5630},{"style":472},[5631],{"type":50,"value":572},{"type":45,"tag":449,"props":5633,"children":5634},{"style":489},[5635],{"type":50,"value":492},{"type":45,"tag":449,"props":5637,"children":5638},{"style":478},[5639],{"type":50,"value":497},{"type":45,"tag":449,"props":5641,"children":5642},{"style":472},[5643],{"type":50,"value":5555},{"type":45,"tag":449,"props":5645,"children":5646},{"style":472},[5647],{"type":50,"value":5560},{"type":45,"tag":449,"props":5649,"children":5650},{"style":478},[5651],{"type":50,"value":789},{"type":45,"tag":449,"props":5653,"children":5654},{"style":472},[5655],{"type":50,"value":5656},"{\"spec\":{\"mode\":\"MutableOnly\"}}",{"type":45,"tag":449,"props":5658,"children":5659},{"style":478},[5660],{"type":50,"value":799},{"type":45,"tag":53,"props":5662,"children":5663},{},[5664],{"type":50,"value":5665},"Restart the operator after evidence is collected:",{"type":45,"tag":438,"props":5667,"children":5669},{"className":440,"code":5668,"language":442,"meta":443,"style":443},"kubectl -n \u003Coperator-namespace> rollout restart deploy\u002Fcockroach-operator\n",[5670],{"type":45,"tag":83,"props":5671,"children":5672},{"__ignoreMap":443},[5673],{"type":45,"tag":449,"props":5674,"children":5675},{"class":451,"line":452},[5676,5680,5684,5688,5692,5696,5700,5704,5709],{"type":45,"tag":449,"props":5677,"children":5678},{"style":466},[5679],{"type":50,"value":729},{"type":45,"tag":449,"props":5681,"children":5682},{"style":472},[5683],{"type":50,"value":475},{"type":45,"tag":449,"props":5685,"children":5686},{"style":478},[5687],{"type":50,"value":481},{"type":45,"tag":449,"props":5689,"children":5690},{"style":472},[5691],{"type":50,"value":486},{"type":45,"tag":449,"props":5693,"children":5694},{"style":489},[5695],{"type":50,"value":492},{"type":45,"tag":449,"props":5697,"children":5698},{"style":478},[5699],{"type":50,"value":497},{"type":45,"tag":449,"props":5701,"children":5702},{"style":472},[5703],{"type":50,"value":1669},{"type":45,"tag":449,"props":5705,"children":5706},{"style":472},[5707],{"type":50,"value":5708}," restart",{"type":45,"tag":449,"props":5710,"children":5711},{"style":472},[5712],{"type":50,"value":5713}," deploy\u002Fcockroach-operator\n",{"type":45,"tag":53,"props":5715,"children":5716},{},[5717],{"type":50,"value":5718},"User-approved timestamp rolling restart:",{"type":45,"tag":438,"props":5720,"children":5721},{"className":440,"code":2230,"language":442,"meta":443,"style":443},[5722],{"type":45,"tag":83,"props":5723,"children":5724},{"__ignoreMap":443},[5725,5792,5803],{"type":45,"tag":449,"props":5726,"children":5727},{"class":451,"line":452},[5728,5732,5736,5740,5744,5748,5752,5756,5760,5764,5768,5772,5776,5780,5784,5788],{"type":45,"tag":449,"props":5729,"children":5730},{"style":466},[5731],{"type":50,"value":469},{"type":45,"tag":449,"props":5733,"children":5734},{"style":472},[5735],{"type":50,"value":475},{"type":45,"tag":449,"props":5737,"children":5738},{"style":478},[5739],{"type":50,"value":481},{"type":45,"tag":449,"props":5741,"children":5742},{"style":472},[5743],{"type":50,"value":551},{"type":45,"tag":449,"props":5745,"children":5746},{"style":489},[5747],{"type":50,"value":492},{"type":45,"tag":449,"props":5749,"children":5750},{"style":478},[5751],{"type":50,"value":497},{"type":45,"tag":449,"props":5753,"children":5754},{"style":472},[5755],{"type":50,"value":2266},{"type":45,"tag":449,"props":5757,"children":5758},{"style":478},[5759],{"type":50,"value":481},{"type":45,"tag":449,"props":5761,"children":5762},{"style":472},[5763],{"type":50,"value":572},{"type":45,"tag":449,"props":5765,"children":5766},{"style":489},[5767],{"type":50,"value":492},{"type":45,"tag":449,"props":5769,"children":5770},{"style":478},[5771],{"type":50,"value":497},{"type":45,"tag":449,"props":5773,"children":5774},{"style":478},[5775],{"type":50,"value":481},{"type":45,"tag":449,"props":5777,"children":5778},{"style":472},[5779],{"type":50,"value":2291},{"type":45,"tag":449,"props":5781,"children":5782},{"style":489},[5783],{"type":50,"value":2296},{"type":45,"tag":449,"props":5785,"children":5786},{"style":478},[5787],{"type":50,"value":497},{"type":45,"tag":449,"props":5789,"children":5790},{"style":489},[5791],{"type":50,"value":2305},{"type":45,"tag":449,"props":5793,"children":5794},{"class":451,"line":462},[5795,5799],{"type":45,"tag":449,"props":5796,"children":5797},{"style":472},[5798],{"type":50,"value":2313},{"type":45,"tag":449,"props":5800,"children":5801},{"style":489},[5802],{"type":50,"value":2305},{"type":45,"tag":449,"props":5804,"children":5805},{"class":451,"line":533},[5806,5810,5814,5818,5822,5826],{"type":45,"tag":449,"props":5807,"children":5808},{"style":472},[5809],{"type":50,"value":2325},{"type":45,"tag":449,"props":5811,"children":5812},{"style":472},[5813],{"type":50,"value":2330},{"type":45,"tag":449,"props":5815,"children":5816},{"style":478},[5817],{"type":50,"value":2335},{"type":45,"tag":449,"props":5819,"children":5820},{"style":466},[5821],{"type":50,"value":2340},{"type":45,"tag":449,"props":5823,"children":5824},{"style":472},[5825],{"type":50,"value":2345},{"type":45,"tag":449,"props":5827,"children":5828},{"style":478},[5829],{"type":50,"value":2350},{"type":45,"tag":68,"props":5831,"children":5833},{"id":5832},"output-format",[5834],{"type":50,"value":5835},"Output Format",{"type":45,"tag":53,"props":5837,"children":5838},{},[5839],{"type":50,"value":5840},"Return findings in this order:",{"type":45,"tag":3028,"props":5842,"children":5843},{},[5844,5849,5854,5859,5864,5869,5874],{"type":45,"tag":79,"props":5845,"children":5846},{},[5847],{"type":50,"value":5848},"Failure class",{"type":45,"tag":79,"props":5850,"children":5851},{},[5852],{"type":50,"value":5853},"Evidence: exact command output or Kubernetes status field",{"type":45,"tag":79,"props":5855,"children":5856},{},[5857],{"type":50,"value":5858},"Root cause or most likely cause",{"type":45,"tag":79,"props":5860,"children":5861},{},[5862],{"type":50,"value":5863},"Minimal remediation",{"type":45,"tag":79,"props":5865,"children":5866},{},[5867],{"type":50,"value":5868},"Verification command",{"type":45,"tag":79,"props":5870,"children":5871},{},[5872],{"type":50,"value":5873},"Data\u002Favailability risk, if any",{"type":45,"tag":79,"props":5875,"children":5876},{},[5877],{"type":50,"value":5878},"Whether escalation packet collection is needed",{"type":45,"tag":68,"props":5880,"children":5882},{"id":5881},"references",[5883],{"type":50,"value":5884},"References",{"type":45,"tag":75,"props":5886,"children":5887},{},[5888,5897,5906,5915,5924],{"type":45,"tag":79,"props":5889,"children":5890},{},[5891],{"type":45,"tag":59,"props":5892,"children":5894},{"href":5893},"..\u002F..\u002F..\u002Fcockroachdb-parent\u002Fdocs\u002FVERSIONING.md",[5895],{"type":50,"value":5896},"CockroachDB Helm Chart Versioning",{"type":45,"tag":79,"props":5898,"children":5899},{},[5900],{"type":45,"tag":59,"props":5901,"children":5903},{"href":5902},"..\u002F..\u002F..\u002Fcockroachdb-parent\u002Fcharts\u002Foperator\u002FREADME.md",[5904],{"type":50,"value":5905},"Operator Helm Chart README",{"type":45,"tag":79,"props":5907,"children":5908},{},[5909],{"type":45,"tag":59,"props":5910,"children":5912},{"href":5911},"..\u002F..\u002F..\u002Fcockroachdb-parent\u002Fcharts\u002Fcockroachdb\u002FREADME.md",[5913],{"type":50,"value":5914},"CockroachDB Helm Chart README",{"type":45,"tag":79,"props":5916,"children":5917},{},[5918],{"type":45,"tag":59,"props":5919,"children":5921},{"href":5920},"..\u002F..\u002F..\u002Fcockroachdb-parent\u002FMIGRATION_v1alpha1_to_v1beta1.md",[5922],{"type":50,"value":5923},"v1alpha1 to v1beta1 Migration Guide",{"type":45,"tag":79,"props":5925,"children":5926},{},[5927],{"type":45,"tag":59,"props":5928,"children":5932},{"href":5929,"rel":5930},"https:\u002F\u002Fwww.cockroachlabs.com\u002Fdocs\u002Fstable\u002Forchestrate-cockroachdb-with-kubernetes",[5931],"nofollow",[5933],{"type":50,"value":5934},"CockroachDB Docs: Kubernetes troubleshooting",{"type":45,"tag":5936,"props":5937,"children":5938},"style",{},[5939],{"type":50,"value":5940},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":5942,"total":6091},[5943,5957,5970,5982,5989,5999,6011,6024,6039,6054,6065,6078],{"slug":64,"name":64,"fn":5944,"description":5945,"org":5946,"tags":5947,"stars":25,"repoUrl":26,"updatedAt":5956},"collect CockroachDB operator escalation packets","Collects a complete CockroachDB Operator escalation packet for TSC\u002FTSE or operator-team handoff, including Helm state, Kubernetes resources, logs, operation-specific evidence, pprof goroutine dumps, metrics, and a customer action timeline. Use when general diagnosis cannot resolve an operator-managed CockroachDB Helm issue or before restarting a stuck operator.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[5948,5949,5952,5953],{"name":13,"slug":14,"type":15},{"name":5950,"slug":5951,"type":15},"Incident Response","incident-response",{"name":20,"slug":21,"type":15},{"name":5954,"slug":5955,"type":15},"Monitoring","monitoring","2026-07-12T07:57:25.288146",{"slug":183,"name":183,"fn":5958,"description":5959,"org":5960,"tags":5961,"stars":25,"repoUrl":26,"updatedAt":5969},"configure TLS for CockroachDB Helm charts","Selects and validates TLS settings for CockroachDB Helm chart deployments, including self-signer, cert-manager, and external certificate modes. Use when a customer needs secure CockroachDB Helm values, certificate secret mapping, cert-manager integration, or TLS install troubleshooting before deploying the chart.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[5962,5963,5966],{"name":17,"slug":18,"type":15},{"name":5964,"slug":5965,"type":15},"Encryption","encryption",{"name":5967,"slug":5968,"type":15},"Security","security","2026-07-12T07:56:37.675396",{"slug":171,"name":171,"fn":5971,"description":5972,"org":5973,"tags":5974,"stars":25,"repoUrl":26,"updatedAt":5981},"debug CockroachDB Operator migration scenarios","Debugs CockroachDB Operator migration scenarios, including Helm StatefulSet to v1beta1 CrdbNode migration and public operator v1alpha1 to v1beta1 migration. Use when migration labels, migration phases, source StatefulSet ownership, converted CRDs, PVC ownership, or post-migration reconciliation are unclear or stuck.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[5975,5976,5977,5978],{"name":13,"slug":14,"type":15},{"name":23,"slug":24,"type":15},{"name":20,"slug":21,"type":15},{"name":5979,"slug":5980,"type":15},"Migration","migration","2026-07-12T07:56:48.360871",{"slug":4,"name":4,"fn":5,"description":6,"org":5983,"tags":5984,"stars":25,"repoUrl":26,"updatedAt":27},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[5985,5986,5987,5988],{"name":13,"slug":14,"type":15},{"name":23,"slug":24,"type":15},{"name":17,"slug":18,"type":15},{"name":20,"slug":21,"type":15},{"slug":5990,"name":5990,"fn":5991,"description":5992,"org":5993,"tags":5994,"stars":25,"repoUrl":26,"updatedAt":5998},"installing-cockroachdb-with-helm","install CockroachDB using Helm","Guides customer-facing installation of CockroachDB on Kubernetes using the CockroachDB split Helm charts and operator-managed v1beta1 resources. Use when installing CockroachDB with Helm, choosing between published and local split charts, verifying a new install, or helping an agent complete first-time Kubernetes onboarding.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[5995,5996,5997],{"name":13,"slug":14,"type":15},{"name":17,"slug":18,"type":15},{"name":20,"slug":21,"type":15},"2026-07-12T07:56:45.777567",{"slug":195,"name":195,"fn":6000,"description":6001,"org":6002,"tags":6003,"stars":25,"repoUrl":26,"updatedAt":6010},"validate CockroachDB multi-region Helm deployments","Validates CockroachDB Helm chart values and Kubernetes prerequisites for operator-managed multi-region deployments. Use before adding a region, deploying CockroachDB across multiple Kubernetes clusters, checking region DNS domains, or confirming that all regions share certificate and networking assumptions.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[6004,6005,6006,6007],{"name":13,"slug":14,"type":15},{"name":17,"slug":18,"type":15},{"name":20,"slug":21,"type":15},{"name":6008,"slug":6009,"type":15},"Operations","operations","2026-07-12T07:56:47.082609",{"slug":6012,"name":6012,"fn":6013,"description":6014,"org":6015,"tags":6016,"stars":533,"repoUrl":6022,"updatedAt":6023},"analyzing-range-distribution","analyze CockroachDB range distribution and health","Analyzes CockroachDB range distribution across tables and indexes using SHOW RANGES to identify range count, size patterns, leaseholder placement, and replication health. Use when investigating hotspots, uneven data distribution, range fragmentation, or validating zone configuration effects without DB Console access.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[6017,6018,6019],{"name":13,"slug":14,"type":15},{"name":5954,"slug":5955,"type":15},{"name":6020,"slug":6021,"type":15},"Performance","performance","https:\u002F\u002Fgithub.com\u002Fcockroachdb\u002Fclaude-plugin","2026-07-12T07:57:18.753533",{"slug":6025,"name":6025,"fn":6026,"description":6027,"org":6028,"tags":6029,"stars":533,"repoUrl":6022,"updatedAt":6038},"analyzing-schema-change-storage-risk","analyze schema change storage requirements","Estimates storage requirements for CockroachDB online schema change backfills using SHOW RANGES WITH DETAILS, KEYS, INDEXES. Use before CREATE INDEX, ADD COLUMN with INDEX\u002FUNIQUE, ALTER PRIMARY KEY, CREATE MATERIALIZED VIEW, CREATE TABLE AS, REFRESH, or SET LOCALITY on tables with large per-index footprints, to avoid mid-backfill disk exhaustion.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[6030,6033,6034,6035],{"name":6031,"slug":6032,"type":15},"Data Modeling","data-modeling",{"name":13,"slug":14,"type":15},{"name":6020,"slug":6021,"type":15},{"name":6036,"slug":6037,"type":15},"SQL","sql","2026-07-12T07:57:22.763788",{"slug":6040,"name":6040,"fn":6041,"description":6042,"org":6043,"tags":6044,"stars":533,"repoUrl":6022,"updatedAt":6053},"auditing-cis-benchmark","audit CockroachDB clusters against CIS benchmarks","Audits a self-hosted CockroachDB cluster against the CIS CockroachDB Benchmark v1.0.0 Level 1 controls. Supports two audit depths — quick automated scans and full CIS audit procedures. Produces a structured PASS\u002FFAIL\u002FMANUAL report covering installation, system hardening, logging, user access, data protection, and CockroachDB settings. Use when preparing for CIS compliance assessments, hardening self-hosted deployments, or validating security posture against industry benchmarks.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[6045,6048,6051,6052],{"name":6046,"slug":6047,"type":15},"Audit","audit",{"name":6049,"slug":6050,"type":15},"Compliance","compliance",{"name":13,"slug":14,"type":15},{"name":5967,"slug":5968,"type":15},"2026-07-18T05:48:00.862384",{"slug":6055,"name":6055,"fn":6056,"description":6057,"org":6058,"tags":6059,"stars":533,"repoUrl":6022,"updatedAt":6064},"auditing-cloud-cluster-security","audit CockroachDB cluster security posture","Audits the security posture of a CockroachDB cluster (Cloud or self-hosted) across network, authentication, authorization, encryption, audit logging, and backup dimensions. Use when assessing cluster security readiness, preparing for compliance reviews, or investigating security configuration gaps.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[6060,6061,6062,6063],{"name":6046,"slug":6047,"type":15},{"name":13,"slug":14,"type":15},{"name":6008,"slug":6009,"type":15},{"name":5967,"slug":5968,"type":15},"2026-07-12T07:57:01.506735",{"slug":6066,"name":6066,"fn":6067,"description":6068,"org":6069,"tags":6070,"stars":533,"repoUrl":6022,"updatedAt":6077},"auditing-table-statistics","audit optimizer table statistics","Audits optimizer table statistics for staleness, missing coverage, and data quality issues using SHOW STATISTICS. Use when diagnosing poor query performance, unexpected plan changes, or after bulk data changes to identify stale statistics requiring refresh via CREATE STATISTICS.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[6071,6072,6075,6076],{"name":6046,"slug":6047,"type":15},{"name":6073,"slug":6074,"type":15},"Data Analysis","data-analysis",{"name":13,"slug":14,"type":15},{"name":6020,"slug":6021,"type":15},"2026-07-12T07:57:16.190081",{"slug":6079,"name":6079,"fn":6080,"description":6081,"org":6082,"tags":6083,"stars":533,"repoUrl":6022,"updatedAt":6090},"benchmarking-transaction-patterns","benchmark CockroachDB transaction patterns","Guides benchmarking and comparing explicit multi-statement transactions versus single-statement CTE transactions in CockroachDB, with fair test methodology, contention analysis, and performance interpretation. Use when comparing transaction formulations, benchmarking CockroachDB workloads under contention, investigating retry pressure, or deciding whether to rewrite multi-step application flows into single SQL statements.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[6084,6085,6088,6089],{"name":13,"slug":14,"type":15},{"name":6086,"slug":6087,"type":15},"Engineering","engineering",{"name":6020,"slug":6021,"type":15},{"name":6036,"slug":6037,"type":15},"2026-07-12T07:57:26.543278",40,{"items":6093,"total":704},[6094,6101,6107,6114,6121,6127],{"slug":64,"name":64,"fn":5944,"description":5945,"org":6095,"tags":6096,"stars":25,"repoUrl":26,"updatedAt":5956},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[6097,6098,6099,6100],{"name":13,"slug":14,"type":15},{"name":5950,"slug":5951,"type":15},{"name":20,"slug":21,"type":15},{"name":5954,"slug":5955,"type":15},{"slug":183,"name":183,"fn":5958,"description":5959,"org":6102,"tags":6103,"stars":25,"repoUrl":26,"updatedAt":5969},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[6104,6105,6106],{"name":17,"slug":18,"type":15},{"name":5964,"slug":5965,"type":15},{"name":5967,"slug":5968,"type":15},{"slug":171,"name":171,"fn":5971,"description":5972,"org":6108,"tags":6109,"stars":25,"repoUrl":26,"updatedAt":5981},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[6110,6111,6112,6113],{"name":13,"slug":14,"type":15},{"name":23,"slug":24,"type":15},{"name":20,"slug":21,"type":15},{"name":5979,"slug":5980,"type":15},{"slug":4,"name":4,"fn":5,"description":6,"org":6115,"tags":6116,"stars":25,"repoUrl":26,"updatedAt":27},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[6117,6118,6119,6120],{"name":13,"slug":14,"type":15},{"name":23,"slug":24,"type":15},{"name":17,"slug":18,"type":15},{"name":20,"slug":21,"type":15},{"slug":5990,"name":5990,"fn":5991,"description":5992,"org":6122,"tags":6123,"stars":25,"repoUrl":26,"updatedAt":5998},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[6124,6125,6126],{"name":13,"slug":14,"type":15},{"name":17,"slug":18,"type":15},{"name":20,"slug":21,"type":15},{"slug":195,"name":195,"fn":6000,"description":6001,"org":6128,"tags":6129,"stars":25,"repoUrl":26,"updatedAt":6010},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[6130,6131,6132,6133],{"name":13,"slug":14,"type":15},{"name":17,"slug":18,"type":15},{"name":20,"slug":21,"type":15},{"name":6008,"slug":6009,"type":15}]