[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-grafana-slo-investigate":3,"mdc--dm8ks4-key":34,"related-repo-grafana-slo-investigate":1830,"related-org-grafana-slo-investigate":1933},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":23,"repoUrl":24,"updatedAt":25,"license":26,"forks":27,"topics":28,"repo":29,"sourceUrl":32,"mdContent":33},"slo-investigate","diagnose breaching Grafana SLOs","Diagnoses breaching Grafana SLOs via gcx - root cause analysis, dimensional breakdown, alert rule correlation, runbook access. Use when a specific SLO is breaching or alerting and the user needs to understand why. Trigger on phrases like \"investigate SLO\", \"why is my SLO breaching\", \"SLO error budget burning\", \"SLO alerting\". For SLO status overview use slo-check-status. For creating or modifying SLOs use slo-manage. For optimization suggestions use slo-optimize.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"grafana","Grafana","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fgrafana.jpg",[12,16,19,20],{"name":13,"slug":14,"type":15},"Observability","observability","tag",{"name":17,"slug":18,"type":15},"Monitoring","monitoring",{"name":9,"slug":8,"type":15},{"name":21,"slug":22,"type":15},"Debugging","debugging",430,"https:\u002F\u002Fgithub.com\u002Fgrafana\u002Fgcx","2026-07-15T05:34:19.396839",null,29,[],{"repoUrl":24,"stars":23,"forks":27,"topics":30,"description":31},[],"A CLI for managing Grafana Cloud resources. Optimized for agentic usage.","https:\u002F\u002Fgithub.com\u002Fgrafana\u002Fgcx\u002Ftree\u002FHEAD\u002Fclaude-plugin\u002Fskills\u002Fslo-investigate","---\nname: slo-investigate\ndescription: Diagnoses breaching Grafana SLOs via gcx - root cause analysis, dimensional breakdown, alert rule correlation, runbook access. Use when a specific SLO is breaching or alerting and the user needs to understand why. Trigger on phrases like \"investigate SLO\", \"why is my SLO breaching\", \"SLO error budget burning\", \"SLO alerting\". For SLO status overview use slo-check-status. For creating or modifying SLOs use slo-manage. For optimization suggestions use slo-optimize.\nallowed-tools: Bash\n---\n\n# SLO Investigator\n\nDeep-dive investigation of breaching SLOs: dimensional breakdown, alert correlation, runbook access. For experienced operators — no hand-holding.\n\n## Core Principles\n\n1. Use gcx commands — do not call Grafana APIs directly (no curl, no HTTP libraries)\n2. Trust the user's expertise — skip obvious context, get to the root cause\n3. Use `-o json` for agent processing, default format for user display; show graphs for time-series data\n4. Errors collected at the end — do not interleave error handling in workflow steps\n5. Use `--from`\u002F`--to` for all time-range commands (never `--start`\u002F`--end`)\n\n## Investigation Workflow\n\n### Step 1: Retrieve SLO Definition\n\n```bash\ngcx slo definitions get \u003CUUID> -o json\n```\n\nExtract from the JSON response:\n- `.metadata.name` — SLO name\n- `.spec.query.type` — query type: `ratio`, `freeform`, or `threshold`\n- For ratio: `.spec.query.ratio.successMetric`, `.spec.query.ratio.totalMetric`, `.spec.query.ratio.groupByLabels[]`\n- For freeform: `.spec.query.freeform.query`\n- `.spec.objectives[0].value` — objective (0–1), `.spec.objectives[0].window` — window\n- `.spec.destinationDatasource.uid` — Prometheus datasource UID\n- `.spec.alerting.fastBurn.annotations`, `.spec.alerting.slowBurn.annotations` — runbook\u002Fdashboard URLs\n- `.metadata.annotations` — additional runbook\u002Fdashboard references\n\nIf no UUID is given, list SLOs and ask which to investigate:\n```bash\ngcx slo definitions list\n```\n\n### Step 2: Check Status with Wide Output\n\n```bash\ngcx slo definitions status \u003CUUID> -o wide\n```\n\nThis shows SLI, ERROR_BUDGET, BURN_RATE, SLI_1H, SLI_1D, and STATUS.\n\n**Early exit — OK status:** If STATUS is OK, report health metrics and stop:\n```\nSLO: \u003Cname> — Status: OK\nSLI: \u003Cvalue>  |  Error budget remaining: \u003Cbudget>%  |  Burn rate: \u003Crate>x\n1h SLI: \u003Csli_1h>  |  1d SLI: \u003Csli_1d>\nNo action needed.\n```\n\n**Early exit — NODATA status:** If STATUS is NODATA, branch to NODATA diagnosis:\n```\nSLO: \u003Cname> — Status: NODATA\nRecording rule metrics unavailable. Likely causes:\n- Destination datasource misconfigured (check .spec.destinationDatasource.uid)\n- Grafana recording rules not yet evaluated (can take 1–2 minutes after creation)\n- Prometheus federation\u002Fremote write issue\n\nCheck: gcx datasources list --type prometheus\nThen verify the destination datasource UID matches what the SLO expects.\n```\n\nWhen diagnosing NODATA, two environment realities save wasted queries: on\nremote-write stacks there are no scrape targets, so `up`-based scrape-health\nchecks return nothing — absence of `up` series is not evidence of a problem.\nAnd if `datasources list --type prometheus` returns an empty list, the stack\nmay leave `type` blank in list payloads (known issue) — rerun without `--type`\nor use the UID from `.spec.destinationDatasource.uid` directly.\n\n**Lifecycle states:** If status is Creating\u002FUpdating\u002FDeleting\u002FError, report that the SLO is in a transient state and investigate the Grafana backend.\n\n### Step 3: Render Timeline\n\n```bash\ngcx slo definitions timeline \u003CUUID> --from now-1h --to now\n```\n\nFor wider trends:\n```bash\ngcx slo definitions timeline \u003CUUID> --from now-24h --to now\n```\n\nShow the graph output (default). Use it to identify when breaching started and how severe it is.\n\n### Step 4: Dimensional Breakdown\n\nResolve the datasource UID. If `.spec.destinationDatasource.uid` is set, use it. Otherwise auto-discover:\n```bash\ngcx datasources list --type prometheus\n```\n\n**For ratio queries** — extract success\u002Ftotal metric selectors and groupByLabels, then query dimensional breakdown:\n\n```bash\n# Success rate by dimension (e.g., cluster, status_code, endpoint)\ngcx metrics query -d \u003Cdatasource-uid> \\\n  'sum by (\u003CgroupByLabel>) (rate(\u003CsuccessMetric>[5m])) \u002F sum by (\u003CgroupByLabel>) (rate(\u003CtotalMetric>[5m]))' \\\n  --from now-1h --to now --step 1m\n\n# Error rate by dimension to spot the bad actor\ngcx metrics query -d \u003Cdatasource-uid> \\\n  'sum by (\u003CgroupByLabel>) (rate(\u003CtotalMetric>[5m])) - sum by (\u003CgroupByLabel>) (rate(\u003CsuccessMetric>[5m]))' \\\n  --from now-1h --to now --step 1m\n```\n\nIf `groupByLabels` is empty, try common dimensions: `cluster`, `namespace`, `service`, `status_code`, `endpoint`.\n\n**For freeform queries** — use the raw PromQL expression and add `by (\u003Clabel>)` grouping:\n\n```bash\ngcx metrics query -d \u003Cdatasource-uid> \\\n  '\u003Cfreeform_expression> by (cluster)' \\\n  --from now-1h --to now --step 1m\n\n# Also try other likely breakdown dimensions\ngcx metrics query -d \u003Cdatasource-uid> \\\n  '\u003Cfreeform_expression> by (namespace)' \\\n  --from now-1h --to now --step 1m\n```\n\nUse graph output to display dimensional trends visually. Use `-o json` to extract exact values for the report.\n\nTo query the SLO's own recording rule metrics instead (`grafana_slo_sli_window`, burn rate, error budget expressions), see [references\u002Fslo-promql-patterns.md](references\u002Fslo-promql-patterns.md) for the metric inventory and ready-made PromQL patterns.\n\n### Step 5: Search for Related Alert Rules\n\n```bash\ngcx alert rules list -o json | jq '[.[] | .rules[]? | select(.name | test(\"\u003Cslo-name>\"; \"i\"))]'\n```\n\nAlso try searching by UUID fragment if the name-based search returns no results:\n```bash\ngcx alert rules list -o json | jq '[.[] | .rules[]? | select(.labels.grafana_slo_uuid == \"\u003CUUID>\" or (.name | test(\"\u003Cslo-name>\"; \"i\")))]'\n```\n\nExtract for each matching rule: name, state (firing\u002Fpending\u002Finactive), labels, and annotations.\n\n### Step 6: Extract Runbook and Dashboard URLs\n\nCollect URLs from:\n- `.spec.alerting.fastBurn.annotations.runbook_url`\n- `.spec.alerting.fastBurn.annotations.dashboard_url`\n- `.spec.alerting.slowBurn.annotations.runbook_url`\n- `.spec.alerting.slowBurn.annotations.dashboard_url`\n- `.metadata.annotations.*`\n\nIf a GitHub URL is found in runbook annotations and `gh` is available:\n```bash\n# Convert GitHub web URL to API path and fetch content\ngh api \u002Frepos\u002F\u003Cowner>\u002F\u003Crepo>\u002Fcontents\u002F\u003Cpath> --jq '.content' | base64 --decode\n```\n\nFor raw GitHub URLs (`raw.githubusercontent.com`), extract the content URL pattern and use `gh api` with the equivalent API endpoint.\n\n## Output Format\n\nAfter completing the investigation, present results in this structure:\n\n```\nSLO: \u003Cname>\nTarget: \u003Cobjective>% over \u003Cwindow>  |  Status: BREACHING\nSLI: \u003Ccurrent>%  |  Error budget remaining: \u003Cbudget>%  |  Burn rate: \u003Crate>x\n1h SLI: \u003Csli_1h>%  |  1d SLI: \u003Csli_1d>%\n\n[Timeline graph — show default output]\n\nDimensional Breakdown:\n  Worst dimension: \u003Clabel>=\u003Cvalue> at \u003Cerror_rate>% error rate\n  [Additional dimensions ranked by error rate]\n\nRelated Alert Rules:\n  - \u003Crule_name>: \u003Cstate> [labels: \u003Ckey>=\u003Cvalue>]\n\nRunbook: \u003Curl>\nDashboard: \u003Curl>\n\n[If runbook fetched]: Key runbook steps:\n  \u003Crelevant excerpt>\n\nNext actions:\n1. \u003Cmost specific actionable step based on findings>\n2. \u003Cfollow-up investigation or escalation path>\n3. \u003Cif budget near zero: suggest slo-optimize for objective review>\n```\n\n## Error Handling\n\n- **gcx slo definitions get fails with 404**: SLO UUID not found. Run `gcx slo definitions list` and confirm the UUID.\n- **gcx slo definitions status returns empty**: No status available — SLO may be newly created. Check if recording rules are running (STATUS may show NODATA).\n- **gcx metrics query fails with datasource error**: Datasource UID may be wrong. Run `gcx datasources list --type prometheus` to find the correct UID. If that list is empty (blank `type` fields in the payload), rerun without `--type` and select by name.\n- **gcx metrics query returns no data**: The SLO metrics may write to a separate datasource (check `.spec.destinationDatasource.uid`). Try both the destination datasource and the default Prometheus datasource.\n- **alert rules list returns empty**: Alert rules may be in a different folder. Try without filters: `gcx alert rules list -o json | jq length` to confirm total count.\n- **gh api fails**: If `gh` is not authenticated or unavailable, report the runbook URL directly and skip content fetching.\n- **SLO has no groupByLabels (ratio query)**: Try common breakdown dimensions: `cluster`, `namespace`, `service`, `endpoint`, `status_code`. Report which ones return data.\n- **Multiple SLOs with similar names**: When searching alert rules by name pattern, report all matches and their states — don't silently drop duplicates.\n",{"data":35,"body":37},{"name":4,"description":6,"allowed-tools":36},"Bash",{"type":38,"children":39},"root",[40,49,55,62,131,137,144,215,220,363,368,395,401,453,458,469,479,489,498,549,559,565,628,633,692,697,703,715,749,759,984,1033,1051,1245,1257,1278,1284,1346,1351,1406,1411,1417,1422,1470,1483,1611,1632,1638,1643,1652,1658,1824],{"type":41,"tag":42,"props":43,"children":45},"element","h1",{"id":44},"slo-investigator",[46],{"type":47,"value":48},"text","SLO Investigator",{"type":41,"tag":50,"props":51,"children":52},"p",{},[53],{"type":47,"value":54},"Deep-dive investigation of breaching SLOs: dimensional breakdown, alert correlation, runbook access. For experienced operators — no hand-holding.",{"type":41,"tag":56,"props":57,"children":59},"h2",{"id":58},"core-principles",[60],{"type":47,"value":61},"Core Principles",{"type":41,"tag":63,"props":64,"children":65},"ol",{},[66,72,77,91,96],{"type":41,"tag":67,"props":68,"children":69},"li",{},[70],{"type":47,"value":71},"Use gcx commands — do not call Grafana APIs directly (no curl, no HTTP libraries)",{"type":41,"tag":67,"props":73,"children":74},{},[75],{"type":47,"value":76},"Trust the user's expertise — skip obvious context, get to the root cause",{"type":41,"tag":67,"props":78,"children":79},{},[80,82,89],{"type":47,"value":81},"Use ",{"type":41,"tag":83,"props":84,"children":86},"code",{"className":85},[],[87],{"type":47,"value":88},"-o json",{"type":47,"value":90}," for agent processing, default format for user display; show graphs for time-series data",{"type":41,"tag":67,"props":92,"children":93},{},[94],{"type":47,"value":95},"Errors collected at the end — do not interleave error handling in workflow steps",{"type":41,"tag":67,"props":97,"children":98},{},[99,100,106,108,114,116,122,123,129],{"type":47,"value":81},{"type":41,"tag":83,"props":101,"children":103},{"className":102},[],[104],{"type":47,"value":105},"--from",{"type":47,"value":107},"\u002F",{"type":41,"tag":83,"props":109,"children":111},{"className":110},[],[112],{"type":47,"value":113},"--to",{"type":47,"value":115}," for all time-range commands (never ",{"type":41,"tag":83,"props":117,"children":119},{"className":118},[],[120],{"type":47,"value":121},"--start",{"type":47,"value":107},{"type":41,"tag":83,"props":124,"children":126},{"className":125},[],[127],{"type":47,"value":128},"--end",{"type":47,"value":130},")",{"type":41,"tag":56,"props":132,"children":134},{"id":133},"investigation-workflow",[135],{"type":47,"value":136},"Investigation Workflow",{"type":41,"tag":138,"props":139,"children":141},"h3",{"id":140},"step-1-retrieve-slo-definition",[142],{"type":47,"value":143},"Step 1: Retrieve SLO Definition",{"type":41,"tag":145,"props":146,"children":151},"pre",{"className":147,"code":148,"language":149,"meta":150,"style":150},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","gcx slo definitions get \u003CUUID> -o json\n","bash","",[152],{"type":41,"tag":83,"props":153,"children":154},{"__ignoreMap":150},[155],{"type":41,"tag":156,"props":157,"children":160},"span",{"class":158,"line":159},"line",1,[161,167,173,178,183,189,194,200,205,210],{"type":41,"tag":156,"props":162,"children":164},{"style":163},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[165],{"type":47,"value":166},"gcx",{"type":41,"tag":156,"props":168,"children":170},{"style":169},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[171],{"type":47,"value":172}," slo",{"type":41,"tag":156,"props":174,"children":175},{"style":169},[176],{"type":47,"value":177}," definitions",{"type":41,"tag":156,"props":179,"children":180},{"style":169},[181],{"type":47,"value":182}," get",{"type":41,"tag":156,"props":184,"children":186},{"style":185},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[187],{"type":47,"value":188}," \u003C",{"type":41,"tag":156,"props":190,"children":191},{"style":169},[192],{"type":47,"value":193},"UUI",{"type":41,"tag":156,"props":195,"children":197},{"style":196},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[198],{"type":47,"value":199},"D",{"type":41,"tag":156,"props":201,"children":202},{"style":185},[203],{"type":47,"value":204},">",{"type":41,"tag":156,"props":206,"children":207},{"style":169},[208],{"type":47,"value":209}," -o",{"type":41,"tag":156,"props":211,"children":212},{"style":169},[213],{"type":47,"value":214}," json\n",{"type":41,"tag":50,"props":216,"children":217},{},[218],{"type":47,"value":219},"Extract from the JSON response:",{"type":41,"tag":221,"props":222,"children":223},"ul",{},[224,235,268,293,304,323,334,352],{"type":41,"tag":67,"props":225,"children":226},{},[227,233],{"type":41,"tag":83,"props":228,"children":230},{"className":229},[],[231],{"type":47,"value":232},".metadata.name",{"type":47,"value":234}," — SLO name",{"type":41,"tag":67,"props":236,"children":237},{},[238,244,246,252,254,260,262],{"type":41,"tag":83,"props":239,"children":241},{"className":240},[],[242],{"type":47,"value":243},".spec.query.type",{"type":47,"value":245}," — query type: ",{"type":41,"tag":83,"props":247,"children":249},{"className":248},[],[250],{"type":47,"value":251},"ratio",{"type":47,"value":253},", ",{"type":41,"tag":83,"props":255,"children":257},{"className":256},[],[258],{"type":47,"value":259},"freeform",{"type":47,"value":261},", or ",{"type":41,"tag":83,"props":263,"children":265},{"className":264},[],[266],{"type":47,"value":267},"threshold",{"type":41,"tag":67,"props":269,"children":270},{},[271,273,279,280,286,287],{"type":47,"value":272},"For ratio: ",{"type":41,"tag":83,"props":274,"children":276},{"className":275},[],[277],{"type":47,"value":278},".spec.query.ratio.successMetric",{"type":47,"value":253},{"type":41,"tag":83,"props":281,"children":283},{"className":282},[],[284],{"type":47,"value":285},".spec.query.ratio.totalMetric",{"type":47,"value":253},{"type":41,"tag":83,"props":288,"children":290},{"className":289},[],[291],{"type":47,"value":292},".spec.query.ratio.groupByLabels[]",{"type":41,"tag":67,"props":294,"children":295},{},[296,298],{"type":47,"value":297},"For freeform: ",{"type":41,"tag":83,"props":299,"children":301},{"className":300},[],[302],{"type":47,"value":303},".spec.query.freeform.query",{"type":41,"tag":67,"props":305,"children":306},{},[307,313,315,321],{"type":41,"tag":83,"props":308,"children":310},{"className":309},[],[311],{"type":47,"value":312},".spec.objectives[0].value",{"type":47,"value":314}," — objective (0–1), ",{"type":41,"tag":83,"props":316,"children":318},{"className":317},[],[319],{"type":47,"value":320},".spec.objectives[0].window",{"type":47,"value":322}," — window",{"type":41,"tag":67,"props":324,"children":325},{},[326,332],{"type":41,"tag":83,"props":327,"children":329},{"className":328},[],[330],{"type":47,"value":331},".spec.destinationDatasource.uid",{"type":47,"value":333}," — Prometheus datasource UID",{"type":41,"tag":67,"props":335,"children":336},{},[337,343,344,350],{"type":41,"tag":83,"props":338,"children":340},{"className":339},[],[341],{"type":47,"value":342},".spec.alerting.fastBurn.annotations",{"type":47,"value":253},{"type":41,"tag":83,"props":345,"children":347},{"className":346},[],[348],{"type":47,"value":349},".spec.alerting.slowBurn.annotations",{"type":47,"value":351}," — runbook\u002Fdashboard URLs",{"type":41,"tag":67,"props":353,"children":354},{},[355,361],{"type":41,"tag":83,"props":356,"children":358},{"className":357},[],[359],{"type":47,"value":360},".metadata.annotations",{"type":47,"value":362}," — additional runbook\u002Fdashboard references",{"type":41,"tag":50,"props":364,"children":365},{},[366],{"type":47,"value":367},"If no UUID is given, list SLOs and ask which to investigate:",{"type":41,"tag":145,"props":369,"children":371},{"className":147,"code":370,"language":149,"meta":150,"style":150},"gcx slo definitions list\n",[372],{"type":41,"tag":83,"props":373,"children":374},{"__ignoreMap":150},[375],{"type":41,"tag":156,"props":376,"children":377},{"class":158,"line":159},[378,382,386,390],{"type":41,"tag":156,"props":379,"children":380},{"style":163},[381],{"type":47,"value":166},{"type":41,"tag":156,"props":383,"children":384},{"style":169},[385],{"type":47,"value":172},{"type":41,"tag":156,"props":387,"children":388},{"style":169},[389],{"type":47,"value":177},{"type":41,"tag":156,"props":391,"children":392},{"style":169},[393],{"type":47,"value":394}," list\n",{"type":41,"tag":138,"props":396,"children":398},{"id":397},"step-2-check-status-with-wide-output",[399],{"type":47,"value":400},"Step 2: Check Status with Wide Output",{"type":41,"tag":145,"props":402,"children":404},{"className":147,"code":403,"language":149,"meta":150,"style":150},"gcx slo definitions status \u003CUUID> -o wide\n",[405],{"type":41,"tag":83,"props":406,"children":407},{"__ignoreMap":150},[408],{"type":41,"tag":156,"props":409,"children":410},{"class":158,"line":159},[411,415,419,423,428,432,436,440,444,448],{"type":41,"tag":156,"props":412,"children":413},{"style":163},[414],{"type":47,"value":166},{"type":41,"tag":156,"props":416,"children":417},{"style":169},[418],{"type":47,"value":172},{"type":41,"tag":156,"props":420,"children":421},{"style":169},[422],{"type":47,"value":177},{"type":41,"tag":156,"props":424,"children":425},{"style":169},[426],{"type":47,"value":427}," status",{"type":41,"tag":156,"props":429,"children":430},{"style":185},[431],{"type":47,"value":188},{"type":41,"tag":156,"props":433,"children":434},{"style":169},[435],{"type":47,"value":193},{"type":41,"tag":156,"props":437,"children":438},{"style":196},[439],{"type":47,"value":199},{"type":41,"tag":156,"props":441,"children":442},{"style":185},[443],{"type":47,"value":204},{"type":41,"tag":156,"props":445,"children":446},{"style":169},[447],{"type":47,"value":209},{"type":41,"tag":156,"props":449,"children":450},{"style":169},[451],{"type":47,"value":452}," wide\n",{"type":41,"tag":50,"props":454,"children":455},{},[456],{"type":47,"value":457},"This shows SLI, ERROR_BUDGET, BURN_RATE, SLI_1H, SLI_1D, and STATUS.",{"type":41,"tag":50,"props":459,"children":460},{},[461,467],{"type":41,"tag":462,"props":463,"children":464},"strong",{},[465],{"type":47,"value":466},"Early exit — OK status:",{"type":47,"value":468}," If STATUS is OK, report health metrics and stop:",{"type":41,"tag":145,"props":470,"children":474},{"className":471,"code":473,"language":47},[472],"language-text","SLO: \u003Cname> — Status: OK\nSLI: \u003Cvalue>  |  Error budget remaining: \u003Cbudget>%  |  Burn rate: \u003Crate>x\n1h SLI: \u003Csli_1h>  |  1d SLI: \u003Csli_1d>\nNo action needed.\n",[475],{"type":41,"tag":83,"props":476,"children":477},{"__ignoreMap":150},[478],{"type":47,"value":473},{"type":41,"tag":50,"props":480,"children":481},{},[482,487],{"type":41,"tag":462,"props":483,"children":484},{},[485],{"type":47,"value":486},"Early exit — NODATA status:",{"type":47,"value":488}," If STATUS is NODATA, branch to NODATA diagnosis:",{"type":41,"tag":145,"props":490,"children":493},{"className":491,"code":492,"language":47},[472],"SLO: \u003Cname> — Status: NODATA\nRecording rule metrics unavailable. Likely causes:\n- Destination datasource misconfigured (check .spec.destinationDatasource.uid)\n- Grafana recording rules not yet evaluated (can take 1–2 minutes after creation)\n- Prometheus federation\u002Fremote write issue\n\nCheck: gcx datasources list --type prometheus\nThen verify the destination datasource UID matches what the SLO expects.\n",[494],{"type":41,"tag":83,"props":495,"children":496},{"__ignoreMap":150},[497],{"type":47,"value":492},{"type":41,"tag":50,"props":499,"children":500},{},[501,503,509,511,516,518,524,526,532,534,540,542,547],{"type":47,"value":502},"When diagnosing NODATA, two environment realities save wasted queries: on\nremote-write stacks there are no scrape targets, so ",{"type":41,"tag":83,"props":504,"children":506},{"className":505},[],[507],{"type":47,"value":508},"up",{"type":47,"value":510},"-based scrape-health\nchecks return nothing — absence of ",{"type":41,"tag":83,"props":512,"children":514},{"className":513},[],[515],{"type":47,"value":508},{"type":47,"value":517}," series is not evidence of a problem.\nAnd if ",{"type":41,"tag":83,"props":519,"children":521},{"className":520},[],[522],{"type":47,"value":523},"datasources list --type prometheus",{"type":47,"value":525}," returns an empty list, the stack\nmay leave ",{"type":41,"tag":83,"props":527,"children":529},{"className":528},[],[530],{"type":47,"value":531},"type",{"type":47,"value":533}," blank in list payloads (known issue) — rerun without ",{"type":41,"tag":83,"props":535,"children":537},{"className":536},[],[538],{"type":47,"value":539},"--type",{"type":47,"value":541},"\nor use the UID from ",{"type":41,"tag":83,"props":543,"children":545},{"className":544},[],[546],{"type":47,"value":331},{"type":47,"value":548}," directly.",{"type":41,"tag":50,"props":550,"children":551},{},[552,557],{"type":41,"tag":462,"props":553,"children":554},{},[555],{"type":47,"value":556},"Lifecycle states:",{"type":47,"value":558}," If status is Creating\u002FUpdating\u002FDeleting\u002FError, report that the SLO is in a transient state and investigate the Grafana backend.",{"type":41,"tag":138,"props":560,"children":562},{"id":561},"step-3-render-timeline",[563],{"type":47,"value":564},"Step 3: Render Timeline",{"type":41,"tag":145,"props":566,"children":568},{"className":147,"code":567,"language":149,"meta":150,"style":150},"gcx slo definitions timeline \u003CUUID> --from now-1h --to now\n",[569],{"type":41,"tag":83,"props":570,"children":571},{"__ignoreMap":150},[572],{"type":41,"tag":156,"props":573,"children":574},{"class":158,"line":159},[575,579,583,587,592,596,600,604,608,613,618,623],{"type":41,"tag":156,"props":576,"children":577},{"style":163},[578],{"type":47,"value":166},{"type":41,"tag":156,"props":580,"children":581},{"style":169},[582],{"type":47,"value":172},{"type":41,"tag":156,"props":584,"children":585},{"style":169},[586],{"type":47,"value":177},{"type":41,"tag":156,"props":588,"children":589},{"style":169},[590],{"type":47,"value":591}," timeline",{"type":41,"tag":156,"props":593,"children":594},{"style":185},[595],{"type":47,"value":188},{"type":41,"tag":156,"props":597,"children":598},{"style":169},[599],{"type":47,"value":193},{"type":41,"tag":156,"props":601,"children":602},{"style":196},[603],{"type":47,"value":199},{"type":41,"tag":156,"props":605,"children":606},{"style":185},[607],{"type":47,"value":204},{"type":41,"tag":156,"props":609,"children":610},{"style":169},[611],{"type":47,"value":612}," --from",{"type":41,"tag":156,"props":614,"children":615},{"style":169},[616],{"type":47,"value":617}," now-1h",{"type":41,"tag":156,"props":619,"children":620},{"style":169},[621],{"type":47,"value":622}," --to",{"type":41,"tag":156,"props":624,"children":625},{"style":169},[626],{"type":47,"value":627}," now\n",{"type":41,"tag":50,"props":629,"children":630},{},[631],{"type":47,"value":632},"For wider trends:",{"type":41,"tag":145,"props":634,"children":636},{"className":147,"code":635,"language":149,"meta":150,"style":150},"gcx slo definitions timeline \u003CUUID> --from now-24h --to now\n",[637],{"type":41,"tag":83,"props":638,"children":639},{"__ignoreMap":150},[640],{"type":41,"tag":156,"props":641,"children":642},{"class":158,"line":159},[643,647,651,655,659,663,667,671,675,679,684,688],{"type":41,"tag":156,"props":644,"children":645},{"style":163},[646],{"type":47,"value":166},{"type":41,"tag":156,"props":648,"children":649},{"style":169},[650],{"type":47,"value":172},{"type":41,"tag":156,"props":652,"children":653},{"style":169},[654],{"type":47,"value":177},{"type":41,"tag":156,"props":656,"children":657},{"style":169},[658],{"type":47,"value":591},{"type":41,"tag":156,"props":660,"children":661},{"style":185},[662],{"type":47,"value":188},{"type":41,"tag":156,"props":664,"children":665},{"style":169},[666],{"type":47,"value":193},{"type":41,"tag":156,"props":668,"children":669},{"style":196},[670],{"type":47,"value":199},{"type":41,"tag":156,"props":672,"children":673},{"style":185},[674],{"type":47,"value":204},{"type":41,"tag":156,"props":676,"children":677},{"style":169},[678],{"type":47,"value":612},{"type":41,"tag":156,"props":680,"children":681},{"style":169},[682],{"type":47,"value":683}," now-24h",{"type":41,"tag":156,"props":685,"children":686},{"style":169},[687],{"type":47,"value":622},{"type":41,"tag":156,"props":689,"children":690},{"style":169},[691],{"type":47,"value":627},{"type":41,"tag":50,"props":693,"children":694},{},[695],{"type":47,"value":696},"Show the graph output (default). Use it to identify when breaching started and how severe it is.",{"type":41,"tag":138,"props":698,"children":700},{"id":699},"step-4-dimensional-breakdown",[701],{"type":47,"value":702},"Step 4: Dimensional Breakdown",{"type":41,"tag":50,"props":704,"children":705},{},[706,708,713],{"type":47,"value":707},"Resolve the datasource UID. If ",{"type":41,"tag":83,"props":709,"children":711},{"className":710},[],[712],{"type":47,"value":331},{"type":47,"value":714}," is set, use it. Otherwise auto-discover:",{"type":41,"tag":145,"props":716,"children":718},{"className":147,"code":717,"language":149,"meta":150,"style":150},"gcx datasources list --type prometheus\n",[719],{"type":41,"tag":83,"props":720,"children":721},{"__ignoreMap":150},[722],{"type":41,"tag":156,"props":723,"children":724},{"class":158,"line":159},[725,729,734,739,744],{"type":41,"tag":156,"props":726,"children":727},{"style":163},[728],{"type":47,"value":166},{"type":41,"tag":156,"props":730,"children":731},{"style":169},[732],{"type":47,"value":733}," datasources",{"type":41,"tag":156,"props":735,"children":736},{"style":169},[737],{"type":47,"value":738}," list",{"type":41,"tag":156,"props":740,"children":741},{"style":169},[742],{"type":47,"value":743}," --type",{"type":41,"tag":156,"props":745,"children":746},{"style":169},[747],{"type":47,"value":748}," prometheus\n",{"type":41,"tag":50,"props":750,"children":751},{},[752,757],{"type":41,"tag":462,"props":753,"children":754},{},[755],{"type":47,"value":756},"For ratio queries",{"type":47,"value":758}," — extract success\u002Ftotal metric selectors and groupByLabels, then query dimensional breakdown:",{"type":41,"tag":145,"props":760,"children":762},{"className":147,"code":761,"language":149,"meta":150,"style":150},"# Success rate by dimension (e.g., cluster, status_code, endpoint)\ngcx metrics query -d \u003Cdatasource-uid> \\\n  'sum by (\u003CgroupByLabel>) (rate(\u003CsuccessMetric>[5m])) \u002F sum by (\u003CgroupByLabel>) (rate(\u003CtotalMetric>[5m]))' \\\n  --from now-1h --to now --step 1m\n\n# Error rate by dimension to spot the bad actor\ngcx metrics query -d \u003Cdatasource-uid> \\\n  'sum by (\u003CgroupByLabel>) (rate(\u003CtotalMetric>[5m])) - sum by (\u003CgroupByLabel>) (rate(\u003CsuccessMetric>[5m]))' \\\n  --from now-1h --to now --step 1m\n",[763],{"type":41,"tag":83,"props":764,"children":765},{"__ignoreMap":150},[766,775,821,844,876,886,895,935,956],{"type":41,"tag":156,"props":767,"children":768},{"class":158,"line":159},[769],{"type":41,"tag":156,"props":770,"children":772},{"style":771},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[773],{"type":47,"value":774},"# Success rate by dimension (e.g., cluster, status_code, endpoint)\n",{"type":41,"tag":156,"props":776,"children":778},{"class":158,"line":777},2,[779,783,788,793,798,802,807,812,816],{"type":41,"tag":156,"props":780,"children":781},{"style":163},[782],{"type":47,"value":166},{"type":41,"tag":156,"props":784,"children":785},{"style":169},[786],{"type":47,"value":787}," metrics",{"type":41,"tag":156,"props":789,"children":790},{"style":169},[791],{"type":47,"value":792}," query",{"type":41,"tag":156,"props":794,"children":795},{"style":169},[796],{"type":47,"value":797}," -d",{"type":41,"tag":156,"props":799,"children":800},{"style":185},[801],{"type":47,"value":188},{"type":41,"tag":156,"props":803,"children":804},{"style":169},[805],{"type":47,"value":806},"datasource-ui",{"type":41,"tag":156,"props":808,"children":809},{"style":196},[810],{"type":47,"value":811},"d",{"type":41,"tag":156,"props":813,"children":814},{"style":185},[815],{"type":47,"value":204},{"type":41,"tag":156,"props":817,"children":818},{"style":196},[819],{"type":47,"value":820}," \\\n",{"type":41,"tag":156,"props":822,"children":824},{"class":158,"line":823},3,[825,830,835,840],{"type":41,"tag":156,"props":826,"children":827},{"style":185},[828],{"type":47,"value":829},"  '",{"type":41,"tag":156,"props":831,"children":832},{"style":169},[833],{"type":47,"value":834},"sum by (\u003CgroupByLabel>) (rate(\u003CsuccessMetric>[5m])) \u002F sum by (\u003CgroupByLabel>) (rate(\u003CtotalMetric>[5m]))",{"type":41,"tag":156,"props":836,"children":837},{"style":185},[838],{"type":47,"value":839},"'",{"type":41,"tag":156,"props":841,"children":842},{"style":196},[843],{"type":47,"value":820},{"type":41,"tag":156,"props":845,"children":847},{"class":158,"line":846},4,[848,853,857,861,866,871],{"type":41,"tag":156,"props":849,"children":850},{"style":169},[851],{"type":47,"value":852},"  --from",{"type":41,"tag":156,"props":854,"children":855},{"style":169},[856],{"type":47,"value":617},{"type":41,"tag":156,"props":858,"children":859},{"style":169},[860],{"type":47,"value":622},{"type":41,"tag":156,"props":862,"children":863},{"style":169},[864],{"type":47,"value":865}," now",{"type":41,"tag":156,"props":867,"children":868},{"style":169},[869],{"type":47,"value":870}," --step",{"type":41,"tag":156,"props":872,"children":873},{"style":169},[874],{"type":47,"value":875}," 1m\n",{"type":41,"tag":156,"props":877,"children":879},{"class":158,"line":878},5,[880],{"type":41,"tag":156,"props":881,"children":883},{"emptyLinePlaceholder":882},true,[884],{"type":47,"value":885},"\n",{"type":41,"tag":156,"props":887,"children":889},{"class":158,"line":888},6,[890],{"type":41,"tag":156,"props":891,"children":892},{"style":771},[893],{"type":47,"value":894},"# Error rate by dimension to spot the bad actor\n",{"type":41,"tag":156,"props":896,"children":898},{"class":158,"line":897},7,[899,903,907,911,915,919,923,927,931],{"type":41,"tag":156,"props":900,"children":901},{"style":163},[902],{"type":47,"value":166},{"type":41,"tag":156,"props":904,"children":905},{"style":169},[906],{"type":47,"value":787},{"type":41,"tag":156,"props":908,"children":909},{"style":169},[910],{"type":47,"value":792},{"type":41,"tag":156,"props":912,"children":913},{"style":169},[914],{"type":47,"value":797},{"type":41,"tag":156,"props":916,"children":917},{"style":185},[918],{"type":47,"value":188},{"type":41,"tag":156,"props":920,"children":921},{"style":169},[922],{"type":47,"value":806},{"type":41,"tag":156,"props":924,"children":925},{"style":196},[926],{"type":47,"value":811},{"type":41,"tag":156,"props":928,"children":929},{"style":185},[930],{"type":47,"value":204},{"type":41,"tag":156,"props":932,"children":933},{"style":196},[934],{"type":47,"value":820},{"type":41,"tag":156,"props":936,"children":938},{"class":158,"line":937},8,[939,943,948,952],{"type":41,"tag":156,"props":940,"children":941},{"style":185},[942],{"type":47,"value":829},{"type":41,"tag":156,"props":944,"children":945},{"style":169},[946],{"type":47,"value":947},"sum by (\u003CgroupByLabel>) (rate(\u003CtotalMetric>[5m])) - sum by (\u003CgroupByLabel>) (rate(\u003CsuccessMetric>[5m]))",{"type":41,"tag":156,"props":949,"children":950},{"style":185},[951],{"type":47,"value":839},{"type":41,"tag":156,"props":953,"children":954},{"style":196},[955],{"type":47,"value":820},{"type":41,"tag":156,"props":957,"children":959},{"class":158,"line":958},9,[960,964,968,972,976,980],{"type":41,"tag":156,"props":961,"children":962},{"style":169},[963],{"type":47,"value":852},{"type":41,"tag":156,"props":965,"children":966},{"style":169},[967],{"type":47,"value":617},{"type":41,"tag":156,"props":969,"children":970},{"style":169},[971],{"type":47,"value":622},{"type":41,"tag":156,"props":973,"children":974},{"style":169},[975],{"type":47,"value":865},{"type":41,"tag":156,"props":977,"children":978},{"style":169},[979],{"type":47,"value":870},{"type":41,"tag":156,"props":981,"children":982},{"style":169},[983],{"type":47,"value":875},{"type":41,"tag":50,"props":985,"children":986},{},[987,989,995,997,1003,1004,1010,1011,1017,1018,1024,1025,1031],{"type":47,"value":988},"If ",{"type":41,"tag":83,"props":990,"children":992},{"className":991},[],[993],{"type":47,"value":994},"groupByLabels",{"type":47,"value":996}," is empty, try common dimensions: ",{"type":41,"tag":83,"props":998,"children":1000},{"className":999},[],[1001],{"type":47,"value":1002},"cluster",{"type":47,"value":253},{"type":41,"tag":83,"props":1005,"children":1007},{"className":1006},[],[1008],{"type":47,"value":1009},"namespace",{"type":47,"value":253},{"type":41,"tag":83,"props":1012,"children":1014},{"className":1013},[],[1015],{"type":47,"value":1016},"service",{"type":47,"value":253},{"type":41,"tag":83,"props":1019,"children":1021},{"className":1020},[],[1022],{"type":47,"value":1023},"status_code",{"type":47,"value":253},{"type":41,"tag":83,"props":1026,"children":1028},{"className":1027},[],[1029],{"type":47,"value":1030},"endpoint",{"type":47,"value":1032},".",{"type":41,"tag":50,"props":1034,"children":1035},{},[1036,1041,1043,1049],{"type":41,"tag":462,"props":1037,"children":1038},{},[1039],{"type":47,"value":1040},"For freeform queries",{"type":47,"value":1042}," — use the raw PromQL expression and add ",{"type":41,"tag":83,"props":1044,"children":1046},{"className":1045},[],[1047],{"type":47,"value":1048},"by (\u003Clabel>)",{"type":47,"value":1050}," grouping:",{"type":41,"tag":145,"props":1052,"children":1054},{"className":147,"code":1053,"language":149,"meta":150,"style":150},"gcx metrics query -d \u003Cdatasource-uid> \\\n  '\u003Cfreeform_expression> by (cluster)' \\\n  --from now-1h --to now --step 1m\n\n# Also try other likely breakdown dimensions\ngcx metrics query -d \u003Cdatasource-uid> \\\n  '\u003Cfreeform_expression> by (namespace)' \\\n  --from now-1h --to now --step 1m\n",[1055],{"type":41,"tag":83,"props":1056,"children":1057},{"__ignoreMap":150},[1058,1097,1117,1144,1151,1159,1198,1218],{"type":41,"tag":156,"props":1059,"children":1060},{"class":158,"line":159},[1061,1065,1069,1073,1077,1081,1085,1089,1093],{"type":41,"tag":156,"props":1062,"children":1063},{"style":163},[1064],{"type":47,"value":166},{"type":41,"tag":156,"props":1066,"children":1067},{"style":169},[1068],{"type":47,"value":787},{"type":41,"tag":156,"props":1070,"children":1071},{"style":169},[1072],{"type":47,"value":792},{"type":41,"tag":156,"props":1074,"children":1075},{"style":169},[1076],{"type":47,"value":797},{"type":41,"tag":156,"props":1078,"children":1079},{"style":185},[1080],{"type":47,"value":188},{"type":41,"tag":156,"props":1082,"children":1083},{"style":169},[1084],{"type":47,"value":806},{"type":41,"tag":156,"props":1086,"children":1087},{"style":196},[1088],{"type":47,"value":811},{"type":41,"tag":156,"props":1090,"children":1091},{"style":185},[1092],{"type":47,"value":204},{"type":41,"tag":156,"props":1094,"children":1095},{"style":196},[1096],{"type":47,"value":820},{"type":41,"tag":156,"props":1098,"children":1099},{"class":158,"line":777},[1100,1104,1109,1113],{"type":41,"tag":156,"props":1101,"children":1102},{"style":185},[1103],{"type":47,"value":829},{"type":41,"tag":156,"props":1105,"children":1106},{"style":169},[1107],{"type":47,"value":1108},"\u003Cfreeform_expression> by (cluster)",{"type":41,"tag":156,"props":1110,"children":1111},{"style":185},[1112],{"type":47,"value":839},{"type":41,"tag":156,"props":1114,"children":1115},{"style":196},[1116],{"type":47,"value":820},{"type":41,"tag":156,"props":1118,"children":1119},{"class":158,"line":823},[1120,1124,1128,1132,1136,1140],{"type":41,"tag":156,"props":1121,"children":1122},{"style":169},[1123],{"type":47,"value":852},{"type":41,"tag":156,"props":1125,"children":1126},{"style":169},[1127],{"type":47,"value":617},{"type":41,"tag":156,"props":1129,"children":1130},{"style":169},[1131],{"type":47,"value":622},{"type":41,"tag":156,"props":1133,"children":1134},{"style":169},[1135],{"type":47,"value":865},{"type":41,"tag":156,"props":1137,"children":1138},{"style":169},[1139],{"type":47,"value":870},{"type":41,"tag":156,"props":1141,"children":1142},{"style":169},[1143],{"type":47,"value":875},{"type":41,"tag":156,"props":1145,"children":1146},{"class":158,"line":846},[1147],{"type":41,"tag":156,"props":1148,"children":1149},{"emptyLinePlaceholder":882},[1150],{"type":47,"value":885},{"type":41,"tag":156,"props":1152,"children":1153},{"class":158,"line":878},[1154],{"type":41,"tag":156,"props":1155,"children":1156},{"style":771},[1157],{"type":47,"value":1158},"# Also try other likely breakdown dimensions\n",{"type":41,"tag":156,"props":1160,"children":1161},{"class":158,"line":888},[1162,1166,1170,1174,1178,1182,1186,1190,1194],{"type":41,"tag":156,"props":1163,"children":1164},{"style":163},[1165],{"type":47,"value":166},{"type":41,"tag":156,"props":1167,"children":1168},{"style":169},[1169],{"type":47,"value":787},{"type":41,"tag":156,"props":1171,"children":1172},{"style":169},[1173],{"type":47,"value":792},{"type":41,"tag":156,"props":1175,"children":1176},{"style":169},[1177],{"type":47,"value":797},{"type":41,"tag":156,"props":1179,"children":1180},{"style":185},[1181],{"type":47,"value":188},{"type":41,"tag":156,"props":1183,"children":1184},{"style":169},[1185],{"type":47,"value":806},{"type":41,"tag":156,"props":1187,"children":1188},{"style":196},[1189],{"type":47,"value":811},{"type":41,"tag":156,"props":1191,"children":1192},{"style":185},[1193],{"type":47,"value":204},{"type":41,"tag":156,"props":1195,"children":1196},{"style":196},[1197],{"type":47,"value":820},{"type":41,"tag":156,"props":1199,"children":1200},{"class":158,"line":897},[1201,1205,1210,1214],{"type":41,"tag":156,"props":1202,"children":1203},{"style":185},[1204],{"type":47,"value":829},{"type":41,"tag":156,"props":1206,"children":1207},{"style":169},[1208],{"type":47,"value":1209},"\u003Cfreeform_expression> by (namespace)",{"type":41,"tag":156,"props":1211,"children":1212},{"style":185},[1213],{"type":47,"value":839},{"type":41,"tag":156,"props":1215,"children":1216},{"style":196},[1217],{"type":47,"value":820},{"type":41,"tag":156,"props":1219,"children":1220},{"class":158,"line":937},[1221,1225,1229,1233,1237,1241],{"type":41,"tag":156,"props":1222,"children":1223},{"style":169},[1224],{"type":47,"value":852},{"type":41,"tag":156,"props":1226,"children":1227},{"style":169},[1228],{"type":47,"value":617},{"type":41,"tag":156,"props":1230,"children":1231},{"style":169},[1232],{"type":47,"value":622},{"type":41,"tag":156,"props":1234,"children":1235},{"style":169},[1236],{"type":47,"value":865},{"type":41,"tag":156,"props":1238,"children":1239},{"style":169},[1240],{"type":47,"value":870},{"type":41,"tag":156,"props":1242,"children":1243},{"style":169},[1244],{"type":47,"value":875},{"type":41,"tag":50,"props":1246,"children":1247},{},[1248,1250,1255],{"type":47,"value":1249},"Use graph output to display dimensional trends visually. Use ",{"type":41,"tag":83,"props":1251,"children":1253},{"className":1252},[],[1254],{"type":47,"value":88},{"type":47,"value":1256}," to extract exact values for the report.",{"type":41,"tag":50,"props":1258,"children":1259},{},[1260,1262,1268,1270,1276],{"type":47,"value":1261},"To query the SLO's own recording rule metrics instead (",{"type":41,"tag":83,"props":1263,"children":1265},{"className":1264},[],[1266],{"type":47,"value":1267},"grafana_slo_sli_window",{"type":47,"value":1269},", burn rate, error budget expressions), see ",{"type":41,"tag":1271,"props":1272,"children":1274},"a",{"href":1273},"references\u002Fslo-promql-patterns.md",[1275],{"type":47,"value":1273},{"type":47,"value":1277}," for the metric inventory and ready-made PromQL patterns.",{"type":41,"tag":138,"props":1279,"children":1281},{"id":1280},"step-5-search-for-related-alert-rules",[1282],{"type":47,"value":1283},"Step 5: Search for Related Alert Rules",{"type":41,"tag":145,"props":1285,"children":1287},{"className":147,"code":1286,"language":149,"meta":150,"style":150},"gcx alert rules list -o json | jq '[.[] | .rules[]? | select(.name | test(\"\u003Cslo-name>\"; \"i\"))]'\n",[1288],{"type":41,"tag":83,"props":1289,"children":1290},{"__ignoreMap":150},[1291],{"type":41,"tag":156,"props":1292,"children":1293},{"class":158,"line":159},[1294,1298,1303,1308,1312,1316,1321,1326,1331,1336,1341],{"type":41,"tag":156,"props":1295,"children":1296},{"style":163},[1297],{"type":47,"value":166},{"type":41,"tag":156,"props":1299,"children":1300},{"style":169},[1301],{"type":47,"value":1302}," alert",{"type":41,"tag":156,"props":1304,"children":1305},{"style":169},[1306],{"type":47,"value":1307}," rules",{"type":41,"tag":156,"props":1309,"children":1310},{"style":169},[1311],{"type":47,"value":738},{"type":41,"tag":156,"props":1313,"children":1314},{"style":169},[1315],{"type":47,"value":209},{"type":41,"tag":156,"props":1317,"children":1318},{"style":169},[1319],{"type":47,"value":1320}," json",{"type":41,"tag":156,"props":1322,"children":1323},{"style":185},[1324],{"type":47,"value":1325}," |",{"type":41,"tag":156,"props":1327,"children":1328},{"style":163},[1329],{"type":47,"value":1330}," jq",{"type":41,"tag":156,"props":1332,"children":1333},{"style":185},[1334],{"type":47,"value":1335}," '",{"type":41,"tag":156,"props":1337,"children":1338},{"style":169},[1339],{"type":47,"value":1340},"[.[] | .rules[]? | select(.name | test(\"\u003Cslo-name>\"; \"i\"))]",{"type":41,"tag":156,"props":1342,"children":1343},{"style":185},[1344],{"type":47,"value":1345},"'\n",{"type":41,"tag":50,"props":1347,"children":1348},{},[1349],{"type":47,"value":1350},"Also try searching by UUID fragment if the name-based search returns no results:",{"type":41,"tag":145,"props":1352,"children":1354},{"className":147,"code":1353,"language":149,"meta":150,"style":150},"gcx alert rules list -o json | jq '[.[] | .rules[]? | select(.labels.grafana_slo_uuid == \"\u003CUUID>\" or (.name | test(\"\u003Cslo-name>\"; \"i\")))]'\n",[1355],{"type":41,"tag":83,"props":1356,"children":1357},{"__ignoreMap":150},[1358],{"type":41,"tag":156,"props":1359,"children":1360},{"class":158,"line":159},[1361,1365,1369,1373,1377,1381,1385,1389,1393,1397,1402],{"type":41,"tag":156,"props":1362,"children":1363},{"style":163},[1364],{"type":47,"value":166},{"type":41,"tag":156,"props":1366,"children":1367},{"style":169},[1368],{"type":47,"value":1302},{"type":41,"tag":156,"props":1370,"children":1371},{"style":169},[1372],{"type":47,"value":1307},{"type":41,"tag":156,"props":1374,"children":1375},{"style":169},[1376],{"type":47,"value":738},{"type":41,"tag":156,"props":1378,"children":1379},{"style":169},[1380],{"type":47,"value":209},{"type":41,"tag":156,"props":1382,"children":1383},{"style":169},[1384],{"type":47,"value":1320},{"type":41,"tag":156,"props":1386,"children":1387},{"style":185},[1388],{"type":47,"value":1325},{"type":41,"tag":156,"props":1390,"children":1391},{"style":163},[1392],{"type":47,"value":1330},{"type":41,"tag":156,"props":1394,"children":1395},{"style":185},[1396],{"type":47,"value":1335},{"type":41,"tag":156,"props":1398,"children":1399},{"style":169},[1400],{"type":47,"value":1401},"[.[] | .rules[]? | select(.labels.grafana_slo_uuid == \"\u003CUUID>\" or (.name | test(\"\u003Cslo-name>\"; \"i\")))]",{"type":41,"tag":156,"props":1403,"children":1404},{"style":185},[1405],{"type":47,"value":1345},{"type":41,"tag":50,"props":1407,"children":1408},{},[1409],{"type":47,"value":1410},"Extract for each matching rule: name, state (firing\u002Fpending\u002Finactive), labels, and annotations.",{"type":41,"tag":138,"props":1412,"children":1414},{"id":1413},"step-6-extract-runbook-and-dashboard-urls",[1415],{"type":47,"value":1416},"Step 6: Extract Runbook and Dashboard URLs",{"type":41,"tag":50,"props":1418,"children":1419},{},[1420],{"type":47,"value":1421},"Collect URLs from:",{"type":41,"tag":221,"props":1423,"children":1424},{},[1425,1434,1443,1452,1461],{"type":41,"tag":67,"props":1426,"children":1427},{},[1428],{"type":41,"tag":83,"props":1429,"children":1431},{"className":1430},[],[1432],{"type":47,"value":1433},".spec.alerting.fastBurn.annotations.runbook_url",{"type":41,"tag":67,"props":1435,"children":1436},{},[1437],{"type":41,"tag":83,"props":1438,"children":1440},{"className":1439},[],[1441],{"type":47,"value":1442},".spec.alerting.fastBurn.annotations.dashboard_url",{"type":41,"tag":67,"props":1444,"children":1445},{},[1446],{"type":41,"tag":83,"props":1447,"children":1449},{"className":1448},[],[1450],{"type":47,"value":1451},".spec.alerting.slowBurn.annotations.runbook_url",{"type":41,"tag":67,"props":1453,"children":1454},{},[1455],{"type":41,"tag":83,"props":1456,"children":1458},{"className":1457},[],[1459],{"type":47,"value":1460},".spec.alerting.slowBurn.annotations.dashboard_url",{"type":41,"tag":67,"props":1462,"children":1463},{},[1464],{"type":41,"tag":83,"props":1465,"children":1467},{"className":1466},[],[1468],{"type":47,"value":1469},".metadata.annotations.*",{"type":41,"tag":50,"props":1471,"children":1472},{},[1473,1475,1481],{"type":47,"value":1474},"If a GitHub URL is found in runbook annotations and ",{"type":41,"tag":83,"props":1476,"children":1478},{"className":1477},[],[1479],{"type":47,"value":1480},"gh",{"type":47,"value":1482}," is available:",{"type":41,"tag":145,"props":1484,"children":1486},{"className":147,"code":1485,"language":149,"meta":150,"style":150},"# Convert GitHub web URL to API path and fetch content\ngh api \u002Frepos\u002F\u003Cowner>\u002F\u003Crepo>\u002Fcontents\u002F\u003Cpath> --jq '.content' | base64 --decode\n",[1487],{"type":41,"tag":83,"props":1488,"children":1489},{"__ignoreMap":150},[1490,1498],{"type":41,"tag":156,"props":1491,"children":1492},{"class":158,"line":159},[1493],{"type":41,"tag":156,"props":1494,"children":1495},{"style":771},[1496],{"type":47,"value":1497},"# Convert GitHub web URL to API path and fetch content\n",{"type":41,"tag":156,"props":1499,"children":1500},{"class":158,"line":777},[1501,1505,1510,1515,1520,1525,1530,1534,1538,1542,1547,1552,1556,1561,1565,1570,1575,1579,1584,1588,1593,1597,1601,1606],{"type":41,"tag":156,"props":1502,"children":1503},{"style":163},[1504],{"type":47,"value":1480},{"type":41,"tag":156,"props":1506,"children":1507},{"style":169},[1508],{"type":47,"value":1509}," api",{"type":41,"tag":156,"props":1511,"children":1512},{"style":169},[1513],{"type":47,"value":1514}," \u002Frepos\u002F",{"type":41,"tag":156,"props":1516,"children":1517},{"style":185},[1518],{"type":47,"value":1519},"\u003C",{"type":41,"tag":156,"props":1521,"children":1522},{"style":169},[1523],{"type":47,"value":1524},"owne",{"type":41,"tag":156,"props":1526,"children":1527},{"style":196},[1528],{"type":47,"value":1529},"r",{"type":41,"tag":156,"props":1531,"children":1532},{"style":185},[1533],{"type":47,"value":204},{"type":41,"tag":156,"props":1535,"children":1536},{"style":169},[1537],{"type":47,"value":107},{"type":41,"tag":156,"props":1539,"children":1540},{"style":185},[1541],{"type":47,"value":1519},{"type":41,"tag":156,"props":1543,"children":1544},{"style":169},[1545],{"type":47,"value":1546},"rep",{"type":41,"tag":156,"props":1548,"children":1549},{"style":196},[1550],{"type":47,"value":1551},"o",{"type":41,"tag":156,"props":1553,"children":1554},{"style":185},[1555],{"type":47,"value":204},{"type":41,"tag":156,"props":1557,"children":1558},{"style":169},[1559],{"type":47,"value":1560},"\u002Fcontents\u002F",{"type":41,"tag":156,"props":1562,"children":1563},{"style":185},[1564],{"type":47,"value":1519},{"type":41,"tag":156,"props":1566,"children":1567},{"style":169},[1568],{"type":47,"value":1569},"pat",{"type":41,"tag":156,"props":1571,"children":1572},{"style":196},[1573],{"type":47,"value":1574},"h",{"type":41,"tag":156,"props":1576,"children":1577},{"style":185},[1578],{"type":47,"value":204},{"type":41,"tag":156,"props":1580,"children":1581},{"style":169},[1582],{"type":47,"value":1583}," --jq",{"type":41,"tag":156,"props":1585,"children":1586},{"style":185},[1587],{"type":47,"value":1335},{"type":41,"tag":156,"props":1589,"children":1590},{"style":169},[1591],{"type":47,"value":1592},".content",{"type":41,"tag":156,"props":1594,"children":1595},{"style":185},[1596],{"type":47,"value":839},{"type":41,"tag":156,"props":1598,"children":1599},{"style":185},[1600],{"type":47,"value":1325},{"type":41,"tag":156,"props":1602,"children":1603},{"style":163},[1604],{"type":47,"value":1605}," base64",{"type":41,"tag":156,"props":1607,"children":1608},{"style":169},[1609],{"type":47,"value":1610}," --decode\n",{"type":41,"tag":50,"props":1612,"children":1613},{},[1614,1616,1622,1624,1630],{"type":47,"value":1615},"For raw GitHub URLs (",{"type":41,"tag":83,"props":1617,"children":1619},{"className":1618},[],[1620],{"type":47,"value":1621},"raw.githubusercontent.com",{"type":47,"value":1623},"), extract the content URL pattern and use ",{"type":41,"tag":83,"props":1625,"children":1627},{"className":1626},[],[1628],{"type":47,"value":1629},"gh api",{"type":47,"value":1631}," with the equivalent API endpoint.",{"type":41,"tag":56,"props":1633,"children":1635},{"id":1634},"output-format",[1636],{"type":47,"value":1637},"Output Format",{"type":41,"tag":50,"props":1639,"children":1640},{},[1641],{"type":47,"value":1642},"After completing the investigation, present results in this structure:",{"type":41,"tag":145,"props":1644,"children":1647},{"className":1645,"code":1646,"language":47},[472],"SLO: \u003Cname>\nTarget: \u003Cobjective>% over \u003Cwindow>  |  Status: BREACHING\nSLI: \u003Ccurrent>%  |  Error budget remaining: \u003Cbudget>%  |  Burn rate: \u003Crate>x\n1h SLI: \u003Csli_1h>%  |  1d SLI: \u003Csli_1d>%\n\n[Timeline graph — show default output]\n\nDimensional Breakdown:\n  Worst dimension: \u003Clabel>=\u003Cvalue> at \u003Cerror_rate>% error rate\n  [Additional dimensions ranked by error rate]\n\nRelated Alert Rules:\n  - \u003Crule_name>: \u003Cstate> [labels: \u003Ckey>=\u003Cvalue>]\n\nRunbook: \u003Curl>\nDashboard: \u003Curl>\n\n[If runbook fetched]: Key runbook steps:\n  \u003Crelevant excerpt>\n\nNext actions:\n1. \u003Cmost specific actionable step based on findings>\n2. \u003Cfollow-up investigation or escalation path>\n3. \u003Cif budget near zero: suggest slo-optimize for objective review>\n",[1648],{"type":41,"tag":83,"props":1649,"children":1650},{"__ignoreMap":150},[1651],{"type":47,"value":1646},{"type":41,"tag":56,"props":1653,"children":1655},{"id":1654},"error-handling",[1656],{"type":47,"value":1657},"Error Handling",{"type":41,"tag":221,"props":1659,"children":1660},{},[1661,1679,1689,1721,1738,1756,1773,1814],{"type":41,"tag":67,"props":1662,"children":1663},{},[1664,1669,1671,1677],{"type":41,"tag":462,"props":1665,"children":1666},{},[1667],{"type":47,"value":1668},"gcx slo definitions get fails with 404",{"type":47,"value":1670},": SLO UUID not found. Run ",{"type":41,"tag":83,"props":1672,"children":1674},{"className":1673},[],[1675],{"type":47,"value":1676},"gcx slo definitions list",{"type":47,"value":1678}," and confirm the UUID.",{"type":41,"tag":67,"props":1680,"children":1681},{},[1682,1687],{"type":41,"tag":462,"props":1683,"children":1684},{},[1685],{"type":47,"value":1686},"gcx slo definitions status returns empty",{"type":47,"value":1688},": No status available — SLO may be newly created. Check if recording rules are running (STATUS may show NODATA).",{"type":41,"tag":67,"props":1690,"children":1691},{},[1692,1697,1699,1705,1707,1712,1714,1719],{"type":41,"tag":462,"props":1693,"children":1694},{},[1695],{"type":47,"value":1696},"gcx metrics query fails with datasource error",{"type":47,"value":1698},": Datasource UID may be wrong. Run ",{"type":41,"tag":83,"props":1700,"children":1702},{"className":1701},[],[1703],{"type":47,"value":1704},"gcx datasources list --type prometheus",{"type":47,"value":1706}," to find the correct UID. If that list is empty (blank ",{"type":41,"tag":83,"props":1708,"children":1710},{"className":1709},[],[1711],{"type":47,"value":531},{"type":47,"value":1713}," fields in the payload), rerun without ",{"type":41,"tag":83,"props":1715,"children":1717},{"className":1716},[],[1718],{"type":47,"value":539},{"type":47,"value":1720}," and select by name.",{"type":41,"tag":67,"props":1722,"children":1723},{},[1724,1729,1731,1736],{"type":41,"tag":462,"props":1725,"children":1726},{},[1727],{"type":47,"value":1728},"gcx metrics query returns no data",{"type":47,"value":1730},": The SLO metrics may write to a separate datasource (check ",{"type":41,"tag":83,"props":1732,"children":1734},{"className":1733},[],[1735],{"type":47,"value":331},{"type":47,"value":1737},"). Try both the destination datasource and the default Prometheus datasource.",{"type":41,"tag":67,"props":1739,"children":1740},{},[1741,1746,1748,1754],{"type":41,"tag":462,"props":1742,"children":1743},{},[1744],{"type":47,"value":1745},"alert rules list returns empty",{"type":47,"value":1747},": Alert rules may be in a different folder. Try without filters: ",{"type":41,"tag":83,"props":1749,"children":1751},{"className":1750},[],[1752],{"type":47,"value":1753},"gcx alert rules list -o json | jq length",{"type":47,"value":1755}," to confirm total count.",{"type":41,"tag":67,"props":1757,"children":1758},{},[1759,1764,1766,1771],{"type":41,"tag":462,"props":1760,"children":1761},{},[1762],{"type":47,"value":1763},"gh api fails",{"type":47,"value":1765},": If ",{"type":41,"tag":83,"props":1767,"children":1769},{"className":1768},[],[1770],{"type":47,"value":1480},{"type":47,"value":1772}," is not authenticated or unavailable, report the runbook URL directly and skip content fetching.",{"type":41,"tag":67,"props":1774,"children":1775},{},[1776,1781,1783,1788,1789,1794,1795,1800,1801,1806,1807,1812],{"type":41,"tag":462,"props":1777,"children":1778},{},[1779],{"type":47,"value":1780},"SLO has no groupByLabels (ratio query)",{"type":47,"value":1782},": Try common breakdown dimensions: ",{"type":41,"tag":83,"props":1784,"children":1786},{"className":1785},[],[1787],{"type":47,"value":1002},{"type":47,"value":253},{"type":41,"tag":83,"props":1790,"children":1792},{"className":1791},[],[1793],{"type":47,"value":1009},{"type":47,"value":253},{"type":41,"tag":83,"props":1796,"children":1798},{"className":1797},[],[1799],{"type":47,"value":1016},{"type":47,"value":253},{"type":41,"tag":83,"props":1802,"children":1804},{"className":1803},[],[1805],{"type":47,"value":1030},{"type":47,"value":253},{"type":41,"tag":83,"props":1808,"children":1810},{"className":1809},[],[1811],{"type":47,"value":1023},{"type":47,"value":1813},". Report which ones return data.",{"type":41,"tag":67,"props":1815,"children":1816},{},[1817,1822],{"type":41,"tag":462,"props":1818,"children":1819},{},[1820],{"type":47,"value":1821},"Multiple SLOs with similar names",{"type":47,"value":1823},": When searching alert rules by name pattern, report all matches and their states — don't silently drop duplicates.",{"type":41,"tag":1825,"props":1826,"children":1827},"style",{},[1828],{"type":47,"value":1829},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":1831,"total":1932},[1832,1845,1861,1874,1889,1906,1919],{"slug":1833,"name":1833,"fn":1834,"description":1835,"org":1836,"tags":1837,"stars":23,"repoUrl":24,"updatedAt":1844},"agento11y","manage Grafana Agent Observability resources","Inspects and manages Grafana Agent Observability resources via gcx: conversations, generations, evaluators, rules, scores, and templates. Use when the user wants to list or search conversations, inspect generations, manage evaluators (upsert, test, delete), set up evaluation rules, check scores, or browse evaluator templates. Trigger on phrases like \"list conversations\", \"search generations\", \"what did the agent do\", \"debug LLM conversation\", \"create evaluator\", \"set up evaluation rule\", \"test evaluator\", \"check scores\", \"evaluate generation quality\", or \"set up online evaluation\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1838,1841,1842,1843],{"name":1839,"slug":1840,"type":15},"Agents","agents",{"name":9,"slug":8,"type":15},{"name":17,"slug":18,"type":15},{"name":13,"slug":14,"type":15},"2026-07-25T05:30:40.29622",{"slug":1846,"name":1846,"fn":1847,"description":1848,"org":1849,"tags":1850,"stars":23,"repoUrl":24,"updatedAt":1860},"agento11y-instrument","instrument LLM apps for agent observability","Sets up and instruments a developer's own LLM app or agent to send generations and agentic workflow to Grafana Agent Observability (the Agent Observability SDKs) — greenfield setup, fixing broken instrumentation, or filling gaps in existing instrumentation. Uses gcx for the parts a static prompt can't do: `gcx login` \u002F `gcx cloud stacks` to find the stack, and `gcx agento11y agents|conversations|generations` to VERIFY that data actually lands — so it iterates (instrument → run → verify → fix) until generations arrive, not blindly. Reads the app's code, detects language\u002Fframework, classifies instrumentation state (none \u002F partial \u002F broken), then runs a fixed gap checklist whose #1 item is the silent failure no other prompt catches: the SDK emits OTel spans\u002Fmetrics but never creates a TracerProvider\u002FMeterProvider, so without them all metrics go to a no-op and are lost. Also checks agent_version (required for per-version Performance charts), set_result completeness, SYNC vs STREAM, parent_generation_ids DAG links, and workflow-step coverage. Recommends changes citing file:line and, only with explicit confirmation, applies minimal diffs that don't change app behavior. Pulls SDK reference from agento11y's llms.txt rather than restating it, and hands off to `agento11y-test-starter` once data flows. It does NOT write test suites or set up tenant evaluations, rules, or guards — offline test suites are `agento11y-test-starter`, tenant eval rules + guards are `agento11y-prod-setup`; does NOT install coding-agent telemetry plugins (that is llms.txt \"Path A\"); does NOT mint or store credentials or invent endpoints. Trigger on phrases like \"instrument my app\", \"send my agent's traces to Grafana\", \"set up AI observability for my app\", \"my generations aren't showing up\", \"why is Performance empty\", \"add Agent Observability to my code\", \"fix my instrumentation\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1851,1852,1853,1856,1859],{"name":1839,"slug":1840,"type":15},{"name":9,"slug":8,"type":15},{"name":1854,"slug":1855,"type":15},"Instrumentation","instrumentation",{"name":1857,"slug":1858,"type":15},"LLM","llm",{"name":13,"slug":14,"type":15},"2026-07-31T05:53:52.580237",{"slug":1862,"name":1862,"fn":1863,"description":1864,"org":1865,"tags":1866,"stars":23,"repoUrl":24,"updatedAt":1873},"agento11y-prod-setup","setup production evaluation for AI agents","Sets up production evaluation and guardrails for a DEPLOYED AI agent in Grafana Agent Observability, grounded in the agent's own code and its real ingested traffic. The judgment layer on top of the `agento11y` skill: it reads the agent's source (system prompt, tools, entrypoint) AND samples its live traffic via gcx, checks what evaluators\u002Frules\u002Fguards already exist, then recommends only what's missing — online eval rules (score live conversations for regressions) and guards (warn-first request-path policies that redact \u002F tool-filter and may later be promoted to deny). It drafts reviewable YAML and, only with explicit confirmation, applies via `gcx agento11y`. New guards are drafted in warn mode (safe on live traffic — warn records but never blocks). It DOES create stack-level objects — that is the point — but every write is confirmed. It never rewrites or redeploys the agent. Trigger on phrases like \"set up production evaluation\", \"my agent is in prod what should I evaluate\", \"catch quality regressions\", \"add guardrails to my agent\", \"redact PII from my agent\", \"block dangerous tools\", \"set up online evals and guards\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1867,1868,1871,1872],{"name":1839,"slug":1840,"type":15},{"name":1869,"slug":1870,"type":15},"Evals","evals",{"name":9,"slug":8,"type":15},{"name":13,"slug":14,"type":15},"2026-07-31T05:53:53.576347",{"slug":1875,"name":1875,"fn":1876,"description":1877,"org":1878,"tags":1879,"stars":23,"repoUrl":24,"updatedAt":1888},"agento11y-test-starter","build and run agent test suites","Use early in an AI-agent project — before ship, before real traffic — to build a starter test suite for the agent and run it offline. Reads the agent's own code (system prompt, tools, task), writes a labeled draft suite of test cases (happy\u002Fedge\u002Fadversarial) grounded in real lines, and recommends how to score each case (the evaluators\u002Fjudges the offline runner uses). Assesses how runnable the agent is: for an easily-invoked agent it generates a runner stub (run_experiment.py) with two holes to fill and can optionally run it (only with permission, only against the endpoint the developer configured); for agents needing a harness or full runtime it points to the existing eval infra. It runs OFFLINE and never creates tenant-level evaluators, rules, or guards — that is `agento11y-prod-setup`, for a deployed agent with real traffic. Trigger on phrases like \"how do I test my agent before shipping\", \"write test cases for my agent\", \"set up tests for my agent\", \"check my agent before prod\", \"I have no traffic yet, how do I evaluate it\", \"test my agent offline\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1880,1881,1882,1885],{"name":1839,"slug":1840,"type":15},{"name":9,"slug":8,"type":15},{"name":1883,"slug":1884,"type":15},"QA","qa",{"name":1886,"slug":1887,"type":15},"Testing","testing","2026-07-31T05:53:51.62785",{"slug":1890,"name":1890,"fn":1891,"description":1892,"org":1893,"tags":1894,"stars":23,"repoUrl":24,"updatedAt":1905},"create-dashboard","create Grafana dashboards with gcx","Designs and creates Grafana dashboards with gcx, using `gcx dashboards snapshot` as a visual feedback loop. Use when the user wants to create a new Grafana dashboard, add panels, variables, or annotations to an existing dashboard, design dashboard panels, variables, queries, or layout, or make a material visual redesign. Triggers on \"create dashboard\", \"new dashboard\", \"build dashboard\", \"dashboard for \u003Cservice>\", \"add panels\", \"add variable\", \"add annotation\", \"improve this dashboard\", or \"iterate on a dashboard\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1895,1898,1901,1904],{"name":1896,"slug":1897,"type":15},"Dashboards","dashboards",{"name":1899,"slug":1900,"type":15},"Data Visualization","data-visualization",{"name":1902,"slug":1903,"type":15},"Design","design",{"name":9,"slug":8,"type":15},"2026-07-25T05:30:46.289717",{"slug":1907,"name":1907,"fn":1908,"description":1909,"org":1910,"tags":1911,"stars":23,"repoUrl":24,"updatedAt":1918},"debug-with-grafana","investigate application issues with Grafana","Structured workflow for investigating application problems with Grafana observability data (metrics, logs, traces) via gcx. Covers live firefighting AND retrospective incident analysis: incident triage, root-cause analysis, blast-radius checks (did an incident spill into other services), verifying whether a deployment or rollout triggered an incident, finding which service, endpoint, or path owns the most errors or slow requests, checking whether retries or queue backlogs piled up, and quantifying error or latency shares over a time window. Trigger on: \"my API is returning 500 errors\", \"latency is spiking\", \"investigate why requests are failing\", \"triage the incident\", \"blast radius\", \"root cause\", \"did the rollout cause it\", \"which endpoint owns the most 5xx\", \"did retries pile up\", or any request to analyse an earlier incident window using telemetry. For authoring dashboards use create-dashboard; for dashboard inventory use manage-dashboards.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1912,1913,1914,1917],{"name":21,"slug":22,"type":15},{"name":9,"slug":8,"type":15},{"name":1915,"slug":1916,"type":15},"Incident Response","incident-response",{"name":13,"slug":14,"type":15},"2026-07-18T05:11:10.445428",{"slug":1920,"name":1920,"fn":1921,"description":1922,"org":1923,"tags":1924,"stars":23,"repoUrl":24,"updatedAt":1931},"diagnose-entity-graph","diagnose Grafana Entity Graph issues","Diagnose Entity Graph problems: missing entities, missing edges, disconnected clusters, or filtering issues. Use when the user reports that Entity Graph doesn't look right, services are missing, edges aren't appearing, or environments can't be filtered. Triggers for: \"entity graph is empty\", \"services missing from entity graph\", \"no edges in entity graph\", \"disconnected services\", \"can't filter entity graph\", \"entity graph not working\", \"diagnose entity graph\", \"debug knowledge graph\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1925,1926,1927,1930],{"name":21,"slug":22,"type":15},{"name":9,"slug":8,"type":15},{"name":1928,"slug":1929,"type":15},"Graph Analysis","graph-analysis",{"name":13,"slug":14,"type":15},"2026-07-25T05:30:39.380934",24,{"items":1934,"total":2067},[1935,1952,1971,1991,1998,2006,2013,2020,2027,2034,2041,2055],{"slug":1936,"name":1936,"fn":1937,"description":1938,"org":1939,"tags":1940,"stars":1949,"repoUrl":1950,"updatedAt":1951},"faro-setup-web","instrument web apps with Grafana Faro","Instruments a web app with Grafana Faro Web SDK for frontend observability. Use when setting up error tracking, Web Vitals, session monitoring, or distributed tracing in a browser app.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1941,1944,1947,1948],{"name":1942,"slug":1943,"type":15},"Distributed Tracing","distributed-tracing",{"name":1945,"slug":1946,"type":15},"Frontend","frontend",{"name":17,"slug":18,"type":15},{"name":13,"slug":14,"type":15},1103,"https:\u002F\u002Fgithub.com\u002Fgrafana\u002Ffaro-web-sdk","2026-07-12T07:43:24.63314",{"slug":1953,"name":1953,"fn":1954,"description":1955,"org":1956,"tags":1957,"stars":1968,"repoUrl":1969,"updatedAt":1970},"configuring-yesoreyeram-infinity-datasource","configure Grafana Infinity data source","Configure the Infinity data source — base URL and allowed hosts, the authentication methods (basic, bearer token, API key, digest, OAuth passthrough, OAuth 2.0 client credentials\u002FJWT, Azure, Azure Blob, AWS), TLS, custom HTTP headers, network and security settings, the custom health check, and provisioning with a config file. Use when a user asks how to set up, configure, or change settings for the Infinity data source; how to authenticate to an API; how to allow hosts; how to provision it as YAML; or how to troubleshoot connection, authentication, or health-check issues.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1958,1961,1964,1967],{"name":1959,"slug":1960,"type":15},"API Development","api-development",{"name":1962,"slug":1963,"type":15},"Authentication","authentication",{"name":1965,"slug":1966,"type":15},"Configuration","configuration",{"name":9,"slug":8,"type":15},1056,"https:\u002F\u002Fgithub.com\u002Fgrafana\u002Fgrafana-infinity-datasource","2026-07-12T07:43:25.939136",{"slug":1972,"name":1972,"fn":1973,"description":1974,"org":1975,"tags":1976,"stars":1968,"repoUrl":1969,"updatedAt":1990},"querying-yesoreyeram-infinity-datasource","query data with Infinity datasource","Build queries with the Infinity data source — the query types (JSON, CSV, TSV, XML, GraphQL, HTML, UQL, GROQ, Google Sheets, Series, Transformations), the parsers (Frontend\u002Fsimple, Backend JSONata, JQ, UQL, GROQ) and which support alerting, the sources (URL, Inline, Reference, Azure Blob, Random walk), output formats (table, timeseries, logs, trace, node graph, dataframe), root selector and columns, computed columns\u002Ffilters\u002Fsummarize, pagination, and template variables. Use when a user asks how to query Infinity; how to fetch JSON\u002FCSV\u002FXML\u002FGraphQL\u002FHTML from a URL or inline; how to select rows and columns; how to transform data with UQL\u002FGROQ\u002FJSONata\u002FJQ; how to make a query work with alerting; how to paginate; or how to use variables in a query.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1977,1980,1983,1984,1987],{"name":1978,"slug":1979,"type":15},"CSV","csv",{"name":1981,"slug":1982,"type":15},"Data Analysis","data-analysis",{"name":9,"slug":8,"type":15},{"name":1985,"slug":1986,"type":15},"GraphQL","graphql",{"name":1988,"slug":1989,"type":15},"JSON","json","2026-07-15T05:34:05.773947",{"slug":1833,"name":1833,"fn":1834,"description":1835,"org":1992,"tags":1993,"stars":23,"repoUrl":24,"updatedAt":1844},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1994,1995,1996,1997],{"name":1839,"slug":1840,"type":15},{"name":9,"slug":8,"type":15},{"name":17,"slug":18,"type":15},{"name":13,"slug":14,"type":15},{"slug":1846,"name":1846,"fn":1847,"description":1848,"org":1999,"tags":2000,"stars":23,"repoUrl":24,"updatedAt":1860},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2001,2002,2003,2004,2005],{"name":1839,"slug":1840,"type":15},{"name":9,"slug":8,"type":15},{"name":1854,"slug":1855,"type":15},{"name":1857,"slug":1858,"type":15},{"name":13,"slug":14,"type":15},{"slug":1862,"name":1862,"fn":1863,"description":1864,"org":2007,"tags":2008,"stars":23,"repoUrl":24,"updatedAt":1873},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2009,2010,2011,2012],{"name":1839,"slug":1840,"type":15},{"name":1869,"slug":1870,"type":15},{"name":9,"slug":8,"type":15},{"name":13,"slug":14,"type":15},{"slug":1875,"name":1875,"fn":1876,"description":1877,"org":2014,"tags":2015,"stars":23,"repoUrl":24,"updatedAt":1888},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2016,2017,2018,2019],{"name":1839,"slug":1840,"type":15},{"name":9,"slug":8,"type":15},{"name":1883,"slug":1884,"type":15},{"name":1886,"slug":1887,"type":15},{"slug":1890,"name":1890,"fn":1891,"description":1892,"org":2021,"tags":2022,"stars":23,"repoUrl":24,"updatedAt":1905},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2023,2024,2025,2026],{"name":1896,"slug":1897,"type":15},{"name":1899,"slug":1900,"type":15},{"name":1902,"slug":1903,"type":15},{"name":9,"slug":8,"type":15},{"slug":1907,"name":1907,"fn":1908,"description":1909,"org":2028,"tags":2029,"stars":23,"repoUrl":24,"updatedAt":1918},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2030,2031,2032,2033],{"name":21,"slug":22,"type":15},{"name":9,"slug":8,"type":15},{"name":1915,"slug":1916,"type":15},{"name":13,"slug":14,"type":15},{"slug":1920,"name":1920,"fn":1921,"description":1922,"org":2035,"tags":2036,"stars":23,"repoUrl":24,"updatedAt":1931},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2037,2038,2039,2040],{"name":21,"slug":22,"type":15},{"name":9,"slug":8,"type":15},{"name":1928,"slug":1929,"type":15},{"name":13,"slug":14,"type":15},{"slug":166,"name":166,"fn":2042,"description":2043,"org":2044,"tags":2045,"stars":23,"repoUrl":24,"updatedAt":2054},"manage Grafana Cloud resources via gcx","Manages Grafana Cloud resources via the gcx CLI. Trigger when the user wants to inspect, create, update, delete, query, or automate any Grafana resource - dashboards, datasources, alerts, SLOs, synthetic checks, oncall, incidents, fleet, k6, knowledge graph, or adaptive telemetry.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2046,2049,2050,2051],{"name":2047,"slug":2048,"type":15},"CLI","cli",{"name":9,"slug":8,"type":15},{"name":17,"slug":18,"type":15},{"name":2052,"slug":2053,"type":15},"Operations","operations","2026-07-31T05:53:50.587304",{"slug":2056,"name":2056,"fn":2057,"description":2058,"org":2059,"tags":2060,"stars":23,"repoUrl":24,"updatedAt":2066},"gcx-demo","present gcx demo tours","Run a narrated, read-only demo tour of gcx for customer or colleague presentations. Showcases the breadth of gcx across every Grafana Cloud product area — resources, datasources, metrics, logs, traces, SLOs, alerts, synthetic monitoring, IRM, k6, fleet, and more. All commands are strictly read-only. Trigger when the user says \"demo gcx\", \"show off gcx\", \"customer demo\", \"gcx tour\", or \"\u002Fgcx-demo\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2061,2062,2063],{"name":2047,"slug":2048,"type":15},{"name":9,"slug":8,"type":15},{"name":2064,"slug":2065,"type":15},"Presentations","presentations","2026-07-25T05:30:45.282458",80]