[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-grafana-agento11y":3,"mdc--q0i9xp-key":34,"related-org-grafana-agento11y":1775,"related-repo-grafana-agento11y":1956},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":23,"repoUrl":24,"updatedAt":25,"license":26,"forks":27,"topics":28,"repo":29,"sourceUrl":32,"mdContent":33},"agento11y","manage Grafana Agent Observability resources","Inspects and manages Grafana Agent Observability resources via gcx: conversations, generations, evaluators, rules, scores, and templates. Use when the user wants to list or search conversations, inspect generations, manage evaluators (upsert, test, delete), set up evaluation rules, check scores, or browse evaluator templates. Trigger on phrases like \"list conversations\", \"search generations\", \"what did the agent do\", \"debug LLM conversation\", \"create evaluator\", \"set up evaluation rule\", \"test evaluator\", \"check scores\", \"evaluate generation quality\", or \"set up online evaluation\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"grafana","Grafana","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fgrafana.jpg",[12,16,19,22],{"name":13,"slug":14,"type":15},"Observability","observability","tag",{"name":17,"slug":18,"type":15},"Monitoring","monitoring",{"name":20,"slug":21,"type":15},"Agents","agents",{"name":9,"slug":8,"type":15},430,"https:\u002F\u002Fgithub.com\u002Fgrafana\u002Fgcx","2026-07-25T05:30:40.29622",null,29,[],{"repoUrl":24,"stars":23,"forks":27,"topics":30,"description":31},[],"A CLI for managing Grafana Cloud resources. Optimized for agentic usage.","https:\u002F\u002Fgithub.com\u002Fgrafana\u002Fgcx\u002Ftree\u002FHEAD\u002Fclaude-plugin\u002Fskills\u002Fagento11y","---\nname: agento11y\ndescription: >\n  Inspects and manages Grafana Agent Observability resources via gcx:\n  conversations, generations, evaluators, rules, scores, and templates.\n  Use when the user wants to list or search conversations, inspect generations,\n  manage evaluators (upsert, test, delete), set up evaluation rules, check scores,\n  or browse evaluator templates. Trigger on phrases like \"list conversations\",\n  \"search generations\", \"what did the agent do\", \"debug LLM conversation\",\n  \"create evaluator\", \"set up evaluation rule\", \"test evaluator\", \"check scores\",\n  \"evaluate generation quality\", or \"set up online evaluation\".\nallowed-tools: Bash, Read, Write, Edit\n---\n\n# Agent Observability\n\nAgent Observability records what LLM-powered applications do in production and scores the quality of their output.\n\nApplications send generations (individual LLM API calls — request, response, model, tokens, tool calls) to Agent Observability. Generations belonging to the same user session are grouped into a conversation.\n\nEvaluators are scoring functions (LLM judge, regex, heuristic, JSON schema, etc.) that assess generation quality. Rules bind evaluators to production traffic, they select which generations to evaluate (e.g. only user-visible turns), filter by agent\u002Fmodel, and control sampling rate. When a rule matches a generation, Agent Observability runs the bound evaluators and writes scores.\n\nAll commands live under `gcx agento11y`. Use `gcx agento11y \u003Csubcommand> --help` for flags and usage.\n\n## Command Groups\n\n| Group | Purpose |\n|-------|---------|\n| `conversations` | List, get, search conversations |\n| `generations` | Get a single generation, list its scores |\n| `agents` | List agents, get details, list version history (`list-versions`) |\n| `evaluators` | List, get, upsert, delete, test evaluators |\n| `rules` | List, get, create, update, delete evaluation rules |\n| `templates` | List, get built-in evaluator templates |\n| `judge` | List judge providers and models |\n| `experiments` | List, get, create, update, cancel runs; `list-scores` and `report` |\n\nDelete commands (`evaluators delete`, `rules delete`) require `--force` to skip confirmation in agent mode (there is no `-f` shorthand on delete). List first to confirm the target ID:\n\n```bash\ngcx agento11y evaluators list\ngcx agento11y evaluators delete \u003Cid> --force\n\ngcx agento11y rules list\ngcx agento11y rules delete \u003Cid> --force\n```\n\nDeleting an evaluator referenced by a rule may leave the rule pointing at a missing evaluator — check `gcx agento11y rules list` after.\n\n## Conversation Search\n\nDefaults to last 24 hours. Filter syntax: `key operator \"value\"`, space-separated.\n\n```bash\ngcx agento11y conversations search --filters 'agent = \"my-agent\" status = \"error\"'\ngcx agento11y conversations search --filters 'agent = \"my-agent\"' --from 2026-04-01T00:00:00Z --to 2026-04-14T00:00:00Z\n```\n\n**Filter keys:** `model`, `provider`, `agent`, `agent.version`, `status`, `error.type`, `error.category`, `duration`, `tool.name`, `operation`, `namespace`, `cluster`, `service`, `generation_count`, `eval.passed`, `eval.evaluator_id`, `eval.score_key`, `eval.score`\n\n**Operators:** `=`, `!=`, `>`, `\u003C`, `>=`, `\u003C=`, `=~` (regex)\n\n## Evaluator Kind Decision Table\n\n| User describes | Kind |\n|----------------|------|\n| \"check if response is helpful \u002F toxic \u002F grounded\" | `llm_judge` |\n| \"combined quality score with explanation\" | `llm_judge` |\n| \"validate JSON output format\" | `json_schema` |\n| \"check if response contains \u002F doesn't contain X\" | `regex` |\n| \"response must be non-empty and at least N chars\" | `heuristic` |\n| \"check multiple conditions (non-empty AND has greeting)\" | `heuristic` |\n\nCopy-paste definitions for each kind, with the constraints the API enforces for each, are in [references\u002Fevaluator-examples.md](references\u002Fevaluator-examples.md).\n\n## Input Format\n\n`gcx agento11y evaluators get -o yaml` and `gcx agento11y rules get -o yaml` emit K8s-style manifests (`apiVersion\u002Fkind\u002Fmetadata\u002Fspec`). `evaluators upsert -f`, `rules create -f`, and `rules update -f` expect top-level fields only. Do not round-trip get output into create\u002Fupdate.\n\nIDs (`evaluator_id`, `rule_id`) accept only letters, digits, `_`, and `.` — hyphens are rejected server-side. `version` is required on evaluator definitions — it versions the evaluator itself, separate from any schema version inside `config` — see [references\u002Fevaluator-examples.md](references\u002Fevaluator-examples.md) for full examples of every kind.\n\nRule definition:\n\n```yaml\nrule_id: my_rule\nenabled: true\nselector: user_visible_turn\nsample_rate: 1.0\nevaluator_ids:\n  - my_evaluator\nmatch:\n  agent_name:\n    - my-agent\n```\n\nThere is no `evaluators update` command; to change an evaluator, re-run `upsert` with the same `evaluator_id` and a new `version` (re-using an existing version is rejected with a 409).\n\n## Setting Up Online Evaluation\n\n1. Pick a template: `gcx agento11y templates list`, then `gcx agento11y templates get \u003Cid> -o yaml`. Template output includes `kind`, `config`, and `output_keys` — copy these into a new evaluator definition and add your own `evaluator_id`. Do not pass the template output directly to `evaluators upsert`.\n2. Write an evaluator YAML using the input format above, create: `gcx agento11y evaluators upsert -f evaluator.yaml`\n3. Test against a real generation: `gcx agento11y evaluators test -e \u003Cevaluator-id> -g \u003Cgeneration-id>`\n4. Iterate until the evaluator scores as expected\n5. Write a rule YAML (see [rule-templates.md](references\u002Frule-templates.md) for copy-paste templates), create: `gcx agento11y rules create -f rule.yaml`\n6. Verify: `gcx agento11y rules list`\n\n## Rule Selectors\n\n| Selector | What it evaluates |\n|----------|-------------------|\n| `user_visible_turn` | Final assistant generation visible to the user |\n| `all_assistant_generations` | Every assistant generation in the conversation |\n| `tool_call_steps` | Tool call generations |\n\n## Rule Match Keys\n\nAll values are arrays. Glob-capable keys support `*`, `?`, `[...]` patterns.\n\n| Key | Glob | Description |\n|-----|------|-------------|\n| `agent_name` | yes | Agent name |\n| `agent_version` | yes | Agent version string |\n| `operation_name` | yes | Operation name |\n| `model.provider` | yes | Model provider (e.g. `openai`, `anthropic`) |\n| `model.name` | yes | Model name (e.g. `gpt-4o`, `claude-sonnet-4-5-20250514`) |\n| `mode` | no | `SYNC` or `STREAM` |\n| `error.type` | no | Error type (also accepts `present`\u002F`absent`) |\n| `error.category` | no | Error category (also accepts `present`\u002F`absent`) |\n| `tags.\u003Ckey>` | no | Custom tag value (e.g. `tags.env`) |\n",{"data":35,"body":37},{"name":4,"description":6,"allowed-tools":36},"Bash, Read, Write, Edit",{"type":38,"children":39},"root",[40,49,55,60,65,87,94,279,316,474,487,493,506,611,747,805,811,933,946,952,1001,1057,1062,1209,1244,1250,1367,1373,1446,1452,1479,1769],{"type":41,"tag":42,"props":43,"children":45},"element","h1",{"id":44},"agent-observability",[46],{"type":47,"value":48},"text","Agent Observability",{"type":41,"tag":50,"props":51,"children":52},"p",{},[53],{"type":47,"value":54},"Agent Observability records what LLM-powered applications do in production and scores the quality of their output.",{"type":41,"tag":50,"props":56,"children":57},{},[58],{"type":47,"value":59},"Applications send generations (individual LLM API calls — request, response, model, tokens, tool calls) to Agent Observability. Generations belonging to the same user session are grouped into a conversation.",{"type":41,"tag":50,"props":61,"children":62},{},[63],{"type":47,"value":64},"Evaluators are scoring functions (LLM judge, regex, heuristic, JSON schema, etc.) that assess generation quality. Rules bind evaluators to production traffic, they select which generations to evaluate (e.g. only user-visible turns), filter by agent\u002Fmodel, and control sampling rate. When a rule matches a generation, Agent Observability runs the bound evaluators and writes scores.",{"type":41,"tag":50,"props":66,"children":67},{},[68,70,77,79,85],{"type":47,"value":69},"All commands live under ",{"type":41,"tag":71,"props":72,"children":74},"code",{"className":73},[],[75],{"type":47,"value":76},"gcx agento11y",{"type":47,"value":78},". Use ",{"type":41,"tag":71,"props":80,"children":82},{"className":81},[],[83],{"type":47,"value":84},"gcx agento11y \u003Csubcommand> --help",{"type":47,"value":86}," for flags and usage.",{"type":41,"tag":88,"props":89,"children":91},"h2",{"id":90},"command-groups",[92],{"type":47,"value":93},"Command Groups",{"type":41,"tag":95,"props":96,"children":97},"table",{},[98,117],{"type":41,"tag":99,"props":100,"children":101},"thead",{},[102],{"type":41,"tag":103,"props":104,"children":105},"tr",{},[106,112],{"type":41,"tag":107,"props":108,"children":109},"th",{},[110],{"type":47,"value":111},"Group",{"type":41,"tag":107,"props":113,"children":114},{},[115],{"type":47,"value":116},"Purpose",{"type":41,"tag":118,"props":119,"children":120},"tbody",{},[121,139,156,180,197,214,231,248],{"type":41,"tag":103,"props":122,"children":123},{},[124,134],{"type":41,"tag":125,"props":126,"children":127},"td",{},[128],{"type":41,"tag":71,"props":129,"children":131},{"className":130},[],[132],{"type":47,"value":133},"conversations",{"type":41,"tag":125,"props":135,"children":136},{},[137],{"type":47,"value":138},"List, get, search conversations",{"type":41,"tag":103,"props":140,"children":141},{},[142,151],{"type":41,"tag":125,"props":143,"children":144},{},[145],{"type":41,"tag":71,"props":146,"children":148},{"className":147},[],[149],{"type":47,"value":150},"generations",{"type":41,"tag":125,"props":152,"children":153},{},[154],{"type":47,"value":155},"Get a single generation, list its scores",{"type":41,"tag":103,"props":157,"children":158},{},[159,167],{"type":41,"tag":125,"props":160,"children":161},{},[162],{"type":41,"tag":71,"props":163,"children":165},{"className":164},[],[166],{"type":47,"value":21},{"type":41,"tag":125,"props":168,"children":169},{},[170,172,178],{"type":47,"value":171},"List agents, get details, list version history (",{"type":41,"tag":71,"props":173,"children":175},{"className":174},[],[176],{"type":47,"value":177},"list-versions",{"type":47,"value":179},")",{"type":41,"tag":103,"props":181,"children":182},{},[183,192],{"type":41,"tag":125,"props":184,"children":185},{},[186],{"type":41,"tag":71,"props":187,"children":189},{"className":188},[],[190],{"type":47,"value":191},"evaluators",{"type":41,"tag":125,"props":193,"children":194},{},[195],{"type":47,"value":196},"List, get, upsert, delete, test evaluators",{"type":41,"tag":103,"props":198,"children":199},{},[200,209],{"type":41,"tag":125,"props":201,"children":202},{},[203],{"type":41,"tag":71,"props":204,"children":206},{"className":205},[],[207],{"type":47,"value":208},"rules",{"type":41,"tag":125,"props":210,"children":211},{},[212],{"type":47,"value":213},"List, get, create, update, delete evaluation rules",{"type":41,"tag":103,"props":215,"children":216},{},[217,226],{"type":41,"tag":125,"props":218,"children":219},{},[220],{"type":41,"tag":71,"props":221,"children":223},{"className":222},[],[224],{"type":47,"value":225},"templates",{"type":41,"tag":125,"props":227,"children":228},{},[229],{"type":47,"value":230},"List, get built-in evaluator templates",{"type":41,"tag":103,"props":232,"children":233},{},[234,243],{"type":41,"tag":125,"props":235,"children":236},{},[237],{"type":41,"tag":71,"props":238,"children":240},{"className":239},[],[241],{"type":47,"value":242},"judge",{"type":41,"tag":125,"props":244,"children":245},{},[246],{"type":47,"value":247},"List judge providers and models",{"type":41,"tag":103,"props":249,"children":250},{},[251,260],{"type":41,"tag":125,"props":252,"children":253},{},[254],{"type":41,"tag":71,"props":255,"children":257},{"className":256},[],[258],{"type":47,"value":259},"experiments",{"type":41,"tag":125,"props":261,"children":262},{},[263,265,271,273],{"type":47,"value":264},"List, get, create, update, cancel runs; ",{"type":41,"tag":71,"props":266,"children":268},{"className":267},[],[269],{"type":47,"value":270},"list-scores",{"type":47,"value":272}," and ",{"type":41,"tag":71,"props":274,"children":276},{"className":275},[],[277],{"type":47,"value":278},"report",{"type":41,"tag":50,"props":280,"children":281},{},[282,284,290,292,298,300,306,308,314],{"type":47,"value":283},"Delete commands (",{"type":41,"tag":71,"props":285,"children":287},{"className":286},[],[288],{"type":47,"value":289},"evaluators delete",{"type":47,"value":291},", ",{"type":41,"tag":71,"props":293,"children":295},{"className":294},[],[296],{"type":47,"value":297},"rules delete",{"type":47,"value":299},") require ",{"type":41,"tag":71,"props":301,"children":303},{"className":302},[],[304],{"type":47,"value":305},"--force",{"type":47,"value":307}," to skip confirmation in agent mode (there is no ",{"type":41,"tag":71,"props":309,"children":311},{"className":310},[],[312],{"type":47,"value":313},"-f",{"type":47,"value":315}," shorthand on delete). List first to confirm the target ID:",{"type":41,"tag":317,"props":318,"children":323},"pre",{"className":319,"code":320,"language":321,"meta":322,"style":322},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","gcx agento11y evaluators list\ngcx agento11y evaluators delete \u003Cid> --force\n\ngcx agento11y rules list\ngcx agento11y rules delete \u003Cid> --force\n","bash","",[324],{"type":41,"tag":71,"props":325,"children":326},{"__ignoreMap":322},[327,355,403,413,434],{"type":41,"tag":328,"props":329,"children":332},"span",{"class":330,"line":331},"line",1,[333,339,345,350],{"type":41,"tag":328,"props":334,"children":336},{"style":335},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[337],{"type":47,"value":338},"gcx",{"type":41,"tag":328,"props":340,"children":342},{"style":341},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[343],{"type":47,"value":344}," agento11y",{"type":41,"tag":328,"props":346,"children":347},{"style":341},[348],{"type":47,"value":349}," evaluators",{"type":41,"tag":328,"props":351,"children":352},{"style":341},[353],{"type":47,"value":354}," list\n",{"type":41,"tag":328,"props":356,"children":358},{"class":330,"line":357},2,[359,363,367,371,376,382,387,393,398],{"type":41,"tag":328,"props":360,"children":361},{"style":335},[362],{"type":47,"value":338},{"type":41,"tag":328,"props":364,"children":365},{"style":341},[366],{"type":47,"value":344},{"type":41,"tag":328,"props":368,"children":369},{"style":341},[370],{"type":47,"value":349},{"type":41,"tag":328,"props":372,"children":373},{"style":341},[374],{"type":47,"value":375}," delete",{"type":41,"tag":328,"props":377,"children":379},{"style":378},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[380],{"type":47,"value":381}," \u003C",{"type":41,"tag":328,"props":383,"children":384},{"style":341},[385],{"type":47,"value":386},"i",{"type":41,"tag":328,"props":388,"children":390},{"style":389},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[391],{"type":47,"value":392},"d",{"type":41,"tag":328,"props":394,"children":395},{"style":378},[396],{"type":47,"value":397},">",{"type":41,"tag":328,"props":399,"children":400},{"style":341},[401],{"type":47,"value":402}," --force\n",{"type":41,"tag":328,"props":404,"children":406},{"class":330,"line":405},3,[407],{"type":41,"tag":328,"props":408,"children":410},{"emptyLinePlaceholder":409},true,[411],{"type":47,"value":412},"\n",{"type":41,"tag":328,"props":414,"children":416},{"class":330,"line":415},4,[417,421,425,430],{"type":41,"tag":328,"props":418,"children":419},{"style":335},[420],{"type":47,"value":338},{"type":41,"tag":328,"props":422,"children":423},{"style":341},[424],{"type":47,"value":344},{"type":41,"tag":328,"props":426,"children":427},{"style":341},[428],{"type":47,"value":429}," rules",{"type":41,"tag":328,"props":431,"children":432},{"style":341},[433],{"type":47,"value":354},{"type":41,"tag":328,"props":435,"children":437},{"class":330,"line":436},5,[438,442,446,450,454,458,462,466,470],{"type":41,"tag":328,"props":439,"children":440},{"style":335},[441],{"type":47,"value":338},{"type":41,"tag":328,"props":443,"children":444},{"style":341},[445],{"type":47,"value":344},{"type":41,"tag":328,"props":447,"children":448},{"style":341},[449],{"type":47,"value":429},{"type":41,"tag":328,"props":451,"children":452},{"style":341},[453],{"type":47,"value":375},{"type":41,"tag":328,"props":455,"children":456},{"style":378},[457],{"type":47,"value":381},{"type":41,"tag":328,"props":459,"children":460},{"style":341},[461],{"type":47,"value":386},{"type":41,"tag":328,"props":463,"children":464},{"style":389},[465],{"type":47,"value":392},{"type":41,"tag":328,"props":467,"children":468},{"style":378},[469],{"type":47,"value":397},{"type":41,"tag":328,"props":471,"children":472},{"style":341},[473],{"type":47,"value":402},{"type":41,"tag":50,"props":475,"children":476},{},[477,479,485],{"type":47,"value":478},"Deleting an evaluator referenced by a rule may leave the rule pointing at a missing evaluator — check ",{"type":41,"tag":71,"props":480,"children":482},{"className":481},[],[483],{"type":47,"value":484},"gcx agento11y rules list",{"type":47,"value":486}," after.",{"type":41,"tag":88,"props":488,"children":490},{"id":489},"conversation-search",[491],{"type":47,"value":492},"Conversation Search",{"type":41,"tag":50,"props":494,"children":495},{},[496,498,504],{"type":47,"value":497},"Defaults to last 24 hours. Filter syntax: ",{"type":41,"tag":71,"props":499,"children":501},{"className":500},[],[502],{"type":47,"value":503},"key operator \"value\"",{"type":47,"value":505},", space-separated.",{"type":41,"tag":317,"props":507,"children":509},{"className":319,"code":508,"language":321,"meta":322,"style":322},"gcx agento11y conversations search --filters 'agent = \"my-agent\" status = \"error\"'\ngcx agento11y conversations search --filters 'agent = \"my-agent\"' --from 2026-04-01T00:00:00Z --to 2026-04-14T00:00:00Z\n",[510],{"type":41,"tag":71,"props":511,"children":512},{"__ignoreMap":322},[513,554],{"type":41,"tag":328,"props":514,"children":515},{"class":330,"line":331},[516,520,524,529,534,539,544,549],{"type":41,"tag":328,"props":517,"children":518},{"style":335},[519],{"type":47,"value":338},{"type":41,"tag":328,"props":521,"children":522},{"style":341},[523],{"type":47,"value":344},{"type":41,"tag":328,"props":525,"children":526},{"style":341},[527],{"type":47,"value":528}," conversations",{"type":41,"tag":328,"props":530,"children":531},{"style":341},[532],{"type":47,"value":533}," search",{"type":41,"tag":328,"props":535,"children":536},{"style":341},[537],{"type":47,"value":538}," --filters",{"type":41,"tag":328,"props":540,"children":541},{"style":378},[542],{"type":47,"value":543}," '",{"type":41,"tag":328,"props":545,"children":546},{"style":341},[547],{"type":47,"value":548},"agent = \"my-agent\" status = \"error\"",{"type":41,"tag":328,"props":550,"children":551},{"style":378},[552],{"type":47,"value":553},"'\n",{"type":41,"tag":328,"props":555,"children":556},{"class":330,"line":357},[557,561,565,569,573,577,581,586,591,596,601,606],{"type":41,"tag":328,"props":558,"children":559},{"style":335},[560],{"type":47,"value":338},{"type":41,"tag":328,"props":562,"children":563},{"style":341},[564],{"type":47,"value":344},{"type":41,"tag":328,"props":566,"children":567},{"style":341},[568],{"type":47,"value":528},{"type":41,"tag":328,"props":570,"children":571},{"style":341},[572],{"type":47,"value":533},{"type":41,"tag":328,"props":574,"children":575},{"style":341},[576],{"type":47,"value":538},{"type":41,"tag":328,"props":578,"children":579},{"style":378},[580],{"type":47,"value":543},{"type":41,"tag":328,"props":582,"children":583},{"style":341},[584],{"type":47,"value":585},"agent = \"my-agent\"",{"type":41,"tag":328,"props":587,"children":588},{"style":378},[589],{"type":47,"value":590},"'",{"type":41,"tag":328,"props":592,"children":593},{"style":341},[594],{"type":47,"value":595}," --from",{"type":41,"tag":328,"props":597,"children":598},{"style":341},[599],{"type":47,"value":600}," 2026-04-01T00:00:00Z",{"type":41,"tag":328,"props":602,"children":603},{"style":341},[604],{"type":47,"value":605}," --to",{"type":41,"tag":328,"props":607,"children":608},{"style":341},[609],{"type":47,"value":610}," 2026-04-14T00:00:00Z\n",{"type":41,"tag":50,"props":612,"children":613},{},[614,620,622,628,629,635,636,642,643,649,650,656,657,663,664,670,671,677,678,684,685,691,692,698,699,705,706,712,713,719,720,726,727,733,734,740,741],{"type":41,"tag":615,"props":616,"children":617},"strong",{},[618],{"type":47,"value":619},"Filter keys:",{"type":47,"value":621}," ",{"type":41,"tag":71,"props":623,"children":625},{"className":624},[],[626],{"type":47,"value":627},"model",{"type":47,"value":291},{"type":41,"tag":71,"props":630,"children":632},{"className":631},[],[633],{"type":47,"value":634},"provider",{"type":47,"value":291},{"type":41,"tag":71,"props":637,"children":639},{"className":638},[],[640],{"type":47,"value":641},"agent",{"type":47,"value":291},{"type":41,"tag":71,"props":644,"children":646},{"className":645},[],[647],{"type":47,"value":648},"agent.version",{"type":47,"value":291},{"type":41,"tag":71,"props":651,"children":653},{"className":652},[],[654],{"type":47,"value":655},"status",{"type":47,"value":291},{"type":41,"tag":71,"props":658,"children":660},{"className":659},[],[661],{"type":47,"value":662},"error.type",{"type":47,"value":291},{"type":41,"tag":71,"props":665,"children":667},{"className":666},[],[668],{"type":47,"value":669},"error.category",{"type":47,"value":291},{"type":41,"tag":71,"props":672,"children":674},{"className":673},[],[675],{"type":47,"value":676},"duration",{"type":47,"value":291},{"type":41,"tag":71,"props":679,"children":681},{"className":680},[],[682],{"type":47,"value":683},"tool.name",{"type":47,"value":291},{"type":41,"tag":71,"props":686,"children":688},{"className":687},[],[689],{"type":47,"value":690},"operation",{"type":47,"value":291},{"type":41,"tag":71,"props":693,"children":695},{"className":694},[],[696],{"type":47,"value":697},"namespace",{"type":47,"value":291},{"type":41,"tag":71,"props":700,"children":702},{"className":701},[],[703],{"type":47,"value":704},"cluster",{"type":47,"value":291},{"type":41,"tag":71,"props":707,"children":709},{"className":708},[],[710],{"type":47,"value":711},"service",{"type":47,"value":291},{"type":41,"tag":71,"props":714,"children":716},{"className":715},[],[717],{"type":47,"value":718},"generation_count",{"type":47,"value":291},{"type":41,"tag":71,"props":721,"children":723},{"className":722},[],[724],{"type":47,"value":725},"eval.passed",{"type":47,"value":291},{"type":41,"tag":71,"props":728,"children":730},{"className":729},[],[731],{"type":47,"value":732},"eval.evaluator_id",{"type":47,"value":291},{"type":41,"tag":71,"props":735,"children":737},{"className":736},[],[738],{"type":47,"value":739},"eval.score_key",{"type":47,"value":291},{"type":41,"tag":71,"props":742,"children":744},{"className":743},[],[745],{"type":47,"value":746},"eval.score",{"type":41,"tag":50,"props":748,"children":749},{},[750,755,756,762,763,769,770,775,776,782,783,789,790,796,797,803],{"type":41,"tag":615,"props":751,"children":752},{},[753],{"type":47,"value":754},"Operators:",{"type":47,"value":621},{"type":41,"tag":71,"props":757,"children":759},{"className":758},[],[760],{"type":47,"value":761},"=",{"type":47,"value":291},{"type":41,"tag":71,"props":764,"children":766},{"className":765},[],[767],{"type":47,"value":768},"!=",{"type":47,"value":291},{"type":41,"tag":71,"props":771,"children":773},{"className":772},[],[774],{"type":47,"value":397},{"type":47,"value":291},{"type":41,"tag":71,"props":777,"children":779},{"className":778},[],[780],{"type":47,"value":781},"\u003C",{"type":47,"value":291},{"type":41,"tag":71,"props":784,"children":786},{"className":785},[],[787],{"type":47,"value":788},">=",{"type":47,"value":291},{"type":41,"tag":71,"props":791,"children":793},{"className":792},[],[794],{"type":47,"value":795},"\u003C=",{"type":47,"value":291},{"type":41,"tag":71,"props":798,"children":800},{"className":799},[],[801],{"type":47,"value":802},"=~",{"type":47,"value":804}," (regex)",{"type":41,"tag":88,"props":806,"children":808},{"id":807},"evaluator-kind-decision-table",[809],{"type":47,"value":810},"Evaluator Kind Decision Table",{"type":41,"tag":95,"props":812,"children":813},{},[814,830],{"type":41,"tag":99,"props":815,"children":816},{},[817],{"type":41,"tag":103,"props":818,"children":819},{},[820,825],{"type":41,"tag":107,"props":821,"children":822},{},[823],{"type":47,"value":824},"User describes",{"type":41,"tag":107,"props":826,"children":827},{},[828],{"type":47,"value":829},"Kind",{"type":41,"tag":118,"props":831,"children":832},{},[833,850,866,883,900,917],{"type":41,"tag":103,"props":834,"children":835},{},[836,841],{"type":41,"tag":125,"props":837,"children":838},{},[839],{"type":47,"value":840},"\"check if response is helpful \u002F toxic \u002F grounded\"",{"type":41,"tag":125,"props":842,"children":843},{},[844],{"type":41,"tag":71,"props":845,"children":847},{"className":846},[],[848],{"type":47,"value":849},"llm_judge",{"type":41,"tag":103,"props":851,"children":852},{},[853,858],{"type":41,"tag":125,"props":854,"children":855},{},[856],{"type":47,"value":857},"\"combined quality score with explanation\"",{"type":41,"tag":125,"props":859,"children":860},{},[861],{"type":41,"tag":71,"props":862,"children":864},{"className":863},[],[865],{"type":47,"value":849},{"type":41,"tag":103,"props":867,"children":868},{},[869,874],{"type":41,"tag":125,"props":870,"children":871},{},[872],{"type":47,"value":873},"\"validate JSON output format\"",{"type":41,"tag":125,"props":875,"children":876},{},[877],{"type":41,"tag":71,"props":878,"children":880},{"className":879},[],[881],{"type":47,"value":882},"json_schema",{"type":41,"tag":103,"props":884,"children":885},{},[886,891],{"type":41,"tag":125,"props":887,"children":888},{},[889],{"type":47,"value":890},"\"check if response contains \u002F doesn't contain X\"",{"type":41,"tag":125,"props":892,"children":893},{},[894],{"type":41,"tag":71,"props":895,"children":897},{"className":896},[],[898],{"type":47,"value":899},"regex",{"type":41,"tag":103,"props":901,"children":902},{},[903,908],{"type":41,"tag":125,"props":904,"children":905},{},[906],{"type":47,"value":907},"\"response must be non-empty and at least N chars\"",{"type":41,"tag":125,"props":909,"children":910},{},[911],{"type":41,"tag":71,"props":912,"children":914},{"className":913},[],[915],{"type":47,"value":916},"heuristic",{"type":41,"tag":103,"props":918,"children":919},{},[920,925],{"type":41,"tag":125,"props":921,"children":922},{},[923],{"type":47,"value":924},"\"check multiple conditions (non-empty AND has greeting)\"",{"type":41,"tag":125,"props":926,"children":927},{},[928],{"type":41,"tag":71,"props":929,"children":931},{"className":930},[],[932],{"type":47,"value":916},{"type":41,"tag":50,"props":934,"children":935},{},[936,938,944],{"type":47,"value":937},"Copy-paste definitions for each kind, with the constraints the API enforces for each, are in ",{"type":41,"tag":939,"props":940,"children":942},"a",{"href":941},"references\u002Fevaluator-examples.md",[943],{"type":47,"value":941},{"type":47,"value":945},".",{"type":41,"tag":88,"props":947,"children":949},{"id":948},"input-format",[950],{"type":47,"value":951},"Input Format",{"type":41,"tag":50,"props":953,"children":954},{},[955,961,962,968,970,976,978,984,985,991,993,999],{"type":41,"tag":71,"props":956,"children":958},{"className":957},[],[959],{"type":47,"value":960},"gcx agento11y evaluators get -o yaml",{"type":47,"value":272},{"type":41,"tag":71,"props":963,"children":965},{"className":964},[],[966],{"type":47,"value":967},"gcx agento11y rules get -o yaml",{"type":47,"value":969}," emit K8s-style manifests (",{"type":41,"tag":71,"props":971,"children":973},{"className":972},[],[974],{"type":47,"value":975},"apiVersion\u002Fkind\u002Fmetadata\u002Fspec",{"type":47,"value":977},"). ",{"type":41,"tag":71,"props":979,"children":981},{"className":980},[],[982],{"type":47,"value":983},"evaluators upsert -f",{"type":47,"value":291},{"type":41,"tag":71,"props":986,"children":988},{"className":987},[],[989],{"type":47,"value":990},"rules create -f",{"type":47,"value":992},", and ",{"type":41,"tag":71,"props":994,"children":996},{"className":995},[],[997],{"type":47,"value":998},"rules update -f",{"type":47,"value":1000}," expect top-level fields only. Do not round-trip get output into create\u002Fupdate.",{"type":41,"tag":50,"props":1002,"children":1003},{},[1004,1006,1012,1013,1019,1021,1027,1028,1033,1035,1041,1043,1049,1051,1055],{"type":47,"value":1005},"IDs (",{"type":41,"tag":71,"props":1007,"children":1009},{"className":1008},[],[1010],{"type":47,"value":1011},"evaluator_id",{"type":47,"value":291},{"type":41,"tag":71,"props":1014,"children":1016},{"className":1015},[],[1017],{"type":47,"value":1018},"rule_id",{"type":47,"value":1020},") accept only letters, digits, ",{"type":41,"tag":71,"props":1022,"children":1024},{"className":1023},[],[1025],{"type":47,"value":1026},"_",{"type":47,"value":992},{"type":41,"tag":71,"props":1029,"children":1031},{"className":1030},[],[1032],{"type":47,"value":945},{"type":47,"value":1034}," — hyphens are rejected server-side. ",{"type":41,"tag":71,"props":1036,"children":1038},{"className":1037},[],[1039],{"type":47,"value":1040},"version",{"type":47,"value":1042}," is required on evaluator definitions — it versions the evaluator itself, separate from any schema version inside ",{"type":41,"tag":71,"props":1044,"children":1046},{"className":1045},[],[1047],{"type":47,"value":1048},"config",{"type":47,"value":1050}," — see ",{"type":41,"tag":939,"props":1052,"children":1053},{"href":941},[1054],{"type":47,"value":941},{"type":47,"value":1056}," for full examples of every kind.",{"type":41,"tag":50,"props":1058,"children":1059},{},[1060],{"type":47,"value":1061},"Rule definition:",{"type":41,"tag":317,"props":1063,"children":1067},{"className":1064,"code":1065,"language":1066,"meta":322,"style":322},"language-yaml shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","rule_id: my_rule\nenabled: true\nselector: user_visible_turn\nsample_rate: 1.0\nevaluator_ids:\n  - my_evaluator\nmatch:\n  agent_name:\n    - my-agent\n","yaml",[1068],{"type":41,"tag":71,"props":1069,"children":1070},{"__ignoreMap":322},[1071,1089,1107,1124,1142,1155,1169,1182,1195],{"type":41,"tag":328,"props":1072,"children":1073},{"class":330,"line":331},[1074,1079,1084],{"type":41,"tag":328,"props":1075,"children":1077},{"style":1076},"--shiki-light:#E53935;--shiki-default:#F07178;--shiki-dark:#F07178",[1078],{"type":47,"value":1018},{"type":41,"tag":328,"props":1080,"children":1081},{"style":378},[1082],{"type":47,"value":1083},":",{"type":41,"tag":328,"props":1085,"children":1086},{"style":341},[1087],{"type":47,"value":1088}," my_rule\n",{"type":41,"tag":328,"props":1090,"children":1091},{"class":330,"line":357},[1092,1097,1101],{"type":41,"tag":328,"props":1093,"children":1094},{"style":1076},[1095],{"type":47,"value":1096},"enabled",{"type":41,"tag":328,"props":1098,"children":1099},{"style":378},[1100],{"type":47,"value":1083},{"type":41,"tag":328,"props":1102,"children":1104},{"style":1103},"--shiki-light:#FF5370;--shiki-default:#FF9CAC;--shiki-dark:#FF9CAC",[1105],{"type":47,"value":1106}," true\n",{"type":41,"tag":328,"props":1108,"children":1109},{"class":330,"line":405},[1110,1115,1119],{"type":41,"tag":328,"props":1111,"children":1112},{"style":1076},[1113],{"type":47,"value":1114},"selector",{"type":41,"tag":328,"props":1116,"children":1117},{"style":378},[1118],{"type":47,"value":1083},{"type":41,"tag":328,"props":1120,"children":1121},{"style":341},[1122],{"type":47,"value":1123}," user_visible_turn\n",{"type":41,"tag":328,"props":1125,"children":1126},{"class":330,"line":415},[1127,1132,1136],{"type":41,"tag":328,"props":1128,"children":1129},{"style":1076},[1130],{"type":47,"value":1131},"sample_rate",{"type":41,"tag":328,"props":1133,"children":1134},{"style":378},[1135],{"type":47,"value":1083},{"type":41,"tag":328,"props":1137,"children":1139},{"style":1138},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[1140],{"type":47,"value":1141}," 1.0\n",{"type":41,"tag":328,"props":1143,"children":1144},{"class":330,"line":436},[1145,1150],{"type":41,"tag":328,"props":1146,"children":1147},{"style":1076},[1148],{"type":47,"value":1149},"evaluator_ids",{"type":41,"tag":328,"props":1151,"children":1152},{"style":378},[1153],{"type":47,"value":1154},":\n",{"type":41,"tag":328,"props":1156,"children":1158},{"class":330,"line":1157},6,[1159,1164],{"type":41,"tag":328,"props":1160,"children":1161},{"style":378},[1162],{"type":47,"value":1163},"  -",{"type":41,"tag":328,"props":1165,"children":1166},{"style":341},[1167],{"type":47,"value":1168}," my_evaluator\n",{"type":41,"tag":328,"props":1170,"children":1172},{"class":330,"line":1171},7,[1173,1178],{"type":41,"tag":328,"props":1174,"children":1175},{"style":1076},[1176],{"type":47,"value":1177},"match",{"type":41,"tag":328,"props":1179,"children":1180},{"style":378},[1181],{"type":47,"value":1154},{"type":41,"tag":328,"props":1183,"children":1185},{"class":330,"line":1184},8,[1186,1191],{"type":41,"tag":328,"props":1187,"children":1188},{"style":1076},[1189],{"type":47,"value":1190},"  agent_name",{"type":41,"tag":328,"props":1192,"children":1193},{"style":378},[1194],{"type":47,"value":1154},{"type":41,"tag":328,"props":1196,"children":1198},{"class":330,"line":1197},9,[1199,1204],{"type":41,"tag":328,"props":1200,"children":1201},{"style":378},[1202],{"type":47,"value":1203},"    -",{"type":41,"tag":328,"props":1205,"children":1206},{"style":341},[1207],{"type":47,"value":1208}," my-agent\n",{"type":41,"tag":50,"props":1210,"children":1211},{},[1212,1214,1220,1222,1228,1230,1235,1237,1242],{"type":47,"value":1213},"There is no ",{"type":41,"tag":71,"props":1215,"children":1217},{"className":1216},[],[1218],{"type":47,"value":1219},"evaluators update",{"type":47,"value":1221}," command; to change an evaluator, re-run ",{"type":41,"tag":71,"props":1223,"children":1225},{"className":1224},[],[1226],{"type":47,"value":1227},"upsert",{"type":47,"value":1229}," with the same ",{"type":41,"tag":71,"props":1231,"children":1233},{"className":1232},[],[1234],{"type":47,"value":1011},{"type":47,"value":1236}," and a new ",{"type":41,"tag":71,"props":1238,"children":1240},{"className":1239},[],[1241],{"type":47,"value":1040},{"type":47,"value":1243}," (re-using an existing version is rejected with a 409).",{"type":41,"tag":88,"props":1245,"children":1247},{"id":1246},"setting-up-online-evaluation",[1248],{"type":47,"value":1249},"Setting Up Online Evaluation",{"type":41,"tag":1251,"props":1252,"children":1253},"ol",{},[1254,1311,1322,1333,1338,1357],{"type":41,"tag":1255,"props":1256,"children":1257},"li",{},[1258,1260,1266,1268,1274,1276,1282,1283,1288,1289,1295,1297,1302,1304,1310],{"type":47,"value":1259},"Pick a template: ",{"type":41,"tag":71,"props":1261,"children":1263},{"className":1262},[],[1264],{"type":47,"value":1265},"gcx agento11y templates list",{"type":47,"value":1267},", then ",{"type":41,"tag":71,"props":1269,"children":1271},{"className":1270},[],[1272],{"type":47,"value":1273},"gcx agento11y templates get \u003Cid> -o yaml",{"type":47,"value":1275},". Template output includes ",{"type":41,"tag":71,"props":1277,"children":1279},{"className":1278},[],[1280],{"type":47,"value":1281},"kind",{"type":47,"value":291},{"type":41,"tag":71,"props":1284,"children":1286},{"className":1285},[],[1287],{"type":47,"value":1048},{"type":47,"value":992},{"type":41,"tag":71,"props":1290,"children":1292},{"className":1291},[],[1293],{"type":47,"value":1294},"output_keys",{"type":47,"value":1296}," — copy these into a new evaluator definition and add your own ",{"type":41,"tag":71,"props":1298,"children":1300},{"className":1299},[],[1301],{"type":47,"value":1011},{"type":47,"value":1303},". Do not pass the template output directly to ",{"type":41,"tag":71,"props":1305,"children":1307},{"className":1306},[],[1308],{"type":47,"value":1309},"evaluators upsert",{"type":47,"value":945},{"type":41,"tag":1255,"props":1312,"children":1313},{},[1314,1316],{"type":47,"value":1315},"Write an evaluator YAML using the input format above, create: ",{"type":41,"tag":71,"props":1317,"children":1319},{"className":1318},[],[1320],{"type":47,"value":1321},"gcx agento11y evaluators upsert -f evaluator.yaml",{"type":41,"tag":1255,"props":1323,"children":1324},{},[1325,1327],{"type":47,"value":1326},"Test against a real generation: ",{"type":41,"tag":71,"props":1328,"children":1330},{"className":1329},[],[1331],{"type":47,"value":1332},"gcx agento11y evaluators test -e \u003Cevaluator-id> -g \u003Cgeneration-id>",{"type":41,"tag":1255,"props":1334,"children":1335},{},[1336],{"type":47,"value":1337},"Iterate until the evaluator scores as expected",{"type":41,"tag":1255,"props":1339,"children":1340},{},[1341,1343,1349,1351],{"type":47,"value":1342},"Write a rule YAML (see ",{"type":41,"tag":939,"props":1344,"children":1346},{"href":1345},"references\u002Frule-templates.md",[1347],{"type":47,"value":1348},"rule-templates.md",{"type":47,"value":1350}," for copy-paste templates), create: ",{"type":41,"tag":71,"props":1352,"children":1354},{"className":1353},[],[1355],{"type":47,"value":1356},"gcx agento11y rules create -f rule.yaml",{"type":41,"tag":1255,"props":1358,"children":1359},{},[1360,1362],{"type":47,"value":1361},"Verify: ",{"type":41,"tag":71,"props":1363,"children":1365},{"className":1364},[],[1366],{"type":47,"value":484},{"type":41,"tag":88,"props":1368,"children":1370},{"id":1369},"rule-selectors",[1371],{"type":47,"value":1372},"Rule Selectors",{"type":41,"tag":95,"props":1374,"children":1375},{},[1376,1392],{"type":41,"tag":99,"props":1377,"children":1378},{},[1379],{"type":41,"tag":103,"props":1380,"children":1381},{},[1382,1387],{"type":41,"tag":107,"props":1383,"children":1384},{},[1385],{"type":47,"value":1386},"Selector",{"type":41,"tag":107,"props":1388,"children":1389},{},[1390],{"type":47,"value":1391},"What it evaluates",{"type":41,"tag":118,"props":1393,"children":1394},{},[1395,1412,1429],{"type":41,"tag":103,"props":1396,"children":1397},{},[1398,1407],{"type":41,"tag":125,"props":1399,"children":1400},{},[1401],{"type":41,"tag":71,"props":1402,"children":1404},{"className":1403},[],[1405],{"type":47,"value":1406},"user_visible_turn",{"type":41,"tag":125,"props":1408,"children":1409},{},[1410],{"type":47,"value":1411},"Final assistant generation visible to the user",{"type":41,"tag":103,"props":1413,"children":1414},{},[1415,1424],{"type":41,"tag":125,"props":1416,"children":1417},{},[1418],{"type":41,"tag":71,"props":1419,"children":1421},{"className":1420},[],[1422],{"type":47,"value":1423},"all_assistant_generations",{"type":41,"tag":125,"props":1425,"children":1426},{},[1427],{"type":47,"value":1428},"Every assistant generation in the conversation",{"type":41,"tag":103,"props":1430,"children":1431},{},[1432,1441],{"type":41,"tag":125,"props":1433,"children":1434},{},[1435],{"type":41,"tag":71,"props":1436,"children":1438},{"className":1437},[],[1439],{"type":47,"value":1440},"tool_call_steps",{"type":41,"tag":125,"props":1442,"children":1443},{},[1444],{"type":47,"value":1445},"Tool call generations",{"type":41,"tag":88,"props":1447,"children":1449},{"id":1448},"rule-match-keys",[1450],{"type":47,"value":1451},"Rule Match Keys",{"type":41,"tag":50,"props":1453,"children":1454},{},[1455,1457,1463,1464,1470,1471,1477],{"type":47,"value":1456},"All values are arrays. Glob-capable keys support ",{"type":41,"tag":71,"props":1458,"children":1460},{"className":1459},[],[1461],{"type":47,"value":1462},"*",{"type":47,"value":291},{"type":41,"tag":71,"props":1465,"children":1467},{"className":1466},[],[1468],{"type":47,"value":1469},"?",{"type":47,"value":291},{"type":41,"tag":71,"props":1472,"children":1474},{"className":1473},[],[1475],{"type":47,"value":1476},"[...]",{"type":47,"value":1478}," patterns.",{"type":41,"tag":95,"props":1480,"children":1481},{},[1482,1503],{"type":41,"tag":99,"props":1483,"children":1484},{},[1485],{"type":41,"tag":103,"props":1486,"children":1487},{},[1488,1493,1498],{"type":41,"tag":107,"props":1489,"children":1490},{},[1491],{"type":47,"value":1492},"Key",{"type":41,"tag":107,"props":1494,"children":1495},{},[1496],{"type":47,"value":1497},"Glob",{"type":41,"tag":107,"props":1499,"children":1500},{},[1501],{"type":47,"value":1502},"Description",{"type":41,"tag":118,"props":1504,"children":1505},{},[1506,1528,1549,1570,1605,1640,1674,1709,1741],{"type":41,"tag":103,"props":1507,"children":1508},{},[1509,1518,1523],{"type":41,"tag":125,"props":1510,"children":1511},{},[1512],{"type":41,"tag":71,"props":1513,"children":1515},{"className":1514},[],[1516],{"type":47,"value":1517},"agent_name",{"type":41,"tag":125,"props":1519,"children":1520},{},[1521],{"type":47,"value":1522},"yes",{"type":41,"tag":125,"props":1524,"children":1525},{},[1526],{"type":47,"value":1527},"Agent name",{"type":41,"tag":103,"props":1529,"children":1530},{},[1531,1540,1544],{"type":41,"tag":125,"props":1532,"children":1533},{},[1534],{"type":41,"tag":71,"props":1535,"children":1537},{"className":1536},[],[1538],{"type":47,"value":1539},"agent_version",{"type":41,"tag":125,"props":1541,"children":1542},{},[1543],{"type":47,"value":1522},{"type":41,"tag":125,"props":1545,"children":1546},{},[1547],{"type":47,"value":1548},"Agent version string",{"type":41,"tag":103,"props":1550,"children":1551},{},[1552,1561,1565],{"type":41,"tag":125,"props":1553,"children":1554},{},[1555],{"type":41,"tag":71,"props":1556,"children":1558},{"className":1557},[],[1559],{"type":47,"value":1560},"operation_name",{"type":41,"tag":125,"props":1562,"children":1563},{},[1564],{"type":47,"value":1522},{"type":41,"tag":125,"props":1566,"children":1567},{},[1568],{"type":47,"value":1569},"Operation name",{"type":41,"tag":103,"props":1571,"children":1572},{},[1573,1582,1586],{"type":41,"tag":125,"props":1574,"children":1575},{},[1576],{"type":41,"tag":71,"props":1577,"children":1579},{"className":1578},[],[1580],{"type":47,"value":1581},"model.provider",{"type":41,"tag":125,"props":1583,"children":1584},{},[1585],{"type":47,"value":1522},{"type":41,"tag":125,"props":1587,"children":1588},{},[1589,1591,1597,1598,1604],{"type":47,"value":1590},"Model provider (e.g. ",{"type":41,"tag":71,"props":1592,"children":1594},{"className":1593},[],[1595],{"type":47,"value":1596},"openai",{"type":47,"value":291},{"type":41,"tag":71,"props":1599,"children":1601},{"className":1600},[],[1602],{"type":47,"value":1603},"anthropic",{"type":47,"value":179},{"type":41,"tag":103,"props":1606,"children":1607},{},[1608,1617,1621],{"type":41,"tag":125,"props":1609,"children":1610},{},[1611],{"type":41,"tag":71,"props":1612,"children":1614},{"className":1613},[],[1615],{"type":47,"value":1616},"model.name",{"type":41,"tag":125,"props":1618,"children":1619},{},[1620],{"type":47,"value":1522},{"type":41,"tag":125,"props":1622,"children":1623},{},[1624,1626,1632,1633,1639],{"type":47,"value":1625},"Model name (e.g. ",{"type":41,"tag":71,"props":1627,"children":1629},{"className":1628},[],[1630],{"type":47,"value":1631},"gpt-4o",{"type":47,"value":291},{"type":41,"tag":71,"props":1634,"children":1636},{"className":1635},[],[1637],{"type":47,"value":1638},"claude-sonnet-4-5-20250514",{"type":47,"value":179},{"type":41,"tag":103,"props":1641,"children":1642},{},[1643,1652,1657],{"type":41,"tag":125,"props":1644,"children":1645},{},[1646],{"type":41,"tag":71,"props":1647,"children":1649},{"className":1648},[],[1650],{"type":47,"value":1651},"mode",{"type":41,"tag":125,"props":1653,"children":1654},{},[1655],{"type":47,"value":1656},"no",{"type":41,"tag":125,"props":1658,"children":1659},{},[1660,1666,1668],{"type":41,"tag":71,"props":1661,"children":1663},{"className":1662},[],[1664],{"type":47,"value":1665},"SYNC",{"type":47,"value":1667}," or ",{"type":41,"tag":71,"props":1669,"children":1671},{"className":1670},[],[1672],{"type":47,"value":1673},"STREAM",{"type":41,"tag":103,"props":1675,"children":1676},{},[1677,1685,1689],{"type":41,"tag":125,"props":1678,"children":1679},{},[1680],{"type":41,"tag":71,"props":1681,"children":1683},{"className":1682},[],[1684],{"type":47,"value":662},{"type":41,"tag":125,"props":1686,"children":1687},{},[1688],{"type":47,"value":1656},{"type":41,"tag":125,"props":1690,"children":1691},{},[1692,1694,1700,1702,1708],{"type":47,"value":1693},"Error type (also accepts ",{"type":41,"tag":71,"props":1695,"children":1697},{"className":1696},[],[1698],{"type":47,"value":1699},"present",{"type":47,"value":1701},"\u002F",{"type":41,"tag":71,"props":1703,"children":1705},{"className":1704},[],[1706],{"type":47,"value":1707},"absent",{"type":47,"value":179},{"type":41,"tag":103,"props":1710,"children":1711},{},[1712,1720,1724],{"type":41,"tag":125,"props":1713,"children":1714},{},[1715],{"type":41,"tag":71,"props":1716,"children":1718},{"className":1717},[],[1719],{"type":47,"value":669},{"type":41,"tag":125,"props":1721,"children":1722},{},[1723],{"type":47,"value":1656},{"type":41,"tag":125,"props":1725,"children":1726},{},[1727,1729,1734,1735,1740],{"type":47,"value":1728},"Error category (also accepts ",{"type":41,"tag":71,"props":1730,"children":1732},{"className":1731},[],[1733],{"type":47,"value":1699},{"type":47,"value":1701},{"type":41,"tag":71,"props":1736,"children":1738},{"className":1737},[],[1739],{"type":47,"value":1707},{"type":47,"value":179},{"type":41,"tag":103,"props":1742,"children":1743},{},[1744,1753,1757],{"type":41,"tag":125,"props":1745,"children":1746},{},[1747],{"type":41,"tag":71,"props":1748,"children":1750},{"className":1749},[],[1751],{"type":47,"value":1752},"tags.\u003Ckey>",{"type":41,"tag":125,"props":1754,"children":1755},{},[1756],{"type":47,"value":1656},{"type":41,"tag":125,"props":1758,"children":1759},{},[1760,1762,1768],{"type":47,"value":1761},"Custom tag value (e.g. ",{"type":41,"tag":71,"props":1763,"children":1765},{"className":1764},[],[1766],{"type":47,"value":1767},"tags.env",{"type":47,"value":179},{"type":41,"tag":1770,"props":1771,"children":1772},"style",{},[1773],{"type":47,"value":1774},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":1776,"total":1955},[1777,1794,1813,1833,1840,1856,1869,1884,1901,1916,1929,1943],{"slug":1778,"name":1778,"fn":1779,"description":1780,"org":1781,"tags":1782,"stars":1791,"repoUrl":1792,"updatedAt":1793},"faro-setup-web","instrument web apps with Grafana Faro","Instruments a web app with Grafana Faro Web SDK for frontend observability. Use when setting up error tracking, Web Vitals, session monitoring, or distributed tracing in a browser app.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1783,1786,1789,1790],{"name":1784,"slug":1785,"type":15},"Distributed Tracing","distributed-tracing",{"name":1787,"slug":1788,"type":15},"Frontend","frontend",{"name":17,"slug":18,"type":15},{"name":13,"slug":14,"type":15},1103,"https:\u002F\u002Fgithub.com\u002Fgrafana\u002Ffaro-web-sdk","2026-07-12T07:43:24.63314",{"slug":1795,"name":1795,"fn":1796,"description":1797,"org":1798,"tags":1799,"stars":1810,"repoUrl":1811,"updatedAt":1812},"configuring-yesoreyeram-infinity-datasource","configure Grafana Infinity data source","Configure the Infinity data source — base URL and allowed hosts, the authentication methods (basic, bearer token, API key, digest, OAuth passthrough, OAuth 2.0 client credentials\u002FJWT, Azure, Azure Blob, AWS), TLS, custom HTTP headers, network and security settings, the custom health check, and provisioning with a config file. Use when a user asks how to set up, configure, or change settings for the Infinity data source; how to authenticate to an API; how to allow hosts; how to provision it as YAML; or how to troubleshoot connection, authentication, or health-check issues.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1800,1803,1806,1809],{"name":1801,"slug":1802,"type":15},"API Development","api-development",{"name":1804,"slug":1805,"type":15},"Authentication","authentication",{"name":1807,"slug":1808,"type":15},"Configuration","configuration",{"name":9,"slug":8,"type":15},1056,"https:\u002F\u002Fgithub.com\u002Fgrafana\u002Fgrafana-infinity-datasource","2026-07-12T07:43:25.939136",{"slug":1814,"name":1814,"fn":1815,"description":1816,"org":1817,"tags":1818,"stars":1810,"repoUrl":1811,"updatedAt":1832},"querying-yesoreyeram-infinity-datasource","query data with Infinity datasource","Build queries with the Infinity data source — the query types (JSON, CSV, TSV, XML, GraphQL, HTML, UQL, GROQ, Google Sheets, Series, Transformations), the parsers (Frontend\u002Fsimple, Backend JSONata, JQ, UQL, GROQ) and which support alerting, the sources (URL, Inline, Reference, Azure Blob, Random walk), output formats (table, timeseries, logs, trace, node graph, dataframe), root selector and columns, computed columns\u002Ffilters\u002Fsummarize, pagination, and template variables. Use when a user asks how to query Infinity; how to fetch JSON\u002FCSV\u002FXML\u002FGraphQL\u002FHTML from a URL or inline; how to select rows and columns; how to transform data with UQL\u002FGROQ\u002FJSONata\u002FJQ; how to make a query work with alerting; how to paginate; or how to use variables in a query.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1819,1822,1825,1826,1829],{"name":1820,"slug":1821,"type":15},"CSV","csv",{"name":1823,"slug":1824,"type":15},"Data Analysis","data-analysis",{"name":9,"slug":8,"type":15},{"name":1827,"slug":1828,"type":15},"GraphQL","graphql",{"name":1830,"slug":1831,"type":15},"JSON","json","2026-07-15T05:34:05.773947",{"slug":4,"name":4,"fn":5,"description":6,"org":1834,"tags":1835,"stars":23,"repoUrl":24,"updatedAt":25},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1836,1837,1838,1839],{"name":20,"slug":21,"type":15},{"name":9,"slug":8,"type":15},{"name":17,"slug":18,"type":15},{"name":13,"slug":14,"type":15},{"slug":1841,"name":1841,"fn":1842,"description":1843,"org":1844,"tags":1845,"stars":23,"repoUrl":24,"updatedAt":1855},"agento11y-instrument","instrument LLM apps for agent observability","Sets up and instruments a developer's own LLM app or agent to send generations and agentic workflow to Grafana Agent Observability (the Agent Observability SDKs) — greenfield setup, fixing broken instrumentation, or filling gaps in existing instrumentation. Uses gcx for the parts a static prompt can't do: `gcx login` \u002F `gcx cloud stacks` to find the stack, and `gcx agento11y agents|conversations|generations` to VERIFY that data actually lands — so it iterates (instrument → run → verify → fix) until generations arrive, not blindly. Reads the app's code, detects language\u002Fframework, classifies instrumentation state (none \u002F partial \u002F broken), then runs a fixed gap checklist whose #1 item is the silent failure no other prompt catches: the SDK emits OTel spans\u002Fmetrics but never creates a TracerProvider\u002FMeterProvider, so without them all metrics go to a no-op and are lost. Also checks agent_version (required for per-version Performance charts), set_result completeness, SYNC vs STREAM, parent_generation_ids DAG links, and workflow-step coverage. Recommends changes citing file:line and, only with explicit confirmation, applies minimal diffs that don't change app behavior. Pulls SDK reference from agento11y's llms.txt rather than restating it, and hands off to `agento11y-test-starter` once data flows. It does NOT write test suites or set up tenant evaluations, rules, or guards — offline test suites are `agento11y-test-starter`, tenant eval rules + guards are `agento11y-prod-setup`; does NOT install coding-agent telemetry plugins (that is llms.txt \"Path A\"); does NOT mint or store credentials or invent endpoints. Trigger on phrases like \"instrument my app\", \"send my agent's traces to Grafana\", \"set up AI observability for my app\", \"my generations aren't showing up\", \"why is Performance empty\", \"add Agent Observability to my code\", \"fix my instrumentation\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1846,1847,1848,1851,1854],{"name":20,"slug":21,"type":15},{"name":9,"slug":8,"type":15},{"name":1849,"slug":1850,"type":15},"Instrumentation","instrumentation",{"name":1852,"slug":1853,"type":15},"LLM","llm",{"name":13,"slug":14,"type":15},"2026-07-31T05:53:52.580237",{"slug":1857,"name":1857,"fn":1858,"description":1859,"org":1860,"tags":1861,"stars":23,"repoUrl":24,"updatedAt":1868},"agento11y-prod-setup","setup production evaluation for AI agents","Sets up production evaluation and guardrails for a DEPLOYED AI agent in Grafana Agent Observability, grounded in the agent's own code and its real ingested traffic. The judgment layer on top of the `agento11y` skill: it reads the agent's source (system prompt, tools, entrypoint) AND samples its live traffic via gcx, checks what evaluators\u002Frules\u002Fguards already exist, then recommends only what's missing — online eval rules (score live conversations for regressions) and guards (warn-first request-path policies that redact \u002F tool-filter and may later be promoted to deny). It drafts reviewable YAML and, only with explicit confirmation, applies via `gcx agento11y`. New guards are drafted in warn mode (safe on live traffic — warn records but never blocks). It DOES create stack-level objects — that is the point — but every write is confirmed. It never rewrites or redeploys the agent. Trigger on phrases like \"set up production evaluation\", \"my agent is in prod what should I evaluate\", \"catch quality regressions\", \"add guardrails to my agent\", \"redact PII from my agent\", \"block dangerous tools\", \"set up online evals and guards\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1862,1863,1866,1867],{"name":20,"slug":21,"type":15},{"name":1864,"slug":1865,"type":15},"Evals","evals",{"name":9,"slug":8,"type":15},{"name":13,"slug":14,"type":15},"2026-07-31T05:53:53.576347",{"slug":1870,"name":1870,"fn":1871,"description":1872,"org":1873,"tags":1874,"stars":23,"repoUrl":24,"updatedAt":1883},"agento11y-test-starter","build and run agent test suites","Use early in an AI-agent project — before ship, before real traffic — to build a starter test suite for the agent and run it offline. Reads the agent's own code (system prompt, tools, task), writes a labeled draft suite of test cases (happy\u002Fedge\u002Fadversarial) grounded in real lines, and recommends how to score each case (the evaluators\u002Fjudges the offline runner uses). Assesses how runnable the agent is: for an easily-invoked agent it generates a runner stub (run_experiment.py) with two holes to fill and can optionally run it (only with permission, only against the endpoint the developer configured); for agents needing a harness or full runtime it points to the existing eval infra. It runs OFFLINE and never creates tenant-level evaluators, rules, or guards — that is `agento11y-prod-setup`, for a deployed agent with real traffic. Trigger on phrases like \"how do I test my agent before shipping\", \"write test cases for my agent\", \"set up tests for my agent\", \"check my agent before prod\", \"I have no traffic yet, how do I evaluate it\", \"test my agent offline\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1875,1876,1877,1880],{"name":20,"slug":21,"type":15},{"name":9,"slug":8,"type":15},{"name":1878,"slug":1879,"type":15},"QA","qa",{"name":1881,"slug":1882,"type":15},"Testing","testing","2026-07-31T05:53:51.62785",{"slug":1885,"name":1885,"fn":1886,"description":1887,"org":1888,"tags":1889,"stars":23,"repoUrl":24,"updatedAt":1900},"create-dashboard","create Grafana dashboards with gcx","Designs and creates Grafana dashboards with gcx, using `gcx dashboards snapshot` as a visual feedback loop. Use when the user wants to create a new Grafana dashboard, add panels, variables, or annotations to an existing dashboard, design dashboard panels, variables, queries, or layout, or make a material visual redesign. Triggers on \"create dashboard\", \"new dashboard\", \"build dashboard\", \"dashboard for \u003Cservice>\", \"add panels\", \"add variable\", \"add annotation\", \"improve this dashboard\", or \"iterate on a dashboard\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1890,1893,1896,1899],{"name":1891,"slug":1892,"type":15},"Dashboards","dashboards",{"name":1894,"slug":1895,"type":15},"Data Visualization","data-visualization",{"name":1897,"slug":1898,"type":15},"Design","design",{"name":9,"slug":8,"type":15},"2026-07-25T05:30:46.289717",{"slug":1902,"name":1902,"fn":1903,"description":1904,"org":1905,"tags":1906,"stars":23,"repoUrl":24,"updatedAt":1915},"debug-with-grafana","investigate application issues with Grafana","Structured workflow for investigating application problems with Grafana observability data (metrics, logs, traces) via gcx. Covers live firefighting AND retrospective incident analysis: incident triage, root-cause analysis, blast-radius checks (did an incident spill into other services), verifying whether a deployment or rollout triggered an incident, finding which service, endpoint, or path owns the most errors or slow requests, checking whether retries or queue backlogs piled up, and quantifying error or latency shares over a time window. Trigger on: \"my API is returning 500 errors\", \"latency is spiking\", \"investigate why requests are failing\", \"triage the incident\", \"blast radius\", \"root cause\", \"did the rollout cause it\", \"which endpoint owns the most 5xx\", \"did retries pile up\", or any request to analyse an earlier incident window using telemetry. For authoring dashboards use create-dashboard; for dashboard inventory use manage-dashboards.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1907,1910,1911,1914],{"name":1908,"slug":1909,"type":15},"Debugging","debugging",{"name":9,"slug":8,"type":15},{"name":1912,"slug":1913,"type":15},"Incident Response","incident-response",{"name":13,"slug":14,"type":15},"2026-07-18T05:11:10.445428",{"slug":1917,"name":1917,"fn":1918,"description":1919,"org":1920,"tags":1921,"stars":23,"repoUrl":24,"updatedAt":1928},"diagnose-entity-graph","diagnose Grafana Entity Graph issues","Diagnose Entity Graph problems: missing entities, missing edges, disconnected clusters, or filtering issues. Use when the user reports that Entity Graph doesn't look right, services are missing, edges aren't appearing, or environments can't be filtered. Triggers for: \"entity graph is empty\", \"services missing from entity graph\", \"no edges in entity graph\", \"disconnected services\", \"can't filter entity graph\", \"entity graph not working\", \"diagnose entity graph\", \"debug knowledge graph\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1922,1923,1924,1927],{"name":1908,"slug":1909,"type":15},{"name":9,"slug":8,"type":15},{"name":1925,"slug":1926,"type":15},"Graph Analysis","graph-analysis",{"name":13,"slug":14,"type":15},"2026-07-25T05:30:39.380934",{"slug":338,"name":338,"fn":1930,"description":1931,"org":1932,"tags":1933,"stars":23,"repoUrl":24,"updatedAt":1942},"manage Grafana Cloud resources via gcx","Manages Grafana Cloud resources via the gcx CLI. Trigger when the user wants to inspect, create, update, delete, query, or automate any Grafana resource - dashboards, datasources, alerts, SLOs, synthetic checks, oncall, incidents, fleet, k6, knowledge graph, or adaptive telemetry.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1934,1937,1938,1939],{"name":1935,"slug":1936,"type":15},"CLI","cli",{"name":9,"slug":8,"type":15},{"name":17,"slug":18,"type":15},{"name":1940,"slug":1941,"type":15},"Operations","operations","2026-07-31T05:53:50.587304",{"slug":1944,"name":1944,"fn":1945,"description":1946,"org":1947,"tags":1948,"stars":23,"repoUrl":24,"updatedAt":1954},"gcx-demo","present gcx demo tours","Run a narrated, read-only demo tour of gcx for customer or colleague presentations. Showcases the breadth of gcx across every Grafana Cloud product area — resources, datasources, metrics, logs, traces, SLOs, alerts, synthetic monitoring, IRM, k6, fleet, and more. All commands are strictly read-only. Trigger when the user says \"demo gcx\", \"show off gcx\", \"customer demo\", \"gcx tour\", or \"\u002Fgcx-demo\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1949,1950,1951],{"name":1935,"slug":1936,"type":15},{"name":9,"slug":8,"type":15},{"name":1952,"slug":1953,"type":15},"Presentations","presentations","2026-07-25T05:30:45.282458",80,{"items":1957,"total":2008},[1958,1965,1973,1980,1987,1994,2001],{"slug":4,"name":4,"fn":5,"description":6,"org":1959,"tags":1960,"stars":23,"repoUrl":24,"updatedAt":25},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1961,1962,1963,1964],{"name":20,"slug":21,"type":15},{"name":9,"slug":8,"type":15},{"name":17,"slug":18,"type":15},{"name":13,"slug":14,"type":15},{"slug":1841,"name":1841,"fn":1842,"description":1843,"org":1966,"tags":1967,"stars":23,"repoUrl":24,"updatedAt":1855},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1968,1969,1970,1971,1972],{"name":20,"slug":21,"type":15},{"name":9,"slug":8,"type":15},{"name":1849,"slug":1850,"type":15},{"name":1852,"slug":1853,"type":15},{"name":13,"slug":14,"type":15},{"slug":1857,"name":1857,"fn":1858,"description":1859,"org":1974,"tags":1975,"stars":23,"repoUrl":24,"updatedAt":1868},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1976,1977,1978,1979],{"name":20,"slug":21,"type":15},{"name":1864,"slug":1865,"type":15},{"name":9,"slug":8,"type":15},{"name":13,"slug":14,"type":15},{"slug":1870,"name":1870,"fn":1871,"description":1872,"org":1981,"tags":1982,"stars":23,"repoUrl":24,"updatedAt":1883},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1983,1984,1985,1986],{"name":20,"slug":21,"type":15},{"name":9,"slug":8,"type":15},{"name":1878,"slug":1879,"type":15},{"name":1881,"slug":1882,"type":15},{"slug":1885,"name":1885,"fn":1886,"description":1887,"org":1988,"tags":1989,"stars":23,"repoUrl":24,"updatedAt":1900},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1990,1991,1992,1993],{"name":1891,"slug":1892,"type":15},{"name":1894,"slug":1895,"type":15},{"name":1897,"slug":1898,"type":15},{"name":9,"slug":8,"type":15},{"slug":1902,"name":1902,"fn":1903,"description":1904,"org":1995,"tags":1996,"stars":23,"repoUrl":24,"updatedAt":1915},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1997,1998,1999,2000],{"name":1908,"slug":1909,"type":15},{"name":9,"slug":8,"type":15},{"name":1912,"slug":1913,"type":15},{"name":13,"slug":14,"type":15},{"slug":1917,"name":1917,"fn":1918,"description":1919,"org":2002,"tags":2003,"stars":23,"repoUrl":24,"updatedAt":1928},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2004,2005,2006,2007],{"name":1908,"slug":1909,"type":15},{"name":9,"slug":8,"type":15},{"name":1925,"slug":1926,"type":15},{"name":13,"slug":14,"type":15},24]