[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-grafana-agento11y-prod-setup":3,"mdc-zgm8cm-key":34,"related-org-grafana-agento11y-prod-setup":4063,"related-repo-grafana-agento11y-prod-setup":4242},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":23,"repoUrl":24,"updatedAt":25,"license":26,"forks":27,"topics":28,"repo":29,"sourceUrl":32,"mdContent":33},"agento11y-prod-setup","setup production evaluation for AI agents","Sets up production evaluation and guardrails for a DEPLOYED AI agent in Grafana Agent Observability, grounded in the agent's own code and its real ingested traffic. The judgment layer on top of the `agento11y` skill: it reads the agent's source (system prompt, tools, entrypoint) AND samples its live traffic via gcx, checks what evaluators\u002Frules\u002Fguards already exist, then recommends only what's missing — online eval rules (score live conversations for regressions) and guards (warn-first request-path policies that redact \u002F tool-filter and may later be promoted to deny). It drafts reviewable YAML and, only with explicit confirmation, applies via `gcx agento11y`. New guards are drafted in warn mode (safe on live traffic — warn records but never blocks). It DOES create stack-level objects — that is the point — but every write is confirmed. It never rewrites or redeploys the agent. Trigger on phrases like \"set up production evaluation\", \"my agent is in prod what should I evaluate\", \"catch quality regressions\", \"add guardrails to my agent\", \"redact PII from my agent\", \"block dangerous tools\", \"set up online evals and guards\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"grafana","Grafana","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fgrafana.jpg",[12,16,19,22],{"name":13,"slug":14,"type":15},"Observability","observability","tag",{"name":17,"slug":18,"type":15},"Evals","evals",{"name":20,"slug":21,"type":15},"Agents","agents",{"name":9,"slug":8,"type":15},430,"https:\u002F\u002Fgithub.com\u002Fgrafana\u002Fgcx","2026-07-31T05:53:53.576347",null,29,[],{"repoUrl":24,"stars":23,"forks":27,"topics":30,"description":31},[],"A CLI for managing Grafana Cloud resources. Optimized for agentic usage.","https:\u002F\u002Fgithub.com\u002Fgrafana\u002Fgcx\u002Ftree\u002FHEAD\u002Fclaude-plugin\u002Fskills\u002Fagento11y-prod-setup","---\nname: agento11y-prod-setup\ndescription: >\n  Sets up production evaluation and guardrails for a DEPLOYED AI agent in Grafana\n  Agent Observability, grounded in the agent's own code and its real ingested traffic.\n  The judgment layer on top of the `agento11y` skill: it reads the agent's source\n  (system prompt, tools, entrypoint) AND samples its live traffic via gcx, checks\n  what evaluators\u002Frules\u002Fguards already exist, then recommends only what's missing —\n  online eval rules (score live conversations for regressions) and guards (warn-first\n  request-path policies that redact \u002F tool-filter and may later be promoted to deny).\n  It drafts reviewable YAML and, only with explicit confirmation, applies via\n  `gcx agento11y`. New guards are drafted in warn mode (safe on live traffic — warn records but\n  never blocks). It DOES create stack-level objects —\n  that is the point — but every write is confirmed. It never rewrites or redeploys the\n  agent. Trigger on phrases like \"set up production evaluation\", \"my agent is in prod\n  what should I evaluate\", \"catch quality regressions\", \"add guardrails to my agent\",\n  \"redact PII from my agent\", \"block dangerous tools\", \"set up online evals and guards\".\nallowed-tools: Bash, Read, Write, Edit\n---\n\n# Agent Observability — production evals & guards setup\n\nThe production counterpart to `agento11y-test-starter` (which runs pre-ship, on code alone,\nproducing an offline test suite). This skill runs **after** ship, when the agent has real\ntraffic, and sets up the two production surfaces the starter deliberately leaves out:\n\n- **Online eval rules** — evaluators that score ingested live conversations, so regressions\n  surface without hand-reviewing every conversation.\n- **Guards** (hook-rules) — policies on the request path that `warn` (and can later be promoted to\n  `deny`) in real time. A guard decides via one of three shapes: `evaluator_ids` (an evaluator\n  judges), `redact` (regex redaction), or `tool_filter` (block tool calls). See Step 4.\n\n## What this skill does that `agento11y` doesn't\n\nThe sibling `agento11y` skill is the **mechanics** layer: exact CLI flags, evaluator\u002Frule YAML\nshapes, create-or-update semantics, the online-eval setup steps. It assumes you already know\n*what* to create.\n\nThis skill is the **judgment** layer. It answers *which* rules and guards this specific agent\nneeds, by grounding in two evidence sources a generic checklist can't use:\n\n1. **The agent's code** — its system prompt, tools, and how it handles user data. Half the value\n   is here; read and cite it (`file:line`).\n2. **The agent's real traffic** — because it's deployed, you can see what it actually does in\n   prod, not just what the code says it might.\n\nTwo gaps this skill fills beyond `agento11y`:\n\n- **Recommendation from evidence** — `agento11y` starts once you know what to create; this decides.\n- **Guards** — `agento11y` documents evaluators and rules but not guards (hook-rules), even though\n  `gcx agento11y guards` exists. This skill carries the guard shapes\n  (`evaluator_ids` \u002F `redact` \u002F `tool_filter`, plus `action_on_fail`) itself.\n\nFor any mechanical detail — exact flags, evaluator\u002Frule YAML fields, the setup flow — defer to\nthe `agento11y` skill and to `gcx agento11y \u003Csub> --help` rather than restating it here.\n\n## Rules\n\n- Every connection to the stack goes through **`gcx agento11y`** — never raw HTTP, never a hand-held\n  token. `gcx` owns Cloud auth (via `gcx login`). Prerequisite: `gcx` installed and authenticated;\n  if it isn't, say so and stop.\n- **Confirm the target stack before any WRITE (Step 0 + Step 5).** Reads run freely once you've\n  shown the context; writes (upsert evaluators, create\u002Fupdate rules and guards) need an explicit yes on the\n  target stack. `gcx` may be pointed at the wrong stack, and this skill creates stack-level\n  objects.\n- **Check before recommending.** Always list what already exists first\n  (`gcx agento11y evaluators list`, `rules list`, `guards list`) and never recommend a duplicate.\n  Compare by **semantic equivalence**, not just id\u002Fname — see Step 2.\n- This skill **does** create stack-level objects — that is its job, the one thing that separates\n  it from `agento11y-test-starter`. But every creation is **explicit and confirmed**: show the exact\n  YAML, get a yes, then create it with the matching `gcx agento11y` command. A yes for one object\n  is not a yes for the next.\n- New guards are always drafted **`action_on_fail: \"warn\"`** — even hard-policy ones. That is what\n  makes a new guard safe: a `warn` guard only records the outcome, it never blocks a request, so it\n  is harmless even while it is live. Never draft a first-time `deny` guard. The developer switches to\n  `deny` themselves, later, after watching it in warn mode (Step 6). Draft `enabled: false` if you\n  can, but don't build extra steps around it — the server may store the guard `enabled: true`\n  regardless (see Step 5), and that is fine, because `warn` carries the safety, not `enabled`.\n- New online rules start with a **conservative `sample_rate`** (e.g. `0.1`), not `1.0` — an\n  `llm_judge` over 100% of traffic costs real money.\n- Prefer **starting from an evaluator template** (`gcx agento11y templates list`, then\n  `gcx agento11y templates get`) over authoring\n  a new evaluator. Only write a fresh one when nothing fits.\n- Do not rewrite the agent's prompt, optimize, or redeploy. This skill configures observation and\n  guardrails around the agent, not the agent itself.\n- If a required input is missing (the agent's name as Agent Observability sees it, or `gcx` auth),\n  ask — don't guess.\n\n## Step 0 — Confirm the target stack\n\nBefore reading traffic or writing anything, show the developer where `gcx` is pointed. The active\ncontext may not be the stack they think:\n\n```\ngcx config current-context        # the active context name\ngcx config view                   # its server URL, org-id, auth method\n```\n\nDisplay the resolved **context name, server URL, and org-id**. Two thresholds:\n\n- **Before reads** (traffic sampling, inventory): show the resolved context so the developer sees\n  where the discovery ran. A wrong stack here wastes effort but doesn't change anything.\n- **Before any write** (Step 5): require an **explicit yes** that this is the intended production\n  stack. Writes are what create stack-level objects, so this confirmation is the hard gate.\n\nIf it's wrong, stop — the developer switches with `gcx config use-context \u003Cname>`, or you pass\n`--context \u003Cname>` on every `gcx` call. (Watch for `localhost` \u002F dev-looking servers — a strong\nsign the active context is not their prod stack.)\n\n## Step 1 — Read the code and sample real traffic\n\nTwo evidence sources. Do both; every later recommendation cites one of them.\n\n**Code** (as `agento11y-test-starter` Step 1). Find and record file:line for: the entrypoint, the\nsystem prompt, the tool\u002Ffunction definitions, and how it handles user data. This tells you what\n*could* go wrong. **The code is the authoritative source for the system prompt and tools** —\ncontent capture is often off in production, so the ingested traffic frequently has an empty\n`system_prompt` and may omit tool definitions. Never conclude \"this agent has no system prompt\"\nfrom the traffic; read it from the code.\n\n**Traffic**, via `gcx` — this tells you what *does* go wrong:\n\n1. Find the agent as Agent Observability sees it: `gcx agento11y agents list` (and `agents get` \u002F\n   `agents list-versions`) to get the exact `agent_name` — this is the `match.agent_name` you'll target. (Tip:\n   `agents list` prints a leading hint line before the JSON; set `GCX_AGENT_MODE=true` or skip\n   that line if you parse it.)\n2. Sample recent conversations: `gcx agento11y conversations search --filters 'agent = \"\u003Cname>\"'`\n   (add `status = \"error\"`, time windows, `tool.name`, `eval.passed = false`) and\n   `gcx agento11y generations get \u003Cid>` for detail. Look for long tool loops, over-refusals, PII\n   echoed back, off-topic drift, malformed outputs, error clusters.\n   - **Some agents have generations but no conversation** (e.g. single-shot agents whose spans\n     aren't grouped into a conversation). If `conversations search` is empty but `agents get` shows a\n     non-zero `generation_count`, don't stop and conclude \"no traffic, wrong skill\" — sample the\n     generations directly (`gcx agento11y generations get \u003Cid>`); there's plenty to work from.\n   - **Pick the selector from how the agent produces generations, not by habit.** The symptom to\n     recognize: a rule that **matches traffic but whose scores stay at zero** — it looks live but\n     silently never fires. That happens when `selector: user_visible_turn` is used on an agent whose\n     generations aren't user-facing turns: a **multi-agent DAG \u002F programmatic pipeline**\n     (fan-out\u002Ffan-in, internal nodes, one conversation per run), or a single-shot agent with no\n     conversation. (The underlying reason is that those generations carry no user-visible-turn flag,\n     but there's no `gcx` command to inspect that directly — diagnose from the zero-score symptom,\n     not by hunting for the field.) For these agents use **`selector: all_assistant_generations`**\n     and scope with `match.agent_name` to the node you care about. Reserve `user_visible_turn` for\n     genuine chat\u002Fassistant agents where a turn is what the user sees. To confirm a selector is\n     working, score over enough sampled traffic — either wait for the rule's `sample_rate` to\n     accumulate hits, or temporarily bump it (and drop it back afterward; `sample_rate: 1.0` costs\n     real judge money, see the rule rules below) — then check `conversations search`. A run that\n     scored nothing shows **`eval_summary` absent or `total_scores: 0`** (the field is omitted when\n     nothing scored, not returned as `0`); persistent zero on a matching agent almost always means\n     the selector is wrong for this agent shape.\n\n**Minimum evidence bar.** Aim for **≥20 recent conversations over ≥7 days** before drafting\nanything. Fewer than that and you risk overfitting one odd conversation into a production rule or\nguard: if the window is thin, either stop and say so, or proceed but mark every recommendation\n**low-confidence** and lean on non-intervening setups (guards in `warn`, low `sample_rate`) rather\nthan anything that blocks. A recommendation from a single conversation is a hypothesis, not a rule. If the\nagent has essentially no traffic, stop — this is the wrong skill; `agento11y-test-starter` (offline\nsuite) is the right one until traffic exists.\n\n## Step 2 — Inventory what already exists\n\nBefore proposing anything: `gcx agento11y evaluators list`, `gcx agento11y rules list`,\n`gcx agento11y guards list` (add `-o yaml` to see full definitions). For each concern you're about to\nraise, decide whether something already covers it — by **semantic equivalence, not id or name**.\nTwo objects are effectively the same when they share:\n\n- the same **surface** (both rules, or both guards),\n- the same **target** (overlapping `match`, especially `agent_name` \u002F `selector`),\n- the same **intent** (evaluator kind + what it checks, or the guard's policy),\n- the same **action** (rule scoring vs. guard `warn`\u002F`deny`\u002F`transform`\u002F`tool_filter`).\n\nIf an existing object matches on those, don't create a second one — say it's covered and stop, or\npropose an `update` to the existing one. A different id over identical intent+target+action is a\nduplicate, and duplicate guards\u002Frules double cost and can conflict.\n\n## Step 3 — Recommend rules and guards\n\nMap each observation to the surface that fits. **Online rules** *observe* (score, detect\nregressions, no user impact, **no agent code change** — the eval worker scores ingested traffic\nasynchronously); **guards** *intervene* (block\u002Fredact on the live request path, **and require a\ncode change in the agent to call them — see Step 5.5**). Pick the surface by whether you want to\nwatch or to stop — and remember a guard is dead config until the agent is wired to it.\n\nFor every online **rule** row below, pick the `selector` per Step 1 (agent shape), not by the\ntemplate's default — `user_visible_turn` for a genuine chat agent, `all_assistant_generations` for a\nDAG\u002Fpipeline node or single-shot agent.\n\n| If, in code or traffic, the agent… | Surface | Shape (prefer a predefined template) |\n| --- | --- | --- |\n| gives answers whose quality can drift | online **rule** | fork `template.helpfulness` \u002F `template.relevance` (`llm_judge`) |\n| does RAG \u002F cites sources | online **rule** | fork `template.groundedness` (`llm_judge`) |\n| must emit JSON \u002F a fixed shape | online **rule** | fork `template.json_valid` (`json_schema`) |\n| over-refuses or drifts off-topic | online **rule** | `regex` \u002F `llm_judge` on `all_assistant_generations` |\n| public-facing text | online **rule** | fork `template.toxicity` \u002F `template.pii` (`llm_judge`) |\n| receives PII\u002Fsecrets in its **input** (pasted into the prompt) | **guard** | `redact` guard, **preflight** — rewrites the request before the model sees it |\n| **generates** PII\u002Fsecrets\u002Fdestructive commands in its final **output** | **guard** | evaluator-backed detector (`regex` \u002F `llm_judge`), **postflight**, `warn` — don't rely on `redact` to scrub already-generated assistant text; detect + warn instead |\n| can call dangerous tools (shell, delete, write) | **guard** | `tool_filter` with `blocked_names` globs, postflight |\n| is subject to prompt-injection \u002F hard policy | **guard** | `llm_judge` evaluator; draft `warn`, later promotable to `deny` |\n\n> **Three guard shapes, all first-class:** `evaluator_ids` (an evaluator decides — most flexible),\n> `redact` (regex redaction), and `tool_filter` (block tool calls). Pick the one that fits the policy;\n> a redaction need is a `redact` guard, not a judge.\n>\n> **`redact` schema is exact — get it right or the create 400s.** The field is `redact` (the server's\n> canonical name; a legacy `transform` alias also works), and each pattern is **only `{id, regex}`** —\n> there is **no `replacement` key**. On a match the server redacts to a placeholder derived from the\n> pattern `id`; supplying `replacement` fails with `400 unknown field \"replacement\"`. Draft it as:\n>\n> ```yaml\n> redact:\n>   patterns:\n>     - id: bearer_token          # the id names the placeholder\n>       regex: 'Bearer\\s+[A-Za-z0-9._-]+'\n> ```\n>\n> After creating any guard, `guards get -o yaml` and confirm the spec stored what you sent (a clean\n> create echo is not proof on its own).\n\nTemplate ids above are the **expected** global blueprints — they can vary by deployment and\nversion, so always resolve the current set with `gcx agento11y templates list` before using a name;\ndon't trust a hardcoded id. Pick 3–6, ranked. Each gets a one-line `why` citing a **file:line or a\nconversation\u002Fgeneration id** from Step 1. For rule mechanics (selectors, match keys, evaluator\nkinds, templates), the `agento11y` skill is the reference — don't restate it here.\n\n**Every candidate lands in exactly one of three states — and the decision is final for this run:**\n\n- **Recommended** — worth setting up now; goes to Step 4 (draft) and Step 5 (apply).\n- **Considered, not recommended** — you evaluated it and it's not worth it (low value, no\n  evidence, would just add cost\u002Fnoise). Record it with a one-line why, and **do NOT draft or apply\n  it**. Don't quietly re-add it later under a different framing — if you're tempted to, it belongs\n  in Recommended, so put it there and own the reasoning. Bias toward fewer objects: recommend only\n  what earns its place. \"It's harmless in warn mode\" is not a reason to create something — an\n  unused guard\u002Frule is still cost and noise.\n- **Skipped (duplicate)** — the stack already covers it (Step 2 semantic-equivalence check);\n  don't create a second one.\n\nThe set you draft in Step 4 and apply in Step 5 is **exactly** the Recommended list — nothing\nfrom the other two states leaks in.\n\n## Step 4 — Draft the definitions as YAML\n\n**First, check whether `.\u002Fagento11y-prod\u002F` already has drafts from an earlier run** (`ls\nagento11y-prod\u002F**` if it exists). If it does, **read them before writing** — don't blind-overwrite\n(a plain `Write` over an existing file also just errors). Treat a prior draft as a peer proposal:\nreconcile rather than replace. If a previous run made a deliberate, well-reasoned choice — e.g.\ndropped an email-redaction pattern because \"this agent's job is to email people, so redacting\nevery address is all-false-positive noise\" — that judgment is usually right; keep it and fold in\nonly what's genuinely new. Overwrite a prior draft only when yours is clearly better, and say why.\n\nWrite the definitions to a **local scratch directory**, `.\u002Fagento11y-prod\u002F`\n(`evaluators\u002F\u003Cid>.yaml`, `rules\u002F\u003Cid>.yaml`, `guards\u002F\u003Cid>.yaml`). These are **working drafts, not\ncommitted artifacts**: they exist so the developer can review a diff before you apply it, and\ntheir source of truth after apply is the stack, not the repo. Add `agento11y-prod\u002F` to `.gitignore`\n(or write under the OS temp dir) so they aren't accidentally committed — they hold the applied\nconfig redundantly and can carry regexes\u002Fprompts the repo shouldn't own. They are exactly what\nyou'll pass to `gcx agento11y \u003Ckind> create -f` (for evaluators: `upsert -f`). Use the\n**top-level-fields** YAML shape that the `create -f`\u002F`upsert -f` commands expect (not the\n`apiVersion\u002Fkind\u002Fspec` manifest that the `get -o yaml` commands emit — don't round-trip get\noutput into create).\n\n**Rules and evaluators**: follow the `agento11y` skill's input format exactly. Start an evaluator\nfrom a template (`gcx agento11y templates get \u003Cid> -o yaml`), give it your own `evaluator_id`, and\n**always include a `version`** — it is required on create (a date like `\"2026-07-15\"` or a semver\nworks; existing evaluators use dates). Omitting it fails with `version is required`. Rule starts\nenabled at a low `sample_rate`.\n\n**Guards** — the shape the `agento11y` skill omits (`gcx agento11y guards create -f guard.yaml`; the\nresource `Kind` is `HookRule`).\n\nThis skill (not the `agento11y` skill) carries the guard shape — draft the guard file directly from\nit, no schema-discovery step needed. A guard drives its decision from one of three (mutually\nexclusive) shapes — `evaluator_ids`, `redact`, or `tool_filter` — plus `action_on_fail`, `phase`,\n`priority`, `selector`. Use the `redact` schema exactly as the callout above shows (`{id, regex}`,\nno `replacement`). If the server ever rejects a field, the 400 names the offending field, so a bad\nshape surfaces at create time rather than needing a probe up front. On create the server fills\ndefaults you don't set — notably `selector: all`, `phase: preflight`, and `short_circuit: false` —\nso a `guards get -o yaml` right after create\nshows more fields than you sent; that's expected, not drift. A guard is drafted in **warn** (and\n`enabled: false` if the server honors it — it may not; see Step 5):\n\n```yaml\n# PROD-SETUP DRAFT — creates a STACK-LEVEL guard (HookRule) via gcx agento11y guards create.\n# warn on purpose: a warn guard records but never blocks, so it is safe on real traffic\n# even while live; watch it, then switch to deny yourself later.\nrule_id: guard.\u003Cagent>.\u003Cpolicy>\nenabled: false         # server may store true anyway; harmless because action_on_fail is warn\npriority: 10           # lower runs first\naction_on_fail: warn   # always draft as warn (see below)\n# Pick exactly ONE decision shape:\n# (a) evaluator-backed — an evaluator (judge\u002Fregex) decides; most flexible.\nevaluator_ids: [\"\u003Cpolicy-judge-id>\"]  # create the evaluator first\n# (b) redact — regex redaction to a placeholder derived from `id`. Schema is {id, regex} ONLY;\n#     a `replacement` key 400s. Field name is `redact` (legacy alias: `transform`).\n# redact:\n#   patterns:\n#     - id: ssn\n#       regex: \"\\\\b\\\\d{3}-\\\\d{2}-\\\\d{4}\\\\b\"\n# (c) tool_filter — block tool calls by name glob.\n# tool_filter:\n#   blocked_names: [\"shell_exec\", \"Bash(*rm*)\"]\n```\n`phase` is `preflight` (default) or `postflight`, and **the correct phase depends on the guard\ntype** — getting this wrong yields a guard that runs but silently does nothing:\n\n- **`redact` → normally `preflight` (input).** Use it to redact sensitive values in the request\n  `input` (`messages`) **before the model sees them** — e.g. a secret pasted into the user prompt or\n  incident text. This is the common case and the one the Grafana UI defaults a Redact guard to.\n  **Do not rely on `redact` to rewrite already-generated assistant text:** for a secret the model\n  *produces* in its final response, a postflight redaction of the assistant text does not reliably\n  scrub it — use a postflight evaluator-backed **detector** (`warn`\u002F`deny`) instead (below).\n  (Postflight `redact` is not useless in general — some runtimes explicitly consume\n  `transformed_input.output` to rewrite **tool-call payloads\u002Farguments** postflight — but that is a\n  narrow tool-arg case, not final-response scrubbing.)\n  - **Disambiguation — decide by where the sensitive value ENTERS, not by where it's most visible.**\n    An agent can both *receive* a secret in its input AND *echo\u002Fgenerate* one in its output; don't\n    let the more eye-catching output occurrence pull you to postflight. If the secret enters through\n    the input (e.g. a key pasted into the prompt \u002F incident text \u002F a config blob), the guard that\n    actually protects it is **`redact` preflight on the input** — that scrubs it before the model,\n    the logs, and every downstream node see it. A postflight guard cannot undo a secret that already\n    entered upstream. Add a postflight **detector** on top only if the model *also* independently\n    produces secrets in its final text; that is a second, separate concern, not a replacement for the\n    preflight redact.\n- **`tool_filter` → `postflight`.** It inspects the tool calls the model wants to make.\n- **`evaluator_ids` (detector) → phase follows what you evaluate.** `preflight` to judge the\n  **input** (block a bad request before spending tokens), `postflight` to judge the **output**\n  (catch a bad response — e.g. detect that the model *generated* a credential or a destructive\n  command). This is the right shape for \"the model produces something risky,\" which `redact` cannot\n  catch.\n\n**Always draft `action_on_fail: warn`, even for\na hard-policy guard** (prompt-injection, deny-list): a first-time `deny` enabled on live traffic\nblocks real users on a false positive. A policy-judge guard references an evaluator id (create the\nevaluator first); the developer changes it to `deny` only later, after watching the false-positive\nrate in warn mode (Step 6). This skill never drafts an enabled `deny` guard.\n\n## Step 5 — Confirm, then apply with `gcx`\n\n> **`upsert`\u002F`create`\u002F`update` write to the stack — never run them before the developer's explicit yes\n> (step 2).** The one thing you CAN run before the yes is `evaluators test -f \u003Crequest>.yaml`,\n> which tests a judge config **without persisting it** (pass `kind`, `config`, `output_keys`,\n> `generation_id` in the file — no evaluator need exist yet). Use it to tune the judge (step 1).\n> There is **no CLI dry-run** for rules or guards — their safety comes from shipping guards in\n> `warn` (records, never blocks) and rules at a low `sample_rate`, not from a preview.\n\nPer object the developer wants, in dependency order (evaluators → rules → guards, since a\nrule\u002Fguard referencing an evaluator needs it to exist first):\n\n1. **For an `llm_judge` evaluator, tune the prompt before creating it — this is the real work,\n   not a formality.** A judge is only as good as its prompt, so don't create it on first draft and\n   move on. Loop: pick 1–2 real generations you know the right answer for\n   (`gcx agento11y generations get \u003Cid>`), run the draft config with\n   `gcx agento11y evaluators test -f \u003Crequest>.yaml -g \u003Cgen-id>` (tests without persisting), and\n   **read both the verdict AND the rationale**. If either disagrees with what you expected, adjust\n   the `system_prompt`\u002F`user_prompt` and re-test. Repeat until verdict and rationale both hold on\n   your known examples. Only then does the evaluator move to step 2. (This tuning loop deserves its\n   own dedicated flow; keep it lightweight here.)\n2. **Confirm.** Restate the target stack from Step 0 (context name + server), show the exact YAML,\n   and get an explicit yes for that object. A yes for one object is not a yes for the next. Nothing\n   is written before this yes.\n3. **Apply** via gcx, only after the yes: `gcx agento11y evaluators upsert -f evaluators\u002F\u003Cid>.yaml`,\n   then `gcx agento11y rules create -f rules\u002F\u003Cid>.yaml`, then\n   `gcx agento11y guards create -f guards\u002F\u003Cid>.yaml`. Evaluators are create-or-update (same id\n   updates). Pass `--context \u003Cname>` on every call if the confirmed stack isn't the default\n   context. gcx handles auth — no tokens here.\n   - **The server may store a new guard `enabled: true` \u002F `short_circuit: true` regardless of what\n     you drafted.** That is a known server default, not a draft error. Do not build a\n     create→get→update dance around it: because you drafted `action_on_fail: warn`, a live guard only\n     records outcomes and blocks no one, so `enabled: true` is harmless here. Create the guard and\n     move on. (What you must never create is an enabled `deny` guard — the `warn` rule above is what\n     prevents that, not the `enabled` value.)\n   - A judge-model 404 when testing\u002Fscoring is usually a stack-side misconfiguration (the stack's\n     judge model id is dead), not your evaluator — flag it; the online rule will hit the same broken\n     judge at runtime until it's fixed.\n4. If `gcx` reports it isn't authenticated, stop and ask the developer to run `gcx login`; do not\n   fall back to raw HTTP.\n\n## Step 5.5 — Wire the agent to call the guard (guards only, REQUIRED)\n\nCreating a guard on the stack does **not** make it do anything. Unlike online rules — which the\neval worker applies asynchronously to already-ingested traffic, with zero agent changes — a guard\nis a **synchronous request-path policy the agent must call itself**. If the agent never calls the\nhooks endpoint, the guard is inert: it exists, shows `enabled`, and never fires. That is also why\nStep 6 would show no `warn`s — nothing invoked the guard.\n\nSo every guard applied in Step 5 has a matching code change the developer owns. This skill does not\nedit or redeploy the agent (see Rules), but it MUST tell the developer exactly what to add, per\nguard, or the guard is dead config. Present this as part of the hand-off — don't leave the guard\nlooking \"done\" after Step 5.\n\nAt each LLM call the agent evaluates the guard via the SDK and honors the verdict. Minimal Python\n(`agento11y` >= 0.11):\n\n```python\nfrom agento11y import (\n    Client, ClientConfig, ApiConfig, HooksConfig,\n    HookEvaluateRequest, HookContext, HookModel, HookInput,\n    HookDeniedError, user_text_message,\n)\n\nclient = Client(config=ClientConfig(\n    api=ApiConfig(endpoint=\"https:\u002F\u002F\u003Cstack>.grafana.net\"),  # scheme+host; SDK appends the hooks path\n    hooks=HooksConfig(enabled=True, phases=[\"preflight\"], fail_open=False),\n))\n\n# preflight: evaluate the INPUT before the LLM call.\n# postflight: run it AFTER the call and pass the produced output instead.\nresp = client.evaluate_hook(HookEvaluateRequest(\n    phase=\"preflight\",\n    context=HookContext(\n        model=HookModel(provider=\"anthropic\", name=\"claude-...\"),\n        agent_name=\"\u003Cthe rule's match.agent_name>\",\n        agent_version=\"\u003Cv>\",\n        conversation_id=conversation_id,   # REQUIRED to record the guard on the conversation\n    ),\n    input=HookInput(messages=[user_text_message(prompt)]),   # postflight: output=[assistant_message]\n))\nif resp.is_deny:\n    raise HookDeniedError(reason=resp.reason, rule_id=resp.rule_id,\n                          evaluations=list(resp.evaluations))\n```\n\nThree gotchas that silently break guards — call each out to the developer:\n\n- **`fail_open=False`** — with the default `True`, a transport error (or a disabled\u002Fmissing guard)\n  resolves to `allow`, so a `deny` never actually blocks. Fail-closed is what makes the guard enforce.\n- **`conversation_id` in the context** — without it, a `deny`\u002F`warn` outcome is NOT persisted onto\n  the conversation, so it's invisible in the UI and Step 6 has nothing to watch. With it, the server\n  records a \"Guard: \u003Crule>\" workflow step on the conversation.\n- **`allow` leaves no trace** — the server persists only `deny` and `warn` outcomes; a clean pass\n  records nothing (just a metric). So \"no guard step on the conversation\" is ambiguous — it means\n  either allow OR not-wired. Distinguish them by whether ANY guard step ever appears for that agent.\n\n**Phase choice mirrors the guard** (see the guard-type rules in Step 4): `redact` is normally\n**preflight** (redact the input before the model sees it; don't use it to scrub already-generated\nassistant text — see Step 4 for the tool-arg exception); `tool_filter` is **postflight**.\nFor an evaluator-backed guard, phase follows `target`: `target: input` → **preflight** (evaluate\n`input.messages` before the call — block a bad request before spending tokens); `target: response`\n→ **postflight** (evaluate `input.output` after — catch a bad response the model produced). The\nrule's `match.agent_name` \u002F `match.model` scopes the guard to this agent; the agent's context must\nsend the same `agent_name`.\n\nOther SDKs (JS, Go) expose the same `evaluate_hook` \u002F hooks-config shape; check the per-language SDK\nreference and verify the exact symbols against the installed `agento11y` package. (As of writing the\ncanonical `llms.txt` does not yet carry a guard-instrumentation section, so don't defer to it for\nthis — the shape above is the reference.)\n\n## Step 6 — Summarize and hand off\n\nOutput, in this order:\n\n1. The three states from Step 3, kept distinct: **Recommended** (each with its surface, evaluator\n   kind, and `why` — file:line or conversation\u002Fgeneration id); **Considered, not recommended**\n   (each with its one-line why); **Skipped as duplicates** (what on the stack already covered it).\n   Don't move an item between states between Step 3 and here.\n2. What was created vs. left as a draft: for each **Recommended** object, the YAML path and whether\n   it was applied via gcx. (Considered-not-recommended items were never drafted, so they have no\n   path — don't list them here.)\n3. The follow-through the developer still owns:\n   - **Guards:** first, **wire the agent to call the guard (Step 5.5)** — until that code ships the\n     guard is inert and you'll see no `warn`s, no matter how it's configured. State this per guard,\n     with the exact call to add. Then watch the `warn` guards on real traffic and flip to `deny` +\n     `enabled` only once the false-positive rate looks acceptable —\n     `gcx agento11y guards update \u003Cid> -f ...`. Both the wiring and the flip are theirs to make, not\n     this skill's.\n   - **Rules:** first confirm the rule is actually scoring — if `eval_summary` is absent or\n     `total_scores` stays 0 on a matching agent, the `selector` is likely wrong for the agent shape\n     (see Step 1), not the sample rate. Once scores appear, raise `sample_rate` as they look sane and\n     cost is understood (`gcx agento11y rules update`); add alerting on regressions if wanted.\n   - Inspect everything in Agent Observability (rules\u002Fguards\u002Fevaluators pages, the conversation\n     Quality view) or via the `gcx agento11y` list and get commands.\n4. A one-line pointer back: for pre-ship offline evaluation of a new agent or version,\n   `agento11y-test-starter` is the counterpart; for control-plane mechanics, the `agento11y` skill.\n",{"data":35,"body":37},{"name":4,"description":6,"allowed-tools":36},"Bash, Read, Write, Edit",{"type":38,"children":39},"root",[40,49,72,137,152,179,198,230,242,313,333,339,626,632,644,656,668,698,734,740,745,784,807,1072,1117,1123,1165,1261,1274,1280,1326,1360,1754,1985,2026,2034,2074,2086,2092,2132,2248,2316,2355,2478,2734,2763,2998,3037,3048,3145,3150,3342,3348,3381,3386,3398,3621,3626,3729,3834,3862,3868,3873,4057],{"type":41,"tag":42,"props":43,"children":45},"element","h1",{"id":44},"agent-observability-production-evals-guards-setup",[46],{"type":47,"value":48},"text","Agent Observability — production evals & guards setup",{"type":41,"tag":50,"props":51,"children":52},"p",{},[53,55,62,64,70],{"type":47,"value":54},"The production counterpart to ",{"type":41,"tag":56,"props":57,"children":59},"code",{"className":58},[],[60],{"type":47,"value":61},"agento11y-test-starter",{"type":47,"value":63}," (which runs pre-ship, on code alone,\nproducing an offline test suite). This skill runs ",{"type":41,"tag":65,"props":66,"children":67},"strong",{},[68],{"type":47,"value":69},"after",{"type":47,"value":71}," ship, when the agent has real\ntraffic, and sets up the two production surfaces the starter deliberately leaves out:",{"type":41,"tag":73,"props":74,"children":75},"ul",{},[76,87],{"type":41,"tag":77,"props":78,"children":79},"li",{},[80,85],{"type":41,"tag":65,"props":81,"children":82},{},[83],{"type":47,"value":84},"Online eval rules",{"type":47,"value":86}," — evaluators that score ingested live conversations, so regressions\nsurface without hand-reviewing every conversation.",{"type":41,"tag":77,"props":88,"children":89},{},[90,95,97,103,105,111,113,119,121,127,129,135],{"type":41,"tag":65,"props":91,"children":92},{},[93],{"type":47,"value":94},"Guards",{"type":47,"value":96}," (hook-rules) — policies on the request path that ",{"type":41,"tag":56,"props":98,"children":100},{"className":99},[],[101],{"type":47,"value":102},"warn",{"type":47,"value":104}," (and can later be promoted to\n",{"type":41,"tag":56,"props":106,"children":108},{"className":107},[],[109],{"type":47,"value":110},"deny",{"type":47,"value":112},") in real time. A guard decides via one of three shapes: ",{"type":41,"tag":56,"props":114,"children":116},{"className":115},[],[117],{"type":47,"value":118},"evaluator_ids",{"type":47,"value":120}," (an evaluator\njudges), ",{"type":41,"tag":56,"props":122,"children":124},{"className":123},[],[125],{"type":47,"value":126},"redact",{"type":47,"value":128}," (regex redaction), or ",{"type":41,"tag":56,"props":130,"children":132},{"className":131},[],[133],{"type":47,"value":134},"tool_filter",{"type":47,"value":136}," (block tool calls). See Step 4.",{"type":41,"tag":138,"props":139,"children":141},"h2",{"id":140},"what-this-skill-does-that-agento11y-doesnt",[142,144,150],{"type":47,"value":143},"What this skill does that ",{"type":41,"tag":56,"props":145,"children":147},{"className":146},[],[148],{"type":47,"value":149},"agento11y",{"type":47,"value":151}," doesn't",{"type":41,"tag":50,"props":153,"children":154},{},[155,157,162,164,169,171,177],{"type":47,"value":156},"The sibling ",{"type":41,"tag":56,"props":158,"children":160},{"className":159},[],[161],{"type":47,"value":149},{"type":47,"value":163}," skill is the ",{"type":41,"tag":65,"props":165,"children":166},{},[167],{"type":47,"value":168},"mechanics",{"type":47,"value":170}," layer: exact CLI flags, evaluator\u002Frule YAML\nshapes, create-or-update semantics, the online-eval setup steps. It assumes you already know\n",{"type":41,"tag":172,"props":173,"children":174},"em",{},[175],{"type":47,"value":176},"what",{"type":47,"value":178}," to create.",{"type":41,"tag":50,"props":180,"children":181},{},[182,184,189,191,196],{"type":47,"value":183},"This skill is the ",{"type":41,"tag":65,"props":185,"children":186},{},[187],{"type":47,"value":188},"judgment",{"type":47,"value":190}," layer. It answers ",{"type":41,"tag":172,"props":192,"children":193},{},[194],{"type":47,"value":195},"which",{"type":47,"value":197}," rules and guards this specific agent\nneeds, by grounding in two evidence sources a generic checklist can't use:",{"type":41,"tag":199,"props":200,"children":201},"ol",{},[202,220],{"type":41,"tag":77,"props":203,"children":204},{},[205,210,212,218],{"type":41,"tag":65,"props":206,"children":207},{},[208],{"type":47,"value":209},"The agent's code",{"type":47,"value":211}," — its system prompt, tools, and how it handles user data. Half the value\nis here; read and cite it (",{"type":41,"tag":56,"props":213,"children":215},{"className":214},[],[216],{"type":47,"value":217},"file:line",{"type":47,"value":219},").",{"type":41,"tag":77,"props":221,"children":222},{},[223,228],{"type":41,"tag":65,"props":224,"children":225},{},[226],{"type":47,"value":227},"The agent's real traffic",{"type":47,"value":229}," — because it's deployed, you can see what it actually does in\nprod, not just what the code says it might.",{"type":41,"tag":50,"props":231,"children":232},{},[233,235,240],{"type":47,"value":234},"Two gaps this skill fills beyond ",{"type":41,"tag":56,"props":236,"children":238},{"className":237},[],[239],{"type":47,"value":149},{"type":47,"value":241},":",{"type":41,"tag":73,"props":243,"children":244},{},[245,262],{"type":41,"tag":77,"props":246,"children":247},{},[248,253,255,260],{"type":41,"tag":65,"props":249,"children":250},{},[251],{"type":47,"value":252},"Recommendation from evidence",{"type":47,"value":254}," — ",{"type":41,"tag":56,"props":256,"children":258},{"className":257},[],[259],{"type":47,"value":149},{"type":47,"value":261}," starts once you know what to create; this decides.",{"type":41,"tag":77,"props":263,"children":264},{},[265,269,270,275,277,283,285,290,292,297,298,303,305,311],{"type":41,"tag":65,"props":266,"children":267},{},[268],{"type":47,"value":94},{"type":47,"value":254},{"type":41,"tag":56,"props":271,"children":273},{"className":272},[],[274],{"type":47,"value":149},{"type":47,"value":276}," documents evaluators and rules but not guards (hook-rules), even though\n",{"type":41,"tag":56,"props":278,"children":280},{"className":279},[],[281],{"type":47,"value":282},"gcx agento11y guards",{"type":47,"value":284}," exists. This skill carries the guard shapes\n(",{"type":41,"tag":56,"props":286,"children":288},{"className":287},[],[289],{"type":47,"value":118},{"type":47,"value":291}," \u002F ",{"type":41,"tag":56,"props":293,"children":295},{"className":294},[],[296],{"type":47,"value":126},{"type":47,"value":291},{"type":41,"tag":56,"props":299,"children":301},{"className":300},[],[302],{"type":47,"value":134},{"type":47,"value":304},", plus ",{"type":41,"tag":56,"props":306,"children":308},{"className":307},[],[309],{"type":47,"value":310},"action_on_fail",{"type":47,"value":312},") itself.",{"type":41,"tag":50,"props":314,"children":315},{},[316,318,323,325,331],{"type":47,"value":317},"For any mechanical detail — exact flags, evaluator\u002Frule YAML fields, the setup flow — defer to\nthe ",{"type":41,"tag":56,"props":319,"children":321},{"className":320},[],[322],{"type":47,"value":149},{"type":47,"value":324}," skill and to ",{"type":41,"tag":56,"props":326,"children":328},{"className":327},[],[329],{"type":47,"value":330},"gcx agento11y \u003Csub> --help",{"type":47,"value":332}," rather than restating it here.",{"type":41,"tag":138,"props":334,"children":336},{"id":335},"rules",[337],{"type":47,"value":338},"Rules",{"type":41,"tag":73,"props":340,"children":341},{},[342,381,398,438,471,539,581,609,614],{"type":41,"tag":77,"props":343,"children":344},{},[345,347,356,358,364,366,372,374,379],{"type":47,"value":346},"Every connection to the stack goes through ",{"type":41,"tag":65,"props":348,"children":349},{},[350],{"type":41,"tag":56,"props":351,"children":353},{"className":352},[],[354],{"type":47,"value":355},"gcx agento11y",{"type":47,"value":357}," — never raw HTTP, never a hand-held\ntoken. ",{"type":41,"tag":56,"props":359,"children":361},{"className":360},[],[362],{"type":47,"value":363},"gcx",{"type":47,"value":365}," owns Cloud auth (via ",{"type":41,"tag":56,"props":367,"children":369},{"className":368},[],[370],{"type":47,"value":371},"gcx login",{"type":47,"value":373},"). Prerequisite: ",{"type":41,"tag":56,"props":375,"children":377},{"className":376},[],[378],{"type":47,"value":363},{"type":47,"value":380}," installed and authenticated;\nif it isn't, say so and stop.",{"type":41,"tag":77,"props":382,"children":383},{},[384,389,391,396],{"type":41,"tag":65,"props":385,"children":386},{},[387],{"type":47,"value":388},"Confirm the target stack before any WRITE (Step 0 + Step 5).",{"type":47,"value":390}," Reads run freely once you've\nshown the context; writes (upsert evaluators, create\u002Fupdate rules and guards) need an explicit yes on the\ntarget stack. ",{"type":41,"tag":56,"props":392,"children":394},{"className":393},[],[395],{"type":47,"value":363},{"type":47,"value":397}," may be pointed at the wrong stack, and this skill creates stack-level\nobjects.",{"type":41,"tag":77,"props":399,"children":400},{},[401,406,408,414,416,422,423,429,431,436],{"type":41,"tag":65,"props":402,"children":403},{},[404],{"type":47,"value":405},"Check before recommending.",{"type":47,"value":407}," Always list what already exists first\n(",{"type":41,"tag":56,"props":409,"children":411},{"className":410},[],[412],{"type":47,"value":413},"gcx agento11y evaluators list",{"type":47,"value":415},", ",{"type":41,"tag":56,"props":417,"children":419},{"className":418},[],[420],{"type":47,"value":421},"rules list",{"type":47,"value":415},{"type":41,"tag":56,"props":424,"children":426},{"className":425},[],[427],{"type":47,"value":428},"guards list",{"type":47,"value":430},") and never recommend a duplicate.\nCompare by ",{"type":41,"tag":65,"props":432,"children":433},{},[434],{"type":47,"value":435},"semantic equivalence",{"type":47,"value":437},", not just id\u002Fname — see Step 2.",{"type":41,"tag":77,"props":439,"children":440},{},[441,443,448,450,455,457,462,464,469],{"type":47,"value":442},"This skill ",{"type":41,"tag":65,"props":444,"children":445},{},[446],{"type":47,"value":447},"does",{"type":47,"value":449}," create stack-level objects — that is its job, the one thing that separates\nit from ",{"type":41,"tag":56,"props":451,"children":453},{"className":452},[],[454],{"type":47,"value":61},{"type":47,"value":456},". But every creation is ",{"type":41,"tag":65,"props":458,"children":459},{},[460],{"type":47,"value":461},"explicit and confirmed",{"type":47,"value":463},": show the exact\nYAML, get a yes, then create it with the matching ",{"type":41,"tag":56,"props":465,"children":467},{"className":466},[],[468],{"type":47,"value":355},{"type":47,"value":470}," command. A yes for one object\nis not a yes for the next.",{"type":41,"tag":77,"props":472,"children":473},{},[474,476,485,487,492,494,499,501,506,508,514,516,522,524,529,531,537],{"type":47,"value":475},"New guards are always drafted ",{"type":41,"tag":65,"props":477,"children":478},{},[479],{"type":41,"tag":56,"props":480,"children":482},{"className":481},[],[483],{"type":47,"value":484},"action_on_fail: \"warn\"",{"type":47,"value":486}," — even hard-policy ones. That is what\nmakes a new guard safe: a ",{"type":41,"tag":56,"props":488,"children":490},{"className":489},[],[491],{"type":47,"value":102},{"type":47,"value":493}," guard only records the outcome, it never blocks a request, so it\nis harmless even while it is live. Never draft a first-time ",{"type":41,"tag":56,"props":495,"children":497},{"className":496},[],[498],{"type":47,"value":110},{"type":47,"value":500}," guard. The developer switches to\n",{"type":41,"tag":56,"props":502,"children":504},{"className":503},[],[505],{"type":47,"value":110},{"type":47,"value":507}," themselves, later, after watching it in warn mode (Step 6). Draft ",{"type":41,"tag":56,"props":509,"children":511},{"className":510},[],[512],{"type":47,"value":513},"enabled: false",{"type":47,"value":515}," if you\ncan, but don't build extra steps around it — the server may store the guard ",{"type":41,"tag":56,"props":517,"children":519},{"className":518},[],[520],{"type":47,"value":521},"enabled: true",{"type":47,"value":523},"\nregardless (see Step 5), and that is fine, because ",{"type":41,"tag":56,"props":525,"children":527},{"className":526},[],[528],{"type":47,"value":102},{"type":47,"value":530}," carries the safety, not ",{"type":41,"tag":56,"props":532,"children":534},{"className":533},[],[535],{"type":47,"value":536},"enabled",{"type":47,"value":538},".",{"type":41,"tag":77,"props":540,"children":541},{},[542,544,555,557,563,565,571,573,579],{"type":47,"value":543},"New online rules start with a ",{"type":41,"tag":65,"props":545,"children":546},{},[547,549],{"type":47,"value":548},"conservative ",{"type":41,"tag":56,"props":550,"children":552},{"className":551},[],[553],{"type":47,"value":554},"sample_rate",{"type":47,"value":556}," (e.g. ",{"type":41,"tag":56,"props":558,"children":560},{"className":559},[],[561],{"type":47,"value":562},"0.1",{"type":47,"value":564},"), not ",{"type":41,"tag":56,"props":566,"children":568},{"className":567},[],[569],{"type":47,"value":570},"1.0",{"type":47,"value":572}," — an\n",{"type":41,"tag":56,"props":574,"children":576},{"className":575},[],[577],{"type":47,"value":578},"llm_judge",{"type":47,"value":580}," over 100% of traffic costs real money.",{"type":41,"tag":77,"props":582,"children":583},{},[584,586,591,593,599,601,607],{"type":47,"value":585},"Prefer ",{"type":41,"tag":65,"props":587,"children":588},{},[589],{"type":47,"value":590},"starting from an evaluator template",{"type":47,"value":592}," (",{"type":41,"tag":56,"props":594,"children":596},{"className":595},[],[597],{"type":47,"value":598},"gcx agento11y templates list",{"type":47,"value":600},", then\n",{"type":41,"tag":56,"props":602,"children":604},{"className":603},[],[605],{"type":47,"value":606},"gcx agento11y templates get",{"type":47,"value":608},") over authoring\na new evaluator. Only write a fresh one when nothing fits.",{"type":41,"tag":77,"props":610,"children":611},{},[612],{"type":47,"value":613},"Do not rewrite the agent's prompt, optimize, or redeploy. This skill configures observation and\nguardrails around the agent, not the agent itself.",{"type":41,"tag":77,"props":615,"children":616},{},[617,619,624],{"type":47,"value":618},"If a required input is missing (the agent's name as Agent Observability sees it, or ",{"type":41,"tag":56,"props":620,"children":622},{"className":621},[],[623],{"type":47,"value":363},{"type":47,"value":625}," auth),\nask — don't guess.",{"type":41,"tag":138,"props":627,"children":629},{"id":628},"step-0-confirm-the-target-stack",[630],{"type":47,"value":631},"Step 0 — Confirm the target stack",{"type":41,"tag":50,"props":633,"children":634},{},[635,637,642],{"type":47,"value":636},"Before reading traffic or writing anything, show the developer where ",{"type":41,"tag":56,"props":638,"children":640},{"className":639},[],[641],{"type":47,"value":363},{"type":47,"value":643}," is pointed. The active\ncontext may not be the stack they think:",{"type":41,"tag":645,"props":646,"children":650},"pre",{"className":647,"code":649,"language":47},[648],"language-text","gcx config current-context        # the active context name\ngcx config view                   # its server URL, org-id, auth method\n",[651],{"type":41,"tag":56,"props":652,"children":654},{"__ignoreMap":653},"",[655],{"type":47,"value":649},{"type":41,"tag":50,"props":657,"children":658},{},[659,661,666],{"type":47,"value":660},"Display the resolved ",{"type":41,"tag":65,"props":662,"children":663},{},[664],{"type":47,"value":665},"context name, server URL, and org-id",{"type":47,"value":667},". Two thresholds:",{"type":41,"tag":73,"props":669,"children":670},{},[671,681],{"type":41,"tag":77,"props":672,"children":673},{},[674,679],{"type":41,"tag":65,"props":675,"children":676},{},[677],{"type":47,"value":678},"Before reads",{"type":47,"value":680}," (traffic sampling, inventory): show the resolved context so the developer sees\nwhere the discovery ran. A wrong stack here wastes effort but doesn't change anything.",{"type":41,"tag":77,"props":682,"children":683},{},[684,689,691,696],{"type":41,"tag":65,"props":685,"children":686},{},[687],{"type":47,"value":688},"Before any write",{"type":47,"value":690}," (Step 5): require an ",{"type":41,"tag":65,"props":692,"children":693},{},[694],{"type":47,"value":695},"explicit yes",{"type":47,"value":697}," that this is the intended production\nstack. Writes are what create stack-level objects, so this confirmation is the hard gate.",{"type":41,"tag":50,"props":699,"children":700},{},[701,703,709,711,717,719,724,726,732],{"type":47,"value":702},"If it's wrong, stop — the developer switches with ",{"type":41,"tag":56,"props":704,"children":706},{"className":705},[],[707],{"type":47,"value":708},"gcx config use-context \u003Cname>",{"type":47,"value":710},", or you pass\n",{"type":41,"tag":56,"props":712,"children":714},{"className":713},[],[715],{"type":47,"value":716},"--context \u003Cname>",{"type":47,"value":718}," on every ",{"type":41,"tag":56,"props":720,"children":722},{"className":721},[],[723],{"type":47,"value":363},{"type":47,"value":725}," call. (Watch for ",{"type":41,"tag":56,"props":727,"children":729},{"className":728},[],[730],{"type":47,"value":731},"localhost",{"type":47,"value":733}," \u002F dev-looking servers — a strong\nsign the active context is not their prod stack.)",{"type":41,"tag":138,"props":735,"children":737},{"id":736},"step-1-read-the-code-and-sample-real-traffic",[738],{"type":47,"value":739},"Step 1 — Read the code and sample real traffic",{"type":41,"tag":50,"props":741,"children":742},{},[743],{"type":47,"value":744},"Two evidence sources. Do both; every later recommendation cites one of them.",{"type":41,"tag":50,"props":746,"children":747},{},[748,753,755,760,762,767,769,774,776,782],{"type":41,"tag":65,"props":749,"children":750},{},[751],{"type":47,"value":752},"Code",{"type":47,"value":754}," (as ",{"type":41,"tag":56,"props":756,"children":758},{"className":757},[],[759],{"type":47,"value":61},{"type":47,"value":761}," Step 1). Find and record file:line for: the entrypoint, the\nsystem prompt, the tool\u002Ffunction definitions, and how it handles user data. This tells you what\n",{"type":41,"tag":172,"props":763,"children":764},{},[765],{"type":47,"value":766},"could",{"type":47,"value":768}," go wrong. ",{"type":41,"tag":65,"props":770,"children":771},{},[772],{"type":47,"value":773},"The code is the authoritative source for the system prompt and tools",{"type":47,"value":775}," —\ncontent capture is often off in production, so the ingested traffic frequently has an empty\n",{"type":41,"tag":56,"props":777,"children":779},{"className":778},[],[780],{"type":47,"value":781},"system_prompt",{"type":47,"value":783}," and may omit tool definitions. Never conclude \"this agent has no system prompt\"\nfrom the traffic; read it from the code.",{"type":41,"tag":50,"props":785,"children":786},{},[787,792,794,799,801,805],{"type":41,"tag":65,"props":788,"children":789},{},[790],{"type":47,"value":791},"Traffic",{"type":47,"value":793},", via ",{"type":41,"tag":56,"props":795,"children":797},{"className":796},[],[798],{"type":47,"value":363},{"type":47,"value":800}," — this tells you what ",{"type":41,"tag":172,"props":802,"children":803},{},[804],{"type":47,"value":447},{"type":47,"value":806}," go wrong:",{"type":41,"tag":199,"props":808,"children":809},{},[810,871],{"type":41,"tag":77,"props":811,"children":812},{},[813,815,821,823,829,831,837,839,845,847,853,855,861,863,869],{"type":47,"value":814},"Find the agent as Agent Observability sees it: ",{"type":41,"tag":56,"props":816,"children":818},{"className":817},[],[819],{"type":47,"value":820},"gcx agento11y agents list",{"type":47,"value":822}," (and ",{"type":41,"tag":56,"props":824,"children":826},{"className":825},[],[827],{"type":47,"value":828},"agents get",{"type":47,"value":830}," \u002F\n",{"type":41,"tag":56,"props":832,"children":834},{"className":833},[],[835],{"type":47,"value":836},"agents list-versions",{"type":47,"value":838},") to get the exact ",{"type":41,"tag":56,"props":840,"children":842},{"className":841},[],[843],{"type":47,"value":844},"agent_name",{"type":47,"value":846}," — this is the ",{"type":41,"tag":56,"props":848,"children":850},{"className":849},[],[851],{"type":47,"value":852},"match.agent_name",{"type":47,"value":854}," you'll target. (Tip:\n",{"type":41,"tag":56,"props":856,"children":858},{"className":857},[],[859],{"type":47,"value":860},"agents list",{"type":47,"value":862}," prints a leading hint line before the JSON; set ",{"type":41,"tag":56,"props":864,"children":866},{"className":865},[],[867],{"type":47,"value":868},"GCX_AGENT_MODE=true",{"type":47,"value":870}," or skip\nthat line if you parse it.)",{"type":41,"tag":77,"props":872,"children":873},{},[874,876,882,884,890,892,898,899,905,907,913,915],{"type":47,"value":875},"Sample recent conversations: ",{"type":41,"tag":56,"props":877,"children":879},{"className":878},[],[880],{"type":47,"value":881},"gcx agento11y conversations search --filters 'agent = \"\u003Cname>\"'",{"type":47,"value":883},"\n(add ",{"type":41,"tag":56,"props":885,"children":887},{"className":886},[],[888],{"type":47,"value":889},"status = \"error\"",{"type":47,"value":891},", time windows, ",{"type":41,"tag":56,"props":893,"children":895},{"className":894},[],[896],{"type":47,"value":897},"tool.name",{"type":47,"value":415},{"type":41,"tag":56,"props":900,"children":902},{"className":901},[],[903],{"type":47,"value":904},"eval.passed = false",{"type":47,"value":906},") and\n",{"type":41,"tag":56,"props":908,"children":910},{"className":909},[],[911],{"type":47,"value":912},"gcx agento11y generations get \u003Cid>",{"type":47,"value":914}," for detail. Look for long tool loops, over-refusals, PII\nechoed back, off-topic drift, malformed outputs, error clusters.\n",{"type":41,"tag":73,"props":916,"children":917},{},[918,958],{"type":41,"tag":77,"props":919,"children":920},{},[921,926,928,934,936,941,943,949,951,956],{"type":41,"tag":65,"props":922,"children":923},{},[924],{"type":47,"value":925},"Some agents have generations but no conversation",{"type":47,"value":927}," (e.g. single-shot agents whose spans\naren't grouped into a conversation). If ",{"type":41,"tag":56,"props":929,"children":931},{"className":930},[],[932],{"type":47,"value":933},"conversations search",{"type":47,"value":935}," is empty but ",{"type":41,"tag":56,"props":937,"children":939},{"className":938},[],[940],{"type":47,"value":828},{"type":47,"value":942}," shows a\nnon-zero ",{"type":41,"tag":56,"props":944,"children":946},{"className":945},[],[947],{"type":47,"value":948},"generation_count",{"type":47,"value":950},", don't stop and conclude \"no traffic, wrong skill\" — sample the\ngenerations directly (",{"type":41,"tag":56,"props":952,"children":954},{"className":953},[],[955],{"type":47,"value":912},{"type":47,"value":957},"); there's plenty to work from.",{"type":41,"tag":77,"props":959,"children":960},{},[961,966,968,973,975,981,983,988,990,995,997,1006,1008,1013,1015,1021,1023,1028,1030,1036,1038,1043,1045,1062,1064,1070],{"type":41,"tag":65,"props":962,"children":963},{},[964],{"type":47,"value":965},"Pick the selector from how the agent produces generations, not by habit.",{"type":47,"value":967}," The symptom to\nrecognize: a rule that ",{"type":41,"tag":65,"props":969,"children":970},{},[971],{"type":47,"value":972},"matches traffic but whose scores stay at zero",{"type":47,"value":974}," — it looks live but\nsilently never fires. That happens when ",{"type":41,"tag":56,"props":976,"children":978},{"className":977},[],[979],{"type":47,"value":980},"selector: user_visible_turn",{"type":47,"value":982}," is used on an agent whose\ngenerations aren't user-facing turns: a ",{"type":41,"tag":65,"props":984,"children":985},{},[986],{"type":47,"value":987},"multi-agent DAG \u002F programmatic pipeline",{"type":47,"value":989},"\n(fan-out\u002Ffan-in, internal nodes, one conversation per run), or a single-shot agent with no\nconversation. (The underlying reason is that those generations carry no user-visible-turn flag,\nbut there's no ",{"type":41,"tag":56,"props":991,"children":993},{"className":992},[],[994],{"type":47,"value":363},{"type":47,"value":996}," command to inspect that directly — diagnose from the zero-score symptom,\nnot by hunting for the field.) For these agents use ",{"type":41,"tag":65,"props":998,"children":999},{},[1000],{"type":41,"tag":56,"props":1001,"children":1003},{"className":1002},[],[1004],{"type":47,"value":1005},"selector: all_assistant_generations",{"type":47,"value":1007},"\nand scope with ",{"type":41,"tag":56,"props":1009,"children":1011},{"className":1010},[],[1012],{"type":47,"value":852},{"type":47,"value":1014}," to the node you care about. Reserve ",{"type":41,"tag":56,"props":1016,"children":1018},{"className":1017},[],[1019],{"type":47,"value":1020},"user_visible_turn",{"type":47,"value":1022}," for\ngenuine chat\u002Fassistant agents where a turn is what the user sees. To confirm a selector is\nworking, score over enough sampled traffic — either wait for the rule's ",{"type":41,"tag":56,"props":1024,"children":1026},{"className":1025},[],[1027],{"type":47,"value":554},{"type":47,"value":1029}," to\naccumulate hits, or temporarily bump it (and drop it back afterward; ",{"type":41,"tag":56,"props":1031,"children":1033},{"className":1032},[],[1034],{"type":47,"value":1035},"sample_rate: 1.0",{"type":47,"value":1037}," costs\nreal judge money, see the rule rules below) — then check ",{"type":41,"tag":56,"props":1039,"children":1041},{"className":1040},[],[1042],{"type":47,"value":933},{"type":47,"value":1044},". A run that\nscored nothing shows ",{"type":41,"tag":65,"props":1046,"children":1047},{},[1048,1054,1056],{"type":41,"tag":56,"props":1049,"children":1051},{"className":1050},[],[1052],{"type":47,"value":1053},"eval_summary",{"type":47,"value":1055}," absent or ",{"type":41,"tag":56,"props":1057,"children":1059},{"className":1058},[],[1060],{"type":47,"value":1061},"total_scores: 0",{"type":47,"value":1063}," (the field is omitted when\nnothing scored, not returned as ",{"type":41,"tag":56,"props":1065,"children":1067},{"className":1066},[],[1068],{"type":47,"value":1069},"0",{"type":47,"value":1071},"); persistent zero on a matching agent almost always means\nthe selector is wrong for this agent shape.",{"type":41,"tag":50,"props":1073,"children":1074},{},[1075,1080,1082,1087,1089,1094,1096,1101,1103,1108,1110,1115],{"type":41,"tag":65,"props":1076,"children":1077},{},[1078],{"type":47,"value":1079},"Minimum evidence bar.",{"type":47,"value":1081}," Aim for ",{"type":41,"tag":65,"props":1083,"children":1084},{},[1085],{"type":47,"value":1086},"≥20 recent conversations over ≥7 days",{"type":47,"value":1088}," before drafting\nanything. Fewer than that and you risk overfitting one odd conversation into a production rule or\nguard: if the window is thin, either stop and say so, or proceed but mark every recommendation\n",{"type":41,"tag":65,"props":1090,"children":1091},{},[1092],{"type":47,"value":1093},"low-confidence",{"type":47,"value":1095}," and lean on non-intervening setups (guards in ",{"type":41,"tag":56,"props":1097,"children":1099},{"className":1098},[],[1100],{"type":47,"value":102},{"type":47,"value":1102},", low ",{"type":41,"tag":56,"props":1104,"children":1106},{"className":1105},[],[1107],{"type":47,"value":554},{"type":47,"value":1109},") rather\nthan anything that blocks. A recommendation from a single conversation is a hypothesis, not a rule. If the\nagent has essentially no traffic, stop — this is the wrong skill; ",{"type":41,"tag":56,"props":1111,"children":1113},{"className":1112},[],[1114],{"type":47,"value":61},{"type":47,"value":1116}," (offline\nsuite) is the right one until traffic exists.",{"type":41,"tag":138,"props":1118,"children":1120},{"id":1119},"step-2-inventory-what-already-exists",[1121],{"type":47,"value":1122},"Step 2 — Inventory what already exists",{"type":41,"tag":50,"props":1124,"children":1125},{},[1126,1128,1133,1134,1140,1142,1148,1150,1156,1158,1163],{"type":47,"value":1127},"Before proposing anything: ",{"type":41,"tag":56,"props":1129,"children":1131},{"className":1130},[],[1132],{"type":47,"value":413},{"type":47,"value":415},{"type":41,"tag":56,"props":1135,"children":1137},{"className":1136},[],[1138],{"type":47,"value":1139},"gcx agento11y rules list",{"type":47,"value":1141},",\n",{"type":41,"tag":56,"props":1143,"children":1145},{"className":1144},[],[1146],{"type":47,"value":1147},"gcx agento11y guards list",{"type":47,"value":1149}," (add ",{"type":41,"tag":56,"props":1151,"children":1153},{"className":1152},[],[1154],{"type":47,"value":1155},"-o yaml",{"type":47,"value":1157}," to see full definitions). For each concern you're about to\nraise, decide whether something already covers it — by ",{"type":41,"tag":65,"props":1159,"children":1160},{},[1161],{"type":47,"value":1162},"semantic equivalence, not id or name",{"type":47,"value":1164},".\nTwo objects are effectively the same when they share:",{"type":41,"tag":73,"props":1166,"children":1167},{},[1168,1180,1213,1224],{"type":41,"tag":77,"props":1169,"children":1170},{},[1171,1173,1178],{"type":47,"value":1172},"the same ",{"type":41,"tag":65,"props":1174,"children":1175},{},[1176],{"type":47,"value":1177},"surface",{"type":47,"value":1179}," (both rules, or both guards),",{"type":41,"tag":77,"props":1181,"children":1182},{},[1183,1184,1189,1191,1197,1199,1204,1205,1211],{"type":47,"value":1172},{"type":41,"tag":65,"props":1185,"children":1186},{},[1187],{"type":47,"value":1188},"target",{"type":47,"value":1190}," (overlapping ",{"type":41,"tag":56,"props":1192,"children":1194},{"className":1193},[],[1195],{"type":47,"value":1196},"match",{"type":47,"value":1198},", especially ",{"type":41,"tag":56,"props":1200,"children":1202},{"className":1201},[],[1203],{"type":47,"value":844},{"type":47,"value":291},{"type":41,"tag":56,"props":1206,"children":1208},{"className":1207},[],[1209],{"type":47,"value":1210},"selector",{"type":47,"value":1212},"),",{"type":41,"tag":77,"props":1214,"children":1215},{},[1216,1217,1222],{"type":47,"value":1172},{"type":41,"tag":65,"props":1218,"children":1219},{},[1220],{"type":47,"value":1221},"intent",{"type":47,"value":1223}," (evaluator kind + what it checks, or the guard's policy),",{"type":41,"tag":77,"props":1225,"children":1226},{},[1227,1228,1233,1235,1240,1242,1247,1248,1254,1255,1260],{"type":47,"value":1172},{"type":41,"tag":65,"props":1229,"children":1230},{},[1231],{"type":47,"value":1232},"action",{"type":47,"value":1234}," (rule scoring vs. guard ",{"type":41,"tag":56,"props":1236,"children":1238},{"className":1237},[],[1239],{"type":47,"value":102},{"type":47,"value":1241},"\u002F",{"type":41,"tag":56,"props":1243,"children":1245},{"className":1244},[],[1246],{"type":47,"value":110},{"type":47,"value":1241},{"type":41,"tag":56,"props":1249,"children":1251},{"className":1250},[],[1252],{"type":47,"value":1253},"transform",{"type":47,"value":1241},{"type":41,"tag":56,"props":1256,"children":1258},{"className":1257},[],[1259],{"type":47,"value":134},{"type":47,"value":219},{"type":41,"tag":50,"props":1262,"children":1263},{},[1264,1266,1272],{"type":47,"value":1265},"If an existing object matches on those, don't create a second one — say it's covered and stop, or\npropose an ",{"type":41,"tag":56,"props":1267,"children":1269},{"className":1268},[],[1270],{"type":47,"value":1271},"update",{"type":47,"value":1273}," to the existing one. A different id over identical intent+target+action is a\nduplicate, and duplicate guards\u002Frules double cost and can conflict.",{"type":41,"tag":138,"props":1275,"children":1277},{"id":1276},"step-3-recommend-rules-and-guards",[1278],{"type":47,"value":1279},"Step 3 — Recommend rules and guards",{"type":41,"tag":50,"props":1281,"children":1282},{},[1283,1285,1290,1292,1297,1299,1304,1306,1311,1312,1317,1319,1324],{"type":47,"value":1284},"Map each observation to the surface that fits. ",{"type":41,"tag":65,"props":1286,"children":1287},{},[1288],{"type":47,"value":1289},"Online rules",{"type":47,"value":1291}," ",{"type":41,"tag":172,"props":1293,"children":1294},{},[1295],{"type":47,"value":1296},"observe",{"type":47,"value":1298}," (score, detect\nregressions, no user impact, ",{"type":41,"tag":65,"props":1300,"children":1301},{},[1302],{"type":47,"value":1303},"no agent code change",{"type":47,"value":1305}," — the eval worker scores ingested traffic\nasynchronously); ",{"type":41,"tag":65,"props":1307,"children":1308},{},[1309],{"type":47,"value":1310},"guards",{"type":47,"value":1291},{"type":41,"tag":172,"props":1313,"children":1314},{},[1315],{"type":47,"value":1316},"intervene",{"type":47,"value":1318}," (block\u002Fredact on the live request path, ",{"type":41,"tag":65,"props":1320,"children":1321},{},[1322],{"type":47,"value":1323},"and require a\ncode change in the agent to call them — see Step 5.5",{"type":47,"value":1325},"). Pick the surface by whether you want to\nwatch or to stop — and remember a guard is dead config until the agent is wired to it.",{"type":41,"tag":50,"props":1327,"children":1328},{},[1329,1331,1336,1338,1343,1345,1350,1352,1358],{"type":47,"value":1330},"For every online ",{"type":41,"tag":65,"props":1332,"children":1333},{},[1334],{"type":47,"value":1335},"rule",{"type":47,"value":1337}," row below, pick the ",{"type":41,"tag":56,"props":1339,"children":1341},{"className":1340},[],[1342],{"type":47,"value":1210},{"type":47,"value":1344}," per Step 1 (agent shape), not by the\ntemplate's default — ",{"type":41,"tag":56,"props":1346,"children":1348},{"className":1347},[],[1349],{"type":47,"value":1020},{"type":47,"value":1351}," for a genuine chat agent, ",{"type":41,"tag":56,"props":1353,"children":1355},{"className":1354},[],[1356],{"type":47,"value":1357},"all_assistant_generations",{"type":47,"value":1359}," for a\nDAG\u002Fpipeline node or single-shot agent.",{"type":41,"tag":1361,"props":1362,"children":1363},"table",{},[1364,1388],{"type":41,"tag":1365,"props":1366,"children":1367},"thead",{},[1368],{"type":41,"tag":1369,"props":1370,"children":1371},"tr",{},[1372,1378,1383],{"type":41,"tag":1373,"props":1374,"children":1375},"th",{},[1376],{"type":47,"value":1377},"If, in code or traffic, the agent…",{"type":41,"tag":1373,"props":1379,"children":1380},{},[1381],{"type":47,"value":1382},"Surface",{"type":41,"tag":1373,"props":1384,"children":1385},{},[1386],{"type":47,"value":1387},"Shape (prefer a predefined template)",{"type":41,"tag":1389,"props":1390,"children":1391},"tbody",{},[1392,1436,1469,1503,1541,1581,1621,1684,1717],{"type":41,"tag":1369,"props":1393,"children":1394},{},[1395,1401,1410],{"type":41,"tag":1396,"props":1397,"children":1398},"td",{},[1399],{"type":47,"value":1400},"gives answers whose quality can drift",{"type":41,"tag":1396,"props":1402,"children":1403},{},[1404,1406],{"type":47,"value":1405},"online ",{"type":41,"tag":65,"props":1407,"children":1408},{},[1409],{"type":47,"value":1335},{"type":41,"tag":1396,"props":1411,"children":1412},{},[1413,1415,1421,1422,1428,1429,1434],{"type":47,"value":1414},"fork ",{"type":41,"tag":56,"props":1416,"children":1418},{"className":1417},[],[1419],{"type":47,"value":1420},"template.helpfulness",{"type":47,"value":291},{"type":41,"tag":56,"props":1423,"children":1425},{"className":1424},[],[1426],{"type":47,"value":1427},"template.relevance",{"type":47,"value":592},{"type":41,"tag":56,"props":1430,"children":1432},{"className":1431},[],[1433],{"type":47,"value":578},{"type":47,"value":1435},")",{"type":41,"tag":1369,"props":1437,"children":1438},{},[1439,1444,1452],{"type":41,"tag":1396,"props":1440,"children":1441},{},[1442],{"type":47,"value":1443},"does RAG \u002F cites sources",{"type":41,"tag":1396,"props":1445,"children":1446},{},[1447,1448],{"type":47,"value":1405},{"type":41,"tag":65,"props":1449,"children":1450},{},[1451],{"type":47,"value":1335},{"type":41,"tag":1396,"props":1453,"children":1454},{},[1455,1456,1462,1463,1468],{"type":47,"value":1414},{"type":41,"tag":56,"props":1457,"children":1459},{"className":1458},[],[1460],{"type":47,"value":1461},"template.groundedness",{"type":47,"value":592},{"type":41,"tag":56,"props":1464,"children":1466},{"className":1465},[],[1467],{"type":47,"value":578},{"type":47,"value":1435},{"type":41,"tag":1369,"props":1470,"children":1471},{},[1472,1477,1485],{"type":41,"tag":1396,"props":1473,"children":1474},{},[1475],{"type":47,"value":1476},"must emit JSON \u002F a fixed shape",{"type":41,"tag":1396,"props":1478,"children":1479},{},[1480,1481],{"type":47,"value":1405},{"type":41,"tag":65,"props":1482,"children":1483},{},[1484],{"type":47,"value":1335},{"type":41,"tag":1396,"props":1486,"children":1487},{},[1488,1489,1495,1496,1502],{"type":47,"value":1414},{"type":41,"tag":56,"props":1490,"children":1492},{"className":1491},[],[1493],{"type":47,"value":1494},"template.json_valid",{"type":47,"value":592},{"type":41,"tag":56,"props":1497,"children":1499},{"className":1498},[],[1500],{"type":47,"value":1501},"json_schema",{"type":47,"value":1435},{"type":41,"tag":1369,"props":1504,"children":1505},{},[1506,1511,1519],{"type":41,"tag":1396,"props":1507,"children":1508},{},[1509],{"type":47,"value":1510},"over-refuses or drifts off-topic",{"type":41,"tag":1396,"props":1512,"children":1513},{},[1514,1515],{"type":47,"value":1405},{"type":41,"tag":65,"props":1516,"children":1517},{},[1518],{"type":47,"value":1335},{"type":41,"tag":1396,"props":1520,"children":1521},{},[1522,1528,1529,1534,1536],{"type":41,"tag":56,"props":1523,"children":1525},{"className":1524},[],[1526],{"type":47,"value":1527},"regex",{"type":47,"value":291},{"type":41,"tag":56,"props":1530,"children":1532},{"className":1531},[],[1533],{"type":47,"value":578},{"type":47,"value":1535}," on ",{"type":41,"tag":56,"props":1537,"children":1539},{"className":1538},[],[1540],{"type":47,"value":1357},{"type":41,"tag":1369,"props":1542,"children":1543},{},[1544,1549,1557],{"type":41,"tag":1396,"props":1545,"children":1546},{},[1547],{"type":47,"value":1548},"public-facing text",{"type":41,"tag":1396,"props":1550,"children":1551},{},[1552,1553],{"type":47,"value":1405},{"type":41,"tag":65,"props":1554,"children":1555},{},[1556],{"type":47,"value":1335},{"type":41,"tag":1396,"props":1558,"children":1559},{},[1560,1561,1567,1568,1574,1575,1580],{"type":47,"value":1414},{"type":41,"tag":56,"props":1562,"children":1564},{"className":1563},[],[1565],{"type":47,"value":1566},"template.toxicity",{"type":47,"value":291},{"type":41,"tag":56,"props":1569,"children":1571},{"className":1570},[],[1572],{"type":47,"value":1573},"template.pii",{"type":47,"value":592},{"type":41,"tag":56,"props":1576,"children":1578},{"className":1577},[],[1579],{"type":47,"value":578},{"type":47,"value":1435},{"type":41,"tag":1369,"props":1582,"children":1583},{},[1584,1596,1604],{"type":41,"tag":1396,"props":1585,"children":1586},{},[1587,1589,1594],{"type":47,"value":1588},"receives PII\u002Fsecrets in its ",{"type":41,"tag":65,"props":1590,"children":1591},{},[1592],{"type":47,"value":1593},"input",{"type":47,"value":1595}," (pasted into the prompt)",{"type":41,"tag":1396,"props":1597,"children":1598},{},[1599],{"type":41,"tag":65,"props":1600,"children":1601},{},[1602],{"type":47,"value":1603},"guard",{"type":41,"tag":1396,"props":1605,"children":1606},{},[1607,1612,1614,1619],{"type":41,"tag":56,"props":1608,"children":1610},{"className":1609},[],[1611],{"type":47,"value":126},{"type":47,"value":1613}," guard, ",{"type":41,"tag":65,"props":1615,"children":1616},{},[1617],{"type":47,"value":1618},"preflight",{"type":47,"value":1620}," — rewrites the request before the model sees it",{"type":41,"tag":1369,"props":1622,"children":1623},{},[1624,1639,1646],{"type":41,"tag":1396,"props":1625,"children":1626},{},[1627,1632,1634],{"type":41,"tag":65,"props":1628,"children":1629},{},[1630],{"type":47,"value":1631},"generates",{"type":47,"value":1633}," PII\u002Fsecrets\u002Fdestructive commands in its final ",{"type":41,"tag":65,"props":1635,"children":1636},{},[1637],{"type":47,"value":1638},"output",{"type":41,"tag":1396,"props":1640,"children":1641},{},[1642],{"type":41,"tag":65,"props":1643,"children":1644},{},[1645],{"type":47,"value":1603},{"type":41,"tag":1396,"props":1647,"children":1648},{},[1649,1651,1656,1657,1662,1664,1669,1670,1675,1677,1682],{"type":47,"value":1650},"evaluator-backed detector (",{"type":41,"tag":56,"props":1652,"children":1654},{"className":1653},[],[1655],{"type":47,"value":1527},{"type":47,"value":291},{"type":41,"tag":56,"props":1658,"children":1660},{"className":1659},[],[1661],{"type":47,"value":578},{"type":47,"value":1663},"), ",{"type":41,"tag":65,"props":1665,"children":1666},{},[1667],{"type":47,"value":1668},"postflight",{"type":47,"value":415},{"type":41,"tag":56,"props":1671,"children":1673},{"className":1672},[],[1674],{"type":47,"value":102},{"type":47,"value":1676}," — don't rely on ",{"type":41,"tag":56,"props":1678,"children":1680},{"className":1679},[],[1681],{"type":47,"value":126},{"type":47,"value":1683}," to scrub already-generated assistant text; detect + warn instead",{"type":41,"tag":1369,"props":1685,"children":1686},{},[1687,1692,1699],{"type":41,"tag":1396,"props":1688,"children":1689},{},[1690],{"type":47,"value":1691},"can call dangerous tools (shell, delete, write)",{"type":41,"tag":1396,"props":1693,"children":1694},{},[1695],{"type":41,"tag":65,"props":1696,"children":1697},{},[1698],{"type":47,"value":1603},{"type":41,"tag":1396,"props":1700,"children":1701},{},[1702,1707,1709,1715],{"type":41,"tag":56,"props":1703,"children":1705},{"className":1704},[],[1706],{"type":47,"value":134},{"type":47,"value":1708}," with ",{"type":41,"tag":56,"props":1710,"children":1712},{"className":1711},[],[1713],{"type":47,"value":1714},"blocked_names",{"type":47,"value":1716}," globs, postflight",{"type":41,"tag":1369,"props":1718,"children":1719},{},[1720,1725,1732],{"type":41,"tag":1396,"props":1721,"children":1722},{},[1723],{"type":47,"value":1724},"is subject to prompt-injection \u002F hard policy",{"type":41,"tag":1396,"props":1726,"children":1727},{},[1728],{"type":41,"tag":65,"props":1729,"children":1730},{},[1731],{"type":47,"value":1603},{"type":41,"tag":1396,"props":1733,"children":1734},{},[1735,1740,1742,1747,1749],{"type":41,"tag":56,"props":1736,"children":1738},{"className":1737},[],[1739],{"type":47,"value":578},{"type":47,"value":1741}," evaluator; draft ",{"type":41,"tag":56,"props":1743,"children":1745},{"className":1744},[],[1746],{"type":47,"value":102},{"type":47,"value":1748},", later promotable to ",{"type":41,"tag":56,"props":1750,"children":1752},{"className":1751},[],[1753],{"type":47,"value":110},{"type":41,"tag":1755,"props":1756,"children":1757},"blockquote",{},[1758,1795,1875,1972],{"type":41,"tag":50,"props":1759,"children":1760},{},[1761,1766,1767,1772,1774,1779,1781,1786,1788,1793],{"type":41,"tag":65,"props":1762,"children":1763},{},[1764],{"type":47,"value":1765},"Three guard shapes, all first-class:",{"type":47,"value":1291},{"type":41,"tag":56,"props":1768,"children":1770},{"className":1769},[],[1771],{"type":47,"value":118},{"type":47,"value":1773}," (an evaluator decides — most flexible),\n",{"type":41,"tag":56,"props":1775,"children":1777},{"className":1776},[],[1778],{"type":47,"value":126},{"type":47,"value":1780}," (regex redaction), and ",{"type":41,"tag":56,"props":1782,"children":1784},{"className":1783},[],[1785],{"type":47,"value":134},{"type":47,"value":1787}," (block tool calls). Pick the one that fits the policy;\na redaction need is a ",{"type":41,"tag":56,"props":1789,"children":1791},{"className":1790},[],[1792],{"type":47,"value":126},{"type":47,"value":1794}," guard, not a judge.",{"type":41,"tag":50,"props":1796,"children":1797},{},[1798,1808,1810,1815,1817,1822,1824,1835,1837,1850,1852,1858,1860,1865,1867,1873],{"type":41,"tag":65,"props":1799,"children":1800},{},[1801,1806],{"type":41,"tag":56,"props":1802,"children":1804},{"className":1803},[],[1805],{"type":47,"value":126},{"type":47,"value":1807}," schema is exact — get it right or the create 400s.",{"type":47,"value":1809}," The field is ",{"type":41,"tag":56,"props":1811,"children":1813},{"className":1812},[],[1814],{"type":47,"value":126},{"type":47,"value":1816}," (the server's\ncanonical name; a legacy ",{"type":41,"tag":56,"props":1818,"children":1820},{"className":1819},[],[1821],{"type":47,"value":1253},{"type":47,"value":1823}," alias also works), and each pattern is ",{"type":41,"tag":65,"props":1825,"children":1826},{},[1827,1829],{"type":47,"value":1828},"only ",{"type":41,"tag":56,"props":1830,"children":1832},{"className":1831},[],[1833],{"type":47,"value":1834},"{id, regex}",{"type":47,"value":1836}," —\nthere is ",{"type":41,"tag":65,"props":1838,"children":1839},{},[1840,1842,1848],{"type":47,"value":1841},"no ",{"type":41,"tag":56,"props":1843,"children":1845},{"className":1844},[],[1846],{"type":47,"value":1847},"replacement",{"type":47,"value":1849}," key",{"type":47,"value":1851},". On a match the server redacts to a placeholder derived from the\npattern ",{"type":41,"tag":56,"props":1853,"children":1855},{"className":1854},[],[1856],{"type":47,"value":1857},"id",{"type":47,"value":1859},"; supplying ",{"type":41,"tag":56,"props":1861,"children":1863},{"className":1862},[],[1864],{"type":47,"value":1847},{"type":47,"value":1866}," fails with ",{"type":41,"tag":56,"props":1868,"children":1870},{"className":1869},[],[1871],{"type":47,"value":1872},"400 unknown field \"replacement\"",{"type":47,"value":1874},". Draft it as:",{"type":41,"tag":645,"props":1876,"children":1880},{"className":1877,"code":1878,"language":1879,"meta":653,"style":653},"language-yaml shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","redact:\n  patterns:\n    - id: bearer_token          # the id names the placeholder\n      regex: 'Bearer\\s+[A-Za-z0-9._-]+'\n","yaml",[1881],{"type":41,"tag":56,"props":1882,"children":1883},{"__ignoreMap":653},[1884,1901,1914,1944],{"type":41,"tag":1885,"props":1886,"children":1889},"span",{"class":1887,"line":1888},"line",1,[1890,1895],{"type":41,"tag":1885,"props":1891,"children":1893},{"style":1892},"--shiki-light:#E53935;--shiki-default:#F07178;--shiki-dark:#F07178",[1894],{"type":47,"value":126},{"type":41,"tag":1885,"props":1896,"children":1898},{"style":1897},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[1899],{"type":47,"value":1900},":\n",{"type":41,"tag":1885,"props":1902,"children":1904},{"class":1887,"line":1903},2,[1905,1910],{"type":41,"tag":1885,"props":1906,"children":1907},{"style":1892},[1908],{"type":47,"value":1909},"  patterns",{"type":41,"tag":1885,"props":1911,"children":1912},{"style":1897},[1913],{"type":47,"value":1900},{"type":41,"tag":1885,"props":1915,"children":1917},{"class":1887,"line":1916},3,[1918,1923,1928,1932,1938],{"type":41,"tag":1885,"props":1919,"children":1920},{"style":1897},[1921],{"type":47,"value":1922},"    -",{"type":41,"tag":1885,"props":1924,"children":1925},{"style":1892},[1926],{"type":47,"value":1927}," id",{"type":41,"tag":1885,"props":1929,"children":1930},{"style":1897},[1931],{"type":47,"value":241},{"type":41,"tag":1885,"props":1933,"children":1935},{"style":1934},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[1936],{"type":47,"value":1937}," bearer_token",{"type":41,"tag":1885,"props":1939,"children":1941},{"style":1940},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[1942],{"type":47,"value":1943},"          # the id names the placeholder\n",{"type":41,"tag":1885,"props":1945,"children":1947},{"class":1887,"line":1946},4,[1948,1953,1957,1962,1967],{"type":41,"tag":1885,"props":1949,"children":1950},{"style":1892},[1951],{"type":47,"value":1952},"      regex",{"type":41,"tag":1885,"props":1954,"children":1955},{"style":1897},[1956],{"type":47,"value":241},{"type":41,"tag":1885,"props":1958,"children":1959},{"style":1897},[1960],{"type":47,"value":1961}," '",{"type":41,"tag":1885,"props":1963,"children":1964},{"style":1934},[1965],{"type":47,"value":1966},"Bearer\\s+[A-Za-z0-9._-]+",{"type":41,"tag":1885,"props":1968,"children":1969},{"style":1897},[1970],{"type":47,"value":1971},"'\n",{"type":41,"tag":50,"props":1973,"children":1974},{},[1975,1977,1983],{"type":47,"value":1976},"After creating any guard, ",{"type":41,"tag":56,"props":1978,"children":1980},{"className":1979},[],[1981],{"type":47,"value":1982},"guards get -o yaml",{"type":47,"value":1984}," and confirm the spec stored what you sent (a clean\ncreate echo is not proof on its own).",{"type":41,"tag":50,"props":1986,"children":1987},{},[1988,1990,1995,1997,2002,2004,2010,2012,2017,2019,2024],{"type":47,"value":1989},"Template ids above are the ",{"type":41,"tag":65,"props":1991,"children":1992},{},[1993],{"type":47,"value":1994},"expected",{"type":47,"value":1996}," global blueprints — they can vary by deployment and\nversion, so always resolve the current set with ",{"type":41,"tag":56,"props":1998,"children":2000},{"className":1999},[],[2001],{"type":47,"value":598},{"type":47,"value":2003}," before using a name;\ndon't trust a hardcoded id. Pick 3–6, ranked. Each gets a one-line ",{"type":41,"tag":56,"props":2005,"children":2007},{"className":2006},[],[2008],{"type":47,"value":2009},"why",{"type":47,"value":2011}," citing a ",{"type":41,"tag":65,"props":2013,"children":2014},{},[2015],{"type":47,"value":2016},"file:line or a\nconversation\u002Fgeneration id",{"type":47,"value":2018}," from Step 1. For rule mechanics (selectors, match keys, evaluator\nkinds, templates), the ",{"type":41,"tag":56,"props":2020,"children":2022},{"className":2021},[],[2023],{"type":47,"value":149},{"type":47,"value":2025}," skill is the reference — don't restate it here.",{"type":41,"tag":50,"props":2027,"children":2028},{},[2029],{"type":41,"tag":65,"props":2030,"children":2031},{},[2032],{"type":47,"value":2033},"Every candidate lands in exactly one of three states — and the decision is final for this run:",{"type":41,"tag":73,"props":2035,"children":2036},{},[2037,2047,2064],{"type":41,"tag":77,"props":2038,"children":2039},{},[2040,2045],{"type":41,"tag":65,"props":2041,"children":2042},{},[2043],{"type":47,"value":2044},"Recommended",{"type":47,"value":2046}," — worth setting up now; goes to Step 4 (draft) and Step 5 (apply).",{"type":41,"tag":77,"props":2048,"children":2049},{},[2050,2055,2057,2062],{"type":41,"tag":65,"props":2051,"children":2052},{},[2053],{"type":47,"value":2054},"Considered, not recommended",{"type":47,"value":2056}," — you evaluated it and it's not worth it (low value, no\nevidence, would just add cost\u002Fnoise). Record it with a one-line why, and ",{"type":41,"tag":65,"props":2058,"children":2059},{},[2060],{"type":47,"value":2061},"do NOT draft or apply\nit",{"type":47,"value":2063},". Don't quietly re-add it later under a different framing — if you're tempted to, it belongs\nin Recommended, so put it there and own the reasoning. Bias toward fewer objects: recommend only\nwhat earns its place. \"It's harmless in warn mode\" is not a reason to create something — an\nunused guard\u002Frule is still cost and noise.",{"type":41,"tag":77,"props":2065,"children":2066},{},[2067,2072],{"type":41,"tag":65,"props":2068,"children":2069},{},[2070],{"type":47,"value":2071},"Skipped (duplicate)",{"type":47,"value":2073}," — the stack already covers it (Step 2 semantic-equivalence check);\ndon't create a second one.",{"type":41,"tag":50,"props":2075,"children":2076},{},[2077,2079,2084],{"type":47,"value":2078},"The set you draft in Step 4 and apply in Step 5 is ",{"type":41,"tag":65,"props":2080,"children":2081},{},[2082],{"type":47,"value":2083},"exactly",{"type":47,"value":2085}," the Recommended list — nothing\nfrom the other two states leaks in.",{"type":41,"tag":138,"props":2087,"children":2089},{"id":2088},"step-4-draft-the-definitions-as-yaml",[2090],{"type":47,"value":2091},"Step 4 — Draft the definitions as YAML",{"type":41,"tag":50,"props":2093,"children":2094},{},[2095,2108,2109,2115,2117,2122,2124,2130],{"type":41,"tag":65,"props":2096,"children":2097},{},[2098,2100,2106],{"type":47,"value":2099},"First, check whether ",{"type":41,"tag":56,"props":2101,"children":2103},{"className":2102},[],[2104],{"type":47,"value":2105},".\u002Fagento11y-prod\u002F",{"type":47,"value":2107}," already has drafts from an earlier run",{"type":47,"value":592},{"type":41,"tag":56,"props":2110,"children":2112},{"className":2111},[],[2113],{"type":47,"value":2114},"ls agento11y-prod\u002F**",{"type":47,"value":2116}," if it exists). If it does, ",{"type":41,"tag":65,"props":2118,"children":2119},{},[2120],{"type":47,"value":2121},"read them before writing",{"type":47,"value":2123}," — don't blind-overwrite\n(a plain ",{"type":41,"tag":56,"props":2125,"children":2127},{"className":2126},[],[2128],{"type":47,"value":2129},"Write",{"type":47,"value":2131}," over an existing file also just errors). Treat a prior draft as a peer proposal:\nreconcile rather than replace. If a previous run made a deliberate, well-reasoned choice — e.g.\ndropped an email-redaction pattern because \"this agent's job is to email people, so redacting\nevery address is all-false-positive noise\" — that judgment is usually right; keep it and fold in\nonly what's genuinely new. Overwrite a prior draft only when yours is clearly better, and say why.",{"type":41,"tag":50,"props":2133,"children":2134},{},[2135,2137,2142,2143,2148,2150,2156,2157,2163,2164,2170,2172,2177,2179,2185,2187,2193,2195,2201,2203,2209,2211,2216,2218,2224,2225,2230,2232,2238,2240,2246],{"type":47,"value":2136},"Write the definitions to a ",{"type":41,"tag":65,"props":2138,"children":2139},{},[2140],{"type":47,"value":2141},"local scratch directory",{"type":47,"value":415},{"type":41,"tag":56,"props":2144,"children":2146},{"className":2145},[],[2147],{"type":47,"value":2105},{"type":47,"value":2149},"\n(",{"type":41,"tag":56,"props":2151,"children":2153},{"className":2152},[],[2154],{"type":47,"value":2155},"evaluators\u002F\u003Cid>.yaml",{"type":47,"value":415},{"type":41,"tag":56,"props":2158,"children":2160},{"className":2159},[],[2161],{"type":47,"value":2162},"rules\u002F\u003Cid>.yaml",{"type":47,"value":415},{"type":41,"tag":56,"props":2165,"children":2167},{"className":2166},[],[2168],{"type":47,"value":2169},"guards\u002F\u003Cid>.yaml",{"type":47,"value":2171},"). These are ",{"type":41,"tag":65,"props":2173,"children":2174},{},[2175],{"type":47,"value":2176},"working drafts, not\ncommitted artifacts",{"type":47,"value":2178},": they exist so the developer can review a diff before you apply it, and\ntheir source of truth after apply is the stack, not the repo. Add ",{"type":41,"tag":56,"props":2180,"children":2182},{"className":2181},[],[2183],{"type":47,"value":2184},"agento11y-prod\u002F",{"type":47,"value":2186}," to ",{"type":41,"tag":56,"props":2188,"children":2190},{"className":2189},[],[2191],{"type":47,"value":2192},".gitignore",{"type":47,"value":2194},"\n(or write under the OS temp dir) so they aren't accidentally committed — they hold the applied\nconfig redundantly and can carry regexes\u002Fprompts the repo shouldn't own. They are exactly what\nyou'll pass to ",{"type":41,"tag":56,"props":2196,"children":2198},{"className":2197},[],[2199],{"type":47,"value":2200},"gcx agento11y \u003Ckind> create -f",{"type":47,"value":2202}," (for evaluators: ",{"type":41,"tag":56,"props":2204,"children":2206},{"className":2205},[],[2207],{"type":47,"value":2208},"upsert -f",{"type":47,"value":2210},"). Use the\n",{"type":41,"tag":65,"props":2212,"children":2213},{},[2214],{"type":47,"value":2215},"top-level-fields",{"type":47,"value":2217}," YAML shape that the ",{"type":41,"tag":56,"props":2219,"children":2221},{"className":2220},[],[2222],{"type":47,"value":2223},"create -f",{"type":47,"value":1241},{"type":41,"tag":56,"props":2226,"children":2228},{"className":2227},[],[2229],{"type":47,"value":2208},{"type":47,"value":2231}," commands expect (not the\n",{"type":41,"tag":56,"props":2233,"children":2235},{"className":2234},[],[2236],{"type":47,"value":2237},"apiVersion\u002Fkind\u002Fspec",{"type":47,"value":2239}," manifest that the ",{"type":41,"tag":56,"props":2241,"children":2243},{"className":2242},[],[2244],{"type":47,"value":2245},"get -o yaml",{"type":47,"value":2247}," commands emit — don't round-trip get\noutput into create).",{"type":41,"tag":50,"props":2249,"children":2250},{},[2251,2256,2258,2263,2265,2271,2273,2279,2281,2292,2294,2300,2302,2308,2310,2315],{"type":41,"tag":65,"props":2252,"children":2253},{},[2254],{"type":47,"value":2255},"Rules and evaluators",{"type":47,"value":2257},": follow the ",{"type":41,"tag":56,"props":2259,"children":2261},{"className":2260},[],[2262],{"type":47,"value":149},{"type":47,"value":2264}," skill's input format exactly. Start an evaluator\nfrom a template (",{"type":41,"tag":56,"props":2266,"children":2268},{"className":2267},[],[2269],{"type":47,"value":2270},"gcx agento11y templates get \u003Cid> -o yaml",{"type":47,"value":2272},"), give it your own ",{"type":41,"tag":56,"props":2274,"children":2276},{"className":2275},[],[2277],{"type":47,"value":2278},"evaluator_id",{"type":47,"value":2280},", and\n",{"type":41,"tag":65,"props":2282,"children":2283},{},[2284,2286],{"type":47,"value":2285},"always include a ",{"type":41,"tag":56,"props":2287,"children":2289},{"className":2288},[],[2290],{"type":47,"value":2291},"version",{"type":47,"value":2293}," — it is required on create (a date like ",{"type":41,"tag":56,"props":2295,"children":2297},{"className":2296},[],[2298],{"type":47,"value":2299},"\"2026-07-15\"",{"type":47,"value":2301}," or a semver\nworks; existing evaluators use dates). Omitting it fails with ",{"type":41,"tag":56,"props":2303,"children":2305},{"className":2304},[],[2306],{"type":47,"value":2307},"version is required",{"type":47,"value":2309},". Rule starts\nenabled at a low ",{"type":41,"tag":56,"props":2311,"children":2313},{"className":2312},[],[2314],{"type":47,"value":554},{"type":47,"value":538},{"type":41,"tag":50,"props":2317,"children":2318},{},[2319,2323,2325,2330,2332,2338,2340,2346,2348,2354],{"type":41,"tag":65,"props":2320,"children":2321},{},[2322],{"type":47,"value":94},{"type":47,"value":2324}," — the shape the ",{"type":41,"tag":56,"props":2326,"children":2328},{"className":2327},[],[2329],{"type":47,"value":149},{"type":47,"value":2331}," skill omits (",{"type":41,"tag":56,"props":2333,"children":2335},{"className":2334},[],[2336],{"type":47,"value":2337},"gcx agento11y guards create -f guard.yaml",{"type":47,"value":2339},"; the\nresource ",{"type":41,"tag":56,"props":2341,"children":2343},{"className":2342},[],[2344],{"type":47,"value":2345},"Kind",{"type":47,"value":2347}," is ",{"type":41,"tag":56,"props":2349,"children":2351},{"className":2350},[],[2352],{"type":47,"value":2353},"HookRule",{"type":47,"value":219},{"type":41,"tag":50,"props":2356,"children":2357},{},[2358,2360,2365,2367,2372,2373,2378,2380,2385,2387,2392,2393,2399,2400,2406,2407,2412,2414,2419,2421,2426,2428,2433,2435,2441,2442,2448,2450,2456,2458,2463,2465,2469,2471,2476],{"type":47,"value":2359},"This skill (not the ",{"type":41,"tag":56,"props":2361,"children":2363},{"className":2362},[],[2364],{"type":47,"value":149},{"type":47,"value":2366}," skill) carries the guard shape — draft the guard file directly from\nit, no schema-discovery step needed. A guard drives its decision from one of three (mutually\nexclusive) shapes — ",{"type":41,"tag":56,"props":2368,"children":2370},{"className":2369},[],[2371],{"type":47,"value":118},{"type":47,"value":415},{"type":41,"tag":56,"props":2374,"children":2376},{"className":2375},[],[2377],{"type":47,"value":126},{"type":47,"value":2379},", or ",{"type":41,"tag":56,"props":2381,"children":2383},{"className":2382},[],[2384],{"type":47,"value":134},{"type":47,"value":2386}," — plus ",{"type":41,"tag":56,"props":2388,"children":2390},{"className":2389},[],[2391],{"type":47,"value":310},{"type":47,"value":415},{"type":41,"tag":56,"props":2394,"children":2396},{"className":2395},[],[2397],{"type":47,"value":2398},"phase",{"type":47,"value":1141},{"type":41,"tag":56,"props":2401,"children":2403},{"className":2402},[],[2404],{"type":47,"value":2405},"priority",{"type":47,"value":415},{"type":41,"tag":56,"props":2408,"children":2410},{"className":2409},[],[2411],{"type":47,"value":1210},{"type":47,"value":2413},". Use the ",{"type":41,"tag":56,"props":2415,"children":2417},{"className":2416},[],[2418],{"type":47,"value":126},{"type":47,"value":2420}," schema exactly as the callout above shows (",{"type":41,"tag":56,"props":2422,"children":2424},{"className":2423},[],[2425],{"type":47,"value":1834},{"type":47,"value":2427},",\nno ",{"type":41,"tag":56,"props":2429,"children":2431},{"className":2430},[],[2432],{"type":47,"value":1847},{"type":47,"value":2434},"). If the server ever rejects a field, the 400 names the offending field, so a bad\nshape surfaces at create time rather than needing a probe up front. On create the server fills\ndefaults you don't set — notably ",{"type":41,"tag":56,"props":2436,"children":2438},{"className":2437},[],[2439],{"type":47,"value":2440},"selector: all",{"type":47,"value":415},{"type":41,"tag":56,"props":2443,"children":2445},{"className":2444},[],[2446],{"type":47,"value":2447},"phase: preflight",{"type":47,"value":2449},", and ",{"type":41,"tag":56,"props":2451,"children":2453},{"className":2452},[],[2454],{"type":47,"value":2455},"short_circuit: false",{"type":47,"value":2457}," —\nso a ",{"type":41,"tag":56,"props":2459,"children":2461},{"className":2460},[],[2462],{"type":47,"value":1982},{"type":47,"value":2464}," right after create\nshows more fields than you sent; that's expected, not drift. A guard is drafted in ",{"type":41,"tag":65,"props":2466,"children":2467},{},[2468],{"type":47,"value":102},{"type":47,"value":2470}," (and\n",{"type":41,"tag":56,"props":2472,"children":2474},{"className":2473},[],[2475],{"type":47,"value":513},{"type":47,"value":2477}," if the server honors it — it may not; see Step 5):",{"type":41,"tag":645,"props":2479,"children":2481},{"className":1877,"code":2480,"language":1879,"meta":653,"style":653},"# PROD-SETUP DRAFT — creates a STACK-LEVEL guard (HookRule) via gcx agento11y guards create.\n# warn on purpose: a warn guard records but never blocks, so it is safe on real traffic\n# even while live; watch it, then switch to deny yourself later.\nrule_id: guard.\u003Cagent>.\u003Cpolicy>\nenabled: false         # server may store true anyway; harmless because action_on_fail is warn\npriority: 10           # lower runs first\naction_on_fail: warn   # always draft as warn (see below)\n# Pick exactly ONE decision shape:\n# (a) evaluator-backed — an evaluator (judge\u002Fregex) decides; most flexible.\nevaluator_ids: [\"\u003Cpolicy-judge-id>\"]  # create the evaluator first\n# (b) redact — regex redaction to a placeholder derived from `id`. Schema is {id, regex} ONLY;\n#     a `replacement` key 400s. Field name is `redact` (legacy alias: `transform`).\n# redact:\n#   patterns:\n#     - id: ssn\n#       regex: \"\\\\b\\\\d{3}-\\\\d{2}-\\\\d{4}\\\\b\"\n# (c) tool_filter — block tool calls by name glob.\n# tool_filter:\n#   blocked_names: [\"shell_exec\", \"Bash(*rm*)\"]\n",[2482],{"type":41,"tag":56,"props":2483,"children":2484},{"__ignoreMap":653},[2485,2493,2501,2509,2526,2549,2572,2594,2603,2612,2653,2662,2671,2680,2689,2698,2707,2716,2725],{"type":41,"tag":1885,"props":2486,"children":2487},{"class":1887,"line":1888},[2488],{"type":41,"tag":1885,"props":2489,"children":2490},{"style":1940},[2491],{"type":47,"value":2492},"# PROD-SETUP DRAFT — creates a STACK-LEVEL guard (HookRule) via gcx agento11y guards create.\n",{"type":41,"tag":1885,"props":2494,"children":2495},{"class":1887,"line":1903},[2496],{"type":41,"tag":1885,"props":2497,"children":2498},{"style":1940},[2499],{"type":47,"value":2500},"# warn on purpose: a warn guard records but never blocks, so it is safe on real traffic\n",{"type":41,"tag":1885,"props":2502,"children":2503},{"class":1887,"line":1916},[2504],{"type":41,"tag":1885,"props":2505,"children":2506},{"style":1940},[2507],{"type":47,"value":2508},"# even while live; watch it, then switch to deny yourself later.\n",{"type":41,"tag":1885,"props":2510,"children":2511},{"class":1887,"line":1946},[2512,2517,2521],{"type":41,"tag":1885,"props":2513,"children":2514},{"style":1892},[2515],{"type":47,"value":2516},"rule_id",{"type":41,"tag":1885,"props":2518,"children":2519},{"style":1897},[2520],{"type":47,"value":241},{"type":41,"tag":1885,"props":2522,"children":2523},{"style":1934},[2524],{"type":47,"value":2525}," guard.\u003Cagent>.\u003Cpolicy>\n",{"type":41,"tag":1885,"props":2527,"children":2529},{"class":1887,"line":2528},5,[2530,2534,2538,2544],{"type":41,"tag":1885,"props":2531,"children":2532},{"style":1892},[2533],{"type":47,"value":536},{"type":41,"tag":1885,"props":2535,"children":2536},{"style":1897},[2537],{"type":47,"value":241},{"type":41,"tag":1885,"props":2539,"children":2541},{"style":2540},"--shiki-light:#FF5370;--shiki-default:#FF9CAC;--shiki-dark:#FF9CAC",[2542],{"type":47,"value":2543}," false",{"type":41,"tag":1885,"props":2545,"children":2546},{"style":1940},[2547],{"type":47,"value":2548},"         # server may store true anyway; harmless because action_on_fail is warn\n",{"type":41,"tag":1885,"props":2550,"children":2552},{"class":1887,"line":2551},6,[2553,2557,2561,2567],{"type":41,"tag":1885,"props":2554,"children":2555},{"style":1892},[2556],{"type":47,"value":2405},{"type":41,"tag":1885,"props":2558,"children":2559},{"style":1897},[2560],{"type":47,"value":241},{"type":41,"tag":1885,"props":2562,"children":2564},{"style":2563},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[2565],{"type":47,"value":2566}," 10",{"type":41,"tag":1885,"props":2568,"children":2569},{"style":1940},[2570],{"type":47,"value":2571},"           # lower runs first\n",{"type":41,"tag":1885,"props":2573,"children":2575},{"class":1887,"line":2574},7,[2576,2580,2584,2589],{"type":41,"tag":1885,"props":2577,"children":2578},{"style":1892},[2579],{"type":47,"value":310},{"type":41,"tag":1885,"props":2581,"children":2582},{"style":1897},[2583],{"type":47,"value":241},{"type":41,"tag":1885,"props":2585,"children":2586},{"style":1934},[2587],{"type":47,"value":2588}," warn",{"type":41,"tag":1885,"props":2590,"children":2591},{"style":1940},[2592],{"type":47,"value":2593},"   # always draft as warn (see below)\n",{"type":41,"tag":1885,"props":2595,"children":2597},{"class":1887,"line":2596},8,[2598],{"type":41,"tag":1885,"props":2599,"children":2600},{"style":1940},[2601],{"type":47,"value":2602},"# Pick exactly ONE decision shape:\n",{"type":41,"tag":1885,"props":2604,"children":2606},{"class":1887,"line":2605},9,[2607],{"type":41,"tag":1885,"props":2608,"children":2609},{"style":1940},[2610],{"type":47,"value":2611},"# (a) evaluator-backed — an evaluator (judge\u002Fregex) decides; most flexible.\n",{"type":41,"tag":1885,"props":2613,"children":2615},{"class":1887,"line":2614},10,[2616,2620,2624,2629,2634,2639,2643,2648],{"type":41,"tag":1885,"props":2617,"children":2618},{"style":1892},[2619],{"type":47,"value":118},{"type":41,"tag":1885,"props":2621,"children":2622},{"style":1897},[2623],{"type":47,"value":241},{"type":41,"tag":1885,"props":2625,"children":2626},{"style":1897},[2627],{"type":47,"value":2628}," [",{"type":41,"tag":1885,"props":2630,"children":2631},{"style":1897},[2632],{"type":47,"value":2633},"\"",{"type":41,"tag":1885,"props":2635,"children":2636},{"style":1934},[2637],{"type":47,"value":2638},"\u003Cpolicy-judge-id>",{"type":41,"tag":1885,"props":2640,"children":2641},{"style":1897},[2642],{"type":47,"value":2633},{"type":41,"tag":1885,"props":2644,"children":2645},{"style":1897},[2646],{"type":47,"value":2647},"]",{"type":41,"tag":1885,"props":2649,"children":2650},{"style":1940},[2651],{"type":47,"value":2652},"  # create the evaluator first\n",{"type":41,"tag":1885,"props":2654,"children":2656},{"class":1887,"line":2655},11,[2657],{"type":41,"tag":1885,"props":2658,"children":2659},{"style":1940},[2660],{"type":47,"value":2661},"# (b) redact — regex redaction to a placeholder derived from `id`. Schema is {id, regex} ONLY;\n",{"type":41,"tag":1885,"props":2663,"children":2665},{"class":1887,"line":2664},12,[2666],{"type":41,"tag":1885,"props":2667,"children":2668},{"style":1940},[2669],{"type":47,"value":2670},"#     a `replacement` key 400s. Field name is `redact` (legacy alias: `transform`).\n",{"type":41,"tag":1885,"props":2672,"children":2674},{"class":1887,"line":2673},13,[2675],{"type":41,"tag":1885,"props":2676,"children":2677},{"style":1940},[2678],{"type":47,"value":2679},"# redact:\n",{"type":41,"tag":1885,"props":2681,"children":2683},{"class":1887,"line":2682},14,[2684],{"type":41,"tag":1885,"props":2685,"children":2686},{"style":1940},[2687],{"type":47,"value":2688},"#   patterns:\n",{"type":41,"tag":1885,"props":2690,"children":2692},{"class":1887,"line":2691},15,[2693],{"type":41,"tag":1885,"props":2694,"children":2695},{"style":1940},[2696],{"type":47,"value":2697},"#     - id: ssn\n",{"type":41,"tag":1885,"props":2699,"children":2701},{"class":1887,"line":2700},16,[2702],{"type":41,"tag":1885,"props":2703,"children":2704},{"style":1940},[2705],{"type":47,"value":2706},"#       regex: \"\\\\b\\\\d{3}-\\\\d{2}-\\\\d{4}\\\\b\"\n",{"type":41,"tag":1885,"props":2708,"children":2710},{"class":1887,"line":2709},17,[2711],{"type":41,"tag":1885,"props":2712,"children":2713},{"style":1940},[2714],{"type":47,"value":2715},"# (c) tool_filter — block tool calls by name glob.\n",{"type":41,"tag":1885,"props":2717,"children":2719},{"class":1887,"line":2718},18,[2720],{"type":41,"tag":1885,"props":2721,"children":2722},{"style":1940},[2723],{"type":47,"value":2724},"# tool_filter:\n",{"type":41,"tag":1885,"props":2726,"children":2728},{"class":1887,"line":2727},19,[2729],{"type":41,"tag":1885,"props":2730,"children":2731},{"style":1940},[2732],{"type":47,"value":2733},"#   blocked_names: [\"shell_exec\", \"Bash(*rm*)\"]\n",{"type":41,"tag":50,"props":2735,"children":2736},{},[2737,2742,2743,2748,2750,2755,2756,2761],{"type":41,"tag":56,"props":2738,"children":2740},{"className":2739},[],[2741],{"type":47,"value":2398},{"type":47,"value":2347},{"type":41,"tag":56,"props":2744,"children":2746},{"className":2745},[],[2747],{"type":47,"value":1618},{"type":47,"value":2749}," (default) or ",{"type":41,"tag":56,"props":2751,"children":2753},{"className":2752},[],[2754],{"type":47,"value":1668},{"type":47,"value":2449},{"type":41,"tag":65,"props":2757,"children":2758},{},[2759],{"type":47,"value":2760},"the correct phase depends on the guard\ntype",{"type":47,"value":2762}," — getting this wrong yields a guard that runs but silently does nothing:",{"type":41,"tag":73,"props":2764,"children":2765},{},[2766,2923,2944],{"type":41,"tag":77,"props":2767,"children":2768},{},[2769,2786,2788,2793,2794,2800,2802,2807,2809,2821,2823,2828,2830,2835,2836,2841,2842,2847,2849,2854,2856,2862,2864,2869,2871],{"type":41,"tag":65,"props":2770,"children":2771},{},[2772,2777,2779,2784],{"type":41,"tag":56,"props":2773,"children":2775},{"className":2774},[],[2776],{"type":47,"value":126},{"type":47,"value":2778}," → normally ",{"type":41,"tag":56,"props":2780,"children":2782},{"className":2781},[],[2783],{"type":47,"value":1618},{"type":47,"value":2785}," (input).",{"type":47,"value":2787}," Use it to redact sensitive values in the request\n",{"type":41,"tag":56,"props":2789,"children":2791},{"className":2790},[],[2792],{"type":47,"value":1593},{"type":47,"value":592},{"type":41,"tag":56,"props":2795,"children":2797},{"className":2796},[],[2798],{"type":47,"value":2799},"messages",{"type":47,"value":2801},") ",{"type":41,"tag":65,"props":2803,"children":2804},{},[2805],{"type":47,"value":2806},"before the model sees them",{"type":47,"value":2808}," — e.g. a secret pasted into the user prompt or\nincident text. This is the common case and the one the Grafana UI defaults a Redact guard to.\n",{"type":41,"tag":65,"props":2810,"children":2811},{},[2812,2814,2819],{"type":47,"value":2813},"Do not rely on ",{"type":41,"tag":56,"props":2815,"children":2817},{"className":2816},[],[2818],{"type":47,"value":126},{"type":47,"value":2820}," to rewrite already-generated assistant text:",{"type":47,"value":2822}," for a secret the model\n",{"type":41,"tag":172,"props":2824,"children":2825},{},[2826],{"type":47,"value":2827},"produces",{"type":47,"value":2829}," in its final response, a postflight redaction of the assistant text does not reliably\nscrub it — use a postflight evaluator-backed ",{"type":41,"tag":65,"props":2831,"children":2832},{},[2833],{"type":47,"value":2834},"detector",{"type":47,"value":592},{"type":41,"tag":56,"props":2837,"children":2839},{"className":2838},[],[2840],{"type":47,"value":102},{"type":47,"value":1241},{"type":41,"tag":56,"props":2843,"children":2845},{"className":2844},[],[2846],{"type":47,"value":110},{"type":47,"value":2848},") instead (below).\n(Postflight ",{"type":41,"tag":56,"props":2850,"children":2852},{"className":2851},[],[2853],{"type":47,"value":126},{"type":47,"value":2855}," is not useless in general — some runtimes explicitly consume\n",{"type":41,"tag":56,"props":2857,"children":2859},{"className":2858},[],[2860],{"type":47,"value":2861},"transformed_input.output",{"type":47,"value":2863}," to rewrite ",{"type":41,"tag":65,"props":2865,"children":2866},{},[2867],{"type":47,"value":2868},"tool-call payloads\u002Farguments",{"type":47,"value":2870}," postflight — but that is a\nnarrow tool-arg case, not final-response scrubbing.)\n",{"type":41,"tag":73,"props":2872,"children":2873},{},[2874],{"type":41,"tag":77,"props":2875,"children":2876},{},[2877,2882,2884,2889,2891,2896,2898,2908,2910,2914,2916,2921],{"type":41,"tag":65,"props":2878,"children":2879},{},[2880],{"type":47,"value":2881},"Disambiguation — decide by where the sensitive value ENTERS, not by where it's most visible.",{"type":47,"value":2883},"\nAn agent can both ",{"type":41,"tag":172,"props":2885,"children":2886},{},[2887],{"type":47,"value":2888},"receive",{"type":47,"value":2890}," a secret in its input AND ",{"type":41,"tag":172,"props":2892,"children":2893},{},[2894],{"type":47,"value":2895},"echo\u002Fgenerate",{"type":47,"value":2897}," one in its output; don't\nlet the more eye-catching output occurrence pull you to postflight. If the secret enters through\nthe input (e.g. a key pasted into the prompt \u002F incident text \u002F a config blob), the guard that\nactually protects it is ",{"type":41,"tag":65,"props":2899,"children":2900},{},[2901,2906],{"type":41,"tag":56,"props":2902,"children":2904},{"className":2903},[],[2905],{"type":47,"value":126},{"type":47,"value":2907}," preflight on the input",{"type":47,"value":2909}," — that scrubs it before the model,\nthe logs, and every downstream node see it. A postflight guard cannot undo a secret that already\nentered upstream. Add a postflight ",{"type":41,"tag":65,"props":2911,"children":2912},{},[2913],{"type":47,"value":2834},{"type":47,"value":2915}," on top only if the model ",{"type":41,"tag":172,"props":2917,"children":2918},{},[2919],{"type":47,"value":2920},"also",{"type":47,"value":2922}," independently\nproduces secrets in its final text; that is a second, separate concern, not a replacement for the\npreflight redact.",{"type":41,"tag":77,"props":2924,"children":2925},{},[2926,2942],{"type":41,"tag":65,"props":2927,"children":2928},{},[2929,2934,2936,2941],{"type":41,"tag":56,"props":2930,"children":2932},{"className":2931},[],[2933],{"type":47,"value":134},{"type":47,"value":2935}," → ",{"type":41,"tag":56,"props":2937,"children":2939},{"className":2938},[],[2940],{"type":47,"value":1668},{"type":47,"value":538},{"type":47,"value":2943}," It inspects the tool calls the model wants to make.",{"type":41,"tag":77,"props":2945,"children":2946},{},[2947,2957,2958,2963,2965,2969,2971,2976,2978,2982,2984,2989,2991,2996],{"type":41,"tag":65,"props":2948,"children":2949},{},[2950,2955],{"type":41,"tag":56,"props":2951,"children":2953},{"className":2952},[],[2954],{"type":47,"value":118},{"type":47,"value":2956}," (detector) → phase follows what you evaluate.",{"type":47,"value":1291},{"type":41,"tag":56,"props":2959,"children":2961},{"className":2960},[],[2962],{"type":47,"value":1618},{"type":47,"value":2964}," to judge the\n",{"type":41,"tag":65,"props":2966,"children":2967},{},[2968],{"type":47,"value":1593},{"type":47,"value":2970}," (block a bad request before spending tokens), ",{"type":41,"tag":56,"props":2972,"children":2974},{"className":2973},[],[2975],{"type":47,"value":1668},{"type":47,"value":2977}," to judge the ",{"type":41,"tag":65,"props":2979,"children":2980},{},[2981],{"type":47,"value":1638},{"type":47,"value":2983},"\n(catch a bad response — e.g. detect that the model ",{"type":41,"tag":172,"props":2985,"children":2986},{},[2987],{"type":47,"value":2988},"generated",{"type":47,"value":2990}," a credential or a destructive\ncommand). This is the right shape for \"the model produces something risky,\" which ",{"type":41,"tag":56,"props":2992,"children":2994},{"className":2993},[],[2995],{"type":47,"value":126},{"type":47,"value":2997}," cannot\ncatch.",{"type":41,"tag":50,"props":2999,"children":3000},{},[3001,3014,3016,3021,3023,3028,3030,3035],{"type":41,"tag":65,"props":3002,"children":3003},{},[3004,3006,3012],{"type":47,"value":3005},"Always draft ",{"type":41,"tag":56,"props":3007,"children":3009},{"className":3008},[],[3010],{"type":47,"value":3011},"action_on_fail: warn",{"type":47,"value":3013},", even for\na hard-policy guard",{"type":47,"value":3015}," (prompt-injection, deny-list): a first-time ",{"type":41,"tag":56,"props":3017,"children":3019},{"className":3018},[],[3020],{"type":47,"value":110},{"type":47,"value":3022}," enabled on live traffic\nblocks real users on a false positive. A policy-judge guard references an evaluator id (create the\nevaluator first); the developer changes it to ",{"type":41,"tag":56,"props":3024,"children":3026},{"className":3025},[],[3027],{"type":47,"value":110},{"type":47,"value":3029}," only later, after watching the false-positive\nrate in warn mode (Step 6). This skill never drafts an enabled ",{"type":41,"tag":56,"props":3031,"children":3033},{"className":3032},[],[3034],{"type":47,"value":110},{"type":47,"value":3036}," guard.",{"type":41,"tag":138,"props":3038,"children":3040},{"id":3039},"step-5-confirm-then-apply-with-gcx",[3041,3043],{"type":47,"value":3042},"Step 5 — Confirm, then apply with ",{"type":41,"tag":56,"props":3044,"children":3046},{"className":3045},[],[3047],{"type":47,"value":363},{"type":41,"tag":1755,"props":3049,"children":3050},{},[3051],{"type":41,"tag":50,"props":3052,"children":3053},{},[3054,3078,3080,3086,3088,3093,3095,3101,3102,3108,3109,3115,3116,3122,3124,3129,3131,3136,3138,3143],{"type":41,"tag":65,"props":3055,"children":3056},{},[3057,3063,3064,3070,3071,3076],{"type":41,"tag":56,"props":3058,"children":3060},{"className":3059},[],[3061],{"type":47,"value":3062},"upsert",{"type":47,"value":1241},{"type":41,"tag":56,"props":3065,"children":3067},{"className":3066},[],[3068],{"type":47,"value":3069},"create",{"type":47,"value":1241},{"type":41,"tag":56,"props":3072,"children":3074},{"className":3073},[],[3075],{"type":47,"value":1271},{"type":47,"value":3077}," write to the stack — never run them before the developer's explicit yes\n(step 2).",{"type":47,"value":3079}," The one thing you CAN run before the yes is ",{"type":41,"tag":56,"props":3081,"children":3083},{"className":3082},[],[3084],{"type":47,"value":3085},"evaluators test -f \u003Crequest>.yaml",{"type":47,"value":3087},",\nwhich tests a judge config ",{"type":41,"tag":65,"props":3089,"children":3090},{},[3091],{"type":47,"value":3092},"without persisting it",{"type":47,"value":3094}," (pass ",{"type":41,"tag":56,"props":3096,"children":3098},{"className":3097},[],[3099],{"type":47,"value":3100},"kind",{"type":47,"value":415},{"type":41,"tag":56,"props":3103,"children":3105},{"className":3104},[],[3106],{"type":47,"value":3107},"config",{"type":47,"value":415},{"type":41,"tag":56,"props":3110,"children":3112},{"className":3111},[],[3113],{"type":47,"value":3114},"output_keys",{"type":47,"value":1141},{"type":41,"tag":56,"props":3117,"children":3119},{"className":3118},[],[3120],{"type":47,"value":3121},"generation_id",{"type":47,"value":3123}," in the file — no evaluator need exist yet). Use it to tune the judge (step 1).\nThere is ",{"type":41,"tag":65,"props":3125,"children":3126},{},[3127],{"type":47,"value":3128},"no CLI dry-run",{"type":47,"value":3130}," for rules or guards — their safety comes from shipping guards in\n",{"type":41,"tag":56,"props":3132,"children":3134},{"className":3133},[],[3135],{"type":47,"value":102},{"type":47,"value":3137}," (records, never blocks) and rules at a low ",{"type":41,"tag":56,"props":3139,"children":3141},{"className":3140},[],[3142],{"type":47,"value":554},{"type":47,"value":3144},", not from a preview.",{"type":41,"tag":50,"props":3146,"children":3147},{},[3148],{"type":47,"value":3149},"Per object the developer wants, in dependency order (evaluators → rules → guards, since a\nrule\u002Fguard referencing an evaluator needs it to exist first):",{"type":41,"tag":199,"props":3151,"children":3152},{},[3153,3206,3216,3323],{"type":41,"tag":77,"props":3154,"children":3155},{},[3156,3168,3170,3175,3177,3183,3185,3190,3192,3197,3198,3204],{"type":41,"tag":65,"props":3157,"children":3158},{},[3159,3161,3166],{"type":47,"value":3160},"For an ",{"type":41,"tag":56,"props":3162,"children":3164},{"className":3163},[],[3165],{"type":47,"value":578},{"type":47,"value":3167}," evaluator, tune the prompt before creating it — this is the real work,\nnot a formality.",{"type":47,"value":3169}," A judge is only as good as its prompt, so don't create it on first draft and\nmove on. Loop: pick 1–2 real generations you know the right answer for\n(",{"type":41,"tag":56,"props":3171,"children":3173},{"className":3172},[],[3174],{"type":47,"value":912},{"type":47,"value":3176},"), run the draft config with\n",{"type":41,"tag":56,"props":3178,"children":3180},{"className":3179},[],[3181],{"type":47,"value":3182},"gcx agento11y evaluators test -f \u003Crequest>.yaml -g \u003Cgen-id>",{"type":47,"value":3184}," (tests without persisting), and\n",{"type":41,"tag":65,"props":3186,"children":3187},{},[3188],{"type":47,"value":3189},"read both the verdict AND the rationale",{"type":47,"value":3191},". If either disagrees with what you expected, adjust\nthe ",{"type":41,"tag":56,"props":3193,"children":3195},{"className":3194},[],[3196],{"type":47,"value":781},{"type":47,"value":1241},{"type":41,"tag":56,"props":3199,"children":3201},{"className":3200},[],[3202],{"type":47,"value":3203},"user_prompt",{"type":47,"value":3205}," and re-test. Repeat until verdict and rationale both hold on\nyour known examples. Only then does the evaluator move to step 2. (This tuning loop deserves its\nown dedicated flow; keep it lightweight here.)",{"type":41,"tag":77,"props":3207,"children":3208},{},[3209,3214],{"type":41,"tag":65,"props":3210,"children":3211},{},[3212],{"type":47,"value":3213},"Confirm.",{"type":47,"value":3215}," Restate the target stack from Step 0 (context name + server), show the exact YAML,\nand get an explicit yes for that object. A yes for one object is not a yes for the next. Nothing\nis written before this yes.",{"type":41,"tag":77,"props":3217,"children":3218},{},[3219,3224,3226,3232,3234,3240,3241,3247,3249,3254,3256],{"type":41,"tag":65,"props":3220,"children":3221},{},[3222],{"type":47,"value":3223},"Apply",{"type":47,"value":3225}," via gcx, only after the yes: ",{"type":41,"tag":56,"props":3227,"children":3229},{"className":3228},[],[3230],{"type":47,"value":3231},"gcx agento11y evaluators upsert -f evaluators\u002F\u003Cid>.yaml",{"type":47,"value":3233},",\nthen ",{"type":41,"tag":56,"props":3235,"children":3237},{"className":3236},[],[3238],{"type":47,"value":3239},"gcx agento11y rules create -f rules\u002F\u003Cid>.yaml",{"type":47,"value":600},{"type":41,"tag":56,"props":3242,"children":3244},{"className":3243},[],[3245],{"type":47,"value":3246},"gcx agento11y guards create -f guards\u002F\u003Cid>.yaml",{"type":47,"value":3248},". Evaluators are create-or-update (same id\nupdates). Pass ",{"type":41,"tag":56,"props":3250,"children":3252},{"className":3251},[],[3253],{"type":47,"value":716},{"type":47,"value":3255}," on every call if the confirmed stack isn't the default\ncontext. gcx handles auth — no tokens here.\n",{"type":41,"tag":73,"props":3257,"children":3258},{},[3259,3318],{"type":41,"tag":77,"props":3260,"children":3261},{},[3262,3281,3283,3288,3290,3295,3297,3302,3304,3309,3311,3316],{"type":41,"tag":65,"props":3263,"children":3264},{},[3265,3267,3272,3273,3279],{"type":47,"value":3266},"The server may store a new guard ",{"type":41,"tag":56,"props":3268,"children":3270},{"className":3269},[],[3271],{"type":47,"value":521},{"type":47,"value":291},{"type":41,"tag":56,"props":3274,"children":3276},{"className":3275},[],[3277],{"type":47,"value":3278},"short_circuit: true",{"type":47,"value":3280}," regardless of what\nyou drafted.",{"type":47,"value":3282}," That is a known server default, not a draft error. Do not build a\ncreate→get→update dance around it: because you drafted ",{"type":41,"tag":56,"props":3284,"children":3286},{"className":3285},[],[3287],{"type":47,"value":3011},{"type":47,"value":3289},", a live guard only\nrecords outcomes and blocks no one, so ",{"type":41,"tag":56,"props":3291,"children":3293},{"className":3292},[],[3294],{"type":47,"value":521},{"type":47,"value":3296}," is harmless here. Create the guard and\nmove on. (What you must never create is an enabled ",{"type":41,"tag":56,"props":3298,"children":3300},{"className":3299},[],[3301],{"type":47,"value":110},{"type":47,"value":3303}," guard — the ",{"type":41,"tag":56,"props":3305,"children":3307},{"className":3306},[],[3308],{"type":47,"value":102},{"type":47,"value":3310}," rule above is what\nprevents that, not the ",{"type":41,"tag":56,"props":3312,"children":3314},{"className":3313},[],[3315],{"type":47,"value":536},{"type":47,"value":3317}," value.)",{"type":41,"tag":77,"props":3319,"children":3320},{},[3321],{"type":47,"value":3322},"A judge-model 404 when testing\u002Fscoring is usually a stack-side misconfiguration (the stack's\njudge model id is dead), not your evaluator — flag it; the online rule will hit the same broken\njudge at runtime until it's fixed.",{"type":41,"tag":77,"props":3324,"children":3325},{},[3326,3328,3333,3335,3340],{"type":47,"value":3327},"If ",{"type":41,"tag":56,"props":3329,"children":3331},{"className":3330},[],[3332],{"type":47,"value":363},{"type":47,"value":3334}," reports it isn't authenticated, stop and ask the developer to run ",{"type":41,"tag":56,"props":3336,"children":3338},{"className":3337},[],[3339],{"type":47,"value":371},{"type":47,"value":3341},"; do not\nfall back to raw HTTP.",{"type":41,"tag":138,"props":3343,"children":3345},{"id":3344},"step-55-wire-the-agent-to-call-the-guard-guards-only-required",[3346],{"type":47,"value":3347},"Step 5.5 — Wire the agent to call the guard (guards only, REQUIRED)",{"type":41,"tag":50,"props":3349,"children":3350},{},[3351,3353,3358,3360,3365,3367,3372,3374,3379],{"type":47,"value":3352},"Creating a guard on the stack does ",{"type":41,"tag":65,"props":3354,"children":3355},{},[3356],{"type":47,"value":3357},"not",{"type":47,"value":3359}," make it do anything. Unlike online rules — which the\neval worker applies asynchronously to already-ingested traffic, with zero agent changes — a guard\nis a ",{"type":41,"tag":65,"props":3361,"children":3362},{},[3363],{"type":47,"value":3364},"synchronous request-path policy the agent must call itself",{"type":47,"value":3366},". If the agent never calls the\nhooks endpoint, the guard is inert: it exists, shows ",{"type":41,"tag":56,"props":3368,"children":3370},{"className":3369},[],[3371],{"type":47,"value":536},{"type":47,"value":3373},", and never fires. That is also why\nStep 6 would show no ",{"type":41,"tag":56,"props":3375,"children":3377},{"className":3376},[],[3378],{"type":47,"value":102},{"type":47,"value":3380},"s — nothing invoked the guard.",{"type":41,"tag":50,"props":3382,"children":3383},{},[3384],{"type":47,"value":3385},"So every guard applied in Step 5 has a matching code change the developer owns. This skill does not\nedit or redeploy the agent (see Rules), but it MUST tell the developer exactly what to add, per\nguard, or the guard is dead config. Present this as part of the hand-off — don't leave the guard\nlooking \"done\" after Step 5.",{"type":41,"tag":50,"props":3387,"children":3388},{},[3389,3391,3396],{"type":47,"value":3390},"At each LLM call the agent evaluates the guard via the SDK and honors the verdict. Minimal Python\n(",{"type":41,"tag":56,"props":3392,"children":3394},{"className":3393},[],[3395],{"type":47,"value":149},{"type":47,"value":3397}," >= 0.11):",{"type":41,"tag":645,"props":3399,"children":3403},{"className":3400,"code":3401,"language":3402,"meta":653,"style":653},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","from agento11y import (\n    Client, ClientConfig, ApiConfig, HooksConfig,\n    HookEvaluateRequest, HookContext, HookModel, HookInput,\n    HookDeniedError, user_text_message,\n)\n\nclient = Client(config=ClientConfig(\n    api=ApiConfig(endpoint=\"https:\u002F\u002F\u003Cstack>.grafana.net\"),  # scheme+host; SDK appends the hooks path\n    hooks=HooksConfig(enabled=True, phases=[\"preflight\"], fail_open=False),\n))\n\n# preflight: evaluate the INPUT before the LLM call.\n# postflight: run it AFTER the call and pass the produced output instead.\nresp = client.evaluate_hook(HookEvaluateRequest(\n    phase=\"preflight\",\n    context=HookContext(\n        model=HookModel(provider=\"anthropic\", name=\"claude-...\"),\n        agent_name=\"\u003Cthe rule's match.agent_name>\",\n        agent_version=\"\u003Cv>\",\n        conversation_id=conversation_id,   # REQUIRED to record the guard on the conversation\n    ),\n    input=HookInput(messages=[user_text_message(prompt)]),   # postflight: output=[assistant_message]\n))\nif resp.is_deny:\n    raise HookDeniedError(reason=resp.reason, rule_id=resp.rule_id,\n                          evaluations=list(resp.evaluations))\n","python",[3404],{"type":41,"tag":56,"props":3405,"children":3406},{"__ignoreMap":653},[3407,3415,3423,3431,3439,3447,3456,3464,3472,3480,3488,3495,3503,3511,3519,3527,3535,3543,3551,3559,3568,3577,3586,3594,3603,3612],{"type":41,"tag":1885,"props":3408,"children":3409},{"class":1887,"line":1888},[3410],{"type":41,"tag":1885,"props":3411,"children":3412},{},[3413],{"type":47,"value":3414},"from agento11y import (\n",{"type":41,"tag":1885,"props":3416,"children":3417},{"class":1887,"line":1903},[3418],{"type":41,"tag":1885,"props":3419,"children":3420},{},[3421],{"type":47,"value":3422},"    Client, ClientConfig, ApiConfig, HooksConfig,\n",{"type":41,"tag":1885,"props":3424,"children":3425},{"class":1887,"line":1916},[3426],{"type":41,"tag":1885,"props":3427,"children":3428},{},[3429],{"type":47,"value":3430},"    HookEvaluateRequest, HookContext, HookModel, HookInput,\n",{"type":41,"tag":1885,"props":3432,"children":3433},{"class":1887,"line":1946},[3434],{"type":41,"tag":1885,"props":3435,"children":3436},{},[3437],{"type":47,"value":3438},"    HookDeniedError, user_text_message,\n",{"type":41,"tag":1885,"props":3440,"children":3441},{"class":1887,"line":2528},[3442],{"type":41,"tag":1885,"props":3443,"children":3444},{},[3445],{"type":47,"value":3446},")\n",{"type":41,"tag":1885,"props":3448,"children":3449},{"class":1887,"line":2551},[3450],{"type":41,"tag":1885,"props":3451,"children":3453},{"emptyLinePlaceholder":3452},true,[3454],{"type":47,"value":3455},"\n",{"type":41,"tag":1885,"props":3457,"children":3458},{"class":1887,"line":2574},[3459],{"type":41,"tag":1885,"props":3460,"children":3461},{},[3462],{"type":47,"value":3463},"client = Client(config=ClientConfig(\n",{"type":41,"tag":1885,"props":3465,"children":3466},{"class":1887,"line":2596},[3467],{"type":41,"tag":1885,"props":3468,"children":3469},{},[3470],{"type":47,"value":3471},"    api=ApiConfig(endpoint=\"https:\u002F\u002F\u003Cstack>.grafana.net\"),  # scheme+host; SDK appends the hooks path\n",{"type":41,"tag":1885,"props":3473,"children":3474},{"class":1887,"line":2605},[3475],{"type":41,"tag":1885,"props":3476,"children":3477},{},[3478],{"type":47,"value":3479},"    hooks=HooksConfig(enabled=True, phases=[\"preflight\"], fail_open=False),\n",{"type":41,"tag":1885,"props":3481,"children":3482},{"class":1887,"line":2614},[3483],{"type":41,"tag":1885,"props":3484,"children":3485},{},[3486],{"type":47,"value":3487},"))\n",{"type":41,"tag":1885,"props":3489,"children":3490},{"class":1887,"line":2655},[3491],{"type":41,"tag":1885,"props":3492,"children":3493},{"emptyLinePlaceholder":3452},[3494],{"type":47,"value":3455},{"type":41,"tag":1885,"props":3496,"children":3497},{"class":1887,"line":2664},[3498],{"type":41,"tag":1885,"props":3499,"children":3500},{},[3501],{"type":47,"value":3502},"# preflight: evaluate the INPUT before the LLM call.\n",{"type":41,"tag":1885,"props":3504,"children":3505},{"class":1887,"line":2673},[3506],{"type":41,"tag":1885,"props":3507,"children":3508},{},[3509],{"type":47,"value":3510},"# postflight: run it AFTER the call and pass the produced output instead.\n",{"type":41,"tag":1885,"props":3512,"children":3513},{"class":1887,"line":2682},[3514],{"type":41,"tag":1885,"props":3515,"children":3516},{},[3517],{"type":47,"value":3518},"resp = client.evaluate_hook(HookEvaluateRequest(\n",{"type":41,"tag":1885,"props":3520,"children":3521},{"class":1887,"line":2691},[3522],{"type":41,"tag":1885,"props":3523,"children":3524},{},[3525],{"type":47,"value":3526},"    phase=\"preflight\",\n",{"type":41,"tag":1885,"props":3528,"children":3529},{"class":1887,"line":2700},[3530],{"type":41,"tag":1885,"props":3531,"children":3532},{},[3533],{"type":47,"value":3534},"    context=HookContext(\n",{"type":41,"tag":1885,"props":3536,"children":3537},{"class":1887,"line":2709},[3538],{"type":41,"tag":1885,"props":3539,"children":3540},{},[3541],{"type":47,"value":3542},"        model=HookModel(provider=\"anthropic\", name=\"claude-...\"),\n",{"type":41,"tag":1885,"props":3544,"children":3545},{"class":1887,"line":2718},[3546],{"type":41,"tag":1885,"props":3547,"children":3548},{},[3549],{"type":47,"value":3550},"        agent_name=\"\u003Cthe rule's match.agent_name>\",\n",{"type":41,"tag":1885,"props":3552,"children":3553},{"class":1887,"line":2727},[3554],{"type":41,"tag":1885,"props":3555,"children":3556},{},[3557],{"type":47,"value":3558},"        agent_version=\"\u003Cv>\",\n",{"type":41,"tag":1885,"props":3560,"children":3562},{"class":1887,"line":3561},20,[3563],{"type":41,"tag":1885,"props":3564,"children":3565},{},[3566],{"type":47,"value":3567},"        conversation_id=conversation_id,   # REQUIRED to record the guard on the conversation\n",{"type":41,"tag":1885,"props":3569,"children":3571},{"class":1887,"line":3570},21,[3572],{"type":41,"tag":1885,"props":3573,"children":3574},{},[3575],{"type":47,"value":3576},"    ),\n",{"type":41,"tag":1885,"props":3578,"children":3580},{"class":1887,"line":3579},22,[3581],{"type":41,"tag":1885,"props":3582,"children":3583},{},[3584],{"type":47,"value":3585},"    input=HookInput(messages=[user_text_message(prompt)]),   # postflight: output=[assistant_message]\n",{"type":41,"tag":1885,"props":3587,"children":3589},{"class":1887,"line":3588},23,[3590],{"type":41,"tag":1885,"props":3591,"children":3592},{},[3593],{"type":47,"value":3487},{"type":41,"tag":1885,"props":3595,"children":3597},{"class":1887,"line":3596},24,[3598],{"type":41,"tag":1885,"props":3599,"children":3600},{},[3601],{"type":47,"value":3602},"if resp.is_deny:\n",{"type":41,"tag":1885,"props":3604,"children":3606},{"class":1887,"line":3605},25,[3607],{"type":41,"tag":1885,"props":3608,"children":3609},{},[3610],{"type":47,"value":3611},"    raise HookDeniedError(reason=resp.reason, rule_id=resp.rule_id,\n",{"type":41,"tag":1885,"props":3613,"children":3615},{"class":1887,"line":3614},26,[3616],{"type":41,"tag":1885,"props":3617,"children":3618},{},[3619],{"type":47,"value":3620},"                          evaluations=list(resp.evaluations))\n",{"type":41,"tag":50,"props":3622,"children":3623},{},[3624],{"type":47,"value":3625},"Three gotchas that silently break guards — call each out to the developer:",{"type":41,"tag":73,"props":3627,"children":3628},{},[3629,3666,3700],{"type":41,"tag":77,"props":3630,"children":3631},{},[3632,3641,3643,3649,3651,3657,3659,3664],{"type":41,"tag":65,"props":3633,"children":3634},{},[3635],{"type":41,"tag":56,"props":3636,"children":3638},{"className":3637},[],[3639],{"type":47,"value":3640},"fail_open=False",{"type":47,"value":3642}," — with the default ",{"type":41,"tag":56,"props":3644,"children":3646},{"className":3645},[],[3647],{"type":47,"value":3648},"True",{"type":47,"value":3650},", a transport error (or a disabled\u002Fmissing guard)\nresolves to ",{"type":41,"tag":56,"props":3652,"children":3654},{"className":3653},[],[3655],{"type":47,"value":3656},"allow",{"type":47,"value":3658},", so a ",{"type":41,"tag":56,"props":3660,"children":3662},{"className":3661},[],[3663],{"type":47,"value":110},{"type":47,"value":3665}," never actually blocks. Fail-closed is what makes the guard enforce.",{"type":41,"tag":77,"props":3667,"children":3668},{},[3669,3680,3682,3687,3688,3693,3695],{"type":41,"tag":65,"props":3670,"children":3671},{},[3672,3678],{"type":41,"tag":56,"props":3673,"children":3675},{"className":3674},[],[3676],{"type":47,"value":3677},"conversation_id",{"type":47,"value":3679}," in the context",{"type":47,"value":3681}," — without it, a ",{"type":41,"tag":56,"props":3683,"children":3685},{"className":3684},[],[3686],{"type":47,"value":110},{"type":47,"value":1241},{"type":41,"tag":56,"props":3689,"children":3691},{"className":3690},[],[3692],{"type":47,"value":102},{"type":47,"value":3694}," outcome is NOT persisted onto\nthe conversation, so it's invisible in the UI and Step 6 has nothing to watch. With it, the server\nrecords a \"Guard: ",{"type":41,"tag":1335,"props":3696,"children":3697},{},[3698],{"type":47,"value":3699},"\" workflow step on the conversation.",{"type":41,"tag":77,"props":3701,"children":3702},{},[3703,3713,3715,3720,3722,3727],{"type":41,"tag":65,"props":3704,"children":3705},{},[3706,3711],{"type":41,"tag":56,"props":3707,"children":3709},{"className":3708},[],[3710],{"type":47,"value":3656},{"type":47,"value":3712}," leaves no trace",{"type":47,"value":3714}," — the server persists only ",{"type":41,"tag":56,"props":3716,"children":3718},{"className":3717},[],[3719],{"type":47,"value":110},{"type":47,"value":3721}," and ",{"type":41,"tag":56,"props":3723,"children":3725},{"className":3724},[],[3726],{"type":47,"value":102},{"type":47,"value":3728}," outcomes; a clean pass\nrecords nothing (just a metric). So \"no guard step on the conversation\" is ambiguous — it means\neither allow OR not-wired. Distinguish them by whether ANY guard step ever appears for that agent.",{"type":41,"tag":50,"props":3730,"children":3731},{},[3732,3737,3739,3744,3746,3750,3752,3757,3758,3762,3764,3769,3771,3777,3778,3782,3784,3790,3792,3798,3800,3804,3806,3812,3814,3819,3820,3826,3828,3833],{"type":41,"tag":65,"props":3733,"children":3734},{},[3735],{"type":47,"value":3736},"Phase choice mirrors the guard",{"type":47,"value":3738}," (see the guard-type rules in Step 4): ",{"type":41,"tag":56,"props":3740,"children":3742},{"className":3741},[],[3743],{"type":47,"value":126},{"type":47,"value":3745}," is normally\n",{"type":41,"tag":65,"props":3747,"children":3748},{},[3749],{"type":47,"value":1618},{"type":47,"value":3751}," (redact the input before the model sees it; don't use it to scrub already-generated\nassistant text — see Step 4 for the tool-arg exception); ",{"type":41,"tag":56,"props":3753,"children":3755},{"className":3754},[],[3756],{"type":47,"value":134},{"type":47,"value":2347},{"type":41,"tag":65,"props":3759,"children":3760},{},[3761],{"type":47,"value":1668},{"type":47,"value":3763},".\nFor an evaluator-backed guard, phase follows ",{"type":41,"tag":56,"props":3765,"children":3767},{"className":3766},[],[3768],{"type":47,"value":1188},{"type":47,"value":3770},": ",{"type":41,"tag":56,"props":3772,"children":3774},{"className":3773},[],[3775],{"type":47,"value":3776},"target: input",{"type":47,"value":2935},{"type":41,"tag":65,"props":3779,"children":3780},{},[3781],{"type":47,"value":1618},{"type":47,"value":3783}," (evaluate\n",{"type":41,"tag":56,"props":3785,"children":3787},{"className":3786},[],[3788],{"type":47,"value":3789},"input.messages",{"type":47,"value":3791}," before the call — block a bad request before spending tokens); ",{"type":41,"tag":56,"props":3793,"children":3795},{"className":3794},[],[3796],{"type":47,"value":3797},"target: response",{"type":47,"value":3799},"\n→ ",{"type":41,"tag":65,"props":3801,"children":3802},{},[3803],{"type":47,"value":1668},{"type":47,"value":3805}," (evaluate ",{"type":41,"tag":56,"props":3807,"children":3809},{"className":3808},[],[3810],{"type":47,"value":3811},"input.output",{"type":47,"value":3813}," after — catch a bad response the model produced). The\nrule's ",{"type":41,"tag":56,"props":3815,"children":3817},{"className":3816},[],[3818],{"type":47,"value":852},{"type":47,"value":291},{"type":41,"tag":56,"props":3821,"children":3823},{"className":3822},[],[3824],{"type":47,"value":3825},"match.model",{"type":47,"value":3827}," scopes the guard to this agent; the agent's context must\nsend the same ",{"type":41,"tag":56,"props":3829,"children":3831},{"className":3830},[],[3832],{"type":47,"value":844},{"type":47,"value":538},{"type":41,"tag":50,"props":3835,"children":3836},{},[3837,3839,3845,3847,3852,3854,3860],{"type":47,"value":3838},"Other SDKs (JS, Go) expose the same ",{"type":41,"tag":56,"props":3840,"children":3842},{"className":3841},[],[3843],{"type":47,"value":3844},"evaluate_hook",{"type":47,"value":3846}," \u002F hooks-config shape; check the per-language SDK\nreference and verify the exact symbols against the installed ",{"type":41,"tag":56,"props":3848,"children":3850},{"className":3849},[],[3851],{"type":47,"value":149},{"type":47,"value":3853}," package. (As of writing the\ncanonical ",{"type":41,"tag":56,"props":3855,"children":3857},{"className":3856},[],[3858],{"type":47,"value":3859},"llms.txt",{"type":47,"value":3861}," does not yet carry a guard-instrumentation section, so don't defer to it for\nthis — the shape above is the reference.)",{"type":41,"tag":138,"props":3863,"children":3865},{"id":3864},"step-6-summarize-and-hand-off",[3866],{"type":47,"value":3867},"Step 6 — Summarize and hand off",{"type":41,"tag":50,"props":3869,"children":3870},{},[3871],{"type":47,"value":3872},"Output, in this order:",{"type":41,"tag":199,"props":3874,"children":3875},{},[3876,3907,3918,4038],{"type":41,"tag":77,"props":3877,"children":3878},{},[3879,3881,3885,3887,3892,3894,3898,3900,3905],{"type":47,"value":3880},"The three states from Step 3, kept distinct: ",{"type":41,"tag":65,"props":3882,"children":3883},{},[3884],{"type":47,"value":2044},{"type":47,"value":3886}," (each with its surface, evaluator\nkind, and ",{"type":41,"tag":56,"props":3888,"children":3890},{"className":3889},[],[3891],{"type":47,"value":2009},{"type":47,"value":3893}," — file:line or conversation\u002Fgeneration id); ",{"type":41,"tag":65,"props":3895,"children":3896},{},[3897],{"type":47,"value":2054},{"type":47,"value":3899},"\n(each with its one-line why); ",{"type":41,"tag":65,"props":3901,"children":3902},{},[3903],{"type":47,"value":3904},"Skipped as duplicates",{"type":47,"value":3906}," (what on the stack already covered it).\nDon't move an item between states between Step 3 and here.",{"type":41,"tag":77,"props":3908,"children":3909},{},[3910,3912,3916],{"type":47,"value":3911},"What was created vs. left as a draft: for each ",{"type":41,"tag":65,"props":3913,"children":3914},{},[3915],{"type":47,"value":2044},{"type":47,"value":3917}," object, the YAML path and whether\nit was applied via gcx. (Considered-not-recommended items were never drafted, so they have no\npath — don't list them here.)",{"type":41,"tag":77,"props":3919,"children":3920},{},[3921,3923],{"type":47,"value":3922},"The follow-through the developer still owns:\n",{"type":41,"tag":73,"props":3924,"children":3925},{},[3926,3979,4026],{"type":41,"tag":77,"props":3927,"children":3928},{},[3929,3934,3936,3941,3943,3948,3950,3955,3957,3962,3964,3969,3971,3977],{"type":41,"tag":65,"props":3930,"children":3931},{},[3932],{"type":47,"value":3933},"Guards:",{"type":47,"value":3935}," first, ",{"type":41,"tag":65,"props":3937,"children":3938},{},[3939],{"type":47,"value":3940},"wire the agent to call the guard (Step 5.5)",{"type":47,"value":3942}," — until that code ships the\nguard is inert and you'll see no ",{"type":41,"tag":56,"props":3944,"children":3946},{"className":3945},[],[3947],{"type":47,"value":102},{"type":47,"value":3949},"s, no matter how it's configured. State this per guard,\nwith the exact call to add. Then watch the ",{"type":41,"tag":56,"props":3951,"children":3953},{"className":3952},[],[3954],{"type":47,"value":102},{"type":47,"value":3956}," guards on real traffic and flip to ",{"type":41,"tag":56,"props":3958,"children":3960},{"className":3959},[],[3961],{"type":47,"value":110},{"type":47,"value":3963}," +\n",{"type":41,"tag":56,"props":3965,"children":3967},{"className":3966},[],[3968],{"type":47,"value":536},{"type":47,"value":3970}," only once the false-positive rate looks acceptable —\n",{"type":41,"tag":56,"props":3972,"children":3974},{"className":3973},[],[3975],{"type":47,"value":3976},"gcx agento11y guards update \u003Cid> -f ...",{"type":47,"value":3978},". Both the wiring and the flip are theirs to make, not\nthis skill's.",{"type":41,"tag":77,"props":3980,"children":3981},{},[3982,3987,3989,3994,3996,4002,4004,4009,4011,4016,4018,4024],{"type":41,"tag":65,"props":3983,"children":3984},{},[3985],{"type":47,"value":3986},"Rules:",{"type":47,"value":3988}," first confirm the rule is actually scoring — if ",{"type":41,"tag":56,"props":3990,"children":3992},{"className":3991},[],[3993],{"type":47,"value":1053},{"type":47,"value":3995}," is absent or\n",{"type":41,"tag":56,"props":3997,"children":3999},{"className":3998},[],[4000],{"type":47,"value":4001},"total_scores",{"type":47,"value":4003}," stays 0 on a matching agent, the ",{"type":41,"tag":56,"props":4005,"children":4007},{"className":4006},[],[4008],{"type":47,"value":1210},{"type":47,"value":4010}," is likely wrong for the agent shape\n(see Step 1), not the sample rate. Once scores appear, raise ",{"type":41,"tag":56,"props":4012,"children":4014},{"className":4013},[],[4015],{"type":47,"value":554},{"type":47,"value":4017}," as they look sane and\ncost is understood (",{"type":41,"tag":56,"props":4019,"children":4021},{"className":4020},[],[4022],{"type":47,"value":4023},"gcx agento11y rules update",{"type":47,"value":4025},"); add alerting on regressions if wanted.",{"type":41,"tag":77,"props":4027,"children":4028},{},[4029,4031,4036],{"type":47,"value":4030},"Inspect everything in Agent Observability (rules\u002Fguards\u002Fevaluators pages, the conversation\nQuality view) or via the ",{"type":41,"tag":56,"props":4032,"children":4034},{"className":4033},[],[4035],{"type":47,"value":355},{"type":47,"value":4037}," list and get commands.",{"type":41,"tag":77,"props":4039,"children":4040},{},[4041,4043,4048,4050,4055],{"type":47,"value":4042},"A one-line pointer back: for pre-ship offline evaluation of a new agent or version,\n",{"type":41,"tag":56,"props":4044,"children":4046},{"className":4045},[],[4047],{"type":47,"value":61},{"type":47,"value":4049}," is the counterpart; for control-plane mechanics, the ",{"type":41,"tag":56,"props":4051,"children":4053},{"className":4052},[],[4054],{"type":47,"value":149},{"type":47,"value":4056}," skill.",{"type":41,"tag":4058,"props":4059,"children":4060},"style",{},[4061],{"type":47,"value":4062},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":4064,"total":4241},[4065,4084,4103,4123,4133,4149,4156,4170,4187,4202,4215,4229],{"slug":4066,"name":4066,"fn":4067,"description":4068,"org":4069,"tags":4070,"stars":4081,"repoUrl":4082,"updatedAt":4083},"faro-setup-web","instrument web apps with Grafana Faro","Instruments a web app with Grafana Faro Web SDK for frontend observability. Use when setting up error tracking, Web Vitals, session monitoring, or distributed tracing in a browser app.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4071,4074,4077,4080],{"name":4072,"slug":4073,"type":15},"Distributed Tracing","distributed-tracing",{"name":4075,"slug":4076,"type":15},"Frontend","frontend",{"name":4078,"slug":4079,"type":15},"Monitoring","monitoring",{"name":13,"slug":14,"type":15},1103,"https:\u002F\u002Fgithub.com\u002Fgrafana\u002Ffaro-web-sdk","2026-07-12T07:43:24.63314",{"slug":4085,"name":4085,"fn":4086,"description":4087,"org":4088,"tags":4089,"stars":4100,"repoUrl":4101,"updatedAt":4102},"configuring-yesoreyeram-infinity-datasource","configure Grafana Infinity data source","Configure the Infinity data source — base URL and allowed hosts, the authentication methods (basic, bearer token, API key, digest, OAuth passthrough, OAuth 2.0 client credentials\u002FJWT, Azure, Azure Blob, AWS), TLS, custom HTTP headers, network and security settings, the custom health check, and provisioning with a config file. Use when a user asks how to set up, configure, or change settings for the Infinity data source; how to authenticate to an API; how to allow hosts; how to provision it as YAML; or how to troubleshoot connection, authentication, or health-check issues.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4090,4093,4096,4099],{"name":4091,"slug":4092,"type":15},"API Development","api-development",{"name":4094,"slug":4095,"type":15},"Authentication","authentication",{"name":4097,"slug":4098,"type":15},"Configuration","configuration",{"name":9,"slug":8,"type":15},1056,"https:\u002F\u002Fgithub.com\u002Fgrafana\u002Fgrafana-infinity-datasource","2026-07-12T07:43:25.939136",{"slug":4104,"name":4104,"fn":4105,"description":4106,"org":4107,"tags":4108,"stars":4100,"repoUrl":4101,"updatedAt":4122},"querying-yesoreyeram-infinity-datasource","query data with Infinity datasource","Build queries with the Infinity data source — the query types (JSON, CSV, TSV, XML, GraphQL, HTML, UQL, GROQ, Google Sheets, Series, Transformations), the parsers (Frontend\u002Fsimple, Backend JSONata, JQ, UQL, GROQ) and which support alerting, the sources (URL, Inline, Reference, Azure Blob, Random walk), output formats (table, timeseries, logs, trace, node graph, dataframe), root selector and columns, computed columns\u002Ffilters\u002Fsummarize, pagination, and template variables. Use when a user asks how to query Infinity; how to fetch JSON\u002FCSV\u002FXML\u002FGraphQL\u002FHTML from a URL or inline; how to select rows and columns; how to transform data with UQL\u002FGROQ\u002FJSONata\u002FJQ; how to make a query work with alerting; how to paginate; or how to use variables in a query.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4109,4112,4115,4116,4119],{"name":4110,"slug":4111,"type":15},"CSV","csv",{"name":4113,"slug":4114,"type":15},"Data Analysis","data-analysis",{"name":9,"slug":8,"type":15},{"name":4117,"slug":4118,"type":15},"GraphQL","graphql",{"name":4120,"slug":4121,"type":15},"JSON","json","2026-07-15T05:34:05.773947",{"slug":149,"name":149,"fn":4124,"description":4125,"org":4126,"tags":4127,"stars":23,"repoUrl":24,"updatedAt":4132},"manage Grafana Agent Observability resources","Inspects and manages Grafana Agent Observability resources via gcx: conversations, generations, evaluators, rules, scores, and templates. Use when the user wants to list or search conversations, inspect generations, manage evaluators (upsert, test, delete), set up evaluation rules, check scores, or browse evaluator templates. Trigger on phrases like \"list conversations\", \"search generations\", \"what did the agent do\", \"debug LLM conversation\", \"create evaluator\", \"set up evaluation rule\", \"test evaluator\", \"check scores\", \"evaluate generation quality\", or \"set up online evaluation\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4128,4129,4130,4131],{"name":20,"slug":21,"type":15},{"name":9,"slug":8,"type":15},{"name":4078,"slug":4079,"type":15},{"name":13,"slug":14,"type":15},"2026-07-25T05:30:40.29622",{"slug":4134,"name":4134,"fn":4135,"description":4136,"org":4137,"tags":4138,"stars":23,"repoUrl":24,"updatedAt":4148},"agento11y-instrument","instrument LLM apps for agent observability","Sets up and instruments a developer's own LLM app or agent to send generations and agentic workflow to Grafana Agent Observability (the Agent Observability SDKs) — greenfield setup, fixing broken instrumentation, or filling gaps in existing instrumentation. Uses gcx for the parts a static prompt can't do: `gcx login` \u002F `gcx cloud stacks` to find the stack, and `gcx agento11y agents|conversations|generations` to VERIFY that data actually lands — so it iterates (instrument → run → verify → fix) until generations arrive, not blindly. Reads the app's code, detects language\u002Fframework, classifies instrumentation state (none \u002F partial \u002F broken), then runs a fixed gap checklist whose #1 item is the silent failure no other prompt catches: the SDK emits OTel spans\u002Fmetrics but never creates a TracerProvider\u002FMeterProvider, so without them all metrics go to a no-op and are lost. Also checks agent_version (required for per-version Performance charts), set_result completeness, SYNC vs STREAM, parent_generation_ids DAG links, and workflow-step coverage. Recommends changes citing file:line and, only with explicit confirmation, applies minimal diffs that don't change app behavior. Pulls SDK reference from agento11y's llms.txt rather than restating it, and hands off to `agento11y-test-starter` once data flows. It does NOT write test suites or set up tenant evaluations, rules, or guards — offline test suites are `agento11y-test-starter`, tenant eval rules + guards are `agento11y-prod-setup`; does NOT install coding-agent telemetry plugins (that is llms.txt \"Path A\"); does NOT mint or store credentials or invent endpoints. Trigger on phrases like \"instrument my app\", \"send my agent's traces to Grafana\", \"set up AI observability for my app\", \"my generations aren't showing up\", \"why is Performance empty\", \"add Agent Observability to my code\", \"fix my instrumentation\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4139,4140,4141,4144,4147],{"name":20,"slug":21,"type":15},{"name":9,"slug":8,"type":15},{"name":4142,"slug":4143,"type":15},"Instrumentation","instrumentation",{"name":4145,"slug":4146,"type":15},"LLM","llm",{"name":13,"slug":14,"type":15},"2026-07-31T05:53:52.580237",{"slug":4,"name":4,"fn":5,"description":6,"org":4150,"tags":4151,"stars":23,"repoUrl":24,"updatedAt":25},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4152,4153,4154,4155],{"name":20,"slug":21,"type":15},{"name":17,"slug":18,"type":15},{"name":9,"slug":8,"type":15},{"name":13,"slug":14,"type":15},{"slug":61,"name":61,"fn":4157,"description":4158,"org":4159,"tags":4160,"stars":23,"repoUrl":24,"updatedAt":4169},"build and run agent test suites","Use early in an AI-agent project — before ship, before real traffic — to build a starter test suite for the agent and run it offline. Reads the agent's own code (system prompt, tools, task), writes a labeled draft suite of test cases (happy\u002Fedge\u002Fadversarial) grounded in real lines, and recommends how to score each case (the evaluators\u002Fjudges the offline runner uses). Assesses how runnable the agent is: for an easily-invoked agent it generates a runner stub (run_experiment.py) with two holes to fill and can optionally run it (only with permission, only against the endpoint the developer configured); for agents needing a harness or full runtime it points to the existing eval infra. It runs OFFLINE and never creates tenant-level evaluators, rules, or guards — that is `agento11y-prod-setup`, for a deployed agent with real traffic. Trigger on phrases like \"how do I test my agent before shipping\", \"write test cases for my agent\", \"set up tests for my agent\", \"check my agent before prod\", \"I have no traffic yet, how do I evaluate it\", \"test my agent offline\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4161,4162,4163,4166],{"name":20,"slug":21,"type":15},{"name":9,"slug":8,"type":15},{"name":4164,"slug":4165,"type":15},"QA","qa",{"name":4167,"slug":4168,"type":15},"Testing","testing","2026-07-31T05:53:51.62785",{"slug":4171,"name":4171,"fn":4172,"description":4173,"org":4174,"tags":4175,"stars":23,"repoUrl":24,"updatedAt":4186},"create-dashboard","create Grafana dashboards with gcx","Designs and creates Grafana dashboards with gcx, using `gcx dashboards snapshot` as a visual feedback loop. Use when the user wants to create a new Grafana dashboard, add panels, variables, or annotations to an existing dashboard, design dashboard panels, variables, queries, or layout, or make a material visual redesign. Triggers on \"create dashboard\", \"new dashboard\", \"build dashboard\", \"dashboard for \u003Cservice>\", \"add panels\", \"add variable\", \"add annotation\", \"improve this dashboard\", or \"iterate on a dashboard\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4176,4179,4182,4185],{"name":4177,"slug":4178,"type":15},"Dashboards","dashboards",{"name":4180,"slug":4181,"type":15},"Data Visualization","data-visualization",{"name":4183,"slug":4184,"type":15},"Design","design",{"name":9,"slug":8,"type":15},"2026-07-25T05:30:46.289717",{"slug":4188,"name":4188,"fn":4189,"description":4190,"org":4191,"tags":4192,"stars":23,"repoUrl":24,"updatedAt":4201},"debug-with-grafana","investigate application issues with Grafana","Structured workflow for investigating application problems with Grafana observability data (metrics, logs, traces) via gcx. Covers live firefighting AND retrospective incident analysis: incident triage, root-cause analysis, blast-radius checks (did an incident spill into other services), verifying whether a deployment or rollout triggered an incident, finding which service, endpoint, or path owns the most errors or slow requests, checking whether retries or queue backlogs piled up, and quantifying error or latency shares over a time window. Trigger on: \"my API is returning 500 errors\", \"latency is spiking\", \"investigate why requests are failing\", \"triage the incident\", \"blast radius\", \"root cause\", \"did the rollout cause it\", \"which endpoint owns the most 5xx\", \"did retries pile up\", or any request to analyse an earlier incident window using telemetry. For authoring dashboards use create-dashboard; for dashboard inventory use manage-dashboards.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4193,4196,4197,4200],{"name":4194,"slug":4195,"type":15},"Debugging","debugging",{"name":9,"slug":8,"type":15},{"name":4198,"slug":4199,"type":15},"Incident Response","incident-response",{"name":13,"slug":14,"type":15},"2026-07-18T05:11:10.445428",{"slug":4203,"name":4203,"fn":4204,"description":4205,"org":4206,"tags":4207,"stars":23,"repoUrl":24,"updatedAt":4214},"diagnose-entity-graph","diagnose Grafana Entity Graph issues","Diagnose Entity Graph problems: missing entities, missing edges, disconnected clusters, or filtering issues. Use when the user reports that Entity Graph doesn't look right, services are missing, edges aren't appearing, or environments can't be filtered. Triggers for: \"entity graph is empty\", \"services missing from entity graph\", \"no edges in entity graph\", \"disconnected services\", \"can't filter entity graph\", \"entity graph not working\", \"diagnose entity graph\", \"debug knowledge graph\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4208,4209,4210,4213],{"name":4194,"slug":4195,"type":15},{"name":9,"slug":8,"type":15},{"name":4211,"slug":4212,"type":15},"Graph Analysis","graph-analysis",{"name":13,"slug":14,"type":15},"2026-07-25T05:30:39.380934",{"slug":363,"name":363,"fn":4216,"description":4217,"org":4218,"tags":4219,"stars":23,"repoUrl":24,"updatedAt":4228},"manage Grafana Cloud resources via gcx","Manages Grafana Cloud resources via the gcx CLI. Trigger when the user wants to inspect, create, update, delete, query, or automate any Grafana resource - dashboards, datasources, alerts, SLOs, synthetic checks, oncall, incidents, fleet, k6, knowledge graph, or adaptive telemetry.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4220,4223,4224,4225],{"name":4221,"slug":4222,"type":15},"CLI","cli",{"name":9,"slug":8,"type":15},{"name":4078,"slug":4079,"type":15},{"name":4226,"slug":4227,"type":15},"Operations","operations","2026-07-31T05:53:50.587304",{"slug":4230,"name":4230,"fn":4231,"description":4232,"org":4233,"tags":4234,"stars":23,"repoUrl":24,"updatedAt":4240},"gcx-demo","present gcx demo tours","Run a narrated, read-only demo tour of gcx for customer or colleague presentations. Showcases the breadth of gcx across every Grafana Cloud product area — resources, datasources, metrics, logs, traces, SLOs, alerts, synthetic monitoring, IRM, k6, fleet, and more. All commands are strictly read-only. Trigger when the user says \"demo gcx\", \"show off gcx\", \"customer demo\", \"gcx tour\", or \"\u002Fgcx-demo\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4235,4236,4237],{"name":4221,"slug":4222,"type":15},{"name":9,"slug":8,"type":15},{"name":4238,"slug":4239,"type":15},"Presentations","presentations","2026-07-25T05:30:45.282458",80,{"items":4243,"total":3596},[4244,4251,4259,4266,4273,4280,4287],{"slug":149,"name":149,"fn":4124,"description":4125,"org":4245,"tags":4246,"stars":23,"repoUrl":24,"updatedAt":4132},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4247,4248,4249,4250],{"name":20,"slug":21,"type":15},{"name":9,"slug":8,"type":15},{"name":4078,"slug":4079,"type":15},{"name":13,"slug":14,"type":15},{"slug":4134,"name":4134,"fn":4135,"description":4136,"org":4252,"tags":4253,"stars":23,"repoUrl":24,"updatedAt":4148},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4254,4255,4256,4257,4258],{"name":20,"slug":21,"type":15},{"name":9,"slug":8,"type":15},{"name":4142,"slug":4143,"type":15},{"name":4145,"slug":4146,"type":15},{"name":13,"slug":14,"type":15},{"slug":4,"name":4,"fn":5,"description":6,"org":4260,"tags":4261,"stars":23,"repoUrl":24,"updatedAt":25},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4262,4263,4264,4265],{"name":20,"slug":21,"type":15},{"name":17,"slug":18,"type":15},{"name":9,"slug":8,"type":15},{"name":13,"slug":14,"type":15},{"slug":61,"name":61,"fn":4157,"description":4158,"org":4267,"tags":4268,"stars":23,"repoUrl":24,"updatedAt":4169},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4269,4270,4271,4272],{"name":20,"slug":21,"type":15},{"name":9,"slug":8,"type":15},{"name":4164,"slug":4165,"type":15},{"name":4167,"slug":4168,"type":15},{"slug":4171,"name":4171,"fn":4172,"description":4173,"org":4274,"tags":4275,"stars":23,"repoUrl":24,"updatedAt":4186},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4276,4277,4278,4279],{"name":4177,"slug":4178,"type":15},{"name":4180,"slug":4181,"type":15},{"name":4183,"slug":4184,"type":15},{"name":9,"slug":8,"type":15},{"slug":4188,"name":4188,"fn":4189,"description":4190,"org":4281,"tags":4282,"stars":23,"repoUrl":24,"updatedAt":4201},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4283,4284,4285,4286],{"name":4194,"slug":4195,"type":15},{"name":9,"slug":8,"type":15},{"name":4198,"slug":4199,"type":15},{"name":13,"slug":14,"type":15},{"slug":4203,"name":4203,"fn":4204,"description":4205,"org":4288,"tags":4289,"stars":23,"repoUrl":24,"updatedAt":4214},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4290,4291,4292,4293],{"name":4194,"slug":4195,"type":15},{"name":9,"slug":8,"type":15},{"name":4211,"slug":4212,"type":15},{"name":13,"slug":14,"type":15}]