[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-datadog-labs-agent-observability-replay-trace":3,"mdc--69gd3j-key":39,"related-repo-datadog-labs-agent-observability-replay-trace":1639,"related-org-datadog-labs-agent-observability-replay-trace":1735},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":28,"repoUrl":29,"updatedAt":30,"license":31,"forks":32,"topics":33,"repo":34,"sourceUrl":37,"mdContent":38},"agent-observability-replay-trace","iterate on LLM traces with local code","Use when a developer wants to iterate on ONE specific Agent Observability \u002F LLM Obs trace whose output they didn't like — re-running that trace against their LOCAL code, seeing a concise diff of the old vs new output, and looping (change code → replay → diff) until satisfied. Invoked as \u002Fagent-observability-replay-trace \u003Ctrace-id> [changes to test]. Signals: \"replay this trace\"; \"iterate on a trace\"; \"this trace's output is wrong, fix it and re-run\"; \"re-run trace \u003Cid> with \u003Cchange>\"; pasting a trace id from the Agent Observability UI with a description of what to fix. It fetches the trace via the datadog-llmo MCP or the pup CLI, edits code, re-runs the app to emit a NEW trace, and diffs the two — no local server, no browser. For agents traced with ddtrace \u002F LLM Obs (Python first-class), with JSON-serializable entry input. Do NOT use for: scored Experiments or the browser \"Replay\" button (that's agent-observability-replay-experiment), building an experiment from a dataset\u002FCSV, writing evaluators, root-causing failed traces, or RUM\u002FHTTP session replay.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"datadog-labs","Datadog Labs","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fdatadog-labs.png",[12,16,19,22,25],{"name":13,"slug":14,"type":15},"Tracing","tracing","tag",{"name":17,"slug":18,"type":15},"Observability","observability",{"name":20,"slug":21,"type":15},"LLM","llm",{"name":23,"slug":24,"type":15},"Datadog","datadog",{"name":26,"slug":27,"type":15},"Debugging","debugging",145,"https:\u002F\u002Fgithub.com\u002Fdatadog-labs\u002Fagent-skills","2026-07-31T05:52:08.594182",null,19,[],{"repoUrl":29,"stars":28,"forks":32,"topics":35,"description":36},[],"Public repository for Datadog Agent Skills","https:\u002F\u002Fgithub.com\u002Fdatadog-labs\u002Fagent-skills\u002Ftree\u002FHEAD\u002Fagent-observability\u002Fagent-observability-replay-trace","---\nname: agent-observability-replay-trace\ndescription: >-\n  Use when a developer wants to iterate on ONE specific Agent Observability \u002F LLM Obs trace whose output\n  they didn't like — re-running that trace against their LOCAL code, seeing a concise diff of the old vs\n  new output, and looping (change code → replay → diff) until satisfied. Invoked as\n  \u002Fagent-observability-replay-trace \u003Ctrace-id> [changes to test]. Signals: \"replay this trace\"; \"iterate on\n  a trace\"; \"this trace's output is wrong, fix it and re-run\"; \"re-run trace \u003Cid> with \u003Cchange>\"; pasting a\n  trace id from the Agent Observability UI with a description of what to fix. It fetches the trace via the\n  datadog-llmo MCP or the pup CLI, edits code, re-runs the app to emit a NEW trace, and\n  diffs the two — no local server, no browser. For agents traced with ddtrace \u002F LLM Obs (Python first-class), with JSON-serializable entry\n  input. Do NOT use for: scored Experiments or the browser \"Replay\" button (that's\n  agent-observability-replay-experiment), building an experiment from a dataset\u002FCSV, writing evaluators,\n  root-causing failed traces, or RUM\u002FHTTP session replay.\n---\n\n# Replay a trace against local code\n\nA fast **iteration loop** on a single production trace: take a trace whose output a developer didn't like,\noptionally change the code, **re-run it against their LOCAL code**, and show a concise diff of old vs new\noutput — repeating until they're happy. Assumes nothing about the project's layout.\n\nInvoked from the developer's coding agent: `\u002Fagent-observability-replay-trace \u003Ctrace-id> [\u003Cchanges to test>]`.\nWith no modification, do the replay + diff only (a reproduce\u002Fregression check), then offer to enter the loop.\n\n**This file is the workflow spine — terse on purpose. The depth lives in `references\u002Fdetails.md` (trace\nbackend + pup flags, the runner contract, export mode, polling, the trace-link scoping fix) and\n`references\u002Flocal-setup.md` (making a deployed-only app locally runnable). Read `details.md` before you touch\npup or generate the runner.**\n\n**Writing code — keep comments minimal to none.** Everything you generate or edit (the annotation, the\nrunner's `ENTRYPOINTS` entries, a local harness, iteration edits) should match the surrounding code and\ncarry **no unnecessary comments** — don't narrate what the code plainly does; add a comment only for a\ngenuinely non-obvious *why*.\n\n## Interaction model — selector gates, never a hard stop\n\nThis is a live loop. At every decision point present the choices as an **`AskUserQuestion` selector** (the\nplan-mode-style menu), not a plain question that ends your turn. Two gates: (a) after you propose code\nchanges, before replaying; (b) after each diff. The selector's free-text option lets the user type detail\n(what to refine) inline — act on it directly, don't ask a follow-up. Keep re-presenting after every replay\nuntil they pick \"stop here\".\n\n## Scope — check first\n\n- **Traced with `ddtrace` \u002F LLM Obs** (an `ml_app` + a discoverable entrypoint). **Python is first-class**;\n  other languages work but you write the runner to the contract in their SDK\u002Fbuild tooling.\n- **JSON-serializable entrypoint input**, and a **callable seam** for the root span (see step 3.5 — not a\n  binary \"is it runnable?\"; deployed-only apps often still expose a plain callable).\n- **A trace-access backend** — the `datadog-llmo` MCP (used when present) or the `pup` CLI (fallback, and\n  the easier install if you have neither) (step 0).\n- **Credentials:** `DD_API_KEY` + `DD_SITE` + provider key(s). **Not `DD_APP_KEY`** — plain trace, not an\n  Experiment (that's `agent-observability-replay-experiment`).\n- **Side effects, irreversible:** replaying re-runs real code (model spend + real writes), and **LLM Obs\n  traces cannot be deleted** — a mis-scoped replay (wrong ml_app) *permanently* pollutes the production app's\n  dashboards\u002Feval sets. That's why the `\u003Cml_app>-local` isolation (steps 4\u002F6\u002F7) is load-bearing, not tidy.\n  Warn before the first replay.\n\n## Workflow\n\n### 0. Ensure a trace-access backend\nPick, in order: (1) the **MCP** if `mcp__datadog-llmo-mcp__*` tools are present — the default (slightly\nricher for reads: structured tree + `content_info`); (2) else **`pup`** if installed and `pup auth` targets\nthe app's org; (3) else the user has neither → guide the **pup install** (it's easier to set up than the\nMCP, so recommend pup here):\n```\nbrew tap datadog-labs\u002Fpack && brew install datadog-labs\u002Fpack\u002Fpup\npup auth login\n```\n(MCP alternative: `claude mcp add --scope user --transport http \"datadog-llmo-mcp\" \"https:\u002F\u002Fmcp.datadoghq.com\u002Fapi\u002Funstable\u002Fmcp-server\u002Fmcp?toolsets=llmobs\"`; see https:\u002F\u002Fdocs.datadoghq.com\u002Fbits_ai\u002Fmcp_server\u002Fsetup\u002F.) Don't proceed without a backend.\nThe backend↔operation mapping and **pup's exact flags\u002Fgotchas are in `details.md` — read that section before\nusing pup.** Two pup musts: (1) results come back at **`data.spans[]`** *or* top-level **`spans[]`**\n(varies by version\u002F`--no-agent`) — parse **whichever is present**, or you get zero hits on an ingested\ntrace (a silent false negative, step 7); (2) check **token expiry** (`pup auth status`), not just that auth\nexists — expiry mid-loop looks like \"trace not found.\"\n\n### 1. Parse the command\n`\u003Ctrace-id>` + optional free-text modification (everything after the id); none → diff-only mode. Determine\nthe `ml_app` from the project (`LLMObs.enable(ml_app=…)` \u002F `DD_LLMOBS_ML_APP`) or the trace; confirm if\nambiguous.\n\n### 2. Fetch the trace + locate the baseline\nFetch via the backend; note `total_duration_ms` (drives step 7), the `trace_url`, and\n`metadata.replay_input`\u002F`replay_entrypoint` if present. **Locate the baseline field — it's not always the\nroot output:** the value the developer dislikes may be a tool-call input or an intermediate output several\nlevels deep, and the app may post-process it before the span records it. Pick the field the code change can\nactually move, or the delta drowns in noise.\n\n### 2.5. Check for fan-out\nIf the root span **fans out into repeated sibling subtrees** (a batch\u002Fmap over N parallel sub-runs), the\nchange under test is usually visible in a **single** branch — replaying the whole root costs ~N× spend and\ntime for no extra signal. Offer to replay one representative branch; **log what you skipped**. **Pick deliberately: the cheapest\nbranch that reached the terminal \u002F side-effecting tool** (most branches are no-ops that prove nothing), and\nreconstruct its input from the **child** span's input, not the root's. Full root only if the change is\ninherently cross-branch.\n\n### 3. Resolve the entrypoint + input\n- **Entrypoint:** `metadata.replay_entrypoint` if present; else infer from the root span (name\u002Fkind) + code\n  and **confirm with the user**.\n- **Input:** `metadata.replay_input` if present; else derive a **suggested** input (prefer the code\n  signature — the rendered prompt is lossy) and have the user **confirm\u002Fedit**.\n\n### 3.5. Ensure a local run path (find the innermost callable seam)\nAsk **\"what is the innermost callable seam for this root span, and can I call it directly with JSON?\"** — not\n\"is the app runnable?\". Already directly callable → **skip, continue**. Buried under a handler\u002Fservice\n(deployed-only, no local `__main__`, live-infra coupling) → follow **`references\u002Flocal-setup.md`** (detect →\npropose → approve → build). The common middle case — a deployed service whose core logic is *already* a plain\ncallable (ports-and-adapters) — just extract\u002Fcall that seam; full local-setup is overkill.\n\n### 4. Ensure the two persistent artifacts (one-time setup)\n- **a) In-entrypoint annotation** on the app's **real** entrypoint, so all future traces (production too)\n  self-describe. Stamp it at **span start, not the success\u002Fdeferred-finish path** — a failed run must still\n  carry `replay_input` (those are the ones you most want to replay):\n  ```python\n  LLMObs.annotate(span=span, metadata={\"replay_entrypoint\": \"\u003Cstable id>\", \"replay_input\": \u003Cextractor>})\n  ```\n  No `replay_output` — the original trace is the baseline. (Non-Python annotate APIs differ — e.g. Go\n  `span.Annotate(llmobs.WithAnnotatedMetadata(...))`; see `details.md`.)\n- **Isolation pre-flight (before writing the runner):** grep the entrypoint's call path for **per-span\u002F\n  per-call ml_app overrides** (Go `llmobs.WithMLApp`; Python `ml_app=` on a decorator or in\n  `LLMObs.annotate`). Those **beat** the init-level `-local`, so the app's spans can still land in\n  production — tracer-level config is **not** proof of isolation. If any exist, the app's ml_app must\n  resolve from env so `-local` wins.\n- **b) The runner** — satisfies the **language-independent runner contract in `details.md`** (load env →\n  derive `\u003Cml_app>-local` → dispatch one entrypoint on JSON → **flush on every exit path incl. errors** →\n  **refuse to start unless ml_app ends in `-local`** → print the `-local` ml_app). **Python:** copy\n  `scripts\u002Freplay_runner_template.py` and fill `ENTRYPOINTS`. **Other languages:** write to the contract —\n  don't assume the Python API carries over (Go APIs + export-mode gotchas in `details.md`), and where the\n  language has no in-process dotenv add a **run wrapper (artifact c)** that sources the project env, unsets\n  ambient provider vars, and exports the `-local` override. Infer + **confirm the run command**; follow the\n  host repo's **build-file conventions** (Bazel\u002FGazelle → `cmd\u002F\u003Cname>\u002F`, run Gazelle, build before replay).\n\n### 5. (If a change was requested) edit, then gate\nMake the code changes, show the developer the diff of your changes, then an `AskUserQuestion` selector:\n**Replay now** \u002F **Adjust the changes first** \u002F **Cancel**. Only replay on \"Replay now\".\n\n### 6. Replay\nBefore the first replay: **warn** (re-running is real — model spend + real writes), and **sanitize the\nenvironment**. The **coding agent's own env** (`ANTHROPIC_API_KEY` \u002F `ANTHROPIC_BASE_URL` set by Claude\nCode, and other provider keys) can make the app's SDK **bypass its configured model gateway** — a fidelity\ngap **invisible in the diff**. **Unset ambient provider vars by default and report that you did** (don't\njust ask); grep the app for its own ambient-key guards. Also **verify the credential's org matches the\ntrace's org** — a mismatch ships the replay somewhere you can't query (looks like ingest lag).\nOn confirmation, record `t0` and run — **source the project's env file, never inline secrets** (the marker\ntag is fine on the command; `DD_API_KEY=\u003Cvalue>` inline is blocked by the permission classifier and leaks to\nhistory\u002Ftranscript — use the wrapper\u002Fenv-file):\n```\nDD_TAGS=replay_run_id:\u003Cunique-id> \u003Crun cmd or wrapper> --entrypoint \u003Cid> --input-file \u003Cpath>\n```\nThe runner emits **under `\u003Cml_app>-local`** (idempotent, so replays never pollute production) and prints that\nname — poll for the new trace **under it**.\n\n### 7. Wait for the new trace\n- **Runner subprocess timeout** = `max(120s, ~3 × total_duration_ms)`.\n- **Ingest poll:** after it returns, poll the backend **every ~5s up to ~2 min** for the `replay_run_id` tag\n  under `\u003Cml_app>-local` (pup: `--query \"replay_run_id:\u003Cid>\"`, plain `key:value`). **Before ever reporting\n  \"not found,\" re-query with no tag filter** (just `\u003Cml_app>-local` + window): if that returns spans, your\n  filter\u002Fparse\u002Fscope is wrong — **not** ingestion. A false \"no trace\" reads as normal and invites a wasteful\n  re-run.\n- **Verify isolation on each hit — a tag match is NOT proof.** `--query`\u002Ftag matching can return a span\n  whose real `ml_app` is a *different* app (the `--ml-app` filter gets ignored). Read `ml_app` off every\n  returned span and **assert it ends in `-local`** before reporting a clean replay — otherwise you report\n  \"clean replay under `-local`\" while the trace is actually in production (which you can't undo). This false\n  *confidence* is worse than the false negative. Don't hard-fail on timeout; offer to keep waiting.\n\n### 8. Diff (with links to both traces)\nConcise summary of how the **new output differs from the old** — meaningful differences only. Note live-world\ndrift; and because any nondeterministic agent varies run-to-run, **default to two replays** (diff-only mode\ntoo, not just model-facing edits) and use **replay-to-replay comparison** — if the two local runs differ\nfrom each other about as much as from production, the delta is sampling variance, not your change. If the\nreplay **disables a side-effecting integration** (dry-run), that integration's subtree is absent — **exclude\nit from both sides** before comparing span counts, or the structural diff is junk. Lead the diff with both\ntrace links:\n- **Old:** `trace_url` **verbatim** — but **under fan-out** (you replayed one branch) link the **branch\n  span**, not the whole-root url.\n- **New (replay):** must carry `ml_app=\u003Cml_app>-local` or it opens **empty** — and the `trace_url` is an\n  org-switch wrapper (`…\u002Fswitch_to_user\u002F\u003Cid>?next=\u003Cencoded \u002Fllm\u002Ftraces …>&flow=org_switch`), so **inject\n  `ml_app=\u003Cml_app>-local` into the decoded `next` query and re-encode; do NOT append to the outer URL**\n  (mechanics in `details.md`). Browser-unverifiable from here — confirm once it opens non-empty.\n\n### 9. Gate — iterate, or stop on a broken harness\n**Harness-failure gate (before the diff):** if a replay reveals the harness is wrong — trace landed under\nthe wrong ml_app, no trace after the step-7 sanity checks, missing flush, or auth\u002Forg misrouted — **do NOT\nproceed to a diff on bad data.** Stop and present a selector to fix the harness (re-scope ml_app \u002F add flush\n\u002F fix env) and re-replay.\nOtherwise, after the diff, an `AskUserQuestion` selector: **Looks good — stop here** (finish; leave the edits\nin the working tree) \u002F **Make more changes** (free-text inline → back to step 5). Re-present after every\nreplay; end only on \"stop here\".\n\n## Reference\n- `references\u002Fdetails.md` — trace backend + **pup exact flags**, the **runner contract** (+ Go, export mode),\n  polling + the false-negative sanity check, the **trace-link scoping fix**, limitations. Read before pup \u002F the runner.\n- `references\u002Flocal-setup.md` — making a deployed-only app locally runnable (step 3.5). Read when that gap shows.\n- `scripts\u002Freplay_runner_template.py` — the Python runner to copy + fill.\n",{"data":40,"body":41},{"name":4,"description":6},{"type":42,"children":43},"root",[44,53,74,88,120,153,160,178,184,344,350,357,410,422,517,523,557,563,607,613,653,659,714,720,764,770,1044,1050,1081,1087,1177,1186,1209,1215,1375,1381,1421,1529,1535,1573,1579,1633],{"type":45,"tag":46,"props":47,"children":49},"element","h1",{"id":48},"replay-a-trace-against-local-code",[50],{"type":51,"value":52},"text","Replay a trace against local code",{"type":45,"tag":54,"props":55,"children":56},"p",{},[57,59,65,67,72],{"type":51,"value":58},"A fast ",{"type":45,"tag":60,"props":61,"children":62},"strong",{},[63],{"type":51,"value":64},"iteration loop",{"type":51,"value":66}," on a single production trace: take a trace whose output a developer didn't like,\noptionally change the code, ",{"type":45,"tag":60,"props":68,"children":69},{},[70],{"type":51,"value":71},"re-run it against their LOCAL code",{"type":51,"value":73},", and show a concise diff of old vs new\noutput — repeating until they're happy. Assumes nothing about the project's layout.",{"type":45,"tag":54,"props":75,"children":76},{},[77,79,86],{"type":51,"value":78},"Invoked from the developer's coding agent: ",{"type":45,"tag":80,"props":81,"children":83},"code",{"className":82},[],[84],{"type":51,"value":85},"\u002Fagent-observability-replay-trace \u003Ctrace-id> [\u003Cchanges to test>]",{"type":51,"value":87},".\nWith no modification, do the replay + diff only (a reproduce\u002Fregression check), then offer to enter the loop.",{"type":45,"tag":54,"props":89,"children":90},{},[91],{"type":45,"tag":60,"props":92,"children":93},{},[94,96,102,104,110,112,118],{"type":51,"value":95},"This file is the workflow spine — terse on purpose. The depth lives in ",{"type":45,"tag":80,"props":97,"children":99},{"className":98},[],[100],{"type":51,"value":101},"references\u002Fdetails.md",{"type":51,"value":103}," (trace\nbackend + pup flags, the runner contract, export mode, polling, the trace-link scoping fix) and\n",{"type":45,"tag":80,"props":105,"children":107},{"className":106},[],[108],{"type":51,"value":109},"references\u002Flocal-setup.md",{"type":51,"value":111}," (making a deployed-only app locally runnable). Read ",{"type":45,"tag":80,"props":113,"children":115},{"className":114},[],[116],{"type":51,"value":117},"details.md",{"type":51,"value":119}," before you touch\npup or generate the runner.",{"type":45,"tag":54,"props":121,"children":122},{},[123,128,130,136,138,143,145,151],{"type":45,"tag":60,"props":124,"children":125},{},[126],{"type":51,"value":127},"Writing code — keep comments minimal to none.",{"type":51,"value":129}," Everything you generate or edit (the annotation, the\nrunner's ",{"type":45,"tag":80,"props":131,"children":133},{"className":132},[],[134],{"type":51,"value":135},"ENTRYPOINTS",{"type":51,"value":137}," entries, a local harness, iteration edits) should match the surrounding code and\ncarry ",{"type":45,"tag":60,"props":139,"children":140},{},[141],{"type":51,"value":142},"no unnecessary comments",{"type":51,"value":144}," — don't narrate what the code plainly does; add a comment only for a\ngenuinely non-obvious ",{"type":45,"tag":146,"props":147,"children":148},"em",{},[149],{"type":51,"value":150},"why",{"type":51,"value":152},".",{"type":45,"tag":154,"props":155,"children":157},"h2",{"id":156},"interaction-model-selector-gates-never-a-hard-stop",[158],{"type":51,"value":159},"Interaction model — selector gates, never a hard stop",{"type":45,"tag":54,"props":161,"children":162},{},[163,165,176],{"type":51,"value":164},"This is a live loop. At every decision point present the choices as an ",{"type":45,"tag":60,"props":166,"children":167},{},[168,174],{"type":45,"tag":80,"props":169,"children":171},{"className":170},[],[172],{"type":51,"value":173},"AskUserQuestion",{"type":51,"value":175}," selector",{"type":51,"value":177}," (the\nplan-mode-style menu), not a plain question that ends your turn. Two gates: (a) after you propose code\nchanges, before replaying; (b) after each diff. The selector's free-text option lets the user type detail\n(what to refine) inline — act on it directly, don't ask a follow-up. Keep re-presenting after every replay\nuntil they pick \"stop here\".",{"type":45,"tag":154,"props":179,"children":181},{"id":180},"scope-check-first",[182],{"type":51,"value":183},"Scope — check first",{"type":45,"tag":185,"props":186,"children":187},"ul",{},[188,222,239,265,312],{"type":45,"tag":189,"props":190,"children":191},"li",{},[192,205,207,213,215,220],{"type":45,"tag":60,"props":193,"children":194},{},[195,197,203],{"type":51,"value":196},"Traced with ",{"type":45,"tag":80,"props":198,"children":200},{"className":199},[],[201],{"type":51,"value":202},"ddtrace",{"type":51,"value":204}," \u002F LLM Obs",{"type":51,"value":206}," (an ",{"type":45,"tag":80,"props":208,"children":210},{"className":209},[],[211],{"type":51,"value":212},"ml_app",{"type":51,"value":214}," + a discoverable entrypoint). ",{"type":45,"tag":60,"props":216,"children":217},{},[218],{"type":51,"value":219},"Python is first-class",{"type":51,"value":221},";\nother languages work but you write the runner to the contract in their SDK\u002Fbuild tooling.",{"type":45,"tag":189,"props":223,"children":224},{},[225,230,232,237],{"type":45,"tag":60,"props":226,"children":227},{},[228],{"type":51,"value":229},"JSON-serializable entrypoint input",{"type":51,"value":231},", and a ",{"type":45,"tag":60,"props":233,"children":234},{},[235],{"type":51,"value":236},"callable seam",{"type":51,"value":238}," for the root span (see step 3.5 — not a\nbinary \"is it runnable?\"; deployed-only apps often still expose a plain callable).",{"type":45,"tag":189,"props":240,"children":241},{},[242,247,249,255,257,263],{"type":45,"tag":60,"props":243,"children":244},{},[245],{"type":51,"value":246},"A trace-access backend",{"type":51,"value":248}," — the ",{"type":45,"tag":80,"props":250,"children":252},{"className":251},[],[253],{"type":51,"value":254},"datadog-llmo",{"type":51,"value":256}," MCP (used when present) or the ",{"type":45,"tag":80,"props":258,"children":260},{"className":259},[],[261],{"type":51,"value":262},"pup",{"type":51,"value":264}," CLI (fallback, and\nthe easier install if you have neither) (step 0).",{"type":45,"tag":189,"props":266,"children":267},{},[268,273,275,281,283,289,291,302,304,310],{"type":45,"tag":60,"props":269,"children":270},{},[271],{"type":51,"value":272},"Credentials:",{"type":51,"value":274}," ",{"type":45,"tag":80,"props":276,"children":278},{"className":277},[],[279],{"type":51,"value":280},"DD_API_KEY",{"type":51,"value":282}," + ",{"type":45,"tag":80,"props":284,"children":286},{"className":285},[],[287],{"type":51,"value":288},"DD_SITE",{"type":51,"value":290}," + provider key(s). ",{"type":45,"tag":60,"props":292,"children":293},{},[294,296],{"type":51,"value":295},"Not ",{"type":45,"tag":80,"props":297,"children":299},{"className":298},[],[300],{"type":51,"value":301},"DD_APP_KEY",{"type":51,"value":303}," — plain trace, not an\nExperiment (that's ",{"type":45,"tag":80,"props":305,"children":307},{"className":306},[],[308],{"type":51,"value":309},"agent-observability-replay-experiment",{"type":51,"value":311},").",{"type":45,"tag":189,"props":313,"children":314},{},[315,320,322,327,329,334,336,342],{"type":45,"tag":60,"props":316,"children":317},{},[318],{"type":51,"value":319},"Side effects, irreversible:",{"type":51,"value":321}," replaying re-runs real code (model spend + real writes), and ",{"type":45,"tag":60,"props":323,"children":324},{},[325],{"type":51,"value":326},"LLM Obs\ntraces cannot be deleted",{"type":51,"value":328}," — a mis-scoped replay (wrong ml_app) ",{"type":45,"tag":146,"props":330,"children":331},{},[332],{"type":51,"value":333},"permanently",{"type":51,"value":335}," pollutes the production app's\ndashboards\u002Feval sets. That's why the ",{"type":45,"tag":80,"props":337,"children":339},{"className":338},[],[340],{"type":51,"value":341},"\u003Cml_app>-local",{"type":51,"value":343}," isolation (steps 4\u002F6\u002F7) is load-bearing, not tidy.\nWarn before the first replay.",{"type":45,"tag":154,"props":345,"children":347},{"id":346},"workflow",[348],{"type":51,"value":349},"Workflow",{"type":45,"tag":351,"props":352,"children":354},"h3",{"id":353},"_0-ensure-a-trace-access-backend",[355],{"type":51,"value":356},"0. Ensure a trace-access backend",{"type":45,"tag":54,"props":358,"children":359},{},[360,362,367,369,375,377,383,385,393,395,401,403,408],{"type":51,"value":361},"Pick, in order: (1) the ",{"type":45,"tag":60,"props":363,"children":364},{},[365],{"type":51,"value":366},"MCP",{"type":51,"value":368}," if ",{"type":45,"tag":80,"props":370,"children":372},{"className":371},[],[373],{"type":51,"value":374},"mcp__datadog-llmo-mcp__*",{"type":51,"value":376}," tools are present — the default (slightly\nricher for reads: structured tree + ",{"type":45,"tag":80,"props":378,"children":380},{"className":379},[],[381],{"type":51,"value":382},"content_info",{"type":51,"value":384},"); (2) else ",{"type":45,"tag":60,"props":386,"children":387},{},[388],{"type":45,"tag":80,"props":389,"children":391},{"className":390},[],[392],{"type":51,"value":262},{"type":51,"value":394}," if installed and ",{"type":45,"tag":80,"props":396,"children":398},{"className":397},[],[399],{"type":51,"value":400},"pup auth",{"type":51,"value":402}," targets\nthe app's org; (3) else the user has neither → guide the ",{"type":45,"tag":60,"props":404,"children":405},{},[406],{"type":51,"value":407},"pup install",{"type":51,"value":409}," (it's easier to set up than the\nMCP, so recommend pup here):",{"type":45,"tag":411,"props":412,"children":416},"pre",{"className":413,"code":415,"language":51},[414],"language-text","brew tap datadog-labs\u002Fpack && brew install datadog-labs\u002Fpack\u002Fpup\npup auth login\n",[417],{"type":45,"tag":80,"props":418,"children":420},{"__ignoreMap":419},"",[421],{"type":51,"value":415},{"type":45,"tag":54,"props":423,"children":424},{},[425,427,433,435,443,445,457,459,468,469,474,476,485,487,493,495,500,502,507,509,515],{"type":51,"value":426},"(MCP alternative: ",{"type":45,"tag":80,"props":428,"children":430},{"className":429},[],[431],{"type":51,"value":432},"claude mcp add --scope user --transport http \"datadog-llmo-mcp\" \"https:\u002F\u002Fmcp.datadoghq.com\u002Fapi\u002Funstable\u002Fmcp-server\u002Fmcp?toolsets=llmobs\"",{"type":51,"value":434},"; see ",{"type":45,"tag":436,"props":437,"children":441},"a",{"href":438,"rel":439},"https:\u002F\u002Fdocs.datadoghq.com\u002Fbits_ai\u002Fmcp_server\u002Fsetup\u002F",[440],"nofollow",[442],{"type":51,"value":438},{"type":51,"value":444},".) Don't proceed without a backend.\nThe backend↔operation mapping and ",{"type":45,"tag":60,"props":446,"children":447},{},[448,450,455],{"type":51,"value":449},"pup's exact flags\u002Fgotchas are in ",{"type":45,"tag":80,"props":451,"children":453},{"className":452},[],[454],{"type":51,"value":117},{"type":51,"value":456}," — read that section before\nusing pup.",{"type":51,"value":458}," Two pup musts: (1) results come back at ",{"type":45,"tag":60,"props":460,"children":461},{},[462],{"type":45,"tag":80,"props":463,"children":465},{"className":464},[],[466],{"type":51,"value":467},"data.spans[]",{"type":51,"value":274},{"type":45,"tag":146,"props":470,"children":471},{},[472],{"type":51,"value":473},"or",{"type":51,"value":475}," top-level ",{"type":45,"tag":60,"props":477,"children":478},{},[479],{"type":45,"tag":80,"props":480,"children":482},{"className":481},[],[483],{"type":51,"value":484},"spans[]",{"type":51,"value":486},"\n(varies by version\u002F",{"type":45,"tag":80,"props":488,"children":490},{"className":489},[],[491],{"type":51,"value":492},"--no-agent",{"type":51,"value":494},") — parse ",{"type":45,"tag":60,"props":496,"children":497},{},[498],{"type":51,"value":499},"whichever is present",{"type":51,"value":501},", or you get zero hits on an ingested\ntrace (a silent false negative, step 7); (2) check ",{"type":45,"tag":60,"props":503,"children":504},{},[505],{"type":51,"value":506},"token expiry",{"type":51,"value":508}," (",{"type":45,"tag":80,"props":510,"children":512},{"className":511},[],[513],{"type":51,"value":514},"pup auth status",{"type":51,"value":516},"), not just that auth\nexists — expiry mid-loop looks like \"trace not found.\"",{"type":45,"tag":351,"props":518,"children":520},{"id":519},"_1-parse-the-command",[521],{"type":51,"value":522},"1. Parse the command",{"type":45,"tag":54,"props":524,"children":525},{},[526,532,534,539,541,547,549,555],{"type":45,"tag":80,"props":527,"children":529},{"className":528},[],[530],{"type":51,"value":531},"\u003Ctrace-id>",{"type":51,"value":533}," + optional free-text modification (everything after the id); none → diff-only mode. Determine\nthe ",{"type":45,"tag":80,"props":535,"children":537},{"className":536},[],[538],{"type":51,"value":212},{"type":51,"value":540}," from the project (",{"type":45,"tag":80,"props":542,"children":544},{"className":543},[],[545],{"type":51,"value":546},"LLMObs.enable(ml_app=…)",{"type":51,"value":548}," \u002F ",{"type":45,"tag":80,"props":550,"children":552},{"className":551},[],[553],{"type":51,"value":554},"DD_LLMOBS_ML_APP",{"type":51,"value":556},") or the trace; confirm if\nambiguous.",{"type":45,"tag":351,"props":558,"children":560},{"id":559},"_2-fetch-the-trace-locate-the-baseline",[561],{"type":51,"value":562},"2. Fetch the trace + locate the baseline",{"type":45,"tag":54,"props":564,"children":565},{},[566,568,574,576,582,584,590,592,598,600,605],{"type":51,"value":567},"Fetch via the backend; note ",{"type":45,"tag":80,"props":569,"children":571},{"className":570},[],[572],{"type":51,"value":573},"total_duration_ms",{"type":51,"value":575}," (drives step 7), the ",{"type":45,"tag":80,"props":577,"children":579},{"className":578},[],[580],{"type":51,"value":581},"trace_url",{"type":51,"value":583},", and\n",{"type":45,"tag":80,"props":585,"children":587},{"className":586},[],[588],{"type":51,"value":589},"metadata.replay_input",{"type":51,"value":591},"\u002F",{"type":45,"tag":80,"props":593,"children":595},{"className":594},[],[596],{"type":51,"value":597},"replay_entrypoint",{"type":51,"value":599}," if present. ",{"type":45,"tag":60,"props":601,"children":602},{},[603],{"type":51,"value":604},"Locate the baseline field — it's not always the\nroot output:",{"type":51,"value":606}," the value the developer dislikes may be a tool-call input or an intermediate output several\nlevels deep, and the app may post-process it before the span records it. Pick the field the code change can\nactually move, or the delta drowns in noise.",{"type":45,"tag":351,"props":608,"children":610},{"id":609},"_25-check-for-fan-out",[611],{"type":51,"value":612},"2.5. Check for fan-out",{"type":45,"tag":54,"props":614,"children":615},{},[616,618,623,625,630,632,637,639,644,646,651],{"type":51,"value":617},"If the root span ",{"type":45,"tag":60,"props":619,"children":620},{},[621],{"type":51,"value":622},"fans out into repeated sibling subtrees",{"type":51,"value":624}," (a batch\u002Fmap over N parallel sub-runs), the\nchange under test is usually visible in a ",{"type":45,"tag":60,"props":626,"children":627},{},[628],{"type":51,"value":629},"single",{"type":51,"value":631}," branch — replaying the whole root costs ~N× spend and\ntime for no extra signal. Offer to replay one representative branch; ",{"type":45,"tag":60,"props":633,"children":634},{},[635],{"type":51,"value":636},"log what you skipped",{"type":51,"value":638},". ",{"type":45,"tag":60,"props":640,"children":641},{},[642],{"type":51,"value":643},"Pick deliberately: the cheapest\nbranch that reached the terminal \u002F side-effecting tool",{"type":51,"value":645}," (most branches are no-ops that prove nothing), and\nreconstruct its input from the ",{"type":45,"tag":60,"props":647,"children":648},{},[649],{"type":51,"value":650},"child",{"type":51,"value":652}," span's input, not the root's. Full root only if the change is\ninherently cross-branch.",{"type":45,"tag":351,"props":654,"children":656},{"id":655},"_3-resolve-the-entrypoint-input",[657],{"type":51,"value":658},"3. Resolve the entrypoint + input",{"type":45,"tag":185,"props":660,"children":661},{},[662,685],{"type":45,"tag":189,"props":663,"children":664},{},[665,670,671,677,679,684],{"type":45,"tag":60,"props":666,"children":667},{},[668],{"type":51,"value":669},"Entrypoint:",{"type":51,"value":274},{"type":45,"tag":80,"props":672,"children":674},{"className":673},[],[675],{"type":51,"value":676},"metadata.replay_entrypoint",{"type":51,"value":678}," if present; else infer from the root span (name\u002Fkind) + code\nand ",{"type":45,"tag":60,"props":680,"children":681},{},[682],{"type":51,"value":683},"confirm with the user",{"type":51,"value":152},{"type":45,"tag":189,"props":686,"children":687},{},[688,693,694,699,701,706,708,713],{"type":45,"tag":60,"props":689,"children":690},{},[691],{"type":51,"value":692},"Input:",{"type":51,"value":274},{"type":45,"tag":80,"props":695,"children":697},{"className":696},[],[698],{"type":51,"value":589},{"type":51,"value":700}," if present; else derive a ",{"type":45,"tag":60,"props":702,"children":703},{},[704],{"type":51,"value":705},"suggested",{"type":51,"value":707}," input (prefer the code\nsignature — the rendered prompt is lossy) and have the user ",{"type":45,"tag":60,"props":709,"children":710},{},[711],{"type":51,"value":712},"confirm\u002Fedit",{"type":51,"value":152},{"type":45,"tag":351,"props":715,"children":717},{"id":716},"_35-ensure-a-local-run-path-find-the-innermost-callable-seam",[718],{"type":51,"value":719},"3.5. Ensure a local run path (find the innermost callable seam)",{"type":45,"tag":54,"props":721,"children":722},{},[723,725,730,732,737,739,745,747,755,757,762],{"type":51,"value":724},"Ask ",{"type":45,"tag":60,"props":726,"children":727},{},[728],{"type":51,"value":729},"\"what is the innermost callable seam for this root span, and can I call it directly with JSON?\"",{"type":51,"value":731}," — not\n\"is the app runnable?\". Already directly callable → ",{"type":45,"tag":60,"props":733,"children":734},{},[735],{"type":51,"value":736},"skip, continue",{"type":51,"value":738},". Buried under a handler\u002Fservice\n(deployed-only, no local ",{"type":45,"tag":80,"props":740,"children":742},{"className":741},[],[743],{"type":51,"value":744},"__main__",{"type":51,"value":746},", live-infra coupling) → follow ",{"type":45,"tag":60,"props":748,"children":749},{},[750],{"type":45,"tag":80,"props":751,"children":753},{"className":752},[],[754],{"type":51,"value":109},{"type":51,"value":756}," (detect →\npropose → approve → build). The common middle case — a deployed service whose core logic is ",{"type":45,"tag":146,"props":758,"children":759},{},[760],{"type":51,"value":761},"already",{"type":51,"value":763}," a plain\ncallable (ports-and-adapters) — just extract\u002Fcall that seam; full local-setup is overkill.",{"type":45,"tag":351,"props":765,"children":767},{"id":766},"_4-ensure-the-two-persistent-artifacts-one-time-setup",[768],{"type":51,"value":769},"4. Ensure the two persistent artifacts (one-time setup)",{"type":45,"tag":185,"props":771,"children":772},{},[773,848,918],{"type":45,"tag":189,"props":774,"children":775},{},[776,781,783,788,790,795,797,803,805,824,826,832,834,840,841,846],{"type":45,"tag":60,"props":777,"children":778},{},[779],{"type":51,"value":780},"a) In-entrypoint annotation",{"type":51,"value":782}," on the app's ",{"type":45,"tag":60,"props":784,"children":785},{},[786],{"type":51,"value":787},"real",{"type":51,"value":789}," entrypoint, so all future traces (production too)\nself-describe. Stamp it at ",{"type":45,"tag":60,"props":791,"children":792},{},[793],{"type":51,"value":794},"span start, not the success\u002Fdeferred-finish path",{"type":51,"value":796}," — a failed run must still\ncarry ",{"type":45,"tag":80,"props":798,"children":800},{"className":799},[],[801],{"type":51,"value":802},"replay_input",{"type":51,"value":804}," (those are the ones you most want to replay):\n",{"type":45,"tag":411,"props":806,"children":810},{"className":807,"code":808,"language":809,"meta":419,"style":419},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","LLMObs.annotate(span=span, metadata={\"replay_entrypoint\": \"\u003Cstable id>\", \"replay_input\": \u003Cextractor>})\n","python",[811],{"type":45,"tag":80,"props":812,"children":813},{"__ignoreMap":419},[814],{"type":45,"tag":815,"props":816,"children":819},"span",{"class":817,"line":818},"line",1,[820],{"type":45,"tag":815,"props":821,"children":822},{},[823],{"type":51,"value":808},{"type":51,"value":825},"\nNo ",{"type":45,"tag":80,"props":827,"children":829},{"className":828},[],[830],{"type":51,"value":831},"replay_output",{"type":51,"value":833}," — the original trace is the baseline. (Non-Python annotate APIs differ — e.g. Go\n",{"type":45,"tag":80,"props":835,"children":837},{"className":836},[],[838],{"type":51,"value":839},"span.Annotate(llmobs.WithAnnotatedMetadata(...))",{"type":51,"value":434},{"type":45,"tag":80,"props":842,"children":844},{"className":843},[],[845],{"type":51,"value":117},{"type":51,"value":847},".)",{"type":45,"tag":189,"props":849,"children":850},{},[851,856,858,863,865,871,873,879,881,887,889,894,896,902,904,909,911,916],{"type":45,"tag":60,"props":852,"children":853},{},[854],{"type":51,"value":855},"Isolation pre-flight (before writing the runner):",{"type":51,"value":857}," grep the entrypoint's call path for ",{"type":45,"tag":60,"props":859,"children":860},{},[861],{"type":51,"value":862},"per-span\u002F\nper-call ml_app overrides",{"type":51,"value":864}," (Go ",{"type":45,"tag":80,"props":866,"children":868},{"className":867},[],[869],{"type":51,"value":870},"llmobs.WithMLApp",{"type":51,"value":872},"; Python ",{"type":45,"tag":80,"props":874,"children":876},{"className":875},[],[877],{"type":51,"value":878},"ml_app=",{"type":51,"value":880}," on a decorator or in\n",{"type":45,"tag":80,"props":882,"children":884},{"className":883},[],[885],{"type":51,"value":886},"LLMObs.annotate",{"type":51,"value":888},"). Those ",{"type":45,"tag":60,"props":890,"children":891},{},[892],{"type":51,"value":893},"beat",{"type":51,"value":895}," the init-level ",{"type":45,"tag":80,"props":897,"children":899},{"className":898},[],[900],{"type":51,"value":901},"-local",{"type":51,"value":903},", so the app's spans can still land in\nproduction — tracer-level config is ",{"type":45,"tag":60,"props":905,"children":906},{},[907],{"type":51,"value":908},"not",{"type":51,"value":910}," proof of isolation. If any exist, the app's ml_app must\nresolve from env so ",{"type":45,"tag":80,"props":912,"children":914},{"className":913},[],[915],{"type":51,"value":901},{"type":51,"value":917}," wins.",{"type":45,"tag":189,"props":919,"children":920},{},[921,926,928,938,940,945,947,952,954,964,966,971,973,978,980,986,988,993,994,999,1001,1006,1008,1013,1015,1020,1022,1027,1029,1034,1036,1042],{"type":45,"tag":60,"props":922,"children":923},{},[924],{"type":51,"value":925},"b) The runner",{"type":51,"value":927}," — satisfies the ",{"type":45,"tag":60,"props":929,"children":930},{},[931,933],{"type":51,"value":932},"language-independent runner contract in ",{"type":45,"tag":80,"props":934,"children":936},{"className":935},[],[937],{"type":51,"value":117},{"type":51,"value":939}," (load env →\nderive ",{"type":45,"tag":80,"props":941,"children":943},{"className":942},[],[944],{"type":51,"value":341},{"type":51,"value":946}," → dispatch one entrypoint on JSON → ",{"type":45,"tag":60,"props":948,"children":949},{},[950],{"type":51,"value":951},"flush on every exit path incl. errors",{"type":51,"value":953}," →\n",{"type":45,"tag":60,"props":955,"children":956},{},[957,959],{"type":51,"value":958},"refuse to start unless ml_app ends in ",{"type":45,"tag":80,"props":960,"children":962},{"className":961},[],[963],{"type":51,"value":901},{"type":51,"value":965}," → print the ",{"type":45,"tag":80,"props":967,"children":969},{"className":968},[],[970],{"type":51,"value":901},{"type":51,"value":972}," ml_app). ",{"type":45,"tag":60,"props":974,"children":975},{},[976],{"type":51,"value":977},"Python:",{"type":51,"value":979}," copy\n",{"type":45,"tag":80,"props":981,"children":983},{"className":982},[],[984],{"type":51,"value":985},"scripts\u002Freplay_runner_template.py",{"type":51,"value":987}," and fill ",{"type":45,"tag":80,"props":989,"children":991},{"className":990},[],[992],{"type":51,"value":135},{"type":51,"value":638},{"type":45,"tag":60,"props":995,"children":996},{},[997],{"type":51,"value":998},"Other languages:",{"type":51,"value":1000}," write to the contract —\ndon't assume the Python API carries over (Go APIs + export-mode gotchas in ",{"type":45,"tag":80,"props":1002,"children":1004},{"className":1003},[],[1005],{"type":51,"value":117},{"type":51,"value":1007},"), and where the\nlanguage has no in-process dotenv add a ",{"type":45,"tag":60,"props":1009,"children":1010},{},[1011],{"type":51,"value":1012},"run wrapper (artifact c)",{"type":51,"value":1014}," that sources the project env, unsets\nambient provider vars, and exports the ",{"type":45,"tag":80,"props":1016,"children":1018},{"className":1017},[],[1019],{"type":51,"value":901},{"type":51,"value":1021}," override. Infer + ",{"type":45,"tag":60,"props":1023,"children":1024},{},[1025],{"type":51,"value":1026},"confirm the run command",{"type":51,"value":1028},"; follow the\nhost repo's ",{"type":45,"tag":60,"props":1030,"children":1031},{},[1032],{"type":51,"value":1033},"build-file conventions",{"type":51,"value":1035}," (Bazel\u002FGazelle → ",{"type":45,"tag":80,"props":1037,"children":1039},{"className":1038},[],[1040],{"type":51,"value":1041},"cmd\u002F\u003Cname>\u002F",{"type":51,"value":1043},", run Gazelle, build before replay).",{"type":45,"tag":351,"props":1045,"children":1047},{"id":1046},"_5-if-a-change-was-requested-edit-then-gate",[1048],{"type":51,"value":1049},"5. (If a change was requested) edit, then gate",{"type":45,"tag":54,"props":1051,"children":1052},{},[1053,1055,1060,1062,1067,1068,1073,1074,1079],{"type":51,"value":1054},"Make the code changes, show the developer the diff of your changes, then an ",{"type":45,"tag":80,"props":1056,"children":1058},{"className":1057},[],[1059],{"type":51,"value":173},{"type":51,"value":1061}," selector:\n",{"type":45,"tag":60,"props":1063,"children":1064},{},[1065],{"type":51,"value":1066},"Replay now",{"type":51,"value":548},{"type":45,"tag":60,"props":1069,"children":1070},{},[1071],{"type":51,"value":1072},"Adjust the changes first",{"type":51,"value":548},{"type":45,"tag":60,"props":1075,"children":1076},{},[1077],{"type":51,"value":1078},"Cancel",{"type":51,"value":1080},". Only replay on \"Replay now\".",{"type":45,"tag":351,"props":1082,"children":1084},{"id":1083},"_6-replay",[1085],{"type":51,"value":1086},"6. Replay",{"type":45,"tag":54,"props":1088,"children":1089},{},[1090,1092,1097,1099,1104,1106,1111,1112,1118,1119,1125,1127,1132,1134,1139,1140,1145,1147,1152,1154,1160,1162,1167,1169,1175],{"type":51,"value":1091},"Before the first replay: ",{"type":45,"tag":60,"props":1093,"children":1094},{},[1095],{"type":51,"value":1096},"warn",{"type":51,"value":1098}," (re-running is real — model spend + real writes), and ",{"type":45,"tag":60,"props":1100,"children":1101},{},[1102],{"type":51,"value":1103},"sanitize the\nenvironment",{"type":51,"value":1105},". The ",{"type":45,"tag":60,"props":1107,"children":1108},{},[1109],{"type":51,"value":1110},"coding agent's own env",{"type":51,"value":508},{"type":45,"tag":80,"props":1113,"children":1115},{"className":1114},[],[1116],{"type":51,"value":1117},"ANTHROPIC_API_KEY",{"type":51,"value":548},{"type":45,"tag":80,"props":1120,"children":1122},{"className":1121},[],[1123],{"type":51,"value":1124},"ANTHROPIC_BASE_URL",{"type":51,"value":1126}," set by Claude\nCode, and other provider keys) can make the app's SDK ",{"type":45,"tag":60,"props":1128,"children":1129},{},[1130],{"type":51,"value":1131},"bypass its configured model gateway",{"type":51,"value":1133}," — a fidelity\ngap ",{"type":45,"tag":60,"props":1135,"children":1136},{},[1137],{"type":51,"value":1138},"invisible in the diff",{"type":51,"value":638},{"type":45,"tag":60,"props":1141,"children":1142},{},[1143],{"type":51,"value":1144},"Unset ambient provider vars by default and report that you did",{"type":51,"value":1146}," (don't\njust ask); grep the app for its own ambient-key guards. Also ",{"type":45,"tag":60,"props":1148,"children":1149},{},[1150],{"type":51,"value":1151},"verify the credential's org matches the\ntrace's org",{"type":51,"value":1153}," — a mismatch ships the replay somewhere you can't query (looks like ingest lag).\nOn confirmation, record ",{"type":45,"tag":80,"props":1155,"children":1157},{"className":1156},[],[1158],{"type":51,"value":1159},"t0",{"type":51,"value":1161}," and run — ",{"type":45,"tag":60,"props":1163,"children":1164},{},[1165],{"type":51,"value":1166},"source the project's env file, never inline secrets",{"type":51,"value":1168}," (the marker\ntag is fine on the command; ",{"type":45,"tag":80,"props":1170,"children":1172},{"className":1171},[],[1173],{"type":51,"value":1174},"DD_API_KEY=\u003Cvalue>",{"type":51,"value":1176}," inline is blocked by the permission classifier and leaks to\nhistory\u002Ftranscript — use the wrapper\u002Fenv-file):",{"type":45,"tag":411,"props":1178,"children":1181},{"className":1179,"code":1180,"language":51},[414],"DD_TAGS=replay_run_id:\u003Cunique-id> \u003Crun cmd or wrapper> --entrypoint \u003Cid> --input-file \u003Cpath>\n",[1182],{"type":45,"tag":80,"props":1183,"children":1184},{"__ignoreMap":419},[1185],{"type":51,"value":1180},{"type":45,"tag":54,"props":1187,"children":1188},{},[1189,1191,1201,1203,1208],{"type":51,"value":1190},"The runner emits ",{"type":45,"tag":60,"props":1192,"children":1193},{},[1194,1196],{"type":51,"value":1195},"under ",{"type":45,"tag":80,"props":1197,"children":1199},{"className":1198},[],[1200],{"type":51,"value":341},{"type":51,"value":1202}," (idempotent, so replays never pollute production) and prints that\nname — poll for the new trace ",{"type":45,"tag":60,"props":1204,"children":1205},{},[1206],{"type":51,"value":1207},"under it",{"type":51,"value":152},{"type":45,"tag":351,"props":1210,"children":1212},{"id":1211},"_7-wait-for-the-new-trace",[1213],{"type":51,"value":1214},"7. Wait for the new trace",{"type":45,"tag":185,"props":1216,"children":1217},{},[1218,1235,1303],{"type":45,"tag":189,"props":1219,"children":1220},{},[1221,1226,1228,1234],{"type":45,"tag":60,"props":1222,"children":1223},{},[1224],{"type":51,"value":1225},"Runner subprocess timeout",{"type":51,"value":1227}," = ",{"type":45,"tag":80,"props":1229,"children":1231},{"className":1230},[],[1232],{"type":51,"value":1233},"max(120s, ~3 × total_duration_ms)",{"type":51,"value":152},{"type":45,"tag":189,"props":1236,"children":1237},{},[1238,1243,1245,1250,1252,1258,1260,1265,1267,1273,1275,1281,1283,1288,1290,1295,1297,1301],{"type":45,"tag":60,"props":1239,"children":1240},{},[1241],{"type":51,"value":1242},"Ingest poll:",{"type":51,"value":1244}," after it returns, poll the backend ",{"type":45,"tag":60,"props":1246,"children":1247},{},[1248],{"type":51,"value":1249},"every ~5s up to ~2 min",{"type":51,"value":1251}," for the ",{"type":45,"tag":80,"props":1253,"children":1255},{"className":1254},[],[1256],{"type":51,"value":1257},"replay_run_id",{"type":51,"value":1259}," tag\nunder ",{"type":45,"tag":80,"props":1261,"children":1263},{"className":1262},[],[1264],{"type":51,"value":341},{"type":51,"value":1266}," (pup: ",{"type":45,"tag":80,"props":1268,"children":1270},{"className":1269},[],[1271],{"type":51,"value":1272},"--query \"replay_run_id:\u003Cid>\"",{"type":51,"value":1274},", plain ",{"type":45,"tag":80,"props":1276,"children":1278},{"className":1277},[],[1279],{"type":51,"value":1280},"key:value",{"type":51,"value":1282},"). ",{"type":45,"tag":60,"props":1284,"children":1285},{},[1286],{"type":51,"value":1287},"Before ever reporting\n\"not found,\" re-query with no tag filter",{"type":51,"value":1289}," (just ",{"type":45,"tag":80,"props":1291,"children":1293},{"className":1292},[],[1294],{"type":51,"value":341},{"type":51,"value":1296}," + window): if that returns spans, your\nfilter\u002Fparse\u002Fscope is wrong — ",{"type":45,"tag":60,"props":1298,"children":1299},{},[1300],{"type":51,"value":908},{"type":51,"value":1302}," ingestion. A false \"no trace\" reads as normal and invites a wasteful\nre-run.",{"type":45,"tag":189,"props":1304,"children":1305},{},[1306,1311,1312,1318,1320,1325,1327,1332,1334,1340,1342,1347,1349,1359,1361,1366,1368,1373],{"type":45,"tag":60,"props":1307,"children":1308},{},[1309],{"type":51,"value":1310},"Verify isolation on each hit — a tag match is NOT proof.",{"type":51,"value":274},{"type":45,"tag":80,"props":1313,"children":1315},{"className":1314},[],[1316],{"type":51,"value":1317},"--query",{"type":51,"value":1319},"\u002Ftag matching can return a span\nwhose real ",{"type":45,"tag":80,"props":1321,"children":1323},{"className":1322},[],[1324],{"type":51,"value":212},{"type":51,"value":1326}," is a ",{"type":45,"tag":146,"props":1328,"children":1329},{},[1330],{"type":51,"value":1331},"different",{"type":51,"value":1333}," app (the ",{"type":45,"tag":80,"props":1335,"children":1337},{"className":1336},[],[1338],{"type":51,"value":1339},"--ml-app",{"type":51,"value":1341}," filter gets ignored). Read ",{"type":45,"tag":80,"props":1343,"children":1345},{"className":1344},[],[1346],{"type":51,"value":212},{"type":51,"value":1348}," off every\nreturned span and ",{"type":45,"tag":60,"props":1350,"children":1351},{},[1352,1354],{"type":51,"value":1353},"assert it ends in ",{"type":45,"tag":80,"props":1355,"children":1357},{"className":1356},[],[1358],{"type":51,"value":901},{"type":51,"value":1360}," before reporting a clean replay — otherwise you report\n\"clean replay under ",{"type":45,"tag":80,"props":1362,"children":1364},{"className":1363},[],[1365],{"type":51,"value":901},{"type":51,"value":1367},"\" while the trace is actually in production (which you can't undo). This false\n",{"type":45,"tag":146,"props":1369,"children":1370},{},[1371],{"type":51,"value":1372},"confidence",{"type":51,"value":1374}," is worse than the false negative. Don't hard-fail on timeout; offer to keep waiting.",{"type":45,"tag":351,"props":1376,"children":1378},{"id":1377},"_8-diff-with-links-to-both-traces",[1379],{"type":51,"value":1380},"8. Diff (with links to both traces)",{"type":45,"tag":54,"props":1382,"children":1383},{},[1384,1386,1391,1393,1398,1400,1405,1407,1412,1414,1419],{"type":51,"value":1385},"Concise summary of how the ",{"type":45,"tag":60,"props":1387,"children":1388},{},[1389],{"type":51,"value":1390},"new output differs from the old",{"type":51,"value":1392}," — meaningful differences only. Note live-world\ndrift; and because any nondeterministic agent varies run-to-run, ",{"type":45,"tag":60,"props":1394,"children":1395},{},[1396],{"type":51,"value":1397},"default to two replays",{"type":51,"value":1399}," (diff-only mode\ntoo, not just model-facing edits) and use ",{"type":45,"tag":60,"props":1401,"children":1402},{},[1403],{"type":51,"value":1404},"replay-to-replay comparison",{"type":51,"value":1406}," — if the two local runs differ\nfrom each other about as much as from production, the delta is sampling variance, not your change. If the\nreplay ",{"type":45,"tag":60,"props":1408,"children":1409},{},[1410],{"type":51,"value":1411},"disables a side-effecting integration",{"type":51,"value":1413}," (dry-run), that integration's subtree is absent — ",{"type":45,"tag":60,"props":1415,"children":1416},{},[1417],{"type":51,"value":1418},"exclude\nit from both sides",{"type":51,"value":1420}," before comparing span counts, or the structural diff is junk. Lead the diff with both\ntrace links:",{"type":45,"tag":185,"props":1422,"children":1423},{},[1424,1460],{"type":45,"tag":189,"props":1425,"children":1426},{},[1427,1432,1433,1438,1439,1444,1446,1451,1453,1458],{"type":45,"tag":60,"props":1428,"children":1429},{},[1430],{"type":51,"value":1431},"Old:",{"type":51,"value":274},{"type":45,"tag":80,"props":1434,"children":1436},{"className":1435},[],[1437],{"type":51,"value":581},{"type":51,"value":274},{"type":45,"tag":60,"props":1440,"children":1441},{},[1442],{"type":51,"value":1443},"verbatim",{"type":51,"value":1445}," — but ",{"type":45,"tag":60,"props":1447,"children":1448},{},[1449],{"type":51,"value":1450},"under fan-out",{"type":51,"value":1452}," (you replayed one branch) link the ",{"type":45,"tag":60,"props":1454,"children":1455},{},[1456],{"type":51,"value":1457},"branch\nspan",{"type":51,"value":1459},", not the whole-root url.",{"type":45,"tag":189,"props":1461,"children":1462},{},[1463,1468,1470,1476,1478,1483,1485,1490,1492,1498,1500,1520,1522,1527],{"type":45,"tag":60,"props":1464,"children":1465},{},[1466],{"type":51,"value":1467},"New (replay):",{"type":51,"value":1469}," must carry ",{"type":45,"tag":80,"props":1471,"children":1473},{"className":1472},[],[1474],{"type":51,"value":1475},"ml_app=\u003Cml_app>-local",{"type":51,"value":1477}," or it opens ",{"type":45,"tag":60,"props":1479,"children":1480},{},[1481],{"type":51,"value":1482},"empty",{"type":51,"value":1484}," — and the ",{"type":45,"tag":80,"props":1486,"children":1488},{"className":1487},[],[1489],{"type":51,"value":581},{"type":51,"value":1491}," is an\norg-switch wrapper (",{"type":45,"tag":80,"props":1493,"children":1495},{"className":1494},[],[1496],{"type":51,"value":1497},"…\u002Fswitch_to_user\u002F\u003Cid>?next=\u003Cencoded \u002Fllm\u002Ftraces …>&flow=org_switch",{"type":51,"value":1499},"), so ",{"type":45,"tag":60,"props":1501,"children":1502},{},[1503,1505,1510,1512,1518],{"type":51,"value":1504},"inject\n",{"type":45,"tag":80,"props":1506,"children":1508},{"className":1507},[],[1509],{"type":51,"value":1475},{"type":51,"value":1511}," into the decoded ",{"type":45,"tag":80,"props":1513,"children":1515},{"className":1514},[],[1516],{"type":51,"value":1517},"next",{"type":51,"value":1519}," query and re-encode; do NOT append to the outer URL",{"type":51,"value":1521},"\n(mechanics in ",{"type":45,"tag":80,"props":1523,"children":1525},{"className":1524},[],[1526],{"type":51,"value":117},{"type":51,"value":1528},"). Browser-unverifiable from here — confirm once it opens non-empty.",{"type":45,"tag":351,"props":1530,"children":1532},{"id":1531},"_9-gate-iterate-or-stop-on-a-broken-harness",[1533],{"type":51,"value":1534},"9. Gate — iterate, or stop on a broken harness",{"type":45,"tag":54,"props":1536,"children":1537},{},[1538,1543,1545,1550,1552,1557,1559,1564,1566,1571],{"type":45,"tag":60,"props":1539,"children":1540},{},[1541],{"type":51,"value":1542},"Harness-failure gate (before the diff):",{"type":51,"value":1544}," if a replay reveals the harness is wrong — trace landed under\nthe wrong ml_app, no trace after the step-7 sanity checks, missing flush, or auth\u002Forg misrouted — ",{"type":45,"tag":60,"props":1546,"children":1547},{},[1548],{"type":51,"value":1549},"do NOT\nproceed to a diff on bad data.",{"type":51,"value":1551}," Stop and present a selector to fix the harness (re-scope ml_app \u002F add flush\n\u002F fix env) and re-replay.\nOtherwise, after the diff, an ",{"type":45,"tag":80,"props":1553,"children":1555},{"className":1554},[],[1556],{"type":51,"value":173},{"type":51,"value":1558}," selector: ",{"type":45,"tag":60,"props":1560,"children":1561},{},[1562],{"type":51,"value":1563},"Looks good — stop here",{"type":51,"value":1565}," (finish; leave the edits\nin the working tree) \u002F ",{"type":45,"tag":60,"props":1567,"children":1568},{},[1569],{"type":51,"value":1570},"Make more changes",{"type":51,"value":1572}," (free-text inline → back to step 5). Re-present after every\nreplay; end only on \"stop here\".",{"type":45,"tag":154,"props":1574,"children":1576},{"id":1575},"reference",[1577],{"type":51,"value":1578},"Reference",{"type":45,"tag":185,"props":1580,"children":1581},{},[1582,1613,1623],{"type":45,"tag":189,"props":1583,"children":1584},{},[1585,1590,1592,1597,1599,1604,1606,1611],{"type":45,"tag":80,"props":1586,"children":1588},{"className":1587},[],[1589],{"type":51,"value":101},{"type":51,"value":1591}," — trace backend + ",{"type":45,"tag":60,"props":1593,"children":1594},{},[1595],{"type":51,"value":1596},"pup exact flags",{"type":51,"value":1598},", the ",{"type":45,"tag":60,"props":1600,"children":1601},{},[1602],{"type":51,"value":1603},"runner contract",{"type":51,"value":1605}," (+ Go, export mode),\npolling + the false-negative sanity check, the ",{"type":45,"tag":60,"props":1607,"children":1608},{},[1609],{"type":51,"value":1610},"trace-link scoping fix",{"type":51,"value":1612},", limitations. Read before pup \u002F the runner.",{"type":45,"tag":189,"props":1614,"children":1615},{},[1616,1621],{"type":45,"tag":80,"props":1617,"children":1619},{"className":1618},[],[1620],{"type":51,"value":109},{"type":51,"value":1622}," — making a deployed-only app locally runnable (step 3.5). Read when that gap shows.",{"type":45,"tag":189,"props":1624,"children":1625},{},[1626,1631],{"type":45,"tag":80,"props":1627,"children":1629},{"className":1628},[],[1630],{"type":51,"value":985},{"type":51,"value":1632}," — the Python runner to copy + fill.",{"type":45,"tag":1634,"props":1635,"children":1636},"style",{},[1637],{"type":51,"value":1638},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":1640,"total":1734},[1641,1656,1670,1681,1697,1711,1726],{"slug":1642,"name":1642,"fn":1643,"description":1644,"org":1645,"tags":1646,"stars":28,"repoUrl":29,"updatedAt":1655},"agent-install","install Datadog Agent on Kubernetes","Install the Datadog Agent on Kubernetes using the Datadog Operator — required before enabling Single Step Instrumentation (SSI), which automatically instruments applications for APM without code changes. Only use if no Datadog Agent is deployed on the cluster yet.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1647,1648,1651,1654],{"name":23,"slug":24,"type":15},{"name":1649,"slug":1650,"type":15},"Deployment","deployment",{"name":1652,"slug":1653,"type":15},"Kubernetes","kubernetes",{"name":17,"slug":18,"type":15},"2026-04-15T04:57:27.489805",{"slug":1657,"name":1657,"fn":1658,"description":1659,"org":1660,"tags":1661,"stars":28,"repoUrl":29,"updatedAt":1669},"agent-observability-auto-experiment","run iterative code improvements using Datadog data","Run an iterative code-improvement hill-climb against real Datadog LLM-Obs data, locally, with Claude Code as the agent. Establishes a baseline eval, makes one focused change, re-scores with the same harness, keeps the change if it improves the score in the goal's direction (labeling within-noise gains tentative), and repeats. Use when the user says \"run an auto experiment\", \"hill-climb this code\", \"iteratively improve X and measure the delta\", \"optimize this prompt\u002Ffile against my traces\", \"auto-optimize against LLM-Obs\", or wants the local equivalent of the auto_experiments worker. Works from a local dataset file, an ml_app, a dataset_id, or a list of trace_ids.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1662,1663,1664,1667,1668],{"name":23,"slug":24,"type":15},{"name":26,"slug":27,"type":15},{"name":1665,"slug":1666,"type":15},"Evals","evals",{"name":20,"slug":21,"type":15},{"name":17,"slug":18,"type":15},"2026-07-31T05:52:13.711906",{"slug":1671,"name":1671,"fn":1672,"description":1673,"org":1674,"tags":1675,"stars":28,"repoUrl":29,"updatedAt":1680},"agent-observability-eval-bootstrap","bootstrap evaluators from production traces","Bootstrap evaluators from production traces — by default propose online LLM-judge evaluators and, after you confirm, create them in Datadog as disabled drafts (never auto-enabled); on request emit Python SDK code or a framework-agnostic JSON spec instead. Use when user says \"bootstrap evaluators\", \"generate evaluators\", \"create evals from traces\", \"eval bootstrap\", \"write evaluators\", \"build eval suite\", \"publish evaluators\", or wants to generate BaseEvaluator\u002FLLMJudge code or online judge configs from production LLM trace data. Works with ml_app and optional RCA report or failure hypothesis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1676,1677,1678,1679],{"name":23,"slug":24,"type":15},{"name":1665,"slug":1666,"type":15},{"name":20,"slug":21,"type":15},{"name":17,"slug":18,"type":15},"2026-06-19T09:04:19.259734",{"slug":1682,"name":1682,"fn":1683,"description":1684,"org":1685,"tags":1686,"stars":28,"repoUrl":29,"updatedAt":1696},"agent-observability-eval-pipeline","run agent observability and evaluation pipelines","End-to-end Agent Observability pipeline for an instrumented ml_app — classify production traces, root-cause failures, bootstrap evaluators, then (optionally) sample + publish a dataset, generate + run an experiment, and analyze results. Six narrated phases with a standardized banner and a \"continue\" checkpoint between each. Pure orchestration over the agent-observability sub-skills (`agent-observability-session-classify`, `agent-observability-trace-rca`, `agent-observability-eval-bootstrap`, `agent-observability-experiment-py-bootstrap`, `agent-observability-experiment-analyzer`). Use when user says \"run the eval pipeline\", \"go from traces to evals\", \"bootstrap evals end to end\", \"classify then RCA then bootstrap\", \"build an eval set from scratch\", \"onboard me to datasets and experiments\", \"walk me through experiments\", \"I have an ml_app, now what\", \"Agent Observability onboarding\", \"guided experiment setup\", \"from traces to experiments\", or wants a deterministic, narrated tour from production data through evaluators, datasets, and experiments. Stop early with `--stop-after \u003Cphase>` to short-circuit at evaluators or dataset, or resume mid-flow with `--start-at \u003Cphase>`.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1687,1690,1693,1694,1695],{"name":1688,"slug":1689,"type":15},"Agents","agents",{"name":1691,"slug":1692,"type":15},"Data Pipeline","data-pipeline",{"name":23,"slug":24,"type":15},{"name":1665,"slug":1666,"type":15},{"name":17,"slug":18,"type":15},"2026-06-19T09:04:17.231423",{"slug":1698,"name":1698,"fn":1699,"description":1700,"org":1701,"tags":1702,"stars":28,"repoUrl":29,"updatedAt":1710},"agent-observability-experiment-analyzer","analyze LLM experiment results","Analyze LLM experiment results. Handles single or comparative experiments, exploratory or Q&A modes. Use when user says \"analyze experiment\", \"compare experiments\", \"analyze against baseline\", or provides one or two experiment IDs for analysis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1703,1706,1707,1708,1709],{"name":1704,"slug":1705,"type":15},"Analytics","analytics",{"name":23,"slug":24,"type":15},{"name":1665,"slug":1666,"type":15},{"name":20,"slug":21,"type":15},{"name":17,"slug":18,"type":15},"2026-06-19T09:04:21.212498",{"slug":1712,"name":1712,"fn":1713,"description":1714,"org":1715,"tags":1716,"stars":28,"repoUrl":29,"updatedAt":1725},"agent-observability-experiment-py-bootstrap","generate Python experiment clients for LLM observability","Generates a self-contained Python experiment client that uses the ddtrace.llmobs SDK. Emits either a runnable .py script or a Jupyter .ipynb notebook matching the canonical DataDog reference notebook style. Use when the user says \"generate Python experiment\", \"write an SDK experiment\", \"create a ddtrace experiment\", \"Python notebook experiment\", \"use the Agent Observability SDK\", or has `ddtrace` installed and wants idiomatic SDK code.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1717,1718,1721,1722,1723],{"name":23,"slug":24,"type":15},{"name":1719,"slug":1720,"type":15},"Jupyter","jupyter",{"name":20,"slug":21,"type":15},{"name":17,"slug":18,"type":15},{"name":1724,"slug":809,"type":15},"Python","2026-06-19T09:04:22.640384",{"slug":4,"name":4,"fn":5,"description":6,"org":1727,"tags":1728,"stars":28,"repoUrl":29,"updatedAt":30},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1729,1730,1731,1732,1733],{"name":23,"slug":24,"type":15},{"name":26,"slug":27,"type":15},{"name":20,"slug":21,"type":15},{"name":17,"slug":18,"type":15},{"name":13,"slug":14,"type":15},34,{"items":1736,"total":1862},[1737,1744,1752,1759,1767,1775,1783,1791,1803,1815,1827,1847],{"slug":1642,"name":1642,"fn":1643,"description":1644,"org":1738,"tags":1739,"stars":28,"repoUrl":29,"updatedAt":1655},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1740,1741,1742,1743],{"name":23,"slug":24,"type":15},{"name":1649,"slug":1650,"type":15},{"name":1652,"slug":1653,"type":15},{"name":17,"slug":18,"type":15},{"slug":1657,"name":1657,"fn":1658,"description":1659,"org":1745,"tags":1746,"stars":28,"repoUrl":29,"updatedAt":1669},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1747,1748,1749,1750,1751],{"name":23,"slug":24,"type":15},{"name":26,"slug":27,"type":15},{"name":1665,"slug":1666,"type":15},{"name":20,"slug":21,"type":15},{"name":17,"slug":18,"type":15},{"slug":1671,"name":1671,"fn":1672,"description":1673,"org":1753,"tags":1754,"stars":28,"repoUrl":29,"updatedAt":1680},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1755,1756,1757,1758],{"name":23,"slug":24,"type":15},{"name":1665,"slug":1666,"type":15},{"name":20,"slug":21,"type":15},{"name":17,"slug":18,"type":15},{"slug":1682,"name":1682,"fn":1683,"description":1684,"org":1760,"tags":1761,"stars":28,"repoUrl":29,"updatedAt":1696},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1762,1763,1764,1765,1766],{"name":1688,"slug":1689,"type":15},{"name":1691,"slug":1692,"type":15},{"name":23,"slug":24,"type":15},{"name":1665,"slug":1666,"type":15},{"name":17,"slug":18,"type":15},{"slug":1698,"name":1698,"fn":1699,"description":1700,"org":1768,"tags":1769,"stars":28,"repoUrl":29,"updatedAt":1710},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1770,1771,1772,1773,1774],{"name":1704,"slug":1705,"type":15},{"name":23,"slug":24,"type":15},{"name":1665,"slug":1666,"type":15},{"name":20,"slug":21,"type":15},{"name":17,"slug":18,"type":15},{"slug":1712,"name":1712,"fn":1713,"description":1714,"org":1776,"tags":1777,"stars":28,"repoUrl":29,"updatedAt":1725},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1778,1779,1780,1781,1782],{"name":23,"slug":24,"type":15},{"name":1719,"slug":1720,"type":15},{"name":20,"slug":21,"type":15},{"name":17,"slug":18,"type":15},{"name":1724,"slug":809,"type":15},{"slug":4,"name":4,"fn":5,"description":6,"org":1784,"tags":1785,"stars":28,"repoUrl":29,"updatedAt":30},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1786,1787,1788,1789,1790],{"name":23,"slug":24,"type":15},{"name":26,"slug":27,"type":15},{"name":20,"slug":21,"type":15},{"name":17,"slug":18,"type":15},{"name":13,"slug":14,"type":15},{"slug":1792,"name":1792,"fn":1793,"description":1794,"org":1795,"tags":1796,"stars":28,"repoUrl":29,"updatedAt":1802},"agent-observability-session-classify","evaluate user intent satisfaction in sessions","Classify whether user intent was satisfied in a Datadog Agent Observability trace or session. Three modes: (1) session_id — classify a single CMD+I assistant session with RUM; (2) trace_id — classify a single Agent Observability trace without RUM; (3) ml_app — sample and classify multiple sessions or traces from a given LLM app. Output is compact by default (verdict + one-sentence reason). Use when evaluating satisfaction, classifying sessions\u002Ftraces, labeling data, or generating signal for agent-observability-eval-pipeline or agent-observability-trace-rca.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1797,1798,1799,1800,1801],{"name":1704,"slug":1705,"type":15},{"name":23,"slug":24,"type":15},{"name":1665,"slug":1666,"type":15},{"name":20,"slug":21,"type":15},{"name":17,"slug":18,"type":15},"2026-06-19T09:04:26.341497",{"slug":1804,"name":1804,"fn":1805,"description":1806,"org":1807,"tags":1808,"stars":28,"repoUrl":29,"updatedAt":1814},"agent-observability-trace-rca","diagnose LLM application failures from traces","Root cause analysis on production LLM traces. Diagnoses why an LLM application is failing — works from eval judge verdicts, runtime errors, or structural anomalies depending on what signals are present. Walks the span tree from symptom to root cause. Use when user says \"what's wrong with my app\", \"why is my eval failing\", \"analyze errors\", \"root cause analysis\", \"diagnose failures\", or wants to understand production failure patterns.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1809,1810,1811,1812,1813],{"name":23,"slug":24,"type":15},{"name":26,"slug":27,"type":15},{"name":1665,"slug":1666,"type":15},{"name":20,"slug":21,"type":15},{"name":17,"slug":18,"type":15},"2026-06-19T09:04:24.316244",{"slug":1816,"name":1816,"fn":1817,"description":1818,"org":1819,"tags":1820,"stars":28,"repoUrl":29,"updatedAt":1826},"agent-skills","use Datadog observability skills","Datadog skills for AI agents. Essential monitoring, logging, tracing and observability.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1821,1822,1825],{"name":23,"slug":24,"type":15},{"name":1823,"slug":1824,"type":15},"Monitoring","monitoring",{"name":17,"slug":18,"type":15},"2026-04-06T18:08:33.337476",{"slug":1828,"name":1828,"fn":1829,"description":1830,"org":1831,"tags":1832,"stars":28,"repoUrl":29,"updatedAt":1846},"datadog-app","build and manage Datadog applications","Guides developers building Datadog Apps with TypeScript, React, the @datadog\u002Fapps scaffolder, and @datadog\u002Fvite-plugin. Use when a user wants to scaffold, run, debug, upgrade, build, upload, publish, upload without publishing (draft upload), add an upload-no-publish script, set up CI\u002FCD, use OAuth or API\u002Fapplication key auth, trigger\u002Fpoll Workflow Automation, choose DDSQL or Action Catalog for backend data access, or query app datastores with DDSQL, including backend function troubleshooting.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1833,1834,1837,1840,1843],{"name":23,"slug":24,"type":15},{"name":1835,"slug":1836,"type":15},"Frontend","frontend",{"name":1838,"slug":1839,"type":15},"React","react",{"name":1841,"slug":1842,"type":15},"TypeScript","typescript",{"name":1844,"slug":1845,"type":15},"Vite","vite","2026-06-18T08:01:32.562331",{"slug":1848,"name":1848,"fn":1849,"description":1850,"org":1851,"tags":1852,"stars":28,"repoUrl":29,"updatedAt":1861},"dd-apm","query Datadog APM traces","APM - install, onboard, instrument, enable, set up, configure, traces, services, dependencies, performance analysis. Use for any request involving Datadog APM setup, instrumentation (SSI, ddtrace, agent install), or analysis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1853,1854,1857,1858],{"name":23,"slug":24,"type":15},{"name":1855,"slug":1856,"type":15},"Distributed Tracing","distributed-tracing",{"name":17,"slug":18,"type":15},{"name":1859,"slug":1860,"type":15},"Performance","performance","2026-04-06T18:08:34.575282",35]