[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-datadog-labs-triage-flaky-test":3,"mdc--gvpy3o-key":39,"related-repo-datadog-labs-triage-flaky-test":1700,"related-org-datadog-labs-triage-flaky-test":1807},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":28,"repoUrl":29,"updatedAt":30,"license":31,"forks":32,"topics":33,"repo":34,"sourceUrl":37,"mdContent":38},"triage-flaky-test","triage and resolve flaky tests","Load when investigating a specific flaky test. Gets history, failure pattern, and category, then recommends fix, quarantine, or escalate.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"datadog-labs","Datadog Labs","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fdatadog-labs.png",[12,16,19,22,25],{"name":13,"slug":14,"type":15},"QA","qa","tag",{"name":17,"slug":18,"type":15},"Datadog","datadog",{"name":20,"slug":21,"type":15},"Triage","triage",{"name":23,"slug":24,"type":15},"Testing","testing",{"name":26,"slug":27,"type":15},"Debugging","debugging",145,"https:\u002F\u002Fgithub.com\u002Fdatadog-labs\u002Fagent-skills","2026-05-28T06:54:41.267229",null,19,[],{"repoUrl":29,"stars":28,"forks":32,"topics":35,"description":36},[],"Public repository for Datadog Agent Skills","https:\u002F\u002Fgithub.com\u002Fdatadog-labs\u002Fagent-skills\u002Ftree\u002FHEAD\u002Fdd-software-delivery\u002Ftriage-flaky-test","---\nname: triage-flaky-test\ndescription: Load when investigating a specific flaky test. Gets history, failure pattern, and category, then recommends fix, quarantine, or escalate.\nmetadata:\n  version: \"1.0.0\"\n  author: datadog-labs\n  repository: https:\u002F\u002Fgithub.com\u002Fdatadog-labs\u002Fagent-skills\n  tags: datadog,ci,cicd,flaky,flaky-tests,test-optimization\n  alwaysApply: \"false\"\n---\n\n# Triage Flaky Test\n\nOne-line summary: Investigate a specific flaky test — get history, failure pattern, and category, then recommend fix, quarantine, or escalate.\n\nRequires: `dd-pup` skill (pup CLI installed and authenticated).\n\n---\n\n## Backend\n\n**Detection** — At the start of every invocation, before taking any action, determine which backend to use:\n\n1. If the user passed `--backend pup` anywhere → use **pup mode** immediately. Skip steps 2–4.\n2. Check whether `get_datadog_flaky_tests` appears in your available tools.\n3. If present → use **MCP mode** throughout. Call tools exactly as named in this skill's workflow sections.\n4. If absent → check whether `pup` is executable: run `pup --version` via Bash. If the command exits successfully (exit code 0), pup is available.\n5. If pup responds → use **pup mode** throughout. Translate every tool call using the Tool Reference appendix at the bottom of this file.\n6. If neither is available → stop and tell the user:\n   > \"Neither the Datadog MCP server nor the pup CLI is available. Connect the MCP server or install pup (`brew install datadog-labs\u002Fpack\u002Fpup`).\"\n\n**pup invocation rules:**\n- Invoke via Bash. pup always outputs JSON — parse directly.\n- Repository IDs passed to pup must be fully lowercase (the API rejects mixed-case).\n- Sort values starting with `-` require `=` syntax: `--sort=\"-last_flaked\"` (not `--sort \"-last_flaked\"`).\n- If pup returns a 401\u002F403, tell the user to run `pup auth refresh` or `pup auth login`.\n\n---\n\n## Input\n\n| Parameter | Description |\n|---|---|\n| Test name | Fully qualified test name (e.g. `TestMyFunc` or `com.example.MyTest`) |\n| Repository | Lowercase, no-schema URL (e.g. `github.com\u002Forg\u002Frepo`). Derive from `git remote get-url origin` if not provided. |\n\n---\n\n## Workflow\n\n### STEP 0 — Parse Input\n\nDerive repository ID from git if not provided:\n```bash\ngit remote get-url origin\n# Strip protocol and trailing .git, then lowercase the result\n# e.g. https:\u002F\u002Fgithub.com\u002FDataDog\u002Fmy-repo.git → github.com\u002Fdatadog\u002Fmy-repo\n```\n\n**Validation fallback:** If STEP 1 returns no results, confirm the correct repository by searching without a repo filter:\n```\nTool: search_datadog_test_events\nquery: @test.name:\"\u003Ctest-name>\"\nfrom: now-30d\ntest_level: test\n```\nExtract `@git.repository.id_v2` from results and retry STEP 1 with the confirmed value.\n\n### STEP 1 — Get Flaky Test Details\n\n**Preferred — use `fingerprint_fqn` if known** (`fingerprint_fqn` is a valid CI Visibility search facet):\n```\nTool: get_datadog_flaky_tests\nquery: fingerprint_fqn:\u003Cfqn>\nsort_field: last_flaked\nsort_order: desc\n```\n\n**Fallback — use name + suite + repo:**\n```\nTool: get_datadog_flaky_tests\nquery: @test.name:\"\u003Ctest-name>\" @test.suite:\"\u003Csuite>\" @git.repository.id_v2:\"\u003Crepo>\"\nsort_field: last_flaked\nsort_order: desc\n```\nOmit `@test.suite` if unknown. Do not filter by `flaky_test_state` — return the test regardless of state.\n\nNote: the query filter facet is `flaky_test_state`; the returned response attribute is `flaky_state` — do not use `flaky_state:active` as a query filter.\n\nExtract from results:\n- `fingerprint_fqn` — unique test identifier; used as the `id` in STEP 5 write call. **If absent, do not proceed to quarantine — see STEP 5.**\n- `flaky_state` — current state (active \u002F quarantined \u002F disabled \u002F fixed)\n- `test_stats.failure_rate_pct` — percentage of runs that fail\n- `flaky_category` — root cause category\n- `codeowners` — owning team\n- `pipeline_stats.total_lost_time_ms` — total CI time lost\n\n### STEP 2 — Get Recent Failure History\n\n```\nTool: search_datadog_test_events\nquery: @test.name:\"\u003Ctest-name>\" @test.suite:\"\u003Csuite>\" @test.status:fail @git.repository.id_v2:\"\u003Crepo>\"\nfrom: now-7d\ntest_level: test\n```\n\nExtract:\n- Error messages and stack traces (`@error.message`, `@error.stack`)\n- Failing branches (`@git.branch`) — branch-specific vs. widespread\n- Frequency pattern — random timing or specific conditions\n- Unique `@ci.pipeline.id` values for blast radius (STEP 3)\n\n### STEP 3 — Check Blast Radius\n\nCount distinct pipelines impacted using pipeline IDs from STEP 2:\n\n```\nTool: aggregate_datadog_ci_pipeline_events\nquery: @ci.status:error @ci.pipeline.id:(\u003Cid1> OR \u003Cid2> OR ...) @git.repository.id_v2:\"\u003Crepo>\"\nci_level: pipeline\naggregation: count\ngroup_by: [\"@ci.pipeline.name\"]\nfrom: now-7d\n```\n\nUse the first 10 pipeline IDs from STEP 2 (cap at 10; if more are available, run a second batch and merge results by summing counts per `@ci.pipeline.name` across batches). Report blast radius as: total number of unique pipelines impacted and whether failures are branch-specific or widespread.\n\nNote: a pipeline failure is not necessarily caused solely by this flaky test — treat blast radius as a signal, not a definitive count.\n\n### STEP 4 — Recommend Fix or Quarantine\n\nUse `flaky_category` from STEP 1 and error messages from STEP 2.\n\n**Root cause first:**\n- Read the full error trace from bottom to top — chained errors hide the real cause; the innermost error is the root cause, not the first line.\n- Identify the exact source of nondeterminism (race, ordering, stale state, timing).\n- If the root cause is a CI infrastructure problem (runner unavailable, Docker daemon failure, network outage) → do NOT propose a code fix; classify as `infra` and recommend retry instead.\n- If root cause is uncertain and cannot be confirmed from the stack trace → skip fix, go to quarantine.\n\n**Fix at the correct layer:**\n- Test issue → fix in test or test helper only.\n- Production bug exposed by the test → fix in production code.\n- Shared helper used by multiple tests → fix the helper AND update all call sites.\n\n**Forbidden — do not propose these:**\n- Timing hacks: increasing timeouts, adding sleeps, widening time windows, adding retries.\n- Masking: relaxing assertions (e.g., exact match → at least 1), dropping validations.\n- Partial fixes: touching one call site when multiple share the root cause.\n\n**Fix patterns by category:**\n\n| Category | Approach |\n|---|---|\n| `timeout` | Identify the slow operation and make it synchronous or deterministic — do NOT simply raise the timeout constant |\n| `concurrency` | Add deterministic synchronization (barriers, channels, locks); remove shared mutable state between tests |\n| `network` | Mock or stub network calls at the boundary; if the test requires a real connection, isolate it with a test server |\n| `time` | Inject a controllable clock; replace wall-clock assertions with relative or event-driven checks |\n| `order_dependency` | Isolate test state with setup\u002Fteardown; eliminate dependencies on execution order or global state |\n| `environment_dependency` | Mock env variables and external config; use test-local fixtures, not shared directories or singletons |\n| `resource_leak` | Ensure every resource opened in a test is closed in teardown; use cleanup hooks that run even on failure |\n| `randomness` | Fix the random seed for the test run; use deterministic inputs instead of random generation |\n| `asynchronous_wait` | Replace fixed sleeps with condition polling or event\u002Fsignal-driven waits with a hard timeout |\n| `io` | Use temp files\u002Fdirs cleaned up in teardown; mock or stub filesystem interactions |\n| `unknown` | Skip fix attempt → go to quarantine |\n\n**Before proposing code changes, verify all of the following — if any fails, skip fix and recommend quarantine:**\n- The root cause is the innermost error in the trace, not a surface-level symptom.\n- The failure is a code problem, not a CI infrastructure problem.\n- The fix eliminates the root cause (not just reduces flake probability).\n- The fix is at the correct layer (test vs. production vs. shared helper).\n- All call sites of any shared code are updated.\n- No timing hacks or relaxed assertions introduced.\n\n**Decision:**\n- If category is `unknown` OR verification above fails → skip fix, recommend quarantine\n- If category is known AND root cause is confirmed AND fix is valid → propose specific code change\n\n### STEP 5 — Produce Triage Brief and Act\n\n```\nFlaky Test Triage Brief\n=======================\nTest:           \u003Cfully qualified test name>\nService:        \u003C@test.service>\nCategory:       \u003Cflaky_category>\nFailure Rate:   \u003Ctest_stats.failure_rate_pct>%\nDuration Lost:  \u003Cpipeline_stats.total_lost_time_ms>ms\nCodeowners:     \u003Ccodeowners>\nBlast Radius:   \u003CN> pipelines (\u003Cbranch-specific | widespread>) [approximate — other failures in the same pipeline runs may not be related]\n\nEvidence:\n  \u003C1-2 key error message lines from STEP 2>\n\nRecommendation: \u003Cfix | quarantine | escalate>\nConfidence:     \u003Chigh | medium | low>\nAction:         \u003Cspecific next step>\n```\n\n**Decision thresholds:**\n- `failure_rate_pct > 10` OR blast radius > 5 pipelines → **quarantine**\n- `failure_rate_pct ≤ 10` AND known category AND clear fix → **fix**\n- `failure_rate_pct ≤ 10` AND category `unknown` → **escalate** to codeowners with triage brief\n\n**If recommending quarantine**, present and require explicit user approval before writing:\n\n```\nProposed action: quarantine \"\u003Ctest-name>\"\n  id (fingerprint_fqn): \u003Cfingerprint_fqn from STEP 1>\n  Effect: test still runs but failures are suppressed (CI will not be blocked)\n  Reversible: yes — set new_state: active to restore\n\nApprove? (yes\u002Fno)\n```\n\n**If `fingerprint_fqn` was not returned in STEP 1** (test not yet in FTM or query returned no results): do not attempt the write. Surface an error and ask the user to open the Flaky Test Management UI directly to quarantine manually.\n\nOnly after explicit approval and a confirmed `fingerprint_fqn`:\n\n**MCP mode:**\n```\nTool: update_datadog_flaky_test_states\ntest_ids: [\"\u003Cfingerprint_fqn>\"]\nnew_state: quarantined\n```\n\n**pup mode:**\n```bash\ncat > \u002Ftmp\u002Fflaky-update.json \u003C\u003C'EOF'\n{\n  \"data\": {\n    \"type\": \"UpdateFlakyTestsRequest\",\n    \"attributes\": {\n      \"tests\": [{\"id\": \"\u003Cfingerprint_fqn>\", \"new_state\": \"quarantined\"}]\n    }\n  }\n}\nEOF\npup test-optimization flaky-tests update --file \u002Ftmp\u002Fflaky-update.json\n```\n\nTo undo: repeat with `new_state: active` \u002F `\"new_state\": \"active\"`.\n\n---\n\n## Tool Reference\n\nThis appendix applies only in **pup mode**. In MCP mode, use the tool names in the workflow sections directly.\n\n| MCP Tool | pup Command |\n|---|---|\n| `get_datadog_flaky_tests` (by fingerprint_fqn) | `pup cicd flaky-tests search --query \"fingerprint_fqn:\u003Cfqn>\" --sort=\"-last_flaked\" --limit 5` |\n| `get_datadog_flaky_tests` (by name + suite + repo) | `pup cicd flaky-tests search --query \"@test.name:\\\"...\\\" @test.suite:\\\"...\\\" @git.repository.id_v2:\\\"...\\\"\" --sort=\"-last_flaked\" --limit 10` |\n| `search_datadog_test_events` (validation fallback) | `pup cicd tests search --query \"@test.name:\\\"\u003Ctest-name>\\\"\" --from 30d --limit 5` |\n| `search_datadog_test_events` (failure history) | `pup cicd tests search --query \"@test.name:\\\"...\\\" @test.suite:\\\"...\\\" @test.status:fail @git.repository.id_v2:\\\"...\\\"\" --from 7d --limit 20` |\n| `aggregate_datadog_ci_pipeline_events` (blast radius) | `pup cicd events aggregate --query \"@ci.status:error @ci.pipeline.id:(...) @git.repository.id_v2:\\\"...\\\"\" --compute count --group-by \"@ci.pipeline.name\" --from 7d` |\n| `update_datadog_flaky_test_states` | Write body to `\u002Ftmp\u002Fflaky-update.json`, then `pup test-optimization flaky-tests update --file \u002Ftmp\u002Fflaky-update.json` |\n",{"data":40,"body":45},{"name":4,"description":6,"metadata":41},{"version":42,"author":8,"repository":29,"tags":43,"alwaysApply":44},"1.0.0","datadog,ci,cicd,flaky,flaky-tests,test-optimization","false",{"type":46,"children":47},"root",[48,56,62,76,80,87,98,202,210,282,285,291,376,379,385,392,397,455,465,475,488,494,519,528,536,545,566,594,599,679,685,694,699,753,759,764,773,786,791,797,809,817,848,856,874,882,900,908,1117,1125,1158,1166,1186,1192,1201,1209,1268,1278,1287,1304,1316,1324,1333,1341,1489,1509,1512,1518,1529,1694],{"type":49,"tag":50,"props":51,"children":52},"element","h1",{"id":4},[53],{"type":54,"value":55},"text","Triage Flaky Test",{"type":49,"tag":57,"props":58,"children":59},"p",{},[60],{"type":54,"value":61},"One-line summary: Investigate a specific flaky test — get history, failure pattern, and category, then recommend fix, quarantine, or escalate.",{"type":49,"tag":57,"props":63,"children":64},{},[65,67,74],{"type":54,"value":66},"Requires: ",{"type":49,"tag":68,"props":69,"children":71},"code",{"className":70},[],[72],{"type":54,"value":73},"dd-pup",{"type":54,"value":75}," skill (pup CLI installed and authenticated).",{"type":49,"tag":77,"props":78,"children":79},"hr",{},[],{"type":49,"tag":81,"props":82,"children":84},"h2",{"id":83},"backend",[85],{"type":54,"value":86},"Backend",{"type":49,"tag":57,"props":88,"children":89},{},[90,96],{"type":49,"tag":91,"props":92,"children":93},"strong",{},[94],{"type":54,"value":95},"Detection",{"type":54,"value":97}," — At the start of every invocation, before taking any action, determine which backend to use:",{"type":49,"tag":99,"props":100,"children":101},"ol",{},[102,123,136,148,169,180],{"type":49,"tag":103,"props":104,"children":105},"li",{},[106,108,114,116,121],{"type":54,"value":107},"If the user passed ",{"type":49,"tag":68,"props":109,"children":111},{"className":110},[],[112],{"type":54,"value":113},"--backend pup",{"type":54,"value":115}," anywhere → use ",{"type":49,"tag":91,"props":117,"children":118},{},[119],{"type":54,"value":120},"pup mode",{"type":54,"value":122}," immediately. Skip steps 2–4.",{"type":49,"tag":103,"props":124,"children":125},{},[126,128,134],{"type":54,"value":127},"Check whether ",{"type":49,"tag":68,"props":129,"children":131},{"className":130},[],[132],{"type":54,"value":133},"get_datadog_flaky_tests",{"type":54,"value":135}," appears in your available tools.",{"type":49,"tag":103,"props":137,"children":138},{},[139,141,146],{"type":54,"value":140},"If present → use ",{"type":49,"tag":91,"props":142,"children":143},{},[144],{"type":54,"value":145},"MCP mode",{"type":54,"value":147}," throughout. Call tools exactly as named in this skill's workflow sections.",{"type":49,"tag":103,"props":149,"children":150},{},[151,153,159,161,167],{"type":54,"value":152},"If absent → check whether ",{"type":49,"tag":68,"props":154,"children":156},{"className":155},[],[157],{"type":54,"value":158},"pup",{"type":54,"value":160}," is executable: run ",{"type":49,"tag":68,"props":162,"children":164},{"className":163},[],[165],{"type":54,"value":166},"pup --version",{"type":54,"value":168}," via Bash. If the command exits successfully (exit code 0), pup is available.",{"type":49,"tag":103,"props":170,"children":171},{},[172,174,178],{"type":54,"value":173},"If pup responds → use ",{"type":49,"tag":91,"props":175,"children":176},{},[177],{"type":54,"value":120},{"type":54,"value":179}," throughout. Translate every tool call using the Tool Reference appendix at the bottom of this file.",{"type":49,"tag":103,"props":181,"children":182},{},[183,185],{"type":54,"value":184},"If neither is available → stop and tell the user:\n",{"type":49,"tag":186,"props":187,"children":188},"blockquote",{},[189],{"type":49,"tag":57,"props":190,"children":191},{},[192,194,200],{"type":54,"value":193},"\"Neither the Datadog MCP server nor the pup CLI is available. Connect the MCP server or install pup (",{"type":49,"tag":68,"props":195,"children":197},{"className":196},[],[198],{"type":54,"value":199},"brew install datadog-labs\u002Fpack\u002Fpup",{"type":54,"value":201},").\"",{"type":49,"tag":57,"props":203,"children":204},{},[205],{"type":49,"tag":91,"props":206,"children":207},{},[208],{"type":54,"value":209},"pup invocation rules:",{"type":49,"tag":211,"props":212,"children":213},"ul",{},[214,219,224,261],{"type":49,"tag":103,"props":215,"children":216},{},[217],{"type":54,"value":218},"Invoke via Bash. pup always outputs JSON — parse directly.",{"type":49,"tag":103,"props":220,"children":221},{},[222],{"type":54,"value":223},"Repository IDs passed to pup must be fully lowercase (the API rejects mixed-case).",{"type":49,"tag":103,"props":225,"children":226},{},[227,229,235,237,243,245,251,253,259],{"type":54,"value":228},"Sort values starting with ",{"type":49,"tag":68,"props":230,"children":232},{"className":231},[],[233],{"type":54,"value":234},"-",{"type":54,"value":236}," require ",{"type":49,"tag":68,"props":238,"children":240},{"className":239},[],[241],{"type":54,"value":242},"=",{"type":54,"value":244}," syntax: ",{"type":49,"tag":68,"props":246,"children":248},{"className":247},[],[249],{"type":54,"value":250},"--sort=\"-last_flaked\"",{"type":54,"value":252}," (not ",{"type":49,"tag":68,"props":254,"children":256},{"className":255},[],[257],{"type":54,"value":258},"--sort \"-last_flaked\"",{"type":54,"value":260},").",{"type":49,"tag":103,"props":262,"children":263},{},[264,266,272,274,280],{"type":54,"value":265},"If pup returns a 401\u002F403, tell the user to run ",{"type":49,"tag":68,"props":267,"children":269},{"className":268},[],[270],{"type":54,"value":271},"pup auth refresh",{"type":54,"value":273}," or ",{"type":49,"tag":68,"props":275,"children":277},{"className":276},[],[278],{"type":54,"value":279},"pup auth login",{"type":54,"value":281},".",{"type":49,"tag":77,"props":283,"children":284},{},[],{"type":49,"tag":81,"props":286,"children":288},{"id":287},"input",[289],{"type":54,"value":290},"Input",{"type":49,"tag":292,"props":293,"children":294},"table",{},[295,314],{"type":49,"tag":296,"props":297,"children":298},"thead",{},[299],{"type":49,"tag":300,"props":301,"children":302},"tr",{},[303,309],{"type":49,"tag":304,"props":305,"children":306},"th",{},[307],{"type":54,"value":308},"Parameter",{"type":49,"tag":304,"props":310,"children":311},{},[312],{"type":54,"value":313},"Description",{"type":49,"tag":315,"props":316,"children":317},"tbody",{},[318,347],{"type":49,"tag":300,"props":319,"children":320},{},[321,327],{"type":49,"tag":322,"props":323,"children":324},"td",{},[325],{"type":54,"value":326},"Test name",{"type":49,"tag":322,"props":328,"children":329},{},[330,332,338,339,345],{"type":54,"value":331},"Fully qualified test name (e.g. ",{"type":49,"tag":68,"props":333,"children":335},{"className":334},[],[336],{"type":54,"value":337},"TestMyFunc",{"type":54,"value":273},{"type":49,"tag":68,"props":340,"children":342},{"className":341},[],[343],{"type":54,"value":344},"com.example.MyTest",{"type":54,"value":346},")",{"type":49,"tag":300,"props":348,"children":349},{},[350,355],{"type":49,"tag":322,"props":351,"children":352},{},[353],{"type":54,"value":354},"Repository",{"type":49,"tag":322,"props":356,"children":357},{},[358,360,366,368,374],{"type":54,"value":359},"Lowercase, no-schema URL (e.g. ",{"type":49,"tag":68,"props":361,"children":363},{"className":362},[],[364],{"type":54,"value":365},"github.com\u002Forg\u002Frepo",{"type":54,"value":367},"). Derive from ",{"type":49,"tag":68,"props":369,"children":371},{"className":370},[],[372],{"type":54,"value":373},"git remote get-url origin",{"type":54,"value":375}," if not provided.",{"type":49,"tag":77,"props":377,"children":378},{},[],{"type":49,"tag":81,"props":380,"children":382},{"id":381},"workflow",[383],{"type":54,"value":384},"Workflow",{"type":49,"tag":386,"props":387,"children":389},"h3",{"id":388},"step-0-parse-input",[390],{"type":54,"value":391},"STEP 0 — Parse Input",{"type":49,"tag":57,"props":393,"children":394},{},[395],{"type":54,"value":396},"Derive repository ID from git if not provided:",{"type":49,"tag":398,"props":399,"children":404},"pre",{"className":400,"code":401,"language":402,"meta":403,"style":403},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","git remote get-url origin\n# Strip protocol and trailing .git, then lowercase the result\n# e.g. https:\u002F\u002Fgithub.com\u002FDataDog\u002Fmy-repo.git → github.com\u002Fdatadog\u002Fmy-repo\n","bash","",[405],{"type":49,"tag":68,"props":406,"children":407},{"__ignoreMap":403},[408,436,446],{"type":49,"tag":409,"props":410,"children":413},"span",{"class":411,"line":412},"line",1,[414,420,426,431],{"type":49,"tag":409,"props":415,"children":417},{"style":416},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[418],{"type":54,"value":419},"git",{"type":49,"tag":409,"props":421,"children":423},{"style":422},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[424],{"type":54,"value":425}," remote",{"type":49,"tag":409,"props":427,"children":428},{"style":422},[429],{"type":54,"value":430}," get-url",{"type":49,"tag":409,"props":432,"children":433},{"style":422},[434],{"type":54,"value":435}," origin\n",{"type":49,"tag":409,"props":437,"children":439},{"class":411,"line":438},2,[440],{"type":49,"tag":409,"props":441,"children":443},{"style":442},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[444],{"type":54,"value":445},"# Strip protocol and trailing .git, then lowercase the result\n",{"type":49,"tag":409,"props":447,"children":449},{"class":411,"line":448},3,[450],{"type":49,"tag":409,"props":451,"children":452},{"style":442},[453],{"type":54,"value":454},"# e.g. https:\u002F\u002Fgithub.com\u002FDataDog\u002Fmy-repo.git → github.com\u002Fdatadog\u002Fmy-repo\n",{"type":49,"tag":57,"props":456,"children":457},{},[458,463],{"type":49,"tag":91,"props":459,"children":460},{},[461],{"type":54,"value":462},"Validation fallback:",{"type":54,"value":464}," If STEP 1 returns no results, confirm the correct repository by searching without a repo filter:",{"type":49,"tag":398,"props":466,"children":470},{"className":467,"code":469,"language":54},[468],"language-text","Tool: search_datadog_test_events\nquery: @test.name:\"\u003Ctest-name>\"\nfrom: now-30d\ntest_level: test\n",[471],{"type":49,"tag":68,"props":472,"children":473},{"__ignoreMap":403},[474],{"type":54,"value":469},{"type":49,"tag":57,"props":476,"children":477},{},[478,480,486],{"type":54,"value":479},"Extract ",{"type":49,"tag":68,"props":481,"children":483},{"className":482},[],[484],{"type":54,"value":485},"@git.repository.id_v2",{"type":54,"value":487}," from results and retry STEP 1 with the confirmed value.",{"type":49,"tag":386,"props":489,"children":491},{"id":490},"step-1-get-flaky-test-details",[492],{"type":54,"value":493},"STEP 1 — Get Flaky Test Details",{"type":49,"tag":57,"props":495,"children":496},{},[497,510,512,517],{"type":49,"tag":91,"props":498,"children":499},{},[500,502,508],{"type":54,"value":501},"Preferred — use ",{"type":49,"tag":68,"props":503,"children":505},{"className":504},[],[506],{"type":54,"value":507},"fingerprint_fqn",{"type":54,"value":509}," if known",{"type":54,"value":511}," (",{"type":49,"tag":68,"props":513,"children":515},{"className":514},[],[516],{"type":54,"value":507},{"type":54,"value":518}," is a valid CI Visibility search facet):",{"type":49,"tag":398,"props":520,"children":523},{"className":521,"code":522,"language":54},[468],"Tool: get_datadog_flaky_tests\nquery: fingerprint_fqn:\u003Cfqn>\nsort_field: last_flaked\nsort_order: desc\n",[524],{"type":49,"tag":68,"props":525,"children":526},{"__ignoreMap":403},[527],{"type":54,"value":522},{"type":49,"tag":57,"props":529,"children":530},{},[531],{"type":49,"tag":91,"props":532,"children":533},{},[534],{"type":54,"value":535},"Fallback — use name + suite + repo:",{"type":49,"tag":398,"props":537,"children":540},{"className":538,"code":539,"language":54},[468],"Tool: get_datadog_flaky_tests\nquery: @test.name:\"\u003Ctest-name>\" @test.suite:\"\u003Csuite>\" @git.repository.id_v2:\"\u003Crepo>\"\nsort_field: last_flaked\nsort_order: desc\n",[541],{"type":49,"tag":68,"props":542,"children":543},{"__ignoreMap":403},[544],{"type":54,"value":539},{"type":49,"tag":57,"props":546,"children":547},{},[548,550,556,558,564],{"type":54,"value":549},"Omit ",{"type":49,"tag":68,"props":551,"children":553},{"className":552},[],[554],{"type":54,"value":555},"@test.suite",{"type":54,"value":557}," if unknown. Do not filter by ",{"type":49,"tag":68,"props":559,"children":561},{"className":560},[],[562],{"type":54,"value":563},"flaky_test_state",{"type":54,"value":565}," — return the test regardless of state.",{"type":49,"tag":57,"props":567,"children":568},{},[569,571,576,578,584,586,592],{"type":54,"value":570},"Note: the query filter facet is ",{"type":49,"tag":68,"props":572,"children":574},{"className":573},[],[575],{"type":54,"value":563},{"type":54,"value":577},"; the returned response attribute is ",{"type":49,"tag":68,"props":579,"children":581},{"className":580},[],[582],{"type":54,"value":583},"flaky_state",{"type":54,"value":585}," — do not use ",{"type":49,"tag":68,"props":587,"children":589},{"className":588},[],[590],{"type":54,"value":591},"flaky_state:active",{"type":54,"value":593}," as a query filter.",{"type":49,"tag":57,"props":595,"children":596},{},[597],{"type":54,"value":598},"Extract from results:",{"type":49,"tag":211,"props":600,"children":601},{},[602,625,635,646,657,668],{"type":49,"tag":103,"props":603,"children":604},{},[605,610,612,618,620],{"type":49,"tag":68,"props":606,"children":608},{"className":607},[],[609],{"type":54,"value":507},{"type":54,"value":611}," — unique test identifier; used as the ",{"type":49,"tag":68,"props":613,"children":615},{"className":614},[],[616],{"type":54,"value":617},"id",{"type":54,"value":619}," in STEP 5 write call. ",{"type":49,"tag":91,"props":621,"children":622},{},[623],{"type":54,"value":624},"If absent, do not proceed to quarantine — see STEP 5.",{"type":49,"tag":103,"props":626,"children":627},{},[628,633],{"type":49,"tag":68,"props":629,"children":631},{"className":630},[],[632],{"type":54,"value":583},{"type":54,"value":634}," — current state (active \u002F quarantined \u002F disabled \u002F fixed)",{"type":49,"tag":103,"props":636,"children":637},{},[638,644],{"type":49,"tag":68,"props":639,"children":641},{"className":640},[],[642],{"type":54,"value":643},"test_stats.failure_rate_pct",{"type":54,"value":645}," — percentage of runs that fail",{"type":49,"tag":103,"props":647,"children":648},{},[649,655],{"type":49,"tag":68,"props":650,"children":652},{"className":651},[],[653],{"type":54,"value":654},"flaky_category",{"type":54,"value":656}," — root cause category",{"type":49,"tag":103,"props":658,"children":659},{},[660,666],{"type":49,"tag":68,"props":661,"children":663},{"className":662},[],[664],{"type":54,"value":665},"codeowners",{"type":54,"value":667}," — owning team",{"type":49,"tag":103,"props":669,"children":670},{},[671,677],{"type":49,"tag":68,"props":672,"children":674},{"className":673},[],[675],{"type":54,"value":676},"pipeline_stats.total_lost_time_ms",{"type":54,"value":678}," — total CI time lost",{"type":49,"tag":386,"props":680,"children":682},{"id":681},"step-2-get-recent-failure-history",[683],{"type":54,"value":684},"STEP 2 — Get Recent Failure History",{"type":49,"tag":398,"props":686,"children":689},{"className":687,"code":688,"language":54},[468],"Tool: search_datadog_test_events\nquery: @test.name:\"\u003Ctest-name>\" @test.suite:\"\u003Csuite>\" @test.status:fail @git.repository.id_v2:\"\u003Crepo>\"\nfrom: now-7d\ntest_level: test\n",[690],{"type":49,"tag":68,"props":691,"children":692},{"__ignoreMap":403},[693],{"type":54,"value":688},{"type":49,"tag":57,"props":695,"children":696},{},[697],{"type":54,"value":698},"Extract:",{"type":49,"tag":211,"props":700,"children":701},{},[702,722,735,740],{"type":49,"tag":103,"props":703,"children":704},{},[705,707,713,715,721],{"type":54,"value":706},"Error messages and stack traces (",{"type":49,"tag":68,"props":708,"children":710},{"className":709},[],[711],{"type":54,"value":712},"@error.message",{"type":54,"value":714},", ",{"type":49,"tag":68,"props":716,"children":718},{"className":717},[],[719],{"type":54,"value":720},"@error.stack",{"type":54,"value":346},{"type":49,"tag":103,"props":723,"children":724},{},[725,727,733],{"type":54,"value":726},"Failing branches (",{"type":49,"tag":68,"props":728,"children":730},{"className":729},[],[731],{"type":54,"value":732},"@git.branch",{"type":54,"value":734},") — branch-specific vs. widespread",{"type":49,"tag":103,"props":736,"children":737},{},[738],{"type":54,"value":739},"Frequency pattern — random timing or specific conditions",{"type":49,"tag":103,"props":741,"children":742},{},[743,745,751],{"type":54,"value":744},"Unique ",{"type":49,"tag":68,"props":746,"children":748},{"className":747},[],[749],{"type":54,"value":750},"@ci.pipeline.id",{"type":54,"value":752}," values for blast radius (STEP 3)",{"type":49,"tag":386,"props":754,"children":756},{"id":755},"step-3-check-blast-radius",[757],{"type":54,"value":758},"STEP 3 — Check Blast Radius",{"type":49,"tag":57,"props":760,"children":761},{},[762],{"type":54,"value":763},"Count distinct pipelines impacted using pipeline IDs from STEP 2:",{"type":49,"tag":398,"props":765,"children":768},{"className":766,"code":767,"language":54},[468],"Tool: aggregate_datadog_ci_pipeline_events\nquery: @ci.status:error @ci.pipeline.id:(\u003Cid1> OR \u003Cid2> OR ...) @git.repository.id_v2:\"\u003Crepo>\"\nci_level: pipeline\naggregation: count\ngroup_by: [\"@ci.pipeline.name\"]\nfrom: now-7d\n",[769],{"type":49,"tag":68,"props":770,"children":771},{"__ignoreMap":403},[772],{"type":54,"value":767},{"type":49,"tag":57,"props":774,"children":775},{},[776,778,784],{"type":54,"value":777},"Use the first 10 pipeline IDs from STEP 2 (cap at 10; if more are available, run a second batch and merge results by summing counts per ",{"type":49,"tag":68,"props":779,"children":781},{"className":780},[],[782],{"type":54,"value":783},"@ci.pipeline.name",{"type":54,"value":785}," across batches). Report blast radius as: total number of unique pipelines impacted and whether failures are branch-specific or widespread.",{"type":49,"tag":57,"props":787,"children":788},{},[789],{"type":54,"value":790},"Note: a pipeline failure is not necessarily caused solely by this flaky test — treat blast radius as a signal, not a definitive count.",{"type":49,"tag":386,"props":792,"children":794},{"id":793},"step-4-recommend-fix-or-quarantine",[795],{"type":54,"value":796},"STEP 4 — Recommend Fix or Quarantine",{"type":49,"tag":57,"props":798,"children":799},{},[800,802,807],{"type":54,"value":801},"Use ",{"type":49,"tag":68,"props":803,"children":805},{"className":804},[],[806],{"type":54,"value":654},{"type":54,"value":808}," from STEP 1 and error messages from STEP 2.",{"type":49,"tag":57,"props":810,"children":811},{},[812],{"type":49,"tag":91,"props":813,"children":814},{},[815],{"type":54,"value":816},"Root cause first:",{"type":49,"tag":211,"props":818,"children":819},{},[820,825,830,843],{"type":49,"tag":103,"props":821,"children":822},{},[823],{"type":54,"value":824},"Read the full error trace from bottom to top — chained errors hide the real cause; the innermost error is the root cause, not the first line.",{"type":49,"tag":103,"props":826,"children":827},{},[828],{"type":54,"value":829},"Identify the exact source of nondeterminism (race, ordering, stale state, timing).",{"type":49,"tag":103,"props":831,"children":832},{},[833,835,841],{"type":54,"value":834},"If the root cause is a CI infrastructure problem (runner unavailable, Docker daemon failure, network outage) → do NOT propose a code fix; classify as ",{"type":49,"tag":68,"props":836,"children":838},{"className":837},[],[839],{"type":54,"value":840},"infra",{"type":54,"value":842}," and recommend retry instead.",{"type":49,"tag":103,"props":844,"children":845},{},[846],{"type":54,"value":847},"If root cause is uncertain and cannot be confirmed from the stack trace → skip fix, go to quarantine.",{"type":49,"tag":57,"props":849,"children":850},{},[851],{"type":49,"tag":91,"props":852,"children":853},{},[854],{"type":54,"value":855},"Fix at the correct layer:",{"type":49,"tag":211,"props":857,"children":858},{},[859,864,869],{"type":49,"tag":103,"props":860,"children":861},{},[862],{"type":54,"value":863},"Test issue → fix in test or test helper only.",{"type":49,"tag":103,"props":865,"children":866},{},[867],{"type":54,"value":868},"Production bug exposed by the test → fix in production code.",{"type":49,"tag":103,"props":870,"children":871},{},[872],{"type":54,"value":873},"Shared helper used by multiple tests → fix the helper AND update all call sites.",{"type":49,"tag":57,"props":875,"children":876},{},[877],{"type":49,"tag":91,"props":878,"children":879},{},[880],{"type":54,"value":881},"Forbidden — do not propose these:",{"type":49,"tag":211,"props":883,"children":884},{},[885,890,895],{"type":49,"tag":103,"props":886,"children":887},{},[888],{"type":54,"value":889},"Timing hacks: increasing timeouts, adding sleeps, widening time windows, adding retries.",{"type":49,"tag":103,"props":891,"children":892},{},[893],{"type":54,"value":894},"Masking: relaxing assertions (e.g., exact match → at least 1), dropping validations.",{"type":49,"tag":103,"props":896,"children":897},{},[898],{"type":54,"value":899},"Partial fixes: touching one call site when multiple share the root cause.",{"type":49,"tag":57,"props":901,"children":902},{},[903],{"type":49,"tag":91,"props":904,"children":905},{},[906],{"type":54,"value":907},"Fix patterns by category:",{"type":49,"tag":292,"props":909,"children":910},{},[911,927],{"type":49,"tag":296,"props":912,"children":913},{},[914],{"type":49,"tag":300,"props":915,"children":916},{},[917,922],{"type":49,"tag":304,"props":918,"children":919},{},[920],{"type":54,"value":921},"Category",{"type":49,"tag":304,"props":923,"children":924},{},[925],{"type":54,"value":926},"Approach",{"type":49,"tag":315,"props":928,"children":929},{},[930,947,964,981,998,1015,1032,1049,1066,1083,1100],{"type":49,"tag":300,"props":931,"children":932},{},[933,942],{"type":49,"tag":322,"props":934,"children":935},{},[936],{"type":49,"tag":68,"props":937,"children":939},{"className":938},[],[940],{"type":54,"value":941},"timeout",{"type":49,"tag":322,"props":943,"children":944},{},[945],{"type":54,"value":946},"Identify the slow operation and make it synchronous or deterministic — do NOT simply raise the timeout constant",{"type":49,"tag":300,"props":948,"children":949},{},[950,959],{"type":49,"tag":322,"props":951,"children":952},{},[953],{"type":49,"tag":68,"props":954,"children":956},{"className":955},[],[957],{"type":54,"value":958},"concurrency",{"type":49,"tag":322,"props":960,"children":961},{},[962],{"type":54,"value":963},"Add deterministic synchronization (barriers, channels, locks); remove shared mutable state between tests",{"type":49,"tag":300,"props":965,"children":966},{},[967,976],{"type":49,"tag":322,"props":968,"children":969},{},[970],{"type":49,"tag":68,"props":971,"children":973},{"className":972},[],[974],{"type":54,"value":975},"network",{"type":49,"tag":322,"props":977,"children":978},{},[979],{"type":54,"value":980},"Mock or stub network calls at the boundary; if the test requires a real connection, isolate it with a test server",{"type":49,"tag":300,"props":982,"children":983},{},[984,993],{"type":49,"tag":322,"props":985,"children":986},{},[987],{"type":49,"tag":68,"props":988,"children":990},{"className":989},[],[991],{"type":54,"value":992},"time",{"type":49,"tag":322,"props":994,"children":995},{},[996],{"type":54,"value":997},"Inject a controllable clock; replace wall-clock assertions with relative or event-driven checks",{"type":49,"tag":300,"props":999,"children":1000},{},[1001,1010],{"type":49,"tag":322,"props":1002,"children":1003},{},[1004],{"type":49,"tag":68,"props":1005,"children":1007},{"className":1006},[],[1008],{"type":54,"value":1009},"order_dependency",{"type":49,"tag":322,"props":1011,"children":1012},{},[1013],{"type":54,"value":1014},"Isolate test state with setup\u002Fteardown; eliminate dependencies on execution order or global state",{"type":49,"tag":300,"props":1016,"children":1017},{},[1018,1027],{"type":49,"tag":322,"props":1019,"children":1020},{},[1021],{"type":49,"tag":68,"props":1022,"children":1024},{"className":1023},[],[1025],{"type":54,"value":1026},"environment_dependency",{"type":49,"tag":322,"props":1028,"children":1029},{},[1030],{"type":54,"value":1031},"Mock env variables and external config; use test-local fixtures, not shared directories or singletons",{"type":49,"tag":300,"props":1033,"children":1034},{},[1035,1044],{"type":49,"tag":322,"props":1036,"children":1037},{},[1038],{"type":49,"tag":68,"props":1039,"children":1041},{"className":1040},[],[1042],{"type":54,"value":1043},"resource_leak",{"type":49,"tag":322,"props":1045,"children":1046},{},[1047],{"type":54,"value":1048},"Ensure every resource opened in a test is closed in teardown; use cleanup hooks that run even on failure",{"type":49,"tag":300,"props":1050,"children":1051},{},[1052,1061],{"type":49,"tag":322,"props":1053,"children":1054},{},[1055],{"type":49,"tag":68,"props":1056,"children":1058},{"className":1057},[],[1059],{"type":54,"value":1060},"randomness",{"type":49,"tag":322,"props":1062,"children":1063},{},[1064],{"type":54,"value":1065},"Fix the random seed for the test run; use deterministic inputs instead of random generation",{"type":49,"tag":300,"props":1067,"children":1068},{},[1069,1078],{"type":49,"tag":322,"props":1070,"children":1071},{},[1072],{"type":49,"tag":68,"props":1073,"children":1075},{"className":1074},[],[1076],{"type":54,"value":1077},"asynchronous_wait",{"type":49,"tag":322,"props":1079,"children":1080},{},[1081],{"type":54,"value":1082},"Replace fixed sleeps with condition polling or event\u002Fsignal-driven waits with a hard timeout",{"type":49,"tag":300,"props":1084,"children":1085},{},[1086,1095],{"type":49,"tag":322,"props":1087,"children":1088},{},[1089],{"type":49,"tag":68,"props":1090,"children":1092},{"className":1091},[],[1093],{"type":54,"value":1094},"io",{"type":49,"tag":322,"props":1096,"children":1097},{},[1098],{"type":54,"value":1099},"Use temp files\u002Fdirs cleaned up in teardown; mock or stub filesystem interactions",{"type":49,"tag":300,"props":1101,"children":1102},{},[1103,1112],{"type":49,"tag":322,"props":1104,"children":1105},{},[1106],{"type":49,"tag":68,"props":1107,"children":1109},{"className":1108},[],[1110],{"type":54,"value":1111},"unknown",{"type":49,"tag":322,"props":1113,"children":1114},{},[1115],{"type":54,"value":1116},"Skip fix attempt → go to quarantine",{"type":49,"tag":57,"props":1118,"children":1119},{},[1120],{"type":49,"tag":91,"props":1121,"children":1122},{},[1123],{"type":54,"value":1124},"Before proposing code changes, verify all of the following — if any fails, skip fix and recommend quarantine:",{"type":49,"tag":211,"props":1126,"children":1127},{},[1128,1133,1138,1143,1148,1153],{"type":49,"tag":103,"props":1129,"children":1130},{},[1131],{"type":54,"value":1132},"The root cause is the innermost error in the trace, not a surface-level symptom.",{"type":49,"tag":103,"props":1134,"children":1135},{},[1136],{"type":54,"value":1137},"The failure is a code problem, not a CI infrastructure problem.",{"type":49,"tag":103,"props":1139,"children":1140},{},[1141],{"type":54,"value":1142},"The fix eliminates the root cause (not just reduces flake probability).",{"type":49,"tag":103,"props":1144,"children":1145},{},[1146],{"type":54,"value":1147},"The fix is at the correct layer (test vs. production vs. shared helper).",{"type":49,"tag":103,"props":1149,"children":1150},{},[1151],{"type":54,"value":1152},"All call sites of any shared code are updated.",{"type":49,"tag":103,"props":1154,"children":1155},{},[1156],{"type":54,"value":1157},"No timing hacks or relaxed assertions introduced.",{"type":49,"tag":57,"props":1159,"children":1160},{},[1161],{"type":49,"tag":91,"props":1162,"children":1163},{},[1164],{"type":54,"value":1165},"Decision:",{"type":49,"tag":211,"props":1167,"children":1168},{},[1169,1181],{"type":49,"tag":103,"props":1170,"children":1171},{},[1172,1174,1179],{"type":54,"value":1173},"If category is ",{"type":49,"tag":68,"props":1175,"children":1177},{"className":1176},[],[1178],{"type":54,"value":1111},{"type":54,"value":1180}," OR verification above fails → skip fix, recommend quarantine",{"type":49,"tag":103,"props":1182,"children":1183},{},[1184],{"type":54,"value":1185},"If category is known AND root cause is confirmed AND fix is valid → propose specific code change",{"type":49,"tag":386,"props":1187,"children":1189},{"id":1188},"step-5-produce-triage-brief-and-act",[1190],{"type":54,"value":1191},"STEP 5 — Produce Triage Brief and Act",{"type":49,"tag":398,"props":1193,"children":1196},{"className":1194,"code":1195,"language":54},[468],"Flaky Test Triage Brief\n=======================\nTest:           \u003Cfully qualified test name>\nService:        \u003C@test.service>\nCategory:       \u003Cflaky_category>\nFailure Rate:   \u003Ctest_stats.failure_rate_pct>%\nDuration Lost:  \u003Cpipeline_stats.total_lost_time_ms>ms\nCodeowners:     \u003Ccodeowners>\nBlast Radius:   \u003CN> pipelines (\u003Cbranch-specific | widespread>) [approximate — other failures in the same pipeline runs may not be related]\n\nEvidence:\n  \u003C1-2 key error message lines from STEP 2>\n\nRecommendation: \u003Cfix | quarantine | escalate>\nConfidence:     \u003Chigh | medium | low>\nAction:         \u003Cspecific next step>\n",[1197],{"type":49,"tag":68,"props":1198,"children":1199},{"__ignoreMap":403},[1200],{"type":54,"value":1195},{"type":49,"tag":57,"props":1202,"children":1203},{},[1204],{"type":49,"tag":91,"props":1205,"children":1206},{},[1207],{"type":54,"value":1208},"Decision thresholds:",{"type":49,"tag":211,"props":1210,"children":1211},{},[1212,1228,1244],{"type":49,"tag":103,"props":1213,"children":1214},{},[1215,1221,1223],{"type":49,"tag":68,"props":1216,"children":1218},{"className":1217},[],[1219],{"type":54,"value":1220},"failure_rate_pct > 10",{"type":54,"value":1222}," OR blast radius > 5 pipelines → ",{"type":49,"tag":91,"props":1224,"children":1225},{},[1226],{"type":54,"value":1227},"quarantine",{"type":49,"tag":103,"props":1229,"children":1230},{},[1231,1237,1239],{"type":49,"tag":68,"props":1232,"children":1234},{"className":1233},[],[1235],{"type":54,"value":1236},"failure_rate_pct ≤ 10",{"type":54,"value":1238}," AND known category AND clear fix → ",{"type":49,"tag":91,"props":1240,"children":1241},{},[1242],{"type":54,"value":1243},"fix",{"type":49,"tag":103,"props":1245,"children":1246},{},[1247,1252,1254,1259,1261,1266],{"type":49,"tag":68,"props":1248,"children":1250},{"className":1249},[],[1251],{"type":54,"value":1236},{"type":54,"value":1253}," AND category ",{"type":49,"tag":68,"props":1255,"children":1257},{"className":1256},[],[1258],{"type":54,"value":1111},{"type":54,"value":1260}," → ",{"type":49,"tag":91,"props":1262,"children":1263},{},[1264],{"type":54,"value":1265},"escalate",{"type":54,"value":1267}," to codeowners with triage brief",{"type":49,"tag":57,"props":1269,"children":1270},{},[1271,1276],{"type":49,"tag":91,"props":1272,"children":1273},{},[1274],{"type":54,"value":1275},"If recommending quarantine",{"type":54,"value":1277},", present and require explicit user approval before writing:",{"type":49,"tag":398,"props":1279,"children":1282},{"className":1280,"code":1281,"language":54},[468],"Proposed action: quarantine \"\u003Ctest-name>\"\n  id (fingerprint_fqn): \u003Cfingerprint_fqn from STEP 1>\n  Effect: test still runs but failures are suppressed (CI will not be blocked)\n  Reversible: yes — set new_state: active to restore\n\nApprove? (yes\u002Fno)\n",[1283],{"type":49,"tag":68,"props":1284,"children":1285},{"__ignoreMap":403},[1286],{"type":54,"value":1281},{"type":49,"tag":57,"props":1288,"children":1289},{},[1290,1302],{"type":49,"tag":91,"props":1291,"children":1292},{},[1293,1295,1300],{"type":54,"value":1294},"If ",{"type":49,"tag":68,"props":1296,"children":1298},{"className":1297},[],[1299],{"type":54,"value":507},{"type":54,"value":1301}," was not returned in STEP 1",{"type":54,"value":1303}," (test not yet in FTM or query returned no results): do not attempt the write. Surface an error and ask the user to open the Flaky Test Management UI directly to quarantine manually.",{"type":49,"tag":57,"props":1305,"children":1306},{},[1307,1309,1314],{"type":54,"value":1308},"Only after explicit approval and a confirmed ",{"type":49,"tag":68,"props":1310,"children":1312},{"className":1311},[],[1313],{"type":54,"value":507},{"type":54,"value":1315},":",{"type":49,"tag":57,"props":1317,"children":1318},{},[1319],{"type":49,"tag":91,"props":1320,"children":1321},{},[1322],{"type":54,"value":1323},"MCP mode:",{"type":49,"tag":398,"props":1325,"children":1328},{"className":1326,"code":1327,"language":54},[468],"Tool: update_datadog_flaky_test_states\ntest_ids: [\"\u003Cfingerprint_fqn>\"]\nnew_state: quarantined\n",[1329],{"type":49,"tag":68,"props":1330,"children":1331},{"__ignoreMap":403},[1332],{"type":54,"value":1327},{"type":49,"tag":57,"props":1334,"children":1335},{},[1336],{"type":49,"tag":91,"props":1337,"children":1338},{},[1339],{"type":54,"value":1340},"pup mode:",{"type":49,"tag":398,"props":1342,"children":1344},{"className":400,"code":1343,"language":402,"meta":403,"style":403},"cat > \u002Ftmp\u002Fflaky-update.json \u003C\u003C'EOF'\n{\n  \"data\": {\n    \"type\": \"UpdateFlakyTestsRequest\",\n    \"attributes\": {\n      \"tests\": [{\"id\": \"\u003Cfingerprint_fqn>\", \"new_state\": \"quarantined\"}]\n    }\n  }\n}\nEOF\npup test-optimization flaky-tests update --file \u002Ftmp\u002Fflaky-update.json\n",[1345],{"type":49,"tag":68,"props":1346,"children":1347},{"__ignoreMap":403},[1348,1377,1385,1393,1402,1411,1420,1429,1438,1447,1456],{"type":49,"tag":409,"props":1349,"children":1350},{"class":411,"line":412},[1351,1356,1362,1367,1372],{"type":49,"tag":409,"props":1352,"children":1353},{"style":416},[1354],{"type":54,"value":1355},"cat",{"type":49,"tag":409,"props":1357,"children":1359},{"style":1358},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[1360],{"type":54,"value":1361}," >",{"type":49,"tag":409,"props":1363,"children":1364},{"style":422},[1365],{"type":54,"value":1366}," \u002Ftmp\u002Fflaky-update.json",{"type":49,"tag":409,"props":1368,"children":1369},{"style":1358},[1370],{"type":54,"value":1371}," \u003C\u003C",{"type":49,"tag":409,"props":1373,"children":1374},{"style":1358},[1375],{"type":54,"value":1376},"'EOF'\n",{"type":49,"tag":409,"props":1378,"children":1379},{"class":411,"line":438},[1380],{"type":49,"tag":409,"props":1381,"children":1382},{"style":422},[1383],{"type":54,"value":1384},"{\n",{"type":49,"tag":409,"props":1386,"children":1387},{"class":411,"line":448},[1388],{"type":49,"tag":409,"props":1389,"children":1390},{"style":422},[1391],{"type":54,"value":1392},"  \"data\": {\n",{"type":49,"tag":409,"props":1394,"children":1396},{"class":411,"line":1395},4,[1397],{"type":49,"tag":409,"props":1398,"children":1399},{"style":422},[1400],{"type":54,"value":1401},"    \"type\": \"UpdateFlakyTestsRequest\",\n",{"type":49,"tag":409,"props":1403,"children":1405},{"class":411,"line":1404},5,[1406],{"type":49,"tag":409,"props":1407,"children":1408},{"style":422},[1409],{"type":54,"value":1410},"    \"attributes\": {\n",{"type":49,"tag":409,"props":1412,"children":1414},{"class":411,"line":1413},6,[1415],{"type":49,"tag":409,"props":1416,"children":1417},{"style":422},[1418],{"type":54,"value":1419},"      \"tests\": [{\"id\": \"\u003Cfingerprint_fqn>\", \"new_state\": \"quarantined\"}]\n",{"type":49,"tag":409,"props":1421,"children":1423},{"class":411,"line":1422},7,[1424],{"type":49,"tag":409,"props":1425,"children":1426},{"style":422},[1427],{"type":54,"value":1428},"    }\n",{"type":49,"tag":409,"props":1430,"children":1432},{"class":411,"line":1431},8,[1433],{"type":49,"tag":409,"props":1434,"children":1435},{"style":422},[1436],{"type":54,"value":1437},"  }\n",{"type":49,"tag":409,"props":1439,"children":1441},{"class":411,"line":1440},9,[1442],{"type":49,"tag":409,"props":1443,"children":1444},{"style":422},[1445],{"type":54,"value":1446},"}\n",{"type":49,"tag":409,"props":1448,"children":1450},{"class":411,"line":1449},10,[1451],{"type":49,"tag":409,"props":1452,"children":1453},{"style":1358},[1454],{"type":54,"value":1455},"EOF\n",{"type":49,"tag":409,"props":1457,"children":1459},{"class":411,"line":1458},11,[1460,1464,1469,1474,1479,1484],{"type":49,"tag":409,"props":1461,"children":1462},{"style":416},[1463],{"type":54,"value":158},{"type":49,"tag":409,"props":1465,"children":1466},{"style":422},[1467],{"type":54,"value":1468}," test-optimization",{"type":49,"tag":409,"props":1470,"children":1471},{"style":422},[1472],{"type":54,"value":1473}," flaky-tests",{"type":49,"tag":409,"props":1475,"children":1476},{"style":422},[1477],{"type":54,"value":1478}," update",{"type":49,"tag":409,"props":1480,"children":1481},{"style":422},[1482],{"type":54,"value":1483}," --file",{"type":49,"tag":409,"props":1485,"children":1486},{"style":422},[1487],{"type":54,"value":1488}," \u002Ftmp\u002Fflaky-update.json\n",{"type":49,"tag":57,"props":1490,"children":1491},{},[1492,1494,1500,1502,1508],{"type":54,"value":1493},"To undo: repeat with ",{"type":49,"tag":68,"props":1495,"children":1497},{"className":1496},[],[1498],{"type":54,"value":1499},"new_state: active",{"type":54,"value":1501}," \u002F ",{"type":49,"tag":68,"props":1503,"children":1505},{"className":1504},[],[1506],{"type":54,"value":1507},"\"new_state\": \"active\"",{"type":54,"value":281},{"type":49,"tag":77,"props":1510,"children":1511},{},[],{"type":49,"tag":81,"props":1513,"children":1515},{"id":1514},"tool-reference",[1516],{"type":54,"value":1517},"Tool Reference",{"type":49,"tag":57,"props":1519,"children":1520},{},[1521,1523,1527],{"type":54,"value":1522},"This appendix applies only in ",{"type":49,"tag":91,"props":1524,"children":1525},{},[1526],{"type":54,"value":120},{"type":54,"value":1528},". In MCP mode, use the tool names in the workflow sections directly.",{"type":49,"tag":292,"props":1530,"children":1531},{},[1532,1548],{"type":49,"tag":296,"props":1533,"children":1534},{},[1535],{"type":49,"tag":300,"props":1536,"children":1537},{},[1538,1543],{"type":49,"tag":304,"props":1539,"children":1540},{},[1541],{"type":54,"value":1542},"MCP Tool",{"type":49,"tag":304,"props":1544,"children":1545},{},[1546],{"type":54,"value":1547},"pup Command",{"type":49,"tag":315,"props":1549,"children":1550},{},[1551,1573,1595,1618,1640,1663],{"type":49,"tag":300,"props":1552,"children":1553},{},[1554,1564],{"type":49,"tag":322,"props":1555,"children":1556},{},[1557,1562],{"type":49,"tag":68,"props":1558,"children":1560},{"className":1559},[],[1561],{"type":54,"value":133},{"type":54,"value":1563}," (by fingerprint_fqn)",{"type":49,"tag":322,"props":1565,"children":1566},{},[1567],{"type":49,"tag":68,"props":1568,"children":1570},{"className":1569},[],[1571],{"type":54,"value":1572},"pup cicd flaky-tests search --query \"fingerprint_fqn:\u003Cfqn>\" --sort=\"-last_flaked\" --limit 5",{"type":49,"tag":300,"props":1574,"children":1575},{},[1576,1586],{"type":49,"tag":322,"props":1577,"children":1578},{},[1579,1584],{"type":49,"tag":68,"props":1580,"children":1582},{"className":1581},[],[1583],{"type":54,"value":133},{"type":54,"value":1585}," (by name + suite + repo)",{"type":49,"tag":322,"props":1587,"children":1588},{},[1589],{"type":49,"tag":68,"props":1590,"children":1592},{"className":1591},[],[1593],{"type":54,"value":1594},"pup cicd flaky-tests search --query \"@test.name:\\\"...\\\" @test.suite:\\\"...\\\" @git.repository.id_v2:\\\"...\\\"\" --sort=\"-last_flaked\" --limit 10",{"type":49,"tag":300,"props":1596,"children":1597},{},[1598,1609],{"type":49,"tag":322,"props":1599,"children":1600},{},[1601,1607],{"type":49,"tag":68,"props":1602,"children":1604},{"className":1603},[],[1605],{"type":54,"value":1606},"search_datadog_test_events",{"type":54,"value":1608}," (validation fallback)",{"type":49,"tag":322,"props":1610,"children":1611},{},[1612],{"type":49,"tag":68,"props":1613,"children":1615},{"className":1614},[],[1616],{"type":54,"value":1617},"pup cicd tests search --query \"@test.name:\\\"\u003Ctest-name>\\\"\" --from 30d --limit 5",{"type":49,"tag":300,"props":1619,"children":1620},{},[1621,1631],{"type":49,"tag":322,"props":1622,"children":1623},{},[1624,1629],{"type":49,"tag":68,"props":1625,"children":1627},{"className":1626},[],[1628],{"type":54,"value":1606},{"type":54,"value":1630}," (failure history)",{"type":49,"tag":322,"props":1632,"children":1633},{},[1634],{"type":49,"tag":68,"props":1635,"children":1637},{"className":1636},[],[1638],{"type":54,"value":1639},"pup cicd tests search --query \"@test.name:\\\"...\\\" @test.suite:\\\"...\\\" @test.status:fail @git.repository.id_v2:\\\"...\\\"\" --from 7d --limit 20",{"type":49,"tag":300,"props":1641,"children":1642},{},[1643,1654],{"type":49,"tag":322,"props":1644,"children":1645},{},[1646,1652],{"type":49,"tag":68,"props":1647,"children":1649},{"className":1648},[],[1650],{"type":54,"value":1651},"aggregate_datadog_ci_pipeline_events",{"type":54,"value":1653}," (blast radius)",{"type":49,"tag":322,"props":1655,"children":1656},{},[1657],{"type":49,"tag":68,"props":1658,"children":1660},{"className":1659},[],[1661],{"type":54,"value":1662},"pup cicd events aggregate --query \"@ci.status:error @ci.pipeline.id:(...) @git.repository.id_v2:\\\"...\\\"\" --compute count --group-by \"@ci.pipeline.name\" --from 7d",{"type":49,"tag":300,"props":1664,"children":1665},{},[1666,1675],{"type":49,"tag":322,"props":1667,"children":1668},{},[1669],{"type":49,"tag":68,"props":1670,"children":1672},{"className":1671},[],[1673],{"type":54,"value":1674},"update_datadog_flaky_test_states",{"type":49,"tag":322,"props":1676,"children":1677},{},[1678,1680,1686,1688],{"type":54,"value":1679},"Write body to ",{"type":49,"tag":68,"props":1681,"children":1683},{"className":1682},[],[1684],{"type":54,"value":1685},"\u002Ftmp\u002Fflaky-update.json",{"type":54,"value":1687},", then ",{"type":49,"tag":68,"props":1689,"children":1691},{"className":1690},[],[1692],{"type":54,"value":1693},"pup test-optimization flaky-tests update --file \u002Ftmp\u002Fflaky-update.json",{"type":49,"tag":1695,"props":1696,"children":1697},"style",{},[1698],{"type":54,"value":1699},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":1701,"total":1806},[1702,1719,1735,1746,1762,1776,1792],{"slug":1703,"name":1703,"fn":1704,"description":1705,"org":1706,"tags":1707,"stars":28,"repoUrl":29,"updatedAt":1718},"agent-install","install Datadog Agent on Kubernetes","Install the Datadog Agent on Kubernetes using the Datadog Operator — required before enabling Single Step Instrumentation (SSI), which automatically instruments applications for APM without code changes. Only use if no Datadog Agent is deployed on the cluster yet.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1708,1709,1712,1715],{"name":17,"slug":18,"type":15},{"name":1710,"slug":1711,"type":15},"Deployment","deployment",{"name":1713,"slug":1714,"type":15},"Kubernetes","kubernetes",{"name":1716,"slug":1717,"type":15},"Observability","observability","2026-04-15T04:57:27.489805",{"slug":1720,"name":1720,"fn":1721,"description":1722,"org":1723,"tags":1724,"stars":28,"repoUrl":29,"updatedAt":1734},"agent-observability-auto-experiment","run iterative code improvements using Datadog data","Run an iterative code-improvement hill-climb against real Datadog LLM-Obs data, locally, with Claude Code as the agent. Establishes a baseline eval, makes one focused change, re-scores with the same harness, keeps the change if it improves the score in the goal's direction (labeling within-noise gains tentative), and repeats. Use when the user says \"run an auto experiment\", \"hill-climb this code\", \"iteratively improve X and measure the delta\", \"optimize this prompt\u002Ffile against my traces\", \"auto-optimize against LLM-Obs\", or wants the local equivalent of the auto_experiments worker. Works from a local dataset file, an ml_app, a dataset_id, or a list of trace_ids.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1725,1726,1727,1730,1733],{"name":17,"slug":18,"type":15},{"name":26,"slug":27,"type":15},{"name":1728,"slug":1729,"type":15},"Evals","evals",{"name":1731,"slug":1732,"type":15},"LLM","llm",{"name":1716,"slug":1717,"type":15},"2026-07-31T05:52:13.711906",{"slug":1736,"name":1736,"fn":1737,"description":1738,"org":1739,"tags":1740,"stars":28,"repoUrl":29,"updatedAt":1745},"agent-observability-eval-bootstrap","bootstrap evaluators from production traces","Bootstrap evaluators from production traces — by default propose online LLM-judge evaluators and, after you confirm, create them in Datadog as disabled drafts (never auto-enabled); on request emit Python SDK code or a framework-agnostic JSON spec instead. Use when user says \"bootstrap evaluators\", \"generate evaluators\", \"create evals from traces\", \"eval bootstrap\", \"write evaluators\", \"build eval suite\", \"publish evaluators\", or wants to generate BaseEvaluator\u002FLLMJudge code or online judge configs from production LLM trace data. Works with ml_app and optional RCA report or failure hypothesis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1741,1742,1743,1744],{"name":17,"slug":18,"type":15},{"name":1728,"slug":1729,"type":15},{"name":1731,"slug":1732,"type":15},{"name":1716,"slug":1717,"type":15},"2026-06-19T09:04:19.259734",{"slug":1747,"name":1747,"fn":1748,"description":1749,"org":1750,"tags":1751,"stars":28,"repoUrl":29,"updatedAt":1761},"agent-observability-eval-pipeline","run agent observability and evaluation pipelines","End-to-end Agent Observability pipeline for an instrumented ml_app — classify production traces, root-cause failures, bootstrap evaluators, then (optionally) sample + publish a dataset, generate + run an experiment, and analyze results. Six narrated phases with a standardized banner and a \"continue\" checkpoint between each. Pure orchestration over the agent-observability sub-skills (`agent-observability-session-classify`, `agent-observability-trace-rca`, `agent-observability-eval-bootstrap`, `agent-observability-experiment-py-bootstrap`, `agent-observability-experiment-analyzer`). Use when user says \"run the eval pipeline\", \"go from traces to evals\", \"bootstrap evals end to end\", \"classify then RCA then bootstrap\", \"build an eval set from scratch\", \"onboard me to datasets and experiments\", \"walk me through experiments\", \"I have an ml_app, now what\", \"Agent Observability onboarding\", \"guided experiment setup\", \"from traces to experiments\", or wants a deterministic, narrated tour from production data through evaluators, datasets, and experiments. Stop early with `--stop-after \u003Cphase>` to short-circuit at evaluators or dataset, or resume mid-flow with `--start-at \u003Cphase>`.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1752,1755,1758,1759,1760],{"name":1753,"slug":1754,"type":15},"Agents","agents",{"name":1756,"slug":1757,"type":15},"Data Pipeline","data-pipeline",{"name":17,"slug":18,"type":15},{"name":1728,"slug":1729,"type":15},{"name":1716,"slug":1717,"type":15},"2026-06-19T09:04:17.231423",{"slug":1763,"name":1763,"fn":1764,"description":1765,"org":1766,"tags":1767,"stars":28,"repoUrl":29,"updatedAt":1775},"agent-observability-experiment-analyzer","analyze LLM experiment results","Analyze LLM experiment results. Handles single or comparative experiments, exploratory or Q&A modes. Use when user says \"analyze experiment\", \"compare experiments\", \"analyze against baseline\", or provides one or two experiment IDs for analysis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1768,1771,1772,1773,1774],{"name":1769,"slug":1770,"type":15},"Analytics","analytics",{"name":17,"slug":18,"type":15},{"name":1728,"slug":1729,"type":15},{"name":1731,"slug":1732,"type":15},{"name":1716,"slug":1717,"type":15},"2026-06-19T09:04:21.212498",{"slug":1777,"name":1777,"fn":1778,"description":1779,"org":1780,"tags":1781,"stars":28,"repoUrl":29,"updatedAt":1791},"agent-observability-experiment-py-bootstrap","generate Python experiment clients for LLM observability","Generates a self-contained Python experiment client that uses the ddtrace.llmobs SDK. Emits either a runnable .py script or a Jupyter .ipynb notebook matching the canonical DataDog reference notebook style. Use when the user says \"generate Python experiment\", \"write an SDK experiment\", \"create a ddtrace experiment\", \"Python notebook experiment\", \"use the Agent Observability SDK\", or has `ddtrace` installed and wants idiomatic SDK code.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1782,1783,1786,1787,1788],{"name":17,"slug":18,"type":15},{"name":1784,"slug":1785,"type":15},"Jupyter","jupyter",{"name":1731,"slug":1732,"type":15},{"name":1716,"slug":1717,"type":15},{"name":1789,"slug":1790,"type":15},"Python","python","2026-06-19T09:04:22.640384",{"slug":1793,"name":1793,"fn":1794,"description":1795,"org":1796,"tags":1797,"stars":28,"repoUrl":29,"updatedAt":1805},"agent-observability-replay-trace","iterate on LLM traces with local code","Use when a developer wants to iterate on ONE specific Agent Observability \u002F LLM Obs trace whose output they didn't like — re-running that trace against their LOCAL code, seeing a concise diff of the old vs new output, and looping (change code → replay → diff) until satisfied. Invoked as \u002Fagent-observability-replay-trace \u003Ctrace-id> [changes to test]. Signals: \"replay this trace\"; \"iterate on a trace\"; \"this trace's output is wrong, fix it and re-run\"; \"re-run trace \u003Cid> with \u003Cchange>\"; pasting a trace id from the Agent Observability UI with a description of what to fix. It fetches the trace via the datadog-llmo MCP or the pup CLI, edits code, re-runs the app to emit a NEW trace, and diffs the two — no local server, no browser. For agents traced with ddtrace \u002F LLM Obs (Python first-class), with JSON-serializable entry input. Do NOT use for: scored Experiments or the browser \"Replay\" button (that's agent-observability-replay-experiment), building an experiment from a dataset\u002FCSV, writing evaluators, root-causing failed traces, or RUM\u002FHTTP session replay.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1798,1799,1800,1801,1802],{"name":17,"slug":18,"type":15},{"name":26,"slug":27,"type":15},{"name":1731,"slug":1732,"type":15},{"name":1716,"slug":1717,"type":15},{"name":1803,"slug":1804,"type":15},"Tracing","tracing","2026-07-31T05:52:08.594182",34,{"items":1808,"total":1934},[1809,1816,1824,1831,1839,1847,1855,1863,1875,1887,1899,1919],{"slug":1703,"name":1703,"fn":1704,"description":1705,"org":1810,"tags":1811,"stars":28,"repoUrl":29,"updatedAt":1718},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1812,1813,1814,1815],{"name":17,"slug":18,"type":15},{"name":1710,"slug":1711,"type":15},{"name":1713,"slug":1714,"type":15},{"name":1716,"slug":1717,"type":15},{"slug":1720,"name":1720,"fn":1721,"description":1722,"org":1817,"tags":1818,"stars":28,"repoUrl":29,"updatedAt":1734},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1819,1820,1821,1822,1823],{"name":17,"slug":18,"type":15},{"name":26,"slug":27,"type":15},{"name":1728,"slug":1729,"type":15},{"name":1731,"slug":1732,"type":15},{"name":1716,"slug":1717,"type":15},{"slug":1736,"name":1736,"fn":1737,"description":1738,"org":1825,"tags":1826,"stars":28,"repoUrl":29,"updatedAt":1745},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1827,1828,1829,1830],{"name":17,"slug":18,"type":15},{"name":1728,"slug":1729,"type":15},{"name":1731,"slug":1732,"type":15},{"name":1716,"slug":1717,"type":15},{"slug":1747,"name":1747,"fn":1748,"description":1749,"org":1832,"tags":1833,"stars":28,"repoUrl":29,"updatedAt":1761},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1834,1835,1836,1837,1838],{"name":1753,"slug":1754,"type":15},{"name":1756,"slug":1757,"type":15},{"name":17,"slug":18,"type":15},{"name":1728,"slug":1729,"type":15},{"name":1716,"slug":1717,"type":15},{"slug":1763,"name":1763,"fn":1764,"description":1765,"org":1840,"tags":1841,"stars":28,"repoUrl":29,"updatedAt":1775},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1842,1843,1844,1845,1846],{"name":1769,"slug":1770,"type":15},{"name":17,"slug":18,"type":15},{"name":1728,"slug":1729,"type":15},{"name":1731,"slug":1732,"type":15},{"name":1716,"slug":1717,"type":15},{"slug":1777,"name":1777,"fn":1778,"description":1779,"org":1848,"tags":1849,"stars":28,"repoUrl":29,"updatedAt":1791},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1850,1851,1852,1853,1854],{"name":17,"slug":18,"type":15},{"name":1784,"slug":1785,"type":15},{"name":1731,"slug":1732,"type":15},{"name":1716,"slug":1717,"type":15},{"name":1789,"slug":1790,"type":15},{"slug":1793,"name":1793,"fn":1794,"description":1795,"org":1856,"tags":1857,"stars":28,"repoUrl":29,"updatedAt":1805},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1858,1859,1860,1861,1862],{"name":17,"slug":18,"type":15},{"name":26,"slug":27,"type":15},{"name":1731,"slug":1732,"type":15},{"name":1716,"slug":1717,"type":15},{"name":1803,"slug":1804,"type":15},{"slug":1864,"name":1864,"fn":1865,"description":1866,"org":1867,"tags":1868,"stars":28,"repoUrl":29,"updatedAt":1874},"agent-observability-session-classify","evaluate user intent satisfaction in sessions","Classify whether user intent was satisfied in a Datadog Agent Observability trace or session. Three modes: (1) session_id — classify a single CMD+I assistant session with RUM; (2) trace_id — classify a single Agent Observability trace without RUM; (3) ml_app — sample and classify multiple sessions or traces from a given LLM app. Output is compact by default (verdict + one-sentence reason). Use when evaluating satisfaction, classifying sessions\u002Ftraces, labeling data, or generating signal for agent-observability-eval-pipeline or agent-observability-trace-rca.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1869,1870,1871,1872,1873],{"name":1769,"slug":1770,"type":15},{"name":17,"slug":18,"type":15},{"name":1728,"slug":1729,"type":15},{"name":1731,"slug":1732,"type":15},{"name":1716,"slug":1717,"type":15},"2026-06-19T09:04:26.341497",{"slug":1876,"name":1876,"fn":1877,"description":1878,"org":1879,"tags":1880,"stars":28,"repoUrl":29,"updatedAt":1886},"agent-observability-trace-rca","diagnose LLM application failures from traces","Root cause analysis on production LLM traces. Diagnoses why an LLM application is failing — works from eval judge verdicts, runtime errors, or structural anomalies depending on what signals are present. Walks the span tree from symptom to root cause. Use when user says \"what's wrong with my app\", \"why is my eval failing\", \"analyze errors\", \"root cause analysis\", \"diagnose failures\", or wants to understand production failure patterns.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1881,1882,1883,1884,1885],{"name":17,"slug":18,"type":15},{"name":26,"slug":27,"type":15},{"name":1728,"slug":1729,"type":15},{"name":1731,"slug":1732,"type":15},{"name":1716,"slug":1717,"type":15},"2026-06-19T09:04:24.316244",{"slug":1888,"name":1888,"fn":1889,"description":1890,"org":1891,"tags":1892,"stars":28,"repoUrl":29,"updatedAt":1898},"agent-skills","use Datadog observability skills","Datadog skills for AI agents. Essential monitoring, logging, tracing and observability.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1893,1894,1897],{"name":17,"slug":18,"type":15},{"name":1895,"slug":1896,"type":15},"Monitoring","monitoring",{"name":1716,"slug":1717,"type":15},"2026-04-06T18:08:33.337476",{"slug":1900,"name":1900,"fn":1901,"description":1902,"org":1903,"tags":1904,"stars":28,"repoUrl":29,"updatedAt":1918},"datadog-app","build and manage Datadog applications","Guides developers building Datadog Apps with TypeScript, React, the @datadog\u002Fapps scaffolder, and @datadog\u002Fvite-plugin. Use when a user wants to scaffold, run, debug, upgrade, build, upload, publish, upload without publishing (draft upload), add an upload-no-publish script, set up CI\u002FCD, use OAuth or API\u002Fapplication key auth, trigger\u002Fpoll Workflow Automation, choose DDSQL or Action Catalog for backend data access, or query app datastores with DDSQL, including backend function troubleshooting.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1905,1906,1909,1912,1915],{"name":17,"slug":18,"type":15},{"name":1907,"slug":1908,"type":15},"Frontend","frontend",{"name":1910,"slug":1911,"type":15},"React","react",{"name":1913,"slug":1914,"type":15},"TypeScript","typescript",{"name":1916,"slug":1917,"type":15},"Vite","vite","2026-06-18T08:01:32.562331",{"slug":1920,"name":1920,"fn":1921,"description":1922,"org":1923,"tags":1924,"stars":28,"repoUrl":29,"updatedAt":1933},"dd-apm","query Datadog APM traces","APM - install, onboard, instrument, enable, set up, configure, traces, services, dependencies, performance analysis. Use for any request involving Datadog APM setup, instrumentation (SSI, ddtrace, agent install), or analysis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1925,1926,1929,1930],{"name":17,"slug":18,"type":15},{"name":1927,"slug":1928,"type":15},"Distributed Tracing","distributed-tracing",{"name":1716,"slug":1717,"type":15},{"name":1931,"slug":1932,"type":15},"Performance","performance","2026-04-06T18:08:34.575282",35]