[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-azure-agentops-config":3,"mdc--ajlunx-key":35,"related-repo-azure-agentops-config":1339,"related-org-azure-agentops-config":1430},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":12,"stars":24,"repoUrl":25,"updatedAt":26,"license":27,"forks":28,"topics":29,"repo":30,"sourceUrl":33,"mdContent":34},"agentops-config","configure AgentOps release-readiness gates","Generate or update agentops.yaml (flat 1.0 schema) for AgentOps release-readiness gates. Trigger on \"configure agentops\", \"agentops.yaml\", \"set up evaluation\", \"what should I evaluate\". Infer the agent target and dataset from the codebase; ask only when nothing can be found.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},"azure","Azure (Microsoft)","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fazure.png","Azure",[13,15,18,21],{"name":11,"slug":8,"type":14},"tag",{"name":16,"slug":17,"type":14},"Configuration","configuration",{"name":19,"slug":20,"type":14},"Evals","evals",{"name":22,"slug":23,"type":14},"Agents","agents",10,"https:\u002F\u002Fgithub.com\u002FAzure\u002Fagentops","2026-07-12T08:19:06.790378",null,7,[],{"repoUrl":25,"stars":24,"forks":28,"topics":31,"description":32},[],"AgentOps Accelerator is an open source framework and CLI for adding continuous evaluation and observability to enterprise AI agents. It standardizes evaluation patterns, automates assessments in CI\u002FCD workflows, and generates structured signals that help teams monitor, control, and safely operate agentic systems at scale.","https:\u002F\u002Fgithub.com\u002FAzure\u002Fagentops\u002Ftree\u002FHEAD\u002Fplugins\u002Fagentops\u002Fskills\u002Fagentops-config","---\nname: agentops-config\ndescription: Generate or update agentops.yaml (flat 1.0 schema) for AgentOps release-readiness gates. Trigger on \"configure agentops\", \"agentops.yaml\", \"set up evaluation\", \"what should I evaluate\". Infer the agent target and dataset from the codebase; ask only when nothing can be found.\n---\n\n# AgentOps Config\n\nGenerate `agentops.yaml` at the project root. This file references the agent\ncandidate and dataset used to answer \"can we ship it?\" The flat schema has only\na handful of fields - most projects need just `version`, `agent`, and\n`dataset`.\n\nThis skill configures AgentOps release gates. It does **not** create or deploy\nFoundry agents. If the user needs to create\u002Fdeploy a Prompt Agent or Hosted\nAgent first, hand off to Foundry Toolkit \u002F the `microsoft-foundry` skill \u002F azd,\nthen return here once there is a `name:version` or URL.\n\n## Step 0 - Prerequisites\n\n1. `pip install \"agentops-accelerator @ git+https:\u002F\u002Fgithub.com\u002FAzure\u002Fagentops.git@main\"` if `agentops` is missing.\n2. Run `agentops eval analyze` first. If it reports missing or ambiguous\n   target\u002Fdataset\u002Fscenario signals, use this skill to adapt the config.\n3. If `agentops.yaml` does not exist, run `agentops init` first. The init\n   wizard already collects the agent reference and dataset path, so\n   `agentops-config` is most useful when the user wants to **tweak** an\n   existing config (add thresholds, switch to a different agent target,\n   add HTTP auth headers, etc.) rather than create one from scratch.\n\n## Step 1 - Detect the agent target\n\nSearch the codebase for the strongest signal and pick one:\n\n| Signal | `agent:` value |\n|---|---|\n| Foundry Prompt Agent ID `name:N` | `\"\u003Cname>:\u003CN>\"` |\n| Foundry Hosted Agent URL `https:\u002F\u002F...services.ai.azure.com\u002F...agents\u002F...` | the full URL |\n| Any other HTTP endpoint your agent serves (FastAPI, Express, ACA, AKS) | the full URL |\n| Direct model use (`openai.chat.completions.create(model=...)`) with no orchestration | `\"model:\u003Cdeployment-name>\"` |\n\nLook in: `README.md`, `main.py`\u002F`server.py`\u002F`app.ts`, `.agentops\u002F.env`,\n`.env`\u002F`.env.local`, `.azure\u002F\u003Cenv>\u002F.env`, `infra\u002F`, IaC outputs. If nothing is\nfound, ask the user once.\n\n## Step 2 - Detect the dataset\n\nIf a JSONL with rows that include `input` already exists in the repo, use\nits path. Otherwise leave the default `.agentops\u002Fdata\u002Fsmoke.jsonl` and\nhand off to the `agentops-dataset` skill before the first run.\n\n## Step 3 - Write agentops.yaml\n\nMinimal example:\n\n```yaml\nversion: 1\nagent: \"my-rag:3\"\ndataset: .agentops\u002Fdata\u002Fsmoke.jsonl\n```\n\nHTTP\u002FJSON example:\n\n```yaml\nversion: 1\nagent: \"https:\u002F\u002Fmy-aca-app.eastus2.azurecontainerapps.io\u002Fchat\"\ndataset: .agentops\u002Fdata\u002Fsmoke.jsonl\nrequest_field: message      # default is \"message\"\nresponse_field: text         # dot-path; default is \"text\"\nauth_header_env: MY_API_TOKEN\n```\n\nOptional extras (only add when the user asks for them):\n\n```yaml\nthresholds:\n  coherence: \">=3\"\n  groundedness: \">=3\"\n  avg_latency_seconds: \"\u003C=30\"\n\n# Prompt-agent only: auto-bootstrap empty Foundry projects on first deploy.\n# When the deploy workflow runs against a Foundry project that does not yet\n# contain the agent named in `agent:`, AgentOps reads this block plus\n# `prompt_file` and creates the first version automatically. Recommended\n# for multi-environment prompt-agent workflows (sandbox → dev → qa → prod)\n# so operators do not have to manually recreate the seed agent in every\n# Foundry project.\nprompt_agent_bootstrap:\n  model: gpt-4o-mini        # required - same deployment name in every env\n  description: \"Helps plan short trips.\"\n  # model_parameters:        # optional - temperature, top_p, etc.\n  #   temperature: 0.2\n  # tools: []                # optional - tool definitions\n\n# Publish results to the Foundry Evaluations panel.\n# - execution: local + publish: true  → Classic Foundry (uploads metrics)\n# - execution: cloud                  → New Foundry (server-side run;\n#                                       publish is implicit, cloud always publishes)\nexecution: local\npublish: true\n# project_endpoint: \"https:\u002F\u002F\u003Cresource>.services.ai.azure.com\u002Fapi\u002Fprojects\u002F\u003Cp>\"\n\n# Cloud dataset submission policy. The local JSONL remains the source of truth;\n# cloud runs sync it to Foundry Data\u002FDatasets by default.\ndataset_sync:\n  mode: auto            # auto | foundry | inline\n  # name: agentops-smoke\n  # version: content-hash\n\nevaluators:           # rare - AgentOps auto-selects from agent + dataset\n  - name: similarity\n    threshold: \">=4\"\n```\n\nGovernance evidence (optional): when the repo already contains ASSERT, ACS, or\nred-team evidence artifacts, wire them into Doctor\u002Frelease evidence without\nexecuting the external tools:\n\n```yaml\nassert_path: .assert\u002Fevaluation-policy.yaml\nacs_path: acs.yaml\nredteam_path: .agentops\u002Fgovernance\u002Fredteam-plan.md\n```\n\nIf the user needs help drafting these files, switch to the\n`agentops-governance` skill. AgentOps validates and hashes these artifacts; it\ndoes not execute ASSERT, apply ACS controls, or run red-team campaigns.\n\n## Step 4 - Validate\n\nRun `agentops eval run` once. If the config is malformed AgentOps prints a\nclear error pointing at the offending key. Adjust and re-run.\n\n## Guardrails\n\n- Do **not** add legacy keys (`bundle`, `target`, `execution`, `output`,\n  `backend`). The 1.0 schema rejects them.\n- Do **not** fabricate agent IDs, endpoint URLs, or model deployment\n  names. Ask the user when uncertain.\n- Keep the file small. Auto-selection covers most metrics.\n- Keep local JSONL canonical. For cloud runs, prefer `dataset_sync.mode: auto`\n  so AgentOps keeps Foundry Data\u002FDatasets in sync; use `inline` only for quick\n  experiments or environments without dataset upload permission.\n",{"data":36,"body":37},{"name":4,"description":6},{"type":38,"children":39},"root",[40,48,87,116,123,194,200,205,318,389,395,424,430,435,514,519,645,650,1147,1152,1210,1223,1229,1241,1247,1333],{"type":41,"tag":42,"props":43,"children":44},"element","h1",{"id":4},[45],{"type":46,"value":47},"text","AgentOps Config",{"type":41,"tag":49,"props":50,"children":51},"p",{},[52,54,61,63,69,71,77,79,85],{"type":46,"value":53},"Generate ",{"type":41,"tag":55,"props":56,"children":58},"code",{"className":57},[],[59],{"type":46,"value":60},"agentops.yaml",{"type":46,"value":62}," at the project root. This file references the agent\ncandidate and dataset used to answer \"can we ship it?\" The flat schema has only\na handful of fields - most projects need just ",{"type":41,"tag":55,"props":64,"children":66},{"className":65},[],[67],{"type":46,"value":68},"version",{"type":46,"value":70},", ",{"type":41,"tag":55,"props":72,"children":74},{"className":73},[],[75],{"type":46,"value":76},"agent",{"type":46,"value":78},", and\n",{"type":41,"tag":55,"props":80,"children":82},{"className":81},[],[83],{"type":46,"value":84},"dataset",{"type":46,"value":86},".",{"type":41,"tag":49,"props":88,"children":89},{},[90,92,98,100,106,108,114],{"type":46,"value":91},"This skill configures AgentOps release gates. It does ",{"type":41,"tag":93,"props":94,"children":95},"strong",{},[96],{"type":46,"value":97},"not",{"type":46,"value":99}," create or deploy\nFoundry agents. If the user needs to create\u002Fdeploy a Prompt Agent or Hosted\nAgent first, hand off to Foundry Toolkit \u002F the ",{"type":41,"tag":55,"props":101,"children":103},{"className":102},[],[104],{"type":46,"value":105},"microsoft-foundry",{"type":46,"value":107}," skill \u002F azd,\nthen return here once there is a ",{"type":41,"tag":55,"props":109,"children":111},{"className":110},[],[112],{"type":46,"value":113},"name:version",{"type":46,"value":115}," or URL.",{"type":41,"tag":117,"props":118,"children":120},"h2",{"id":119},"step-0-prerequisites",[121],{"type":46,"value":122},"Step 0 - Prerequisites",{"type":41,"tag":124,"props":125,"children":126},"ol",{},[127,147,160],{"type":41,"tag":128,"props":129,"children":130},"li",{},[131,137,139,145],{"type":41,"tag":55,"props":132,"children":134},{"className":133},[],[135],{"type":46,"value":136},"pip install \"agentops-accelerator @ git+https:\u002F\u002Fgithub.com\u002FAzure\u002Fagentops.git@main\"",{"type":46,"value":138}," if ",{"type":41,"tag":55,"props":140,"children":142},{"className":141},[],[143],{"type":46,"value":144},"agentops",{"type":46,"value":146}," is missing.",{"type":41,"tag":128,"props":148,"children":149},{},[150,152,158],{"type":46,"value":151},"Run ",{"type":41,"tag":55,"props":153,"children":155},{"className":154},[],[156],{"type":46,"value":157},"agentops eval analyze",{"type":46,"value":159}," first. If it reports missing or ambiguous\ntarget\u002Fdataset\u002Fscenario signals, use this skill to adapt the config.",{"type":41,"tag":128,"props":161,"children":162},{},[163,165,170,172,178,180,185,187,192],{"type":46,"value":164},"If ",{"type":41,"tag":55,"props":166,"children":168},{"className":167},[],[169],{"type":46,"value":60},{"type":46,"value":171}," does not exist, run ",{"type":41,"tag":55,"props":173,"children":175},{"className":174},[],[176],{"type":46,"value":177},"agentops init",{"type":46,"value":179}," first. The init\nwizard already collects the agent reference and dataset path, so\n",{"type":41,"tag":55,"props":181,"children":183},{"className":182},[],[184],{"type":46,"value":4},{"type":46,"value":186}," is most useful when the user wants to ",{"type":41,"tag":93,"props":188,"children":189},{},[190],{"type":46,"value":191},"tweak",{"type":46,"value":193}," an\nexisting config (add thresholds, switch to a different agent target,\nadd HTTP auth headers, etc.) rather than create one from scratch.",{"type":41,"tag":117,"props":195,"children":197},{"id":196},"step-1-detect-the-agent-target",[198],{"type":46,"value":199},"Step 1 - Detect the agent target",{"type":41,"tag":49,"props":201,"children":202},{},[203],{"type":46,"value":204},"Search the codebase for the strongest signal and pick one:",{"type":41,"tag":206,"props":207,"children":208},"table",{},[209,234],{"type":41,"tag":210,"props":211,"children":212},"thead",{},[213],{"type":41,"tag":214,"props":215,"children":216},"tr",{},[217,223],{"type":41,"tag":218,"props":219,"children":220},"th",{},[221],{"type":46,"value":222},"Signal",{"type":41,"tag":218,"props":224,"children":225},{},[226,232],{"type":41,"tag":55,"props":227,"children":229},{"className":228},[],[230],{"type":46,"value":231},"agent:",{"type":46,"value":233}," value",{"type":41,"tag":235,"props":236,"children":237},"tbody",{},[238,262,281,293],{"type":41,"tag":214,"props":239,"children":240},{},[241,253],{"type":41,"tag":242,"props":243,"children":244},"td",{},[245,247],{"type":46,"value":246},"Foundry Prompt Agent ID ",{"type":41,"tag":55,"props":248,"children":250},{"className":249},[],[251],{"type":46,"value":252},"name:N",{"type":41,"tag":242,"props":254,"children":255},{},[256],{"type":41,"tag":55,"props":257,"children":259},{"className":258},[],[260],{"type":46,"value":261},"\"\u003Cname>:\u003CN>\"",{"type":41,"tag":214,"props":263,"children":264},{},[265,276],{"type":41,"tag":242,"props":266,"children":267},{},[268,270],{"type":46,"value":269},"Foundry Hosted Agent URL ",{"type":41,"tag":55,"props":271,"children":273},{"className":272},[],[274],{"type":46,"value":275},"https:\u002F\u002F...services.ai.azure.com\u002F...agents\u002F...",{"type":41,"tag":242,"props":277,"children":278},{},[279],{"type":46,"value":280},"the full URL",{"type":41,"tag":214,"props":282,"children":283},{},[284,289],{"type":41,"tag":242,"props":285,"children":286},{},[287],{"type":46,"value":288},"Any other HTTP endpoint your agent serves (FastAPI, Express, ACA, AKS)",{"type":41,"tag":242,"props":290,"children":291},{},[292],{"type":46,"value":280},{"type":41,"tag":214,"props":294,"children":295},{},[296,309],{"type":41,"tag":242,"props":297,"children":298},{},[299,301,307],{"type":46,"value":300},"Direct model use (",{"type":41,"tag":55,"props":302,"children":304},{"className":303},[],[305],{"type":46,"value":306},"openai.chat.completions.create(model=...)",{"type":46,"value":308},") with no orchestration",{"type":41,"tag":242,"props":310,"children":311},{},[312],{"type":41,"tag":55,"props":313,"children":315},{"className":314},[],[316],{"type":46,"value":317},"\"model:\u003Cdeployment-name>\"",{"type":41,"tag":49,"props":319,"children":320},{},[321,323,329,330,336,338,344,345,351,352,358,360,366,367,373,374,380,381,387],{"type":46,"value":322},"Look in: ",{"type":41,"tag":55,"props":324,"children":326},{"className":325},[],[327],{"type":46,"value":328},"README.md",{"type":46,"value":70},{"type":41,"tag":55,"props":331,"children":333},{"className":332},[],[334],{"type":46,"value":335},"main.py",{"type":46,"value":337},"\u002F",{"type":41,"tag":55,"props":339,"children":341},{"className":340},[],[342],{"type":46,"value":343},"server.py",{"type":46,"value":337},{"type":41,"tag":55,"props":346,"children":348},{"className":347},[],[349],{"type":46,"value":350},"app.ts",{"type":46,"value":70},{"type":41,"tag":55,"props":353,"children":355},{"className":354},[],[356],{"type":46,"value":357},".agentops\u002F.env",{"type":46,"value":359},",\n",{"type":41,"tag":55,"props":361,"children":363},{"className":362},[],[364],{"type":46,"value":365},".env",{"type":46,"value":337},{"type":41,"tag":55,"props":368,"children":370},{"className":369},[],[371],{"type":46,"value":372},".env.local",{"type":46,"value":70},{"type":41,"tag":55,"props":375,"children":377},{"className":376},[],[378],{"type":46,"value":379},".azure\u002F\u003Cenv>\u002F.env",{"type":46,"value":70},{"type":41,"tag":55,"props":382,"children":384},{"className":383},[],[385],{"type":46,"value":386},"infra\u002F",{"type":46,"value":388},", IaC outputs. If nothing is\nfound, ask the user once.",{"type":41,"tag":117,"props":390,"children":392},{"id":391},"step-2-detect-the-dataset",[393],{"type":46,"value":394},"Step 2 - Detect the dataset",{"type":41,"tag":49,"props":396,"children":397},{},[398,400,406,408,414,416,422],{"type":46,"value":399},"If a JSONL with rows that include ",{"type":41,"tag":55,"props":401,"children":403},{"className":402},[],[404],{"type":46,"value":405},"input",{"type":46,"value":407}," already exists in the repo, use\nits path. Otherwise leave the default ",{"type":41,"tag":55,"props":409,"children":411},{"className":410},[],[412],{"type":46,"value":413},".agentops\u002Fdata\u002Fsmoke.jsonl",{"type":46,"value":415}," and\nhand off to the ",{"type":41,"tag":55,"props":417,"children":419},{"className":418},[],[420],{"type":46,"value":421},"agentops-dataset",{"type":46,"value":423}," skill before the first run.",{"type":41,"tag":117,"props":425,"children":427},{"id":426},"step-3-write-agentopsyaml",[428],{"type":46,"value":429},"Step 3 - Write agentops.yaml",{"type":41,"tag":49,"props":431,"children":432},{},[433],{"type":46,"value":434},"Minimal example:",{"type":41,"tag":436,"props":437,"children":442},"pre",{"className":438,"code":439,"language":440,"meta":441,"style":441},"language-yaml shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","version: 1\nagent: \"my-rag:3\"\ndataset: .agentops\u002Fdata\u002Fsmoke.jsonl\n","yaml","",[443],{"type":41,"tag":55,"props":444,"children":445},{"__ignoreMap":441},[446,469,497],{"type":41,"tag":447,"props":448,"children":451},"span",{"class":449,"line":450},"line",1,[452,457,463],{"type":41,"tag":447,"props":453,"children":455},{"style":454},"--shiki-light:#E53935;--shiki-default:#F07178;--shiki-dark:#F07178",[456],{"type":46,"value":68},{"type":41,"tag":447,"props":458,"children":460},{"style":459},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[461],{"type":46,"value":462},":",{"type":41,"tag":447,"props":464,"children":466},{"style":465},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[467],{"type":46,"value":468}," 1\n",{"type":41,"tag":447,"props":470,"children":472},{"class":449,"line":471},2,[473,477,481,486,492],{"type":41,"tag":447,"props":474,"children":475},{"style":454},[476],{"type":46,"value":76},{"type":41,"tag":447,"props":478,"children":479},{"style":459},[480],{"type":46,"value":462},{"type":41,"tag":447,"props":482,"children":483},{"style":459},[484],{"type":46,"value":485}," \"",{"type":41,"tag":447,"props":487,"children":489},{"style":488},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[490],{"type":46,"value":491},"my-rag:3",{"type":41,"tag":447,"props":493,"children":494},{"style":459},[495],{"type":46,"value":496},"\"\n",{"type":41,"tag":447,"props":498,"children":500},{"class":449,"line":499},3,[501,505,509],{"type":41,"tag":447,"props":502,"children":503},{"style":454},[504],{"type":46,"value":84},{"type":41,"tag":447,"props":506,"children":507},{"style":459},[508],{"type":46,"value":462},{"type":41,"tag":447,"props":510,"children":511},{"style":488},[512],{"type":46,"value":513}," .agentops\u002Fdata\u002Fsmoke.jsonl\n",{"type":41,"tag":49,"props":515,"children":516},{},[517],{"type":46,"value":518},"HTTP\u002FJSON example:",{"type":41,"tag":436,"props":520,"children":522},{"className":438,"code":521,"language":440,"meta":441,"style":441},"version: 1\nagent: \"https:\u002F\u002Fmy-aca-app.eastus2.azurecontainerapps.io\u002Fchat\"\ndataset: .agentops\u002Fdata\u002Fsmoke.jsonl\nrequest_field: message      # default is \"message\"\nresponse_field: text         # dot-path; default is \"text\"\nauth_header_env: MY_API_TOKEN\n",[523],{"type":41,"tag":55,"props":524,"children":525},{"__ignoreMap":441},[526,541,565,580,604,627],{"type":41,"tag":447,"props":527,"children":528},{"class":449,"line":450},[529,533,537],{"type":41,"tag":447,"props":530,"children":531},{"style":454},[532],{"type":46,"value":68},{"type":41,"tag":447,"props":534,"children":535},{"style":459},[536],{"type":46,"value":462},{"type":41,"tag":447,"props":538,"children":539},{"style":465},[540],{"type":46,"value":468},{"type":41,"tag":447,"props":542,"children":543},{"class":449,"line":471},[544,548,552,556,561],{"type":41,"tag":447,"props":545,"children":546},{"style":454},[547],{"type":46,"value":76},{"type":41,"tag":447,"props":549,"children":550},{"style":459},[551],{"type":46,"value":462},{"type":41,"tag":447,"props":553,"children":554},{"style":459},[555],{"type":46,"value":485},{"type":41,"tag":447,"props":557,"children":558},{"style":488},[559],{"type":46,"value":560},"https:\u002F\u002Fmy-aca-app.eastus2.azurecontainerapps.io\u002Fchat",{"type":41,"tag":447,"props":562,"children":563},{"style":459},[564],{"type":46,"value":496},{"type":41,"tag":447,"props":566,"children":567},{"class":449,"line":499},[568,572,576],{"type":41,"tag":447,"props":569,"children":570},{"style":454},[571],{"type":46,"value":84},{"type":41,"tag":447,"props":573,"children":574},{"style":459},[575],{"type":46,"value":462},{"type":41,"tag":447,"props":577,"children":578},{"style":488},[579],{"type":46,"value":513},{"type":41,"tag":447,"props":581,"children":583},{"class":449,"line":582},4,[584,589,593,598],{"type":41,"tag":447,"props":585,"children":586},{"style":454},[587],{"type":46,"value":588},"request_field",{"type":41,"tag":447,"props":590,"children":591},{"style":459},[592],{"type":46,"value":462},{"type":41,"tag":447,"props":594,"children":595},{"style":488},[596],{"type":46,"value":597}," message",{"type":41,"tag":447,"props":599,"children":601},{"style":600},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[602],{"type":46,"value":603},"      # default is \"message\"\n",{"type":41,"tag":447,"props":605,"children":607},{"class":449,"line":606},5,[608,613,617,622],{"type":41,"tag":447,"props":609,"children":610},{"style":454},[611],{"type":46,"value":612},"response_field",{"type":41,"tag":447,"props":614,"children":615},{"style":459},[616],{"type":46,"value":462},{"type":41,"tag":447,"props":618,"children":619},{"style":488},[620],{"type":46,"value":621}," text",{"type":41,"tag":447,"props":623,"children":624},{"style":600},[625],{"type":46,"value":626},"         # dot-path; default is \"text\"\n",{"type":41,"tag":447,"props":628,"children":630},{"class":449,"line":629},6,[631,636,640],{"type":41,"tag":447,"props":632,"children":633},{"style":454},[634],{"type":46,"value":635},"auth_header_env",{"type":41,"tag":447,"props":637,"children":638},{"style":459},[639],{"type":46,"value":462},{"type":41,"tag":447,"props":641,"children":642},{"style":488},[643],{"type":46,"value":644}," MY_API_TOKEN\n",{"type":41,"tag":49,"props":646,"children":647},{},[648],{"type":46,"value":649},"Optional extras (only add when the user asks for them):",{"type":41,"tag":436,"props":651,"children":653},{"className":438,"code":652,"language":440,"meta":441,"style":441},"thresholds:\n  coherence: \">=3\"\n  groundedness: \">=3\"\n  avg_latency_seconds: \"\u003C=30\"\n\n# Prompt-agent only: auto-bootstrap empty Foundry projects on first deploy.\n# When the deploy workflow runs against a Foundry project that does not yet\n# contain the agent named in `agent:`, AgentOps reads this block plus\n# `prompt_file` and creates the first version automatically. Recommended\n# for multi-environment prompt-agent workflows (sandbox → dev → qa → prod)\n# so operators do not have to manually recreate the seed agent in every\n# Foundry project.\nprompt_agent_bootstrap:\n  model: gpt-4o-mini        # required - same deployment name in every env\n  description: \"Helps plan short trips.\"\n  # model_parameters:        # optional - temperature, top_p, etc.\n  #   temperature: 0.2\n  # tools: []                # optional - tool definitions\n\n# Publish results to the Foundry Evaluations panel.\n# - execution: local + publish: true  → Classic Foundry (uploads metrics)\n# - execution: cloud                  → New Foundry (server-side run;\n#                                       publish is implicit, cloud always publishes)\nexecution: local\npublish: true\n# project_endpoint: \"https:\u002F\u002F\u003Cresource>.services.ai.azure.com\u002Fapi\u002Fprojects\u002F\u003Cp>\"\n\n# Cloud dataset submission policy. The local JSONL remains the source of truth;\n# cloud runs sync it to Foundry Data\u002FDatasets by default.\ndataset_sync:\n  mode: auto            # auto | foundry | inline\n  # name: agentops-smoke\n  # version: content-hash\n\nevaluators:           # rare - AgentOps auto-selects from agent + dataset\n  - name: similarity\n    threshold: \">=4\"\n",[654],{"type":41,"tag":55,"props":655,"children":656},{"__ignoreMap":441},[657,670,695,719,744,753,761,769,778,787,795,804,813,826,849,875,884,893,902,910,919,928,937,946,964,983,992,1000,1009,1018,1031,1054,1063,1072,1080,1098,1121],{"type":41,"tag":447,"props":658,"children":659},{"class":449,"line":450},[660,665],{"type":41,"tag":447,"props":661,"children":662},{"style":454},[663],{"type":46,"value":664},"thresholds",{"type":41,"tag":447,"props":666,"children":667},{"style":459},[668],{"type":46,"value":669},":\n",{"type":41,"tag":447,"props":671,"children":672},{"class":449,"line":471},[673,678,682,686,691],{"type":41,"tag":447,"props":674,"children":675},{"style":454},[676],{"type":46,"value":677},"  coherence",{"type":41,"tag":447,"props":679,"children":680},{"style":459},[681],{"type":46,"value":462},{"type":41,"tag":447,"props":683,"children":684},{"style":459},[685],{"type":46,"value":485},{"type":41,"tag":447,"props":687,"children":688},{"style":488},[689],{"type":46,"value":690},">=3",{"type":41,"tag":447,"props":692,"children":693},{"style":459},[694],{"type":46,"value":496},{"type":41,"tag":447,"props":696,"children":697},{"class":449,"line":499},[698,703,707,711,715],{"type":41,"tag":447,"props":699,"children":700},{"style":454},[701],{"type":46,"value":702},"  groundedness",{"type":41,"tag":447,"props":704,"children":705},{"style":459},[706],{"type":46,"value":462},{"type":41,"tag":447,"props":708,"children":709},{"style":459},[710],{"type":46,"value":485},{"type":41,"tag":447,"props":712,"children":713},{"style":488},[714],{"type":46,"value":690},{"type":41,"tag":447,"props":716,"children":717},{"style":459},[718],{"type":46,"value":496},{"type":41,"tag":447,"props":720,"children":721},{"class":449,"line":582},[722,727,731,735,740],{"type":41,"tag":447,"props":723,"children":724},{"style":454},[725],{"type":46,"value":726},"  avg_latency_seconds",{"type":41,"tag":447,"props":728,"children":729},{"style":459},[730],{"type":46,"value":462},{"type":41,"tag":447,"props":732,"children":733},{"style":459},[734],{"type":46,"value":485},{"type":41,"tag":447,"props":736,"children":737},{"style":488},[738],{"type":46,"value":739},"\u003C=30",{"type":41,"tag":447,"props":741,"children":742},{"style":459},[743],{"type":46,"value":496},{"type":41,"tag":447,"props":745,"children":746},{"class":449,"line":606},[747],{"type":41,"tag":447,"props":748,"children":750},{"emptyLinePlaceholder":749},true,[751],{"type":46,"value":752},"\n",{"type":41,"tag":447,"props":754,"children":755},{"class":449,"line":629},[756],{"type":41,"tag":447,"props":757,"children":758},{"style":600},[759],{"type":46,"value":760},"# Prompt-agent only: auto-bootstrap empty Foundry projects on first deploy.\n",{"type":41,"tag":447,"props":762,"children":763},{"class":449,"line":28},[764],{"type":41,"tag":447,"props":765,"children":766},{"style":600},[767],{"type":46,"value":768},"# When the deploy workflow runs against a Foundry project that does not yet\n",{"type":41,"tag":447,"props":770,"children":772},{"class":449,"line":771},8,[773],{"type":41,"tag":447,"props":774,"children":775},{"style":600},[776],{"type":46,"value":777},"# contain the agent named in `agent:`, AgentOps reads this block plus\n",{"type":41,"tag":447,"props":779,"children":781},{"class":449,"line":780},9,[782],{"type":41,"tag":447,"props":783,"children":784},{"style":600},[785],{"type":46,"value":786},"# `prompt_file` and creates the first version automatically. Recommended\n",{"type":41,"tag":447,"props":788,"children":789},{"class":449,"line":24},[790],{"type":41,"tag":447,"props":791,"children":792},{"style":600},[793],{"type":46,"value":794},"# for multi-environment prompt-agent workflows (sandbox → dev → qa → prod)\n",{"type":41,"tag":447,"props":796,"children":798},{"class":449,"line":797},11,[799],{"type":41,"tag":447,"props":800,"children":801},{"style":600},[802],{"type":46,"value":803},"# so operators do not have to manually recreate the seed agent in every\n",{"type":41,"tag":447,"props":805,"children":807},{"class":449,"line":806},12,[808],{"type":41,"tag":447,"props":809,"children":810},{"style":600},[811],{"type":46,"value":812},"# Foundry project.\n",{"type":41,"tag":447,"props":814,"children":816},{"class":449,"line":815},13,[817,822],{"type":41,"tag":447,"props":818,"children":819},{"style":454},[820],{"type":46,"value":821},"prompt_agent_bootstrap",{"type":41,"tag":447,"props":823,"children":824},{"style":459},[825],{"type":46,"value":669},{"type":41,"tag":447,"props":827,"children":829},{"class":449,"line":828},14,[830,835,839,844],{"type":41,"tag":447,"props":831,"children":832},{"style":454},[833],{"type":46,"value":834},"  model",{"type":41,"tag":447,"props":836,"children":837},{"style":459},[838],{"type":46,"value":462},{"type":41,"tag":447,"props":840,"children":841},{"style":488},[842],{"type":46,"value":843}," gpt-4o-mini",{"type":41,"tag":447,"props":845,"children":846},{"style":600},[847],{"type":46,"value":848},"        # required - same deployment name in every env\n",{"type":41,"tag":447,"props":850,"children":852},{"class":449,"line":851},15,[853,858,862,866,871],{"type":41,"tag":447,"props":854,"children":855},{"style":454},[856],{"type":46,"value":857},"  description",{"type":41,"tag":447,"props":859,"children":860},{"style":459},[861],{"type":46,"value":462},{"type":41,"tag":447,"props":863,"children":864},{"style":459},[865],{"type":46,"value":485},{"type":41,"tag":447,"props":867,"children":868},{"style":488},[869],{"type":46,"value":870},"Helps plan short trips.",{"type":41,"tag":447,"props":872,"children":873},{"style":459},[874],{"type":46,"value":496},{"type":41,"tag":447,"props":876,"children":878},{"class":449,"line":877},16,[879],{"type":41,"tag":447,"props":880,"children":881},{"style":600},[882],{"type":46,"value":883},"  # model_parameters:        # optional - temperature, top_p, etc.\n",{"type":41,"tag":447,"props":885,"children":887},{"class":449,"line":886},17,[888],{"type":41,"tag":447,"props":889,"children":890},{"style":600},[891],{"type":46,"value":892},"  #   temperature: 0.2\n",{"type":41,"tag":447,"props":894,"children":896},{"class":449,"line":895},18,[897],{"type":41,"tag":447,"props":898,"children":899},{"style":600},[900],{"type":46,"value":901},"  # tools: []                # optional - tool definitions\n",{"type":41,"tag":447,"props":903,"children":905},{"class":449,"line":904},19,[906],{"type":41,"tag":447,"props":907,"children":908},{"emptyLinePlaceholder":749},[909],{"type":46,"value":752},{"type":41,"tag":447,"props":911,"children":913},{"class":449,"line":912},20,[914],{"type":41,"tag":447,"props":915,"children":916},{"style":600},[917],{"type":46,"value":918},"# Publish results to the Foundry Evaluations panel.\n",{"type":41,"tag":447,"props":920,"children":922},{"class":449,"line":921},21,[923],{"type":41,"tag":447,"props":924,"children":925},{"style":600},[926],{"type":46,"value":927},"# - execution: local + publish: true  → Classic Foundry (uploads metrics)\n",{"type":41,"tag":447,"props":929,"children":931},{"class":449,"line":930},22,[932],{"type":41,"tag":447,"props":933,"children":934},{"style":600},[935],{"type":46,"value":936},"# - execution: cloud                  → New Foundry (server-side run;\n",{"type":41,"tag":447,"props":938,"children":940},{"class":449,"line":939},23,[941],{"type":41,"tag":447,"props":942,"children":943},{"style":600},[944],{"type":46,"value":945},"#                                       publish is implicit, cloud always publishes)\n",{"type":41,"tag":447,"props":947,"children":949},{"class":449,"line":948},24,[950,955,959],{"type":41,"tag":447,"props":951,"children":952},{"style":454},[953],{"type":46,"value":954},"execution",{"type":41,"tag":447,"props":956,"children":957},{"style":459},[958],{"type":46,"value":462},{"type":41,"tag":447,"props":960,"children":961},{"style":488},[962],{"type":46,"value":963}," local\n",{"type":41,"tag":447,"props":965,"children":967},{"class":449,"line":966},25,[968,973,977],{"type":41,"tag":447,"props":969,"children":970},{"style":454},[971],{"type":46,"value":972},"publish",{"type":41,"tag":447,"props":974,"children":975},{"style":459},[976],{"type":46,"value":462},{"type":41,"tag":447,"props":978,"children":980},{"style":979},"--shiki-light:#FF5370;--shiki-default:#FF9CAC;--shiki-dark:#FF9CAC",[981],{"type":46,"value":982}," true\n",{"type":41,"tag":447,"props":984,"children":986},{"class":449,"line":985},26,[987],{"type":41,"tag":447,"props":988,"children":989},{"style":600},[990],{"type":46,"value":991},"# project_endpoint: \"https:\u002F\u002F\u003Cresource>.services.ai.azure.com\u002Fapi\u002Fprojects\u002F\u003Cp>\"\n",{"type":41,"tag":447,"props":993,"children":995},{"class":449,"line":994},27,[996],{"type":41,"tag":447,"props":997,"children":998},{"emptyLinePlaceholder":749},[999],{"type":46,"value":752},{"type":41,"tag":447,"props":1001,"children":1003},{"class":449,"line":1002},28,[1004],{"type":41,"tag":447,"props":1005,"children":1006},{"style":600},[1007],{"type":46,"value":1008},"# Cloud dataset submission policy. The local JSONL remains the source of truth;\n",{"type":41,"tag":447,"props":1010,"children":1012},{"class":449,"line":1011},29,[1013],{"type":41,"tag":447,"props":1014,"children":1015},{"style":600},[1016],{"type":46,"value":1017},"# cloud runs sync it to Foundry Data\u002FDatasets by default.\n",{"type":41,"tag":447,"props":1019,"children":1021},{"class":449,"line":1020},30,[1022,1027],{"type":41,"tag":447,"props":1023,"children":1024},{"style":454},[1025],{"type":46,"value":1026},"dataset_sync",{"type":41,"tag":447,"props":1028,"children":1029},{"style":459},[1030],{"type":46,"value":669},{"type":41,"tag":447,"props":1032,"children":1034},{"class":449,"line":1033},31,[1035,1040,1044,1049],{"type":41,"tag":447,"props":1036,"children":1037},{"style":454},[1038],{"type":46,"value":1039},"  mode",{"type":41,"tag":447,"props":1041,"children":1042},{"style":459},[1043],{"type":46,"value":462},{"type":41,"tag":447,"props":1045,"children":1046},{"style":488},[1047],{"type":46,"value":1048}," auto",{"type":41,"tag":447,"props":1050,"children":1051},{"style":600},[1052],{"type":46,"value":1053},"            # auto | foundry | inline\n",{"type":41,"tag":447,"props":1055,"children":1057},{"class":449,"line":1056},32,[1058],{"type":41,"tag":447,"props":1059,"children":1060},{"style":600},[1061],{"type":46,"value":1062},"  # name: agentops-smoke\n",{"type":41,"tag":447,"props":1064,"children":1066},{"class":449,"line":1065},33,[1067],{"type":41,"tag":447,"props":1068,"children":1069},{"style":600},[1070],{"type":46,"value":1071},"  # version: content-hash\n",{"type":41,"tag":447,"props":1073,"children":1075},{"class":449,"line":1074},34,[1076],{"type":41,"tag":447,"props":1077,"children":1078},{"emptyLinePlaceholder":749},[1079],{"type":46,"value":752},{"type":41,"tag":447,"props":1081,"children":1083},{"class":449,"line":1082},35,[1084,1089,1093],{"type":41,"tag":447,"props":1085,"children":1086},{"style":454},[1087],{"type":46,"value":1088},"evaluators",{"type":41,"tag":447,"props":1090,"children":1091},{"style":459},[1092],{"type":46,"value":462},{"type":41,"tag":447,"props":1094,"children":1095},{"style":600},[1096],{"type":46,"value":1097},"           # rare - AgentOps auto-selects from agent + dataset\n",{"type":41,"tag":447,"props":1099,"children":1101},{"class":449,"line":1100},36,[1102,1107,1112,1116],{"type":41,"tag":447,"props":1103,"children":1104},{"style":459},[1105],{"type":46,"value":1106},"  -",{"type":41,"tag":447,"props":1108,"children":1109},{"style":454},[1110],{"type":46,"value":1111}," name",{"type":41,"tag":447,"props":1113,"children":1114},{"style":459},[1115],{"type":46,"value":462},{"type":41,"tag":447,"props":1117,"children":1118},{"style":488},[1119],{"type":46,"value":1120}," similarity\n",{"type":41,"tag":447,"props":1122,"children":1124},{"class":449,"line":1123},37,[1125,1130,1134,1138,1143],{"type":41,"tag":447,"props":1126,"children":1127},{"style":454},[1128],{"type":46,"value":1129},"    threshold",{"type":41,"tag":447,"props":1131,"children":1132},{"style":459},[1133],{"type":46,"value":462},{"type":41,"tag":447,"props":1135,"children":1136},{"style":459},[1137],{"type":46,"value":485},{"type":41,"tag":447,"props":1139,"children":1140},{"style":488},[1141],{"type":46,"value":1142},">=4",{"type":41,"tag":447,"props":1144,"children":1145},{"style":459},[1146],{"type":46,"value":496},{"type":41,"tag":49,"props":1148,"children":1149},{},[1150],{"type":46,"value":1151},"Governance evidence (optional): when the repo already contains ASSERT, ACS, or\nred-team evidence artifacts, wire them into Doctor\u002Frelease evidence without\nexecuting the external tools:",{"type":41,"tag":436,"props":1153,"children":1155},{"className":438,"code":1154,"language":440,"meta":441,"style":441},"assert_path: .assert\u002Fevaluation-policy.yaml\nacs_path: acs.yaml\nredteam_path: .agentops\u002Fgovernance\u002Fredteam-plan.md\n",[1156],{"type":41,"tag":55,"props":1157,"children":1158},{"__ignoreMap":441},[1159,1176,1193],{"type":41,"tag":447,"props":1160,"children":1161},{"class":449,"line":450},[1162,1167,1171],{"type":41,"tag":447,"props":1163,"children":1164},{"style":454},[1165],{"type":46,"value":1166},"assert_path",{"type":41,"tag":447,"props":1168,"children":1169},{"style":459},[1170],{"type":46,"value":462},{"type":41,"tag":447,"props":1172,"children":1173},{"style":488},[1174],{"type":46,"value":1175}," .assert\u002Fevaluation-policy.yaml\n",{"type":41,"tag":447,"props":1177,"children":1178},{"class":449,"line":471},[1179,1184,1188],{"type":41,"tag":447,"props":1180,"children":1181},{"style":454},[1182],{"type":46,"value":1183},"acs_path",{"type":41,"tag":447,"props":1185,"children":1186},{"style":459},[1187],{"type":46,"value":462},{"type":41,"tag":447,"props":1189,"children":1190},{"style":488},[1191],{"type":46,"value":1192}," acs.yaml\n",{"type":41,"tag":447,"props":1194,"children":1195},{"class":449,"line":499},[1196,1201,1205],{"type":41,"tag":447,"props":1197,"children":1198},{"style":454},[1199],{"type":46,"value":1200},"redteam_path",{"type":41,"tag":447,"props":1202,"children":1203},{"style":459},[1204],{"type":46,"value":462},{"type":41,"tag":447,"props":1206,"children":1207},{"style":488},[1208],{"type":46,"value":1209}," .agentops\u002Fgovernance\u002Fredteam-plan.md\n",{"type":41,"tag":49,"props":1211,"children":1212},{},[1213,1215,1221],{"type":46,"value":1214},"If the user needs help drafting these files, switch to the\n",{"type":41,"tag":55,"props":1216,"children":1218},{"className":1217},[],[1219],{"type":46,"value":1220},"agentops-governance",{"type":46,"value":1222}," skill. AgentOps validates and hashes these artifacts; it\ndoes not execute ASSERT, apply ACS controls, or run red-team campaigns.",{"type":41,"tag":117,"props":1224,"children":1226},{"id":1225},"step-4-validate",[1227],{"type":46,"value":1228},"Step 4 - Validate",{"type":41,"tag":49,"props":1230,"children":1231},{},[1232,1233,1239],{"type":46,"value":151},{"type":41,"tag":55,"props":1234,"children":1236},{"className":1235},[],[1237],{"type":46,"value":1238},"agentops eval run",{"type":46,"value":1240}," once. If the config is malformed AgentOps prints a\nclear error pointing at the offending key. Adjust and re-run.",{"type":41,"tag":117,"props":1242,"children":1244},{"id":1243},"guardrails",[1245],{"type":46,"value":1246},"Guardrails",{"type":41,"tag":1248,"props":1249,"children":1250},"ul",{},[1251,1297,1307,1312],{"type":41,"tag":128,"props":1252,"children":1253},{},[1254,1256,1260,1262,1268,1269,1275,1276,1281,1282,1288,1289,1295],{"type":46,"value":1255},"Do ",{"type":41,"tag":93,"props":1257,"children":1258},{},[1259],{"type":46,"value":97},{"type":46,"value":1261}," add legacy keys (",{"type":41,"tag":55,"props":1263,"children":1265},{"className":1264},[],[1266],{"type":46,"value":1267},"bundle",{"type":46,"value":70},{"type":41,"tag":55,"props":1270,"children":1272},{"className":1271},[],[1273],{"type":46,"value":1274},"target",{"type":46,"value":70},{"type":41,"tag":55,"props":1277,"children":1279},{"className":1278},[],[1280],{"type":46,"value":954},{"type":46,"value":70},{"type":41,"tag":55,"props":1283,"children":1285},{"className":1284},[],[1286],{"type":46,"value":1287},"output",{"type":46,"value":359},{"type":41,"tag":55,"props":1290,"children":1292},{"className":1291},[],[1293],{"type":46,"value":1294},"backend",{"type":46,"value":1296},"). The 1.0 schema rejects them.",{"type":41,"tag":128,"props":1298,"children":1299},{},[1300,1301,1305],{"type":46,"value":1255},{"type":41,"tag":93,"props":1302,"children":1303},{},[1304],{"type":46,"value":97},{"type":46,"value":1306}," fabricate agent IDs, endpoint URLs, or model deployment\nnames. Ask the user when uncertain.",{"type":41,"tag":128,"props":1308,"children":1309},{},[1310],{"type":46,"value":1311},"Keep the file small. Auto-selection covers most metrics.",{"type":41,"tag":128,"props":1313,"children":1314},{},[1315,1317,1323,1325,1331],{"type":46,"value":1316},"Keep local JSONL canonical. For cloud runs, prefer ",{"type":41,"tag":55,"props":1318,"children":1320},{"className":1319},[],[1321],{"type":46,"value":1322},"dataset_sync.mode: auto",{"type":46,"value":1324},"\nso AgentOps keeps Foundry Data\u002FDatasets in sync; use ",{"type":41,"tag":55,"props":1326,"children":1328},{"className":1327},[],[1329],{"type":46,"value":1330},"inline",{"type":46,"value":1332}," only for quick\nexperiments or environments without dataset upload permission.",{"type":41,"tag":1334,"props":1335,"children":1336},"style",{},[1337],{"type":46,"value":1338},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":1340,"total":28},[1341,1357,1364,1376,1386,1400,1413],{"slug":1342,"name":1342,"fn":1343,"description":1344,"org":1345,"tags":1346,"stars":24,"repoUrl":25,"updatedAt":1356},"agentops-agent","diagnose agent performance with AgentOps","AgentOps Doctor - surface release-readiness findings, regressions, latency spikes, error rates, and safety hits across AgentOps eval history, Azure Monitor traces, and Foundry control plane.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1347,1348,1349,1352,1353],{"name":22,"slug":23,"type":14},{"name":11,"slug":8,"type":14},{"name":1350,"slug":1351,"type":14},"Debugging","debugging",{"name":19,"slug":20,"type":14},{"name":1354,"slug":1355,"type":14},"Observability","observability","2026-07-12T08:19:05.564606",{"slug":4,"name":4,"fn":5,"description":6,"org":1358,"tags":1359,"stars":24,"repoUrl":25,"updatedAt":26},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1360,1361,1362,1363],{"name":22,"slug":23,"type":14},{"name":11,"slug":8,"type":14},{"name":16,"slug":17,"type":14},{"name":19,"slug":20,"type":14},{"slug":421,"name":421,"fn":1365,"description":1366,"org":1367,"tags":1368,"stars":24,"repoUrl":25,"updatedAt":1375},"create datasets for AgentOps release readiness","Create or extend a small JSONL dataset for AgentOps release-readiness gates. Trigger on \"create dataset\", \"generate test data\", \"JSONL\", \"more eval rows\". Infer the agent's domain from the codebase and produce realistic rows; never fabricate data when the domain is unclear.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1369,1370,1371,1374],{"name":22,"slug":23,"type":14},{"name":11,"slug":8,"type":14},{"name":1372,"slug":1373,"type":14},"Datasets","datasets",{"name":19,"slug":20,"type":14},"2026-07-12T08:19:08.00408",{"slug":1377,"name":1377,"fn":1378,"description":1379,"org":1380,"tags":1381,"stars":24,"repoUrl":25,"updatedAt":1385},"agentops-eval","run release-readiness evaluations for agents","Run AgentOps release-readiness evaluations against Foundry prompt agents, Foundry hosted endpoints, HTTP\u002FJSON agents, or raw model deployments. Trigger on phrases like \"run eval\", \"evaluate my agent\", \"benchmark\", \"agentops eval\", \"compare runs\", \"can we ship\". Uses the flat agentops.yaml schema.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1382,1383,1384],{"name":22,"slug":23,"type":14},{"name":11,"slug":8,"type":14},{"name":19,"slug":20,"type":14},"2026-07-15T06:02:50.097169",{"slug":1220,"name":1220,"fn":1387,"description":1388,"org":1389,"tags":1390,"stars":24,"repoUrl":25,"updatedAt":1399},"draft governance evidence for AI agents","Scaffold ASSERT and Red Team runners for the release gate, and draft reviewable governance evidence for ASSERT, Agent Control Specification (ACS), Guided Guardrail readiness, and red-team planning. Trigger on \"ASSERT\", \"ACS\", \"agent control\", \"guardrail\", \"red team\", \"governance\", \"release evidence\", \"scaffold assert\", \"set up red team\", \"add safety gate\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1391,1392,1393,1396],{"name":22,"slug":23,"type":14},{"name":11,"slug":8,"type":14},{"name":1394,"slug":1395,"type":14},"Compliance","compliance",{"name":1397,"slug":1398,"type":14},"Governance","governance","2026-07-12T08:19:04.026045",{"slug":1401,"name":1401,"fn":1402,"description":1403,"org":1404,"tags":1405,"stars":24,"repoUrl":25,"updatedAt":1412},"agentops-report","generate and explain AgentOps release-gate reports","Read, regenerate, and explain AgentOps release-gate reports. Trigger on \"show report\", \"explain scores\", \"regenerate report\", \"what do these metrics mean\", \"where is the proof\". Operates on results.json and report.md produced by `agentops eval run`.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1406,1407,1408,1409],{"name":22,"slug":23,"type":14},{"name":11,"slug":8,"type":14},{"name":1354,"slug":1355,"type":14},{"name":1410,"slug":1411,"type":14},"Reporting","reporting","2026-07-12T08:19:09.241071",{"slug":1414,"name":1414,"fn":1415,"description":1416,"org":1417,"tags":1418,"stars":24,"repoUrl":25,"updatedAt":1429},"agentops-workflow","configure AgentOps release readiness workflows","Set up AgentOps release-readiness workflows: PR eval gates, Doctor\u002Fevidence artifacts, and safe deploy handoffs to azd or Foundry prompt-agent tooling. Trigger on CI, CD, pipeline, workflow, GitHub Actions, Azure DevOps, ADO, PR gate, deploy, environments, GitFlow, release branch, promote to prod, DevOps, can we ship.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1419,1420,1423,1426],{"name":11,"slug":8,"type":14},{"name":1421,"slug":1422,"type":14},"CI\u002FCD","ci-cd",{"name":1424,"slug":1425,"type":14},"Deployment","deployment",{"name":1427,"slug":1428,"type":14},"GitHub Actions","github-actions","2026-07-15T06:02:51.371749",{"items":1431,"total":1600},[1432,1447,1464,1483,1498,1513,1524,1539,1550,1564,1575,1588],{"slug":1433,"name":1433,"fn":1434,"description":1435,"org":1436,"tags":1437,"stars":1444,"repoUrl":1445,"updatedAt":1446},"azure-arg-external-evaluation-policy-author","author and test Azure Resource Graph policies","Use when the user wants to author, design, or test an Azure Policy that queries Azure Resource Graph (ARG) at request-time — i.e. a policy whose deny\u002Faudit decision depends on data from elsewhere in the subscription (sibling\u002Fparent resource state, RG-wide invariants, multi-hop relationships, etc.). Formally called Azure Policy External Evaluation; sometimes referred to colloquially as \"Invoke\". Drives an iterative KQL co-design loop against the user's real subscription via `az graph query`, then emits a policy definition, assignment, `.http` test flow, and an `EXPLANATION.md` companion. Read-only; never provisions anything.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1438,1439,1440,1441],{"name":11,"slug":8,"type":14},{"name":1394,"slug":1395,"type":14},{"name":1397,"slug":1398,"type":14},{"name":1442,"slug":1443,"type":14},"Policy","policy",1686,"https:\u002F\u002Fgithub.com\u002FAzure\u002Fazure-policy","2026-07-12T08:17:48.378432",{"slug":1448,"name":1448,"fn":1449,"description":1450,"org":1451,"tags":1452,"stars":1461,"repoUrl":1462,"updatedAt":1463},"azure-blueprints-migration","migrate Azure Blueprints to Template Specs","Use when a user needs to migrate off Azure Blueprints (definitions and\u002For assignments) to Template Specs and Deployment Stacks before the January 31, 2027 retirement. Covers inventory, export, conversion to Bicep, policy decoupling, Template Spec publishing, Deployment Stack deployment with deny-settings, validation, and cutover.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1453,1454,1455,1458],{"name":11,"slug":8,"type":14},{"name":1424,"slug":1425,"type":14},{"name":1456,"slug":1457,"type":14},"Infrastructure as Code","infrastructure-as-code",{"name":1459,"slug":1460,"type":14},"Migration","migration",260,"https:\u002F\u002Fgithub.com\u002FAzure\u002Fazure-blueprints","2026-07-12T08:17:49.646405",{"slug":1465,"name":1465,"fn":1466,"description":1467,"org":1468,"tags":1469,"stars":1480,"repoUrl":1481,"updatedAt":1482},"apiview-feedback-resolution","resolve APIView feedback on Azure SDKs","Analyze and resolve APIView review feedback on Azure SDK PRs. **UTILITY SKILL**. USE FOR: APIView comments, API review feedback, SDK API surface changes. DO NOT USE FOR: general code review, non-APIView feedback. INVOKES: azure-sdk-mcp:azsdk_apiview_get_comments, azure-sdk-mcp:azsdk_typespec_customized_code_update.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1470,1473,1474,1477],{"name":1471,"slug":1472,"type":14},"API Development","api-development",{"name":11,"slug":8,"type":14},{"name":1475,"slug":1476,"type":14},"Code Review","code-review",{"name":1478,"slug":1479,"type":14},"Documentation","documentation",133,"https:\u002F\u002Fgithub.com\u002FAzure\u002Fazure-sdk-tools","2026-07-12T08:17:43.350876",{"slug":1484,"name":1484,"fn":1485,"description":1486,"org":1487,"tags":1488,"stars":1480,"repoUrl":1481,"updatedAt":1497},"azsdk-common-live-and-recorded-tests","deploy resources and run Azure SDK tests","Deploy test resources and run Azure SDK tests in live, record, or playback mode. WHEN: \"run live tests\", \"run recorded tests\", \"deploy test resources\", \"record tests\", \"run tests in record mode\", \"clean up test resources\", \"run tests against live resources\". DO NOT USE FOR: writing new tests, authoring Bicep templates, playback-only test runs without resource deployment. INVOKES: azure-sdk-mcp:azsdk_package_run_tests.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1489,1490,1491,1494],{"name":11,"slug":8,"type":14},{"name":1424,"slug":1425,"type":14},{"name":1492,"slug":1493,"type":14},"SDK","sdk",{"name":1495,"slug":1496,"type":14},"Testing","testing","2026-07-12T08:17:44.718943",{"slug":1499,"name":1499,"fn":1500,"description":1501,"org":1502,"tags":1503,"stars":1480,"repoUrl":1481,"updatedAt":1512},"azsdk-common-prepare-release-plan","manage Azure SDK release plan work items","Create, get, update, abandon, and link SDK PRs to release plan work items for Azure SDK releases. **UTILITY SKILL**. USE FOR: \"create release plan\", \"get release plan\", \"update release plan\", \"update API spec in release plan\", \"update SDK details in release plan\", \"abandon release plan\", \"link SDK PR to plan\", \"namespace approval\", \"check release plan status\". DO NOT USE FOR: SDK code generation, pipeline troubleshooting, API review feedback. INVOKES: azure-sdk-mcp:azsdk_create_release_plan, azure-sdk-mcp:azsdk_get_release_plan, azure-sdk-mcp:azsdk_get_release_plan_for_spec_pr, azure-sdk-mcp:azsdk_update_release_plan, azure-sdk-mcp:azsdk_update_api_spec_pull_request_in_release_plan, azure-sdk-mcp:azsdk_update_sdk_details_in_release_plan, azure-sdk-mcp:azsdk_abandon_release_plan, azure-sdk-mcp:azsdk_link_sdk_pull_request_to_release_plan, azure-sdk-mcp:azsdk_link_namespace_approval_issue.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1504,1505,1508,1511],{"name":11,"slug":8,"type":14},{"name":1506,"slug":1507,"type":14},"GitHub","github",{"name":1509,"slug":1510,"type":14},"Project Management","project-management",{"name":1492,"slug":1493,"type":14},"2026-07-12T08:17:38.345387",{"slug":1514,"name":1514,"fn":1515,"description":1516,"org":1517,"tags":1518,"stars":1480,"repoUrl":1481,"updatedAt":1523},"azsdk-common-sdk-release","release Azure SDK packages","Check release readiness and trigger the release pipeline for Azure SDK packages. **UTILITY SKILL**. USE FOR: \"release SDK\", \"trigger release\", \"check release readiness\", \"release pipeline\", \"publish package\", \"ship SDK\". DO NOT USE FOR: SDK development, code generation, pipeline debugging, release plan creation. INVOKES: azure-sdk-mcp:azsdk_release_sdk.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1519,1520,1521,1522],{"name":11,"slug":8,"type":14},{"name":1421,"slug":1422,"type":14},{"name":1424,"slug":1425,"type":14},{"name":1492,"slug":1493,"type":14},"2026-07-12T08:17:34.27607",{"slug":1525,"name":1525,"fn":1526,"description":1527,"org":1528,"tags":1529,"stars":1480,"repoUrl":1481,"updatedAt":1538},"azure-typespec-author","author and modify Azure TypeSpec API specifications","Authors and modifies Azure TypeSpec (.tsp) API specifications. USE FOR: any TypeSpec\u002Ftsp change — api versions (add, bump, preview, stable, promote), resources, operations, models, properties, decorators, visibility, constraints, breaking changes, LRO, suppressions, operationId, spread model. Covers ARM resource-manager and data-plane services. DO NOT USE FOR: SDK generation, releasing SDK packages, or single MCP tool calls. INVOKES: azure-sdk-mcp:azsdk_typespec_generate_authoring_plan, azure-sdk-mcp:azsdk_run_typespec_validation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1530,1531,1532,1535],{"name":1471,"slug":1472,"type":14},{"name":11,"slug":8,"type":14},{"name":1533,"slug":1534,"type":14},"OpenAPI","openapi",{"name":1536,"slug":1537,"type":14},"Technical Writing","technical-writing","2026-07-12T08:17:39.603232",{"slug":1540,"name":1540,"fn":1541,"description":1542,"org":1543,"tags":1544,"stars":1480,"repoUrl":1481,"updatedAt":1549},"generate-sdk-locally","generate and test Azure SDKs locally","Generate, build, and test Azure SDKs locally from TypeSpec with automatic customization. WHEN: \"generate SDK locally\", \"build SDK\", \"run SDK tests\", \"run CI checks\", \"validate package\", \"run checks\", \"update changelog\", \"fix SDK build errors\", \"fix breaking changes\", \"resolve SDK generation errors\", \"customize TypeSpec\", \"rename SDK client\", \"rename SDK model\", \"hide operation from SDK\", \"fix analyzer errors\", \"resolve customization drift\", \"create subclient\", \"update metadata\", \"update version\". DO NOT USE FOR: publishing to package registries, CI pipeline configuration, API design review. INVOKES: azsdk_verify_setup, azsdk_package_generate_code, azsdk_package_build_code, azsdk_package_run_check, azsdk_package_run_tests, azsdk_customized_code_update, azsdk_package_update_changelog_content, azsdk_package_update_metadata, azsdk_package_update_version.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1545,1546,1547,1548],{"name":11,"slug":8,"type":14},{"name":1421,"slug":1422,"type":14},{"name":1492,"slug":1493,"type":14},{"name":1495,"slug":1496,"type":14},"2026-07-12T08:17:37.08523",{"slug":1551,"name":1551,"fn":1552,"description":1553,"org":1554,"tags":1555,"stars":1480,"repoUrl":1481,"updatedAt":1563},"markdown-token-optimizer","optimize markdown files for token efficiency","Analyze markdown files for token efficiency and reduce context-window bloat. **UTILITY SKILL**. DO NOT USE FOR: code optimization, general file editing, non-markdown files. TRIGGERS: optimize markdown, reduce tokens, token count, token bloat, too many tokens, make concise, shrink file, file too large, optimize for AI, token efficiency, verbose markdown, reduce file size. INVOKES: waza CLI.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1556,1559,1562],{"name":1557,"slug":1558,"type":14},"LLM","llm",{"name":1560,"slug":1561,"type":14},"Performance","performance",{"name":1536,"slug":1537,"type":14},"2026-07-12T08:17:42.080413",{"slug":1565,"name":1565,"fn":1566,"description":1567,"org":1568,"tags":1569,"stars":1480,"repoUrl":1481,"updatedAt":1574},"pipeline-troubleshooting","troubleshoot Azure SDK CI pipelines","Diagnose and resolve failures in Azure SDK CI and generation pipelines. **UTILITY SKILL**. USE FOR: \"pipeline failed\", \"build failure\", \"CI check failing\", \"SDK generation error\", \"reproduce pipeline locally\", \"debug SDK pipeline\". DO NOT USE FOR: local build issues without pipeline context, API design review, SDK publishing. INVOKES: azure-sdk-mcp:azsdk_analyze_pipeline, azure-sdk-mcp:azsdk_package_build_code, azure-sdk-mcp:azsdk_package_run_check.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1570,1571,1572,1573],{"name":11,"slug":8,"type":14},{"name":1421,"slug":1422,"type":14},{"name":1350,"slug":1351,"type":14},{"name":1492,"slug":1493,"type":14},"2026-07-12T08:17:40.821512",{"slug":1576,"name":1576,"fn":1577,"description":1578,"org":1579,"tags":1580,"stars":1480,"repoUrl":1481,"updatedAt":1587},"sensei","improve skill frontmatter compliance","**WORKFLOW SKILL** — Iteratively improve skill frontmatter compliance using the Ralph loop pattern. WHEN: \"run sensei\", \"sensei help\", \"improve skill\", \"fix frontmatter\", \"skill compliance\", \"frontmatter audit\", \"score skill\", \"check skill tokens\". INVOKES: token counting tools, test runners, git commands. FOR SINGLE OPERATIONS: use token CLI directly for counts\u002Fchecks.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1581,1582,1583,1586],{"name":11,"slug":8,"type":14},{"name":1394,"slug":1395,"type":14},{"name":1584,"slug":1585,"type":14},"Process Optimization","process-optimization",{"name":1536,"slug":1537,"type":14},"2026-07-12T08:17:32.970921",{"slug":1589,"name":1589,"fn":1590,"description":1591,"org":1592,"tags":1593,"stars":1480,"repoUrl":1481,"updatedAt":1599},"skill-authoring","author agent skills for agentskills.io","Write Agent Skills that comply with the agentskills.io specification. WHEN: \"create a skill\", \"new skill\", \"write a skill\", \"skill template\", \"skill structure\", \"review skill\", \"skill PR\", \"skill compliance\", \"SKILL.md format\", \"skill frontmatter\", \"skill best practices\". DO NOT USE FOR: improving existing skills (use sensei), general documentation. INVOKES: waza CLI.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1594,1595,1598],{"name":1478,"slug":1479,"type":14},{"name":1596,"slug":1597,"type":14},"Plugin Development","plugin-development",{"name":1536,"slug":1537,"type":14},"2026-07-12T08:17:35.873862",109]