[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-azure-agentops-dataset":3,"mdc-g9hhj4-key":35,"related-repo-azure-agentops-dataset":694,"related-org-azure-agentops-dataset":787},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":12,"stars":24,"repoUrl":25,"updatedAt":26,"license":27,"forks":28,"topics":29,"repo":30,"sourceUrl":33,"mdContent":34},"agentops-dataset","create datasets for AgentOps release readiness","Create or extend a small JSONL dataset for AgentOps release-readiness gates. Trigger on \"create dataset\", \"generate test data\", \"JSONL\", \"more eval rows\". Infer the agent's domain from the codebase and produce realistic rows; never fabricate data when the domain is unclear.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},"azure","Azure (Microsoft)","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fazure.png","Azure",[13,15,18,21],{"name":11,"slug":8,"type":14},"tag",{"name":16,"slug":17,"type":14},"Datasets","datasets",{"name":19,"slug":20,"type":14},"Evals","evals",{"name":22,"slug":23,"type":14},"Agents","agents",10,"https:\u002F\u002Fgithub.com\u002FAzure\u002Fagentops","2026-07-12T08:19:08.00408",null,7,[],{"repoUrl":25,"stars":24,"forks":28,"topics":31,"description":32},[],"AgentOps Accelerator is an open source framework and CLI for adding continuous evaluation and observability to enterprise AI agents. It standardizes evaluation patterns, automates assessments in CI\u002FCD workflows, and generates structured signals that help teams monitor, control, and safely operate agentic systems at scale.","https:\u002F\u002Fgithub.com\u002FAzure\u002Fagentops\u002Ftree\u002FHEAD\u002Fplugins\u002Fagentops\u002Fskills\u002Fagentops-dataset","---\nname: agentops-dataset\ndescription: Create or extend a small JSONL dataset for AgentOps release-readiness gates. Trigger on \"create dataset\", \"generate test data\", \"JSONL\", \"more eval rows\". Infer the agent's domain from the codebase and produce realistic rows; never fabricate data when the domain is unclear.\n---\n\n# AgentOps Dataset\n\nGenerate a small, realistic JSONL dataset for the agent under evaluation.\nDefault location: `.agentops\u002Fdata\u002Fsmoke.jsonl` (referenced from\n`agentops.yaml`). These rows are repo-side release-gate inputs: keep them\nreviewable and deterministic, not a full replacement for Foundry dataset\nmanagement.\n\n## Step 0 - Prerequisites\n\n1. `pip install \"agentops-accelerator @ git+https:\u002F\u002Fgithub.com\u002FAzure\u002Fagentops.git@main\"` if `agentops` is missing.\n2. Run `agentops eval analyze` first. If it reports missing dataset columns or\n   recommends `agentops-dataset`, use this skill before the first eval run.\n3. If `agentops.yaml` does not exist, run `agentops init` first (the init\n   wizard will prompt for the agent reference, project endpoint, and\n   dataset path, then create a starter `.agentops\u002Fdata\u002Fsmoke.jsonl`).\n\n## Step 1 - Pick the columns\n\nRead `agentops.yaml` (and the agent code) to figure out the agent type,\nthen choose the row schema:\n\n| Agent type | Required columns | Optional columns |\n|---|---|---|\n| Direct model \u002F Q&A | `input`, `expected` | - |\n| RAG | `input`, `expected`, `context` | - |\n| Conversational | `input`, `expected` | - |\n| Tool-using agent | `input`, `expected`, `tool_calls` | `tool_definitions` |\n\n`input` is always the user prompt. `expected` is the gold answer.\n`context` is the retrieved passage(s). `tool_calls` is a list of\n`{name, arguments}` describing the expected tool invocations.\n\n## Step 2 - Ground the rows in the codebase\n\n- Read the README, system prompt, tool definitions, and any sample\n  fixtures.\n- Generate **5–10 rows** that exercise the agent's actual capabilities.\n- If the domain is unclear, generate a tiny generic draft and clearly\n  flag it as a placeholder.\n\n## Step 3 - Write the JSONL\n\nOne JSON object per line, no trailing commas, UTF-8:\n\n```json\n{\"input\": \"What is the refund policy?\", \"expected\": \"Refunds within 30 days...\", \"context\": \"Refund policy: ...\"}\n```\n\nSave to the path referenced by `dataset:` in `agentops.yaml` (default\n`.agentops\u002Fdata\u002Fsmoke.jsonl`).\n\nThis file is the AgentOps source of truth. In Foundry cloud evaluation,\nAgentOps syncs it to a stable Foundry dataset version by default and reuses the\nsame Foundry dataset version while the JSONL content is unchanged. If the user\nforces `dataset_sync.mode: inline`, Foundry may show generated `eval-data-*`\nbacking assets in the project Data\u002FDatasets page.\n\n## Step 4 - Sanity-check\n\nRun a quick eval and confirm rows are picked up:\n\n```bash\nagentops eval run\n```\n\nOpen `.agentops\u002Fresults\u002Flatest\u002Freport.md` and confirm the row count\nmatches.\n\n## Guardrails\n\n- Do not invent customer data, real names, or sensitive content.\n- Keep rows short - datasets are meant to be quick gates, not full QA\n  suites.\n- If the user already has a domain dataset, prefer pointing\n  `agentops.yaml` at that file rather than generating new rows.\n- If the user asks why Foundry shows `eval-data-*`, explain that those are\n  cloud-eval backing assets from inline compatibility mode; normal cloud runs\n  should use the stable `agentops-*` Foundry dataset.\n",{"data":36,"body":37},{"name":4,"description":6},{"type":38,"children":39},"root",[40,48,71,78,149,155,167,327,366,372,399,405,410,539,565,586,592,597,624,637,643,688],{"type":41,"tag":42,"props":43,"children":44},"element","h1",{"id":4},[45],{"type":46,"value":47},"text","AgentOps Dataset",{"type":41,"tag":49,"props":50,"children":51},"p",{},[52,54,61,63,69],{"type":46,"value":53},"Generate a small, realistic JSONL dataset for the agent under evaluation.\nDefault location: ",{"type":41,"tag":55,"props":56,"children":58},"code",{"className":57},[],[59],{"type":46,"value":60},".agentops\u002Fdata\u002Fsmoke.jsonl",{"type":46,"value":62}," (referenced from\n",{"type":41,"tag":55,"props":64,"children":66},{"className":65},[],[67],{"type":46,"value":68},"agentops.yaml",{"type":46,"value":70},"). These rows are repo-side release-gate inputs: keep them\nreviewable and deterministic, not a full replacement for Foundry dataset\nmanagement.",{"type":41,"tag":72,"props":73,"children":75},"h2",{"id":74},"step-0-prerequisites",[76],{"type":46,"value":77},"Step 0 - Prerequisites",{"type":41,"tag":79,"props":80,"children":81},"ol",{},[82,102,122],{"type":41,"tag":83,"props":84,"children":85},"li",{},[86,92,94,100],{"type":41,"tag":55,"props":87,"children":89},{"className":88},[],[90],{"type":46,"value":91},"pip install \"agentops-accelerator @ git+https:\u002F\u002Fgithub.com\u002FAzure\u002Fagentops.git@main\"",{"type":46,"value":93}," if ",{"type":41,"tag":55,"props":95,"children":97},{"className":96},[],[98],{"type":46,"value":99},"agentops",{"type":46,"value":101}," is missing.",{"type":41,"tag":83,"props":103,"children":104},{},[105,107,113,115,120],{"type":46,"value":106},"Run ",{"type":41,"tag":55,"props":108,"children":110},{"className":109},[],[111],{"type":46,"value":112},"agentops eval analyze",{"type":46,"value":114}," first. If it reports missing dataset columns or\nrecommends ",{"type":41,"tag":55,"props":116,"children":118},{"className":117},[],[119],{"type":46,"value":4},{"type":46,"value":121},", use this skill before the first eval run.",{"type":41,"tag":83,"props":123,"children":124},{},[125,127,132,134,140,142,147],{"type":46,"value":126},"If ",{"type":41,"tag":55,"props":128,"children":130},{"className":129},[],[131],{"type":46,"value":68},{"type":46,"value":133}," does not exist, run ",{"type":41,"tag":55,"props":135,"children":137},{"className":136},[],[138],{"type":46,"value":139},"agentops init",{"type":46,"value":141}," first (the init\nwizard will prompt for the agent reference, project endpoint, and\ndataset path, then create a starter ",{"type":41,"tag":55,"props":143,"children":145},{"className":144},[],[146],{"type":46,"value":60},{"type":46,"value":148},").",{"type":41,"tag":72,"props":150,"children":152},{"id":151},"step-1-pick-the-columns",[153],{"type":46,"value":154},"Step 1 - Pick the columns",{"type":41,"tag":49,"props":156,"children":157},{},[158,160,165],{"type":46,"value":159},"Read ",{"type":41,"tag":55,"props":161,"children":163},{"className":162},[],[164],{"type":46,"value":68},{"type":46,"value":166}," (and the agent code) to figure out the agent type,\nthen choose the row schema:",{"type":41,"tag":168,"props":169,"children":170},"table",{},[171,195],{"type":41,"tag":172,"props":173,"children":174},"thead",{},[175],{"type":41,"tag":176,"props":177,"children":178},"tr",{},[179,185,190],{"type":41,"tag":180,"props":181,"children":182},"th",{},[183],{"type":46,"value":184},"Agent type",{"type":41,"tag":180,"props":186,"children":187},{},[188],{"type":46,"value":189},"Required columns",{"type":41,"tag":180,"props":191,"children":192},{},[193],{"type":46,"value":194},"Optional columns",{"type":41,"tag":196,"props":197,"children":198},"tbody",{},[199,230,263,289],{"type":41,"tag":176,"props":200,"children":201},{},[202,208,225],{"type":41,"tag":203,"props":204,"children":205},"td",{},[206],{"type":46,"value":207},"Direct model \u002F Q&A",{"type":41,"tag":203,"props":209,"children":210},{},[211,217,219],{"type":41,"tag":55,"props":212,"children":214},{"className":213},[],[215],{"type":46,"value":216},"input",{"type":46,"value":218},", ",{"type":41,"tag":55,"props":220,"children":222},{"className":221},[],[223],{"type":46,"value":224},"expected",{"type":41,"tag":203,"props":226,"children":227},{},[228],{"type":46,"value":229},"-",{"type":41,"tag":176,"props":231,"children":232},{},[233,238,259],{"type":41,"tag":203,"props":234,"children":235},{},[236],{"type":46,"value":237},"RAG",{"type":41,"tag":203,"props":239,"children":240},{},[241,246,247,252,253],{"type":41,"tag":55,"props":242,"children":244},{"className":243},[],[245],{"type":46,"value":216},{"type":46,"value":218},{"type":41,"tag":55,"props":248,"children":250},{"className":249},[],[251],{"type":46,"value":224},{"type":46,"value":218},{"type":41,"tag":55,"props":254,"children":256},{"className":255},[],[257],{"type":46,"value":258},"context",{"type":41,"tag":203,"props":260,"children":261},{},[262],{"type":46,"value":229},{"type":41,"tag":176,"props":264,"children":265},{},[266,271,285],{"type":41,"tag":203,"props":267,"children":268},{},[269],{"type":46,"value":270},"Conversational",{"type":41,"tag":203,"props":272,"children":273},{},[274,279,280],{"type":41,"tag":55,"props":275,"children":277},{"className":276},[],[278],{"type":46,"value":216},{"type":46,"value":218},{"type":41,"tag":55,"props":281,"children":283},{"className":282},[],[284],{"type":46,"value":224},{"type":41,"tag":203,"props":286,"children":287},{},[288],{"type":46,"value":229},{"type":41,"tag":176,"props":290,"children":291},{},[292,297,318],{"type":41,"tag":203,"props":293,"children":294},{},[295],{"type":46,"value":296},"Tool-using agent",{"type":41,"tag":203,"props":298,"children":299},{},[300,305,306,311,312],{"type":41,"tag":55,"props":301,"children":303},{"className":302},[],[304],{"type":46,"value":216},{"type":46,"value":218},{"type":41,"tag":55,"props":307,"children":309},{"className":308},[],[310],{"type":46,"value":224},{"type":46,"value":218},{"type":41,"tag":55,"props":313,"children":315},{"className":314},[],[316],{"type":46,"value":317},"tool_calls",{"type":41,"tag":203,"props":319,"children":320},{},[321],{"type":41,"tag":55,"props":322,"children":324},{"className":323},[],[325],{"type":46,"value":326},"tool_definitions",{"type":41,"tag":49,"props":328,"children":329},{},[330,335,337,342,344,349,351,356,358,364],{"type":41,"tag":55,"props":331,"children":333},{"className":332},[],[334],{"type":46,"value":216},{"type":46,"value":336}," is always the user prompt. ",{"type":41,"tag":55,"props":338,"children":340},{"className":339},[],[341],{"type":46,"value":224},{"type":46,"value":343}," is the gold answer.\n",{"type":41,"tag":55,"props":345,"children":347},{"className":346},[],[348],{"type":46,"value":258},{"type":46,"value":350}," is the retrieved passage(s). ",{"type":41,"tag":55,"props":352,"children":354},{"className":353},[],[355],{"type":46,"value":317},{"type":46,"value":357}," is a list of\n",{"type":41,"tag":55,"props":359,"children":361},{"className":360},[],[362],{"type":46,"value":363},"{name, arguments}",{"type":46,"value":365}," describing the expected tool invocations.",{"type":41,"tag":72,"props":367,"children":369},{"id":368},"step-2-ground-the-rows-in-the-codebase",[370],{"type":46,"value":371},"Step 2 - Ground the rows in the codebase",{"type":41,"tag":373,"props":374,"children":375},"ul",{},[376,381,394],{"type":41,"tag":83,"props":377,"children":378},{},[379],{"type":46,"value":380},"Read the README, system prompt, tool definitions, and any sample\nfixtures.",{"type":41,"tag":83,"props":382,"children":383},{},[384,386,392],{"type":46,"value":385},"Generate ",{"type":41,"tag":387,"props":388,"children":389},"strong",{},[390],{"type":46,"value":391},"5–10 rows",{"type":46,"value":393}," that exercise the agent's actual capabilities.",{"type":41,"tag":83,"props":395,"children":396},{},[397],{"type":46,"value":398},"If the domain is unclear, generate a tiny generic draft and clearly\nflag it as a placeholder.",{"type":41,"tag":72,"props":400,"children":402},{"id":401},"step-3-write-the-jsonl",[403],{"type":46,"value":404},"Step 3 - Write the JSONL",{"type":41,"tag":49,"props":406,"children":407},{},[408],{"type":46,"value":409},"One JSON object per line, no trailing commas, UTF-8:",{"type":41,"tag":411,"props":412,"children":417},"pre",{"className":413,"code":414,"language":415,"meta":416,"style":416},"language-json shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","{\"input\": \"What is the refund policy?\", \"expected\": \"Refunds within 30 days...\", \"context\": \"Refund policy: ...\"}\n","json","",[418],{"type":41,"tag":55,"props":419,"children":420},{"__ignoreMap":416},[421],{"type":41,"tag":422,"props":423,"children":426},"span",{"class":424,"line":425},"line",1,[427,433,438,443,447,452,457,463,467,472,476,480,484,488,492,497,501,505,509,513,517,521,525,530,534],{"type":41,"tag":422,"props":428,"children":430},{"style":429},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[431],{"type":46,"value":432},"{",{"type":41,"tag":422,"props":434,"children":435},{"style":429},[436],{"type":46,"value":437},"\"",{"type":41,"tag":422,"props":439,"children":441},{"style":440},"--shiki-light:#9C3EDA;--shiki-default:#C792EA;--shiki-dark:#C792EA",[442],{"type":46,"value":216},{"type":41,"tag":422,"props":444,"children":445},{"style":429},[446],{"type":46,"value":437},{"type":41,"tag":422,"props":448,"children":449},{"style":429},[450],{"type":46,"value":451},":",{"type":41,"tag":422,"props":453,"children":454},{"style":429},[455],{"type":46,"value":456}," \"",{"type":41,"tag":422,"props":458,"children":460},{"style":459},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[461],{"type":46,"value":462},"What is the refund policy?",{"type":41,"tag":422,"props":464,"children":465},{"style":429},[466],{"type":46,"value":437},{"type":41,"tag":422,"props":468,"children":469},{"style":429},[470],{"type":46,"value":471},",",{"type":41,"tag":422,"props":473,"children":474},{"style":429},[475],{"type":46,"value":456},{"type":41,"tag":422,"props":477,"children":478},{"style":440},[479],{"type":46,"value":224},{"type":41,"tag":422,"props":481,"children":482},{"style":429},[483],{"type":46,"value":437},{"type":41,"tag":422,"props":485,"children":486},{"style":429},[487],{"type":46,"value":451},{"type":41,"tag":422,"props":489,"children":490},{"style":429},[491],{"type":46,"value":456},{"type":41,"tag":422,"props":493,"children":494},{"style":459},[495],{"type":46,"value":496},"Refunds within 30 days...",{"type":41,"tag":422,"props":498,"children":499},{"style":429},[500],{"type":46,"value":437},{"type":41,"tag":422,"props":502,"children":503},{"style":429},[504],{"type":46,"value":471},{"type":41,"tag":422,"props":506,"children":507},{"style":429},[508],{"type":46,"value":456},{"type":41,"tag":422,"props":510,"children":511},{"style":440},[512],{"type":46,"value":258},{"type":41,"tag":422,"props":514,"children":515},{"style":429},[516],{"type":46,"value":437},{"type":41,"tag":422,"props":518,"children":519},{"style":429},[520],{"type":46,"value":451},{"type":41,"tag":422,"props":522,"children":523},{"style":429},[524],{"type":46,"value":456},{"type":41,"tag":422,"props":526,"children":527},{"style":459},[528],{"type":46,"value":529},"Refund policy: ...",{"type":41,"tag":422,"props":531,"children":532},{"style":429},[533],{"type":46,"value":437},{"type":41,"tag":422,"props":535,"children":536},{"style":429},[537],{"type":46,"value":538},"}\n",{"type":41,"tag":49,"props":540,"children":541},{},[542,544,550,552,557,559,564],{"type":46,"value":543},"Save to the path referenced by ",{"type":41,"tag":55,"props":545,"children":547},{"className":546},[],[548],{"type":46,"value":549},"dataset:",{"type":46,"value":551}," in ",{"type":41,"tag":55,"props":553,"children":555},{"className":554},[],[556],{"type":46,"value":68},{"type":46,"value":558}," (default\n",{"type":41,"tag":55,"props":560,"children":562},{"className":561},[],[563],{"type":46,"value":60},{"type":46,"value":148},{"type":41,"tag":49,"props":566,"children":567},{},[568,570,576,578,584],{"type":46,"value":569},"This file is the AgentOps source of truth. In Foundry cloud evaluation,\nAgentOps syncs it to a stable Foundry dataset version by default and reuses the\nsame Foundry dataset version while the JSONL content is unchanged. If the user\nforces ",{"type":41,"tag":55,"props":571,"children":573},{"className":572},[],[574],{"type":46,"value":575},"dataset_sync.mode: inline",{"type":46,"value":577},", Foundry may show generated ",{"type":41,"tag":55,"props":579,"children":581},{"className":580},[],[582],{"type":46,"value":583},"eval-data-*",{"type":46,"value":585},"\nbacking assets in the project Data\u002FDatasets page.",{"type":41,"tag":72,"props":587,"children":589},{"id":588},"step-4-sanity-check",[590],{"type":46,"value":591},"Step 4 - Sanity-check",{"type":41,"tag":49,"props":593,"children":594},{},[595],{"type":46,"value":596},"Run a quick eval and confirm rows are picked up:",{"type":41,"tag":411,"props":598,"children":602},{"className":599,"code":600,"language":601,"meta":416,"style":416},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","agentops eval run\n","bash",[603],{"type":41,"tag":55,"props":604,"children":605},{"__ignoreMap":416},[606],{"type":41,"tag":422,"props":607,"children":608},{"class":424,"line":425},[609,614,619],{"type":41,"tag":422,"props":610,"children":612},{"style":611},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[613],{"type":46,"value":99},{"type":41,"tag":422,"props":615,"children":616},{"style":459},[617],{"type":46,"value":618}," eval",{"type":41,"tag":422,"props":620,"children":621},{"style":459},[622],{"type":46,"value":623}," run\n",{"type":41,"tag":49,"props":625,"children":626},{},[627,629,635],{"type":46,"value":628},"Open ",{"type":41,"tag":55,"props":630,"children":632},{"className":631},[],[633],{"type":46,"value":634},".agentops\u002Fresults\u002Flatest\u002Freport.md",{"type":46,"value":636}," and confirm the row count\nmatches.",{"type":41,"tag":72,"props":638,"children":640},{"id":639},"guardrails",[641],{"type":46,"value":642},"Guardrails",{"type":41,"tag":373,"props":644,"children":645},{},[646,651,656,668],{"type":41,"tag":83,"props":647,"children":648},{},[649],{"type":46,"value":650},"Do not invent customer data, real names, or sensitive content.",{"type":41,"tag":83,"props":652,"children":653},{},[654],{"type":46,"value":655},"Keep rows short - datasets are meant to be quick gates, not full QA\nsuites.",{"type":41,"tag":83,"props":657,"children":658},{},[659,661,666],{"type":46,"value":660},"If the user already has a domain dataset, prefer pointing\n",{"type":41,"tag":55,"props":662,"children":664},{"className":663},[],[665],{"type":46,"value":68},{"type":46,"value":667}," at that file rather than generating new rows.",{"type":41,"tag":83,"props":669,"children":670},{},[671,673,678,680,686],{"type":46,"value":672},"If the user asks why Foundry shows ",{"type":41,"tag":55,"props":674,"children":676},{"className":675},[],[677],{"type":46,"value":583},{"type":46,"value":679},", explain that those are\ncloud-eval backing assets from inline compatibility mode; normal cloud runs\nshould use the stable ",{"type":41,"tag":55,"props":681,"children":683},{"className":682},[],[684],{"type":46,"value":685},"agentops-*",{"type":46,"value":687}," Foundry dataset.",{"type":41,"tag":689,"props":690,"children":691},"style",{},[692],{"type":46,"value":693},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":695,"total":28},[696,712,725,732,742,757,770],{"slug":697,"name":697,"fn":698,"description":699,"org":700,"tags":701,"stars":24,"repoUrl":25,"updatedAt":711},"agentops-agent","diagnose agent performance with AgentOps","AgentOps Doctor - surface release-readiness findings, regressions, latency spikes, error rates, and safety hits across AgentOps eval history, Azure Monitor traces, and Foundry control plane.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[702,703,704,707,708],{"name":22,"slug":23,"type":14},{"name":11,"slug":8,"type":14},{"name":705,"slug":706,"type":14},"Debugging","debugging",{"name":19,"slug":20,"type":14},{"name":709,"slug":710,"type":14},"Observability","observability","2026-07-12T08:19:05.564606",{"slug":713,"name":713,"fn":714,"description":715,"org":716,"tags":717,"stars":24,"repoUrl":25,"updatedAt":724},"agentops-config","configure AgentOps release-readiness gates","Generate or update agentops.yaml (flat 1.0 schema) for AgentOps release-readiness gates. Trigger on \"configure agentops\", \"agentops.yaml\", \"set up evaluation\", \"what should I evaluate\". Infer the agent target and dataset from the codebase; ask only when nothing can be found.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[718,719,720,723],{"name":22,"slug":23,"type":14},{"name":11,"slug":8,"type":14},{"name":721,"slug":722,"type":14},"Configuration","configuration",{"name":19,"slug":20,"type":14},"2026-07-12T08:19:06.790378",{"slug":4,"name":4,"fn":5,"description":6,"org":726,"tags":727,"stars":24,"repoUrl":25,"updatedAt":26},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[728,729,730,731],{"name":22,"slug":23,"type":14},{"name":11,"slug":8,"type":14},{"name":16,"slug":17,"type":14},{"name":19,"slug":20,"type":14},{"slug":733,"name":733,"fn":734,"description":735,"org":736,"tags":737,"stars":24,"repoUrl":25,"updatedAt":741},"agentops-eval","run release-readiness evaluations for agents","Run AgentOps release-readiness evaluations against Foundry prompt agents, Foundry hosted endpoints, HTTP\u002FJSON agents, or raw model deployments. Trigger on phrases like \"run eval\", \"evaluate my agent\", \"benchmark\", \"agentops eval\", \"compare runs\", \"can we ship\". Uses the flat agentops.yaml schema.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[738,739,740],{"name":22,"slug":23,"type":14},{"name":11,"slug":8,"type":14},{"name":19,"slug":20,"type":14},"2026-07-15T06:02:50.097169",{"slug":743,"name":743,"fn":744,"description":745,"org":746,"tags":747,"stars":24,"repoUrl":25,"updatedAt":756},"agentops-governance","draft governance evidence for AI agents","Scaffold ASSERT and Red Team runners for the release gate, and draft reviewable governance evidence for ASSERT, Agent Control Specification (ACS), Guided Guardrail readiness, and red-team planning. Trigger on \"ASSERT\", \"ACS\", \"agent control\", \"guardrail\", \"red team\", \"governance\", \"release evidence\", \"scaffold assert\", \"set up red team\", \"add safety gate\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[748,749,750,753],{"name":22,"slug":23,"type":14},{"name":11,"slug":8,"type":14},{"name":751,"slug":752,"type":14},"Compliance","compliance",{"name":754,"slug":755,"type":14},"Governance","governance","2026-07-12T08:19:04.026045",{"slug":758,"name":758,"fn":759,"description":760,"org":761,"tags":762,"stars":24,"repoUrl":25,"updatedAt":769},"agentops-report","generate and explain AgentOps release-gate reports","Read, regenerate, and explain AgentOps release-gate reports. Trigger on \"show report\", \"explain scores\", \"regenerate report\", \"what do these metrics mean\", \"where is the proof\". Operates on results.json and report.md produced by `agentops eval run`.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[763,764,765,766],{"name":22,"slug":23,"type":14},{"name":11,"slug":8,"type":14},{"name":709,"slug":710,"type":14},{"name":767,"slug":768,"type":14},"Reporting","reporting","2026-07-12T08:19:09.241071",{"slug":771,"name":771,"fn":772,"description":773,"org":774,"tags":775,"stars":24,"repoUrl":25,"updatedAt":786},"agentops-workflow","configure AgentOps release readiness workflows","Set up AgentOps release-readiness workflows: PR eval gates, Doctor\u002Fevidence artifacts, and safe deploy handoffs to azd or Foundry prompt-agent tooling. Trigger on CI, CD, pipeline, workflow, GitHub Actions, Azure DevOps, ADO, PR gate, deploy, environments, GitFlow, release branch, promote to prod, DevOps, can we ship.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[776,777,780,783],{"name":11,"slug":8,"type":14},{"name":778,"slug":779,"type":14},"CI\u002FCD","ci-cd",{"name":781,"slug":782,"type":14},"Deployment","deployment",{"name":784,"slug":785,"type":14},"GitHub Actions","github-actions","2026-07-15T06:02:51.371749",{"items":788,"total":957},[789,804,821,840,855,870,881,896,907,921,932,945],{"slug":790,"name":790,"fn":791,"description":792,"org":793,"tags":794,"stars":801,"repoUrl":802,"updatedAt":803},"azure-arg-external-evaluation-policy-author","author and test Azure Resource Graph policies","Use when the user wants to author, design, or test an Azure Policy that queries Azure Resource Graph (ARG) at request-time — i.e. a policy whose deny\u002Faudit decision depends on data from elsewhere in the subscription (sibling\u002Fparent resource state, RG-wide invariants, multi-hop relationships, etc.). Formally called Azure Policy External Evaluation; sometimes referred to colloquially as \"Invoke\". Drives an iterative KQL co-design loop against the user's real subscription via `az graph query`, then emits a policy definition, assignment, `.http` test flow, and an `EXPLANATION.md` companion. Read-only; never provisions anything.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[795,796,797,798],{"name":11,"slug":8,"type":14},{"name":751,"slug":752,"type":14},{"name":754,"slug":755,"type":14},{"name":799,"slug":800,"type":14},"Policy","policy",1686,"https:\u002F\u002Fgithub.com\u002FAzure\u002Fazure-policy","2026-07-12T08:17:48.378432",{"slug":805,"name":805,"fn":806,"description":807,"org":808,"tags":809,"stars":818,"repoUrl":819,"updatedAt":820},"azure-blueprints-migration","migrate Azure Blueprints to Template Specs","Use when a user needs to migrate off Azure Blueprints (definitions and\u002For assignments) to Template Specs and Deployment Stacks before the January 31, 2027 retirement. Covers inventory, export, conversion to Bicep, policy decoupling, Template Spec publishing, Deployment Stack deployment with deny-settings, validation, and cutover.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[810,811,812,815],{"name":11,"slug":8,"type":14},{"name":781,"slug":782,"type":14},{"name":813,"slug":814,"type":14},"Infrastructure as Code","infrastructure-as-code",{"name":816,"slug":817,"type":14},"Migration","migration",260,"https:\u002F\u002Fgithub.com\u002FAzure\u002Fazure-blueprints","2026-07-12T08:17:49.646405",{"slug":822,"name":822,"fn":823,"description":824,"org":825,"tags":826,"stars":837,"repoUrl":838,"updatedAt":839},"apiview-feedback-resolution","resolve APIView feedback on Azure SDKs","Analyze and resolve APIView review feedback on Azure SDK PRs. **UTILITY SKILL**. USE FOR: APIView comments, API review feedback, SDK API surface changes. DO NOT USE FOR: general code review, non-APIView feedback. INVOKES: azure-sdk-mcp:azsdk_apiview_get_comments, azure-sdk-mcp:azsdk_typespec_customized_code_update.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[827,830,831,834],{"name":828,"slug":829,"type":14},"API Development","api-development",{"name":11,"slug":8,"type":14},{"name":832,"slug":833,"type":14},"Code Review","code-review",{"name":835,"slug":836,"type":14},"Documentation","documentation",133,"https:\u002F\u002Fgithub.com\u002FAzure\u002Fazure-sdk-tools","2026-07-12T08:17:43.350876",{"slug":841,"name":841,"fn":842,"description":843,"org":844,"tags":845,"stars":837,"repoUrl":838,"updatedAt":854},"azsdk-common-live-and-recorded-tests","deploy resources and run Azure SDK tests","Deploy test resources and run Azure SDK tests in live, record, or playback mode. WHEN: \"run live tests\", \"run recorded tests\", \"deploy test resources\", \"record tests\", \"run tests in record mode\", \"clean up test resources\", \"run tests against live resources\". DO NOT USE FOR: writing new tests, authoring Bicep templates, playback-only test runs without resource deployment. INVOKES: azure-sdk-mcp:azsdk_package_run_tests.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[846,847,848,851],{"name":11,"slug":8,"type":14},{"name":781,"slug":782,"type":14},{"name":849,"slug":850,"type":14},"SDK","sdk",{"name":852,"slug":853,"type":14},"Testing","testing","2026-07-12T08:17:44.718943",{"slug":856,"name":856,"fn":857,"description":858,"org":859,"tags":860,"stars":837,"repoUrl":838,"updatedAt":869},"azsdk-common-prepare-release-plan","manage Azure SDK release plan work items","Create, get, update, abandon, and link SDK PRs to release plan work items for Azure SDK releases. **UTILITY SKILL**. USE FOR: \"create release plan\", \"get release plan\", \"update release plan\", \"update API spec in release plan\", \"update SDK details in release plan\", \"abandon release plan\", \"link SDK PR to plan\", \"namespace approval\", \"check release plan status\". DO NOT USE FOR: SDK code generation, pipeline troubleshooting, API review feedback. INVOKES: azure-sdk-mcp:azsdk_create_release_plan, azure-sdk-mcp:azsdk_get_release_plan, azure-sdk-mcp:azsdk_get_release_plan_for_spec_pr, azure-sdk-mcp:azsdk_update_release_plan, azure-sdk-mcp:azsdk_update_api_spec_pull_request_in_release_plan, azure-sdk-mcp:azsdk_update_sdk_details_in_release_plan, azure-sdk-mcp:azsdk_abandon_release_plan, azure-sdk-mcp:azsdk_link_sdk_pull_request_to_release_plan, azure-sdk-mcp:azsdk_link_namespace_approval_issue.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[861,862,865,868],{"name":11,"slug":8,"type":14},{"name":863,"slug":864,"type":14},"GitHub","github",{"name":866,"slug":867,"type":14},"Project Management","project-management",{"name":849,"slug":850,"type":14},"2026-07-12T08:17:38.345387",{"slug":871,"name":871,"fn":872,"description":873,"org":874,"tags":875,"stars":837,"repoUrl":838,"updatedAt":880},"azsdk-common-sdk-release","release Azure SDK packages","Check release readiness and trigger the release pipeline for Azure SDK packages. **UTILITY SKILL**. USE FOR: \"release SDK\", \"trigger release\", \"check release readiness\", \"release pipeline\", \"publish package\", \"ship SDK\". DO NOT USE FOR: SDK development, code generation, pipeline debugging, release plan creation. INVOKES: azure-sdk-mcp:azsdk_release_sdk.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[876,877,878,879],{"name":11,"slug":8,"type":14},{"name":778,"slug":779,"type":14},{"name":781,"slug":782,"type":14},{"name":849,"slug":850,"type":14},"2026-07-12T08:17:34.27607",{"slug":882,"name":882,"fn":883,"description":884,"org":885,"tags":886,"stars":837,"repoUrl":838,"updatedAt":895},"azure-typespec-author","author and modify Azure TypeSpec API specifications","Authors and modifies Azure TypeSpec (.tsp) API specifications. USE FOR: any TypeSpec\u002Ftsp change — api versions (add, bump, preview, stable, promote), resources, operations, models, properties, decorators, visibility, constraints, breaking changes, LRO, suppressions, operationId, spread model. Covers ARM resource-manager and data-plane services. DO NOT USE FOR: SDK generation, releasing SDK packages, or single MCP tool calls. INVOKES: azure-sdk-mcp:azsdk_typespec_generate_authoring_plan, azure-sdk-mcp:azsdk_run_typespec_validation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[887,888,889,892],{"name":828,"slug":829,"type":14},{"name":11,"slug":8,"type":14},{"name":890,"slug":891,"type":14},"OpenAPI","openapi",{"name":893,"slug":894,"type":14},"Technical Writing","technical-writing","2026-07-12T08:17:39.603232",{"slug":897,"name":897,"fn":898,"description":899,"org":900,"tags":901,"stars":837,"repoUrl":838,"updatedAt":906},"generate-sdk-locally","generate and test Azure SDKs locally","Generate, build, and test Azure SDKs locally from TypeSpec with automatic customization. WHEN: \"generate SDK locally\", \"build SDK\", \"run SDK tests\", \"run CI checks\", \"validate package\", \"run checks\", \"update changelog\", \"fix SDK build errors\", \"fix breaking changes\", \"resolve SDK generation errors\", \"customize TypeSpec\", \"rename SDK client\", \"rename SDK model\", \"hide operation from SDK\", \"fix analyzer errors\", \"resolve customization drift\", \"create subclient\", \"update metadata\", \"update version\". DO NOT USE FOR: publishing to package registries, CI pipeline configuration, API design review. INVOKES: azsdk_verify_setup, azsdk_package_generate_code, azsdk_package_build_code, azsdk_package_run_check, azsdk_package_run_tests, azsdk_customized_code_update, azsdk_package_update_changelog_content, azsdk_package_update_metadata, azsdk_package_update_version.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[902,903,904,905],{"name":11,"slug":8,"type":14},{"name":778,"slug":779,"type":14},{"name":849,"slug":850,"type":14},{"name":852,"slug":853,"type":14},"2026-07-12T08:17:37.08523",{"slug":908,"name":908,"fn":909,"description":910,"org":911,"tags":912,"stars":837,"repoUrl":838,"updatedAt":920},"markdown-token-optimizer","optimize markdown files for token efficiency","Analyze markdown files for token efficiency and reduce context-window bloat. **UTILITY SKILL**. DO NOT USE FOR: code optimization, general file editing, non-markdown files. TRIGGERS: optimize markdown, reduce tokens, token count, token bloat, too many tokens, make concise, shrink file, file too large, optimize for AI, token efficiency, verbose markdown, reduce file size. INVOKES: waza CLI.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[913,916,919],{"name":914,"slug":915,"type":14},"LLM","llm",{"name":917,"slug":918,"type":14},"Performance","performance",{"name":893,"slug":894,"type":14},"2026-07-12T08:17:42.080413",{"slug":922,"name":922,"fn":923,"description":924,"org":925,"tags":926,"stars":837,"repoUrl":838,"updatedAt":931},"pipeline-troubleshooting","troubleshoot Azure SDK CI pipelines","Diagnose and resolve failures in Azure SDK CI and generation pipelines. **UTILITY SKILL**. USE FOR: \"pipeline failed\", \"build failure\", \"CI check failing\", \"SDK generation error\", \"reproduce pipeline locally\", \"debug SDK pipeline\". DO NOT USE FOR: local build issues without pipeline context, API design review, SDK publishing. INVOKES: azure-sdk-mcp:azsdk_analyze_pipeline, azure-sdk-mcp:azsdk_package_build_code, azure-sdk-mcp:azsdk_package_run_check.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[927,928,929,930],{"name":11,"slug":8,"type":14},{"name":778,"slug":779,"type":14},{"name":705,"slug":706,"type":14},{"name":849,"slug":850,"type":14},"2026-07-12T08:17:40.821512",{"slug":933,"name":933,"fn":934,"description":935,"org":936,"tags":937,"stars":837,"repoUrl":838,"updatedAt":944},"sensei","improve skill frontmatter compliance","**WORKFLOW SKILL** — Iteratively improve skill frontmatter compliance using the Ralph loop pattern. WHEN: \"run sensei\", \"sensei help\", \"improve skill\", \"fix frontmatter\", \"skill compliance\", \"frontmatter audit\", \"score skill\", \"check skill tokens\". INVOKES: token counting tools, test runners, git commands. FOR SINGLE OPERATIONS: use token CLI directly for counts\u002Fchecks.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[938,939,940,943],{"name":11,"slug":8,"type":14},{"name":751,"slug":752,"type":14},{"name":941,"slug":942,"type":14},"Process Optimization","process-optimization",{"name":893,"slug":894,"type":14},"2026-07-12T08:17:32.970921",{"slug":946,"name":946,"fn":947,"description":948,"org":949,"tags":950,"stars":837,"repoUrl":838,"updatedAt":956},"skill-authoring","author agent skills for agentskills.io","Write Agent Skills that comply with the agentskills.io specification. WHEN: \"create a skill\", \"new skill\", \"write a skill\", \"skill template\", \"skill structure\", \"review skill\", \"skill PR\", \"skill compliance\", \"SKILL.md format\", \"skill frontmatter\", \"skill best practices\". DO NOT USE FOR: improving existing skills (use sensei), general documentation. INVOKES: waza CLI.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[951,952,955],{"name":835,"slug":836,"type":14},{"name":953,"slug":954,"type":14},"Plugin Development","plugin-development",{"name":893,"slug":894,"type":14},"2026-07-12T08:17:35.873862",109]