[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-arize-datasets":3,"mdc-smcs1g-key":47,"related-repo-arize-datasets":530,"related-org-arize-datasets":615},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":12,"stars":22,"repoUrl":23,"updatedAt":24,"license":25,"forks":26,"topics":27,"repo":42,"sourceUrl":45,"mdContent":46},"datasets","reason about Phoenix dataset structure","Understand what a Phoenix dataset is and reason well about its examples, outputs, splits, and how it feeds evaluators and experiments. Load this whenever a dataset is in view or the user asks what a dataset is, how splits work, what an output \"means\", or how datasets relate to experiments and evals. This skill governs the judgment; any tool descriptions govern the mechanics.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},"arize","Arize AI","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Farize.jpg","Arize-ai",[13,16,19],{"name":14,"slug":4,"type":15},"Datasets","tag",{"name":17,"slug":18,"type":15},"Evals","evals",{"name":20,"slug":21,"type":15},"Data Analysis","data-analysis",10513,"https:\u002F\u002Fgithub.com\u002FArize-ai\u002Fphoenix","2026-07-12T08:08:21.695457",null,977,[28,29,30,31,32,4,18,33,34,35,36,37,38,39,40,41],"agents","ai-monitoring","ai-observability","aiengineering","anthropic","langchain","llamaindex","llm-eval","llm-evaluation","llmops","llms","openai","prompt-engineering","smolagents",{"repoUrl":23,"stars":22,"forks":26,"topics":43,"description":44},[28,29,30,31,32,4,18,33,34,35,36,37,38,39,40,41],"AI Observability & Evaluation","https:\u002F\u002Fgithub.com\u002FArize-ai\u002Fphoenix\u002Ftree\u002FHEAD\u002Fsrc\u002Fphoenix\u002Fserver\u002Fagents\u002Fprompts\u002Fskills\u002Fdatasets","---\nname: datasets\ndescription: >\n  Understand what a Phoenix dataset is and reason well about its examples, outputs, splits, and how it feeds evaluators and experiments. Load this whenever a dataset is in view or the user asks what a dataset is, how splits work, what an output \"means\", or how datasets relate to experiments and evals. This skill governs the judgment; any tool descriptions govern the mechanics.\nsummary: Reason well about Phoenix datasets — examples, outputs, splits, labels — and how they feed evaluators and experiments.\n---\n\n# Datasets\n\nA dataset is a table of **examples**. Each example (row) has an **input**, an optional **output**, and optional **metadata**. A dataset is the unit you evaluate a prompt or application against: you run something over every example and compare what comes out against what the example says.\n\nRows in one dataset should look roughly alike — the same shape of input, the same shape of output, the same metadata keys. The dataset has no enforced schema, so this consistency is a *convention you maintain*, not something the system guarantees. When you add or edit rows, match the shape of the rows already there; a dataset where row 3 has a `question` field and row 4 has a `prompt` field is harder to evaluate and harder to reason about.\n\n## What an output actually means\n\nThe most common mistake is to treat a saved output as \"the correct answer.\" It usually is not.\n\nAn output on a row is a **reference**, not a verdict. Treat it as \"an answer that was recorded for this input,\" and ask *where it came from* before trusting it:\n\n- A dataset is **golden** only when its outputs are genuinely ideal — hand-written or hand-verified to be the answer you want. Here the output really is the target.\n- Far more often, outputs are **baselines**: captured from whatever produced them when the dataset was built (a model run, a production trace, an import). A baseline output records what the system *did*, which may be mediocre, outdated, or wrong. It is a point of comparison, not a goal.\n\nSo: never assume an output is right because it is present. If the user implies the dataset is golden, you can lean on the outputs as targets; if you don't know its provenance, say so and treat outputs as references. A row with no output at all is normal and fine — it just means \"we have an input to run, but no recorded answer to compare against.\"\n\n## How datasets feed evaluators and experiments\n\n- **Evaluators** are attached to a dataset to judge outputs along a dimension (correctness, relevance, format, and so on).\n- Running a prompt or application over every example is an **experiment**. Each example produces an experiment output, and the evaluators score those outputs — often against the example's reference output. This is why the reference\u002Fgolden distinction matters: an experiment compared against weak baselines tells you \"did this change behavior,\" while an experiment compared against golden outputs tells you \"is this correct.\"\n- Every example-level change creates a new dataset **version**, recording who made it. Experiments are run against a specific version, so the dataset's history is part of the evidence.\n\n## Matching a dataset to the prompt you run over it\n\nWhen you run a prompt over a dataset in the playground, each prompt template variable `{{x}}` is filled from the matching example field `input.x` — **bound by name**. So the dataset's `input` keys have to cover the prompt's template variables, or the run produces nothing useful.\n\nReconcile the two before you run (or before you build a dataset for a run):\n\n- List the prompt's template variables and the dataset's `input` keys and make them line up. A `{{customer_message}}` variable needs an `input.customer_message` field — not `input.message` or `input.question`. A field that matches no variable is unused; a variable with no matching field renders empty.\n- Cover **every** variable the prompt uses. A prompt with five variables run over a dataset that only carries one field fills one variable and blanks the other four.\n- A playground run that *finishes with empty output and no error* almost always means this binding did not match — treat that as a misconfiguration to fix (align the field names, or map them), not as success.\n\nIf you are creating a dataset specifically to exercise a prompt, read the prompt's variables first and name the example `input` keys to match them from the start.\n\n## Splits\n\nA **split** is a named slice of the examples. The same mechanism serves three distinct purposes — know which one the user means before acting:\n\n1. **Honest measurement (train \u002F validation \u002F test).** The classic ML division. A held-out test split that you never tune against gives a trustworthy estimate of real performance; tuning until the test numbers look good (overfitting) destroys that. If the user is optimizing a prompt or a model, respect the test split as blind.\n2. **Facets (category \u002F difficulty \u002F type).** Splits like `single-hop` vs. `multi-hop`, `easy` vs. `hard`, or by topic let you break an experiment down and see *where* a task is weak instead of reading one aggregate number. Use these when the user asks \"where is it failing,\" not just \"how well does it do.\"\n3. **Quick iteration (small chunks).** A small split is useful for a fast pass in the playground *before* committing to a full regression run over the whole dataset. Cheaper and faster to learn from while iterating.\n\nA single example can belong to more than one split, and a split is just a label on rows — moving a row between splits doesn't change the row's input or output.\n\n## How to work with datasets well\n\n- **Look before you change.** Read existing rows before adding — to match their shape — and before editing or deleting — to act on the real current content, not an assumption. This is the same habit the prompt tools follow.\n- **Be honest about outputs.** When you add a row, only present its output as the right answer if it genuinely is; otherwise call it a baseline\u002Freference.\n- **Keep rows uniform.** New and edited rows should match the field names and structure of the rest of the dataset.\n- **Respect splits.** Don't blur a held-out test split into training material, and surface per-split breakdowns when the user wants to know where a task is weak.\n- **Treat changes as versioned and attributable.** Edits and deletes are real mutations to a shared artifact others may be using, and each creates a new dataset version. Preview and confirm before applying.\n",{"data":48,"body":50},{"name":4,"description":6,"summary":49},"Reason well about Phoenix datasets — examples, outputs, splits, labels — and how they feed evaluators and experiments.",{"type":51,"children":52},"root",[53,60,95,125,132,137,156,192,197,203,240,246,281,286,357,369,375,387,466,471,477],{"type":54,"tag":55,"props":56,"children":57},"element","h1",{"id":4},[58],{"type":59,"value":14},"text",{"type":54,"tag":61,"props":62,"children":63},"p",{},[64,66,72,74,79,81,86,88,93],{"type":59,"value":65},"A dataset is a table of ",{"type":54,"tag":67,"props":68,"children":69},"strong",{},[70],{"type":59,"value":71},"examples",{"type":59,"value":73},". Each example (row) has an ",{"type":54,"tag":67,"props":75,"children":76},{},[77],{"type":59,"value":78},"input",{"type":59,"value":80},", an optional ",{"type":54,"tag":67,"props":82,"children":83},{},[84],{"type":59,"value":85},"output",{"type":59,"value":87},", and optional ",{"type":54,"tag":67,"props":89,"children":90},{},[91],{"type":59,"value":92},"metadata",{"type":59,"value":94},". A dataset is the unit you evaluate a prompt or application against: you run something over every example and compare what comes out against what the example says.",{"type":54,"tag":61,"props":96,"children":97},{},[98,100,106,108,115,117,123],{"type":59,"value":99},"Rows in one dataset should look roughly alike — the same shape of input, the same shape of output, the same metadata keys. The dataset has no enforced schema, so this consistency is a ",{"type":54,"tag":101,"props":102,"children":103},"em",{},[104],{"type":59,"value":105},"convention you maintain",{"type":59,"value":107},", not something the system guarantees. When you add or edit rows, match the shape of the rows already there; a dataset where row 3 has a ",{"type":54,"tag":109,"props":110,"children":112},"code",{"className":111},[],[113],{"type":59,"value":114},"question",{"type":59,"value":116}," field and row 4 has a ",{"type":54,"tag":109,"props":118,"children":120},{"className":119},[],[121],{"type":59,"value":122},"prompt",{"type":59,"value":124}," field is harder to evaluate and harder to reason about.",{"type":54,"tag":126,"props":127,"children":129},"h2",{"id":128},"what-an-output-actually-means",[130],{"type":59,"value":131},"What an output actually means",{"type":54,"tag":61,"props":133,"children":134},{},[135],{"type":59,"value":136},"The most common mistake is to treat a saved output as \"the correct answer.\" It usually is not.",{"type":54,"tag":61,"props":138,"children":139},{},[140,142,147,149,154],{"type":59,"value":141},"An output on a row is a ",{"type":54,"tag":67,"props":143,"children":144},{},[145],{"type":59,"value":146},"reference",{"type":59,"value":148},", not a verdict. Treat it as \"an answer that was recorded for this input,\" and ask ",{"type":54,"tag":101,"props":150,"children":151},{},[152],{"type":59,"value":153},"where it came from",{"type":59,"value":155}," before trusting it:",{"type":54,"tag":157,"props":158,"children":159},"ul",{},[160,173],{"type":54,"tag":161,"props":162,"children":163},"li",{},[164,166,171],{"type":59,"value":165},"A dataset is ",{"type":54,"tag":67,"props":167,"children":168},{},[169],{"type":59,"value":170},"golden",{"type":59,"value":172}," only when its outputs are genuinely ideal — hand-written or hand-verified to be the answer you want. Here the output really is the target.",{"type":54,"tag":161,"props":174,"children":175},{},[176,178,183,185,190],{"type":59,"value":177},"Far more often, outputs are ",{"type":54,"tag":67,"props":179,"children":180},{},[181],{"type":59,"value":182},"baselines",{"type":59,"value":184},": captured from whatever produced them when the dataset was built (a model run, a production trace, an import). A baseline output records what the system ",{"type":54,"tag":101,"props":186,"children":187},{},[188],{"type":59,"value":189},"did",{"type":59,"value":191},", which may be mediocre, outdated, or wrong. It is a point of comparison, not a goal.",{"type":54,"tag":61,"props":193,"children":194},{},[195],{"type":59,"value":196},"So: never assume an output is right because it is present. If the user implies the dataset is golden, you can lean on the outputs as targets; if you don't know its provenance, say so and treat outputs as references. A row with no output at all is normal and fine — it just means \"we have an input to run, but no recorded answer to compare against.\"",{"type":54,"tag":126,"props":198,"children":200},{"id":199},"how-datasets-feed-evaluators-and-experiments",[201],{"type":59,"value":202},"How datasets feed evaluators and experiments",{"type":54,"tag":157,"props":204,"children":205},{},[206,216,228],{"type":54,"tag":161,"props":207,"children":208},{},[209,214],{"type":54,"tag":67,"props":210,"children":211},{},[212],{"type":59,"value":213},"Evaluators",{"type":59,"value":215}," are attached to a dataset to judge outputs along a dimension (correctness, relevance, format, and so on).",{"type":54,"tag":161,"props":217,"children":218},{},[219,221,226],{"type":59,"value":220},"Running a prompt or application over every example is an ",{"type":54,"tag":67,"props":222,"children":223},{},[224],{"type":59,"value":225},"experiment",{"type":59,"value":227},". Each example produces an experiment output, and the evaluators score those outputs — often against the example's reference output. This is why the reference\u002Fgolden distinction matters: an experiment compared against weak baselines tells you \"did this change behavior,\" while an experiment compared against golden outputs tells you \"is this correct.\"",{"type":54,"tag":161,"props":229,"children":230},{},[231,233,238],{"type":59,"value":232},"Every example-level change creates a new dataset ",{"type":54,"tag":67,"props":234,"children":235},{},[236],{"type":59,"value":237},"version",{"type":59,"value":239},", recording who made it. Experiments are run against a specific version, so the dataset's history is part of the evidence.",{"type":54,"tag":126,"props":241,"children":243},{"id":242},"matching-a-dataset-to-the-prompt-you-run-over-it",[244],{"type":59,"value":245},"Matching a dataset to the prompt you run over it",{"type":54,"tag":61,"props":247,"children":248},{},[249,251,257,259,265,267,272,274,279],{"type":59,"value":250},"When you run a prompt over a dataset in the playground, each prompt template variable ",{"type":54,"tag":109,"props":252,"children":254},{"className":253},[],[255],{"type":59,"value":256},"{{x}}",{"type":59,"value":258}," is filled from the matching example field ",{"type":54,"tag":109,"props":260,"children":262},{"className":261},[],[263],{"type":59,"value":264},"input.x",{"type":59,"value":266}," — ",{"type":54,"tag":67,"props":268,"children":269},{},[270],{"type":59,"value":271},"bound by name",{"type":59,"value":273},". So the dataset's ",{"type":54,"tag":109,"props":275,"children":277},{"className":276},[],[278],{"type":59,"value":78},{"type":59,"value":280}," keys have to cover the prompt's template variables, or the run produces nothing useful.",{"type":54,"tag":61,"props":282,"children":283},{},[284],{"type":59,"value":285},"Reconcile the two before you run (or before you build a dataset for a run):",{"type":54,"tag":157,"props":287,"children":288},{},[289,333,345],{"type":54,"tag":161,"props":290,"children":291},{},[292,294,299,301,307,309,315,317,323,325,331],{"type":59,"value":293},"List the prompt's template variables and the dataset's ",{"type":54,"tag":109,"props":295,"children":297},{"className":296},[],[298],{"type":59,"value":78},{"type":59,"value":300}," keys and make them line up. A ",{"type":54,"tag":109,"props":302,"children":304},{"className":303},[],[305],{"type":59,"value":306},"{{customer_message}}",{"type":59,"value":308}," variable needs an ",{"type":54,"tag":109,"props":310,"children":312},{"className":311},[],[313],{"type":59,"value":314},"input.customer_message",{"type":59,"value":316}," field — not ",{"type":54,"tag":109,"props":318,"children":320},{"className":319},[],[321],{"type":59,"value":322},"input.message",{"type":59,"value":324}," or ",{"type":54,"tag":109,"props":326,"children":328},{"className":327},[],[329],{"type":59,"value":330},"input.question",{"type":59,"value":332},". A field that matches no variable is unused; a variable with no matching field renders empty.",{"type":54,"tag":161,"props":334,"children":335},{},[336,338,343],{"type":59,"value":337},"Cover ",{"type":54,"tag":67,"props":339,"children":340},{},[341],{"type":59,"value":342},"every",{"type":59,"value":344}," variable the prompt uses. A prompt with five variables run over a dataset that only carries one field fills one variable and blanks the other four.",{"type":54,"tag":161,"props":346,"children":347},{},[348,350,355],{"type":59,"value":349},"A playground run that ",{"type":54,"tag":101,"props":351,"children":352},{},[353],{"type":59,"value":354},"finishes with empty output and no error",{"type":59,"value":356}," almost always means this binding did not match — treat that as a misconfiguration to fix (align the field names, or map them), not as success.",{"type":54,"tag":61,"props":358,"children":359},{},[360,362,367],{"type":59,"value":361},"If you are creating a dataset specifically to exercise a prompt, read the prompt's variables first and name the example ",{"type":54,"tag":109,"props":363,"children":365},{"className":364},[],[366],{"type":59,"value":78},{"type":59,"value":368}," keys to match them from the start.",{"type":54,"tag":126,"props":370,"children":372},{"id":371},"splits",[373],{"type":59,"value":374},"Splits",{"type":54,"tag":61,"props":376,"children":377},{},[378,380,385],{"type":59,"value":379},"A ",{"type":54,"tag":67,"props":381,"children":382},{},[383],{"type":59,"value":384},"split",{"type":59,"value":386}," is a named slice of the examples. The same mechanism serves three distinct purposes — know which one the user means before acting:",{"type":54,"tag":388,"props":389,"children":390},"ol",{},[391,401,449],{"type":54,"tag":161,"props":392,"children":393},{},[394,399],{"type":54,"tag":67,"props":395,"children":396},{},[397],{"type":59,"value":398},"Honest measurement (train \u002F validation \u002F test).",{"type":59,"value":400}," The classic ML division. A held-out test split that you never tune against gives a trustworthy estimate of real performance; tuning until the test numbers look good (overfitting) destroys that. If the user is optimizing a prompt or a model, respect the test split as blind.",{"type":54,"tag":161,"props":402,"children":403},{},[404,409,411,417,419,425,427,433,434,440,442,447],{"type":54,"tag":67,"props":405,"children":406},{},[407],{"type":59,"value":408},"Facets (category \u002F difficulty \u002F type).",{"type":59,"value":410}," Splits like ",{"type":54,"tag":109,"props":412,"children":414},{"className":413},[],[415],{"type":59,"value":416},"single-hop",{"type":59,"value":418}," vs. ",{"type":54,"tag":109,"props":420,"children":422},{"className":421},[],[423],{"type":59,"value":424},"multi-hop",{"type":59,"value":426},", ",{"type":54,"tag":109,"props":428,"children":430},{"className":429},[],[431],{"type":59,"value":432},"easy",{"type":59,"value":418},{"type":54,"tag":109,"props":435,"children":437},{"className":436},[],[438],{"type":59,"value":439},"hard",{"type":59,"value":441},", or by topic let you break an experiment down and see ",{"type":54,"tag":101,"props":443,"children":444},{},[445],{"type":59,"value":446},"where",{"type":59,"value":448}," a task is weak instead of reading one aggregate number. Use these when the user asks \"where is it failing,\" not just \"how well does it do.\"",{"type":54,"tag":161,"props":450,"children":451},{},[452,457,459,464],{"type":54,"tag":67,"props":453,"children":454},{},[455],{"type":59,"value":456},"Quick iteration (small chunks).",{"type":59,"value":458}," A small split is useful for a fast pass in the playground ",{"type":54,"tag":101,"props":460,"children":461},{},[462],{"type":59,"value":463},"before",{"type":59,"value":465}," committing to a full regression run over the whole dataset. Cheaper and faster to learn from while iterating.",{"type":54,"tag":61,"props":467,"children":468},{},[469],{"type":59,"value":470},"A single example can belong to more than one split, and a split is just a label on rows — moving a row between splits doesn't change the row's input or output.",{"type":54,"tag":126,"props":472,"children":474},{"id":473},"how-to-work-with-datasets-well",[475],{"type":59,"value":476},"How to work with datasets well",{"type":54,"tag":157,"props":478,"children":479},{},[480,490,500,510,520],{"type":54,"tag":161,"props":481,"children":482},{},[483,488],{"type":54,"tag":67,"props":484,"children":485},{},[486],{"type":59,"value":487},"Look before you change.",{"type":59,"value":489}," Read existing rows before adding — to match their shape — and before editing or deleting — to act on the real current content, not an assumption. This is the same habit the prompt tools follow.",{"type":54,"tag":161,"props":491,"children":492},{},[493,498],{"type":54,"tag":67,"props":494,"children":495},{},[496],{"type":59,"value":497},"Be honest about outputs.",{"type":59,"value":499}," When you add a row, only present its output as the right answer if it genuinely is; otherwise call it a baseline\u002Freference.",{"type":54,"tag":161,"props":501,"children":502},{},[503,508],{"type":54,"tag":67,"props":504,"children":505},{},[506],{"type":59,"value":507},"Keep rows uniform.",{"type":59,"value":509}," New and edited rows should match the field names and structure of the rest of the dataset.",{"type":54,"tag":161,"props":511,"children":512},{},[513,518],{"type":54,"tag":67,"props":514,"children":515},{},[516],{"type":59,"value":517},"Respect splits.",{"type":59,"value":519}," Don't blur a held-out test split into training material, and surface per-split breakdowns when the user wants to know where a task is weak.",{"type":54,"tag":161,"props":521,"children":522},{},[523,528],{"type":54,"tag":67,"props":524,"children":525},{},[526],{"type":59,"value":527},"Treat changes as versioned and attributable.",{"type":59,"value":529}," Edits and deletes are real mutations to a shared artifact others may be using, and each creates a new dataset version. Preview and confirm before applying.",{"items":531,"total":614},[532,549,555,567,579,590,604],{"slug":533,"name":533,"fn":534,"description":535,"org":536,"tags":537,"stars":22,"repoUrl":23,"updatedAt":548},"annotate-spans","annotate LLM spans and traces","Write effective, consistent annotations on LLM\u002Fagent spans and traces, and coach the user on annotation practice. Load this whenever you are about to record structured feedback with the `batch_span_annotate` tool, or when the user asks how to annotate, label, score, or review spans\u002Ftraces, build a failure taxonomy, or set up human\u002FLLM review. Do NOT load for: pure analysis with no intent to save feedback (use debug-trace), latency or cost statistics, or prompt authoring (use playground).\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[538,539,542,545],{"name":17,"slug":18,"type":15},{"name":540,"slug":541,"type":15},"LLM","llm",{"name":543,"slug":544,"type":15},"Observability","observability",{"name":546,"slug":547,"type":15},"Tracing","tracing","2026-07-12T08:08:14.140984",{"slug":4,"name":4,"fn":5,"description":6,"org":550,"tags":551,"stars":22,"repoUrl":23,"updatedAt":24},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[552,553,554],{"name":20,"slug":21,"type":15},{"name":14,"slug":4,"type":15},{"name":17,"slug":18,"type":15},{"slug":556,"name":556,"fn":557,"description":558,"org":559,"tags":560,"stars":22,"repoUrl":23,"updatedAt":566},"debug-trace","diagnose failures using trace investigation","Diagnose failure modes by systematically investigating traces. Trigger when the user explicitly asks for cross-trace diagnosis: \"what's going wrong?\", \"were there errors?\", \"debug this\", \"where is my agent struggling?\". Do NOT trigger on: (1) advice questions (\"what should I do?\"), (2) statistical questions (\"what's the average latency?\"), (3) summarize requests, (4) trace filtering (\"show me traces with errors\"), (5) vague questions (\"is there a problem?\"), (6) unrelated requests.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[561,564,565],{"name":562,"slug":563,"type":15},"Debugging","debugging",{"name":543,"slug":544,"type":15},{"name":546,"slug":547,"type":15},"2026-07-12T08:08:10.44243",{"slug":568,"name":568,"fn":569,"description":570,"org":571,"tags":572,"stars":22,"repoUrl":23,"updatedAt":578},"evaluators","author and refine Phoenix evaluators","Author or refine a Phoenix evaluator — code or LLM-as-a-judge — that scores a run's output. Trigger when the user wants to create a new evaluator, improve an existing one's logic or rubric, choose labels, or decide what to measure on a dataset or experiment. Do NOT trigger on: (1) manual prompt drafting (use `playground`), (2) running or comparing experiments themselves (use `experiments`), (3) cross-trace failure diagnosis with no evaluator in scope (use `debug-trace`).\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[573,574,575],{"name":17,"slug":18,"type":15},{"name":540,"slug":541,"type":15},{"name":576,"slug":577,"type":15},"Testing","testing","2026-07-31T05:58:09.13624",{"slug":580,"name":580,"fn":581,"description":582,"org":583,"tags":584,"stars":22,"repoUrl":23,"updatedAt":589},"experiments","run and compare dataset-backed experiments","Run, read, and compare dataset-backed experiments to find evidence that a prompt or pipeline is improving. Trigger when the user wants to iterate over a dataset with experiments, compare experiment runs, read experiment quality\u002Flatency\u002Fcost, or decide whether a change actually helped. Running a prompt over a dataset is implicitly an experiment — load this skill when dataset-backed work begins, before authoring evaluators for the experiment and before starting the recorded run, not only when reading results. Do NOT trigger on: (1) manual prompt drafting with no dataset-backed evaluation in scope (use `playground`), (2) authoring or refining an evaluator's logic or rubric (use `evaluators`), (3) cross-trace failure diagnosis with no experiment in scope (use `debug-trace`).\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[585,586,587,588],{"name":14,"slug":4,"type":15},{"name":17,"slug":18,"type":15},{"name":540,"slug":541,"type":15},{"name":576,"slug":577,"type":15},"2026-07-12T08:08:11.691477",{"slug":591,"name":591,"fn":592,"description":593,"org":594,"tags":595,"stars":22,"repoUrl":23,"updatedAt":603},"phoenix-graphql","query Phoenix API with GraphQL","Write efficient GraphQL queries against the Phoenix API. Load this skill in two cases: (1) before composing any non-trivial GraphQL query yourself for data analysis (via the `phoenix-gql` bash command) — it contains schema entrypoints and patterns that eliminate the need for introspection; (2) when the user asks for help writing GraphQL queries for their own scripts, tools, or integrations against Phoenix — it covers the endpoint, authentication, and client examples.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[596,599,600],{"name":597,"slug":598,"type":15},"Analytics","analytics",{"name":20,"slug":21,"type":15},{"name":601,"slug":602,"type":15},"GraphQL","graphql","2026-07-12T08:08:17.163493",{"slug":605,"name":605,"fn":606,"description":607,"org":608,"tags":609,"stars":22,"repoUrl":23,"updatedAt":613},"playground","author and iterate on prompts in Phoenix","Author, edit, or iterate on prompts in the Phoenix prompt playground, including running experiments over a dataset. Load before any playground tool call, including single-shot prompt rewrites.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[610,611,612],{"name":17,"slug":18,"type":15},{"name":540,"slug":541,"type":15},{"name":576,"slug":577,"type":15},"2026-07-12T08:08:12.920792",8,{"items":616,"total":740},[617,624,630,636,642,649,655,661,672,690,708,718],{"slug":533,"name":533,"fn":534,"description":535,"org":618,"tags":619,"stars":22,"repoUrl":23,"updatedAt":548},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[620,621,622,623],{"name":17,"slug":18,"type":15},{"name":540,"slug":541,"type":15},{"name":543,"slug":544,"type":15},{"name":546,"slug":547,"type":15},{"slug":4,"name":4,"fn":5,"description":6,"org":625,"tags":626,"stars":22,"repoUrl":23,"updatedAt":24},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[627,628,629],{"name":20,"slug":21,"type":15},{"name":14,"slug":4,"type":15},{"name":17,"slug":18,"type":15},{"slug":556,"name":556,"fn":557,"description":558,"org":631,"tags":632,"stars":22,"repoUrl":23,"updatedAt":566},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[633,634,635],{"name":562,"slug":563,"type":15},{"name":543,"slug":544,"type":15},{"name":546,"slug":547,"type":15},{"slug":568,"name":568,"fn":569,"description":570,"org":637,"tags":638,"stars":22,"repoUrl":23,"updatedAt":578},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[639,640,641],{"name":17,"slug":18,"type":15},{"name":540,"slug":541,"type":15},{"name":576,"slug":577,"type":15},{"slug":580,"name":580,"fn":581,"description":582,"org":643,"tags":644,"stars":22,"repoUrl":23,"updatedAt":589},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[645,646,647,648],{"name":14,"slug":4,"type":15},{"name":17,"slug":18,"type":15},{"name":540,"slug":541,"type":15},{"name":576,"slug":577,"type":15},{"slug":591,"name":591,"fn":592,"description":593,"org":650,"tags":651,"stars":22,"repoUrl":23,"updatedAt":603},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[652,653,654],{"name":597,"slug":598,"type":15},{"name":20,"slug":21,"type":15},{"name":601,"slug":602,"type":15},{"slug":605,"name":605,"fn":606,"description":607,"org":656,"tags":657,"stars":22,"repoUrl":23,"updatedAt":613},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[658,659,660],{"name":17,"slug":18,"type":15},{"name":540,"slug":541,"type":15},{"name":576,"slug":577,"type":15},{"slug":662,"name":662,"fn":663,"description":664,"org":665,"tags":666,"stars":22,"repoUrl":23,"updatedAt":671},"span-coding","analyze and code Phoenix spans","Open-code Phoenix spans with PXI-owned notes, recover those notes for axial coding, and promote stable categories into structured annotations. Load this when analyzing spans to discover failure patterns before a taxonomy exists.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[667,668,669,670],{"name":562,"slug":563,"type":15},{"name":540,"slug":541,"type":15},{"name":543,"slug":544,"type":15},{"name":546,"slug":547,"type":15},"2026-07-12T08:08:19.597239",{"slug":673,"name":673,"fn":674,"description":675,"org":676,"tags":677,"stars":687,"repoUrl":688,"updatedAt":689},"arize-admin","manage Arize enterprise user access","Manages Arize users, organizations, spaces, projects, roles, role bindings, resource restrictions, and API keys via the ax CLI. Use for enterprise admin workflows: inviting and offboarding users, onboarding new teams, creating custom roles for SAML\u002FSSO mappings, assigning roles to users, restricting project-level access, and managing service keys for multi-tenant architectures. Covers ax users, ax organizations, ax spaces, ax projects, ax roles, ax role-bindings, and ax api-keys.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[678,681,684],{"name":679,"slug":680,"type":15},"CLI","cli",{"name":682,"slug":683,"type":15},"Operations","operations",{"name":685,"slug":686,"type":15},"Permissions","permissions",38,"https:\u002F\u002Fgithub.com\u002FArize-ai\u002Farize-skills","2026-07-22T05:37:21.991338",{"slug":691,"name":691,"fn":692,"description":693,"org":694,"tags":695,"stars":687,"repoUrl":688,"updatedAt":707},"arize-ai-provider-integration","manage Arize AI provider integrations","Creates, reads, updates, and deletes Arize AI integrations that store LLM provider credentials used by evaluators and other Arize features. Supports any LLM provider (e.g. OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Vertex AI, Gemini, NVIDIA NIM). Use when the user mentions AI integration, LLM provider credentials, create integration, list integrations, update credentials, delete integration, or connecting an LLM provider to Arize.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[696,698,701,704,705],{"name":697,"slug":32,"type":15},"Anthropic",{"name":699,"slug":700,"type":15},"Azure","azure",{"name":702,"slug":703,"type":15},"Integrations","integrations",{"name":540,"slug":541,"type":15},{"name":706,"slug":39,"type":15},"OpenAI","2026-07-22T05:37:23.90468",{"slug":709,"name":709,"fn":710,"description":711,"org":712,"tags":713,"stars":687,"repoUrl":688,"updatedAt":717},"arize-annotation","manage Arize annotation workflows","Creates and manages annotation configs (categorical, continuous, freeform label schemas) and annotation queues (human review workflows) on Arize. Applies human annotations to project spans via the Python SDK. Use when the user mentions annotation config, annotation queue, label schema, human feedback, bulk annotate spans, update_annotations, labeling queue, annotate record, or human review.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[714,715,716],{"name":20,"slug":21,"type":15},{"name":540,"slug":541,"type":15},{"name":543,"slug":544,"type":15},"2026-07-22T05:37:19.010776",{"slug":719,"name":719,"fn":720,"description":721,"org":722,"tags":723,"stars":687,"repoUrl":688,"updatedAt":739},"arize-compliance-audit","audit AI agents for regulatory compliance","INVOKE THIS SKILL when auditing an AI agent or LLM app for regulatory compliance. Covers EU AI Act, GPAI Code of Practice, GDPR, NIST AI RMF, Colorado AI Act, HIPAA, and ISO 42001. Scans the codebase for compliance gaps, cross-references Arize instrumentation for audit trail coverage, and produces an actionable remediation checklist tailored to the selected frameworks.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[724,727,730,733,736],{"name":725,"slug":726,"type":15},"Audit","audit",{"name":728,"slug":729,"type":15},"Compliance","compliance",{"name":731,"slug":732,"type":15},"GDPR","gdpr",{"name":734,"slug":735,"type":15},"Legal","legal",{"name":737,"slug":738,"type":15},"Security","security","2026-07-19T05:39:42.632738",23]