[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-mlflow-analyzing-mlflow-trace":3,"mdc--1frcn2-key":33,"related-repo-mlflow-analyzing-mlflow-trace":1268,"related-org-mlflow-analyzing-mlflow-trace":1365},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":23,"repoUrl":24,"updatedAt":25,"license":26,"forks":27,"topics":28,"repo":29,"sourceUrl":31,"mdContent":32},"analyzing-mlflow-trace","analyze MLflow traces","Analyzes a single MLflow trace to answer a user query about it. Use when the user provides a trace ID and asks to debug, investigate, find issues, root-cause errors, understand behavior, or analyze quality. Triggers on \"analyze this trace\", \"what went wrong with this trace\", \"debug trace\", \"investigate trace\", \"why did this trace fail\", \"root cause this trace\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"mlflow","MLflow","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fmlflow.png",[12,16,19,20],{"name":13,"slug":14,"type":15},"Tracing","tracing","tag",{"name":17,"slug":18,"type":15},"Observability","observability",{"name":9,"slug":8,"type":15},{"name":21,"slug":22,"type":15},"Debugging","debugging",60,"https:\u002F\u002Fgithub.com\u002Fmlflow\u002Fskills","2026-07-14T05:39:02.874441",null,19,[],{"repoUrl":24,"stars":23,"forks":27,"topics":30,"description":26},[],"https:\u002F\u002Fgithub.com\u002Fmlflow\u002Fskills\u002Ftree\u002FHEAD\u002Fanalyze-mlflow-trace","---\nname: analyzing-mlflow-trace\ndescription: Analyzes a single MLflow trace to answer a user query about it. Use when the user provides a trace ID and asks to debug, investigate, find issues, root-cause errors, understand behavior, or analyze quality. Triggers on \"analyze this trace\", \"what went wrong with this trace\", \"debug trace\", \"investigate trace\", \"why did this trace fail\", \"root cause this trace\".\n---\n\n# Analyzing a Single MLflow Trace\n\n## Trace Structure\n\nA trace captures the full execution of an AI\u002FML application as a tree of **spans**. Each span represents one operation (LLM call, tool invocation, retrieval step, etc.) and records its inputs, outputs, timing, and status. Traces also carry **assessments** — feedback from humans or LLM judges about quality.\n\nIt is recommended to read [references\u002Ftrace-structure.md](references\u002Ftrace-structure.md) before analyzing a trace — it covers the complete data model, all fields and types, analysis guidance, and OpenTelemetry compatibility notes.\n\n## Handling CLI Output\n\nTraces can be 100KB+ for complex agent executions. **Always redirect output to a file** — do not pipe `mlflow traces get` directly to `jq`, `head`, or other commands, as piping can silently produce no output.\n\n```bash\n# Fetch full trace to a file (traces get always outputs JSON, no --output flag needed)\nmlflow traces get --trace-id \u003CID> > \u002Ftmp\u002Ftrace.json\n\n# Then process the file\njq '.info.state' \u002Ftmp\u002Ftrace.json\njq '.data.spans | length' \u002Ftmp\u002Ftrace.json\n```\n\n**Prefer fetching the full trace and parsing the JSON directly** rather than using `--extract-fields`. The `--extract-fields` flag has limited support for nested span data (e.g., span inputs\u002Foutputs may return empty objects). Fetch the complete trace once and parse it as needed.\n\n## JSON Structure\n\nThe trace JSON has two top-level keys: `info` (metadata, assessments) and `data` (spans).\n\n```\n{\n  \"info\": { \"trace_id\", \"state\", \"request_time\", \"assessments\", ... },\n  \"data\": { \"spans\": [ { \"span_id\", \"name\", \"status\", \"attributes\", ... } ] }\n}\n```\n\n**Key paths** (verified against actual CLI output):\n\n| What | jq path |\n|---|---|\n| Trace state | `.info.state` |\n| All spans | `.data.spans` |\n| Root span | `.data.spans[] \\| select(.parent_span_id == null)` |\n| Span status code | `.data.spans[].status.code` (values: `STATUS_CODE_OK`, `STATUS_CODE_ERROR`, `STATUS_CODE_UNSET`) |\n| Span status message | `.data.spans[].status.message` |\n| Span inputs | `.data.spans[].attributes[\"mlflow.spanInputs\"]` |\n| Span outputs | `.data.spans[].attributes[\"mlflow.spanOutputs\"]` |\n| Assessments | `.info.assessments` |\n| Assessment name | `.info.assessments[].assessment_name` |\n| Feedback value | `.info.assessments[].feedback.value` |\n| Feedback error | `.info.assessments[].feedback.error` |\n| Assessment rationale | `.info.assessments[].rationale` |\n\n**Important**: Span inputs and outputs are stored as serialized JSON strings inside `attributes`, not as top-level span fields. Traces from third-party OpenTelemetry clients may use different attribute names (e.g., GenAI Semantic Conventions, OpenInference, or custom keys) — check the raw `attributes` dict to find the equivalent fields.\n\n**If paths don't match** (structure may vary by MLflow version), discover them:\n\n```bash\n# Top-level keys\njq 'keys' \u002Ftmp\u002Ftrace.json\n\n# Span keys\njq '.data.spans[0] | keys' \u002Ftmp\u002Ftrace.json\n\n# Status structure\njq '.data.spans[0].status' \u002Ftmp\u002Ftrace.json\n```\n\n## Quick Health Check\n\nAfter fetching a trace to a file, run this to get a summary:\n\n```bash\njq '{\n  state: .info.state,\n  span_count: (.data.spans | length),\n  error_spans: [.data.spans[] | select(.status.code == \"STATUS_CODE_ERROR\") | .name],\n  assessment_errors: [.info.assessments[] | select(.feedback.error) | .assessment_name]\n}' \u002Ftmp\u002Ftrace.json\n```\n\n## Analysis Insights\n\n- **`state: OK` does not mean correct output.** It only means no unhandled exception. Check assessments for quality signals, and if none exist, analyze the trace's inputs, outputs, and intermediate span data directly for issues.\n- **Always consult the `rationale` when interpreting assessment values.** The `value` alone can be misleading — for example, a `user_frustration` assessment with `value: \"no\"` could mean \"no frustration detected\" or \"the frustration check did not pass\" (i.e., frustration *is* present), depending on how the scorer was configured. The `.rationale` field (a top-level assessment field, **not** nested under `.feedback`) explains what the value means in context and often describes the issue in plain language before you need to examine any spans.\n- **Assessments tell you *what* went wrong; spans tell you *where*.** If assessments exist, use feedback\u002Fexpectations to form a hypothesis, then confirm it in the span tree. If no assessments exist, examine span inputs\u002Foutputs to identify where the execution diverged from expected behavior.\n- **Assessment errors are not trace errors.** If an assessment has an `error` field, it means the scorer or judge that evaluated the trace failed — not that the trace itself has a problem. The trace may be perfectly fine; the assessment's `value` is just unreliable. This can happen when a scorer crashes (e.g., timed out, returned unparseable output) or when a scorer was applied to a trace type it wasn't designed for (e.g., a retrieval relevance scorer applied to a trace with no retrieval steps). The latter is a scorer configuration issue, not a trace issue.\n- **Span timing reveals performance issues.** Gaps between parent and child spans indicate overhead; repeated span names suggest retries; compare individual span durations to find bottlenecks.\n- **Token usage explains latency and cost.** Look for token usage in trace metadata (e.g., `mlflow.trace.tokenUsage`) or span attributes (e.g., `mlflow.chat.tokenUsage`). Not all clients set these — check the raw `attributes` dict for equivalent fields. Spikes in input tokens may indicate prompt injection or overly large context.\n\n## Codebase Correlation\n\nMLflow Tracing captures inputs, outputs, and metadata from different parts of an application's call stack. By correlating trace contents with the source code, issues can be root-caused more precisely than from the trace alone.\n\n- **Span names map to functions.** Span names typically match the function decorated with `@mlflow.trace` or wrapped in `mlflow.start_span()`. For autologged spans (LangChain, OpenAI, etc.), names follow framework conventions instead (e.g., `ChatOpenAI`, `RetrievalQA`).\n- **The span tree mirrors the call stack.** If span A is the parent of span B, then function A called function B.\n- **Span inputs\u002Foutputs correspond to function parameters\u002Freturn values.** Comparing them against the code logic reveals whether the function behaved as designed or produced an unexpected result.\n- **The trace shows *what happened*; the code shows *why*.** A retriever returning irrelevant results might trace back to a faulty similarity threshold. Incorrect span inputs might reveal wrong model parameters or missing environment variables set in code.\n\n## Example: Investigating a Wrong Answer\n\nA user reports that their customer support agent gave an incorrect answer for the query \"What is our refund policy?\" There are no assessments on the trace.\n\n**1. Fetch the trace and check high-level signals.**\n\nThe trace has `state: OK` — no crash occurred. No assessments are present, so examine the trace's inputs and outputs directly. The `response_preview` says *\"Our shipping policy states that orders are delivered within 3-5 business days...\"* — this answers a different question than what was asked.\n\n**2. Examine spans to locate the problem.**\n\nThe span tree shows:\n\n```\ncustomer_support_agent (AGENT) — OK\n├── plan_action (LLM) — OK\n│   outputs: {\"tool_call\": \"search_knowledge_base\", \"args\": {\"query\": \"refund policy\"}}\n├── search_knowledge_base (TOOL) — OK\n│   inputs: {\"query\": \"refund policy\"}\n│   outputs: [{\"doc\": \"Shipping takes 3-5 business days...\", \"score\": 0.82}]\n├── generate_response (LLM) — OK\n│   inputs: {\"messages\": [..., {\"role\": \"user\", \"content\": \"Context: Shipping takes 3-5 business days...\"}]}\n│   outputs: {\"content\": \"Our shipping policy states...\"}\n```\n\nThe agent correctly decided to search for \"refund policy,\" but the `search_knowledge_base` tool returned a shipping document. The LLM then faithfully answered using the wrong context. The problem is in the tool's retrieval, not the agent's reasoning or the LLM's generation.\n\n**3. Correlate with the codebase.**\n\nThe span `search_knowledge_base` maps to a function in the application code. Investigating reveals the vector index was built from only the shipping FAQ — the refund policy documents were never indexed.\n\n**4. Recommendations.**\n\n- Re-index the knowledge base to include refund policy documents.\n- Add a retrieval relevance scorer to detect when retrieved context doesn't match the query topic.\n- Consider adding expectation assessments with correct answers for common queries to enable regression testing.\n",{"data":34,"body":35},{"name":4,"description":6},{"type":36,"children":37},"root",[38,47,54,75,88,94,131,282,307,313,334,344,354,609,634,644,763,769,774,845,851,1037,1043,1048,1135,1141,1146,1154,1181,1189,1194,1203,1216,1224,1236,1244,1262],{"type":39,"tag":40,"props":41,"children":43},"element","h1",{"id":42},"analyzing-a-single-mlflow-trace",[44],{"type":45,"value":46},"text","Analyzing a Single MLflow Trace",{"type":39,"tag":48,"props":49,"children":51},"h2",{"id":50},"trace-structure",[52],{"type":45,"value":53},"Trace Structure",{"type":39,"tag":55,"props":56,"children":57},"p",{},[58,60,66,68,73],{"type":45,"value":59},"A trace captures the full execution of an AI\u002FML application as a tree of ",{"type":39,"tag":61,"props":62,"children":63},"strong",{},[64],{"type":45,"value":65},"spans",{"type":45,"value":67},". Each span represents one operation (LLM call, tool invocation, retrieval step, etc.) and records its inputs, outputs, timing, and status. Traces also carry ",{"type":39,"tag":61,"props":69,"children":70},{},[71],{"type":45,"value":72},"assessments",{"type":45,"value":74}," — feedback from humans or LLM judges about quality.",{"type":39,"tag":55,"props":76,"children":77},{},[78,80,86],{"type":45,"value":79},"It is recommended to read ",{"type":39,"tag":81,"props":82,"children":84},"a",{"href":83},"references\u002Ftrace-structure.md",[85],{"type":45,"value":83},{"type":45,"value":87}," before analyzing a trace — it covers the complete data model, all fields and types, analysis guidance, and OpenTelemetry compatibility notes.",{"type":39,"tag":48,"props":89,"children":91},{"id":90},"handling-cli-output",[92],{"type":45,"value":93},"Handling CLI Output",{"type":39,"tag":55,"props":95,"children":96},{},[97,99,104,106,113,115,121,123,129],{"type":45,"value":98},"Traces can be 100KB+ for complex agent executions. ",{"type":39,"tag":61,"props":100,"children":101},{},[102],{"type":45,"value":103},"Always redirect output to a file",{"type":45,"value":105}," — do not pipe ",{"type":39,"tag":107,"props":108,"children":110},"code",{"className":109},[],[111],{"type":45,"value":112},"mlflow traces get",{"type":45,"value":114}," directly to ",{"type":39,"tag":107,"props":116,"children":118},{"className":117},[],[119],{"type":45,"value":120},"jq",{"type":45,"value":122},", ",{"type":39,"tag":107,"props":124,"children":126},{"className":125},[],[127],{"type":45,"value":128},"head",{"type":45,"value":130},", or other commands, as piping can silently produce no output.",{"type":39,"tag":132,"props":133,"children":138},"pre",{"className":134,"code":135,"language":136,"meta":137,"style":137},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","# Fetch full trace to a file (traces get always outputs JSON, no --output flag needed)\nmlflow traces get --trace-id \u003CID> > \u002Ftmp\u002Ftrace.json\n\n# Then process the file\njq '.info.state' \u002Ftmp\u002Ftrace.json\njq '.data.spans | length' \u002Ftmp\u002Ftrace.json\n","bash","",[139],{"type":39,"tag":107,"props":140,"children":141},{"__ignoreMap":137},[142,154,211,221,230,257],{"type":39,"tag":143,"props":144,"children":147},"span",{"class":145,"line":146},"line",1,[148],{"type":39,"tag":143,"props":149,"children":151},{"style":150},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[152],{"type":45,"value":153},"# Fetch full trace to a file (traces get always outputs JSON, no --output flag needed)\n",{"type":39,"tag":143,"props":155,"children":157},{"class":145,"line":156},2,[158,163,169,174,179,185,190,196,201,206],{"type":39,"tag":143,"props":159,"children":161},{"style":160},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[162],{"type":45,"value":8},{"type":39,"tag":143,"props":164,"children":166},{"style":165},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[167],{"type":45,"value":168}," traces",{"type":39,"tag":143,"props":170,"children":171},{"style":165},[172],{"type":45,"value":173}," get",{"type":39,"tag":143,"props":175,"children":176},{"style":165},[177],{"type":45,"value":178}," --trace-id",{"type":39,"tag":143,"props":180,"children":182},{"style":181},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[183],{"type":45,"value":184}," \u003C",{"type":39,"tag":143,"props":186,"children":187},{"style":165},[188],{"type":45,"value":189},"I",{"type":39,"tag":143,"props":191,"children":193},{"style":192},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[194],{"type":45,"value":195},"D",{"type":39,"tag":143,"props":197,"children":198},{"style":181},[199],{"type":45,"value":200},">",{"type":39,"tag":143,"props":202,"children":203},{"style":181},[204],{"type":45,"value":205}," >",{"type":39,"tag":143,"props":207,"children":208},{"style":165},[209],{"type":45,"value":210}," \u002Ftmp\u002Ftrace.json\n",{"type":39,"tag":143,"props":212,"children":214},{"class":145,"line":213},3,[215],{"type":39,"tag":143,"props":216,"children":218},{"emptyLinePlaceholder":217},true,[219],{"type":45,"value":220},"\n",{"type":39,"tag":143,"props":222,"children":224},{"class":145,"line":223},4,[225],{"type":39,"tag":143,"props":226,"children":227},{"style":150},[228],{"type":45,"value":229},"# Then process the file\n",{"type":39,"tag":143,"props":231,"children":233},{"class":145,"line":232},5,[234,238,243,248,253],{"type":39,"tag":143,"props":235,"children":236},{"style":160},[237],{"type":45,"value":120},{"type":39,"tag":143,"props":239,"children":240},{"style":181},[241],{"type":45,"value":242}," '",{"type":39,"tag":143,"props":244,"children":245},{"style":165},[246],{"type":45,"value":247},".info.state",{"type":39,"tag":143,"props":249,"children":250},{"style":181},[251],{"type":45,"value":252},"'",{"type":39,"tag":143,"props":254,"children":255},{"style":165},[256],{"type":45,"value":210},{"type":39,"tag":143,"props":258,"children":260},{"class":145,"line":259},6,[261,265,269,274,278],{"type":39,"tag":143,"props":262,"children":263},{"style":160},[264],{"type":45,"value":120},{"type":39,"tag":143,"props":266,"children":267},{"style":181},[268],{"type":45,"value":242},{"type":39,"tag":143,"props":270,"children":271},{"style":165},[272],{"type":45,"value":273},".data.spans | length",{"type":39,"tag":143,"props":275,"children":276},{"style":181},[277],{"type":45,"value":252},{"type":39,"tag":143,"props":279,"children":280},{"style":165},[281],{"type":45,"value":210},{"type":39,"tag":55,"props":283,"children":284},{},[285,290,292,298,300,305],{"type":39,"tag":61,"props":286,"children":287},{},[288],{"type":45,"value":289},"Prefer fetching the full trace and parsing the JSON directly",{"type":45,"value":291}," rather than using ",{"type":39,"tag":107,"props":293,"children":295},{"className":294},[],[296],{"type":45,"value":297},"--extract-fields",{"type":45,"value":299},". The ",{"type":39,"tag":107,"props":301,"children":303},{"className":302},[],[304],{"type":45,"value":297},{"type":45,"value":306}," flag has limited support for nested span data (e.g., span inputs\u002Foutputs may return empty objects). Fetch the complete trace once and parse it as needed.",{"type":39,"tag":48,"props":308,"children":310},{"id":309},"json-structure",[311],{"type":45,"value":312},"JSON Structure",{"type":39,"tag":55,"props":314,"children":315},{},[316,318,324,326,332],{"type":45,"value":317},"The trace JSON has two top-level keys: ",{"type":39,"tag":107,"props":319,"children":321},{"className":320},[],[322],{"type":45,"value":323},"info",{"type":45,"value":325}," (metadata, assessments) and ",{"type":39,"tag":107,"props":327,"children":329},{"className":328},[],[330],{"type":45,"value":331},"data",{"type":45,"value":333}," (spans).",{"type":39,"tag":132,"props":335,"children":339},{"className":336,"code":338,"language":45},[337],"language-text","{\n  \"info\": { \"trace_id\", \"state\", \"request_time\", \"assessments\", ... },\n  \"data\": { \"spans\": [ { \"span_id\", \"name\", \"status\", \"attributes\", ... } ] }\n}\n",[340],{"type":39,"tag":107,"props":341,"children":342},{"__ignoreMap":137},[343],{"type":45,"value":338},{"type":39,"tag":55,"props":345,"children":346},{},[347,352],{"type":39,"tag":61,"props":348,"children":349},{},[350],{"type":45,"value":351},"Key paths",{"type":45,"value":353}," (verified against actual CLI output):",{"type":39,"tag":355,"props":356,"children":357},"table",{},[358,377],{"type":39,"tag":359,"props":360,"children":361},"thead",{},[362],{"type":39,"tag":363,"props":364,"children":365},"tr",{},[366,372],{"type":39,"tag":367,"props":368,"children":369},"th",{},[370],{"type":45,"value":371},"What",{"type":39,"tag":367,"props":373,"children":374},{},[375],{"type":45,"value":376},"jq path",{"type":39,"tag":378,"props":379,"children":380},"tbody",{},[381,398,415,432,473,490,507,524,541,558,575,592],{"type":39,"tag":363,"props":382,"children":383},{},[384,390],{"type":39,"tag":385,"props":386,"children":387},"td",{},[388],{"type":45,"value":389},"Trace state",{"type":39,"tag":385,"props":391,"children":392},{},[393],{"type":39,"tag":107,"props":394,"children":396},{"className":395},[],[397],{"type":45,"value":247},{"type":39,"tag":363,"props":399,"children":400},{},[401,406],{"type":39,"tag":385,"props":402,"children":403},{},[404],{"type":45,"value":405},"All spans",{"type":39,"tag":385,"props":407,"children":408},{},[409],{"type":39,"tag":107,"props":410,"children":412},{"className":411},[],[413],{"type":45,"value":414},".data.spans",{"type":39,"tag":363,"props":416,"children":417},{},[418,423],{"type":39,"tag":385,"props":419,"children":420},{},[421],{"type":45,"value":422},"Root span",{"type":39,"tag":385,"props":424,"children":425},{},[426],{"type":39,"tag":107,"props":427,"children":429},{"className":428},[],[430],{"type":45,"value":431},".data.spans[] | select(.parent_span_id == null)",{"type":39,"tag":363,"props":433,"children":434},{},[435,440],{"type":39,"tag":385,"props":436,"children":437},{},[438],{"type":45,"value":439},"Span status code",{"type":39,"tag":385,"props":441,"children":442},{},[443,449,451,457,458,464,465,471],{"type":39,"tag":107,"props":444,"children":446},{"className":445},[],[447],{"type":45,"value":448},".data.spans[].status.code",{"type":45,"value":450}," (values: ",{"type":39,"tag":107,"props":452,"children":454},{"className":453},[],[455],{"type":45,"value":456},"STATUS_CODE_OK",{"type":45,"value":122},{"type":39,"tag":107,"props":459,"children":461},{"className":460},[],[462],{"type":45,"value":463},"STATUS_CODE_ERROR",{"type":45,"value":122},{"type":39,"tag":107,"props":466,"children":468},{"className":467},[],[469],{"type":45,"value":470},"STATUS_CODE_UNSET",{"type":45,"value":472},")",{"type":39,"tag":363,"props":474,"children":475},{},[476,481],{"type":39,"tag":385,"props":477,"children":478},{},[479],{"type":45,"value":480},"Span status message",{"type":39,"tag":385,"props":482,"children":483},{},[484],{"type":39,"tag":107,"props":485,"children":487},{"className":486},[],[488],{"type":45,"value":489},".data.spans[].status.message",{"type":39,"tag":363,"props":491,"children":492},{},[493,498],{"type":39,"tag":385,"props":494,"children":495},{},[496],{"type":45,"value":497},"Span inputs",{"type":39,"tag":385,"props":499,"children":500},{},[501],{"type":39,"tag":107,"props":502,"children":504},{"className":503},[],[505],{"type":45,"value":506},".data.spans[].attributes[\"mlflow.spanInputs\"]",{"type":39,"tag":363,"props":508,"children":509},{},[510,515],{"type":39,"tag":385,"props":511,"children":512},{},[513],{"type":45,"value":514},"Span outputs",{"type":39,"tag":385,"props":516,"children":517},{},[518],{"type":39,"tag":107,"props":519,"children":521},{"className":520},[],[522],{"type":45,"value":523},".data.spans[].attributes[\"mlflow.spanOutputs\"]",{"type":39,"tag":363,"props":525,"children":526},{},[527,532],{"type":39,"tag":385,"props":528,"children":529},{},[530],{"type":45,"value":531},"Assessments",{"type":39,"tag":385,"props":533,"children":534},{},[535],{"type":39,"tag":107,"props":536,"children":538},{"className":537},[],[539],{"type":45,"value":540},".info.assessments",{"type":39,"tag":363,"props":542,"children":543},{},[544,549],{"type":39,"tag":385,"props":545,"children":546},{},[547],{"type":45,"value":548},"Assessment name",{"type":39,"tag":385,"props":550,"children":551},{},[552],{"type":39,"tag":107,"props":553,"children":555},{"className":554},[],[556],{"type":45,"value":557},".info.assessments[].assessment_name",{"type":39,"tag":363,"props":559,"children":560},{},[561,566],{"type":39,"tag":385,"props":562,"children":563},{},[564],{"type":45,"value":565},"Feedback value",{"type":39,"tag":385,"props":567,"children":568},{},[569],{"type":39,"tag":107,"props":570,"children":572},{"className":571},[],[573],{"type":45,"value":574},".info.assessments[].feedback.value",{"type":39,"tag":363,"props":576,"children":577},{},[578,583],{"type":39,"tag":385,"props":579,"children":580},{},[581],{"type":45,"value":582},"Feedback error",{"type":39,"tag":385,"props":584,"children":585},{},[586],{"type":39,"tag":107,"props":587,"children":589},{"className":588},[],[590],{"type":45,"value":591},".info.assessments[].feedback.error",{"type":39,"tag":363,"props":593,"children":594},{},[595,600],{"type":39,"tag":385,"props":596,"children":597},{},[598],{"type":45,"value":599},"Assessment rationale",{"type":39,"tag":385,"props":601,"children":602},{},[603],{"type":39,"tag":107,"props":604,"children":606},{"className":605},[],[607],{"type":45,"value":608},".info.assessments[].rationale",{"type":39,"tag":55,"props":610,"children":611},{},[612,617,619,625,627,632],{"type":39,"tag":61,"props":613,"children":614},{},[615],{"type":45,"value":616},"Important",{"type":45,"value":618},": Span inputs and outputs are stored as serialized JSON strings inside ",{"type":39,"tag":107,"props":620,"children":622},{"className":621},[],[623],{"type":45,"value":624},"attributes",{"type":45,"value":626},", not as top-level span fields. Traces from third-party OpenTelemetry clients may use different attribute names (e.g., GenAI Semantic Conventions, OpenInference, or custom keys) — check the raw ",{"type":39,"tag":107,"props":628,"children":630},{"className":629},[],[631],{"type":45,"value":624},{"type":45,"value":633}," dict to find the equivalent fields.",{"type":39,"tag":55,"props":635,"children":636},{},[637,642],{"type":39,"tag":61,"props":638,"children":639},{},[640],{"type":45,"value":641},"If paths don't match",{"type":45,"value":643}," (structure may vary by MLflow version), discover them:",{"type":39,"tag":132,"props":645,"children":647},{"className":134,"code":646,"language":136,"meta":137,"style":137},"# Top-level keys\njq 'keys' \u002Ftmp\u002Ftrace.json\n\n# Span keys\njq '.data.spans[0] | keys' \u002Ftmp\u002Ftrace.json\n\n# Status structure\njq '.data.spans[0].status' \u002Ftmp\u002Ftrace.json\n",[648],{"type":39,"tag":107,"props":649,"children":650},{"__ignoreMap":137},[651,659,683,690,698,722,729,738],{"type":39,"tag":143,"props":652,"children":653},{"class":145,"line":146},[654],{"type":39,"tag":143,"props":655,"children":656},{"style":150},[657],{"type":45,"value":658},"# Top-level keys\n",{"type":39,"tag":143,"props":660,"children":661},{"class":145,"line":156},[662,666,670,675,679],{"type":39,"tag":143,"props":663,"children":664},{"style":160},[665],{"type":45,"value":120},{"type":39,"tag":143,"props":667,"children":668},{"style":181},[669],{"type":45,"value":242},{"type":39,"tag":143,"props":671,"children":672},{"style":165},[673],{"type":45,"value":674},"keys",{"type":39,"tag":143,"props":676,"children":677},{"style":181},[678],{"type":45,"value":252},{"type":39,"tag":143,"props":680,"children":681},{"style":165},[682],{"type":45,"value":210},{"type":39,"tag":143,"props":684,"children":685},{"class":145,"line":213},[686],{"type":39,"tag":143,"props":687,"children":688},{"emptyLinePlaceholder":217},[689],{"type":45,"value":220},{"type":39,"tag":143,"props":691,"children":692},{"class":145,"line":223},[693],{"type":39,"tag":143,"props":694,"children":695},{"style":150},[696],{"type":45,"value":697},"# Span keys\n",{"type":39,"tag":143,"props":699,"children":700},{"class":145,"line":232},[701,705,709,714,718],{"type":39,"tag":143,"props":702,"children":703},{"style":160},[704],{"type":45,"value":120},{"type":39,"tag":143,"props":706,"children":707},{"style":181},[708],{"type":45,"value":242},{"type":39,"tag":143,"props":710,"children":711},{"style":165},[712],{"type":45,"value":713},".data.spans[0] | keys",{"type":39,"tag":143,"props":715,"children":716},{"style":181},[717],{"type":45,"value":252},{"type":39,"tag":143,"props":719,"children":720},{"style":165},[721],{"type":45,"value":210},{"type":39,"tag":143,"props":723,"children":724},{"class":145,"line":259},[725],{"type":39,"tag":143,"props":726,"children":727},{"emptyLinePlaceholder":217},[728],{"type":45,"value":220},{"type":39,"tag":143,"props":730,"children":732},{"class":145,"line":731},7,[733],{"type":39,"tag":143,"props":734,"children":735},{"style":150},[736],{"type":45,"value":737},"# Status structure\n",{"type":39,"tag":143,"props":739,"children":741},{"class":145,"line":740},8,[742,746,750,755,759],{"type":39,"tag":143,"props":743,"children":744},{"style":160},[745],{"type":45,"value":120},{"type":39,"tag":143,"props":747,"children":748},{"style":181},[749],{"type":45,"value":242},{"type":39,"tag":143,"props":751,"children":752},{"style":165},[753],{"type":45,"value":754},".data.spans[0].status",{"type":39,"tag":143,"props":756,"children":757},{"style":181},[758],{"type":45,"value":252},{"type":39,"tag":143,"props":760,"children":761},{"style":165},[762],{"type":45,"value":210},{"type":39,"tag":48,"props":764,"children":766},{"id":765},"quick-health-check",[767],{"type":45,"value":768},"Quick Health Check",{"type":39,"tag":55,"props":770,"children":771},{},[772],{"type":45,"value":773},"After fetching a trace to a file, run this to get a summary:",{"type":39,"tag":132,"props":775,"children":777},{"className":134,"code":776,"language":136,"meta":137,"style":137},"jq '{\n  state: .info.state,\n  span_count: (.data.spans | length),\n  error_spans: [.data.spans[] | select(.status.code == \"STATUS_CODE_ERROR\") | .name],\n  assessment_errors: [.info.assessments[] | select(.feedback.error) | .assessment_name]\n}' \u002Ftmp\u002Ftrace.json\n",[778],{"type":39,"tag":107,"props":779,"children":780},{"__ignoreMap":137},[781,797,805,813,821,829],{"type":39,"tag":143,"props":782,"children":783},{"class":145,"line":146},[784,788,792],{"type":39,"tag":143,"props":785,"children":786},{"style":160},[787],{"type":45,"value":120},{"type":39,"tag":143,"props":789,"children":790},{"style":181},[791],{"type":45,"value":242},{"type":39,"tag":143,"props":793,"children":794},{"style":165},[795],{"type":45,"value":796},"{\n",{"type":39,"tag":143,"props":798,"children":799},{"class":145,"line":156},[800],{"type":39,"tag":143,"props":801,"children":802},{"style":165},[803],{"type":45,"value":804},"  state: .info.state,\n",{"type":39,"tag":143,"props":806,"children":807},{"class":145,"line":213},[808],{"type":39,"tag":143,"props":809,"children":810},{"style":165},[811],{"type":45,"value":812},"  span_count: (.data.spans | length),\n",{"type":39,"tag":143,"props":814,"children":815},{"class":145,"line":223},[816],{"type":39,"tag":143,"props":817,"children":818},{"style":165},[819],{"type":45,"value":820},"  error_spans: [.data.spans[] | select(.status.code == \"STATUS_CODE_ERROR\") | .name],\n",{"type":39,"tag":143,"props":822,"children":823},{"class":145,"line":232},[824],{"type":39,"tag":143,"props":825,"children":826},{"style":165},[827],{"type":45,"value":828},"  assessment_errors: [.info.assessments[] | select(.feedback.error) | .assessment_name]\n",{"type":39,"tag":143,"props":830,"children":831},{"class":145,"line":259},[832,837,841],{"type":39,"tag":143,"props":833,"children":834},{"style":165},[835],{"type":45,"value":836},"}",{"type":39,"tag":143,"props":838,"children":839},{"style":181},[840],{"type":45,"value":252},{"type":39,"tag":143,"props":842,"children":843},{"style":165},[844],{"type":45,"value":210},{"type":39,"tag":48,"props":846,"children":848},{"id":847},"analysis-insights",[849],{"type":45,"value":850},"Analysis Insights",{"type":39,"tag":852,"props":853,"children":854},"ul",{},[855,872,945,969,994,1004],{"type":39,"tag":856,"props":857,"children":858},"li",{},[859,870],{"type":39,"tag":61,"props":860,"children":861},{},[862,868],{"type":39,"tag":107,"props":863,"children":865},{"className":864},[],[866],{"type":45,"value":867},"state: OK",{"type":45,"value":869}," does not mean correct output.",{"type":45,"value":871}," It only means no unhandled exception. Check assessments for quality signals, and if none exist, analyze the trace's inputs, outputs, and intermediate span data directly for issues.",{"type":39,"tag":856,"props":873,"children":874},{},[875,888,890,896,898,904,906,912,914,920,922,928,930,935,937,943],{"type":39,"tag":61,"props":876,"children":877},{},[878,880,886],{"type":45,"value":879},"Always consult the ",{"type":39,"tag":107,"props":881,"children":883},{"className":882},[],[884],{"type":45,"value":885},"rationale",{"type":45,"value":887}," when interpreting assessment values.",{"type":45,"value":889}," The ",{"type":39,"tag":107,"props":891,"children":893},{"className":892},[],[894],{"type":45,"value":895},"value",{"type":45,"value":897}," alone can be misleading — for example, a ",{"type":39,"tag":107,"props":899,"children":901},{"className":900},[],[902],{"type":45,"value":903},"user_frustration",{"type":45,"value":905}," assessment with ",{"type":39,"tag":107,"props":907,"children":909},{"className":908},[],[910],{"type":45,"value":911},"value: \"no\"",{"type":45,"value":913}," could mean \"no frustration detected\" or \"the frustration check did not pass\" (i.e., frustration ",{"type":39,"tag":915,"props":916,"children":917},"em",{},[918],{"type":45,"value":919},"is",{"type":45,"value":921}," present), depending on how the scorer was configured. The ",{"type":39,"tag":107,"props":923,"children":925},{"className":924},[],[926],{"type":45,"value":927},".rationale",{"type":45,"value":929}," field (a top-level assessment field, ",{"type":39,"tag":61,"props":931,"children":932},{},[933],{"type":45,"value":934},"not",{"type":45,"value":936}," nested under ",{"type":39,"tag":107,"props":938,"children":940},{"className":939},[],[941],{"type":45,"value":942},".feedback",{"type":45,"value":944},") explains what the value means in context and often describes the issue in plain language before you need to examine any spans.",{"type":39,"tag":856,"props":946,"children":947},{},[948,967],{"type":39,"tag":61,"props":949,"children":950},{},[951,953,958,960,965],{"type":45,"value":952},"Assessments tell you ",{"type":39,"tag":915,"props":954,"children":955},{},[956],{"type":45,"value":957},"what",{"type":45,"value":959}," went wrong; spans tell you ",{"type":39,"tag":915,"props":961,"children":962},{},[963],{"type":45,"value":964},"where",{"type":45,"value":966},".",{"type":45,"value":968}," If assessments exist, use feedback\u002Fexpectations to form a hypothesis, then confirm it in the span tree. If no assessments exist, examine span inputs\u002Foutputs to identify where the execution diverged from expected behavior.",{"type":39,"tag":856,"props":970,"children":971},{},[972,977,979,985,987,992],{"type":39,"tag":61,"props":973,"children":974},{},[975],{"type":45,"value":976},"Assessment errors are not trace errors.",{"type":45,"value":978}," If an assessment has an ",{"type":39,"tag":107,"props":980,"children":982},{"className":981},[],[983],{"type":45,"value":984},"error",{"type":45,"value":986}," field, it means the scorer or judge that evaluated the trace failed — not that the trace itself has a problem. The trace may be perfectly fine; the assessment's ",{"type":39,"tag":107,"props":988,"children":990},{"className":989},[],[991],{"type":45,"value":895},{"type":45,"value":993}," is just unreliable. This can happen when a scorer crashes (e.g., timed out, returned unparseable output) or when a scorer was applied to a trace type it wasn't designed for (e.g., a retrieval relevance scorer applied to a trace with no retrieval steps). The latter is a scorer configuration issue, not a trace issue.",{"type":39,"tag":856,"props":995,"children":996},{},[997,1002],{"type":39,"tag":61,"props":998,"children":999},{},[1000],{"type":45,"value":1001},"Span timing reveals performance issues.",{"type":45,"value":1003}," Gaps between parent and child spans indicate overhead; repeated span names suggest retries; compare individual span durations to find bottlenecks.",{"type":39,"tag":856,"props":1005,"children":1006},{},[1007,1012,1014,1020,1022,1028,1030,1035],{"type":39,"tag":61,"props":1008,"children":1009},{},[1010],{"type":45,"value":1011},"Token usage explains latency and cost.",{"type":45,"value":1013}," Look for token usage in trace metadata (e.g., ",{"type":39,"tag":107,"props":1015,"children":1017},{"className":1016},[],[1018],{"type":45,"value":1019},"mlflow.trace.tokenUsage",{"type":45,"value":1021},") or span attributes (e.g., ",{"type":39,"tag":107,"props":1023,"children":1025},{"className":1024},[],[1026],{"type":45,"value":1027},"mlflow.chat.tokenUsage",{"type":45,"value":1029},"). Not all clients set these — check the raw ",{"type":39,"tag":107,"props":1031,"children":1033},{"className":1032},[],[1034],{"type":45,"value":624},{"type":45,"value":1036}," dict for equivalent fields. Spikes in input tokens may indicate prompt injection or overly large context.",{"type":39,"tag":48,"props":1038,"children":1040},{"id":1039},"codebase-correlation",[1041],{"type":45,"value":1042},"Codebase Correlation",{"type":39,"tag":55,"props":1044,"children":1045},{},[1046],{"type":45,"value":1047},"MLflow Tracing captures inputs, outputs, and metadata from different parts of an application's call stack. By correlating trace contents with the source code, issues can be root-caused more precisely than from the trace alone.",{"type":39,"tag":852,"props":1049,"children":1050},{},[1051,1092,1102,1112],{"type":39,"tag":856,"props":1052,"children":1053},{},[1054,1059,1061,1067,1069,1075,1077,1083,1084,1090],{"type":39,"tag":61,"props":1055,"children":1056},{},[1057],{"type":45,"value":1058},"Span names map to functions.",{"type":45,"value":1060}," Span names typically match the function decorated with ",{"type":39,"tag":107,"props":1062,"children":1064},{"className":1063},[],[1065],{"type":45,"value":1066},"@mlflow.trace",{"type":45,"value":1068}," or wrapped in ",{"type":39,"tag":107,"props":1070,"children":1072},{"className":1071},[],[1073],{"type":45,"value":1074},"mlflow.start_span()",{"type":45,"value":1076},". For autologged spans (LangChain, OpenAI, etc.), names follow framework conventions instead (e.g., ",{"type":39,"tag":107,"props":1078,"children":1080},{"className":1079},[],[1081],{"type":45,"value":1082},"ChatOpenAI",{"type":45,"value":122},{"type":39,"tag":107,"props":1085,"children":1087},{"className":1086},[],[1088],{"type":45,"value":1089},"RetrievalQA",{"type":45,"value":1091},").",{"type":39,"tag":856,"props":1093,"children":1094},{},[1095,1100],{"type":39,"tag":61,"props":1096,"children":1097},{},[1098],{"type":45,"value":1099},"The span tree mirrors the call stack.",{"type":45,"value":1101}," If span A is the parent of span B, then function A called function B.",{"type":39,"tag":856,"props":1103,"children":1104},{},[1105,1110],{"type":39,"tag":61,"props":1106,"children":1107},{},[1108],{"type":45,"value":1109},"Span inputs\u002Foutputs correspond to function parameters\u002Freturn values.",{"type":45,"value":1111}," Comparing them against the code logic reveals whether the function behaved as designed or produced an unexpected result.",{"type":39,"tag":856,"props":1113,"children":1114},{},[1115,1133],{"type":39,"tag":61,"props":1116,"children":1117},{},[1118,1120,1125,1127,1132],{"type":45,"value":1119},"The trace shows ",{"type":39,"tag":915,"props":1121,"children":1122},{},[1123],{"type":45,"value":1124},"what happened",{"type":45,"value":1126},"; the code shows ",{"type":39,"tag":915,"props":1128,"children":1129},{},[1130],{"type":45,"value":1131},"why",{"type":45,"value":966},{"type":45,"value":1134}," A retriever returning irrelevant results might trace back to a faulty similarity threshold. Incorrect span inputs might reveal wrong model parameters or missing environment variables set in code.",{"type":39,"tag":48,"props":1136,"children":1138},{"id":1137},"example-investigating-a-wrong-answer",[1139],{"type":45,"value":1140},"Example: Investigating a Wrong Answer",{"type":39,"tag":55,"props":1142,"children":1143},{},[1144],{"type":45,"value":1145},"A user reports that their customer support agent gave an incorrect answer for the query \"What is our refund policy?\" There are no assessments on the trace.",{"type":39,"tag":55,"props":1147,"children":1148},{},[1149],{"type":39,"tag":61,"props":1150,"children":1151},{},[1152],{"type":45,"value":1153},"1. Fetch the trace and check high-level signals.",{"type":39,"tag":55,"props":1155,"children":1156},{},[1157,1159,1164,1166,1172,1174,1179],{"type":45,"value":1158},"The trace has ",{"type":39,"tag":107,"props":1160,"children":1162},{"className":1161},[],[1163],{"type":45,"value":867},{"type":45,"value":1165}," — no crash occurred. No assessments are present, so examine the trace's inputs and outputs directly. The ",{"type":39,"tag":107,"props":1167,"children":1169},{"className":1168},[],[1170],{"type":45,"value":1171},"response_preview",{"type":45,"value":1173}," says ",{"type":39,"tag":915,"props":1175,"children":1176},{},[1177],{"type":45,"value":1178},"\"Our shipping policy states that orders are delivered within 3-5 business days...\"",{"type":45,"value":1180}," — this answers a different question than what was asked.",{"type":39,"tag":55,"props":1182,"children":1183},{},[1184],{"type":39,"tag":61,"props":1185,"children":1186},{},[1187],{"type":45,"value":1188},"2. Examine spans to locate the problem.",{"type":39,"tag":55,"props":1190,"children":1191},{},[1192],{"type":45,"value":1193},"The span tree shows:",{"type":39,"tag":132,"props":1195,"children":1198},{"className":1196,"code":1197,"language":45},[337],"customer_support_agent (AGENT) — OK\n├── plan_action (LLM) — OK\n│   outputs: {\"tool_call\": \"search_knowledge_base\", \"args\": {\"query\": \"refund policy\"}}\n├── search_knowledge_base (TOOL) — OK\n│   inputs: {\"query\": \"refund policy\"}\n│   outputs: [{\"doc\": \"Shipping takes 3-5 business days...\", \"score\": 0.82}]\n├── generate_response (LLM) — OK\n│   inputs: {\"messages\": [..., {\"role\": \"user\", \"content\": \"Context: Shipping takes 3-5 business days...\"}]}\n│   outputs: {\"content\": \"Our shipping policy states...\"}\n",[1199],{"type":39,"tag":107,"props":1200,"children":1201},{"__ignoreMap":137},[1202],{"type":45,"value":1197},{"type":39,"tag":55,"props":1204,"children":1205},{},[1206,1208,1214],{"type":45,"value":1207},"The agent correctly decided to search for \"refund policy,\" but the ",{"type":39,"tag":107,"props":1209,"children":1211},{"className":1210},[],[1212],{"type":45,"value":1213},"search_knowledge_base",{"type":45,"value":1215}," tool returned a shipping document. The LLM then faithfully answered using the wrong context. The problem is in the tool's retrieval, not the agent's reasoning or the LLM's generation.",{"type":39,"tag":55,"props":1217,"children":1218},{},[1219],{"type":39,"tag":61,"props":1220,"children":1221},{},[1222],{"type":45,"value":1223},"3. Correlate with the codebase.",{"type":39,"tag":55,"props":1225,"children":1226},{},[1227,1229,1234],{"type":45,"value":1228},"The span ",{"type":39,"tag":107,"props":1230,"children":1232},{"className":1231},[],[1233],{"type":45,"value":1213},{"type":45,"value":1235}," maps to a function in the application code. Investigating reveals the vector index was built from only the shipping FAQ — the refund policy documents were never indexed.",{"type":39,"tag":55,"props":1237,"children":1238},{},[1239],{"type":39,"tag":61,"props":1240,"children":1241},{},[1242],{"type":45,"value":1243},"4. Recommendations.",{"type":39,"tag":852,"props":1245,"children":1246},{},[1247,1252,1257],{"type":39,"tag":856,"props":1248,"children":1249},{},[1250],{"type":45,"value":1251},"Re-index the knowledge base to include refund policy documents.",{"type":39,"tag":856,"props":1253,"children":1254},{},[1255],{"type":45,"value":1256},"Add a retrieval relevance scorer to detect when retrieved context doesn't match the query topic.",{"type":39,"tag":856,"props":1258,"children":1259},{},[1260],{"type":45,"value":1261},"Consider adding expectation assessments with correct answers for common queries to enable regression testing.",{"type":39,"tag":1263,"props":1264,"children":1265},"style",{},[1266],{"type":45,"value":1267},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":1269,"total":1364},[1270,1287,1298,1305,1321,1339,1352],{"slug":1271,"name":1271,"fn":1272,"description":1273,"org":1274,"tags":1275,"stars":23,"repoUrl":24,"updatedAt":1286},"agent-evaluation","evaluate and optimize LLM agent output","Use this when you need to EVALUATE OR IMPROVE or OPTIMIZE an existing LLM agent's output quality - including improving tool selection accuracy, answer quality, reducing costs, or fixing issues where the agent gives wrong\u002Fincomplete responses. Evaluates agents systematically using MLflow evaluation with datasets, scorers, and tracing. IMPORTANT - Always also load the instrumenting-with-mlflow-tracing skill before starting any work. Covers end-to-end evaluation workflow or individual components (tracing setup, dataset creation, scorer definition, evaluation execution).",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1276,1279,1282,1283],{"name":1277,"slug":1278,"type":15},"Agents","agents",{"name":1280,"slug":1281,"type":15},"Evals","evals",{"name":9,"slug":8,"type":15},{"name":1284,"slug":1285,"type":15},"Performance","performance","2026-07-14T05:39:15.600492",{"slug":1288,"name":1288,"fn":1289,"description":1290,"org":1291,"tags":1292,"stars":23,"repoUrl":24,"updatedAt":1297},"analyzing-mlflow-session","analyze MLflow chat conversation sessions","Analyzes an MLflow session — a sequence of traces from a multi-turn chat conversation or interaction. Use when the user asks to debug a chat conversation, review session or chat history, find where a multi-turn chat went wrong, or analyze patterns across turns. Triggers on \"analyze this session\", \"what happened in this conversation\", \"debug session\", \"review chat history\", \"where did this chat go wrong\", \"session traces\", \"analyze chat\", \"debug this chat\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1293,1294,1295,1296],{"name":21,"slug":22,"type":15},{"name":9,"slug":8,"type":15},{"name":17,"slug":18,"type":15},{"name":13,"slug":14,"type":15},"2026-07-14T05:39:10.542342",{"slug":4,"name":4,"fn":5,"description":6,"org":1299,"tags":1300,"stars":23,"repoUrl":24,"updatedAt":25},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1301,1302,1303,1304],{"name":21,"slug":22,"type":15},{"name":9,"slug":8,"type":15},{"name":17,"slug":18,"type":15},{"name":13,"slug":14,"type":15},{"slug":1306,"name":1306,"fn":1307,"description":1308,"org":1309,"tags":1310,"stars":23,"repoUrl":24,"updatedAt":1320},"fix-agent-issue","fix and update AI agent behavior","Drives a disciplined explore → plan → implement → verify loop for changing an AI agent's behavior with confidence — whether fixing a reported failure or introducing a new requirement, business rule, or policy. Grounds the diagnosis in MLflow traces, codifies the desired behavior as a regression test suite (`mlflow.genai.evaluate` assertions in `@mlflow.test` pytest tests), and iterates the agent — not the test — until green, resisting quick system-prompt patches when the real fix is upstream (missing tool, retrieval source, or capability). Use whenever the user wants to fix or change how an agent behaves — e.g. \"fix this issue in my agent\", \"this answer is wrong\", \"the agent is hallucinating\", \"improve my agent based on this trace\", \"make the agent do X instead of Y\", \"I want the agent to lead with\u002Fprioritize\u002Frecommend X\", \"new business rule: the agent should X\", \"always\u002Fnever do X\", \"change the agent's default behavior\" — or shares a trace they want addressed.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1311,1312,1315,1316,1319],{"name":1277,"slug":1278,"type":15},{"name":1313,"slug":1314,"type":15},"Best Practices","best-practices",{"name":21,"slug":22,"type":15},{"name":1317,"slug":1318,"type":15},"Engineering","engineering",{"name":9,"slug":8,"type":15},"2026-07-30T05:53:39.749464",{"slug":1322,"name":1322,"fn":1323,"description":1324,"org":1325,"tags":1326,"stars":23,"repoUrl":24,"updatedAt":1338},"instrumenting-with-mlflow-tracing","instrument Python and TypeScript with MLflow Tracing","Instruments Python and TypeScript code with MLflow Tracing for observability. Must be loaded when setting up tracing as part of any workflow including agent evaluation. Triggers on adding tracing, instrumenting agents\u002FLLM apps, getting started with MLflow tracing, tracing specific frameworks (LangGraph, LangChain, OpenAI, Gemini, DSPy, CrewAI, AutoGen), or when another skill references tracing setup. Examples - \"How do I add tracing?\", \"Instrument my agent\", \"Trace my LangChain app\", \"Set up tracing for evaluation\"",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1327,1328,1329,1330,1331,1334,1335],{"name":1277,"slug":1278,"type":15},{"name":1280,"slug":1281,"type":15},{"name":9,"slug":8,"type":15},{"name":17,"slug":18,"type":15},{"name":1332,"slug":1333,"type":15},"Python","python",{"name":13,"slug":14,"type":15},{"name":1336,"slug":1337,"type":15},"TypeScript","typescript","2026-07-20T05:58:52.968218",{"slug":1340,"name":1340,"fn":1341,"description":1342,"org":1343,"tags":1344,"stars":23,"repoUrl":24,"updatedAt":1351},"mlflow-agent","dispatch MLflow workflows and agent tasks","Master dispatcher for all MLflow workflows. Use this skill when the user wants to do anything with MLflow — tracing, evaluating, debugging, or improving an agent. Routes to the right MLflow sub-skill automatically. Triggers on: \"use mlflow\", \"help with mlflow\", \"mlflow agent\", \"add mlflow to my project\", \"trace my agent\", \"evaluate my agent\", or any MLflow task without a specific skill in mind.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1345,1346,1347,1350],{"name":1277,"slug":1278,"type":15},{"name":9,"slug":8,"type":15},{"name":1348,"slug":1349,"type":15},"MLOps","mlops",{"name":13,"slug":14,"type":15},"2026-07-14T05:39:04.133424",{"slug":1353,"name":1353,"fn":1354,"description":1355,"org":1356,"tags":1357,"stars":23,"repoUrl":24,"updatedAt":1363},"mlflow-onboarding","onboard users to MLflow workflows","Onboards users to MLflow by determining their use case (GenAI agents\u002Fapps or traditional ML\u002Fdeep learning) and guiding them through relevant quickstart tutorials and initial integration. If an experiment ID is available, it should be supplied as input to help determine the use case. Use when the user asks to get started with MLflow, set up tracking, add observability, or integrate MLflow into their project. Triggers on \"get started with MLflow\", \"set up MLflow\", \"onboard to MLflow\", \"add MLflow to my project\", \"how do I use MLflow\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1358,1359,1360],{"name":9,"slug":8,"type":15},{"name":1348,"slug":1349,"type":15},{"name":1361,"slug":1362,"type":15},"Onboarding","onboarding","2026-07-14T05:39:06.681003",11,{"items":1366,"total":1484},[1367,1381,1391,1398,1405,1412,1420,1430,1437,1443,1458,1469],{"slug":1368,"name":1368,"fn":1369,"description":1370,"org":1371,"tags":1372,"stars":1378,"repoUrl":1379,"updatedAt":1380},"setup","configure MLflow tracing","Configure MLflow tracing for Claude Code.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1373,1376,1377],{"name":1374,"slug":1375,"type":15},"Claude Code","claude-code",{"name":9,"slug":8,"type":15},{"name":13,"slug":14,"type":15},27014,"https:\u002F\u002Fgithub.com\u002Fmlflow\u002Fmlflow","2026-07-14T05:39:00.297769",{"slug":1382,"name":1382,"fn":1383,"description":1384,"org":1385,"tags":1386,"stars":1378,"repoUrl":1379,"updatedAt":1390},"status","display MLflow tracing configuration","Show the current MLflow tracing configuration for Claude Code.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1387,1388,1389],{"name":1374,"slug":1375,"type":15},{"name":9,"slug":8,"type":15},{"name":13,"slug":14,"type":15},"2026-07-14T05:39:01.540537",{"slug":1271,"name":1271,"fn":1272,"description":1273,"org":1392,"tags":1393,"stars":23,"repoUrl":24,"updatedAt":1286},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1394,1395,1396,1397],{"name":1277,"slug":1278,"type":15},{"name":1280,"slug":1281,"type":15},{"name":9,"slug":8,"type":15},{"name":1284,"slug":1285,"type":15},{"slug":1288,"name":1288,"fn":1289,"description":1290,"org":1399,"tags":1400,"stars":23,"repoUrl":24,"updatedAt":1297},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1401,1402,1403,1404],{"name":21,"slug":22,"type":15},{"name":9,"slug":8,"type":15},{"name":17,"slug":18,"type":15},{"name":13,"slug":14,"type":15},{"slug":4,"name":4,"fn":5,"description":6,"org":1406,"tags":1407,"stars":23,"repoUrl":24,"updatedAt":25},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1408,1409,1410,1411],{"name":21,"slug":22,"type":15},{"name":9,"slug":8,"type":15},{"name":17,"slug":18,"type":15},{"name":13,"slug":14,"type":15},{"slug":1306,"name":1306,"fn":1307,"description":1308,"org":1413,"tags":1414,"stars":23,"repoUrl":24,"updatedAt":1320},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1415,1416,1417,1418,1419],{"name":1277,"slug":1278,"type":15},{"name":1313,"slug":1314,"type":15},{"name":21,"slug":22,"type":15},{"name":1317,"slug":1318,"type":15},{"name":9,"slug":8,"type":15},{"slug":1322,"name":1322,"fn":1323,"description":1324,"org":1421,"tags":1422,"stars":23,"repoUrl":24,"updatedAt":1338},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1423,1424,1425,1426,1427,1428,1429],{"name":1277,"slug":1278,"type":15},{"name":1280,"slug":1281,"type":15},{"name":9,"slug":8,"type":15},{"name":17,"slug":18,"type":15},{"name":1332,"slug":1333,"type":15},{"name":13,"slug":14,"type":15},{"name":1336,"slug":1337,"type":15},{"slug":1340,"name":1340,"fn":1341,"description":1342,"org":1431,"tags":1432,"stars":23,"repoUrl":24,"updatedAt":1351},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1433,1434,1435,1436],{"name":1277,"slug":1278,"type":15},{"name":9,"slug":8,"type":15},{"name":1348,"slug":1349,"type":15},{"name":13,"slug":14,"type":15},{"slug":1353,"name":1353,"fn":1354,"description":1355,"org":1438,"tags":1439,"stars":23,"repoUrl":24,"updatedAt":1363},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1440,1441,1442],{"name":9,"slug":8,"type":15},{"name":1348,"slug":1349,"type":15},{"name":1361,"slug":1362,"type":15},{"slug":1444,"name":1444,"fn":1445,"description":1446,"org":1447,"tags":1448,"stars":23,"repoUrl":24,"updatedAt":1457},"querying-mlflow-metrics","fetch trace metrics from MLflow servers","Fetches aggregated trace metrics (token usage, latency, trace counts, quality evaluations) from MLflow tracking servers. Triggers on requests to show metrics, analyze token usage, view LLM costs, check usage trends, or query trace statistics.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1449,1452,1455,1456],{"name":1450,"slug":1451,"type":15},"Analytics","analytics",{"name":1453,"slug":1454,"type":15},"Metrics","metrics",{"name":9,"slug":8,"type":15},{"name":17,"slug":18,"type":15},"2026-07-14T05:39:13.07056",{"slug":1459,"name":1459,"fn":1460,"description":1461,"org":1462,"tags":1463,"stars":23,"repoUrl":24,"updatedAt":1468},"retrieving-mlflow-traces","retrieve and query MLflow traces","Retrieves MLflow traces using CLI or Python API. Use when the user asks to get a trace by ID, find traces, filter traces by status\u002Ftags\u002Fmetadata\u002Fexecution time, query traces, or debug failed traces. Triggers on \"get trace\", \"search traces\", \"find failed traces\", \"filter traces by\", \"traces slower than\", \"query MLflow traces\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1464,1465,1466,1467],{"name":21,"slug":22,"type":15},{"name":9,"slug":8,"type":15},{"name":17,"slug":18,"type":15},{"name":13,"slug":14,"type":15},"2026-07-14T05:39:09.22888",{"slug":1470,"name":1470,"fn":1471,"description":1472,"org":1473,"tags":1474,"stars":23,"repoUrl":24,"updatedAt":1483},"sagemaker-mlflow","connect to SageMaker Managed MLflow","Connect to SageMaker Managed MLflow (mlflow-app or mlflow-tracking-server ARN) as an MLflow backend, then hand off to the other MLflow skills. Triggers on a SageMaker MLflow ARN (arn:aws:sagemaker:...:mlflow-app\u002F... or arn:aws:sagemaker:...:mlflow-tracking-server\u002F...) or \"SageMaker Managed MLflow\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1475,1478,1481,1482],{"name":1476,"slug":1477,"type":15},"AI Infrastructure","ai-infrastructure",{"name":1479,"slug":1480,"type":15},"AWS","aws",{"name":9,"slug":8,"type":15},{"name":1348,"slug":1349,"type":15},"2026-07-14T05:39:05.401801",13]