[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-mlflow-mlflow-onboarding":3,"mdc-3aobi-key":30,"related-org-mlflow-mlflow-onboarding":2028,"related-repo-mlflow-mlflow-onboarding":2189},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":20,"repoUrl":21,"updatedAt":22,"license":23,"forks":24,"topics":25,"repo":26,"sourceUrl":28,"mdContent":29},"mlflow-onboarding","onboard users to MLflow workflows","Onboards users to MLflow by determining their use case (GenAI agents\u002Fapps or traditional ML\u002Fdeep learning) and guiding them through relevant quickstart tutorials and initial integration. If an experiment ID is available, it should be supplied as input to help determine the use case. Use when the user asks to get started with MLflow, set up tracking, add observability, or integrate MLflow into their project. Triggers on \"get started with MLflow\", \"set up MLflow\", \"onboard to MLflow\", \"add MLflow to my project\", \"how do I use MLflow\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"mlflow","MLflow","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fmlflow.png",[12,16,17],{"name":13,"slug":14,"type":15},"Onboarding","onboarding","tag",{"name":9,"slug":8,"type":15},{"name":18,"slug":19,"type":15},"MLOps","mlops",60,"https:\u002F\u002Fgithub.com\u002Fmlflow\u002Fskills","2026-07-14T05:39:06.681003",null,19,[],{"repoUrl":21,"stars":20,"forks":24,"topics":27,"description":23},[],"https:\u002F\u002Fgithub.com\u002Fmlflow\u002Fskills\u002Ftree\u002FHEAD\u002Fmlflow-onboarding","---\nname: mlflow-onboarding\ndescription: Onboards users to MLflow by determining their use case (GenAI agents\u002Fapps or traditional ML\u002Fdeep learning) and guiding them through relevant quickstart tutorials and initial integration. If an experiment ID is available, it should be supplied as input to help determine the use case. Use when the user asks to get started with MLflow, set up tracking, add observability, or integrate MLflow into their project. Triggers on \"get started with MLflow\", \"set up MLflow\", \"onboard to MLflow\", \"add MLflow to my project\", \"how do I use MLflow\".\n---\n\n# MLflow Onboarding\n\nMLflow supports two broad use cases that require different onboarding paths:\n\n- **GenAI applications and agents**: LLM-powered apps, chatbots, RAG pipelines, tool-calling agents. Key MLflow features include **tracing** for observability, **evaluation** with LLM judges, and **prompt management**, among others.\n- **Traditional ML \u002F deep learning models**: scikit-learn, PyTorch, TensorFlow, XGBoost, etc. Key MLflow features include **experiment tracking** (parameters, metrics, artifacts), **model logging**, and **model deployment**, among others.\n\nDetermining which use case applies is the first and most important step. The onboarding path, quickstart tutorials, and integration steps differ significantly between the two.\n\n## Step 1: Determine the Use Case\n\nBefore recommending tutorials or integration steps, determine which use case the user is working on. Use the signals below, checking them in order. **If the signals are ambiguous or absent, you MUST ask the user directly.**\n\n### Signal 1: Check the Codebase\n\nSearch the user's project for imports and usage patterns that indicate the use case:\n\n**GenAI indicators** (any of these suggest GenAI):\n- Imports from LLM client libraries: `openai`, `anthropic`, `google.generativeai`, `google.genai`, `langchain`, `langchain_openai`, `langgraph`, `llamaindex`, `litellm`, `autogen`, `crewai`, `dspy`\n- Imports from MLflow GenAI modules: `mlflow.genai`, `mlflow.tracing`, `mlflow.openai`, `mlflow.langchain`\n- Usage of chat completions, embeddings, or agent frameworks\n- Prompt templates or prompt engineering code\n\n**Traditional ML indicators** (any of these suggest ML):\n- Imports from ML frameworks: `sklearn`, `torch`, `tensorflow`, `keras`, `xgboost`, `lightgbm`, `catboost`, `statsmodels`, `scipy`\n- Imports from MLflow ML modules: `mlflow.sklearn`, `mlflow.pytorch`, `mlflow.tensorflow`\n- Model training loops, `.fit()` calls, hyperparameter tuning code\n- Dataset loading with tabular\u002Fimage\u002Ftime-series data\n\n```bash\n# Search for GenAI indicators\ngrep -rl --include='*.py' -E '(import openai|import anthropic|from langchain|from langgraph|import litellm|from mlflow\\.genai|from mlflow\\.tracing|mlflow\\.openai|mlflow\\.langchain|ChatCompletion|chat\\.completions)' .\n\n# Search for ML indicators\ngrep -rl --include='*.py' -E '(from sklearn|import torch|import tensorflow|import keras|import xgboost|import lightgbm|mlflow\\.sklearn|mlflow\\.pytorch|mlflow\\.tensorflow|\\.fit\\()' .\n```\n\n### Signal 2: Check the Experiment Type Tag\n\nIf the codebase or project directory is the MLflow repository itself, skip to Signal 3 — the MLflow repo contains code for all use cases and does not indicate the user's intent.\n\nIf the experiment ID is known, check its `mlflow.experimentKind` tag. This tag is set by MLflow to indicate the experiment type:\n\n```bash\nmlflow experiments get --experiment-id \u003CEXPERIMENT_ID> --output json > \u002Ftmp\u002Fexp_detail.json\njq -r '.tags[\"mlflow.experimentKind\"] \u002F\u002F \"not set\"' \u002Ftmp\u002Fexp_detail.json\n```\n\n- **`genai_development`** → GenAI use case\n- **`custom_model_development`** → Traditional ML use case\n- **Not set** → Proceed to Signal 3\n\nIf the experiment ID is not known, skip to Signal 3.\n\n### Signal 3: Ask the User\n\nIf the codebase and experiment signals are inconclusive, ask directly:\n\n> Are you building a **GenAI application** (e.g., an LLM-powered chatbot, RAG pipeline, or tool-calling agent) or a **traditional ML\u002Fdeep learning model** (e.g., training a classifier, regression model, or neural network)?\n\n**Do not guess.** The onboarding paths are different enough that starting down the wrong one wastes the user's time.\n\n## Step 2: Recommend Quickstart Tutorials\n\nOnce the use case is determined, recommend the appropriate quickstart tutorials from the MLflow documentation. Present them to the user and ask if they'd like to follow along or jump directly to integrating MLflow into their project.\n\n### GenAI Path\n\nThe MLflow GenAI documentation is at: https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fgenai\u002Fgetting-started\u002F\n\nChoose the most relevant tutorials based on the user's context and what they've told you. Available tutorials include:\n\n- **Tracing Quickstart** (https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fgenai\u002Ftracing\u002Fquickstart\u002F) — Enabling automatic tracing for LLM calls. Covers starting an MLflow server, creating an experiment, enabling autologging, and viewing traces in the UI.\n  - Python + OpenAI variant: https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fgenai\u002Ftracing\u002Fquickstart\u002Fpython-openai\u002F\n  - TypeScript + OpenAI variant: https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fgenai\u002Ftracing\u002Fquickstart\u002Ftypescript-openai\n  - OpenTelemetry (language-agnostic) variant: also linked from the quickstart page\n- **Evaluation Quickstart** (https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fgenai\u002Feval-monitor\u002Fquickstart\u002F) — Evaluating GenAI application quality using LLM judges (scorers). Covers defining datasets, prediction functions, and built-in + custom scorers.\n- **Version Tracking Quickstart** (https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fgenai\u002Fversion-tracking\u002Fquickstart\u002F) — Prompt management, application versioning, and connecting tracing to versioned prompts.\n\nIf none of these match the user's needs, look up the MLflow GenAI documentation for more relevant guides.\n\n### Traditional ML Path\n\nThe MLflow ML documentation is at: https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fml\u002Fgetting-started\u002F\n\nChoose the most relevant tutorials based on the user's context and what they've told you. Available tutorials include:\n\n- **Tracking Quickstart** (https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fml\u002Ftracking\u002Fquickstart\u002F) — Experiment tracking with scikit-learn: autologging, manual parameter\u002Fmetric\u002Fmodel logging, and exploring results in the MLflow UI.\n- **Deep Learning Tutorial** (https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fml\u002Fgetting-started\u002Fdeep-learning\u002F) — Training a PyTorch model with MLflow logging: parameters, metrics, checkpoints, and system metrics (GPU utilization, memory).\n- **Hyperparameter Tuning Tutorial** (https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fml\u002Fgetting-started\u002Fhyperparameter-tuning\u002F) — Running hyperparameter searches with Optuna + MLflow, comparing results, and selecting the best model.\n\nIf none of these match the user's needs, look up the MLflow ML documentation for more relevant guides.\n\n## Step 3: Integrate MLflow into the User's Project\n\nAfter the user has reviewed the quickstart tutorials (or opted to skip them), offer to help integrate MLflow directly into their codebase. **Always ask for the user's consent before making changes to their code.**\n\n### GenAI Integration\n\nThe core integration for GenAI apps is **tracing** — capturing LLM calls, tool invocations, and agent steps automatically.\n\n**If asked to create an example project:** Do not assume the user has LLM API keys (e.g., OpenAI, Anthropic). Instead, create traces with mock data using `@mlflow.trace` and `mlflow.start_span()` to demonstrate tracing without requiring external API access. For example:\n\n```python\nimport mlflow\n\nmlflow.set_experiment(\"example-genai-app\")\n\n@mlflow.trace\ndef mock_chat(query: str) -> str:\n    with mlflow.start_span(name=\"retrieve_context\") as span:\n        context = \"Mock retrieved context for: \" + query\n        span.set_inputs({\"query\": query})\n        span.set_outputs({\"context\": context})\n    with mlflow.start_span(name=\"generate_response\") as span:\n        response = \"Mock response based on: \" + context\n        span.set_inputs({\"context\": context, \"query\": query})\n        span.set_outputs({\"response\": response})\n    return response\n\nmock_chat(\"What is MLflow?\")\n```\n\n**What to set up (for an existing project):**\n\n1. **Autologging** — If the user's code uses a supported framework, a single line automatically traces all calls to their LLM provider. See https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fgenai\u002Ftracing\u002F for the full list of supported providers. If the provider is supported:\n\n   ```python\n   import mlflow\n\n   # Pick the one that matches the user's LLM provider:\n   mlflow.openai.autolog()       # OpenAI SDK\n   mlflow.anthropic.autolog()    # Anthropic SDK\n   mlflow.gemini.autolog()       # Google Gemini (google-genai SDK)\n   mlflow.langchain.autolog()    # LangChain \u002F LangGraph\n   mlflow.litellm.autolog()      # LiteLLM\n   ```\n\n   Add this call once at application startup (e.g., top of `main.py`, `app.py`, or the entry point module). It must execute before any LLM calls are made.\n\n   If the provider is **not** supported by autologging, skip to step 3 (Custom tracing) and use `@mlflow.trace` to manually instrument the relevant functions.\n\n2. **Experiment configuration** — Set the experiment so traces are organized:\n\n   ```python\n   mlflow.set_experiment(\"my-genai-app\")\n   ```\n\n   Or via environment variable: `export MLFLOW_EXPERIMENT_NAME=\"my-genai-app\"`\n\n3. **Custom tracing** (optional) — For functions that aren't automatically traced (custom tools, business logic), use the `@mlflow.trace` decorator:\n\n   ```python\n   @mlflow.trace\n   def my_custom_tool(query: str) -> str:\n       # ... tool logic ...\n       return result\n   ```\n\n**Where to add it:** Find the application's entry point or initialization module and add the autologging call there. Search for the main LLM client instantiation (e.g., `openai.OpenAI()`, `ChatOpenAI()`) to find the right location.\n\n### Traditional ML Integration\n\nThe core integration for ML is **experiment tracking** — capturing parameters, metrics, and models from training runs.\n\n**What to set up:**\n\n1. **Autologging** — If the user's code uses a supported framework, a single line automatically logs parameters, metrics, and models during training. See https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fml\u002F for the full list of supported frameworks. If the framework is supported:\n\n   ```python\n   import mlflow\n\n   # Pick the one that matches the user's ML framework:\n   mlflow.sklearn.autolog()      # scikit-learn\n   mlflow.pytorch.autolog()      # PyTorch \u002F PyTorch Lightning\n   mlflow.tensorflow.autolog()   # TensorFlow \u002F Keras\n   mlflow.xgboost.autolog()      # XGBoost\n   mlflow.lightgbm.autolog()     # LightGBM\n   ```\n\n   Add this call once before training starts. It automatically captures `model.fit()` calls, logged metrics, and model artifacts.\n\n   If the framework is **not** supported by autologging, skip to step 3 (Manual logging) and use `mlflow.log_param()`, `mlflow.log_metric()`, and `mlflow.log_artifact()` to log data explicitly.\n\n2. **Experiment configuration** — Set the experiment so runs are organized:\n\n   ```python\n   mlflow.set_experiment(\"my-ml-experiment\")\n   ```\n\n   Or via environment variable: `export MLFLOW_EXPERIMENT_NAME=\"my-ml-experiment\"`\n\n3. **Manual logging** (optional) — For metrics or parameters not captured by autologging:\n\n   ```python\n   with mlflow.start_run():\n       mlflow.log_param(\"custom_param\", value)\n       mlflow.log_metric(\"custom_metric\", value)\n   ```\n\n**Where to add it:** Find the training script or module where `model.fit()` (or equivalent) is called. Add the autologging call before the training loop begins.\n\n## Verification\n\nAfter integration, verify that MLflow is capturing data correctly:\n\n### GenAI Verification\n\n1. Run the application and trigger at least one LLM call\n2. Check for traces:\n   ```bash\n   mlflow traces search \\\n     --experiment-id \u003CEXPERIMENT_ID> \\\n     --max-results 5 \\\n     --extract-fields 'info.trace_id,info.state,info.request_time' \\\n     --output json > \u002Ftmp\u002Fverify_traces.json\n   jq '.traces | length' \u002Ftmp\u002Fverify_traces.json\n   ```\n3. If traces appear, open the MLflow UI to inspect them visually\n\n### ML Verification\n\n1. Run the training script\n2. Check for runs:\n   ```bash\n   mlflow runs search \\\n     --experiment-id \u003CEXPERIMENT_ID> \\\n     --max-results 5 \\\n     --output json > \u002Ftmp\u002Fverify_runs.json\n   jq '.runs | length' \u002Ftmp\u002Fverify_runs.json\n   ```\n3. If runs appear, open the MLflow UI to inspect logged parameters, metrics, and artifacts\n",{"data":31,"body":32},{"name":4,"description":6},{"type":33,"children":34},"root",[35,43,49,116,121,128,138,145,150,160,295,305,418,569,575,580,593,693,734,739,745,750,773,783,789,794,800,813,818,903,908,914,925,929,983,988,994,1004,1010,1021,1047,1201,1209,1430,1455,1461,1472,1480,1689,1705,1711,1716,1722,1885,1891,2022],{"type":36,"tag":37,"props":38,"children":39},"element","h1",{"id":4},[40],{"type":41,"value":42},"text","MLflow Onboarding",{"type":36,"tag":44,"props":45,"children":46},"p",{},[47],{"type":41,"value":48},"MLflow supports two broad use cases that require different onboarding paths:",{"type":36,"tag":50,"props":51,"children":52},"ul",{},[53,86],{"type":36,"tag":54,"props":55,"children":56},"li",{},[57,63,65,70,72,77,79,84],{"type":36,"tag":58,"props":59,"children":60},"strong",{},[61],{"type":41,"value":62},"GenAI applications and agents",{"type":41,"value":64},": LLM-powered apps, chatbots, RAG pipelines, tool-calling agents. Key MLflow features include ",{"type":36,"tag":58,"props":66,"children":67},{},[68],{"type":41,"value":69},"tracing",{"type":41,"value":71}," for observability, ",{"type":36,"tag":58,"props":73,"children":74},{},[75],{"type":41,"value":76},"evaluation",{"type":41,"value":78}," with LLM judges, and ",{"type":36,"tag":58,"props":80,"children":81},{},[82],{"type":41,"value":83},"prompt management",{"type":41,"value":85},", among others.",{"type":36,"tag":54,"props":87,"children":88},{},[89,94,96,101,103,108,110,115],{"type":36,"tag":58,"props":90,"children":91},{},[92],{"type":41,"value":93},"Traditional ML \u002F deep learning models",{"type":41,"value":95},": scikit-learn, PyTorch, TensorFlow, XGBoost, etc. Key MLflow features include ",{"type":36,"tag":58,"props":97,"children":98},{},[99],{"type":41,"value":100},"experiment tracking",{"type":41,"value":102}," (parameters, metrics, artifacts), ",{"type":36,"tag":58,"props":104,"children":105},{},[106],{"type":41,"value":107},"model logging",{"type":41,"value":109},", and ",{"type":36,"tag":58,"props":111,"children":112},{},[113],{"type":41,"value":114},"model deployment",{"type":41,"value":85},{"type":36,"tag":44,"props":117,"children":118},{},[119],{"type":41,"value":120},"Determining which use case applies is the first and most important step. The onboarding path, quickstart tutorials, and integration steps differ significantly between the two.",{"type":36,"tag":122,"props":123,"children":125},"h2",{"id":124},"step-1-determine-the-use-case",[126],{"type":41,"value":127},"Step 1: Determine the Use Case",{"type":36,"tag":44,"props":129,"children":130},{},[131,133],{"type":41,"value":132},"Before recommending tutorials or integration steps, determine which use case the user is working on. Use the signals below, checking them in order. ",{"type":36,"tag":58,"props":134,"children":135},{},[136],{"type":41,"value":137},"If the signals are ambiguous or absent, you MUST ask the user directly.",{"type":36,"tag":139,"props":140,"children":142},"h3",{"id":141},"signal-1-check-the-codebase",[143],{"type":41,"value":144},"Signal 1: Check the Codebase",{"type":36,"tag":44,"props":146,"children":147},{},[148],{"type":41,"value":149},"Search the user's project for imports and usage patterns that indicate the use case:",{"type":36,"tag":44,"props":151,"children":152},{},[153,158],{"type":36,"tag":58,"props":154,"children":155},{},[156],{"type":41,"value":157},"GenAI indicators",{"type":41,"value":159}," (any of these suggest GenAI):",{"type":36,"tag":50,"props":161,"children":162},{},[163,253,285,290],{"type":36,"tag":54,"props":164,"children":165},{},[166,168,175,177,183,184,190,191,197,198,204,205,211,212,218,219,225,226,232,233,239,240,246,247],{"type":41,"value":167},"Imports from LLM client libraries: ",{"type":36,"tag":169,"props":170,"children":172},"code",{"className":171},[],[173],{"type":41,"value":174},"openai",{"type":41,"value":176},", ",{"type":36,"tag":169,"props":178,"children":180},{"className":179},[],[181],{"type":41,"value":182},"anthropic",{"type":41,"value":176},{"type":36,"tag":169,"props":185,"children":187},{"className":186},[],[188],{"type":41,"value":189},"google.generativeai",{"type":41,"value":176},{"type":36,"tag":169,"props":192,"children":194},{"className":193},[],[195],{"type":41,"value":196},"google.genai",{"type":41,"value":176},{"type":36,"tag":169,"props":199,"children":201},{"className":200},[],[202],{"type":41,"value":203},"langchain",{"type":41,"value":176},{"type":36,"tag":169,"props":206,"children":208},{"className":207},[],[209],{"type":41,"value":210},"langchain_openai",{"type":41,"value":176},{"type":36,"tag":169,"props":213,"children":215},{"className":214},[],[216],{"type":41,"value":217},"langgraph",{"type":41,"value":176},{"type":36,"tag":169,"props":220,"children":222},{"className":221},[],[223],{"type":41,"value":224},"llamaindex",{"type":41,"value":176},{"type":36,"tag":169,"props":227,"children":229},{"className":228},[],[230],{"type":41,"value":231},"litellm",{"type":41,"value":176},{"type":36,"tag":169,"props":234,"children":236},{"className":235},[],[237],{"type":41,"value":238},"autogen",{"type":41,"value":176},{"type":36,"tag":169,"props":241,"children":243},{"className":242},[],[244],{"type":41,"value":245},"crewai",{"type":41,"value":176},{"type":36,"tag":169,"props":248,"children":250},{"className":249},[],[251],{"type":41,"value":252},"dspy",{"type":36,"tag":54,"props":254,"children":255},{},[256,258,264,265,271,272,278,279],{"type":41,"value":257},"Imports from MLflow GenAI modules: ",{"type":36,"tag":169,"props":259,"children":261},{"className":260},[],[262],{"type":41,"value":263},"mlflow.genai",{"type":41,"value":176},{"type":36,"tag":169,"props":266,"children":268},{"className":267},[],[269],{"type":41,"value":270},"mlflow.tracing",{"type":41,"value":176},{"type":36,"tag":169,"props":273,"children":275},{"className":274},[],[276],{"type":41,"value":277},"mlflow.openai",{"type":41,"value":176},{"type":36,"tag":169,"props":280,"children":282},{"className":281},[],[283],{"type":41,"value":284},"mlflow.langchain",{"type":36,"tag":54,"props":286,"children":287},{},[288],{"type":41,"value":289},"Usage of chat completions, embeddings, or agent frameworks",{"type":36,"tag":54,"props":291,"children":292},{},[293],{"type":41,"value":294},"Prompt templates or prompt engineering code",{"type":36,"tag":44,"props":296,"children":297},{},[298,303],{"type":36,"tag":58,"props":299,"children":300},{},[301],{"type":41,"value":302},"Traditional ML indicators",{"type":41,"value":304}," (any of these suggest ML):",{"type":36,"tag":50,"props":306,"children":307},{},[308,375,400,413],{"type":36,"tag":54,"props":309,"children":310},{},[311,313,319,320,326,327,333,334,340,341,347,348,354,355,361,362,368,369],{"type":41,"value":312},"Imports from ML frameworks: ",{"type":36,"tag":169,"props":314,"children":316},{"className":315},[],[317],{"type":41,"value":318},"sklearn",{"type":41,"value":176},{"type":36,"tag":169,"props":321,"children":323},{"className":322},[],[324],{"type":41,"value":325},"torch",{"type":41,"value":176},{"type":36,"tag":169,"props":328,"children":330},{"className":329},[],[331],{"type":41,"value":332},"tensorflow",{"type":41,"value":176},{"type":36,"tag":169,"props":335,"children":337},{"className":336},[],[338],{"type":41,"value":339},"keras",{"type":41,"value":176},{"type":36,"tag":169,"props":342,"children":344},{"className":343},[],[345],{"type":41,"value":346},"xgboost",{"type":41,"value":176},{"type":36,"tag":169,"props":349,"children":351},{"className":350},[],[352],{"type":41,"value":353},"lightgbm",{"type":41,"value":176},{"type":36,"tag":169,"props":356,"children":358},{"className":357},[],[359],{"type":41,"value":360},"catboost",{"type":41,"value":176},{"type":36,"tag":169,"props":363,"children":365},{"className":364},[],[366],{"type":41,"value":367},"statsmodels",{"type":41,"value":176},{"type":36,"tag":169,"props":370,"children":372},{"className":371},[],[373],{"type":41,"value":374},"scipy",{"type":36,"tag":54,"props":376,"children":377},{},[378,380,386,387,393,394],{"type":41,"value":379},"Imports from MLflow ML modules: ",{"type":36,"tag":169,"props":381,"children":383},{"className":382},[],[384],{"type":41,"value":385},"mlflow.sklearn",{"type":41,"value":176},{"type":36,"tag":169,"props":388,"children":390},{"className":389},[],[391],{"type":41,"value":392},"mlflow.pytorch",{"type":41,"value":176},{"type":36,"tag":169,"props":395,"children":397},{"className":396},[],[398],{"type":41,"value":399},"mlflow.tensorflow",{"type":36,"tag":54,"props":401,"children":402},{},[403,405,411],{"type":41,"value":404},"Model training loops, ",{"type":36,"tag":169,"props":406,"children":408},{"className":407},[],[409],{"type":41,"value":410},".fit()",{"type":41,"value":412}," calls, hyperparameter tuning code",{"type":36,"tag":54,"props":414,"children":415},{},[416],{"type":41,"value":417},"Dataset loading with tabular\u002Fimage\u002Ftime-series data",{"type":36,"tag":419,"props":420,"children":425},"pre",{"className":421,"code":422,"language":423,"meta":424,"style":424},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","# Search for GenAI indicators\ngrep -rl --include='*.py' -E '(import openai|import anthropic|from langchain|from langgraph|import litellm|from mlflow\\.genai|from mlflow\\.tracing|mlflow\\.openai|mlflow\\.langchain|ChatCompletion|chat\\.completions)' .\n\n# Search for ML indicators\ngrep -rl --include='*.py' -E '(from sklearn|import torch|import tensorflow|import keras|import xgboost|import lightgbm|mlflow\\.sklearn|mlflow\\.pytorch|mlflow\\.tensorflow|\\.fit\\()' .\n","bash","",[426],{"type":36,"tag":169,"props":427,"children":428},{"__ignoreMap":424},[429,441,501,511,520],{"type":36,"tag":430,"props":431,"children":434},"span",{"class":432,"line":433},"line",1,[435],{"type":36,"tag":430,"props":436,"children":438},{"style":437},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[439],{"type":41,"value":440},"# Search for GenAI indicators\n",{"type":36,"tag":430,"props":442,"children":444},{"class":432,"line":443},2,[445,451,457,462,468,473,477,482,487,492,496],{"type":36,"tag":430,"props":446,"children":448},{"style":447},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[449],{"type":41,"value":450},"grep",{"type":36,"tag":430,"props":452,"children":454},{"style":453},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[455],{"type":41,"value":456}," -rl",{"type":36,"tag":430,"props":458,"children":459},{"style":453},[460],{"type":41,"value":461}," --include=",{"type":36,"tag":430,"props":463,"children":465},{"style":464},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[466],{"type":41,"value":467},"'",{"type":36,"tag":430,"props":469,"children":470},{"style":453},[471],{"type":41,"value":472},"*.py",{"type":36,"tag":430,"props":474,"children":475},{"style":464},[476],{"type":41,"value":467},{"type":36,"tag":430,"props":478,"children":479},{"style":453},[480],{"type":41,"value":481}," -E",{"type":36,"tag":430,"props":483,"children":484},{"style":464},[485],{"type":41,"value":486}," '",{"type":36,"tag":430,"props":488,"children":489},{"style":453},[490],{"type":41,"value":491},"(import openai|import anthropic|from langchain|from langgraph|import litellm|from mlflow\\.genai|from mlflow\\.tracing|mlflow\\.openai|mlflow\\.langchain|ChatCompletion|chat\\.completions)",{"type":36,"tag":430,"props":493,"children":494},{"style":464},[495],{"type":41,"value":467},{"type":36,"tag":430,"props":497,"children":498},{"style":453},[499],{"type":41,"value":500}," .\n",{"type":36,"tag":430,"props":502,"children":504},{"class":432,"line":503},3,[505],{"type":36,"tag":430,"props":506,"children":508},{"emptyLinePlaceholder":507},true,[509],{"type":41,"value":510},"\n",{"type":36,"tag":430,"props":512,"children":514},{"class":432,"line":513},4,[515],{"type":36,"tag":430,"props":516,"children":517},{"style":437},[518],{"type":41,"value":519},"# Search for ML indicators\n",{"type":36,"tag":430,"props":521,"children":523},{"class":432,"line":522},5,[524,528,532,536,540,544,548,552,556,561,565],{"type":36,"tag":430,"props":525,"children":526},{"style":447},[527],{"type":41,"value":450},{"type":36,"tag":430,"props":529,"children":530},{"style":453},[531],{"type":41,"value":456},{"type":36,"tag":430,"props":533,"children":534},{"style":453},[535],{"type":41,"value":461},{"type":36,"tag":430,"props":537,"children":538},{"style":464},[539],{"type":41,"value":467},{"type":36,"tag":430,"props":541,"children":542},{"style":453},[543],{"type":41,"value":472},{"type":36,"tag":430,"props":545,"children":546},{"style":464},[547],{"type":41,"value":467},{"type":36,"tag":430,"props":549,"children":550},{"style":453},[551],{"type":41,"value":481},{"type":36,"tag":430,"props":553,"children":554},{"style":464},[555],{"type":41,"value":486},{"type":36,"tag":430,"props":557,"children":558},{"style":453},[559],{"type":41,"value":560},"(from sklearn|import torch|import tensorflow|import keras|import xgboost|import lightgbm|mlflow\\.sklearn|mlflow\\.pytorch|mlflow\\.tensorflow|\\.fit\\()",{"type":36,"tag":430,"props":562,"children":563},{"style":464},[564],{"type":41,"value":467},{"type":36,"tag":430,"props":566,"children":567},{"style":453},[568],{"type":41,"value":500},{"type":36,"tag":139,"props":570,"children":572},{"id":571},"signal-2-check-the-experiment-type-tag",[573],{"type":41,"value":574},"Signal 2: Check the Experiment Type Tag",{"type":36,"tag":44,"props":576,"children":577},{},[578],{"type":41,"value":579},"If the codebase or project directory is the MLflow repository itself, skip to Signal 3 — the MLflow repo contains code for all use cases and does not indicate the user's intent.",{"type":36,"tag":44,"props":581,"children":582},{},[583,585,591],{"type":41,"value":584},"If the experiment ID is known, check its ",{"type":36,"tag":169,"props":586,"children":588},{"className":587},[],[589],{"type":41,"value":590},"mlflow.experimentKind",{"type":41,"value":592}," tag. This tag is set by MLflow to indicate the experiment type:",{"type":36,"tag":419,"props":594,"children":596},{"className":421,"code":595,"language":423,"meta":424,"style":424},"mlflow experiments get --experiment-id \u003CEXPERIMENT_ID> --output json > \u002Ftmp\u002Fexp_detail.json\njq -r '.tags[\"mlflow.experimentKind\"] \u002F\u002F \"not set\"' \u002Ftmp\u002Fexp_detail.json\n",[597],{"type":36,"tag":169,"props":598,"children":599},{"__ignoreMap":424},[600,663],{"type":36,"tag":430,"props":601,"children":602},{"class":432,"line":433},[603,607,612,617,622,627,632,638,643,648,653,658],{"type":36,"tag":430,"props":604,"children":605},{"style":447},[606],{"type":41,"value":8},{"type":36,"tag":430,"props":608,"children":609},{"style":453},[610],{"type":41,"value":611}," experiments",{"type":36,"tag":430,"props":613,"children":614},{"style":453},[615],{"type":41,"value":616}," get",{"type":36,"tag":430,"props":618,"children":619},{"style":453},[620],{"type":41,"value":621}," --experiment-id",{"type":36,"tag":430,"props":623,"children":624},{"style":464},[625],{"type":41,"value":626}," \u003C",{"type":36,"tag":430,"props":628,"children":629},{"style":453},[630],{"type":41,"value":631},"EXPERIMENT_I",{"type":36,"tag":430,"props":633,"children":635},{"style":634},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[636],{"type":41,"value":637},"D",{"type":36,"tag":430,"props":639,"children":640},{"style":464},[641],{"type":41,"value":642},">",{"type":36,"tag":430,"props":644,"children":645},{"style":453},[646],{"type":41,"value":647}," --output",{"type":36,"tag":430,"props":649,"children":650},{"style":453},[651],{"type":41,"value":652}," json",{"type":36,"tag":430,"props":654,"children":655},{"style":464},[656],{"type":41,"value":657}," >",{"type":36,"tag":430,"props":659,"children":660},{"style":453},[661],{"type":41,"value":662}," \u002Ftmp\u002Fexp_detail.json\n",{"type":36,"tag":430,"props":664,"children":665},{"class":432,"line":443},[666,671,676,680,685,689],{"type":36,"tag":430,"props":667,"children":668},{"style":447},[669],{"type":41,"value":670},"jq",{"type":36,"tag":430,"props":672,"children":673},{"style":453},[674],{"type":41,"value":675}," -r",{"type":36,"tag":430,"props":677,"children":678},{"style":464},[679],{"type":41,"value":486},{"type":36,"tag":430,"props":681,"children":682},{"style":453},[683],{"type":41,"value":684},".tags[\"mlflow.experimentKind\"] \u002F\u002F \"not set\"",{"type":36,"tag":430,"props":686,"children":687},{"style":464},[688],{"type":41,"value":467},{"type":36,"tag":430,"props":690,"children":691},{"style":453},[692],{"type":41,"value":662},{"type":36,"tag":50,"props":694,"children":695},{},[696,710,724],{"type":36,"tag":54,"props":697,"children":698},{},[699,708],{"type":36,"tag":58,"props":700,"children":701},{},[702],{"type":36,"tag":169,"props":703,"children":705},{"className":704},[],[706],{"type":41,"value":707},"genai_development",{"type":41,"value":709}," → GenAI use case",{"type":36,"tag":54,"props":711,"children":712},{},[713,722],{"type":36,"tag":58,"props":714,"children":715},{},[716],{"type":36,"tag":169,"props":717,"children":719},{"className":718},[],[720],{"type":41,"value":721},"custom_model_development",{"type":41,"value":723}," → Traditional ML use case",{"type":36,"tag":54,"props":725,"children":726},{},[727,732],{"type":36,"tag":58,"props":728,"children":729},{},[730],{"type":41,"value":731},"Not set",{"type":41,"value":733}," → Proceed to Signal 3",{"type":36,"tag":44,"props":735,"children":736},{},[737],{"type":41,"value":738},"If the experiment ID is not known, skip to Signal 3.",{"type":36,"tag":139,"props":740,"children":742},{"id":741},"signal-3-ask-the-user",[743],{"type":41,"value":744},"Signal 3: Ask the User",{"type":36,"tag":44,"props":746,"children":747},{},[748],{"type":41,"value":749},"If the codebase and experiment signals are inconclusive, ask directly:",{"type":36,"tag":751,"props":752,"children":753},"blockquote",{},[754],{"type":36,"tag":44,"props":755,"children":756},{},[757,759,764,766,771],{"type":41,"value":758},"Are you building a ",{"type":36,"tag":58,"props":760,"children":761},{},[762],{"type":41,"value":763},"GenAI application",{"type":41,"value":765}," (e.g., an LLM-powered chatbot, RAG pipeline, or tool-calling agent) or a ",{"type":36,"tag":58,"props":767,"children":768},{},[769],{"type":41,"value":770},"traditional ML\u002Fdeep learning model",{"type":41,"value":772}," (e.g., training a classifier, regression model, or neural network)?",{"type":36,"tag":44,"props":774,"children":775},{},[776,781],{"type":36,"tag":58,"props":777,"children":778},{},[779],{"type":41,"value":780},"Do not guess.",{"type":41,"value":782}," The onboarding paths are different enough that starting down the wrong one wastes the user's time.",{"type":36,"tag":122,"props":784,"children":786},{"id":785},"step-2-recommend-quickstart-tutorials",[787],{"type":41,"value":788},"Step 2: Recommend Quickstart Tutorials",{"type":36,"tag":44,"props":790,"children":791},{},[792],{"type":41,"value":793},"Once the use case is determined, recommend the appropriate quickstart tutorials from the MLflow documentation. Present them to the user and ask if they'd like to follow along or jump directly to integrating MLflow into their project.",{"type":36,"tag":139,"props":795,"children":797},{"id":796},"genai-path",[798],{"type":41,"value":799},"GenAI Path",{"type":36,"tag":44,"props":801,"children":802},{},[803,805],{"type":41,"value":804},"The MLflow GenAI documentation is at: ",{"type":36,"tag":806,"props":807,"children":811},"a",{"href":808,"rel":809},"https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fgenai\u002Fgetting-started\u002F",[810],"nofollow",[812],{"type":41,"value":808},{"type":36,"tag":44,"props":814,"children":815},{},[816],{"type":41,"value":817},"Choose the most relevant tutorials based on the user's context and what they've told you. Available tutorials include:",{"type":36,"tag":50,"props":819,"children":820},{},[821,869,886],{"type":36,"tag":54,"props":822,"children":823},{},[824,829,831,837,839],{"type":36,"tag":58,"props":825,"children":826},{},[827],{"type":41,"value":828},"Tracing Quickstart",{"type":41,"value":830}," (",{"type":36,"tag":806,"props":832,"children":835},{"href":833,"rel":834},"https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fgenai\u002Ftracing\u002Fquickstart\u002F",[810],[836],{"type":41,"value":833},{"type":41,"value":838},") — Enabling automatic tracing for LLM calls. Covers starting an MLflow server, creating an experiment, enabling autologging, and viewing traces in the UI.\n",{"type":36,"tag":50,"props":840,"children":841},{},[842,853,864],{"type":36,"tag":54,"props":843,"children":844},{},[845,847],{"type":41,"value":846},"Python + OpenAI variant: ",{"type":36,"tag":806,"props":848,"children":851},{"href":849,"rel":850},"https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fgenai\u002Ftracing\u002Fquickstart\u002Fpython-openai\u002F",[810],[852],{"type":41,"value":849},{"type":36,"tag":54,"props":854,"children":855},{},[856,858],{"type":41,"value":857},"TypeScript + OpenAI variant: ",{"type":36,"tag":806,"props":859,"children":862},{"href":860,"rel":861},"https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fgenai\u002Ftracing\u002Fquickstart\u002Ftypescript-openai",[810],[863],{"type":41,"value":860},{"type":36,"tag":54,"props":865,"children":866},{},[867],{"type":41,"value":868},"OpenTelemetry (language-agnostic) variant: also linked from the quickstart page",{"type":36,"tag":54,"props":870,"children":871},{},[872,877,878,884],{"type":36,"tag":58,"props":873,"children":874},{},[875],{"type":41,"value":876},"Evaluation Quickstart",{"type":41,"value":830},{"type":36,"tag":806,"props":879,"children":882},{"href":880,"rel":881},"https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fgenai\u002Feval-monitor\u002Fquickstart\u002F",[810],[883],{"type":41,"value":880},{"type":41,"value":885},") — Evaluating GenAI application quality using LLM judges (scorers). Covers defining datasets, prediction functions, and built-in + custom scorers.",{"type":36,"tag":54,"props":887,"children":888},{},[889,894,895,901],{"type":36,"tag":58,"props":890,"children":891},{},[892],{"type":41,"value":893},"Version Tracking Quickstart",{"type":41,"value":830},{"type":36,"tag":806,"props":896,"children":899},{"href":897,"rel":898},"https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fgenai\u002Fversion-tracking\u002Fquickstart\u002F",[810],[900],{"type":41,"value":897},{"type":41,"value":902},") — Prompt management, application versioning, and connecting tracing to versioned prompts.",{"type":36,"tag":44,"props":904,"children":905},{},[906],{"type":41,"value":907},"If none of these match the user's needs, look up the MLflow GenAI documentation for more relevant guides.",{"type":36,"tag":139,"props":909,"children":911},{"id":910},"traditional-ml-path",[912],{"type":41,"value":913},"Traditional ML Path",{"type":36,"tag":44,"props":915,"children":916},{},[917,919],{"type":41,"value":918},"The MLflow ML documentation is at: ",{"type":36,"tag":806,"props":920,"children":923},{"href":921,"rel":922},"https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fml\u002Fgetting-started\u002F",[810],[924],{"type":41,"value":921},{"type":36,"tag":44,"props":926,"children":927},{},[928],{"type":41,"value":817},{"type":36,"tag":50,"props":930,"children":931},{},[932,949,966],{"type":36,"tag":54,"props":933,"children":934},{},[935,940,941,947],{"type":36,"tag":58,"props":936,"children":937},{},[938],{"type":41,"value":939},"Tracking Quickstart",{"type":41,"value":830},{"type":36,"tag":806,"props":942,"children":945},{"href":943,"rel":944},"https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fml\u002Ftracking\u002Fquickstart\u002F",[810],[946],{"type":41,"value":943},{"type":41,"value":948},") — Experiment tracking with scikit-learn: autologging, manual parameter\u002Fmetric\u002Fmodel logging, and exploring results in the MLflow UI.",{"type":36,"tag":54,"props":950,"children":951},{},[952,957,958,964],{"type":36,"tag":58,"props":953,"children":954},{},[955],{"type":41,"value":956},"Deep Learning Tutorial",{"type":41,"value":830},{"type":36,"tag":806,"props":959,"children":962},{"href":960,"rel":961},"https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fml\u002Fgetting-started\u002Fdeep-learning\u002F",[810],[963],{"type":41,"value":960},{"type":41,"value":965},") — Training a PyTorch model with MLflow logging: parameters, metrics, checkpoints, and system metrics (GPU utilization, memory).",{"type":36,"tag":54,"props":967,"children":968},{},[969,974,975,981],{"type":36,"tag":58,"props":970,"children":971},{},[972],{"type":41,"value":973},"Hyperparameter Tuning Tutorial",{"type":41,"value":830},{"type":36,"tag":806,"props":976,"children":979},{"href":977,"rel":978},"https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fml\u002Fgetting-started\u002Fhyperparameter-tuning\u002F",[810],[980],{"type":41,"value":977},{"type":41,"value":982},") — Running hyperparameter searches with Optuna + MLflow, comparing results, and selecting the best model.",{"type":36,"tag":44,"props":984,"children":985},{},[986],{"type":41,"value":987},"If none of these match the user's needs, look up the MLflow ML documentation for more relevant guides.",{"type":36,"tag":122,"props":989,"children":991},{"id":990},"step-3-integrate-mlflow-into-the-users-project",[992],{"type":41,"value":993},"Step 3: Integrate MLflow into the User's Project",{"type":36,"tag":44,"props":995,"children":996},{},[997,999],{"type":41,"value":998},"After the user has reviewed the quickstart tutorials (or opted to skip them), offer to help integrate MLflow directly into their codebase. ",{"type":36,"tag":58,"props":1000,"children":1001},{},[1002],{"type":41,"value":1003},"Always ask for the user's consent before making changes to their code.",{"type":36,"tag":139,"props":1005,"children":1007},{"id":1006},"genai-integration",[1008],{"type":41,"value":1009},"GenAI Integration",{"type":36,"tag":44,"props":1011,"children":1012},{},[1013,1015,1019],{"type":41,"value":1014},"The core integration for GenAI apps is ",{"type":36,"tag":58,"props":1016,"children":1017},{},[1018],{"type":41,"value":69},{"type":41,"value":1020}," — capturing LLM calls, tool invocations, and agent steps automatically.",{"type":36,"tag":44,"props":1022,"children":1023},{},[1024,1029,1031,1037,1039,1045],{"type":36,"tag":58,"props":1025,"children":1026},{},[1027],{"type":41,"value":1028},"If asked to create an example project:",{"type":41,"value":1030}," Do not assume the user has LLM API keys (e.g., OpenAI, Anthropic). Instead, create traces with mock data using ",{"type":36,"tag":169,"props":1032,"children":1034},{"className":1033},[],[1035],{"type":41,"value":1036},"@mlflow.trace",{"type":41,"value":1038}," and ",{"type":36,"tag":169,"props":1040,"children":1042},{"className":1041},[],[1043],{"type":41,"value":1044},"mlflow.start_span()",{"type":41,"value":1046}," to demonstrate tracing without requiring external API access. For example:",{"type":36,"tag":419,"props":1048,"children":1052},{"className":1049,"code":1050,"language":1051,"meta":424,"style":424},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","import mlflow\n\nmlflow.set_experiment(\"example-genai-app\")\n\n@mlflow.trace\ndef mock_chat(query: str) -> str:\n    with mlflow.start_span(name=\"retrieve_context\") as span:\n        context = \"Mock retrieved context for: \" + query\n        span.set_inputs({\"query\": query})\n        span.set_outputs({\"context\": context})\n    with mlflow.start_span(name=\"generate_response\") as span:\n        response = \"Mock response based on: \" + context\n        span.set_inputs({\"context\": context, \"query\": query})\n        span.set_outputs({\"response\": response})\n    return response\n\nmock_chat(\"What is MLflow?\")\n","python",[1053],{"type":36,"tag":169,"props":1054,"children":1055},{"__ignoreMap":424},[1056,1064,1071,1079,1086,1094,1103,1112,1121,1130,1139,1148,1157,1166,1175,1184,1192],{"type":36,"tag":430,"props":1057,"children":1058},{"class":432,"line":433},[1059],{"type":36,"tag":430,"props":1060,"children":1061},{},[1062],{"type":41,"value":1063},"import mlflow\n",{"type":36,"tag":430,"props":1065,"children":1066},{"class":432,"line":443},[1067],{"type":36,"tag":430,"props":1068,"children":1069},{"emptyLinePlaceholder":507},[1070],{"type":41,"value":510},{"type":36,"tag":430,"props":1072,"children":1073},{"class":432,"line":503},[1074],{"type":36,"tag":430,"props":1075,"children":1076},{},[1077],{"type":41,"value":1078},"mlflow.set_experiment(\"example-genai-app\")\n",{"type":36,"tag":430,"props":1080,"children":1081},{"class":432,"line":513},[1082],{"type":36,"tag":430,"props":1083,"children":1084},{"emptyLinePlaceholder":507},[1085],{"type":41,"value":510},{"type":36,"tag":430,"props":1087,"children":1088},{"class":432,"line":522},[1089],{"type":36,"tag":430,"props":1090,"children":1091},{},[1092],{"type":41,"value":1093},"@mlflow.trace\n",{"type":36,"tag":430,"props":1095,"children":1097},{"class":432,"line":1096},6,[1098],{"type":36,"tag":430,"props":1099,"children":1100},{},[1101],{"type":41,"value":1102},"def mock_chat(query: str) -> str:\n",{"type":36,"tag":430,"props":1104,"children":1106},{"class":432,"line":1105},7,[1107],{"type":36,"tag":430,"props":1108,"children":1109},{},[1110],{"type":41,"value":1111},"    with mlflow.start_span(name=\"retrieve_context\") as span:\n",{"type":36,"tag":430,"props":1113,"children":1115},{"class":432,"line":1114},8,[1116],{"type":36,"tag":430,"props":1117,"children":1118},{},[1119],{"type":41,"value":1120},"        context = \"Mock retrieved context for: \" + query\n",{"type":36,"tag":430,"props":1122,"children":1124},{"class":432,"line":1123},9,[1125],{"type":36,"tag":430,"props":1126,"children":1127},{},[1128],{"type":41,"value":1129},"        span.set_inputs({\"query\": query})\n",{"type":36,"tag":430,"props":1131,"children":1133},{"class":432,"line":1132},10,[1134],{"type":36,"tag":430,"props":1135,"children":1136},{},[1137],{"type":41,"value":1138},"        span.set_outputs({\"context\": context})\n",{"type":36,"tag":430,"props":1140,"children":1142},{"class":432,"line":1141},11,[1143],{"type":36,"tag":430,"props":1144,"children":1145},{},[1146],{"type":41,"value":1147},"    with mlflow.start_span(name=\"generate_response\") as span:\n",{"type":36,"tag":430,"props":1149,"children":1151},{"class":432,"line":1150},12,[1152],{"type":36,"tag":430,"props":1153,"children":1154},{},[1155],{"type":41,"value":1156},"        response = \"Mock response based on: \" + context\n",{"type":36,"tag":430,"props":1158,"children":1160},{"class":432,"line":1159},13,[1161],{"type":36,"tag":430,"props":1162,"children":1163},{},[1164],{"type":41,"value":1165},"        span.set_inputs({\"context\": context, \"query\": query})\n",{"type":36,"tag":430,"props":1167,"children":1169},{"class":432,"line":1168},14,[1170],{"type":36,"tag":430,"props":1171,"children":1172},{},[1173],{"type":41,"value":1174},"        span.set_outputs({\"response\": response})\n",{"type":36,"tag":430,"props":1176,"children":1178},{"class":432,"line":1177},15,[1179],{"type":36,"tag":430,"props":1180,"children":1181},{},[1182],{"type":41,"value":1183},"    return response\n",{"type":36,"tag":430,"props":1185,"children":1187},{"class":432,"line":1186},16,[1188],{"type":36,"tag":430,"props":1189,"children":1190},{"emptyLinePlaceholder":507},[1191],{"type":41,"value":510},{"type":36,"tag":430,"props":1193,"children":1195},{"class":432,"line":1194},17,[1196],{"type":36,"tag":430,"props":1197,"children":1198},{},[1199],{"type":41,"value":1200},"mock_chat(\"What is MLflow?\")\n",{"type":36,"tag":44,"props":1202,"children":1203},{},[1204],{"type":36,"tag":58,"props":1205,"children":1206},{},[1207],{"type":41,"value":1208},"What to set up (for an existing project):",{"type":36,"tag":1210,"props":1211,"children":1212},"ol",{},[1213,1340,1375],{"type":36,"tag":54,"props":1214,"children":1215},{},[1216,1221,1223,1229,1231,1300,1304,1306,1312,1313,1319,1321,1324,1326,1331,1333,1338],{"type":36,"tag":58,"props":1217,"children":1218},{},[1219],{"type":41,"value":1220},"Autologging",{"type":41,"value":1222}," — If the user's code uses a supported framework, a single line automatically traces all calls to their LLM provider. See ",{"type":36,"tag":806,"props":1224,"children":1227},{"href":1225,"rel":1226},"https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fgenai\u002Ftracing\u002F",[810],[1228],{"type":41,"value":1225},{"type":41,"value":1230}," for the full list of supported providers. If the provider is supported:",{"type":36,"tag":419,"props":1232,"children":1234},{"className":1049,"code":1233,"language":1051,"meta":424,"style":424},"import mlflow\n\n# Pick the one that matches the user's LLM provider:\nmlflow.openai.autolog()       # OpenAI SDK\nmlflow.anthropic.autolog()    # Anthropic SDK\nmlflow.gemini.autolog()       # Google Gemini (google-genai SDK)\nmlflow.langchain.autolog()    # LangChain \u002F LangGraph\nmlflow.litellm.autolog()      # LiteLLM\n",[1235],{"type":36,"tag":169,"props":1236,"children":1237},{"__ignoreMap":424},[1238,1245,1252,1260,1268,1276,1284,1292],{"type":36,"tag":430,"props":1239,"children":1240},{"class":432,"line":433},[1241],{"type":36,"tag":430,"props":1242,"children":1243},{},[1244],{"type":41,"value":1063},{"type":36,"tag":430,"props":1246,"children":1247},{"class":432,"line":443},[1248],{"type":36,"tag":430,"props":1249,"children":1250},{"emptyLinePlaceholder":507},[1251],{"type":41,"value":510},{"type":36,"tag":430,"props":1253,"children":1254},{"class":432,"line":503},[1255],{"type":36,"tag":430,"props":1256,"children":1257},{},[1258],{"type":41,"value":1259},"# Pick the one that matches the user's LLM provider:\n",{"type":36,"tag":430,"props":1261,"children":1262},{"class":432,"line":513},[1263],{"type":36,"tag":430,"props":1264,"children":1265},{},[1266],{"type":41,"value":1267},"mlflow.openai.autolog()       # OpenAI SDK\n",{"type":36,"tag":430,"props":1269,"children":1270},{"class":432,"line":522},[1271],{"type":36,"tag":430,"props":1272,"children":1273},{},[1274],{"type":41,"value":1275},"mlflow.anthropic.autolog()    # Anthropic SDK\n",{"type":36,"tag":430,"props":1277,"children":1278},{"class":432,"line":1096},[1279],{"type":36,"tag":430,"props":1280,"children":1281},{},[1282],{"type":41,"value":1283},"mlflow.gemini.autolog()       # Google Gemini (google-genai SDK)\n",{"type":36,"tag":430,"props":1285,"children":1286},{"class":432,"line":1105},[1287],{"type":36,"tag":430,"props":1288,"children":1289},{},[1290],{"type":41,"value":1291},"mlflow.langchain.autolog()    # LangChain \u002F LangGraph\n",{"type":36,"tag":430,"props":1293,"children":1294},{"class":432,"line":1114},[1295],{"type":36,"tag":430,"props":1296,"children":1297},{},[1298],{"type":41,"value":1299},"mlflow.litellm.autolog()      # LiteLLM\n",{"type":36,"tag":1301,"props":1302,"children":1303},"br",{},[],{"type":41,"value":1305},"Add this call once at application startup (e.g., top of ",{"type":36,"tag":169,"props":1307,"children":1309},{"className":1308},[],[1310],{"type":41,"value":1311},"main.py",{"type":41,"value":176},{"type":36,"tag":169,"props":1314,"children":1316},{"className":1315},[],[1317],{"type":41,"value":1318},"app.py",{"type":41,"value":1320},", or the entry point module). It must execute before any LLM calls are made.",{"type":36,"tag":1301,"props":1322,"children":1323},{},[],{"type":41,"value":1325},"If the provider is ",{"type":36,"tag":58,"props":1327,"children":1328},{},[1329],{"type":41,"value":1330},"not",{"type":41,"value":1332}," supported by autologging, skip to step 3 (Custom tracing) and use ",{"type":36,"tag":169,"props":1334,"children":1336},{"className":1335},[],[1337],{"type":41,"value":1036},{"type":41,"value":1339}," to manually instrument the relevant functions.",{"type":36,"tag":54,"props":1341,"children":1342},{},[1343,1348,1350,1364,1367,1369],{"type":36,"tag":58,"props":1344,"children":1345},{},[1346],{"type":41,"value":1347},"Experiment configuration",{"type":41,"value":1349}," — Set the experiment so traces are organized:",{"type":36,"tag":419,"props":1351,"children":1353},{"className":1049,"code":1352,"language":1051,"meta":424,"style":424},"mlflow.set_experiment(\"my-genai-app\")\n",[1354],{"type":36,"tag":169,"props":1355,"children":1356},{"__ignoreMap":424},[1357],{"type":36,"tag":430,"props":1358,"children":1359},{"class":432,"line":433},[1360],{"type":36,"tag":430,"props":1361,"children":1362},{},[1363],{"type":41,"value":1352},{"type":36,"tag":1301,"props":1365,"children":1366},{},[],{"type":41,"value":1368},"Or via environment variable: ",{"type":36,"tag":169,"props":1370,"children":1372},{"className":1371},[],[1373],{"type":41,"value":1374},"export MLFLOW_EXPERIMENT_NAME=\"my-genai-app\"",{"type":36,"tag":54,"props":1376,"children":1377},{},[1378,1383,1385,1390,1392],{"type":36,"tag":58,"props":1379,"children":1380},{},[1381],{"type":41,"value":1382},"Custom tracing",{"type":41,"value":1384}," (optional) — For functions that aren't automatically traced (custom tools, business logic), use the ",{"type":36,"tag":169,"props":1386,"children":1388},{"className":1387},[],[1389],{"type":41,"value":1036},{"type":41,"value":1391}," decorator:",{"type":36,"tag":419,"props":1393,"children":1395},{"className":1049,"code":1394,"language":1051,"meta":424,"style":424},"@mlflow.trace\ndef my_custom_tool(query: str) -> str:\n    # ... tool logic ...\n    return result\n",[1396],{"type":36,"tag":169,"props":1397,"children":1398},{"__ignoreMap":424},[1399,1406,1414,1422],{"type":36,"tag":430,"props":1400,"children":1401},{"class":432,"line":433},[1402],{"type":36,"tag":430,"props":1403,"children":1404},{},[1405],{"type":41,"value":1093},{"type":36,"tag":430,"props":1407,"children":1408},{"class":432,"line":443},[1409],{"type":36,"tag":430,"props":1410,"children":1411},{},[1412],{"type":41,"value":1413},"def my_custom_tool(query: str) -> str:\n",{"type":36,"tag":430,"props":1415,"children":1416},{"class":432,"line":503},[1417],{"type":36,"tag":430,"props":1418,"children":1419},{},[1420],{"type":41,"value":1421},"    # ... tool logic ...\n",{"type":36,"tag":430,"props":1423,"children":1424},{"class":432,"line":513},[1425],{"type":36,"tag":430,"props":1426,"children":1427},{},[1428],{"type":41,"value":1429},"    return result\n",{"type":36,"tag":44,"props":1431,"children":1432},{},[1433,1438,1440,1446,1447,1453],{"type":36,"tag":58,"props":1434,"children":1435},{},[1436],{"type":41,"value":1437},"Where to add it:",{"type":41,"value":1439}," Find the application's entry point or initialization module and add the autologging call there. Search for the main LLM client instantiation (e.g., ",{"type":36,"tag":169,"props":1441,"children":1443},{"className":1442},[],[1444],{"type":41,"value":1445},"openai.OpenAI()",{"type":41,"value":176},{"type":36,"tag":169,"props":1448,"children":1450},{"className":1449},[],[1451],{"type":41,"value":1452},"ChatOpenAI()",{"type":41,"value":1454},") to find the right location.",{"type":36,"tag":139,"props":1456,"children":1458},{"id":1457},"traditional-ml-integration",[1459],{"type":41,"value":1460},"Traditional ML Integration",{"type":36,"tag":44,"props":1462,"children":1463},{},[1464,1466,1470],{"type":41,"value":1465},"The core integration for ML is ",{"type":36,"tag":58,"props":1467,"children":1468},{},[1469],{"type":41,"value":100},{"type":41,"value":1471}," — capturing parameters, metrics, and models from training runs.",{"type":36,"tag":44,"props":1473,"children":1474},{},[1475],{"type":36,"tag":58,"props":1476,"children":1477},{},[1478],{"type":41,"value":1479},"What to set up:",{"type":36,"tag":1210,"props":1481,"children":1482},{},[1483,1615,1648],{"type":36,"tag":54,"props":1484,"children":1485},{},[1486,1490,1492,1498,1500,1569,1572,1574,1580,1582,1585,1587,1591,1593,1599,1600,1606,1607,1613],{"type":36,"tag":58,"props":1487,"children":1488},{},[1489],{"type":41,"value":1220},{"type":41,"value":1491}," — If the user's code uses a supported framework, a single line automatically logs parameters, metrics, and models during training. See ",{"type":36,"tag":806,"props":1493,"children":1496},{"href":1494,"rel":1495},"https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fml\u002F",[810],[1497],{"type":41,"value":1494},{"type":41,"value":1499}," for the full list of supported frameworks. If the framework is supported:",{"type":36,"tag":419,"props":1501,"children":1503},{"className":1049,"code":1502,"language":1051,"meta":424,"style":424},"import mlflow\n\n# Pick the one that matches the user's ML framework:\nmlflow.sklearn.autolog()      # scikit-learn\nmlflow.pytorch.autolog()      # PyTorch \u002F PyTorch Lightning\nmlflow.tensorflow.autolog()   # TensorFlow \u002F Keras\nmlflow.xgboost.autolog()      # XGBoost\nmlflow.lightgbm.autolog()     # LightGBM\n",[1504],{"type":36,"tag":169,"props":1505,"children":1506},{"__ignoreMap":424},[1507,1514,1521,1529,1537,1545,1553,1561],{"type":36,"tag":430,"props":1508,"children":1509},{"class":432,"line":433},[1510],{"type":36,"tag":430,"props":1511,"children":1512},{},[1513],{"type":41,"value":1063},{"type":36,"tag":430,"props":1515,"children":1516},{"class":432,"line":443},[1517],{"type":36,"tag":430,"props":1518,"children":1519},{"emptyLinePlaceholder":507},[1520],{"type":41,"value":510},{"type":36,"tag":430,"props":1522,"children":1523},{"class":432,"line":503},[1524],{"type":36,"tag":430,"props":1525,"children":1526},{},[1527],{"type":41,"value":1528},"# Pick the one that matches the user's ML framework:\n",{"type":36,"tag":430,"props":1530,"children":1531},{"class":432,"line":513},[1532],{"type":36,"tag":430,"props":1533,"children":1534},{},[1535],{"type":41,"value":1536},"mlflow.sklearn.autolog()      # scikit-learn\n",{"type":36,"tag":430,"props":1538,"children":1539},{"class":432,"line":522},[1540],{"type":36,"tag":430,"props":1541,"children":1542},{},[1543],{"type":41,"value":1544},"mlflow.pytorch.autolog()      # PyTorch \u002F PyTorch Lightning\n",{"type":36,"tag":430,"props":1546,"children":1547},{"class":432,"line":1096},[1548],{"type":36,"tag":430,"props":1549,"children":1550},{},[1551],{"type":41,"value":1552},"mlflow.tensorflow.autolog()   # TensorFlow \u002F Keras\n",{"type":36,"tag":430,"props":1554,"children":1555},{"class":432,"line":1105},[1556],{"type":36,"tag":430,"props":1557,"children":1558},{},[1559],{"type":41,"value":1560},"mlflow.xgboost.autolog()      # XGBoost\n",{"type":36,"tag":430,"props":1562,"children":1563},{"class":432,"line":1114},[1564],{"type":36,"tag":430,"props":1565,"children":1566},{},[1567],{"type":41,"value":1568},"mlflow.lightgbm.autolog()     # LightGBM\n",{"type":36,"tag":1301,"props":1570,"children":1571},{},[],{"type":41,"value":1573},"Add this call once before training starts. It automatically captures ",{"type":36,"tag":169,"props":1575,"children":1577},{"className":1576},[],[1578],{"type":41,"value":1579},"model.fit()",{"type":41,"value":1581}," calls, logged metrics, and model artifacts.",{"type":36,"tag":1301,"props":1583,"children":1584},{},[],{"type":41,"value":1586},"If the framework is ",{"type":36,"tag":58,"props":1588,"children":1589},{},[1590],{"type":41,"value":1330},{"type":41,"value":1592}," supported by autologging, skip to step 3 (Manual logging) and use ",{"type":36,"tag":169,"props":1594,"children":1596},{"className":1595},[],[1597],{"type":41,"value":1598},"mlflow.log_param()",{"type":41,"value":176},{"type":36,"tag":169,"props":1601,"children":1603},{"className":1602},[],[1604],{"type":41,"value":1605},"mlflow.log_metric()",{"type":41,"value":109},{"type":36,"tag":169,"props":1608,"children":1610},{"className":1609},[],[1611],{"type":41,"value":1612},"mlflow.log_artifact()",{"type":41,"value":1614}," to log data explicitly.",{"type":36,"tag":54,"props":1616,"children":1617},{},[1618,1622,1624,1638,1641,1642],{"type":36,"tag":58,"props":1619,"children":1620},{},[1621],{"type":41,"value":1347},{"type":41,"value":1623}," — Set the experiment so runs are organized:",{"type":36,"tag":419,"props":1625,"children":1627},{"className":1049,"code":1626,"language":1051,"meta":424,"style":424},"mlflow.set_experiment(\"my-ml-experiment\")\n",[1628],{"type":36,"tag":169,"props":1629,"children":1630},{"__ignoreMap":424},[1631],{"type":36,"tag":430,"props":1632,"children":1633},{"class":432,"line":433},[1634],{"type":36,"tag":430,"props":1635,"children":1636},{},[1637],{"type":41,"value":1626},{"type":36,"tag":1301,"props":1639,"children":1640},{},[],{"type":41,"value":1368},{"type":36,"tag":169,"props":1643,"children":1645},{"className":1644},[],[1646],{"type":41,"value":1647},"export MLFLOW_EXPERIMENT_NAME=\"my-ml-experiment\"",{"type":36,"tag":54,"props":1649,"children":1650},{},[1651,1656,1658],{"type":36,"tag":58,"props":1652,"children":1653},{},[1654],{"type":41,"value":1655},"Manual logging",{"type":41,"value":1657}," (optional) — For metrics or parameters not captured by autologging:",{"type":36,"tag":419,"props":1659,"children":1661},{"className":1049,"code":1660,"language":1051,"meta":424,"style":424},"with mlflow.start_run():\n    mlflow.log_param(\"custom_param\", value)\n    mlflow.log_metric(\"custom_metric\", value)\n",[1662],{"type":36,"tag":169,"props":1663,"children":1664},{"__ignoreMap":424},[1665,1673,1681],{"type":36,"tag":430,"props":1666,"children":1667},{"class":432,"line":433},[1668],{"type":36,"tag":430,"props":1669,"children":1670},{},[1671],{"type":41,"value":1672},"with mlflow.start_run():\n",{"type":36,"tag":430,"props":1674,"children":1675},{"class":432,"line":443},[1676],{"type":36,"tag":430,"props":1677,"children":1678},{},[1679],{"type":41,"value":1680},"    mlflow.log_param(\"custom_param\", value)\n",{"type":36,"tag":430,"props":1682,"children":1683},{"class":432,"line":503},[1684],{"type":36,"tag":430,"props":1685,"children":1686},{},[1687],{"type":41,"value":1688},"    mlflow.log_metric(\"custom_metric\", value)\n",{"type":36,"tag":44,"props":1690,"children":1691},{},[1692,1696,1698,1703],{"type":36,"tag":58,"props":1693,"children":1694},{},[1695],{"type":41,"value":1437},{"type":41,"value":1697}," Find the training script or module where ",{"type":36,"tag":169,"props":1699,"children":1701},{"className":1700},[],[1702],{"type":41,"value":1579},{"type":41,"value":1704}," (or equivalent) is called. Add the autologging call before the training loop begins.",{"type":36,"tag":122,"props":1706,"children":1708},{"id":1707},"verification",[1709],{"type":41,"value":1710},"Verification",{"type":36,"tag":44,"props":1712,"children":1713},{},[1714],{"type":41,"value":1715},"After integration, verify that MLflow is capturing data correctly:",{"type":36,"tag":139,"props":1717,"children":1719},{"id":1718},"genai-verification",[1720],{"type":41,"value":1721},"GenAI Verification",{"type":36,"tag":1210,"props":1723,"children":1724},{},[1725,1730,1880],{"type":36,"tag":54,"props":1726,"children":1727},{},[1728],{"type":41,"value":1729},"Run the application and trigger at least one LLM call",{"type":36,"tag":54,"props":1731,"children":1732},{},[1733,1735],{"type":41,"value":1734},"Check for traces:\n",{"type":36,"tag":419,"props":1736,"children":1738},{"className":421,"code":1737,"language":423,"meta":424,"style":424},"mlflow traces search \\\n  --experiment-id \u003CEXPERIMENT_ID> \\\n  --max-results 5 \\\n  --extract-fields 'info.trace_id,info.state,info.request_time' \\\n  --output json > \u002Ftmp\u002Fverify_traces.json\njq '.traces | length' \u002Ftmp\u002Fverify_traces.json\n",[1739],{"type":36,"tag":169,"props":1740,"children":1741},{"__ignoreMap":424},[1742,1764,1792,1810,1835,1856],{"type":36,"tag":430,"props":1743,"children":1744},{"class":432,"line":433},[1745,1749,1754,1759],{"type":36,"tag":430,"props":1746,"children":1747},{"style":447},[1748],{"type":41,"value":8},{"type":36,"tag":430,"props":1750,"children":1751},{"style":453},[1752],{"type":41,"value":1753}," traces",{"type":36,"tag":430,"props":1755,"children":1756},{"style":453},[1757],{"type":41,"value":1758}," search",{"type":36,"tag":430,"props":1760,"children":1761},{"style":634},[1762],{"type":41,"value":1763}," \\\n",{"type":36,"tag":430,"props":1765,"children":1766},{"class":432,"line":443},[1767,1772,1776,1780,1784,1788],{"type":36,"tag":430,"props":1768,"children":1769},{"style":453},[1770],{"type":41,"value":1771},"  --experiment-id",{"type":36,"tag":430,"props":1773,"children":1774},{"style":464},[1775],{"type":41,"value":626},{"type":36,"tag":430,"props":1777,"children":1778},{"style":453},[1779],{"type":41,"value":631},{"type":36,"tag":430,"props":1781,"children":1782},{"style":634},[1783],{"type":41,"value":637},{"type":36,"tag":430,"props":1785,"children":1786},{"style":464},[1787],{"type":41,"value":642},{"type":36,"tag":430,"props":1789,"children":1790},{"style":634},[1791],{"type":41,"value":1763},{"type":36,"tag":430,"props":1793,"children":1794},{"class":432,"line":503},[1795,1800,1806],{"type":36,"tag":430,"props":1796,"children":1797},{"style":453},[1798],{"type":41,"value":1799},"  --max-results",{"type":36,"tag":430,"props":1801,"children":1803},{"style":1802},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[1804],{"type":41,"value":1805}," 5",{"type":36,"tag":430,"props":1807,"children":1808},{"style":634},[1809],{"type":41,"value":1763},{"type":36,"tag":430,"props":1811,"children":1812},{"class":432,"line":513},[1813,1818,1822,1827,1831],{"type":36,"tag":430,"props":1814,"children":1815},{"style":453},[1816],{"type":41,"value":1817},"  --extract-fields",{"type":36,"tag":430,"props":1819,"children":1820},{"style":464},[1821],{"type":41,"value":486},{"type":36,"tag":430,"props":1823,"children":1824},{"style":453},[1825],{"type":41,"value":1826},"info.trace_id,info.state,info.request_time",{"type":36,"tag":430,"props":1828,"children":1829},{"style":464},[1830],{"type":41,"value":467},{"type":36,"tag":430,"props":1832,"children":1833},{"style":634},[1834],{"type":41,"value":1763},{"type":36,"tag":430,"props":1836,"children":1837},{"class":432,"line":522},[1838,1843,1847,1851],{"type":36,"tag":430,"props":1839,"children":1840},{"style":453},[1841],{"type":41,"value":1842},"  --output",{"type":36,"tag":430,"props":1844,"children":1845},{"style":453},[1846],{"type":41,"value":652},{"type":36,"tag":430,"props":1848,"children":1849},{"style":464},[1850],{"type":41,"value":657},{"type":36,"tag":430,"props":1852,"children":1853},{"style":453},[1854],{"type":41,"value":1855}," \u002Ftmp\u002Fverify_traces.json\n",{"type":36,"tag":430,"props":1857,"children":1858},{"class":432,"line":1096},[1859,1863,1867,1872,1876],{"type":36,"tag":430,"props":1860,"children":1861},{"style":447},[1862],{"type":41,"value":670},{"type":36,"tag":430,"props":1864,"children":1865},{"style":464},[1866],{"type":41,"value":486},{"type":36,"tag":430,"props":1868,"children":1869},{"style":453},[1870],{"type":41,"value":1871},".traces | length",{"type":36,"tag":430,"props":1873,"children":1874},{"style":464},[1875],{"type":41,"value":467},{"type":36,"tag":430,"props":1877,"children":1878},{"style":453},[1879],{"type":41,"value":1855},{"type":36,"tag":54,"props":1881,"children":1882},{},[1883],{"type":41,"value":1884},"If traces appear, open the MLflow UI to inspect them visually",{"type":36,"tag":139,"props":1886,"children":1888},{"id":1887},"ml-verification",[1889],{"type":41,"value":1890},"ML Verification",{"type":36,"tag":1210,"props":1892,"children":1893},{},[1894,1899,2017],{"type":36,"tag":54,"props":1895,"children":1896},{},[1897],{"type":41,"value":1898},"Run the training script",{"type":36,"tag":54,"props":1900,"children":1901},{},[1902,1904],{"type":41,"value":1903},"Check for runs:\n",{"type":36,"tag":419,"props":1905,"children":1907},{"className":421,"code":1906,"language":423,"meta":424,"style":424},"mlflow runs search \\\n  --experiment-id \u003CEXPERIMENT_ID> \\\n  --max-results 5 \\\n  --output json > \u002Ftmp\u002Fverify_runs.json\njq '.runs | length' \u002Ftmp\u002Fverify_runs.json\n",[1908],{"type":36,"tag":169,"props":1909,"children":1910},{"__ignoreMap":424},[1911,1931,1958,1973,1993],{"type":36,"tag":430,"props":1912,"children":1913},{"class":432,"line":433},[1914,1918,1923,1927],{"type":36,"tag":430,"props":1915,"children":1916},{"style":447},[1917],{"type":41,"value":8},{"type":36,"tag":430,"props":1919,"children":1920},{"style":453},[1921],{"type":41,"value":1922}," runs",{"type":36,"tag":430,"props":1924,"children":1925},{"style":453},[1926],{"type":41,"value":1758},{"type":36,"tag":430,"props":1928,"children":1929},{"style":634},[1930],{"type":41,"value":1763},{"type":36,"tag":430,"props":1932,"children":1933},{"class":432,"line":443},[1934,1938,1942,1946,1950,1954],{"type":36,"tag":430,"props":1935,"children":1936},{"style":453},[1937],{"type":41,"value":1771},{"type":36,"tag":430,"props":1939,"children":1940},{"style":464},[1941],{"type":41,"value":626},{"type":36,"tag":430,"props":1943,"children":1944},{"style":453},[1945],{"type":41,"value":631},{"type":36,"tag":430,"props":1947,"children":1948},{"style":634},[1949],{"type":41,"value":637},{"type":36,"tag":430,"props":1951,"children":1952},{"style":464},[1953],{"type":41,"value":642},{"type":36,"tag":430,"props":1955,"children":1956},{"style":634},[1957],{"type":41,"value":1763},{"type":36,"tag":430,"props":1959,"children":1960},{"class":432,"line":503},[1961,1965,1969],{"type":36,"tag":430,"props":1962,"children":1963},{"style":453},[1964],{"type":41,"value":1799},{"type":36,"tag":430,"props":1966,"children":1967},{"style":1802},[1968],{"type":41,"value":1805},{"type":36,"tag":430,"props":1970,"children":1971},{"style":634},[1972],{"type":41,"value":1763},{"type":36,"tag":430,"props":1974,"children":1975},{"class":432,"line":513},[1976,1980,1984,1988],{"type":36,"tag":430,"props":1977,"children":1978},{"style":453},[1979],{"type":41,"value":1842},{"type":36,"tag":430,"props":1981,"children":1982},{"style":453},[1983],{"type":41,"value":652},{"type":36,"tag":430,"props":1985,"children":1986},{"style":464},[1987],{"type":41,"value":657},{"type":36,"tag":430,"props":1989,"children":1990},{"style":453},[1991],{"type":41,"value":1992}," \u002Ftmp\u002Fverify_runs.json\n",{"type":36,"tag":430,"props":1994,"children":1995},{"class":432,"line":522},[1996,2000,2004,2009,2013],{"type":36,"tag":430,"props":1997,"children":1998},{"style":447},[1999],{"type":41,"value":670},{"type":36,"tag":430,"props":2001,"children":2002},{"style":464},[2003],{"type":41,"value":486},{"type":36,"tag":430,"props":2005,"children":2006},{"style":453},[2007],{"type":41,"value":2008},".runs | length",{"type":36,"tag":430,"props":2010,"children":2011},{"style":464},[2012],{"type":41,"value":467},{"type":36,"tag":430,"props":2014,"children":2015},{"style":453},[2016],{"type":41,"value":1992},{"type":36,"tag":54,"props":2018,"children":2019},{},[2020],{"type":41,"value":2021},"If runs appear, open the MLflow UI to inspect logged parameters, metrics, and artifacts",{"type":36,"tag":2023,"props":2024,"children":2025},"style",{},[2026],{"type":41,"value":2027},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":2029,"total":1159},[2030,2045,2055,2072,2087,2098,2114,2131,2142,2148,2163,2174],{"slug":2031,"name":2031,"fn":2032,"description":2033,"org":2034,"tags":2035,"stars":2042,"repoUrl":2043,"updatedAt":2044},"setup","configure MLflow tracing","Configure MLflow tracing for Claude Code.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2036,2039,2040],{"name":2037,"slug":2038,"type":15},"Claude Code","claude-code",{"name":9,"slug":8,"type":15},{"name":2041,"slug":69,"type":15},"Tracing",27014,"https:\u002F\u002Fgithub.com\u002Fmlflow\u002Fmlflow","2026-07-14T05:39:00.297769",{"slug":2046,"name":2046,"fn":2047,"description":2048,"org":2049,"tags":2050,"stars":2042,"repoUrl":2043,"updatedAt":2054},"status","display MLflow tracing configuration","Show the current MLflow tracing configuration for Claude Code.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2051,2052,2053],{"name":2037,"slug":2038,"type":15},{"name":9,"slug":8,"type":15},{"name":2041,"slug":69,"type":15},"2026-07-14T05:39:01.540537",{"slug":2056,"name":2056,"fn":2057,"description":2058,"org":2059,"tags":2060,"stars":20,"repoUrl":21,"updatedAt":2071},"agent-evaluation","evaluate and optimize LLM agent output","Use this when you need to EVALUATE OR IMPROVE or OPTIMIZE an existing LLM agent's output quality - including improving tool selection accuracy, answer quality, reducing costs, or fixing issues where the agent gives wrong\u002Fincomplete responses. Evaluates agents systematically using MLflow evaluation with datasets, scorers, and tracing. IMPORTANT - Always also load the instrumenting-with-mlflow-tracing skill before starting any work. Covers end-to-end evaluation workflow or individual components (tracing setup, dataset creation, scorer definition, evaluation execution).",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2061,2064,2067,2068],{"name":2062,"slug":2063,"type":15},"Agents","agents",{"name":2065,"slug":2066,"type":15},"Evals","evals",{"name":9,"slug":8,"type":15},{"name":2069,"slug":2070,"type":15},"Performance","performance","2026-07-14T05:39:15.600492",{"slug":2073,"name":2073,"fn":2074,"description":2075,"org":2076,"tags":2077,"stars":20,"repoUrl":21,"updatedAt":2086},"analyzing-mlflow-session","analyze MLflow chat conversation sessions","Analyzes an MLflow session — a sequence of traces from a multi-turn chat conversation or interaction. Use when the user asks to debug a chat conversation, review session or chat history, find where a multi-turn chat went wrong, or analyze patterns across turns. Triggers on \"analyze this session\", \"what happened in this conversation\", \"debug session\", \"review chat history\", \"where did this chat go wrong\", \"session traces\", \"analyze chat\", \"debug this chat\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2078,2081,2082,2085],{"name":2079,"slug":2080,"type":15},"Debugging","debugging",{"name":9,"slug":8,"type":15},{"name":2083,"slug":2084,"type":15},"Observability","observability",{"name":2041,"slug":69,"type":15},"2026-07-14T05:39:10.542342",{"slug":2088,"name":2088,"fn":2089,"description":2090,"org":2091,"tags":2092,"stars":20,"repoUrl":21,"updatedAt":2097},"analyzing-mlflow-trace","analyze MLflow traces","Analyzes a single MLflow trace to answer a user query about it. Use when the user provides a trace ID and asks to debug, investigate, find issues, root-cause errors, understand behavior, or analyze quality. Triggers on \"analyze this trace\", \"what went wrong with this trace\", \"debug trace\", \"investigate trace\", \"why did this trace fail\", \"root cause this trace\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2093,2094,2095,2096],{"name":2079,"slug":2080,"type":15},{"name":9,"slug":8,"type":15},{"name":2083,"slug":2084,"type":15},{"name":2041,"slug":69,"type":15},"2026-07-14T05:39:02.874441",{"slug":2099,"name":2099,"fn":2100,"description":2101,"org":2102,"tags":2103,"stars":20,"repoUrl":21,"updatedAt":2113},"fix-agent-issue","fix and update AI agent behavior","Drives a disciplined explore → plan → implement → verify loop for changing an AI agent's behavior with confidence — whether fixing a reported failure or introducing a new requirement, business rule, or policy. Grounds the diagnosis in MLflow traces, codifies the desired behavior as a regression test suite (`mlflow.genai.evaluate` assertions in `@mlflow.test` pytest tests), and iterates the agent — not the test — until green, resisting quick system-prompt patches when the real fix is upstream (missing tool, retrieval source, or capability). Use whenever the user wants to fix or change how an agent behaves — e.g. \"fix this issue in my agent\", \"this answer is wrong\", \"the agent is hallucinating\", \"improve my agent based on this trace\", \"make the agent do X instead of Y\", \"I want the agent to lead with\u002Fprioritize\u002Frecommend X\", \"new business rule: the agent should X\", \"always\u002Fnever do X\", \"change the agent's default behavior\" — or shares a trace they want addressed.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2104,2105,2108,2109,2112],{"name":2062,"slug":2063,"type":15},{"name":2106,"slug":2107,"type":15},"Best Practices","best-practices",{"name":2079,"slug":2080,"type":15},{"name":2110,"slug":2111,"type":15},"Engineering","engineering",{"name":9,"slug":8,"type":15},"2026-07-30T05:53:39.749464",{"slug":2115,"name":2115,"fn":2116,"description":2117,"org":2118,"tags":2119,"stars":20,"repoUrl":21,"updatedAt":2130},"instrumenting-with-mlflow-tracing","instrument Python and TypeScript with MLflow Tracing","Instruments Python and TypeScript code with MLflow Tracing for observability. Must be loaded when setting up tracing as part of any workflow including agent evaluation. Triggers on adding tracing, instrumenting agents\u002FLLM apps, getting started with MLflow tracing, tracing specific frameworks (LangGraph, LangChain, OpenAI, Gemini, DSPy, CrewAI, AutoGen), or when another skill references tracing setup. Examples - \"How do I add tracing?\", \"Instrument my agent\", \"Trace my LangChain app\", \"Set up tracing for evaluation\"",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2120,2121,2122,2123,2124,2126,2127],{"name":2062,"slug":2063,"type":15},{"name":2065,"slug":2066,"type":15},{"name":9,"slug":8,"type":15},{"name":2083,"slug":2084,"type":15},{"name":2125,"slug":1051,"type":15},"Python",{"name":2041,"slug":69,"type":15},{"name":2128,"slug":2129,"type":15},"TypeScript","typescript","2026-07-20T05:58:52.968218",{"slug":2132,"name":2132,"fn":2133,"description":2134,"org":2135,"tags":2136,"stars":20,"repoUrl":21,"updatedAt":2141},"mlflow-agent","dispatch MLflow workflows and agent tasks","Master dispatcher for all MLflow workflows. Use this skill when the user wants to do anything with MLflow — tracing, evaluating, debugging, or improving an agent. Routes to the right MLflow sub-skill automatically. Triggers on: \"use mlflow\", \"help with mlflow\", \"mlflow agent\", \"add mlflow to my project\", \"trace my agent\", \"evaluate my agent\", or any MLflow task without a specific skill in mind.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2137,2138,2139,2140],{"name":2062,"slug":2063,"type":15},{"name":9,"slug":8,"type":15},{"name":18,"slug":19,"type":15},{"name":2041,"slug":69,"type":15},"2026-07-14T05:39:04.133424",{"slug":4,"name":4,"fn":5,"description":6,"org":2143,"tags":2144,"stars":20,"repoUrl":21,"updatedAt":22},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2145,2146,2147],{"name":9,"slug":8,"type":15},{"name":18,"slug":19,"type":15},{"name":13,"slug":14,"type":15},{"slug":2149,"name":2149,"fn":2150,"description":2151,"org":2152,"tags":2153,"stars":20,"repoUrl":21,"updatedAt":2162},"querying-mlflow-metrics","fetch trace metrics from MLflow servers","Fetches aggregated trace metrics (token usage, latency, trace counts, quality evaluations) from MLflow tracking servers. Triggers on requests to show metrics, analyze token usage, view LLM costs, check usage trends, or query trace statistics.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2154,2157,2160,2161],{"name":2155,"slug":2156,"type":15},"Analytics","analytics",{"name":2158,"slug":2159,"type":15},"Metrics","metrics",{"name":9,"slug":8,"type":15},{"name":2083,"slug":2084,"type":15},"2026-07-14T05:39:13.07056",{"slug":2164,"name":2164,"fn":2165,"description":2166,"org":2167,"tags":2168,"stars":20,"repoUrl":21,"updatedAt":2173},"retrieving-mlflow-traces","retrieve and query MLflow traces","Retrieves MLflow traces using CLI or Python API. Use when the user asks to get a trace by ID, find traces, filter traces by status\u002Ftags\u002Fmetadata\u002Fexecution time, query traces, or debug failed traces. Triggers on \"get trace\", \"search traces\", \"find failed traces\", \"filter traces by\", \"traces slower than\", \"query MLflow traces\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2169,2170,2171,2172],{"name":2079,"slug":2080,"type":15},{"name":9,"slug":8,"type":15},{"name":2083,"slug":2084,"type":15},{"name":2041,"slug":69,"type":15},"2026-07-14T05:39:09.22888",{"slug":2175,"name":2175,"fn":2176,"description":2177,"org":2178,"tags":2179,"stars":20,"repoUrl":21,"updatedAt":2188},"sagemaker-mlflow","connect to SageMaker Managed MLflow","Connect to SageMaker Managed MLflow (mlflow-app or mlflow-tracking-server ARN) as an MLflow backend, then hand off to the other MLflow skills. Triggers on a SageMaker MLflow ARN (arn:aws:sagemaker:...:mlflow-app\u002F... or arn:aws:sagemaker:...:mlflow-tracking-server\u002F...) or \"SageMaker Managed MLflow\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2180,2183,2186,2187],{"name":2181,"slug":2182,"type":15},"AI Infrastructure","ai-infrastructure",{"name":2184,"slug":2185,"type":15},"AWS","aws",{"name":9,"slug":8,"type":15},{"name":18,"slug":19,"type":15},"2026-07-14T05:39:05.401801",{"items":2190,"total":1141},[2191,2198,2205,2212,2220,2230,2237],{"slug":2056,"name":2056,"fn":2057,"description":2058,"org":2192,"tags":2193,"stars":20,"repoUrl":21,"updatedAt":2071},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2194,2195,2196,2197],{"name":2062,"slug":2063,"type":15},{"name":2065,"slug":2066,"type":15},{"name":9,"slug":8,"type":15},{"name":2069,"slug":2070,"type":15},{"slug":2073,"name":2073,"fn":2074,"description":2075,"org":2199,"tags":2200,"stars":20,"repoUrl":21,"updatedAt":2086},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2201,2202,2203,2204],{"name":2079,"slug":2080,"type":15},{"name":9,"slug":8,"type":15},{"name":2083,"slug":2084,"type":15},{"name":2041,"slug":69,"type":15},{"slug":2088,"name":2088,"fn":2089,"description":2090,"org":2206,"tags":2207,"stars":20,"repoUrl":21,"updatedAt":2097},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2208,2209,2210,2211],{"name":2079,"slug":2080,"type":15},{"name":9,"slug":8,"type":15},{"name":2083,"slug":2084,"type":15},{"name":2041,"slug":69,"type":15},{"slug":2099,"name":2099,"fn":2100,"description":2101,"org":2213,"tags":2214,"stars":20,"repoUrl":21,"updatedAt":2113},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2215,2216,2217,2218,2219],{"name":2062,"slug":2063,"type":15},{"name":2106,"slug":2107,"type":15},{"name":2079,"slug":2080,"type":15},{"name":2110,"slug":2111,"type":15},{"name":9,"slug":8,"type":15},{"slug":2115,"name":2115,"fn":2116,"description":2117,"org":2221,"tags":2222,"stars":20,"repoUrl":21,"updatedAt":2130},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2223,2224,2225,2226,2227,2228,2229],{"name":2062,"slug":2063,"type":15},{"name":2065,"slug":2066,"type":15},{"name":9,"slug":8,"type":15},{"name":2083,"slug":2084,"type":15},{"name":2125,"slug":1051,"type":15},{"name":2041,"slug":69,"type":15},{"name":2128,"slug":2129,"type":15},{"slug":2132,"name":2132,"fn":2133,"description":2134,"org":2231,"tags":2232,"stars":20,"repoUrl":21,"updatedAt":2141},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2233,2234,2235,2236],{"name":2062,"slug":2063,"type":15},{"name":9,"slug":8,"type":15},{"name":18,"slug":19,"type":15},{"name":2041,"slug":69,"type":15},{"slug":4,"name":4,"fn":5,"description":6,"org":2238,"tags":2239,"stars":20,"repoUrl":21,"updatedAt":22},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2240,2241,2242],{"name":9,"slug":8,"type":15},{"name":18,"slug":19,"type":15},{"name":13,"slug":14,"type":15}]