[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-nvidia-tao-finetune-huggingface-model":3,"mdc--kd1eo3-key":37,"related-repo-nvidia-tao-finetune-huggingface-model":2842,"related-org-nvidia-tao-finetune-huggingface-model":2948},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":26,"repoUrl":27,"updatedAt":28,"license":29,"forks":30,"topics":31,"repo":32,"sourceUrl":35,"mdContent":36},"tao-finetune-huggingface-model","fine-tune Hugging Face models on NVIDIA GPUs","Fine-tune any HuggingFace CV \u002F VLM \u002F LLM model on local NVIDIA GPUs inside an NGC PyTorch container. Use when the user wants to fine-tune a HuggingFace model (full or LoRA), train a vision \u002F VLM \u002F LLM model end-to-end, generate a reproducible HF training pipeline, smoke-test a HuggingFace model locally before scale-up, push a fine-tuned model to the HF Hub with a model card, or emit a self-contained rerun skill for an existing HuggingFace finetune. Supports image classification, object detection, semantic \u002F instance \u002F panoptic segmentation, depth estimation, image-text-to-text VLM (SFT \u002F LoRA), and LLM SFT \u002F DPO \u002F GRPO. Six-step workflow: inspect and qualify, hardware and NGC image, research, generate and smoke, train + eval + infer, push and emit rerun skill.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},"nvidia","NVIDIA","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fnvidia.png",[12,16,19,22,25],{"name":13,"slug":14,"type":15},"Hugging Face","hugging-face","tag",{"name":17,"slug":18,"type":15},"LLM","llm",{"name":20,"slug":21,"type":15},"Deep Learning","deep-learning",{"name":23,"slug":24,"type":15},"Machine Learning","machine-learning",{"name":9,"slug":8,"type":15},2473,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fskills","2026-07-14T05:26:21.432694","Apache-2.0",281,[],{"repoUrl":27,"stars":26,"forks":30,"topics":33,"description":34},[],"AI agent skills published by NVIDIA","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fskills\u002Ftree\u002FHEAD\u002Fskills\u002Ftao-finetune-huggingface-model","---\nname: tao-finetune-huggingface-model\ndescription: >\n  Fine-tune any HuggingFace CV \u002F VLM \u002F LLM model on local NVIDIA GPUs inside an\n  NGC PyTorch container. Use when the user wants to fine-tune a HuggingFace\n  model (full or LoRA), train a vision \u002F VLM \u002F LLM model end-to-end, generate a\n  reproducible HF training pipeline, smoke-test a HuggingFace model locally\n  before scale-up, push a fine-tuned model to the HF Hub with a model card, or\n  emit a self-contained rerun skill for an existing HuggingFace finetune.\n  Supports image classification, object detection, semantic \u002F instance \u002F\n  panoptic segmentation, depth estimation, image-text-to-text VLM (SFT \u002F LoRA),\n  and LLM SFT \u002F DPO \u002F GRPO. Six-step workflow: inspect and qualify, hardware\n  and NGC image, research, generate and smoke, train + eval + infer, push and\n  emit rerun skill.\nlicense: Apache-2.0\ntags:\n  - finetuning\n  - huggingface\n  - nvidia-tao\n  - computer-vision\n  - training\ncompatibility: Requires docker + nvidia-container-toolkit, NVIDIA GPU (driver ≥ 545, ≥ 24 GB VRAM for ≤3B models), ~40 GB free disk. Optional credentials (read from the session environment, exported before launching) — HF_TOKEN is read only when the model\u002Fdataset is gated or `push_to_hub` is on; WANDB_API_KEY and WANDB_PROJECT only when WandB logging is enabled.\nmetadata:\n  author: NVIDIA Corporation\n  version: \"0.1.0\"\nallowed-tools: Read Bash Write\n---\n\u003C!-- Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. Licensed under the Apache License, Version 2.0; see http:\u002F\u002Fwww.apache.org\u002Flicenses\u002FLICENSE-2.0 -->\n\n# tao-finetune-huggingface-model\n\nLocal NVIDIA GPU fine-tuning for HuggingFace models, grounded in live-fetched\ndocumentation with curated references as a fallback safety net. One NGC container,\na few focused scripts, one push to HF Hub. Follow the rules in this file; don't\nimprovise.\n\n**Order of authority (highest first):**\n\n1. **User input** — explicit `model_id`, `dataset_id`, `training_method`, `config.yaml` overrides.\n2. **Live research** — model card, HF repo example, author finetune script, HF task docs, paper; always fetched (Step 3 + `references\u002Fresearch-priorities.md`).\n3. **Curated references** (`references\u002F*.md`) — fallback when live research is silent\u002Fambiguous.\n4. **Your training-data memory** — last resort; suspect, cross-check against (2)\u002F(3).\n\nConflict resolution between (2) and (3) and the source-line discrepancy note are\nin `references\u002Fresearch-priorities.md`.\n\n---\n\n## Inputs\n\n**Required:**\n- `model_id` — HuggingFace model ID, e.g. `google\u002Fvit-base-patch16-224`\n\n**Conditional credentials (read from the session environment, exported before launching when present):**\n- `HF_TOKEN` — only when the model\u002Fdataset is **gated** (read) or `push_to_hub` is on (write); public + public + `push_to_hub: false` needs none. Value never read — presence-only via `[ -n \"$HF_TOKEN\" ]`.\n- `WANDB_API_KEY`, `WANDB_PROJECT` — only when WandB is enabled; `WANDB_MODE=disabled` opts out.\n\n**Dataset — exactly one:**\n- `dataset_id` — HuggingFace dataset ID *(source: `hf`)*\n- `local_dataset_path` — local folder or file *(source: `local`)*; optional\n  `local_dataset_format` ∈ {auto, imagefolder, coco, voc, jsonl, arrow, parquet,\n  csv} (default: auto-detect).\n- *(omit)* — agent recommends popular datasets *(source: `recommend`)*\n\n**Optional (have defaults):**\n- `task_type` — auto-detected from config + model card\n- `n_train=10000`, `n_eval=1000`, `n_epochs=3`, `lora_r=16`\n- `output_dir=.\u002Foutput\u002F\u003Cmodel_short_name>`\n- `hf_model_repo` — push target; if unset and HF_TOKEN has write access,\n  auto-derived as `\u003Cwhoami>\u002F\u003Cmodel_short_name>-finetuned`.\n- `push_to_hub=True` — set to `False` to skip\n- `skip_baseline=False` — skip zero-shot baseline eval\n\n**Optional deliverables (off by default):**\n```yaml\nemit_progress_log: false   # output_dir\u002FPROGRESS.md (per-step journal)\nemit_report:       false   # reports\u002Freport.{pdf,html} with curves & samples\nemit_unit_tests:   false   # tests\u002F with fake-data heterogeneous-batch tests\n```\n\nAll values live in `output_dir\u002Fconfig.yaml`. Never hardcode in Python.\n\n---\n\n## Execution platform\n\nThis skill orchestrates *what* to run; the platform skills own *how* to run it on\na GPU host — read them first.\n\n| Concern | Authoritative skill |\n|---|---|\n| GPU host runtime (driver 580, CUDA Toolkit 13.0, NVIDIA Container Toolkit 1.19.0) | [`tao-skill-bank:tao-setup-nvidia-gpu-host`](..\u002F..\u002Fplatform\u002Ftao-setup-nvidia-gpu-host\u002FSKILL.md) |\n| `docker run` flags, NGC auth, mounts, env passthrough | [`tao-skill-bank:tao-run-on-docker`](..\u002F..\u002Fplatform\u002Ftao-run-on-docker\u002FSKILL.md) |\n| Local Docker job preflight (daemon, GPU smoke) | [`tao-skill-bank:tao-run-on-local-docker`](..\u002F..\u002Fplatform\u002Ftao-run-on-local-docker\u002FSKILL.md) |\n\n**Default platform:** `local-docker` — build a one-off image (`run-\u003Cshort>:latest`)\nand run it on the local Docker daemon. Ask only when the user explicitly needs a\ndifferent backend (Brev remote GPU, SLURM\u002FKubernetes); then run that platform's\nPreflight first and route the Steps 4–5 `docker run` commands through it. The\nGPU-runtime and presence-only credential preflights (values never read), the\ncanonical `docker run` flag set, the `list_tao_platforms.py` selection command, and\nthe workflow-specific flags (`--entrypoint \u002Fbin\u002Fbash -lc`, `PYTORCH_CUDA_ALLOC_CONF`,\n`--name hft_train`) are in `references\u002Fworkflow-intake-preflight.md`.\n\n---\n\n## References — fallback safety net\n\nConsulted **only** when live research is silent, ambiguous, or unavailable; live\ndocs always win for the specific model and current API. Each step links the\nreferences it needs; full catalog in `references\u002Fdetailed-workflow.md`.\n\nAlways-on: `core-rules.md`, `error-playbook.md`, `compat-workarounds.md`,\n`model-discovery.md`, `dataset-recommendations.md`, `dataset-sources.md`,\n`dataset-patterns.md`, `hardware-container.md`, `research-priorities.md`,\n`cv-scripts.md`, `vlm-scripts.md`, `docker-runs.md`, `hub-push.md`,\n`pipeline-skill-template.md`, `deliverables.md`. Opt-in (when their flag\u002Fneed\napplies): `progress-tracking.md`, `testing.md`, `reporting.md`,\n`workflow-intake-preflight.md`, `workflow-generate-train.md`, `workflow-push-rerun.md`.\n\n**Rule:** before falling back, log the live source you tried and why it was\ninsufficient (`config.yaml` `notes:`, and PROGRESS.md if enabled). `[FETCH LIVE]`\nmarkers in `cv-scripts.md` \u002F `vlm-scripts.md` are a research checklist, not code to\ninline — refetch the listed URL if a block has no Step 3 finding.\n\n---\n\n## Core rules\n\nNon-negotiable behaviors. **Short version** (full enumeration —\nhallucinated-imports list, never-without-approval list, full error-recovery and\nhardware-sizing tables — in `references\u002Fcore-rules.md`, consult before any\ntraining-time decision):\n\n- **Your HF-library knowledge is outdated.** Fetch live docs (model card, HF\n  repo example, task doc) before writing any ML code — don't generate trainer\n  args \u002F collator \u002F transforms from memory (Step 3).\n- **Smoke-test on real data with `--max_steps 1`** before any full run; no batch\n  launches without a verified smoke.\n- **Never silently substitute** model_id, dataset_id, or training_method — if\n  what the user asked for doesn't load, stop and ask.\n- **Error recovery is minimal-change.** OOM → halve batch, double grad_accum,\n  enable gradient checkpointing (no LoRA switch without approval); NaN → reduce\n  LR 10×; flat loss → inspect collator; same error 3× → stop and ask. Don't loop.\n- **Dataset columns verified BEFORE the collator** — rename in `prepare_data.py`;\n  restructuring needed → stop and ask.\n- **Hardware-sizing thumb (bf16):** ≤3B → 24 GB, 7–13B → 80 GB, 30B+ → multi-GPU\n  or LoRA on 1× 80 GB, 70B+ → 8× 80 GB or LoRA. Full finetune won't fit and no\n  LoRA requested → ask before switching.\n\n---\n\n## Workflow — 6 steps\n\nSingle pass, sequential; each step has a clear gate before the next begins.\n\n### Step 1 — Inspect & qualify\n\n**Goal:** decide whether to proceed. Probe model + dataset, apply accept\u002Freject,\nregister applicable compat fixes, write the initial `config.yaml`.\n\nPrerequisites: `MODEL_ID`, optional `DATASET_ID` \u002F `local_dataset_path`,\noptional `HF_TOKEN`, `OUTPUT_DIR` (default `.\u002Foutput\u002F\u003Cmodel_short_name>`). Probes\nrun in a CPU-only `python:3.12-slim` Docker container (bind-mounted `.probe\u002F`\nscratch) so the host needs no virtualenv — Docker must exist first. Docker-presence\nguard, container env, full probe invocation, and the model\u002Fdataset probe scripts\nare in `references\u002Fworkflow-intake-preflight.md`, `references\u002Fmodel-discovery.md`,\nand `references\u002Fdataset-sources.md`.\n\nProbe requirements:\n\n- Model: load `AutoConfig`, read model-card tags, detect task from\n  `architectures` + tags + card examples (fallback logging in `model-discovery.md`).\n- Dataset: for recommended datasets, first present 3-5 choices from\n  `dataset-recommendations.md`; for local data, bind-mount read-only and use\n  `dataset-sources.md` format detection.\n- Reject early if the model config fails, the task is out of scope, no recipe\n  source exists, or the dataset cannot load \u002F match the task schema.\n- Evaluate `compat-workarounds.md` against the model\u002Ftask; defer hardware-dependent\n  rules to Step 2.\n\nWrite the initial `config.yaml` (`model_id`, `task`, `dataset_id` or\n`local_dataset_path`, `research_sources: []` filled in Step 3,\n`applicable_workarounds:` from Step 1, `notes: []` for reference fallbacks,\n`push_to_hub: true` default — annotated template in\n`references\u002Fworkflow-intake-preflight.md`). Optionally `rm -rf \"$OUTPUT_DIR\u002F.probe\"`\nonce the gate is met.\n\n**Gate:** `config.yaml` exists with model, dataset, task, applicable_workarounds;\ndo not proceed if any field is missing.\n\n---\n\n### Step 2 — Hardware audit & NGC image\n\n**Goal:** verify Docker + GPU + disk, pick the NGC PyTorch image live, finalize\nhardware-dependent compat rules.\n\n**2a. Audit (hard gate)** — three checks (commands in\n`references\u002Fworkflow-intake-preflight.md`):\n1. GPU host runtime — `tao-setup-nvidia-gpu-host`'s\n   `setup-nvidia-gpu-host.sh --backend docker --check-only`; on fail, ask approval\n   then re-run with `--install --yes`.\n2. Free-disk soft-warn — override via `MIN_DISK_GB` (default 100 GB); recommend\n   ≥ 100 GB for NGC base (~20 GB) + HF cache + checkpoints + data.\n3. Conditional credential presence (from the session environment, values never\n   read) — `HF_TOKEN` only when gated or `push_to_hub` is on; `WANDB_*` only when\n   WandB is on.\n\n**Do not proceed to Step 4 on a hard-fail** — Step 4's `docker build` pulls a\n20+ GB NGC base, and a missing `nvidia-container-toolkit` only surfaces later as\n`could not select device driver \"\" with capabilities: [[gpu]]`. Record `gpu_count`,\n`gpu_name`, `driver_major`, `vram_gb_per_gpu` in `config.yaml`.\n\n**2b. Pick NGC image (live):** from the NVIDIA Deep Learning Frameworks support\nmatrix (\u003Chttps:\u002F\u002Fdocs.nvidia.com\u002Fdeeplearning\u002Fframeworks\u002Fsupport-matrix\u002Findex.html>),\nPyTorch NGC container section, pick the highest-versioned image where\n`Min driver ≤ detected driver_major` and container CUDA `≤` host CUDA Toolkit\n(match closely so cuDNN \u002F TensorRT line up). Do **not** reject an image for an\n`aN`\u002F`bN`\u002F`rcN` PyTorch tag — NGC validates the full image; pick the newest\nCUDA-aligned one and let `compat-workarounds.md` handle per-version issues. If the\nmatrix is unreachable, use the fallbacks in `references\u002Fhardware-container.md`;\ndefault `nvcr.io\u002Fnvidia\u002Fpytorch:24.09-py3` (driver ≥ 545; SDPA+GQA bug — if\n`num_key_value_heads \u003C num_attention_heads`, set `attn_implementation: \"eager\"`).\nRecord `ngc_image` in `config.yaml`.\n\n**2c. Re-evaluate hardware-dependent compat rules:** re-run the\n`compat-workarounds.md` walk for entries whose `detect` needs `hw`; update\n`applicable_workarounds:` in place.\n\n**2d. Model-fit check:** estimate `param_bytes ≈ 2×param_count` (bf16); if\n> 60% of `vram_gb_per_gpu × 1e9`, recommend LoRA in the user-facing summary.\n\n**Gate:** `config.yaml` has `ngc_image`, `gpu_count`, `gpu_name`, `driver_major`,\n`vram_gb_per_gpu`; hardware-dependent compat fixes recorded.\n\n---\n\n### Step 3 — Research the recipe\n\n**Goal:** fetch the live recipe — training-data knowledge of\n`transformers`\u002F`trl`\u002F`peft` is suspect, so Step 3 is non-negotiable. Walk\n`references\u002Fresearch-priorities.md` in priority order (Priority 1 → 6); stop once\nyou have, for the detected task:\n\n- `AutoModel` \u002F processor class\n- Train + eval transforms\n- Collator\n- `compute_metrics`\n- Hyperparameter hints (LR, batch size, epochs, scheduler)\n\nRecord findings in `meta\u002Frecipe.md`, append source URLs to\n`config.yaml: research_sources:`. A slot with no live finding falls back to the\nmatching scaffold (`cv-scripts.md` \u002F `vlm-scripts.md`), logged as \"fallback to\nscaffold — no live source for \u003Cslot>\" under `notes:`. Conflict-resolution rules\nare in `references\u002Fresearch-priorities.md`.\n\n**Gate:** every required slot filled, with a source URL or scaffold-fallback note.\n\n---\n\n### Step 4 — Generate project & smoke-test\n\n**Goal:** write all scripts, build the image, prepare data, run a 1-step smoke on\nreal data (one `docker build`, two `docker run`s).\n\n**4a. Generate project files** in `output_dir\u002F`: `config.yaml`, `Dockerfile`,\n`requirements.txt`, `prepare_data.py`, `train.py`, `run_eval.py`, `infer.py`,\noptional `merge_lora.py`, optional `tests\u002F`, `.gitignore`. Live Step 3 research is\nauthority; `cv-scripts.md` \u002F `vlm-scripts.md` give scaffold shape only. Apply every\n`applicable_workarounds` entry as a Dockerfile block, requirement pin, config\noverride, or runtime env var. Hard rules: `run_eval.py` keeps that exact filename\n(avoids colliding with the HF `evaluate` package); every generated `.py` starts\nwith the NVIDIA Apache-2.0 copyright header and any emitter fails when it is\nmissing; `emit_unit_tests: true` generates and runs tests per\n`references\u002Ftesting.md`. Script bodies, Dockerfile shape, and the emitter contract\nare in `references\u002Fworkflow-generate-train.md`.\n\n**4b. Build, prepare, smoke** — `docker build -t run-\u003Cshort>:latest .`, then\n`prepare_data` and the `--smoke --max_steps 1` run (`references\u002Fdocker-runs.md`\n§1-3). Smoke pass criteria (in `logs\u002Fsmoke.log`):\n- No exception\n- Loss is finite (not `0.0`, not `NaN`)\n- `grad_norm > 0` at step 1\n\nIf `emit_unit_tests: true`, also run `pytest tests\u002F` in the container. Any failure → STOP.\n\n**4c. Preflight summary** — before full training, print and verify: reference URL,\ndataset columns, Hub target, monitoring target, NGC image, hardware, smoke loss\u002Fgrad norm.\n\n**Gate:** project files written, image built, smoke PASSED, preflight has no\nblank fields.\n\n---\n\n### Step 5 — Train, evaluate, infer\n\n**Goal:** baseline eval, full training, post-train eval, optional LoRA merge, 5\ninference samples (all commands: `references\u002Fdocker-runs.md` §4-8).\n\n| Sub-step | docker-runs.md | Skip if |\n|---|---|---|\n| 5a. Baseline eval (zero-shot) | §4 | `skip_baseline: true` |\n| 5b. Full training (detached) | §5 | — |\n| 5c. LoRA merge | §6 | not VLM+LoRA |\n| 5d. Post-train eval | §7 | — |\n| 5e. Inference (5 samples) | §8 | — |\n\nMulti-GPU: prepend `torchrun --nproc_per_node=$gpu_count` to `python train.py`.\n\nWhile training streams, watch `docker logs -f hft_train`: loss should drop within\n10-20 steps; flat loss (collator\u002Flabel-masking bug), NaN (LR too high), and OOM\nall stop the run — recovery in `references\u002Fcore-rules.md`. If `emit_report: true`,\nrun `report.py` after Step 5e per `references\u002Freporting.md`.\n\n**Gate:** all of:\n- `checkpoints\u002Ffinal\u002F` (or `checkpoints\u002Fmerged\u002F` for LoRA) exists\n- `reports\u002Feval_results.json` has a numeric primary metric\n- `reports\u002Fbaseline_results.json` exists (unless skipped)\n- `reports\u002Finference_samples\u002F` has 5 samples\n- wandb URL shows descending loss\n\n---\n\n### Step 6 — Push & emit rerun skill\n\n**Goal:** publish the run and make it reproducible without re-research.\n\nPush per `references\u002Fhub-push.md` (weights, model card, eval\u002Fbaseline JSONs,\n`config.yaml`, `Dockerfile`, `requirements.txt`, inference samples, reports when\nemitted) unless `push_to_hub: false` is explicit. Emit\n`\u003Coutput_dir>\u002Fskills\u002Frun-\u003Cshort>\u002FSKILL.md` from\n`references\u002Fpipeline-skill-template.md` — substitute every placeholder, include\nfull YAML metadata + the NVIDIA copyright HTML comment, and make any emitter fail\nif those are missing.\n\n**Gate (Done criteria):** all of:\n- Step 5 gate met\n- HF Hub repo exists at the resolved URL with weights + card + `results\u002F`\n  (unless `push_to_hub: false`)\n- `\u003Coutput_dir>\u002Fskills\u002Frun-\u003Cshort>\u002FSKILL.md` exists, no `\u003Cplaceholder>` left,\n  with metadata + copyright HTML comment per `pipeline-skill-template.md`\n\nFinal message: wandb URL, HF Hub URL, baseline -> fine-tuned primary metric,\n`reports\u002Finference_samples\u002F`, and the rerun skill path.\n\n---\n\n## Error playbook\n\nOn a known runtime error, consult the symptom → minimal-fix table in\n`references\u002Ferror-playbook.md` (NGC entrypoint, PyTorch\u002FTransformers regressions,\nnumpy ABI, Albumentations bbox, PEFT\u002Fcheckpointing, LoRA target breadth, CV\naugmentation gaps, OOM at step 0) before redesigning anything. When a row there\nfires twice across runs, lift it into `compat-workarounds.md` with a `detect` rule\n— auto-applied in Step 1 before the error can fire.\n\n---\n\n## Communication style\n\n- Terse. No filler, no restating the request; one-word answers when appropriate.\n- Always include direct Hub and wandb URLs when referencing artifacts.\n- On error: state what went wrong, why, what you changed — no menus.\n- Never present \"Option A\u002FB\u002FC\" for a request with a clear answer. Act.\n\n## Example pipelines\n\n- [tao-rerun-convnext-cifar10](references\u002Ftao-rerun-convnext-cifar10.md)\n- [tao-rerun-detr-cppe5](references\u002Ftao-rerun-detr-cppe5.md)\n- [tao-rerun-segformer-foodseg103](references\u002Ftao-rerun-segformer-foodseg103.md)\n- [tao-rerun-smolvlm-vqav2](references\u002Ftao-rerun-smolvlm-vqav2.md)\n",{"data":38,"body":50},{"name":4,"description":6,"license":29,"tags":39,"compatibility":45,"metadata":46,"allowed-tools":49},[40,41,42,43,44],"finetuning","huggingface","nvidia-tao","computer-vision","training","Requires docker + nvidia-container-toolkit, NVIDIA GPU (driver ≥ 545, ≥ 24 GB VRAM for ≤3B models), ~40 GB free disk. Optional credentials (read from the session environment, exported before launching) — HF_TOKEN is read only when the model\u002Fdataset is gated or `push_to_hub` is on; WANDB_API_KEY and WANDB_PROJECT only when WandB logging is enabled.",{"author":47,"version":48},"NVIDIA Corporation","0.1.0","Read Bash Write",{"type":51,"children":52},"root",[53,60,66,75,167,179,183,190,198,218,226,296,304,384,392,493,501,588,601,604,610,629,727,805,808,814,833,986,1032,1035,1041,1061,1138,1141,1147,1152,1159,1175,1261,1266,1332,1416,1432,1435,1441,1450,1467,1538,1607,1724,1764,1782,1799,1845,1848,1854,1892,1930,1983,1992,1995,2001,2024,2177,2226,2265,2285,2295,2304,2307,2313,2329,2447,2467,2510,2519,2579,2582,2588,2597,2652,2661,2711,2723,2726,2732,2759,2762,2768,2791,2797,2836],{"type":54,"tag":55,"props":56,"children":57},"element","h1",{"id":4},[58],{"type":59,"value":4},"text",{"type":54,"tag":61,"props":62,"children":63},"p",{},[64],{"type":59,"value":65},"Local NVIDIA GPU fine-tuning for HuggingFace models, grounded in live-fetched\ndocumentation with curated references as a fallback safety net. One NGC container,\na few focused scripts, one push to HF Hub. Follow the rules in this file; don't\nimprovise.",{"type":54,"tag":61,"props":67,"children":68},{},[69],{"type":54,"tag":70,"props":71,"children":72},"strong",{},[73],{"type":59,"value":74},"Order of authority (highest first):",{"type":54,"tag":76,"props":77,"children":78},"ol",{},[79,121,139,157],{"type":54,"tag":80,"props":81,"children":82},"li",{},[83,88,90,97,99,105,106,112,113,119],{"type":54,"tag":70,"props":84,"children":85},{},[86],{"type":59,"value":87},"User input",{"type":59,"value":89}," — explicit ",{"type":54,"tag":91,"props":92,"children":94},"code",{"className":93},[],[95],{"type":59,"value":96},"model_id",{"type":59,"value":98},", ",{"type":54,"tag":91,"props":100,"children":102},{"className":101},[],[103],{"type":59,"value":104},"dataset_id",{"type":59,"value":98},{"type":54,"tag":91,"props":107,"children":109},{"className":108},[],[110],{"type":59,"value":111},"training_method",{"type":59,"value":98},{"type":54,"tag":91,"props":114,"children":116},{"className":115},[],[117],{"type":59,"value":118},"config.yaml",{"type":59,"value":120}," overrides.",{"type":54,"tag":80,"props":122,"children":123},{},[124,129,131,137],{"type":54,"tag":70,"props":125,"children":126},{},[127],{"type":59,"value":128},"Live research",{"type":59,"value":130}," — model card, HF repo example, author finetune script, HF task docs, paper; always fetched (Step 3 + ",{"type":54,"tag":91,"props":132,"children":134},{"className":133},[],[135],{"type":59,"value":136},"references\u002Fresearch-priorities.md",{"type":59,"value":138},").",{"type":54,"tag":80,"props":140,"children":141},{},[142,147,149,155],{"type":54,"tag":70,"props":143,"children":144},{},[145],{"type":59,"value":146},"Curated references",{"type":59,"value":148}," (",{"type":54,"tag":91,"props":150,"children":152},{"className":151},[],[153],{"type":59,"value":154},"references\u002F*.md",{"type":59,"value":156},") — fallback when live research is silent\u002Fambiguous.",{"type":54,"tag":80,"props":158,"children":159},{},[160,165],{"type":54,"tag":70,"props":161,"children":162},{},[163],{"type":59,"value":164},"Your training-data memory",{"type":59,"value":166}," — last resort; suspect, cross-check against (2)\u002F(3).",{"type":54,"tag":61,"props":168,"children":169},{},[170,172,177],{"type":59,"value":171},"Conflict resolution between (2) and (3) and the source-line discrepancy note are\nin ",{"type":54,"tag":91,"props":173,"children":175},{"className":174},[],[176],{"type":59,"value":136},{"type":59,"value":178},".",{"type":54,"tag":180,"props":181,"children":182},"hr",{},[],{"type":54,"tag":184,"props":185,"children":187},"h2",{"id":186},"inputs",[188],{"type":59,"value":189},"Inputs",{"type":54,"tag":61,"props":191,"children":192},{},[193],{"type":54,"tag":70,"props":194,"children":195},{},[196],{"type":59,"value":197},"Required:",{"type":54,"tag":199,"props":200,"children":201},"ul",{},[202],{"type":54,"tag":80,"props":203,"children":204},{},[205,210,212],{"type":54,"tag":91,"props":206,"children":208},{"className":207},[],[209],{"type":59,"value":96},{"type":59,"value":211}," — HuggingFace model ID, e.g. ",{"type":54,"tag":91,"props":213,"children":215},{"className":214},[],[216],{"type":59,"value":217},"google\u002Fvit-base-patch16-224",{"type":54,"tag":61,"props":219,"children":220},{},[221],{"type":54,"tag":70,"props":222,"children":223},{},[224],{"type":59,"value":225},"Conditional credentials (read from the session environment, exported before launching when present):",{"type":54,"tag":199,"props":227,"children":228},{},[229,270],{"type":54,"tag":80,"props":230,"children":231},{},[232,238,240,245,247,253,255,261,263,269],{"type":54,"tag":91,"props":233,"children":235},{"className":234},[],[236],{"type":59,"value":237},"HF_TOKEN",{"type":59,"value":239}," — only when the model\u002Fdataset is ",{"type":54,"tag":70,"props":241,"children":242},{},[243],{"type":59,"value":244},"gated",{"type":59,"value":246}," (read) or ",{"type":54,"tag":91,"props":248,"children":250},{"className":249},[],[251],{"type":59,"value":252},"push_to_hub",{"type":59,"value":254}," is on (write); public + public + ",{"type":54,"tag":91,"props":256,"children":258},{"className":257},[],[259],{"type":59,"value":260},"push_to_hub: false",{"type":59,"value":262}," needs none. Value never read — presence-only via ",{"type":54,"tag":91,"props":264,"children":266},{"className":265},[],[267],{"type":59,"value":268},"[ -n \"$HF_TOKEN\" ]",{"type":59,"value":178},{"type":54,"tag":80,"props":271,"children":272},{},[273,279,280,286,288,294],{"type":54,"tag":91,"props":274,"children":276},{"className":275},[],[277],{"type":59,"value":278},"WANDB_API_KEY",{"type":59,"value":98},{"type":54,"tag":91,"props":281,"children":283},{"className":282},[],[284],{"type":59,"value":285},"WANDB_PROJECT",{"type":59,"value":287}," — only when WandB is enabled; ",{"type":54,"tag":91,"props":289,"children":291},{"className":290},[],[292],{"type":59,"value":293},"WANDB_MODE=disabled",{"type":59,"value":295}," opts out.",{"type":54,"tag":61,"props":297,"children":298},{},[299],{"type":54,"tag":70,"props":300,"children":301},{},[302],{"type":59,"value":303},"Dataset — exactly one:",{"type":54,"tag":199,"props":305,"children":306},{},[307,331,363],{"type":54,"tag":80,"props":308,"children":309},{},[310,315,317],{"type":54,"tag":91,"props":311,"children":313},{"className":312},[],[314],{"type":59,"value":104},{"type":59,"value":316}," — HuggingFace dataset ID ",{"type":54,"tag":318,"props":319,"children":320},"em",{},[321,323,329],{"type":59,"value":322},"(source: ",{"type":54,"tag":91,"props":324,"children":326},{"className":325},[],[327],{"type":59,"value":328},"hf",{"type":59,"value":330},")",{"type":54,"tag":80,"props":332,"children":333},{},[334,340,342,353,355,361],{"type":54,"tag":91,"props":335,"children":337},{"className":336},[],[338],{"type":59,"value":339},"local_dataset_path",{"type":59,"value":341}," — local folder or file ",{"type":54,"tag":318,"props":343,"children":344},{},[345,346,352],{"type":59,"value":322},{"type":54,"tag":91,"props":347,"children":349},{"className":348},[],[350],{"type":59,"value":351},"local",{"type":59,"value":330},{"type":59,"value":354},"; optional\n",{"type":54,"tag":91,"props":356,"children":358},{"className":357},[],[359],{"type":59,"value":360},"local_dataset_format",{"type":59,"value":362}," ∈ {auto, imagefolder, coco, voc, jsonl, arrow, parquet,\ncsv} (default: auto-detect).",{"type":54,"tag":80,"props":364,"children":365},{},[366,371,373],{"type":54,"tag":318,"props":367,"children":368},{},[369],{"type":59,"value":370},"(omit)",{"type":59,"value":372}," — agent recommends popular datasets ",{"type":54,"tag":318,"props":374,"children":375},{},[376,377,383],{"type":59,"value":322},{"type":54,"tag":91,"props":378,"children":380},{"className":379},[],[381],{"type":59,"value":382},"recommend",{"type":59,"value":330},{"type":54,"tag":61,"props":385,"children":386},{},[387],{"type":54,"tag":70,"props":388,"children":389},{},[390],{"type":59,"value":391},"Optional (have defaults):",{"type":54,"tag":199,"props":393,"children":394},{},[395,406,436,445,463,482],{"type":54,"tag":80,"props":396,"children":397},{},[398,404],{"type":54,"tag":91,"props":399,"children":401},{"className":400},[],[402],{"type":59,"value":403},"task_type",{"type":59,"value":405}," — auto-detected from config + model card",{"type":54,"tag":80,"props":407,"children":408},{},[409,415,416,422,423,429,430],{"type":54,"tag":91,"props":410,"children":412},{"className":411},[],[413],{"type":59,"value":414},"n_train=10000",{"type":59,"value":98},{"type":54,"tag":91,"props":417,"children":419},{"className":418},[],[420],{"type":59,"value":421},"n_eval=1000",{"type":59,"value":98},{"type":54,"tag":91,"props":424,"children":426},{"className":425},[],[427],{"type":59,"value":428},"n_epochs=3",{"type":59,"value":98},{"type":54,"tag":91,"props":431,"children":433},{"className":432},[],[434],{"type":59,"value":435},"lora_r=16",{"type":54,"tag":80,"props":437,"children":438},{},[439],{"type":54,"tag":91,"props":440,"children":442},{"className":441},[],[443],{"type":59,"value":444},"output_dir=.\u002Foutput\u002F\u003Cmodel_short_name>",{"type":54,"tag":80,"props":446,"children":447},{},[448,454,456,462],{"type":54,"tag":91,"props":449,"children":451},{"className":450},[],[452],{"type":59,"value":453},"hf_model_repo",{"type":59,"value":455}," — push target; if unset and HF_TOKEN has write access,\nauto-derived as ",{"type":54,"tag":91,"props":457,"children":459},{"className":458},[],[460],{"type":59,"value":461},"\u003Cwhoami>\u002F\u003Cmodel_short_name>-finetuned",{"type":59,"value":178},{"type":54,"tag":80,"props":464,"children":465},{},[466,472,474,480],{"type":54,"tag":91,"props":467,"children":469},{"className":468},[],[470],{"type":59,"value":471},"push_to_hub=True",{"type":59,"value":473}," — set to ",{"type":54,"tag":91,"props":475,"children":477},{"className":476},[],[478],{"type":59,"value":479},"False",{"type":59,"value":481}," to skip",{"type":54,"tag":80,"props":483,"children":484},{},[485,491],{"type":54,"tag":91,"props":486,"children":488},{"className":487},[],[489],{"type":59,"value":490},"skip_baseline=False",{"type":59,"value":492}," — skip zero-shot baseline eval",{"type":54,"tag":61,"props":494,"children":495},{},[496],{"type":54,"tag":70,"props":497,"children":498},{},[499],{"type":59,"value":500},"Optional deliverables (off by default):",{"type":54,"tag":502,"props":503,"children":508},"pre",{"className":504,"code":505,"language":506,"meta":507,"style":507},"language-yaml shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","emit_progress_log: false   # output_dir\u002FPROGRESS.md (per-step journal)\nemit_report:       false   # reports\u002Freport.{pdf,html} with curves & samples\nemit_unit_tests:   false   # tests\u002F with fake-data heterogeneous-batch tests\n","yaml","",[509],{"type":54,"tag":91,"props":510,"children":511},{"__ignoreMap":507},[512,542,565],{"type":54,"tag":513,"props":514,"children":517},"span",{"class":515,"line":516},"line",1,[518,524,530,536],{"type":54,"tag":513,"props":519,"children":521},{"style":520},"--shiki-light:#E53935;--shiki-default:#F07178;--shiki-dark:#F07178",[522],{"type":59,"value":523},"emit_progress_log",{"type":54,"tag":513,"props":525,"children":527},{"style":526},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[528],{"type":59,"value":529},":",{"type":54,"tag":513,"props":531,"children":533},{"style":532},"--shiki-light:#FF5370;--shiki-default:#FF9CAC;--shiki-dark:#FF9CAC",[534],{"type":59,"value":535}," false",{"type":54,"tag":513,"props":537,"children":539},{"style":538},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[540],{"type":59,"value":541},"   # output_dir\u002FPROGRESS.md (per-step journal)\n",{"type":54,"tag":513,"props":543,"children":545},{"class":515,"line":544},2,[546,551,555,560],{"type":54,"tag":513,"props":547,"children":548},{"style":520},[549],{"type":59,"value":550},"emit_report",{"type":54,"tag":513,"props":552,"children":553},{"style":526},[554],{"type":59,"value":529},{"type":54,"tag":513,"props":556,"children":557},{"style":532},[558],{"type":59,"value":559},"       false",{"type":54,"tag":513,"props":561,"children":562},{"style":538},[563],{"type":59,"value":564},"   # reports\u002Freport.{pdf,html} with curves & samples\n",{"type":54,"tag":513,"props":566,"children":568},{"class":515,"line":567},3,[569,574,578,583],{"type":54,"tag":513,"props":570,"children":571},{"style":520},[572],{"type":59,"value":573},"emit_unit_tests",{"type":54,"tag":513,"props":575,"children":576},{"style":526},[577],{"type":59,"value":529},{"type":54,"tag":513,"props":579,"children":580},{"style":532},[581],{"type":59,"value":582},"   false",{"type":54,"tag":513,"props":584,"children":585},{"style":538},[586],{"type":59,"value":587},"   # tests\u002F with fake-data heterogeneous-batch tests\n",{"type":54,"tag":61,"props":589,"children":590},{},[591,593,599],{"type":59,"value":592},"All values live in ",{"type":54,"tag":91,"props":594,"children":596},{"className":595},[],[597],{"type":59,"value":598},"output_dir\u002Fconfig.yaml",{"type":59,"value":600},". Never hardcode in Python.",{"type":54,"tag":180,"props":602,"children":603},{},[],{"type":54,"tag":184,"props":605,"children":607},{"id":606},"execution-platform",[608],{"type":59,"value":609},"Execution platform",{"type":54,"tag":61,"props":611,"children":612},{},[613,615,620,622,627],{"type":59,"value":614},"This skill orchestrates ",{"type":54,"tag":318,"props":616,"children":617},{},[618],{"type":59,"value":619},"what",{"type":59,"value":621}," to run; the platform skills own ",{"type":54,"tag":318,"props":623,"children":624},{},[625],{"type":59,"value":626},"how",{"type":59,"value":628}," to run it on\na GPU host — read them first.",{"type":54,"tag":630,"props":631,"children":632},"table",{},[633,652],{"type":54,"tag":634,"props":635,"children":636},"thead",{},[637],{"type":54,"tag":638,"props":639,"children":640},"tr",{},[641,647],{"type":54,"tag":642,"props":643,"children":644},"th",{},[645],{"type":59,"value":646},"Concern",{"type":54,"tag":642,"props":648,"children":649},{},[650],{"type":59,"value":651},"Authoritative skill",{"type":54,"tag":653,"props":654,"children":655},"tbody",{},[656,679,706],{"type":54,"tag":638,"props":657,"children":658},{},[659,665],{"type":54,"tag":660,"props":661,"children":662},"td",{},[663],{"type":59,"value":664},"GPU host runtime (driver 580, CUDA Toolkit 13.0, NVIDIA Container Toolkit 1.19.0)",{"type":54,"tag":660,"props":666,"children":667},{},[668],{"type":54,"tag":669,"props":670,"children":672},"a",{"href":671},"..\u002F..\u002Fplatform\u002Ftao-setup-nvidia-gpu-host\u002FSKILL.md",[673],{"type":54,"tag":91,"props":674,"children":676},{"className":675},[],[677],{"type":59,"value":678},"tao-skill-bank:tao-setup-nvidia-gpu-host",{"type":54,"tag":638,"props":680,"children":681},{},[682,693],{"type":54,"tag":660,"props":683,"children":684},{},[685,691],{"type":54,"tag":91,"props":686,"children":688},{"className":687},[],[689],{"type":59,"value":690},"docker run",{"type":59,"value":692}," flags, NGC auth, mounts, env passthrough",{"type":54,"tag":660,"props":694,"children":695},{},[696],{"type":54,"tag":669,"props":697,"children":699},{"href":698},"..\u002F..\u002Fplatform\u002Ftao-run-on-docker\u002FSKILL.md",[700],{"type":54,"tag":91,"props":701,"children":703},{"className":702},[],[704],{"type":59,"value":705},"tao-skill-bank:tao-run-on-docker",{"type":54,"tag":638,"props":707,"children":708},{},[709,714],{"type":54,"tag":660,"props":710,"children":711},{},[712],{"type":59,"value":713},"Local Docker job preflight (daemon, GPU smoke)",{"type":54,"tag":660,"props":715,"children":716},{},[717],{"type":54,"tag":669,"props":718,"children":720},{"href":719},"..\u002F..\u002Fplatform\u002Ftao-run-on-local-docker\u002FSKILL.md",[721],{"type":54,"tag":91,"props":722,"children":724},{"className":723},[],[725],{"type":59,"value":726},"tao-skill-bank:tao-run-on-local-docker",{"type":54,"tag":61,"props":728,"children":729},{},[730,735,737,743,745,751,753,758,760,765,767,773,775,781,782,788,790,796,798,804],{"type":54,"tag":70,"props":731,"children":732},{},[733],{"type":59,"value":734},"Default platform:",{"type":59,"value":736}," ",{"type":54,"tag":91,"props":738,"children":740},{"className":739},[],[741],{"type":59,"value":742},"local-docker",{"type":59,"value":744}," — build a one-off image (",{"type":54,"tag":91,"props":746,"children":748},{"className":747},[],[749],{"type":59,"value":750},"run-\u003Cshort>:latest",{"type":59,"value":752},")\nand run it on the local Docker daemon. Ask only when the user explicitly needs a\ndifferent backend (Brev remote GPU, SLURM\u002FKubernetes); then run that platform's\nPreflight first and route the Steps 4–5 ",{"type":54,"tag":91,"props":754,"children":756},{"className":755},[],[757],{"type":59,"value":690},{"type":59,"value":759}," commands through it. The\nGPU-runtime and presence-only credential preflights (values never read), the\ncanonical ",{"type":54,"tag":91,"props":761,"children":763},{"className":762},[],[764],{"type":59,"value":690},{"type":59,"value":766}," flag set, the ",{"type":54,"tag":91,"props":768,"children":770},{"className":769},[],[771],{"type":59,"value":772},"list_tao_platforms.py",{"type":59,"value":774}," selection command, and\nthe workflow-specific flags (",{"type":54,"tag":91,"props":776,"children":778},{"className":777},[],[779],{"type":59,"value":780},"--entrypoint \u002Fbin\u002Fbash -lc",{"type":59,"value":98},{"type":54,"tag":91,"props":783,"children":785},{"className":784},[],[786],{"type":59,"value":787},"PYTORCH_CUDA_ALLOC_CONF",{"type":59,"value":789},",\n",{"type":54,"tag":91,"props":791,"children":793},{"className":792},[],[794],{"type":59,"value":795},"--name hft_train",{"type":59,"value":797},") are in ",{"type":54,"tag":91,"props":799,"children":801},{"className":800},[],[802],{"type":59,"value":803},"references\u002Fworkflow-intake-preflight.md",{"type":59,"value":178},{"type":54,"tag":180,"props":806,"children":807},{},[],{"type":54,"tag":184,"props":809,"children":811},{"id":810},"references-fallback-safety-net",[812],{"type":59,"value":813},"References — fallback safety net",{"type":54,"tag":61,"props":815,"children":816},{},[817,819,824,826,832],{"type":59,"value":818},"Consulted ",{"type":54,"tag":70,"props":820,"children":821},{},[822],{"type":59,"value":823},"only",{"type":59,"value":825}," when live research is silent, ambiguous, or unavailable; live\ndocs always win for the specific model and current API. Each step links the\nreferences it needs; full catalog in ",{"type":54,"tag":91,"props":827,"children":829},{"className":828},[],[830],{"type":59,"value":831},"references\u002Fdetailed-workflow.md",{"type":59,"value":178},{"type":54,"tag":61,"props":834,"children":835},{},[836,838,844,845,851,852,858,859,865,866,872,873,879,880,886,887,893,894,900,901,907,908,914,915,921,922,928,929,935,936,942,944,950,951,957,958,964,965,971,972,978,979,985],{"type":59,"value":837},"Always-on: ",{"type":54,"tag":91,"props":839,"children":841},{"className":840},[],[842],{"type":59,"value":843},"core-rules.md",{"type":59,"value":98},{"type":54,"tag":91,"props":846,"children":848},{"className":847},[],[849],{"type":59,"value":850},"error-playbook.md",{"type":59,"value":98},{"type":54,"tag":91,"props":853,"children":855},{"className":854},[],[856],{"type":59,"value":857},"compat-workarounds.md",{"type":59,"value":789},{"type":54,"tag":91,"props":860,"children":862},{"className":861},[],[863],{"type":59,"value":864},"model-discovery.md",{"type":59,"value":98},{"type":54,"tag":91,"props":867,"children":869},{"className":868},[],[870],{"type":59,"value":871},"dataset-recommendations.md",{"type":59,"value":98},{"type":54,"tag":91,"props":874,"children":876},{"className":875},[],[877],{"type":59,"value":878},"dataset-sources.md",{"type":59,"value":789},{"type":54,"tag":91,"props":881,"children":883},{"className":882},[],[884],{"type":59,"value":885},"dataset-patterns.md",{"type":59,"value":98},{"type":54,"tag":91,"props":888,"children":890},{"className":889},[],[891],{"type":59,"value":892},"hardware-container.md",{"type":59,"value":98},{"type":54,"tag":91,"props":895,"children":897},{"className":896},[],[898],{"type":59,"value":899},"research-priorities.md",{"type":59,"value":789},{"type":54,"tag":91,"props":902,"children":904},{"className":903},[],[905],{"type":59,"value":906},"cv-scripts.md",{"type":59,"value":98},{"type":54,"tag":91,"props":909,"children":911},{"className":910},[],[912],{"type":59,"value":913},"vlm-scripts.md",{"type":59,"value":98},{"type":54,"tag":91,"props":916,"children":918},{"className":917},[],[919],{"type":59,"value":920},"docker-runs.md",{"type":59,"value":98},{"type":54,"tag":91,"props":923,"children":925},{"className":924},[],[926],{"type":59,"value":927},"hub-push.md",{"type":59,"value":789},{"type":54,"tag":91,"props":930,"children":932},{"className":931},[],[933],{"type":59,"value":934},"pipeline-skill-template.md",{"type":59,"value":98},{"type":54,"tag":91,"props":937,"children":939},{"className":938},[],[940],{"type":59,"value":941},"deliverables.md",{"type":59,"value":943},". Opt-in (when their flag\u002Fneed\napplies): ",{"type":54,"tag":91,"props":945,"children":947},{"className":946},[],[948],{"type":59,"value":949},"progress-tracking.md",{"type":59,"value":98},{"type":54,"tag":91,"props":952,"children":954},{"className":953},[],[955],{"type":59,"value":956},"testing.md",{"type":59,"value":98},{"type":54,"tag":91,"props":959,"children":961},{"className":960},[],[962],{"type":59,"value":963},"reporting.md",{"type":59,"value":789},{"type":54,"tag":91,"props":966,"children":968},{"className":967},[],[969],{"type":59,"value":970},"workflow-intake-preflight.md",{"type":59,"value":98},{"type":54,"tag":91,"props":973,"children":975},{"className":974},[],[976],{"type":59,"value":977},"workflow-generate-train.md",{"type":59,"value":98},{"type":54,"tag":91,"props":980,"children":982},{"className":981},[],[983],{"type":59,"value":984},"workflow-push-rerun.md",{"type":59,"value":178},{"type":54,"tag":61,"props":987,"children":988},{},[989,994,996,1001,1002,1008,1010,1016,1018,1023,1025,1030],{"type":54,"tag":70,"props":990,"children":991},{},[992],{"type":59,"value":993},"Rule:",{"type":59,"value":995}," before falling back, log the live source you tried and why it was\ninsufficient (",{"type":54,"tag":91,"props":997,"children":999},{"className":998},[],[1000],{"type":59,"value":118},{"type":59,"value":736},{"type":54,"tag":91,"props":1003,"children":1005},{"className":1004},[],[1006],{"type":59,"value":1007},"notes:",{"type":59,"value":1009},", and PROGRESS.md if enabled). ",{"type":54,"tag":91,"props":1011,"children":1013},{"className":1012},[],[1014],{"type":59,"value":1015},"[FETCH LIVE]",{"type":59,"value":1017},"\nmarkers in ",{"type":54,"tag":91,"props":1019,"children":1021},{"className":1020},[],[1022],{"type":59,"value":906},{"type":59,"value":1024}," \u002F ",{"type":54,"tag":91,"props":1026,"children":1028},{"className":1027},[],[1029],{"type":59,"value":913},{"type":59,"value":1031}," are a research checklist, not code to\ninline — refetch the listed URL if a block has no Step 3 finding.",{"type":54,"tag":180,"props":1033,"children":1034},{},[],{"type":54,"tag":184,"props":1036,"children":1038},{"id":1037},"core-rules",[1039],{"type":59,"value":1040},"Core rules",{"type":54,"tag":61,"props":1042,"children":1043},{},[1044,1046,1051,1053,1059],{"type":59,"value":1045},"Non-negotiable behaviors. ",{"type":54,"tag":70,"props":1047,"children":1048},{},[1049],{"type":59,"value":1050},"Short version",{"type":59,"value":1052}," (full enumeration —\nhallucinated-imports list, never-without-approval list, full error-recovery and\nhardware-sizing tables — in ",{"type":54,"tag":91,"props":1054,"children":1056},{"className":1055},[],[1057],{"type":59,"value":1058},"references\u002Fcore-rules.md",{"type":59,"value":1060},", consult before any\ntraining-time decision):",{"type":54,"tag":199,"props":1062,"children":1063},{},[1064,1074,1090,1100,1110,1128],{"type":54,"tag":80,"props":1065,"children":1066},{},[1067,1072],{"type":54,"tag":70,"props":1068,"children":1069},{},[1070],{"type":59,"value":1071},"Your HF-library knowledge is outdated.",{"type":59,"value":1073}," Fetch live docs (model card, HF\nrepo example, task doc) before writing any ML code — don't generate trainer\nargs \u002F collator \u002F transforms from memory (Step 3).",{"type":54,"tag":80,"props":1075,"children":1076},{},[1077,1088],{"type":54,"tag":70,"props":1078,"children":1079},{},[1080,1082],{"type":59,"value":1081},"Smoke-test on real data with ",{"type":54,"tag":91,"props":1083,"children":1085},{"className":1084},[],[1086],{"type":59,"value":1087},"--max_steps 1",{"type":59,"value":1089}," before any full run; no batch\nlaunches without a verified smoke.",{"type":54,"tag":80,"props":1091,"children":1092},{},[1093,1098],{"type":54,"tag":70,"props":1094,"children":1095},{},[1096],{"type":59,"value":1097},"Never silently substitute",{"type":59,"value":1099}," model_id, dataset_id, or training_method — if\nwhat the user asked for doesn't load, stop and ask.",{"type":54,"tag":80,"props":1101,"children":1102},{},[1103,1108],{"type":54,"tag":70,"props":1104,"children":1105},{},[1106],{"type":59,"value":1107},"Error recovery is minimal-change.",{"type":59,"value":1109}," OOM → halve batch, double grad_accum,\nenable gradient checkpointing (no LoRA switch without approval); NaN → reduce\nLR 10×; flat loss → inspect collator; same error 3× → stop and ask. Don't loop.",{"type":54,"tag":80,"props":1111,"children":1112},{},[1113,1118,1120,1126],{"type":54,"tag":70,"props":1114,"children":1115},{},[1116],{"type":59,"value":1117},"Dataset columns verified BEFORE the collator",{"type":59,"value":1119}," — rename in ",{"type":54,"tag":91,"props":1121,"children":1123},{"className":1122},[],[1124],{"type":59,"value":1125},"prepare_data.py",{"type":59,"value":1127},";\nrestructuring needed → stop and ask.",{"type":54,"tag":80,"props":1129,"children":1130},{},[1131,1136],{"type":54,"tag":70,"props":1132,"children":1133},{},[1134],{"type":59,"value":1135},"Hardware-sizing thumb (bf16):",{"type":59,"value":1137}," ≤3B → 24 GB, 7–13B → 80 GB, 30B+ → multi-GPU\nor LoRA on 1× 80 GB, 70B+ → 8× 80 GB or LoRA. Full finetune won't fit and no\nLoRA requested → ask before switching.",{"type":54,"tag":180,"props":1139,"children":1140},{},[],{"type":54,"tag":184,"props":1142,"children":1144},{"id":1143},"workflow-6-steps",[1145],{"type":59,"value":1146},"Workflow — 6 steps",{"type":54,"tag":61,"props":1148,"children":1149},{},[1150],{"type":59,"value":1151},"Single pass, sequential; each step has a clear gate before the next begins.",{"type":54,"tag":1153,"props":1154,"children":1156},"h3",{"id":1155},"step-1-inspect-qualify",[1157],{"type":59,"value":1158},"Step 1 — Inspect & qualify",{"type":54,"tag":61,"props":1160,"children":1161},{},[1162,1167,1169,1174],{"type":54,"tag":70,"props":1163,"children":1164},{},[1165],{"type":59,"value":1166},"Goal:",{"type":59,"value":1168}," decide whether to proceed. Probe model + dataset, apply accept\u002Freject,\nregister applicable compat fixes, write the initial ",{"type":54,"tag":91,"props":1170,"children":1172},{"className":1171},[],[1173],{"type":59,"value":118},{"type":59,"value":178},{"type":54,"tag":61,"props":1176,"children":1177},{},[1178,1180,1186,1188,1194,1195,1200,1202,1207,1208,1214,1216,1222,1224,1230,1232,1238,1240,1245,1246,1252,1254,1260],{"type":59,"value":1179},"Prerequisites: ",{"type":54,"tag":91,"props":1181,"children":1183},{"className":1182},[],[1184],{"type":59,"value":1185},"MODEL_ID",{"type":59,"value":1187},", optional ",{"type":54,"tag":91,"props":1189,"children":1191},{"className":1190},[],[1192],{"type":59,"value":1193},"DATASET_ID",{"type":59,"value":1024},{"type":54,"tag":91,"props":1196,"children":1198},{"className":1197},[],[1199],{"type":59,"value":339},{"type":59,"value":1201},",\noptional ",{"type":54,"tag":91,"props":1203,"children":1205},{"className":1204},[],[1206],{"type":59,"value":237},{"type":59,"value":98},{"type":54,"tag":91,"props":1209,"children":1211},{"className":1210},[],[1212],{"type":59,"value":1213},"OUTPUT_DIR",{"type":59,"value":1215}," (default ",{"type":54,"tag":91,"props":1217,"children":1219},{"className":1218},[],[1220],{"type":59,"value":1221},".\u002Foutput\u002F\u003Cmodel_short_name>",{"type":59,"value":1223},"). Probes\nrun in a CPU-only ",{"type":54,"tag":91,"props":1225,"children":1227},{"className":1226},[],[1228],{"type":59,"value":1229},"python:3.12-slim",{"type":59,"value":1231}," Docker container (bind-mounted ",{"type":54,"tag":91,"props":1233,"children":1235},{"className":1234},[],[1236],{"type":59,"value":1237},".probe\u002F",{"type":59,"value":1239},"\nscratch) so the host needs no virtualenv — Docker must exist first. Docker-presence\nguard, container env, full probe invocation, and the model\u002Fdataset probe scripts\nare in ",{"type":54,"tag":91,"props":1241,"children":1243},{"className":1242},[],[1244],{"type":59,"value":803},{"type":59,"value":98},{"type":54,"tag":91,"props":1247,"children":1249},{"className":1248},[],[1250],{"type":59,"value":1251},"references\u002Fmodel-discovery.md",{"type":59,"value":1253},",\nand ",{"type":54,"tag":91,"props":1255,"children":1257},{"className":1256},[],[1258],{"type":59,"value":1259},"references\u002Fdataset-sources.md",{"type":59,"value":178},{"type":54,"tag":61,"props":1262,"children":1263},{},[1264],{"type":59,"value":1265},"Probe requirements:",{"type":54,"tag":199,"props":1267,"children":1268},{},[1269,1296,1315,1320],{"type":54,"tag":80,"props":1270,"children":1271},{},[1272,1274,1280,1282,1288,1290,1295],{"type":59,"value":1273},"Model: load ",{"type":54,"tag":91,"props":1275,"children":1277},{"className":1276},[],[1278],{"type":59,"value":1279},"AutoConfig",{"type":59,"value":1281},", read model-card tags, detect task from\n",{"type":54,"tag":91,"props":1283,"children":1285},{"className":1284},[],[1286],{"type":59,"value":1287},"architectures",{"type":59,"value":1289}," + tags + card examples (fallback logging in ",{"type":54,"tag":91,"props":1291,"children":1293},{"className":1292},[],[1294],{"type":59,"value":864},{"type":59,"value":138},{"type":54,"tag":80,"props":1297,"children":1298},{},[1299,1301,1306,1308,1313],{"type":59,"value":1300},"Dataset: for recommended datasets, first present 3-5 choices from\n",{"type":54,"tag":91,"props":1302,"children":1304},{"className":1303},[],[1305],{"type":59,"value":871},{"type":59,"value":1307},"; for local data, bind-mount read-only and use\n",{"type":54,"tag":91,"props":1309,"children":1311},{"className":1310},[],[1312],{"type":59,"value":878},{"type":59,"value":1314}," format detection.",{"type":54,"tag":80,"props":1316,"children":1317},{},[1318],{"type":59,"value":1319},"Reject early if the model config fails, the task is out of scope, no recipe\nsource exists, or the dataset cannot load \u002F match the task schema.",{"type":54,"tag":80,"props":1321,"children":1322},{},[1323,1325,1330],{"type":59,"value":1324},"Evaluate ",{"type":54,"tag":91,"props":1326,"children":1328},{"className":1327},[],[1329],{"type":59,"value":857},{"type":59,"value":1331}," against the model\u002Ftask; defer hardware-dependent\nrules to Step 2.",{"type":54,"tag":61,"props":1333,"children":1334},{},[1335,1337,1342,1343,1348,1349,1355,1356,1361,1363,1368,1369,1375,1377,1383,1385,1391,1393,1399,1401,1406,1408,1414],{"type":59,"value":1336},"Write the initial ",{"type":54,"tag":91,"props":1338,"children":1340},{"className":1339},[],[1341],{"type":59,"value":118},{"type":59,"value":148},{"type":54,"tag":91,"props":1344,"children":1346},{"className":1345},[],[1347],{"type":59,"value":96},{"type":59,"value":98},{"type":54,"tag":91,"props":1350,"children":1352},{"className":1351},[],[1353],{"type":59,"value":1354},"task",{"type":59,"value":98},{"type":54,"tag":91,"props":1357,"children":1359},{"className":1358},[],[1360],{"type":59,"value":104},{"type":59,"value":1362}," or\n",{"type":54,"tag":91,"props":1364,"children":1366},{"className":1365},[],[1367],{"type":59,"value":339},{"type":59,"value":98},{"type":54,"tag":91,"props":1370,"children":1372},{"className":1371},[],[1373],{"type":59,"value":1374},"research_sources: []",{"type":59,"value":1376}," filled in Step 3,\n",{"type":54,"tag":91,"props":1378,"children":1380},{"className":1379},[],[1381],{"type":59,"value":1382},"applicable_workarounds:",{"type":59,"value":1384}," from Step 1, ",{"type":54,"tag":91,"props":1386,"children":1388},{"className":1387},[],[1389],{"type":59,"value":1390},"notes: []",{"type":59,"value":1392}," for reference fallbacks,\n",{"type":54,"tag":91,"props":1394,"children":1396},{"className":1395},[],[1397],{"type":59,"value":1398},"push_to_hub: true",{"type":59,"value":1400}," default — annotated template in\n",{"type":54,"tag":91,"props":1402,"children":1404},{"className":1403},[],[1405],{"type":59,"value":803},{"type":59,"value":1407},"). Optionally ",{"type":54,"tag":91,"props":1409,"children":1411},{"className":1410},[],[1412],{"type":59,"value":1413},"rm -rf \"$OUTPUT_DIR\u002F.probe\"",{"type":59,"value":1415},"\nonce the gate is met.",{"type":54,"tag":61,"props":1417,"children":1418},{},[1419,1424,1425,1430],{"type":54,"tag":70,"props":1420,"children":1421},{},[1422],{"type":59,"value":1423},"Gate:",{"type":59,"value":736},{"type":54,"tag":91,"props":1426,"children":1428},{"className":1427},[],[1429],{"type":59,"value":118},{"type":59,"value":1431}," exists with model, dataset, task, applicable_workarounds;\ndo not proceed if any field is missing.",{"type":54,"tag":180,"props":1433,"children":1434},{},[],{"type":54,"tag":1153,"props":1436,"children":1438},{"id":1437},"step-2-hardware-audit-ngc-image",[1439],{"type":59,"value":1440},"Step 2 — Hardware audit & NGC image",{"type":54,"tag":61,"props":1442,"children":1443},{},[1444,1448],{"type":54,"tag":70,"props":1445,"children":1446},{},[1447],{"type":59,"value":1166},{"type":59,"value":1449}," verify Docker + GPU + disk, pick the NGC PyTorch image live, finalize\nhardware-dependent compat rules.",{"type":54,"tag":61,"props":1451,"children":1452},{},[1453,1458,1460,1465],{"type":54,"tag":70,"props":1454,"children":1455},{},[1456],{"type":59,"value":1457},"2a. Audit (hard gate)",{"type":59,"value":1459}," — three checks (commands in\n",{"type":54,"tag":91,"props":1461,"children":1463},{"className":1462},[],[1464],{"type":59,"value":803},{"type":59,"value":1466},"):",{"type":54,"tag":76,"props":1468,"children":1469},{},[1470,1498,1511],{"type":54,"tag":80,"props":1471,"children":1472},{},[1473,1475,1481,1483,1489,1491,1497],{"type":59,"value":1474},"GPU host runtime — ",{"type":54,"tag":91,"props":1476,"children":1478},{"className":1477},[],[1479],{"type":59,"value":1480},"tao-setup-nvidia-gpu-host",{"type":59,"value":1482},"'s\n",{"type":54,"tag":91,"props":1484,"children":1486},{"className":1485},[],[1487],{"type":59,"value":1488},"setup-nvidia-gpu-host.sh --backend docker --check-only",{"type":59,"value":1490},"; on fail, ask approval\nthen re-run with ",{"type":54,"tag":91,"props":1492,"children":1494},{"className":1493},[],[1495],{"type":59,"value":1496},"--install --yes",{"type":59,"value":178},{"type":54,"tag":80,"props":1499,"children":1500},{},[1501,1503,1509],{"type":59,"value":1502},"Free-disk soft-warn — override via ",{"type":54,"tag":91,"props":1504,"children":1506},{"className":1505},[],[1507],{"type":59,"value":1508},"MIN_DISK_GB",{"type":59,"value":1510}," (default 100 GB); recommend\n≥ 100 GB for NGC base (~20 GB) + HF cache + checkpoints + data.",{"type":54,"tag":80,"props":1512,"children":1513},{},[1514,1516,1521,1523,1528,1530,1536],{"type":59,"value":1515},"Conditional credential presence (from the session environment, values never\nread) — ",{"type":54,"tag":91,"props":1517,"children":1519},{"className":1518},[],[1520],{"type":59,"value":237},{"type":59,"value":1522}," only when gated or ",{"type":54,"tag":91,"props":1524,"children":1526},{"className":1525},[],[1527],{"type":59,"value":252},{"type":59,"value":1529}," is on; ",{"type":54,"tag":91,"props":1531,"children":1533},{"className":1532},[],[1534],{"type":59,"value":1535},"WANDB_*",{"type":59,"value":1537}," only when\nWandB is on.",{"type":54,"tag":61,"props":1539,"children":1540},{},[1541,1546,1548,1554,1556,1562,1564,1570,1572,1578,1579,1585,1586,1592,1593,1599,1601,1606],{"type":54,"tag":70,"props":1542,"children":1543},{},[1544],{"type":59,"value":1545},"Do not proceed to Step 4 on a hard-fail",{"type":59,"value":1547}," — Step 4's ",{"type":54,"tag":91,"props":1549,"children":1551},{"className":1550},[],[1552],{"type":59,"value":1553},"docker build",{"type":59,"value":1555}," pulls a\n20+ GB NGC base, and a missing ",{"type":54,"tag":91,"props":1557,"children":1559},{"className":1558},[],[1560],{"type":59,"value":1561},"nvidia-container-toolkit",{"type":59,"value":1563}," only surfaces later as\n",{"type":54,"tag":91,"props":1565,"children":1567},{"className":1566},[],[1568],{"type":59,"value":1569},"could not select device driver \"\" with capabilities: [[gpu]]",{"type":59,"value":1571},". Record ",{"type":54,"tag":91,"props":1573,"children":1575},{"className":1574},[],[1576],{"type":59,"value":1577},"gpu_count",{"type":59,"value":789},{"type":54,"tag":91,"props":1580,"children":1582},{"className":1581},[],[1583],{"type":59,"value":1584},"gpu_name",{"type":59,"value":98},{"type":54,"tag":91,"props":1587,"children":1589},{"className":1588},[],[1590],{"type":59,"value":1591},"driver_major",{"type":59,"value":98},{"type":54,"tag":91,"props":1594,"children":1596},{"className":1595},[],[1597],{"type":59,"value":1598},"vram_gb_per_gpu",{"type":59,"value":1600}," in ",{"type":54,"tag":91,"props":1602,"children":1604},{"className":1603},[],[1605],{"type":59,"value":118},{"type":59,"value":178},{"type":54,"tag":61,"props":1608,"children":1609},{},[1610,1615,1617,1624,1626,1632,1634,1640,1642,1647,1649,1655,1657,1663,1664,1670,1672,1677,1679,1685,1687,1693,1695,1701,1703,1709,1711,1717,1718,1723],{"type":54,"tag":70,"props":1611,"children":1612},{},[1613],{"type":59,"value":1614},"2b. Pick NGC image (live):",{"type":59,"value":1616}," from the NVIDIA Deep Learning Frameworks support\nmatrix (",{"type":54,"tag":669,"props":1618,"children":1622},{"href":1619,"rel":1620},"https:\u002F\u002Fdocs.nvidia.com\u002Fdeeplearning\u002Fframeworks\u002Fsupport-matrix\u002Findex.html",[1621],"nofollow",[1623],{"type":59,"value":1619},{"type":59,"value":1625},"),\nPyTorch NGC container section, pick the highest-versioned image where\n",{"type":54,"tag":91,"props":1627,"children":1629},{"className":1628},[],[1630],{"type":59,"value":1631},"Min driver ≤ detected driver_major",{"type":59,"value":1633}," and container CUDA ",{"type":54,"tag":91,"props":1635,"children":1637},{"className":1636},[],[1638],{"type":59,"value":1639},"≤",{"type":59,"value":1641}," host CUDA Toolkit\n(match closely so cuDNN \u002F TensorRT line up). Do ",{"type":54,"tag":70,"props":1643,"children":1644},{},[1645],{"type":59,"value":1646},"not",{"type":59,"value":1648}," reject an image for an\n",{"type":54,"tag":91,"props":1650,"children":1652},{"className":1651},[],[1653],{"type":59,"value":1654},"aN",{"type":59,"value":1656},"\u002F",{"type":54,"tag":91,"props":1658,"children":1660},{"className":1659},[],[1661],{"type":59,"value":1662},"bN",{"type":59,"value":1656},{"type":54,"tag":91,"props":1665,"children":1667},{"className":1666},[],[1668],{"type":59,"value":1669},"rcN",{"type":59,"value":1671}," PyTorch tag — NGC validates the full image; pick the newest\nCUDA-aligned one and let ",{"type":54,"tag":91,"props":1673,"children":1675},{"className":1674},[],[1676],{"type":59,"value":857},{"type":59,"value":1678}," handle per-version issues. If the\nmatrix is unreachable, use the fallbacks in ",{"type":54,"tag":91,"props":1680,"children":1682},{"className":1681},[],[1683],{"type":59,"value":1684},"references\u002Fhardware-container.md",{"type":59,"value":1686},";\ndefault ",{"type":54,"tag":91,"props":1688,"children":1690},{"className":1689},[],[1691],{"type":59,"value":1692},"nvcr.io\u002Fnvidia\u002Fpytorch:24.09-py3",{"type":59,"value":1694}," (driver ≥ 545; SDPA+GQA bug — if\n",{"type":54,"tag":91,"props":1696,"children":1698},{"className":1697},[],[1699],{"type":59,"value":1700},"num_key_value_heads \u003C num_attention_heads",{"type":59,"value":1702},", set ",{"type":54,"tag":91,"props":1704,"children":1706},{"className":1705},[],[1707],{"type":59,"value":1708},"attn_implementation: \"eager\"",{"type":59,"value":1710},").\nRecord ",{"type":54,"tag":91,"props":1712,"children":1714},{"className":1713},[],[1715],{"type":59,"value":1716},"ngc_image",{"type":59,"value":1600},{"type":54,"tag":91,"props":1719,"children":1721},{"className":1720},[],[1722],{"type":59,"value":118},{"type":59,"value":178},{"type":54,"tag":61,"props":1725,"children":1726},{},[1727,1732,1734,1739,1741,1747,1749,1755,1757,1762],{"type":54,"tag":70,"props":1728,"children":1729},{},[1730],{"type":59,"value":1731},"2c. Re-evaluate hardware-dependent compat rules:",{"type":59,"value":1733}," re-run the\n",{"type":54,"tag":91,"props":1735,"children":1737},{"className":1736},[],[1738],{"type":59,"value":857},{"type":59,"value":1740}," walk for entries whose ",{"type":54,"tag":91,"props":1742,"children":1744},{"className":1743},[],[1745],{"type":59,"value":1746},"detect",{"type":59,"value":1748}," needs ",{"type":54,"tag":91,"props":1750,"children":1752},{"className":1751},[],[1753],{"type":59,"value":1754},"hw",{"type":59,"value":1756},"; update\n",{"type":54,"tag":91,"props":1758,"children":1760},{"className":1759},[],[1761],{"type":59,"value":1382},{"type":59,"value":1763}," in place.",{"type":54,"tag":61,"props":1765,"children":1766},{},[1767,1772,1774,1780],{"type":54,"tag":70,"props":1768,"children":1769},{},[1770],{"type":59,"value":1771},"2d. Model-fit check:",{"type":59,"value":1773}," estimate ",{"type":54,"tag":91,"props":1775,"children":1777},{"className":1776},[],[1778],{"type":59,"value":1779},"param_bytes ≈ 2×param_count",{"type":59,"value":1781}," (bf16); if",{"type":54,"tag":1783,"props":1784,"children":1785},"blockquote",{},[1786],{"type":54,"tag":61,"props":1787,"children":1788},{},[1789,1791,1797],{"type":59,"value":1790},"60% of ",{"type":54,"tag":91,"props":1792,"children":1794},{"className":1793},[],[1795],{"type":59,"value":1796},"vram_gb_per_gpu × 1e9",{"type":59,"value":1798},", recommend LoRA in the user-facing summary.",{"type":54,"tag":61,"props":1800,"children":1801},{},[1802,1806,1807,1812,1814,1819,1820,1825,1826,1831,1832,1837,1838,1843],{"type":54,"tag":70,"props":1803,"children":1804},{},[1805],{"type":59,"value":1423},{"type":59,"value":736},{"type":54,"tag":91,"props":1808,"children":1810},{"className":1809},[],[1811],{"type":59,"value":118},{"type":59,"value":1813}," has ",{"type":54,"tag":91,"props":1815,"children":1817},{"className":1816},[],[1818],{"type":59,"value":1716},{"type":59,"value":98},{"type":54,"tag":91,"props":1821,"children":1823},{"className":1822},[],[1824],{"type":59,"value":1577},{"type":59,"value":98},{"type":54,"tag":91,"props":1827,"children":1829},{"className":1828},[],[1830],{"type":59,"value":1584},{"type":59,"value":98},{"type":54,"tag":91,"props":1833,"children":1835},{"className":1834},[],[1836],{"type":59,"value":1591},{"type":59,"value":789},{"type":54,"tag":91,"props":1839,"children":1841},{"className":1840},[],[1842],{"type":59,"value":1598},{"type":59,"value":1844},"; hardware-dependent compat fixes recorded.",{"type":54,"tag":180,"props":1846,"children":1847},{},[],{"type":54,"tag":1153,"props":1849,"children":1851},{"id":1850},"step-3-research-the-recipe",[1852],{"type":59,"value":1853},"Step 3 — Research the recipe",{"type":54,"tag":61,"props":1855,"children":1856},{},[1857,1861,1863,1869,1870,1876,1877,1883,1885,1890],{"type":54,"tag":70,"props":1858,"children":1859},{},[1860],{"type":59,"value":1166},{"type":59,"value":1862}," fetch the live recipe — training-data knowledge of\n",{"type":54,"tag":91,"props":1864,"children":1866},{"className":1865},[],[1867],{"type":59,"value":1868},"transformers",{"type":59,"value":1656},{"type":54,"tag":91,"props":1871,"children":1873},{"className":1872},[],[1874],{"type":59,"value":1875},"trl",{"type":59,"value":1656},{"type":54,"tag":91,"props":1878,"children":1880},{"className":1879},[],[1881],{"type":59,"value":1882},"peft",{"type":59,"value":1884}," is suspect, so Step 3 is non-negotiable. Walk\n",{"type":54,"tag":91,"props":1886,"children":1888},{"className":1887},[],[1889],{"type":59,"value":136},{"type":59,"value":1891}," in priority order (Priority 1 → 6); stop once\nyou have, for the detected task:",{"type":54,"tag":199,"props":1893,"children":1894},{},[1895,1906,1911,1916,1925],{"type":54,"tag":80,"props":1896,"children":1897},{},[1898,1904],{"type":54,"tag":91,"props":1899,"children":1901},{"className":1900},[],[1902],{"type":59,"value":1903},"AutoModel",{"type":59,"value":1905}," \u002F processor class",{"type":54,"tag":80,"props":1907,"children":1908},{},[1909],{"type":59,"value":1910},"Train + eval transforms",{"type":54,"tag":80,"props":1912,"children":1913},{},[1914],{"type":59,"value":1915},"Collator",{"type":54,"tag":80,"props":1917,"children":1918},{},[1919],{"type":54,"tag":91,"props":1920,"children":1922},{"className":1921},[],[1923],{"type":59,"value":1924},"compute_metrics",{"type":54,"tag":80,"props":1926,"children":1927},{},[1928],{"type":59,"value":1929},"Hyperparameter hints (LR, batch size, epochs, scheduler)",{"type":54,"tag":61,"props":1931,"children":1932},{},[1933,1935,1941,1943,1949,1951,1956,1957,1962,1964],{"type":59,"value":1934},"Record findings in ",{"type":54,"tag":91,"props":1936,"children":1938},{"className":1937},[],[1939],{"type":59,"value":1940},"meta\u002Frecipe.md",{"type":59,"value":1942},", append source URLs to\n",{"type":54,"tag":91,"props":1944,"children":1946},{"className":1945},[],[1947],{"type":59,"value":1948},"config.yaml: research_sources:",{"type":59,"value":1950},". A slot with no live finding falls back to the\nmatching scaffold (",{"type":54,"tag":91,"props":1952,"children":1954},{"className":1953},[],[1955],{"type":59,"value":906},{"type":59,"value":1024},{"type":54,"tag":91,"props":1958,"children":1960},{"className":1959},[],[1961],{"type":59,"value":913},{"type":59,"value":1963},"), logged as \"fallback to\nscaffold — no live source for ",{"type":54,"tag":1965,"props":1966,"children":1967},"slot",{},[1968,1970,1975,1977,1982],{"type":59,"value":1969},"\" under ",{"type":54,"tag":91,"props":1971,"children":1973},{"className":1972},[],[1974],{"type":59,"value":1007},{"type":59,"value":1976},". Conflict-resolution rules\nare in ",{"type":54,"tag":91,"props":1978,"children":1980},{"className":1979},[],[1981],{"type":59,"value":136},{"type":59,"value":178},{"type":54,"tag":61,"props":1984,"children":1985},{},[1986,1990],{"type":54,"tag":70,"props":1987,"children":1988},{},[1989],{"type":59,"value":1423},{"type":59,"value":1991}," every required slot filled, with a source URL or scaffold-fallback note.",{"type":54,"tag":180,"props":1993,"children":1994},{},[],{"type":54,"tag":1153,"props":1996,"children":1998},{"id":1997},"step-4-generate-project-smoke-test",[1999],{"type":59,"value":2000},"Step 4 — Generate project & smoke-test",{"type":54,"tag":61,"props":2002,"children":2003},{},[2004,2008,2010,2015,2017,2022],{"type":54,"tag":70,"props":2005,"children":2006},{},[2007],{"type":59,"value":1166},{"type":59,"value":2009}," write all scripts, build the image, prepare data, run a 1-step smoke on\nreal data (one ",{"type":54,"tag":91,"props":2011,"children":2013},{"className":2012},[],[2014],{"type":59,"value":1553},{"type":59,"value":2016},", two ",{"type":54,"tag":91,"props":2018,"children":2020},{"className":2019},[],[2021],{"type":59,"value":690},{"type":59,"value":2023},"s).",{"type":54,"tag":61,"props":2025,"children":2026},{},[2027,2032,2033,2039,2041,2046,2047,2053,2054,2060,2061,2066,2067,2073,2074,2080,2081,2087,2088,2094,2095,2101,2102,2108,2110,2115,2116,2121,2123,2129,2131,2136,2138,2144,2146,2152,2154,2160,2162,2168,2170,2176],{"type":54,"tag":70,"props":2028,"children":2029},{},[2030],{"type":59,"value":2031},"4a. Generate project files",{"type":59,"value":1600},{"type":54,"tag":91,"props":2034,"children":2036},{"className":2035},[],[2037],{"type":59,"value":2038},"output_dir\u002F",{"type":59,"value":2040},": ",{"type":54,"tag":91,"props":2042,"children":2044},{"className":2043},[],[2045],{"type":59,"value":118},{"type":59,"value":98},{"type":54,"tag":91,"props":2048,"children":2050},{"className":2049},[],[2051],{"type":59,"value":2052},"Dockerfile",{"type":59,"value":789},{"type":54,"tag":91,"props":2055,"children":2057},{"className":2056},[],[2058],{"type":59,"value":2059},"requirements.txt",{"type":59,"value":98},{"type":54,"tag":91,"props":2062,"children":2064},{"className":2063},[],[2065],{"type":59,"value":1125},{"type":59,"value":98},{"type":54,"tag":91,"props":2068,"children":2070},{"className":2069},[],[2071],{"type":59,"value":2072},"train.py",{"type":59,"value":98},{"type":54,"tag":91,"props":2075,"children":2077},{"className":2076},[],[2078],{"type":59,"value":2079},"run_eval.py",{"type":59,"value":98},{"type":54,"tag":91,"props":2082,"children":2084},{"className":2083},[],[2085],{"type":59,"value":2086},"infer.py",{"type":59,"value":1201},{"type":54,"tag":91,"props":2089,"children":2091},{"className":2090},[],[2092],{"type":59,"value":2093},"merge_lora.py",{"type":59,"value":1187},{"type":54,"tag":91,"props":2096,"children":2098},{"className":2097},[],[2099],{"type":59,"value":2100},"tests\u002F",{"type":59,"value":98},{"type":54,"tag":91,"props":2103,"children":2105},{"className":2104},[],[2106],{"type":59,"value":2107},".gitignore",{"type":59,"value":2109},". Live Step 3 research is\nauthority; ",{"type":54,"tag":91,"props":2111,"children":2113},{"className":2112},[],[2114],{"type":59,"value":906},{"type":59,"value":1024},{"type":54,"tag":91,"props":2117,"children":2119},{"className":2118},[],[2120],{"type":59,"value":913},{"type":59,"value":2122}," give scaffold shape only. Apply every\n",{"type":54,"tag":91,"props":2124,"children":2126},{"className":2125},[],[2127],{"type":59,"value":2128},"applicable_workarounds",{"type":59,"value":2130}," entry as a Dockerfile block, requirement pin, config\noverride, or runtime env var. Hard rules: ",{"type":54,"tag":91,"props":2132,"children":2134},{"className":2133},[],[2135],{"type":59,"value":2079},{"type":59,"value":2137}," keeps that exact filename\n(avoids colliding with the HF ",{"type":54,"tag":91,"props":2139,"children":2141},{"className":2140},[],[2142],{"type":59,"value":2143},"evaluate",{"type":59,"value":2145}," package); every generated ",{"type":54,"tag":91,"props":2147,"children":2149},{"className":2148},[],[2150],{"type":59,"value":2151},".py",{"type":59,"value":2153}," starts\nwith the NVIDIA Apache-2.0 copyright header and any emitter fails when it is\nmissing; ",{"type":54,"tag":91,"props":2155,"children":2157},{"className":2156},[],[2158],{"type":59,"value":2159},"emit_unit_tests: true",{"type":59,"value":2161}," generates and runs tests per\n",{"type":54,"tag":91,"props":2163,"children":2165},{"className":2164},[],[2166],{"type":59,"value":2167},"references\u002Ftesting.md",{"type":59,"value":2169},". Script bodies, Dockerfile shape, and the emitter contract\nare in ",{"type":54,"tag":91,"props":2171,"children":2173},{"className":2172},[],[2174],{"type":59,"value":2175},"references\u002Fworkflow-generate-train.md",{"type":59,"value":178},{"type":54,"tag":61,"props":2178,"children":2179},{},[2180,2185,2187,2193,2195,2201,2203,2209,2211,2217,2219,2225],{"type":54,"tag":70,"props":2181,"children":2182},{},[2183],{"type":59,"value":2184},"4b. Build, prepare, smoke",{"type":59,"value":2186}," — ",{"type":54,"tag":91,"props":2188,"children":2190},{"className":2189},[],[2191],{"type":59,"value":2192},"docker build -t run-\u003Cshort>:latest .",{"type":59,"value":2194},", then\n",{"type":54,"tag":91,"props":2196,"children":2198},{"className":2197},[],[2199],{"type":59,"value":2200},"prepare_data",{"type":59,"value":2202}," and the ",{"type":54,"tag":91,"props":2204,"children":2206},{"className":2205},[],[2207],{"type":59,"value":2208},"--smoke --max_steps 1",{"type":59,"value":2210}," run (",{"type":54,"tag":91,"props":2212,"children":2214},{"className":2213},[],[2215],{"type":59,"value":2216},"references\u002Fdocker-runs.md",{"type":59,"value":2218},"\n§1-3). Smoke pass criteria (in ",{"type":54,"tag":91,"props":2220,"children":2222},{"className":2221},[],[2223],{"type":59,"value":2224},"logs\u002Fsmoke.log",{"type":59,"value":1466},{"type":54,"tag":199,"props":2227,"children":2228},{},[2229,2234,2254],{"type":54,"tag":80,"props":2230,"children":2231},{},[2232],{"type":59,"value":2233},"No exception",{"type":54,"tag":80,"props":2235,"children":2236},{},[2237,2239,2245,2247,2253],{"type":59,"value":2238},"Loss is finite (not ",{"type":54,"tag":91,"props":2240,"children":2242},{"className":2241},[],[2243],{"type":59,"value":2244},"0.0",{"type":59,"value":2246},", not ",{"type":54,"tag":91,"props":2248,"children":2250},{"className":2249},[],[2251],{"type":59,"value":2252},"NaN",{"type":59,"value":330},{"type":54,"tag":80,"props":2255,"children":2256},{},[2257,2263],{"type":54,"tag":91,"props":2258,"children":2260},{"className":2259},[],[2261],{"type":59,"value":2262},"grad_norm > 0",{"type":59,"value":2264}," at step 1",{"type":54,"tag":61,"props":2266,"children":2267},{},[2268,2270,2275,2277,2283],{"type":59,"value":2269},"If ",{"type":54,"tag":91,"props":2271,"children":2273},{"className":2272},[],[2274],{"type":59,"value":2159},{"type":59,"value":2276},", also run ",{"type":54,"tag":91,"props":2278,"children":2280},{"className":2279},[],[2281],{"type":59,"value":2282},"pytest tests\u002F",{"type":59,"value":2284}," in the container. Any failure → STOP.",{"type":54,"tag":61,"props":2286,"children":2287},{},[2288,2293],{"type":54,"tag":70,"props":2289,"children":2290},{},[2291],{"type":59,"value":2292},"4c. Preflight summary",{"type":59,"value":2294}," — before full training, print and verify: reference URL,\ndataset columns, Hub target, monitoring target, NGC image, hardware, smoke loss\u002Fgrad norm.",{"type":54,"tag":61,"props":2296,"children":2297},{},[2298,2302],{"type":54,"tag":70,"props":2299,"children":2300},{},[2301],{"type":59,"value":1423},{"type":59,"value":2303}," project files written, image built, smoke PASSED, preflight has no\nblank fields.",{"type":54,"tag":180,"props":2305,"children":2306},{},[],{"type":54,"tag":1153,"props":2308,"children":2310},{"id":2309},"step-5-train-evaluate-infer",[2311],{"type":59,"value":2312},"Step 5 — Train, evaluate, infer",{"type":54,"tag":61,"props":2314,"children":2315},{},[2316,2320,2322,2327],{"type":54,"tag":70,"props":2317,"children":2318},{},[2319],{"type":59,"value":1166},{"type":59,"value":2321}," baseline eval, full training, post-train eval, optional LoRA merge, 5\ninference samples (all commands: ",{"type":54,"tag":91,"props":2323,"children":2325},{"className":2324},[],[2326],{"type":59,"value":2216},{"type":59,"value":2328}," §4-8).",{"type":54,"tag":630,"props":2330,"children":2331},{},[2332,2352],{"type":54,"tag":634,"props":2333,"children":2334},{},[2335],{"type":54,"tag":638,"props":2336,"children":2337},{},[2338,2343,2347],{"type":54,"tag":642,"props":2339,"children":2340},{},[2341],{"type":59,"value":2342},"Sub-step",{"type":54,"tag":642,"props":2344,"children":2345},{},[2346],{"type":59,"value":920},{"type":54,"tag":642,"props":2348,"children":2349},{},[2350],{"type":59,"value":2351},"Skip if",{"type":54,"tag":653,"props":2353,"children":2354},{},[2355,2377,2395,2413,2430],{"type":54,"tag":638,"props":2356,"children":2357},{},[2358,2363,2368],{"type":54,"tag":660,"props":2359,"children":2360},{},[2361],{"type":59,"value":2362},"5a. Baseline eval (zero-shot)",{"type":54,"tag":660,"props":2364,"children":2365},{},[2366],{"type":59,"value":2367},"§4",{"type":54,"tag":660,"props":2369,"children":2370},{},[2371],{"type":54,"tag":91,"props":2372,"children":2374},{"className":2373},[],[2375],{"type":59,"value":2376},"skip_baseline: true",{"type":54,"tag":638,"props":2378,"children":2379},{},[2380,2385,2390],{"type":54,"tag":660,"props":2381,"children":2382},{},[2383],{"type":59,"value":2384},"5b. Full training (detached)",{"type":54,"tag":660,"props":2386,"children":2387},{},[2388],{"type":59,"value":2389},"§5",{"type":54,"tag":660,"props":2391,"children":2392},{},[2393],{"type":59,"value":2394},"—",{"type":54,"tag":638,"props":2396,"children":2397},{},[2398,2403,2408],{"type":54,"tag":660,"props":2399,"children":2400},{},[2401],{"type":59,"value":2402},"5c. LoRA merge",{"type":54,"tag":660,"props":2404,"children":2405},{},[2406],{"type":59,"value":2407},"§6",{"type":54,"tag":660,"props":2409,"children":2410},{},[2411],{"type":59,"value":2412},"not VLM+LoRA",{"type":54,"tag":638,"props":2414,"children":2415},{},[2416,2421,2426],{"type":54,"tag":660,"props":2417,"children":2418},{},[2419],{"type":59,"value":2420},"5d. Post-train eval",{"type":54,"tag":660,"props":2422,"children":2423},{},[2424],{"type":59,"value":2425},"§7",{"type":54,"tag":660,"props":2427,"children":2428},{},[2429],{"type":59,"value":2394},{"type":54,"tag":638,"props":2431,"children":2432},{},[2433,2438,2443],{"type":54,"tag":660,"props":2434,"children":2435},{},[2436],{"type":59,"value":2437},"5e. Inference (5 samples)",{"type":54,"tag":660,"props":2439,"children":2440},{},[2441],{"type":59,"value":2442},"§8",{"type":54,"tag":660,"props":2444,"children":2445},{},[2446],{"type":59,"value":2394},{"type":54,"tag":61,"props":2448,"children":2449},{},[2450,2452,2458,2460,2466],{"type":59,"value":2451},"Multi-GPU: prepend ",{"type":54,"tag":91,"props":2453,"children":2455},{"className":2454},[],[2456],{"type":59,"value":2457},"torchrun --nproc_per_node=$gpu_count",{"type":59,"value":2459}," to ",{"type":54,"tag":91,"props":2461,"children":2463},{"className":2462},[],[2464],{"type":59,"value":2465},"python train.py",{"type":59,"value":178},{"type":54,"tag":61,"props":2468,"children":2469},{},[2470,2472,2478,2480,2485,2487,2493,2495,2501,2503,2509],{"type":59,"value":2471},"While training streams, watch ",{"type":54,"tag":91,"props":2473,"children":2475},{"className":2474},[],[2476],{"type":59,"value":2477},"docker logs -f hft_train",{"type":59,"value":2479},": loss should drop within\n10-20 steps; flat loss (collator\u002Flabel-masking bug), NaN (LR too high), and OOM\nall stop the run — recovery in ",{"type":54,"tag":91,"props":2481,"children":2483},{"className":2482},[],[2484],{"type":59,"value":1058},{"type":59,"value":2486},". If ",{"type":54,"tag":91,"props":2488,"children":2490},{"className":2489},[],[2491],{"type":59,"value":2492},"emit_report: true",{"type":59,"value":2494},",\nrun ",{"type":54,"tag":91,"props":2496,"children":2498},{"className":2497},[],[2499],{"type":59,"value":2500},"report.py",{"type":59,"value":2502}," after Step 5e per ",{"type":54,"tag":91,"props":2504,"children":2506},{"className":2505},[],[2507],{"type":59,"value":2508},"references\u002Freporting.md",{"type":59,"value":178},{"type":54,"tag":61,"props":2511,"children":2512},{},[2513,2517],{"type":54,"tag":70,"props":2514,"children":2515},{},[2516],{"type":59,"value":1423},{"type":59,"value":2518}," all of:",{"type":54,"tag":199,"props":2520,"children":2521},{},[2522,2541,2552,2563,2574],{"type":54,"tag":80,"props":2523,"children":2524},{},[2525,2531,2533,2539],{"type":54,"tag":91,"props":2526,"children":2528},{"className":2527},[],[2529],{"type":59,"value":2530},"checkpoints\u002Ffinal\u002F",{"type":59,"value":2532}," (or ",{"type":54,"tag":91,"props":2534,"children":2536},{"className":2535},[],[2537],{"type":59,"value":2538},"checkpoints\u002Fmerged\u002F",{"type":59,"value":2540}," for LoRA) exists",{"type":54,"tag":80,"props":2542,"children":2543},{},[2544,2550],{"type":54,"tag":91,"props":2545,"children":2547},{"className":2546},[],[2548],{"type":59,"value":2549},"reports\u002Feval_results.json",{"type":59,"value":2551}," has a numeric primary metric",{"type":54,"tag":80,"props":2553,"children":2554},{},[2555,2561],{"type":54,"tag":91,"props":2556,"children":2558},{"className":2557},[],[2559],{"type":59,"value":2560},"reports\u002Fbaseline_results.json",{"type":59,"value":2562}," exists (unless skipped)",{"type":54,"tag":80,"props":2564,"children":2565},{},[2566,2572],{"type":54,"tag":91,"props":2567,"children":2569},{"className":2568},[],[2570],{"type":59,"value":2571},"reports\u002Finference_samples\u002F",{"type":59,"value":2573}," has 5 samples",{"type":54,"tag":80,"props":2575,"children":2576},{},[2577],{"type":59,"value":2578},"wandb URL shows descending loss",{"type":54,"tag":180,"props":2580,"children":2581},{},[],{"type":54,"tag":1153,"props":2583,"children":2585},{"id":2584},"step-6-push-emit-rerun-skill",[2586],{"type":59,"value":2587},"Step 6 — Push & emit rerun skill",{"type":54,"tag":61,"props":2589,"children":2590},{},[2591,2595],{"type":54,"tag":70,"props":2592,"children":2593},{},[2594],{"type":59,"value":1166},{"type":59,"value":2596}," publish the run and make it reproducible without re-research.",{"type":54,"tag":61,"props":2598,"children":2599},{},[2600,2602,2608,2610,2615,2616,2621,2622,2627,2629,2634,2636,2642,2644,2650],{"type":59,"value":2601},"Push per ",{"type":54,"tag":91,"props":2603,"children":2605},{"className":2604},[],[2606],{"type":59,"value":2607},"references\u002Fhub-push.md",{"type":59,"value":2609}," (weights, model card, eval\u002Fbaseline JSONs,\n",{"type":54,"tag":91,"props":2611,"children":2613},{"className":2612},[],[2614],{"type":59,"value":118},{"type":59,"value":98},{"type":54,"tag":91,"props":2617,"children":2619},{"className":2618},[],[2620],{"type":59,"value":2052},{"type":59,"value":98},{"type":54,"tag":91,"props":2623,"children":2625},{"className":2624},[],[2626],{"type":59,"value":2059},{"type":59,"value":2628},", inference samples, reports when\nemitted) unless ",{"type":54,"tag":91,"props":2630,"children":2632},{"className":2631},[],[2633],{"type":59,"value":260},{"type":59,"value":2635}," is explicit. Emit\n",{"type":54,"tag":91,"props":2637,"children":2639},{"className":2638},[],[2640],{"type":59,"value":2641},"\u003Coutput_dir>\u002Fskills\u002Frun-\u003Cshort>\u002FSKILL.md",{"type":59,"value":2643}," from\n",{"type":54,"tag":91,"props":2645,"children":2647},{"className":2646},[],[2648],{"type":59,"value":2649},"references\u002Fpipeline-skill-template.md",{"type":59,"value":2651}," — substitute every placeholder, include\nfull YAML metadata + the NVIDIA copyright HTML comment, and make any emitter fail\nif those are missing.",{"type":54,"tag":61,"props":2653,"children":2654},{},[2655,2660],{"type":54,"tag":70,"props":2656,"children":2657},{},[2658],{"type":59,"value":2659},"Gate (Done criteria):",{"type":59,"value":2518},{"type":54,"tag":199,"props":2662,"children":2663},{},[2664,2669,2688],{"type":54,"tag":80,"props":2665,"children":2666},{},[2667],{"type":59,"value":2668},"Step 5 gate met",{"type":54,"tag":80,"props":2670,"children":2671},{},[2672,2674,2680,2682,2687],{"type":59,"value":2673},"HF Hub repo exists at the resolved URL with weights + card + ",{"type":54,"tag":91,"props":2675,"children":2677},{"className":2676},[],[2678],{"type":59,"value":2679},"results\u002F",{"type":59,"value":2681},"\n(unless ",{"type":54,"tag":91,"props":2683,"children":2685},{"className":2684},[],[2686],{"type":59,"value":260},{"type":59,"value":330},{"type":54,"tag":80,"props":2689,"children":2690},{},[2691,2696,2698,2704,2706],{"type":54,"tag":91,"props":2692,"children":2694},{"className":2693},[],[2695],{"type":59,"value":2641},{"type":59,"value":2697}," exists, no ",{"type":54,"tag":91,"props":2699,"children":2701},{"className":2700},[],[2702],{"type":59,"value":2703},"\u003Cplaceholder>",{"type":59,"value":2705}," left,\nwith metadata + copyright HTML comment per ",{"type":54,"tag":91,"props":2707,"children":2709},{"className":2708},[],[2710],{"type":59,"value":934},{"type":54,"tag":61,"props":2712,"children":2713},{},[2714,2716,2721],{"type":59,"value":2715},"Final message: wandb URL, HF Hub URL, baseline -> fine-tuned primary metric,\n",{"type":54,"tag":91,"props":2717,"children":2719},{"className":2718},[],[2720],{"type":59,"value":2571},{"type":59,"value":2722},", and the rerun skill path.",{"type":54,"tag":180,"props":2724,"children":2725},{},[],{"type":54,"tag":184,"props":2727,"children":2729},{"id":2728},"error-playbook",[2730],{"type":59,"value":2731},"Error playbook",{"type":54,"tag":61,"props":2733,"children":2734},{},[2735,2737,2743,2745,2750,2752,2757],{"type":59,"value":2736},"On a known runtime error, consult the symptom → minimal-fix table in\n",{"type":54,"tag":91,"props":2738,"children":2740},{"className":2739},[],[2741],{"type":59,"value":2742},"references\u002Ferror-playbook.md",{"type":59,"value":2744}," (NGC entrypoint, PyTorch\u002FTransformers regressions,\nnumpy ABI, Albumentations bbox, PEFT\u002Fcheckpointing, LoRA target breadth, CV\naugmentation gaps, OOM at step 0) before redesigning anything. When a row there\nfires twice across runs, lift it into ",{"type":54,"tag":91,"props":2746,"children":2748},{"className":2747},[],[2749],{"type":59,"value":857},{"type":59,"value":2751}," with a ",{"type":54,"tag":91,"props":2753,"children":2755},{"className":2754},[],[2756],{"type":59,"value":1746},{"type":59,"value":2758}," rule\n— auto-applied in Step 1 before the error can fire.",{"type":54,"tag":180,"props":2760,"children":2761},{},[],{"type":54,"tag":184,"props":2763,"children":2765},{"id":2764},"communication-style",[2766],{"type":59,"value":2767},"Communication style",{"type":54,"tag":199,"props":2769,"children":2770},{},[2771,2776,2781,2786],{"type":54,"tag":80,"props":2772,"children":2773},{},[2774],{"type":59,"value":2775},"Terse. No filler, no restating the request; one-word answers when appropriate.",{"type":54,"tag":80,"props":2777,"children":2778},{},[2779],{"type":59,"value":2780},"Always include direct Hub and wandb URLs when referencing artifacts.",{"type":54,"tag":80,"props":2782,"children":2783},{},[2784],{"type":59,"value":2785},"On error: state what went wrong, why, what you changed — no menus.",{"type":54,"tag":80,"props":2787,"children":2788},{},[2789],{"type":59,"value":2790},"Never present \"Option A\u002FB\u002FC\" for a request with a clear answer. Act.",{"type":54,"tag":184,"props":2792,"children":2794},{"id":2793},"example-pipelines",[2795],{"type":59,"value":2796},"Example pipelines",{"type":54,"tag":199,"props":2798,"children":2799},{},[2800,2809,2818,2827],{"type":54,"tag":80,"props":2801,"children":2802},{},[2803],{"type":54,"tag":669,"props":2804,"children":2806},{"href":2805},"references\u002Ftao-rerun-convnext-cifar10.md",[2807],{"type":59,"value":2808},"tao-rerun-convnext-cifar10",{"type":54,"tag":80,"props":2810,"children":2811},{},[2812],{"type":54,"tag":669,"props":2813,"children":2815},{"href":2814},"references\u002Ftao-rerun-detr-cppe5.md",[2816],{"type":59,"value":2817},"tao-rerun-detr-cppe5",{"type":54,"tag":80,"props":2819,"children":2820},{},[2821],{"type":54,"tag":669,"props":2822,"children":2824},{"href":2823},"references\u002Ftao-rerun-segformer-foodseg103.md",[2825],{"type":59,"value":2826},"tao-rerun-segformer-foodseg103",{"type":54,"tag":80,"props":2828,"children":2829},{},[2830],{"type":54,"tag":669,"props":2831,"children":2833},{"href":2832},"references\u002Ftao-rerun-smolvlm-vqav2.md",[2834],{"type":59,"value":2835},"tao-rerun-smolvlm-vqav2",{"type":54,"tag":2837,"props":2838,"children":2839},"style",{},[2840],{"type":59,"value":2841},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":2843,"total":2947},[2844,2861,2875,2889,2901,2918,2933],{"slug":2845,"name":2845,"fn":2846,"description":2847,"org":2848,"tags":2849,"stars":26,"repoUrl":27,"updatedAt":2860},"accelerated-computing-cudf","accelerate data processing with cuDF","Official NVIDIA-authored guidance for NVIDIA cuDF GPU DataFrames, pandas acceleration, dask-cuDF, ETL, joins, groupby, CSV\u002FParquet I\u002FO, nullable semantics, and multi-GPU DataFrame workloads.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2850,2853,2856,2857],{"name":2851,"slug":2852,"type":15},"Data Analysis","data-analysis",{"name":2854,"slug":2855,"type":15},"Data Engineering","data-engineering",{"name":9,"slug":8,"type":15},{"name":2858,"slug":2859,"type":15},"Performance","performance","2026-07-14T05:28:43.176466",{"slug":2862,"name":2862,"fn":2863,"description":2864,"org":2865,"tags":2866,"stars":26,"repoUrl":27,"updatedAt":2874},"aiq-deploy","deploy and manage NVIDIA AI-Q infrastructure","Use when asked to install, deploy, run, validate, troubleshoot, or stop NVIDIA AI-Q Blueprint infrastructure.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2867,2870,2873],{"name":2868,"slug":2869,"type":15},"Deployment","deployment",{"name":2871,"slug":2872,"type":15},"Infrastructure","infrastructure",{"name":9,"slug":8,"type":15},"2026-07-14T05:29:06.667109",{"slug":2876,"name":2876,"fn":2877,"description":2878,"org":2879,"tags":2880,"stars":26,"repoUrl":27,"updatedAt":2888},"aiq-research","conduct deep research with AI-Q","Use when asked to run deep research or AI-Q research through a reachable NVIDIA AI-Q Blueprint backend.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2881,2884,2885],{"name":2882,"slug":2883,"type":15},"Agents","agents",{"name":9,"slug":8,"type":15},{"name":2886,"slug":2887,"type":15},"Research","research","2026-07-14T05:28:06.816956",{"slug":2890,"name":2890,"fn":2891,"description":2892,"org":2893,"tags":2894,"stars":26,"repoUrl":27,"updatedAt":2900},"amc-run-sample-calibration","run AMC sample dataset calibration","Run end-to-end calibration on the shipped sample dataset (sdg_08_2_sample_data_010926.zip) against a running AMC microservice. Use when user says 'test sample dataset', 'run sample calibration', 'verify AMC install', or 'launch and test'.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2895,2896,2897],{"name":2851,"slug":2852,"type":15},{"name":9,"slug":8,"type":15},{"name":2898,"slug":2899,"type":15},"Testing","testing","2026-07-17T05:29:03.913266",{"slug":2902,"name":2902,"fn":2903,"description":2904,"org":2905,"tags":2906,"stars":26,"repoUrl":27,"updatedAt":2917},"amc-run-video-calibration","calibrate video datasets with AutoMagicCalib","Calibrate a new dataset from pre-recorded video files via the AutoMagicCalib REST API. Use when user has local MP4s and says 'calibrate my videos', 'run AMC on these videos', or similar. For RTSP\u002Flive streams, use amc-run-rtsp-calibration instead.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2907,2910,2913,2914],{"name":2908,"slug":2909,"type":15},"Automation","automation",{"name":2911,"slug":2912,"type":15},"Imaging","imaging",{"name":9,"slug":8,"type":15},{"name":2915,"slug":2916,"type":15},"Video","video","2026-07-17T05:28:53.905004",{"slug":2919,"name":2919,"fn":2920,"description":2921,"org":2922,"tags":2923,"stars":26,"repoUrl":27,"updatedAt":2932},"amc-setup-calibration-stack","deploy AutoMagicCalib microservice with Docker","Launch AutoMagicCalib microservice and web UI from NGC release images via Docker Compose. Use when user says 'deploy auto calibration', 'launch auto calibration', 'launch AMC', 'start MS+UI', or 'set up auto-magic-calib'. Requires NGC API key.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2924,2925,2928,2929],{"name":2868,"slug":2869,"type":15},{"name":2926,"slug":2927,"type":15},"Docker","docker",{"name":9,"slug":8,"type":15},{"name":2930,"slug":2931,"type":15},"Operations","operations","2026-07-17T05:28:56.913999",{"slug":2934,"name":2934,"fn":2935,"description":2936,"org":2937,"tags":2938,"stars":26,"repoUrl":27,"updatedAt":2946},"cudaq-guide","develop quantum applications with CUDA-Q","CUDA-Q onboarding guide for installation, test programs, GPU simulation, QPU hardware, and quantum applications.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2939,2940,2943],{"name":9,"slug":8,"type":15},{"name":2941,"slug":2942,"type":15},"Quantum Computing","quantum-computing",{"name":2944,"slug":2945,"type":15},"Simulation","simulation","2026-07-14T05:26:58.898253",305,{"items":2949,"total":3098},[2950,2968,2984,2995,3007,3021,3034,3046,3057,3066,3080,3089],{"slug":2951,"name":2951,"fn":2952,"description":2953,"org":2954,"tags":2955,"stars":2965,"repoUrl":2966,"updatedAt":2967},"nemoclaw-user-guide","retrieve NemoClaw documentation and configuration","Guides human users' AI agents to the NemoClaw docs MCP server and canonical Fern documentation in Markdown form. Use when users ask how to install, configure, operate, troubleshoot, secure, or learn NemoClaw with an AI coding assistant. Trigger keywords - nemoclaw docs, use nemoclaw with ai agent, nemoclaw mcp docs, nemoclaw install help, nemoclaw quickstart, nemoclaw markdown docs, llms.txt, agent skills.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2956,2959,2962],{"name":2957,"slug":2958,"type":15},"Documentation","documentation",{"name":2960,"slug":2961,"type":15},"MCP","mcp",{"name":2963,"slug":2964,"type":15},"Search","search",21777,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FNemoClaw","2026-07-20T06:00:01.461044",{"slug":2969,"name":2969,"fn":2970,"description":2971,"org":2972,"tags":2973,"stars":2981,"repoUrl":2982,"updatedAt":2983},"mcore-build-and-dependency","manage Megatron-LM development environments","Container-based dev environment setup and dependency management for Megatron-LM. Covers acquiring and launching the CI container, uv package management, and updating uv.lock.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2974,2977,2978],{"name":2975,"slug":2976,"type":15},"Containers","containers",{"name":2868,"slug":2869,"type":15},{"name":2979,"slug":2980,"type":15},"Python","python",17049,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FMegatron-LM","2026-07-27T06:06:11.249662",{"slug":2985,"name":2985,"fn":2986,"description":2987,"org":2988,"tags":2989,"stars":2981,"repoUrl":2982,"updatedAt":2994},"mcore-bump-base-image","update NVIDIA PyTorch base images","Bump the NVIDIA PyTorch base image (`nvcr.io\u002Fnvidia\u002Fpytorch:YY.MM-py3`) used by Megatron-LM CI. Covers the two pin sites (GitHub CI in `docker\u002F.ngc_version.dev` and GitLab CI in `.gitlab\u002Fstages\u002F01.build.yml`), the post-bump CI loop (re-run functional tests, refresh golden values, mark broken tests), and the gotchas that bit PRs",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2990,2993],{"name":2991,"slug":2992,"type":15},"CI\u002FCD","ci-cd",{"name":2868,"slug":2869,"type":15},"2026-07-14T05:25:59.97109",{"slug":2996,"name":2996,"fn":2997,"description":2998,"org":2999,"tags":3000,"stars":2981,"repoUrl":2982,"updatedAt":3006},"mcore-cicd","manage CI\u002FCD pipelines for Megatron-LM","CI\u002FCD reference for Megatron-LM. Covers CI pipeline structure, PR scope labels, triggering internal GitLab CI (which force-pushes the current branch to a pull-request\u002FBRANCH ref — always dry-run and verify the destination first; never run against shared or protected branches), and CI failure investigation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3001,3002,3003],{"name":2991,"slug":2992,"type":15},{"name":2868,"slug":2869,"type":15},{"name":3004,"slug":3005,"type":15},"GitHub","github","2026-07-27T06:06:12.278222",{"slug":3008,"name":3008,"fn":3009,"description":3010,"org":3011,"tags":3012,"stars":2981,"repoUrl":2982,"updatedAt":3020},"mcore-create-issue","investigate CI failures and create issues","Investigate a failing GitHub Actions run or job and create a GitHub issue for the failure.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3013,3016,3017],{"name":3014,"slug":3015,"type":15},"Debugging","debugging",{"name":3004,"slug":3005,"type":15},{"name":3018,"slug":3019,"type":15},"Triage","triage","2026-07-14T05:25:57.442089",{"slug":3022,"name":3022,"fn":3023,"description":3024,"org":3025,"tags":3026,"stars":2981,"repoUrl":2982,"updatedAt":3033},"mcore-linting-and-formatting","lint and format Megatron-LM code","Linting and formatting for Megatron-LM. Covers running autoformat.sh, tools (ruff, black, isort, pylint, mypy), and code style rules.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3027,3030],{"name":3028,"slug":3029,"type":15},"Best Practices","best-practices",{"name":3031,"slug":3032,"type":15},"Code Analysis","code-analysis","2026-07-14T05:25:56.18433",{"slug":3035,"name":3035,"fn":3036,"description":3037,"org":3038,"tags":3039,"stars":2981,"repoUrl":2982,"updatedAt":3045},"mcore-migrate-gpt-to-hybrid","migrate Megatron-LM models to HybridModel","Migration guide for moving Megatron Core GPTModel checkpoints, model providers, training commands, and layer mappings to HybridModel.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3040,3041,3044],{"name":23,"slug":24,"type":15},{"name":3042,"slug":3043,"type":15},"Migration","migration",{"name":9,"slug":8,"type":15},"2026-07-17T06:07:11.777011",{"slug":3047,"name":3047,"fn":3048,"description":3049,"org":3050,"tags":3051,"stars":2981,"repoUrl":2982,"updatedAt":3056},"mcore-onboard-gb200-1node-tests","onboard functional tests for GB200","Onboard 1-node GitHub MR functional tests for GB200 from existing mr-scoped 2-node tests.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3052,3055],{"name":3053,"slug":3054,"type":15},"QA","qa",{"name":2898,"slug":2899,"type":15},"2026-07-14T05:25:53.673039",{"slug":3058,"name":3058,"fn":3059,"description":3060,"org":3061,"tags":3062,"stars":2981,"repoUrl":2982,"updatedAt":3065},"mcore-run-on-slurm","launch distributed training jobs on SLURM","How to launch distributed Megatron-LM training jobs on a SLURM cluster. Covers a minimal sbatch skeleton, environment-variable setup for torch.distributed.run, CUDA_DEVICE_MAX_CONNECTIONS rules across hardware and parallelism modes, container conventions, monitoring, and per-rank failure diagnosis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3063,3064],{"name":2868,"slug":2869,"type":15},{"name":2871,"slug":2872,"type":15},"2026-07-14T05:25:49.362534",{"slug":3067,"name":3067,"fn":3068,"description":3069,"org":3070,"tags":3071,"stars":2981,"repoUrl":2982,"updatedAt":3079},"mcore-split-pr","split pull requests to reduce review load","Split a PR into multiple PRs to reduce the number of required CODEOWNERS reviewer groups.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3072,3075,3076],{"name":3073,"slug":3074,"type":15},"Code Review","code-review",{"name":3004,"slug":3005,"type":15},{"name":3077,"slug":3078,"type":15},"Pull Requests","pull-requests","2026-07-14T05:26:01.226578",{"slug":3081,"name":3081,"fn":3082,"description":3083,"org":3084,"tags":3085,"stars":2981,"repoUrl":2982,"updatedAt":3088},"mcore-testing","run and manage Megatron-LM tests","Test system for Megatron-LM. Covers test layout, recipe YAML structure, adding and running unit and functional tests, golden values, marker filters, and CI parity.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3086,3087],{"name":3053,"slug":3054,"type":15},{"name":2898,"slug":2899,"type":15},"2026-07-14T05:25:54.928983",{"slug":3090,"name":3090,"fn":3091,"description":3092,"org":3093,"tags":3094,"stars":2981,"repoUrl":2982,"updatedAt":3097},"nightly-sync","manage nightly main-to-dev sync workflows","Domain knowledge for the nightly main-to-dev sync workflow. Covers merge strategy, CI architecture, failure investigation, and known issues.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3095,3096],{"name":2908,"slug":2909,"type":15},{"name":2991,"slug":2992,"type":15},"2026-07-30T05:29:03.275638",496]