[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-nvidia-tao-run-on-local-docker":3,"mdc--w8i0p9-key":34,"related-repo-nvidia-tao-run-on-local-docker":2309,"related-org-nvidia-tao-run-on-local-docker":2411},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":23,"repoUrl":24,"updatedAt":25,"license":26,"forks":27,"topics":28,"repo":29,"sourceUrl":32,"mdContent":33},"tao-run-on-local-docker","run TAO SDK jobs in Docker","Local or remote Docker execution for TAO SDK job containers using a Docker daemon with NVIDIA GPU runtime. Use when running TAO jobs on the current machine, a directly attached Docker host, or a remote GPU box exposed through DOCKER_HOST. Trigger phrases include \"run locally\", \"local Docker\", \"remote Docker\", \"use my GPU\", \"run on my machine\", \"host Docker daemon\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},"nvidia","NVIDIA","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fnvidia.png",[12,16,19,20],{"name":13,"slug":14,"type":15},"Machine Learning","machine-learning","tag",{"name":17,"slug":18,"type":15},"Deployment","deployment",{"name":9,"slug":8,"type":15},{"name":21,"slug":22,"type":15},"Docker","docker",2473,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fskills","2026-07-14T05:29:10.44444","Apache-2.0",281,[],{"repoUrl":24,"stars":23,"forks":27,"topics":30,"description":31},[],"AI agent skills published by NVIDIA","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fskills\u002Ftree\u002FHEAD\u002Fskills\u002Ftao-run-on-local-docker","---\nname: tao-run-on-local-docker\ndescription: Local or remote Docker execution for TAO SDK job containers using a Docker daemon with NVIDIA GPU runtime. Use\n  when running TAO jobs on the current machine, a directly attached Docker host, or a remote GPU box exposed through\n  DOCKER_HOST. Trigger phrases include \"run locally\", \"local Docker\", \"remote Docker\", \"use my GPU\", \"run on my\n  machine\", \"host Docker daemon\".\nlicense: Apache-2.0\ncompatibility: Requires NVIDIA driver branch 580, CUDA Toolkit 13.0, Docker, and NVIDIA Container Toolkit 1.19.0. The TAO SDK with the docker extra (pip install 'nvidia-tao-sdk[docker]') is needed only if you want Job handles, S3 I\u002FO wrapping, or run-folder durability via ActionWorkflow.\nmetadata:\n  author: NVIDIA Corporation\n  version: \"0.1.0\"\nallowed-tools: Read Bash\ntags:\n- platform\n- local\n- docker\n---\n\n# Local Docker\n\nSingle-node execution platform that runs TAO jobs as named Docker containers on\na Docker daemon. The daemon can be local to the agent host or remote through\n`DOCKER_HOST=ssh:\u002F\u002Fuser@host` \u002F a Docker context. It is useful for development,\ndebugging, small runs, and workflows where a local coding agent submits jobs to\na remote GPU box.\n\nUse local Docker when the data is local to the Docker host or accessible through\nmounted volumes\u002Fcloud credentials. Do not use it for remote cluster scheduling,\nmulti-node training, or jobs that need SLURM queueing.\n\nUse remote Docker when the agent is running on a workstation or laptop but the\nDocker daemon and GPUs are on another single GPU server. In remote Docker mode,\nall local filesystem paths in specs are interpreted on the remote Docker host,\nnot on the agent machine.\n\n## Preflight\n\nThe workflow must verify the host GPU runtime before starting Docker jobs. If\nthe check fails, prompt the user to approve the install, run the printed install\ncommand, and rerun the preflight.\n\n```bash\n# Host GPU runtime: NVIDIA driver 580, CUDA 13.0, NVIDIA Container Toolkit 1.19.0.\nTAO_SKILL_BANK_ROOT=\"${TAO_SKILL_BANK_ROOT:-$PWD}\"\nSETUP_SCRIPT=\"${TAO_SKILL_BANK_ROOT}\u002Fskills\u002Fplatform\u002Ftao-setup-nvidia-gpu-host\u002Fscripts\u002Fsetup-nvidia-gpu-host.sh\"\n\nbash \"$SETUP_SCRIPT\" --backend docker --check-only || {\n  echo \"MISSING: TAO GPU host runtime is not ready.\"\n  echo \"After user approval, run:\"\n  echo \"  bash \\\"$SETUP_SCRIPT\\\" --backend docker --install --yes\"\n  exit 1\n}\n\n# Mode 1 — direct docker (no Python). All you need is docker + the GPU runtime.\ndocker info >\u002Fdev\u002Fnull 2>&1 || { echo \"MISSING: docker daemon not reachable. Start Docker.\"; exit 1; }\ndocker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi >\u002Fdev\u002Fnull 2>&1 || {\n  echo \"MISSING: NVIDIA Container Toolkit not installed\u002Fconfigured. See:\"\n  echo \"  bash \\\"$SETUP_SCRIPT\\\" --backend docker --install --yes\"\n  exit 1\n}\n\n# Mode 2 — TAO SDK wrapper. Adds Job handles, S3 I\u002FO wrapping, ActionWorkflow.\n# Skip this block if Mode 1 is sufficient for the user's request.\n# When Mode 2 is in scope, read `tao-skill-bank:tao-run-platform` for the DockerSDK\n# kwarg contract, build_entrypoint, and monitoring patterns.\n# nvidia-tao-sdk is on public PyPI; pin lives in versions.yaml (wheels.tao_sdk_docker).\nPIN=$(\"${TAO_SKILL_BANK_PATH:?}\u002Fscripts\u002Fresolve_versions_key.py\" wheels.tao_sdk_docker)\npython -c \"import tao_sdk\" 2>\u002Fdev\u002Fnull || python -m pip install \"$PIN\"\npython -c \"import docker\" 2>\u002Fdev\u002Fnull || python -m pip install \"$PIN\"\npython -c \"import tao_sdk, docker\"\n\n# DockerSDK attaches every job container to ${DOCKER_NETWORK:-tao_default}.\n# Create the network if it is missing; the operation is local and idempotent.\nDOCKER_NETWORK_NAME=\"${DOCKER_NETWORK:-tao_default}\"\ndocker network inspect \"$DOCKER_NETWORK_NAME\" >\u002Fdev\u002Fnull 2>&1 || \\\n  docker network create \"$DOCKER_NETWORK_NAME\" >\u002Fdev\u002Fnull\n```\n\nIf a check fails, the agent prompts the user to authorize the install\u002Ffix via Bash before proceeding. Pip-installable Python requirements and Docker network creation above are exceptions: install\u002Fcreate them automatically, then rerun preflight.\n\n## Credentials\n\nThere are no platform credentials required beyond access to the Docker daemon.\n\nOptional environment:\n\n- **DOCKER_HOST**: Optional Docker daemon URL. If unset, the SDK uses the\n  Docker Python client's normal environment\u002Fdefault socket resolution. Required\n  for the `remote-docker` platform option.\n- **DOCKER_NETWORK**: Docker network for job containers. Default is\n  `tao_default`.\n- **DOCKER_USERNAME**: Registry username. Default is `$oauthtoken` for NGC.\n- **NGC_KEY**: Used when pulling private images from `nvcr.io`.\n- **HOST_SSH_PATH**: Mounted into AutoML brain containers when they need SSH keys\n  to monitor remote SLURM child jobs.\n- **ACCESS_KEY**, **SECRET_KEY**, **S3_ENDPOINT_URL**, **S3_BUCKET_NAME**:\n  Optional S3-compatible storage settings for jobs that still read\u002Fwrite cloud\n  storage from a local container.\n\n## Launch Preflight\n\nBefore generating scripts or starting containers:\n\n1. Verify the Docker daemon is reachable, NVIDIA Container Toolkit is registered\n   as a Docker runtime, GPUs and driver version are reported, and a smoke\n   container can see GPUs before launch. For remote Docker, query GPUs through\n   `docker run ... nvidia-smi` against the remote daemon; do not use local\n   `nvidia-smi` from the agent machine.\n2. Verify every local\u002Ffile dataset annotation and media path exists on the\n   Docker host.\n3. For `s3:\u002F\u002F` datasets\u002Fresults, verify `ACCESS_KEY` and `SECRET_KEY` are set\n   and the exact paths are readable with `aws s3 ls`. If `aws` is missing,\n   report the missing dependency and ask before installing it; rerun preflight\n   after installation.\n4. Verify model-specific credentials such as `HF_TOKEN` before launch.\n5. Check current GPU occupancy with `nvidia-smi` and avoid GPUs already used by\n   other running jobs when the user requested that constraint. Show the selected\n   GPU ids in the launch review.\n6. For model\u002Fcontainer combinations with known architecture limits, compare\n   host GPU compute capability with the container stack before launch. If the\n   selected image cannot JIT or run kernels for the host architecture, block\n   early and ask for a compatible image or platform.\n\nUse the packaged helper for these checks when possible:\n\n```bash\n${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external}\u002Fscripts\u002Fcheck_tao_launch_preflight.py \\\n  --platform local-docker \\\n  --container-image \"\u003Cselected-image>\" \\\n  --path train_annotation=\u002Fabs\u002Fpath\u002Fto\u002Fannotations.json \\\n  --path train_media=\u002Fabs\u002Fpath\u002Fto\u002Fmedia\n```\n\nFor a remote Docker daemon, use the `remote-docker` platform and pass or export\n`DOCKER_HOST`. The helper verifies remote GPU\u002Fruntime readiness and checks\nremote-host dataset paths through read-only bind mounts:\n\n```bash\n${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external}\u002Fscripts\u002Fcheck_tao_launch_preflight.py \\\n  --platform remote-docker \\\n  --docker-host ssh:\u002F\u002Fuser@gpu-host \\\n  --container-image \"\u003Cselected-image>\" \\\n  --gpu-smoke-image ubuntu:22.04 \\\n  --path train_annotation=\u002Fremote\u002Fdata\u002Ftrain\u002Fannotations.json \\\n  --path train_media=\u002Fremote\u002Fdata\u002Ftrain\n```\n\nThe `--path` values above must exist on the remote Docker host. Do not pass\npaths that exist only on the local laptop or Codex host.\n\n## Multi-GPU and multi-node\n\n**Multi-node is not supported on local Docker.** One job runs on the local Docker daemon's host with no cross-host coordination.\n\nMulti-GPU **on the local host** is supported via the NVIDIA Container Toolkit's `--gpus` flag (`--gpus all` or `--gpus '\"device=0,1,2,3\"'`). `DockerSDK.create_job(gpu_count=N)` plumbs through to `--gpus`. Single-host distributed init uses `localhost`; `torchrun --nproc-per-node=N` or PyTorch DDP work as usual.\n\n## Backend Details\n\nUse the SDK backend value `local-docker`. The local backend schema has no extra\nbackend details, so most routing is controlled by environment and job\nparameters:\n\n```json\n{\n  \"backend_type\": \"local-docker\",\n  \"num_gpu\": 1\n}\n```\n\nFollowing the Brev SDK design, platform\u002Fcontrol-plane values stay in SDK\nstate and Docker labels. The SDK does not inject `BACKEND`, `HOST_PLATFORM`,\n`MONGOSECRET`, `DOCKER_HOST`, or `DOCKER_NETWORK` into the training container.\n\n## Container Execution\n\nThe TAO SDK local Docker handler starts containers through the Docker Python\nclient:\n\n- Backend job name uses the `tao-job-\u003Cjob_id>` form used by SDK handlers.\n- Command is usually `[\"\u002Fbin\u002Fbash\", \"-c\", \"\u003Cjob command>\"]`.\n- Containers run detached. The SDK keeps containers by default so status and\n  logs remain inspectable, unless `DOCKER_AUTO_REMOVE=true`.\n- `\u002Fdev\u002Fshm` is mounted as tmpfs.\n- The configured Docker network is applied by the Docker daemon for the job\n  container; it is not passed through as a process environment variable.\n- Existing containers with the same job id are stopped and removed before a\n  replacement starts.\n\nFor GPU access, the handler auto-detects the host type:\n\n- Tegra or Jetson hosts use `runtime=\"nvidia\"` plus\n  `NVIDIA_VISIBLE_DEVICES` and `NVIDIA_DRIVER_CAPABILITIES=all`.\n- Standard x86 hosts use Docker `device_requests` with GPU capabilities.\n\nIf `num_gpus` is `0`, no GPUs are assigned. If `num_gpus` is `-1`, all visible\nGPUs are requested. Prefer explicit GPU counts for shared development machines.\nWhen explicit device ids are available, prefer them over count-only selection\non shared machines so the launch does not steal GPUs occupied by other tasks.\n\n## Storage\n\nLocal Docker accepts local and `file:\u002F\u002F` paths because the container runs on the\nsame Docker host. Make sure every path in the spec is either:\n\n- mounted into the container by the handler or surrounding service,\n- reachable from inside the container already, or\n- a cloud URI with matching credentials.\n\nFor remote\u002Fshared filesystems, prefer the platform that owns that filesystem.\nFor example, use SLURM plus `lustre:\u002F\u002F\u002F...` for Lustre paths on a cluster.\n\n## Monitoring\n\n- The SDK handler maps Docker container state directly: created -> Pending,\n  running\u002Frestarting -> Running, paused -> Paused, exit code 0 -> Complete,\n  nonzero exit -> Error.\n- Logs come directly from the named container through the Docker Python client\n  (`docker logs tao-job-\u003Cjob_id>`).\n\nIf the container has exited, died, is being removed, or cannot be found, status\nreconciliation treats the backend process as terminated.\n\n## Cancellation\n\nCancellation stops the named container. GPU ownership is managed by Docker \u002F\nthe NVIDIA runtime, not by TAO Core's local GPU manager.\n\n## Optional: via the TAO SDK\n\nIf you want Job handles, S3 I\u002FO wrapping via the SDK's `script_runner`, or\ndurability across sessions:\n\n```python\nfrom tao_sdk.platforms.docker import DockerSDK\n\nsdk = DockerSDK()  # reads DOCKER_HOST, NGC_KEY, S3 creds from env\njob = sdk.create_job(\n    image='nvcr.io\u002Fnvidia\u002Ftao\u002Ftao-toolkit:6.26.3-pyt',\n    command='dino train -e \u002Ftmp\u002Fspec.yaml',\n    gpu_count=1,\n    inputs={'\u002Fdata\u002Ftrain.json': 's3:\u002F\u002Fbucket\u002Fcoco\u002Ftrain.json'},\n    outputs=['\u002Fresults\u002F'],\n)\n\nstatus = sdk.get_job_status(job.id)\nlogs = sdk.get_job_logs(job.id, tail=200)\n```\n\nThis wraps the same `docker run` invocation under a `Job` handle and routes\nthe entrypoint through `script_runner` so `inputs`\u002F`outputs` get downloaded\nfrom \u002F uploaded to S3 automatically. If you don't need those, just use\n`docker run` directly — no SDK install required.\n\n## Failure Modes\n\n**Docker client not initialized**: Verify the Docker Python package is installed,\nset `DOCKER_HOST` if you are not using the default local socket, and confirm the\nprocess can talk to the daemon.\n\n**GPU assignment failed**: Requested GPUs are unavailable, the NVIDIA Container\nToolkit is not configured, or the Docker daemon cannot create GPU device\nrequests. Use fewer GPUs, wait for another job to finish, or verify\n`docker run --gpus ...` works on the host.\n\n**Image pull auth failed**: Set a valid `NGC_KEY` for private `nvcr.io` images\nor run `docker login nvcr.io -u '$oauthtoken'` on the Docker host.\n\n**Container exited unexpectedly**: Check `docker logs tao-job-\u003Cjob_id>`, the\nconfigured `DOCKER_NETWORK`, and the command produced by the SDK action runner.\n\n**Path missing inside container**: A local path on the host is not necessarily\nmounted into the job container. Use a path convention supported by the action\nrunner or configure an explicit volume through the surrounding service.\n",{"data":35,"body":44},{"name":4,"description":6,"license":26,"compatibility":36,"metadata":37,"allowed-tools":40,"tags":41},"Requires NVIDIA driver branch 580, CUDA Toolkit 13.0, Docker, and NVIDIA Container Toolkit 1.19.0. The TAO SDK with the docker extra (pip install 'nvidia-tao-sdk[docker]') is needed only if you want Job handles, S3 I\u002FO wrapping, or run-folder durability via ActionWorkflow.",{"author":38,"version":39},"NVIDIA Corporation","0.1.0","Read Bash",[42,43,22],"platform","local",{"type":45,"children":46},"root",[47,56,71,76,81,88,93,995,1000,1006,1011,1016,1130,1136,1141,1244,1249,1366,1385,1528,1541,1547,1557,1624,1630,1642,1730,1770,1776,1781,1842,1847,1890,1925,1931,1944,1962,1975,1981,2002,2007,2013,2018,2024,2037,2146,2196,2202,2219,2237,2269,2293,2303],{"type":48,"tag":49,"props":50,"children":52},"element","h1",{"id":51},"local-docker",[53],{"type":54,"value":55},"text","Local Docker",{"type":48,"tag":57,"props":58,"children":59},"p",{},[60,62,69],{"type":54,"value":61},"Single-node execution platform that runs TAO jobs as named Docker containers on\na Docker daemon. The daemon can be local to the agent host or remote through\n",{"type":48,"tag":63,"props":64,"children":66},"code",{"className":65},[],[67],{"type":54,"value":68},"DOCKER_HOST=ssh:\u002F\u002Fuser@host",{"type":54,"value":70}," \u002F a Docker context. It is useful for development,\ndebugging, small runs, and workflows where a local coding agent submits jobs to\na remote GPU box.",{"type":48,"tag":57,"props":72,"children":73},{},[74],{"type":54,"value":75},"Use local Docker when the data is local to the Docker host or accessible through\nmounted volumes\u002Fcloud credentials. Do not use it for remote cluster scheduling,\nmulti-node training, or jobs that need SLURM queueing.",{"type":48,"tag":57,"props":77,"children":78},{},[79],{"type":54,"value":80},"Use remote Docker when the agent is running on a workstation or laptop but the\nDocker daemon and GPUs are on another single GPU server. In remote Docker mode,\nall local filesystem paths in specs are interpreted on the remote Docker host,\nnot on the agent machine.",{"type":48,"tag":82,"props":83,"children":85},"h2",{"id":84},"preflight",[86],{"type":54,"value":87},"Preflight",{"type":48,"tag":57,"props":89,"children":90},{},[91],{"type":54,"value":92},"The workflow must verify the host GPU runtime before starting Docker jobs. If\nthe check fails, prompt the user to approve the install, run the printed install\ncommand, and rerun the preflight.",{"type":48,"tag":94,"props":95,"children":100},"pre",{"className":96,"code":97,"language":98,"meta":99,"style":99},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","# Host GPU runtime: NVIDIA driver 580, CUDA 13.0, NVIDIA Container Toolkit 1.19.0.\nTAO_SKILL_BANK_ROOT=\"${TAO_SKILL_BANK_ROOT:-$PWD}\"\nSETUP_SCRIPT=\"${TAO_SKILL_BANK_ROOT}\u002Fskills\u002Fplatform\u002Ftao-setup-nvidia-gpu-host\u002Fscripts\u002Fsetup-nvidia-gpu-host.sh\"\n\nbash \"$SETUP_SCRIPT\" --backend docker --check-only || {\n  echo \"MISSING: TAO GPU host runtime is not ready.\"\n  echo \"After user approval, run:\"\n  echo \"  bash \\\"$SETUP_SCRIPT\\\" --backend docker --install --yes\"\n  exit 1\n}\n\n# Mode 1 — direct docker (no Python). All you need is docker + the GPU runtime.\ndocker info >\u002Fdev\u002Fnull 2>&1 || { echo \"MISSING: docker daemon not reachable. Start Docker.\"; exit 1; }\ndocker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi >\u002Fdev\u002Fnull 2>&1 || {\n  echo \"MISSING: NVIDIA Container Toolkit not installed\u002Fconfigured. See:\"\n  echo \"  bash \\\"$SETUP_SCRIPT\\\" --backend docker --install --yes\"\n  exit 1\n}\n\n# Mode 2 — TAO SDK wrapper. Adds Job handles, S3 I\u002FO wrapping, ActionWorkflow.\n# Skip this block if Mode 1 is sufficient for the user's request.\n# When Mode 2 is in scope, read `tao-skill-bank:tao-run-platform` for the DockerSDK\n# kwarg contract, build_entrypoint, and monitoring patterns.\n# nvidia-tao-sdk is on public PyPI; pin lives in versions.yaml (wheels.tao_sdk_docker).\nPIN=$(\"${TAO_SKILL_BANK_PATH:?}\u002Fscripts\u002Fresolve_versions_key.py\" wheels.tao_sdk_docker)\npython -c \"import tao_sdk\" 2>\u002Fdev\u002Fnull || python -m pip install \"$PIN\"\npython -c \"import docker\" 2>\u002Fdev\u002Fnull || python -m pip install \"$PIN\"\npython -c \"import tao_sdk, docker\"\n\n# DockerSDK attaches every job container to ${DOCKER_NETWORK:-tao_default}.\n# Create the network if it is missing; the operation is local and idempotent.\nDOCKER_NETWORK_NAME=\"${DOCKER_NETWORK:-tao_default}\"\ndocker network inspect \"$DOCKER_NETWORK_NAME\" >\u002Fdev\u002Fnull 2>&1 || \\\n  docker network create \"$DOCKER_NETWORK_NAME\" >\u002Fdev\u002Fnull\n","bash","",[101],{"type":48,"tag":63,"props":102,"children":103},{"__ignoreMap":99},[104,116,156,193,203,252,275,296,327,342,351,359,368,447,510,531,559,571,579,587,596,605,614,623,632,680,753,818,843,851,860,869,904,956],{"type":48,"tag":105,"props":106,"children":109},"span",{"class":107,"line":108},"line",1,[110],{"type":48,"tag":105,"props":111,"children":113},{"style":112},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[114],{"type":54,"value":115},"# Host GPU runtime: NVIDIA driver 580, CUDA 13.0, NVIDIA Container Toolkit 1.19.0.\n",{"type":48,"tag":105,"props":117,"children":119},{"class":107,"line":118},2,[120,126,132,137,141,146,151],{"type":48,"tag":105,"props":121,"children":123},{"style":122},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[124],{"type":54,"value":125},"TAO_SKILL_BANK_ROOT",{"type":48,"tag":105,"props":127,"children":129},{"style":128},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[130],{"type":54,"value":131},"=",{"type":48,"tag":105,"props":133,"children":134},{"style":128},[135],{"type":54,"value":136},"\"${",{"type":48,"tag":105,"props":138,"children":139},{"style":122},[140],{"type":54,"value":125},{"type":48,"tag":105,"props":142,"children":143},{"style":128},[144],{"type":54,"value":145},":-",{"type":48,"tag":105,"props":147,"children":148},{"style":122},[149],{"type":54,"value":150},"$PWD",{"type":48,"tag":105,"props":152,"children":153},{"style":128},[154],{"type":54,"value":155},"}\"\n",{"type":48,"tag":105,"props":157,"children":159},{"class":107,"line":158},3,[160,165,169,173,177,182,188],{"type":48,"tag":105,"props":161,"children":162},{"style":122},[163],{"type":54,"value":164},"SETUP_SCRIPT",{"type":48,"tag":105,"props":166,"children":167},{"style":128},[168],{"type":54,"value":131},{"type":48,"tag":105,"props":170,"children":171},{"style":128},[172],{"type":54,"value":136},{"type":48,"tag":105,"props":174,"children":175},{"style":122},[176],{"type":54,"value":125},{"type":48,"tag":105,"props":178,"children":179},{"style":128},[180],{"type":54,"value":181},"}",{"type":48,"tag":105,"props":183,"children":185},{"style":184},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[186],{"type":54,"value":187},"\u002Fskills\u002Fplatform\u002Ftao-setup-nvidia-gpu-host\u002Fscripts\u002Fsetup-nvidia-gpu-host.sh",{"type":48,"tag":105,"props":189,"children":190},{"style":128},[191],{"type":54,"value":192},"\"\n",{"type":48,"tag":105,"props":194,"children":196},{"class":107,"line":195},4,[197],{"type":48,"tag":105,"props":198,"children":200},{"emptyLinePlaceholder":199},true,[201],{"type":54,"value":202},"\n",{"type":48,"tag":105,"props":204,"children":206},{"class":107,"line":205},5,[207,212,217,222,227,232,237,242,247],{"type":48,"tag":105,"props":208,"children":210},{"style":209},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[211],{"type":54,"value":98},{"type":48,"tag":105,"props":213,"children":214},{"style":128},[215],{"type":54,"value":216}," \"",{"type":48,"tag":105,"props":218,"children":219},{"style":122},[220],{"type":54,"value":221},"$SETUP_SCRIPT",{"type":48,"tag":105,"props":223,"children":224},{"style":128},[225],{"type":54,"value":226},"\"",{"type":48,"tag":105,"props":228,"children":229},{"style":184},[230],{"type":54,"value":231}," --backend",{"type":48,"tag":105,"props":233,"children":234},{"style":184},[235],{"type":54,"value":236}," docker",{"type":48,"tag":105,"props":238,"children":239},{"style":184},[240],{"type":54,"value":241}," --check-only",{"type":48,"tag":105,"props":243,"children":244},{"style":128},[245],{"type":54,"value":246}," ||",{"type":48,"tag":105,"props":248,"children":249},{"style":128},[250],{"type":54,"value":251}," {\n",{"type":48,"tag":105,"props":253,"children":255},{"class":107,"line":254},6,[256,262,266,271],{"type":48,"tag":105,"props":257,"children":259},{"style":258},"--shiki-light:#6182B8;--shiki-default:#82AAFF;--shiki-dark:#82AAFF",[260],{"type":54,"value":261},"  echo",{"type":48,"tag":105,"props":263,"children":264},{"style":128},[265],{"type":54,"value":216},{"type":48,"tag":105,"props":267,"children":268},{"style":184},[269],{"type":54,"value":270},"MISSING: TAO GPU host runtime is not ready.",{"type":48,"tag":105,"props":272,"children":273},{"style":128},[274],{"type":54,"value":192},{"type":48,"tag":105,"props":276,"children":278},{"class":107,"line":277},7,[279,283,287,292],{"type":48,"tag":105,"props":280,"children":281},{"style":258},[282],{"type":54,"value":261},{"type":48,"tag":105,"props":284,"children":285},{"style":128},[286],{"type":54,"value":216},{"type":48,"tag":105,"props":288,"children":289},{"style":184},[290],{"type":54,"value":291},"After user approval, run:",{"type":48,"tag":105,"props":293,"children":294},{"style":128},[295],{"type":54,"value":192},{"type":48,"tag":105,"props":297,"children":299},{"class":107,"line":298},8,[300,304,308,313,318,323],{"type":48,"tag":105,"props":301,"children":302},{"style":258},[303],{"type":54,"value":261},{"type":48,"tag":105,"props":305,"children":306},{"style":128},[307],{"type":54,"value":216},{"type":48,"tag":105,"props":309,"children":310},{"style":184},[311],{"type":54,"value":312},"  bash ",{"type":48,"tag":105,"props":314,"children":315},{"style":122},[316],{"type":54,"value":317},"\\\"$SETUP_SCRIPT\\\"",{"type":48,"tag":105,"props":319,"children":320},{"style":184},[321],{"type":54,"value":322}," --backend docker --install --yes",{"type":48,"tag":105,"props":324,"children":325},{"style":128},[326],{"type":54,"value":192},{"type":48,"tag":105,"props":328,"children":330},{"class":107,"line":329},9,[331,336],{"type":48,"tag":105,"props":332,"children":333},{"style":258},[334],{"type":54,"value":335},"  exit",{"type":48,"tag":105,"props":337,"children":339},{"style":338},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[340],{"type":54,"value":341}," 1\n",{"type":48,"tag":105,"props":343,"children":345},{"class":107,"line":344},10,[346],{"type":48,"tag":105,"props":347,"children":348},{"style":128},[349],{"type":54,"value":350},"}\n",{"type":48,"tag":105,"props":352,"children":354},{"class":107,"line":353},11,[355],{"type":48,"tag":105,"props":356,"children":357},{"emptyLinePlaceholder":199},[358],{"type":54,"value":202},{"type":48,"tag":105,"props":360,"children":362},{"class":107,"line":361},12,[363],{"type":48,"tag":105,"props":364,"children":365},{"style":112},[366],{"type":54,"value":367},"# Mode 1 — direct docker (no Python). All you need is docker + the GPU runtime.\n",{"type":48,"tag":105,"props":369,"children":371},{"class":107,"line":370},13,[372,376,381,386,391,396,400,405,410,414,419,423,428,433,438,442],{"type":48,"tag":105,"props":373,"children":374},{"style":209},[375],{"type":54,"value":22},{"type":48,"tag":105,"props":377,"children":378},{"style":184},[379],{"type":54,"value":380}," info",{"type":48,"tag":105,"props":382,"children":383},{"style":128},[384],{"type":54,"value":385}," >",{"type":48,"tag":105,"props":387,"children":388},{"style":184},[389],{"type":54,"value":390},"\u002Fdev\u002Fnull",{"type":48,"tag":105,"props":392,"children":393},{"style":128},[394],{"type":54,"value":395}," 2>&1",{"type":48,"tag":105,"props":397,"children":398},{"style":128},[399],{"type":54,"value":246},{"type":48,"tag":105,"props":401,"children":402},{"style":128},[403],{"type":54,"value":404}," {",{"type":48,"tag":105,"props":406,"children":407},{"style":258},[408],{"type":54,"value":409}," echo",{"type":48,"tag":105,"props":411,"children":412},{"style":128},[413],{"type":54,"value":216},{"type":48,"tag":105,"props":415,"children":416},{"style":184},[417],{"type":54,"value":418},"MISSING: docker daemon not reachable. Start Docker.",{"type":48,"tag":105,"props":420,"children":421},{"style":128},[422],{"type":54,"value":226},{"type":48,"tag":105,"props":424,"children":425},{"style":128},[426],{"type":54,"value":427},";",{"type":48,"tag":105,"props":429,"children":430},{"style":258},[431],{"type":54,"value":432}," exit",{"type":48,"tag":105,"props":434,"children":435},{"style":338},[436],{"type":54,"value":437}," 1",{"type":48,"tag":105,"props":439,"children":440},{"style":128},[441],{"type":54,"value":427},{"type":48,"tag":105,"props":443,"children":444},{"style":128},[445],{"type":54,"value":446}," }\n",{"type":48,"tag":105,"props":448,"children":450},{"class":107,"line":449},14,[451,455,460,465,470,475,480,485,490,494,498,502,506],{"type":48,"tag":105,"props":452,"children":453},{"style":209},[454],{"type":54,"value":22},{"type":48,"tag":105,"props":456,"children":457},{"style":184},[458],{"type":54,"value":459}," run",{"type":48,"tag":105,"props":461,"children":462},{"style":184},[463],{"type":54,"value":464}," --rm",{"type":48,"tag":105,"props":466,"children":467},{"style":184},[468],{"type":54,"value":469}," --runtime=nvidia",{"type":48,"tag":105,"props":471,"children":472},{"style":184},[473],{"type":54,"value":474}," --gpus",{"type":48,"tag":105,"props":476,"children":477},{"style":184},[478],{"type":54,"value":479}," all",{"type":48,"tag":105,"props":481,"children":482},{"style":184},[483],{"type":54,"value":484}," ubuntu",{"type":48,"tag":105,"props":486,"children":487},{"style":184},[488],{"type":54,"value":489}," nvidia-smi",{"type":48,"tag":105,"props":491,"children":492},{"style":128},[493],{"type":54,"value":385},{"type":48,"tag":105,"props":495,"children":496},{"style":184},[497],{"type":54,"value":390},{"type":48,"tag":105,"props":499,"children":500},{"style":128},[501],{"type":54,"value":395},{"type":48,"tag":105,"props":503,"children":504},{"style":128},[505],{"type":54,"value":246},{"type":48,"tag":105,"props":507,"children":508},{"style":128},[509],{"type":54,"value":251},{"type":48,"tag":105,"props":511,"children":513},{"class":107,"line":512},15,[514,518,522,527],{"type":48,"tag":105,"props":515,"children":516},{"style":258},[517],{"type":54,"value":261},{"type":48,"tag":105,"props":519,"children":520},{"style":128},[521],{"type":54,"value":216},{"type":48,"tag":105,"props":523,"children":524},{"style":184},[525],{"type":54,"value":526},"MISSING: NVIDIA Container Toolkit not installed\u002Fconfigured. See:",{"type":48,"tag":105,"props":528,"children":529},{"style":128},[530],{"type":54,"value":192},{"type":48,"tag":105,"props":532,"children":534},{"class":107,"line":533},16,[535,539,543,547,551,555],{"type":48,"tag":105,"props":536,"children":537},{"style":258},[538],{"type":54,"value":261},{"type":48,"tag":105,"props":540,"children":541},{"style":128},[542],{"type":54,"value":216},{"type":48,"tag":105,"props":544,"children":545},{"style":184},[546],{"type":54,"value":312},{"type":48,"tag":105,"props":548,"children":549},{"style":122},[550],{"type":54,"value":317},{"type":48,"tag":105,"props":552,"children":553},{"style":184},[554],{"type":54,"value":322},{"type":48,"tag":105,"props":556,"children":557},{"style":128},[558],{"type":54,"value":192},{"type":48,"tag":105,"props":560,"children":562},{"class":107,"line":561},17,[563,567],{"type":48,"tag":105,"props":564,"children":565},{"style":258},[566],{"type":54,"value":335},{"type":48,"tag":105,"props":568,"children":569},{"style":338},[570],{"type":54,"value":341},{"type":48,"tag":105,"props":572,"children":574},{"class":107,"line":573},18,[575],{"type":48,"tag":105,"props":576,"children":577},{"style":128},[578],{"type":54,"value":350},{"type":48,"tag":105,"props":580,"children":582},{"class":107,"line":581},19,[583],{"type":48,"tag":105,"props":584,"children":585},{"emptyLinePlaceholder":199},[586],{"type":54,"value":202},{"type":48,"tag":105,"props":588,"children":590},{"class":107,"line":589},20,[591],{"type":48,"tag":105,"props":592,"children":593},{"style":112},[594],{"type":54,"value":595},"# Mode 2 — TAO SDK wrapper. Adds Job handles, S3 I\u002FO wrapping, ActionWorkflow.\n",{"type":48,"tag":105,"props":597,"children":599},{"class":107,"line":598},21,[600],{"type":48,"tag":105,"props":601,"children":602},{"style":112},[603],{"type":54,"value":604},"# Skip this block if Mode 1 is sufficient for the user's request.\n",{"type":48,"tag":105,"props":606,"children":608},{"class":107,"line":607},22,[609],{"type":48,"tag":105,"props":610,"children":611},{"style":112},[612],{"type":54,"value":613},"# When Mode 2 is in scope, read `tao-skill-bank:tao-run-platform` for the DockerSDK\n",{"type":48,"tag":105,"props":615,"children":617},{"class":107,"line":616},23,[618],{"type":48,"tag":105,"props":619,"children":620},{"style":112},[621],{"type":54,"value":622},"# kwarg contract, build_entrypoint, and monitoring patterns.\n",{"type":48,"tag":105,"props":624,"children":626},{"class":107,"line":625},24,[627],{"type":48,"tag":105,"props":628,"children":629},{"style":112},[630],{"type":54,"value":631},"# nvidia-tao-sdk is on public PyPI; pin lives in versions.yaml (wheels.tao_sdk_docker).\n",{"type":48,"tag":105,"props":633,"children":635},{"class":107,"line":634},25,[636,641,646,650,655,660,665,670,675],{"type":48,"tag":105,"props":637,"children":638},{"style":122},[639],{"type":54,"value":640},"PIN",{"type":48,"tag":105,"props":642,"children":643},{"style":128},[644],{"type":54,"value":645},"=$(",{"type":48,"tag":105,"props":647,"children":648},{"style":209},[649],{"type":54,"value":226},{"type":48,"tag":105,"props":651,"children":652},{"style":128},[653],{"type":54,"value":654},"${",{"type":48,"tag":105,"props":656,"children":657},{"style":122},[658],{"type":54,"value":659},"TAO_SKILL_BANK_PATH",{"type":48,"tag":105,"props":661,"children":662},{"style":128},[663],{"type":54,"value":664},":?}",{"type":48,"tag":105,"props":666,"children":667},{"style":209},[668],{"type":54,"value":669},"\u002Fscripts\u002Fresolve_versions_key.py\"",{"type":48,"tag":105,"props":671,"children":672},{"style":184},[673],{"type":54,"value":674}," wheels.tao_sdk_docker",{"type":48,"tag":105,"props":676,"children":677},{"style":128},[678],{"type":54,"value":679},")\n",{"type":48,"tag":105,"props":681,"children":683},{"class":107,"line":682},26,[684,689,694,698,703,707,712,716,720,725,730,735,740,744,749],{"type":48,"tag":105,"props":685,"children":686},{"style":209},[687],{"type":54,"value":688},"python",{"type":48,"tag":105,"props":690,"children":691},{"style":184},[692],{"type":54,"value":693}," -c",{"type":48,"tag":105,"props":695,"children":696},{"style":128},[697],{"type":54,"value":216},{"type":48,"tag":105,"props":699,"children":700},{"style":184},[701],{"type":54,"value":702},"import tao_sdk",{"type":48,"tag":105,"props":704,"children":705},{"style":128},[706],{"type":54,"value":226},{"type":48,"tag":105,"props":708,"children":709},{"style":128},[710],{"type":54,"value":711}," 2>",{"type":48,"tag":105,"props":713,"children":714},{"style":184},[715],{"type":54,"value":390},{"type":48,"tag":105,"props":717,"children":718},{"style":128},[719],{"type":54,"value":246},{"type":48,"tag":105,"props":721,"children":722},{"style":209},[723],{"type":54,"value":724}," python",{"type":48,"tag":105,"props":726,"children":727},{"style":184},[728],{"type":54,"value":729}," -m",{"type":48,"tag":105,"props":731,"children":732},{"style":184},[733],{"type":54,"value":734}," pip",{"type":48,"tag":105,"props":736,"children":737},{"style":184},[738],{"type":54,"value":739}," install",{"type":48,"tag":105,"props":741,"children":742},{"style":128},[743],{"type":54,"value":216},{"type":48,"tag":105,"props":745,"children":746},{"style":122},[747],{"type":54,"value":748},"$PIN",{"type":48,"tag":105,"props":750,"children":751},{"style":128},[752],{"type":54,"value":192},{"type":48,"tag":105,"props":754,"children":756},{"class":107,"line":755},27,[757,761,765,769,774,778,782,786,790,794,798,802,806,810,814],{"type":48,"tag":105,"props":758,"children":759},{"style":209},[760],{"type":54,"value":688},{"type":48,"tag":105,"props":762,"children":763},{"style":184},[764],{"type":54,"value":693},{"type":48,"tag":105,"props":766,"children":767},{"style":128},[768],{"type":54,"value":216},{"type":48,"tag":105,"props":770,"children":771},{"style":184},[772],{"type":54,"value":773},"import docker",{"type":48,"tag":105,"props":775,"children":776},{"style":128},[777],{"type":54,"value":226},{"type":48,"tag":105,"props":779,"children":780},{"style":128},[781],{"type":54,"value":711},{"type":48,"tag":105,"props":783,"children":784},{"style":184},[785],{"type":54,"value":390},{"type":48,"tag":105,"props":787,"children":788},{"style":128},[789],{"type":54,"value":246},{"type":48,"tag":105,"props":791,"children":792},{"style":209},[793],{"type":54,"value":724},{"type":48,"tag":105,"props":795,"children":796},{"style":184},[797],{"type":54,"value":729},{"type":48,"tag":105,"props":799,"children":800},{"style":184},[801],{"type":54,"value":734},{"type":48,"tag":105,"props":803,"children":804},{"style":184},[805],{"type":54,"value":739},{"type":48,"tag":105,"props":807,"children":808},{"style":128},[809],{"type":54,"value":216},{"type":48,"tag":105,"props":811,"children":812},{"style":122},[813],{"type":54,"value":748},{"type":48,"tag":105,"props":815,"children":816},{"style":128},[817],{"type":54,"value":192},{"type":48,"tag":105,"props":819,"children":821},{"class":107,"line":820},28,[822,826,830,834,839],{"type":48,"tag":105,"props":823,"children":824},{"style":209},[825],{"type":54,"value":688},{"type":48,"tag":105,"props":827,"children":828},{"style":184},[829],{"type":54,"value":693},{"type":48,"tag":105,"props":831,"children":832},{"style":128},[833],{"type":54,"value":216},{"type":48,"tag":105,"props":835,"children":836},{"style":184},[837],{"type":54,"value":838},"import tao_sdk, docker",{"type":48,"tag":105,"props":840,"children":841},{"style":128},[842],{"type":54,"value":192},{"type":48,"tag":105,"props":844,"children":846},{"class":107,"line":845},29,[847],{"type":48,"tag":105,"props":848,"children":849},{"emptyLinePlaceholder":199},[850],{"type":54,"value":202},{"type":48,"tag":105,"props":852,"children":854},{"class":107,"line":853},30,[855],{"type":48,"tag":105,"props":856,"children":857},{"style":112},[858],{"type":54,"value":859},"# DockerSDK attaches every job container to ${DOCKER_NETWORK:-tao_default}.\n",{"type":48,"tag":105,"props":861,"children":863},{"class":107,"line":862},31,[864],{"type":48,"tag":105,"props":865,"children":866},{"style":112},[867],{"type":54,"value":868},"# Create the network if it is missing; the operation is local and idempotent.\n",{"type":48,"tag":105,"props":870,"children":872},{"class":107,"line":871},32,[873,878,882,886,891,895,900],{"type":48,"tag":105,"props":874,"children":875},{"style":122},[876],{"type":54,"value":877},"DOCKER_NETWORK_NAME",{"type":48,"tag":105,"props":879,"children":880},{"style":128},[881],{"type":54,"value":131},{"type":48,"tag":105,"props":883,"children":884},{"style":128},[885],{"type":54,"value":136},{"type":48,"tag":105,"props":887,"children":888},{"style":122},[889],{"type":54,"value":890},"DOCKER_NETWORK",{"type":48,"tag":105,"props":892,"children":893},{"style":128},[894],{"type":54,"value":145},{"type":48,"tag":105,"props":896,"children":897},{"style":122},[898],{"type":54,"value":899},"tao_default",{"type":48,"tag":105,"props":901,"children":902},{"style":128},[903],{"type":54,"value":155},{"type":48,"tag":105,"props":905,"children":907},{"class":107,"line":906},33,[908,912,917,922,926,931,935,939,943,947,951],{"type":48,"tag":105,"props":909,"children":910},{"style":209},[911],{"type":54,"value":22},{"type":48,"tag":105,"props":913,"children":914},{"style":184},[915],{"type":54,"value":916}," network",{"type":48,"tag":105,"props":918,"children":919},{"style":184},[920],{"type":54,"value":921}," inspect",{"type":48,"tag":105,"props":923,"children":924},{"style":128},[925],{"type":54,"value":216},{"type":48,"tag":105,"props":927,"children":928},{"style":122},[929],{"type":54,"value":930},"$DOCKER_NETWORK_NAME",{"type":48,"tag":105,"props":932,"children":933},{"style":128},[934],{"type":54,"value":226},{"type":48,"tag":105,"props":936,"children":937},{"style":128},[938],{"type":54,"value":385},{"type":48,"tag":105,"props":940,"children":941},{"style":184},[942],{"type":54,"value":390},{"type":48,"tag":105,"props":944,"children":945},{"style":128},[946],{"type":54,"value":395},{"type":48,"tag":105,"props":948,"children":949},{"style":128},[950],{"type":54,"value":246},{"type":48,"tag":105,"props":952,"children":953},{"style":122},[954],{"type":54,"value":955}," \\\n",{"type":48,"tag":105,"props":957,"children":959},{"class":107,"line":958},34,[960,965,969,974,978,982,986,990],{"type":48,"tag":105,"props":961,"children":962},{"style":209},[963],{"type":54,"value":964},"  docker",{"type":48,"tag":105,"props":966,"children":967},{"style":184},[968],{"type":54,"value":916},{"type":48,"tag":105,"props":970,"children":971},{"style":184},[972],{"type":54,"value":973}," create",{"type":48,"tag":105,"props":975,"children":976},{"style":128},[977],{"type":54,"value":216},{"type":48,"tag":105,"props":979,"children":980},{"style":122},[981],{"type":54,"value":930},{"type":48,"tag":105,"props":983,"children":984},{"style":128},[985],{"type":54,"value":226},{"type":48,"tag":105,"props":987,"children":988},{"style":128},[989],{"type":54,"value":385},{"type":48,"tag":105,"props":991,"children":992},{"style":184},[993],{"type":54,"value":994},"\u002Fdev\u002Fnull\n",{"type":48,"tag":57,"props":996,"children":997},{},[998],{"type":54,"value":999},"If a check fails, the agent prompts the user to authorize the install\u002Ffix via Bash before proceeding. Pip-installable Python requirements and Docker network creation above are exceptions: install\u002Fcreate them automatically, then rerun preflight.",{"type":48,"tag":82,"props":1001,"children":1003},{"id":1002},"credentials",[1004],{"type":54,"value":1005},"Credentials",{"type":48,"tag":57,"props":1007,"children":1008},{},[1009],{"type":54,"value":1010},"There are no platform credentials required beyond access to the Docker daemon.",{"type":48,"tag":57,"props":1012,"children":1013},{},[1014],{"type":54,"value":1015},"Optional environment:",{"type":48,"tag":1017,"props":1018,"children":1019},"ul",{},[1020,1040,1056,1074,1091,1101],{"type":48,"tag":1021,"props":1022,"children":1023},"li",{},[1024,1030,1032,1038],{"type":48,"tag":1025,"props":1026,"children":1027},"strong",{},[1028],{"type":54,"value":1029},"DOCKER_HOST",{"type":54,"value":1031},": Optional Docker daemon URL. If unset, the SDK uses the\nDocker Python client's normal environment\u002Fdefault socket resolution. Required\nfor the ",{"type":48,"tag":63,"props":1033,"children":1035},{"className":1034},[],[1036],{"type":54,"value":1037},"remote-docker",{"type":54,"value":1039}," platform option.",{"type":48,"tag":1021,"props":1041,"children":1042},{},[1043,1047,1049,1054],{"type":48,"tag":1025,"props":1044,"children":1045},{},[1046],{"type":54,"value":890},{"type":54,"value":1048},": Docker network for job containers. Default is\n",{"type":48,"tag":63,"props":1050,"children":1052},{"className":1051},[],[1053],{"type":54,"value":899},{"type":54,"value":1055},".",{"type":48,"tag":1021,"props":1057,"children":1058},{},[1059,1064,1066,1072],{"type":48,"tag":1025,"props":1060,"children":1061},{},[1062],{"type":54,"value":1063},"DOCKER_USERNAME",{"type":54,"value":1065},": Registry username. Default is ",{"type":48,"tag":63,"props":1067,"children":1069},{"className":1068},[],[1070],{"type":54,"value":1071},"$oauthtoken",{"type":54,"value":1073}," for NGC.",{"type":48,"tag":1021,"props":1075,"children":1076},{},[1077,1082,1084,1090],{"type":48,"tag":1025,"props":1078,"children":1079},{},[1080],{"type":54,"value":1081},"NGC_KEY",{"type":54,"value":1083},": Used when pulling private images from ",{"type":48,"tag":63,"props":1085,"children":1087},{"className":1086},[],[1088],{"type":54,"value":1089},"nvcr.io",{"type":54,"value":1055},{"type":48,"tag":1021,"props":1092,"children":1093},{},[1094,1099],{"type":48,"tag":1025,"props":1095,"children":1096},{},[1097],{"type":54,"value":1098},"HOST_SSH_PATH",{"type":54,"value":1100},": Mounted into AutoML brain containers when they need SSH keys\nto monitor remote SLURM child jobs.",{"type":48,"tag":1021,"props":1102,"children":1103},{},[1104,1109,1111,1116,1117,1122,1123,1128],{"type":48,"tag":1025,"props":1105,"children":1106},{},[1107],{"type":54,"value":1108},"ACCESS_KEY",{"type":54,"value":1110},", ",{"type":48,"tag":1025,"props":1112,"children":1113},{},[1114],{"type":54,"value":1115},"SECRET_KEY",{"type":54,"value":1110},{"type":48,"tag":1025,"props":1118,"children":1119},{},[1120],{"type":54,"value":1121},"S3_ENDPOINT_URL",{"type":54,"value":1110},{"type":48,"tag":1025,"props":1124,"children":1125},{},[1126],{"type":54,"value":1127},"S3_BUCKET_NAME",{"type":54,"value":1129},":\nOptional S3-compatible storage settings for jobs that still read\u002Fwrite cloud\nstorage from a local container.",{"type":48,"tag":82,"props":1131,"children":1133},{"id":1132},"launch-preflight",[1134],{"type":54,"value":1135},"Launch Preflight",{"type":48,"tag":57,"props":1137,"children":1138},{},[1139],{"type":54,"value":1140},"Before generating scripts or starting containers:",{"type":48,"tag":1142,"props":1143,"children":1144},"ol",{},[1145,1166,1171,1214,1227,1239],{"type":48,"tag":1021,"props":1146,"children":1147},{},[1148,1150,1156,1158,1164],{"type":54,"value":1149},"Verify the Docker daemon is reachable, NVIDIA Container Toolkit is registered\nas a Docker runtime, GPUs and driver version are reported, and a smoke\ncontainer can see GPUs before launch. For remote Docker, query GPUs through\n",{"type":48,"tag":63,"props":1151,"children":1153},{"className":1152},[],[1154],{"type":54,"value":1155},"docker run ... nvidia-smi",{"type":54,"value":1157}," against the remote daemon; do not use local\n",{"type":48,"tag":63,"props":1159,"children":1161},{"className":1160},[],[1162],{"type":54,"value":1163},"nvidia-smi",{"type":54,"value":1165}," from the agent machine.",{"type":48,"tag":1021,"props":1167,"children":1168},{},[1169],{"type":54,"value":1170},"Verify every local\u002Ffile dataset annotation and media path exists on the\nDocker host.",{"type":48,"tag":1021,"props":1172,"children":1173},{},[1174,1176,1182,1184,1189,1191,1196,1198,1204,1206,1212],{"type":54,"value":1175},"For ",{"type":48,"tag":63,"props":1177,"children":1179},{"className":1178},[],[1180],{"type":54,"value":1181},"s3:\u002F\u002F",{"type":54,"value":1183}," datasets\u002Fresults, verify ",{"type":48,"tag":63,"props":1185,"children":1187},{"className":1186},[],[1188],{"type":54,"value":1108},{"type":54,"value":1190}," and ",{"type":48,"tag":63,"props":1192,"children":1194},{"className":1193},[],[1195],{"type":54,"value":1115},{"type":54,"value":1197}," are set\nand the exact paths are readable with ",{"type":48,"tag":63,"props":1199,"children":1201},{"className":1200},[],[1202],{"type":54,"value":1203},"aws s3 ls",{"type":54,"value":1205},". If ",{"type":48,"tag":63,"props":1207,"children":1209},{"className":1208},[],[1210],{"type":54,"value":1211},"aws",{"type":54,"value":1213}," is missing,\nreport the missing dependency and ask before installing it; rerun preflight\nafter installation.",{"type":48,"tag":1021,"props":1215,"children":1216},{},[1217,1219,1225],{"type":54,"value":1218},"Verify model-specific credentials such as ",{"type":48,"tag":63,"props":1220,"children":1222},{"className":1221},[],[1223],{"type":54,"value":1224},"HF_TOKEN",{"type":54,"value":1226}," before launch.",{"type":48,"tag":1021,"props":1228,"children":1229},{},[1230,1232,1237],{"type":54,"value":1231},"Check current GPU occupancy with ",{"type":48,"tag":63,"props":1233,"children":1235},{"className":1234},[],[1236],{"type":54,"value":1163},{"type":54,"value":1238}," and avoid GPUs already used by\nother running jobs when the user requested that constraint. Show the selected\nGPU ids in the launch review.",{"type":48,"tag":1021,"props":1240,"children":1241},{},[1242],{"type":54,"value":1243},"For model\u002Fcontainer combinations with known architecture limits, compare\nhost GPU compute capability with the container stack before launch. If the\nselected image cannot JIT or run kernels for the host architecture, block\nearly and ask for a compatible image or platform.",{"type":48,"tag":57,"props":1245,"children":1246},{},[1247],{"type":54,"value":1248},"Use the packaged helper for these checks when possible:",{"type":48,"tag":94,"props":1250,"children":1252},{"className":96,"code":1251,"language":98,"meta":99,"style":99},"${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external}\u002Fscripts\u002Fcheck_tao_launch_preflight.py \\\n  --platform local-docker \\\n  --container-image \"\u003Cselected-image>\" \\\n  --path train_annotation=\u002Fabs\u002Fpath\u002Fto\u002Fannotations.json \\\n  --path train_media=\u002Fabs\u002Fpath\u002Fto\u002Fmedia\n",[1253],{"type":48,"tag":63,"props":1254,"children":1255},{"__ignoreMap":99},[1256,1295,1312,1337,1354],{"type":48,"tag":105,"props":1257,"children":1258},{"class":107,"line":108},[1259,1263,1267,1271,1276,1281,1286,1290],{"type":48,"tag":105,"props":1260,"children":1261},{"style":128},[1262],{"type":54,"value":654},{"type":48,"tag":105,"props":1264,"children":1265},{"style":122},[1266],{"type":54,"value":659},{"type":48,"tag":105,"props":1268,"children":1269},{"style":128},[1270],{"type":54,"value":145},{"type":48,"tag":105,"props":1272,"children":1273},{"style":122},[1274],{"type":54,"value":1275},"~",{"type":48,"tag":105,"props":1277,"children":1278},{"style":128},[1279],{"type":54,"value":1280},"\u002F",{"type":48,"tag":105,"props":1282,"children":1283},{"style":122},[1284],{"type":54,"value":1285},"tao-skills-external",{"type":48,"tag":105,"props":1287,"children":1288},{"style":128},[1289],{"type":54,"value":181},{"type":48,"tag":105,"props":1291,"children":1292},{"style":122},[1293],{"type":54,"value":1294},"\u002Fscripts\u002Fcheck_tao_launch_preflight.py \\\n",{"type":48,"tag":105,"props":1296,"children":1297},{"class":107,"line":118},[1298,1303,1308],{"type":48,"tag":105,"props":1299,"children":1300},{"style":209},[1301],{"type":54,"value":1302},"  --platform",{"type":48,"tag":105,"props":1304,"children":1305},{"style":184},[1306],{"type":54,"value":1307}," local-docker",{"type":48,"tag":105,"props":1309,"children":1310},{"style":122},[1311],{"type":54,"value":955},{"type":48,"tag":105,"props":1313,"children":1314},{"class":107,"line":158},[1315,1320,1324,1329,1333],{"type":48,"tag":105,"props":1316,"children":1317},{"style":184},[1318],{"type":54,"value":1319},"  --container-image",{"type":48,"tag":105,"props":1321,"children":1322},{"style":128},[1323],{"type":54,"value":216},{"type":48,"tag":105,"props":1325,"children":1326},{"style":184},[1327],{"type":54,"value":1328},"\u003Cselected-image>",{"type":48,"tag":105,"props":1330,"children":1331},{"style":128},[1332],{"type":54,"value":226},{"type":48,"tag":105,"props":1334,"children":1335},{"style":122},[1336],{"type":54,"value":955},{"type":48,"tag":105,"props":1338,"children":1339},{"class":107,"line":195},[1340,1345,1350],{"type":48,"tag":105,"props":1341,"children":1342},{"style":184},[1343],{"type":54,"value":1344},"  --path",{"type":48,"tag":105,"props":1346,"children":1347},{"style":184},[1348],{"type":54,"value":1349}," train_annotation=\u002Fabs\u002Fpath\u002Fto\u002Fannotations.json",{"type":48,"tag":105,"props":1351,"children":1352},{"style":122},[1353],{"type":54,"value":955},{"type":48,"tag":105,"props":1355,"children":1356},{"class":107,"line":205},[1357,1361],{"type":48,"tag":105,"props":1358,"children":1359},{"style":184},[1360],{"type":54,"value":1344},{"type":48,"tag":105,"props":1362,"children":1363},{"style":184},[1364],{"type":54,"value":1365}," train_media=\u002Fabs\u002Fpath\u002Fto\u002Fmedia\n",{"type":48,"tag":57,"props":1367,"children":1368},{},[1369,1371,1376,1378,1383],{"type":54,"value":1370},"For a remote Docker daemon, use the ",{"type":48,"tag":63,"props":1372,"children":1374},{"className":1373},[],[1375],{"type":54,"value":1037},{"type":54,"value":1377}," platform and pass or export\n",{"type":48,"tag":63,"props":1379,"children":1381},{"className":1380},[],[1382],{"type":54,"value":1029},{"type":54,"value":1384},". The helper verifies remote GPU\u002Fruntime readiness and checks\nremote-host dataset paths through read-only bind mounts:",{"type":48,"tag":94,"props":1386,"children":1388},{"className":96,"code":1387,"language":98,"meta":99,"style":99},"${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external}\u002Fscripts\u002Fcheck_tao_launch_preflight.py \\\n  --platform remote-docker \\\n  --docker-host ssh:\u002F\u002Fuser@gpu-host \\\n  --container-image \"\u003Cselected-image>\" \\\n  --gpu-smoke-image ubuntu:22.04 \\\n  --path train_annotation=\u002Fremote\u002Fdata\u002Ftrain\u002Fannotations.json \\\n  --path train_media=\u002Fremote\u002Fdata\u002Ftrain\n",[1389],{"type":48,"tag":63,"props":1390,"children":1391},{"__ignoreMap":99},[1392,1427,1443,1460,1483,1500,1516],{"type":48,"tag":105,"props":1393,"children":1394},{"class":107,"line":108},[1395,1399,1403,1407,1411,1415,1419,1423],{"type":48,"tag":105,"props":1396,"children":1397},{"style":128},[1398],{"type":54,"value":654},{"type":48,"tag":105,"props":1400,"children":1401},{"style":122},[1402],{"type":54,"value":659},{"type":48,"tag":105,"props":1404,"children":1405},{"style":128},[1406],{"type":54,"value":145},{"type":48,"tag":105,"props":1408,"children":1409},{"style":122},[1410],{"type":54,"value":1275},{"type":48,"tag":105,"props":1412,"children":1413},{"style":128},[1414],{"type":54,"value":1280},{"type":48,"tag":105,"props":1416,"children":1417},{"style":122},[1418],{"type":54,"value":1285},{"type":48,"tag":105,"props":1420,"children":1421},{"style":128},[1422],{"type":54,"value":181},{"type":48,"tag":105,"props":1424,"children":1425},{"style":122},[1426],{"type":54,"value":1294},{"type":48,"tag":105,"props":1428,"children":1429},{"class":107,"line":118},[1430,1434,1439],{"type":48,"tag":105,"props":1431,"children":1432},{"style":209},[1433],{"type":54,"value":1302},{"type":48,"tag":105,"props":1435,"children":1436},{"style":184},[1437],{"type":54,"value":1438}," remote-docker",{"type":48,"tag":105,"props":1440,"children":1441},{"style":122},[1442],{"type":54,"value":955},{"type":48,"tag":105,"props":1444,"children":1445},{"class":107,"line":158},[1446,1451,1456],{"type":48,"tag":105,"props":1447,"children":1448},{"style":184},[1449],{"type":54,"value":1450},"  --docker-host",{"type":48,"tag":105,"props":1452,"children":1453},{"style":184},[1454],{"type":54,"value":1455}," ssh:\u002F\u002Fuser@gpu-host",{"type":48,"tag":105,"props":1457,"children":1458},{"style":122},[1459],{"type":54,"value":955},{"type":48,"tag":105,"props":1461,"children":1462},{"class":107,"line":195},[1463,1467,1471,1475,1479],{"type":48,"tag":105,"props":1464,"children":1465},{"style":184},[1466],{"type":54,"value":1319},{"type":48,"tag":105,"props":1468,"children":1469},{"style":128},[1470],{"type":54,"value":216},{"type":48,"tag":105,"props":1472,"children":1473},{"style":184},[1474],{"type":54,"value":1328},{"type":48,"tag":105,"props":1476,"children":1477},{"style":128},[1478],{"type":54,"value":226},{"type":48,"tag":105,"props":1480,"children":1481},{"style":122},[1482],{"type":54,"value":955},{"type":48,"tag":105,"props":1484,"children":1485},{"class":107,"line":205},[1486,1491,1496],{"type":48,"tag":105,"props":1487,"children":1488},{"style":184},[1489],{"type":54,"value":1490},"  --gpu-smoke-image",{"type":48,"tag":105,"props":1492,"children":1493},{"style":184},[1494],{"type":54,"value":1495}," ubuntu:22.04",{"type":48,"tag":105,"props":1497,"children":1498},{"style":122},[1499],{"type":54,"value":955},{"type":48,"tag":105,"props":1501,"children":1502},{"class":107,"line":254},[1503,1507,1512],{"type":48,"tag":105,"props":1504,"children":1505},{"style":184},[1506],{"type":54,"value":1344},{"type":48,"tag":105,"props":1508,"children":1509},{"style":184},[1510],{"type":54,"value":1511}," train_annotation=\u002Fremote\u002Fdata\u002Ftrain\u002Fannotations.json",{"type":48,"tag":105,"props":1513,"children":1514},{"style":122},[1515],{"type":54,"value":955},{"type":48,"tag":105,"props":1517,"children":1518},{"class":107,"line":277},[1519,1523],{"type":48,"tag":105,"props":1520,"children":1521},{"style":184},[1522],{"type":54,"value":1344},{"type":48,"tag":105,"props":1524,"children":1525},{"style":184},[1526],{"type":54,"value":1527}," train_media=\u002Fremote\u002Fdata\u002Ftrain\n",{"type":48,"tag":57,"props":1529,"children":1530},{},[1531,1533,1539],{"type":54,"value":1532},"The ",{"type":48,"tag":63,"props":1534,"children":1536},{"className":1535},[],[1537],{"type":54,"value":1538},"--path",{"type":54,"value":1540}," values above must exist on the remote Docker host. Do not pass\npaths that exist only on the local laptop or Codex host.",{"type":48,"tag":82,"props":1542,"children":1544},{"id":1543},"multi-gpu-and-multi-node",[1545],{"type":54,"value":1546},"Multi-GPU and multi-node",{"type":48,"tag":57,"props":1548,"children":1549},{},[1550,1555],{"type":48,"tag":1025,"props":1551,"children":1552},{},[1553],{"type":54,"value":1554},"Multi-node is not supported on local Docker.",{"type":54,"value":1556}," One job runs on the local Docker daemon's host with no cross-host coordination.",{"type":48,"tag":57,"props":1558,"children":1559},{},[1560,1562,1567,1569,1575,1577,1583,1585,1591,1593,1599,1601,1606,1608,1614,1616,1622],{"type":54,"value":1561},"Multi-GPU ",{"type":48,"tag":1025,"props":1563,"children":1564},{},[1565],{"type":54,"value":1566},"on the local host",{"type":54,"value":1568}," is supported via the NVIDIA Container Toolkit's ",{"type":48,"tag":63,"props":1570,"children":1572},{"className":1571},[],[1573],{"type":54,"value":1574},"--gpus",{"type":54,"value":1576}," flag (",{"type":48,"tag":63,"props":1578,"children":1580},{"className":1579},[],[1581],{"type":54,"value":1582},"--gpus all",{"type":54,"value":1584}," or ",{"type":48,"tag":63,"props":1586,"children":1588},{"className":1587},[],[1589],{"type":54,"value":1590},"--gpus '\"device=0,1,2,3\"'",{"type":54,"value":1592},"). ",{"type":48,"tag":63,"props":1594,"children":1596},{"className":1595},[],[1597],{"type":54,"value":1598},"DockerSDK.create_job(gpu_count=N)",{"type":54,"value":1600}," plumbs through to ",{"type":48,"tag":63,"props":1602,"children":1604},{"className":1603},[],[1605],{"type":54,"value":1574},{"type":54,"value":1607},". Single-host distributed init uses ",{"type":48,"tag":63,"props":1609,"children":1611},{"className":1610},[],[1612],{"type":54,"value":1613},"localhost",{"type":54,"value":1615},"; ",{"type":48,"tag":63,"props":1617,"children":1619},{"className":1618},[],[1620],{"type":54,"value":1621},"torchrun --nproc-per-node=N",{"type":54,"value":1623}," or PyTorch DDP work as usual.",{"type":48,"tag":82,"props":1625,"children":1627},{"id":1626},"backend-details",[1628],{"type":54,"value":1629},"Backend Details",{"type":48,"tag":57,"props":1631,"children":1632},{},[1633,1635,1640],{"type":54,"value":1634},"Use the SDK backend value ",{"type":48,"tag":63,"props":1636,"children":1638},{"className":1637},[],[1639],{"type":54,"value":51},{"type":54,"value":1641},". The local backend schema has no extra\nbackend details, so most routing is controlled by environment and job\nparameters:",{"type":48,"tag":94,"props":1643,"children":1647},{"className":1644,"code":1645,"language":1646,"meta":99,"style":99},"language-json shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","{\n  \"backend_type\": \"local-docker\",\n  \"num_gpu\": 1\n}\n","json",[1648],{"type":48,"tag":63,"props":1649,"children":1650},{"__ignoreMap":99},[1651,1659,1699,1723],{"type":48,"tag":105,"props":1652,"children":1653},{"class":107,"line":108},[1654],{"type":48,"tag":105,"props":1655,"children":1656},{"style":128},[1657],{"type":54,"value":1658},"{\n",{"type":48,"tag":105,"props":1660,"children":1661},{"class":107,"line":118},[1662,1667,1673,1677,1682,1686,1690,1694],{"type":48,"tag":105,"props":1663,"children":1664},{"style":128},[1665],{"type":54,"value":1666},"  \"",{"type":48,"tag":105,"props":1668,"children":1670},{"style":1669},"--shiki-light:#9C3EDA;--shiki-default:#C792EA;--shiki-dark:#C792EA",[1671],{"type":54,"value":1672},"backend_type",{"type":48,"tag":105,"props":1674,"children":1675},{"style":128},[1676],{"type":54,"value":226},{"type":48,"tag":105,"props":1678,"children":1679},{"style":128},[1680],{"type":54,"value":1681},":",{"type":48,"tag":105,"props":1683,"children":1684},{"style":128},[1685],{"type":54,"value":216},{"type":48,"tag":105,"props":1687,"children":1688},{"style":184},[1689],{"type":54,"value":51},{"type":48,"tag":105,"props":1691,"children":1692},{"style":128},[1693],{"type":54,"value":226},{"type":48,"tag":105,"props":1695,"children":1696},{"style":128},[1697],{"type":54,"value":1698},",\n",{"type":48,"tag":105,"props":1700,"children":1701},{"class":107,"line":158},[1702,1706,1711,1715,1719],{"type":48,"tag":105,"props":1703,"children":1704},{"style":128},[1705],{"type":54,"value":1666},{"type":48,"tag":105,"props":1707,"children":1708},{"style":1669},[1709],{"type":54,"value":1710},"num_gpu",{"type":48,"tag":105,"props":1712,"children":1713},{"style":128},[1714],{"type":54,"value":226},{"type":48,"tag":105,"props":1716,"children":1717},{"style":128},[1718],{"type":54,"value":1681},{"type":48,"tag":105,"props":1720,"children":1721},{"style":338},[1722],{"type":54,"value":341},{"type":48,"tag":105,"props":1724,"children":1725},{"class":107,"line":195},[1726],{"type":48,"tag":105,"props":1727,"children":1728},{"style":128},[1729],{"type":54,"value":350},{"type":48,"tag":57,"props":1731,"children":1732},{},[1733,1735,1741,1742,1748,1749,1755,1756,1761,1763,1768],{"type":54,"value":1734},"Following the Brev SDK design, platform\u002Fcontrol-plane values stay in SDK\nstate and Docker labels. The SDK does not inject ",{"type":48,"tag":63,"props":1736,"children":1738},{"className":1737},[],[1739],{"type":54,"value":1740},"BACKEND",{"type":54,"value":1110},{"type":48,"tag":63,"props":1743,"children":1745},{"className":1744},[],[1746],{"type":54,"value":1747},"HOST_PLATFORM",{"type":54,"value":1698},{"type":48,"tag":63,"props":1750,"children":1752},{"className":1751},[],[1753],{"type":54,"value":1754},"MONGOSECRET",{"type":54,"value":1110},{"type":48,"tag":63,"props":1757,"children":1759},{"className":1758},[],[1760],{"type":54,"value":1029},{"type":54,"value":1762},", or ",{"type":48,"tag":63,"props":1764,"children":1766},{"className":1765},[],[1767],{"type":54,"value":890},{"type":54,"value":1769}," into the training container.",{"type":48,"tag":82,"props":1771,"children":1773},{"id":1772},"container-execution",[1774],{"type":54,"value":1775},"Container Execution",{"type":48,"tag":57,"props":1777,"children":1778},{},[1779],{"type":54,"value":1780},"The TAO SDK local Docker handler starts containers through the Docker Python\nclient:",{"type":48,"tag":1017,"props":1782,"children":1783},{},[1784,1797,1809,1821,1832,1837],{"type":48,"tag":1021,"props":1785,"children":1786},{},[1787,1789,1795],{"type":54,"value":1788},"Backend job name uses the ",{"type":48,"tag":63,"props":1790,"children":1792},{"className":1791},[],[1793],{"type":54,"value":1794},"tao-job-\u003Cjob_id>",{"type":54,"value":1796}," form used by SDK handlers.",{"type":48,"tag":1021,"props":1798,"children":1799},{},[1800,1802,1808],{"type":54,"value":1801},"Command is usually ",{"type":48,"tag":63,"props":1803,"children":1805},{"className":1804},[],[1806],{"type":54,"value":1807},"[\"\u002Fbin\u002Fbash\", \"-c\", \"\u003Cjob command>\"]",{"type":54,"value":1055},{"type":48,"tag":1021,"props":1810,"children":1811},{},[1812,1814,1820],{"type":54,"value":1813},"Containers run detached. The SDK keeps containers by default so status and\nlogs remain inspectable, unless ",{"type":48,"tag":63,"props":1815,"children":1817},{"className":1816},[],[1818],{"type":54,"value":1819},"DOCKER_AUTO_REMOVE=true",{"type":54,"value":1055},{"type":48,"tag":1021,"props":1822,"children":1823},{},[1824,1830],{"type":48,"tag":63,"props":1825,"children":1827},{"className":1826},[],[1828],{"type":54,"value":1829},"\u002Fdev\u002Fshm",{"type":54,"value":1831}," is mounted as tmpfs.",{"type":48,"tag":1021,"props":1833,"children":1834},{},[1835],{"type":54,"value":1836},"The configured Docker network is applied by the Docker daemon for the job\ncontainer; it is not passed through as a process environment variable.",{"type":48,"tag":1021,"props":1838,"children":1839},{},[1840],{"type":54,"value":1841},"Existing containers with the same job id are stopped and removed before a\nreplacement starts.",{"type":48,"tag":57,"props":1843,"children":1844},{},[1845],{"type":54,"value":1846},"For GPU access, the handler auto-detects the host type:",{"type":48,"tag":1017,"props":1848,"children":1849},{},[1850,1877],{"type":48,"tag":1021,"props":1851,"children":1852},{},[1853,1855,1861,1863,1869,1870,1876],{"type":54,"value":1854},"Tegra or Jetson hosts use ",{"type":48,"tag":63,"props":1856,"children":1858},{"className":1857},[],[1859],{"type":54,"value":1860},"runtime=\"nvidia\"",{"type":54,"value":1862}," plus\n",{"type":48,"tag":63,"props":1864,"children":1866},{"className":1865},[],[1867],{"type":54,"value":1868},"NVIDIA_VISIBLE_DEVICES",{"type":54,"value":1190},{"type":48,"tag":63,"props":1871,"children":1873},{"className":1872},[],[1874],{"type":54,"value":1875},"NVIDIA_DRIVER_CAPABILITIES=all",{"type":54,"value":1055},{"type":48,"tag":1021,"props":1878,"children":1879},{},[1880,1882,1888],{"type":54,"value":1881},"Standard x86 hosts use Docker ",{"type":48,"tag":63,"props":1883,"children":1885},{"className":1884},[],[1886],{"type":54,"value":1887},"device_requests",{"type":54,"value":1889}," with GPU capabilities.",{"type":48,"tag":57,"props":1891,"children":1892},{},[1893,1895,1901,1903,1909,1911,1916,1917,1923],{"type":54,"value":1894},"If ",{"type":48,"tag":63,"props":1896,"children":1898},{"className":1897},[],[1899],{"type":54,"value":1900},"num_gpus",{"type":54,"value":1902}," is ",{"type":48,"tag":63,"props":1904,"children":1906},{"className":1905},[],[1907],{"type":54,"value":1908},"0",{"type":54,"value":1910},", no GPUs are assigned. If ",{"type":48,"tag":63,"props":1912,"children":1914},{"className":1913},[],[1915],{"type":54,"value":1900},{"type":54,"value":1902},{"type":48,"tag":63,"props":1918,"children":1920},{"className":1919},[],[1921],{"type":54,"value":1922},"-1",{"type":54,"value":1924},", all visible\nGPUs are requested. Prefer explicit GPU counts for shared development machines.\nWhen explicit device ids are available, prefer them over count-only selection\non shared machines so the launch does not steal GPUs occupied by other tasks.",{"type":48,"tag":82,"props":1926,"children":1928},{"id":1927},"storage",[1929],{"type":54,"value":1930},"Storage",{"type":48,"tag":57,"props":1932,"children":1933},{},[1934,1936,1942],{"type":54,"value":1935},"Local Docker accepts local and ",{"type":48,"tag":63,"props":1937,"children":1939},{"className":1938},[],[1940],{"type":54,"value":1941},"file:\u002F\u002F",{"type":54,"value":1943}," paths because the container runs on the\nsame Docker host. Make sure every path in the spec is either:",{"type":48,"tag":1017,"props":1945,"children":1946},{},[1947,1952,1957],{"type":48,"tag":1021,"props":1948,"children":1949},{},[1950],{"type":54,"value":1951},"mounted into the container by the handler or surrounding service,",{"type":48,"tag":1021,"props":1953,"children":1954},{},[1955],{"type":54,"value":1956},"reachable from inside the container already, or",{"type":48,"tag":1021,"props":1958,"children":1959},{},[1960],{"type":54,"value":1961},"a cloud URI with matching credentials.",{"type":48,"tag":57,"props":1963,"children":1964},{},[1965,1967,1973],{"type":54,"value":1966},"For remote\u002Fshared filesystems, prefer the platform that owns that filesystem.\nFor example, use SLURM plus ",{"type":48,"tag":63,"props":1968,"children":1970},{"className":1969},[],[1971],{"type":54,"value":1972},"lustre:\u002F\u002F\u002F...",{"type":54,"value":1974}," for Lustre paths on a cluster.",{"type":48,"tag":82,"props":1976,"children":1978},{"id":1977},"monitoring",[1979],{"type":54,"value":1980},"Monitoring",{"type":48,"tag":1017,"props":1982,"children":1983},{},[1984,1989],{"type":48,"tag":1021,"props":1985,"children":1986},{},[1987],{"type":54,"value":1988},"The SDK handler maps Docker container state directly: created -> Pending,\nrunning\u002Frestarting -> Running, paused -> Paused, exit code 0 -> Complete,\nnonzero exit -> Error.",{"type":48,"tag":1021,"props":1990,"children":1991},{},[1992,1994,2000],{"type":54,"value":1993},"Logs come directly from the named container through the Docker Python client\n(",{"type":48,"tag":63,"props":1995,"children":1997},{"className":1996},[],[1998],{"type":54,"value":1999},"docker logs tao-job-\u003Cjob_id>",{"type":54,"value":2001},").",{"type":48,"tag":57,"props":2003,"children":2004},{},[2005],{"type":54,"value":2006},"If the container has exited, died, is being removed, or cannot be found, status\nreconciliation treats the backend process as terminated.",{"type":48,"tag":82,"props":2008,"children":2010},{"id":2009},"cancellation",[2011],{"type":54,"value":2012},"Cancellation",{"type":48,"tag":57,"props":2014,"children":2015},{},[2016],{"type":54,"value":2017},"Cancellation stops the named container. GPU ownership is managed by Docker \u002F\nthe NVIDIA runtime, not by TAO Core's local GPU manager.",{"type":48,"tag":82,"props":2019,"children":2021},{"id":2020},"optional-via-the-tao-sdk",[2022],{"type":54,"value":2023},"Optional: via the TAO SDK",{"type":48,"tag":57,"props":2025,"children":2026},{},[2027,2029,2035],{"type":54,"value":2028},"If you want Job handles, S3 I\u002FO wrapping via the SDK's ",{"type":48,"tag":63,"props":2030,"children":2032},{"className":2031},[],[2033],{"type":54,"value":2034},"script_runner",{"type":54,"value":2036},", or\ndurability across sessions:",{"type":48,"tag":94,"props":2038,"children":2041},{"className":2039,"code":2040,"language":688,"meta":99,"style":99},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","from tao_sdk.platforms.docker import DockerSDK\n\nsdk = DockerSDK()  # reads DOCKER_HOST, NGC_KEY, S3 creds from env\njob = sdk.create_job(\n    image='nvcr.io\u002Fnvidia\u002Ftao\u002Ftao-toolkit:6.26.3-pyt',\n    command='dino train -e \u002Ftmp\u002Fspec.yaml',\n    gpu_count=1,\n    inputs={'\u002Fdata\u002Ftrain.json': 's3:\u002F\u002Fbucket\u002Fcoco\u002Ftrain.json'},\n    outputs=['\u002Fresults\u002F'],\n)\n\nstatus = sdk.get_job_status(job.id)\nlogs = sdk.get_job_logs(job.id, tail=200)\n",[2042],{"type":48,"tag":63,"props":2043,"children":2044},{"__ignoreMap":99},[2045,2053,2060,2068,2076,2084,2092,2100,2108,2116,2123,2130,2138],{"type":48,"tag":105,"props":2046,"children":2047},{"class":107,"line":108},[2048],{"type":48,"tag":105,"props":2049,"children":2050},{},[2051],{"type":54,"value":2052},"from tao_sdk.platforms.docker import DockerSDK\n",{"type":48,"tag":105,"props":2054,"children":2055},{"class":107,"line":118},[2056],{"type":48,"tag":105,"props":2057,"children":2058},{"emptyLinePlaceholder":199},[2059],{"type":54,"value":202},{"type":48,"tag":105,"props":2061,"children":2062},{"class":107,"line":158},[2063],{"type":48,"tag":105,"props":2064,"children":2065},{},[2066],{"type":54,"value":2067},"sdk = DockerSDK()  # reads DOCKER_HOST, NGC_KEY, S3 creds from env\n",{"type":48,"tag":105,"props":2069,"children":2070},{"class":107,"line":195},[2071],{"type":48,"tag":105,"props":2072,"children":2073},{},[2074],{"type":54,"value":2075},"job = sdk.create_job(\n",{"type":48,"tag":105,"props":2077,"children":2078},{"class":107,"line":205},[2079],{"type":48,"tag":105,"props":2080,"children":2081},{},[2082],{"type":54,"value":2083},"    image='nvcr.io\u002Fnvidia\u002Ftao\u002Ftao-toolkit:6.26.3-pyt',\n",{"type":48,"tag":105,"props":2085,"children":2086},{"class":107,"line":254},[2087],{"type":48,"tag":105,"props":2088,"children":2089},{},[2090],{"type":54,"value":2091},"    command='dino train -e \u002Ftmp\u002Fspec.yaml',\n",{"type":48,"tag":105,"props":2093,"children":2094},{"class":107,"line":277},[2095],{"type":48,"tag":105,"props":2096,"children":2097},{},[2098],{"type":54,"value":2099},"    gpu_count=1,\n",{"type":48,"tag":105,"props":2101,"children":2102},{"class":107,"line":298},[2103],{"type":48,"tag":105,"props":2104,"children":2105},{},[2106],{"type":54,"value":2107},"    inputs={'\u002Fdata\u002Ftrain.json': 's3:\u002F\u002Fbucket\u002Fcoco\u002Ftrain.json'},\n",{"type":48,"tag":105,"props":2109,"children":2110},{"class":107,"line":329},[2111],{"type":48,"tag":105,"props":2112,"children":2113},{},[2114],{"type":54,"value":2115},"    outputs=['\u002Fresults\u002F'],\n",{"type":48,"tag":105,"props":2117,"children":2118},{"class":107,"line":344},[2119],{"type":48,"tag":105,"props":2120,"children":2121},{},[2122],{"type":54,"value":679},{"type":48,"tag":105,"props":2124,"children":2125},{"class":107,"line":353},[2126],{"type":48,"tag":105,"props":2127,"children":2128},{"emptyLinePlaceholder":199},[2129],{"type":54,"value":202},{"type":48,"tag":105,"props":2131,"children":2132},{"class":107,"line":361},[2133],{"type":48,"tag":105,"props":2134,"children":2135},{},[2136],{"type":54,"value":2137},"status = sdk.get_job_status(job.id)\n",{"type":48,"tag":105,"props":2139,"children":2140},{"class":107,"line":370},[2141],{"type":48,"tag":105,"props":2142,"children":2143},{},[2144],{"type":54,"value":2145},"logs = sdk.get_job_logs(job.id, tail=200)\n",{"type":48,"tag":57,"props":2147,"children":2148},{},[2149,2151,2157,2159,2165,2167,2172,2174,2180,2181,2187,2189,2194],{"type":54,"value":2150},"This wraps the same ",{"type":48,"tag":63,"props":2152,"children":2154},{"className":2153},[],[2155],{"type":54,"value":2156},"docker run",{"type":54,"value":2158}," invocation under a ",{"type":48,"tag":63,"props":2160,"children":2162},{"className":2161},[],[2163],{"type":54,"value":2164},"Job",{"type":54,"value":2166}," handle and routes\nthe entrypoint through ",{"type":48,"tag":63,"props":2168,"children":2170},{"className":2169},[],[2171],{"type":54,"value":2034},{"type":54,"value":2173}," so ",{"type":48,"tag":63,"props":2175,"children":2177},{"className":2176},[],[2178],{"type":54,"value":2179},"inputs",{"type":54,"value":1280},{"type":48,"tag":63,"props":2182,"children":2184},{"className":2183},[],[2185],{"type":54,"value":2186},"outputs",{"type":54,"value":2188}," get downloaded\nfrom \u002F uploaded to S3 automatically. If you don't need those, just use\n",{"type":48,"tag":63,"props":2190,"children":2192},{"className":2191},[],[2193],{"type":54,"value":2156},{"type":54,"value":2195}," directly — no SDK install required.",{"type":48,"tag":82,"props":2197,"children":2199},{"id":2198},"failure-modes",[2200],{"type":54,"value":2201},"Failure Modes",{"type":48,"tag":57,"props":2203,"children":2204},{},[2205,2210,2212,2217],{"type":48,"tag":1025,"props":2206,"children":2207},{},[2208],{"type":54,"value":2209},"Docker client not initialized",{"type":54,"value":2211},": Verify the Docker Python package is installed,\nset ",{"type":48,"tag":63,"props":2213,"children":2215},{"className":2214},[],[2216],{"type":54,"value":1029},{"type":54,"value":2218}," if you are not using the default local socket, and confirm the\nprocess can talk to the daemon.",{"type":48,"tag":57,"props":2220,"children":2221},{},[2222,2227,2229,2235],{"type":48,"tag":1025,"props":2223,"children":2224},{},[2225],{"type":54,"value":2226},"GPU assignment failed",{"type":54,"value":2228},": Requested GPUs are unavailable, the NVIDIA Container\nToolkit is not configured, or the Docker daemon cannot create GPU device\nrequests. Use fewer GPUs, wait for another job to finish, or verify\n",{"type":48,"tag":63,"props":2230,"children":2232},{"className":2231},[],[2233],{"type":54,"value":2234},"docker run --gpus ...",{"type":54,"value":2236}," works on the host.",{"type":48,"tag":57,"props":2238,"children":2239},{},[2240,2245,2247,2252,2254,2259,2261,2267],{"type":48,"tag":1025,"props":2241,"children":2242},{},[2243],{"type":54,"value":2244},"Image pull auth failed",{"type":54,"value":2246},": Set a valid ",{"type":48,"tag":63,"props":2248,"children":2250},{"className":2249},[],[2251],{"type":54,"value":1081},{"type":54,"value":2253}," for private ",{"type":48,"tag":63,"props":2255,"children":2257},{"className":2256},[],[2258],{"type":54,"value":1089},{"type":54,"value":2260}," images\nor run ",{"type":48,"tag":63,"props":2262,"children":2264},{"className":2263},[],[2265],{"type":54,"value":2266},"docker login nvcr.io -u '$oauthtoken'",{"type":54,"value":2268}," on the Docker host.",{"type":48,"tag":57,"props":2270,"children":2271},{},[2272,2277,2279,2284,2286,2291],{"type":48,"tag":1025,"props":2273,"children":2274},{},[2275],{"type":54,"value":2276},"Container exited unexpectedly",{"type":54,"value":2278},": Check ",{"type":48,"tag":63,"props":2280,"children":2282},{"className":2281},[],[2283],{"type":54,"value":1999},{"type":54,"value":2285},", the\nconfigured ",{"type":48,"tag":63,"props":2287,"children":2289},{"className":2288},[],[2290],{"type":54,"value":890},{"type":54,"value":2292},", and the command produced by the SDK action runner.",{"type":48,"tag":57,"props":2294,"children":2295},{},[2296,2301],{"type":48,"tag":1025,"props":2297,"children":2298},{},[2299],{"type":54,"value":2300},"Path missing inside container",{"type":54,"value":2302},": A local path on the host is not necessarily\nmounted into the job container. Use a path convention supported by the action\nrunner or configure an explicit volume through the surrounding service.",{"type":48,"tag":2304,"props":2305,"children":2306},"style",{},[2307],{"type":54,"value":2308},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":2310,"total":2410},[2311,2328,2340,2354,2366,2383,2396],{"slug":2312,"name":2312,"fn":2313,"description":2314,"org":2315,"tags":2316,"stars":23,"repoUrl":24,"updatedAt":2327},"accelerated-computing-cudf","accelerate data processing with cuDF","Official NVIDIA-authored guidance for NVIDIA cuDF GPU DataFrames, pandas acceleration, dask-cuDF, ETL, joins, groupby, CSV\u002FParquet I\u002FO, nullable semantics, and multi-GPU DataFrame workloads.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2317,2320,2323,2324],{"name":2318,"slug":2319,"type":15},"Data Analysis","data-analysis",{"name":2321,"slug":2322,"type":15},"Data Engineering","data-engineering",{"name":9,"slug":8,"type":15},{"name":2325,"slug":2326,"type":15},"Performance","performance","2026-07-14T05:28:43.176466",{"slug":2329,"name":2329,"fn":2330,"description":2331,"org":2332,"tags":2333,"stars":23,"repoUrl":24,"updatedAt":2339},"aiq-deploy","deploy and manage NVIDIA AI-Q infrastructure","Use when asked to install, deploy, run, validate, troubleshoot, or stop NVIDIA AI-Q Blueprint infrastructure.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2334,2335,2338],{"name":17,"slug":18,"type":15},{"name":2336,"slug":2337,"type":15},"Infrastructure","infrastructure",{"name":9,"slug":8,"type":15},"2026-07-14T05:29:06.667109",{"slug":2341,"name":2341,"fn":2342,"description":2343,"org":2344,"tags":2345,"stars":23,"repoUrl":24,"updatedAt":2353},"aiq-research","conduct deep research with AI-Q","Use when asked to run deep research or AI-Q research through a reachable NVIDIA AI-Q Blueprint backend.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2346,2349,2350],{"name":2347,"slug":2348,"type":15},"Agents","agents",{"name":9,"slug":8,"type":15},{"name":2351,"slug":2352,"type":15},"Research","research","2026-07-14T05:28:06.816956",{"slug":2355,"name":2355,"fn":2356,"description":2357,"org":2358,"tags":2359,"stars":23,"repoUrl":24,"updatedAt":2365},"amc-run-sample-calibration","run AMC sample dataset calibration","Run end-to-end calibration on the shipped sample dataset (sdg_08_2_sample_data_010926.zip) against a running AMC microservice. Use when user says 'test sample dataset', 'run sample calibration', 'verify AMC install', or 'launch and test'.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2360,2361,2362],{"name":2318,"slug":2319,"type":15},{"name":9,"slug":8,"type":15},{"name":2363,"slug":2364,"type":15},"Testing","testing","2026-07-17T05:29:03.913266",{"slug":2367,"name":2367,"fn":2368,"description":2369,"org":2370,"tags":2371,"stars":23,"repoUrl":24,"updatedAt":2382},"amc-run-video-calibration","calibrate video datasets with AutoMagicCalib","Calibrate a new dataset from pre-recorded video files via the AutoMagicCalib REST API. Use when user has local MP4s and says 'calibrate my videos', 'run AMC on these videos', or similar. For RTSP\u002Flive streams, use amc-run-rtsp-calibration instead.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2372,2375,2378,2379],{"name":2373,"slug":2374,"type":15},"Automation","automation",{"name":2376,"slug":2377,"type":15},"Imaging","imaging",{"name":9,"slug":8,"type":15},{"name":2380,"slug":2381,"type":15},"Video","video","2026-07-17T05:28:53.905004",{"slug":2384,"name":2384,"fn":2385,"description":2386,"org":2387,"tags":2388,"stars":23,"repoUrl":24,"updatedAt":2395},"amc-setup-calibration-stack","deploy AutoMagicCalib microservice with Docker","Launch AutoMagicCalib microservice and web UI from NGC release images via Docker Compose. Use when user says 'deploy auto calibration', 'launch auto calibration', 'launch AMC', 'start MS+UI', or 'set up auto-magic-calib'. Requires NGC API key.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2389,2390,2391,2392],{"name":17,"slug":18,"type":15},{"name":21,"slug":22,"type":15},{"name":9,"slug":8,"type":15},{"name":2393,"slug":2394,"type":15},"Operations","operations","2026-07-17T05:28:56.913999",{"slug":2397,"name":2397,"fn":2398,"description":2399,"org":2400,"tags":2401,"stars":23,"repoUrl":24,"updatedAt":2409},"cudaq-guide","develop quantum applications with CUDA-Q","CUDA-Q onboarding guide for installation, test programs, GPU simulation, QPU hardware, and quantum applications.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2402,2403,2406],{"name":9,"slug":8,"type":15},{"name":2404,"slug":2405,"type":15},"Quantum Computing","quantum-computing",{"name":2407,"slug":2408,"type":15},"Simulation","simulation","2026-07-14T05:26:58.898253",305,{"items":2412,"total":2560},[2413,2431,2446,2457,2469,2483,2496,2508,2519,2528,2542,2551],{"slug":2414,"name":2414,"fn":2415,"description":2416,"org":2417,"tags":2418,"stars":2428,"repoUrl":2429,"updatedAt":2430},"nemoclaw-user-guide","retrieve NemoClaw documentation and configuration","Guides human users' AI agents to the NemoClaw docs MCP server and canonical Fern documentation in Markdown form. Use when users ask how to install, configure, operate, troubleshoot, secure, or learn NemoClaw with an AI coding assistant. Trigger keywords - nemoclaw docs, use nemoclaw with ai agent, nemoclaw mcp docs, nemoclaw install help, nemoclaw quickstart, nemoclaw markdown docs, llms.txt, agent skills.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2419,2422,2425],{"name":2420,"slug":2421,"type":15},"Documentation","documentation",{"name":2423,"slug":2424,"type":15},"MCP","mcp",{"name":2426,"slug":2427,"type":15},"Search","search",21777,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FNemoClaw","2026-07-20T06:00:01.461044",{"slug":2432,"name":2432,"fn":2433,"description":2434,"org":2435,"tags":2436,"stars":2443,"repoUrl":2444,"updatedAt":2445},"mcore-build-and-dependency","manage Megatron-LM development environments","Container-based dev environment setup and dependency management for Megatron-LM. Covers acquiring and launching the CI container, uv package management, and updating uv.lock.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2437,2440,2441],{"name":2438,"slug":2439,"type":15},"Containers","containers",{"name":17,"slug":18,"type":15},{"name":2442,"slug":688,"type":15},"Python",17049,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FMegatron-LM","2026-07-27T06:06:11.249662",{"slug":2447,"name":2447,"fn":2448,"description":2449,"org":2450,"tags":2451,"stars":2443,"repoUrl":2444,"updatedAt":2456},"mcore-bump-base-image","update NVIDIA PyTorch base images","Bump the NVIDIA PyTorch base image (`nvcr.io\u002Fnvidia\u002Fpytorch:YY.MM-py3`) used by Megatron-LM CI. Covers the two pin sites (GitHub CI in `docker\u002F.ngc_version.dev` and GitLab CI in `.gitlab\u002Fstages\u002F01.build.yml`), the post-bump CI loop (re-run functional tests, refresh golden values, mark broken tests), and the gotchas that bit PRs",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2452,2455],{"name":2453,"slug":2454,"type":15},"CI\u002FCD","ci-cd",{"name":17,"slug":18,"type":15},"2026-07-14T05:25:59.97109",{"slug":2458,"name":2458,"fn":2459,"description":2460,"org":2461,"tags":2462,"stars":2443,"repoUrl":2444,"updatedAt":2468},"mcore-cicd","manage CI\u002FCD pipelines for Megatron-LM","CI\u002FCD reference for Megatron-LM. Covers CI pipeline structure, PR scope labels, triggering internal GitLab CI (which force-pushes the current branch to a pull-request\u002FBRANCH ref — always dry-run and verify the destination first; never run against shared or protected branches), and CI failure investigation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2463,2464,2465],{"name":2453,"slug":2454,"type":15},{"name":17,"slug":18,"type":15},{"name":2466,"slug":2467,"type":15},"GitHub","github","2026-07-27T06:06:12.278222",{"slug":2470,"name":2470,"fn":2471,"description":2472,"org":2473,"tags":2474,"stars":2443,"repoUrl":2444,"updatedAt":2482},"mcore-create-issue","investigate CI failures and create issues","Investigate a failing GitHub Actions run or job and create a GitHub issue for the failure.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2475,2478,2479],{"name":2476,"slug":2477,"type":15},"Debugging","debugging",{"name":2466,"slug":2467,"type":15},{"name":2480,"slug":2481,"type":15},"Triage","triage","2026-07-14T05:25:57.442089",{"slug":2484,"name":2484,"fn":2485,"description":2486,"org":2487,"tags":2488,"stars":2443,"repoUrl":2444,"updatedAt":2495},"mcore-linting-and-formatting","lint and format Megatron-LM code","Linting and formatting for Megatron-LM. Covers running autoformat.sh, tools (ruff, black, isort, pylint, mypy), and code style rules.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2489,2492],{"name":2490,"slug":2491,"type":15},"Best Practices","best-practices",{"name":2493,"slug":2494,"type":15},"Code Analysis","code-analysis","2026-07-14T05:25:56.18433",{"slug":2497,"name":2497,"fn":2498,"description":2499,"org":2500,"tags":2501,"stars":2443,"repoUrl":2444,"updatedAt":2507},"mcore-migrate-gpt-to-hybrid","migrate Megatron-LM models to HybridModel","Migration guide for moving Megatron Core GPTModel checkpoints, model providers, training commands, and layer mappings to HybridModel.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2502,2503,2506],{"name":13,"slug":14,"type":15},{"name":2504,"slug":2505,"type":15},"Migration","migration",{"name":9,"slug":8,"type":15},"2026-07-17T06:07:11.777011",{"slug":2509,"name":2509,"fn":2510,"description":2511,"org":2512,"tags":2513,"stars":2443,"repoUrl":2444,"updatedAt":2518},"mcore-onboard-gb200-1node-tests","onboard functional tests for GB200","Onboard 1-node GitHub MR functional tests for GB200 from existing mr-scoped 2-node tests.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2514,2517],{"name":2515,"slug":2516,"type":15},"QA","qa",{"name":2363,"slug":2364,"type":15},"2026-07-14T05:25:53.673039",{"slug":2520,"name":2520,"fn":2521,"description":2522,"org":2523,"tags":2524,"stars":2443,"repoUrl":2444,"updatedAt":2527},"mcore-run-on-slurm","launch distributed training jobs on SLURM","How to launch distributed Megatron-LM training jobs on a SLURM cluster. Covers a minimal sbatch skeleton, environment-variable setup for torch.distributed.run, CUDA_DEVICE_MAX_CONNECTIONS rules across hardware and parallelism modes, container conventions, monitoring, and per-rank failure diagnosis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2525,2526],{"name":17,"slug":18,"type":15},{"name":2336,"slug":2337,"type":15},"2026-07-14T05:25:49.362534",{"slug":2529,"name":2529,"fn":2530,"description":2531,"org":2532,"tags":2533,"stars":2443,"repoUrl":2444,"updatedAt":2541},"mcore-split-pr","split pull requests to reduce review load","Split a PR into multiple PRs to reduce the number of required CODEOWNERS reviewer groups.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2534,2537,2538],{"name":2535,"slug":2536,"type":15},"Code Review","code-review",{"name":2466,"slug":2467,"type":15},{"name":2539,"slug":2540,"type":15},"Pull Requests","pull-requests","2026-07-14T05:26:01.226578",{"slug":2543,"name":2543,"fn":2544,"description":2545,"org":2546,"tags":2547,"stars":2443,"repoUrl":2444,"updatedAt":2550},"mcore-testing","run and manage Megatron-LM tests","Test system for Megatron-LM. Covers test layout, recipe YAML structure, adding and running unit and functional tests, golden values, marker filters, and CI parity.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2548,2549],{"name":2515,"slug":2516,"type":15},{"name":2363,"slug":2364,"type":15},"2026-07-14T05:25:54.928983",{"slug":2552,"name":2552,"fn":2553,"description":2554,"org":2555,"tags":2556,"stars":2443,"repoUrl":2444,"updatedAt":2559},"nightly-sync","manage nightly main-to-dev sync workflows","Domain knowledge for the nightly main-to-dev sync workflow. Covers merge strategy, CI architecture, failure investigation, and known issues.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2557,2558],{"name":2373,"slug":2374,"type":15},{"name":2453,"slug":2454,"type":15},"2026-07-30T05:29:03.275638",496]