[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-nvidia-tao-run-inference-service":3,"mdc--sg2loq-key":31,"related-org-nvidia-tao-run-inference-service":3353,"related-repo-nvidia-tao-run-inference-service":3509},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":20,"repoUrl":21,"updatedAt":22,"license":23,"forks":24,"topics":25,"repo":26,"sourceUrl":29,"mdContent":30},"tao-run-inference-service","run TAO inference microservices","Start, query, and stop a network-specific TAO inference microservice ({network_arch}-inference-microservice) by delegating container execution to the appropriate platform skill. Handles container image resolution, job-payload JSON construction, and the service registry. Use when the user wants to run inference on a TAO model checkpoint using a microservice container, deploy a TAO inference endpoint, or stop a running inference container.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},"nvidia","NVIDIA","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fnvidia.png",[12,16,19],{"name":13,"slug":14,"type":15},"Containers","containers","tag",{"name":17,"slug":18,"type":15},"Deployment","deployment",{"name":9,"slug":8,"type":15},2473,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fskills","2026-07-14T05:28:15.894019","Apache-2.0",281,[],{"repoUrl":21,"stars":20,"forks":24,"topics":27,"description":28},[],"AI agent skills published by NVIDIA","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fskills\u002Ftree\u002FHEAD\u002Fskills\u002Ftao-run-inference-service","---\nname: tao-run-inference-service\ndescription: >\n  Start, query, and stop a network-specific TAO inference microservice\n  ({network_arch}-inference-microservice) by delegating container execution to\n  the appropriate platform skill. Handles container image resolution,\n  job-payload JSON construction, and the service registry. Use when the user\n  wants to run inference on a TAO model checkpoint using a microservice\n  container, deploy a TAO inference endpoint, or stop a running inference\n  container.\nlicense: Apache-2.0\ncompatibility: The inference service has no cloud-storage dependency — model weights come from the HuggingFace Hub (HF_TOKEN env var for gated models) or a local container path. Platform prerequisites are checked by each platform skill.\nmetadata:\n  author: NVIDIA Corporation\n  version: \"0.1.0\"\nallowed-tools: Read Bash Write\ntags:\n- inference\n- microservice\n- workflow\n---\n\n# TAO Inference Microservice\n\n## Instructions\n\n**To start an inference service:**\n1. Collect required inputs (Section 1) and resolve the container image (Section 2).\n2. Build the job payload and inner command (Sections 3–4.1); use `references\u002Fcode-templates.yaml` → `job_payload_builder`.\n3. Read `skills\u002Fplatform\u002F\u003Cplatform>\u002FSKILL.md` and start the container (Section 4.2).\n4. Write the service registry and poll readiness (Section 4.3); use `references\u002Fcode-templates.yaml` → `registry_write.\u003Cplatform>` and `readiness_check`.\n\n**To send an inference request:**\n1. Resolve which service receives the request per Section 6.0 (by `job_id`, by `network_arch`, or by explicit user choice when multiple services run — **never silently default to `\"latest\"` when more than one service exists**), then read the endpoint from `references\u002Fcode-templates.yaml` → `request.registry_read` with the resolved `job_id`.\n2. **Before building the request body, prompt the user for the vLLM-style sampling parameters (Section 6.1).** Present `max_tokens`, `top_p`, `temperature` (and any per-arch extras) with their defaults; let the user override or skip each one to accept the default. Never silently use defaults.\n3. Build and send the body per Section 6.2; handle the response per Section 6.3.\n\n**To stop a service:** Read `references\u002Fcode-templates.yaml` → `stop.registry_read` to resolve the job_id, read `skills\u002Fplatform\u002F\u003Cplatform>\u002FSKILL.md`, then follow Section 5.\n\n**Reference data** (schemas, mappings, valid values — no instructions):\n- **`references\u002Fservice.yaml`** — image mappings, valid `network_arch` names, job payload schema, env var names, secrets classification.\n- **`references\u002Frequest.yaml`** — endpoint definition, request field schema, response shapes, code examples.\n- **`references\u002Fcode-templates.yaml`** — Python templates for payload building, registry writes, readiness checks, and stop\u002Frequest flows.\n\n---\n\n## Secrets rule (applies to every generated code block in this skill)\n\n**Never ask the user to type a secret value into a prompt.** For every secret value:\n1. Tell the user which environment variable to set (e.g. `export HF_TOKEN=...`).\n2. Generate code that reads it with `os.environ[\"VAR_NAME\"]` — never hard-code, interpolate, or prompt for the value.\n\n**Secret env vars** (full list in `references\u002Fservice.yaml` → `secrets_handling`):\n`HF_TOKEN`, `WANDB_API_KEY`, `CLEARML_API_ACCESS_KEY`, `CLEARML_API_SECRET_KEY`, `TAO_API_KEY`, `TAO_USER_KEY`.\n\n**Safe to collect in the prompt:** `network_arch`, `model_path`, `num_gpus`, prompt text, `WANDB_*` config URLs, `CLEARML_*_HOST` URLs.\n\n---\n\n## 1. What to collect from the user\n\n| Input | Role |\n|--------|------|\n| **`network_arch`** | Chooses container image, the per-arch inner command shape (`references\u002Fservice.yaml` → `container_commands.\u003Cnetwork_arch>`), and `neural_network_name` in the job JSON when applicable. Must match a basename in `valid_network_arch_config_basenames` in `references\u002Fservice.yaml` (e.g. `cosmos-rl`, `cosmos-predict2.5`). |\n| **`model_path`** | The trained model checkpoint. Valid forms: `hf_model:\u002F\u002F\u003Corg>\u002F\u003Cmodel>` (HuggingFace Hub — set `HF_TOKEN` for gated models) or a local container filesystem path. Cloud URIs (`s3:\u002F\u002F`, `gs:\u002F\u002F`, `az:\u002F\u002F`) are NOT supported — the inference service has no cloud-storage dependency. Always ask the user; never substitute a placeholder. See `references\u002Fservice.yaml` → `model_path_protocols`. |\n| **`platform`** | Compute platform: `local-docker`, `brev`, `slurm`, or `kubernetes`. |\n| **`num_gpus`** | Defaults to **1**; minimum **1** for inference. |\n\n---\n\n## 2. Image resolution\n\nEach `network_arch` has a sidecar config file named `{network_arch}.config.json`. Resolve the container image as follows:\n\n1. Read `{network_arch}.config.json` and take `api_params.image` (e.g. `COSMOS_RL`). This is a key into `docker_image_defaults.mapping` in `references\u002Fservice.yaml`.\n2. Look up that key in the mapping. If the host env var `IMAGE_\u003CKEY>` is set (e.g. `IMAGE_COSMOS_RL`), it overrides the mapped default.\n3. The mapped value is normally a dotted key into the repo-root `versions.yaml` manifest (e.g. `tao_toolkit.cosmos_rl`). Resolve it to a concrete `nvcr.io\u002F...` image URI by looking up `versions.yaml` → `images.\u003Cgroup>.\u003Cname>`. Absolute URIs pass through unchanged, so an `IMAGE_\u003CKEY>` env-var override that contains a full URI still works. The Python helper for this lives in `references\u002Fcode-templates.yaml`.\n4. If the config file is missing or `api_params.image` is empty, fall back to the `COSMOS_RL` key.\n\nThe config file also has `spec_params.inference.model_path` which drives **folder vs file** path semantics: if the value contains the substring `folder`, the container treats the path as a directory.\n\n---\n\n## 3. Environment variables (no callbacks)\n\nSet these in `env_payload` before encoding `env_json`. Do **not** set `TAO_LOGGING_SERVER_URL` or `TAO_ADMIN_KEY`.\n\n**`TAO_EXECUTION_BACKEND`** — must match the platform:\n\n| Platform | `TAO_EXECUTION_BACKEND` value |\n|----------|-------------------------------|\n| local-docker | `local-docker` |\n| brev | `local-docker` |\n| slurm | `slurm` |\n| kubernetes | `local-k8s` |\n\n**`CLOUD_BASED`** — always `\"False\"` for this skill (disables callback posting to `TAO_LOGGING_SERVER_URL`).\n\n**GPU env vars** — only needed when the platform skill does not handle GPU injection automatically:\n- Tegra \u002F Jetson: `--runtime=nvidia` with `NVIDIA_DRIVER_CAPABILITIES=all` and `NVIDIA_VISIBLE_DEVICES=\u003Cids>`.\n- Standard x86 + nvidia-container-toolkit: use Docker `device_requests`. The platform skill handles this.\n\n---\n\n## 4. Executing across platforms\n\nThe job payload and inner command (Sections 1–3) are **platform-agnostic**. For each platform, read **`skills\u002Fplatform\u002F\u003Cname>\u002FSKILL.md`** for preflight checks and credentials **before** generating any execution code.\n\n### 4.1 Build the inner command (per arch)\n\nThe inner-command shape is **per `network_arch`** — there is no uniform template. Look up the per-arch entry in `references\u002Fservice.yaml` → `container_commands.\u003Cnetwork_arch>`; if not present, the arch is unsupported — stop and ask. Pick the matching sub-block in `references\u002Fcode-templates.yaml` → `job_payload_builder.\u003Cnetwork_arch>`. Prefix the command with `umask 0 &&` and keep it **identical across platforms** (local-docker, brev, slurm, kubernetes).\n\nCommon across arches:\n\n- `job_id`: fresh `uuid.uuid4()` — becomes the container name and registry key.\n- `image`: resolve per Section 2.\n- Secrets (`access_key`, `secret_key`, `HF_TOKEN`, etc.) are read from env vars at runtime — never hard-code, never log or print.\n\nArch-specific notes (full details in `references\u002Fservice.yaml` → `container_commands`):\n\n- **`cosmos-rl`** — single `--job '\u003CJOB_JSON>' --docker_env_vars '\u003CENV_JSON>'` blob; `json.dumps(...)` + `shlex.quote(...)`. `env_payload` carries `TAO_EXECUTION_BACKEND` (per Section 3 table), `TAO_API_JOB_ID`, `CLOUD_BASED=False`. The inference service has no cloud-storage dependency; `HF_TOKEN` is the only cred env var that ever applies (for gated HuggingFace models).\n- **`cosmos-predict2.5`** — flag-style `cosmos_predict inference_microservice start ... --port 8080` (no `setup.` prefix; uses `tyro.conf.OmitArgPrefixes`). `--job`\u002F`--docker_env_vars` are **not** accepted. Translate `model_path` to `--checkpoint-path` (local path) or `--model \u003Cregistered_key>` (`hf_model:\u002F\u002F`); cloud URIs are rejected. The only cred env var that ever applies is `HF_TOKEN` for gated HuggingFace models. Per-request params (prompt, inference_type, num_output_frames, guidance, seed, num_steps, negative_prompt) go in the request body, not at startup. `TAO_EXECUTION_BACKEND`\u002F`TAO_API_JOB_ID`\u002F`CLOUD_BASED` are unused and may be omitted.\n\n### 4.2 Delegate execution to the platform skill\n\nRead **`skills\u002Fplatform\u002F\u003Cplatform>\u002FSKILL.md`** and follow it to start the container.\n\n**Base parameters (all platforms):**\n\n| Parameter | Value |\n|-----------|-------|\n| `image` | resolved container image (Section 2) |\n| `command` | `inner` — the shell string built in Section 4.1 |\n| `gpu_count` | `num_gpus` |\n| `env_vars` | `env_payload` |\n| job \u002F container name | `job_id` — must equal the UUID from 4.1 so the registry can reference it |\n| `host_port` *(local-docker, brev)* | host-side port to bind to container port 8080. Default `8080`, but **must be unique per concurrent service** — see the port-allocation rule below. |\n\n**Platform-specific additional inputs:**\n\n| Platform | Additional inputs |\n|----------|------------------|\n| **local-docker** | None beyond base |\n| **brev** | `instance_id` (optional — reuse an existing instance); on multi-credential \u002F multi-workspace accounts also `cloud_cred_id` and `workspace_group_id` for first-create — see `skills\u002Fplatform\u002Ftao-run-on-brev\u002FSKILL.md` |\n| **slurm** | `partition` and `account` — check `SLURM_PARTITION`\u002F`SLURM_ACCOUNT` env vars; ask user if unset |\n| **kubernetes** | `namespace` (default: `default`); `image_pull_secret` (required for `nvcr.io` images) |\n\n**Port binding (local-docker and brev):** use **direct docker run** (not DockerSDK) so that `-p \u003Chost_port>:8080` can be passed and the container name equals `job_id` exactly.\n\n**Port allocation rule (local-docker and brev, REQUIRED for concurrent services):** Before starting a service, read the registry (`\u002Ftmp\u002Ftao-inf-ms-state.json`) and collect the set of `host_port` values from every existing entry on the same platform (and, for brev, the same `instance_id`). Pick the **lowest free port starting from 8080** that is not in that set — e.g. `host_port = next(p for p in range(8080, 8200) if p not in used_ports)`. The default `8080` only applies when no other service is running. This is what makes \"start 3 services, each reachable at a distinct `host_url`\" work; without it, services 2 and 3 fail with `bind: address already in use`. SLURM and kubernetes get distinct endpoints from their own platform mechanisms and do not need this step.\n\n### 4.3 After start: service registry and endpoint\n\nWrite the service registry immediately after the platform confirms the container is running. The registry (`\u002Ftmp\u002Ftao-inf-ms-state.json`) is keyed by `job_id`; `\"latest\"` always points to the most recently started service.\n\nSee `references\u002Fcode-templates.yaml` → `registry_write.\u003Cplatform>` for the Python template.\n\n| Platform | `host_url` | `platform_job_id` | Extra step before writing |\n|----------|-----------|-------------------|--------------------------|\n| **local-docker** | `http:\u002F\u002Flocalhost:{host_port}` | — | None |\n| **brev** | `http:\u002F\u002F{brev_ip}:{host_port}` | — | `brev ls` → get instance IP (`localhost` is invalid on remote VM) |\n| **slurm** | `http:\u002F\u002Flocalhost:{host_port}` | SLURM scheduler job ID | Wait until Running; SSH port-forward `localhost:{host_port}→{node}:8080` |\n| **kubernetes** | `http:\u002F\u002F{external_ip}:8080` | k8s job name | `kubectl expose job … --type=LoadBalancer`; wait for external IP |\n\nAfter writing the registry, print the job_id and URL:\n\n```python\nprint(f\"Inference service started.\")\nprint(f\"  Job ID : {job_id}\")\nprint(f\"  Arch   : {network_arch}\")\nprint(f\"  URL    : {state[job_id]['host_url']}\u002Fv1\u002Fchat\u002Fcompletions\")\nprint(f\"Use this Job ID to send requests or stop the service.\")\n```\n\nThen poll for readiness — see `references\u002Fcode-templates.yaml` → `readiness_check`. The container loads the model in the background; do not send requests before it returns 200.\n\n---\n\n## 5. Stopping the inference service\n\nAsk the user for the `job_id` to stop. If they don't provide one, default to `state[\"latest\"]` and confirm which job_id is being stopped. Read the registry using `references\u002Fcode-templates.yaml` → `stop.registry_read`, then read **`skills\u002Fplatform\u002F\u003Cplatform>\u002FSKILL.md`** and use its cancellation \u002F stop mechanism.\n\n| Platform | Identifier to pass | Extra cleanup |\n|----------|--------------------|---------------|\n| **local-docker** | `job_id_to_stop` — container name | None |\n| **brev** | `job_id_to_stop` — container name | None |\n| **slurm** | `entry[\"platform_job_id\"]` — SLURM job ID | `pkill -f \"ssh.*-L.*{entry['host_port']}\"` |\n| **kubernetes** | `entry[\"platform_job_id\"]` — k8s job name | `kubectl delete svc {entry[\"platform_job_id\"]} -n \u003Cnamespace>` |\n\nwhere `entry = state[job_id_to_stop]`. After stopping, clean up the registry: `references\u002Fcode-templates.yaml` → `stop.registry_cleanup`.\n\n---\n\n## 6. Sending inference requests\n\n### 6.0 Resolve which service receives this request (REQUIRED)\n\nEach request must be routed to the **specific** service that runs the matching model. Routing happens by `job_id` — the registry stores `network_arch` per entry, so you can resolve a target by arch when the user names a model instead of a `job_id`. Apply these rules in order:\n\n1. **User provided an explicit `job_id`** → use it. Verify it exists in `state`.\n2. **User named a `network_arch`** (e.g. \"send this to the cosmos-rl service\") → look up matching entries: `candidates = [j for j, e in state.items() if j != \"latest\" and isinstance(e, dict) and e[\"network_arch\"] == arch]`.\n   - Exactly one match → use it.\n   - Multiple matches → **prompt the user** with the candidate `job_id`s and their `started_at`; do not auto-pick.\n   - No match → stop and tell the user no service for that arch is running.\n3. **No `job_id` and no `network_arch`** → count non-`\"latest\"` entries in `state`:\n   - Exactly one running service → use it.\n   - Two or more → **do not silently default to `state[\"latest\"]`**. Prompt the user with the full list (`job_id`, `network_arch`, `host_url`) and require an explicit choice. The `\"latest\"` pointer is a convenience for single-service workflows, not a routing fallback when multiple services coexist.\n   - Zero → stop and tell the user to start a service first.\n\nAfter resolving, read the endpoint from the registry (`references\u002Fcode-templates.yaml` → `request.registry_read`), passing the resolved `job_id` as `user_provided_job_id`. Confirm to the user: \"Sending to job_id=… arch=… url=…\". If the service may still be loading, poll readiness first (`references\u002Fcode-templates.yaml` → `readiness_check`).\n\n**Cross-check before sending:** if the user-supplied request body contains arch-specific fields (e.g. `guidance` \u002F `num_steps` \u002F `seed` \u002F `negative_prompt` → cosmos-predict2.5; required `image_url`\u002F`video_url` content items → cosmos-rl), verify they are consistent with `state[job_id][\"network_arch\"]`. On mismatch, stop and ask — sending a cosmos-predict2.5 body to a cosmos-rl service will fail at the container with a 4xx\u002F5xx that is harder to diagnose than catching it here.\n\n### 6.1 Sampling parameters — REQUIRED user prompt before each request\n\nBefore constructing the request body, you **MUST** explicitly prompt the user for the vLLM-style sampling parameters. Do **not** silently apply defaults. Use a structured prompt, one question per field, that:\n\n1. Lists every applicable field with its **type** and **default value**.\n2. Lets the user skip \u002F accept any field to take that field's default — entering a value is never required.\n3. Collects all fields in one round.\n\nAfter the prompt, apply each user-entered value verbatim and substitute the default for any skipped field. Do not invent values or silently clamp.\n\n**Field list, defaults, and per-arch applicability:** `references\u002Frequest.yaml` → `chat_completions_request_body` (base sampling fields: `max_tokens`, `top_p`, `temperature`) and `network_arch_constraints.\u003Cnetwork_arch>` (per-arch overrides and extras such as `guidance`\u002F`num_steps`\u002F`seed`\u002F`negative_prompt` for `cosmos-predict2.5`). If a field is marked unsupported for the active arch, do **not** prompt for it and do **not** include it in the body.\n\n### 6.2 Request format\n\nSend a `POST` to `{BASE_URL}\u002Fv1\u002Fchat\u002Fcompletions` with `Content-Type: application\u002Fjson` and a timeout of **at least 300 s**. The body is OpenAI-compatible (vLLM chat completions); see `references\u002Frequest.yaml` → `chat_completions_request_body` for the full field schema and content-item shapes (text \u002F image_url \u002F video_url), and `code_examples` for ready-to-run Python and curl samples.\n\n**Constraints:** only the first user message is processed. No secret values in request bodies. **Per-network constraints** (e.g. cosmos-rl requires every request to include an image or video; cosmos-rl rejects `data:` URIs) are in `references\u002Frequest.yaml` → `network_arch_constraints`.\n\n### 6.3 Response handling\n\n| HTTP status | Meaning | Action |\n|-------------|---------|--------|\n| **200** | Success — `choices[0].message.content` has the generated text | Read result |\n| **202** | Server still initializing or model still loading | Retry after a delay |\n| **503** | Initialization failed, model load failed, **or model not yet ready** | Inspect `error.type`: `model_not_ready` → retry; `initialization_error` \u002F `model_load_error` → give up and check logs |\n| **400** | Missing or empty JSON body | Fix request |\n| **500** | Unhandled exception during inference | Check container logs |\n\nFor 202 and 503, the body contains `{\"error\": {\"type\": \"\u003Cerror_type>\", \"message\": \"\u003Creason>\"}}`. See `container_response_shapes` in `references\u002Frequest.yaml` for error type strings.\n",{"data":32,"body":42},{"name":4,"description":6,"license":23,"compatibility":33,"metadata":34,"allowed-tools":37,"tags":38},"The inference service has no cloud-storage dependency — model weights come from the HuggingFace Hub (HF_TOKEN env var for gated models) or a local container path. Platform prerequisites are checked by each platform skill.",{"author":35,"version":36},"NVIDIA Corporation","0.1.0","Read Bash Write",[39,40,41],"inference","microservice","workflow",{"type":43,"children":44},"root",[45,54,61,71,142,150,247,278,288,340,344,350,360,389,455,502,505,511,759,762,768,788,926,954,957,963,1006,1020,1108,1136,1146,1189,1192,1198,1228,1235,1294,1299,1357,1376,1568,1574,1588,1596,1754,1762,1928,1960,2030,2036,2062,2080,2258,2263,2321,2339,2342,2348,2391,2524,2550,2553,2559,2565,2598,2778,2823,2886,2892,2910,2940,2945,3039,3045,3100,3138,3144,3320,3347],{"type":46,"tag":47,"props":48,"children":50},"element","h1",{"id":49},"tao-inference-microservice",[51],{"type":52,"value":53},"text","TAO Inference Microservice",{"type":46,"tag":55,"props":56,"children":58},"h2",{"id":57},"instructions",[59],{"type":52,"value":60},"Instructions",{"type":46,"tag":62,"props":63,"children":64},"p",{},[65],{"type":46,"tag":66,"props":67,"children":68},"strong",{},[69],{"type":52,"value":70},"To start an inference service:",{"type":46,"tag":72,"props":73,"children":74},"ol",{},[75,81,103,116],{"type":46,"tag":76,"props":77,"children":78},"li",{},[79],{"type":52,"value":80},"Collect required inputs (Section 1) and resolve the container image (Section 2).",{"type":46,"tag":76,"props":82,"children":83},{},[84,86,93,95,101],{"type":52,"value":85},"Build the job payload and inner command (Sections 3–4.1); use ",{"type":46,"tag":87,"props":88,"children":90},"code",{"className":89},[],[91],{"type":52,"value":92},"references\u002Fcode-templates.yaml",{"type":52,"value":94}," → ",{"type":46,"tag":87,"props":96,"children":98},{"className":97},[],[99],{"type":52,"value":100},"job_payload_builder",{"type":52,"value":102},".",{"type":46,"tag":76,"props":104,"children":105},{},[106,108,114],{"type":52,"value":107},"Read ",{"type":46,"tag":87,"props":109,"children":111},{"className":110},[],[112],{"type":52,"value":113},"skills\u002Fplatform\u002F\u003Cplatform>\u002FSKILL.md",{"type":52,"value":115}," and start the container (Section 4.2).",{"type":46,"tag":76,"props":117,"children":118},{},[119,121,126,127,133,135,141],{"type":52,"value":120},"Write the service registry and poll readiness (Section 4.3); use ",{"type":46,"tag":87,"props":122,"children":124},{"className":123},[],[125],{"type":52,"value":92},{"type":52,"value":94},{"type":46,"tag":87,"props":128,"children":130},{"className":129},[],[131],{"type":52,"value":132},"registry_write.\u003Cplatform>",{"type":52,"value":134}," and ",{"type":46,"tag":87,"props":136,"children":138},{"className":137},[],[139],{"type":52,"value":140},"readiness_check",{"type":52,"value":102},{"type":46,"tag":62,"props":143,"children":144},{},[145],{"type":46,"tag":66,"props":146,"children":147},{},[148],{"type":52,"value":149},"To send an inference request:",{"type":46,"tag":72,"props":151,"children":152},{},[153,209,242],{"type":46,"tag":76,"props":154,"children":155},{},[156,158,164,166,172,174,187,189,194,195,201,203,208],{"type":52,"value":157},"Resolve which service receives the request per Section 6.0 (by ",{"type":46,"tag":87,"props":159,"children":161},{"className":160},[],[162],{"type":52,"value":163},"job_id",{"type":52,"value":165},", by ",{"type":46,"tag":87,"props":167,"children":169},{"className":168},[],[170],{"type":52,"value":171},"network_arch",{"type":52,"value":173},", or by explicit user choice when multiple services run — ",{"type":46,"tag":66,"props":175,"children":176},{},[177,179,185],{"type":52,"value":178},"never silently default to ",{"type":46,"tag":87,"props":180,"children":182},{"className":181},[],[183],{"type":52,"value":184},"\"latest\"",{"type":52,"value":186}," when more than one service exists",{"type":52,"value":188},"), then read the endpoint from ",{"type":46,"tag":87,"props":190,"children":192},{"className":191},[],[193],{"type":52,"value":92},{"type":52,"value":94},{"type":46,"tag":87,"props":196,"children":198},{"className":197},[],[199],{"type":52,"value":200},"request.registry_read",{"type":52,"value":202}," with the resolved ",{"type":46,"tag":87,"props":204,"children":206},{"className":205},[],[207],{"type":52,"value":163},{"type":52,"value":102},{"type":46,"tag":76,"props":210,"children":211},{},[212,217,219,225,227,233,234,240],{"type":46,"tag":66,"props":213,"children":214},{},[215],{"type":52,"value":216},"Before building the request body, prompt the user for the vLLM-style sampling parameters (Section 6.1).",{"type":52,"value":218}," Present ",{"type":46,"tag":87,"props":220,"children":222},{"className":221},[],[223],{"type":52,"value":224},"max_tokens",{"type":52,"value":226},", ",{"type":46,"tag":87,"props":228,"children":230},{"className":229},[],[231],{"type":52,"value":232},"top_p",{"type":52,"value":226},{"type":46,"tag":87,"props":235,"children":237},{"className":236},[],[238],{"type":52,"value":239},"temperature",{"type":52,"value":241}," (and any per-arch extras) with their defaults; let the user override or skip each one to accept the default. Never silently use defaults.",{"type":46,"tag":76,"props":243,"children":244},{},[245],{"type":52,"value":246},"Build and send the body per Section 6.2; handle the response per Section 6.3.",{"type":46,"tag":62,"props":248,"children":249},{},[250,255,257,262,263,269,271,276],{"type":46,"tag":66,"props":251,"children":252},{},[253],{"type":52,"value":254},"To stop a service:",{"type":52,"value":256}," Read ",{"type":46,"tag":87,"props":258,"children":260},{"className":259},[],[261],{"type":52,"value":92},{"type":52,"value":94},{"type":46,"tag":87,"props":264,"children":266},{"className":265},[],[267],{"type":52,"value":268},"stop.registry_read",{"type":52,"value":270}," to resolve the job_id, read ",{"type":46,"tag":87,"props":272,"children":274},{"className":273},[],[275],{"type":52,"value":113},{"type":52,"value":277},", then follow Section 5.",{"type":46,"tag":62,"props":279,"children":280},{},[281,286],{"type":46,"tag":66,"props":282,"children":283},{},[284],{"type":52,"value":285},"Reference data",{"type":52,"value":287}," (schemas, mappings, valid values — no instructions):",{"type":46,"tag":289,"props":290,"children":291},"ul",{},[292,313,327],{"type":46,"tag":76,"props":293,"children":294},{},[295,304,306,311],{"type":46,"tag":66,"props":296,"children":297},{},[298],{"type":46,"tag":87,"props":299,"children":301},{"className":300},[],[302],{"type":52,"value":303},"references\u002Fservice.yaml",{"type":52,"value":305}," — image mappings, valid ",{"type":46,"tag":87,"props":307,"children":309},{"className":308},[],[310],{"type":52,"value":171},{"type":52,"value":312}," names, job payload schema, env var names, secrets classification.",{"type":46,"tag":76,"props":314,"children":315},{},[316,325],{"type":46,"tag":66,"props":317,"children":318},{},[319],{"type":46,"tag":87,"props":320,"children":322},{"className":321},[],[323],{"type":52,"value":324},"references\u002Frequest.yaml",{"type":52,"value":326}," — endpoint definition, request field schema, response shapes, code examples.",{"type":46,"tag":76,"props":328,"children":329},{},[330,338],{"type":46,"tag":66,"props":331,"children":332},{},[333],{"type":46,"tag":87,"props":334,"children":336},{"className":335},[],[337],{"type":52,"value":92},{"type":52,"value":339}," — Python templates for payload building, registry writes, readiness checks, and stop\u002Frequest flows.",{"type":46,"tag":341,"props":342,"children":343},"hr",{},[],{"type":46,"tag":55,"props":345,"children":347},{"id":346},"secrets-rule-applies-to-every-generated-code-block-in-this-skill",[348],{"type":52,"value":349},"Secrets rule (applies to every generated code block in this skill)",{"type":46,"tag":62,"props":351,"children":352},{},[353,358],{"type":46,"tag":66,"props":354,"children":355},{},[356],{"type":52,"value":357},"Never ask the user to type a secret value into a prompt.",{"type":52,"value":359}," For every secret value:",{"type":46,"tag":72,"props":361,"children":362},{},[363,376],{"type":46,"tag":76,"props":364,"children":365},{},[366,368,374],{"type":52,"value":367},"Tell the user which environment variable to set (e.g. ",{"type":46,"tag":87,"props":369,"children":371},{"className":370},[],[372],{"type":52,"value":373},"export HF_TOKEN=...",{"type":52,"value":375},").",{"type":46,"tag":76,"props":377,"children":378},{},[379,381,387],{"type":52,"value":380},"Generate code that reads it with ",{"type":46,"tag":87,"props":382,"children":384},{"className":383},[],[385],{"type":52,"value":386},"os.environ[\"VAR_NAME\"]",{"type":52,"value":388}," — never hard-code, interpolate, or prompt for the value.",{"type":46,"tag":62,"props":390,"children":391},{},[392,397,399,404,405,411,413,419,420,426,427,433,434,440,441,447,448,454],{"type":46,"tag":66,"props":393,"children":394},{},[395],{"type":52,"value":396},"Secret env vars",{"type":52,"value":398}," (full list in ",{"type":46,"tag":87,"props":400,"children":402},{"className":401},[],[403],{"type":52,"value":303},{"type":52,"value":94},{"type":46,"tag":87,"props":406,"children":408},{"className":407},[],[409],{"type":52,"value":410},"secrets_handling",{"type":52,"value":412},"):\n",{"type":46,"tag":87,"props":414,"children":416},{"className":415},[],[417],{"type":52,"value":418},"HF_TOKEN",{"type":52,"value":226},{"type":46,"tag":87,"props":421,"children":423},{"className":422},[],[424],{"type":52,"value":425},"WANDB_API_KEY",{"type":52,"value":226},{"type":46,"tag":87,"props":428,"children":430},{"className":429},[],[431],{"type":52,"value":432},"CLEARML_API_ACCESS_KEY",{"type":52,"value":226},{"type":46,"tag":87,"props":435,"children":437},{"className":436},[],[438],{"type":52,"value":439},"CLEARML_API_SECRET_KEY",{"type":52,"value":226},{"type":46,"tag":87,"props":442,"children":444},{"className":443},[],[445],{"type":52,"value":446},"TAO_API_KEY",{"type":52,"value":226},{"type":46,"tag":87,"props":449,"children":451},{"className":450},[],[452],{"type":52,"value":453},"TAO_USER_KEY",{"type":52,"value":102},{"type":46,"tag":62,"props":456,"children":457},{},[458,463,465,470,471,477,478,484,486,492,494,500],{"type":46,"tag":66,"props":459,"children":460},{},[461],{"type":52,"value":462},"Safe to collect in the prompt:",{"type":52,"value":464}," ",{"type":46,"tag":87,"props":466,"children":468},{"className":467},[],[469],{"type":52,"value":171},{"type":52,"value":226},{"type":46,"tag":87,"props":472,"children":474},{"className":473},[],[475],{"type":52,"value":476},"model_path",{"type":52,"value":226},{"type":46,"tag":87,"props":479,"children":481},{"className":480},[],[482],{"type":52,"value":483},"num_gpus",{"type":52,"value":485},", prompt text, ",{"type":46,"tag":87,"props":487,"children":489},{"className":488},[],[490],{"type":52,"value":491},"WANDB_*",{"type":52,"value":493}," config URLs, ",{"type":46,"tag":87,"props":495,"children":497},{"className":496},[],[498],{"type":52,"value":499},"CLEARML_*_HOST",{"type":52,"value":501}," URLs.",{"type":46,"tag":341,"props":503,"children":504},{},[],{"type":46,"tag":55,"props":506,"children":508},{"id":507},"_1-what-to-collect-from-the-user",[509],{"type":52,"value":510},"1. What to collect from the user",{"type":46,"tag":512,"props":513,"children":514},"table",{},[515,534],{"type":46,"tag":516,"props":517,"children":518},"thead",{},[519],{"type":46,"tag":520,"props":521,"children":522},"tr",{},[523,529],{"type":46,"tag":524,"props":525,"children":526},"th",{},[527],{"type":52,"value":528},"Input",{"type":46,"tag":524,"props":530,"children":531},{},[532],{"type":52,"value":533},"Role",{"type":46,"tag":535,"props":536,"children":537},"tbody",{},[538,609,678,727],{"type":46,"tag":520,"props":539,"children":540},{},[541,553],{"type":46,"tag":542,"props":543,"children":544},"td",{},[545],{"type":46,"tag":66,"props":546,"children":547},{},[548],{"type":46,"tag":87,"props":549,"children":551},{"className":550},[],[552],{"type":52,"value":171},{"type":46,"tag":542,"props":554,"children":555},{},[556,558,563,564,570,572,578,580,586,588,593,595,601,602,608],{"type":52,"value":557},"Chooses container image, the per-arch inner command shape (",{"type":46,"tag":87,"props":559,"children":561},{"className":560},[],[562],{"type":52,"value":303},{"type":52,"value":94},{"type":46,"tag":87,"props":565,"children":567},{"className":566},[],[568],{"type":52,"value":569},"container_commands.\u003Cnetwork_arch>",{"type":52,"value":571},"), and ",{"type":46,"tag":87,"props":573,"children":575},{"className":574},[],[576],{"type":52,"value":577},"neural_network_name",{"type":52,"value":579}," in the job JSON when applicable. Must match a basename in ",{"type":46,"tag":87,"props":581,"children":583},{"className":582},[],[584],{"type":52,"value":585},"valid_network_arch_config_basenames",{"type":52,"value":587}," in ",{"type":46,"tag":87,"props":589,"children":591},{"className":590},[],[592],{"type":52,"value":303},{"type":52,"value":594}," (e.g. ",{"type":46,"tag":87,"props":596,"children":598},{"className":597},[],[599],{"type":52,"value":600},"cosmos-rl",{"type":52,"value":226},{"type":46,"tag":87,"props":603,"children":605},{"className":604},[],[606],{"type":52,"value":607},"cosmos-predict2.5",{"type":52,"value":375},{"type":46,"tag":520,"props":610,"children":611},{},[612,623],{"type":46,"tag":542,"props":613,"children":614},{},[615],{"type":46,"tag":66,"props":616,"children":617},{},[618],{"type":46,"tag":87,"props":619,"children":621},{"className":620},[],[622],{"type":52,"value":476},{"type":46,"tag":542,"props":624,"children":625},{},[626,628,634,636,641,643,649,650,656,657,663,665,670,671,677],{"type":52,"value":627},"The trained model checkpoint. Valid forms: ",{"type":46,"tag":87,"props":629,"children":631},{"className":630},[],[632],{"type":52,"value":633},"hf_model:\u002F\u002F\u003Corg>\u002F\u003Cmodel>",{"type":52,"value":635}," (HuggingFace Hub — set ",{"type":46,"tag":87,"props":637,"children":639},{"className":638},[],[640],{"type":52,"value":418},{"type":52,"value":642}," for gated models) or a local container filesystem path. Cloud URIs (",{"type":46,"tag":87,"props":644,"children":646},{"className":645},[],[647],{"type":52,"value":648},"s3:\u002F\u002F",{"type":52,"value":226},{"type":46,"tag":87,"props":651,"children":653},{"className":652},[],[654],{"type":52,"value":655},"gs:\u002F\u002F",{"type":52,"value":226},{"type":46,"tag":87,"props":658,"children":660},{"className":659},[],[661],{"type":52,"value":662},"az:\u002F\u002F",{"type":52,"value":664},") are NOT supported — the inference service has no cloud-storage dependency. Always ask the user; never substitute a placeholder. See ",{"type":46,"tag":87,"props":666,"children":668},{"className":667},[],[669],{"type":52,"value":303},{"type":52,"value":94},{"type":46,"tag":87,"props":672,"children":674},{"className":673},[],[675],{"type":52,"value":676},"model_path_protocols",{"type":52,"value":102},{"type":46,"tag":520,"props":679,"children":680},{},[681,693],{"type":46,"tag":542,"props":682,"children":683},{},[684],{"type":46,"tag":66,"props":685,"children":686},{},[687],{"type":46,"tag":87,"props":688,"children":690},{"className":689},[],[691],{"type":52,"value":692},"platform",{"type":46,"tag":542,"props":694,"children":695},{},[696,698,704,705,711,712,718,720,726],{"type":52,"value":697},"Compute platform: ",{"type":46,"tag":87,"props":699,"children":701},{"className":700},[],[702],{"type":52,"value":703},"local-docker",{"type":52,"value":226},{"type":46,"tag":87,"props":706,"children":708},{"className":707},[],[709],{"type":52,"value":710},"brev",{"type":52,"value":226},{"type":46,"tag":87,"props":713,"children":715},{"className":714},[],[716],{"type":52,"value":717},"slurm",{"type":52,"value":719},", or ",{"type":46,"tag":87,"props":721,"children":723},{"className":722},[],[724],{"type":52,"value":725},"kubernetes",{"type":52,"value":102},{"type":46,"tag":520,"props":728,"children":729},{},[730,741],{"type":46,"tag":542,"props":731,"children":732},{},[733],{"type":46,"tag":66,"props":734,"children":735},{},[736],{"type":46,"tag":87,"props":737,"children":739},{"className":738},[],[740],{"type":52,"value":483},{"type":46,"tag":542,"props":742,"children":743},{},[744,746,751,753,757],{"type":52,"value":745},"Defaults to ",{"type":46,"tag":66,"props":747,"children":748},{},[749],{"type":52,"value":750},"1",{"type":52,"value":752},"; minimum ",{"type":46,"tag":66,"props":754,"children":755},{},[756],{"type":52,"value":750},{"type":52,"value":758}," for inference.",{"type":46,"tag":341,"props":760,"children":761},{},[],{"type":46,"tag":55,"props":763,"children":765},{"id":764},"_2-image-resolution",[766],{"type":52,"value":767},"2. Image resolution",{"type":46,"tag":62,"props":769,"children":770},{},[771,773,778,780,786],{"type":52,"value":772},"Each ",{"type":46,"tag":87,"props":774,"children":776},{"className":775},[],[777],{"type":52,"value":171},{"type":52,"value":779}," has a sidecar config file named ",{"type":46,"tag":87,"props":781,"children":783},{"className":782},[],[784],{"type":52,"value":785},"{network_arch}.config.json",{"type":52,"value":787},". Resolve the container image as follows:",{"type":46,"tag":72,"props":789,"children":790},{},[791,830,851,907],{"type":46,"tag":76,"props":792,"children":793},{},[794,795,800,802,808,809,815,817,823,824,829],{"type":52,"value":107},{"type":46,"tag":87,"props":796,"children":798},{"className":797},[],[799],{"type":52,"value":785},{"type":52,"value":801}," and take ",{"type":46,"tag":87,"props":803,"children":805},{"className":804},[],[806],{"type":52,"value":807},"api_params.image",{"type":52,"value":594},{"type":46,"tag":87,"props":810,"children":812},{"className":811},[],[813],{"type":52,"value":814},"COSMOS_RL",{"type":52,"value":816},"). This is a key into ",{"type":46,"tag":87,"props":818,"children":820},{"className":819},[],[821],{"type":52,"value":822},"docker_image_defaults.mapping",{"type":52,"value":587},{"type":46,"tag":87,"props":825,"children":827},{"className":826},[],[828],{"type":52,"value":303},{"type":52,"value":102},{"type":46,"tag":76,"props":831,"children":832},{},[833,835,841,843,849],{"type":52,"value":834},"Look up that key in the mapping. If the host env var ",{"type":46,"tag":87,"props":836,"children":838},{"className":837},[],[839],{"type":52,"value":840},"IMAGE_\u003CKEY>",{"type":52,"value":842}," is set (e.g. ",{"type":46,"tag":87,"props":844,"children":846},{"className":845},[],[847],{"type":52,"value":848},"IMAGE_COSMOS_RL",{"type":52,"value":850},"), it overrides the mapped default.",{"type":46,"tag":76,"props":852,"children":853},{},[854,856,862,864,870,872,878,880,885,886,892,894,899,901,906],{"type":52,"value":855},"The mapped value is normally a dotted key into the repo-root ",{"type":46,"tag":87,"props":857,"children":859},{"className":858},[],[860],{"type":52,"value":861},"versions.yaml",{"type":52,"value":863}," manifest (e.g. ",{"type":46,"tag":87,"props":865,"children":867},{"className":866},[],[868],{"type":52,"value":869},"tao_toolkit.cosmos_rl",{"type":52,"value":871},"). Resolve it to a concrete ",{"type":46,"tag":87,"props":873,"children":875},{"className":874},[],[876],{"type":52,"value":877},"nvcr.io\u002F...",{"type":52,"value":879}," image URI by looking up ",{"type":46,"tag":87,"props":881,"children":883},{"className":882},[],[884],{"type":52,"value":861},{"type":52,"value":94},{"type":46,"tag":87,"props":887,"children":889},{"className":888},[],[890],{"type":52,"value":891},"images.\u003Cgroup>.\u003Cname>",{"type":52,"value":893},". Absolute URIs pass through unchanged, so an ",{"type":46,"tag":87,"props":895,"children":897},{"className":896},[],[898],{"type":52,"value":840},{"type":52,"value":900}," env-var override that contains a full URI still works. The Python helper for this lives in ",{"type":46,"tag":87,"props":902,"children":904},{"className":903},[],[905],{"type":52,"value":92},{"type":52,"value":102},{"type":46,"tag":76,"props":908,"children":909},{},[910,912,917,919,924],{"type":52,"value":911},"If the config file is missing or ",{"type":46,"tag":87,"props":913,"children":915},{"className":914},[],[916],{"type":52,"value":807},{"type":52,"value":918}," is empty, fall back to the ",{"type":46,"tag":87,"props":920,"children":922},{"className":921},[],[923],{"type":52,"value":814},{"type":52,"value":925}," key.",{"type":46,"tag":62,"props":927,"children":928},{},[929,931,937,939,944,946,952],{"type":52,"value":930},"The config file also has ",{"type":46,"tag":87,"props":932,"children":934},{"className":933},[],[935],{"type":52,"value":936},"spec_params.inference.model_path",{"type":52,"value":938}," which drives ",{"type":46,"tag":66,"props":940,"children":941},{},[942],{"type":52,"value":943},"folder vs file",{"type":52,"value":945}," path semantics: if the value contains the substring ",{"type":46,"tag":87,"props":947,"children":949},{"className":948},[],[950],{"type":52,"value":951},"folder",{"type":52,"value":953},", the container treats the path as a directory.",{"type":46,"tag":341,"props":955,"children":956},{},[],{"type":46,"tag":55,"props":958,"children":960},{"id":959},"_3-environment-variables-no-callbacks",[961],{"type":52,"value":962},"3. Environment variables (no callbacks)",{"type":46,"tag":62,"props":964,"children":965},{},[966,968,974,976,982,984,989,991,997,999,1005],{"type":52,"value":967},"Set these in ",{"type":46,"tag":87,"props":969,"children":971},{"className":970},[],[972],{"type":52,"value":973},"env_payload",{"type":52,"value":975}," before encoding ",{"type":46,"tag":87,"props":977,"children":979},{"className":978},[],[980],{"type":52,"value":981},"env_json",{"type":52,"value":983},". Do ",{"type":46,"tag":66,"props":985,"children":986},{},[987],{"type":52,"value":988},"not",{"type":52,"value":990}," set ",{"type":46,"tag":87,"props":992,"children":994},{"className":993},[],[995],{"type":52,"value":996},"TAO_LOGGING_SERVER_URL",{"type":52,"value":998}," or ",{"type":46,"tag":87,"props":1000,"children":1002},{"className":1001},[],[1003],{"type":52,"value":1004},"TAO_ADMIN_KEY",{"type":52,"value":102},{"type":46,"tag":62,"props":1007,"children":1008},{},[1009,1018],{"type":46,"tag":66,"props":1010,"children":1011},{},[1012],{"type":46,"tag":87,"props":1013,"children":1015},{"className":1014},[],[1016],{"type":52,"value":1017},"TAO_EXECUTION_BACKEND",{"type":52,"value":1019}," — must match the platform:",{"type":46,"tag":512,"props":1021,"children":1022},{},[1023,1044],{"type":46,"tag":516,"props":1024,"children":1025},{},[1026],{"type":46,"tag":520,"props":1027,"children":1028},{},[1029,1034],{"type":46,"tag":524,"props":1030,"children":1031},{},[1032],{"type":52,"value":1033},"Platform",{"type":46,"tag":524,"props":1035,"children":1036},{},[1037,1042],{"type":46,"tag":87,"props":1038,"children":1040},{"className":1039},[],[1041],{"type":52,"value":1017},{"type":52,"value":1043}," value",{"type":46,"tag":535,"props":1045,"children":1046},{},[1047,1062,1077,1092],{"type":46,"tag":520,"props":1048,"children":1049},{},[1050,1054],{"type":46,"tag":542,"props":1051,"children":1052},{},[1053],{"type":52,"value":703},{"type":46,"tag":542,"props":1055,"children":1056},{},[1057],{"type":46,"tag":87,"props":1058,"children":1060},{"className":1059},[],[1061],{"type":52,"value":703},{"type":46,"tag":520,"props":1063,"children":1064},{},[1065,1069],{"type":46,"tag":542,"props":1066,"children":1067},{},[1068],{"type":52,"value":710},{"type":46,"tag":542,"props":1070,"children":1071},{},[1072],{"type":46,"tag":87,"props":1073,"children":1075},{"className":1074},[],[1076],{"type":52,"value":703},{"type":46,"tag":520,"props":1078,"children":1079},{},[1080,1084],{"type":46,"tag":542,"props":1081,"children":1082},{},[1083],{"type":52,"value":717},{"type":46,"tag":542,"props":1085,"children":1086},{},[1087],{"type":46,"tag":87,"props":1088,"children":1090},{"className":1089},[],[1091],{"type":52,"value":717},{"type":46,"tag":520,"props":1093,"children":1094},{},[1095,1099],{"type":46,"tag":542,"props":1096,"children":1097},{},[1098],{"type":52,"value":725},{"type":46,"tag":542,"props":1100,"children":1101},{},[1102],{"type":46,"tag":87,"props":1103,"children":1105},{"className":1104},[],[1106],{"type":52,"value":1107},"local-k8s",{"type":46,"tag":62,"props":1109,"children":1110},{},[1111,1120,1122,1128,1130,1135],{"type":46,"tag":66,"props":1112,"children":1113},{},[1114],{"type":46,"tag":87,"props":1115,"children":1117},{"className":1116},[],[1118],{"type":52,"value":1119},"CLOUD_BASED",{"type":52,"value":1121}," — always ",{"type":46,"tag":87,"props":1123,"children":1125},{"className":1124},[],[1126],{"type":52,"value":1127},"\"False\"",{"type":52,"value":1129}," for this skill (disables callback posting to ",{"type":46,"tag":87,"props":1131,"children":1133},{"className":1132},[],[1134],{"type":52,"value":996},{"type":52,"value":375},{"type":46,"tag":62,"props":1137,"children":1138},{},[1139,1144],{"type":46,"tag":66,"props":1140,"children":1141},{},[1142],{"type":52,"value":1143},"GPU env vars",{"type":52,"value":1145}," — only needed when the platform skill does not handle GPU injection automatically:",{"type":46,"tag":289,"props":1147,"children":1148},{},[1149,1176],{"type":46,"tag":76,"props":1150,"children":1151},{},[1152,1154,1160,1162,1168,1169,1175],{"type":52,"value":1153},"Tegra \u002F Jetson: ",{"type":46,"tag":87,"props":1155,"children":1157},{"className":1156},[],[1158],{"type":52,"value":1159},"--runtime=nvidia",{"type":52,"value":1161}," with ",{"type":46,"tag":87,"props":1163,"children":1165},{"className":1164},[],[1166],{"type":52,"value":1167},"NVIDIA_DRIVER_CAPABILITIES=all",{"type":52,"value":134},{"type":46,"tag":87,"props":1170,"children":1172},{"className":1171},[],[1173],{"type":52,"value":1174},"NVIDIA_VISIBLE_DEVICES=\u003Cids>",{"type":52,"value":102},{"type":46,"tag":76,"props":1177,"children":1178},{},[1179,1181,1187],{"type":52,"value":1180},"Standard x86 + nvidia-container-toolkit: use Docker ",{"type":46,"tag":87,"props":1182,"children":1184},{"className":1183},[],[1185],{"type":52,"value":1186},"device_requests",{"type":52,"value":1188},". The platform skill handles this.",{"type":46,"tag":341,"props":1190,"children":1191},{},[],{"type":46,"tag":55,"props":1193,"children":1195},{"id":1194},"_4-executing-across-platforms",[1196],{"type":52,"value":1197},"4. Executing across platforms",{"type":46,"tag":62,"props":1199,"children":1200},{},[1201,1203,1208,1210,1219,1221,1226],{"type":52,"value":1202},"The job payload and inner command (Sections 1–3) are ",{"type":46,"tag":66,"props":1204,"children":1205},{},[1206],{"type":52,"value":1207},"platform-agnostic",{"type":52,"value":1209},". For each platform, read ",{"type":46,"tag":66,"props":1211,"children":1212},{},[1213],{"type":46,"tag":87,"props":1214,"children":1216},{"className":1215},[],[1217],{"type":52,"value":1218},"skills\u002Fplatform\u002F\u003Cname>\u002FSKILL.md",{"type":52,"value":1220}," for preflight checks and credentials ",{"type":46,"tag":66,"props":1222,"children":1223},{},[1224],{"type":52,"value":1225},"before",{"type":52,"value":1227}," generating any execution code.",{"type":46,"tag":1229,"props":1230,"children":1232},"h3",{"id":1231},"_41-build-the-inner-command-per-arch",[1233],{"type":52,"value":1234},"4.1 Build the inner command (per arch)",{"type":46,"tag":62,"props":1236,"children":1237},{},[1238,1240,1250,1252,1257,1258,1263,1265,1270,1271,1277,1279,1285,1287,1292],{"type":52,"value":1239},"The inner-command shape is ",{"type":46,"tag":66,"props":1241,"children":1242},{},[1243,1245],{"type":52,"value":1244},"per ",{"type":46,"tag":87,"props":1246,"children":1248},{"className":1247},[],[1249],{"type":52,"value":171},{"type":52,"value":1251}," — there is no uniform template. Look up the per-arch entry in ",{"type":46,"tag":87,"props":1253,"children":1255},{"className":1254},[],[1256],{"type":52,"value":303},{"type":52,"value":94},{"type":46,"tag":87,"props":1259,"children":1261},{"className":1260},[],[1262],{"type":52,"value":569},{"type":52,"value":1264},"; if not present, the arch is unsupported — stop and ask. Pick the matching sub-block in ",{"type":46,"tag":87,"props":1266,"children":1268},{"className":1267},[],[1269],{"type":52,"value":92},{"type":52,"value":94},{"type":46,"tag":87,"props":1272,"children":1274},{"className":1273},[],[1275],{"type":52,"value":1276},"job_payload_builder.\u003Cnetwork_arch>",{"type":52,"value":1278},". Prefix the command with ",{"type":46,"tag":87,"props":1280,"children":1282},{"className":1281},[],[1283],{"type":52,"value":1284},"umask 0 &&",{"type":52,"value":1286}," and keep it ",{"type":46,"tag":66,"props":1288,"children":1289},{},[1290],{"type":52,"value":1291},"identical across platforms",{"type":52,"value":1293}," (local-docker, brev, slurm, kubernetes).",{"type":46,"tag":62,"props":1295,"children":1296},{},[1297],{"type":52,"value":1298},"Common across arches:",{"type":46,"tag":289,"props":1300,"children":1301},{},[1302,1320,1331],{"type":46,"tag":76,"props":1303,"children":1304},{},[1305,1310,1312,1318],{"type":46,"tag":87,"props":1306,"children":1308},{"className":1307},[],[1309],{"type":52,"value":163},{"type":52,"value":1311},": fresh ",{"type":46,"tag":87,"props":1313,"children":1315},{"className":1314},[],[1316],{"type":52,"value":1317},"uuid.uuid4()",{"type":52,"value":1319}," — becomes the container name and registry key.",{"type":46,"tag":76,"props":1321,"children":1322},{},[1323,1329],{"type":46,"tag":87,"props":1324,"children":1326},{"className":1325},[],[1327],{"type":52,"value":1328},"image",{"type":52,"value":1330},": resolve per Section 2.",{"type":46,"tag":76,"props":1332,"children":1333},{},[1334,1336,1342,1343,1349,1350,1355],{"type":52,"value":1335},"Secrets (",{"type":46,"tag":87,"props":1337,"children":1339},{"className":1338},[],[1340],{"type":52,"value":1341},"access_key",{"type":52,"value":226},{"type":46,"tag":87,"props":1344,"children":1346},{"className":1345},[],[1347],{"type":52,"value":1348},"secret_key",{"type":52,"value":226},{"type":46,"tag":87,"props":1351,"children":1353},{"className":1352},[],[1354],{"type":52,"value":418},{"type":52,"value":1356},", etc.) are read from env vars at runtime — never hard-code, never log or print.",{"type":46,"tag":62,"props":1358,"children":1359},{},[1360,1362,1367,1368,1374],{"type":52,"value":1361},"Arch-specific notes (full details in ",{"type":46,"tag":87,"props":1363,"children":1365},{"className":1364},[],[1366],{"type":52,"value":303},{"type":52,"value":94},{"type":46,"tag":87,"props":1369,"children":1371},{"className":1370},[],[1372],{"type":52,"value":1373},"container_commands",{"type":52,"value":1375},"):",{"type":46,"tag":289,"props":1377,"children":1378},{},[1379,1452],{"type":46,"tag":76,"props":1380,"children":1381},{},[1382,1390,1392,1398,1400,1406,1408,1414,1416,1421,1423,1428,1430,1436,1437,1443,1445,1450],{"type":46,"tag":66,"props":1383,"children":1384},{},[1385],{"type":46,"tag":87,"props":1386,"children":1388},{"className":1387},[],[1389],{"type":52,"value":600},{"type":52,"value":1391}," — single ",{"type":46,"tag":87,"props":1393,"children":1395},{"className":1394},[],[1396],{"type":52,"value":1397},"--job '\u003CJOB_JSON>' --docker_env_vars '\u003CENV_JSON>'",{"type":52,"value":1399}," blob; ",{"type":46,"tag":87,"props":1401,"children":1403},{"className":1402},[],[1404],{"type":52,"value":1405},"json.dumps(...)",{"type":52,"value":1407}," + ",{"type":46,"tag":87,"props":1409,"children":1411},{"className":1410},[],[1412],{"type":52,"value":1413},"shlex.quote(...)",{"type":52,"value":1415},". ",{"type":46,"tag":87,"props":1417,"children":1419},{"className":1418},[],[1420],{"type":52,"value":973},{"type":52,"value":1422}," carries ",{"type":46,"tag":87,"props":1424,"children":1426},{"className":1425},[],[1427],{"type":52,"value":1017},{"type":52,"value":1429}," (per Section 3 table), ",{"type":46,"tag":87,"props":1431,"children":1433},{"className":1432},[],[1434],{"type":52,"value":1435},"TAO_API_JOB_ID",{"type":52,"value":226},{"type":46,"tag":87,"props":1438,"children":1440},{"className":1439},[],[1441],{"type":52,"value":1442},"CLOUD_BASED=False",{"type":52,"value":1444},". The inference service has no cloud-storage dependency; ",{"type":46,"tag":87,"props":1446,"children":1448},{"className":1447},[],[1449],{"type":52,"value":418},{"type":52,"value":1451}," is the only cred env var that ever applies (for gated HuggingFace models).",{"type":46,"tag":76,"props":1453,"children":1454},{},[1455,1463,1465,1471,1473,1479,1481,1487,1489,1495,1497,1503,1505,1509,1511,1516,1518,1524,1526,1532,1534,1540,1542,1547,1549,1554,1555,1560,1561,1566],{"type":46,"tag":66,"props":1456,"children":1457},{},[1458],{"type":46,"tag":87,"props":1459,"children":1461},{"className":1460},[],[1462],{"type":52,"value":607},{"type":52,"value":1464}," — flag-style ",{"type":46,"tag":87,"props":1466,"children":1468},{"className":1467},[],[1469],{"type":52,"value":1470},"cosmos_predict inference_microservice start ... --port 8080",{"type":52,"value":1472}," (no ",{"type":46,"tag":87,"props":1474,"children":1476},{"className":1475},[],[1477],{"type":52,"value":1478},"setup.",{"type":52,"value":1480}," prefix; uses ",{"type":46,"tag":87,"props":1482,"children":1484},{"className":1483},[],[1485],{"type":52,"value":1486},"tyro.conf.OmitArgPrefixes",{"type":52,"value":1488},"). ",{"type":46,"tag":87,"props":1490,"children":1492},{"className":1491},[],[1493],{"type":52,"value":1494},"--job",{"type":52,"value":1496},"\u002F",{"type":46,"tag":87,"props":1498,"children":1500},{"className":1499},[],[1501],{"type":52,"value":1502},"--docker_env_vars",{"type":52,"value":1504}," are ",{"type":46,"tag":66,"props":1506,"children":1507},{},[1508],{"type":52,"value":988},{"type":52,"value":1510}," accepted. Translate ",{"type":46,"tag":87,"props":1512,"children":1514},{"className":1513},[],[1515],{"type":52,"value":476},{"type":52,"value":1517}," to ",{"type":46,"tag":87,"props":1519,"children":1521},{"className":1520},[],[1522],{"type":52,"value":1523},"--checkpoint-path",{"type":52,"value":1525}," (local path) or ",{"type":46,"tag":87,"props":1527,"children":1529},{"className":1528},[],[1530],{"type":52,"value":1531},"--model \u003Cregistered_key>",{"type":52,"value":1533}," (",{"type":46,"tag":87,"props":1535,"children":1537},{"className":1536},[],[1538],{"type":52,"value":1539},"hf_model:\u002F\u002F",{"type":52,"value":1541},"); cloud URIs are rejected. The only cred env var that ever applies is ",{"type":46,"tag":87,"props":1543,"children":1545},{"className":1544},[],[1546],{"type":52,"value":418},{"type":52,"value":1548}," for gated HuggingFace models. Per-request params (prompt, inference_type, num_output_frames, guidance, seed, num_steps, negative_prompt) go in the request body, not at startup. ",{"type":46,"tag":87,"props":1550,"children":1552},{"className":1551},[],[1553],{"type":52,"value":1017},{"type":52,"value":1496},{"type":46,"tag":87,"props":1556,"children":1558},{"className":1557},[],[1559],{"type":52,"value":1435},{"type":52,"value":1496},{"type":46,"tag":87,"props":1562,"children":1564},{"className":1563},[],[1565],{"type":52,"value":1119},{"type":52,"value":1567}," are unused and may be omitted.",{"type":46,"tag":1229,"props":1569,"children":1571},{"id":1570},"_42-delegate-execution-to-the-platform-skill",[1572],{"type":52,"value":1573},"4.2 Delegate execution to the platform skill",{"type":46,"tag":62,"props":1575,"children":1576},{},[1577,1578,1586],{"type":52,"value":107},{"type":46,"tag":66,"props":1579,"children":1580},{},[1581],{"type":46,"tag":87,"props":1582,"children":1584},{"className":1583},[],[1585],{"type":52,"value":113},{"type":52,"value":1587}," and follow it to start the container.",{"type":46,"tag":62,"props":1589,"children":1590},{},[1591],{"type":46,"tag":66,"props":1592,"children":1593},{},[1594],{"type":52,"value":1595},"Base parameters (all platforms):",{"type":46,"tag":512,"props":1597,"children":1598},{},[1599,1615],{"type":46,"tag":516,"props":1600,"children":1601},{},[1602],{"type":46,"tag":520,"props":1603,"children":1604},{},[1605,1610],{"type":46,"tag":524,"props":1606,"children":1607},{},[1608],{"type":52,"value":1609},"Parameter",{"type":46,"tag":524,"props":1611,"children":1612},{},[1613],{"type":52,"value":1614},"Value",{"type":46,"tag":535,"props":1616,"children":1617},{},[1618,1634,1657,1677,1697,1715],{"type":46,"tag":520,"props":1619,"children":1620},{},[1621,1629],{"type":46,"tag":542,"props":1622,"children":1623},{},[1624],{"type":46,"tag":87,"props":1625,"children":1627},{"className":1626},[],[1628],{"type":52,"value":1328},{"type":46,"tag":542,"props":1630,"children":1631},{},[1632],{"type":52,"value":1633},"resolved container image (Section 2)",{"type":46,"tag":520,"props":1635,"children":1636},{},[1637,1646],{"type":46,"tag":542,"props":1638,"children":1639},{},[1640],{"type":46,"tag":87,"props":1641,"children":1643},{"className":1642},[],[1644],{"type":52,"value":1645},"command",{"type":46,"tag":542,"props":1647,"children":1648},{},[1649,1655],{"type":46,"tag":87,"props":1650,"children":1652},{"className":1651},[],[1653],{"type":52,"value":1654},"inner",{"type":52,"value":1656}," — the shell string built in Section 4.1",{"type":46,"tag":520,"props":1658,"children":1659},{},[1660,1669],{"type":46,"tag":542,"props":1661,"children":1662},{},[1663],{"type":46,"tag":87,"props":1664,"children":1666},{"className":1665},[],[1667],{"type":52,"value":1668},"gpu_count",{"type":46,"tag":542,"props":1670,"children":1671},{},[1672],{"type":46,"tag":87,"props":1673,"children":1675},{"className":1674},[],[1676],{"type":52,"value":483},{"type":46,"tag":520,"props":1678,"children":1679},{},[1680,1689],{"type":46,"tag":542,"props":1681,"children":1682},{},[1683],{"type":46,"tag":87,"props":1684,"children":1686},{"className":1685},[],[1687],{"type":52,"value":1688},"env_vars",{"type":46,"tag":542,"props":1690,"children":1691},{},[1692],{"type":46,"tag":87,"props":1693,"children":1695},{"className":1694},[],[1696],{"type":52,"value":973},{"type":46,"tag":520,"props":1698,"children":1699},{},[1700,1705],{"type":46,"tag":542,"props":1701,"children":1702},{},[1703],{"type":52,"value":1704},"job \u002F container name",{"type":46,"tag":542,"props":1706,"children":1707},{},[1708,1713],{"type":46,"tag":87,"props":1709,"children":1711},{"className":1710},[],[1712],{"type":52,"value":163},{"type":52,"value":1714}," — must equal the UUID from 4.1 so the registry can reference it",{"type":46,"tag":520,"props":1716,"children":1717},{},[1718,1734],{"type":46,"tag":542,"props":1719,"children":1720},{},[1721,1727,1728],{"type":46,"tag":87,"props":1722,"children":1724},{"className":1723},[],[1725],{"type":52,"value":1726},"host_port",{"type":52,"value":464},{"type":46,"tag":1729,"props":1730,"children":1731},"em",{},[1732],{"type":52,"value":1733},"(local-docker, brev)",{"type":46,"tag":542,"props":1735,"children":1736},{},[1737,1739,1745,1747,1752],{"type":52,"value":1738},"host-side port to bind to container port 8080. Default ",{"type":46,"tag":87,"props":1740,"children":1742},{"className":1741},[],[1743],{"type":52,"value":1744},"8080",{"type":52,"value":1746},", but ",{"type":46,"tag":66,"props":1748,"children":1749},{},[1750],{"type":52,"value":1751},"must be unique per concurrent service",{"type":52,"value":1753}," — see the port-allocation rule below.",{"type":46,"tag":62,"props":1755,"children":1756},{},[1757],{"type":46,"tag":66,"props":1758,"children":1759},{},[1760],{"type":52,"value":1761},"Platform-specific additional inputs:",{"type":46,"tag":512,"props":1763,"children":1764},{},[1765,1780],{"type":46,"tag":516,"props":1766,"children":1767},{},[1768],{"type":46,"tag":520,"props":1769,"children":1770},{},[1771,1775],{"type":46,"tag":524,"props":1772,"children":1773},{},[1774],{"type":52,"value":1033},{"type":46,"tag":524,"props":1776,"children":1777},{},[1778],{"type":52,"value":1779},"Additional inputs",{"type":46,"tag":535,"props":1781,"children":1782},{},[1783,1798,1840,1883],{"type":46,"tag":520,"props":1784,"children":1785},{},[1786,1793],{"type":46,"tag":542,"props":1787,"children":1788},{},[1789],{"type":46,"tag":66,"props":1790,"children":1791},{},[1792],{"type":52,"value":703},{"type":46,"tag":542,"props":1794,"children":1795},{},[1796],{"type":52,"value":1797},"None beyond base",{"type":46,"tag":520,"props":1799,"children":1800},{},[1801,1808],{"type":46,"tag":542,"props":1802,"children":1803},{},[1804],{"type":46,"tag":66,"props":1805,"children":1806},{},[1807],{"type":52,"value":710},{"type":46,"tag":542,"props":1809,"children":1810},{},[1811,1817,1819,1825,1826,1832,1834],{"type":46,"tag":87,"props":1812,"children":1814},{"className":1813},[],[1815],{"type":52,"value":1816},"instance_id",{"type":52,"value":1818}," (optional — reuse an existing instance); on multi-credential \u002F multi-workspace accounts also ",{"type":46,"tag":87,"props":1820,"children":1822},{"className":1821},[],[1823],{"type":52,"value":1824},"cloud_cred_id",{"type":52,"value":134},{"type":46,"tag":87,"props":1827,"children":1829},{"className":1828},[],[1830],{"type":52,"value":1831},"workspace_group_id",{"type":52,"value":1833}," for first-create — see ",{"type":46,"tag":87,"props":1835,"children":1837},{"className":1836},[],[1838],{"type":52,"value":1839},"skills\u002Fplatform\u002Ftao-run-on-brev\u002FSKILL.md",{"type":46,"tag":520,"props":1841,"children":1842},{},[1843,1850],{"type":46,"tag":542,"props":1844,"children":1845},{},[1846],{"type":46,"tag":66,"props":1847,"children":1848},{},[1849],{"type":52,"value":717},{"type":46,"tag":542,"props":1851,"children":1852},{},[1853,1859,1860,1866,1868,1874,1875,1881],{"type":46,"tag":87,"props":1854,"children":1856},{"className":1855},[],[1857],{"type":52,"value":1858},"partition",{"type":52,"value":134},{"type":46,"tag":87,"props":1861,"children":1863},{"className":1862},[],[1864],{"type":52,"value":1865},"account",{"type":52,"value":1867}," — check ",{"type":46,"tag":87,"props":1869,"children":1871},{"className":1870},[],[1872],{"type":52,"value":1873},"SLURM_PARTITION",{"type":52,"value":1496},{"type":46,"tag":87,"props":1876,"children":1878},{"className":1877},[],[1879],{"type":52,"value":1880},"SLURM_ACCOUNT",{"type":52,"value":1882}," env vars; ask user if unset",{"type":46,"tag":520,"props":1884,"children":1885},{},[1886,1893],{"type":46,"tag":542,"props":1887,"children":1888},{},[1889],{"type":46,"tag":66,"props":1890,"children":1891},{},[1892],{"type":52,"value":725},{"type":46,"tag":542,"props":1894,"children":1895},{},[1896,1902,1904,1910,1912,1918,1920,1926],{"type":46,"tag":87,"props":1897,"children":1899},{"className":1898},[],[1900],{"type":52,"value":1901},"namespace",{"type":52,"value":1903}," (default: ",{"type":46,"tag":87,"props":1905,"children":1907},{"className":1906},[],[1908],{"type":52,"value":1909},"default",{"type":52,"value":1911},"); ",{"type":46,"tag":87,"props":1913,"children":1915},{"className":1914},[],[1916],{"type":52,"value":1917},"image_pull_secret",{"type":52,"value":1919}," (required for ",{"type":46,"tag":87,"props":1921,"children":1923},{"className":1922},[],[1924],{"type":52,"value":1925},"nvcr.io",{"type":52,"value":1927}," images)",{"type":46,"tag":62,"props":1929,"children":1930},{},[1931,1936,1938,1943,1945,1951,1953,1958],{"type":46,"tag":66,"props":1932,"children":1933},{},[1934],{"type":52,"value":1935},"Port binding (local-docker and brev):",{"type":52,"value":1937}," use ",{"type":46,"tag":66,"props":1939,"children":1940},{},[1941],{"type":52,"value":1942},"direct docker run",{"type":52,"value":1944}," (not DockerSDK) so that ",{"type":46,"tag":87,"props":1946,"children":1948},{"className":1947},[],[1949],{"type":52,"value":1950},"-p \u003Chost_port>:8080",{"type":52,"value":1952}," can be passed and the container name equals ",{"type":46,"tag":87,"props":1954,"children":1956},{"className":1955},[],[1957],{"type":52,"value":163},{"type":52,"value":1959}," exactly.",{"type":46,"tag":62,"props":1961,"children":1962},{},[1963,1968,1970,1976,1978,1983,1985,1990,1992,1997,1999,2005,2007,2012,2014,2020,2022,2028],{"type":46,"tag":66,"props":1964,"children":1965},{},[1966],{"type":52,"value":1967},"Port allocation rule (local-docker and brev, REQUIRED for concurrent services):",{"type":52,"value":1969}," Before starting a service, read the registry (",{"type":46,"tag":87,"props":1971,"children":1973},{"className":1972},[],[1974],{"type":52,"value":1975},"\u002Ftmp\u002Ftao-inf-ms-state.json",{"type":52,"value":1977},") and collect the set of ",{"type":46,"tag":87,"props":1979,"children":1981},{"className":1980},[],[1982],{"type":52,"value":1726},{"type":52,"value":1984}," values from every existing entry on the same platform (and, for brev, the same ",{"type":46,"tag":87,"props":1986,"children":1988},{"className":1987},[],[1989],{"type":52,"value":1816},{"type":52,"value":1991},"). Pick the ",{"type":46,"tag":66,"props":1993,"children":1994},{},[1995],{"type":52,"value":1996},"lowest free port starting from 8080",{"type":52,"value":1998}," that is not in that set — e.g. ",{"type":46,"tag":87,"props":2000,"children":2002},{"className":2001},[],[2003],{"type":52,"value":2004},"host_port = next(p for p in range(8080, 8200) if p not in used_ports)",{"type":52,"value":2006},". The default ",{"type":46,"tag":87,"props":2008,"children":2010},{"className":2009},[],[2011],{"type":52,"value":1744},{"type":52,"value":2013}," only applies when no other service is running. This is what makes \"start 3 services, each reachable at a distinct ",{"type":46,"tag":87,"props":2015,"children":2017},{"className":2016},[],[2018],{"type":52,"value":2019},"host_url",{"type":52,"value":2021},"\" work; without it, services 2 and 3 fail with ",{"type":46,"tag":87,"props":2023,"children":2025},{"className":2024},[],[2026],{"type":52,"value":2027},"bind: address already in use",{"type":52,"value":2029},". SLURM and kubernetes get distinct endpoints from their own platform mechanisms and do not need this step.",{"type":46,"tag":1229,"props":2031,"children":2033},{"id":2032},"_43-after-start-service-registry-and-endpoint",[2034],{"type":52,"value":2035},"4.3 After start: service registry and endpoint",{"type":46,"tag":62,"props":2037,"children":2038},{},[2039,2041,2046,2048,2053,2055,2060],{"type":52,"value":2040},"Write the service registry immediately after the platform confirms the container is running. The registry (",{"type":46,"tag":87,"props":2042,"children":2044},{"className":2043},[],[2045],{"type":52,"value":1975},{"type":52,"value":2047},") is keyed by ",{"type":46,"tag":87,"props":2049,"children":2051},{"className":2050},[],[2052],{"type":52,"value":163},{"type":52,"value":2054},"; ",{"type":46,"tag":87,"props":2056,"children":2058},{"className":2057},[],[2059],{"type":52,"value":184},{"type":52,"value":2061}," always points to the most recently started service.",{"type":46,"tag":62,"props":2063,"children":2064},{},[2065,2067,2072,2073,2078],{"type":52,"value":2066},"See ",{"type":46,"tag":87,"props":2068,"children":2070},{"className":2069},[],[2071],{"type":52,"value":92},{"type":52,"value":94},{"type":46,"tag":87,"props":2074,"children":2076},{"className":2075},[],[2077],{"type":52,"value":132},{"type":52,"value":2079}," for the Python template.",{"type":46,"tag":512,"props":2081,"children":2082},{},[2083,2115],{"type":46,"tag":516,"props":2084,"children":2085},{},[2086],{"type":46,"tag":520,"props":2087,"children":2088},{},[2089,2093,2101,2110],{"type":46,"tag":524,"props":2090,"children":2091},{},[2092],{"type":52,"value":1033},{"type":46,"tag":524,"props":2094,"children":2095},{},[2096],{"type":46,"tag":87,"props":2097,"children":2099},{"className":2098},[],[2100],{"type":52,"value":2019},{"type":46,"tag":524,"props":2102,"children":2103},{},[2104],{"type":46,"tag":87,"props":2105,"children":2107},{"className":2106},[],[2108],{"type":52,"value":2109},"platform_job_id",{"type":46,"tag":524,"props":2111,"children":2112},{},[2113],{"type":52,"value":2114},"Extra step before writing",{"type":46,"tag":535,"props":2116,"children":2117},{},[2118,2147,2189,2223],{"type":46,"tag":520,"props":2119,"children":2120},{},[2121,2128,2137,2142],{"type":46,"tag":542,"props":2122,"children":2123},{},[2124],{"type":46,"tag":66,"props":2125,"children":2126},{},[2127],{"type":52,"value":703},{"type":46,"tag":542,"props":2129,"children":2130},{},[2131],{"type":46,"tag":87,"props":2132,"children":2134},{"className":2133},[],[2135],{"type":52,"value":2136},"http:\u002F\u002Flocalhost:{host_port}",{"type":46,"tag":542,"props":2138,"children":2139},{},[2140],{"type":52,"value":2141},"—",{"type":46,"tag":542,"props":2143,"children":2144},{},[2145],{"type":52,"value":2146},"None",{"type":46,"tag":520,"props":2148,"children":2149},{},[2150,2157,2166,2170],{"type":46,"tag":542,"props":2151,"children":2152},{},[2153],{"type":46,"tag":66,"props":2154,"children":2155},{},[2156],{"type":52,"value":710},{"type":46,"tag":542,"props":2158,"children":2159},{},[2160],{"type":46,"tag":87,"props":2161,"children":2163},{"className":2162},[],[2164],{"type":52,"value":2165},"http:\u002F\u002F{brev_ip}:{host_port}",{"type":46,"tag":542,"props":2167,"children":2168},{},[2169],{"type":52,"value":2141},{"type":46,"tag":542,"props":2171,"children":2172},{},[2173,2179,2181,2187],{"type":46,"tag":87,"props":2174,"children":2176},{"className":2175},[],[2177],{"type":52,"value":2178},"brev ls",{"type":52,"value":2180}," → get instance IP (",{"type":46,"tag":87,"props":2182,"children":2184},{"className":2183},[],[2185],{"type":52,"value":2186},"localhost",{"type":52,"value":2188}," is invalid on remote VM)",{"type":46,"tag":520,"props":2190,"children":2191},{},[2192,2199,2207,2212],{"type":46,"tag":542,"props":2193,"children":2194},{},[2195],{"type":46,"tag":66,"props":2196,"children":2197},{},[2198],{"type":52,"value":717},{"type":46,"tag":542,"props":2200,"children":2201},{},[2202],{"type":46,"tag":87,"props":2203,"children":2205},{"className":2204},[],[2206],{"type":52,"value":2136},{"type":46,"tag":542,"props":2208,"children":2209},{},[2210],{"type":52,"value":2211},"SLURM scheduler job ID",{"type":46,"tag":542,"props":2213,"children":2214},{},[2215,2217],{"type":52,"value":2216},"Wait until Running; SSH port-forward ",{"type":46,"tag":87,"props":2218,"children":2220},{"className":2219},[],[2221],{"type":52,"value":2222},"localhost:{host_port}→{node}:8080",{"type":46,"tag":520,"props":2224,"children":2225},{},[2226,2233,2242,2247],{"type":46,"tag":542,"props":2227,"children":2228},{},[2229],{"type":46,"tag":66,"props":2230,"children":2231},{},[2232],{"type":52,"value":725},{"type":46,"tag":542,"props":2234,"children":2235},{},[2236],{"type":46,"tag":87,"props":2237,"children":2239},{"className":2238},[],[2240],{"type":52,"value":2241},"http:\u002F\u002F{external_ip}:8080",{"type":46,"tag":542,"props":2243,"children":2244},{},[2245],{"type":52,"value":2246},"k8s job name",{"type":46,"tag":542,"props":2248,"children":2249},{},[2250,2256],{"type":46,"tag":87,"props":2251,"children":2253},{"className":2252},[],[2254],{"type":52,"value":2255},"kubectl expose job … --type=LoadBalancer",{"type":52,"value":2257},"; wait for external IP",{"type":46,"tag":62,"props":2259,"children":2260},{},[2261],{"type":52,"value":2262},"After writing the registry, print the job_id and URL:",{"type":46,"tag":2264,"props":2265,"children":2270},"pre",{"className":2266,"code":2267,"language":2268,"meta":2269,"style":2269},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","print(f\"Inference service started.\")\nprint(f\"  Job ID : {job_id}\")\nprint(f\"  Arch   : {network_arch}\")\nprint(f\"  URL    : {state[job_id]['host_url']}\u002Fv1\u002Fchat\u002Fcompletions\")\nprint(f\"Use this Job ID to send requests or stop the service.\")\n","python","",[2271],{"type":46,"tag":87,"props":2272,"children":2273},{"__ignoreMap":2269},[2274,2285,2294,2303,2312],{"type":46,"tag":2275,"props":2276,"children":2279},"span",{"class":2277,"line":2278},"line",1,[2280],{"type":46,"tag":2275,"props":2281,"children":2282},{},[2283],{"type":52,"value":2284},"print(f\"Inference service started.\")\n",{"type":46,"tag":2275,"props":2286,"children":2288},{"class":2277,"line":2287},2,[2289],{"type":46,"tag":2275,"props":2290,"children":2291},{},[2292],{"type":52,"value":2293},"print(f\"  Job ID : {job_id}\")\n",{"type":46,"tag":2275,"props":2295,"children":2297},{"class":2277,"line":2296},3,[2298],{"type":46,"tag":2275,"props":2299,"children":2300},{},[2301],{"type":52,"value":2302},"print(f\"  Arch   : {network_arch}\")\n",{"type":46,"tag":2275,"props":2304,"children":2306},{"class":2277,"line":2305},4,[2307],{"type":46,"tag":2275,"props":2308,"children":2309},{},[2310],{"type":52,"value":2311},"print(f\"  URL    : {state[job_id]['host_url']}\u002Fv1\u002Fchat\u002Fcompletions\")\n",{"type":46,"tag":2275,"props":2313,"children":2315},{"class":2277,"line":2314},5,[2316],{"type":46,"tag":2275,"props":2317,"children":2318},{},[2319],{"type":52,"value":2320},"print(f\"Use this Job ID to send requests or stop the service.\")\n",{"type":46,"tag":62,"props":2322,"children":2323},{},[2324,2326,2331,2332,2337],{"type":52,"value":2325},"Then poll for readiness — see ",{"type":46,"tag":87,"props":2327,"children":2329},{"className":2328},[],[2330],{"type":52,"value":92},{"type":52,"value":94},{"type":46,"tag":87,"props":2333,"children":2335},{"className":2334},[],[2336],{"type":52,"value":140},{"type":52,"value":2338},". The container loads the model in the background; do not send requests before it returns 200.",{"type":46,"tag":341,"props":2340,"children":2341},{},[],{"type":46,"tag":55,"props":2343,"children":2345},{"id":2344},"_5-stopping-the-inference-service",[2346],{"type":52,"value":2347},"5. Stopping the inference service",{"type":46,"tag":62,"props":2349,"children":2350},{},[2351,2353,2358,2360,2366,2368,2373,2374,2379,2381,2389],{"type":52,"value":2352},"Ask the user for the ",{"type":46,"tag":87,"props":2354,"children":2356},{"className":2355},[],[2357],{"type":52,"value":163},{"type":52,"value":2359}," to stop. If they don't provide one, default to ",{"type":46,"tag":87,"props":2361,"children":2363},{"className":2362},[],[2364],{"type":52,"value":2365},"state[\"latest\"]",{"type":52,"value":2367}," and confirm which job_id is being stopped. Read the registry using ",{"type":46,"tag":87,"props":2369,"children":2371},{"className":2370},[],[2372],{"type":52,"value":92},{"type":52,"value":94},{"type":46,"tag":87,"props":2375,"children":2377},{"className":2376},[],[2378],{"type":52,"value":268},{"type":52,"value":2380},", then read ",{"type":46,"tag":66,"props":2382,"children":2383},{},[2384],{"type":46,"tag":87,"props":2385,"children":2387},{"className":2386},[],[2388],{"type":52,"value":113},{"type":52,"value":2390}," and use its cancellation \u002F stop mechanism.",{"type":46,"tag":512,"props":2392,"children":2393},{},[2394,2414],{"type":46,"tag":516,"props":2395,"children":2396},{},[2397],{"type":46,"tag":520,"props":2398,"children":2399},{},[2400,2404,2409],{"type":46,"tag":524,"props":2401,"children":2402},{},[2403],{"type":52,"value":1033},{"type":46,"tag":524,"props":2405,"children":2406},{},[2407],{"type":52,"value":2408},"Identifier to pass",{"type":46,"tag":524,"props":2410,"children":2411},{},[2412],{"type":52,"value":2413},"Extra cleanup",{"type":46,"tag":535,"props":2415,"children":2416},{},[2417,2442,2465,2495],{"type":46,"tag":520,"props":2418,"children":2419},{},[2420,2427,2438],{"type":46,"tag":542,"props":2421,"children":2422},{},[2423],{"type":46,"tag":66,"props":2424,"children":2425},{},[2426],{"type":52,"value":703},{"type":46,"tag":542,"props":2428,"children":2429},{},[2430,2436],{"type":46,"tag":87,"props":2431,"children":2433},{"className":2432},[],[2434],{"type":52,"value":2435},"job_id_to_stop",{"type":52,"value":2437}," — container name",{"type":46,"tag":542,"props":2439,"children":2440},{},[2441],{"type":52,"value":2146},{"type":46,"tag":520,"props":2443,"children":2444},{},[2445,2452,2461],{"type":46,"tag":542,"props":2446,"children":2447},{},[2448],{"type":46,"tag":66,"props":2449,"children":2450},{},[2451],{"type":52,"value":710},{"type":46,"tag":542,"props":2453,"children":2454},{},[2455,2460],{"type":46,"tag":87,"props":2456,"children":2458},{"className":2457},[],[2459],{"type":52,"value":2435},{"type":52,"value":2437},{"type":46,"tag":542,"props":2462,"children":2463},{},[2464],{"type":52,"value":2146},{"type":46,"tag":520,"props":2466,"children":2467},{},[2468,2475,2486],{"type":46,"tag":542,"props":2469,"children":2470},{},[2471],{"type":46,"tag":66,"props":2472,"children":2473},{},[2474],{"type":52,"value":717},{"type":46,"tag":542,"props":2476,"children":2477},{},[2478,2484],{"type":46,"tag":87,"props":2479,"children":2481},{"className":2480},[],[2482],{"type":52,"value":2483},"entry[\"platform_job_id\"]",{"type":52,"value":2485}," — SLURM job ID",{"type":46,"tag":542,"props":2487,"children":2488},{},[2489],{"type":46,"tag":87,"props":2490,"children":2492},{"className":2491},[],[2493],{"type":52,"value":2494},"pkill -f \"ssh.*-L.*{entry['host_port']}\"",{"type":46,"tag":520,"props":2496,"children":2497},{},[2498,2505,2515],{"type":46,"tag":542,"props":2499,"children":2500},{},[2501],{"type":46,"tag":66,"props":2502,"children":2503},{},[2504],{"type":52,"value":725},{"type":46,"tag":542,"props":2506,"children":2507},{},[2508,2513],{"type":46,"tag":87,"props":2509,"children":2511},{"className":2510},[],[2512],{"type":52,"value":2483},{"type":52,"value":2514}," — k8s job name",{"type":46,"tag":542,"props":2516,"children":2517},{},[2518],{"type":46,"tag":87,"props":2519,"children":2521},{"className":2520},[],[2522],{"type":52,"value":2523},"kubectl delete svc {entry[\"platform_job_id\"]} -n \u003Cnamespace>",{"type":46,"tag":62,"props":2525,"children":2526},{},[2527,2529,2535,2537,2542,2543,2549],{"type":52,"value":2528},"where ",{"type":46,"tag":87,"props":2530,"children":2532},{"className":2531},[],[2533],{"type":52,"value":2534},"entry = state[job_id_to_stop]",{"type":52,"value":2536},". After stopping, clean up the registry: ",{"type":46,"tag":87,"props":2538,"children":2540},{"className":2539},[],[2541],{"type":52,"value":92},{"type":52,"value":94},{"type":46,"tag":87,"props":2544,"children":2546},{"className":2545},[],[2547],{"type":52,"value":2548},"stop.registry_cleanup",{"type":52,"value":102},{"type":46,"tag":341,"props":2551,"children":2552},{},[],{"type":46,"tag":55,"props":2554,"children":2556},{"id":2555},"_6-sending-inference-requests",[2557],{"type":52,"value":2558},"6. Sending inference requests",{"type":46,"tag":1229,"props":2560,"children":2562},{"id":2561},"_60-resolve-which-service-receives-this-request-required",[2563],{"type":52,"value":2564},"6.0 Resolve which service receives this request (REQUIRED)",{"type":46,"tag":62,"props":2566,"children":2567},{},[2568,2570,2575,2577,2582,2584,2589,2591,2596],{"type":52,"value":2569},"Each request must be routed to the ",{"type":46,"tag":66,"props":2571,"children":2572},{},[2573],{"type":52,"value":2574},"specific",{"type":52,"value":2576}," service that runs the matching model. Routing happens by ",{"type":46,"tag":87,"props":2578,"children":2580},{"className":2579},[],[2581],{"type":52,"value":163},{"type":52,"value":2583}," — the registry stores ",{"type":46,"tag":87,"props":2585,"children":2587},{"className":2586},[],[2588],{"type":52,"value":171},{"type":52,"value":2590}," per entry, so you can resolve a target by arch when the user names a model instead of a ",{"type":46,"tag":87,"props":2592,"children":2594},{"className":2593},[],[2595],{"type":52,"value":163},{"type":52,"value":2597},". Apply these rules in order:",{"type":46,"tag":72,"props":2599,"children":2600},{},[2601,2623,2686],{"type":46,"tag":76,"props":2602,"children":2603},{},[2604,2614,2616,2622],{"type":46,"tag":66,"props":2605,"children":2606},{},[2607,2609],{"type":52,"value":2608},"User provided an explicit ",{"type":46,"tag":87,"props":2610,"children":2612},{"className":2611},[],[2613],{"type":52,"value":163},{"type":52,"value":2615}," → use it. Verify it exists in ",{"type":46,"tag":87,"props":2617,"children":2619},{"className":2618},[],[2620],{"type":52,"value":2621},"state",{"type":52,"value":102},{"type":46,"tag":76,"props":2624,"children":2625},{},[2626,2636,2638,2644,2646],{"type":46,"tag":66,"props":2627,"children":2628},{},[2629,2631],{"type":52,"value":2630},"User named a ",{"type":46,"tag":87,"props":2632,"children":2634},{"className":2633},[],[2635],{"type":52,"value":171},{"type":52,"value":2637}," (e.g. \"send this to the cosmos-rl service\") → look up matching entries: ",{"type":46,"tag":87,"props":2639,"children":2641},{"className":2640},[],[2642],{"type":52,"value":2643},"candidates = [j for j, e in state.items() if j != \"latest\" and isinstance(e, dict) and e[\"network_arch\"] == arch]",{"type":52,"value":2645},".\n",{"type":46,"tag":289,"props":2647,"children":2648},{},[2649,2654,2681],{"type":46,"tag":76,"props":2650,"children":2651},{},[2652],{"type":52,"value":2653},"Exactly one match → use it.",{"type":46,"tag":76,"props":2655,"children":2656},{},[2657,2659,2664,2666,2671,2673,2679],{"type":52,"value":2658},"Multiple matches → ",{"type":46,"tag":66,"props":2660,"children":2661},{},[2662],{"type":52,"value":2663},"prompt the user",{"type":52,"value":2665}," with the candidate ",{"type":46,"tag":87,"props":2667,"children":2669},{"className":2668},[],[2670],{"type":52,"value":163},{"type":52,"value":2672},"s and their ",{"type":46,"tag":87,"props":2674,"children":2676},{"className":2675},[],[2677],{"type":52,"value":2678},"started_at",{"type":52,"value":2680},"; do not auto-pick.",{"type":46,"tag":76,"props":2682,"children":2683},{},[2684],{"type":52,"value":2685},"No match → stop and tell the user no service for that arch is running.",{"type":46,"tag":76,"props":2687,"children":2688},{},[2689,2706,2708,2713,2715,2720,2722],{"type":46,"tag":66,"props":2690,"children":2691},{},[2692,2694,2699,2701],{"type":52,"value":2693},"No ",{"type":46,"tag":87,"props":2695,"children":2697},{"className":2696},[],[2698],{"type":52,"value":163},{"type":52,"value":2700}," and no ",{"type":46,"tag":87,"props":2702,"children":2704},{"className":2703},[],[2705],{"type":52,"value":171},{"type":52,"value":2707}," → count non-",{"type":46,"tag":87,"props":2709,"children":2711},{"className":2710},[],[2712],{"type":52,"value":184},{"type":52,"value":2714}," entries in ",{"type":46,"tag":87,"props":2716,"children":2718},{"className":2717},[],[2719],{"type":52,"value":2621},{"type":52,"value":2721},":\n",{"type":46,"tag":289,"props":2723,"children":2724},{},[2725,2730,2773],{"type":46,"tag":76,"props":2726,"children":2727},{},[2728],{"type":52,"value":2729},"Exactly one running service → use it.",{"type":46,"tag":76,"props":2731,"children":2732},{},[2733,2735,2745,2747,2752,2753,2758,2759,2764,2766,2771],{"type":52,"value":2734},"Two or more → ",{"type":46,"tag":66,"props":2736,"children":2737},{},[2738,2740],{"type":52,"value":2739},"do not silently default to ",{"type":46,"tag":87,"props":2741,"children":2743},{"className":2742},[],[2744],{"type":52,"value":2365},{"type":52,"value":2746},". Prompt the user with the full list (",{"type":46,"tag":87,"props":2748,"children":2750},{"className":2749},[],[2751],{"type":52,"value":163},{"type":52,"value":226},{"type":46,"tag":87,"props":2754,"children":2756},{"className":2755},[],[2757],{"type":52,"value":171},{"type":52,"value":226},{"type":46,"tag":87,"props":2760,"children":2762},{"className":2761},[],[2763],{"type":52,"value":2019},{"type":52,"value":2765},") and require an explicit choice. The ",{"type":46,"tag":87,"props":2767,"children":2769},{"className":2768},[],[2770],{"type":52,"value":184},{"type":52,"value":2772}," pointer is a convenience for single-service workflows, not a routing fallback when multiple services coexist.",{"type":46,"tag":76,"props":2774,"children":2775},{},[2776],{"type":52,"value":2777},"Zero → stop and tell the user to start a service first.",{"type":46,"tag":62,"props":2779,"children":2780},{},[2781,2783,2788,2789,2794,2796,2801,2803,2809,2811,2816,2817,2822],{"type":52,"value":2782},"After resolving, read the endpoint from the registry (",{"type":46,"tag":87,"props":2784,"children":2786},{"className":2785},[],[2787],{"type":52,"value":92},{"type":52,"value":94},{"type":46,"tag":87,"props":2790,"children":2792},{"className":2791},[],[2793],{"type":52,"value":200},{"type":52,"value":2795},"), passing the resolved ",{"type":46,"tag":87,"props":2797,"children":2799},{"className":2798},[],[2800],{"type":52,"value":163},{"type":52,"value":2802}," as ",{"type":46,"tag":87,"props":2804,"children":2806},{"className":2805},[],[2807],{"type":52,"value":2808},"user_provided_job_id",{"type":52,"value":2810},". Confirm to the user: \"Sending to job_id=… arch=… url=…\". If the service may still be loading, poll readiness first (",{"type":46,"tag":87,"props":2812,"children":2814},{"className":2813},[],[2815],{"type":52,"value":92},{"type":52,"value":94},{"type":46,"tag":87,"props":2818,"children":2820},{"className":2819},[],[2821],{"type":52,"value":140},{"type":52,"value":375},{"type":46,"tag":62,"props":2824,"children":2825},{},[2826,2831,2833,2839,2841,2847,2848,2854,2855,2861,2863,2869,2870,2876,2878,2884],{"type":46,"tag":66,"props":2827,"children":2828},{},[2829],{"type":52,"value":2830},"Cross-check before sending:",{"type":52,"value":2832}," if the user-supplied request body contains arch-specific fields (e.g. ",{"type":46,"tag":87,"props":2834,"children":2836},{"className":2835},[],[2837],{"type":52,"value":2838},"guidance",{"type":52,"value":2840}," \u002F ",{"type":46,"tag":87,"props":2842,"children":2844},{"className":2843},[],[2845],{"type":52,"value":2846},"num_steps",{"type":52,"value":2840},{"type":46,"tag":87,"props":2849,"children":2851},{"className":2850},[],[2852],{"type":52,"value":2853},"seed",{"type":52,"value":2840},{"type":46,"tag":87,"props":2856,"children":2858},{"className":2857},[],[2859],{"type":52,"value":2860},"negative_prompt",{"type":52,"value":2862}," → cosmos-predict2.5; required ",{"type":46,"tag":87,"props":2864,"children":2866},{"className":2865},[],[2867],{"type":52,"value":2868},"image_url",{"type":52,"value":1496},{"type":46,"tag":87,"props":2871,"children":2873},{"className":2872},[],[2874],{"type":52,"value":2875},"video_url",{"type":52,"value":2877}," content items → cosmos-rl), verify they are consistent with ",{"type":46,"tag":87,"props":2879,"children":2881},{"className":2880},[],[2882],{"type":52,"value":2883},"state[job_id][\"network_arch\"]",{"type":52,"value":2885},". On mismatch, stop and ask — sending a cosmos-predict2.5 body to a cosmos-rl service will fail at the container with a 4xx\u002F5xx that is harder to diagnose than catching it here.",{"type":46,"tag":1229,"props":2887,"children":2889},{"id":2888},"_61-sampling-parameters-required-user-prompt-before-each-request",[2890],{"type":52,"value":2891},"6.1 Sampling parameters — REQUIRED user prompt before each request",{"type":46,"tag":62,"props":2893,"children":2894},{},[2895,2897,2902,2904,2908],{"type":52,"value":2896},"Before constructing the request body, you ",{"type":46,"tag":66,"props":2898,"children":2899},{},[2900],{"type":52,"value":2901},"MUST",{"type":52,"value":2903}," explicitly prompt the user for the vLLM-style sampling parameters. Do ",{"type":46,"tag":66,"props":2905,"children":2906},{},[2907],{"type":52,"value":988},{"type":52,"value":2909}," silently apply defaults. Use a structured prompt, one question per field, that:",{"type":46,"tag":72,"props":2911,"children":2912},{},[2913,2930,2935],{"type":46,"tag":76,"props":2914,"children":2915},{},[2916,2918,2923,2924,2929],{"type":52,"value":2917},"Lists every applicable field with its ",{"type":46,"tag":66,"props":2919,"children":2920},{},[2921],{"type":52,"value":2922},"type",{"type":52,"value":134},{"type":46,"tag":66,"props":2925,"children":2926},{},[2927],{"type":52,"value":2928},"default value",{"type":52,"value":102},{"type":46,"tag":76,"props":2931,"children":2932},{},[2933],{"type":52,"value":2934},"Lets the user skip \u002F accept any field to take that field's default — entering a value is never required.",{"type":46,"tag":76,"props":2936,"children":2937},{},[2938],{"type":52,"value":2939},"Collects all fields in one round.",{"type":46,"tag":62,"props":2941,"children":2942},{},[2943],{"type":52,"value":2944},"After the prompt, apply each user-entered value verbatim and substitute the default for any skipped field. Do not invent values or silently clamp.",{"type":46,"tag":62,"props":2946,"children":2947},{},[2948,2953,2954,2959,2960,2966,2968,2973,2974,2979,2980,2985,2987,2993,2995,3000,3001,3006,3007,3012,3013,3018,3020,3025,3027,3031,3033,3037],{"type":46,"tag":66,"props":2949,"children":2950},{},[2951],{"type":52,"value":2952},"Field list, defaults, and per-arch applicability:",{"type":52,"value":464},{"type":46,"tag":87,"props":2955,"children":2957},{"className":2956},[],[2958],{"type":52,"value":324},{"type":52,"value":94},{"type":46,"tag":87,"props":2961,"children":2963},{"className":2962},[],[2964],{"type":52,"value":2965},"chat_completions_request_body",{"type":52,"value":2967}," (base sampling fields: ",{"type":46,"tag":87,"props":2969,"children":2971},{"className":2970},[],[2972],{"type":52,"value":224},{"type":52,"value":226},{"type":46,"tag":87,"props":2975,"children":2977},{"className":2976},[],[2978],{"type":52,"value":232},{"type":52,"value":226},{"type":46,"tag":87,"props":2981,"children":2983},{"className":2982},[],[2984],{"type":52,"value":239},{"type":52,"value":2986},") and ",{"type":46,"tag":87,"props":2988,"children":2990},{"className":2989},[],[2991],{"type":52,"value":2992},"network_arch_constraints.\u003Cnetwork_arch>",{"type":52,"value":2994}," (per-arch overrides and extras such as ",{"type":46,"tag":87,"props":2996,"children":2998},{"className":2997},[],[2999],{"type":52,"value":2838},{"type":52,"value":1496},{"type":46,"tag":87,"props":3002,"children":3004},{"className":3003},[],[3005],{"type":52,"value":2846},{"type":52,"value":1496},{"type":46,"tag":87,"props":3008,"children":3010},{"className":3009},[],[3011],{"type":52,"value":2853},{"type":52,"value":1496},{"type":46,"tag":87,"props":3014,"children":3016},{"className":3015},[],[3017],{"type":52,"value":2860},{"type":52,"value":3019}," for ",{"type":46,"tag":87,"props":3021,"children":3023},{"className":3022},[],[3024],{"type":52,"value":607},{"type":52,"value":3026},"). If a field is marked unsupported for the active arch, do ",{"type":46,"tag":66,"props":3028,"children":3029},{},[3030],{"type":52,"value":988},{"type":52,"value":3032}," prompt for it and do ",{"type":46,"tag":66,"props":3034,"children":3035},{},[3036],{"type":52,"value":988},{"type":52,"value":3038}," include it in the body.",{"type":46,"tag":1229,"props":3040,"children":3042},{"id":3041},"_62-request-format",[3043],{"type":52,"value":3044},"6.2 Request format",{"type":46,"tag":62,"props":3046,"children":3047},{},[3048,3050,3056,3057,3063,3064,3070,3072,3077,3079,3084,3085,3090,3092,3098],{"type":52,"value":3049},"Send a ",{"type":46,"tag":87,"props":3051,"children":3053},{"className":3052},[],[3054],{"type":52,"value":3055},"POST",{"type":52,"value":1517},{"type":46,"tag":87,"props":3058,"children":3060},{"className":3059},[],[3061],{"type":52,"value":3062},"{BASE_URL}\u002Fv1\u002Fchat\u002Fcompletions",{"type":52,"value":1161},{"type":46,"tag":87,"props":3065,"children":3067},{"className":3066},[],[3068],{"type":52,"value":3069},"Content-Type: application\u002Fjson",{"type":52,"value":3071}," and a timeout of ",{"type":46,"tag":66,"props":3073,"children":3074},{},[3075],{"type":52,"value":3076},"at least 300 s",{"type":52,"value":3078},". The body is OpenAI-compatible (vLLM chat completions); see ",{"type":46,"tag":87,"props":3080,"children":3082},{"className":3081},[],[3083],{"type":52,"value":324},{"type":52,"value":94},{"type":46,"tag":87,"props":3086,"children":3088},{"className":3087},[],[3089],{"type":52,"value":2965},{"type":52,"value":3091}," for the full field schema and content-item shapes (text \u002F image_url \u002F video_url), and ",{"type":46,"tag":87,"props":3093,"children":3095},{"className":3094},[],[3096],{"type":52,"value":3097},"code_examples",{"type":52,"value":3099}," for ready-to-run Python and curl samples.",{"type":46,"tag":62,"props":3101,"children":3102},{},[3103,3108,3110,3115,3117,3123,3125,3130,3131,3137],{"type":46,"tag":66,"props":3104,"children":3105},{},[3106],{"type":52,"value":3107},"Constraints:",{"type":52,"value":3109}," only the first user message is processed. No secret values in request bodies. ",{"type":46,"tag":66,"props":3111,"children":3112},{},[3113],{"type":52,"value":3114},"Per-network constraints",{"type":52,"value":3116}," (e.g. cosmos-rl requires every request to include an image or video; cosmos-rl rejects ",{"type":46,"tag":87,"props":3118,"children":3120},{"className":3119},[],[3121],{"type":52,"value":3122},"data:",{"type":52,"value":3124}," URIs) are in ",{"type":46,"tag":87,"props":3126,"children":3128},{"className":3127},[],[3129],{"type":52,"value":324},{"type":52,"value":94},{"type":46,"tag":87,"props":3132,"children":3134},{"className":3133},[],[3135],{"type":52,"value":3136},"network_arch_constraints",{"type":52,"value":102},{"type":46,"tag":1229,"props":3139,"children":3141},{"id":3140},"_63-response-handling",[3142],{"type":52,"value":3143},"6.3 Response handling",{"type":46,"tag":512,"props":3145,"children":3146},{},[3147,3168],{"type":46,"tag":516,"props":3148,"children":3149},{},[3150],{"type":46,"tag":520,"props":3151,"children":3152},{},[3153,3158,3163],{"type":46,"tag":524,"props":3154,"children":3155},{},[3156],{"type":52,"value":3157},"HTTP status",{"type":46,"tag":524,"props":3159,"children":3160},{},[3161],{"type":52,"value":3162},"Meaning",{"type":46,"tag":524,"props":3164,"children":3165},{},[3166],{"type":52,"value":3167},"Action",{"type":46,"tag":535,"props":3169,"children":3170},{},[3171,3200,3221,3278,3299],{"type":46,"tag":520,"props":3172,"children":3173},{},[3174,3182,3195],{"type":46,"tag":542,"props":3175,"children":3176},{},[3177],{"type":46,"tag":66,"props":3178,"children":3179},{},[3180],{"type":52,"value":3181},"200",{"type":46,"tag":542,"props":3183,"children":3184},{},[3185,3187,3193],{"type":52,"value":3186},"Success — ",{"type":46,"tag":87,"props":3188,"children":3190},{"className":3189},[],[3191],{"type":52,"value":3192},"choices[0].message.content",{"type":52,"value":3194}," has the generated text",{"type":46,"tag":542,"props":3196,"children":3197},{},[3198],{"type":52,"value":3199},"Read result",{"type":46,"tag":520,"props":3201,"children":3202},{},[3203,3211,3216],{"type":46,"tag":542,"props":3204,"children":3205},{},[3206],{"type":46,"tag":66,"props":3207,"children":3208},{},[3209],{"type":52,"value":3210},"202",{"type":46,"tag":542,"props":3212,"children":3213},{},[3214],{"type":52,"value":3215},"Server still initializing or model still loading",{"type":46,"tag":542,"props":3217,"children":3218},{},[3219],{"type":52,"value":3220},"Retry after a delay",{"type":46,"tag":520,"props":3222,"children":3223},{},[3224,3232,3242],{"type":46,"tag":542,"props":3225,"children":3226},{},[3227],{"type":46,"tag":66,"props":3228,"children":3229},{},[3230],{"type":52,"value":3231},"503",{"type":46,"tag":542,"props":3233,"children":3234},{},[3235,3237],{"type":52,"value":3236},"Initialization failed, model load failed, ",{"type":46,"tag":66,"props":3238,"children":3239},{},[3240],{"type":52,"value":3241},"or model not yet ready",{"type":46,"tag":542,"props":3243,"children":3244},{},[3245,3247,3253,3255,3261,3263,3269,3270,3276],{"type":52,"value":3246},"Inspect ",{"type":46,"tag":87,"props":3248,"children":3250},{"className":3249},[],[3251],{"type":52,"value":3252},"error.type",{"type":52,"value":3254},": ",{"type":46,"tag":87,"props":3256,"children":3258},{"className":3257},[],[3259],{"type":52,"value":3260},"model_not_ready",{"type":52,"value":3262}," → retry; ",{"type":46,"tag":87,"props":3264,"children":3266},{"className":3265},[],[3267],{"type":52,"value":3268},"initialization_error",{"type":52,"value":2840},{"type":46,"tag":87,"props":3271,"children":3273},{"className":3272},[],[3274],{"type":52,"value":3275},"model_load_error",{"type":52,"value":3277}," → give up and check logs",{"type":46,"tag":520,"props":3279,"children":3280},{},[3281,3289,3294],{"type":46,"tag":542,"props":3282,"children":3283},{},[3284],{"type":46,"tag":66,"props":3285,"children":3286},{},[3287],{"type":52,"value":3288},"400",{"type":46,"tag":542,"props":3290,"children":3291},{},[3292],{"type":52,"value":3293},"Missing or empty JSON body",{"type":46,"tag":542,"props":3295,"children":3296},{},[3297],{"type":52,"value":3298},"Fix request",{"type":46,"tag":520,"props":3300,"children":3301},{},[3302,3310,3315],{"type":46,"tag":542,"props":3303,"children":3304},{},[3305],{"type":46,"tag":66,"props":3306,"children":3307},{},[3308],{"type":52,"value":3309},"500",{"type":46,"tag":542,"props":3311,"children":3312},{},[3313],{"type":52,"value":3314},"Unhandled exception during inference",{"type":46,"tag":542,"props":3316,"children":3317},{},[3318],{"type":52,"value":3319},"Check container logs",{"type":46,"tag":62,"props":3321,"children":3322},{},[3323,3325,3331,3333,3339,3340,3345],{"type":52,"value":3324},"For 202 and 503, the body contains ",{"type":46,"tag":87,"props":3326,"children":3328},{"className":3327},[],[3329],{"type":52,"value":3330},"{\"error\": {\"type\": \"\u003Cerror_type>\", \"message\": \"\u003Creason>\"}}",{"type":52,"value":3332},". See ",{"type":46,"tag":87,"props":3334,"children":3336},{"className":3335},[],[3337],{"type":52,"value":3338},"container_response_shapes",{"type":52,"value":587},{"type":46,"tag":87,"props":3341,"children":3343},{"className":3342},[],[3344],{"type":52,"value":324},{"type":52,"value":3346}," for error type strings.",{"type":46,"tag":3348,"props":3349,"children":3350},"style",{},[3351],{"type":52,"value":3352},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":3354,"total":3508},[3355,3373,3386,3397,3409,3423,3436,3450,3463,3474,3488,3497],{"slug":3356,"name":3356,"fn":3357,"description":3358,"org":3359,"tags":3360,"stars":3370,"repoUrl":3371,"updatedAt":3372},"nemoclaw-user-guide","retrieve NemoClaw documentation and configuration","Guides human users' AI agents to the NemoClaw docs MCP server and canonical Fern documentation in Markdown form. Use when users ask how to install, configure, operate, troubleshoot, secure, or learn NemoClaw with an AI coding assistant. Trigger keywords - nemoclaw docs, use nemoclaw with ai agent, nemoclaw mcp docs, nemoclaw install help, nemoclaw quickstart, nemoclaw markdown docs, llms.txt, agent skills.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3361,3364,3367],{"name":3362,"slug":3363,"type":15},"Documentation","documentation",{"name":3365,"slug":3366,"type":15},"MCP","mcp",{"name":3368,"slug":3369,"type":15},"Search","search",21777,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FNemoClaw","2026-07-20T06:00:01.461044",{"slug":3374,"name":3374,"fn":3375,"description":3376,"org":3377,"tags":3378,"stars":3383,"repoUrl":3384,"updatedAt":3385},"mcore-build-and-dependency","manage Megatron-LM development environments","Container-based dev environment setup and dependency management for Megatron-LM. Covers acquiring and launching the CI container, uv package management, and updating uv.lock.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3379,3380,3381],{"name":13,"slug":14,"type":15},{"name":17,"slug":18,"type":15},{"name":3382,"slug":2268,"type":15},"Python",17049,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FMegatron-LM","2026-07-27T06:06:11.249662",{"slug":3387,"name":3387,"fn":3388,"description":3389,"org":3390,"tags":3391,"stars":3383,"repoUrl":3384,"updatedAt":3396},"mcore-bump-base-image","update NVIDIA PyTorch base images","Bump the NVIDIA PyTorch base image (`nvcr.io\u002Fnvidia\u002Fpytorch:YY.MM-py3`) used by Megatron-LM CI. Covers the two pin sites (GitHub CI in `docker\u002F.ngc_version.dev` and GitLab CI in `.gitlab\u002Fstages\u002F01.build.yml`), the post-bump CI loop (re-run functional tests, refresh golden values, mark broken tests), and the gotchas that bit PRs",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3392,3395],{"name":3393,"slug":3394,"type":15},"CI\u002FCD","ci-cd",{"name":17,"slug":18,"type":15},"2026-07-14T05:25:59.97109",{"slug":3398,"name":3398,"fn":3399,"description":3400,"org":3401,"tags":3402,"stars":3383,"repoUrl":3384,"updatedAt":3408},"mcore-cicd","manage CI\u002FCD pipelines for Megatron-LM","CI\u002FCD reference for Megatron-LM. Covers CI pipeline structure, PR scope labels, triggering internal GitLab CI (which force-pushes the current branch to a pull-request\u002FBRANCH ref — always dry-run and verify the destination first; never run against shared or protected branches), and CI failure investigation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3403,3404,3405],{"name":3393,"slug":3394,"type":15},{"name":17,"slug":18,"type":15},{"name":3406,"slug":3407,"type":15},"GitHub","github","2026-07-27T06:06:12.278222",{"slug":3410,"name":3410,"fn":3411,"description":3412,"org":3413,"tags":3414,"stars":3383,"repoUrl":3384,"updatedAt":3422},"mcore-create-issue","investigate CI failures and create issues","Investigate a failing GitHub Actions run or job and create a GitHub issue for the failure.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3415,3418,3419],{"name":3416,"slug":3417,"type":15},"Debugging","debugging",{"name":3406,"slug":3407,"type":15},{"name":3420,"slug":3421,"type":15},"Triage","triage","2026-07-14T05:25:57.442089",{"slug":3424,"name":3424,"fn":3425,"description":3426,"org":3427,"tags":3428,"stars":3383,"repoUrl":3384,"updatedAt":3435},"mcore-linting-and-formatting","lint and format Megatron-LM code","Linting and formatting for Megatron-LM. Covers running autoformat.sh, tools (ruff, black, isort, pylint, mypy), and code style rules.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3429,3432],{"name":3430,"slug":3431,"type":15},"Best Practices","best-practices",{"name":3433,"slug":3434,"type":15},"Code Analysis","code-analysis","2026-07-14T05:25:56.18433",{"slug":3437,"name":3437,"fn":3438,"description":3439,"org":3440,"tags":3441,"stars":3383,"repoUrl":3384,"updatedAt":3449},"mcore-migrate-gpt-to-hybrid","migrate Megatron-LM models to HybridModel","Migration guide for moving Megatron Core GPTModel checkpoints, model providers, training commands, and layer mappings to HybridModel.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3442,3445,3448],{"name":3443,"slug":3444,"type":15},"Machine Learning","machine-learning",{"name":3446,"slug":3447,"type":15},"Migration","migration",{"name":9,"slug":8,"type":15},"2026-07-17T06:07:11.777011",{"slug":3451,"name":3451,"fn":3452,"description":3453,"org":3454,"tags":3455,"stars":3383,"repoUrl":3384,"updatedAt":3462},"mcore-onboard-gb200-1node-tests","onboard functional tests for GB200","Onboard 1-node GitHub MR functional tests for GB200 from existing mr-scoped 2-node tests.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3456,3459],{"name":3457,"slug":3458,"type":15},"QA","qa",{"name":3460,"slug":3461,"type":15},"Testing","testing","2026-07-14T05:25:53.673039",{"slug":3464,"name":3464,"fn":3465,"description":3466,"org":3467,"tags":3468,"stars":3383,"repoUrl":3384,"updatedAt":3473},"mcore-run-on-slurm","launch distributed training jobs on SLURM","How to launch distributed Megatron-LM training jobs on a SLURM cluster. Covers a minimal sbatch skeleton, environment-variable setup for torch.distributed.run, CUDA_DEVICE_MAX_CONNECTIONS rules across hardware and parallelism modes, container conventions, monitoring, and per-rank failure diagnosis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3469,3470],{"name":17,"slug":18,"type":15},{"name":3471,"slug":3472,"type":15},"Infrastructure","infrastructure","2026-07-14T05:25:49.362534",{"slug":3475,"name":3475,"fn":3476,"description":3477,"org":3478,"tags":3479,"stars":3383,"repoUrl":3384,"updatedAt":3487},"mcore-split-pr","split pull requests to reduce review load","Split a PR into multiple PRs to reduce the number of required CODEOWNERS reviewer groups.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3480,3483,3484],{"name":3481,"slug":3482,"type":15},"Code Review","code-review",{"name":3406,"slug":3407,"type":15},{"name":3485,"slug":3486,"type":15},"Pull Requests","pull-requests","2026-07-14T05:26:01.226578",{"slug":3489,"name":3489,"fn":3490,"description":3491,"org":3492,"tags":3493,"stars":3383,"repoUrl":3384,"updatedAt":3496},"mcore-testing","run and manage Megatron-LM tests","Test system for Megatron-LM. Covers test layout, recipe YAML structure, adding and running unit and functional tests, golden values, marker filters, and CI parity.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3494,3495],{"name":3457,"slug":3458,"type":15},{"name":3460,"slug":3461,"type":15},"2026-07-14T05:25:54.928983",{"slug":3498,"name":3498,"fn":3499,"description":3500,"org":3501,"tags":3502,"stars":3383,"repoUrl":3384,"updatedAt":3507},"nightly-sync","manage nightly main-to-dev sync workflows","Domain knowledge for the nightly main-to-dev sync workflow. Covers merge strategy, CI architecture, failure investigation, and known issues.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3503,3506],{"name":3504,"slug":3505,"type":15},"Automation","automation",{"name":3393,"slug":3394,"type":15},"2026-07-30T05:29:03.275638",496,{"items":3510,"total":3606},[3511,3528,3538,3552,3562,3577,3592],{"slug":3512,"name":3512,"fn":3513,"description":3514,"org":3515,"tags":3516,"stars":20,"repoUrl":21,"updatedAt":3527},"accelerated-computing-cudf","accelerate data processing with cuDF","Official NVIDIA-authored guidance for NVIDIA cuDF GPU DataFrames, pandas acceleration, dask-cuDF, ETL, joins, groupby, CSV\u002FParquet I\u002FO, nullable semantics, and multi-GPU DataFrame workloads.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3517,3520,3523,3524],{"name":3518,"slug":3519,"type":15},"Data Analysis","data-analysis",{"name":3521,"slug":3522,"type":15},"Data Engineering","data-engineering",{"name":9,"slug":8,"type":15},{"name":3525,"slug":3526,"type":15},"Performance","performance","2026-07-14T05:28:43.176466",{"slug":3529,"name":3529,"fn":3530,"description":3531,"org":3532,"tags":3533,"stars":20,"repoUrl":21,"updatedAt":3537},"aiq-deploy","deploy and manage NVIDIA AI-Q infrastructure","Use when asked to install, deploy, run, validate, troubleshoot, or stop NVIDIA AI-Q Blueprint infrastructure.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3534,3535,3536],{"name":17,"slug":18,"type":15},{"name":3471,"slug":3472,"type":15},{"name":9,"slug":8,"type":15},"2026-07-14T05:29:06.667109",{"slug":3539,"name":3539,"fn":3540,"description":3541,"org":3542,"tags":3543,"stars":20,"repoUrl":21,"updatedAt":3551},"aiq-research","conduct deep research with AI-Q","Use when asked to run deep research or AI-Q research through a reachable NVIDIA AI-Q Blueprint backend.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3544,3547,3548],{"name":3545,"slug":3546,"type":15},"Agents","agents",{"name":9,"slug":8,"type":15},{"name":3549,"slug":3550,"type":15},"Research","research","2026-07-14T05:28:06.816956",{"slug":3553,"name":3553,"fn":3554,"description":3555,"org":3556,"tags":3557,"stars":20,"repoUrl":21,"updatedAt":3561},"amc-run-sample-calibration","run AMC sample dataset calibration","Run end-to-end calibration on the shipped sample dataset (sdg_08_2_sample_data_010926.zip) against a running AMC microservice. Use when user says 'test sample dataset', 'run sample calibration', 'verify AMC install', or 'launch and test'.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3558,3559,3560],{"name":3518,"slug":3519,"type":15},{"name":9,"slug":8,"type":15},{"name":3460,"slug":3461,"type":15},"2026-07-17T05:29:03.913266",{"slug":3563,"name":3563,"fn":3564,"description":3565,"org":3566,"tags":3567,"stars":20,"repoUrl":21,"updatedAt":3576},"amc-run-video-calibration","calibrate video datasets with AutoMagicCalib","Calibrate a new dataset from pre-recorded video files via the AutoMagicCalib REST API. Use when user has local MP4s and says 'calibrate my videos', 'run AMC on these videos', or similar. For RTSP\u002Flive streams, use amc-run-rtsp-calibration instead.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3568,3569,3572,3573],{"name":3504,"slug":3505,"type":15},{"name":3570,"slug":3571,"type":15},"Imaging","imaging",{"name":9,"slug":8,"type":15},{"name":3574,"slug":3575,"type":15},"Video","video","2026-07-17T05:28:53.905004",{"slug":3578,"name":3578,"fn":3579,"description":3580,"org":3581,"tags":3582,"stars":20,"repoUrl":21,"updatedAt":3591},"amc-setup-calibration-stack","deploy AutoMagicCalib microservice with Docker","Launch AutoMagicCalib microservice and web UI from NGC release images via Docker Compose. Use when user says 'deploy auto calibration', 'launch auto calibration', 'launch AMC', 'start MS+UI', or 'set up auto-magic-calib'. Requires NGC API key.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3583,3584,3587,3588],{"name":17,"slug":18,"type":15},{"name":3585,"slug":3586,"type":15},"Docker","docker",{"name":9,"slug":8,"type":15},{"name":3589,"slug":3590,"type":15},"Operations","operations","2026-07-17T05:28:56.913999",{"slug":3593,"name":3593,"fn":3594,"description":3595,"org":3596,"tags":3597,"stars":20,"repoUrl":21,"updatedAt":3605},"cudaq-guide","develop quantum applications with CUDA-Q","CUDA-Q onboarding guide for installation, test programs, GPU simulation, QPU hardware, and quantum applications.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3598,3599,3602],{"name":9,"slug":8,"type":15},{"name":3600,"slug":3601,"type":15},"Quantum Computing","quantum-computing",{"name":3603,"slug":3604,"type":15},"Simulation","simulation","2026-07-14T05:26:58.898253",305]