[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-nvidia-tao-finetune-clip":3,"mdc--r3933n-key":34,"related-repo-nvidia-tao-finetune-clip":2175,"related-org-nvidia-tao-finetune-clip":2279},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":23,"repoUrl":24,"updatedAt":25,"license":26,"forks":27,"topics":28,"repo":29,"sourceUrl":32,"mdContent":33},"tao-finetune-clip","fine-tune CLIP vision-language models","CLIP vision-language model for image-text retrieval, zero-shot classification, embedding extraction, ONNX export, and TensorRT deployment. Use when fine-tuning or training CLIP, running zero-shot classification, computing image embeddings, or deploying CLIP to ONNX\u002FTensorRT.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},"nvidia","NVIDIA","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fnvidia.png",[12,16,19,20],{"name":13,"slug":14,"type":15},"Deep Learning","deep-learning","tag",{"name":17,"slug":18,"type":15},"Multimodal","multimodal",{"name":9,"slug":8,"type":15},{"name":21,"slug":22,"type":15},"Computer Vision","computer-vision",2473,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fskills","2026-07-14T05:29:38.186491","Apache-2.0",281,[],{"repoUrl":24,"stars":23,"forks":27,"topics":30,"description":31},[],"AI agent skills published by NVIDIA","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fskills\u002Ftree\u002FHEAD\u002Fskills\u002Ftao-finetune-clip","---\nname: tao-finetune-clip\ndescription: CLIP vision-language model for image-text retrieval, zero-shot classification, embedding extraction, ONNX\n  export, and TensorRT deployment. Use when fine-tuning or training CLIP, running zero-shot classification, computing image\n  embeddings, or deploying CLIP to ONNX\u002FTensorRT.\nlicense: Apache-2.0\ncompatibility: Requires docker + nvidia-container-toolkit.\nmetadata:\n  author: NVIDIA Corporation\n  version: \"0.1.0\"\nallowed-tools: Read Bash\ntags:\n- vision-language\n- classification\n- embedding\n- zero-shot\n- deployment\n---\n\n# CLIP\n\nContrastive Language-Image Pre-training model for zero-shot and fine-tuned image classification, image-text retrieval, and embedding extraction. Fine-tuning adapts CLIP's shared image-text embedding space to domain-specific image-caption data.\n\nNo default NGC pretrained checkpoint is required for spec construction, but unset checkpoint behavior is action-specific. In the validation-fixes PyTorch image, `export.checkpoint: null` exports the selected CLIP architecture and may initialize weights when pretrained weights are unavailable. Do not assume `inference.checkpoint: null` loads pretrained weights: `clip inference` currently calls the checkpoint loader with `None` and fails before embedding extraction. For PyTorch inference, checkpoint-backed evaluation\u002Fexport, resume, and retrain flows, resolve and pass an exact checkpoint from the parent train output. For trusted TAO checkpoints produced by the current run or a known parent job, set `TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1` on checkpoint-dependent PyTorch actions so PyTorch 2.6 can load the Lightning checkpoint metadata; do not set this for untrusted checkpoints.\n\nSupported actions: `train`, `evaluate`, `inference`, `export`, `gen_trt_engine`.\n\n## Train Action Policy\n\nThis model is AutoML-enabled at the model layer. Before handling any train-stage request, read `references\u002Fskill_info.yaml` and resolve the run override from either an explicit `automl_policy` value or the user's workflow request. Use `automl_policy: on` by default and only expose `on` \u002F `off` in new launch prompts. Treat phrases like \"turn off AutoML\", \"disable AutoML\", \"no HPO\", or \"plain training\" as `automl_policy: off` for this run only. When `automl_policy: on`, `automl_enabled: true`, and both `schemas\u002Ftrain.schema.json` and `references\u002Fspec_template_train.yaml` are packaged, route the train action through `tao-skill-bank:tao-run-automl` by default with this model's `skill_dir`. Preserve workflow\u002Fapplication overrides for datasets, specs, output directories, GPU\u002Fplatform settings, parent checkpoints, and `automl_policy`. Use direct model training only when `automl_policy: off` or the packaged train schema\u002Ftemplate is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.\n\nThe packaged CLIP train schema enables `train.optim.vision_lr` and `train.optim.text_lr` as default AutoML search parameters. For smoke tests, keep the search small by using the Bayesian algorithm with two recommendations and narrow LR ranges.\n\nNon-train actions such as `evaluate`, `inference`, `export`, and deploy flows stay in this model skill. The per-run `automl_policy` override does not change model metadata.\n\n## Instructions\n\nUse this skill for NVIDIA TAO CLIP jobs: training, evaluation, embedding inference, ONNX export, and TensorRT engine generation. Start by identifying the requested action, then load only the referenced files needed for that action: `defaults.json` for default parameters, `config.json` for action\u002Fdata-source wiring, `references\u002Fspec_template.yaml` for full spec shape, and `references\u002Fmodel_info.yaml` for SDK metadata.\n\nFor dataset-backed actions, collect the required image, caption, list, or prompt files from the user and place the resolved paths in `spec_overrides`. For local Docker runs, mount extracted folders in the container and point `image_dir` \u002F `caption_dir` at those folders; if a data source provides `.tar.gz` archives, extract them before running the in-container CLIP commands. For `export` and `gen_trt_engine`, infer parent artifacts from the upstream job when available; otherwise require explicit checkpoint, ONNX, or engine paths. Run `gen_trt_engine`, TensorRT `evaluate`, and TensorRT `inference` in the TAO Deploy image.\n\nFor TAO Deploy TensorRT actions (`gen_trt_engine`, TensorRT `evaluate`, and TensorRT `inference`), read `references\u002Ftao-deploy-clip.md` first. Deploy spec templates live in this skill's `references\u002F` folder with the `spec_template_deploy_*.yaml` prefix.\n\n## Training Requirements\n\n- **Dataset type:** image_text\n- **Formats:** custom image\u002Fcaption folders or WebDataset shards\n- **Monitoring metric:** val\u002Ft2i_mAP\n\n### Supported Models\n\n- **OpenCLIP \u002F NV-CLIP:** `ViT-L-14-SigLIP-CLIPA-224` (default), `ViT-L-14-SigLIP-CLIPA-336`, `ViT-H-14-SigLIP-CLIPA-224`, `ViT-H-14-SigLIP-CLIPA-336`, `ViT-H-14-SigLIP-CLIPA-574`\n- **Radio-CLIP:** `c-radio_v3-b`, `c-radio_v3-l`, `c-radio_v3-h`, `c-radio_v3-g`\n- **SigLIP2:** `siglip2-so400m-patch16-256`, `siglip2-so400m-patch14-224`, `siglip2-so400m-patch14-384`, `siglip2-so400m-patch16-384`, `siglip2-so400m-patch16-512`, `siglip2-so400m-patch16-naflex`\n\nRadio-CLIP requires `model.adaptor_name` to be set to `siglip` or `clip`.\n\n### Per-Action Dataset Requirements\n\n| Action | Spec Key | Source | Files | List? |\n|---|---|---|---|---|\n| train | dataset.train.datasets | train_datasets | image_dir: images.tar.gz, image_list_file: image_list.txt, caption_dir: captions.tar.gz | Yes |\n| train | dataset.train.wds.root_dir | train_wds_dataset | root directory containing `.tar` shards | No |\n| train | dataset.train.wds.shard_list_file | train_wds_dataset | shards.txt listing shard paths | No |\n| train | dataset.val.datasets | eval_dataset | image_dir: images.tar.gz, image_list_file: image_list.txt, caption_dir: captions.tar.gz | Yes |\n| evaluate | dataset.val.datasets | eval_dataset | image_dir: images.tar.gz, image_list_file: image_list.txt, caption_dir: captions.tar.gz | Yes |\n| inference | inference.datasets | inference_dataset | image_dir: images.tar.gz | Yes |\n| inference | inference.text_file | inference_dataset | prompts.txt | No |\n| export | export.checkpoint | parent train job or explicit checkpoint | checkpoint .pth, optional for pretrained export | No |\n| gen_trt_engine | gen_trt_engine.onnx_file | parent export job or explicit ONNX | clip_model.onnx | No |\n\nFor custom training, set `dataset.train.type: custom` and provide `dataset.train.datasets` entries. Image and caption files must share the same base name. `caption_file_suffix` defaults to `.txt`, and `image_list_file` is optional.\n\nWhen no native CLIP image-caption dataset is available, do not silently treat image-classification data as CLIP data. If the user explicitly allows a plumbing-only validation fallback, derive caption files from class labels, document that the captions are generated from labels, and keep each image\u002Fcaption pair on the same base filename. Without an `image_list_file`, the TAO custom loader scans the configured image directory for image files; keep validation folders flat unless you provide a list file.\n\nFor WDS training, set `dataset.train.type: wds` and provide at least one of `dataset.train.wds.root_dir` or `dataset.train.wds.shard_list_file`. `root_dir` is scanned recursively for `.tar` shards. `shard_list_file` is a text file with one shard path per line; relative lines resolve under the list-file directory unless `root_dir` is also supplied, in which case they resolve under `root_dir`. Validation\u002Fevaluation data remains custom format via `dataset.val.datasets`.\n\n### Typical Spec Overrides\n\nData source overrides are mandatory for dataset-backed actions. Construct paths from the Per-Action Dataset Requirements table and include them in `spec_overrides`. For inference, provide at least one of `inference.datasets` or `inference.text_file`.\n\n```python\nS3_TRAIN = \"s3:\u002F\u002Fbucket\u002Fdata\u002Ftrain\"\nS3_WDS = \"s3:\u002F\u002Fbucket\u002Fdata\u002Fwds\"\nS3_EVAL = \"s3:\u002F\u002Fbucket\u002Fdata\u002Feval\"\nS3_INFER = \"s3:\u002F\u002Fbucket\u002Fdata\u002Finfer\"\n```\n\n**train, custom dataset:**\n```python\n{\n    \"train.num_epochs\": 10,\n    \"dataset.train.type\": \"custom\",\n    \"dataset.train.datasets\": [{\"image_dir\": f\"{S3_TRAIN}\u002Fimages.tar.gz\", \"image_list_file\": f\"{S3_TRAIN}\u002Fimage_list.txt\", \"caption_dir\": f\"{S3_TRAIN}\u002Fcaptions.tar.gz\"}],\n    \"dataset.val.datasets\": [{\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"image_list_file\": f\"{S3_EVAL}\u002Fimage_list.txt\", \"caption_dir\": f\"{S3_EVAL}\u002Fcaptions.tar.gz\"}],\n}\n```\n\n**train, WDS dataset:**\n```python\n{\n    \"train.num_epochs\": 10,\n    \"dataset.train.type\": \"wds\",\n    \"dataset.train.wds.root_dir\": f\"{S3_WDS}\",\n    \"dataset.train.wds.shard_list_file\": f\"{S3_WDS}\u002Fshards.txt\",\n    \"dataset.train.wds.samples_per_shard\": 10000,\n    \"dataset.val.datasets\": [{\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"image_list_file\": f\"{S3_EVAL}\u002Fimage_list.txt\", \"caption_dir\": f\"{S3_EVAL}\u002Fcaptions.tar.gz\"}],\n}\n```\n\n**evaluate:**\n```python\n{\n    \"dataset.val.datasets\": [{\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"image_list_file\": f\"{S3_EVAL}\u002Fimage_list.txt\", \"caption_dir\": f\"{S3_EVAL}\u002Fcaptions.tar.gz\"}],\n}\n```\n\nLeave `evaluate.checkpoint` unset for zero-shot evaluation with pretrained weights. Set `evaluate.trt_engine` instead of `evaluate.checkpoint` for TensorRT evaluation.\n\n**inference:**\n```python\n{\n    \"inference.datasets\": [{\"image_dir\": f\"{S3_INFER}\u002Fimages.tar.gz\"}],\n    \"inference.text_file\": f\"{S3_INFER}\u002Fprompts.txt\",\n}\n```\n\nInference writes `image_embeddings.h5` and\u002For `text_embeddings.h5` under `results_dir`. The saved embeddings are L2-normalized.\n\n**export:**\n```python\n{\n    \"export.onnx_file\": \"${results_dir}\u002Fexport\u002Fclip_model.onnx\",\n    \"export.encoder_type\": \"combined\",\n    \"export.batch_size\": -1,\n}\n```\n\nSet `export.encoder_type: separate` when deployment should use independent vision and text encoders. Separate export writes `_vision.onnx` and `_text.onnx` variants derived from the base `export.onnx_file`.\n\nFor checkpoint-dependent actions, use the model-specific checkpoint resolver output from the parent train job. CLIP training writes checkpoints such as `model_epoch_000_step_00020.pth` and a `clip_latest.pth` symlink. Use the exact resolved checkpoint for `evaluate.checkpoint`, `inference.checkpoint`, `export.checkpoint`, and `train.resume_training_checkpoint_path`; use `clip_latest.pth` only when the user explicitly asks for latest.\n\nWhen the resolved checkpoint is trusted TAO output, checkpoint-backed PyTorch `evaluate`, `inference`, `export`, and resume training should run with `TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1`. PyTorch 2.6 otherwise defaults checkpoint loading to weights-only mode and can reject CLIP Lightning checkpoints containing NumPy scalar metadata.\n\n**gen_trt_engine:**\n```python\n{\n    \"gen_trt_engine.onnx_file\": \"${results_dir}\u002Fexport\u002Fclip_model.onnx\",\n    \"gen_trt_engine.trt_engine\": \"${results_dir}\u002Fdeploy\u002Fclip_model.engine\",\n    \"gen_trt_engine.batch_size\": -1,\n    \"gen_trt_engine.tensorrt.data_type\": \"fp16\",\n    \"gen_trt_engine.tensorrt.min_batch_size\": 1,\n    \"gen_trt_engine.tensorrt.opt_batch_size\": 1,\n    \"gen_trt_engine.tensorrt.max_batch_size\": 16,\n}\n```\n\n## Eval Dataset\n\nOptional for training. If provided, validation metrics are computed at validation intervals. Required for `evaluate`.\n\n## Deploy Workflow\n\nThe skill exposes `gen_trt_engine` as the deploy action. In generated SDK runners, use `model_info[\"actions\"][\"gen_trt_engine\"]` and run it in the TAO Deploy image, not the PyTorch training image. The in-container command is `clip gen_trt_engine -e {config_path}`; direct TAO Launcher usage spells the same action as `tao deploy clip gen_trt_engine -e \u002Fpath\u002Fto\u002Fspec.yaml`.\n\nTAO Deploy inference can discover combined engines, paired separate engines, or single-pillar `_vision.engine` \u002F `_text.engine` files. For full TensorRT retrieval evaluation or image+text TensorRT inference, export with `export.encoder_type: separate` and run `clip gen_trt_engine` twice: build `clip_model_vision.onnx` to an engine ending in `_vision.engine`, then build `clip_model_text.onnx` to the matching `_text.engine` in the same directory. For image-only TensorRT inference, building only the `_vision.engine` is sufficient and `inference.text_file` must be `null`. TensorRT `evaluate` and text inference require a text-capable engine; if only a vision engine is present, deploy evaluation fails because text embeddings cannot be extracted.\n\nUse `evaluate.trt_engine` for TensorRT evaluation and `inference.trt_engine` for TensorRT embedding extraction. These TensorRT paths also run in the TAO Deploy image. Direct TAO Launcher usage spells these as `tao deploy clip evaluate` and `tao deploy clip inference`.\n\n## Important Parameters\n\n- **model.type**: Backbone family and resolution. Use a TAO-registered CLIP model ID such as `ViT-L-14-SigLIP-CLIPA-224`. Prefer the listed OpenCLIP \u002F NV-CLIP IDs for AutoML smoke tests because the current TAO container registry routes them through the supported augmentation adapter.\n- **model.adaptor_name**: Required for Radio-CLIP. Set to `siglip` or `clip`.\n- **model.image_size**: Training transform image resolution. Keep it aligned with the selected fixed-resolution backbone.\n- **train.num_epochs**: CLIP fine-tuning often converges quickly. Start with 10-20 epochs for domain adaptation, then increase only if validation loss is still improving.\n- **train.optim.vision_lr \u002F train.optim.text_lr**: Learning rates for the two encoders. CLIP is sensitive to high learning rates; reduce both if loss is unstable.\n- **model.freeze_vision_encoder \u002F model.freeze_text_encoder**: Defaults are false. Freezing one encoder can help when the dataset is small or only one modality needs adaptation.\n- **train.loss_type**: `siglip` is recommended for SigLIP2 and Radio-CLIP. Use `clip` for CLIP-style softmax loss.\n- **export.encoder_type**: `combined` exports one ONNX graph. `separate` exports independent vision and text graphs.\n- **gen_trt_engine.tensorrt.data_type**: TensorRT deployment supports `fp16` and `fp32`.\n\n## Hardware\n\nSingle-GPU training works for small datasets. Use 4+ GPUs for datasets with more than 100k images or large backbones. Use 16GB+ VRAM per GPU for small\u002Ffixed-resolution runs and larger GPUs for Radio-CLIP or high-resolution OpenCLIP variants.\n\n## Error Patterns\n\nSee `references\u002Ferror-patterns.md` for the full list of CLIP error symptoms and fixes (CUDA OOM, NaN loss, retrieval quality, dataset format\u002Fsize, Radio-CLIP and model-ID validation, ONNX external data, TensorRT shape mismatch, PyTorch 2.6 checkpoint load, null-checkpoint inference, TensorRT text\u002Fretrieval failures, `attention_mask` handling, and spec\u002Fschema merge errors).\n\n## Spec Param \u002F Parent Model Inference\n\nSee `references\u002Fspec-param-inference.md` for the model-specific inference mappings (the full `clip.config.json` action\u002Fspec-field\u002Finference-function table) that generated runners apply with SDK helpers before `create_job()`, plus `parent_job_id` resolution rules.\n\n## Deployment\n\n- [tao-deploy-clip](references\u002Ftao-deploy-clip.md)\n",{"data":35,"body":47},{"name":4,"description":6,"license":26,"compatibility":36,"metadata":37,"allowed-tools":40,"tags":41},"Requires docker + nvidia-container-toolkit.",{"author":38,"version":39},"NVIDIA Corporation","0.1.0","Read Bash",[42,43,44,45,46],"vision-language","classification","embedding","zero-shot","deployment",{"type":48,"children":49},"root",[50,59,65,111,153,160,273,293,324,330,367,437,485,491,527,534,668,695,701,982,1026,1038,1107,1113,1137,1186,1194,1251,1259,1328,1336,1364,1392,1400,1437,1466,1474,1519,1554,1609,1640,1648,1726,1732,1743,1749,1784,1878,1912,1918,2072,2078,2083,2089,2110,2116,2152,2157,2169],{"type":51,"tag":52,"props":53,"children":55},"element","h1",{"id":54},"clip",[56],{"type":57,"value":58},"text","CLIP",{"type":51,"tag":60,"props":61,"children":62},"p",{},[63],{"type":57,"value":64},"Contrastive Language-Image Pre-training model for zero-shot and fine-tuned image classification, image-text retrieval, and embedding extraction. Fine-tuning adapts CLIP's shared image-text embedding space to domain-specific image-caption data.",{"type":51,"tag":60,"props":66,"children":67},{},[68,70,77,79,85,87,93,95,101,103,109],{"type":57,"value":69},"No default NGC pretrained checkpoint is required for spec construction, but unset checkpoint behavior is action-specific. In the validation-fixes PyTorch image, ",{"type":51,"tag":71,"props":72,"children":74},"code",{"className":73},[],[75],{"type":57,"value":76},"export.checkpoint: null",{"type":57,"value":78}," exports the selected CLIP architecture and may initialize weights when pretrained weights are unavailable. Do not assume ",{"type":51,"tag":71,"props":80,"children":82},{"className":81},[],[83],{"type":57,"value":84},"inference.checkpoint: null",{"type":57,"value":86}," loads pretrained weights: ",{"type":51,"tag":71,"props":88,"children":90},{"className":89},[],[91],{"type":57,"value":92},"clip inference",{"type":57,"value":94}," currently calls the checkpoint loader with ",{"type":51,"tag":71,"props":96,"children":98},{"className":97},[],[99],{"type":57,"value":100},"None",{"type":57,"value":102}," and fails before embedding extraction. For PyTorch inference, checkpoint-backed evaluation\u002Fexport, resume, and retrain flows, resolve and pass an exact checkpoint from the parent train output. For trusted TAO checkpoints produced by the current run or a known parent job, set ",{"type":51,"tag":71,"props":104,"children":106},{"className":105},[],[107],{"type":57,"value":108},"TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1",{"type":57,"value":110}," on checkpoint-dependent PyTorch actions so PyTorch 2.6 can load the Lightning checkpoint metadata; do not set this for untrusted checkpoints.",{"type":51,"tag":60,"props":112,"children":113},{},[114,116,122,124,130,131,137,138,144,145,151],{"type":57,"value":115},"Supported actions: ",{"type":51,"tag":71,"props":117,"children":119},{"className":118},[],[120],{"type":57,"value":121},"train",{"type":57,"value":123},", ",{"type":51,"tag":71,"props":125,"children":127},{"className":126},[],[128],{"type":57,"value":129},"evaluate",{"type":57,"value":123},{"type":51,"tag":71,"props":132,"children":134},{"className":133},[],[135],{"type":57,"value":136},"inference",{"type":57,"value":123},{"type":51,"tag":71,"props":139,"children":141},{"className":140},[],[142],{"type":57,"value":143},"export",{"type":57,"value":123},{"type":51,"tag":71,"props":146,"children":148},{"className":147},[],[149],{"type":57,"value":150},"gen_trt_engine",{"type":57,"value":152},".",{"type":51,"tag":154,"props":155,"children":157},"h2",{"id":156},"train-action-policy",[158],{"type":57,"value":159},"Train Action Policy",{"type":51,"tag":60,"props":161,"children":162},{},[163,165,171,173,179,181,187,189,195,197,203,205,211,213,218,219,225,227,233,235,241,243,249,251,257,259,264,266,271],{"type":57,"value":164},"This model is AutoML-enabled at the model layer. Before handling any train-stage request, read ",{"type":51,"tag":71,"props":166,"children":168},{"className":167},[],[169],{"type":57,"value":170},"references\u002Fskill_info.yaml",{"type":57,"value":172}," and resolve the run override from either an explicit ",{"type":51,"tag":71,"props":174,"children":176},{"className":175},[],[177],{"type":57,"value":178},"automl_policy",{"type":57,"value":180}," value or the user's workflow request. Use ",{"type":51,"tag":71,"props":182,"children":184},{"className":183},[],[185],{"type":57,"value":186},"automl_policy: on",{"type":57,"value":188}," by default and only expose ",{"type":51,"tag":71,"props":190,"children":192},{"className":191},[],[193],{"type":57,"value":194},"on",{"type":57,"value":196}," \u002F ",{"type":51,"tag":71,"props":198,"children":200},{"className":199},[],[201],{"type":57,"value":202},"off",{"type":57,"value":204}," in new launch prompts. Treat phrases like \"turn off AutoML\", \"disable AutoML\", \"no HPO\", or \"plain training\" as ",{"type":51,"tag":71,"props":206,"children":208},{"className":207},[],[209],{"type":57,"value":210},"automl_policy: off",{"type":57,"value":212}," for this run only. When ",{"type":51,"tag":71,"props":214,"children":216},{"className":215},[],[217],{"type":57,"value":186},{"type":57,"value":123},{"type":51,"tag":71,"props":220,"children":222},{"className":221},[],[223],{"type":57,"value":224},"automl_enabled: true",{"type":57,"value":226},", and both ",{"type":51,"tag":71,"props":228,"children":230},{"className":229},[],[231],{"type":57,"value":232},"schemas\u002Ftrain.schema.json",{"type":57,"value":234}," and ",{"type":51,"tag":71,"props":236,"children":238},{"className":237},[],[239],{"type":57,"value":240},"references\u002Fspec_template_train.yaml",{"type":57,"value":242}," are packaged, route the train action through ",{"type":51,"tag":71,"props":244,"children":246},{"className":245},[],[247],{"type":57,"value":248},"tao-skill-bank:tao-run-automl",{"type":57,"value":250}," by default with this model's ",{"type":51,"tag":71,"props":252,"children":254},{"className":253},[],[255],{"type":57,"value":256},"skill_dir",{"type":57,"value":258},". Preserve workflow\u002Fapplication overrides for datasets, specs, output directories, GPU\u002Fplatform settings, parent checkpoints, and ",{"type":51,"tag":71,"props":260,"children":262},{"className":261},[],[263],{"type":57,"value":178},{"type":57,"value":265},". Use direct model training only when ",{"type":51,"tag":71,"props":267,"children":269},{"className":268},[],[270],{"type":57,"value":210},{"type":57,"value":272}," or the packaged train schema\u002Ftemplate is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.",{"type":51,"tag":60,"props":274,"children":275},{},[276,278,284,285,291],{"type":57,"value":277},"The packaged CLIP train schema enables ",{"type":51,"tag":71,"props":279,"children":281},{"className":280},[],[282],{"type":57,"value":283},"train.optim.vision_lr",{"type":57,"value":234},{"type":51,"tag":71,"props":286,"children":288},{"className":287},[],[289],{"type":57,"value":290},"train.optim.text_lr",{"type":57,"value":292}," as default AutoML search parameters. For smoke tests, keep the search small by using the Bayesian algorithm with two recommendations and narrow LR ranges.",{"type":51,"tag":60,"props":294,"children":295},{},[296,298,303,304,309,310,315,317,322],{"type":57,"value":297},"Non-train actions such as ",{"type":51,"tag":71,"props":299,"children":301},{"className":300},[],[302],{"type":57,"value":129},{"type":57,"value":123},{"type":51,"tag":71,"props":305,"children":307},{"className":306},[],[308],{"type":57,"value":136},{"type":57,"value":123},{"type":51,"tag":71,"props":311,"children":313},{"className":312},[],[314],{"type":57,"value":143},{"type":57,"value":316},", and deploy flows stay in this model skill. The per-run ",{"type":51,"tag":71,"props":318,"children":320},{"className":319},[],[321],{"type":57,"value":178},{"type":57,"value":323}," override does not change model metadata.",{"type":51,"tag":154,"props":325,"children":327},{"id":326},"instructions",[328],{"type":57,"value":329},"Instructions",{"type":51,"tag":60,"props":331,"children":332},{},[333,335,341,343,349,351,357,359,365],{"type":57,"value":334},"Use this skill for NVIDIA TAO CLIP jobs: training, evaluation, embedding inference, ONNX export, and TensorRT engine generation. Start by identifying the requested action, then load only the referenced files needed for that action: ",{"type":51,"tag":71,"props":336,"children":338},{"className":337},[],[339],{"type":57,"value":340},"defaults.json",{"type":57,"value":342}," for default parameters, ",{"type":51,"tag":71,"props":344,"children":346},{"className":345},[],[347],{"type":57,"value":348},"config.json",{"type":57,"value":350}," for action\u002Fdata-source wiring, ",{"type":51,"tag":71,"props":352,"children":354},{"className":353},[],[355],{"type":57,"value":356},"references\u002Fspec_template.yaml",{"type":57,"value":358}," for full spec shape, and ",{"type":51,"tag":71,"props":360,"children":362},{"className":361},[],[363],{"type":57,"value":364},"references\u002Fmodel_info.yaml",{"type":57,"value":366}," for SDK metadata.",{"type":51,"tag":60,"props":368,"children":369},{},[370,372,378,380,386,387,393,395,401,403,408,409,414,416,421,423,428,430,435],{"type":57,"value":371},"For dataset-backed actions, collect the required image, caption, list, or prompt files from the user and place the resolved paths in ",{"type":51,"tag":71,"props":373,"children":375},{"className":374},[],[376],{"type":57,"value":377},"spec_overrides",{"type":57,"value":379},". For local Docker runs, mount extracted folders in the container and point ",{"type":51,"tag":71,"props":381,"children":383},{"className":382},[],[384],{"type":57,"value":385},"image_dir",{"type":57,"value":196},{"type":51,"tag":71,"props":388,"children":390},{"className":389},[],[391],{"type":57,"value":392},"caption_dir",{"type":57,"value":394}," at those folders; if a data source provides ",{"type":51,"tag":71,"props":396,"children":398},{"className":397},[],[399],{"type":57,"value":400},".tar.gz",{"type":57,"value":402}," archives, extract them before running the in-container CLIP commands. For ",{"type":51,"tag":71,"props":404,"children":406},{"className":405},[],[407],{"type":57,"value":143},{"type":57,"value":234},{"type":51,"tag":71,"props":410,"children":412},{"className":411},[],[413],{"type":57,"value":150},{"type":57,"value":415},", infer parent artifacts from the upstream job when available; otherwise require explicit checkpoint, ONNX, or engine paths. Run ",{"type":51,"tag":71,"props":417,"children":419},{"className":418},[],[420],{"type":57,"value":150},{"type":57,"value":422},", TensorRT ",{"type":51,"tag":71,"props":424,"children":426},{"className":425},[],[427],{"type":57,"value":129},{"type":57,"value":429},", and TensorRT ",{"type":51,"tag":71,"props":431,"children":433},{"className":432},[],[434],{"type":57,"value":136},{"type":57,"value":436}," in the TAO Deploy image.",{"type":51,"tag":60,"props":438,"children":439},{},[440,442,447,448,453,454,459,461,467,469,475,477,483],{"type":57,"value":441},"For TAO Deploy TensorRT actions (",{"type":51,"tag":71,"props":443,"children":445},{"className":444},[],[446],{"type":57,"value":150},{"type":57,"value":422},{"type":51,"tag":71,"props":449,"children":451},{"className":450},[],[452],{"type":57,"value":129},{"type":57,"value":429},{"type":51,"tag":71,"props":455,"children":457},{"className":456},[],[458],{"type":57,"value":136},{"type":57,"value":460},"), read ",{"type":51,"tag":71,"props":462,"children":464},{"className":463},[],[465],{"type":57,"value":466},"references\u002Ftao-deploy-clip.md",{"type":57,"value":468}," first. Deploy spec templates live in this skill's ",{"type":51,"tag":71,"props":470,"children":472},{"className":471},[],[473],{"type":57,"value":474},"references\u002F",{"type":57,"value":476}," folder with the ",{"type":51,"tag":71,"props":478,"children":480},{"className":479},[],[481],{"type":57,"value":482},"spec_template_deploy_*.yaml",{"type":57,"value":484}," prefix.",{"type":51,"tag":154,"props":486,"children":488},{"id":487},"training-requirements",[489],{"type":57,"value":490},"Training Requirements",{"type":51,"tag":492,"props":493,"children":494},"ul",{},[495,507,517],{"type":51,"tag":496,"props":497,"children":498},"li",{},[499,505],{"type":51,"tag":500,"props":501,"children":502},"strong",{},[503],{"type":57,"value":504},"Dataset type:",{"type":57,"value":506}," image_text",{"type":51,"tag":496,"props":508,"children":509},{},[510,515],{"type":51,"tag":500,"props":511,"children":512},{},[513],{"type":57,"value":514},"Formats:",{"type":57,"value":516}," custom image\u002Fcaption folders or WebDataset shards",{"type":51,"tag":496,"props":518,"children":519},{},[520,525],{"type":51,"tag":500,"props":521,"children":522},{},[523],{"type":57,"value":524},"Monitoring metric:",{"type":57,"value":526}," val\u002Ft2i_mAP",{"type":51,"tag":528,"props":529,"children":531},"h3",{"id":530},"supported-models",[532],{"type":57,"value":533},"Supported Models",{"type":51,"tag":492,"props":535,"children":536},{},[537,582,618],{"type":51,"tag":496,"props":538,"children":539},{},[540,545,547,553,555,561,562,568,569,575,576],{"type":51,"tag":500,"props":541,"children":542},{},[543],{"type":57,"value":544},"OpenCLIP \u002F NV-CLIP:",{"type":57,"value":546}," ",{"type":51,"tag":71,"props":548,"children":550},{"className":549},[],[551],{"type":57,"value":552},"ViT-L-14-SigLIP-CLIPA-224",{"type":57,"value":554}," (default), ",{"type":51,"tag":71,"props":556,"children":558},{"className":557},[],[559],{"type":57,"value":560},"ViT-L-14-SigLIP-CLIPA-336",{"type":57,"value":123},{"type":51,"tag":71,"props":563,"children":565},{"className":564},[],[566],{"type":57,"value":567},"ViT-H-14-SigLIP-CLIPA-224",{"type":57,"value":123},{"type":51,"tag":71,"props":570,"children":572},{"className":571},[],[573],{"type":57,"value":574},"ViT-H-14-SigLIP-CLIPA-336",{"type":57,"value":123},{"type":51,"tag":71,"props":577,"children":579},{"className":578},[],[580],{"type":57,"value":581},"ViT-H-14-SigLIP-CLIPA-574",{"type":51,"tag":496,"props":583,"children":584},{},[585,590,591,597,598,604,605,611,612],{"type":51,"tag":500,"props":586,"children":587},{},[588],{"type":57,"value":589},"Radio-CLIP:",{"type":57,"value":546},{"type":51,"tag":71,"props":592,"children":594},{"className":593},[],[595],{"type":57,"value":596},"c-radio_v3-b",{"type":57,"value":123},{"type":51,"tag":71,"props":599,"children":601},{"className":600},[],[602],{"type":57,"value":603},"c-radio_v3-l",{"type":57,"value":123},{"type":51,"tag":71,"props":606,"children":608},{"className":607},[],[609],{"type":57,"value":610},"c-radio_v3-h",{"type":57,"value":123},{"type":51,"tag":71,"props":613,"children":615},{"className":614},[],[616],{"type":57,"value":617},"c-radio_v3-g",{"type":51,"tag":496,"props":619,"children":620},{},[621,626,627,633,634,640,641,647,648,654,655,661,662],{"type":51,"tag":500,"props":622,"children":623},{},[624],{"type":57,"value":625},"SigLIP2:",{"type":57,"value":546},{"type":51,"tag":71,"props":628,"children":630},{"className":629},[],[631],{"type":57,"value":632},"siglip2-so400m-patch16-256",{"type":57,"value":123},{"type":51,"tag":71,"props":635,"children":637},{"className":636},[],[638],{"type":57,"value":639},"siglip2-so400m-patch14-224",{"type":57,"value":123},{"type":51,"tag":71,"props":642,"children":644},{"className":643},[],[645],{"type":57,"value":646},"siglip2-so400m-patch14-384",{"type":57,"value":123},{"type":51,"tag":71,"props":649,"children":651},{"className":650},[],[652],{"type":57,"value":653},"siglip2-so400m-patch16-384",{"type":57,"value":123},{"type":51,"tag":71,"props":656,"children":658},{"className":657},[],[659],{"type":57,"value":660},"siglip2-so400m-patch16-512",{"type":57,"value":123},{"type":51,"tag":71,"props":663,"children":665},{"className":664},[],[666],{"type":57,"value":667},"siglip2-so400m-patch16-naflex",{"type":51,"tag":60,"props":669,"children":670},{},[671,673,679,681,687,689,694],{"type":57,"value":672},"Radio-CLIP requires ",{"type":51,"tag":71,"props":674,"children":676},{"className":675},[],[677],{"type":57,"value":678},"model.adaptor_name",{"type":57,"value":680}," to be set to ",{"type":51,"tag":71,"props":682,"children":684},{"className":683},[],[685],{"type":57,"value":686},"siglip",{"type":57,"value":688}," or ",{"type":51,"tag":71,"props":690,"children":692},{"className":691},[],[693],{"type":57,"value":54},{"type":57,"value":152},{"type":51,"tag":528,"props":696,"children":698},{"id":697},"per-action-dataset-requirements",[699],{"type":57,"value":700},"Per-Action Dataset Requirements",{"type":51,"tag":702,"props":703,"children":704},"table",{},[705,739],{"type":51,"tag":706,"props":707,"children":708},"thead",{},[709],{"type":51,"tag":710,"props":711,"children":712},"tr",{},[713,719,724,729,734],{"type":51,"tag":714,"props":715,"children":716},"th",{},[717],{"type":57,"value":718},"Action",{"type":51,"tag":714,"props":720,"children":721},{},[722],{"type":57,"value":723},"Spec Key",{"type":51,"tag":714,"props":725,"children":726},{},[727],{"type":57,"value":728},"Source",{"type":51,"tag":714,"props":730,"children":731},{},[732],{"type":57,"value":733},"Files",{"type":51,"tag":714,"props":735,"children":736},{},[737],{"type":57,"value":738},"List?",{"type":51,"tag":740,"props":741,"children":742},"tbody",{},[743,771,806,831,856,879,905,930,956],{"type":51,"tag":710,"props":744,"children":745},{},[746,751,756,761,766],{"type":51,"tag":747,"props":748,"children":749},"td",{},[750],{"type":57,"value":121},{"type":51,"tag":747,"props":752,"children":753},{},[754],{"type":57,"value":755},"dataset.train.datasets",{"type":51,"tag":747,"props":757,"children":758},{},[759],{"type":57,"value":760},"train_datasets",{"type":51,"tag":747,"props":762,"children":763},{},[764],{"type":57,"value":765},"image_dir: images.tar.gz, image_list_file: image_list.txt, caption_dir: captions.tar.gz",{"type":51,"tag":747,"props":767,"children":768},{},[769],{"type":57,"value":770},"Yes",{"type":51,"tag":710,"props":772,"children":773},{},[774,778,783,788,801],{"type":51,"tag":747,"props":775,"children":776},{},[777],{"type":57,"value":121},{"type":51,"tag":747,"props":779,"children":780},{},[781],{"type":57,"value":782},"dataset.train.wds.root_dir",{"type":51,"tag":747,"props":784,"children":785},{},[786],{"type":57,"value":787},"train_wds_dataset",{"type":51,"tag":747,"props":789,"children":790},{},[791,793,799],{"type":57,"value":792},"root directory containing ",{"type":51,"tag":71,"props":794,"children":796},{"className":795},[],[797],{"type":57,"value":798},".tar",{"type":57,"value":800}," shards",{"type":51,"tag":747,"props":802,"children":803},{},[804],{"type":57,"value":805},"No",{"type":51,"tag":710,"props":807,"children":808},{},[809,813,818,822,827],{"type":51,"tag":747,"props":810,"children":811},{},[812],{"type":57,"value":121},{"type":51,"tag":747,"props":814,"children":815},{},[816],{"type":57,"value":817},"dataset.train.wds.shard_list_file",{"type":51,"tag":747,"props":819,"children":820},{},[821],{"type":57,"value":787},{"type":51,"tag":747,"props":823,"children":824},{},[825],{"type":57,"value":826},"shards.txt listing shard paths",{"type":51,"tag":747,"props":828,"children":829},{},[830],{"type":57,"value":805},{"type":51,"tag":710,"props":832,"children":833},{},[834,838,843,848,852],{"type":51,"tag":747,"props":835,"children":836},{},[837],{"type":57,"value":121},{"type":51,"tag":747,"props":839,"children":840},{},[841],{"type":57,"value":842},"dataset.val.datasets",{"type":51,"tag":747,"props":844,"children":845},{},[846],{"type":57,"value":847},"eval_dataset",{"type":51,"tag":747,"props":849,"children":850},{},[851],{"type":57,"value":765},{"type":51,"tag":747,"props":853,"children":854},{},[855],{"type":57,"value":770},{"type":51,"tag":710,"props":857,"children":858},{},[859,863,867,871,875],{"type":51,"tag":747,"props":860,"children":861},{},[862],{"type":57,"value":129},{"type":51,"tag":747,"props":864,"children":865},{},[866],{"type":57,"value":842},{"type":51,"tag":747,"props":868,"children":869},{},[870],{"type":57,"value":847},{"type":51,"tag":747,"props":872,"children":873},{},[874],{"type":57,"value":765},{"type":51,"tag":747,"props":876,"children":877},{},[878],{"type":57,"value":770},{"type":51,"tag":710,"props":880,"children":881},{},[882,886,891,896,901],{"type":51,"tag":747,"props":883,"children":884},{},[885],{"type":57,"value":136},{"type":51,"tag":747,"props":887,"children":888},{},[889],{"type":57,"value":890},"inference.datasets",{"type":51,"tag":747,"props":892,"children":893},{},[894],{"type":57,"value":895},"inference_dataset",{"type":51,"tag":747,"props":897,"children":898},{},[899],{"type":57,"value":900},"image_dir: images.tar.gz",{"type":51,"tag":747,"props":902,"children":903},{},[904],{"type":57,"value":770},{"type":51,"tag":710,"props":906,"children":907},{},[908,912,917,921,926],{"type":51,"tag":747,"props":909,"children":910},{},[911],{"type":57,"value":136},{"type":51,"tag":747,"props":913,"children":914},{},[915],{"type":57,"value":916},"inference.text_file",{"type":51,"tag":747,"props":918,"children":919},{},[920],{"type":57,"value":895},{"type":51,"tag":747,"props":922,"children":923},{},[924],{"type":57,"value":925},"prompts.txt",{"type":51,"tag":747,"props":927,"children":928},{},[929],{"type":57,"value":805},{"type":51,"tag":710,"props":931,"children":932},{},[933,937,942,947,952],{"type":51,"tag":747,"props":934,"children":935},{},[936],{"type":57,"value":143},{"type":51,"tag":747,"props":938,"children":939},{},[940],{"type":57,"value":941},"export.checkpoint",{"type":51,"tag":747,"props":943,"children":944},{},[945],{"type":57,"value":946},"parent train job or explicit checkpoint",{"type":51,"tag":747,"props":948,"children":949},{},[950],{"type":57,"value":951},"checkpoint .pth, optional for pretrained export",{"type":51,"tag":747,"props":953,"children":954},{},[955],{"type":57,"value":805},{"type":51,"tag":710,"props":957,"children":958},{},[959,963,968,973,978],{"type":51,"tag":747,"props":960,"children":961},{},[962],{"type":57,"value":150},{"type":51,"tag":747,"props":964,"children":965},{},[966],{"type":57,"value":967},"gen_trt_engine.onnx_file",{"type":51,"tag":747,"props":969,"children":970},{},[971],{"type":57,"value":972},"parent export job or explicit ONNX",{"type":51,"tag":747,"props":974,"children":975},{},[976],{"type":57,"value":977},"clip_model.onnx",{"type":51,"tag":747,"props":979,"children":980},{},[981],{"type":57,"value":805},{"type":51,"tag":60,"props":983,"children":984},{},[985,987,993,995,1000,1002,1008,1010,1016,1018,1024],{"type":57,"value":986},"For custom training, set ",{"type":51,"tag":71,"props":988,"children":990},{"className":989},[],[991],{"type":57,"value":992},"dataset.train.type: custom",{"type":57,"value":994}," and provide ",{"type":51,"tag":71,"props":996,"children":998},{"className":997},[],[999],{"type":57,"value":755},{"type":57,"value":1001}," entries. Image and caption files must share the same base name. ",{"type":51,"tag":71,"props":1003,"children":1005},{"className":1004},[],[1006],{"type":57,"value":1007},"caption_file_suffix",{"type":57,"value":1009}," defaults to ",{"type":51,"tag":71,"props":1011,"children":1013},{"className":1012},[],[1014],{"type":57,"value":1015},".txt",{"type":57,"value":1017},", and ",{"type":51,"tag":71,"props":1019,"children":1021},{"className":1020},[],[1022],{"type":57,"value":1023},"image_list_file",{"type":57,"value":1025}," is optional.",{"type":51,"tag":60,"props":1027,"children":1028},{},[1029,1031,1036],{"type":57,"value":1030},"When no native CLIP image-caption dataset is available, do not silently treat image-classification data as CLIP data. If the user explicitly allows a plumbing-only validation fallback, derive caption files from class labels, document that the captions are generated from labels, and keep each image\u002Fcaption pair on the same base filename. Without an ",{"type":51,"tag":71,"props":1032,"children":1034},{"className":1033},[],[1035],{"type":57,"value":1023},{"type":57,"value":1037},", the TAO custom loader scans the configured image directory for image files; keep validation folders flat unless you provide a list file.",{"type":51,"tag":60,"props":1039,"children":1040},{},[1041,1043,1049,1051,1056,1057,1062,1064,1070,1072,1077,1079,1085,1087,1092,1094,1099,1101,1106],{"type":57,"value":1042},"For WDS training, set ",{"type":51,"tag":71,"props":1044,"children":1046},{"className":1045},[],[1047],{"type":57,"value":1048},"dataset.train.type: wds",{"type":57,"value":1050}," and provide at least one of ",{"type":51,"tag":71,"props":1052,"children":1054},{"className":1053},[],[1055],{"type":57,"value":782},{"type":57,"value":688},{"type":51,"tag":71,"props":1058,"children":1060},{"className":1059},[],[1061],{"type":57,"value":817},{"type":57,"value":1063},". ",{"type":51,"tag":71,"props":1065,"children":1067},{"className":1066},[],[1068],{"type":57,"value":1069},"root_dir",{"type":57,"value":1071}," is scanned recursively for ",{"type":51,"tag":71,"props":1073,"children":1075},{"className":1074},[],[1076],{"type":57,"value":798},{"type":57,"value":1078}," shards. ",{"type":51,"tag":71,"props":1080,"children":1082},{"className":1081},[],[1083],{"type":57,"value":1084},"shard_list_file",{"type":57,"value":1086}," is a text file with one shard path per line; relative lines resolve under the list-file directory unless ",{"type":51,"tag":71,"props":1088,"children":1090},{"className":1089},[],[1091],{"type":57,"value":1069},{"type":57,"value":1093}," is also supplied, in which case they resolve under ",{"type":51,"tag":71,"props":1095,"children":1097},{"className":1096},[],[1098],{"type":57,"value":1069},{"type":57,"value":1100},". Validation\u002Fevaluation data remains custom format via ",{"type":51,"tag":71,"props":1102,"children":1104},{"className":1103},[],[1105],{"type":57,"value":842},{"type":57,"value":152},{"type":51,"tag":528,"props":1108,"children":1110},{"id":1109},"typical-spec-overrides",[1111],{"type":57,"value":1112},"Typical Spec Overrides",{"type":51,"tag":60,"props":1114,"children":1115},{},[1116,1118,1123,1125,1130,1131,1136],{"type":57,"value":1117},"Data source overrides are mandatory for dataset-backed actions. Construct paths from the Per-Action Dataset Requirements table and include them in ",{"type":51,"tag":71,"props":1119,"children":1121},{"className":1120},[],[1122],{"type":57,"value":377},{"type":57,"value":1124},". For inference, provide at least one of ",{"type":51,"tag":71,"props":1126,"children":1128},{"className":1127},[],[1129],{"type":57,"value":890},{"type":57,"value":688},{"type":51,"tag":71,"props":1132,"children":1134},{"className":1133},[],[1135],{"type":57,"value":916},{"type":57,"value":152},{"type":51,"tag":1138,"props":1139,"children":1144},"pre",{"className":1140,"code":1141,"language":1142,"meta":1143,"style":1143},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","S3_TRAIN = \"s3:\u002F\u002Fbucket\u002Fdata\u002Ftrain\"\nS3_WDS = \"s3:\u002F\u002Fbucket\u002Fdata\u002Fwds\"\nS3_EVAL = \"s3:\u002F\u002Fbucket\u002Fdata\u002Feval\"\nS3_INFER = \"s3:\u002F\u002Fbucket\u002Fdata\u002Finfer\"\n","python","",[1145],{"type":51,"tag":71,"props":1146,"children":1147},{"__ignoreMap":1143},[1148,1159,1168,1177],{"type":51,"tag":1149,"props":1150,"children":1153},"span",{"class":1151,"line":1152},"line",1,[1154],{"type":51,"tag":1149,"props":1155,"children":1156},{},[1157],{"type":57,"value":1158},"S3_TRAIN = \"s3:\u002F\u002Fbucket\u002Fdata\u002Ftrain\"\n",{"type":51,"tag":1149,"props":1160,"children":1162},{"class":1151,"line":1161},2,[1163],{"type":51,"tag":1149,"props":1164,"children":1165},{},[1166],{"type":57,"value":1167},"S3_WDS = \"s3:\u002F\u002Fbucket\u002Fdata\u002Fwds\"\n",{"type":51,"tag":1149,"props":1169,"children":1171},{"class":1151,"line":1170},3,[1172],{"type":51,"tag":1149,"props":1173,"children":1174},{},[1175],{"type":57,"value":1176},"S3_EVAL = \"s3:\u002F\u002Fbucket\u002Fdata\u002Feval\"\n",{"type":51,"tag":1149,"props":1178,"children":1180},{"class":1151,"line":1179},4,[1181],{"type":51,"tag":1149,"props":1182,"children":1183},{},[1184],{"type":57,"value":1185},"S3_INFER = \"s3:\u002F\u002Fbucket\u002Fdata\u002Finfer\"\n",{"type":51,"tag":60,"props":1187,"children":1188},{},[1189],{"type":51,"tag":500,"props":1190,"children":1191},{},[1192],{"type":57,"value":1193},"train, custom dataset:",{"type":51,"tag":1138,"props":1195,"children":1197},{"className":1140,"code":1196,"language":1142,"meta":1143,"style":1143},"{\n    \"train.num_epochs\": 10,\n    \"dataset.train.type\": \"custom\",\n    \"dataset.train.datasets\": [{\"image_dir\": f\"{S3_TRAIN}\u002Fimages.tar.gz\", \"image_list_file\": f\"{S3_TRAIN}\u002Fimage_list.txt\", \"caption_dir\": f\"{S3_TRAIN}\u002Fcaptions.tar.gz\"}],\n    \"dataset.val.datasets\": [{\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"image_list_file\": f\"{S3_EVAL}\u002Fimage_list.txt\", \"caption_dir\": f\"{S3_EVAL}\u002Fcaptions.tar.gz\"}],\n}\n",[1198],{"type":51,"tag":71,"props":1199,"children":1200},{"__ignoreMap":1143},[1201,1209,1217,1225,1233,1242],{"type":51,"tag":1149,"props":1202,"children":1203},{"class":1151,"line":1152},[1204],{"type":51,"tag":1149,"props":1205,"children":1206},{},[1207],{"type":57,"value":1208},"{\n",{"type":51,"tag":1149,"props":1210,"children":1211},{"class":1151,"line":1161},[1212],{"type":51,"tag":1149,"props":1213,"children":1214},{},[1215],{"type":57,"value":1216},"    \"train.num_epochs\": 10,\n",{"type":51,"tag":1149,"props":1218,"children":1219},{"class":1151,"line":1170},[1220],{"type":51,"tag":1149,"props":1221,"children":1222},{},[1223],{"type":57,"value":1224},"    \"dataset.train.type\": \"custom\",\n",{"type":51,"tag":1149,"props":1226,"children":1227},{"class":1151,"line":1179},[1228],{"type":51,"tag":1149,"props":1229,"children":1230},{},[1231],{"type":57,"value":1232},"    \"dataset.train.datasets\": [{\"image_dir\": f\"{S3_TRAIN}\u002Fimages.tar.gz\", \"image_list_file\": f\"{S3_TRAIN}\u002Fimage_list.txt\", \"caption_dir\": f\"{S3_TRAIN}\u002Fcaptions.tar.gz\"}],\n",{"type":51,"tag":1149,"props":1234,"children":1236},{"class":1151,"line":1235},5,[1237],{"type":51,"tag":1149,"props":1238,"children":1239},{},[1240],{"type":57,"value":1241},"    \"dataset.val.datasets\": [{\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"image_list_file\": f\"{S3_EVAL}\u002Fimage_list.txt\", \"caption_dir\": f\"{S3_EVAL}\u002Fcaptions.tar.gz\"}],\n",{"type":51,"tag":1149,"props":1243,"children":1245},{"class":1151,"line":1244},6,[1246],{"type":51,"tag":1149,"props":1247,"children":1248},{},[1249],{"type":57,"value":1250},"}\n",{"type":51,"tag":60,"props":1252,"children":1253},{},[1254],{"type":51,"tag":500,"props":1255,"children":1256},{},[1257],{"type":57,"value":1258},"train, WDS dataset:",{"type":51,"tag":1138,"props":1260,"children":1262},{"className":1140,"code":1261,"language":1142,"meta":1143,"style":1143},"{\n    \"train.num_epochs\": 10,\n    \"dataset.train.type\": \"wds\",\n    \"dataset.train.wds.root_dir\": f\"{S3_WDS}\",\n    \"dataset.train.wds.shard_list_file\": f\"{S3_WDS}\u002Fshards.txt\",\n    \"dataset.train.wds.samples_per_shard\": 10000,\n    \"dataset.val.datasets\": [{\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"image_list_file\": f\"{S3_EVAL}\u002Fimage_list.txt\", \"caption_dir\": f\"{S3_EVAL}\u002Fcaptions.tar.gz\"}],\n}\n",[1263],{"type":51,"tag":71,"props":1264,"children":1265},{"__ignoreMap":1143},[1266,1273,1280,1288,1296,1304,1312,1320],{"type":51,"tag":1149,"props":1267,"children":1268},{"class":1151,"line":1152},[1269],{"type":51,"tag":1149,"props":1270,"children":1271},{},[1272],{"type":57,"value":1208},{"type":51,"tag":1149,"props":1274,"children":1275},{"class":1151,"line":1161},[1276],{"type":51,"tag":1149,"props":1277,"children":1278},{},[1279],{"type":57,"value":1216},{"type":51,"tag":1149,"props":1281,"children":1282},{"class":1151,"line":1170},[1283],{"type":51,"tag":1149,"props":1284,"children":1285},{},[1286],{"type":57,"value":1287},"    \"dataset.train.type\": \"wds\",\n",{"type":51,"tag":1149,"props":1289,"children":1290},{"class":1151,"line":1179},[1291],{"type":51,"tag":1149,"props":1292,"children":1293},{},[1294],{"type":57,"value":1295},"    \"dataset.train.wds.root_dir\": f\"{S3_WDS}\",\n",{"type":51,"tag":1149,"props":1297,"children":1298},{"class":1151,"line":1235},[1299],{"type":51,"tag":1149,"props":1300,"children":1301},{},[1302],{"type":57,"value":1303},"    \"dataset.train.wds.shard_list_file\": f\"{S3_WDS}\u002Fshards.txt\",\n",{"type":51,"tag":1149,"props":1305,"children":1306},{"class":1151,"line":1244},[1307],{"type":51,"tag":1149,"props":1308,"children":1309},{},[1310],{"type":57,"value":1311},"    \"dataset.train.wds.samples_per_shard\": 10000,\n",{"type":51,"tag":1149,"props":1313,"children":1315},{"class":1151,"line":1314},7,[1316],{"type":51,"tag":1149,"props":1317,"children":1318},{},[1319],{"type":57,"value":1241},{"type":51,"tag":1149,"props":1321,"children":1323},{"class":1151,"line":1322},8,[1324],{"type":51,"tag":1149,"props":1325,"children":1326},{},[1327],{"type":57,"value":1250},{"type":51,"tag":60,"props":1329,"children":1330},{},[1331],{"type":51,"tag":500,"props":1332,"children":1333},{},[1334],{"type":57,"value":1335},"evaluate:",{"type":51,"tag":1138,"props":1337,"children":1339},{"className":1140,"code":1338,"language":1142,"meta":1143,"style":1143},"{\n    \"dataset.val.datasets\": [{\"image_dir\": f\"{S3_EVAL}\u002Fimages.tar.gz\", \"image_list_file\": f\"{S3_EVAL}\u002Fimage_list.txt\", \"caption_dir\": f\"{S3_EVAL}\u002Fcaptions.tar.gz\"}],\n}\n",[1340],{"type":51,"tag":71,"props":1341,"children":1342},{"__ignoreMap":1143},[1343,1350,1357],{"type":51,"tag":1149,"props":1344,"children":1345},{"class":1151,"line":1152},[1346],{"type":51,"tag":1149,"props":1347,"children":1348},{},[1349],{"type":57,"value":1208},{"type":51,"tag":1149,"props":1351,"children":1352},{"class":1151,"line":1161},[1353],{"type":51,"tag":1149,"props":1354,"children":1355},{},[1356],{"type":57,"value":1241},{"type":51,"tag":1149,"props":1358,"children":1359},{"class":1151,"line":1170},[1360],{"type":51,"tag":1149,"props":1361,"children":1362},{},[1363],{"type":57,"value":1250},{"type":51,"tag":60,"props":1365,"children":1366},{},[1367,1369,1375,1377,1383,1385,1390],{"type":57,"value":1368},"Leave ",{"type":51,"tag":71,"props":1370,"children":1372},{"className":1371},[],[1373],{"type":57,"value":1374},"evaluate.checkpoint",{"type":57,"value":1376}," unset for zero-shot evaluation with pretrained weights. Set ",{"type":51,"tag":71,"props":1378,"children":1380},{"className":1379},[],[1381],{"type":57,"value":1382},"evaluate.trt_engine",{"type":57,"value":1384}," instead of ",{"type":51,"tag":71,"props":1386,"children":1388},{"className":1387},[],[1389],{"type":57,"value":1374},{"type":57,"value":1391}," for TensorRT evaluation.",{"type":51,"tag":60,"props":1393,"children":1394},{},[1395],{"type":51,"tag":500,"props":1396,"children":1397},{},[1398],{"type":57,"value":1399},"inference:",{"type":51,"tag":1138,"props":1401,"children":1403},{"className":1140,"code":1402,"language":1142,"meta":1143,"style":1143},"{\n    \"inference.datasets\": [{\"image_dir\": f\"{S3_INFER}\u002Fimages.tar.gz\"}],\n    \"inference.text_file\": f\"{S3_INFER}\u002Fprompts.txt\",\n}\n",[1404],{"type":51,"tag":71,"props":1405,"children":1406},{"__ignoreMap":1143},[1407,1414,1422,1430],{"type":51,"tag":1149,"props":1408,"children":1409},{"class":1151,"line":1152},[1410],{"type":51,"tag":1149,"props":1411,"children":1412},{},[1413],{"type":57,"value":1208},{"type":51,"tag":1149,"props":1415,"children":1416},{"class":1151,"line":1161},[1417],{"type":51,"tag":1149,"props":1418,"children":1419},{},[1420],{"type":57,"value":1421},"    \"inference.datasets\": [{\"image_dir\": f\"{S3_INFER}\u002Fimages.tar.gz\"}],\n",{"type":51,"tag":1149,"props":1423,"children":1424},{"class":1151,"line":1170},[1425],{"type":51,"tag":1149,"props":1426,"children":1427},{},[1428],{"type":57,"value":1429},"    \"inference.text_file\": f\"{S3_INFER}\u002Fprompts.txt\",\n",{"type":51,"tag":1149,"props":1431,"children":1432},{"class":1151,"line":1179},[1433],{"type":51,"tag":1149,"props":1434,"children":1435},{},[1436],{"type":57,"value":1250},{"type":51,"tag":60,"props":1438,"children":1439},{},[1440,1442,1448,1450,1456,1458,1464],{"type":57,"value":1441},"Inference writes ",{"type":51,"tag":71,"props":1443,"children":1445},{"className":1444},[],[1446],{"type":57,"value":1447},"image_embeddings.h5",{"type":57,"value":1449}," and\u002For ",{"type":51,"tag":71,"props":1451,"children":1453},{"className":1452},[],[1454],{"type":57,"value":1455},"text_embeddings.h5",{"type":57,"value":1457}," under ",{"type":51,"tag":71,"props":1459,"children":1461},{"className":1460},[],[1462],{"type":57,"value":1463},"results_dir",{"type":57,"value":1465},". The saved embeddings are L2-normalized.",{"type":51,"tag":60,"props":1467,"children":1468},{},[1469],{"type":51,"tag":500,"props":1470,"children":1471},{},[1472],{"type":57,"value":1473},"export:",{"type":51,"tag":1138,"props":1475,"children":1477},{"className":1140,"code":1476,"language":1142,"meta":1143,"style":1143},"{\n    \"export.onnx_file\": \"${results_dir}\u002Fexport\u002Fclip_model.onnx\",\n    \"export.encoder_type\": \"combined\",\n    \"export.batch_size\": -1,\n}\n",[1478],{"type":51,"tag":71,"props":1479,"children":1480},{"__ignoreMap":1143},[1481,1488,1496,1504,1512],{"type":51,"tag":1149,"props":1482,"children":1483},{"class":1151,"line":1152},[1484],{"type":51,"tag":1149,"props":1485,"children":1486},{},[1487],{"type":57,"value":1208},{"type":51,"tag":1149,"props":1489,"children":1490},{"class":1151,"line":1161},[1491],{"type":51,"tag":1149,"props":1492,"children":1493},{},[1494],{"type":57,"value":1495},"    \"export.onnx_file\": \"${results_dir}\u002Fexport\u002Fclip_model.onnx\",\n",{"type":51,"tag":1149,"props":1497,"children":1498},{"class":1151,"line":1170},[1499],{"type":51,"tag":1149,"props":1500,"children":1501},{},[1502],{"type":57,"value":1503},"    \"export.encoder_type\": \"combined\",\n",{"type":51,"tag":1149,"props":1505,"children":1506},{"class":1151,"line":1179},[1507],{"type":51,"tag":1149,"props":1508,"children":1509},{},[1510],{"type":57,"value":1511},"    \"export.batch_size\": -1,\n",{"type":51,"tag":1149,"props":1513,"children":1514},{"class":1151,"line":1235},[1515],{"type":51,"tag":1149,"props":1516,"children":1517},{},[1518],{"type":57,"value":1250},{"type":51,"tag":60,"props":1520,"children":1521},{},[1522,1524,1530,1532,1538,1539,1545,1547,1553],{"type":57,"value":1523},"Set ",{"type":51,"tag":71,"props":1525,"children":1527},{"className":1526},[],[1528],{"type":57,"value":1529},"export.encoder_type: separate",{"type":57,"value":1531}," when deployment should use independent vision and text encoders. Separate export writes ",{"type":51,"tag":71,"props":1533,"children":1535},{"className":1534},[],[1536],{"type":57,"value":1537},"_vision.onnx",{"type":57,"value":234},{"type":51,"tag":71,"props":1540,"children":1542},{"className":1541},[],[1543],{"type":57,"value":1544},"_text.onnx",{"type":57,"value":1546}," variants derived from the base ",{"type":51,"tag":71,"props":1548,"children":1550},{"className":1549},[],[1551],{"type":57,"value":1552},"export.onnx_file",{"type":57,"value":152},{"type":51,"tag":60,"props":1555,"children":1556},{},[1557,1559,1565,1567,1573,1575,1580,1581,1587,1588,1593,1594,1600,1602,1607],{"type":57,"value":1558},"For checkpoint-dependent actions, use the model-specific checkpoint resolver output from the parent train job. CLIP training writes checkpoints such as ",{"type":51,"tag":71,"props":1560,"children":1562},{"className":1561},[],[1563],{"type":57,"value":1564},"model_epoch_000_step_00020.pth",{"type":57,"value":1566}," and a ",{"type":51,"tag":71,"props":1568,"children":1570},{"className":1569},[],[1571],{"type":57,"value":1572},"clip_latest.pth",{"type":57,"value":1574}," symlink. Use the exact resolved checkpoint for ",{"type":51,"tag":71,"props":1576,"children":1578},{"className":1577},[],[1579],{"type":57,"value":1374},{"type":57,"value":123},{"type":51,"tag":71,"props":1582,"children":1584},{"className":1583},[],[1585],{"type":57,"value":1586},"inference.checkpoint",{"type":57,"value":123},{"type":51,"tag":71,"props":1589,"children":1591},{"className":1590},[],[1592],{"type":57,"value":941},{"type":57,"value":1017},{"type":51,"tag":71,"props":1595,"children":1597},{"className":1596},[],[1598],{"type":57,"value":1599},"train.resume_training_checkpoint_path",{"type":57,"value":1601},"; use ",{"type":51,"tag":71,"props":1603,"children":1605},{"className":1604},[],[1606],{"type":57,"value":1572},{"type":57,"value":1608}," only when the user explicitly asks for latest.",{"type":51,"tag":60,"props":1610,"children":1611},{},[1612,1614,1619,1620,1625,1626,1631,1633,1638],{"type":57,"value":1613},"When the resolved checkpoint is trusted TAO output, checkpoint-backed PyTorch ",{"type":51,"tag":71,"props":1615,"children":1617},{"className":1616},[],[1618],{"type":57,"value":129},{"type":57,"value":123},{"type":51,"tag":71,"props":1621,"children":1623},{"className":1622},[],[1624],{"type":57,"value":136},{"type":57,"value":123},{"type":51,"tag":71,"props":1627,"children":1629},{"className":1628},[],[1630],{"type":57,"value":143},{"type":57,"value":1632},", and resume training should run with ",{"type":51,"tag":71,"props":1634,"children":1636},{"className":1635},[],[1637],{"type":57,"value":108},{"type":57,"value":1639},". PyTorch 2.6 otherwise defaults checkpoint loading to weights-only mode and can reject CLIP Lightning checkpoints containing NumPy scalar metadata.",{"type":51,"tag":60,"props":1641,"children":1642},{},[1643],{"type":51,"tag":500,"props":1644,"children":1645},{},[1646],{"type":57,"value":1647},"gen_trt_engine:",{"type":51,"tag":1138,"props":1649,"children":1651},{"className":1140,"code":1650,"language":1142,"meta":1143,"style":1143},"{\n    \"gen_trt_engine.onnx_file\": \"${results_dir}\u002Fexport\u002Fclip_model.onnx\",\n    \"gen_trt_engine.trt_engine\": \"${results_dir}\u002Fdeploy\u002Fclip_model.engine\",\n    \"gen_trt_engine.batch_size\": -1,\n    \"gen_trt_engine.tensorrt.data_type\": \"fp16\",\n    \"gen_trt_engine.tensorrt.min_batch_size\": 1,\n    \"gen_trt_engine.tensorrt.opt_batch_size\": 1,\n    \"gen_trt_engine.tensorrt.max_batch_size\": 16,\n}\n",[1652],{"type":51,"tag":71,"props":1653,"children":1654},{"__ignoreMap":1143},[1655,1662,1670,1678,1686,1694,1702,1710,1718],{"type":51,"tag":1149,"props":1656,"children":1657},{"class":1151,"line":1152},[1658],{"type":51,"tag":1149,"props":1659,"children":1660},{},[1661],{"type":57,"value":1208},{"type":51,"tag":1149,"props":1663,"children":1664},{"class":1151,"line":1161},[1665],{"type":51,"tag":1149,"props":1666,"children":1667},{},[1668],{"type":57,"value":1669},"    \"gen_trt_engine.onnx_file\": \"${results_dir}\u002Fexport\u002Fclip_model.onnx\",\n",{"type":51,"tag":1149,"props":1671,"children":1672},{"class":1151,"line":1170},[1673],{"type":51,"tag":1149,"props":1674,"children":1675},{},[1676],{"type":57,"value":1677},"    \"gen_trt_engine.trt_engine\": \"${results_dir}\u002Fdeploy\u002Fclip_model.engine\",\n",{"type":51,"tag":1149,"props":1679,"children":1680},{"class":1151,"line":1179},[1681],{"type":51,"tag":1149,"props":1682,"children":1683},{},[1684],{"type":57,"value":1685},"    \"gen_trt_engine.batch_size\": -1,\n",{"type":51,"tag":1149,"props":1687,"children":1688},{"class":1151,"line":1235},[1689],{"type":51,"tag":1149,"props":1690,"children":1691},{},[1692],{"type":57,"value":1693},"    \"gen_trt_engine.tensorrt.data_type\": \"fp16\",\n",{"type":51,"tag":1149,"props":1695,"children":1696},{"class":1151,"line":1244},[1697],{"type":51,"tag":1149,"props":1698,"children":1699},{},[1700],{"type":57,"value":1701},"    \"gen_trt_engine.tensorrt.min_batch_size\": 1,\n",{"type":51,"tag":1149,"props":1703,"children":1704},{"class":1151,"line":1314},[1705],{"type":51,"tag":1149,"props":1706,"children":1707},{},[1708],{"type":57,"value":1709},"    \"gen_trt_engine.tensorrt.opt_batch_size\": 1,\n",{"type":51,"tag":1149,"props":1711,"children":1712},{"class":1151,"line":1322},[1713],{"type":51,"tag":1149,"props":1714,"children":1715},{},[1716],{"type":57,"value":1717},"    \"gen_trt_engine.tensorrt.max_batch_size\": 16,\n",{"type":51,"tag":1149,"props":1719,"children":1721},{"class":1151,"line":1720},9,[1722],{"type":51,"tag":1149,"props":1723,"children":1724},{},[1725],{"type":57,"value":1250},{"type":51,"tag":154,"props":1727,"children":1729},{"id":1728},"eval-dataset",[1730],{"type":57,"value":1731},"Eval Dataset",{"type":51,"tag":60,"props":1733,"children":1734},{},[1735,1737,1742],{"type":57,"value":1736},"Optional for training. If provided, validation metrics are computed at validation intervals. Required for ",{"type":51,"tag":71,"props":1738,"children":1740},{"className":1739},[],[1741],{"type":57,"value":129},{"type":57,"value":152},{"type":51,"tag":154,"props":1744,"children":1746},{"id":1745},"deploy-workflow",[1747],{"type":57,"value":1748},"Deploy Workflow",{"type":51,"tag":60,"props":1750,"children":1751},{},[1752,1754,1759,1761,1767,1769,1775,1777,1783],{"type":57,"value":1753},"The skill exposes ",{"type":51,"tag":71,"props":1755,"children":1757},{"className":1756},[],[1758],{"type":57,"value":150},{"type":57,"value":1760}," as the deploy action. In generated SDK runners, use ",{"type":51,"tag":71,"props":1762,"children":1764},{"className":1763},[],[1765],{"type":57,"value":1766},"model_info[\"actions\"][\"gen_trt_engine\"]",{"type":57,"value":1768}," and run it in the TAO Deploy image, not the PyTorch training image. The in-container command is ",{"type":51,"tag":71,"props":1770,"children":1772},{"className":1771},[],[1773],{"type":57,"value":1774},"clip gen_trt_engine -e {config_path}",{"type":57,"value":1776},"; direct TAO Launcher usage spells the same action as ",{"type":51,"tag":71,"props":1778,"children":1780},{"className":1779},[],[1781],{"type":57,"value":1782},"tao deploy clip gen_trt_engine -e \u002Fpath\u002Fto\u002Fspec.yaml",{"type":57,"value":152},{"type":51,"tag":60,"props":1785,"children":1786},{},[1787,1789,1795,1796,1802,1804,1809,1811,1817,1819,1825,1827,1832,1834,1840,1842,1847,1849,1854,1856,1861,1863,1869,1871,1876],{"type":57,"value":1788},"TAO Deploy inference can discover combined engines, paired separate engines, or single-pillar ",{"type":51,"tag":71,"props":1790,"children":1792},{"className":1791},[],[1793],{"type":57,"value":1794},"_vision.engine",{"type":57,"value":196},{"type":51,"tag":71,"props":1797,"children":1799},{"className":1798},[],[1800],{"type":57,"value":1801},"_text.engine",{"type":57,"value":1803}," files. For full TensorRT retrieval evaluation or image+text TensorRT inference, export with ",{"type":51,"tag":71,"props":1805,"children":1807},{"className":1806},[],[1808],{"type":57,"value":1529},{"type":57,"value":1810}," and run ",{"type":51,"tag":71,"props":1812,"children":1814},{"className":1813},[],[1815],{"type":57,"value":1816},"clip gen_trt_engine",{"type":57,"value":1818}," twice: build ",{"type":51,"tag":71,"props":1820,"children":1822},{"className":1821},[],[1823],{"type":57,"value":1824},"clip_model_vision.onnx",{"type":57,"value":1826}," to an engine ending in ",{"type":51,"tag":71,"props":1828,"children":1830},{"className":1829},[],[1831],{"type":57,"value":1794},{"type":57,"value":1833},", then build ",{"type":51,"tag":71,"props":1835,"children":1837},{"className":1836},[],[1838],{"type":57,"value":1839},"clip_model_text.onnx",{"type":57,"value":1841}," to the matching ",{"type":51,"tag":71,"props":1843,"children":1845},{"className":1844},[],[1846],{"type":57,"value":1801},{"type":57,"value":1848}," in the same directory. For image-only TensorRT inference, building only the ",{"type":51,"tag":71,"props":1850,"children":1852},{"className":1851},[],[1853],{"type":57,"value":1794},{"type":57,"value":1855}," is sufficient and ",{"type":51,"tag":71,"props":1857,"children":1859},{"className":1858},[],[1860],{"type":57,"value":916},{"type":57,"value":1862}," must be ",{"type":51,"tag":71,"props":1864,"children":1866},{"className":1865},[],[1867],{"type":57,"value":1868},"null",{"type":57,"value":1870},". TensorRT ",{"type":51,"tag":71,"props":1872,"children":1874},{"className":1873},[],[1875],{"type":57,"value":129},{"type":57,"value":1877}," and text inference require a text-capable engine; if only a vision engine is present, deploy evaluation fails because text embeddings cannot be extracted.",{"type":51,"tag":60,"props":1879,"children":1880},{},[1881,1883,1888,1890,1896,1898,1904,1905,1911],{"type":57,"value":1882},"Use ",{"type":51,"tag":71,"props":1884,"children":1886},{"className":1885},[],[1887],{"type":57,"value":1382},{"type":57,"value":1889}," for TensorRT evaluation and ",{"type":51,"tag":71,"props":1891,"children":1893},{"className":1892},[],[1894],{"type":57,"value":1895},"inference.trt_engine",{"type":57,"value":1897}," for TensorRT embedding extraction. These TensorRT paths also run in the TAO Deploy image. Direct TAO Launcher usage spells these as ",{"type":51,"tag":71,"props":1899,"children":1901},{"className":1900},[],[1902],{"type":57,"value":1903},"tao deploy clip evaluate",{"type":57,"value":234},{"type":51,"tag":71,"props":1906,"children":1908},{"className":1907},[],[1909],{"type":57,"value":1910},"tao deploy clip inference",{"type":57,"value":152},{"type":51,"tag":154,"props":1913,"children":1915},{"id":1914},"important-parameters",[1916],{"type":57,"value":1917},"Important Parameters",{"type":51,"tag":492,"props":1919,"children":1920},{},[1921,1938,1959,1969,1979,1989,1999,2023,2048],{"type":51,"tag":496,"props":1922,"children":1923},{},[1924,1929,1931,1936],{"type":51,"tag":500,"props":1925,"children":1926},{},[1927],{"type":57,"value":1928},"model.type",{"type":57,"value":1930},": Backbone family and resolution. Use a TAO-registered CLIP model ID such as ",{"type":51,"tag":71,"props":1932,"children":1934},{"className":1933},[],[1935],{"type":57,"value":552},{"type":57,"value":1937},". Prefer the listed OpenCLIP \u002F NV-CLIP IDs for AutoML smoke tests because the current TAO container registry routes them through the supported augmentation adapter.",{"type":51,"tag":496,"props":1939,"children":1940},{},[1941,1945,1947,1952,1953,1958],{"type":51,"tag":500,"props":1942,"children":1943},{},[1944],{"type":57,"value":678},{"type":57,"value":1946},": Required for Radio-CLIP. Set to ",{"type":51,"tag":71,"props":1948,"children":1950},{"className":1949},[],[1951],{"type":57,"value":686},{"type":57,"value":688},{"type":51,"tag":71,"props":1954,"children":1956},{"className":1955},[],[1957],{"type":57,"value":54},{"type":57,"value":152},{"type":51,"tag":496,"props":1960,"children":1961},{},[1962,1967],{"type":51,"tag":500,"props":1963,"children":1964},{},[1965],{"type":57,"value":1966},"model.image_size",{"type":57,"value":1968},": Training transform image resolution. Keep it aligned with the selected fixed-resolution backbone.",{"type":51,"tag":496,"props":1970,"children":1971},{},[1972,1977],{"type":51,"tag":500,"props":1973,"children":1974},{},[1975],{"type":57,"value":1976},"train.num_epochs",{"type":57,"value":1978},": CLIP fine-tuning often converges quickly. Start with 10-20 epochs for domain adaptation, then increase only if validation loss is still improving.",{"type":51,"tag":496,"props":1980,"children":1981},{},[1982,1987],{"type":51,"tag":500,"props":1983,"children":1984},{},[1985],{"type":57,"value":1986},"train.optim.vision_lr \u002F train.optim.text_lr",{"type":57,"value":1988},": Learning rates for the two encoders. CLIP is sensitive to high learning rates; reduce both if loss is unstable.",{"type":51,"tag":496,"props":1990,"children":1991},{},[1992,1997],{"type":51,"tag":500,"props":1993,"children":1994},{},[1995],{"type":57,"value":1996},"model.freeze_vision_encoder \u002F model.freeze_text_encoder",{"type":57,"value":1998},": Defaults are false. Freezing one encoder can help when the dataset is small or only one modality needs adaptation.",{"type":51,"tag":496,"props":2000,"children":2001},{},[2002,2007,2009,2014,2016,2021],{"type":51,"tag":500,"props":2003,"children":2004},{},[2005],{"type":57,"value":2006},"train.loss_type",{"type":57,"value":2008},": ",{"type":51,"tag":71,"props":2010,"children":2012},{"className":2011},[],[2013],{"type":57,"value":686},{"type":57,"value":2015}," is recommended for SigLIP2 and Radio-CLIP. Use ",{"type":51,"tag":71,"props":2017,"children":2019},{"className":2018},[],[2020],{"type":57,"value":54},{"type":57,"value":2022}," for CLIP-style softmax loss.",{"type":51,"tag":496,"props":2024,"children":2025},{},[2026,2031,2032,2038,2040,2046],{"type":51,"tag":500,"props":2027,"children":2028},{},[2029],{"type":57,"value":2030},"export.encoder_type",{"type":57,"value":2008},{"type":51,"tag":71,"props":2033,"children":2035},{"className":2034},[],[2036],{"type":57,"value":2037},"combined",{"type":57,"value":2039}," exports one ONNX graph. ",{"type":51,"tag":71,"props":2041,"children":2043},{"className":2042},[],[2044],{"type":57,"value":2045},"separate",{"type":57,"value":2047}," exports independent vision and text graphs.",{"type":51,"tag":496,"props":2049,"children":2050},{},[2051,2056,2058,2064,2065,2071],{"type":51,"tag":500,"props":2052,"children":2053},{},[2054],{"type":57,"value":2055},"gen_trt_engine.tensorrt.data_type",{"type":57,"value":2057},": TensorRT deployment supports ",{"type":51,"tag":71,"props":2059,"children":2061},{"className":2060},[],[2062],{"type":57,"value":2063},"fp16",{"type":57,"value":234},{"type":51,"tag":71,"props":2066,"children":2068},{"className":2067},[],[2069],{"type":57,"value":2070},"fp32",{"type":57,"value":152},{"type":51,"tag":154,"props":2073,"children":2075},{"id":2074},"hardware",[2076],{"type":57,"value":2077},"Hardware",{"type":51,"tag":60,"props":2079,"children":2080},{},[2081],{"type":57,"value":2082},"Single-GPU training works for small datasets. Use 4+ GPUs for datasets with more than 100k images or large backbones. Use 16GB+ VRAM per GPU for small\u002Ffixed-resolution runs and larger GPUs for Radio-CLIP or high-resolution OpenCLIP variants.",{"type":51,"tag":154,"props":2084,"children":2086},{"id":2085},"error-patterns",[2087],{"type":57,"value":2088},"Error Patterns",{"type":51,"tag":60,"props":2090,"children":2091},{},[2092,2094,2100,2102,2108],{"type":57,"value":2093},"See ",{"type":51,"tag":71,"props":2095,"children":2097},{"className":2096},[],[2098],{"type":57,"value":2099},"references\u002Ferror-patterns.md",{"type":57,"value":2101}," for the full list of CLIP error symptoms and fixes (CUDA OOM, NaN loss, retrieval quality, dataset format\u002Fsize, Radio-CLIP and model-ID validation, ONNX external data, TensorRT shape mismatch, PyTorch 2.6 checkpoint load, null-checkpoint inference, TensorRT text\u002Fretrieval failures, ",{"type":51,"tag":71,"props":2103,"children":2105},{"className":2104},[],[2106],{"type":57,"value":2107},"attention_mask",{"type":57,"value":2109}," handling, and spec\u002Fschema merge errors).",{"type":51,"tag":154,"props":2111,"children":2113},{"id":2112},"spec-param-parent-model-inference",[2114],{"type":57,"value":2115},"Spec Param \u002F Parent Model Inference",{"type":51,"tag":60,"props":2117,"children":2118},{},[2119,2120,2126,2128,2134,2136,2142,2144,2150],{"type":57,"value":2093},{"type":51,"tag":71,"props":2121,"children":2123},{"className":2122},[],[2124],{"type":57,"value":2125},"references\u002Fspec-param-inference.md",{"type":57,"value":2127}," for the model-specific inference mappings (the full ",{"type":51,"tag":71,"props":2129,"children":2131},{"className":2130},[],[2132],{"type":57,"value":2133},"clip.config.json",{"type":57,"value":2135}," action\u002Fspec-field\u002Finference-function table) that generated runners apply with SDK helpers before ",{"type":51,"tag":71,"props":2137,"children":2139},{"className":2138},[],[2140],{"type":57,"value":2141},"create_job()",{"type":57,"value":2143},", plus ",{"type":51,"tag":71,"props":2145,"children":2147},{"className":2146},[],[2148],{"type":57,"value":2149},"parent_job_id",{"type":57,"value":2151}," resolution rules.",{"type":51,"tag":154,"props":2153,"children":2154},{"id":46},[2155],{"type":57,"value":2156},"Deployment",{"type":51,"tag":492,"props":2158,"children":2159},{},[2160],{"type":51,"tag":496,"props":2161,"children":2162},{},[2163],{"type":51,"tag":2164,"props":2165,"children":2166},"a",{"href":466},[2167],{"type":57,"value":2168},"tao-deploy-clip",{"type":51,"tag":2170,"props":2171,"children":2172},"style",{},[2173],{"type":57,"value":2174},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":2176,"total":2278},[2177,2194,2206,2220,2232,2249,2264],{"slug":2178,"name":2178,"fn":2179,"description":2180,"org":2181,"tags":2182,"stars":23,"repoUrl":24,"updatedAt":2193},"accelerated-computing-cudf","accelerate data processing with cuDF","Official NVIDIA-authored guidance for NVIDIA cuDF GPU DataFrames, pandas acceleration, dask-cuDF, ETL, joins, groupby, CSV\u002FParquet I\u002FO, nullable semantics, and multi-GPU DataFrame workloads.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2183,2186,2189,2190],{"name":2184,"slug":2185,"type":15},"Data Analysis","data-analysis",{"name":2187,"slug":2188,"type":15},"Data Engineering","data-engineering",{"name":9,"slug":8,"type":15},{"name":2191,"slug":2192,"type":15},"Performance","performance","2026-07-14T05:28:43.176466",{"slug":2195,"name":2195,"fn":2196,"description":2197,"org":2198,"tags":2199,"stars":23,"repoUrl":24,"updatedAt":2205},"aiq-deploy","deploy and manage NVIDIA AI-Q infrastructure","Use when asked to install, deploy, run, validate, troubleshoot, or stop NVIDIA AI-Q Blueprint infrastructure.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2200,2201,2204],{"name":2156,"slug":46,"type":15},{"name":2202,"slug":2203,"type":15},"Infrastructure","infrastructure",{"name":9,"slug":8,"type":15},"2026-07-14T05:29:06.667109",{"slug":2207,"name":2207,"fn":2208,"description":2209,"org":2210,"tags":2211,"stars":23,"repoUrl":24,"updatedAt":2219},"aiq-research","conduct deep research with AI-Q","Use when asked to run deep research or AI-Q research through a reachable NVIDIA AI-Q Blueprint backend.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2212,2215,2216],{"name":2213,"slug":2214,"type":15},"Agents","agents",{"name":9,"slug":8,"type":15},{"name":2217,"slug":2218,"type":15},"Research","research","2026-07-14T05:28:06.816956",{"slug":2221,"name":2221,"fn":2222,"description":2223,"org":2224,"tags":2225,"stars":23,"repoUrl":24,"updatedAt":2231},"amc-run-sample-calibration","run AMC sample dataset calibration","Run end-to-end calibration on the shipped sample dataset (sdg_08_2_sample_data_010926.zip) against a running AMC microservice. Use when user says 'test sample dataset', 'run sample calibration', 'verify AMC install', or 'launch and test'.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2226,2227,2228],{"name":2184,"slug":2185,"type":15},{"name":9,"slug":8,"type":15},{"name":2229,"slug":2230,"type":15},"Testing","testing","2026-07-17T05:29:03.913266",{"slug":2233,"name":2233,"fn":2234,"description":2235,"org":2236,"tags":2237,"stars":23,"repoUrl":24,"updatedAt":2248},"amc-run-video-calibration","calibrate video datasets with AutoMagicCalib","Calibrate a new dataset from pre-recorded video files via the AutoMagicCalib REST API. Use when user has local MP4s and says 'calibrate my videos', 'run AMC on these videos', or similar. For RTSP\u002Flive streams, use amc-run-rtsp-calibration instead.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2238,2241,2244,2245],{"name":2239,"slug":2240,"type":15},"Automation","automation",{"name":2242,"slug":2243,"type":15},"Imaging","imaging",{"name":9,"slug":8,"type":15},{"name":2246,"slug":2247,"type":15},"Video","video","2026-07-17T05:28:53.905004",{"slug":2250,"name":2250,"fn":2251,"description":2252,"org":2253,"tags":2254,"stars":23,"repoUrl":24,"updatedAt":2263},"amc-setup-calibration-stack","deploy AutoMagicCalib microservice with Docker","Launch AutoMagicCalib microservice and web UI from NGC release images via Docker Compose. Use when user says 'deploy auto calibration', 'launch auto calibration', 'launch AMC', 'start MS+UI', or 'set up auto-magic-calib'. Requires NGC API key.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2255,2256,2259,2260],{"name":2156,"slug":46,"type":15},{"name":2257,"slug":2258,"type":15},"Docker","docker",{"name":9,"slug":8,"type":15},{"name":2261,"slug":2262,"type":15},"Operations","operations","2026-07-17T05:28:56.913999",{"slug":2265,"name":2265,"fn":2266,"description":2267,"org":2268,"tags":2269,"stars":23,"repoUrl":24,"updatedAt":2277},"cudaq-guide","develop quantum applications with CUDA-Q","CUDA-Q onboarding guide for installation, test programs, GPU simulation, QPU hardware, and quantum applications.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2270,2271,2274],{"name":9,"slug":8,"type":15},{"name":2272,"slug":2273,"type":15},"Quantum Computing","quantum-computing",{"name":2275,"slug":2276,"type":15},"Simulation","simulation","2026-07-14T05:26:58.898253",305,{"items":2280,"total":2430},[2281,2299,2314,2325,2337,2351,2364,2378,2389,2398,2412,2421],{"slug":2282,"name":2282,"fn":2283,"description":2284,"org":2285,"tags":2286,"stars":2296,"repoUrl":2297,"updatedAt":2298},"nemoclaw-user-guide","retrieve NemoClaw documentation and configuration","Guides human users' AI agents to the NemoClaw docs MCP server and canonical Fern documentation in Markdown form. Use when users ask how to install, configure, operate, troubleshoot, secure, or learn NemoClaw with an AI coding assistant. Trigger keywords - nemoclaw docs, use nemoclaw with ai agent, nemoclaw mcp docs, nemoclaw install help, nemoclaw quickstart, nemoclaw markdown docs, llms.txt, agent skills.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2287,2290,2293],{"name":2288,"slug":2289,"type":15},"Documentation","documentation",{"name":2291,"slug":2292,"type":15},"MCP","mcp",{"name":2294,"slug":2295,"type":15},"Search","search",21777,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FNemoClaw","2026-07-20T06:00:01.461044",{"slug":2300,"name":2300,"fn":2301,"description":2302,"org":2303,"tags":2304,"stars":2311,"repoUrl":2312,"updatedAt":2313},"mcore-build-and-dependency","manage Megatron-LM development environments","Container-based dev environment setup and dependency management for Megatron-LM. Covers acquiring and launching the CI container, uv package management, and updating uv.lock.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2305,2308,2309],{"name":2306,"slug":2307,"type":15},"Containers","containers",{"name":2156,"slug":46,"type":15},{"name":2310,"slug":1142,"type":15},"Python",17049,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FMegatron-LM","2026-07-27T06:06:11.249662",{"slug":2315,"name":2315,"fn":2316,"description":2317,"org":2318,"tags":2319,"stars":2311,"repoUrl":2312,"updatedAt":2324},"mcore-bump-base-image","update NVIDIA PyTorch base images","Bump the NVIDIA PyTorch base image (`nvcr.io\u002Fnvidia\u002Fpytorch:YY.MM-py3`) used by Megatron-LM CI. Covers the two pin sites (GitHub CI in `docker\u002F.ngc_version.dev` and GitLab CI in `.gitlab\u002Fstages\u002F01.build.yml`), the post-bump CI loop (re-run functional tests, refresh golden values, mark broken tests), and the gotchas that bit PRs",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2320,2323],{"name":2321,"slug":2322,"type":15},"CI\u002FCD","ci-cd",{"name":2156,"slug":46,"type":15},"2026-07-14T05:25:59.97109",{"slug":2326,"name":2326,"fn":2327,"description":2328,"org":2329,"tags":2330,"stars":2311,"repoUrl":2312,"updatedAt":2336},"mcore-cicd","manage CI\u002FCD pipelines for Megatron-LM","CI\u002FCD reference for Megatron-LM. Covers CI pipeline structure, PR scope labels, triggering internal GitLab CI (which force-pushes the current branch to a pull-request\u002FBRANCH ref — always dry-run and verify the destination first; never run against shared or protected branches), and CI failure investigation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2331,2332,2333],{"name":2321,"slug":2322,"type":15},{"name":2156,"slug":46,"type":15},{"name":2334,"slug":2335,"type":15},"GitHub","github","2026-07-27T06:06:12.278222",{"slug":2338,"name":2338,"fn":2339,"description":2340,"org":2341,"tags":2342,"stars":2311,"repoUrl":2312,"updatedAt":2350},"mcore-create-issue","investigate CI failures and create issues","Investigate a failing GitHub Actions run or job and create a GitHub issue for the failure.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2343,2346,2347],{"name":2344,"slug":2345,"type":15},"Debugging","debugging",{"name":2334,"slug":2335,"type":15},{"name":2348,"slug":2349,"type":15},"Triage","triage","2026-07-14T05:25:57.442089",{"slug":2352,"name":2352,"fn":2353,"description":2354,"org":2355,"tags":2356,"stars":2311,"repoUrl":2312,"updatedAt":2363},"mcore-linting-and-formatting","lint and format Megatron-LM code","Linting and formatting for Megatron-LM. Covers running autoformat.sh, tools (ruff, black, isort, pylint, mypy), and code style rules.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2357,2360],{"name":2358,"slug":2359,"type":15},"Best Practices","best-practices",{"name":2361,"slug":2362,"type":15},"Code Analysis","code-analysis","2026-07-14T05:25:56.18433",{"slug":2365,"name":2365,"fn":2366,"description":2367,"org":2368,"tags":2369,"stars":2311,"repoUrl":2312,"updatedAt":2377},"mcore-migrate-gpt-to-hybrid","migrate Megatron-LM models to HybridModel","Migration guide for moving Megatron Core GPTModel checkpoints, model providers, training commands, and layer mappings to HybridModel.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2370,2373,2376],{"name":2371,"slug":2372,"type":15},"Machine Learning","machine-learning",{"name":2374,"slug":2375,"type":15},"Migration","migration",{"name":9,"slug":8,"type":15},"2026-07-17T06:07:11.777011",{"slug":2379,"name":2379,"fn":2380,"description":2381,"org":2382,"tags":2383,"stars":2311,"repoUrl":2312,"updatedAt":2388},"mcore-onboard-gb200-1node-tests","onboard functional tests for GB200","Onboard 1-node GitHub MR functional tests for GB200 from existing mr-scoped 2-node tests.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2384,2387],{"name":2385,"slug":2386,"type":15},"QA","qa",{"name":2229,"slug":2230,"type":15},"2026-07-14T05:25:53.673039",{"slug":2390,"name":2390,"fn":2391,"description":2392,"org":2393,"tags":2394,"stars":2311,"repoUrl":2312,"updatedAt":2397},"mcore-run-on-slurm","launch distributed training jobs on SLURM","How to launch distributed Megatron-LM training jobs on a SLURM cluster. Covers a minimal sbatch skeleton, environment-variable setup for torch.distributed.run, CUDA_DEVICE_MAX_CONNECTIONS rules across hardware and parallelism modes, container conventions, monitoring, and per-rank failure diagnosis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2395,2396],{"name":2156,"slug":46,"type":15},{"name":2202,"slug":2203,"type":15},"2026-07-14T05:25:49.362534",{"slug":2399,"name":2399,"fn":2400,"description":2401,"org":2402,"tags":2403,"stars":2311,"repoUrl":2312,"updatedAt":2411},"mcore-split-pr","split pull requests to reduce review load","Split a PR into multiple PRs to reduce the number of required CODEOWNERS reviewer groups.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2404,2407,2408],{"name":2405,"slug":2406,"type":15},"Code Review","code-review",{"name":2334,"slug":2335,"type":15},{"name":2409,"slug":2410,"type":15},"Pull Requests","pull-requests","2026-07-14T05:26:01.226578",{"slug":2413,"name":2413,"fn":2414,"description":2415,"org":2416,"tags":2417,"stars":2311,"repoUrl":2312,"updatedAt":2420},"mcore-testing","run and manage Megatron-LM tests","Test system for Megatron-LM. Covers test layout, recipe YAML structure, adding and running unit and functional tests, golden values, marker filters, and CI parity.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2418,2419],{"name":2385,"slug":2386,"type":15},{"name":2229,"slug":2230,"type":15},"2026-07-14T05:25:54.928983",{"slug":2422,"name":2422,"fn":2423,"description":2424,"org":2425,"tags":2426,"stars":2311,"repoUrl":2312,"updatedAt":2429},"nightly-sync","manage nightly main-to-dev sync workflows","Domain knowledge for the nightly main-to-dev sync workflow. Covers merge strategy, CI architecture, failure investigation, and known issues.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2427,2428],{"name":2239,"slug":2240,"type":15},{"name":2321,"slug":2322,"type":15},"2026-07-30T05:29:03.275638",496]