[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-nvidia-nemo-evaluator-plugin":3,"mdc-bsntxh-key":34,"related-org-nvidia-nemo-evaluator-plugin":1136,"related-repo-nvidia-nemo-evaluator-plugin":1296},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":23,"repoUrl":24,"updatedAt":25,"license":26,"forks":27,"topics":28,"repo":29,"sourceUrl":32,"mdContent":33},"nemo-evaluator-plugin","manage NeMo evaluator jobs","Use when working on the Evaluator plugin CLI, jobs, SDK-backed specs, metric types, or plugin-owned Evaluator skills.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},"nvidia","NVIDIA","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fnvidia.png",[12,16,19,20],{"name":13,"slug":14,"type":15},"LLM","llm","tag",{"name":17,"slug":18,"type":15},"Evals","evals",{"name":9,"slug":8,"type":15},{"name":21,"slug":22,"type":15},"SDK","sdk",2473,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fskills","2026-07-14T05:27:33.014436","Apache-2.0",281,[],{"repoUrl":24,"stars":23,"forks":27,"topics":30,"description":31},[],"AI agent skills published by NVIDIA","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fskills\u002Ftree\u002FHEAD\u002Fskills\u002Fnemo-evaluator-plugin","---\nname: nemo-evaluator-plugin\ndescription: Use when working on the Evaluator plugin CLI, jobs, SDK-backed specs, metric types, or plugin-owned Evaluator skills.\nmetadata:\n  owner: nemo-platform\n  maturity: active\nlicense: Apache-2.0\n---\n\n# Evaluator Plugin\n\nUse this skill for evaluation tasks against a running NeMo Platform server. The plugin-backed CLI interface is `nemo evaluator`; the legacy generated `nemo evaluation` API command group is not the target surface for new guidance.\n\n## CLI Interface\n\n### Prerequisites\n\n- all commands in this file assume that the shell's working dir is at the root of the Nvidia-NeMo\u002Fnemo-platform repo\n- activate the Python virtual environment before invoking the `nemo` CLI: `source .venv\u002Fbin\u002Factivate`\n\nCheck plugin status from the CLI:\n\n```bash\nnemo evaluator info\n```\n\n## Metric Types\n\n### Explore Available Metrics\n\nTo view available metric names, run:\n\n```bash\nnemo evaluator metric-types\n```\n\nTo view a specific metric schema, pass a metric name from the `metric_types` list above:\n\n```bash\nnemo evaluator metric-types \u003Cmetric-name>\n```\n\nInspect all the registered metric schema contracts:\n\n```bash\nnemo evaluator evaluate explain\n```\n\n> Note: use `nemo evaluator evaluate explain` as the source of truth for the current plugin input schema. It will return a large json schema response, so strongly prefer `nemo evaluator metric-types` when you only need metric names and corresponding schemas.\n\n## Evaluation Spec\n\nEvaluation spec is a payload that is provided to CLI as an input to execute evaluation.\n\nAt a high level, a spec describes:\n\n- `metrics`: bundled Evaluator SDK metric configurations\n- `dataset`: inline rows to evaluate or platform FilesetRef that contains the dataset\n- `params`: optional Evaluator SDK execution parameters\n- `target`: optional model or agent target for online evaluation\n\nSee the LLM-judge spec example at [assets\u002Fspecs\u002Fllm_as_judge.json](.\u002Fassets\u002Fspecs\u002Fllm_as_judge.json).\n\n### Metric Bundle Payloads\n\nThe checked-in [spec examples](.\u002Fassets\u002Fspecs) use bundled SDK metrics. The fields under `metrics[*].payload` are generated by `bundle_metric(metric, CloudpickleMetricBundlePackager())`.\n\nTo see the pattern for configuring a pre-defined SDK metric, for example `ExactMatchMetric`, and converting it into bundled metric JSON, inspect `build_metric_bundle_example()` in [generate_example_specs.py](.\u002Fscripts\u002Fgenerate_example_specs.py) and run:\n\n```bash\nuv run --frozen python skills\u002Fnemo-evaluator-plugin\u002Fscripts\u002Fgenerate_example_specs.py\n```\n\n## Run Evaluations\n\n### Run Using File Spec Reference\n\nWhen using the `nemo evaluator evaluate run` command, results are saved into local temporary directories and the link is printed to stdout.\nPrefer the `--spec-file` named argument over inline shell JSON because metric bundles include serialized payloads.\nExamples of various specs are provided in the [assets\u002Fspecs](.\u002Fassets\u002Fspecs\u002F) directory.\n\n#### Evaluate using `exact-match` metric\n\nSee the spec example at [assets\u002Fspecs\u002Fexact_match_metric.json](.\u002Fassets\u002Fspecs\u002Fexact_match_metric.json).\n\n```bash\nnemo evaluator evaluate run --spec-file skills\u002Fnemo-evaluator-plugin\u002Fassets\u002Fspecs\u002Fexact_match_metric.json\n```\n\n#### Evaluate using a benchmark metric set\n\n```bash\nnemo evaluator evaluate run --spec-file skills\u002Fnemo-evaluator-plugin\u002Fassets\u002Fspecs\u002Fexact_match_benchmark.json\n```\n\n#### Evaluate using `LLM-Judge` metric\n\nUses an LLM to score responses. See the spec example at [assets\u002Fspecs\u002Fllm_as_judge.json](.\u002Fassets\u002Fspecs\u002Fllm_as_judge.json).\n\n```bash\nnemo evaluator evaluate run --spec-file skills\u002Fnemo-evaluator-plugin\u002Fassets\u002Fspecs\u002Fllm_as_judge.json\n```\n\n### Run Evaluation As A Durable Job\n\nUse the `nemo evaluator evaluate submit` command to create a durable evaluation job. The response of this command returns a job handler object instead of the evaluation result.\n\n```bash\nnemo evaluator evaluate submit \\\n  --spec-file skills\u002Fnemo-evaluator-plugin\u002Fassets\u002Fspecs\u002Fexact_match_metric.json\n```\n\nThe submit response includes the generated job's `name` field, for example `nemo-evaluator-zlhn1ecd`. Wait for the job to complete, then list and download the job results.\n\n```bash\nnemo jobs get-status \u003Cjob-name>\nnemo jobs get \u003Cjob-name>\nnemo jobs results list \u003Cjob-name>\nnemo jobs results download aggregate-scores --job \u003Cjob-name> --output-file aggregate-scores.json\nnemo jobs results download row-scores --job \u003Cjob-name> --output-file row-scores.jsonl\n```\n\n## Python SDK Interface\n\nEvaluator Python SDK client is exposed as `evaluator` variable on `NeMoPlatform` instance:\n\n```python\nfrom nemo_platform import NeMoPlatform\n\nplatform_client = NeMoPlatform(base_url=\"http:\u002F\u002Flocalhost:8080\")\nstatus = platform_client.evaluator.plugin_status()\n```\n\nSee examples of using the plugin SDK interface in [plugin_sdk_examples.py](.\u002Fassets\u002Fexamples\u002Fplugin_sdk_examples.py).\n\n## Security\nMake sure not to print any secrets to stdout since this can be collected as logs\n\n## Additional Resources\n\nFor LLM-judge setup notes, see [LLM Judge Notes](references\u002Fllm-judge.md).\n\nFor evaluator API key auth, see [Evaluator API Auth](references\u002Fapi-auth.md).\n\nFor local and cluster troubleshooting, see [Evaluation Troubleshooting](references\u002Ftroubleshooting.md).\n",{"data":35,"body":39},{"name":4,"description":6,"metadata":36,"license":26},{"owner":37,"maturity":38},"nemo-platform","active",{"type":40,"children":41},"root",[42,51,74,81,88,117,122,155,161,167,172,195,208,253,258,286,311,317,322,327,374,388,394,422,451,486,492,498,527,542,554,590,596,631,643,653,688,694,707,752,773,996,1002,1023,1065,1077,1083,1088,1094,1106,1118,1130],{"type":43,"tag":44,"props":45,"children":47},"element","h1",{"id":46},"evaluator-plugin",[48],{"type":49,"value":50},"text","Evaluator Plugin",{"type":43,"tag":52,"props":53,"children":54},"p",{},[55,57,64,66,72],{"type":49,"value":56},"Use this skill for evaluation tasks against a running NeMo Platform server. The plugin-backed CLI interface is ",{"type":43,"tag":58,"props":59,"children":61},"code",{"className":60},[],[62],{"type":49,"value":63},"nemo evaluator",{"type":49,"value":65},"; the legacy generated ",{"type":43,"tag":58,"props":67,"children":69},{"className":68},[],[70],{"type":49,"value":71},"nemo evaluation",{"type":49,"value":73}," API command group is not the target surface for new guidance.",{"type":43,"tag":75,"props":76,"children":78},"h2",{"id":77},"cli-interface",[79],{"type":49,"value":80},"CLI Interface",{"type":43,"tag":82,"props":83,"children":85},"h3",{"id":84},"prerequisites",[86],{"type":49,"value":87},"Prerequisites",{"type":43,"tag":89,"props":90,"children":91},"ul",{},[92,98],{"type":43,"tag":93,"props":94,"children":95},"li",{},[96],{"type":49,"value":97},"all commands in this file assume that the shell's working dir is at the root of the Nvidia-NeMo\u002Fnemo-platform repo",{"type":43,"tag":93,"props":99,"children":100},{},[101,103,109,111],{"type":49,"value":102},"activate the Python virtual environment before invoking the ",{"type":43,"tag":58,"props":104,"children":106},{"className":105},[],[107],{"type":49,"value":108},"nemo",{"type":49,"value":110}," CLI: ",{"type":43,"tag":58,"props":112,"children":114},{"className":113},[],[115],{"type":49,"value":116},"source .venv\u002Fbin\u002Factivate",{"type":43,"tag":52,"props":118,"children":119},{},[120],{"type":49,"value":121},"Check plugin status from the CLI:",{"type":43,"tag":123,"props":124,"children":129},"pre",{"className":125,"code":126,"language":127,"meta":128,"style":128},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","nemo evaluator info\n","bash","",[130],{"type":43,"tag":58,"props":131,"children":132},{"__ignoreMap":128},[133],{"type":43,"tag":134,"props":135,"children":138},"span",{"class":136,"line":137},"line",1,[139,144,150],{"type":43,"tag":134,"props":140,"children":142},{"style":141},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[143],{"type":49,"value":108},{"type":43,"tag":134,"props":145,"children":147},{"style":146},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[148],{"type":49,"value":149}," evaluator",{"type":43,"tag":134,"props":151,"children":152},{"style":146},[153],{"type":49,"value":154}," info\n",{"type":43,"tag":75,"props":156,"children":158},{"id":157},"metric-types",[159],{"type":49,"value":160},"Metric Types",{"type":43,"tag":82,"props":162,"children":164},{"id":163},"explore-available-metrics",[165],{"type":49,"value":166},"Explore Available Metrics",{"type":43,"tag":52,"props":168,"children":169},{},[170],{"type":49,"value":171},"To view available metric names, run:",{"type":43,"tag":123,"props":173,"children":175},{"className":125,"code":174,"language":127,"meta":128,"style":128},"nemo evaluator metric-types\n",[176],{"type":43,"tag":58,"props":177,"children":178},{"__ignoreMap":128},[179],{"type":43,"tag":134,"props":180,"children":181},{"class":136,"line":137},[182,186,190],{"type":43,"tag":134,"props":183,"children":184},{"style":141},[185],{"type":49,"value":108},{"type":43,"tag":134,"props":187,"children":188},{"style":146},[189],{"type":49,"value":149},{"type":43,"tag":134,"props":191,"children":192},{"style":146},[193],{"type":49,"value":194}," metric-types\n",{"type":43,"tag":52,"props":196,"children":197},{},[198,200,206],{"type":49,"value":199},"To view a specific metric schema, pass a metric name from the ",{"type":43,"tag":58,"props":201,"children":203},{"className":202},[],[204],{"type":49,"value":205},"metric_types",{"type":49,"value":207}," list above:",{"type":43,"tag":123,"props":209,"children":211},{"className":125,"code":210,"language":127,"meta":128,"style":128},"nemo evaluator metric-types \u003Cmetric-name>\n",[212],{"type":43,"tag":58,"props":213,"children":214},{"__ignoreMap":128},[215],{"type":43,"tag":134,"props":216,"children":217},{"class":136,"line":137},[218,222,226,231,237,242,248],{"type":43,"tag":134,"props":219,"children":220},{"style":141},[221],{"type":49,"value":108},{"type":43,"tag":134,"props":223,"children":224},{"style":146},[225],{"type":49,"value":149},{"type":43,"tag":134,"props":227,"children":228},{"style":146},[229],{"type":49,"value":230}," metric-types",{"type":43,"tag":134,"props":232,"children":234},{"style":233},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[235],{"type":49,"value":236}," \u003C",{"type":43,"tag":134,"props":238,"children":239},{"style":146},[240],{"type":49,"value":241},"metric-nam",{"type":43,"tag":134,"props":243,"children":245},{"style":244},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[246],{"type":49,"value":247},"e",{"type":43,"tag":134,"props":249,"children":250},{"style":233},[251],{"type":49,"value":252},">\n",{"type":43,"tag":52,"props":254,"children":255},{},[256],{"type":49,"value":257},"Inspect all the registered metric schema contracts:",{"type":43,"tag":123,"props":259,"children":261},{"className":125,"code":260,"language":127,"meta":128,"style":128},"nemo evaluator evaluate explain\n",[262],{"type":43,"tag":58,"props":263,"children":264},{"__ignoreMap":128},[265],{"type":43,"tag":134,"props":266,"children":267},{"class":136,"line":137},[268,272,276,281],{"type":43,"tag":134,"props":269,"children":270},{"style":141},[271],{"type":49,"value":108},{"type":43,"tag":134,"props":273,"children":274},{"style":146},[275],{"type":49,"value":149},{"type":43,"tag":134,"props":277,"children":278},{"style":146},[279],{"type":49,"value":280}," evaluate",{"type":43,"tag":134,"props":282,"children":283},{"style":146},[284],{"type":49,"value":285}," explain\n",{"type":43,"tag":287,"props":288,"children":289},"blockquote",{},[290],{"type":43,"tag":52,"props":291,"children":292},{},[293,295,301,303,309],{"type":49,"value":294},"Note: use ",{"type":43,"tag":58,"props":296,"children":298},{"className":297},[],[299],{"type":49,"value":300},"nemo evaluator evaluate explain",{"type":49,"value":302}," as the source of truth for the current plugin input schema. It will return a large json schema response, so strongly prefer ",{"type":43,"tag":58,"props":304,"children":306},{"className":305},[],[307],{"type":49,"value":308},"nemo evaluator metric-types",{"type":49,"value":310}," when you only need metric names and corresponding schemas.",{"type":43,"tag":75,"props":312,"children":314},{"id":313},"evaluation-spec",[315],{"type":49,"value":316},"Evaluation Spec",{"type":43,"tag":52,"props":318,"children":319},{},[320],{"type":49,"value":321},"Evaluation spec is a payload that is provided to CLI as an input to execute evaluation.",{"type":43,"tag":52,"props":323,"children":324},{},[325],{"type":49,"value":326},"At a high level, a spec describes:",{"type":43,"tag":89,"props":328,"children":329},{},[330,341,352,363],{"type":43,"tag":93,"props":331,"children":332},{},[333,339],{"type":43,"tag":58,"props":334,"children":336},{"className":335},[],[337],{"type":49,"value":338},"metrics",{"type":49,"value":340},": bundled Evaluator SDK metric configurations",{"type":43,"tag":93,"props":342,"children":343},{},[344,350],{"type":43,"tag":58,"props":345,"children":347},{"className":346},[],[348],{"type":49,"value":349},"dataset",{"type":49,"value":351},": inline rows to evaluate or platform FilesetRef that contains the dataset",{"type":43,"tag":93,"props":353,"children":354},{},[355,361],{"type":43,"tag":58,"props":356,"children":358},{"className":357},[],[359],{"type":49,"value":360},"params",{"type":49,"value":362},": optional Evaluator SDK execution parameters",{"type":43,"tag":93,"props":364,"children":365},{},[366,372],{"type":43,"tag":58,"props":367,"children":369},{"className":368},[],[370],{"type":49,"value":371},"target",{"type":49,"value":373},": optional model or agent target for online evaluation",{"type":43,"tag":52,"props":375,"children":376},{},[377,379,386],{"type":49,"value":378},"See the LLM-judge spec example at ",{"type":43,"tag":380,"props":381,"children":383},"a",{"href":382},".\u002Fassets\u002Fspecs\u002Fllm_as_judge.json",[384],{"type":49,"value":385},"assets\u002Fspecs\u002Fllm_as_judge.json",{"type":49,"value":387},".",{"type":43,"tag":82,"props":389,"children":391},{"id":390},"metric-bundle-payloads",[392],{"type":49,"value":393},"Metric Bundle Payloads",{"type":43,"tag":52,"props":395,"children":396},{},[397,399,405,407,413,415,421],{"type":49,"value":398},"The checked-in ",{"type":43,"tag":380,"props":400,"children":402},{"href":401},".\u002Fassets\u002Fspecs",[403],{"type":49,"value":404},"spec examples",{"type":49,"value":406}," use bundled SDK metrics. The fields under ",{"type":43,"tag":58,"props":408,"children":410},{"className":409},[],[411],{"type":49,"value":412},"metrics[*].payload",{"type":49,"value":414}," are generated by ",{"type":43,"tag":58,"props":416,"children":418},{"className":417},[],[419],{"type":49,"value":420},"bundle_metric(metric, CloudpickleMetricBundlePackager())",{"type":49,"value":387},{"type":43,"tag":52,"props":423,"children":424},{},[425,427,433,435,441,443,449],{"type":49,"value":426},"To see the pattern for configuring a pre-defined SDK metric, for example ",{"type":43,"tag":58,"props":428,"children":430},{"className":429},[],[431],{"type":49,"value":432},"ExactMatchMetric",{"type":49,"value":434},", and converting it into bundled metric JSON, inspect ",{"type":43,"tag":58,"props":436,"children":438},{"className":437},[],[439],{"type":49,"value":440},"build_metric_bundle_example()",{"type":49,"value":442}," in ",{"type":43,"tag":380,"props":444,"children":446},{"href":445},".\u002Fscripts\u002Fgenerate_example_specs.py",[447],{"type":49,"value":448},"generate_example_specs.py",{"type":49,"value":450}," and run:",{"type":43,"tag":123,"props":452,"children":454},{"className":125,"code":453,"language":127,"meta":128,"style":128},"uv run --frozen python skills\u002Fnemo-evaluator-plugin\u002Fscripts\u002Fgenerate_example_specs.py\n",[455],{"type":43,"tag":58,"props":456,"children":457},{"__ignoreMap":128},[458],{"type":43,"tag":134,"props":459,"children":460},{"class":136,"line":137},[461,466,471,476,481],{"type":43,"tag":134,"props":462,"children":463},{"style":141},[464],{"type":49,"value":465},"uv",{"type":43,"tag":134,"props":467,"children":468},{"style":146},[469],{"type":49,"value":470}," run",{"type":43,"tag":134,"props":472,"children":473},{"style":146},[474],{"type":49,"value":475}," --frozen",{"type":43,"tag":134,"props":477,"children":478},{"style":146},[479],{"type":49,"value":480}," python",{"type":43,"tag":134,"props":482,"children":483},{"style":146},[484],{"type":49,"value":485}," skills\u002Fnemo-evaluator-plugin\u002Fscripts\u002Fgenerate_example_specs.py\n",{"type":43,"tag":75,"props":487,"children":489},{"id":488},"run-evaluations",[490],{"type":49,"value":491},"Run Evaluations",{"type":43,"tag":82,"props":493,"children":495},{"id":494},"run-using-file-spec-reference",[496],{"type":49,"value":497},"Run Using File Spec Reference",{"type":43,"tag":52,"props":499,"children":500},{},[501,503,509,511,517,519,525],{"type":49,"value":502},"When using the ",{"type":43,"tag":58,"props":504,"children":506},{"className":505},[],[507],{"type":49,"value":508},"nemo evaluator evaluate run",{"type":49,"value":510}," command, results are saved into local temporary directories and the link is printed to stdout.\nPrefer the ",{"type":43,"tag":58,"props":512,"children":514},{"className":513},[],[515],{"type":49,"value":516},"--spec-file",{"type":49,"value":518}," named argument over inline shell JSON because metric bundles include serialized payloads.\nExamples of various specs are provided in the ",{"type":43,"tag":380,"props":520,"children":522},{"href":521},".\u002Fassets\u002Fspecs\u002F",[523],{"type":49,"value":524},"assets\u002Fspecs",{"type":49,"value":526}," directory.",{"type":43,"tag":528,"props":529,"children":531},"h4",{"id":530},"evaluate-using-exact-match-metric",[532,534,540],{"type":49,"value":533},"Evaluate using ",{"type":43,"tag":58,"props":535,"children":537},{"className":536},[],[538],{"type":49,"value":539},"exact-match",{"type":49,"value":541}," metric",{"type":43,"tag":52,"props":543,"children":544},{},[545,547,553],{"type":49,"value":546},"See the spec example at ",{"type":43,"tag":380,"props":548,"children":550},{"href":549},".\u002Fassets\u002Fspecs\u002Fexact_match_metric.json",[551],{"type":49,"value":552},"assets\u002Fspecs\u002Fexact_match_metric.json",{"type":49,"value":387},{"type":43,"tag":123,"props":555,"children":557},{"className":125,"code":556,"language":127,"meta":128,"style":128},"nemo evaluator evaluate run --spec-file skills\u002Fnemo-evaluator-plugin\u002Fassets\u002Fspecs\u002Fexact_match_metric.json\n",[558],{"type":43,"tag":58,"props":559,"children":560},{"__ignoreMap":128},[561],{"type":43,"tag":134,"props":562,"children":563},{"class":136,"line":137},[564,568,572,576,580,585],{"type":43,"tag":134,"props":565,"children":566},{"style":141},[567],{"type":49,"value":108},{"type":43,"tag":134,"props":569,"children":570},{"style":146},[571],{"type":49,"value":149},{"type":43,"tag":134,"props":573,"children":574},{"style":146},[575],{"type":49,"value":280},{"type":43,"tag":134,"props":577,"children":578},{"style":146},[579],{"type":49,"value":470},{"type":43,"tag":134,"props":581,"children":582},{"style":146},[583],{"type":49,"value":584}," --spec-file",{"type":43,"tag":134,"props":586,"children":587},{"style":146},[588],{"type":49,"value":589}," skills\u002Fnemo-evaluator-plugin\u002Fassets\u002Fspecs\u002Fexact_match_metric.json\n",{"type":43,"tag":528,"props":591,"children":593},{"id":592},"evaluate-using-a-benchmark-metric-set",[594],{"type":49,"value":595},"Evaluate using a benchmark metric set",{"type":43,"tag":123,"props":597,"children":599},{"className":125,"code":598,"language":127,"meta":128,"style":128},"nemo evaluator evaluate run --spec-file skills\u002Fnemo-evaluator-plugin\u002Fassets\u002Fspecs\u002Fexact_match_benchmark.json\n",[600],{"type":43,"tag":58,"props":601,"children":602},{"__ignoreMap":128},[603],{"type":43,"tag":134,"props":604,"children":605},{"class":136,"line":137},[606,610,614,618,622,626],{"type":43,"tag":134,"props":607,"children":608},{"style":141},[609],{"type":49,"value":108},{"type":43,"tag":134,"props":611,"children":612},{"style":146},[613],{"type":49,"value":149},{"type":43,"tag":134,"props":615,"children":616},{"style":146},[617],{"type":49,"value":280},{"type":43,"tag":134,"props":619,"children":620},{"style":146},[621],{"type":49,"value":470},{"type":43,"tag":134,"props":623,"children":624},{"style":146},[625],{"type":49,"value":584},{"type":43,"tag":134,"props":627,"children":628},{"style":146},[629],{"type":49,"value":630}," skills\u002Fnemo-evaluator-plugin\u002Fassets\u002Fspecs\u002Fexact_match_benchmark.json\n",{"type":43,"tag":528,"props":632,"children":634},{"id":633},"evaluate-using-llm-judge-metric",[635,636,642],{"type":49,"value":533},{"type":43,"tag":58,"props":637,"children":639},{"className":638},[],[640],{"type":49,"value":641},"LLM-Judge",{"type":49,"value":541},{"type":43,"tag":52,"props":644,"children":645},{},[646,648,652],{"type":49,"value":647},"Uses an LLM to score responses. See the spec example at ",{"type":43,"tag":380,"props":649,"children":650},{"href":382},[651],{"type":49,"value":385},{"type":49,"value":387},{"type":43,"tag":123,"props":654,"children":656},{"className":125,"code":655,"language":127,"meta":128,"style":128},"nemo evaluator evaluate run --spec-file skills\u002Fnemo-evaluator-plugin\u002Fassets\u002Fspecs\u002Fllm_as_judge.json\n",[657],{"type":43,"tag":58,"props":658,"children":659},{"__ignoreMap":128},[660],{"type":43,"tag":134,"props":661,"children":662},{"class":136,"line":137},[663,667,671,675,679,683],{"type":43,"tag":134,"props":664,"children":665},{"style":141},[666],{"type":49,"value":108},{"type":43,"tag":134,"props":668,"children":669},{"style":146},[670],{"type":49,"value":149},{"type":43,"tag":134,"props":672,"children":673},{"style":146},[674],{"type":49,"value":280},{"type":43,"tag":134,"props":676,"children":677},{"style":146},[678],{"type":49,"value":470},{"type":43,"tag":134,"props":680,"children":681},{"style":146},[682],{"type":49,"value":584},{"type":43,"tag":134,"props":684,"children":685},{"style":146},[686],{"type":49,"value":687}," skills\u002Fnemo-evaluator-plugin\u002Fassets\u002Fspecs\u002Fllm_as_judge.json\n",{"type":43,"tag":82,"props":689,"children":691},{"id":690},"run-evaluation-as-a-durable-job",[692],{"type":49,"value":693},"Run Evaluation As A Durable Job",{"type":43,"tag":52,"props":695,"children":696},{},[697,699,705],{"type":49,"value":698},"Use the ",{"type":43,"tag":58,"props":700,"children":702},{"className":701},[],[703],{"type":49,"value":704},"nemo evaluator evaluate submit",{"type":49,"value":706}," command to create a durable evaluation job. The response of this command returns a job handler object instead of the evaluation result.",{"type":43,"tag":123,"props":708,"children":710},{"className":125,"code":709,"language":127,"meta":128,"style":128},"nemo evaluator evaluate submit \\\n  --spec-file skills\u002Fnemo-evaluator-plugin\u002Fassets\u002Fspecs\u002Fexact_match_metric.json\n",[711],{"type":43,"tag":58,"props":712,"children":713},{"__ignoreMap":128},[714,739],{"type":43,"tag":134,"props":715,"children":716},{"class":136,"line":137},[717,721,725,729,734],{"type":43,"tag":134,"props":718,"children":719},{"style":141},[720],{"type":49,"value":108},{"type":43,"tag":134,"props":722,"children":723},{"style":146},[724],{"type":49,"value":149},{"type":43,"tag":134,"props":726,"children":727},{"style":146},[728],{"type":49,"value":280},{"type":43,"tag":134,"props":730,"children":731},{"style":146},[732],{"type":49,"value":733}," submit",{"type":43,"tag":134,"props":735,"children":736},{"style":244},[737],{"type":49,"value":738}," \\\n",{"type":43,"tag":134,"props":740,"children":742},{"class":136,"line":741},2,[743,748],{"type":43,"tag":134,"props":744,"children":745},{"style":146},[746],{"type":49,"value":747},"  --spec-file",{"type":43,"tag":134,"props":749,"children":750},{"style":146},[751],{"type":49,"value":589},{"type":43,"tag":52,"props":753,"children":754},{},[755,757,763,765,771],{"type":49,"value":756},"The submit response includes the generated job's ",{"type":43,"tag":58,"props":758,"children":760},{"className":759},[],[761],{"type":49,"value":762},"name",{"type":49,"value":764}," field, for example ",{"type":43,"tag":58,"props":766,"children":768},{"className":767},[],[769],{"type":49,"value":770},"nemo-evaluator-zlhn1ecd",{"type":49,"value":772},". Wait for the job to complete, then list and download the job results.",{"type":43,"tag":123,"props":774,"children":776},{"className":125,"code":775,"language":127,"meta":128,"style":128},"nemo jobs get-status \u003Cjob-name>\nnemo jobs get \u003Cjob-name>\nnemo jobs results list \u003Cjob-name>\nnemo jobs results download aggregate-scores --job \u003Cjob-name> --output-file aggregate-scores.json\nnemo jobs results download row-scores --job \u003Cjob-name> --output-file row-scores.jsonl\n",[777],{"type":43,"tag":58,"props":778,"children":779},{"__ignoreMap":128},[780,814,846,884,942],{"type":43,"tag":134,"props":781,"children":782},{"class":136,"line":137},[783,787,792,797,801,806,810],{"type":43,"tag":134,"props":784,"children":785},{"style":141},[786],{"type":49,"value":108},{"type":43,"tag":134,"props":788,"children":789},{"style":146},[790],{"type":49,"value":791}," jobs",{"type":43,"tag":134,"props":793,"children":794},{"style":146},[795],{"type":49,"value":796}," get-status",{"type":43,"tag":134,"props":798,"children":799},{"style":233},[800],{"type":49,"value":236},{"type":43,"tag":134,"props":802,"children":803},{"style":146},[804],{"type":49,"value":805},"job-nam",{"type":43,"tag":134,"props":807,"children":808},{"style":244},[809],{"type":49,"value":247},{"type":43,"tag":134,"props":811,"children":812},{"style":233},[813],{"type":49,"value":252},{"type":43,"tag":134,"props":815,"children":816},{"class":136,"line":741},[817,821,825,830,834,838,842],{"type":43,"tag":134,"props":818,"children":819},{"style":141},[820],{"type":49,"value":108},{"type":43,"tag":134,"props":822,"children":823},{"style":146},[824],{"type":49,"value":791},{"type":43,"tag":134,"props":826,"children":827},{"style":146},[828],{"type":49,"value":829}," get",{"type":43,"tag":134,"props":831,"children":832},{"style":233},[833],{"type":49,"value":236},{"type":43,"tag":134,"props":835,"children":836},{"style":146},[837],{"type":49,"value":805},{"type":43,"tag":134,"props":839,"children":840},{"style":244},[841],{"type":49,"value":247},{"type":43,"tag":134,"props":843,"children":844},{"style":233},[845],{"type":49,"value":252},{"type":43,"tag":134,"props":847,"children":849},{"class":136,"line":848},3,[850,854,858,863,868,872,876,880],{"type":43,"tag":134,"props":851,"children":852},{"style":141},[853],{"type":49,"value":108},{"type":43,"tag":134,"props":855,"children":856},{"style":146},[857],{"type":49,"value":791},{"type":43,"tag":134,"props":859,"children":860},{"style":146},[861],{"type":49,"value":862}," results",{"type":43,"tag":134,"props":864,"children":865},{"style":146},[866],{"type":49,"value":867}," list",{"type":43,"tag":134,"props":869,"children":870},{"style":233},[871],{"type":49,"value":236},{"type":43,"tag":134,"props":873,"children":874},{"style":146},[875],{"type":49,"value":805},{"type":43,"tag":134,"props":877,"children":878},{"style":244},[879],{"type":49,"value":247},{"type":43,"tag":134,"props":881,"children":882},{"style":233},[883],{"type":49,"value":252},{"type":43,"tag":134,"props":885,"children":887},{"class":136,"line":886},4,[888,892,896,900,905,910,915,919,923,927,932,937],{"type":43,"tag":134,"props":889,"children":890},{"style":141},[891],{"type":49,"value":108},{"type":43,"tag":134,"props":893,"children":894},{"style":146},[895],{"type":49,"value":791},{"type":43,"tag":134,"props":897,"children":898},{"style":146},[899],{"type":49,"value":862},{"type":43,"tag":134,"props":901,"children":902},{"style":146},[903],{"type":49,"value":904}," download",{"type":43,"tag":134,"props":906,"children":907},{"style":146},[908],{"type":49,"value":909}," aggregate-scores",{"type":43,"tag":134,"props":911,"children":912},{"style":146},[913],{"type":49,"value":914}," --job",{"type":43,"tag":134,"props":916,"children":917},{"style":233},[918],{"type":49,"value":236},{"type":43,"tag":134,"props":920,"children":921},{"style":146},[922],{"type":49,"value":805},{"type":43,"tag":134,"props":924,"children":925},{"style":244},[926],{"type":49,"value":247},{"type":43,"tag":134,"props":928,"children":929},{"style":233},[930],{"type":49,"value":931},">",{"type":43,"tag":134,"props":933,"children":934},{"style":146},[935],{"type":49,"value":936}," --output-file",{"type":43,"tag":134,"props":938,"children":939},{"style":146},[940],{"type":49,"value":941}," aggregate-scores.json\n",{"type":43,"tag":134,"props":943,"children":945},{"class":136,"line":944},5,[946,950,954,958,962,967,971,975,979,983,987,991],{"type":43,"tag":134,"props":947,"children":948},{"style":141},[949],{"type":49,"value":108},{"type":43,"tag":134,"props":951,"children":952},{"style":146},[953],{"type":49,"value":791},{"type":43,"tag":134,"props":955,"children":956},{"style":146},[957],{"type":49,"value":862},{"type":43,"tag":134,"props":959,"children":960},{"style":146},[961],{"type":49,"value":904},{"type":43,"tag":134,"props":963,"children":964},{"style":146},[965],{"type":49,"value":966}," row-scores",{"type":43,"tag":134,"props":968,"children":969},{"style":146},[970],{"type":49,"value":914},{"type":43,"tag":134,"props":972,"children":973},{"style":233},[974],{"type":49,"value":236},{"type":43,"tag":134,"props":976,"children":977},{"style":146},[978],{"type":49,"value":805},{"type":43,"tag":134,"props":980,"children":981},{"style":244},[982],{"type":49,"value":247},{"type":43,"tag":134,"props":984,"children":985},{"style":233},[986],{"type":49,"value":931},{"type":43,"tag":134,"props":988,"children":989},{"style":146},[990],{"type":49,"value":936},{"type":43,"tag":134,"props":992,"children":993},{"style":146},[994],{"type":49,"value":995}," row-scores.jsonl\n",{"type":43,"tag":75,"props":997,"children":999},{"id":998},"python-sdk-interface",[1000],{"type":49,"value":1001},"Python SDK Interface",{"type":43,"tag":52,"props":1003,"children":1004},{},[1005,1007,1013,1015,1021],{"type":49,"value":1006},"Evaluator Python SDK client is exposed as ",{"type":43,"tag":58,"props":1008,"children":1010},{"className":1009},[],[1011],{"type":49,"value":1012},"evaluator",{"type":49,"value":1014}," variable on ",{"type":43,"tag":58,"props":1016,"children":1018},{"className":1017},[],[1019],{"type":49,"value":1020},"NeMoPlatform",{"type":49,"value":1022}," instance:",{"type":43,"tag":123,"props":1024,"children":1028},{"className":1025,"code":1026,"language":1027,"meta":128,"style":128},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","from nemo_platform import NeMoPlatform\n\nplatform_client = NeMoPlatform(base_url=\"http:\u002F\u002Flocalhost:8080\")\nstatus = platform_client.evaluator.plugin_status()\n","python",[1029],{"type":43,"tag":58,"props":1030,"children":1031},{"__ignoreMap":128},[1032,1040,1049,1057],{"type":43,"tag":134,"props":1033,"children":1034},{"class":136,"line":137},[1035],{"type":43,"tag":134,"props":1036,"children":1037},{},[1038],{"type":49,"value":1039},"from nemo_platform import NeMoPlatform\n",{"type":43,"tag":134,"props":1041,"children":1042},{"class":136,"line":741},[1043],{"type":43,"tag":134,"props":1044,"children":1046},{"emptyLinePlaceholder":1045},true,[1047],{"type":49,"value":1048},"\n",{"type":43,"tag":134,"props":1050,"children":1051},{"class":136,"line":848},[1052],{"type":43,"tag":134,"props":1053,"children":1054},{},[1055],{"type":49,"value":1056},"platform_client = NeMoPlatform(base_url=\"http:\u002F\u002Flocalhost:8080\")\n",{"type":43,"tag":134,"props":1058,"children":1059},{"class":136,"line":886},[1060],{"type":43,"tag":134,"props":1061,"children":1062},{},[1063],{"type":49,"value":1064},"status = platform_client.evaluator.plugin_status()\n",{"type":43,"tag":52,"props":1066,"children":1067},{},[1068,1070,1076],{"type":49,"value":1069},"See examples of using the plugin SDK interface in ",{"type":43,"tag":380,"props":1071,"children":1073},{"href":1072},".\u002Fassets\u002Fexamples\u002Fplugin_sdk_examples.py",[1074],{"type":49,"value":1075},"plugin_sdk_examples.py",{"type":49,"value":387},{"type":43,"tag":75,"props":1078,"children":1080},{"id":1079},"security",[1081],{"type":49,"value":1082},"Security",{"type":43,"tag":52,"props":1084,"children":1085},{},[1086],{"type":49,"value":1087},"Make sure not to print any secrets to stdout since this can be collected as logs",{"type":43,"tag":75,"props":1089,"children":1091},{"id":1090},"additional-resources",[1092],{"type":49,"value":1093},"Additional Resources",{"type":43,"tag":52,"props":1095,"children":1096},{},[1097,1099,1105],{"type":49,"value":1098},"For LLM-judge setup notes, see ",{"type":43,"tag":380,"props":1100,"children":1102},{"href":1101},"references\u002Fllm-judge.md",[1103],{"type":49,"value":1104},"LLM Judge Notes",{"type":49,"value":387},{"type":43,"tag":52,"props":1107,"children":1108},{},[1109,1111,1117],{"type":49,"value":1110},"For evaluator API key auth, see ",{"type":43,"tag":380,"props":1112,"children":1114},{"href":1113},"references\u002Fapi-auth.md",[1115],{"type":49,"value":1116},"Evaluator API Auth",{"type":49,"value":387},{"type":43,"tag":52,"props":1119,"children":1120},{},[1121,1123,1129],{"type":49,"value":1122},"For local and cluster troubleshooting, see ",{"type":43,"tag":380,"props":1124,"children":1126},{"href":1125},"references\u002Ftroubleshooting.md",[1127],{"type":49,"value":1128},"Evaluation Troubleshooting",{"type":49,"value":387},{"type":43,"tag":1131,"props":1132,"children":1133},"style",{},[1134],{"type":49,"value":1135},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":1137,"total":1295},[1138,1156,1173,1184,1196,1210,1223,1237,1250,1261,1275,1284],{"slug":1139,"name":1139,"fn":1140,"description":1141,"org":1142,"tags":1143,"stars":1153,"repoUrl":1154,"updatedAt":1155},"nemoclaw-user-guide","retrieve NemoClaw documentation and configuration","Guides human users' AI agents to the NemoClaw docs MCP server and canonical Fern documentation in Markdown form. Use when users ask how to install, configure, operate, troubleshoot, secure, or learn NemoClaw with an AI coding assistant. Trigger keywords - nemoclaw docs, use nemoclaw with ai agent, nemoclaw mcp docs, nemoclaw install help, nemoclaw quickstart, nemoclaw markdown docs, llms.txt, agent skills.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1144,1147,1150],{"name":1145,"slug":1146,"type":15},"Documentation","documentation",{"name":1148,"slug":1149,"type":15},"MCP","mcp",{"name":1151,"slug":1152,"type":15},"Search","search",21777,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FNemoClaw","2026-07-20T06:00:01.461044",{"slug":1157,"name":1157,"fn":1158,"description":1159,"org":1160,"tags":1161,"stars":1170,"repoUrl":1171,"updatedAt":1172},"mcore-build-and-dependency","manage Megatron-LM development environments","Container-based dev environment setup and dependency management for Megatron-LM. Covers acquiring and launching the CI container, uv package management, and updating uv.lock.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1162,1165,1168],{"name":1163,"slug":1164,"type":15},"Containers","containers",{"name":1166,"slug":1167,"type":15},"Deployment","deployment",{"name":1169,"slug":1027,"type":15},"Python",17049,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FMegatron-LM","2026-07-27T06:06:11.249662",{"slug":1174,"name":1174,"fn":1175,"description":1176,"org":1177,"tags":1178,"stars":1170,"repoUrl":1171,"updatedAt":1183},"mcore-bump-base-image","update NVIDIA PyTorch base images","Bump the NVIDIA PyTorch base image (`nvcr.io\u002Fnvidia\u002Fpytorch:YY.MM-py3`) used by Megatron-LM CI. Covers the two pin sites (GitHub CI in `docker\u002F.ngc_version.dev` and GitLab CI in `.gitlab\u002Fstages\u002F01.build.yml`), the post-bump CI loop (re-run functional tests, refresh golden values, mark broken tests), and the gotchas that bit PRs",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1179,1182],{"name":1180,"slug":1181,"type":15},"CI\u002FCD","ci-cd",{"name":1166,"slug":1167,"type":15},"2026-07-14T05:25:59.97109",{"slug":1185,"name":1185,"fn":1186,"description":1187,"org":1188,"tags":1189,"stars":1170,"repoUrl":1171,"updatedAt":1195},"mcore-cicd","manage CI\u002FCD pipelines for Megatron-LM","CI\u002FCD reference for Megatron-LM. Covers CI pipeline structure, PR scope labels, triggering internal GitLab CI (which force-pushes the current branch to a pull-request\u002FBRANCH ref — always dry-run and verify the destination first; never run against shared or protected branches), and CI failure investigation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1190,1191,1192],{"name":1180,"slug":1181,"type":15},{"name":1166,"slug":1167,"type":15},{"name":1193,"slug":1194,"type":15},"GitHub","github","2026-07-27T06:06:12.278222",{"slug":1197,"name":1197,"fn":1198,"description":1199,"org":1200,"tags":1201,"stars":1170,"repoUrl":1171,"updatedAt":1209},"mcore-create-issue","investigate CI failures and create issues","Investigate a failing GitHub Actions run or job and create a GitHub issue for the failure.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1202,1205,1206],{"name":1203,"slug":1204,"type":15},"Debugging","debugging",{"name":1193,"slug":1194,"type":15},{"name":1207,"slug":1208,"type":15},"Triage","triage","2026-07-14T05:25:57.442089",{"slug":1211,"name":1211,"fn":1212,"description":1213,"org":1214,"tags":1215,"stars":1170,"repoUrl":1171,"updatedAt":1222},"mcore-linting-and-formatting","lint and format Megatron-LM code","Linting and formatting for Megatron-LM. Covers running autoformat.sh, tools (ruff, black, isort, pylint, mypy), and code style rules.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1216,1219],{"name":1217,"slug":1218,"type":15},"Best Practices","best-practices",{"name":1220,"slug":1221,"type":15},"Code Analysis","code-analysis","2026-07-14T05:25:56.18433",{"slug":1224,"name":1224,"fn":1225,"description":1226,"org":1227,"tags":1228,"stars":1170,"repoUrl":1171,"updatedAt":1236},"mcore-migrate-gpt-to-hybrid","migrate Megatron-LM models to HybridModel","Migration guide for moving Megatron Core GPTModel checkpoints, model providers, training commands, and layer mappings to HybridModel.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1229,1232,1235],{"name":1230,"slug":1231,"type":15},"Machine Learning","machine-learning",{"name":1233,"slug":1234,"type":15},"Migration","migration",{"name":9,"slug":8,"type":15},"2026-07-17T06:07:11.777011",{"slug":1238,"name":1238,"fn":1239,"description":1240,"org":1241,"tags":1242,"stars":1170,"repoUrl":1171,"updatedAt":1249},"mcore-onboard-gb200-1node-tests","onboard functional tests for GB200","Onboard 1-node GitHub MR functional tests for GB200 from existing mr-scoped 2-node tests.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1243,1246],{"name":1244,"slug":1245,"type":15},"QA","qa",{"name":1247,"slug":1248,"type":15},"Testing","testing","2026-07-14T05:25:53.673039",{"slug":1251,"name":1251,"fn":1252,"description":1253,"org":1254,"tags":1255,"stars":1170,"repoUrl":1171,"updatedAt":1260},"mcore-run-on-slurm","launch distributed training jobs on SLURM","How to launch distributed Megatron-LM training jobs on a SLURM cluster. Covers a minimal sbatch skeleton, environment-variable setup for torch.distributed.run, CUDA_DEVICE_MAX_CONNECTIONS rules across hardware and parallelism modes, container conventions, monitoring, and per-rank failure diagnosis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1256,1257],{"name":1166,"slug":1167,"type":15},{"name":1258,"slug":1259,"type":15},"Infrastructure","infrastructure","2026-07-14T05:25:49.362534",{"slug":1262,"name":1262,"fn":1263,"description":1264,"org":1265,"tags":1266,"stars":1170,"repoUrl":1171,"updatedAt":1274},"mcore-split-pr","split pull requests to reduce review load","Split a PR into multiple PRs to reduce the number of required CODEOWNERS reviewer groups.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1267,1270,1271],{"name":1268,"slug":1269,"type":15},"Code Review","code-review",{"name":1193,"slug":1194,"type":15},{"name":1272,"slug":1273,"type":15},"Pull Requests","pull-requests","2026-07-14T05:26:01.226578",{"slug":1276,"name":1276,"fn":1277,"description":1278,"org":1279,"tags":1280,"stars":1170,"repoUrl":1171,"updatedAt":1283},"mcore-testing","run and manage Megatron-LM tests","Test system for Megatron-LM. Covers test layout, recipe YAML structure, adding and running unit and functional tests, golden values, marker filters, and CI parity.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1281,1282],{"name":1244,"slug":1245,"type":15},{"name":1247,"slug":1248,"type":15},"2026-07-14T05:25:54.928983",{"slug":1285,"name":1285,"fn":1286,"description":1287,"org":1288,"tags":1289,"stars":1170,"repoUrl":1171,"updatedAt":1294},"nightly-sync","manage nightly main-to-dev sync workflows","Domain knowledge for the nightly main-to-dev sync workflow. Covers merge strategy, CI architecture, failure investigation, and known issues.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1290,1293],{"name":1291,"slug":1292,"type":15},"Automation","automation",{"name":1180,"slug":1181,"type":15},"2026-07-30T05:29:03.275638",496,{"items":1297,"total":1393},[1298,1315,1325,1339,1349,1364,1379],{"slug":1299,"name":1299,"fn":1300,"description":1301,"org":1302,"tags":1303,"stars":23,"repoUrl":24,"updatedAt":1314},"accelerated-computing-cudf","accelerate data processing with cuDF","Official NVIDIA-authored guidance for NVIDIA cuDF GPU DataFrames, pandas acceleration, dask-cuDF, ETL, joins, groupby, CSV\u002FParquet I\u002FO, nullable semantics, and multi-GPU DataFrame workloads.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1304,1307,1310,1311],{"name":1305,"slug":1306,"type":15},"Data Analysis","data-analysis",{"name":1308,"slug":1309,"type":15},"Data Engineering","data-engineering",{"name":9,"slug":8,"type":15},{"name":1312,"slug":1313,"type":15},"Performance","performance","2026-07-14T05:28:43.176466",{"slug":1316,"name":1316,"fn":1317,"description":1318,"org":1319,"tags":1320,"stars":23,"repoUrl":24,"updatedAt":1324},"aiq-deploy","deploy and manage NVIDIA AI-Q infrastructure","Use when asked to install, deploy, run, validate, troubleshoot, or stop NVIDIA AI-Q Blueprint infrastructure.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1321,1322,1323],{"name":1166,"slug":1167,"type":15},{"name":1258,"slug":1259,"type":15},{"name":9,"slug":8,"type":15},"2026-07-14T05:29:06.667109",{"slug":1326,"name":1326,"fn":1327,"description":1328,"org":1329,"tags":1330,"stars":23,"repoUrl":24,"updatedAt":1338},"aiq-research","conduct deep research with AI-Q","Use when asked to run deep research or AI-Q research through a reachable NVIDIA AI-Q Blueprint backend.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1331,1334,1335],{"name":1332,"slug":1333,"type":15},"Agents","agents",{"name":9,"slug":8,"type":15},{"name":1336,"slug":1337,"type":15},"Research","research","2026-07-14T05:28:06.816956",{"slug":1340,"name":1340,"fn":1341,"description":1342,"org":1343,"tags":1344,"stars":23,"repoUrl":24,"updatedAt":1348},"amc-run-sample-calibration","run AMC sample dataset calibration","Run end-to-end calibration on the shipped sample dataset (sdg_08_2_sample_data_010926.zip) against a running AMC microservice. Use when user says 'test sample dataset', 'run sample calibration', 'verify AMC install', or 'launch and test'.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1345,1346,1347],{"name":1305,"slug":1306,"type":15},{"name":9,"slug":8,"type":15},{"name":1247,"slug":1248,"type":15},"2026-07-17T05:29:03.913266",{"slug":1350,"name":1350,"fn":1351,"description":1352,"org":1353,"tags":1354,"stars":23,"repoUrl":24,"updatedAt":1363},"amc-run-video-calibration","calibrate video datasets with AutoMagicCalib","Calibrate a new dataset from pre-recorded video files via the AutoMagicCalib REST API. Use when user has local MP4s and says 'calibrate my videos', 'run AMC on these videos', or similar. For RTSP\u002Flive streams, use amc-run-rtsp-calibration instead.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1355,1356,1359,1360],{"name":1291,"slug":1292,"type":15},{"name":1357,"slug":1358,"type":15},"Imaging","imaging",{"name":9,"slug":8,"type":15},{"name":1361,"slug":1362,"type":15},"Video","video","2026-07-17T05:28:53.905004",{"slug":1365,"name":1365,"fn":1366,"description":1367,"org":1368,"tags":1369,"stars":23,"repoUrl":24,"updatedAt":1378},"amc-setup-calibration-stack","deploy AutoMagicCalib microservice with Docker","Launch AutoMagicCalib microservice and web UI from NGC release images via Docker Compose. Use when user says 'deploy auto calibration', 'launch auto calibration', 'launch AMC', 'start MS+UI', or 'set up auto-magic-calib'. Requires NGC API key.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1370,1371,1374,1375],{"name":1166,"slug":1167,"type":15},{"name":1372,"slug":1373,"type":15},"Docker","docker",{"name":9,"slug":8,"type":15},{"name":1376,"slug":1377,"type":15},"Operations","operations","2026-07-17T05:28:56.913999",{"slug":1380,"name":1380,"fn":1381,"description":1382,"org":1383,"tags":1384,"stars":23,"repoUrl":24,"updatedAt":1392},"cudaq-guide","develop quantum applications with CUDA-Q","CUDA-Q onboarding guide for installation, test programs, GPU simulation, QPU hardware, and quantum applications.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1385,1386,1389],{"name":9,"slug":8,"type":15},{"name":1387,"slug":1388,"type":15},"Quantum Computing","quantum-computing",{"name":1390,"slug":1391,"type":15},"Simulation","simulation","2026-07-14T05:26:58.898253",305]