[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-nvidia-nvshmem-configure-nic-pe-mapping":3,"mdc--prufw3-key":36,"related-org-nvidia-nvshmem-configure-nic-pe-mapping":1280,"related-repo-nvidia-nvshmem-configure-nic-pe-mapping":1440},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":20,"repoUrl":21,"updatedAt":22,"license":23,"forks":24,"topics":25,"repo":31,"sourceUrl":34,"mdContent":35},"nvshmem-configure-nic-pe-mapping","configure NVSHMEM NIC-to-PE mappings","Recommend NVSHMEM NIC-to-PE mappings and environment exports. Use for HCA selection, multi-NIC configuration, or topology-based mapping diagnostics.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},"nvidia","NVIDIA","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fnvidia.png",[12,16,19],{"name":13,"slug":14,"type":15},"Networking","networking","tag",{"name":17,"slug":18,"type":15},"Configuration","configuration",{"name":9,"slug":8,"type":15},579,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fnvshmem","2026-08-31T09:19:06.10294","Apache-2.0",104,[26,27,28,29,8,30],"communciations","cpp","cuda","deep-learning","python",{"repoUrl":21,"stars":20,"forks":24,"topics":32,"description":33},[26,27,28,29,8,30],"NVIDIA NVSHMEM is a parallel programming interface for NVIDIA GPUs based on OpenSHMEM. NVSHMEM can significantly reduce multi-process communication and coordination overheads by allowing programmers to perform one-sided communication from within CUDA kernels and on CUDA streams.","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fnvshmem\u002Ftree\u002FHEAD\u002Fskills\u002Fnvshmem-configure-nic-pe-mapping","---\nname: nvshmem-configure-nic-pe-mapping\ndescription: Recommend NVSHMEM NIC-to-PE mappings and environment exports. Use for HCA selection, multi-NIC configuration, or topology-based mapping diagnostics.\nlicense: Apache-2.0\nmetadata:\n  version: \"1.0.0\"\n  author: NVIDIA NVSHMEM Team \u003Cnvshmem@nvidia.com>\n  tags:\n    - nvshmem\n    - nic\n    - mapping\n---\n\n# Tune NVSHMEM NIC-to-PE Mapping\n\n## Purpose\n\nProduce a reproducible recommendation from the target node's actual GPU\u002FNIC topology. Do not promise a performance improvement; present the mapping as a topology-informed starting point that must be validated.\n\n## Requirements\n\n- Use an allocated target compute node with visible GPUs and RDMA devices; do not use login-node topology as evidence.\n- Make `nvidia-smi` and `\u002Fsys\u002Fclass\u002Finfiniband` readable on that node. If `\u002Fsys` is unavailable, make `ibv_devinfo` available or provide equivalent per-port evidence.\n- Make `nvshmem-info` available on the target node, or supply the installed NVSHMEM version explicitly.\n- Use the bundled read-only collector when possible. It requires no API keys, credentials, package installation, or writable filesystem access.\n\n## Inputs\n\nResolve every required input before emitting exact exports. If values from different sources conflict, identify the conflict and request clarification rather than selecting a value silently.\n\n### Required inputs\n\n- **NVSHMEM version:** Prefer the user-confirmed installed version; otherwise use the collector's `[nvshmem-version]` output, then `nvshmem-info -n` on the target node. Retain the complete reported version after normalizing a leading `v`.\n- **Remote transport:** Prefer the explicitly selected transport for the target run; otherwise use the target node's NVSHMEM initialization log. Do not infer a transport from topology; request a log when automatic selection is ambiguous.\n- **Local PE count:** Prefer observed per-rank logs, then launcher configuration, then an explicit user-provided count. Confirm that all evidence describes the same node allocation.\n- **Ordered local `PE -> GPU index or PCI BDF` binding:** Prefer observed per-rank binding logs, then launcher binding configuration, then an explicit user mapping. Use `CUDA_VISIBLE_DEVICES` only when its rank order is explicitly established.\n- **Target-node GPU\u002FNIC and HCA-port topology:** Prefer the newest complete output from `scripts\u002Fcollect-nic-topology.sh` on the target node. Otherwise use the listed `nvidia-smi` commands plus per-port `\u002Fsys\u002Fclass\u002Finfiniband` or `ibv_devinfo` evidence. Include GPU and NIC PCI BDFs, port state, and link layer.\n\n### Optional inputs\n\n- **Requested link layer:** Use the user's explicit InfiniBand or Ethernet\u002FRoCE preference. If absent, choose from observed active ports as described in step 5.\n- **Excess-port policy:** Use the user's explicit request to select a closest subset or use multiple NICs per PE. If absent, present supported alternatives conditionally and make no selection.\n- **Current official documentation or citations:** Obtain only when the user requests them or the version\u002Ftransport matrix is insufficient. Use `$nvshmem-docs` for that check.\n\n## Scripts\n\n| Script | Use | Invocation |\n| --- | --- | --- |\n| `scripts\u002Fcollect-nic-topology.sh` | Collect read-only NVSHMEM version, GPU PCI, GPU\u002FNIC topology, and RDMA-port evidence. It emits all evidence to stdout and may exit nonzero after producing useful partial output. | Resolve the script from this skill directory and use `run_script` on the target compute node. If `run_script` is unavailable, run `bash scripts\u002Fcollect-nic-topology.sh`. |\n\n## Read the Required References\n\n- Read [references\u002Fversion-transport-matrix.md](references\u002Fversion-transport-matrix.md) before interpreting or emitting any mapping variable.\n- Read [references\u002Ftopology-and-output.md](references\u002Ftopology-and-output.md) before ranking NICs or formatting a recommendation.\n- Treat the bundled **NVSHMEM 3.8 or later BLOCK contract** as authoritative within its recorded transport scope. Do not invoke `$nvshmem-docs` merely to revalidate behavior that this contract resolves, including for releases later than 3.8.\n\n## Instructions\n\n### 1. Resolve the NVSHMEM Version and Transport\n\nObtain the version from the user first. Otherwise use the collector's `[nvshmem-version]` evidence or run `nvshmem-info -n` when it is already available on the target node. Normalize a leading `v` and retain the complete reported version.\n\nClassify the release as:\n\n- 3.7 or earlier;\n- 3.8 or later.\n\nDo not assume `latest` or silently treat an unknown version as 3.8. If the version remains unknown, give only conditional guidance and request the installed version before emitting final exports.\n\nDetermine the selected remote transport: IBRC, IBDevX, IBGDA, GPUNetIO, UCX, or libfabric. If automatic selection makes the transport ambiguous, ask for the relevant initialization log or the intended transport.\n\n### 2. Resolve Documentation Uncertainty\n\nUse [references\u002Fversion-transport-matrix.md](references\u002Fversion-transport-matrix.md) as the default authority for mapping behavior. Do not invoke `$nvshmem-docs` when the matrix or the bundled **NVSHMEM 3.8 or later BLOCK contract** resolves the version, transport, feature gate, and required mapping controls, including for a release later than 3.8.\n\nInvoke the sibling `$nvshmem-docs` skill at `..\u002Fnvshmem-docs\u002FSKILL.md` only when the matrix and bundled contract are insufficient or conflicting, exact feature-version applicability remains unclear, or the user explicitly requests live official documentation or citations. Follow that skill's version and citation workflow.\n\nIf a documentation check is needed but the sibling skill, an HTTPS retrieval tool, or the live official pages are unavailable:\n\n- continue from the matrix only when it resolves the needed behavior;\n- otherwise withhold final exports and identify the unresolved behavior;\n- label `Documentation freshness: unverified` only when a documentation check was attempted;\n- state which official routes could not be checked; and\n- never invent a behavior that the matrix does not establish.\n\nWhen a documentation check is performed, report the exact documentation edition consulted and any remaining uncertainty. Otherwise identify the applicable matrix section as the basis for the recommendation.\n\n### 3. Collect Topology Evidence\n\nWhen already running on the target compute node, tell the user that the skill is running a read-only collector, then run:\n\n```bash\nbash scripts\u002Fcollect-nic-topology.sh\n```\n\nResolve the bundled script path from `SKILL.md`. Use `run_script` when available; otherwise run the shell command above. The collector writes evidence only to stdout and may return a nonzero status with useful partial output. Do not use `run_script` or the shell fallback from a login node as topology evidence.\n\nWhen not on the target node, ask the user to run that command there and paste the complete output. If the script is unavailable remotely, request all of:\n\n```bash\nnvshmem-info -n\nnvidia-smi --query-gpu=index,pci.bus_id,name --format=csv,noheader\nnvidia-smi topo -m\n```\n\nAlso request, for every candidate HCA port, its HCA name, port number, PCI BDF, state, and link layer from `\u002Fsys\u002Fclass\u002Finfiniband` or `ibv_devinfo`.\n\nDo not run a collector from a login node and treat that topology as compute-node evidence.\n\n### 4. Establish PE Placement\n\nObtain the number of local PEs per node. Assume nodes are homogeneous only after stating that assumption.\n\nObtain the ordered local `PE -> GPU index or PCI BDF` binding from explicit user input, launcher configuration, `CUDA_VISIBLE_DEVICES`, or per-rank logs. Do not infer a local PE's GPU solely from PE count or the order displayed by `nvidia-smi topo -m`.\n\nIf the binding is ambiguous, request it before producing an exact closest-NIC mapping. A conditional table is acceptable, but final exports are not. Do not bypass the missing binding by duplicating every usable NIC for every PE; that adds non-closest paths and is a different multi-port policy. Consider such an all-ports assignment only after the user explicitly requests it and accepts the distance tradeoff.\n\n### 5. Filter and Rank Candidate Ports\n\nTreat an HCA port, not merely an HCA device, as the selectable NIC unit.\n\n- Exclude inactive or physically down ports and record why each was excluded.\n- Partition active ports by link layer. Do not mix InfiniBand with Ethernet\u002FRoCE by default.\n- Honor an explicitly requested link layer. Otherwise choose the larger active partition; prefer InfiniBand on an exact tie. Ask the user if that default conflicts with their fabric or provider intent.\n- Rank each bound GPU-to-port pair by `PIX \u003C PXB \u003C PHB \u003C NODE \u003C SYS` from `nvidia-smi topo -m`.\n- Break equal-distance choices by lower assigned-port load, then canonical `hca:port` order.\n- Share a port when fewer usable ports than PEs makes sharing necessary, while preserving distance first and balance second.\n\n### 6. Resolve Excess NICs Explicitly\n\nWhen usable NIC ports outnumber local PEs, stop before final exports and ask whether to:\n\n1. select the closest balanced subset; or\n2. use multiple NICs per PE.\n\nOffer option 2 only when the resolved version and transport support it according to the matrix. Explain whether multi-NIC requires automatic distance assignment or supports an explicit BLOCK mapping. If the user does not answer, show both supported alternatives as conditional recommendations and do not choose silently.\n\nIf the user asked to use all NICs but the resolved version\u002Ftransport does not support multi-NIC, explain the restriction and request approval to use the closest subset. Do not treat the unsupported request as implicit approval for a subset, and do not emit final exports until the user confirms it. When only the subset is supported, present that single supported choice for confirmation instead of offering a disabled multi-NIC option.\n\n### 7. Synthesize Exactly One Selection Mechanism\n\nPrefer `NVSHMEM_HCA_PE_MAPPING` for exact assignments. Use `NVSHMEM_HCA_LIST` only when selecting an eligible set and accepting transport-canonical ordering. Never set both variables.\n\nApply the version\u002Ftransport rules in the matrix exactly:\n\n- For 3.7 and earlier, preserve one expanded mapping slot per PE for explicit IB mappings. Use automatic distance assignment for supported multi-NIC paths.\n- For 3.8 or later, apply the bundled BLOCK contract: set `NVSHMEM_ENABLE_NIC_PE_MAPPING=1` for explicit assignments and `NVSHMEM_ENABLE_MULTI_PORT=1` whenever recommending more than one NIC per PE.\n- For UCX and libfabric, explain the provider-controlled boundary and do not fabricate common-HCA exact mappings.\n\nBuild `NVSHMEM_HCA_PE_MAPPING` with explicit `hca:port:count` entries. Expand every count before explaining which slots each PE receives. Preserve duplicates when intentional sharing is required.\n\nKeep a mapping-only export block limited to NIC-assignment controls. Do not add `NVSHMEM_REMOTE_TRANSPORT` or provider\u002Flauncher settings merely because the transport is known; include them only when the user separately asks to select the transport and the matrix, or documentation when needed, verifies the value.\n\n### 8. Return the Recommendation\n\nExecute the [Output Contract](references\u002Ftopology-and-output.md#output-contract) in order; that reference is the single authority for output fields, formatting, withholding messages, and validation guidance.\n\nTreat its `Mapping` and `Exact Exports` sections as mandatory gates. Do not return until each gate is either completed or explicitly withheld for the reason allowed by the contract. Do not duplicate or paraphrase the contract in this file.\n\n## Examples\n\n- “My 4 local PEs are bound to GPUs 0–3. Given this target-node collector output and NVSHMEM 3.8.0 IBRC, recommend the closest balanced InfiniBand port mapping and exact exports.”\n- “Interpret this `nvidia-smi topo -m` and per-port state\u002Flink-layer output. NVSHMEM is 3.7 with IBGDA; identify whether multiple NICs per PE are supported and what extra evidence is needed before emitting exports.”\n- “We have 2 PEs and 5 active Ethernet\u002FRoCE ports. Use this PE-to-GPU binding and initialization log to show the supported subset and multi-NIC alternatives without selecting one silently.”\n\n## Troubleshooting\n\n| Symptom | Response |\n| --- | --- |\n| The collector reports `status=partial`, missing GPUs, or no HCA ports. | Re-run it on an allocated target compute node with NVIDIA and RDMA device visibility. Paste the complete labeled output; a nonzero exit may still contain usable evidence. |\n| No exact mapping can be produced. | Obtain the NVSHMEM version, resolved remote transport, local PE-to-GPU binding, and active-port evidence. Give only conditional guidance until all are available. |\n| Ports have mixed link layers or a requested port is inactive. | Exclude inactive ports and select one link-layer partition unless the user explicitly requests another supported fabric. |\n| NVSHMEM behavior is unclear for the selected version or transport. | Check the version\u002Ftransport matrix first, then invoke `$nvshmem-docs` only if it remains insufficient or the user requires current official citations. |\n| Logs select unexpected ports after applying the mapping. | Verify the export block, PE binding, selected transport, and provider constraints; compare against the unforced automatic configuration before changing other settings. |\n\n## Limitations\n\n- Limit recommendations to the observed target-node topology and port state; repeat evidence collection after hardware, fabric, launcher, or binding changes.\n- Treat topology proximity as a selection heuristic, not a performance guarantee; validate with a representative workload.\n- Do not produce an exact mapping until the NVSHMEM version, transport, local PE-to-GPU binding, and usable-port evidence are resolved.\n- For UCX and libfabric, remain within the provider-controlled boundary and do not fabricate common-HCA mapping settings.\n\n## Guardrails\n\n- Do not select inactive ports or silently mix link layers.\n- Do not assume that a numerically matching GPU and NIC index implies proximity.\n- Do not turn an unknown PE-to-GPU binding into an all-ports-per-PE mapping.\n- Do not emit 3.8-or-later BLOCK behavior for 3.7 or earlier.\n- Do not emit legacy IBGDA multi-port controls as the primary 3.8-or-later control.\n- Do not generate provider-native UCX or libfabric settings; explain them without inventing values.\n- Do not overwrite user files or change the system while collecting evidence.\n",{"data":37,"body":45},{"name":4,"description":6,"license":23,"metadata":38},{"version":39,"author":40,"tags":41},"1.0.0","NVIDIA NVSHMEM Team \u003Cnvshmem@nvidia.com>",[42,43,44],"nvshmem","nic","mapping",{"type":46,"children":47},"root",[48,57,64,70,76,141,147,152,159,282,288,329,335,411,417,463,469,475,501,506,519,532,537,543,567,587,592,628,633,639,644,672,699,704,759,776,781,787,792,818,823,829,834,889,895,900,914,919,924,930,951,956,990,1010,1023,1029,1042,1062,1068,1093,1099,1201,1207,1230,1236,1274],{"type":49,"tag":50,"props":51,"children":53},"element","h1",{"id":52},"tune-nvshmem-nic-to-pe-mapping",[54],{"type":55,"value":56},"text","Tune NVSHMEM NIC-to-PE Mapping",{"type":49,"tag":58,"props":59,"children":61},"h2",{"id":60},"purpose",[62],{"type":55,"value":63},"Purpose",{"type":49,"tag":65,"props":66,"children":67},"p",{},[68],{"type":55,"value":69},"Produce a reproducible recommendation from the target node's actual GPU\u002FNIC topology. Do not promise a performance improvement; present the mapping as a topology-informed starting point that must be validated.",{"type":49,"tag":58,"props":71,"children":73},{"id":72},"requirements",[74],{"type":55,"value":75},"Requirements",{"type":49,"tag":77,"props":78,"children":79},"ul",{},[80,86,124,136],{"type":49,"tag":81,"props":82,"children":83},"li",{},[84],{"type":55,"value":85},"Use an allocated target compute node with visible GPUs and RDMA devices; do not use login-node topology as evidence.",{"type":49,"tag":81,"props":87,"children":88},{},[89,91,98,100,106,108,114,116,122],{"type":55,"value":90},"Make ",{"type":49,"tag":92,"props":93,"children":95},"code",{"className":94},[],[96],{"type":55,"value":97},"nvidia-smi",{"type":55,"value":99}," and ",{"type":49,"tag":92,"props":101,"children":103},{"className":102},[],[104],{"type":55,"value":105},"\u002Fsys\u002Fclass\u002Finfiniband",{"type":55,"value":107}," readable on that node. If ",{"type":49,"tag":92,"props":109,"children":111},{"className":110},[],[112],{"type":55,"value":113},"\u002Fsys",{"type":55,"value":115}," is unavailable, make ",{"type":49,"tag":92,"props":117,"children":119},{"className":118},[],[120],{"type":55,"value":121},"ibv_devinfo",{"type":55,"value":123}," available or provide equivalent per-port evidence.",{"type":49,"tag":81,"props":125,"children":126},{},[127,128,134],{"type":55,"value":90},{"type":49,"tag":92,"props":129,"children":131},{"className":130},[],[132],{"type":55,"value":133},"nvshmem-info",{"type":55,"value":135}," available on the target node, or supply the installed NVSHMEM version explicitly.",{"type":49,"tag":81,"props":137,"children":138},{},[139],{"type":55,"value":140},"Use the bundled read-only collector when possible. It requires no API keys, credentials, package installation, or writable filesystem access.",{"type":49,"tag":58,"props":142,"children":144},{"id":143},"inputs",[145],{"type":55,"value":146},"Inputs",{"type":49,"tag":65,"props":148,"children":149},{},[150],{"type":55,"value":151},"Resolve every required input before emitting exact exports. If values from different sources conflict, identify the conflict and request clarification rather than selecting a value silently.",{"type":49,"tag":153,"props":154,"children":156},"h3",{"id":155},"required-inputs",[157],{"type":55,"value":158},"Required inputs",{"type":49,"tag":77,"props":160,"children":161},{},[162,197,207,217,243],{"type":49,"tag":81,"props":163,"children":164},{},[165,171,173,179,181,187,189,195],{"type":49,"tag":166,"props":167,"children":168},"strong",{},[169],{"type":55,"value":170},"NVSHMEM version:",{"type":55,"value":172}," Prefer the user-confirmed installed version; otherwise use the collector's ",{"type":49,"tag":92,"props":174,"children":176},{"className":175},[],[177],{"type":55,"value":178},"[nvshmem-version]",{"type":55,"value":180}," output, then ",{"type":49,"tag":92,"props":182,"children":184},{"className":183},[],[185],{"type":55,"value":186},"nvshmem-info -n",{"type":55,"value":188}," on the target node. Retain the complete reported version after normalizing a leading ",{"type":49,"tag":92,"props":190,"children":192},{"className":191},[],[193],{"type":55,"value":194},"v",{"type":55,"value":196},".",{"type":49,"tag":81,"props":198,"children":199},{},[200,205],{"type":49,"tag":166,"props":201,"children":202},{},[203],{"type":55,"value":204},"Remote transport:",{"type":55,"value":206}," Prefer the explicitly selected transport for the target run; otherwise use the target node's NVSHMEM initialization log. Do not infer a transport from topology; request a log when automatic selection is ambiguous.",{"type":49,"tag":81,"props":208,"children":209},{},[210,215],{"type":49,"tag":166,"props":211,"children":212},{},[213],{"type":55,"value":214},"Local PE count:",{"type":55,"value":216}," Prefer observed per-rank logs, then launcher configuration, then an explicit user-provided count. Confirm that all evidence describes the same node allocation.",{"type":49,"tag":81,"props":218,"children":219},{},[220,233,235,241],{"type":49,"tag":166,"props":221,"children":222},{},[223,225,231],{"type":55,"value":224},"Ordered local ",{"type":49,"tag":92,"props":226,"children":228},{"className":227},[],[229],{"type":55,"value":230},"PE -> GPU index or PCI BDF",{"type":55,"value":232}," binding:",{"type":55,"value":234}," Prefer observed per-rank binding logs, then launcher binding configuration, then an explicit user mapping. Use ",{"type":49,"tag":92,"props":236,"children":238},{"className":237},[],[239],{"type":55,"value":240},"CUDA_VISIBLE_DEVICES",{"type":55,"value":242}," only when its rank order is explicitly established.",{"type":49,"tag":81,"props":244,"children":245},{},[246,251,253,259,261,266,268,273,275,280],{"type":49,"tag":166,"props":247,"children":248},{},[249],{"type":55,"value":250},"Target-node GPU\u002FNIC and HCA-port topology:",{"type":55,"value":252}," Prefer the newest complete output from ",{"type":49,"tag":92,"props":254,"children":256},{"className":255},[],[257],{"type":55,"value":258},"scripts\u002Fcollect-nic-topology.sh",{"type":55,"value":260}," on the target node. Otherwise use the listed ",{"type":49,"tag":92,"props":262,"children":264},{"className":263},[],[265],{"type":55,"value":97},{"type":55,"value":267}," commands plus per-port ",{"type":49,"tag":92,"props":269,"children":271},{"className":270},[],[272],{"type":55,"value":105},{"type":55,"value":274}," or ",{"type":49,"tag":92,"props":276,"children":278},{"className":277},[],[279],{"type":55,"value":121},{"type":55,"value":281}," evidence. Include GPU and NIC PCI BDFs, port state, and link layer.",{"type":49,"tag":153,"props":283,"children":285},{"id":284},"optional-inputs",[286],{"type":55,"value":287},"Optional inputs",{"type":49,"tag":77,"props":289,"children":290},{},[291,301,311],{"type":49,"tag":81,"props":292,"children":293},{},[294,299],{"type":49,"tag":166,"props":295,"children":296},{},[297],{"type":55,"value":298},"Requested link layer:",{"type":55,"value":300}," Use the user's explicit InfiniBand or Ethernet\u002FRoCE preference. If absent, choose from observed active ports as described in step 5.",{"type":49,"tag":81,"props":302,"children":303},{},[304,309],{"type":49,"tag":166,"props":305,"children":306},{},[307],{"type":55,"value":308},"Excess-port policy:",{"type":55,"value":310}," Use the user's explicit request to select a closest subset or use multiple NICs per PE. If absent, present supported alternatives conditionally and make no selection.",{"type":49,"tag":81,"props":312,"children":313},{},[314,319,321,327],{"type":49,"tag":166,"props":315,"children":316},{},[317],{"type":55,"value":318},"Current official documentation or citations:",{"type":55,"value":320}," Obtain only when the user requests them or the version\u002Ftransport matrix is insufficient. Use ",{"type":49,"tag":92,"props":322,"children":324},{"className":323},[],[325],{"type":55,"value":326},"$nvshmem-docs",{"type":55,"value":328}," for that check.",{"type":49,"tag":58,"props":330,"children":332},{"id":331},"scripts",[333],{"type":55,"value":334},"Scripts",{"type":49,"tag":336,"props":337,"children":338},"table",{},[339,363],{"type":49,"tag":340,"props":341,"children":342},"thead",{},[343],{"type":49,"tag":344,"props":345,"children":346},"tr",{},[347,353,358],{"type":49,"tag":348,"props":349,"children":350},"th",{},[351],{"type":55,"value":352},"Script",{"type":49,"tag":348,"props":354,"children":355},{},[356],{"type":55,"value":357},"Use",{"type":49,"tag":348,"props":359,"children":360},{},[361],{"type":55,"value":362},"Invocation",{"type":49,"tag":364,"props":365,"children":366},"tbody",{},[367],{"type":49,"tag":344,"props":368,"children":369},{},[370,379,384],{"type":49,"tag":371,"props":372,"children":373},"td",{},[374],{"type":49,"tag":92,"props":375,"children":377},{"className":376},[],[378],{"type":55,"value":258},{"type":49,"tag":371,"props":380,"children":381},{},[382],{"type":55,"value":383},"Collect read-only NVSHMEM version, GPU PCI, GPU\u002FNIC topology, and RDMA-port evidence. It emits all evidence to stdout and may exit nonzero after producing useful partial output.",{"type":49,"tag":371,"props":385,"children":386},{},[387,389,395,397,402,404,410],{"type":55,"value":388},"Resolve the script from this skill directory and use ",{"type":49,"tag":92,"props":390,"children":392},{"className":391},[],[393],{"type":55,"value":394},"run_script",{"type":55,"value":396}," on the target compute node. If ",{"type":49,"tag":92,"props":398,"children":400},{"className":399},[],[401],{"type":55,"value":394},{"type":55,"value":403}," is unavailable, run ",{"type":49,"tag":92,"props":405,"children":407},{"className":406},[],[408],{"type":55,"value":409},"bash scripts\u002Fcollect-nic-topology.sh",{"type":55,"value":196},{"type":49,"tag":58,"props":412,"children":414},{"id":413},"read-the-required-references",[415],{"type":55,"value":416},"Read the Required References",{"type":49,"tag":77,"props":418,"children":419},{},[420,433,444],{"type":49,"tag":81,"props":421,"children":422},{},[423,425,431],{"type":55,"value":424},"Read ",{"type":49,"tag":426,"props":427,"children":429},"a",{"href":428},"references\u002Fversion-transport-matrix.md",[430],{"type":55,"value":428},{"type":55,"value":432}," before interpreting or emitting any mapping variable.",{"type":49,"tag":81,"props":434,"children":435},{},[436,437,442],{"type":55,"value":424},{"type":49,"tag":426,"props":438,"children":440},{"href":439},"references\u002Ftopology-and-output.md",[441],{"type":55,"value":439},{"type":55,"value":443}," before ranking NICs or formatting a recommendation.",{"type":49,"tag":81,"props":445,"children":446},{},[447,449,454,456,461],{"type":55,"value":448},"Treat the bundled ",{"type":49,"tag":166,"props":450,"children":451},{},[452],{"type":55,"value":453},"NVSHMEM 3.8 or later BLOCK contract",{"type":55,"value":455}," as authoritative within its recorded transport scope. Do not invoke ",{"type":49,"tag":92,"props":457,"children":459},{"className":458},[],[460],{"type":55,"value":326},{"type":55,"value":462}," merely to revalidate behavior that this contract resolves, including for releases later than 3.8.",{"type":49,"tag":58,"props":464,"children":466},{"id":465},"instructions",[467],{"type":55,"value":468},"Instructions",{"type":49,"tag":153,"props":470,"children":472},{"id":471},"_1-resolve-the-nvshmem-version-and-transport",[473],{"type":55,"value":474},"1. Resolve the NVSHMEM Version and Transport",{"type":49,"tag":65,"props":476,"children":477},{},[478,480,485,487,492,494,499],{"type":55,"value":479},"Obtain the version from the user first. Otherwise use the collector's ",{"type":49,"tag":92,"props":481,"children":483},{"className":482},[],[484],{"type":55,"value":178},{"type":55,"value":486}," evidence or run ",{"type":49,"tag":92,"props":488,"children":490},{"className":489},[],[491],{"type":55,"value":186},{"type":55,"value":493}," when it is already available on the target node. Normalize a leading ",{"type":49,"tag":92,"props":495,"children":497},{"className":496},[],[498],{"type":55,"value":194},{"type":55,"value":500}," and retain the complete reported version.",{"type":49,"tag":65,"props":502,"children":503},{},[504],{"type":55,"value":505},"Classify the release as:",{"type":49,"tag":77,"props":507,"children":508},{},[509,514],{"type":49,"tag":81,"props":510,"children":511},{},[512],{"type":55,"value":513},"3.7 or earlier;",{"type":49,"tag":81,"props":515,"children":516},{},[517],{"type":55,"value":518},"3.8 or later.",{"type":49,"tag":65,"props":520,"children":521},{},[522,524,530],{"type":55,"value":523},"Do not assume ",{"type":49,"tag":92,"props":525,"children":527},{"className":526},[],[528],{"type":55,"value":529},"latest",{"type":55,"value":531}," or silently treat an unknown version as 3.8. If the version remains unknown, give only conditional guidance and request the installed version before emitting final exports.",{"type":49,"tag":65,"props":533,"children":534},{},[535],{"type":55,"value":536},"Determine the selected remote transport: IBRC, IBDevX, IBGDA, GPUNetIO, UCX, or libfabric. If automatic selection makes the transport ambiguous, ask for the relevant initialization log or the intended transport.",{"type":49,"tag":153,"props":538,"children":540},{"id":539},"_2-resolve-documentation-uncertainty",[541],{"type":55,"value":542},"2. Resolve Documentation Uncertainty",{"type":49,"tag":65,"props":544,"children":545},{},[546,548,552,554,559,561,565],{"type":55,"value":547},"Use ",{"type":49,"tag":426,"props":549,"children":550},{"href":428},[551],{"type":55,"value":428},{"type":55,"value":553}," as the default authority for mapping behavior. Do not invoke ",{"type":49,"tag":92,"props":555,"children":557},{"className":556},[],[558],{"type":55,"value":326},{"type":55,"value":560}," when the matrix or the bundled ",{"type":49,"tag":166,"props":562,"children":563},{},[564],{"type":55,"value":453},{"type":55,"value":566}," resolves the version, transport, feature gate, and required mapping controls, including for a release later than 3.8.",{"type":49,"tag":65,"props":568,"children":569},{},[570,572,577,579,585],{"type":55,"value":571},"Invoke the sibling ",{"type":49,"tag":92,"props":573,"children":575},{"className":574},[],[576],{"type":55,"value":326},{"type":55,"value":578}," skill at ",{"type":49,"tag":92,"props":580,"children":582},{"className":581},[],[583],{"type":55,"value":584},"..\u002Fnvshmem-docs\u002FSKILL.md",{"type":55,"value":586}," only when the matrix and bundled contract are insufficient or conflicting, exact feature-version applicability remains unclear, or the user explicitly requests live official documentation or citations. Follow that skill's version and citation workflow.",{"type":49,"tag":65,"props":588,"children":589},{},[590],{"type":55,"value":591},"If a documentation check is needed but the sibling skill, an HTTPS retrieval tool, or the live official pages are unavailable:",{"type":49,"tag":77,"props":593,"children":594},{},[595,600,605,618,623],{"type":49,"tag":81,"props":596,"children":597},{},[598],{"type":55,"value":599},"continue from the matrix only when it resolves the needed behavior;",{"type":49,"tag":81,"props":601,"children":602},{},[603],{"type":55,"value":604},"otherwise withhold final exports and identify the unresolved behavior;",{"type":49,"tag":81,"props":606,"children":607},{},[608,610,616],{"type":55,"value":609},"label ",{"type":49,"tag":92,"props":611,"children":613},{"className":612},[],[614],{"type":55,"value":615},"Documentation freshness: unverified",{"type":55,"value":617}," only when a documentation check was attempted;",{"type":49,"tag":81,"props":619,"children":620},{},[621],{"type":55,"value":622},"state which official routes could not be checked; and",{"type":49,"tag":81,"props":624,"children":625},{},[626],{"type":55,"value":627},"never invent a behavior that the matrix does not establish.",{"type":49,"tag":65,"props":629,"children":630},{},[631],{"type":55,"value":632},"When a documentation check is performed, report the exact documentation edition consulted and any remaining uncertainty. Otherwise identify the applicable matrix section as the basis for the recommendation.",{"type":49,"tag":153,"props":634,"children":636},{"id":635},"_3-collect-topology-evidence",[637],{"type":55,"value":638},"3. Collect Topology Evidence",{"type":49,"tag":65,"props":640,"children":641},{},[642],{"type":55,"value":643},"When already running on the target compute node, tell the user that the skill is running a read-only collector, then run:",{"type":49,"tag":645,"props":646,"children":651},"pre",{"className":647,"code":648,"language":649,"meta":650,"style":650},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","bash scripts\u002Fcollect-nic-topology.sh\n","bash","",[652],{"type":49,"tag":92,"props":653,"children":654},{"__ignoreMap":650},[655],{"type":49,"tag":656,"props":657,"children":660},"span",{"class":658,"line":659},"line",1,[661,666],{"type":49,"tag":656,"props":662,"children":664},{"style":663},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[665],{"type":55,"value":649},{"type":49,"tag":656,"props":667,"children":669},{"style":668},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[670],{"type":55,"value":671}," scripts\u002Fcollect-nic-topology.sh\n",{"type":49,"tag":65,"props":673,"children":674},{},[675,677,683,685,690,692,697],{"type":55,"value":676},"Resolve the bundled script path from ",{"type":49,"tag":92,"props":678,"children":680},{"className":679},[],[681],{"type":55,"value":682},"SKILL.md",{"type":55,"value":684},". Use ",{"type":49,"tag":92,"props":686,"children":688},{"className":687},[],[689],{"type":55,"value":394},{"type":55,"value":691}," when available; otherwise run the shell command above. The collector writes evidence only to stdout and may return a nonzero status with useful partial output. Do not use ",{"type":49,"tag":92,"props":693,"children":695},{"className":694},[],[696],{"type":55,"value":394},{"type":55,"value":698}," or the shell fallback from a login node as topology evidence.",{"type":49,"tag":65,"props":700,"children":701},{},[702],{"type":55,"value":703},"When not on the target node, ask the user to run that command there and paste the complete output. If the script is unavailable remotely, request all of:",{"type":49,"tag":645,"props":705,"children":707},{"className":647,"code":706,"language":649,"meta":650,"style":650},"nvshmem-info -n\nnvidia-smi --query-gpu=index,pci.bus_id,name --format=csv,noheader\nnvidia-smi topo -m\n",[708],{"type":49,"tag":92,"props":709,"children":710},{"__ignoreMap":650},[711,723,741],{"type":49,"tag":656,"props":712,"children":713},{"class":658,"line":659},[714,718],{"type":49,"tag":656,"props":715,"children":716},{"style":663},[717],{"type":55,"value":133},{"type":49,"tag":656,"props":719,"children":720},{"style":668},[721],{"type":55,"value":722}," -n\n",{"type":49,"tag":656,"props":724,"children":726},{"class":658,"line":725},2,[727,731,736],{"type":49,"tag":656,"props":728,"children":729},{"style":663},[730],{"type":55,"value":97},{"type":49,"tag":656,"props":732,"children":733},{"style":668},[734],{"type":55,"value":735}," --query-gpu=index,pci.bus_id,name",{"type":49,"tag":656,"props":737,"children":738},{"style":668},[739],{"type":55,"value":740}," --format=csv,noheader\n",{"type":49,"tag":656,"props":742,"children":744},{"class":658,"line":743},3,[745,749,754],{"type":49,"tag":656,"props":746,"children":747},{"style":663},[748],{"type":55,"value":97},{"type":49,"tag":656,"props":750,"children":751},{"style":668},[752],{"type":55,"value":753}," topo",{"type":49,"tag":656,"props":755,"children":756},{"style":668},[757],{"type":55,"value":758}," -m\n",{"type":49,"tag":65,"props":760,"children":761},{},[762,764,769,770,775],{"type":55,"value":763},"Also request, for every candidate HCA port, its HCA name, port number, PCI BDF, state, and link layer from ",{"type":49,"tag":92,"props":765,"children":767},{"className":766},[],[768],{"type":55,"value":105},{"type":55,"value":274},{"type":49,"tag":92,"props":771,"children":773},{"className":772},[],[774],{"type":55,"value":121},{"type":55,"value":196},{"type":49,"tag":65,"props":777,"children":778},{},[779],{"type":55,"value":780},"Do not run a collector from a login node and treat that topology as compute-node evidence.",{"type":49,"tag":153,"props":782,"children":784},{"id":783},"_4-establish-pe-placement",[785],{"type":55,"value":786},"4. Establish PE Placement",{"type":49,"tag":65,"props":788,"children":789},{},[790],{"type":55,"value":791},"Obtain the number of local PEs per node. Assume nodes are homogeneous only after stating that assumption.",{"type":49,"tag":65,"props":793,"children":794},{},[795,797,802,804,809,811,817],{"type":55,"value":796},"Obtain the ordered local ",{"type":49,"tag":92,"props":798,"children":800},{"className":799},[],[801],{"type":55,"value":230},{"type":55,"value":803}," binding from explicit user input, launcher configuration, ",{"type":49,"tag":92,"props":805,"children":807},{"className":806},[],[808],{"type":55,"value":240},{"type":55,"value":810},", or per-rank logs. Do not infer a local PE's GPU solely from PE count or the order displayed by ",{"type":49,"tag":92,"props":812,"children":814},{"className":813},[],[815],{"type":55,"value":816},"nvidia-smi topo -m",{"type":55,"value":196},{"type":49,"tag":65,"props":819,"children":820},{},[821],{"type":55,"value":822},"If the binding is ambiguous, request it before producing an exact closest-NIC mapping. A conditional table is acceptable, but final exports are not. Do not bypass the missing binding by duplicating every usable NIC for every PE; that adds non-closest paths and is a different multi-port policy. Consider such an all-ports assignment only after the user explicitly requests it and accepts the distance tradeoff.",{"type":49,"tag":153,"props":824,"children":826},{"id":825},"_5-filter-and-rank-candidate-ports",[827],{"type":55,"value":828},"5. Filter and Rank Candidate Ports",{"type":49,"tag":65,"props":830,"children":831},{},[832],{"type":55,"value":833},"Treat an HCA port, not merely an HCA device, as the selectable NIC unit.",{"type":49,"tag":77,"props":835,"children":836},{},[837,842,847,852,871,884],{"type":49,"tag":81,"props":838,"children":839},{},[840],{"type":55,"value":841},"Exclude inactive or physically down ports and record why each was excluded.",{"type":49,"tag":81,"props":843,"children":844},{},[845],{"type":55,"value":846},"Partition active ports by link layer. Do not mix InfiniBand with Ethernet\u002FRoCE by default.",{"type":49,"tag":81,"props":848,"children":849},{},[850],{"type":55,"value":851},"Honor an explicitly requested link layer. Otherwise choose the larger active partition; prefer InfiniBand on an exact tie. Ask the user if that default conflicts with their fabric or provider intent.",{"type":49,"tag":81,"props":853,"children":854},{},[855,857,863,865,870],{"type":55,"value":856},"Rank each bound GPU-to-port pair by ",{"type":49,"tag":92,"props":858,"children":860},{"className":859},[],[861],{"type":55,"value":862},"PIX \u003C PXB \u003C PHB \u003C NODE \u003C SYS",{"type":55,"value":864}," from ",{"type":49,"tag":92,"props":866,"children":868},{"className":867},[],[869],{"type":55,"value":816},{"type":55,"value":196},{"type":49,"tag":81,"props":872,"children":873},{},[874,876,882],{"type":55,"value":875},"Break equal-distance choices by lower assigned-port load, then canonical ",{"type":49,"tag":92,"props":877,"children":879},{"className":878},[],[880],{"type":55,"value":881},"hca:port",{"type":55,"value":883}," order.",{"type":49,"tag":81,"props":885,"children":886},{},[887],{"type":55,"value":888},"Share a port when fewer usable ports than PEs makes sharing necessary, while preserving distance first and balance second.",{"type":49,"tag":153,"props":890,"children":892},{"id":891},"_6-resolve-excess-nics-explicitly",[893],{"type":55,"value":894},"6. Resolve Excess NICs Explicitly",{"type":49,"tag":65,"props":896,"children":897},{},[898],{"type":55,"value":899},"When usable NIC ports outnumber local PEs, stop before final exports and ask whether to:",{"type":49,"tag":901,"props":902,"children":903},"ol",{},[904,909],{"type":49,"tag":81,"props":905,"children":906},{},[907],{"type":55,"value":908},"select the closest balanced subset; or",{"type":49,"tag":81,"props":910,"children":911},{},[912],{"type":55,"value":913},"use multiple NICs per PE.",{"type":49,"tag":65,"props":915,"children":916},{},[917],{"type":55,"value":918},"Offer option 2 only when the resolved version and transport support it according to the matrix. Explain whether multi-NIC requires automatic distance assignment or supports an explicit BLOCK mapping. If the user does not answer, show both supported alternatives as conditional recommendations and do not choose silently.",{"type":49,"tag":65,"props":920,"children":921},{},[922],{"type":55,"value":923},"If the user asked to use all NICs but the resolved version\u002Ftransport does not support multi-NIC, explain the restriction and request approval to use the closest subset. Do not treat the unsupported request as implicit approval for a subset, and do not emit final exports until the user confirms it. When only the subset is supported, present that single supported choice for confirmation instead of offering a disabled multi-NIC option.",{"type":49,"tag":153,"props":925,"children":927},{"id":926},"_7-synthesize-exactly-one-selection-mechanism",[928],{"type":55,"value":929},"7. Synthesize Exactly One Selection Mechanism",{"type":49,"tag":65,"props":931,"children":932},{},[933,935,941,943,949],{"type":55,"value":934},"Prefer ",{"type":49,"tag":92,"props":936,"children":938},{"className":937},[],[939],{"type":55,"value":940},"NVSHMEM_HCA_PE_MAPPING",{"type":55,"value":942}," for exact assignments. Use ",{"type":49,"tag":92,"props":944,"children":946},{"className":945},[],[947],{"type":55,"value":948},"NVSHMEM_HCA_LIST",{"type":55,"value":950}," only when selecting an eligible set and accepting transport-canonical ordering. Never set both variables.",{"type":49,"tag":65,"props":952,"children":953},{},[954],{"type":55,"value":955},"Apply the version\u002Ftransport rules in the matrix exactly:",{"type":49,"tag":77,"props":957,"children":958},{},[959,964,985],{"type":49,"tag":81,"props":960,"children":961},{},[962],{"type":55,"value":963},"For 3.7 and earlier, preserve one expanded mapping slot per PE for explicit IB mappings. Use automatic distance assignment for supported multi-NIC paths.",{"type":49,"tag":81,"props":965,"children":966},{},[967,969,975,977,983],{"type":55,"value":968},"For 3.8 or later, apply the bundled BLOCK contract: set ",{"type":49,"tag":92,"props":970,"children":972},{"className":971},[],[973],{"type":55,"value":974},"NVSHMEM_ENABLE_NIC_PE_MAPPING=1",{"type":55,"value":976}," for explicit assignments and ",{"type":49,"tag":92,"props":978,"children":980},{"className":979},[],[981],{"type":55,"value":982},"NVSHMEM_ENABLE_MULTI_PORT=1",{"type":55,"value":984}," whenever recommending more than one NIC per PE.",{"type":49,"tag":81,"props":986,"children":987},{},[988],{"type":55,"value":989},"For UCX and libfabric, explain the provider-controlled boundary and do not fabricate common-HCA exact mappings.",{"type":49,"tag":65,"props":991,"children":992},{},[993,995,1000,1002,1008],{"type":55,"value":994},"Build ",{"type":49,"tag":92,"props":996,"children":998},{"className":997},[],[999],{"type":55,"value":940},{"type":55,"value":1001}," with explicit ",{"type":49,"tag":92,"props":1003,"children":1005},{"className":1004},[],[1006],{"type":55,"value":1007},"hca:port:count",{"type":55,"value":1009}," entries. Expand every count before explaining which slots each PE receives. Preserve duplicates when intentional sharing is required.",{"type":49,"tag":65,"props":1011,"children":1012},{},[1013,1015,1021],{"type":55,"value":1014},"Keep a mapping-only export block limited to NIC-assignment controls. Do not add ",{"type":49,"tag":92,"props":1016,"children":1018},{"className":1017},[],[1019],{"type":55,"value":1020},"NVSHMEM_REMOTE_TRANSPORT",{"type":55,"value":1022}," or provider\u002Flauncher settings merely because the transport is known; include them only when the user separately asks to select the transport and the matrix, or documentation when needed, verifies the value.",{"type":49,"tag":153,"props":1024,"children":1026},{"id":1025},"_8-return-the-recommendation",[1027],{"type":55,"value":1028},"8. Return the Recommendation",{"type":49,"tag":65,"props":1030,"children":1031},{},[1032,1034,1040],{"type":55,"value":1033},"Execute the ",{"type":49,"tag":426,"props":1035,"children":1037},{"href":1036},"references\u002Ftopology-and-output.md#output-contract",[1038],{"type":55,"value":1039},"Output Contract",{"type":55,"value":1041}," in order; that reference is the single authority for output fields, formatting, withholding messages, and validation guidance.",{"type":49,"tag":65,"props":1043,"children":1044},{},[1045,1047,1053,1054,1060],{"type":55,"value":1046},"Treat its ",{"type":49,"tag":92,"props":1048,"children":1050},{"className":1049},[],[1051],{"type":55,"value":1052},"Mapping",{"type":55,"value":99},{"type":49,"tag":92,"props":1055,"children":1057},{"className":1056},[],[1058],{"type":55,"value":1059},"Exact Exports",{"type":55,"value":1061}," sections as mandatory gates. Do not return until each gate is either completed or explicitly withheld for the reason allowed by the contract. Do not duplicate or paraphrase the contract in this file.",{"type":49,"tag":58,"props":1063,"children":1065},{"id":1064},"examples",[1066],{"type":55,"value":1067},"Examples",{"type":49,"tag":77,"props":1069,"children":1070},{},[1071,1076,1088],{"type":49,"tag":81,"props":1072,"children":1073},{},[1074],{"type":55,"value":1075},"“My 4 local PEs are bound to GPUs 0–3. Given this target-node collector output and NVSHMEM 3.8.0 IBRC, recommend the closest balanced InfiniBand port mapping and exact exports.”",{"type":49,"tag":81,"props":1077,"children":1078},{},[1079,1081,1086],{"type":55,"value":1080},"“Interpret this ",{"type":49,"tag":92,"props":1082,"children":1084},{"className":1083},[],[1085],{"type":55,"value":816},{"type":55,"value":1087}," and per-port state\u002Flink-layer output. NVSHMEM is 3.7 with IBGDA; identify whether multiple NICs per PE are supported and what extra evidence is needed before emitting exports.”",{"type":49,"tag":81,"props":1089,"children":1090},{},[1091],{"type":55,"value":1092},"“We have 2 PEs and 5 active Ethernet\u002FRoCE ports. Use this PE-to-GPU binding and initialization log to show the supported subset and multi-NIC alternatives without selecting one silently.”",{"type":49,"tag":58,"props":1094,"children":1096},{"id":1095},"troubleshooting",[1097],{"type":55,"value":1098},"Troubleshooting",{"type":49,"tag":336,"props":1100,"children":1101},{},[1102,1118],{"type":49,"tag":340,"props":1103,"children":1104},{},[1105],{"type":49,"tag":344,"props":1106,"children":1107},{},[1108,1113],{"type":49,"tag":348,"props":1109,"children":1110},{},[1111],{"type":55,"value":1112},"Symptom",{"type":49,"tag":348,"props":1114,"children":1115},{},[1116],{"type":55,"value":1117},"Response",{"type":49,"tag":364,"props":1119,"children":1120},{},[1121,1142,1155,1168,1188],{"type":49,"tag":344,"props":1122,"children":1123},{},[1124,1137],{"type":49,"tag":371,"props":1125,"children":1126},{},[1127,1129,1135],{"type":55,"value":1128},"The collector reports ",{"type":49,"tag":92,"props":1130,"children":1132},{"className":1131},[],[1133],{"type":55,"value":1134},"status=partial",{"type":55,"value":1136},", missing GPUs, or no HCA ports.",{"type":49,"tag":371,"props":1138,"children":1139},{},[1140],{"type":55,"value":1141},"Re-run it on an allocated target compute node with NVIDIA and RDMA device visibility. Paste the complete labeled output; a nonzero exit may still contain usable evidence.",{"type":49,"tag":344,"props":1143,"children":1144},{},[1145,1150],{"type":49,"tag":371,"props":1146,"children":1147},{},[1148],{"type":55,"value":1149},"No exact mapping can be produced.",{"type":49,"tag":371,"props":1151,"children":1152},{},[1153],{"type":55,"value":1154},"Obtain the NVSHMEM version, resolved remote transport, local PE-to-GPU binding, and active-port evidence. Give only conditional guidance until all are available.",{"type":49,"tag":344,"props":1156,"children":1157},{},[1158,1163],{"type":49,"tag":371,"props":1159,"children":1160},{},[1161],{"type":55,"value":1162},"Ports have mixed link layers or a requested port is inactive.",{"type":49,"tag":371,"props":1164,"children":1165},{},[1166],{"type":55,"value":1167},"Exclude inactive ports and select one link-layer partition unless the user explicitly requests another supported fabric.",{"type":49,"tag":344,"props":1169,"children":1170},{},[1171,1176],{"type":49,"tag":371,"props":1172,"children":1173},{},[1174],{"type":55,"value":1175},"NVSHMEM behavior is unclear for the selected version or transport.",{"type":49,"tag":371,"props":1177,"children":1178},{},[1179,1181,1186],{"type":55,"value":1180},"Check the version\u002Ftransport matrix first, then invoke ",{"type":49,"tag":92,"props":1182,"children":1184},{"className":1183},[],[1185],{"type":55,"value":326},{"type":55,"value":1187}," only if it remains insufficient or the user requires current official citations.",{"type":49,"tag":344,"props":1189,"children":1190},{},[1191,1196],{"type":49,"tag":371,"props":1192,"children":1193},{},[1194],{"type":55,"value":1195},"Logs select unexpected ports after applying the mapping.",{"type":49,"tag":371,"props":1197,"children":1198},{},[1199],{"type":55,"value":1200},"Verify the export block, PE binding, selected transport, and provider constraints; compare against the unforced automatic configuration before changing other settings.",{"type":49,"tag":58,"props":1202,"children":1204},{"id":1203},"limitations",[1205],{"type":55,"value":1206},"Limitations",{"type":49,"tag":77,"props":1208,"children":1209},{},[1210,1215,1220,1225],{"type":49,"tag":81,"props":1211,"children":1212},{},[1213],{"type":55,"value":1214},"Limit recommendations to the observed target-node topology and port state; repeat evidence collection after hardware, fabric, launcher, or binding changes.",{"type":49,"tag":81,"props":1216,"children":1217},{},[1218],{"type":55,"value":1219},"Treat topology proximity as a selection heuristic, not a performance guarantee; validate with a representative workload.",{"type":49,"tag":81,"props":1221,"children":1222},{},[1223],{"type":55,"value":1224},"Do not produce an exact mapping until the NVSHMEM version, transport, local PE-to-GPU binding, and usable-port evidence are resolved.",{"type":49,"tag":81,"props":1226,"children":1227},{},[1228],{"type":55,"value":1229},"For UCX and libfabric, remain within the provider-controlled boundary and do not fabricate common-HCA mapping settings.",{"type":49,"tag":58,"props":1231,"children":1233},{"id":1232},"guardrails",[1234],{"type":55,"value":1235},"Guardrails",{"type":49,"tag":77,"props":1237,"children":1238},{},[1239,1244,1249,1254,1259,1264,1269],{"type":49,"tag":81,"props":1240,"children":1241},{},[1242],{"type":55,"value":1243},"Do not select inactive ports or silently mix link layers.",{"type":49,"tag":81,"props":1245,"children":1246},{},[1247],{"type":55,"value":1248},"Do not assume that a numerically matching GPU and NIC index implies proximity.",{"type":49,"tag":81,"props":1250,"children":1251},{},[1252],{"type":55,"value":1253},"Do not turn an unknown PE-to-GPU binding into an all-ports-per-PE mapping.",{"type":49,"tag":81,"props":1255,"children":1256},{},[1257],{"type":55,"value":1258},"Do not emit 3.8-or-later BLOCK behavior for 3.7 or earlier.",{"type":49,"tag":81,"props":1260,"children":1261},{},[1262],{"type":55,"value":1263},"Do not emit legacy IBGDA multi-port controls as the primary 3.8-or-later control.",{"type":49,"tag":81,"props":1265,"children":1266},{},[1267],{"type":55,"value":1268},"Do not generate provider-native UCX or libfabric settings; explain them without inventing values.",{"type":49,"tag":81,"props":1270,"children":1271},{},[1272],{"type":55,"value":1273},"Do not overwrite user files or change the system while collecting evidence.",{"type":49,"tag":1275,"props":1276,"children":1277},"style",{},[1278],{"type":55,"value":1279},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":1281,"total":1439},[1282,1300,1317,1328,1340,1354,1367,1381,1394,1405,1419,1428],{"slug":1283,"name":1283,"fn":1284,"description":1285,"org":1286,"tags":1287,"stars":1297,"repoUrl":1298,"updatedAt":1299},"nemoclaw-user-guide","retrieve NemoClaw documentation and configuration","Guides human users' AI agents to the NemoClaw docs MCP server and canonical Fern documentation in Markdown form. Use when users ask how to install, configure, operate, troubleshoot, secure, or learn NemoClaw with an AI coding assistant. Trigger keywords - nemoclaw docs, use nemoclaw with ai agent, nemoclaw mcp docs, nemoclaw install help, nemoclaw quickstart, nemoclaw markdown docs, llms.txt, agent skills.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1288,1291,1294],{"name":1289,"slug":1290,"type":15},"Documentation","documentation",{"name":1292,"slug":1293,"type":15},"MCP","mcp",{"name":1295,"slug":1296,"type":15},"Search","search",21777,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FNemoClaw","2026-08-25T03:29:57.273192",{"slug":1301,"name":1301,"fn":1302,"description":1303,"org":1304,"tags":1305,"stars":1314,"repoUrl":1315,"updatedAt":1316},"mcore-build-and-dependency","manage Megatron-LM development environments","Container-based dev environment setup and dependency management for Megatron-LM. Covers acquiring and launching the CI container, uv package management, and updating uv.lock.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1306,1309,1312],{"name":1307,"slug":1308,"type":15},"Containers","containers",{"name":1310,"slug":1311,"type":15},"Deployment","deployment",{"name":1313,"slug":30,"type":15},"Python",17049,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FMegatron-LM","2026-07-27T06:06:11.249662",{"slug":1318,"name":1318,"fn":1319,"description":1320,"org":1321,"tags":1322,"stars":1314,"repoUrl":1315,"updatedAt":1327},"mcore-bump-base-image","update NVIDIA PyTorch base images","Bump the NVIDIA PyTorch base image (`nvcr.io\u002Fnvidia\u002Fpytorch:YY.MM-py3`) used by Megatron-LM CI. Covers the two pin sites (GitHub CI in `docker\u002F.ngc_version.dev` and GitLab CI in `.gitlab\u002Fstages\u002F01.build.yml`), the post-bump CI loop (re-run functional tests, refresh golden values, mark broken tests), and the gotchas that bit PRs",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1323,1326],{"name":1324,"slug":1325,"type":15},"CI\u002FCD","ci-cd",{"name":1310,"slug":1311,"type":15},"2026-07-14T05:25:59.97109",{"slug":1329,"name":1329,"fn":1330,"description":1331,"org":1332,"tags":1333,"stars":1314,"repoUrl":1315,"updatedAt":1339},"mcore-cicd","manage CI\u002FCD pipelines for Megatron-LM","CI\u002FCD reference for Megatron-LM. Covers CI pipeline structure, PR scope labels, triggering internal GitLab CI (which force-pushes the current branch to a pull-request\u002FBRANCH ref — always dry-run and verify the destination first; never run against shared or protected branches), and CI failure investigation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1334,1335,1336],{"name":1324,"slug":1325,"type":15},{"name":1310,"slug":1311,"type":15},{"name":1337,"slug":1338,"type":15},"GitHub","github","2026-08-28T14:38:16.959248",{"slug":1341,"name":1341,"fn":1342,"description":1343,"org":1344,"tags":1345,"stars":1314,"repoUrl":1315,"updatedAt":1353},"mcore-create-issue","investigate CI failures and create issues","Investigate a failing GitHub Actions run or job and create a GitHub issue for the failure.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1346,1349,1350],{"name":1347,"slug":1348,"type":15},"Debugging","debugging",{"name":1337,"slug":1338,"type":15},{"name":1351,"slug":1352,"type":15},"Triage","triage","2026-07-14T05:25:57.442089",{"slug":1355,"name":1355,"fn":1356,"description":1357,"org":1358,"tags":1359,"stars":1314,"repoUrl":1315,"updatedAt":1366},"mcore-linting-and-formatting","lint and format Megatron-LM code","Linting and formatting for Megatron-LM. Covers running autoformat.sh, tools (ruff, black, isort, pylint, mypy), and code style rules.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1360,1363],{"name":1361,"slug":1362,"type":15},"Best Practices","best-practices",{"name":1364,"slug":1365,"type":15},"Code Analysis","code-analysis","2026-07-14T05:25:56.18433",{"slug":1368,"name":1368,"fn":1369,"description":1370,"org":1371,"tags":1372,"stars":1314,"repoUrl":1315,"updatedAt":1380},"mcore-migrate-gpt-to-hybrid","migrate Megatron-LM models to HybridModel","Migration guide for moving Megatron Core GPTModel checkpoints, model providers, training commands, and layer mappings to HybridModel, including the mechanical steps for transferring an existing pretrain_gpt.py launch script.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1373,1376,1379],{"name":1374,"slug":1375,"type":15},"Machine Learning","machine-learning",{"name":1377,"slug":1378,"type":15},"Migration","migration",{"name":9,"slug":8,"type":15},"2026-08-21T03:36:57.470256",{"slug":1382,"name":1382,"fn":1383,"description":1384,"org":1385,"tags":1386,"stars":1314,"repoUrl":1315,"updatedAt":1393},"mcore-onboard-gb200-1node-tests","onboard functional tests for GB200","Onboard 1-node GitHub MR functional tests for GB200 from existing mr-scoped 2-node tests.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1387,1390],{"name":1388,"slug":1389,"type":15},"QA","qa",{"name":1391,"slug":1392,"type":15},"Testing","testing","2026-07-14T05:25:53.673039",{"slug":1395,"name":1395,"fn":1396,"description":1397,"org":1398,"tags":1399,"stars":1314,"repoUrl":1315,"updatedAt":1404},"mcore-run-on-slurm","launch distributed training jobs on SLURM","How to launch distributed Megatron-LM training jobs on a SLURM cluster. Covers a minimal sbatch skeleton, environment-variable setup for torch.distributed.run, CUDA_DEVICE_MAX_CONNECTIONS rules across hardware and parallelism modes, container conventions, monitoring, and per-rank failure diagnosis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1400,1401],{"name":1310,"slug":1311,"type":15},{"name":1402,"slug":1403,"type":15},"Infrastructure","infrastructure","2026-07-14T05:25:49.362534",{"slug":1406,"name":1406,"fn":1407,"description":1408,"org":1409,"tags":1410,"stars":1314,"repoUrl":1315,"updatedAt":1418},"mcore-split-pr","split pull requests to reduce review load","Split a PR into multiple PRs to reduce the number of required CODEOWNERS reviewer groups.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1411,1414,1415],{"name":1412,"slug":1413,"type":15},"Code Review","code-review",{"name":1337,"slug":1338,"type":15},{"name":1416,"slug":1417,"type":15},"Pull Requests","pull-requests","2026-08-25T03:29:16.211287",{"slug":1420,"name":1420,"fn":1421,"description":1422,"org":1423,"tags":1424,"stars":1314,"repoUrl":1315,"updatedAt":1427},"mcore-testing","run and manage Megatron-LM tests","Test system for Megatron-LM. Covers test layout, recipe YAML structure, adding and running unit and functional tests, golden values, marker filters, and CI parity.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1425,1426],{"name":1388,"slug":1389,"type":15},{"name":1391,"slug":1392,"type":15},"2026-07-14T05:25:54.928983",{"slug":1429,"name":1429,"fn":1430,"description":1431,"org":1432,"tags":1433,"stars":1314,"repoUrl":1315,"updatedAt":1438},"nightly-sync","manage nightly main-to-dev sync workflows","Domain knowledge for the nightly main-to-dev sync workflow. Covers merge strategy, CI architecture, failure investigation, and known issues.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1434,1437],{"name":1435,"slug":1436,"type":15},"Automation","automation",{"name":1324,"slug":1325,"type":15},"2026-07-30T05:29:03.275638",563,{"items":1441,"total":1518},[1442,1456,1462,1474,1486,1498,1508],{"slug":1443,"name":1443,"fn":1444,"description":1445,"org":1446,"tags":1447,"stars":20,"repoUrl":21,"updatedAt":1455},"nvshmem-collect-performance-data","collect NVSHMEM performance metrics","Collect and package NVSHMEM put\u002Fget bandwidth, latency, and other perftest results with system and topology evidence for performance sanity checks.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1448,1451,1452],{"name":1449,"slug":1450,"type":15},"Monitoring","monitoring",{"name":9,"slug":8,"type":15},{"name":1453,"slug":1454,"type":15},"Performance","performance","2026-08-31T09:19:00.917887",{"slug":4,"name":4,"fn":5,"description":6,"org":1457,"tags":1458,"stars":20,"repoUrl":21,"updatedAt":22},{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1459,1460,1461],{"name":17,"slug":18,"type":15},{"name":13,"slug":14,"type":15},{"name":9,"slug":8,"type":15},{"slug":1463,"name":1463,"fn":1464,"description":1465,"org":1466,"tags":1467,"stars":20,"repoUrl":21,"updatedAt":1473},"nvshmem-docs","fetch official NVSHMEM documentation","Find version-aware official NVSHMEM and NVSHMEM4Py documentation for releases, installation, APIs, runtime settings, transports, containers, and troubleshooting.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1468,1469,1470],{"name":1289,"slug":1290,"type":15},{"name":9,"slug":8,"type":15},{"name":1471,"slug":1472,"type":15},"Reference","reference","2026-08-31T09:19:00.214841",{"slug":1475,"name":1475,"fn":1476,"description":1477,"org":1478,"tags":1479,"stars":20,"repoUrl":21,"updatedAt":1485},"nvshmem-enable-tma","optimize NVSHMEM CUDA kernels for TMA","Prepare or review NVSHMEM CUDA kernels for TMA SMEM registration and direct-SMEM transfers. Do not use for unrelated CUDA tuning.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1480,1483,1484],{"name":1481,"slug":1482,"type":15},"Engineering","engineering",{"name":9,"slug":8,"type":15},{"name":1453,"slug":1454,"type":15},"2026-08-31T09:18:50.449676",{"slug":1487,"name":1487,"fn":1488,"description":1489,"org":1490,"tags":1491,"stars":20,"repoUrl":21,"updatedAt":1497},"nvshmem-get-started","onboard developers to NVSHMEM","Guide NVSHMEM beginners through fit assessment, mental models, first C\u002FC++ or Python NVSHMEM programs, compilation, launching, and next steps. Use for onboarding.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1492,1493,1494],{"name":1289,"slug":1290,"type":15},{"name":9,"slug":8,"type":15},{"name":1495,"slug":1496,"type":15},"Onboarding","onboarding","2026-08-31T09:18:55.48573",{"slug":1499,"name":1499,"fn":1500,"description":1501,"org":1502,"tags":1503,"stars":20,"repoUrl":21,"updatedAt":1507},"nvshmem-install","install and validate NVSHMEM","Plan and validate NVSHMEM and NVSHMEM4Py installations. Use for package, container, or source deployments.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1504,1505,1506],{"name":1310,"slug":1311,"type":15},{"name":1481,"slug":1482,"type":15},{"name":9,"slug":8,"type":15},"2026-08-31T09:18:50.105521",{"slug":1509,"name":1509,"fn":1510,"description":1511,"org":1512,"tags":1513,"stars":20,"repoUrl":21,"updatedAt":1517},"nvshmem-select-remote-transport","select NVSHMEM remote transport configurations","Select an NVSHMEM remote transport from target system and kernel evidence. Use for inter-node selection, compatibility checks, or configuration.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[1514,1515,1516],{"name":17,"slug":18,"type":15},{"name":9,"slug":8,"type":15},{"name":1453,"slug":1454,"type":15},"2026-08-31T09:18:59.493722",9]