[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-nvidia-diagnose-driver-install":3,"mdc--ixpo6f-key":31,"related-repo-nvidia-diagnose-driver-install":512,"related-org-nvidia-diagnose-driver-install":561},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":20,"repoUrl":21,"updatedAt":22,"license":23,"forks":24,"topics":25,"repo":26,"sourceUrl":29,"mdContent":30},"diagnose-driver-install","diagnose NVIDIA driver installation failures","Diagnose NVIDIA driver installation failures on DeepOps-managed nodes — nvidia-smi errors, \"No devices were found\", DKMS build failures, or GPU pods crash-looping. Use before reinstalling anything.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},"nvidia","NVIDIA","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fnvidia.png",[12,14,17],{"name":9,"slug":8,"type":13},"tag",{"name":15,"slug":16,"type":13},"Infrastructure","infrastructure",{"name":18,"slug":19,"type":13},"Debugging","debugging",1460,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fdeepops","2026-07-17T06:05:02.811247",null,353,[],{"repoUrl":21,"stars":20,"forks":24,"topics":27,"description":28},[],"Tools for building GPU clusters","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fdeepops\u002Ftree\u002FHEAD\u002Fskills\u002Fdiagnose-driver-install","---\nname: diagnose-driver-install\ndescription: Diagnose NVIDIA driver installation failures on DeepOps-managed nodes — nvidia-smi errors, \"No devices were found\", DKMS build failures, or GPU pods crash-looping. Use before reinstalling anything.\n---\n\n# Diagnose an NVIDIA driver install\n\nWork through these in order; most \"driver failures\" are one of the first\nthree and need no reinstall.\n\n## 1. Are you being fooled by GPU hiding?\n\nOn DeepOps **Slurm** nodes, GPUs are hidden from ordinary SSH sessions by\ndesign. Bare `nvidia-smi` over SSH reporting `No devices were found` on an\notherwise healthy node is expected.\n\n```bash\nsrun --gpus=1 nvidia-smi          # the authoritative test on Slurm nodes\n```\n\nIf the srun job sees the GPU, the driver is fine. Stop here.\n\n## 2. Is the hardware visible at all?\n\n```bash\nlspci | grep -i nvidia\n```\n\nNo output → not a driver problem. The GPU is absent, unseated, or bound by\nVFIO passthrough or platform firmware; escalate to hardware support before\ntouching software.\n\n## 3. Open vs proprietary kernel modules\n\n`nvidia-smi: No devices were found` immediately after a clean install is the\nclassic symptom of the wrong module flavor for the GPU generation:\n\n- Turing and newer (T4, A100, H100, RTX PRO Blackwell): open kernel modules\n  supported — DeepOps default `nvidia_driver_ubuntu_use_open_kernel_modules: true`\n  is correct.\n- Pascal and older (P100, GTX 10xx): open modules are **not** supported —\n  set `nvidia_driver_ubuntu_use_open_kernel_modules: false` in\n  `config\u002Fgroup_vars\u002Fall.yml` and rerun the driver play.\n\nCheck what is loaded: `modinfo nvidia | grep -i license` (open modules say\nMIT\u002FGPL, proprietary says NVIDIA).\n\n## 4. Did the kernel module actually build and load?\n\n```bash\ndkms status                        # driver module state per kernel\ndmesg | grep -iE 'nvidia|nvrm' | tail -20\nlsmod | grep nvidia\n```\n\n- DKMS shows an error for the running kernel → usually missing headers\n  (`linux-headers-$(uname -r)`) or a kernel updated after the driver\n  install. Install headers or reboot into the matching kernel, then rerun\n  the driver play.\n- Module loaded but `nvidia-smi` fails → check `dmesg` for RmInitAdapter or\n  fallen-off-the-bus errors; these are hardware\u002Ffirmware territory.\n\n## 5. Rerun the play, don't hand-fix\n\nAfter correcting configuration, converge with the playbook rather than\nmanual package surgery, then validate:\n\n```bash\nansible-playbook -l \u003Chost> playbooks\u002Fnvidia-software\u002Fnvidia-driver.yml\npython3 scripts\u002Fvalidation\u002Fvalidate_slurm.py --json    # or validate_k8s.py\n```\n\nReruns are idempotent; a healthy converged rerun reports `changed=0`.\n\n## Kubernetes-specific notes\n\nGPU Operator crash-looping driver pods: `kubectl logs -n \u003Cgpu-operator\nnamespace> \u003Cdriver-pod>` usually names the same root causes — missing\nheaders, wrong module flavor, or a node reboot needed. Fix via the operator\nvalues or node state, not by installing drivers by hand on the host (the\noperator owns the driver on Kubernetes nodes unless DeepOps was configured\nfor host drivers).\n",{"data":32,"body":33},{"name":4,"description":6},{"type":34,"children":35},"root",[36,45,51,58,88,128,133,139,175,180,186,197,243,256,262,359,394,400,405,474,487,493,506],{"type":37,"tag":38,"props":39,"children":41},"element","h1",{"id":40},"diagnose-an-nvidia-driver-install",[42],{"type":43,"value":44},"text","Diagnose an NVIDIA driver install",{"type":37,"tag":46,"props":47,"children":48},"p",{},[49],{"type":43,"value":50},"Work through these in order; most \"driver failures\" are one of the first\nthree and need no reinstall.",{"type":37,"tag":52,"props":53,"children":55},"h2",{"id":54},"_1-are-you-being-fooled-by-gpu-hiding",[56],{"type":43,"value":57},"1. Are you being fooled by GPU hiding?",{"type":37,"tag":46,"props":59,"children":60},{},[61,63,69,71,78,80,86],{"type":43,"value":62},"On DeepOps ",{"type":37,"tag":64,"props":65,"children":66},"strong",{},[67],{"type":43,"value":68},"Slurm",{"type":43,"value":70}," nodes, GPUs are hidden from ordinary SSH sessions by\ndesign. Bare ",{"type":37,"tag":72,"props":73,"children":75},"code",{"className":74},[],[76],{"type":43,"value":77},"nvidia-smi",{"type":43,"value":79}," over SSH reporting ",{"type":37,"tag":72,"props":81,"children":83},{"className":82},[],[84],{"type":43,"value":85},"No devices were found",{"type":43,"value":87}," on an\notherwise healthy node is expected.",{"type":37,"tag":89,"props":90,"children":95},"pre",{"className":91,"code":92,"language":93,"meta":94,"style":94},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","srun --gpus=1 nvidia-smi          # the authoritative test on Slurm nodes\n","bash","",[96],{"type":37,"tag":72,"props":97,"children":98},{"__ignoreMap":94},[99],{"type":37,"tag":100,"props":101,"children":104},"span",{"class":102,"line":103},"line",1,[105,111,117,122],{"type":37,"tag":100,"props":106,"children":108},{"style":107},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[109],{"type":43,"value":110},"srun",{"type":37,"tag":100,"props":112,"children":114},{"style":113},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[115],{"type":43,"value":116}," --gpus=1",{"type":37,"tag":100,"props":118,"children":119},{"style":113},[120],{"type":43,"value":121}," nvidia-smi",{"type":37,"tag":100,"props":123,"children":125},{"style":124},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[126],{"type":43,"value":127},"          # the authoritative test on Slurm nodes\n",{"type":37,"tag":46,"props":129,"children":130},{},[131],{"type":43,"value":132},"If the srun job sees the GPU, the driver is fine. Stop here.",{"type":37,"tag":52,"props":134,"children":136},{"id":135},"_2-is-the-hardware-visible-at-all",[137],{"type":43,"value":138},"2. Is the hardware visible at all?",{"type":37,"tag":89,"props":140,"children":142},{"className":91,"code":141,"language":93,"meta":94,"style":94},"lspci | grep -i nvidia\n",[143],{"type":37,"tag":72,"props":144,"children":145},{"__ignoreMap":94},[146],{"type":37,"tag":100,"props":147,"children":148},{"class":102,"line":103},[149,154,160,165,170],{"type":37,"tag":100,"props":150,"children":151},{"style":107},[152],{"type":43,"value":153},"lspci",{"type":37,"tag":100,"props":155,"children":157},{"style":156},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[158],{"type":43,"value":159}," |",{"type":37,"tag":100,"props":161,"children":162},{"style":107},[163],{"type":43,"value":164}," grep",{"type":37,"tag":100,"props":166,"children":167},{"style":113},[168],{"type":43,"value":169}," -i",{"type":37,"tag":100,"props":171,"children":172},{"style":113},[173],{"type":43,"value":174}," nvidia\n",{"type":37,"tag":46,"props":176,"children":177},{},[178],{"type":43,"value":179},"No output → not a driver problem. The GPU is absent, unseated, or bound by\nVFIO passthrough or platform firmware; escalate to hardware support before\ntouching software.",{"type":37,"tag":52,"props":181,"children":183},{"id":182},"_3-open-vs-proprietary-kernel-modules",[184],{"type":43,"value":185},"3. Open vs proprietary kernel modules",{"type":37,"tag":46,"props":187,"children":188},{},[189,195],{"type":37,"tag":72,"props":190,"children":192},{"className":191},[],[193],{"type":43,"value":194},"nvidia-smi: No devices were found",{"type":43,"value":196}," immediately after a clean install is the\nclassic symptom of the wrong module flavor for the GPU generation:",{"type":37,"tag":198,"props":199,"children":200},"ul",{},[201,215],{"type":37,"tag":202,"props":203,"children":204},"li",{},[205,207,213],{"type":43,"value":206},"Turing and newer (T4, A100, H100, RTX PRO Blackwell): open kernel modules\nsupported — DeepOps default ",{"type":37,"tag":72,"props":208,"children":210},{"className":209},[],[211],{"type":43,"value":212},"nvidia_driver_ubuntu_use_open_kernel_modules: true",{"type":43,"value":214},"\nis correct.",{"type":37,"tag":202,"props":216,"children":217},{},[218,220,225,227,233,235,241],{"type":43,"value":219},"Pascal and older (P100, GTX 10xx): open modules are ",{"type":37,"tag":64,"props":221,"children":222},{},[223],{"type":43,"value":224},"not",{"type":43,"value":226}," supported —\nset ",{"type":37,"tag":72,"props":228,"children":230},{"className":229},[],[231],{"type":43,"value":232},"nvidia_driver_ubuntu_use_open_kernel_modules: false",{"type":43,"value":234}," in\n",{"type":37,"tag":72,"props":236,"children":238},{"className":237},[],[239],{"type":43,"value":240},"config\u002Fgroup_vars\u002Fall.yml",{"type":43,"value":242}," and rerun the driver play.",{"type":37,"tag":46,"props":244,"children":245},{},[246,248,254],{"type":43,"value":247},"Check what is loaded: ",{"type":37,"tag":72,"props":249,"children":251},{"className":250},[],[252],{"type":43,"value":253},"modinfo nvidia | grep -i license",{"type":43,"value":255}," (open modules say\nMIT\u002FGPL, proprietary says NVIDIA).",{"type":37,"tag":52,"props":257,"children":259},{"id":258},"_4-did-the-kernel-module-actually-build-and-load",[260],{"type":43,"value":261},"4. Did the kernel module actually build and load?",{"type":37,"tag":89,"props":263,"children":265},{"className":91,"code":264,"language":93,"meta":94,"style":94},"dkms status                        # driver module state per kernel\ndmesg | grep -iE 'nvidia|nvrm' | tail -20\nlsmod | grep nvidia\n",[266],{"type":37,"tag":72,"props":267,"children":268},{"__ignoreMap":94},[269,287,338],{"type":37,"tag":100,"props":270,"children":271},{"class":102,"line":103},[272,277,282],{"type":37,"tag":100,"props":273,"children":274},{"style":107},[275],{"type":43,"value":276},"dkms",{"type":37,"tag":100,"props":278,"children":279},{"style":113},[280],{"type":43,"value":281}," status",{"type":37,"tag":100,"props":283,"children":284},{"style":124},[285],{"type":43,"value":286},"                        # driver module state per kernel\n",{"type":37,"tag":100,"props":288,"children":290},{"class":102,"line":289},2,[291,296,300,304,309,314,319,324,328,333],{"type":37,"tag":100,"props":292,"children":293},{"style":107},[294],{"type":43,"value":295},"dmesg",{"type":37,"tag":100,"props":297,"children":298},{"style":156},[299],{"type":43,"value":159},{"type":37,"tag":100,"props":301,"children":302},{"style":107},[303],{"type":43,"value":164},{"type":37,"tag":100,"props":305,"children":306},{"style":113},[307],{"type":43,"value":308}," -iE",{"type":37,"tag":100,"props":310,"children":311},{"style":156},[312],{"type":43,"value":313}," '",{"type":37,"tag":100,"props":315,"children":316},{"style":113},[317],{"type":43,"value":318},"nvidia|nvrm",{"type":37,"tag":100,"props":320,"children":321},{"style":156},[322],{"type":43,"value":323},"'",{"type":37,"tag":100,"props":325,"children":326},{"style":156},[327],{"type":43,"value":159},{"type":37,"tag":100,"props":329,"children":330},{"style":107},[331],{"type":43,"value":332}," tail",{"type":37,"tag":100,"props":334,"children":335},{"style":113},[336],{"type":43,"value":337}," -20\n",{"type":37,"tag":100,"props":339,"children":341},{"class":102,"line":340},3,[342,347,351,355],{"type":37,"tag":100,"props":343,"children":344},{"style":107},[345],{"type":43,"value":346},"lsmod",{"type":37,"tag":100,"props":348,"children":349},{"style":156},[350],{"type":43,"value":159},{"type":37,"tag":100,"props":352,"children":353},{"style":107},[354],{"type":43,"value":164},{"type":37,"tag":100,"props":356,"children":357},{"style":113},[358],{"type":43,"value":174},{"type":37,"tag":198,"props":360,"children":361},{},[362,375],{"type":37,"tag":202,"props":363,"children":364},{},[365,367,373],{"type":43,"value":366},"DKMS shows an error for the running kernel → usually missing headers\n(",{"type":37,"tag":72,"props":368,"children":370},{"className":369},[],[371],{"type":43,"value":372},"linux-headers-$(uname -r)",{"type":43,"value":374},") or a kernel updated after the driver\ninstall. Install headers or reboot into the matching kernel, then rerun\nthe driver play.",{"type":37,"tag":202,"props":376,"children":377},{},[378,380,385,387,392],{"type":43,"value":379},"Module loaded but ",{"type":37,"tag":72,"props":381,"children":383},{"className":382},[],[384],{"type":43,"value":77},{"type":43,"value":386}," fails → check ",{"type":37,"tag":72,"props":388,"children":390},{"className":389},[],[391],{"type":43,"value":295},{"type":43,"value":393}," for RmInitAdapter or\nfallen-off-the-bus errors; these are hardware\u002Ffirmware territory.",{"type":37,"tag":52,"props":395,"children":397},{"id":396},"_5-rerun-the-play-dont-hand-fix",[398],{"type":43,"value":399},"5. Rerun the play, don't hand-fix",{"type":37,"tag":46,"props":401,"children":402},{},[403],{"type":43,"value":404},"After correcting configuration, converge with the playbook rather than\nmanual package surgery, then validate:",{"type":37,"tag":89,"props":406,"children":408},{"className":91,"code":407,"language":93,"meta":94,"style":94},"ansible-playbook -l \u003Chost> playbooks\u002Fnvidia-software\u002Fnvidia-driver.yml\npython3 scripts\u002Fvalidation\u002Fvalidate_slurm.py --json    # or validate_k8s.py\n",[409],{"type":37,"tag":72,"props":410,"children":411},{"__ignoreMap":94},[412,451],{"type":37,"tag":100,"props":413,"children":414},{"class":102,"line":103},[415,420,425,430,435,441,446],{"type":37,"tag":100,"props":416,"children":417},{"style":107},[418],{"type":43,"value":419},"ansible-playbook",{"type":37,"tag":100,"props":421,"children":422},{"style":113},[423],{"type":43,"value":424}," -l",{"type":37,"tag":100,"props":426,"children":427},{"style":156},[428],{"type":43,"value":429}," \u003C",{"type":37,"tag":100,"props":431,"children":432},{"style":113},[433],{"type":43,"value":434},"hos",{"type":37,"tag":100,"props":436,"children":438},{"style":437},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[439],{"type":43,"value":440},"t",{"type":37,"tag":100,"props":442,"children":443},{"style":156},[444],{"type":43,"value":445},">",{"type":37,"tag":100,"props":447,"children":448},{"style":113},[449],{"type":43,"value":450}," playbooks\u002Fnvidia-software\u002Fnvidia-driver.yml\n",{"type":37,"tag":100,"props":452,"children":453},{"class":102,"line":289},[454,459,464,469],{"type":37,"tag":100,"props":455,"children":456},{"style":107},[457],{"type":43,"value":458},"python3",{"type":37,"tag":100,"props":460,"children":461},{"style":113},[462],{"type":43,"value":463}," scripts\u002Fvalidation\u002Fvalidate_slurm.py",{"type":37,"tag":100,"props":465,"children":466},{"style":113},[467],{"type":43,"value":468}," --json",{"type":37,"tag":100,"props":470,"children":471},{"style":124},[472],{"type":43,"value":473},"    # or validate_k8s.py\n",{"type":37,"tag":46,"props":475,"children":476},{},[477,479,485],{"type":43,"value":478},"Reruns are idempotent; a healthy converged rerun reports ",{"type":37,"tag":72,"props":480,"children":482},{"className":481},[],[483],{"type":43,"value":484},"changed=0",{"type":43,"value":486},".",{"type":37,"tag":52,"props":488,"children":490},{"id":489},"kubernetes-specific-notes",[491],{"type":43,"value":492},"Kubernetes-specific notes",{"type":37,"tag":46,"props":494,"children":495},{},[496,498,504],{"type":43,"value":497},"GPU Operator crash-looping driver pods: ",{"type":37,"tag":72,"props":499,"children":501},{"className":500},[],[502],{"type":43,"value":503},"kubectl logs -n \u003Cgpu-operator namespace> \u003Cdriver-pod>",{"type":43,"value":505}," usually names the same root causes — missing\nheaders, wrong module flavor, or a node reboot needed. Fix via the operator\nvalues or node state, not by installing drivers by hand on the host (the\noperator owns the driver on Kubernetes nodes unless DeepOps was configured\nfor host drivers).",{"type":37,"tag":507,"props":508,"children":509},"style",{},[510],{"type":43,"value":511},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":513,"total":560},[514,529,539,545],{"slug":515,"name":515,"fn":516,"description":517,"org":518,"tags":519,"stars":20,"repoUrl":21,"updatedAt":528},"deploy-k8s-gpu-cluster","deploy Kubernetes GPU clusters","Deploy a Kubernetes GPU cluster with DeepOps (Kubespray + GPU Operator) and prove it schedules GPU pods. Use when asked to deploy or rebuild Kubernetes on GPU servers with this repository.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[520,523,524,527],{"name":521,"slug":522,"type":13},"Deployment","deployment",{"name":15,"slug":16,"type":13},{"name":525,"slug":526,"type":13},"Kubernetes","kubernetes",{"name":9,"slug":8,"type":13},"2026-07-17T06:05:02.11504",{"slug":530,"name":530,"fn":531,"description":532,"org":533,"tags":534,"stars":20,"repoUrl":21,"updatedAt":538},"deploy-slurm-cluster","deploy Slurm GPU clusters","Deploy a Slurm GPU cluster with DeepOps and prove it works. Use when asked to deploy, install, or rebuild Slurm on one or more GPU servers with this repository.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[535,536,537],{"name":521,"slug":522,"type":13},{"name":15,"slug":16,"type":13},{"name":9,"slug":8,"type":13},"2026-07-17T06:05:02.462676",{"slug":4,"name":4,"fn":5,"description":6,"org":540,"tags":541,"stars":20,"repoUrl":21,"updatedAt":22},{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[542,543,544],{"name":18,"slug":19,"type":13},{"name":15,"slug":16,"type":13},{"name":9,"slug":8,"type":13},{"slug":546,"name":546,"fn":547,"description":548,"org":549,"tags":550,"stars":20,"repoUrl":21,"updatedAt":559},"validate-gpu-cluster","validate GPU cluster health","Check whether a DeepOps-deployed Slurm or Kubernetes GPU cluster is healthy and report a machine-readable verdict. Use for health checks, post-deploy verification, \"is the cluster working?\" questions, and after any node or driver change.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[551,552,555,556],{"name":15,"slug":16,"type":13},{"name":553,"slug":554,"type":13},"Monitoring","monitoring",{"name":9,"slug":8,"type":13},{"name":557,"slug":558,"type":13},"QA","qa","2026-07-17T06:07:15.184954",4,{"items":562,"total":713},[563,581,597,608,620,632,645,659,670,679,693,702],{"slug":564,"name":564,"fn":565,"description":566,"org":567,"tags":568,"stars":578,"repoUrl":579,"updatedAt":580},"nemoclaw-user-guide","retrieve NemoClaw documentation and configuration","Guides human users' AI agents to the NemoClaw docs MCP server and canonical Fern documentation in Markdown form. Use when users ask how to install, configure, operate, troubleshoot, secure, or learn NemoClaw with an AI coding assistant. Trigger keywords - nemoclaw docs, use nemoclaw with ai agent, nemoclaw mcp docs, nemoclaw install help, nemoclaw quickstart, nemoclaw markdown docs, llms.txt, agent skills.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[569,572,575],{"name":570,"slug":571,"type":13},"Documentation","documentation",{"name":573,"slug":574,"type":13},"MCP","mcp",{"name":576,"slug":577,"type":13},"Search","search",21777,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FNemoClaw","2026-07-20T06:00:01.461044",{"slug":582,"name":582,"fn":583,"description":584,"org":585,"tags":586,"stars":594,"repoUrl":595,"updatedAt":596},"mcore-build-and-dependency","manage Megatron-LM development environments","Container-based dev environment setup and dependency management for Megatron-LM. Covers acquiring and launching the CI container, uv package management, and updating uv.lock.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[587,590,591],{"name":588,"slug":589,"type":13},"Containers","containers",{"name":521,"slug":522,"type":13},{"name":592,"slug":593,"type":13},"Python","python",17049,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FMegatron-LM","2026-07-27T06:06:11.249662",{"slug":598,"name":598,"fn":599,"description":600,"org":601,"tags":602,"stars":594,"repoUrl":595,"updatedAt":607},"mcore-bump-base-image","update NVIDIA PyTorch base images","Bump the NVIDIA PyTorch base image (`nvcr.io\u002Fnvidia\u002Fpytorch:YY.MM-py3`) used by Megatron-LM CI. Covers the two pin sites (GitHub CI in `docker\u002F.ngc_version.dev` and GitLab CI in `.gitlab\u002Fstages\u002F01.build.yml`), the post-bump CI loop (re-run functional tests, refresh golden values, mark broken tests), and the gotchas that bit PRs",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[603,606],{"name":604,"slug":605,"type":13},"CI\u002FCD","ci-cd",{"name":521,"slug":522,"type":13},"2026-07-14T05:25:59.97109",{"slug":609,"name":609,"fn":610,"description":611,"org":612,"tags":613,"stars":594,"repoUrl":595,"updatedAt":619},"mcore-cicd","manage CI\u002FCD pipelines for Megatron-LM","CI\u002FCD reference for Megatron-LM. Covers CI pipeline structure, PR scope labels, triggering internal GitLab CI (which force-pushes the current branch to a pull-request\u002FBRANCH ref — always dry-run and verify the destination first; never run against shared or protected branches), and CI failure investigation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[614,615,616],{"name":604,"slug":605,"type":13},{"name":521,"slug":522,"type":13},{"name":617,"slug":618,"type":13},"GitHub","github","2026-07-27T06:06:12.278222",{"slug":621,"name":621,"fn":622,"description":623,"org":624,"tags":625,"stars":594,"repoUrl":595,"updatedAt":631},"mcore-create-issue","investigate CI failures and create issues","Investigate a failing GitHub Actions run or job and create a GitHub issue for the failure.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[626,627,628],{"name":18,"slug":19,"type":13},{"name":617,"slug":618,"type":13},{"name":629,"slug":630,"type":13},"Triage","triage","2026-07-14T05:25:57.442089",{"slug":633,"name":633,"fn":634,"description":635,"org":636,"tags":637,"stars":594,"repoUrl":595,"updatedAt":644},"mcore-linting-and-formatting","lint and format Megatron-LM code","Linting and formatting for Megatron-LM. Covers running autoformat.sh, tools (ruff, black, isort, pylint, mypy), and code style rules.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[638,641],{"name":639,"slug":640,"type":13},"Best Practices","best-practices",{"name":642,"slug":643,"type":13},"Code Analysis","code-analysis","2026-07-14T05:25:56.18433",{"slug":646,"name":646,"fn":647,"description":648,"org":649,"tags":650,"stars":594,"repoUrl":595,"updatedAt":658},"mcore-migrate-gpt-to-hybrid","migrate Megatron-LM models to HybridModel","Migration guide for moving Megatron Core GPTModel checkpoints, model providers, training commands, and layer mappings to HybridModel.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[651,654,657],{"name":652,"slug":653,"type":13},"Machine Learning","machine-learning",{"name":655,"slug":656,"type":13},"Migration","migration",{"name":9,"slug":8,"type":13},"2026-07-17T06:07:11.777011",{"slug":660,"name":660,"fn":661,"description":662,"org":663,"tags":664,"stars":594,"repoUrl":595,"updatedAt":669},"mcore-onboard-gb200-1node-tests","onboard functional tests for GB200","Onboard 1-node GitHub MR functional tests for GB200 from existing mr-scoped 2-node tests.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[665,666],{"name":557,"slug":558,"type":13},{"name":667,"slug":668,"type":13},"Testing","testing","2026-07-14T05:25:53.673039",{"slug":671,"name":671,"fn":672,"description":673,"org":674,"tags":675,"stars":594,"repoUrl":595,"updatedAt":678},"mcore-run-on-slurm","launch distributed training jobs on SLURM","How to launch distributed Megatron-LM training jobs on a SLURM cluster. Covers a minimal sbatch skeleton, environment-variable setup for torch.distributed.run, CUDA_DEVICE_MAX_CONNECTIONS rules across hardware and parallelism modes, container conventions, monitoring, and per-rank failure diagnosis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[676,677],{"name":521,"slug":522,"type":13},{"name":15,"slug":16,"type":13},"2026-07-14T05:25:49.362534",{"slug":680,"name":680,"fn":681,"description":682,"org":683,"tags":684,"stars":594,"repoUrl":595,"updatedAt":692},"mcore-split-pr","split pull requests to reduce review load","Split a PR into multiple PRs to reduce the number of required CODEOWNERS reviewer groups.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[685,688,689],{"name":686,"slug":687,"type":13},"Code Review","code-review",{"name":617,"slug":618,"type":13},{"name":690,"slug":691,"type":13},"Pull Requests","pull-requests","2026-07-14T05:26:01.226578",{"slug":694,"name":694,"fn":695,"description":696,"org":697,"tags":698,"stars":594,"repoUrl":595,"updatedAt":701},"mcore-testing","run and manage Megatron-LM tests","Test system for Megatron-LM. Covers test layout, recipe YAML structure, adding and running unit and functional tests, golden values, marker filters, and CI parity.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[699,700],{"name":557,"slug":558,"type":13},{"name":667,"slug":668,"type":13},"2026-07-14T05:25:54.928983",{"slug":703,"name":703,"fn":704,"description":705,"org":706,"tags":707,"stars":594,"repoUrl":595,"updatedAt":712},"nightly-sync","manage nightly main-to-dev sync workflows","Domain knowledge for the nightly main-to-dev sync workflow. Covers merge strategy, CI architecture, failure investigation, and known issues.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[708,711],{"name":709,"slug":710,"type":13},"Automation","automation",{"name":604,"slug":605,"type":13},"2026-07-30T05:29:03.275638",496]