[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-nvidia-nemo-automodel-model-onboarding":3,"mdc--6h5e1o-key":31,"related-repo-nvidia-nemo-automodel-model-onboarding":2692,"related-org-nvidia-nemo-automodel-model-onboarding":2798},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":20,"repoUrl":21,"updatedAt":22,"license":23,"forks":24,"topics":25,"repo":26,"sourceUrl":29,"mdContent":30},"nemo-automodel-model-onboarding","onboard model architectures to NeMo AutoModel","Guide for onboarding new model architectures into NeMo AutoModel, including architecture discovery, implementation patterns, registration, and validation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},"nvidia","NVIDIA","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fnvidia.png",[12,16,17],{"name":13,"slug":14,"type":15},"Machine Learning","machine-learning","tag",{"name":9,"slug":8,"type":15},{"name":18,"slug":19,"type":15},"Engineering","engineering",2473,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fskills","2026-07-14T05:28:54.273629","Apache-2.0",281,[],{"repoUrl":21,"stars":20,"forks":24,"topics":27,"description":28},[],"AI agent skills published by NVIDIA","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fskills\u002Ftree\u002FHEAD\u002Fskills\u002Fnemo-automodel-model-onboarding","---\nname: nemo-automodel-model-onboarding\ndescription: Guide for onboarding new model architectures into NeMo AutoModel, including architecture discovery, implementation patterns, registration, and validation.\nwhen_to_use: Adding or modifying model architecture support in NeMo AutoModel, such as LLM\u002FVLM\u002FMoE model files, custom layers, state-dict adapters, registry entries, Hugging Face config mapping, or capability flags.\nlicense: Apache-2.0\nmetadata:\n author: NVIDIA\n tags:\n - nemo-automodel\n - model-onboarding\n---\n\n# Adding Model Support to NeMo AutoModel\n\n## Purpose\n\nThis skill guides implementation of new model architectures in NeMo AutoModel. Follow the five phases in order.\n\u003C!-- NVSkills signature refresh requested for AM-519. -->\n\n## Instructions\n\nWhen answering an onboarding question, keep the response in this order:\n\n1. Classify the architecture from `config.json`.\n2. Name the exact implementation files under `components\u002Fmodels\u002F\u003Cname>\u002F`.\n3. Identify registry and optional custom-config updates.\n4. State the validation tests that must be added before full checkpoint use.\n\nFor conceptual onboarding questions, answer from this skill without opening the\npattern files unless the user asks you to edit code. Mention pattern filenames\nas references, then give the direct checklist.\n\nUse direct action verbs: classify the model, name the files, map the weights,\nregister the class, and add tests. Do not discuss distributed strategy,\nlauncher configuration, or general recipe authoring unless the user explicitly\nconnects it to onboarding a new architecture.\n\n## Examples\n\nUse these compact answer patterns for common questions:\n\n- Dense causal LM: classify as dense only when `architectures` contains a\n `ForCausalLM` class and expert fields such as `num_local_experts`,\n `n_routed_experts`, or `num_experts_per_tok` are absent. Create\n `components\u002Fmodels\u002F\u003Cname>\u002Fmodel.py`, `state_dict_adapter.py`, `__init__.py`,\n and optional `config.py`, register `MODEL_ARCH_MAPPING` in\n `_transformers\u002Fregistry.py`, add example YAML, and add tiny-config unit tests\n plus layer-equivalence tests for rewritten layers.\n- MoE state dict: identify expert fields in `config.json`, reference\n `moe-patterns.md`, map router tensors separately, preserve routed-expert\n index order, map routed experts, shared experts, and gate\u002Fup\u002Fdown projections,\n add adapter key-map tests and tiny-config numerical equivalence tests, and do\n not rely only on `from_pretrained()` or silent tensor reshapes.\n- VLM onboarding: classify as VLM only when `vision_config`, `text_config`, and\n a `ForConditionalGeneration` architecture are present. Reference\n `vlm-patterns.md` and existing VLM implementations such as `mistral4`,\n `kimivl`, or `kimi_k25_vl`; check text backbone, vision tower, projector,\n processor assumptions, text and vision `state_dict_adapter.py` mappings,\n registry registration, and tiny image-text tests before full checkpoints.\n Do not treat VLM onboarding as a pure causal-LM path or skip processor\u002Fimage\n tests.\n\nFor MoE state-dict questions, always include the safety checklist:\n\n- Map router tensors separately from expert tensors.\n- Preserve routed-expert index order; never sort, drop, merge, or silently\n reshape expert weights to make loading pass.\n- Map gate, up, and down projections explicitly, including combined projection\n layouts and shared experts when present.\n- Add adapter key-map tests and tiny-config numerical equivalence tests before\n relying on full checkpoint loading.\n\nFor VLM questions, explicitly check `vision_config`, `text_config`, the\nconditional-generation architecture, text backbone, vision tower, projector,\nprocessor assumptions, registry entry, and tiny image-text tests.\n\n## Routing Boundary\n\nUse this skill only when the user is adding or modifying model architecture support: model files, custom layers, state-dict adapters, Hugging Face config mapping, registry entries, or model capability flags.\n\nDo not use this skill for standalone training recipe YAML questions about optimizers, datasets, schedulers, validation datasets, or trainer wiring unless they are explicitly part of onboarding a new model architecture. Those recipe questions belong to the nemo-automodel-recipe-development skill.\n\nIn-scope examples:\n\n- \"Add support for a new Hugging Face causal LM architecture.\"\n- \"Map MoE router and expert weights from a Hugging Face checkpoint.\"\n- \"Register a new model class in NeMo AutoModel.\"\n\nOut-of-scope examples:\n\n- \"Write a finetuning recipe YAML with optimizer and dataset sections.\"\n- \"Choose FSDP2, DDP, tensor parallel, or context parallel settings.\"\n- \"Configure Slurm, SkyPilot, containers, mounts, or launch dispatch.\"\n\n## Phase 1: Discovery\n\nBefore writing code, gather information about the target model.\n\n### 1.1 Fetch HuggingFace config.json\n\nDownload the model's `config.json` from the HuggingFace Hub (or use `AutoConfig.from_pretrained`). Key fields to extract:\n\n- `architectures` -- determines the class name and registration key (e.g., `\"LlamaForCausalLM\"`, `\"Qwen3MoeForCausalLM\"`, `\"Mistral3ForConditionalGeneration\"`)\n- `model_type` -- used for custom config registration in `_CUSTOM_CONFIG_REGISTRATIONS` if HF does not have a built-in config class\n- `hidden_size`, `intermediate_size`, `num_hidden_layers`, `num_attention_heads`, `num_key_value_heads` -- sizing\n- `vocab_size` -- needed for tiny test configs\n- `tie_word_embeddings` -- whether lm_head shares weights with embed_tokens\n- `hidden_act` -- activation function (e.g., `\"silu\"` for SwiGLU)\n\n### 1.2 Determine model type\n\n| Type | Indicators | Pattern file |\n|------|-----------|-------------|\n| **Dense LLM** | `ForCausalLM` in architectures, no expert fields | [llm-patterns.md](.\u002Fllm-patterns.md) |\n| **MoE LLM** | `n_routed_experts`, `num_local_experts`, `num_experts_per_tok` in config | [moe-patterns.md](.\u002Fmoe-patterns.md) |\n| **VLM** | `ForConditionalGeneration` in architectures, has `vision_config` + `text_config` | [vlm-patterns.md](.\u002Fvlm-patterns.md) |\n\n### 1.3 Check for existing similar architectures\n\nLook in `components\u002Fmodels\u002F` for architectures with similar attention or MLP patterns:\n\n```\ncomponents\u002Fmodels\u002F\n llama\u002F # Standard GQA + SwiGLU (CombinedQKV + CombinedGateUpMLP)\n qwen2\u002F # Same as Llama but with attention bias + QKV bias\n baichuan\u002F # ALiBi attention variant\n deepseek_v3\u002F # MLA attention + MoE (DeepSeek-style grouped experts)\n mistral4\u002F # MLA + MoE + VLM (Pixtral vision)\n kimivl\u002F # DeepSeek-V3 backbone + MoonVit vision\n kimi_k25_vl\u002F # Updated KimiVL with different projector\n qwen3_moe\u002F # Qwen3 with MoE layers\n nemotron_v3\u002F # Hybrid mamba-attention\n```\n\n### 1.4 Identify custom components\n\nCheck whether the model needs:\n\n- **Custom attention**: GQA (standard), MLA (DeepSeek\u002FMistral4), sliding window, bidirectional\n- **Custom RoPE**: Standard (Llama), YaRN scaling, NTK-aware, complex-number (DeepSeek)\n- **Custom normalization**: RMSNorm (standard), LayerNorm, different eps values\n- **Custom MLP**: SwiGLU (standard), GeGLU, ReLU-squared, MoE routing\n- **Custom config class**: Needed only if HF `AutoConfig` cannot parse the model's `config.json` (check `auto_map` field)\n\n### 1.5 Note dimensions for test config\n\nFor unit tests, create a tiny config. Target: ~1M parameters or less.\n\n```python\n# Example tiny config for a Llama-like model:\ntiny_config = LlamaConfig(\n hidden_size=64,\n intermediate_size=128,\n num_hidden_layers=2,\n num_attention_heads=4,\n num_key_value_heads=2,\n vocab_size=256,\n max_position_embeddings=128,\n)\n```\n\n---\n\n## Phase 2: Implementation\n\n### 2.1 Create directory structure\n\n```\ncomponents\u002Fmodels\u002F\u003Cname>\u002F\n __init__.py\n model.py\n state_dict_adapter.py\n config.py # Only if HF config is insufficient\n layers.py # Only for MoE \u002F MLA \u002F other non-standard layers\n rope_utils.py # Only for custom RoPE\n```\n\n### 2.2 Implementation order\n\nImplement files in dependency order:\n\n1. **config.py** (if needed) -- Custom `PretrainedConfig` subclass\n2. **rope_utils.py** (if needed) -- RoPE implementation\n3. **layers.py** (if needed) -- Attention, MLP, decoder block classes\n4. **model.py** -- The main `ForCausalLM` (or `ForConditionalGeneration`) class\n5. **state_dict_adapter.py** -- HF weight conversion\n6. **__init__.py** -- Re-export the main model class\n\nSee the pattern files for detailed implementation guidance:\n\n- Dense LLM: [llm-patterns.md](.\u002Fllm-patterns.md)\n- MoE: [moe-patterns.md](.\u002Fmoe-patterns.md)\n- VLM: [vlm-patterns.md](.\u002Fvlm-patterns.md)\n- Capabilities and fp32 precision: [capabilities-and-precision.md](.\u002Fcapabilities-and-precision.md)\n\n### 2.3 Causal LM weight tying\n\nFor any CausalLM-style class whose config can enable `tie_word_embeddings`,\nmake tying explicit: declare `_tied_weights_keys`, implement `tie_weights()`\nwith the actual `lm_head` and input-embedding FQNs, and add tiny tests for\ntied and untied configs. Do not tie architectures with intentionally separate\nheads, asymmetric vocab sizes, or stages that do not own both tensors.\n\n### 2.4 MoE state-dict adapter checklist\n\nFor MoE models, do not stop at generic loading. The adapter must explicitly map:\n\n- Router weights, including gate bias or correction-bias tensors when the Hugging Face model has them.\n- Expert weights, preserving expert index order across local and routed experts.\n- Gate\u002Fup\u002Fdown projections, including combined or split projection layouts.\n- Shared experts separately from routed experts when the architecture has both.\n\nAdd tests that assert expected key mappings and run numerical equivalence with tiny configs before trying full checkpoints.\n\nDo not use these shortcuts:\n\n- Do not validate the adapter only by calling `from_pretrained()`.\n- Do not accept missing or extra expert keys without an explicit mapping reason.\n- Do not change dtype, transpose dimensions, or reshape tensors unless the HF\n and NeMo layouts require it and a test proves the conversion is reversible.\n- Do not skip router or shared-expert tests because dense-layer tests pass.\n\n### 2.5 VLM onboarding checklist\n\nFor VLMs, confirm the Hugging Face config has `vision_config` and `text_config`\nand that `architectures` points to a conditional-generation class. Start from\nthe closest VLM pattern file, usually [vlm-patterns.md](.\u002Fvlm-patterns.md), and\ncompare existing implementations such as `mistral4`, `kimivl`, or\n`kimi_k25_vl`.\n\nThe implementation should explicitly cover:\n\n- Text backbone, vision tower, projector, and processor or image preprocessing assumptions.\n- Weight mapping for both text and vision modules in `state_dict_adapter.py`.\n- Registration of the `ForConditionalGeneration` class in `_transformers\u002Fregistry.py`.\n- Tiny tests that exercise image-text inputs and verify the adapter round-trip.\n\n### 2.6 Register in registry\n\nAdd the model to `MODEL_ARCH_MAPPING` in `_transformers\u002Fregistry.py`:\n\n```python\n# In _transformers\u002Fregistry.py\nMODEL_ARCH_MAPPING = OrderedDict([\n # ... existing entries ...\n (\n \"NewModelForCausalLM\",\n (\"nemo_automodel.components.models.new_model.model\", \"NewModelForCausalLM\"),\n ),\n])\n```\n\nIf the model has a custom config class with `auto_map` in its `config.json`, also register in `_CUSTOM_CONFIG_REGISTRATIONS`:\n\n```python\n_CUSTOM_CONFIG_REGISTRATIONS: Dict[str, Tuple[str, str]] = {\n # ... existing entries ...\n \"new_model\": (\"nemo_automodel.components.models.new_model.configuration\", \"NewModelConfig\"),\n}\n```\n\n### 2.7 Declare capabilities and precision-sensitive params\n\nEvery class registered in `MODEL_ARCH_MAPPING` must declare parallelism\ncapabilities, either with a static nested `ModelCapabilities` dataclass or a\nvariant-aware `get_capabilities(cls, config)` method. Pick exactly one pattern.\nCapabilities should reflect recipe YAMLs that have been validated end to end.\n\nIf the model has precision-sensitive parameters such as Mamba `A_log` \u002F\n`dt_bias`, MoE sigmoid gate bias, attention-sink bias, or per-head `scale`,\ndeclare `_keep_in_fp32_modules_strict` so sharding keeps those params in fp32\ncompute. See [capabilities-and-precision.md](.\u002Fcapabilities-and-precision.md)\nfor examples, variant dispatch rules, and frozen-submodule dtype guidance.\n\n---\n\n## Phase 3: Onboarding Example Config\n\nThis phase is only for adding a minimal example config that proves the newly\nonboarded architecture can load and run. Use nemo-automodel-recipe-development for general\nrecipe authoring or existing recipe modifications.\n\n### 3.1 Create example YAML config\n\nCreate an example config under `examples\u002Fllm_finetune\u002F\u003Cname>\u002F` (or `examples\u002Fvlm_finetune\u002F\u003Cname>\u002F`):\n\n```yaml\nmodel:\n _target_: nemo_automodel.NeMoAutoModelForCausalLM.from_pretrained\n pretrained_model_name_or_path: \u003Corg>\u002F\u003Cmodel-name>\n\ntrainer:\n max_steps: 100\n gradient_clip_val: 1.0\n accumulate_grad_batches: 1\n\n# ... data, optimizer config ...\n```\n\n### 3.2 Verify model loads\n\nTest that the model loads from a HuggingFace checkpoint:\n\n```python\nfrom nemo_automodel import NeMoAutoModelForCausalLM\n\nmodel = NeMoAutoModelForCausalLM.from_pretrained(\"\u003Corg>\u002F\u003Cmodel-name>\")\n```\n\n### 3.3 Test with tiny config first\n\nBefore using full-size models, verify with a tiny config (1-2 layers, small hidden dim) to catch shape mismatches early.\n\n## Phase 4: Tests\n\nCreate `tests\u002Funit_tests\u002Fmodels\u002F\u003Cname>\u002F` and cover the checks below before\nloading full checkpoints:\n\n- Forward-shape smoke test with a tiny config.\n- State-dict adapter round-trip: `from_hf -> to_hf` preserves mapped names,\n shapes, dtypes, and values.\n- Layer equivalence tests for every rewritten attention, MLP, normalization,\n RoPE, or MoE layer. Use the model dtype from config, identical seeded weights,\n identical inputs, and dtype-appropriate `torch.allclose` tolerances.\n- Short functional test that verifies loss decreases over a few training steps.\n\n---\n\n## Phase 5: Documentation\n\n### 5.1 Update model coverage page\n\nEdit the appropriate file in `docs\u002Fmodel-coverage\u002F`:\n- LLM\u002FMoE: `docs\u002Fmodel-coverage\u002Fllm\u002Findex.md`\n- VLM: `docs\u002Fmodel-coverage\u002Fvlm\u002Findex.md`\n\nAdd a row with the model name, supported features (TP, PP, FSDP, LoRA, QLoRA), and any limitations.\n\n---\n\n## Phase 6: Parity Testing\n\nAfter implementation and unit tests are complete, run the full parity-testing\nworkflow to verify that the new model produces numerically equivalent results to\nthe reference HuggingFace implementation.\n\nRun three levels of comparison:\n\n1. State-dict round-trip: load a reference HuggingFace checkpoint, convert it\n into the NeMo AutoModel layout, export it back, and verify that all mapped\n tensors match the reference names, shapes, dtypes, and values within the\n expected tolerance.\n2. Component-level parity: compare rewritten attention, MLP, normalization,\n RoPE, and MoE components against the HuggingFace implementation with fixed\n seeds and identical dtype.\n3. End-to-end forward pass: run the full NeMo AutoModel and HuggingFace model\n on the same tokenized input and compare logits, hidden states, and loss.\n\nDo not skip this phase. A model that passes unit tests can still diverge from HF\ndue to subtle weight-conversion bugs, backend differences, or RoPE mismatches\nthat only surface in a full parity comparison.\n\n---\n\n## Key Files Reference\n\n| File | Purpose |\n|------|---------|\n| `_transformers\u002Fregistry.py` | `MODEL_ARCH_MAPPING` and `_CUSTOM_CONFIG_REGISTRATIONS` |\n| `components\u002Fmodels\u002Fcommon\u002F__init__.py` | Exports `CombinedQKVAttentionMixin`, `CombinedGateUpMLP`, `BackendConfig`, `HFCheckpointingMixin`, etc. |\n| `components\u002Fmodels\u002Fcommon\u002Fcombined_projection\u002Fcombined_qkv.py` | `CombinedQKVAttentionMixin` with `setup_qkv_projection()` and `compute_qkv()` |\n| `components\u002Fmodels\u002Fcommon\u002Fcombined_projection\u002Fcombined_mlp.py` | `CombinedGateUpMLP` with interleaved gate\u002Fup layout |\n| `components\u002Fmodels\u002Fcommon\u002Fcombined_projection\u002Fstate_dict_adapter.py` | `CombinedProjectionStateDictAdapter` base class |\n| `components\u002Fmodels\u002Fcommon\u002Fhf_checkpointing_mixin.py` | `HFCheckpointingMixin` for save\u002Fload |\n| `components\u002Fmodels\u002Fcommon\u002Futils.py` | `BackendConfig`, `initialize_rms_norm_module`, `initialize_linear_module`, `get_rope_config` |\n| `components\u002Fmoe\u002Fconfig.py` | `MoEConfig` dataclass |\n| `components\u002Fmoe\u002Ffsdp_mixin.py` | `MoEFSDPSyncMixin` for distributed expert handling |\n| `components\u002Fmoe\u002Flayers.py` | `MoE` layer, `MLP` (dense) for MoE blocks |\n| `components\u002Fmoe\u002Fexperts.py` | `GroupedExperts`, `GroupedExpertsDeepEP`, `GroupedExpertsTE` |\n\n---\n\n## Checklist\n\n- [ ] Fetched and analyzed `config.json` from HuggingFace\n- [ ] Determined model type (dense LLM \u002F MoE \u002F VLM)\n- [ ] Identified custom components (attention, RoPE, normalization, MLP)\n- [ ] Created `components\u002Fmodels\u002F\u003Cname>\u002F` directory\n- [ ] Implemented config.py (if custom config needed)\n- [ ] Implemented layers.py (if custom layers needed)\n- [ ] Implemented rope_utils.py (if custom RoPE needed)\n- [ ] Implemented model.py with `HFCheckpointingMixin`\n- [ ] Implemented state_dict_adapter.py\n- [ ] Implemented __init__.py with re-export\n- [ ] Registered in `MODEL_ARCH_MAPPING` in `_transformers\u002Fregistry.py`\n- [ ] Registered custom config in `_CUSTOM_CONFIG_REGISTRATIONS` (if applicable)\n- [ ] Declared `ModelCapabilities` nested dataclass (static) OR `get_capabilities(cls, config)` classmethod (variant dispatch, e.g. ERNIE-4.5 MoE vs dense) — never both, never neither\n- [ ] Created example YAML config\n- [ ] Verified model loads via `NeMoAutoModelForCausalLM.from_pretrained()`\n- [ ] Created unit tests (forward shape, state_dict round-trip)\n- [ ] Declared `_keep_in_fp32_modules_strict` for every intrinsically-fp32 param (SSM `A_log`\u002F`dt_bias`, Mamba `D` when reference-fp32, MoE gate bias, attention-sink bias, `scale`, …) — see §2.7\n- [ ] Created layer equivalence tests for every rewritten layer (matching model dtype)\n- [ ] Created functional tests (training loss decreases)\n- [ ] Updated docs\u002Fmodel-coverage page\n- [ ] Ran state-dict round-trip, component parity, and E2E forward-pass parity checks\n- [ ] Set `ModelClass = \u003CName>ForCausalLM` at module bottom\n",{"data":32,"body":38},{"name":4,"description":6,"when_to_use":33,"license":23,"metadata":34},"Adding or modifying model architecture support in NeMo AutoModel, such as LLM\u002FVLM\u002FMoE model files, custom layers, state-dict adapters, registry entries, Hugging Face config mapping, or capability flags.",{"author":9,"tags":35},[36,37],"nemo-automodel","model-onboarding",{"type":39,"children":40},"root",[41,50,57,63,69,74,115,120,125,131,136,325,330,353,371,377,382,387,392,410,415,433,439,444,451,471,605,611,758,764,777,789,795,800,876,882,887,988,992,998,1004,1013,1019,1024,1112,1117,1158,1164,1200,1206,1211,1234,1239,1244,1273,1279,1330,1335,1377,1383,1402,1473,1498,1536,1542,1570,1613,1616,1622,1627,1633,1653,1801,1807,1812,1842,1848,1853,1859,1872,1911,1914,1920,1926,1938,1962,1967,1970,1976,1981,1986,2004,2009,2012,2018,2365,2368,2374,2686],{"type":42,"tag":43,"props":44,"children":46},"element","h1",{"id":45},"adding-model-support-to-nemo-automodel",[47],{"type":48,"value":49},"text","Adding Model Support to NeMo AutoModel",{"type":42,"tag":51,"props":52,"children":54},"h2",{"id":53},"purpose",[55],{"type":48,"value":56},"Purpose",{"type":42,"tag":58,"props":59,"children":60},"p",{},[61],{"type":48,"value":62},"This skill guides implementation of new model architectures in NeMo AutoModel. Follow the five phases in order.",{"type":42,"tag":51,"props":64,"children":66},{"id":65},"instructions",[67],{"type":48,"value":68},"Instructions",{"type":42,"tag":58,"props":70,"children":71},{},[72],{"type":48,"value":73},"When answering an onboarding question, keep the response in this order:",{"type":42,"tag":75,"props":76,"children":77},"ol",{},[78,93,105,110],{"type":42,"tag":79,"props":80,"children":81},"li",{},[82,84,91],{"type":48,"value":83},"Classify the architecture from ",{"type":42,"tag":85,"props":86,"children":88},"code",{"className":87},[],[89],{"type":48,"value":90},"config.json",{"type":48,"value":92},".",{"type":42,"tag":79,"props":94,"children":95},{},[96,98,104],{"type":48,"value":97},"Name the exact implementation files under ",{"type":42,"tag":85,"props":99,"children":101},{"className":100},[],[102],{"type":48,"value":103},"components\u002Fmodels\u002F\u003Cname>\u002F",{"type":48,"value":92},{"type":42,"tag":79,"props":106,"children":107},{},[108],{"type":48,"value":109},"Identify registry and optional custom-config updates.",{"type":42,"tag":79,"props":111,"children":112},{},[113],{"type":48,"value":114},"State the validation tests that must be added before full checkpoint use.",{"type":42,"tag":58,"props":116,"children":117},{},[118],{"type":48,"value":119},"For conceptual onboarding questions, answer from this skill without opening the\npattern files unless the user asks you to edit code. Mention pattern filenames\nas references, then give the direct checklist.",{"type":42,"tag":58,"props":121,"children":122},{},[123],{"type":48,"value":124},"Use direct action verbs: classify the model, name the files, map the weights,\nregister the class, and add tests. Do not discuss distributed strategy,\nlauncher configuration, or general recipe authoring unless the user explicitly\nconnects it to onboarding a new architecture.",{"type":42,"tag":51,"props":126,"children":128},{"id":127},"examples",[129],{"type":48,"value":130},"Examples",{"type":42,"tag":58,"props":132,"children":133},{},[134],{"type":48,"value":135},"Use these compact answer patterns for common questions:",{"type":42,"tag":137,"props":138,"children":139},"ul",{},[140,232,260],{"type":42,"tag":79,"props":141,"children":142},{},[143,145,151,153,159,161,167,169,175,177,183,185,191,193,199,200,206,208,214,216,222,224,230],{"type":48,"value":144},"Dense causal LM: classify as dense only when ",{"type":42,"tag":85,"props":146,"children":148},{"className":147},[],[149],{"type":48,"value":150},"architectures",{"type":48,"value":152}," contains a\n",{"type":42,"tag":85,"props":154,"children":156},{"className":155},[],[157],{"type":48,"value":158},"ForCausalLM",{"type":48,"value":160}," class and expert fields such as ",{"type":42,"tag":85,"props":162,"children":164},{"className":163},[],[165],{"type":48,"value":166},"num_local_experts",{"type":48,"value":168},",\n",{"type":42,"tag":85,"props":170,"children":172},{"className":171},[],[173],{"type":48,"value":174},"n_routed_experts",{"type":48,"value":176},", or ",{"type":42,"tag":85,"props":178,"children":180},{"className":179},[],[181],{"type":48,"value":182},"num_experts_per_tok",{"type":48,"value":184}," are absent. Create\n",{"type":42,"tag":85,"props":186,"children":188},{"className":187},[],[189],{"type":48,"value":190},"components\u002Fmodels\u002F\u003Cname>\u002Fmodel.py",{"type":48,"value":192},", ",{"type":42,"tag":85,"props":194,"children":196},{"className":195},[],[197],{"type":48,"value":198},"state_dict_adapter.py",{"type":48,"value":192},{"type":42,"tag":85,"props":201,"children":203},{"className":202},[],[204],{"type":48,"value":205},"__init__.py",{"type":48,"value":207},",\nand optional ",{"type":42,"tag":85,"props":209,"children":211},{"className":210},[],[212],{"type":48,"value":213},"config.py",{"type":48,"value":215},", register ",{"type":42,"tag":85,"props":217,"children":219},{"className":218},[],[220],{"type":48,"value":221},"MODEL_ARCH_MAPPING",{"type":48,"value":223}," in\n",{"type":42,"tag":85,"props":225,"children":227},{"className":226},[],[228],{"type":48,"value":229},"_transformers\u002Fregistry.py",{"type":48,"value":231},", add example YAML, and add tiny-config unit tests\nplus layer-equivalence tests for rewritten layers.",{"type":42,"tag":79,"props":233,"children":234},{},[235,237,242,244,250,252,258],{"type":48,"value":236},"MoE state dict: identify expert fields in ",{"type":42,"tag":85,"props":238,"children":240},{"className":239},[],[241],{"type":48,"value":90},{"type":48,"value":243},", reference\n",{"type":42,"tag":85,"props":245,"children":247},{"className":246},[],[248],{"type":48,"value":249},"moe-patterns.md",{"type":48,"value":251},", map router tensors separately, preserve routed-expert\nindex order, map routed experts, shared experts, and gate\u002Fup\u002Fdown projections,\nadd adapter key-map tests and tiny-config numerical equivalence tests, and do\nnot rely only on ",{"type":42,"tag":85,"props":253,"children":255},{"className":254},[],[256],{"type":48,"value":257},"from_pretrained()",{"type":48,"value":259}," or silent tensor reshapes.",{"type":42,"tag":79,"props":261,"children":262},{},[263,265,271,272,278,280,286,288,294,296,302,303,309,310,316,318,323],{"type":48,"value":264},"VLM onboarding: classify as VLM only when ",{"type":42,"tag":85,"props":266,"children":268},{"className":267},[],[269],{"type":48,"value":270},"vision_config",{"type":48,"value":192},{"type":42,"tag":85,"props":273,"children":275},{"className":274},[],[276],{"type":48,"value":277},"text_config",{"type":48,"value":279},", and\na ",{"type":42,"tag":85,"props":281,"children":283},{"className":282},[],[284],{"type":48,"value":285},"ForConditionalGeneration",{"type":48,"value":287}," architecture are present. Reference\n",{"type":42,"tag":85,"props":289,"children":291},{"className":290},[],[292],{"type":48,"value":293},"vlm-patterns.md",{"type":48,"value":295}," and existing VLM implementations such as ",{"type":42,"tag":85,"props":297,"children":299},{"className":298},[],[300],{"type":48,"value":301},"mistral4",{"type":48,"value":168},{"type":42,"tag":85,"props":304,"children":306},{"className":305},[],[307],{"type":48,"value":308},"kimivl",{"type":48,"value":176},{"type":42,"tag":85,"props":311,"children":313},{"className":312},[],[314],{"type":48,"value":315},"kimi_k25_vl",{"type":48,"value":317},"; check text backbone, vision tower, projector,\nprocessor assumptions, text and vision ",{"type":42,"tag":85,"props":319,"children":321},{"className":320},[],[322],{"type":48,"value":198},{"type":48,"value":324}," mappings,\nregistry registration, and tiny image-text tests before full checkpoints.\nDo not treat VLM onboarding as a pure causal-LM path or skip processor\u002Fimage\ntests.",{"type":42,"tag":58,"props":326,"children":327},{},[328],{"type":48,"value":329},"For MoE state-dict questions, always include the safety checklist:",{"type":42,"tag":137,"props":331,"children":332},{},[333,338,343,348],{"type":42,"tag":79,"props":334,"children":335},{},[336],{"type":48,"value":337},"Map router tensors separately from expert tensors.",{"type":42,"tag":79,"props":339,"children":340},{},[341],{"type":48,"value":342},"Preserve routed-expert index order; never sort, drop, merge, or silently\nreshape expert weights to make loading pass.",{"type":42,"tag":79,"props":344,"children":345},{},[346],{"type":48,"value":347},"Map gate, up, and down projections explicitly, including combined projection\nlayouts and shared experts when present.",{"type":42,"tag":79,"props":349,"children":350},{},[351],{"type":48,"value":352},"Add adapter key-map tests and tiny-config numerical equivalence tests before\nrelying on full checkpoint loading.",{"type":42,"tag":58,"props":354,"children":355},{},[356,358,363,364,369],{"type":48,"value":357},"For VLM questions, explicitly check ",{"type":42,"tag":85,"props":359,"children":361},{"className":360},[],[362],{"type":48,"value":270},{"type":48,"value":192},{"type":42,"tag":85,"props":365,"children":367},{"className":366},[],[368],{"type":48,"value":277},{"type":48,"value":370},", the\nconditional-generation architecture, text backbone, vision tower, projector,\nprocessor assumptions, registry entry, and tiny image-text tests.",{"type":42,"tag":51,"props":372,"children":374},{"id":373},"routing-boundary",[375],{"type":48,"value":376},"Routing Boundary",{"type":42,"tag":58,"props":378,"children":379},{},[380],{"type":48,"value":381},"Use this skill only when the user is adding or modifying model architecture support: model files, custom layers, state-dict adapters, Hugging Face config mapping, registry entries, or model capability flags.",{"type":42,"tag":58,"props":383,"children":384},{},[385],{"type":48,"value":386},"Do not use this skill for standalone training recipe YAML questions about optimizers, datasets, schedulers, validation datasets, or trainer wiring unless they are explicitly part of onboarding a new model architecture. Those recipe questions belong to the nemo-automodel-recipe-development skill.",{"type":42,"tag":58,"props":388,"children":389},{},[390],{"type":48,"value":391},"In-scope examples:",{"type":42,"tag":137,"props":393,"children":394},{},[395,400,405],{"type":42,"tag":79,"props":396,"children":397},{},[398],{"type":48,"value":399},"\"Add support for a new Hugging Face causal LM architecture.\"",{"type":42,"tag":79,"props":401,"children":402},{},[403],{"type":48,"value":404},"\"Map MoE router and expert weights from a Hugging Face checkpoint.\"",{"type":42,"tag":79,"props":406,"children":407},{},[408],{"type":48,"value":409},"\"Register a new model class in NeMo AutoModel.\"",{"type":42,"tag":58,"props":411,"children":412},{},[413],{"type":48,"value":414},"Out-of-scope examples:",{"type":42,"tag":137,"props":416,"children":417},{},[418,423,428],{"type":42,"tag":79,"props":419,"children":420},{},[421],{"type":48,"value":422},"\"Write a finetuning recipe YAML with optimizer and dataset sections.\"",{"type":42,"tag":79,"props":424,"children":425},{},[426],{"type":48,"value":427},"\"Choose FSDP2, DDP, tensor parallel, or context parallel settings.\"",{"type":42,"tag":79,"props":429,"children":430},{},[431],{"type":48,"value":432},"\"Configure Slurm, SkyPilot, containers, mounts, or launch dispatch.\"",{"type":42,"tag":51,"props":434,"children":436},{"id":435},"phase-1-discovery",[437],{"type":48,"value":438},"Phase 1: Discovery",{"type":42,"tag":58,"props":440,"children":441},{},[442],{"type":48,"value":443},"Before writing code, gather information about the target model.",{"type":42,"tag":445,"props":446,"children":448},"h3",{"id":447},"_11-fetch-huggingface-configjson",[449],{"type":48,"value":450},"1.1 Fetch HuggingFace config.json",{"type":42,"tag":58,"props":452,"children":453},{},[454,456,461,463,469],{"type":48,"value":455},"Download the model's ",{"type":42,"tag":85,"props":457,"children":459},{"className":458},[],[460],{"type":48,"value":90},{"type":48,"value":462}," from the HuggingFace Hub (or use ",{"type":42,"tag":85,"props":464,"children":466},{"className":465},[],[467],{"type":48,"value":468},"AutoConfig.from_pretrained",{"type":48,"value":470},"). Key fields to extract:",{"type":42,"tag":137,"props":472,"children":473},{},[474,506,525,564,575,586],{"type":42,"tag":79,"props":475,"children":476},{},[477,482,484,490,491,497,498,504],{"type":42,"tag":85,"props":478,"children":480},{"className":479},[],[481],{"type":48,"value":150},{"type":48,"value":483}," -- determines the class name and registration key (e.g., ",{"type":42,"tag":85,"props":485,"children":487},{"className":486},[],[488],{"type":48,"value":489},"\"LlamaForCausalLM\"",{"type":48,"value":192},{"type":42,"tag":85,"props":492,"children":494},{"className":493},[],[495],{"type":48,"value":496},"\"Qwen3MoeForCausalLM\"",{"type":48,"value":192},{"type":42,"tag":85,"props":499,"children":501},{"className":500},[],[502],{"type":48,"value":503},"\"Mistral3ForConditionalGeneration\"",{"type":48,"value":505},")",{"type":42,"tag":79,"props":507,"children":508},{},[509,515,517,523],{"type":42,"tag":85,"props":510,"children":512},{"className":511},[],[513],{"type":48,"value":514},"model_type",{"type":48,"value":516}," -- used for custom config registration in ",{"type":42,"tag":85,"props":518,"children":520},{"className":519},[],[521],{"type":48,"value":522},"_CUSTOM_CONFIG_REGISTRATIONS",{"type":48,"value":524}," if HF does not have a built-in config class",{"type":42,"tag":79,"props":526,"children":527},{},[528,534,535,541,542,548,549,555,556,562],{"type":42,"tag":85,"props":529,"children":531},{"className":530},[],[532],{"type":48,"value":533},"hidden_size",{"type":48,"value":192},{"type":42,"tag":85,"props":536,"children":538},{"className":537},[],[539],{"type":48,"value":540},"intermediate_size",{"type":48,"value":192},{"type":42,"tag":85,"props":543,"children":545},{"className":544},[],[546],{"type":48,"value":547},"num_hidden_layers",{"type":48,"value":192},{"type":42,"tag":85,"props":550,"children":552},{"className":551},[],[553],{"type":48,"value":554},"num_attention_heads",{"type":48,"value":192},{"type":42,"tag":85,"props":557,"children":559},{"className":558},[],[560],{"type":48,"value":561},"num_key_value_heads",{"type":48,"value":563}," -- sizing",{"type":42,"tag":79,"props":565,"children":566},{},[567,573],{"type":42,"tag":85,"props":568,"children":570},{"className":569},[],[571],{"type":48,"value":572},"vocab_size",{"type":48,"value":574}," -- needed for tiny test configs",{"type":42,"tag":79,"props":576,"children":577},{},[578,584],{"type":42,"tag":85,"props":579,"children":581},{"className":580},[],[582],{"type":48,"value":583},"tie_word_embeddings",{"type":48,"value":585}," -- whether lm_head shares weights with embed_tokens",{"type":42,"tag":79,"props":587,"children":588},{},[589,595,597,603],{"type":42,"tag":85,"props":590,"children":592},{"className":591},[],[593],{"type":48,"value":594},"hidden_act",{"type":48,"value":596}," -- activation function (e.g., ",{"type":42,"tag":85,"props":598,"children":600},{"className":599},[],[601],{"type":48,"value":602},"\"silu\"",{"type":48,"value":604}," for SwiGLU)",{"type":42,"tag":445,"props":606,"children":608},{"id":607},"_12-determine-model-type",[609],{"type":48,"value":610},"1.2 Determine model type",{"type":42,"tag":612,"props":613,"children":614},"table",{},[615,639],{"type":42,"tag":616,"props":617,"children":618},"thead",{},[619],{"type":42,"tag":620,"props":621,"children":622},"tr",{},[623,629,634],{"type":42,"tag":624,"props":625,"children":626},"th",{},[627],{"type":48,"value":628},"Type",{"type":42,"tag":624,"props":630,"children":631},{},[632],{"type":48,"value":633},"Indicators",{"type":42,"tag":624,"props":635,"children":636},{},[637],{"type":48,"value":638},"Pattern file",{"type":42,"tag":640,"props":641,"children":642},"tbody",{},[643,676,717],{"type":42,"tag":620,"props":644,"children":645},{},[646,656,666],{"type":42,"tag":647,"props":648,"children":649},"td",{},[650],{"type":42,"tag":651,"props":652,"children":653},"strong",{},[654],{"type":48,"value":655},"Dense LLM",{"type":42,"tag":647,"props":657,"children":658},{},[659,664],{"type":42,"tag":85,"props":660,"children":662},{"className":661},[],[663],{"type":48,"value":158},{"type":48,"value":665}," in architectures, no expert fields",{"type":42,"tag":647,"props":667,"children":668},{},[669],{"type":42,"tag":670,"props":671,"children":673},"a",{"href":672},".\u002Fllm-patterns.md",[674],{"type":48,"value":675},"llm-patterns.md",{"type":42,"tag":620,"props":677,"children":678},{},[679,687,709],{"type":42,"tag":647,"props":680,"children":681},{},[682],{"type":42,"tag":651,"props":683,"children":684},{},[685],{"type":48,"value":686},"MoE LLM",{"type":42,"tag":647,"props":688,"children":689},{},[690,695,696,701,702,707],{"type":42,"tag":85,"props":691,"children":693},{"className":692},[],[694],{"type":48,"value":174},{"type":48,"value":192},{"type":42,"tag":85,"props":697,"children":699},{"className":698},[],[700],{"type":48,"value":166},{"type":48,"value":192},{"type":42,"tag":85,"props":703,"children":705},{"className":704},[],[706],{"type":48,"value":182},{"type":48,"value":708}," in config",{"type":42,"tag":647,"props":710,"children":711},{},[712],{"type":42,"tag":670,"props":713,"children":715},{"href":714},".\u002Fmoe-patterns.md",[716],{"type":48,"value":249},{"type":42,"tag":620,"props":718,"children":719},{},[720,728,750],{"type":42,"tag":647,"props":721,"children":722},{},[723],{"type":42,"tag":651,"props":724,"children":725},{},[726],{"type":48,"value":727},"VLM",{"type":42,"tag":647,"props":729,"children":730},{},[731,736,738,743,745],{"type":42,"tag":85,"props":732,"children":734},{"className":733},[],[735],{"type":48,"value":285},{"type":48,"value":737}," in architectures, has ",{"type":42,"tag":85,"props":739,"children":741},{"className":740},[],[742],{"type":48,"value":270},{"type":48,"value":744}," + ",{"type":42,"tag":85,"props":746,"children":748},{"className":747},[],[749],{"type":48,"value":277},{"type":42,"tag":647,"props":751,"children":752},{},[753],{"type":42,"tag":670,"props":754,"children":756},{"href":755},".\u002Fvlm-patterns.md",[757],{"type":48,"value":293},{"type":42,"tag":445,"props":759,"children":761},{"id":760},"_13-check-for-existing-similar-architectures",[762],{"type":48,"value":763},"1.3 Check for existing similar architectures",{"type":42,"tag":58,"props":765,"children":766},{},[767,769,775],{"type":48,"value":768},"Look in ",{"type":42,"tag":85,"props":770,"children":772},{"className":771},[],[773],{"type":48,"value":774},"components\u002Fmodels\u002F",{"type":48,"value":776}," for architectures with similar attention or MLP patterns:",{"type":42,"tag":778,"props":779,"children":783},"pre",{"className":780,"code":782,"language":48},[781],"language-text","components\u002Fmodels\u002F\n llama\u002F # Standard GQA + SwiGLU (CombinedQKV + CombinedGateUpMLP)\n qwen2\u002F # Same as Llama but with attention bias + QKV bias\n baichuan\u002F # ALiBi attention variant\n deepseek_v3\u002F # MLA attention + MoE (DeepSeek-style grouped experts)\n mistral4\u002F # MLA + MoE + VLM (Pixtral vision)\n kimivl\u002F # DeepSeek-V3 backbone + MoonVit vision\n kimi_k25_vl\u002F # Updated KimiVL with different projector\n qwen3_moe\u002F # Qwen3 with MoE layers\n nemotron_v3\u002F # Hybrid mamba-attention\n",[784],{"type":42,"tag":85,"props":785,"children":787},{"__ignoreMap":786},"",[788],{"type":48,"value":782},{"type":42,"tag":445,"props":790,"children":792},{"id":791},"_14-identify-custom-components",[793],{"type":48,"value":794},"1.4 Identify custom components",{"type":42,"tag":58,"props":796,"children":797},{},[798],{"type":48,"value":799},"Check whether the model needs:",{"type":42,"tag":137,"props":801,"children":802},{},[803,813,823,833,843],{"type":42,"tag":79,"props":804,"children":805},{},[806,811],{"type":42,"tag":651,"props":807,"children":808},{},[809],{"type":48,"value":810},"Custom attention",{"type":48,"value":812},": GQA (standard), MLA (DeepSeek\u002FMistral4), sliding window, bidirectional",{"type":42,"tag":79,"props":814,"children":815},{},[816,821],{"type":42,"tag":651,"props":817,"children":818},{},[819],{"type":48,"value":820},"Custom RoPE",{"type":48,"value":822},": Standard (Llama), YaRN scaling, NTK-aware, complex-number (DeepSeek)",{"type":42,"tag":79,"props":824,"children":825},{},[826,831],{"type":42,"tag":651,"props":827,"children":828},{},[829],{"type":48,"value":830},"Custom normalization",{"type":48,"value":832},": RMSNorm (standard), LayerNorm, different eps values",{"type":42,"tag":79,"props":834,"children":835},{},[836,841],{"type":42,"tag":651,"props":837,"children":838},{},[839],{"type":48,"value":840},"Custom MLP",{"type":48,"value":842},": SwiGLU (standard), GeGLU, ReLU-squared, MoE routing",{"type":42,"tag":79,"props":844,"children":845},{},[846,851,853,859,861,866,868,874],{"type":42,"tag":651,"props":847,"children":848},{},[849],{"type":48,"value":850},"Custom config class",{"type":48,"value":852},": Needed only if HF ",{"type":42,"tag":85,"props":854,"children":856},{"className":855},[],[857],{"type":48,"value":858},"AutoConfig",{"type":48,"value":860}," cannot parse the model's ",{"type":42,"tag":85,"props":862,"children":864},{"className":863},[],[865],{"type":48,"value":90},{"type":48,"value":867}," (check ",{"type":42,"tag":85,"props":869,"children":871},{"className":870},[],[872],{"type":48,"value":873},"auto_map",{"type":48,"value":875}," field)",{"type":42,"tag":445,"props":877,"children":879},{"id":878},"_15-note-dimensions-for-test-config",[880],{"type":48,"value":881},"1.5 Note dimensions for test config",{"type":42,"tag":58,"props":883,"children":884},{},[885],{"type":48,"value":886},"For unit tests, create a tiny config. Target: ~1M parameters or less.",{"type":42,"tag":778,"props":888,"children":892},{"className":889,"code":890,"language":891,"meta":786,"style":786},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","# Example tiny config for a Llama-like model:\ntiny_config = LlamaConfig(\n hidden_size=64,\n intermediate_size=128,\n num_hidden_layers=2,\n num_attention_heads=4,\n num_key_value_heads=2,\n vocab_size=256,\n max_position_embeddings=128,\n)\n","python",[893],{"type":42,"tag":85,"props":894,"children":895},{"__ignoreMap":786},[896,907,916,925,934,943,952,961,970,979],{"type":42,"tag":897,"props":898,"children":901},"span",{"class":899,"line":900},"line",1,[902],{"type":42,"tag":897,"props":903,"children":904},{},[905],{"type":48,"value":906},"# Example tiny config for a Llama-like model:\n",{"type":42,"tag":897,"props":908,"children":910},{"class":899,"line":909},2,[911],{"type":42,"tag":897,"props":912,"children":913},{},[914],{"type":48,"value":915},"tiny_config = LlamaConfig(\n",{"type":42,"tag":897,"props":917,"children":919},{"class":899,"line":918},3,[920],{"type":42,"tag":897,"props":921,"children":922},{},[923],{"type":48,"value":924}," hidden_size=64,\n",{"type":42,"tag":897,"props":926,"children":928},{"class":899,"line":927},4,[929],{"type":42,"tag":897,"props":930,"children":931},{},[932],{"type":48,"value":933}," intermediate_size=128,\n",{"type":42,"tag":897,"props":935,"children":937},{"class":899,"line":936},5,[938],{"type":42,"tag":897,"props":939,"children":940},{},[941],{"type":48,"value":942}," num_hidden_layers=2,\n",{"type":42,"tag":897,"props":944,"children":946},{"class":899,"line":945},6,[947],{"type":42,"tag":897,"props":948,"children":949},{},[950],{"type":48,"value":951}," num_attention_heads=4,\n",{"type":42,"tag":897,"props":953,"children":955},{"class":899,"line":954},7,[956],{"type":42,"tag":897,"props":957,"children":958},{},[959],{"type":48,"value":960}," num_key_value_heads=2,\n",{"type":42,"tag":897,"props":962,"children":964},{"class":899,"line":963},8,[965],{"type":42,"tag":897,"props":966,"children":967},{},[968],{"type":48,"value":969}," vocab_size=256,\n",{"type":42,"tag":897,"props":971,"children":973},{"class":899,"line":972},9,[974],{"type":42,"tag":897,"props":975,"children":976},{},[977],{"type":48,"value":978}," max_position_embeddings=128,\n",{"type":42,"tag":897,"props":980,"children":982},{"class":899,"line":981},10,[983],{"type":42,"tag":897,"props":984,"children":985},{},[986],{"type":48,"value":987},")\n",{"type":42,"tag":989,"props":990,"children":991},"hr",{},[],{"type":42,"tag":51,"props":993,"children":995},{"id":994},"phase-2-implementation",[996],{"type":48,"value":997},"Phase 2: Implementation",{"type":42,"tag":445,"props":999,"children":1001},{"id":1000},"_21-create-directory-structure",[1002],{"type":48,"value":1003},"2.1 Create directory structure",{"type":42,"tag":778,"props":1005,"children":1008},{"className":1006,"code":1007,"language":48},[781],"components\u002Fmodels\u002F\u003Cname>\u002F\n __init__.py\n model.py\n state_dict_adapter.py\n config.py # Only if HF config is insufficient\n layers.py # Only for MoE \u002F MLA \u002F other non-standard layers\n rope_utils.py # Only for custom RoPE\n",[1009],{"type":42,"tag":85,"props":1010,"children":1011},{"__ignoreMap":786},[1012],{"type":48,"value":1007},{"type":42,"tag":445,"props":1014,"children":1016},{"id":1015},"_22-implementation-order",[1017],{"type":48,"value":1018},"2.2 Implementation order",{"type":42,"tag":58,"props":1020,"children":1021},{},[1022],{"type":48,"value":1023},"Implement files in dependency order:",{"type":42,"tag":75,"props":1025,"children":1026},{},[1027,1044,1054,1064,1088,1097],{"type":42,"tag":79,"props":1028,"children":1029},{},[1030,1034,1036,1042],{"type":42,"tag":651,"props":1031,"children":1032},{},[1033],{"type":48,"value":213},{"type":48,"value":1035}," (if needed) -- Custom ",{"type":42,"tag":85,"props":1037,"children":1039},{"className":1038},[],[1040],{"type":48,"value":1041},"PretrainedConfig",{"type":48,"value":1043}," subclass",{"type":42,"tag":79,"props":1045,"children":1046},{},[1047,1052],{"type":42,"tag":651,"props":1048,"children":1049},{},[1050],{"type":48,"value":1051},"rope_utils.py",{"type":48,"value":1053}," (if needed) -- RoPE implementation",{"type":42,"tag":79,"props":1055,"children":1056},{},[1057,1062],{"type":42,"tag":651,"props":1058,"children":1059},{},[1060],{"type":48,"value":1061},"layers.py",{"type":48,"value":1063}," (if needed) -- Attention, MLP, decoder block classes",{"type":42,"tag":79,"props":1065,"children":1066},{},[1067,1072,1074,1079,1081,1086],{"type":42,"tag":651,"props":1068,"children":1069},{},[1070],{"type":48,"value":1071},"model.py",{"type":48,"value":1073}," -- The main ",{"type":42,"tag":85,"props":1075,"children":1077},{"className":1076},[],[1078],{"type":48,"value":158},{"type":48,"value":1080}," (or ",{"type":42,"tag":85,"props":1082,"children":1084},{"className":1083},[],[1085],{"type":48,"value":285},{"type":48,"value":1087},") class",{"type":42,"tag":79,"props":1089,"children":1090},{},[1091,1095],{"type":42,"tag":651,"props":1092,"children":1093},{},[1094],{"type":48,"value":198},{"type":48,"value":1096}," -- HF weight conversion",{"type":42,"tag":79,"props":1098,"children":1099},{},[1100,1110],{"type":42,"tag":651,"props":1101,"children":1102},{},[1103,1108],{"type":42,"tag":651,"props":1104,"children":1105},{},[1106],{"type":48,"value":1107},"init",{"type":48,"value":1109},".py",{"type":48,"value":1111}," -- Re-export the main model class",{"type":42,"tag":58,"props":1113,"children":1114},{},[1115],{"type":48,"value":1116},"See the pattern files for detailed implementation guidance:",{"type":42,"tag":137,"props":1118,"children":1119},{},[1120,1129,1138,1147],{"type":42,"tag":79,"props":1121,"children":1122},{},[1123,1125],{"type":48,"value":1124},"Dense LLM: ",{"type":42,"tag":670,"props":1126,"children":1127},{"href":672},[1128],{"type":48,"value":675},{"type":42,"tag":79,"props":1130,"children":1131},{},[1132,1134],{"type":48,"value":1133},"MoE: ",{"type":42,"tag":670,"props":1135,"children":1136},{"href":714},[1137],{"type":48,"value":249},{"type":42,"tag":79,"props":1139,"children":1140},{},[1141,1143],{"type":48,"value":1142},"VLM: ",{"type":42,"tag":670,"props":1144,"children":1145},{"href":755},[1146],{"type":48,"value":293},{"type":42,"tag":79,"props":1148,"children":1149},{},[1150,1152],{"type":48,"value":1151},"Capabilities and fp32 precision: ",{"type":42,"tag":670,"props":1153,"children":1155},{"href":1154},".\u002Fcapabilities-and-precision.md",[1156],{"type":48,"value":1157},"capabilities-and-precision.md",{"type":42,"tag":445,"props":1159,"children":1161},{"id":1160},"_23-causal-lm-weight-tying",[1162],{"type":48,"value":1163},"2.3 Causal LM weight tying",{"type":42,"tag":58,"props":1165,"children":1166},{},[1167,1169,1174,1176,1182,1184,1190,1192,1198],{"type":48,"value":1168},"For any CausalLM-style class whose config can enable ",{"type":42,"tag":85,"props":1170,"children":1172},{"className":1171},[],[1173],{"type":48,"value":583},{"type":48,"value":1175},",\nmake tying explicit: declare ",{"type":42,"tag":85,"props":1177,"children":1179},{"className":1178},[],[1180],{"type":48,"value":1181},"_tied_weights_keys",{"type":48,"value":1183},", implement ",{"type":42,"tag":85,"props":1185,"children":1187},{"className":1186},[],[1188],{"type":48,"value":1189},"tie_weights()",{"type":48,"value":1191},"\nwith the actual ",{"type":42,"tag":85,"props":1193,"children":1195},{"className":1194},[],[1196],{"type":48,"value":1197},"lm_head",{"type":48,"value":1199}," and input-embedding FQNs, and add tiny tests for\ntied and untied configs. Do not tie architectures with intentionally separate\nheads, asymmetric vocab sizes, or stages that do not own both tensors.",{"type":42,"tag":445,"props":1201,"children":1203},{"id":1202},"_24-moe-state-dict-adapter-checklist",[1204],{"type":48,"value":1205},"2.4 MoE state-dict adapter checklist",{"type":42,"tag":58,"props":1207,"children":1208},{},[1209],{"type":48,"value":1210},"For MoE models, do not stop at generic loading. The adapter must explicitly map:",{"type":42,"tag":137,"props":1212,"children":1213},{},[1214,1219,1224,1229],{"type":42,"tag":79,"props":1215,"children":1216},{},[1217],{"type":48,"value":1218},"Router weights, including gate bias or correction-bias tensors when the Hugging Face model has them.",{"type":42,"tag":79,"props":1220,"children":1221},{},[1222],{"type":48,"value":1223},"Expert weights, preserving expert index order across local and routed experts.",{"type":42,"tag":79,"props":1225,"children":1226},{},[1227],{"type":48,"value":1228},"Gate\u002Fup\u002Fdown projections, including combined or split projection layouts.",{"type":42,"tag":79,"props":1230,"children":1231},{},[1232],{"type":48,"value":1233},"Shared experts separately from routed experts when the architecture has both.",{"type":42,"tag":58,"props":1235,"children":1236},{},[1237],{"type":48,"value":1238},"Add tests that assert expected key mappings and run numerical equivalence with tiny configs before trying full checkpoints.",{"type":42,"tag":58,"props":1240,"children":1241},{},[1242],{"type":48,"value":1243},"Do not use these shortcuts:",{"type":42,"tag":137,"props":1245,"children":1246},{},[1247,1258,1263,1268],{"type":42,"tag":79,"props":1248,"children":1249},{},[1250,1252,1257],{"type":48,"value":1251},"Do not validate the adapter only by calling ",{"type":42,"tag":85,"props":1253,"children":1255},{"className":1254},[],[1256],{"type":48,"value":257},{"type":48,"value":92},{"type":42,"tag":79,"props":1259,"children":1260},{},[1261],{"type":48,"value":1262},"Do not accept missing or extra expert keys without an explicit mapping reason.",{"type":42,"tag":79,"props":1264,"children":1265},{},[1266],{"type":48,"value":1267},"Do not change dtype, transpose dimensions, or reshape tensors unless the HF\nand NeMo layouts require it and a test proves the conversion is reversible.",{"type":42,"tag":79,"props":1269,"children":1270},{},[1271],{"type":48,"value":1272},"Do not skip router or shared-expert tests because dense-layer tests pass.",{"type":42,"tag":445,"props":1274,"children":1276},{"id":1275},"_25-vlm-onboarding-checklist",[1277],{"type":48,"value":1278},"2.5 VLM onboarding checklist",{"type":42,"tag":58,"props":1280,"children":1281},{},[1282,1284,1289,1291,1296,1298,1303,1305,1309,1311,1316,1317,1322,1324,1329],{"type":48,"value":1283},"For VLMs, confirm the Hugging Face config has ",{"type":42,"tag":85,"props":1285,"children":1287},{"className":1286},[],[1288],{"type":48,"value":270},{"type":48,"value":1290}," and ",{"type":42,"tag":85,"props":1292,"children":1294},{"className":1293},[],[1295],{"type":48,"value":277},{"type":48,"value":1297},"\nand that ",{"type":42,"tag":85,"props":1299,"children":1301},{"className":1300},[],[1302],{"type":48,"value":150},{"type":48,"value":1304}," points to a conditional-generation class. Start from\nthe closest VLM pattern file, usually ",{"type":42,"tag":670,"props":1306,"children":1307},{"href":755},[1308],{"type":48,"value":293},{"type":48,"value":1310},", and\ncompare existing implementations such as ",{"type":42,"tag":85,"props":1312,"children":1314},{"className":1313},[],[1315],{"type":48,"value":301},{"type":48,"value":192},{"type":42,"tag":85,"props":1318,"children":1320},{"className":1319},[],[1321],{"type":48,"value":308},{"type":48,"value":1323},", or\n",{"type":42,"tag":85,"props":1325,"children":1327},{"className":1326},[],[1328],{"type":48,"value":315},{"type":48,"value":92},{"type":42,"tag":58,"props":1331,"children":1332},{},[1333],{"type":48,"value":1334},"The implementation should explicitly cover:",{"type":42,"tag":137,"props":1336,"children":1337},{},[1338,1343,1354,1372],{"type":42,"tag":79,"props":1339,"children":1340},{},[1341],{"type":48,"value":1342},"Text backbone, vision tower, projector, and processor or image preprocessing assumptions.",{"type":42,"tag":79,"props":1344,"children":1345},{},[1346,1348,1353],{"type":48,"value":1347},"Weight mapping for both text and vision modules in ",{"type":42,"tag":85,"props":1349,"children":1351},{"className":1350},[],[1352],{"type":48,"value":198},{"type":48,"value":92},{"type":42,"tag":79,"props":1355,"children":1356},{},[1357,1359,1364,1366,1371],{"type":48,"value":1358},"Registration of the ",{"type":42,"tag":85,"props":1360,"children":1362},{"className":1361},[],[1363],{"type":48,"value":285},{"type":48,"value":1365}," class in ",{"type":42,"tag":85,"props":1367,"children":1369},{"className":1368},[],[1370],{"type":48,"value":229},{"type":48,"value":92},{"type":42,"tag":79,"props":1373,"children":1374},{},[1375],{"type":48,"value":1376},"Tiny tests that exercise image-text inputs and verify the adapter round-trip.",{"type":42,"tag":445,"props":1378,"children":1380},{"id":1379},"_26-register-in-registry",[1381],{"type":48,"value":1382},"2.6 Register in registry",{"type":42,"tag":58,"props":1384,"children":1385},{},[1386,1388,1393,1395,1400],{"type":48,"value":1387},"Add the model to ",{"type":42,"tag":85,"props":1389,"children":1391},{"className":1390},[],[1392],{"type":48,"value":221},{"type":48,"value":1394}," in ",{"type":42,"tag":85,"props":1396,"children":1398},{"className":1397},[],[1399],{"type":48,"value":229},{"type":48,"value":1401},":",{"type":42,"tag":778,"props":1403,"children":1405},{"className":889,"code":1404,"language":891,"meta":786,"style":786},"# In _transformers\u002Fregistry.py\nMODEL_ARCH_MAPPING = OrderedDict([\n # ... existing entries ...\n (\n \"NewModelForCausalLM\",\n (\"nemo_automodel.components.models.new_model.model\", \"NewModelForCausalLM\"),\n ),\n])\n",[1406],{"type":42,"tag":85,"props":1407,"children":1408},{"__ignoreMap":786},[1409,1417,1425,1433,1441,1449,1457,1465],{"type":42,"tag":897,"props":1410,"children":1411},{"class":899,"line":900},[1412],{"type":42,"tag":897,"props":1413,"children":1414},{},[1415],{"type":48,"value":1416},"# In _transformers\u002Fregistry.py\n",{"type":42,"tag":897,"props":1418,"children":1419},{"class":899,"line":909},[1420],{"type":42,"tag":897,"props":1421,"children":1422},{},[1423],{"type":48,"value":1424},"MODEL_ARCH_MAPPING = OrderedDict([\n",{"type":42,"tag":897,"props":1426,"children":1427},{"class":899,"line":918},[1428],{"type":42,"tag":897,"props":1429,"children":1430},{},[1431],{"type":48,"value":1432}," # ... existing entries ...\n",{"type":42,"tag":897,"props":1434,"children":1435},{"class":899,"line":927},[1436],{"type":42,"tag":897,"props":1437,"children":1438},{},[1439],{"type":48,"value":1440}," (\n",{"type":42,"tag":897,"props":1442,"children":1443},{"class":899,"line":936},[1444],{"type":42,"tag":897,"props":1445,"children":1446},{},[1447],{"type":48,"value":1448}," \"NewModelForCausalLM\",\n",{"type":42,"tag":897,"props":1450,"children":1451},{"class":899,"line":945},[1452],{"type":42,"tag":897,"props":1453,"children":1454},{},[1455],{"type":48,"value":1456}," (\"nemo_automodel.components.models.new_model.model\", \"NewModelForCausalLM\"),\n",{"type":42,"tag":897,"props":1458,"children":1459},{"class":899,"line":954},[1460],{"type":42,"tag":897,"props":1461,"children":1462},{},[1463],{"type":48,"value":1464}," ),\n",{"type":42,"tag":897,"props":1466,"children":1467},{"class":899,"line":963},[1468],{"type":42,"tag":897,"props":1469,"children":1470},{},[1471],{"type":48,"value":1472},"])\n",{"type":42,"tag":58,"props":1474,"children":1475},{},[1476,1478,1483,1485,1490,1492,1497],{"type":48,"value":1477},"If the model has a custom config class with ",{"type":42,"tag":85,"props":1479,"children":1481},{"className":1480},[],[1482],{"type":48,"value":873},{"type":48,"value":1484}," in its ",{"type":42,"tag":85,"props":1486,"children":1488},{"className":1487},[],[1489],{"type":48,"value":90},{"type":48,"value":1491},", also register in ",{"type":42,"tag":85,"props":1493,"children":1495},{"className":1494},[],[1496],{"type":48,"value":522},{"type":48,"value":1401},{"type":42,"tag":778,"props":1499,"children":1501},{"className":889,"code":1500,"language":891,"meta":786,"style":786},"_CUSTOM_CONFIG_REGISTRATIONS: Dict[str, Tuple[str, str]] = {\n # ... existing entries ...\n \"new_model\": (\"nemo_automodel.components.models.new_model.configuration\", \"NewModelConfig\"),\n}\n",[1502],{"type":42,"tag":85,"props":1503,"children":1504},{"__ignoreMap":786},[1505,1513,1520,1528],{"type":42,"tag":897,"props":1506,"children":1507},{"class":899,"line":900},[1508],{"type":42,"tag":897,"props":1509,"children":1510},{},[1511],{"type":48,"value":1512},"_CUSTOM_CONFIG_REGISTRATIONS: Dict[str, Tuple[str, str]] = {\n",{"type":42,"tag":897,"props":1514,"children":1515},{"class":899,"line":909},[1516],{"type":42,"tag":897,"props":1517,"children":1518},{},[1519],{"type":48,"value":1432},{"type":42,"tag":897,"props":1521,"children":1522},{"class":899,"line":918},[1523],{"type":42,"tag":897,"props":1524,"children":1525},{},[1526],{"type":48,"value":1527}," \"new_model\": (\"nemo_automodel.components.models.new_model.configuration\", \"NewModelConfig\"),\n",{"type":42,"tag":897,"props":1529,"children":1530},{"class":899,"line":927},[1531],{"type":42,"tag":897,"props":1532,"children":1533},{},[1534],{"type":48,"value":1535},"}\n",{"type":42,"tag":445,"props":1537,"children":1539},{"id":1538},"_27-declare-capabilities-and-precision-sensitive-params",[1540],{"type":48,"value":1541},"2.7 Declare capabilities and precision-sensitive params",{"type":42,"tag":58,"props":1543,"children":1544},{},[1545,1547,1552,1554,1560,1562,1568],{"type":48,"value":1546},"Every class registered in ",{"type":42,"tag":85,"props":1548,"children":1550},{"className":1549},[],[1551],{"type":48,"value":221},{"type":48,"value":1553}," must declare parallelism\ncapabilities, either with a static nested ",{"type":42,"tag":85,"props":1555,"children":1557},{"className":1556},[],[1558],{"type":48,"value":1559},"ModelCapabilities",{"type":48,"value":1561}," dataclass or a\nvariant-aware ",{"type":42,"tag":85,"props":1563,"children":1565},{"className":1564},[],[1566],{"type":48,"value":1567},"get_capabilities(cls, config)",{"type":48,"value":1569}," method. Pick exactly one pattern.\nCapabilities should reflect recipe YAMLs that have been validated end to end.",{"type":42,"tag":58,"props":1571,"children":1572},{},[1573,1575,1581,1583,1589,1591,1597,1599,1605,1607,1611],{"type":48,"value":1574},"If the model has precision-sensitive parameters such as Mamba ",{"type":42,"tag":85,"props":1576,"children":1578},{"className":1577},[],[1579],{"type":48,"value":1580},"A_log",{"type":48,"value":1582}," \u002F\n",{"type":42,"tag":85,"props":1584,"children":1586},{"className":1585},[],[1587],{"type":48,"value":1588},"dt_bias",{"type":48,"value":1590},", MoE sigmoid gate bias, attention-sink bias, or per-head ",{"type":42,"tag":85,"props":1592,"children":1594},{"className":1593},[],[1595],{"type":48,"value":1596},"scale",{"type":48,"value":1598},",\ndeclare ",{"type":42,"tag":85,"props":1600,"children":1602},{"className":1601},[],[1603],{"type":48,"value":1604},"_keep_in_fp32_modules_strict",{"type":48,"value":1606}," so sharding keeps those params in fp32\ncompute. See ",{"type":42,"tag":670,"props":1608,"children":1609},{"href":1154},[1610],{"type":48,"value":1157},{"type":48,"value":1612},"\nfor examples, variant dispatch rules, and frozen-submodule dtype guidance.",{"type":42,"tag":989,"props":1614,"children":1615},{},[],{"type":42,"tag":51,"props":1617,"children":1619},{"id":1618},"phase-3-onboarding-example-config",[1620],{"type":48,"value":1621},"Phase 3: Onboarding Example Config",{"type":42,"tag":58,"props":1623,"children":1624},{},[1625],{"type":48,"value":1626},"This phase is only for adding a minimal example config that proves the newly\nonboarded architecture can load and run. Use nemo-automodel-recipe-development for general\nrecipe authoring or existing recipe modifications.",{"type":42,"tag":445,"props":1628,"children":1630},{"id":1629},"_31-create-example-yaml-config",[1631],{"type":48,"value":1632},"3.1 Create example YAML config",{"type":42,"tag":58,"props":1634,"children":1635},{},[1636,1638,1644,1645,1651],{"type":48,"value":1637},"Create an example config under ",{"type":42,"tag":85,"props":1639,"children":1641},{"className":1640},[],[1642],{"type":48,"value":1643},"examples\u002Fllm_finetune\u002F\u003Cname>\u002F",{"type":48,"value":1080},{"type":42,"tag":85,"props":1646,"children":1648},{"className":1647},[],[1649],{"type":48,"value":1650},"examples\u002Fvlm_finetune\u002F\u003Cname>\u002F",{"type":48,"value":1652},"):",{"type":42,"tag":778,"props":1654,"children":1658},{"className":1655,"code":1656,"language":1657,"meta":786,"style":786},"language-yaml shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","model:\n _target_: nemo_automodel.NeMoAutoModelForCausalLM.from_pretrained\n pretrained_model_name_or_path: \u003Corg>\u002F\u003Cmodel-name>\n\ntrainer:\n max_steps: 100\n gradient_clip_val: 1.0\n accumulate_grad_batches: 1\n\n# ... data, optimizer config ...\n","yaml",[1659],{"type":42,"tag":85,"props":1660,"children":1661},{"__ignoreMap":786},[1662,1677,1695,1712,1721,1733,1751,1768,1785,1792],{"type":42,"tag":897,"props":1663,"children":1664},{"class":899,"line":900},[1665,1671],{"type":42,"tag":897,"props":1666,"children":1668},{"style":1667},"--shiki-light:#E53935;--shiki-default:#F07178;--shiki-dark:#F07178",[1669],{"type":48,"value":1670},"model",{"type":42,"tag":897,"props":1672,"children":1674},{"style":1673},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[1675],{"type":48,"value":1676},":\n",{"type":42,"tag":897,"props":1678,"children":1679},{"class":899,"line":909},[1680,1685,1689],{"type":42,"tag":897,"props":1681,"children":1682},{"style":1667},[1683],{"type":48,"value":1684}," _target_",{"type":42,"tag":897,"props":1686,"children":1687},{"style":1673},[1688],{"type":48,"value":1401},{"type":42,"tag":897,"props":1690,"children":1692},{"style":1691},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[1693],{"type":48,"value":1694}," nemo_automodel.NeMoAutoModelForCausalLM.from_pretrained\n",{"type":42,"tag":897,"props":1696,"children":1697},{"class":899,"line":918},[1698,1703,1707],{"type":42,"tag":897,"props":1699,"children":1700},{"style":1667},[1701],{"type":48,"value":1702}," pretrained_model_name_or_path",{"type":42,"tag":897,"props":1704,"children":1705},{"style":1673},[1706],{"type":48,"value":1401},{"type":42,"tag":897,"props":1708,"children":1709},{"style":1691},[1710],{"type":48,"value":1711}," \u003Corg>\u002F\u003Cmodel-name>\n",{"type":42,"tag":897,"props":1713,"children":1714},{"class":899,"line":927},[1715],{"type":42,"tag":897,"props":1716,"children":1718},{"emptyLinePlaceholder":1717},true,[1719],{"type":48,"value":1720},"\n",{"type":42,"tag":897,"props":1722,"children":1723},{"class":899,"line":936},[1724,1729],{"type":42,"tag":897,"props":1725,"children":1726},{"style":1667},[1727],{"type":48,"value":1728},"trainer",{"type":42,"tag":897,"props":1730,"children":1731},{"style":1673},[1732],{"type":48,"value":1676},{"type":42,"tag":897,"props":1734,"children":1735},{"class":899,"line":945},[1736,1741,1745],{"type":42,"tag":897,"props":1737,"children":1738},{"style":1667},[1739],{"type":48,"value":1740}," max_steps",{"type":42,"tag":897,"props":1742,"children":1743},{"style":1673},[1744],{"type":48,"value":1401},{"type":42,"tag":897,"props":1746,"children":1748},{"style":1747},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[1749],{"type":48,"value":1750}," 100\n",{"type":42,"tag":897,"props":1752,"children":1753},{"class":899,"line":954},[1754,1759,1763],{"type":42,"tag":897,"props":1755,"children":1756},{"style":1667},[1757],{"type":48,"value":1758}," gradient_clip_val",{"type":42,"tag":897,"props":1760,"children":1761},{"style":1673},[1762],{"type":48,"value":1401},{"type":42,"tag":897,"props":1764,"children":1765},{"style":1747},[1766],{"type":48,"value":1767}," 1.0\n",{"type":42,"tag":897,"props":1769,"children":1770},{"class":899,"line":963},[1771,1776,1780],{"type":42,"tag":897,"props":1772,"children":1773},{"style":1667},[1774],{"type":48,"value":1775}," accumulate_grad_batches",{"type":42,"tag":897,"props":1777,"children":1778},{"style":1673},[1779],{"type":48,"value":1401},{"type":42,"tag":897,"props":1781,"children":1782},{"style":1747},[1783],{"type":48,"value":1784}," 1\n",{"type":42,"tag":897,"props":1786,"children":1787},{"class":899,"line":972},[1788],{"type":42,"tag":897,"props":1789,"children":1790},{"emptyLinePlaceholder":1717},[1791],{"type":48,"value":1720},{"type":42,"tag":897,"props":1793,"children":1794},{"class":899,"line":981},[1795],{"type":42,"tag":897,"props":1796,"children":1798},{"style":1797},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[1799],{"type":48,"value":1800},"# ... data, optimizer config ...\n",{"type":42,"tag":445,"props":1802,"children":1804},{"id":1803},"_32-verify-model-loads",[1805],{"type":48,"value":1806},"3.2 Verify model loads",{"type":42,"tag":58,"props":1808,"children":1809},{},[1810],{"type":48,"value":1811},"Test that the model loads from a HuggingFace checkpoint:",{"type":42,"tag":778,"props":1813,"children":1815},{"className":889,"code":1814,"language":891,"meta":786,"style":786},"from nemo_automodel import NeMoAutoModelForCausalLM\n\nmodel = NeMoAutoModelForCausalLM.from_pretrained(\"\u003Corg>\u002F\u003Cmodel-name>\")\n",[1816],{"type":42,"tag":85,"props":1817,"children":1818},{"__ignoreMap":786},[1819,1827,1834],{"type":42,"tag":897,"props":1820,"children":1821},{"class":899,"line":900},[1822],{"type":42,"tag":897,"props":1823,"children":1824},{},[1825],{"type":48,"value":1826},"from nemo_automodel import NeMoAutoModelForCausalLM\n",{"type":42,"tag":897,"props":1828,"children":1829},{"class":899,"line":909},[1830],{"type":42,"tag":897,"props":1831,"children":1832},{"emptyLinePlaceholder":1717},[1833],{"type":48,"value":1720},{"type":42,"tag":897,"props":1835,"children":1836},{"class":899,"line":918},[1837],{"type":42,"tag":897,"props":1838,"children":1839},{},[1840],{"type":48,"value":1841},"model = NeMoAutoModelForCausalLM.from_pretrained(\"\u003Corg>\u002F\u003Cmodel-name>\")\n",{"type":42,"tag":445,"props":1843,"children":1845},{"id":1844},"_33-test-with-tiny-config-first",[1846],{"type":48,"value":1847},"3.3 Test with tiny config first",{"type":42,"tag":58,"props":1849,"children":1850},{},[1851],{"type":48,"value":1852},"Before using full-size models, verify with a tiny config (1-2 layers, small hidden dim) to catch shape mismatches early.",{"type":42,"tag":51,"props":1854,"children":1856},{"id":1855},"phase-4-tests",[1857],{"type":48,"value":1858},"Phase 4: Tests",{"type":42,"tag":58,"props":1860,"children":1861},{},[1862,1864,1870],{"type":48,"value":1863},"Create ",{"type":42,"tag":85,"props":1865,"children":1867},{"className":1866},[],[1868],{"type":48,"value":1869},"tests\u002Funit_tests\u002Fmodels\u002F\u003Cname>\u002F",{"type":48,"value":1871}," and cover the checks below before\nloading full checkpoints:",{"type":42,"tag":137,"props":1873,"children":1874},{},[1875,1880,1893,1906],{"type":42,"tag":79,"props":1876,"children":1877},{},[1878],{"type":48,"value":1879},"Forward-shape smoke test with a tiny config.",{"type":42,"tag":79,"props":1881,"children":1882},{},[1883,1885,1891],{"type":48,"value":1884},"State-dict adapter round-trip: ",{"type":42,"tag":85,"props":1886,"children":1888},{"className":1887},[],[1889],{"type":48,"value":1890},"from_hf -> to_hf",{"type":48,"value":1892}," preserves mapped names,\nshapes, dtypes, and values.",{"type":42,"tag":79,"props":1894,"children":1895},{},[1896,1898,1904],{"type":48,"value":1897},"Layer equivalence tests for every rewritten attention, MLP, normalization,\nRoPE, or MoE layer. Use the model dtype from config, identical seeded weights,\nidentical inputs, and dtype-appropriate ",{"type":42,"tag":85,"props":1899,"children":1901},{"className":1900},[],[1902],{"type":48,"value":1903},"torch.allclose",{"type":48,"value":1905}," tolerances.",{"type":42,"tag":79,"props":1907,"children":1908},{},[1909],{"type":48,"value":1910},"Short functional test that verifies loss decreases over a few training steps.",{"type":42,"tag":989,"props":1912,"children":1913},{},[],{"type":42,"tag":51,"props":1915,"children":1917},{"id":1916},"phase-5-documentation",[1918],{"type":48,"value":1919},"Phase 5: Documentation",{"type":42,"tag":445,"props":1921,"children":1923},{"id":1922},"_51-update-model-coverage-page",[1924],{"type":48,"value":1925},"5.1 Update model coverage page",{"type":42,"tag":58,"props":1927,"children":1928},{},[1929,1931,1937],{"type":48,"value":1930},"Edit the appropriate file in ",{"type":42,"tag":85,"props":1932,"children":1934},{"className":1933},[],[1935],{"type":48,"value":1936},"docs\u002Fmodel-coverage\u002F",{"type":48,"value":1401},{"type":42,"tag":137,"props":1939,"children":1940},{},[1941,1952],{"type":42,"tag":79,"props":1942,"children":1943},{},[1944,1946],{"type":48,"value":1945},"LLM\u002FMoE: ",{"type":42,"tag":85,"props":1947,"children":1949},{"className":1948},[],[1950],{"type":48,"value":1951},"docs\u002Fmodel-coverage\u002Fllm\u002Findex.md",{"type":42,"tag":79,"props":1953,"children":1954},{},[1955,1956],{"type":48,"value":1142},{"type":42,"tag":85,"props":1957,"children":1959},{"className":1958},[],[1960],{"type":48,"value":1961},"docs\u002Fmodel-coverage\u002Fvlm\u002Findex.md",{"type":42,"tag":58,"props":1963,"children":1964},{},[1965],{"type":48,"value":1966},"Add a row with the model name, supported features (TP, PP, FSDP, LoRA, QLoRA), and any limitations.",{"type":42,"tag":989,"props":1968,"children":1969},{},[],{"type":42,"tag":51,"props":1971,"children":1973},{"id":1972},"phase-6-parity-testing",[1974],{"type":48,"value":1975},"Phase 6: Parity Testing",{"type":42,"tag":58,"props":1977,"children":1978},{},[1979],{"type":48,"value":1980},"After implementation and unit tests are complete, run the full parity-testing\nworkflow to verify that the new model produces numerically equivalent results to\nthe reference HuggingFace implementation.",{"type":42,"tag":58,"props":1982,"children":1983},{},[1984],{"type":48,"value":1985},"Run three levels of comparison:",{"type":42,"tag":75,"props":1987,"children":1988},{},[1989,1994,1999],{"type":42,"tag":79,"props":1990,"children":1991},{},[1992],{"type":48,"value":1993},"State-dict round-trip: load a reference HuggingFace checkpoint, convert it\ninto the NeMo AutoModel layout, export it back, and verify that all mapped\ntensors match the reference names, shapes, dtypes, and values within the\nexpected tolerance.",{"type":42,"tag":79,"props":1995,"children":1996},{},[1997],{"type":48,"value":1998},"Component-level parity: compare rewritten attention, MLP, normalization,\nRoPE, and MoE components against the HuggingFace implementation with fixed\nseeds and identical dtype.",{"type":42,"tag":79,"props":2000,"children":2001},{},[2002],{"type":48,"value":2003},"End-to-end forward pass: run the full NeMo AutoModel and HuggingFace model\non the same tokenized input and compare logits, hidden states, and loss.",{"type":42,"tag":58,"props":2005,"children":2006},{},[2007],{"type":48,"value":2008},"Do not skip this phase. A model that passes unit tests can still diverge from HF\ndue to subtle weight-conversion bugs, backend differences, or RoPE mismatches\nthat only surface in a full parity comparison.",{"type":42,"tag":989,"props":2010,"children":2011},{},[],{"type":42,"tag":51,"props":2013,"children":2015},{"id":2014},"key-files-reference",[2016],{"type":48,"value":2017},"Key Files Reference",{"type":42,"tag":612,"props":2019,"children":2020},{},[2021,2036],{"type":42,"tag":616,"props":2022,"children":2023},{},[2024],{"type":42,"tag":620,"props":2025,"children":2026},{},[2027,2032],{"type":42,"tag":624,"props":2028,"children":2029},{},[2030],{"type":48,"value":2031},"File",{"type":42,"tag":624,"props":2033,"children":2034},{},[2035],{"type":48,"value":56},{"type":42,"tag":640,"props":2037,"children":2038},{},[2039,2064,2110,2145,2167,2190,2212,2253,2276,2299,2330],{"type":42,"tag":620,"props":2040,"children":2041},{},[2042,2050],{"type":42,"tag":647,"props":2043,"children":2044},{},[2045],{"type":42,"tag":85,"props":2046,"children":2048},{"className":2047},[],[2049],{"type":48,"value":229},{"type":42,"tag":647,"props":2051,"children":2052},{},[2053,2058,2059],{"type":42,"tag":85,"props":2054,"children":2056},{"className":2055},[],[2057],{"type":48,"value":221},{"type":48,"value":1290},{"type":42,"tag":85,"props":2060,"children":2062},{"className":2061},[],[2063],{"type":48,"value":522},{"type":42,"tag":620,"props":2065,"children":2066},{},[2067,2076],{"type":42,"tag":647,"props":2068,"children":2069},{},[2070],{"type":42,"tag":85,"props":2071,"children":2073},{"className":2072},[],[2074],{"type":48,"value":2075},"components\u002Fmodels\u002Fcommon\u002F__init__.py",{"type":42,"tag":647,"props":2077,"children":2078},{},[2079,2081,2087,2088,2094,2095,2101,2102,2108],{"type":48,"value":2080},"Exports ",{"type":42,"tag":85,"props":2082,"children":2084},{"className":2083},[],[2085],{"type":48,"value":2086},"CombinedQKVAttentionMixin",{"type":48,"value":192},{"type":42,"tag":85,"props":2089,"children":2091},{"className":2090},[],[2092],{"type":48,"value":2093},"CombinedGateUpMLP",{"type":48,"value":192},{"type":42,"tag":85,"props":2096,"children":2098},{"className":2097},[],[2099],{"type":48,"value":2100},"BackendConfig",{"type":48,"value":192},{"type":42,"tag":85,"props":2103,"children":2105},{"className":2104},[],[2106],{"type":48,"value":2107},"HFCheckpointingMixin",{"type":48,"value":2109},", etc.",{"type":42,"tag":620,"props":2111,"children":2112},{},[2113,2122],{"type":42,"tag":647,"props":2114,"children":2115},{},[2116],{"type":42,"tag":85,"props":2117,"children":2119},{"className":2118},[],[2120],{"type":48,"value":2121},"components\u002Fmodels\u002Fcommon\u002Fcombined_projection\u002Fcombined_qkv.py",{"type":42,"tag":647,"props":2123,"children":2124},{},[2125,2130,2132,2138,2139],{"type":42,"tag":85,"props":2126,"children":2128},{"className":2127},[],[2129],{"type":48,"value":2086},{"type":48,"value":2131}," with ",{"type":42,"tag":85,"props":2133,"children":2135},{"className":2134},[],[2136],{"type":48,"value":2137},"setup_qkv_projection()",{"type":48,"value":1290},{"type":42,"tag":85,"props":2140,"children":2142},{"className":2141},[],[2143],{"type":48,"value":2144},"compute_qkv()",{"type":42,"tag":620,"props":2146,"children":2147},{},[2148,2157],{"type":42,"tag":647,"props":2149,"children":2150},{},[2151],{"type":42,"tag":85,"props":2152,"children":2154},{"className":2153},[],[2155],{"type":48,"value":2156},"components\u002Fmodels\u002Fcommon\u002Fcombined_projection\u002Fcombined_mlp.py",{"type":42,"tag":647,"props":2158,"children":2159},{},[2160,2165],{"type":42,"tag":85,"props":2161,"children":2163},{"className":2162},[],[2164],{"type":48,"value":2093},{"type":48,"value":2166}," with interleaved gate\u002Fup layout",{"type":42,"tag":620,"props":2168,"children":2169},{},[2170,2179],{"type":42,"tag":647,"props":2171,"children":2172},{},[2173],{"type":42,"tag":85,"props":2174,"children":2176},{"className":2175},[],[2177],{"type":48,"value":2178},"components\u002Fmodels\u002Fcommon\u002Fcombined_projection\u002Fstate_dict_adapter.py",{"type":42,"tag":647,"props":2180,"children":2181},{},[2182,2188],{"type":42,"tag":85,"props":2183,"children":2185},{"className":2184},[],[2186],{"type":48,"value":2187},"CombinedProjectionStateDictAdapter",{"type":48,"value":2189}," base class",{"type":42,"tag":620,"props":2191,"children":2192},{},[2193,2202],{"type":42,"tag":647,"props":2194,"children":2195},{},[2196],{"type":42,"tag":85,"props":2197,"children":2199},{"className":2198},[],[2200],{"type":48,"value":2201},"components\u002Fmodels\u002Fcommon\u002Fhf_checkpointing_mixin.py",{"type":42,"tag":647,"props":2203,"children":2204},{},[2205,2210],{"type":42,"tag":85,"props":2206,"children":2208},{"className":2207},[],[2209],{"type":48,"value":2107},{"type":48,"value":2211}," for save\u002Fload",{"type":42,"tag":620,"props":2213,"children":2214},{},[2215,2224],{"type":42,"tag":647,"props":2216,"children":2217},{},[2218],{"type":42,"tag":85,"props":2219,"children":2221},{"className":2220},[],[2222],{"type":48,"value":2223},"components\u002Fmodels\u002Fcommon\u002Futils.py",{"type":42,"tag":647,"props":2225,"children":2226},{},[2227,2232,2233,2239,2240,2246,2247],{"type":42,"tag":85,"props":2228,"children":2230},{"className":2229},[],[2231],{"type":48,"value":2100},{"type":48,"value":192},{"type":42,"tag":85,"props":2234,"children":2236},{"className":2235},[],[2237],{"type":48,"value":2238},"initialize_rms_norm_module",{"type":48,"value":192},{"type":42,"tag":85,"props":2241,"children":2243},{"className":2242},[],[2244],{"type":48,"value":2245},"initialize_linear_module",{"type":48,"value":192},{"type":42,"tag":85,"props":2248,"children":2250},{"className":2249},[],[2251],{"type":48,"value":2252},"get_rope_config",{"type":42,"tag":620,"props":2254,"children":2255},{},[2256,2265],{"type":42,"tag":647,"props":2257,"children":2258},{},[2259],{"type":42,"tag":85,"props":2260,"children":2262},{"className":2261},[],[2263],{"type":48,"value":2264},"components\u002Fmoe\u002Fconfig.py",{"type":42,"tag":647,"props":2266,"children":2267},{},[2268,2274],{"type":42,"tag":85,"props":2269,"children":2271},{"className":2270},[],[2272],{"type":48,"value":2273},"MoEConfig",{"type":48,"value":2275}," dataclass",{"type":42,"tag":620,"props":2277,"children":2278},{},[2279,2288],{"type":42,"tag":647,"props":2280,"children":2281},{},[2282],{"type":42,"tag":85,"props":2283,"children":2285},{"className":2284},[],[2286],{"type":48,"value":2287},"components\u002Fmoe\u002Ffsdp_mixin.py",{"type":42,"tag":647,"props":2289,"children":2290},{},[2291,2297],{"type":42,"tag":85,"props":2292,"children":2294},{"className":2293},[],[2295],{"type":48,"value":2296},"MoEFSDPSyncMixin",{"type":48,"value":2298}," for distributed expert handling",{"type":42,"tag":620,"props":2300,"children":2301},{},[2302,2311],{"type":42,"tag":647,"props":2303,"children":2304},{},[2305],{"type":42,"tag":85,"props":2306,"children":2308},{"className":2307},[],[2309],{"type":48,"value":2310},"components\u002Fmoe\u002Flayers.py",{"type":42,"tag":647,"props":2312,"children":2313},{},[2314,2320,2322,2328],{"type":42,"tag":85,"props":2315,"children":2317},{"className":2316},[],[2318],{"type":48,"value":2319},"MoE",{"type":48,"value":2321}," layer, ",{"type":42,"tag":85,"props":2323,"children":2325},{"className":2324},[],[2326],{"type":48,"value":2327},"MLP",{"type":48,"value":2329}," (dense) for MoE blocks",{"type":42,"tag":620,"props":2331,"children":2332},{},[2333,2342],{"type":42,"tag":647,"props":2334,"children":2335},{},[2336],{"type":42,"tag":85,"props":2337,"children":2339},{"className":2338},[],[2340],{"type":48,"value":2341},"components\u002Fmoe\u002Fexperts.py",{"type":42,"tag":647,"props":2343,"children":2344},{},[2345,2351,2352,2358,2359],{"type":42,"tag":85,"props":2346,"children":2348},{"className":2347},[],[2349],{"type":48,"value":2350},"GroupedExperts",{"type":48,"value":192},{"type":42,"tag":85,"props":2353,"children":2355},{"className":2354},[],[2356],{"type":48,"value":2357},"GroupedExpertsDeepEP",{"type":48,"value":192},{"type":42,"tag":85,"props":2360,"children":2362},{"className":2361},[],[2363],{"type":48,"value":2364},"GroupedExpertsTE",{"type":42,"tag":989,"props":2366,"children":2367},{},[],{"type":42,"tag":51,"props":2369,"children":2371},{"id":2370},"checklist",[2372],{"type":48,"value":2373},"Checklist",{"type":42,"tag":137,"props":2375,"children":2378},{"className":2376},[2377],"contains-task-list",[2379,2398,2407,2416,2432,2441,2450,2459,2473,2482,2497,2517,2533,2556,2565,2580,2589,2633,2642,2651,2660,2669],{"type":42,"tag":79,"props":2380,"children":2383},{"className":2381},[2382],"task-list-item",[2384,2389,2391,2396],{"type":42,"tag":2385,"props":2386,"children":2388},"input",{"disabled":1717,"type":2387},"checkbox",[],{"type":48,"value":2390}," Fetched and analyzed ",{"type":42,"tag":85,"props":2392,"children":2394},{"className":2393},[],[2395],{"type":48,"value":90},{"type":48,"value":2397}," from HuggingFace",{"type":42,"tag":79,"props":2399,"children":2401},{"className":2400},[2382],[2402,2405],{"type":42,"tag":2385,"props":2403,"children":2404},{"disabled":1717,"type":2387},[],{"type":48,"value":2406}," Determined model type (dense LLM \u002F MoE \u002F VLM)",{"type":42,"tag":79,"props":2408,"children":2410},{"className":2409},[2382],[2411,2414],{"type":42,"tag":2385,"props":2412,"children":2413},{"disabled":1717,"type":2387},[],{"type":48,"value":2415}," Identified custom components (attention, RoPE, normalization, MLP)",{"type":42,"tag":79,"props":2417,"children":2419},{"className":2418},[2382],[2420,2423,2425,2430],{"type":42,"tag":2385,"props":2421,"children":2422},{"disabled":1717,"type":2387},[],{"type":48,"value":2424}," Created ",{"type":42,"tag":85,"props":2426,"children":2428},{"className":2427},[],[2429],{"type":48,"value":103},{"type":48,"value":2431}," directory",{"type":42,"tag":79,"props":2433,"children":2435},{"className":2434},[2382],[2436,2439],{"type":42,"tag":2385,"props":2437,"children":2438},{"disabled":1717,"type":2387},[],{"type":48,"value":2440}," Implemented config.py (if custom config needed)",{"type":42,"tag":79,"props":2442,"children":2444},{"className":2443},[2382],[2445,2448],{"type":42,"tag":2385,"props":2446,"children":2447},{"disabled":1717,"type":2387},[],{"type":48,"value":2449}," Implemented layers.py (if custom layers needed)",{"type":42,"tag":79,"props":2451,"children":2453},{"className":2452},[2382],[2454,2457],{"type":42,"tag":2385,"props":2455,"children":2456},{"disabled":1717,"type":2387},[],{"type":48,"value":2458}," Implemented rope_utils.py (if custom RoPE needed)",{"type":42,"tag":79,"props":2460,"children":2462},{"className":2461},[2382],[2463,2466,2468],{"type":42,"tag":2385,"props":2464,"children":2465},{"disabled":1717,"type":2387},[],{"type":48,"value":2467}," Implemented model.py with ",{"type":42,"tag":85,"props":2469,"children":2471},{"className":2470},[],[2472],{"type":48,"value":2107},{"type":42,"tag":79,"props":2474,"children":2476},{"className":2475},[2382],[2477,2480],{"type":42,"tag":2385,"props":2478,"children":2479},{"disabled":1717,"type":2387},[],{"type":48,"value":2481}," Implemented state_dict_adapter.py",{"type":42,"tag":79,"props":2483,"children":2485},{"className":2484},[2382],[2486,2489,2491,2495],{"type":42,"tag":2385,"props":2487,"children":2488},{"disabled":1717,"type":2387},[],{"type":48,"value":2490}," Implemented ",{"type":42,"tag":651,"props":2492,"children":2493},{},[2494],{"type":48,"value":1107},{"type":48,"value":2496},".py with re-export",{"type":42,"tag":79,"props":2498,"children":2500},{"className":2499},[2382],[2501,2504,2506,2511,2512],{"type":42,"tag":2385,"props":2502,"children":2503},{"disabled":1717,"type":2387},[],{"type":48,"value":2505}," Registered in ",{"type":42,"tag":85,"props":2507,"children":2509},{"className":2508},[],[2510],{"type":48,"value":221},{"type":48,"value":1394},{"type":42,"tag":85,"props":2513,"children":2515},{"className":2514},[],[2516],{"type":48,"value":229},{"type":42,"tag":79,"props":2518,"children":2520},{"className":2519},[2382],[2521,2524,2526,2531],{"type":42,"tag":2385,"props":2522,"children":2523},{"disabled":1717,"type":2387},[],{"type":48,"value":2525}," Registered custom config in ",{"type":42,"tag":85,"props":2527,"children":2529},{"className":2528},[],[2530],{"type":48,"value":522},{"type":48,"value":2532}," (if applicable)",{"type":42,"tag":79,"props":2534,"children":2536},{"className":2535},[2382],[2537,2540,2542,2547,2549,2554],{"type":42,"tag":2385,"props":2538,"children":2539},{"disabled":1717,"type":2387},[],{"type":48,"value":2541}," Declared ",{"type":42,"tag":85,"props":2543,"children":2545},{"className":2544},[],[2546],{"type":48,"value":1559},{"type":48,"value":2548}," nested dataclass (static) OR ",{"type":42,"tag":85,"props":2550,"children":2552},{"className":2551},[],[2553],{"type":48,"value":1567},{"type":48,"value":2555}," classmethod (variant dispatch, e.g. ERNIE-4.5 MoE vs dense) — never both, never neither",{"type":42,"tag":79,"props":2557,"children":2559},{"className":2558},[2382],[2560,2563],{"type":42,"tag":2385,"props":2561,"children":2562},{"disabled":1717,"type":2387},[],{"type":48,"value":2564}," Created example YAML config",{"type":42,"tag":79,"props":2566,"children":2568},{"className":2567},[2382],[2569,2572,2574],{"type":42,"tag":2385,"props":2570,"children":2571},{"disabled":1717,"type":2387},[],{"type":48,"value":2573}," Verified model loads via ",{"type":42,"tag":85,"props":2575,"children":2577},{"className":2576},[],[2578],{"type":48,"value":2579},"NeMoAutoModelForCausalLM.from_pretrained()",{"type":42,"tag":79,"props":2581,"children":2583},{"className":2582},[2382],[2584,2587],{"type":42,"tag":2385,"props":2585,"children":2586},{"disabled":1717,"type":2387},[],{"type":48,"value":2588}," Created unit tests (forward shape, state_dict round-trip)",{"type":42,"tag":79,"props":2590,"children":2592},{"className":2591},[2382],[2593,2596,2597,2602,2604,2609,2611,2616,2618,2624,2626,2631],{"type":42,"tag":2385,"props":2594,"children":2595},{"disabled":1717,"type":2387},[],{"type":48,"value":2541},{"type":42,"tag":85,"props":2598,"children":2600},{"className":2599},[],[2601],{"type":48,"value":1604},{"type":48,"value":2603}," for every intrinsically-fp32 param (SSM ",{"type":42,"tag":85,"props":2605,"children":2607},{"className":2606},[],[2608],{"type":48,"value":1580},{"type":48,"value":2610},"\u002F",{"type":42,"tag":85,"props":2612,"children":2614},{"className":2613},[],[2615],{"type":48,"value":1588},{"type":48,"value":2617},", Mamba ",{"type":42,"tag":85,"props":2619,"children":2621},{"className":2620},[],[2622],{"type":48,"value":2623},"D",{"type":48,"value":2625}," when reference-fp32, MoE gate bias, attention-sink bias, ",{"type":42,"tag":85,"props":2627,"children":2629},{"className":2628},[],[2630],{"type":48,"value":1596},{"type":48,"value":2632},", …) — see §2.7",{"type":42,"tag":79,"props":2634,"children":2636},{"className":2635},[2382],[2637,2640],{"type":42,"tag":2385,"props":2638,"children":2639},{"disabled":1717,"type":2387},[],{"type":48,"value":2641}," Created layer equivalence tests for every rewritten layer (matching model dtype)",{"type":42,"tag":79,"props":2643,"children":2645},{"className":2644},[2382],[2646,2649],{"type":42,"tag":2385,"props":2647,"children":2648},{"disabled":1717,"type":2387},[],{"type":48,"value":2650}," Created functional tests (training loss decreases)",{"type":42,"tag":79,"props":2652,"children":2654},{"className":2653},[2382],[2655,2658],{"type":42,"tag":2385,"props":2656,"children":2657},{"disabled":1717,"type":2387},[],{"type":48,"value":2659}," Updated docs\u002Fmodel-coverage page",{"type":42,"tag":79,"props":2661,"children":2663},{"className":2662},[2382],[2664,2667],{"type":42,"tag":2385,"props":2665,"children":2666},{"disabled":1717,"type":2387},[],{"type":48,"value":2668}," Ran state-dict round-trip, component parity, and E2E forward-pass parity checks",{"type":42,"tag":79,"props":2670,"children":2672},{"className":2671},[2382],[2673,2676,2678,2684],{"type":42,"tag":2385,"props":2674,"children":2675},{"disabled":1717,"type":2387},[],{"type":48,"value":2677}," Set ",{"type":42,"tag":85,"props":2679,"children":2681},{"className":2680},[],[2682],{"type":48,"value":2683},"ModelClass = \u003CName>ForCausalLM",{"type":48,"value":2685}," at module bottom",{"type":42,"tag":2687,"props":2688,"children":2689},"style",{},[2690],{"type":48,"value":2691},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":2693,"total":2797},[2694,2711,2725,2739,2751,2768,2783],{"slug":2695,"name":2695,"fn":2696,"description":2697,"org":2698,"tags":2699,"stars":20,"repoUrl":21,"updatedAt":2710},"accelerated-computing-cudf","accelerate data processing with cuDF","Official NVIDIA-authored guidance for NVIDIA cuDF GPU DataFrames, pandas acceleration, dask-cuDF, ETL, joins, groupby, CSV\u002FParquet I\u002FO, nullable semantics, and multi-GPU DataFrame workloads.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2700,2703,2706,2707],{"name":2701,"slug":2702,"type":15},"Data Analysis","data-analysis",{"name":2704,"slug":2705,"type":15},"Data Engineering","data-engineering",{"name":9,"slug":8,"type":15},{"name":2708,"slug":2709,"type":15},"Performance","performance","2026-07-14T05:28:43.176466",{"slug":2712,"name":2712,"fn":2713,"description":2714,"org":2715,"tags":2716,"stars":20,"repoUrl":21,"updatedAt":2724},"aiq-deploy","deploy and manage NVIDIA AI-Q infrastructure","Use when asked to install, deploy, run, validate, troubleshoot, or stop NVIDIA AI-Q Blueprint infrastructure.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2717,2720,2723],{"name":2718,"slug":2719,"type":15},"Deployment","deployment",{"name":2721,"slug":2722,"type":15},"Infrastructure","infrastructure",{"name":9,"slug":8,"type":15},"2026-07-14T05:29:06.667109",{"slug":2726,"name":2726,"fn":2727,"description":2728,"org":2729,"tags":2730,"stars":20,"repoUrl":21,"updatedAt":2738},"aiq-research","conduct deep research with AI-Q","Use when asked to run deep research or AI-Q research through a reachable NVIDIA AI-Q Blueprint backend.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2731,2734,2735],{"name":2732,"slug":2733,"type":15},"Agents","agents",{"name":9,"slug":8,"type":15},{"name":2736,"slug":2737,"type":15},"Research","research","2026-07-14T05:28:06.816956",{"slug":2740,"name":2740,"fn":2741,"description":2742,"org":2743,"tags":2744,"stars":20,"repoUrl":21,"updatedAt":2750},"amc-run-sample-calibration","run AMC sample dataset calibration","Run end-to-end calibration on the shipped sample dataset (sdg_08_2_sample_data_010926.zip) against a running AMC microservice. Use when user says 'test sample dataset', 'run sample calibration', 'verify AMC install', or 'launch and test'.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2745,2746,2747],{"name":2701,"slug":2702,"type":15},{"name":9,"slug":8,"type":15},{"name":2748,"slug":2749,"type":15},"Testing","testing","2026-07-17T05:29:03.913266",{"slug":2752,"name":2752,"fn":2753,"description":2754,"org":2755,"tags":2756,"stars":20,"repoUrl":21,"updatedAt":2767},"amc-run-video-calibration","calibrate video datasets with AutoMagicCalib","Calibrate a new dataset from pre-recorded video files via the AutoMagicCalib REST API. Use when user has local MP4s and says 'calibrate my videos', 'run AMC on these videos', or similar. For RTSP\u002Flive streams, use amc-run-rtsp-calibration instead.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2757,2760,2763,2764],{"name":2758,"slug":2759,"type":15},"Automation","automation",{"name":2761,"slug":2762,"type":15},"Imaging","imaging",{"name":9,"slug":8,"type":15},{"name":2765,"slug":2766,"type":15},"Video","video","2026-07-17T05:28:53.905004",{"slug":2769,"name":2769,"fn":2770,"description":2771,"org":2772,"tags":2773,"stars":20,"repoUrl":21,"updatedAt":2782},"amc-setup-calibration-stack","deploy AutoMagicCalib microservice with Docker","Launch AutoMagicCalib microservice and web UI from NGC release images via Docker Compose. Use when user says 'deploy auto calibration', 'launch auto calibration', 'launch AMC', 'start MS+UI', or 'set up auto-magic-calib'. Requires NGC API key.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2774,2775,2778,2779],{"name":2718,"slug":2719,"type":15},{"name":2776,"slug":2777,"type":15},"Docker","docker",{"name":9,"slug":8,"type":15},{"name":2780,"slug":2781,"type":15},"Operations","operations","2026-07-17T05:28:56.913999",{"slug":2784,"name":2784,"fn":2785,"description":2786,"org":2787,"tags":2788,"stars":20,"repoUrl":21,"updatedAt":2796},"cudaq-guide","develop quantum applications with CUDA-Q","CUDA-Q onboarding guide for installation, test programs, GPU simulation, QPU hardware, and quantum applications.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2789,2790,2793],{"name":9,"slug":8,"type":15},{"name":2791,"slug":2792,"type":15},"Quantum Computing","quantum-computing",{"name":2794,"slug":2795,"type":15},"Simulation","simulation","2026-07-14T05:26:58.898253",305,{"items":2799,"total":2947},[2800,2818,2833,2844,2856,2870,2883,2895,2906,2915,2929,2938],{"slug":2801,"name":2801,"fn":2802,"description":2803,"org":2804,"tags":2805,"stars":2815,"repoUrl":2816,"updatedAt":2817},"nemoclaw-user-guide","retrieve NemoClaw documentation and configuration","Guides human users' AI agents to the NemoClaw docs MCP server and canonical Fern documentation in Markdown form. Use when users ask how to install, configure, operate, troubleshoot, secure, or learn NemoClaw with an AI coding assistant. Trigger keywords - nemoclaw docs, use nemoclaw with ai agent, nemoclaw mcp docs, nemoclaw install help, nemoclaw quickstart, nemoclaw markdown docs, llms.txt, agent skills.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2806,2809,2812],{"name":2807,"slug":2808,"type":15},"Documentation","documentation",{"name":2810,"slug":2811,"type":15},"MCP","mcp",{"name":2813,"slug":2814,"type":15},"Search","search",21777,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FNemoClaw","2026-07-20T06:00:01.461044",{"slug":2819,"name":2819,"fn":2820,"description":2821,"org":2822,"tags":2823,"stars":2830,"repoUrl":2831,"updatedAt":2832},"mcore-build-and-dependency","manage Megatron-LM development environments","Container-based dev environment setup and dependency management for Megatron-LM. Covers acquiring and launching the CI container, uv package management, and updating uv.lock.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2824,2827,2828],{"name":2825,"slug":2826,"type":15},"Containers","containers",{"name":2718,"slug":2719,"type":15},{"name":2829,"slug":891,"type":15},"Python",17049,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FMegatron-LM","2026-07-27T06:06:11.249662",{"slug":2834,"name":2834,"fn":2835,"description":2836,"org":2837,"tags":2838,"stars":2830,"repoUrl":2831,"updatedAt":2843},"mcore-bump-base-image","update NVIDIA PyTorch base images","Bump the NVIDIA PyTorch base image (`nvcr.io\u002Fnvidia\u002Fpytorch:YY.MM-py3`) used by Megatron-LM CI. Covers the two pin sites (GitHub CI in `docker\u002F.ngc_version.dev` and GitLab CI in `.gitlab\u002Fstages\u002F01.build.yml`), the post-bump CI loop (re-run functional tests, refresh golden values, mark broken tests), and the gotchas that bit PRs",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2839,2842],{"name":2840,"slug":2841,"type":15},"CI\u002FCD","ci-cd",{"name":2718,"slug":2719,"type":15},"2026-07-14T05:25:59.97109",{"slug":2845,"name":2845,"fn":2846,"description":2847,"org":2848,"tags":2849,"stars":2830,"repoUrl":2831,"updatedAt":2855},"mcore-cicd","manage CI\u002FCD pipelines for Megatron-LM","CI\u002FCD reference for Megatron-LM. Covers CI pipeline structure, PR scope labels, triggering internal GitLab CI (which force-pushes the current branch to a pull-request\u002FBRANCH ref — always dry-run and verify the destination first; never run against shared or protected branches), and CI failure investigation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2850,2851,2852],{"name":2840,"slug":2841,"type":15},{"name":2718,"slug":2719,"type":15},{"name":2853,"slug":2854,"type":15},"GitHub","github","2026-07-27T06:06:12.278222",{"slug":2857,"name":2857,"fn":2858,"description":2859,"org":2860,"tags":2861,"stars":2830,"repoUrl":2831,"updatedAt":2869},"mcore-create-issue","investigate CI failures and create issues","Investigate a failing GitHub Actions run or job and create a GitHub issue for the failure.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2862,2865,2866],{"name":2863,"slug":2864,"type":15},"Debugging","debugging",{"name":2853,"slug":2854,"type":15},{"name":2867,"slug":2868,"type":15},"Triage","triage","2026-07-14T05:25:57.442089",{"slug":2871,"name":2871,"fn":2872,"description":2873,"org":2874,"tags":2875,"stars":2830,"repoUrl":2831,"updatedAt":2882},"mcore-linting-and-formatting","lint and format Megatron-LM code","Linting and formatting for Megatron-LM. Covers running autoformat.sh, tools (ruff, black, isort, pylint, mypy), and code style rules.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2876,2879],{"name":2877,"slug":2878,"type":15},"Best Practices","best-practices",{"name":2880,"slug":2881,"type":15},"Code Analysis","code-analysis","2026-07-14T05:25:56.18433",{"slug":2884,"name":2884,"fn":2885,"description":2886,"org":2887,"tags":2888,"stars":2830,"repoUrl":2831,"updatedAt":2894},"mcore-migrate-gpt-to-hybrid","migrate Megatron-LM models to HybridModel","Migration guide for moving Megatron Core GPTModel checkpoints, model providers, training commands, and layer mappings to HybridModel.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2889,2890,2893],{"name":13,"slug":14,"type":15},{"name":2891,"slug":2892,"type":15},"Migration","migration",{"name":9,"slug":8,"type":15},"2026-07-17T06:07:11.777011",{"slug":2896,"name":2896,"fn":2897,"description":2898,"org":2899,"tags":2900,"stars":2830,"repoUrl":2831,"updatedAt":2905},"mcore-onboard-gb200-1node-tests","onboard functional tests for GB200","Onboard 1-node GitHub MR functional tests for GB200 from existing mr-scoped 2-node tests.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2901,2904],{"name":2902,"slug":2903,"type":15},"QA","qa",{"name":2748,"slug":2749,"type":15},"2026-07-14T05:25:53.673039",{"slug":2907,"name":2907,"fn":2908,"description":2909,"org":2910,"tags":2911,"stars":2830,"repoUrl":2831,"updatedAt":2914},"mcore-run-on-slurm","launch distributed training jobs on SLURM","How to launch distributed Megatron-LM training jobs on a SLURM cluster. Covers a minimal sbatch skeleton, environment-variable setup for torch.distributed.run, CUDA_DEVICE_MAX_CONNECTIONS rules across hardware and parallelism modes, container conventions, monitoring, and per-rank failure diagnosis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2912,2913],{"name":2718,"slug":2719,"type":15},{"name":2721,"slug":2722,"type":15},"2026-07-14T05:25:49.362534",{"slug":2916,"name":2916,"fn":2917,"description":2918,"org":2919,"tags":2920,"stars":2830,"repoUrl":2831,"updatedAt":2928},"mcore-split-pr","split pull requests to reduce review load","Split a PR into multiple PRs to reduce the number of required CODEOWNERS reviewer groups.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2921,2924,2925],{"name":2922,"slug":2923,"type":15},"Code Review","code-review",{"name":2853,"slug":2854,"type":15},{"name":2926,"slug":2927,"type":15},"Pull Requests","pull-requests","2026-07-14T05:26:01.226578",{"slug":2930,"name":2930,"fn":2931,"description":2932,"org":2933,"tags":2934,"stars":2830,"repoUrl":2831,"updatedAt":2937},"mcore-testing","run and manage Megatron-LM tests","Test system for Megatron-LM. Covers test layout, recipe YAML structure, adding and running unit and functional tests, golden values, marker filters, and CI parity.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2935,2936],{"name":2902,"slug":2903,"type":15},{"name":2748,"slug":2749,"type":15},"2026-07-14T05:25:54.928983",{"slug":2939,"name":2939,"fn":2940,"description":2941,"org":2942,"tags":2943,"stars":2830,"repoUrl":2831,"updatedAt":2946},"nightly-sync","manage nightly main-to-dev sync workflows","Domain knowledge for the nightly main-to-dev sync workflow. Covers merge strategy, CI architecture, failure investigation, and known issues.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2944,2945],{"name":2758,"slug":2759,"type":15},{"name":2840,"slug":2841,"type":15},"2026-07-30T05:29:03.275638",496]