[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-huggingface-transformers-to-mlx":3,"mdc-6z9zcc-key":38,"related-repo-huggingface-transformers-to-mlx":3506,"related-org-huggingface-transformers-to-mlx":3516},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":27,"repoUrl":28,"updatedAt":29,"license":30,"forks":31,"topics":32,"repo":33,"sourceUrl":36,"mdContent":37},"transformers-to-mlx","convert Transformers models to MLX format","Convert Hugging Face transformers language models to MLX format for Apple Silicon inference. Use when asked to convert a model from transformers to MLX, port a model to MLX, create an MLX implementation of a model, or test\u002Fvalidate an MLX model conversion. Triggers include mentions of \"convert to MLX\", \"MLX implementation\", \"port to MLX\", \"mlx-lm\", or requests involving both transformers and MLX frameworks.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"huggingface","Hugging Face","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fhuggingface.png",[12,15,18,21,24],{"name":9,"slug":13,"type":14},"hugging-face","tag",{"name":16,"slug":17,"type":14},"MLX","mlx",{"name":19,"slug":20,"type":14},"Migration","migration",{"name":22,"slug":23,"type":14},"AI Infrastructure","ai-infrastructure",{"name":25,"slug":26,"type":14},"Apple Silicon","apple-silicon",50,"https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftransformers-to-mlx","2026-04-17T05:02:32.560708",null,6,[],{"repoUrl":28,"stars":27,"forks":31,"topics":34,"description":35},[],"Agent Skill to help convert transformer LLMs to mlx-lm","https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftransformers-to-mlx\u002Ftree\u002FHEAD\u002Fskills\u002Ftransformers-to-mlx","---\nname: transformers-to-mlx\ndescription: Convert Hugging Face transformers language models to MLX format for Apple Silicon inference. Use when asked to convert a model from transformers to MLX, port a model to MLX, create an MLX implementation of a model, or test\u002Fvalidate an MLX model conversion. Triggers include mentions of \"convert to MLX\", \"MLX implementation\", \"port to MLX\", \"mlx-lm\", or requests involving both transformers and MLX frameworks.\n---\n\n# Transformers to MLX Model Conversion\n\n## MLX-LM repo\n\nIn all circumstances refer to the following repo and branch unless overridden by the user.\n\nGitHub repo: https:\u002F\u002Fgithub.com\u002Fml-explore\u002Fmlx-lm\nBranch: main\n\n## Workflow\n\n### Phase 0: Set up working environment\n\n1. Verify `uv` is installed, use it to create a virtual environment. Always use `uv pip install` (not plain `pip install`) to install packages in the environment — this avoids resolver conflicts and is significantly faster.\n2. git clone the mlx-repo from GitHub. ALWAYS USE the repo and branch specified in the `MLX-LM repo` section above.\n3. Install mlx-lm in editable mode in the environment\n4. Install transformers locally too, to ensure we have the latest version of the code: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftransformers. You can clone with `--depth 1` because it has many refs. Also install `accelerate` for `device_map` support.\n5. Make sure `huggingface_hub` is installed too, to use the `hf` CLI for model download. It comes with the previous libraries as a dependency, so you may need to wait for it if you run stuff in parallel.\n6. **Use the venv binaries directly** rather than activating: `.venv\u002Fbin\u002Fpython script.py`, `.venv\u002Fbin\u002Fhf download ...`, `.venv\u002Fbin\u002Fmlx_lm.generate ...`. This avoids `source .venv\u002Fbin\u002Factivate` (which executes a shell script that triggers permission prompts and doesn't propagate to background commands). The CLI entry points and Python interpreter resolve their own environment from their installed location, so no activation is needed.\n7. Discover and select target models, if the user did not provide a list of models to consider. Search the Hub for all models using the target architecture to convert, identify canonical variants across sizes, and confirm targets with the user. Use [Model Discovery and Config Analysis](references\u002Fmodel-discovery-and-config-analysis.md) to learn how.\n8. Download a local copy of the model(s) from Hugging Face using the `hf` CLI. Save the models to local folders inside the project using `--local-dir`, this will allow you to read contents easily.\n\n### Phase 1: Analyze Source Models\n\n1. Read `config.json` for `model_type` and architecture details\n2. Verify native transformers support in `...\u002Ftransformers\u002Fmodels\u002F\u003Cmodel_type>\u002F`, residing in the virtual environment\n3. Read the transformers `modeling_\u003Cmodel_type>.py` implementation carefully\n4. If the model is a vision-language model, convert the text-generation backbone, but inform the user in your report. Do not proceed if the model is not a language model at all (and explain why).\n5. Analyze config variance across target models and cross-reference with conditional code paths in transformers. See [Model Discovery and Config Analysis](references\u002Fmodel-discovery-and-config-analysis.md).\n6. Pay special attention to unique or novel architecture variants the model may introduce. Highlight them in your report, and be vigilant for subtle bugs that may be difficult to detect in tests. Do some ad-hoc testing for them if necessary.\n\n**Some key config fields to check:**\n- `rope_parameters`, `position_embedding_type` - See section on RoPE later\n- `layer_types` - For hybrid models (e.g., mamba + attention)\n- Scalar multipliers: `embedding_multiplier`, `attention_multiplier`, `residual_multiplier`, `logits_scaling`\n- `torch_dtype` or `dtype` or `text_config.dtype` — The model's intended runtime precision. Some models (especially VL wrappers) nest this inside `text_config` rather than exposing it as top-level `torch_dtype` (or `dtype`). If the converter can't find it, it skips the dtype casting step entirely, leaving stray float32 weights in the converted model. Verify which field the model uses and confirm `mlx_lm.convert` detects it. Use [`scripts\u002Fresolve_dtype.py`](scripts\u002Fresolve_dtype.py) to determine the expected dtype from the config (or the safetensors header as a fallback) and record the source — this becomes the basis for the dtype tests in Phase 4 and the manifest in Phase 8.\n\n### Phase 2: Find Reference MLX Models\n\nExamples from `mlx-lm\u002Fmlx_lm\u002Fmodels\u002F`:\n- Standard attention: `llama.py`\n- MLA (Multi-head Latent Attention): `deepseek_v3.py` — see [MLA Reference](references\u002Fmla.md)\n- MoE (Mixture of Experts): `deepseek_v3.py`, `mixtral.py`\n- GatedDeltaNet (linear attention): `qwen3_next.py`, `qwen3_5.py`, `olmo_hybrid.py`\n- Hybrid (Mamba + Attention): `nemotron_h.py`\n- Sliding window: `mistral.py`\n\nStart from the **closest existing model** rather than building from scratch — see [Code Selection and Simplification](references\u002Fcode-selection-simplification.md).\n\n### Phase 3: Create MLX Model\n\nCreate `mlx-lm\u002Fmlx_lm\u002Fmodels\u002F\u003Cmodel_type>.py` (self-contained single file):\n\n```python\n@dataclass\nclass ModelArgs:\n    # Config params with defaults\n    position_embedding_type: str = \"rope\"  # Check for \"nope\"!\n\nclass Attention(nn.Module): ...\nclass MLP(nn.Module): ...\nclass DecoderLayer(nn.Module): ...\nclass LanguageModel(nn.Module): ...\n\nclass Model(nn.Module):\n    def sanitize(self, weights): ...  # Convert weight names\u002Fshapes\n    def shard(self, group): ...       # Distributed inference\n\n    @property\n    def cast_predicate(self):         # Keep specific weights in float32\n        return lambda _, m, _: isinstance(m, MoEGate) and \"bias\" in m\n```\n\n**sanitize() patterns** — see [Weight Sanitization](references\u002Fweight-sanitization.md) for detailed guidance on how to make weights loading compatible with both transformers and MLX-converted weights. Here are some common examples:\n- Stack experts: `experts.{N}.gate_proj` → `switch_mlp.gate_proj` shape `(num_experts, ...)`\n- Remove: `rotary_emb.inv_freq`, extra layers (e.g., MTP heads)\n- 3D MoE weights: May need reshape from `(num_experts, hidden, intermediate)`\n- Shared\u002Ftied weights: Replicate from source blocks to all tied layers\n- Different model sizes may use different weight formats — detect and branch\n- MLA `kv_b_proj` splitting: dequantize → reshape by heads → split into `embed_q`\u002F`unembed_out` — see [MLA Reference](references\u002Fmla.md)\n\n**RoPE:**\n- `rope_interleave=True` → `traditional=True` in MLX\n- `position_embedding_type=\"nope\"` → Do NOT apply RoPE at all\n- `rope_parameters` nested dict: some models store RoPE config as `{\"rope_theta\": ..., \"rope_type\": ...}` instead of top-level fields — extract in `__post_init__`. See [MLA Reference](references\u002Fmla.md).\n- **Null handling:** `rope_parameters` can be `null`, absent, or a dict with `null` values (e.g., `{\"rope_theta\": null}`). Use `.get()` with defaults and test for `None` explicitly in `__post_init__`. Verify the different paths the transformers codebase uses to deal with all variants present in the config files.\n\n**Conversion and quantization:** Use `mlx_lm.convert` to validate sanitize and produce quantized models — see [Conversion and Quantization](references\u002Fconversion-and-quantization.md). Only do this after testing transformers weights, or if the model is [too large to run unquantized](references\u002Flarge-models.md).\n\n### Phase 4: Testing\n\n**Note:** For very large models that can't fit in fp16 \u002F bf16, the standard comparison workflow isn't feasible. Use the quantize-first strategy instead — see [Large Models](references\u002Flarge-models.md). For models that require multiple machines, see [Distributed Inference](references\u002Fdistributed-inference.md).\n\n**Test 1: Generation comparison** (do first - easier to spot garbage output)\n\nPre-validation: run `mlx_lm.generate` with a prompt and verify the output is coherent. This will use the default sampling method taken from the model's `generation_config.json`.\n\nExample:\n\n```bash\nmlx_lm.generate --model zai-org\u002FGLM-4.7-Flash --prompt \"Who are you?\"\n```\n\nUse an appropriate prompt depending on the model capabilities (for example, is it a base model or does it use a chat template). If it's a base model, use an inviting prefix sentence to be continued, not just a single word. If it's a conversational model, ask an interesting question. If the model card mentions specific capabilities (for example, the model was trained for code generation), use a relevant topic.\n\nIf generated text is garbage, use the layer-by-layer tests to try to identify implementation mismatches.\n\nFor more controlled generation (greedy mode) and comparison with transformers, use the [compare_generation.py script](scripts\u002Fcompare_generation.py).\n\nWhen using this script, make a judgement call about whether the generations are semantically similar. They will rarely match exactly, because of differences in the compute device, kernel implementations, and other reasons.\n\nDo not declare success if generation is not reasonable - check for numerical and implementation differences and iterate until you get a good quality output, using the transformers one as the quality reference.\n\n**Test 2: Output dtype**\n\nResolve the expected dtype with [`scripts\u002Fresolve_dtype.py`](scripts\u002Fresolve_dtype.py) — it tries config fields first, then falls back to the most common dtype in the safetensors header. Always record the **source** in the conversion report (e.g., `config.dtype`, `safetensors header (BF16=292\u002F293, F32=1\u002F293)`) so reviewers know the basis for the check.\n\n```bash\npython scripts\u002Fresolve_dtype.py \u002Fpath\u002Fto\u002Fmodel\n# dtype: bfloat16\n# source: safetensors header (BF16=292\u002F293, F32=1\u002F293)\n```\n\nThen run a forward pass and verify the output dtype matches with [`scripts\u002Fcheck_dtype.py`](scripts\u002Fcheck_dtype.py):\n\n```bash\npython scripts\u002Fcheck_dtype.py \u002Fpath\u002Fto\u002Fmodel bfloat16\n# {\"output_dtype\": \"mlx.core.bfloat16\", \"expected_dtype\": \"bfloat16\", \"match\": true}\n```\n\nExits non-zero with a `FAIL:` message on stderr if the runtime dtype doesn't match.\n\nA single float32 weight (e.g., a norm weight promoted by `v + 1.0` in sanitize) will silently propagate float32 through the entire model via dtype promotion, causing up to 5x slower inference on Apple Silicon.\n\nThe same script works after quantization — the runtime output dtype is unaffected by weight quantization (only the storage of matmul weights changes). Re-run it on the quantized model in Phase 5 to confirm.\n\n**Test 3: Numerical comparison**\n\nUse the [compare_predictions script](scripts\u002Fcompare_predictions.py) to compare logits from a single forward pass and extract some statistics. **Use a prompt of at least 100 tokens** (e.g., a paragraph of text) — short prompts (5-10 tokens) can hide positional encoding bugs because RoPE error grows with position. A wrong `rope_theta` may pass top-1 comparison at 5 tokens but diverge significantly at 100+.\n\nPoint out any major differences you observe. A max logits difference in ~0.5-1.0 (or larger, for models with many layers where errors accumulate) is usual between CPU transformers and Metal MLX. If you see suspicious values, run a layer-by-layer comparison even if the generation test seemed to pass.\n\nIf you run a top-5 or top-10 comparison, include the top-10 list in the GitHub PR you'll open when you're done (unless they are identical).\n\n**Test 4: Layer-by-layer** (if predictions don't match — and as a routine sanity check)\n\nUse [`scripts\u002Fcompare_layers.py`](scripts\u002Fcompare_layers.py). One forward pass per framework captures the output of every decoder layer, the post-final-norm state, and the logits, then compares them side-by-side with absolute and relative diffs.\n\n```bash\npython scripts\u002Fcompare_layers.py \u002Fpath\u002Fto\u002Fmodel \"\u003Cprompt>\"\n```\n\nRead the table from the bottom up:\n- **`logits` row**: the most important. If logits agree, the implementation is end-to-end correct even if hidden states drift along the way.\n- **`post-norm` row**: hidden state after the final RMSNorm. Should be small relative diff.\n- **`layer N` rows**: per-layer outputs. Look for the first sharp jump in `rel diff` — that's where divergence is introduced.\n\nThe script is architecture-agnostic: it wraps each entry in `model.[language_model.]model.layers` with a thin capture proxy (preserving the layer's class so `isinstance` checks in hybrid models keep working). It does NOT compare the embedding output, since pre-layer transforms (multipliers, per-layer scaling) differ across architectures and the embedding row is rarely informative.\n\nHealthy implementations show \u003C1% relative diff at every position. A pattern where mid-layer divergence is relatively large but the `logits` row is small usually means the divergence cancels out at the head — annoying but not a correctness bug.\n\nInclude results in the report.\n\n**Test 5: Long sequence degradation**\n- Run a prompt that generates a long sequence, such as \"Write an HTML and JavaScript page implementing space invaders\", or something appropriate depending on model capabilities. You can use `mlx_lm.generate -m 16384` to allow up to 16K tokens to be generated.\n- Observe if the output turns to garbage after a while. This usually signals mistakes in RoPE.\n\n### Phase 5: Quants\n\nUse `mlx_lm.convert` to quantize the model. First, run `scripts\u002Fresolve_dtype.py` on the source checkpoint to confirm the storage dtype. If the model already deals with native quantized weights, skip this phase and inform the user. If it doesn't, try a simple quantization to 4 bits and run generation tests to verify whether results are coherent.\n\nAfter quantization, re-run `scripts\u002Fcheck_dtype.py` against the quantized model with the same expected dtype as the unquantized one. The runtime dtype shouldn't change with quantization — only the storage of matmul weights does. A mismatch here usually means a stray float32 weight survived the quantize step.\n\n### Phase 6: MLX Tests\n\nWhen appropriate, create tests for the model you just converted, following the structure and criteria used in the mlx-lm repo.\n\n### Phase 7: Report\n\nAlways include the following in your output report:\n\n- The list of models and variants selected for conversion, and tested.\n- Notable architecture decisions and novelties introduced.\n- Non-trivial implementation details that are worth noting.\n- The expected output dtype per variant and where it came from (e.g., `config.dtype` vs `safetensors header`). Use the results from `scripts\u002Fresolve_dtype.py`.\n- At least a meaningful generation example of ~200 tokens or more.\n- A summary of the numerical differences you found.\n- Per-layer comparison results.\n- Errors that you couldn't solve and pointers to analyze them.\n- If changes to `mlx-lm` were needed, explain in detail what they were, and provide links to the equivalent transformers implementation. This should be rare (bugs or missing features).\n- Reflect upon learnings that we could consider to incorporate back into the Skill for future reference. Trivial discoveries that are easy for you to replicate in future sessions are not necessary; focus on initial error cases that required some iteration.\n\nPlease, output this report in your conversation, and also create a markdown file for reference.\n\n### Phase 8: GitHub PR\n\n- **Create a feature branch** for your changes (e.g., `add-\u003Cmodel_type>`) — never commit directly to the main working branch. See [GitHub PR Workflow](references\u002Fgithub-pr-workflow.md) for details.\n- ALWAYS ask for confirmation before submitting a PR. The user may have questions or want to run additional tests, or include more variants.\n- Make sure the `gh` cli is authenticated.\n- Ensure you use the github repo specified at the beginning of this document (even if it's not the canonical one).\n- In the PR, include:\n    * A suitable title. But please, prefix it with `[transformers-to-mlx skill]` for disclosure.\n    * Also disclose that all tests and results were obtained with the skill.\n    * Your summary report in the PR.\n    * In addition to the summary report, include test commands and their outputs. For example, if you ran a long sequence generation with `mlx_lm.generate`, include the command and the full output. Also include the source code you used to compare numerical results, when appropriate. Same thing for the dtype tests and everything else.\n- **After the mlx-lm PR is created**, generate a test manifest and open a PR to the test harness repo. See [GitHub PR Workflow — Test Manifest](references\u002Fgithub-pr-workflow.md#test-manifest) for the format and procedure.\n\n## Critical Lessons\n\n\u003C!-- Note for the skill author: we may want to extract these to separate files depending\non the model type or type of pitfall -->\n\n### Garbage Output Despite Loading\n\nModel loading and generating without crashes does NOT mean it works. Always verify:\n1. Output is coherent text (not \"lylyly Usesfinite\")\n2. Top predictions match transformers\n3. Logits statistics are in same ballpark\n\n### Position Embeddings\n\nThis is a usual source of problems.\n\nCheck `position_embedding_type` in config:\n- `\"rope\"` - Apply rotary position embeddings\n- `\"nope\"` - Do NOT apply any position embeddings (common in hybrid models)\n\n**RoPE config can vary across variants of the same architecture.** For example, OLMo-Hybrid base\u002FSFT use `rope_theta=10000` while the DPO variant uses NoPE (no positional embeddings at all, via `rope_parameters: {rope_theta: null}`). Always diff RoPE-related config fields across ALL target checkpoints — base, SFT, DPO, and different sizes — before implementation.\n\n```python\n# Conditional RoPE\nuse_rope = getattr(args, 'position_embedding_type', 'rope') != 'nope'\nif use_rope:\n    self.rope = initialize_rope(...)\nelse:\n    self.rope = None\n```\n\n### Scalar Multipliers\n\nSome models (Granite, etc.) use scaling factors that MUST be applied exactly:\n- `embedding_multiplier` - Scale embeddings before first layer\n- `attention_multiplier` - Replace `1\u002Fsqrt(head_dim)` in attention\n- `residual_multiplier` - Scale residual connections\n- `logits_scaling` - Scale final logits\n\nMissing any of these causes completely wrong predictions.\n\n### Weight Key Names: Checkpoint vs Transformers Attributes\n\nWhen building the MLX model, use the **transformers model attribute names** for your module definitions — not the raw checkpoint key names. These often differ: the checkpoint may use names like `Wqkv` or `gate_up_proj` while the transformers model class uses `q_proj`, `k_proj`, `gate_proj`, `up_proj`, etc.\n\nThe `sanitize()` method is where you remap checkpoint names to match the MLX model structure (which mirrors transformers attribute names). If the MLX model uses checkpoint names instead, weight loading will silently fail or require a convoluted sanitize.\n\nTo find the authoritative mapping, check `conversion_mapping.py` in the transformers model directory — see [Weight Sanitization](references\u002Fweight-sanitization.md#transformers-conversion_mappingpy).\n\n### Numerical Precision\n\nSome models define custom operations (e.g., gated RMSNorm, specialized activations) that are sensitive to precision. If the transformers code explicitly casts to float32, the MLX implementation must do the same:\n\n```python\nclass GatedRMSNorm(nn.Module):\n    def __call__(self, x, residual):\n        original_dtype = x.dtype\n        x = x.astype(mx.float32)\n        norm = mx.rsqrt(mx.mean(x * x, axis=-1, keepdims=True) + self.eps)\n        x = (x * norm).astype(original_dtype) * self.weight\n        return x\n```\n\nMissing a float32 cast may not cause obvious errors in short sequences but leads to degraded output quality (e.g., repetition) in longer generations. When reading the transformers source, watch for `.float()`, `.to(torch.float32)`, or similar casts — they are there for a reason.\n\nHowever, do check for MLX ops that may be internally upcasting in their implementation.\n\n### Dtype Contamination After Conversion\n\nA single float32 parameter in a non-quantized layer (norms, biases, gating weights) can silently poison the entire forward pass through MLX's dtype promotion rules. This causes no correctness errors — the model generates coherent text — but inference speed degrades dramatically (up to 5x slower) because all quantized matmuls run in float32 mode instead of bfloat16.\n\nCommon causes:\n- Sanitize operations like `v + 1.0` promote bfloat16 norm weights to float32 (Python's `1.0` is float64\u002Ffloat32)\n- Parameters initialized with `mx.ones()` or `mx.zeros()` default to float32 and persist if the checkpoint doesn't include them\n\nThe contamination path: float32 norm weight → `rms_norm` outputs float32 → quantized matmul receives float32 input → outputs float32 → residual addition promotes hidden state to float32 → every subsequent layer inherits float32.\n\n**Always verify output dtype after conversion and after quantization** (see Phase 5).\n\n### Hybrid and SSM Architectures\n\nFor models mixing layer types (e.g., Mamba + Attention) — see [Hybrid and SSM Models](references\u002Fhybrid-and-ssm-models.md) for detailed guidance:\n1. Check `layer_types` config for per-layer architecture\n2. Each layer may have MULTIPLE blocks (not just one):\n   - Block 1: Mamba OR Attention\n   - Block 2: MoE + shared_mlp (for ALL layers)\n3. Both Mamba and Attention layers often share the same MoE\u002FMLP block\n4. `make_cache()` must return the right cache type per layer (e.g., `CacheList(KVCache(), ArraysCache())` for hybrid layers)\n5. CUDA-only dependencies (`mamba_ssm`, `causal_conv1d`) won't install on macOS — transformers uses fallback implementations\n\n### 3D MoE Weights\n\nExpert weights may be stored as 3D tensors:\n- Shape: `(num_experts, hidden_size, intermediate_size)`\n- Need proper reshape in sanitize()\n- SwitchGLU expects specific shapes\n\n### Debugging Checklist\n\nWhen predictions don't match:\n1. **Embeddings** - Should match exactly (mean, std)\n2. **After embedding scaling** - Check multiplier applied\n3. **Layer-by-layer** - Find first divergence point\n4. **Check config fields** - position_embedding_type, multipliers\n5. **Layer structure** - All blocks applied? (MoE often missed)\n6. **Attention scaling** - Verify the scale factor matches transformers exactly (some models use non-standard formulas like `(head_dim \u002F\u002F 2) ** -0.5`)\n7. **Precision casts** - Check for `.float()` \u002F `.to(torch.float32)` in transformers code, especially in norms and activations (see Numerical Precision above)\n8. **Repetitive output** - If the model generates repetitive or looping text, this may indicate a shape or transpose error in the attention or linear attention layers (e.g., head dimensions swapped, wrong reshape order), not a sampling issue\n\n### Large Models\n\nModels that consume a significant fraction of available RAM require special handling:\n- Set `sudo sysctl iogpu.wired_limit_mb=\u003Cvalue>` to increase the wired memory limit\n- Set `MLX_METAL_FAST_SYNCH=1` for faster GPU synchronization\n- `mlx_lm.generate` may OOM due to `mx.async_eval` double-buffering — use a manual generation loop\n- Very large MoE models suffer memory system thrashing even with selective `gather_qmm`\n- For models too large for one machine, use tensor parallelism via `mlx.launch`\n\nSee [Large Models](references\u002Flarge-models.md) and [Distributed Inference](references\u002Fdistributed-inference.md) for detailed guidance.\n\n### Changes to Shared mlx-lm Code\n\nSome conversions require modifications to shared mlx-lm infrastructure (`ssm.py`, Metal kernels, `cache.py`), not just a new model file. These changes affect all models using the same code. See [Common Infrastructure Changes](references\u002Fmlx-common-infrastructure.md) for guidance on localizing fixes and avoiding regressions.\n\n## General Guidance and Common Pitfalls\n\n- **Minimize code comments.** Do not add comments unless the code would be legitimately confusing without one (which should be very rare). Comments must be reviewed just like code, so every unnecessary comment increases review burden. The PR description and tests provide all the context a reviewer needs.\n- Greedy MLX decoding: `sampler=lambda x: mx.argmax(x, axis=-1)`, not `temperature=0`\n- Don't override `eos_token_id` - use generation_config\n- Use separate tokenizers for transformers\u002FMLX tests\n- `apply_chat_template`: use `tokenize=True, return_tensors=\"pt\"` directly\n- MPS limitations: use CPU for comparison tests\n- Hybrid models: RoPE is often disabled (`position_embedding_type: \"nope\"`)\n- Every layer may need MoE block, not just attention layers\n\n## Output\n\nUpon success, please save the mlx implementation in the appropriate place inside `mlx-lm\u002Fmlx_lm\u002Fmodels`, and generate a summary report with the results from the tests.\n\nOn failure (only declare when you run out of ideas after multiple iterations), please report your analysis as well as the results from tests you run.\n\n## List of scripts\n\nBundled scripts in `scripts\u002F` (relative to this skill):\n\n| Script | Purpose |\n|--------|---------|\n| `resolve_dtype.py` | Resolve expected runtime dtype from config or safetensors header (Phase 1, Phase 4 \u002F Test 2, Phase 5) |\n| `check_dtype.py` | Verify a model's forward-pass output dtype matches an expected value (Phase 4 \u002F Test 2, Phase 5) |\n| `compare_generation.py` | Full text generation comparison (Phase 4 \u002F Test 1) |\n| `compare_predictions.py` | Forward pass comparison with tolerance analysis and top-k overlap (Phase 4 \u002F Test 3) |\n| `compare_layers.py` | Per-layer hidden state + logits comparison (Phase 4 \u002F Test 4) |\n| `debug_transformers.py` | Analyze transformers model (embeddings, logits, top predictions) |\n| `debug_mlx.py` | Analyze MLX model (same output format as `debug_transformers.py`) |\n\n**Recommended order for the comparison flow:**\n1. `compare_generation.py` — quick sanity check (is output coherent?)\n2. `compare_predictions.py` — numerical analysis (do predictions match?)\n3. `compare_layers.py` — find divergence (where does it break?)\n\nUsage:\n```bash\n# Resolve and verify dtype\npython scripts\u002Fresolve_dtype.py \u002Fpath\u002Fto\u002Fmodel\npython scripts\u002Fcheck_dtype.py \u002Fpath\u002Fto\u002Fmodel bfloat16\n\n# Full generation comparison\npython scripts\u002Fcompare_generation.py \u002Fpath\u002Fto\u002Fmodel --message \"What is 2+2?\"\n\n# Numerical comparison with tolerance analysis\npython scripts\u002Fcompare_predictions.py \u002Fpath\u002Fto\u002Fmodel \"The quick brown fox...\"\n\n# Per-layer + logits comparison\npython scripts\u002Fcompare_layers.py \u002Fpath\u002Fto\u002Fmodel \"The quick brown fox...\"\n```\n\n## References\n\n| Reference | Purpose |\n|-----------|---------|\n| [Model Discovery and Config Analysis](references\u002Fmodel-discovery-and-config-analysis.md) | Hub search, config diffing, weight format detection |\n| [Weight Sanitization](references\u002Fweight-sanitization.md) | sanitize() patterns, idempotency, shared weights, ordering |\n| [Conversion and Quantization](references\u002Fconversion-and-quantization.md) | mlx_lm.convert usage, quantize-first workflow |\n| [MLA](references\u002Fmla.md) | Multi-head Latent Attention, kv_b_proj splitting, rope_parameters |\n| [Hybrid and SSM Models](references\u002Fhybrid-and-ssm-models.md) | Cache construction, layer dispatch, CUDA-only deps |\n| [Large Models](references\u002Flarge-models.md) | Wired memory, OOM workarounds, MoE memory pressure |\n| [Distributed Inference](references\u002Fdistributed-inference.md) | mlx.launch, hostfiles, rsync, tensor parallelism |\n| [Common Infrastructure Changes](references\u002Fmlx-common-infrastructure.md) | Safe modifications to shared mlx-lm code |\n| [GitHub PR Workflow](references\u002Fgithub-pr-workflow.md) | Feature branches, targeting correct repo, `gh pr edit` tips |\n| [Code Simplification](references\u002Fcode-selection-simplification.md) | Dead branch removal, reference model patterns |\n\n\u003C!-- Missing: training -->\n",{"data":39,"body":40},{"name":4,"description":6},{"type":41,"children":42},"root",[43,52,59,65,80,86,93,278,284,354,362,494,500,513,610,629,635,648,812,829,925,933,1059,1091,1097,1121,1131,1151,1156,1205,1210,1215,1227,1232,1237,1245,1282,1323,1338,1375,1388,1401,1406,1414,1442,1447,1452,1462,1478,1514,1519,1577,1598,1610,1615,1623,1644,1650,1668,1680,1686,1691,1697,1702,1784,1789,1795,1908,1914,1920,1925,1943,1949,1954,1966,1991,2017,2072,2078,2083,2134,2139,2145,2201,2214,2233,2239,2244,2307,2327,2332,2338,2343,2348,2391,2404,2414,2420,2433,2509,2515,2520,2544,2550,2555,2660,2665,2670,2738,2755,2761,2790,2796,2887,2893,2906,2911,2917,2930,3083,3091,3124,3129,3313,3319,3500],{"type":44,"tag":45,"props":46,"children":48},"element","h1",{"id":47},"transformers-to-mlx-model-conversion",[49],{"type":50,"value":51},"text","Transformers to MLX Model Conversion",{"type":44,"tag":53,"props":54,"children":56},"h2",{"id":55},"mlx-lm-repo",[57],{"type":50,"value":58},"MLX-LM repo",{"type":44,"tag":60,"props":61,"children":62},"p",{},[63],{"type":50,"value":64},"In all circumstances refer to the following repo and branch unless overridden by the user.",{"type":44,"tag":60,"props":66,"children":67},{},[68,70,78],{"type":50,"value":69},"GitHub repo: ",{"type":44,"tag":71,"props":72,"children":76},"a",{"href":73,"rel":74},"https:\u002F\u002Fgithub.com\u002Fml-explore\u002Fmlx-lm",[75],"nofollow",[77],{"type":50,"value":73},{"type":50,"value":79},"\nBranch: main",{"type":44,"tag":53,"props":81,"children":83},{"id":82},"workflow",[84],{"type":50,"value":85},"Workflow",{"type":44,"tag":87,"props":88,"children":90},"h3",{"id":89},"phase-0-set-up-working-environment",[91],{"type":50,"value":92},"Phase 0: Set up working environment",{"type":44,"tag":94,"props":95,"children":96},"ol",{},[97,128,140,145,182,203,245,258],{"type":44,"tag":98,"props":99,"children":100},"li",{},[101,103,110,112,118,120,126],{"type":50,"value":102},"Verify ",{"type":44,"tag":104,"props":105,"children":107},"code",{"className":106},[],[108],{"type":50,"value":109},"uv",{"type":50,"value":111}," is installed, use it to create a virtual environment. Always use ",{"type":44,"tag":104,"props":113,"children":115},{"className":114},[],[116],{"type":50,"value":117},"uv pip install",{"type":50,"value":119}," (not plain ",{"type":44,"tag":104,"props":121,"children":123},{"className":122},[],[124],{"type":50,"value":125},"pip install",{"type":50,"value":127},") to install packages in the environment — this avoids resolver conflicts and is significantly faster.",{"type":44,"tag":98,"props":129,"children":130},{},[131,133,138],{"type":50,"value":132},"git clone the mlx-repo from GitHub. ALWAYS USE the repo and branch specified in the ",{"type":44,"tag":104,"props":134,"children":136},{"className":135},[],[137],{"type":50,"value":58},{"type":50,"value":139}," section above.",{"type":44,"tag":98,"props":141,"children":142},{},[143],{"type":50,"value":144},"Install mlx-lm in editable mode in the environment",{"type":44,"tag":98,"props":146,"children":147},{},[148,150,156,158,164,166,172,174,180],{"type":50,"value":149},"Install transformers locally too, to ensure we have the latest version of the code: ",{"type":44,"tag":71,"props":151,"children":154},{"href":152,"rel":153},"https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftransformers",[75],[155],{"type":50,"value":152},{"type":50,"value":157},". You can clone with ",{"type":44,"tag":104,"props":159,"children":161},{"className":160},[],[162],{"type":50,"value":163},"--depth 1",{"type":50,"value":165}," because it has many refs. Also install ",{"type":44,"tag":104,"props":167,"children":169},{"className":168},[],[170],{"type":50,"value":171},"accelerate",{"type":50,"value":173}," for ",{"type":44,"tag":104,"props":175,"children":177},{"className":176},[],[178],{"type":50,"value":179},"device_map",{"type":50,"value":181}," support.",{"type":44,"tag":98,"props":183,"children":184},{},[185,187,193,195,201],{"type":50,"value":186},"Make sure ",{"type":44,"tag":104,"props":188,"children":190},{"className":189},[],[191],{"type":50,"value":192},"huggingface_hub",{"type":50,"value":194}," is installed too, to use the ",{"type":44,"tag":104,"props":196,"children":198},{"className":197},[],[199],{"type":50,"value":200},"hf",{"type":50,"value":202}," CLI for model download. It comes with the previous libraries as a dependency, so you may need to wait for it if you run stuff in parallel.",{"type":44,"tag":98,"props":204,"children":205},{},[206,212,214,220,222,228,229,235,237,243],{"type":44,"tag":207,"props":208,"children":209},"strong",{},[210],{"type":50,"value":211},"Use the venv binaries directly",{"type":50,"value":213}," rather than activating: ",{"type":44,"tag":104,"props":215,"children":217},{"className":216},[],[218],{"type":50,"value":219},".venv\u002Fbin\u002Fpython script.py",{"type":50,"value":221},", ",{"type":44,"tag":104,"props":223,"children":225},{"className":224},[],[226],{"type":50,"value":227},".venv\u002Fbin\u002Fhf download ...",{"type":50,"value":221},{"type":44,"tag":104,"props":230,"children":232},{"className":231},[],[233],{"type":50,"value":234},".venv\u002Fbin\u002Fmlx_lm.generate ...",{"type":50,"value":236},". This avoids ",{"type":44,"tag":104,"props":238,"children":240},{"className":239},[],[241],{"type":50,"value":242},"source .venv\u002Fbin\u002Factivate",{"type":50,"value":244}," (which executes a shell script that triggers permission prompts and doesn't propagate to background commands). The CLI entry points and Python interpreter resolve their own environment from their installed location, so no activation is needed.",{"type":44,"tag":98,"props":246,"children":247},{},[248,250,256],{"type":50,"value":249},"Discover and select target models, if the user did not provide a list of models to consider. Search the Hub for all models using the target architecture to convert, identify canonical variants across sizes, and confirm targets with the user. Use ",{"type":44,"tag":71,"props":251,"children":253},{"href":252},"references\u002Fmodel-discovery-and-config-analysis.md",[254],{"type":50,"value":255},"Model Discovery and Config Analysis",{"type":50,"value":257}," to learn how.",{"type":44,"tag":98,"props":259,"children":260},{},[261,263,268,270,276],{"type":50,"value":262},"Download a local copy of the model(s) from Hugging Face using the ",{"type":44,"tag":104,"props":264,"children":266},{"className":265},[],[267],{"type":50,"value":200},{"type":50,"value":269}," CLI. Save the models to local folders inside the project using ",{"type":44,"tag":104,"props":271,"children":273},{"className":272},[],[274],{"type":50,"value":275},"--local-dir",{"type":50,"value":277},", this will allow you to read contents easily.",{"type":44,"tag":87,"props":279,"children":281},{"id":280},"phase-1-analyze-source-models",[282],{"type":50,"value":283},"Phase 1: Analyze Source Models",{"type":44,"tag":94,"props":285,"children":286},{},[287,307,320,333,338,349],{"type":44,"tag":98,"props":288,"children":289},{},[290,292,298,299,305],{"type":50,"value":291},"Read ",{"type":44,"tag":104,"props":293,"children":295},{"className":294},[],[296],{"type":50,"value":297},"config.json",{"type":50,"value":173},{"type":44,"tag":104,"props":300,"children":302},{"className":301},[],[303],{"type":50,"value":304},"model_type",{"type":50,"value":306}," and architecture details",{"type":44,"tag":98,"props":308,"children":309},{},[310,312,318],{"type":50,"value":311},"Verify native transformers support in ",{"type":44,"tag":104,"props":313,"children":315},{"className":314},[],[316],{"type":50,"value":317},"...\u002Ftransformers\u002Fmodels\u002F\u003Cmodel_type>\u002F",{"type":50,"value":319},", residing in the virtual environment",{"type":44,"tag":98,"props":321,"children":322},{},[323,325,331],{"type":50,"value":324},"Read the transformers ",{"type":44,"tag":104,"props":326,"children":328},{"className":327},[],[329],{"type":50,"value":330},"modeling_\u003Cmodel_type>.py",{"type":50,"value":332}," implementation carefully",{"type":44,"tag":98,"props":334,"children":335},{},[336],{"type":50,"value":337},"If the model is a vision-language model, convert the text-generation backbone, but inform the user in your report. Do not proceed if the model is not a language model at all (and explain why).",{"type":44,"tag":98,"props":339,"children":340},{},[341,343,347],{"type":50,"value":342},"Analyze config variance across target models and cross-reference with conditional code paths in transformers. See ",{"type":44,"tag":71,"props":344,"children":345},{"href":252},[346],{"type":50,"value":255},{"type":50,"value":348},".",{"type":44,"tag":98,"props":350,"children":351},{},[352],{"type":50,"value":353},"Pay special attention to unique or novel architecture variants the model may introduce. Highlight them in your report, and be vigilant for subtle bugs that may be difficult to detect in tests. Do some ad-hoc testing for them if necessary.",{"type":44,"tag":60,"props":355,"children":356},{},[357],{"type":44,"tag":207,"props":358,"children":359},{},[360],{"type":50,"value":361},"Some key config fields to check:",{"type":44,"tag":363,"props":364,"children":365},"ul",{},[366,384,395,427],{"type":44,"tag":98,"props":367,"children":368},{},[369,375,376,382],{"type":44,"tag":104,"props":370,"children":372},{"className":371},[],[373],{"type":50,"value":374},"rope_parameters",{"type":50,"value":221},{"type":44,"tag":104,"props":377,"children":379},{"className":378},[],[380],{"type":50,"value":381},"position_embedding_type",{"type":50,"value":383}," - See section on RoPE later",{"type":44,"tag":98,"props":385,"children":386},{},[387,393],{"type":44,"tag":104,"props":388,"children":390},{"className":389},[],[391],{"type":50,"value":392},"layer_types",{"type":50,"value":394}," - For hybrid models (e.g., mamba + attention)",{"type":44,"tag":98,"props":396,"children":397},{},[398,400,406,407,413,414,420,421],{"type":50,"value":399},"Scalar multipliers: ",{"type":44,"tag":104,"props":401,"children":403},{"className":402},[],[404],{"type":50,"value":405},"embedding_multiplier",{"type":50,"value":221},{"type":44,"tag":104,"props":408,"children":410},{"className":409},[],[411],{"type":50,"value":412},"attention_multiplier",{"type":50,"value":221},{"type":44,"tag":104,"props":415,"children":417},{"className":416},[],[418],{"type":50,"value":419},"residual_multiplier",{"type":50,"value":221},{"type":44,"tag":104,"props":422,"children":424},{"className":423},[],[425],{"type":50,"value":426},"logits_scaling",{"type":44,"tag":98,"props":428,"children":429},{},[430,436,438,444,445,451,453,459,461,466,468,473,475,481,483,492],{"type":44,"tag":104,"props":431,"children":433},{"className":432},[],[434],{"type":50,"value":435},"torch_dtype",{"type":50,"value":437}," or ",{"type":44,"tag":104,"props":439,"children":441},{"className":440},[],[442],{"type":50,"value":443},"dtype",{"type":50,"value":437},{"type":44,"tag":104,"props":446,"children":448},{"className":447},[],[449],{"type":50,"value":450},"text_config.dtype",{"type":50,"value":452}," — The model's intended runtime precision. Some models (especially VL wrappers) nest this inside ",{"type":44,"tag":104,"props":454,"children":456},{"className":455},[],[457],{"type":50,"value":458},"text_config",{"type":50,"value":460}," rather than exposing it as top-level ",{"type":44,"tag":104,"props":462,"children":464},{"className":463},[],[465],{"type":50,"value":435},{"type":50,"value":467}," (or ",{"type":44,"tag":104,"props":469,"children":471},{"className":470},[],[472],{"type":50,"value":443},{"type":50,"value":474},"). If the converter can't find it, it skips the dtype casting step entirely, leaving stray float32 weights in the converted model. Verify which field the model uses and confirm ",{"type":44,"tag":104,"props":476,"children":478},{"className":477},[],[479],{"type":50,"value":480},"mlx_lm.convert",{"type":50,"value":482}," detects it. Use ",{"type":44,"tag":71,"props":484,"children":486},{"href":485},"scripts\u002Fresolve_dtype.py",[487],{"type":44,"tag":104,"props":488,"children":490},{"className":489},[],[491],{"type":50,"value":485},{"type":50,"value":493}," to determine the expected dtype from the config (or the safetensors header as a fallback) and record the source — this becomes the basis for the dtype tests in Phase 4 and the manifest in Phase 8.",{"type":44,"tag":87,"props":495,"children":497},{"id":496},"phase-2-find-reference-mlx-models",[498],{"type":50,"value":499},"Phase 2: Find Reference MLX Models",{"type":44,"tag":60,"props":501,"children":502},{},[503,505,511],{"type":50,"value":504},"Examples from ",{"type":44,"tag":104,"props":506,"children":508},{"className":507},[],[509],{"type":50,"value":510},"mlx-lm\u002Fmlx_lm\u002Fmodels\u002F",{"type":50,"value":512},":",{"type":44,"tag":363,"props":514,"children":515},{},[516,527,546,563,588,599],{"type":44,"tag":98,"props":517,"children":518},{},[519,521],{"type":50,"value":520},"Standard attention: ",{"type":44,"tag":104,"props":522,"children":524},{"className":523},[],[525],{"type":50,"value":526},"llama.py",{"type":44,"tag":98,"props":528,"children":529},{},[530,532,538,540],{"type":50,"value":531},"MLA (Multi-head Latent Attention): ",{"type":44,"tag":104,"props":533,"children":535},{"className":534},[],[536],{"type":50,"value":537},"deepseek_v3.py",{"type":50,"value":539}," — see ",{"type":44,"tag":71,"props":541,"children":543},{"href":542},"references\u002Fmla.md",[544],{"type":50,"value":545},"MLA Reference",{"type":44,"tag":98,"props":547,"children":548},{},[549,551,556,557],{"type":50,"value":550},"MoE (Mixture of Experts): ",{"type":44,"tag":104,"props":552,"children":554},{"className":553},[],[555],{"type":50,"value":537},{"type":50,"value":221},{"type":44,"tag":104,"props":558,"children":560},{"className":559},[],[561],{"type":50,"value":562},"mixtral.py",{"type":44,"tag":98,"props":564,"children":565},{},[566,568,574,575,581,582],{"type":50,"value":567},"GatedDeltaNet (linear attention): ",{"type":44,"tag":104,"props":569,"children":571},{"className":570},[],[572],{"type":50,"value":573},"qwen3_next.py",{"type":50,"value":221},{"type":44,"tag":104,"props":576,"children":578},{"className":577},[],[579],{"type":50,"value":580},"qwen3_5.py",{"type":50,"value":221},{"type":44,"tag":104,"props":583,"children":585},{"className":584},[],[586],{"type":50,"value":587},"olmo_hybrid.py",{"type":44,"tag":98,"props":589,"children":590},{},[591,593],{"type":50,"value":592},"Hybrid (Mamba + Attention): ",{"type":44,"tag":104,"props":594,"children":596},{"className":595},[],[597],{"type":50,"value":598},"nemotron_h.py",{"type":44,"tag":98,"props":600,"children":601},{},[602,604],{"type":50,"value":603},"Sliding window: ",{"type":44,"tag":104,"props":605,"children":607},{"className":606},[],[608],{"type":50,"value":609},"mistral.py",{"type":44,"tag":60,"props":611,"children":612},{},[613,615,620,622,628],{"type":50,"value":614},"Start from the ",{"type":44,"tag":207,"props":616,"children":617},{},[618],{"type":50,"value":619},"closest existing model",{"type":50,"value":621}," rather than building from scratch — see ",{"type":44,"tag":71,"props":623,"children":625},{"href":624},"references\u002Fcode-selection-simplification.md",[626],{"type":50,"value":627},"Code Selection and Simplification",{"type":50,"value":348},{"type":44,"tag":87,"props":630,"children":632},{"id":631},"phase-3-create-mlx-model",[633],{"type":50,"value":634},"Phase 3: Create MLX Model",{"type":44,"tag":60,"props":636,"children":637},{},[638,640,646],{"type":50,"value":639},"Create ",{"type":44,"tag":104,"props":641,"children":643},{"className":642},[],[644],{"type":50,"value":645},"mlx-lm\u002Fmlx_lm\u002Fmodels\u002F\u003Cmodel_type>.py",{"type":50,"value":647}," (self-contained single file):",{"type":44,"tag":649,"props":650,"children":655},"pre",{"className":651,"code":652,"language":653,"meta":654,"style":654},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","@dataclass\nclass ModelArgs:\n    # Config params with defaults\n    position_embedding_type: str = \"rope\"  # Check for \"nope\"!\n\nclass Attention(nn.Module): ...\nclass MLP(nn.Module): ...\nclass DecoderLayer(nn.Module): ...\nclass LanguageModel(nn.Module): ...\n\nclass Model(nn.Module):\n    def sanitize(self, weights): ...  # Convert weight names\u002Fshapes\n    def shard(self, group): ...       # Distributed inference\n\n    @property\n    def cast_predicate(self):         # Keep specific weights in float32\n        return lambda _, m, _: isinstance(m, MoEGate) and \"bias\" in m\n","python","",[656],{"type":44,"tag":104,"props":657,"children":658},{"__ignoreMap":654},[659,670,679,688,697,707,715,724,733,742,750,759,768,777,785,794,803],{"type":44,"tag":660,"props":661,"children":664},"span",{"class":662,"line":663},"line",1,[665],{"type":44,"tag":660,"props":666,"children":667},{},[668],{"type":50,"value":669},"@dataclass\n",{"type":44,"tag":660,"props":671,"children":673},{"class":662,"line":672},2,[674],{"type":44,"tag":660,"props":675,"children":676},{},[677],{"type":50,"value":678},"class ModelArgs:\n",{"type":44,"tag":660,"props":680,"children":682},{"class":662,"line":681},3,[683],{"type":44,"tag":660,"props":684,"children":685},{},[686],{"type":50,"value":687},"    # Config params with defaults\n",{"type":44,"tag":660,"props":689,"children":691},{"class":662,"line":690},4,[692],{"type":44,"tag":660,"props":693,"children":694},{},[695],{"type":50,"value":696},"    position_embedding_type: str = \"rope\"  # Check for \"nope\"!\n",{"type":44,"tag":660,"props":698,"children":700},{"class":662,"line":699},5,[701],{"type":44,"tag":660,"props":702,"children":704},{"emptyLinePlaceholder":703},true,[705],{"type":50,"value":706},"\n",{"type":44,"tag":660,"props":708,"children":709},{"class":662,"line":31},[710],{"type":44,"tag":660,"props":711,"children":712},{},[713],{"type":50,"value":714},"class Attention(nn.Module): ...\n",{"type":44,"tag":660,"props":716,"children":718},{"class":662,"line":717},7,[719],{"type":44,"tag":660,"props":720,"children":721},{},[722],{"type":50,"value":723},"class MLP(nn.Module): ...\n",{"type":44,"tag":660,"props":725,"children":727},{"class":662,"line":726},8,[728],{"type":44,"tag":660,"props":729,"children":730},{},[731],{"type":50,"value":732},"class DecoderLayer(nn.Module): ...\n",{"type":44,"tag":660,"props":734,"children":736},{"class":662,"line":735},9,[737],{"type":44,"tag":660,"props":738,"children":739},{},[740],{"type":50,"value":741},"class LanguageModel(nn.Module): ...\n",{"type":44,"tag":660,"props":743,"children":745},{"class":662,"line":744},10,[746],{"type":44,"tag":660,"props":747,"children":748},{"emptyLinePlaceholder":703},[749],{"type":50,"value":706},{"type":44,"tag":660,"props":751,"children":753},{"class":662,"line":752},11,[754],{"type":44,"tag":660,"props":755,"children":756},{},[757],{"type":50,"value":758},"class Model(nn.Module):\n",{"type":44,"tag":660,"props":760,"children":762},{"class":662,"line":761},12,[763],{"type":44,"tag":660,"props":764,"children":765},{},[766],{"type":50,"value":767},"    def sanitize(self, weights): ...  # Convert weight names\u002Fshapes\n",{"type":44,"tag":660,"props":769,"children":771},{"class":662,"line":770},13,[772],{"type":44,"tag":660,"props":773,"children":774},{},[775],{"type":50,"value":776},"    def shard(self, group): ...       # Distributed inference\n",{"type":44,"tag":660,"props":778,"children":780},{"class":662,"line":779},14,[781],{"type":44,"tag":660,"props":782,"children":783},{"emptyLinePlaceholder":703},[784],{"type":50,"value":706},{"type":44,"tag":660,"props":786,"children":788},{"class":662,"line":787},15,[789],{"type":44,"tag":660,"props":790,"children":791},{},[792],{"type":50,"value":793},"    @property\n",{"type":44,"tag":660,"props":795,"children":797},{"class":662,"line":796},16,[798],{"type":44,"tag":660,"props":799,"children":800},{},[801],{"type":50,"value":802},"    def cast_predicate(self):         # Keep specific weights in float32\n",{"type":44,"tag":660,"props":804,"children":806},{"class":662,"line":805},17,[807],{"type":44,"tag":660,"props":808,"children":809},{},[810],{"type":50,"value":811},"        return lambda _, m, _: isinstance(m, MoEGate) and \"bias\" in m\n",{"type":44,"tag":60,"props":813,"children":814},{},[815,820,821,827],{"type":44,"tag":207,"props":816,"children":817},{},[818],{"type":50,"value":819},"sanitize() patterns",{"type":50,"value":539},{"type":44,"tag":71,"props":822,"children":824},{"href":823},"references\u002Fweight-sanitization.md",[825],{"type":50,"value":826},"Weight Sanitization",{"type":50,"value":828}," for detailed guidance on how to make weights loading compatible with both transformers and MLX-converted weights. Here are some common examples:",{"type":44,"tag":363,"props":830,"children":831},{},[832,859,872,883,888,893],{"type":44,"tag":98,"props":833,"children":834},{},[835,837,843,845,851,853],{"type":50,"value":836},"Stack experts: ",{"type":44,"tag":104,"props":838,"children":840},{"className":839},[],[841],{"type":50,"value":842},"experts.{N}.gate_proj",{"type":50,"value":844}," → ",{"type":44,"tag":104,"props":846,"children":848},{"className":847},[],[849],{"type":50,"value":850},"switch_mlp.gate_proj",{"type":50,"value":852}," shape ",{"type":44,"tag":104,"props":854,"children":856},{"className":855},[],[857],{"type":50,"value":858},"(num_experts, ...)",{"type":44,"tag":98,"props":860,"children":861},{},[862,864,870],{"type":50,"value":863},"Remove: ",{"type":44,"tag":104,"props":865,"children":867},{"className":866},[],[868],{"type":50,"value":869},"rotary_emb.inv_freq",{"type":50,"value":871},", extra layers (e.g., MTP heads)",{"type":44,"tag":98,"props":873,"children":874},{},[875,877],{"type":50,"value":876},"3D MoE weights: May need reshape from ",{"type":44,"tag":104,"props":878,"children":880},{"className":879},[],[881],{"type":50,"value":882},"(num_experts, hidden, intermediate)",{"type":44,"tag":98,"props":884,"children":885},{},[886],{"type":50,"value":887},"Shared\u002Ftied weights: Replicate from source blocks to all tied layers",{"type":44,"tag":98,"props":889,"children":890},{},[891],{"type":50,"value":892},"Different model sizes may use different weight formats — detect and branch",{"type":44,"tag":98,"props":894,"children":895},{},[896,898,904,906,912,914,920,921],{"type":50,"value":897},"MLA ",{"type":44,"tag":104,"props":899,"children":901},{"className":900},[],[902],{"type":50,"value":903},"kv_b_proj",{"type":50,"value":905}," splitting: dequantize → reshape by heads → split into ",{"type":44,"tag":104,"props":907,"children":909},{"className":908},[],[910],{"type":50,"value":911},"embed_q",{"type":50,"value":913},"\u002F",{"type":44,"tag":104,"props":915,"children":917},{"className":916},[],[918],{"type":50,"value":919},"unembed_out",{"type":50,"value":539},{"type":44,"tag":71,"props":922,"children":923},{"href":542},[924],{"type":50,"value":545},{"type":44,"tag":60,"props":926,"children":927},{},[928],{"type":44,"tag":207,"props":929,"children":930},{},[931],{"type":50,"value":932},"RoPE:",{"type":44,"tag":363,"props":934,"children":935},{},[936,954,965,996],{"type":44,"tag":98,"props":937,"children":938},{},[939,945,946,952],{"type":44,"tag":104,"props":940,"children":942},{"className":941},[],[943],{"type":50,"value":944},"rope_interleave=True",{"type":50,"value":844},{"type":44,"tag":104,"props":947,"children":949},{"className":948},[],[950],{"type":50,"value":951},"traditional=True",{"type":50,"value":953}," in MLX",{"type":44,"tag":98,"props":955,"children":956},{},[957,963],{"type":44,"tag":104,"props":958,"children":960},{"className":959},[],[961],{"type":50,"value":962},"position_embedding_type=\"nope\"",{"type":50,"value":964}," → Do NOT apply RoPE at all",{"type":44,"tag":98,"props":966,"children":967},{},[968,973,975,981,983,989,991,995],{"type":44,"tag":104,"props":969,"children":971},{"className":970},[],[972],{"type":50,"value":374},{"type":50,"value":974}," nested dict: some models store RoPE config as ",{"type":44,"tag":104,"props":976,"children":978},{"className":977},[],[979],{"type":50,"value":980},"{\"rope_theta\": ..., \"rope_type\": ...}",{"type":50,"value":982}," instead of top-level fields — extract in ",{"type":44,"tag":104,"props":984,"children":986},{"className":985},[],[987],{"type":50,"value":988},"__post_init__",{"type":50,"value":990},". See ",{"type":44,"tag":71,"props":992,"children":993},{"href":542},[994],{"type":50,"value":545},{"type":50,"value":348},{"type":44,"tag":98,"props":997,"children":998},{},[999,1004,1006,1011,1013,1019,1021,1026,1028,1034,1036,1042,1044,1050,1052,1057],{"type":44,"tag":207,"props":1000,"children":1001},{},[1002],{"type":50,"value":1003},"Null handling:",{"type":50,"value":1005}," ",{"type":44,"tag":104,"props":1007,"children":1009},{"className":1008},[],[1010],{"type":50,"value":374},{"type":50,"value":1012}," can be ",{"type":44,"tag":104,"props":1014,"children":1016},{"className":1015},[],[1017],{"type":50,"value":1018},"null",{"type":50,"value":1020},", absent, or a dict with ",{"type":44,"tag":104,"props":1022,"children":1024},{"className":1023},[],[1025],{"type":50,"value":1018},{"type":50,"value":1027}," values (e.g., ",{"type":44,"tag":104,"props":1029,"children":1031},{"className":1030},[],[1032],{"type":50,"value":1033},"{\"rope_theta\": null}",{"type":50,"value":1035},"). Use ",{"type":44,"tag":104,"props":1037,"children":1039},{"className":1038},[],[1040],{"type":50,"value":1041},".get()",{"type":50,"value":1043}," with defaults and test for ",{"type":44,"tag":104,"props":1045,"children":1047},{"className":1046},[],[1048],{"type":50,"value":1049},"None",{"type":50,"value":1051}," explicitly in ",{"type":44,"tag":104,"props":1053,"children":1055},{"className":1054},[],[1056],{"type":50,"value":988},{"type":50,"value":1058},". Verify the different paths the transformers codebase uses to deal with all variants present in the config files.",{"type":44,"tag":60,"props":1060,"children":1061},{},[1062,1067,1069,1074,1076,1082,1084,1090],{"type":44,"tag":207,"props":1063,"children":1064},{},[1065],{"type":50,"value":1066},"Conversion and quantization:",{"type":50,"value":1068}," Use ",{"type":44,"tag":104,"props":1070,"children":1072},{"className":1071},[],[1073],{"type":50,"value":480},{"type":50,"value":1075}," to validate sanitize and produce quantized models — see ",{"type":44,"tag":71,"props":1077,"children":1079},{"href":1078},"references\u002Fconversion-and-quantization.md",[1080],{"type":50,"value":1081},"Conversion and Quantization",{"type":50,"value":1083},". Only do this after testing transformers weights, or if the model is ",{"type":44,"tag":71,"props":1085,"children":1087},{"href":1086},"references\u002Flarge-models.md",[1088],{"type":50,"value":1089},"too large to run unquantized",{"type":50,"value":348},{"type":44,"tag":87,"props":1092,"children":1094},{"id":1093},"phase-4-testing",[1095],{"type":50,"value":1096},"Phase 4: Testing",{"type":44,"tag":60,"props":1098,"children":1099},{},[1100,1105,1107,1112,1114,1120],{"type":44,"tag":207,"props":1101,"children":1102},{},[1103],{"type":50,"value":1104},"Note:",{"type":50,"value":1106}," For very large models that can't fit in fp16 \u002F bf16, the standard comparison workflow isn't feasible. Use the quantize-first strategy instead — see ",{"type":44,"tag":71,"props":1108,"children":1109},{"href":1086},[1110],{"type":50,"value":1111},"Large Models",{"type":50,"value":1113},". For models that require multiple machines, see ",{"type":44,"tag":71,"props":1115,"children":1117},{"href":1116},"references\u002Fdistributed-inference.md",[1118],{"type":50,"value":1119},"Distributed Inference",{"type":50,"value":348},{"type":44,"tag":60,"props":1122,"children":1123},{},[1124,1129],{"type":44,"tag":207,"props":1125,"children":1126},{},[1127],{"type":50,"value":1128},"Test 1: Generation comparison",{"type":50,"value":1130}," (do first - easier to spot garbage output)",{"type":44,"tag":60,"props":1132,"children":1133},{},[1134,1136,1142,1144,1150],{"type":50,"value":1135},"Pre-validation: run ",{"type":44,"tag":104,"props":1137,"children":1139},{"className":1138},[],[1140],{"type":50,"value":1141},"mlx_lm.generate",{"type":50,"value":1143}," with a prompt and verify the output is coherent. This will use the default sampling method taken from the model's ",{"type":44,"tag":104,"props":1145,"children":1147},{"className":1146},[],[1148],{"type":50,"value":1149},"generation_config.json",{"type":50,"value":348},{"type":44,"tag":60,"props":1152,"children":1153},{},[1154],{"type":50,"value":1155},"Example:",{"type":44,"tag":649,"props":1157,"children":1161},{"className":1158,"code":1159,"language":1160,"meta":654,"style":654},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","mlx_lm.generate --model zai-org\u002FGLM-4.7-Flash --prompt \"Who are you?\"\n","bash",[1162],{"type":44,"tag":104,"props":1163,"children":1164},{"__ignoreMap":654},[1165],{"type":44,"tag":660,"props":1166,"children":1167},{"class":662,"line":663},[1168,1173,1179,1184,1189,1195,1200],{"type":44,"tag":660,"props":1169,"children":1171},{"style":1170},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[1172],{"type":50,"value":1141},{"type":44,"tag":660,"props":1174,"children":1176},{"style":1175},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[1177],{"type":50,"value":1178}," --model",{"type":44,"tag":660,"props":1180,"children":1181},{"style":1175},[1182],{"type":50,"value":1183}," zai-org\u002FGLM-4.7-Flash",{"type":44,"tag":660,"props":1185,"children":1186},{"style":1175},[1187],{"type":50,"value":1188}," --prompt",{"type":44,"tag":660,"props":1190,"children":1192},{"style":1191},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[1193],{"type":50,"value":1194}," \"",{"type":44,"tag":660,"props":1196,"children":1197},{"style":1175},[1198],{"type":50,"value":1199},"Who are you?",{"type":44,"tag":660,"props":1201,"children":1202},{"style":1191},[1203],{"type":50,"value":1204},"\"\n",{"type":44,"tag":60,"props":1206,"children":1207},{},[1208],{"type":50,"value":1209},"Use an appropriate prompt depending on the model capabilities (for example, is it a base model or does it use a chat template). If it's a base model, use an inviting prefix sentence to be continued, not just a single word. If it's a conversational model, ask an interesting question. If the model card mentions specific capabilities (for example, the model was trained for code generation), use a relevant topic.",{"type":44,"tag":60,"props":1211,"children":1212},{},[1213],{"type":50,"value":1214},"If generated text is garbage, use the layer-by-layer tests to try to identify implementation mismatches.",{"type":44,"tag":60,"props":1216,"children":1217},{},[1218,1220,1226],{"type":50,"value":1219},"For more controlled generation (greedy mode) and comparison with transformers, use the ",{"type":44,"tag":71,"props":1221,"children":1223},{"href":1222},"scripts\u002Fcompare_generation.py",[1224],{"type":50,"value":1225},"compare_generation.py script",{"type":50,"value":348},{"type":44,"tag":60,"props":1228,"children":1229},{},[1230],{"type":50,"value":1231},"When using this script, make a judgement call about whether the generations are semantically similar. They will rarely match exactly, because of differences in the compute device, kernel implementations, and other reasons.",{"type":44,"tag":60,"props":1233,"children":1234},{},[1235],{"type":50,"value":1236},"Do not declare success if generation is not reasonable - check for numerical and implementation differences and iterate until you get a good quality output, using the transformers one as the quality reference.",{"type":44,"tag":60,"props":1238,"children":1239},{},[1240],{"type":44,"tag":207,"props":1241,"children":1242},{},[1243],{"type":50,"value":1244},"Test 2: Output dtype",{"type":44,"tag":60,"props":1246,"children":1247},{},[1248,1250,1258,1260,1265,1267,1273,1274,1280],{"type":50,"value":1249},"Resolve the expected dtype with ",{"type":44,"tag":71,"props":1251,"children":1252},{"href":485},[1253],{"type":44,"tag":104,"props":1254,"children":1256},{"className":1255},[],[1257],{"type":50,"value":485},{"type":50,"value":1259}," — it tries config fields first, then falls back to the most common dtype in the safetensors header. Always record the ",{"type":44,"tag":207,"props":1261,"children":1262},{},[1263],{"type":50,"value":1264},"source",{"type":50,"value":1266}," in the conversion report (e.g., ",{"type":44,"tag":104,"props":1268,"children":1270},{"className":1269},[],[1271],{"type":50,"value":1272},"config.dtype",{"type":50,"value":221},{"type":44,"tag":104,"props":1275,"children":1277},{"className":1276},[],[1278],{"type":50,"value":1279},"safetensors header (BF16=292\u002F293, F32=1\u002F293)",{"type":50,"value":1281},") so reviewers know the basis for the check.",{"type":44,"tag":649,"props":1283,"children":1285},{"className":1158,"code":1284,"language":1160,"meta":654,"style":654},"python scripts\u002Fresolve_dtype.py \u002Fpath\u002Fto\u002Fmodel\n# dtype: bfloat16\n# source: safetensors header (BF16=292\u002F293, F32=1\u002F293)\n",[1286],{"type":44,"tag":104,"props":1287,"children":1288},{"__ignoreMap":654},[1289,1306,1315],{"type":44,"tag":660,"props":1290,"children":1291},{"class":662,"line":663},[1292,1296,1301],{"type":44,"tag":660,"props":1293,"children":1294},{"style":1170},[1295],{"type":50,"value":653},{"type":44,"tag":660,"props":1297,"children":1298},{"style":1175},[1299],{"type":50,"value":1300}," scripts\u002Fresolve_dtype.py",{"type":44,"tag":660,"props":1302,"children":1303},{"style":1175},[1304],{"type":50,"value":1305}," \u002Fpath\u002Fto\u002Fmodel\n",{"type":44,"tag":660,"props":1307,"children":1308},{"class":662,"line":672},[1309],{"type":44,"tag":660,"props":1310,"children":1312},{"style":1311},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[1313],{"type":50,"value":1314},"# dtype: bfloat16\n",{"type":44,"tag":660,"props":1316,"children":1317},{"class":662,"line":681},[1318],{"type":44,"tag":660,"props":1319,"children":1320},{"style":1311},[1321],{"type":50,"value":1322},"# source: safetensors header (BF16=292\u002F293, F32=1\u002F293)\n",{"type":44,"tag":60,"props":1324,"children":1325},{},[1326,1328,1337],{"type":50,"value":1327},"Then run a forward pass and verify the output dtype matches with ",{"type":44,"tag":71,"props":1329,"children":1331},{"href":1330},"scripts\u002Fcheck_dtype.py",[1332],{"type":44,"tag":104,"props":1333,"children":1335},{"className":1334},[],[1336],{"type":50,"value":1330},{"type":50,"value":512},{"type":44,"tag":649,"props":1339,"children":1341},{"className":1158,"code":1340,"language":1160,"meta":654,"style":654},"python scripts\u002Fcheck_dtype.py \u002Fpath\u002Fto\u002Fmodel bfloat16\n# {\"output_dtype\": \"mlx.core.bfloat16\", \"expected_dtype\": \"bfloat16\", \"match\": true}\n",[1342],{"type":44,"tag":104,"props":1343,"children":1344},{"__ignoreMap":654},[1345,1367],{"type":44,"tag":660,"props":1346,"children":1347},{"class":662,"line":663},[1348,1352,1357,1362],{"type":44,"tag":660,"props":1349,"children":1350},{"style":1170},[1351],{"type":50,"value":653},{"type":44,"tag":660,"props":1353,"children":1354},{"style":1175},[1355],{"type":50,"value":1356}," scripts\u002Fcheck_dtype.py",{"type":44,"tag":660,"props":1358,"children":1359},{"style":1175},[1360],{"type":50,"value":1361}," \u002Fpath\u002Fto\u002Fmodel",{"type":44,"tag":660,"props":1363,"children":1364},{"style":1175},[1365],{"type":50,"value":1366}," bfloat16\n",{"type":44,"tag":660,"props":1368,"children":1369},{"class":662,"line":672},[1370],{"type":44,"tag":660,"props":1371,"children":1372},{"style":1311},[1373],{"type":50,"value":1374},"# {\"output_dtype\": \"mlx.core.bfloat16\", \"expected_dtype\": \"bfloat16\", \"match\": true}\n",{"type":44,"tag":60,"props":1376,"children":1377},{},[1378,1380,1386],{"type":50,"value":1379},"Exits non-zero with a ",{"type":44,"tag":104,"props":1381,"children":1383},{"className":1382},[],[1384],{"type":50,"value":1385},"FAIL:",{"type":50,"value":1387}," message on stderr if the runtime dtype doesn't match.",{"type":44,"tag":60,"props":1389,"children":1390},{},[1391,1393,1399],{"type":50,"value":1392},"A single float32 weight (e.g., a norm weight promoted by ",{"type":44,"tag":104,"props":1394,"children":1396},{"className":1395},[],[1397],{"type":50,"value":1398},"v + 1.0",{"type":50,"value":1400}," in sanitize) will silently propagate float32 through the entire model via dtype promotion, causing up to 5x slower inference on Apple Silicon.",{"type":44,"tag":60,"props":1402,"children":1403},{},[1404],{"type":50,"value":1405},"The same script works after quantization — the runtime output dtype is unaffected by weight quantization (only the storage of matmul weights changes). Re-run it on the quantized model in Phase 5 to confirm.",{"type":44,"tag":60,"props":1407,"children":1408},{},[1409],{"type":44,"tag":207,"props":1410,"children":1411},{},[1412],{"type":50,"value":1413},"Test 3: Numerical comparison",{"type":44,"tag":60,"props":1415,"children":1416},{},[1417,1419,1425,1427,1432,1434,1440],{"type":50,"value":1418},"Use the ",{"type":44,"tag":71,"props":1420,"children":1422},{"href":1421},"scripts\u002Fcompare_predictions.py",[1423],{"type":50,"value":1424},"compare_predictions script",{"type":50,"value":1426}," to compare logits from a single forward pass and extract some statistics. ",{"type":44,"tag":207,"props":1428,"children":1429},{},[1430],{"type":50,"value":1431},"Use a prompt of at least 100 tokens",{"type":50,"value":1433}," (e.g., a paragraph of text) — short prompts (5-10 tokens) can hide positional encoding bugs because RoPE error grows with position. A wrong ",{"type":44,"tag":104,"props":1435,"children":1437},{"className":1436},[],[1438],{"type":50,"value":1439},"rope_theta",{"type":50,"value":1441}," may pass top-1 comparison at 5 tokens but diverge significantly at 100+.",{"type":44,"tag":60,"props":1443,"children":1444},{},[1445],{"type":50,"value":1446},"Point out any major differences you observe. A max logits difference in ~0.5-1.0 (or larger, for models with many layers where errors accumulate) is usual between CPU transformers and Metal MLX. If you see suspicious values, run a layer-by-layer comparison even if the generation test seemed to pass.",{"type":44,"tag":60,"props":1448,"children":1449},{},[1450],{"type":50,"value":1451},"If you run a top-5 or top-10 comparison, include the top-10 list in the GitHub PR you'll open when you're done (unless they are identical).",{"type":44,"tag":60,"props":1453,"children":1454},{},[1455,1460],{"type":44,"tag":207,"props":1456,"children":1457},{},[1458],{"type":50,"value":1459},"Test 4: Layer-by-layer",{"type":50,"value":1461}," (if predictions don't match — and as a routine sanity check)",{"type":44,"tag":60,"props":1463,"children":1464},{},[1465,1467,1476],{"type":50,"value":1466},"Use ",{"type":44,"tag":71,"props":1468,"children":1470},{"href":1469},"scripts\u002Fcompare_layers.py",[1471],{"type":44,"tag":104,"props":1472,"children":1474},{"className":1473},[],[1475],{"type":50,"value":1469},{"type":50,"value":1477},". One forward pass per framework captures the output of every decoder layer, the post-final-norm state, and the logits, then compares them side-by-side with absolute and relative diffs.",{"type":44,"tag":649,"props":1479,"children":1481},{"className":1158,"code":1480,"language":1160,"meta":654,"style":654},"python scripts\u002Fcompare_layers.py \u002Fpath\u002Fto\u002Fmodel \"\u003Cprompt>\"\n",[1482],{"type":44,"tag":104,"props":1483,"children":1484},{"__ignoreMap":654},[1485],{"type":44,"tag":660,"props":1486,"children":1487},{"class":662,"line":663},[1488,1492,1497,1501,1505,1510],{"type":44,"tag":660,"props":1489,"children":1490},{"style":1170},[1491],{"type":50,"value":653},{"type":44,"tag":660,"props":1493,"children":1494},{"style":1175},[1495],{"type":50,"value":1496}," scripts\u002Fcompare_layers.py",{"type":44,"tag":660,"props":1498,"children":1499},{"style":1175},[1500],{"type":50,"value":1361},{"type":44,"tag":660,"props":1502,"children":1503},{"style":1191},[1504],{"type":50,"value":1194},{"type":44,"tag":660,"props":1506,"children":1507},{"style":1175},[1508],{"type":50,"value":1509},"\u003Cprompt>",{"type":44,"tag":660,"props":1511,"children":1512},{"style":1191},[1513],{"type":50,"value":1204},{"type":44,"tag":60,"props":1515,"children":1516},{},[1517],{"type":50,"value":1518},"Read the table from the bottom up:",{"type":44,"tag":363,"props":1520,"children":1521},{},[1522,1538,1553],{"type":44,"tag":98,"props":1523,"children":1524},{},[1525,1536],{"type":44,"tag":207,"props":1526,"children":1527},{},[1528,1534],{"type":44,"tag":104,"props":1529,"children":1531},{"className":1530},[],[1532],{"type":50,"value":1533},"logits",{"type":50,"value":1535}," row",{"type":50,"value":1537},": the most important. If logits agree, the implementation is end-to-end correct even if hidden states drift along the way.",{"type":44,"tag":98,"props":1539,"children":1540},{},[1541,1551],{"type":44,"tag":207,"props":1542,"children":1543},{},[1544,1550],{"type":44,"tag":104,"props":1545,"children":1547},{"className":1546},[],[1548],{"type":50,"value":1549},"post-norm",{"type":50,"value":1535},{"type":50,"value":1552},": hidden state after the final RMSNorm. Should be small relative diff.",{"type":44,"tag":98,"props":1554,"children":1555},{},[1556,1567,1569,1575],{"type":44,"tag":207,"props":1557,"children":1558},{},[1559,1565],{"type":44,"tag":104,"props":1560,"children":1562},{"className":1561},[],[1563],{"type":50,"value":1564},"layer N",{"type":50,"value":1566}," rows",{"type":50,"value":1568},": per-layer outputs. Look for the first sharp jump in ",{"type":44,"tag":104,"props":1570,"children":1572},{"className":1571},[],[1573],{"type":50,"value":1574},"rel diff",{"type":50,"value":1576}," — that's where divergence is introduced.",{"type":44,"tag":60,"props":1578,"children":1579},{},[1580,1582,1588,1590,1596],{"type":50,"value":1581},"The script is architecture-agnostic: it wraps each entry in ",{"type":44,"tag":104,"props":1583,"children":1585},{"className":1584},[],[1586],{"type":50,"value":1587},"model.[language_model.]model.layers",{"type":50,"value":1589}," with a thin capture proxy (preserving the layer's class so ",{"type":44,"tag":104,"props":1591,"children":1593},{"className":1592},[],[1594],{"type":50,"value":1595},"isinstance",{"type":50,"value":1597}," checks in hybrid models keep working). It does NOT compare the embedding output, since pre-layer transforms (multipliers, per-layer scaling) differ across architectures and the embedding row is rarely informative.",{"type":44,"tag":60,"props":1599,"children":1600},{},[1601,1603,1608],{"type":50,"value":1602},"Healthy implementations show \u003C1% relative diff at every position. A pattern where mid-layer divergence is relatively large but the ",{"type":44,"tag":104,"props":1604,"children":1606},{"className":1605},[],[1607],{"type":50,"value":1533},{"type":50,"value":1609}," row is small usually means the divergence cancels out at the head — annoying but not a correctness bug.",{"type":44,"tag":60,"props":1611,"children":1612},{},[1613],{"type":50,"value":1614},"Include results in the report.",{"type":44,"tag":60,"props":1616,"children":1617},{},[1618],{"type":44,"tag":207,"props":1619,"children":1620},{},[1621],{"type":50,"value":1622},"Test 5: Long sequence degradation",{"type":44,"tag":363,"props":1624,"children":1625},{},[1626,1639],{"type":44,"tag":98,"props":1627,"children":1628},{},[1629,1631,1637],{"type":50,"value":1630},"Run a prompt that generates a long sequence, such as \"Write an HTML and JavaScript page implementing space invaders\", or something appropriate depending on model capabilities. You can use ",{"type":44,"tag":104,"props":1632,"children":1634},{"className":1633},[],[1635],{"type":50,"value":1636},"mlx_lm.generate -m 16384",{"type":50,"value":1638}," to allow up to 16K tokens to be generated.",{"type":44,"tag":98,"props":1640,"children":1641},{},[1642],{"type":50,"value":1643},"Observe if the output turns to garbage after a while. This usually signals mistakes in RoPE.",{"type":44,"tag":87,"props":1645,"children":1647},{"id":1646},"phase-5-quants",[1648],{"type":50,"value":1649},"Phase 5: Quants",{"type":44,"tag":60,"props":1651,"children":1652},{},[1653,1654,1659,1661,1666],{"type":50,"value":1466},{"type":44,"tag":104,"props":1655,"children":1657},{"className":1656},[],[1658],{"type":50,"value":480},{"type":50,"value":1660}," to quantize the model. First, run ",{"type":44,"tag":104,"props":1662,"children":1664},{"className":1663},[],[1665],{"type":50,"value":485},{"type":50,"value":1667}," on the source checkpoint to confirm the storage dtype. If the model already deals with native quantized weights, skip this phase and inform the user. If it doesn't, try a simple quantization to 4 bits and run generation tests to verify whether results are coherent.",{"type":44,"tag":60,"props":1669,"children":1670},{},[1671,1673,1678],{"type":50,"value":1672},"After quantization, re-run ",{"type":44,"tag":104,"props":1674,"children":1676},{"className":1675},[],[1677],{"type":50,"value":1330},{"type":50,"value":1679}," against the quantized model with the same expected dtype as the unquantized one. The runtime dtype shouldn't change with quantization — only the storage of matmul weights does. A mismatch here usually means a stray float32 weight survived the quantize step.",{"type":44,"tag":87,"props":1681,"children":1683},{"id":1682},"phase-6-mlx-tests",[1684],{"type":50,"value":1685},"Phase 6: MLX Tests",{"type":44,"tag":60,"props":1687,"children":1688},{},[1689],{"type":50,"value":1690},"When appropriate, create tests for the model you just converted, following the structure and criteria used in the mlx-lm repo.",{"type":44,"tag":87,"props":1692,"children":1694},{"id":1693},"phase-7-report",[1695],{"type":50,"value":1696},"Phase 7: Report",{"type":44,"tag":60,"props":1698,"children":1699},{},[1700],{"type":50,"value":1701},"Always include the following in your output report:",{"type":44,"tag":363,"props":1703,"children":1704},{},[1705,1710,1715,1720,1746,1751,1756,1761,1766,1779],{"type":44,"tag":98,"props":1706,"children":1707},{},[1708],{"type":50,"value":1709},"The list of models and variants selected for conversion, and tested.",{"type":44,"tag":98,"props":1711,"children":1712},{},[1713],{"type":50,"value":1714},"Notable architecture decisions and novelties introduced.",{"type":44,"tag":98,"props":1716,"children":1717},{},[1718],{"type":50,"value":1719},"Non-trivial implementation details that are worth noting.",{"type":44,"tag":98,"props":1721,"children":1722},{},[1723,1725,1730,1732,1738,1740,1745],{"type":50,"value":1724},"The expected output dtype per variant and where it came from (e.g., ",{"type":44,"tag":104,"props":1726,"children":1728},{"className":1727},[],[1729],{"type":50,"value":1272},{"type":50,"value":1731}," vs ",{"type":44,"tag":104,"props":1733,"children":1735},{"className":1734},[],[1736],{"type":50,"value":1737},"safetensors header",{"type":50,"value":1739},"). Use the results from ",{"type":44,"tag":104,"props":1741,"children":1743},{"className":1742},[],[1744],{"type":50,"value":485},{"type":50,"value":348},{"type":44,"tag":98,"props":1747,"children":1748},{},[1749],{"type":50,"value":1750},"At least a meaningful generation example of ~200 tokens or more.",{"type":44,"tag":98,"props":1752,"children":1753},{},[1754],{"type":50,"value":1755},"A summary of the numerical differences you found.",{"type":44,"tag":98,"props":1757,"children":1758},{},[1759],{"type":50,"value":1760},"Per-layer comparison results.",{"type":44,"tag":98,"props":1762,"children":1763},{},[1764],{"type":50,"value":1765},"Errors that you couldn't solve and pointers to analyze them.",{"type":44,"tag":98,"props":1767,"children":1768},{},[1769,1771,1777],{"type":50,"value":1770},"If changes to ",{"type":44,"tag":104,"props":1772,"children":1774},{"className":1773},[],[1775],{"type":50,"value":1776},"mlx-lm",{"type":50,"value":1778}," were needed, explain in detail what they were, and provide links to the equivalent transformers implementation. This should be rare (bugs or missing features).",{"type":44,"tag":98,"props":1780,"children":1781},{},[1782],{"type":50,"value":1783},"Reflect upon learnings that we could consider to incorporate back into the Skill for future reference. Trivial discoveries that are easy for you to replicate in future sessions are not necessary; focus on initial error cases that required some iteration.",{"type":44,"tag":60,"props":1785,"children":1786},{},[1787],{"type":50,"value":1788},"Please, output this report in your conversation, and also create a markdown file for reference.",{"type":44,"tag":87,"props":1790,"children":1792},{"id":1791},"phase-8-github-pr",[1793],{"type":50,"value":1794},"Phase 8: GitHub PR",{"type":44,"tag":363,"props":1796,"children":1797},{},[1798,1824,1829,1842,1847,1890],{"type":44,"tag":98,"props":1799,"children":1800},{},[1801,1806,1808,1814,1816,1822],{"type":44,"tag":207,"props":1802,"children":1803},{},[1804],{"type":50,"value":1805},"Create a feature branch",{"type":50,"value":1807}," for your changes (e.g., ",{"type":44,"tag":104,"props":1809,"children":1811},{"className":1810},[],[1812],{"type":50,"value":1813},"add-\u003Cmodel_type>",{"type":50,"value":1815},") — never commit directly to the main working branch. See ",{"type":44,"tag":71,"props":1817,"children":1819},{"href":1818},"references\u002Fgithub-pr-workflow.md",[1820],{"type":50,"value":1821},"GitHub PR Workflow",{"type":50,"value":1823}," for details.",{"type":44,"tag":98,"props":1825,"children":1826},{},[1827],{"type":50,"value":1828},"ALWAYS ask for confirmation before submitting a PR. The user may have questions or want to run additional tests, or include more variants.",{"type":44,"tag":98,"props":1830,"children":1831},{},[1832,1834,1840],{"type":50,"value":1833},"Make sure the ",{"type":44,"tag":104,"props":1835,"children":1837},{"className":1836},[],[1838],{"type":50,"value":1839},"gh",{"type":50,"value":1841}," cli is authenticated.",{"type":44,"tag":98,"props":1843,"children":1844},{},[1845],{"type":50,"value":1846},"Ensure you use the github repo specified at the beginning of this document (even if it's not the canonical one).",{"type":44,"tag":98,"props":1848,"children":1849},{},[1850,1852],{"type":50,"value":1851},"In the PR, include:\n",{"type":44,"tag":363,"props":1853,"children":1854},{},[1855,1868,1873,1878],{"type":44,"tag":98,"props":1856,"children":1857},{},[1858,1860,1866],{"type":50,"value":1859},"A suitable title. But please, prefix it with ",{"type":44,"tag":104,"props":1861,"children":1863},{"className":1862},[],[1864],{"type":50,"value":1865},"[transformers-to-mlx skill]",{"type":50,"value":1867}," for disclosure.",{"type":44,"tag":98,"props":1869,"children":1870},{},[1871],{"type":50,"value":1872},"Also disclose that all tests and results were obtained with the skill.",{"type":44,"tag":98,"props":1874,"children":1875},{},[1876],{"type":50,"value":1877},"Your summary report in the PR.",{"type":44,"tag":98,"props":1879,"children":1880},{},[1881,1883,1888],{"type":50,"value":1882},"In addition to the summary report, include test commands and their outputs. For example, if you ran a long sequence generation with ",{"type":44,"tag":104,"props":1884,"children":1886},{"className":1885},[],[1887],{"type":50,"value":1141},{"type":50,"value":1889},", include the command and the full output. Also include the source code you used to compare numerical results, when appropriate. Same thing for the dtype tests and everything else.",{"type":44,"tag":98,"props":1891,"children":1892},{},[1893,1898,1900,1906],{"type":44,"tag":207,"props":1894,"children":1895},{},[1896],{"type":50,"value":1897},"After the mlx-lm PR is created",{"type":50,"value":1899},", generate a test manifest and open a PR to the test harness repo. See ",{"type":44,"tag":71,"props":1901,"children":1903},{"href":1902},"references\u002Fgithub-pr-workflow.md#test-manifest",[1904],{"type":50,"value":1905},"GitHub PR Workflow — Test Manifest",{"type":50,"value":1907}," for the format and procedure.",{"type":44,"tag":53,"props":1909,"children":1911},{"id":1910},"critical-lessons",[1912],{"type":50,"value":1913},"Critical Lessons",{"type":44,"tag":87,"props":1915,"children":1917},{"id":1916},"garbage-output-despite-loading",[1918],{"type":50,"value":1919},"Garbage Output Despite Loading",{"type":44,"tag":60,"props":1921,"children":1922},{},[1923],{"type":50,"value":1924},"Model loading and generating without crashes does NOT mean it works. Always verify:",{"type":44,"tag":94,"props":1926,"children":1927},{},[1928,1933,1938],{"type":44,"tag":98,"props":1929,"children":1930},{},[1931],{"type":50,"value":1932},"Output is coherent text (not \"lylyly Usesfinite\")",{"type":44,"tag":98,"props":1934,"children":1935},{},[1936],{"type":50,"value":1937},"Top predictions match transformers",{"type":44,"tag":98,"props":1939,"children":1940},{},[1941],{"type":50,"value":1942},"Logits statistics are in same ballpark",{"type":44,"tag":87,"props":1944,"children":1946},{"id":1945},"position-embeddings",[1947],{"type":50,"value":1948},"Position Embeddings",{"type":44,"tag":60,"props":1950,"children":1951},{},[1952],{"type":50,"value":1953},"This is a usual source of problems.",{"type":44,"tag":60,"props":1955,"children":1956},{},[1957,1959,1964],{"type":50,"value":1958},"Check ",{"type":44,"tag":104,"props":1960,"children":1962},{"className":1961},[],[1963],{"type":50,"value":381},{"type":50,"value":1965}," in config:",{"type":44,"tag":363,"props":1967,"children":1968},{},[1969,1980],{"type":44,"tag":98,"props":1970,"children":1971},{},[1972,1978],{"type":44,"tag":104,"props":1973,"children":1975},{"className":1974},[],[1976],{"type":50,"value":1977},"\"rope\"",{"type":50,"value":1979}," - Apply rotary position embeddings",{"type":44,"tag":98,"props":1981,"children":1982},{},[1983,1989],{"type":44,"tag":104,"props":1984,"children":1986},{"className":1985},[],[1987],{"type":50,"value":1988},"\"nope\"",{"type":50,"value":1990}," - Do NOT apply any position embeddings (common in hybrid models)",{"type":44,"tag":60,"props":1992,"children":1993},{},[1994,1999,2001,2007,2009,2015],{"type":44,"tag":207,"props":1995,"children":1996},{},[1997],{"type":50,"value":1998},"RoPE config can vary across variants of the same architecture.",{"type":50,"value":2000}," For example, OLMo-Hybrid base\u002FSFT use ",{"type":44,"tag":104,"props":2002,"children":2004},{"className":2003},[],[2005],{"type":50,"value":2006},"rope_theta=10000",{"type":50,"value":2008}," while the DPO variant uses NoPE (no positional embeddings at all, via ",{"type":44,"tag":104,"props":2010,"children":2012},{"className":2011},[],[2013],{"type":50,"value":2014},"rope_parameters: {rope_theta: null}",{"type":50,"value":2016},"). Always diff RoPE-related config fields across ALL target checkpoints — base, SFT, DPO, and different sizes — before implementation.",{"type":44,"tag":649,"props":2018,"children":2020},{"className":651,"code":2019,"language":653,"meta":654,"style":654},"# Conditional RoPE\nuse_rope = getattr(args, 'position_embedding_type', 'rope') != 'nope'\nif use_rope:\n    self.rope = initialize_rope(...)\nelse:\n    self.rope = None\n",[2021],{"type":44,"tag":104,"props":2022,"children":2023},{"__ignoreMap":654},[2024,2032,2040,2048,2056,2064],{"type":44,"tag":660,"props":2025,"children":2026},{"class":662,"line":663},[2027],{"type":44,"tag":660,"props":2028,"children":2029},{},[2030],{"type":50,"value":2031},"# Conditional RoPE\n",{"type":44,"tag":660,"props":2033,"children":2034},{"class":662,"line":672},[2035],{"type":44,"tag":660,"props":2036,"children":2037},{},[2038],{"type":50,"value":2039},"use_rope = getattr(args, 'position_embedding_type', 'rope') != 'nope'\n",{"type":44,"tag":660,"props":2041,"children":2042},{"class":662,"line":681},[2043],{"type":44,"tag":660,"props":2044,"children":2045},{},[2046],{"type":50,"value":2047},"if use_rope:\n",{"type":44,"tag":660,"props":2049,"children":2050},{"class":662,"line":690},[2051],{"type":44,"tag":660,"props":2052,"children":2053},{},[2054],{"type":50,"value":2055},"    self.rope = initialize_rope(...)\n",{"type":44,"tag":660,"props":2057,"children":2058},{"class":662,"line":699},[2059],{"type":44,"tag":660,"props":2060,"children":2061},{},[2062],{"type":50,"value":2063},"else:\n",{"type":44,"tag":660,"props":2065,"children":2066},{"class":662,"line":31},[2067],{"type":44,"tag":660,"props":2068,"children":2069},{},[2070],{"type":50,"value":2071},"    self.rope = None\n",{"type":44,"tag":87,"props":2073,"children":2075},{"id":2074},"scalar-multipliers",[2076],{"type":50,"value":2077},"Scalar Multipliers",{"type":44,"tag":60,"props":2079,"children":2080},{},[2081],{"type":50,"value":2082},"Some models (Granite, etc.) use scaling factors that MUST be applied exactly:",{"type":44,"tag":363,"props":2084,"children":2085},{},[2086,2096,2114,2124],{"type":44,"tag":98,"props":2087,"children":2088},{},[2089,2094],{"type":44,"tag":104,"props":2090,"children":2092},{"className":2091},[],[2093],{"type":50,"value":405},{"type":50,"value":2095}," - Scale embeddings before first layer",{"type":44,"tag":98,"props":2097,"children":2098},{},[2099,2104,2106,2112],{"type":44,"tag":104,"props":2100,"children":2102},{"className":2101},[],[2103],{"type":50,"value":412},{"type":50,"value":2105}," - Replace ",{"type":44,"tag":104,"props":2107,"children":2109},{"className":2108},[],[2110],{"type":50,"value":2111},"1\u002Fsqrt(head_dim)",{"type":50,"value":2113}," in attention",{"type":44,"tag":98,"props":2115,"children":2116},{},[2117,2122],{"type":44,"tag":104,"props":2118,"children":2120},{"className":2119},[],[2121],{"type":50,"value":419},{"type":50,"value":2123}," - Scale residual connections",{"type":44,"tag":98,"props":2125,"children":2126},{},[2127,2132],{"type":44,"tag":104,"props":2128,"children":2130},{"className":2129},[],[2131],{"type":50,"value":426},{"type":50,"value":2133}," - Scale final logits",{"type":44,"tag":60,"props":2135,"children":2136},{},[2137],{"type":50,"value":2138},"Missing any of these causes completely wrong predictions.",{"type":44,"tag":87,"props":2140,"children":2142},{"id":2141},"weight-key-names-checkpoint-vs-transformers-attributes",[2143],{"type":50,"value":2144},"Weight Key Names: Checkpoint vs Transformers Attributes",{"type":44,"tag":60,"props":2146,"children":2147},{},[2148,2150,2155,2157,2163,2164,2170,2172,2178,2179,2185,2186,2192,2193,2199],{"type":50,"value":2149},"When building the MLX model, use the ",{"type":44,"tag":207,"props":2151,"children":2152},{},[2153],{"type":50,"value":2154},"transformers model attribute names",{"type":50,"value":2156}," for your module definitions — not the raw checkpoint key names. These often differ: the checkpoint may use names like ",{"type":44,"tag":104,"props":2158,"children":2160},{"className":2159},[],[2161],{"type":50,"value":2162},"Wqkv",{"type":50,"value":437},{"type":44,"tag":104,"props":2165,"children":2167},{"className":2166},[],[2168],{"type":50,"value":2169},"gate_up_proj",{"type":50,"value":2171}," while the transformers model class uses ",{"type":44,"tag":104,"props":2173,"children":2175},{"className":2174},[],[2176],{"type":50,"value":2177},"q_proj",{"type":50,"value":221},{"type":44,"tag":104,"props":2180,"children":2182},{"className":2181},[],[2183],{"type":50,"value":2184},"k_proj",{"type":50,"value":221},{"type":44,"tag":104,"props":2187,"children":2189},{"className":2188},[],[2190],{"type":50,"value":2191},"gate_proj",{"type":50,"value":221},{"type":44,"tag":104,"props":2194,"children":2196},{"className":2195},[],[2197],{"type":50,"value":2198},"up_proj",{"type":50,"value":2200},", etc.",{"type":44,"tag":60,"props":2202,"children":2203},{},[2204,2206,2212],{"type":50,"value":2205},"The ",{"type":44,"tag":104,"props":2207,"children":2209},{"className":2208},[],[2210],{"type":50,"value":2211},"sanitize()",{"type":50,"value":2213}," method is where you remap checkpoint names to match the MLX model structure (which mirrors transformers attribute names). If the MLX model uses checkpoint names instead, weight loading will silently fail or require a convoluted sanitize.",{"type":44,"tag":60,"props":2215,"children":2216},{},[2217,2219,2225,2227,2232],{"type":50,"value":2218},"To find the authoritative mapping, check ",{"type":44,"tag":104,"props":2220,"children":2222},{"className":2221},[],[2223],{"type":50,"value":2224},"conversion_mapping.py",{"type":50,"value":2226}," in the transformers model directory — see ",{"type":44,"tag":71,"props":2228,"children":2230},{"href":2229},"references\u002Fweight-sanitization.md#transformers-conversion_mappingpy",[2231],{"type":50,"value":826},{"type":50,"value":348},{"type":44,"tag":87,"props":2234,"children":2236},{"id":2235},"numerical-precision",[2237],{"type":50,"value":2238},"Numerical Precision",{"type":44,"tag":60,"props":2240,"children":2241},{},[2242],{"type":50,"value":2243},"Some models define custom operations (e.g., gated RMSNorm, specialized activations) that are sensitive to precision. If the transformers code explicitly casts to float32, the MLX implementation must do the same:",{"type":44,"tag":649,"props":2245,"children":2247},{"className":651,"code":2246,"language":653,"meta":654,"style":654},"class GatedRMSNorm(nn.Module):\n    def __call__(self, x, residual):\n        original_dtype = x.dtype\n        x = x.astype(mx.float32)\n        norm = mx.rsqrt(mx.mean(x * x, axis=-1, keepdims=True) + self.eps)\n        x = (x * norm).astype(original_dtype) * self.weight\n        return x\n",[2248],{"type":44,"tag":104,"props":2249,"children":2250},{"__ignoreMap":654},[2251,2259,2267,2275,2283,2291,2299],{"type":44,"tag":660,"props":2252,"children":2253},{"class":662,"line":663},[2254],{"type":44,"tag":660,"props":2255,"children":2256},{},[2257],{"type":50,"value":2258},"class GatedRMSNorm(nn.Module):\n",{"type":44,"tag":660,"props":2260,"children":2261},{"class":662,"line":672},[2262],{"type":44,"tag":660,"props":2263,"children":2264},{},[2265],{"type":50,"value":2266},"    def __call__(self, x, residual):\n",{"type":44,"tag":660,"props":2268,"children":2269},{"class":662,"line":681},[2270],{"type":44,"tag":660,"props":2271,"children":2272},{},[2273],{"type":50,"value":2274},"        original_dtype = x.dtype\n",{"type":44,"tag":660,"props":2276,"children":2277},{"class":662,"line":690},[2278],{"type":44,"tag":660,"props":2279,"children":2280},{},[2281],{"type":50,"value":2282},"        x = x.astype(mx.float32)\n",{"type":44,"tag":660,"props":2284,"children":2285},{"class":662,"line":699},[2286],{"type":44,"tag":660,"props":2287,"children":2288},{},[2289],{"type":50,"value":2290},"        norm = mx.rsqrt(mx.mean(x * x, axis=-1, keepdims=True) + self.eps)\n",{"type":44,"tag":660,"props":2292,"children":2293},{"class":662,"line":31},[2294],{"type":44,"tag":660,"props":2295,"children":2296},{},[2297],{"type":50,"value":2298},"        x = (x * norm).astype(original_dtype) * self.weight\n",{"type":44,"tag":660,"props":2300,"children":2301},{"class":662,"line":717},[2302],{"type":44,"tag":660,"props":2303,"children":2304},{},[2305],{"type":50,"value":2306},"        return x\n",{"type":44,"tag":60,"props":2308,"children":2309},{},[2310,2312,2318,2319,2325],{"type":50,"value":2311},"Missing a float32 cast may not cause obvious errors in short sequences but leads to degraded output quality (e.g., repetition) in longer generations. When reading the transformers source, watch for ",{"type":44,"tag":104,"props":2313,"children":2315},{"className":2314},[],[2316],{"type":50,"value":2317},".float()",{"type":50,"value":221},{"type":44,"tag":104,"props":2320,"children":2322},{"className":2321},[],[2323],{"type":50,"value":2324},".to(torch.float32)",{"type":50,"value":2326},", or similar casts — they are there for a reason.",{"type":44,"tag":60,"props":2328,"children":2329},{},[2330],{"type":50,"value":2331},"However, do check for MLX ops that may be internally upcasting in their implementation.",{"type":44,"tag":87,"props":2333,"children":2335},{"id":2334},"dtype-contamination-after-conversion",[2336],{"type":50,"value":2337},"Dtype Contamination After Conversion",{"type":44,"tag":60,"props":2339,"children":2340},{},[2341],{"type":50,"value":2342},"A single float32 parameter in a non-quantized layer (norms, biases, gating weights) can silently poison the entire forward pass through MLX's dtype promotion rules. This causes no correctness errors — the model generates coherent text — but inference speed degrades dramatically (up to 5x slower) because all quantized matmuls run in float32 mode instead of bfloat16.",{"type":44,"tag":60,"props":2344,"children":2345},{},[2346],{"type":50,"value":2347},"Common causes:",{"type":44,"tag":363,"props":2349,"children":2350},{},[2351,2371],{"type":44,"tag":98,"props":2352,"children":2353},{},[2354,2356,2361,2363,2369],{"type":50,"value":2355},"Sanitize operations like ",{"type":44,"tag":104,"props":2357,"children":2359},{"className":2358},[],[2360],{"type":50,"value":1398},{"type":50,"value":2362}," promote bfloat16 norm weights to float32 (Python's ",{"type":44,"tag":104,"props":2364,"children":2366},{"className":2365},[],[2367],{"type":50,"value":2368},"1.0",{"type":50,"value":2370}," is float64\u002Ffloat32)",{"type":44,"tag":98,"props":2372,"children":2373},{},[2374,2376,2382,2383,2389],{"type":50,"value":2375},"Parameters initialized with ",{"type":44,"tag":104,"props":2377,"children":2379},{"className":2378},[],[2380],{"type":50,"value":2381},"mx.ones()",{"type":50,"value":437},{"type":44,"tag":104,"props":2384,"children":2386},{"className":2385},[],[2387],{"type":50,"value":2388},"mx.zeros()",{"type":50,"value":2390}," default to float32 and persist if the checkpoint doesn't include them",{"type":44,"tag":60,"props":2392,"children":2393},{},[2394,2396,2402],{"type":50,"value":2395},"The contamination path: float32 norm weight → ",{"type":44,"tag":104,"props":2397,"children":2399},{"className":2398},[],[2400],{"type":50,"value":2401},"rms_norm",{"type":50,"value":2403}," outputs float32 → quantized matmul receives float32 input → outputs float32 → residual addition promotes hidden state to float32 → every subsequent layer inherits float32.",{"type":44,"tag":60,"props":2405,"children":2406},{},[2407,2412],{"type":44,"tag":207,"props":2408,"children":2409},{},[2410],{"type":50,"value":2411},"Always verify output dtype after conversion and after quantization",{"type":50,"value":2413}," (see Phase 5).",{"type":44,"tag":87,"props":2415,"children":2417},{"id":2416},"hybrid-and-ssm-architectures",[2418],{"type":50,"value":2419},"Hybrid and SSM Architectures",{"type":44,"tag":60,"props":2421,"children":2422},{},[2423,2425,2431],{"type":50,"value":2424},"For models mixing layer types (e.g., Mamba + Attention) — see ",{"type":44,"tag":71,"props":2426,"children":2428},{"href":2427},"references\u002Fhybrid-and-ssm-models.md",[2429],{"type":50,"value":2430},"Hybrid and SSM Models",{"type":50,"value":2432}," for detailed guidance:",{"type":44,"tag":94,"props":2434,"children":2435},{},[2436,2447,2465,2470,2489],{"type":44,"tag":98,"props":2437,"children":2438},{},[2439,2440,2445],{"type":50,"value":1958},{"type":44,"tag":104,"props":2441,"children":2443},{"className":2442},[],[2444],{"type":50,"value":392},{"type":50,"value":2446}," config for per-layer architecture",{"type":44,"tag":98,"props":2448,"children":2449},{},[2450,2452],{"type":50,"value":2451},"Each layer may have MULTIPLE blocks (not just one):\n",{"type":44,"tag":363,"props":2453,"children":2454},{},[2455,2460],{"type":44,"tag":98,"props":2456,"children":2457},{},[2458],{"type":50,"value":2459},"Block 1: Mamba OR Attention",{"type":44,"tag":98,"props":2461,"children":2462},{},[2463],{"type":50,"value":2464},"Block 2: MoE + shared_mlp (for ALL layers)",{"type":44,"tag":98,"props":2466,"children":2467},{},[2468],{"type":50,"value":2469},"Both Mamba and Attention layers often share the same MoE\u002FMLP block",{"type":44,"tag":98,"props":2471,"children":2472},{},[2473,2479,2481,2487],{"type":44,"tag":104,"props":2474,"children":2476},{"className":2475},[],[2477],{"type":50,"value":2478},"make_cache()",{"type":50,"value":2480}," must return the right cache type per layer (e.g., ",{"type":44,"tag":104,"props":2482,"children":2484},{"className":2483},[],[2485],{"type":50,"value":2486},"CacheList(KVCache(), ArraysCache())",{"type":50,"value":2488}," for hybrid layers)",{"type":44,"tag":98,"props":2490,"children":2491},{},[2492,2494,2500,2501,2507],{"type":50,"value":2493},"CUDA-only dependencies (",{"type":44,"tag":104,"props":2495,"children":2497},{"className":2496},[],[2498],{"type":50,"value":2499},"mamba_ssm",{"type":50,"value":221},{"type":44,"tag":104,"props":2502,"children":2504},{"className":2503},[],[2505],{"type":50,"value":2506},"causal_conv1d",{"type":50,"value":2508},") won't install on macOS — transformers uses fallback implementations",{"type":44,"tag":87,"props":2510,"children":2512},{"id":2511},"_3d-moe-weights",[2513],{"type":50,"value":2514},"3D MoE Weights",{"type":44,"tag":60,"props":2516,"children":2517},{},[2518],{"type":50,"value":2519},"Expert weights may be stored as 3D tensors:",{"type":44,"tag":363,"props":2521,"children":2522},{},[2523,2534,2539],{"type":44,"tag":98,"props":2524,"children":2525},{},[2526,2528],{"type":50,"value":2527},"Shape: ",{"type":44,"tag":104,"props":2529,"children":2531},{"className":2530},[],[2532],{"type":50,"value":2533},"(num_experts, hidden_size, intermediate_size)",{"type":44,"tag":98,"props":2535,"children":2536},{},[2537],{"type":50,"value":2538},"Need proper reshape in sanitize()",{"type":44,"tag":98,"props":2540,"children":2541},{},[2542],{"type":50,"value":2543},"SwitchGLU expects specific shapes",{"type":44,"tag":87,"props":2545,"children":2547},{"id":2546},"debugging-checklist",[2548],{"type":50,"value":2549},"Debugging Checklist",{"type":44,"tag":60,"props":2551,"children":2552},{},[2553],{"type":50,"value":2554},"When predictions don't match:",{"type":44,"tag":94,"props":2556,"children":2557},{},[2558,2568,2578,2588,2598,2608,2626,2650],{"type":44,"tag":98,"props":2559,"children":2560},{},[2561,2566],{"type":44,"tag":207,"props":2562,"children":2563},{},[2564],{"type":50,"value":2565},"Embeddings",{"type":50,"value":2567}," - Should match exactly (mean, std)",{"type":44,"tag":98,"props":2569,"children":2570},{},[2571,2576],{"type":44,"tag":207,"props":2572,"children":2573},{},[2574],{"type":50,"value":2575},"After embedding scaling",{"type":50,"value":2577}," - Check multiplier applied",{"type":44,"tag":98,"props":2579,"children":2580},{},[2581,2586],{"type":44,"tag":207,"props":2582,"children":2583},{},[2584],{"type":50,"value":2585},"Layer-by-layer",{"type":50,"value":2587}," - Find first divergence point",{"type":44,"tag":98,"props":2589,"children":2590},{},[2591,2596],{"type":44,"tag":207,"props":2592,"children":2593},{},[2594],{"type":50,"value":2595},"Check config fields",{"type":50,"value":2597}," - position_embedding_type, multipliers",{"type":44,"tag":98,"props":2599,"children":2600},{},[2601,2606],{"type":44,"tag":207,"props":2602,"children":2603},{},[2604],{"type":50,"value":2605},"Layer structure",{"type":50,"value":2607}," - All blocks applied? (MoE often missed)",{"type":44,"tag":98,"props":2609,"children":2610},{},[2611,2616,2618,2624],{"type":44,"tag":207,"props":2612,"children":2613},{},[2614],{"type":50,"value":2615},"Attention scaling",{"type":50,"value":2617}," - Verify the scale factor matches transformers exactly (some models use non-standard formulas like ",{"type":44,"tag":104,"props":2619,"children":2621},{"className":2620},[],[2622],{"type":50,"value":2623},"(head_dim \u002F\u002F 2) ** -0.5",{"type":50,"value":2625},")",{"type":44,"tag":98,"props":2627,"children":2628},{},[2629,2634,2636,2641,2643,2648],{"type":44,"tag":207,"props":2630,"children":2631},{},[2632],{"type":50,"value":2633},"Precision casts",{"type":50,"value":2635}," - Check for ",{"type":44,"tag":104,"props":2637,"children":2639},{"className":2638},[],[2640],{"type":50,"value":2317},{"type":50,"value":2642}," \u002F ",{"type":44,"tag":104,"props":2644,"children":2646},{"className":2645},[],[2647],{"type":50,"value":2324},{"type":50,"value":2649}," in transformers code, especially in norms and activations (see Numerical Precision above)",{"type":44,"tag":98,"props":2651,"children":2652},{},[2653,2658],{"type":44,"tag":207,"props":2654,"children":2655},{},[2656],{"type":50,"value":2657},"Repetitive output",{"type":50,"value":2659}," - If the model generates repetitive or looping text, this may indicate a shape or transpose error in the attention or linear attention layers (e.g., head dimensions swapped, wrong reshape order), not a sampling issue",{"type":44,"tag":87,"props":2661,"children":2663},{"id":2662},"large-models",[2664],{"type":50,"value":1111},{"type":44,"tag":60,"props":2666,"children":2667},{},[2668],{"type":50,"value":2669},"Models that consume a significant fraction of available RAM require special handling:",{"type":44,"tag":363,"props":2671,"children":2672},{},[2673,2686,2698,2716,2727],{"type":44,"tag":98,"props":2674,"children":2675},{},[2676,2678,2684],{"type":50,"value":2677},"Set ",{"type":44,"tag":104,"props":2679,"children":2681},{"className":2680},[],[2682],{"type":50,"value":2683},"sudo sysctl iogpu.wired_limit_mb=\u003Cvalue>",{"type":50,"value":2685}," to increase the wired memory limit",{"type":44,"tag":98,"props":2687,"children":2688},{},[2689,2690,2696],{"type":50,"value":2677},{"type":44,"tag":104,"props":2691,"children":2693},{"className":2692},[],[2694],{"type":50,"value":2695},"MLX_METAL_FAST_SYNCH=1",{"type":50,"value":2697}," for faster GPU synchronization",{"type":44,"tag":98,"props":2699,"children":2700},{},[2701,2706,2708,2714],{"type":44,"tag":104,"props":2702,"children":2704},{"className":2703},[],[2705],{"type":50,"value":1141},{"type":50,"value":2707}," may OOM due to ",{"type":44,"tag":104,"props":2709,"children":2711},{"className":2710},[],[2712],{"type":50,"value":2713},"mx.async_eval",{"type":50,"value":2715}," double-buffering — use a manual generation loop",{"type":44,"tag":98,"props":2717,"children":2718},{},[2719,2721],{"type":50,"value":2720},"Very large MoE models suffer memory system thrashing even with selective ",{"type":44,"tag":104,"props":2722,"children":2724},{"className":2723},[],[2725],{"type":50,"value":2726},"gather_qmm",{"type":44,"tag":98,"props":2728,"children":2729},{},[2730,2732],{"type":50,"value":2731},"For models too large for one machine, use tensor parallelism via ",{"type":44,"tag":104,"props":2733,"children":2735},{"className":2734},[],[2736],{"type":50,"value":2737},"mlx.launch",{"type":44,"tag":60,"props":2739,"children":2740},{},[2741,2743,2747,2749,2753],{"type":50,"value":2742},"See ",{"type":44,"tag":71,"props":2744,"children":2745},{"href":1086},[2746],{"type":50,"value":1111},{"type":50,"value":2748}," and ",{"type":44,"tag":71,"props":2750,"children":2751},{"href":1116},[2752],{"type":50,"value":1119},{"type":50,"value":2754}," for detailed guidance.",{"type":44,"tag":87,"props":2756,"children":2758},{"id":2757},"changes-to-shared-mlx-lm-code",[2759],{"type":50,"value":2760},"Changes to Shared mlx-lm Code",{"type":44,"tag":60,"props":2762,"children":2763},{},[2764,2766,2772,2774,2780,2782,2788],{"type":50,"value":2765},"Some conversions require modifications to shared mlx-lm infrastructure (",{"type":44,"tag":104,"props":2767,"children":2769},{"className":2768},[],[2770],{"type":50,"value":2771},"ssm.py",{"type":50,"value":2773},", Metal kernels, ",{"type":44,"tag":104,"props":2775,"children":2777},{"className":2776},[],[2778],{"type":50,"value":2779},"cache.py",{"type":50,"value":2781},"), not just a new model file. These changes affect all models using the same code. See ",{"type":44,"tag":71,"props":2783,"children":2785},{"href":2784},"references\u002Fmlx-common-infrastructure.md",[2786],{"type":50,"value":2787},"Common Infrastructure Changes",{"type":50,"value":2789}," for guidance on localizing fixes and avoiding regressions.",{"type":44,"tag":53,"props":2791,"children":2793},{"id":2792},"general-guidance-and-common-pitfalls",[2794],{"type":50,"value":2795},"General Guidance and Common Pitfalls",{"type":44,"tag":363,"props":2797,"children":2798},{},[2799,2809,2828,2841,2846,2865,2870,2882],{"type":44,"tag":98,"props":2800,"children":2801},{},[2802,2807],{"type":44,"tag":207,"props":2803,"children":2804},{},[2805],{"type":50,"value":2806},"Minimize code comments.",{"type":50,"value":2808}," Do not add comments unless the code would be legitimately confusing without one (which should be very rare). Comments must be reviewed just like code, so every unnecessary comment increases review burden. The PR description and tests provide all the context a reviewer needs.",{"type":44,"tag":98,"props":2810,"children":2811},{},[2812,2814,2820,2822],{"type":50,"value":2813},"Greedy MLX decoding: ",{"type":44,"tag":104,"props":2815,"children":2817},{"className":2816},[],[2818],{"type":50,"value":2819},"sampler=lambda x: mx.argmax(x, axis=-1)",{"type":50,"value":2821},", not ",{"type":44,"tag":104,"props":2823,"children":2825},{"className":2824},[],[2826],{"type":50,"value":2827},"temperature=0",{"type":44,"tag":98,"props":2829,"children":2830},{},[2831,2833,2839],{"type":50,"value":2832},"Don't override ",{"type":44,"tag":104,"props":2834,"children":2836},{"className":2835},[],[2837],{"type":50,"value":2838},"eos_token_id",{"type":50,"value":2840}," - use generation_config",{"type":44,"tag":98,"props":2842,"children":2843},{},[2844],{"type":50,"value":2845},"Use separate tokenizers for transformers\u002FMLX tests",{"type":44,"tag":98,"props":2847,"children":2848},{},[2849,2855,2857,2863],{"type":44,"tag":104,"props":2850,"children":2852},{"className":2851},[],[2853],{"type":50,"value":2854},"apply_chat_template",{"type":50,"value":2856},": use ",{"type":44,"tag":104,"props":2858,"children":2860},{"className":2859},[],[2861],{"type":50,"value":2862},"tokenize=True, return_tensors=\"pt\"",{"type":50,"value":2864}," directly",{"type":44,"tag":98,"props":2866,"children":2867},{},[2868],{"type":50,"value":2869},"MPS limitations: use CPU for comparison tests",{"type":44,"tag":98,"props":2871,"children":2872},{},[2873,2875,2881],{"type":50,"value":2874},"Hybrid models: RoPE is often disabled (",{"type":44,"tag":104,"props":2876,"children":2878},{"className":2877},[],[2879],{"type":50,"value":2880},"position_embedding_type: \"nope\"",{"type":50,"value":2625},{"type":44,"tag":98,"props":2883,"children":2884},{},[2885],{"type":50,"value":2886},"Every layer may need MoE block, not just attention layers",{"type":44,"tag":53,"props":2888,"children":2890},{"id":2889},"output",[2891],{"type":50,"value":2892},"Output",{"type":44,"tag":60,"props":2894,"children":2895},{},[2896,2898,2904],{"type":50,"value":2897},"Upon success, please save the mlx implementation in the appropriate place inside ",{"type":44,"tag":104,"props":2899,"children":2901},{"className":2900},[],[2902],{"type":50,"value":2903},"mlx-lm\u002Fmlx_lm\u002Fmodels",{"type":50,"value":2905},", and generate a summary report with the results from the tests.",{"type":44,"tag":60,"props":2907,"children":2908},{},[2909],{"type":50,"value":2910},"On failure (only declare when you run out of ideas after multiple iterations), please report your analysis as well as the results from tests you run.",{"type":44,"tag":53,"props":2912,"children":2914},{"id":2913},"list-of-scripts",[2915],{"type":50,"value":2916},"List of scripts",{"type":44,"tag":60,"props":2918,"children":2919},{},[2920,2922,2928],{"type":50,"value":2921},"Bundled scripts in ",{"type":44,"tag":104,"props":2923,"children":2925},{"className":2924},[],[2926],{"type":50,"value":2927},"scripts\u002F",{"type":50,"value":2929}," (relative to this skill):",{"type":44,"tag":2931,"props":2932,"children":2933},"table",{},[2934,2953],{"type":44,"tag":2935,"props":2936,"children":2937},"thead",{},[2938],{"type":44,"tag":2939,"props":2940,"children":2941},"tr",{},[2942,2948],{"type":44,"tag":2943,"props":2944,"children":2945},"th",{},[2946],{"type":50,"value":2947},"Script",{"type":44,"tag":2943,"props":2949,"children":2950},{},[2951],{"type":50,"value":2952},"Purpose",{"type":44,"tag":2954,"props":2955,"children":2956},"tbody",{},[2957,2975,2992,3009,3026,3043,3060],{"type":44,"tag":2939,"props":2958,"children":2959},{},[2960,2970],{"type":44,"tag":2961,"props":2962,"children":2963},"td",{},[2964],{"type":44,"tag":104,"props":2965,"children":2967},{"className":2966},[],[2968],{"type":50,"value":2969},"resolve_dtype.py",{"type":44,"tag":2961,"props":2971,"children":2972},{},[2973],{"type":50,"value":2974},"Resolve expected runtime dtype from config or safetensors header (Phase 1, Phase 4 \u002F Test 2, Phase 5)",{"type":44,"tag":2939,"props":2976,"children":2977},{},[2978,2987],{"type":44,"tag":2961,"props":2979,"children":2980},{},[2981],{"type":44,"tag":104,"props":2982,"children":2984},{"className":2983},[],[2985],{"type":50,"value":2986},"check_dtype.py",{"type":44,"tag":2961,"props":2988,"children":2989},{},[2990],{"type":50,"value":2991},"Verify a model's forward-pass output dtype matches an expected value (Phase 4 \u002F Test 2, Phase 5)",{"type":44,"tag":2939,"props":2993,"children":2994},{},[2995,3004],{"type":44,"tag":2961,"props":2996,"children":2997},{},[2998],{"type":44,"tag":104,"props":2999,"children":3001},{"className":3000},[],[3002],{"type":50,"value":3003},"compare_generation.py",{"type":44,"tag":2961,"props":3005,"children":3006},{},[3007],{"type":50,"value":3008},"Full text generation comparison (Phase 4 \u002F Test 1)",{"type":44,"tag":2939,"props":3010,"children":3011},{},[3012,3021],{"type":44,"tag":2961,"props":3013,"children":3014},{},[3015],{"type":44,"tag":104,"props":3016,"children":3018},{"className":3017},[],[3019],{"type":50,"value":3020},"compare_predictions.py",{"type":44,"tag":2961,"props":3022,"children":3023},{},[3024],{"type":50,"value":3025},"Forward pass comparison with tolerance analysis and top-k overlap (Phase 4 \u002F Test 3)",{"type":44,"tag":2939,"props":3027,"children":3028},{},[3029,3038],{"type":44,"tag":2961,"props":3030,"children":3031},{},[3032],{"type":44,"tag":104,"props":3033,"children":3035},{"className":3034},[],[3036],{"type":50,"value":3037},"compare_layers.py",{"type":44,"tag":2961,"props":3039,"children":3040},{},[3041],{"type":50,"value":3042},"Per-layer hidden state + logits comparison (Phase 4 \u002F Test 4)",{"type":44,"tag":2939,"props":3044,"children":3045},{},[3046,3055],{"type":44,"tag":2961,"props":3047,"children":3048},{},[3049],{"type":44,"tag":104,"props":3050,"children":3052},{"className":3051},[],[3053],{"type":50,"value":3054},"debug_transformers.py",{"type":44,"tag":2961,"props":3056,"children":3057},{},[3058],{"type":50,"value":3059},"Analyze transformers model (embeddings, logits, top predictions)",{"type":44,"tag":2939,"props":3061,"children":3062},{},[3063,3072],{"type":44,"tag":2961,"props":3064,"children":3065},{},[3066],{"type":44,"tag":104,"props":3067,"children":3069},{"className":3068},[],[3070],{"type":50,"value":3071},"debug_mlx.py",{"type":44,"tag":2961,"props":3073,"children":3074},{},[3075,3077,3082],{"type":50,"value":3076},"Analyze MLX model (same output format as ",{"type":44,"tag":104,"props":3078,"children":3080},{"className":3079},[],[3081],{"type":50,"value":3054},{"type":50,"value":2625},{"type":44,"tag":60,"props":3084,"children":3085},{},[3086],{"type":44,"tag":207,"props":3087,"children":3088},{},[3089],{"type":50,"value":3090},"Recommended order for the comparison flow:",{"type":44,"tag":94,"props":3092,"children":3093},{},[3094,3104,3114],{"type":44,"tag":98,"props":3095,"children":3096},{},[3097,3102],{"type":44,"tag":104,"props":3098,"children":3100},{"className":3099},[],[3101],{"type":50,"value":3003},{"type":50,"value":3103}," — quick sanity check (is output coherent?)",{"type":44,"tag":98,"props":3105,"children":3106},{},[3107,3112],{"type":44,"tag":104,"props":3108,"children":3110},{"className":3109},[],[3111],{"type":50,"value":3020},{"type":50,"value":3113}," — numerical analysis (do predictions match?)",{"type":44,"tag":98,"props":3115,"children":3116},{},[3117,3122],{"type":44,"tag":104,"props":3118,"children":3120},{"className":3119},[],[3121],{"type":50,"value":3037},{"type":50,"value":3123}," — find divergence (where does it break?)",{"type":44,"tag":60,"props":3125,"children":3126},{},[3127],{"type":50,"value":3128},"Usage:",{"type":44,"tag":649,"props":3130,"children":3132},{"className":1158,"code":3131,"language":1160,"meta":654,"style":654},"# Resolve and verify dtype\npython scripts\u002Fresolve_dtype.py \u002Fpath\u002Fto\u002Fmodel\npython scripts\u002Fcheck_dtype.py \u002Fpath\u002Fto\u002Fmodel bfloat16\n\n# Full generation comparison\npython scripts\u002Fcompare_generation.py \u002Fpath\u002Fto\u002Fmodel --message \"What is 2+2?\"\n\n# Numerical comparison with tolerance analysis\npython scripts\u002Fcompare_predictions.py \u002Fpath\u002Fto\u002Fmodel \"The quick brown fox...\"\n\n# Per-layer + logits comparison\npython scripts\u002Fcompare_layers.py \u002Fpath\u002Fto\u002Fmodel \"The quick brown fox...\"\n",[3133],{"type":44,"tag":104,"props":3134,"children":3135},{"__ignoreMap":654},[3136,3144,3159,3178,3185,3193,3227,3234,3242,3271,3278,3286],{"type":44,"tag":660,"props":3137,"children":3138},{"class":662,"line":663},[3139],{"type":44,"tag":660,"props":3140,"children":3141},{"style":1311},[3142],{"type":50,"value":3143},"# Resolve and verify dtype\n",{"type":44,"tag":660,"props":3145,"children":3146},{"class":662,"line":672},[3147,3151,3155],{"type":44,"tag":660,"props":3148,"children":3149},{"style":1170},[3150],{"type":50,"value":653},{"type":44,"tag":660,"props":3152,"children":3153},{"style":1175},[3154],{"type":50,"value":1300},{"type":44,"tag":660,"props":3156,"children":3157},{"style":1175},[3158],{"type":50,"value":1305},{"type":44,"tag":660,"props":3160,"children":3161},{"class":662,"line":681},[3162,3166,3170,3174],{"type":44,"tag":660,"props":3163,"children":3164},{"style":1170},[3165],{"type":50,"value":653},{"type":44,"tag":660,"props":3167,"children":3168},{"style":1175},[3169],{"type":50,"value":1356},{"type":44,"tag":660,"props":3171,"children":3172},{"style":1175},[3173],{"type":50,"value":1361},{"type":44,"tag":660,"props":3175,"children":3176},{"style":1175},[3177],{"type":50,"value":1366},{"type":44,"tag":660,"props":3179,"children":3180},{"class":662,"line":690},[3181],{"type":44,"tag":660,"props":3182,"children":3183},{"emptyLinePlaceholder":703},[3184],{"type":50,"value":706},{"type":44,"tag":660,"props":3186,"children":3187},{"class":662,"line":699},[3188],{"type":44,"tag":660,"props":3189,"children":3190},{"style":1311},[3191],{"type":50,"value":3192},"# Full generation comparison\n",{"type":44,"tag":660,"props":3194,"children":3195},{"class":662,"line":31},[3196,3200,3205,3209,3214,3218,3223],{"type":44,"tag":660,"props":3197,"children":3198},{"style":1170},[3199],{"type":50,"value":653},{"type":44,"tag":660,"props":3201,"children":3202},{"style":1175},[3203],{"type":50,"value":3204}," scripts\u002Fcompare_generation.py",{"type":44,"tag":660,"props":3206,"children":3207},{"style":1175},[3208],{"type":50,"value":1361},{"type":44,"tag":660,"props":3210,"children":3211},{"style":1175},[3212],{"type":50,"value":3213}," --message",{"type":44,"tag":660,"props":3215,"children":3216},{"style":1191},[3217],{"type":50,"value":1194},{"type":44,"tag":660,"props":3219,"children":3220},{"style":1175},[3221],{"type":50,"value":3222},"What is 2+2?",{"type":44,"tag":660,"props":3224,"children":3225},{"style":1191},[3226],{"type":50,"value":1204},{"type":44,"tag":660,"props":3228,"children":3229},{"class":662,"line":717},[3230],{"type":44,"tag":660,"props":3231,"children":3232},{"emptyLinePlaceholder":703},[3233],{"type":50,"value":706},{"type":44,"tag":660,"props":3235,"children":3236},{"class":662,"line":726},[3237],{"type":44,"tag":660,"props":3238,"children":3239},{"style":1311},[3240],{"type":50,"value":3241},"# Numerical comparison with tolerance analysis\n",{"type":44,"tag":660,"props":3243,"children":3244},{"class":662,"line":735},[3245,3249,3254,3258,3262,3267],{"type":44,"tag":660,"props":3246,"children":3247},{"style":1170},[3248],{"type":50,"value":653},{"type":44,"tag":660,"props":3250,"children":3251},{"style":1175},[3252],{"type":50,"value":3253}," scripts\u002Fcompare_predictions.py",{"type":44,"tag":660,"props":3255,"children":3256},{"style":1175},[3257],{"type":50,"value":1361},{"type":44,"tag":660,"props":3259,"children":3260},{"style":1191},[3261],{"type":50,"value":1194},{"type":44,"tag":660,"props":3263,"children":3264},{"style":1175},[3265],{"type":50,"value":3266},"The quick brown fox...",{"type":44,"tag":660,"props":3268,"children":3269},{"style":1191},[3270],{"type":50,"value":1204},{"type":44,"tag":660,"props":3272,"children":3273},{"class":662,"line":744},[3274],{"type":44,"tag":660,"props":3275,"children":3276},{"emptyLinePlaceholder":703},[3277],{"type":50,"value":706},{"type":44,"tag":660,"props":3279,"children":3280},{"class":662,"line":752},[3281],{"type":44,"tag":660,"props":3282,"children":3283},{"style":1311},[3284],{"type":50,"value":3285},"# Per-layer + logits comparison\n",{"type":44,"tag":660,"props":3287,"children":3288},{"class":662,"line":761},[3289,3293,3297,3301,3305,3309],{"type":44,"tag":660,"props":3290,"children":3291},{"style":1170},[3292],{"type":50,"value":653},{"type":44,"tag":660,"props":3294,"children":3295},{"style":1175},[3296],{"type":50,"value":1496},{"type":44,"tag":660,"props":3298,"children":3299},{"style":1175},[3300],{"type":50,"value":1361},{"type":44,"tag":660,"props":3302,"children":3303},{"style":1191},[3304],{"type":50,"value":1194},{"type":44,"tag":660,"props":3306,"children":3307},{"style":1175},[3308],{"type":50,"value":3266},{"type":44,"tag":660,"props":3310,"children":3311},{"style":1191},[3312],{"type":50,"value":1204},{"type":44,"tag":53,"props":3314,"children":3316},{"id":3315},"references",[3317],{"type":50,"value":3318},"References",{"type":44,"tag":2931,"props":3320,"children":3321},{},[3322,3337],{"type":44,"tag":2935,"props":3323,"children":3324},{},[3325],{"type":44,"tag":2939,"props":3326,"children":3327},{},[3328,3333],{"type":44,"tag":2943,"props":3329,"children":3330},{},[3331],{"type":50,"value":3332},"Reference",{"type":44,"tag":2943,"props":3334,"children":3335},{},[3336],{"type":50,"value":2952},{"type":44,"tag":2954,"props":3338,"children":3339},{},[3340,3355,3370,3385,3401,3416,3431,3446,3461,3484],{"type":44,"tag":2939,"props":3341,"children":3342},{},[3343,3350],{"type":44,"tag":2961,"props":3344,"children":3345},{},[3346],{"type":44,"tag":71,"props":3347,"children":3348},{"href":252},[3349],{"type":50,"value":255},{"type":44,"tag":2961,"props":3351,"children":3352},{},[3353],{"type":50,"value":3354},"Hub search, config diffing, weight format detection",{"type":44,"tag":2939,"props":3356,"children":3357},{},[3358,3365],{"type":44,"tag":2961,"props":3359,"children":3360},{},[3361],{"type":44,"tag":71,"props":3362,"children":3363},{"href":823},[3364],{"type":50,"value":826},{"type":44,"tag":2961,"props":3366,"children":3367},{},[3368],{"type":50,"value":3369},"sanitize() patterns, idempotency, shared weights, ordering",{"type":44,"tag":2939,"props":3371,"children":3372},{},[3373,3380],{"type":44,"tag":2961,"props":3374,"children":3375},{},[3376],{"type":44,"tag":71,"props":3377,"children":3378},{"href":1078},[3379],{"type":50,"value":1081},{"type":44,"tag":2961,"props":3381,"children":3382},{},[3383],{"type":50,"value":3384},"mlx_lm.convert usage, quantize-first workflow",{"type":44,"tag":2939,"props":3386,"children":3387},{},[3388,3396],{"type":44,"tag":2961,"props":3389,"children":3390},{},[3391],{"type":44,"tag":71,"props":3392,"children":3393},{"href":542},[3394],{"type":50,"value":3395},"MLA",{"type":44,"tag":2961,"props":3397,"children":3398},{},[3399],{"type":50,"value":3400},"Multi-head Latent Attention, kv_b_proj splitting, rope_parameters",{"type":44,"tag":2939,"props":3402,"children":3403},{},[3404,3411],{"type":44,"tag":2961,"props":3405,"children":3406},{},[3407],{"type":44,"tag":71,"props":3408,"children":3409},{"href":2427},[3410],{"type":50,"value":2430},{"type":44,"tag":2961,"props":3412,"children":3413},{},[3414],{"type":50,"value":3415},"Cache construction, layer dispatch, CUDA-only deps",{"type":44,"tag":2939,"props":3417,"children":3418},{},[3419,3426],{"type":44,"tag":2961,"props":3420,"children":3421},{},[3422],{"type":44,"tag":71,"props":3423,"children":3424},{"href":1086},[3425],{"type":50,"value":1111},{"type":44,"tag":2961,"props":3427,"children":3428},{},[3429],{"type":50,"value":3430},"Wired memory, OOM workarounds, MoE memory pressure",{"type":44,"tag":2939,"props":3432,"children":3433},{},[3434,3441],{"type":44,"tag":2961,"props":3435,"children":3436},{},[3437],{"type":44,"tag":71,"props":3438,"children":3439},{"href":1116},[3440],{"type":50,"value":1119},{"type":44,"tag":2961,"props":3442,"children":3443},{},[3444],{"type":50,"value":3445},"mlx.launch, hostfiles, rsync, tensor parallelism",{"type":44,"tag":2939,"props":3447,"children":3448},{},[3449,3456],{"type":44,"tag":2961,"props":3450,"children":3451},{},[3452],{"type":44,"tag":71,"props":3453,"children":3454},{"href":2784},[3455],{"type":50,"value":2787},{"type":44,"tag":2961,"props":3457,"children":3458},{},[3459],{"type":50,"value":3460},"Safe modifications to shared mlx-lm code",{"type":44,"tag":2939,"props":3462,"children":3463},{},[3464,3471],{"type":44,"tag":2961,"props":3465,"children":3466},{},[3467],{"type":44,"tag":71,"props":3468,"children":3469},{"href":1818},[3470],{"type":50,"value":1821},{"type":44,"tag":2961,"props":3472,"children":3473},{},[3474,3476,3482],{"type":50,"value":3475},"Feature branches, targeting correct repo, ",{"type":44,"tag":104,"props":3477,"children":3479},{"className":3478},[],[3480],{"type":50,"value":3481},"gh pr edit",{"type":50,"value":3483}," tips",{"type":44,"tag":2939,"props":3485,"children":3486},{},[3487,3495],{"type":44,"tag":2961,"props":3488,"children":3489},{},[3490],{"type":44,"tag":71,"props":3491,"children":3492},{"href":624},[3493],{"type":50,"value":3494},"Code Simplification",{"type":44,"tag":2961,"props":3496,"children":3497},{},[3498],{"type":50,"value":3499},"Dead branch removal, reference model patterns",{"type":44,"tag":3501,"props":3502,"children":3503},"style",{},[3504],{"type":50,"value":3505},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":3507,"total":663},[3508],{"slug":4,"name":4,"fn":5,"description":6,"org":3509,"tags":3510,"stars":27,"repoUrl":28,"updatedAt":29},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3511,3512,3513,3514,3515],{"name":22,"slug":23,"type":14},{"name":25,"slug":26,"type":14},{"name":9,"slug":13,"type":14},{"name":19,"slug":20,"type":14},{"name":16,"slug":17,"type":14},{"items":3517,"total":3691},[3518,3539,3553,3570,3584,3597,3610,3625,3639,3649,3662,3676],{"slug":3519,"name":3519,"fn":3520,"description":3521,"org":3522,"tags":3523,"stars":3536,"repoUrl":3537,"updatedAt":3538},"train-sentence-transformers","train sentence-transformers models","Train or fine-tune sentence-transformers models across `SentenceTransformer` (bi-encoder; dense or static embedding model; for retrieval, similarity, clustering, classification, paraphrase mining, dedup, multimodal), `CrossEncoder` (reranker; pair scoring for two-stage retrieval \u002F pair classification), and `SparseEncoder` (SPLADE, sparse embedding model; for learned-sparse retrieval). Covers loss selection, hard-negative mining, evaluators, distillation, LoRA, Matryoshka, and Hugging Face Hub publishing. Use for any sentence-transformers training task.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3524,3527,3528,3531,3533],{"name":3525,"slug":3526,"type":14},"Deep Learning","deep-learning",{"name":9,"slug":13,"type":14},{"name":3529,"slug":3530,"type":14},"LLM","llm",{"name":3532,"slug":653,"type":14},"Python",{"name":3534,"slug":3535,"type":14},"Search","search",18914,"https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fsentence-transformers","2026-05-08T05:09:16.820066",{"slug":3540,"name":3540,"fn":3541,"description":3542,"org":3543,"tags":3544,"stars":3550,"repoUrl":3551,"updatedAt":3552},"trl-training","train and fine-tune LLMs with TRL","Train and fine-tune transformer language models using TRL (Transformers Reinforcement Learning). Supports SFT, DPO, GRPO, KTO, RLOO and Reward Model training via CLI commands.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3545,3546,3547,3548,3549],{"name":22,"slug":23,"type":14},{"name":3525,"slug":3526,"type":14},{"name":9,"slug":13,"type":14},{"name":3529,"slug":3530,"type":14},{"name":3532,"slug":653,"type":14},18850,"https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftrl","2026-04-06T18:25:32.746828",{"slug":3554,"name":3554,"fn":3555,"description":3556,"org":3557,"tags":3558,"stars":3567,"repoUrl":3568,"updatedAt":3569},"hf-cli","manage Hugging Face Hub resources via CLI","Hugging Face Hub CLI (`hf`) for downloading, uploading, and managing models, datasets, spaces, buckets, repos, papers, jobs, and more on the Hugging Face Hub. Use when: handling authentication; managing local cache; managing Hugging Face Buckets; running or scheduling jobs on Hugging Face infrastructure; managing Hugging Face repos; discussions and pull requests; browsing models, datasets and spaces; reading, searching, or browsing academic papers; managing collections; querying datasets; configuring spaces; setting up webhooks; or deploying and managing HF Inference Endpoints. Make sure to use this skill whenever the user mentions 'hf', 'huggingface', 'Hugging Face', 'huggingface-cli', or 'hugging face cli', or wants to do anything related to the Hugging Face ecosystem and to AI and ML in general. Also use for cloud storage needs like training checkpoints, data pipelines, or agent traces. Use even if the user doesn't explicitly ask for a CLI command. Replaces the deprecated `huggingface-cli`.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3559,3562,3565,3566],{"name":3560,"slug":3561,"type":14},"CLI","cli",{"name":3563,"slug":3564,"type":14},"Datasets","datasets",{"name":9,"slug":13,"type":14},{"name":3529,"slug":3530,"type":14},10861,"https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fskills","2026-04-06T18:25:34.020855",{"slug":3571,"name":3571,"fn":3572,"description":3573,"org":3574,"tags":3575,"stars":3567,"repoUrl":3568,"updatedAt":3583},"hf-cloud-aws-context-discovery","discover local AWS environment context","Discover the user's local AWS context (active profile, region, account ID, caller identity) at the start of any AWS task. Use this skill before any other AWS work — deploying to SageMaker, creating resources, calling AWS APIs, or anything that touches an AWS account. Use it especially when the user has not specified a region or profile explicitly, when they say things like \"use my AWS account\", \"deploy to AWS\", \"use my profile\", or when about to make any AWS CLI or SDK call. Never guess the region or account ID — always use this skill to read it from the local configuration first.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3576,3579,3580],{"name":3577,"slug":3578,"type":14},"AWS","aws",{"name":3560,"slug":3561,"type":14},{"name":3581,"slug":3582,"type":14},"Configuration","configuration","2026-07-08T05:55:33.716099",{"slug":3585,"name":3585,"fn":3586,"description":3587,"org":3588,"tags":3589,"stars":3567,"repoUrl":3568,"updatedAt":3596},"hf-cloud-python-env-setup","set up Python environments for AWS","Set up an isolated Python environment for SageMaker \u002F AWS work, with the right Python version and current boto3. Use this skill whenever Python code will be executed for a SageMaker deployment, training job, or any AWS automation — including when about to run `pip install`, when about to invoke `boto3`, when creating or activating a virtualenv, or when the user asks to \"set up the environment\". Never use system Python and never `pip install` into it. Always isolate. This skill prevents the most common failure modes: wrong Python version, dependency conflicts, and stale SDKs.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3590,3591,3592,3595],{"name":22,"slug":23,"type":14},{"name":3577,"slug":3578,"type":14},{"name":3593,"slug":3594,"type":14},"Engineering","engineering",{"name":3532,"slug":653,"type":14},"2026-07-08T05:55:32.505017",{"slug":3598,"name":3598,"fn":3599,"description":3600,"org":3601,"tags":3602,"stars":3567,"repoUrl":3568,"updatedAt":3609},"hf-cloud-sagemaker-deployment-planner","plan model deployments to Amazon SageMaker","Plan and coordinate the deployment of a model to Amazon SageMaker AI. Use this skill whenever the user wants to deploy, host, serve, or expose a model on SageMaker or AWS — including phrases like \"deploy a model\", \"host this LLM on AWS\", \"serve this embedding model\", \"deploy a reranker\", \"deploy a text-to-image \u002F diffusion model\", \"host this for async inference\", \"create an endpoint\", \"serve my fine-tuned model\", or any request that involves making a model available for inference on AWS. Use this even when the user is vague (e.g. \"I just want to get this running on AWS, you figure it out\"). Works for text-generation LLMs, embedding models, rerankers, classifiers, text-to-image \u002F diffusion models — picks the right serving stack and chooses between real-time and async inference. This is the entry-point skill for SageMaker deployment work — it asks clarifying questions, picks a deployment pathway, and coordinates the other deployment skills.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3603,3604,3605,3608],{"name":22,"slug":23,"type":14},{"name":3577,"slug":3578,"type":14},{"name":3606,"slug":3607,"type":14},"Deployment","deployment",{"name":9,"slug":13,"type":14},"2026-07-08T05:55:37.387689",{"slug":3611,"name":3611,"fn":3612,"description":3613,"org":3614,"tags":3615,"stars":3567,"repoUrl":3568,"updatedAt":3624},"hf-cloud-sagemaker-iam-preflight","configure SageMaker IAM roles","Ensure a usable SageMaker execution role exists before deploying or training. Use this skill whenever about to create a SageMaker endpoint, model, training job, or any resource that requires an execution role. Use it especially when the user has not provided a role ARN explicitly, when scripts are about to call `iam:CreateRole`, or when an AccessDenied error mentions an IAM action. Never blindly call `iam:CreateRole` — always check for existing roles first. This skill prevents the most common SageMaker deployment failure: trying to create IAM resources from an SSO principal that has no IAM write permissions.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3616,3617,3618,3621],{"name":22,"slug":23,"type":14},{"name":3577,"slug":3578,"type":14},{"name":3619,"slug":3620,"type":14},"Permissions","permissions",{"name":3622,"slug":3623,"type":14},"Security","security","2026-07-08T05:55:34.948771",{"slug":3626,"name":3626,"fn":3627,"description":3628,"org":3629,"tags":3630,"stars":3567,"repoUrl":3568,"updatedAt":3638},"hf-cloud-sagemaker-production-defaults","create production-ready SageMaker endpoints","Create a SageMaker endpoint (real-time or async) with autoscaling, CloudWatch alarms, and tagging enabled by default. Use this skill whenever about to create a SageMaker endpoint, write deployment code that calls `create_endpoint`, or finalize a deployment after the image URI and IAM role are known. Provides deploy.py for real-time endpoints and deploy_async.py for async endpoints (with genuine scale-to-zero support). This is the last step in the SageMaker deployment workflow. Never generate a bare `create_endpoint` call without these defaults — endpoints without autoscaling or alarms are demos, not deployments.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3631,3632,3633,3634,3635],{"name":22,"slug":23,"type":14},{"name":3577,"slug":3578,"type":14},{"name":3606,"slug":3607,"type":14},{"name":9,"slug":13,"type":14},{"name":3636,"slug":3637,"type":14},"Monitoring","monitoring","2026-07-08T05:55:38.664702",{"slug":3640,"name":3640,"fn":3641,"description":3642,"org":3643,"tags":3644,"stars":3567,"repoUrl":3568,"updatedAt":3648},"hf-cloud-serving-image-selection","select SageMaker serving containers","Pick the right serving container for a SageMaker model deployment and find its current image URI. Use this skill whenever about to deploy a model to a SageMaker endpoint and an image URI needs to be chosen — including when the user says \"deploy this LLM\", \"host this HuggingFace model\", \"serve this fine-tuned model\", \"deploy this embedding model\", \"host a reranker\", \"serve a sentence-transformers model\", or when about to hardcode any container URI in deployment code. HuggingFace-curated Deep Learning Containers are ALWAYS preferred: HuggingFace vLLM (LLMs and generative rerankers), HuggingFace vLLM-Omni (multimodal), TEI (embeddings\u002Fcross-encoder rerankers), HF Inference Toolkit (other transformers). Generic images (AWS vLLM, DJL-LMI, SGLang) are used only when no HuggingFace image is compatible — never merely because they carry a newer version. Never hardcode a container URI from memory and never default to TGI. Prevents stale-image failures and wrong-region URIs.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3645,3646,3647],{"name":22,"slug":23,"type":14},{"name":3577,"slug":3578,"type":14},{"name":3606,"slug":3607,"type":14},"2026-07-08T05:55:36.173465",{"slug":3650,"name":3650,"fn":3651,"description":3652,"org":3653,"tags":3654,"stars":3567,"repoUrl":3568,"updatedAt":3661},"hf-mcp","access Hugging Face Hub via MCP","Use Hugging Face Hub via MCP server tools. Search models, datasets, Spaces, papers. Get repo details, fetch documentation, run compute jobs, and use Gradio Spaces as AI tools. Available when connected to the HF MCP server.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3655,3656,3657,3658],{"name":3563,"slug":3564,"type":14},{"name":9,"slug":13,"type":14},{"name":3529,"slug":3530,"type":14},{"name":3659,"slug":3660,"type":14},"MCP","mcp","2026-04-06T18:25:50.364185",{"slug":3663,"name":3663,"fn":3664,"description":3665,"org":3666,"tags":3667,"stars":3567,"repoUrl":3568,"updatedAt":3675},"hf-mem","estimate memory for Hugging Face models","Hugging Face CLI to estimate the required memory to load Safetensors or GGUF model weights for inference from the Hugging Face Hub",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3668,3669,3670,3671,3672],{"name":22,"slug":23,"type":14},{"name":3560,"slug":3561,"type":14},{"name":9,"slug":13,"type":14},{"name":3529,"slug":3530,"type":14},{"name":3673,"slug":3674,"type":14},"Performance","performance","2026-06-13T07:23:57.101435",{"slug":3677,"name":3677,"fn":3678,"description":3679,"org":3680,"tags":3681,"stars":3567,"repoUrl":3568,"updatedAt":3690},"huggingface-best","find and compare Hugging Face models","Use when the user asks about finding the best, top, or recommended model for a task, wants to know what AI model to use, or wants to compare models by benchmark scores. Triggers on: \"best model for X\", \"what model should I use for\", \"top models for [task]\", \"which model runs on my laptop\u002Fmachine\u002Fdevice\", \"recommend a model for\", \"what LLM should I use for\", \"compare models for\", \"what's state of the art for\", or any question about choosing an AI model for a specific use case. Always use this skill when the user wants model recommendations or comparisons, even if they don't explicitly mention HuggingFace or benchmarks.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3682,3685,3686,3687],{"name":3683,"slug":3684,"type":14},"Analytics","analytics",{"name":9,"slug":13,"type":14},{"name":3529,"slug":3530,"type":14},{"name":3688,"slug":3689,"type":14},"Research","research","2026-04-24T05:09:45.870658",37]