[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-nvidia-tilegym-cutile-autotuning":3,"mdc-fkxg8y-key":34,"related-org-nvidia-tilegym-cutile-autotuning":2509,"related-repo-nvidia-tilegym-cutile-autotuning":2667},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":23,"repoUrl":24,"updatedAt":25,"license":26,"forks":27,"topics":28,"repo":29,"sourceUrl":32,"mdContent":33},"tilegym-cutile-autotuning","optimize CuTile autotuning configurations","Use when adding, modifying, optimizing, or debugging CuTile autotuning code. Trigger signals: `exhaustive_search` \u002F `replace_hints` \u002F `hints_fn` \u002F `cuda.tile.tune` in code, `autotune` in filenames, or correctness\u002Fperformance issues in autotuned CuTile kernels. Covers: tune-once\u002Fcache\u002Flaunch pattern, per-architecture configs (sm80–sm120), parameter space design (tile sizes, occupancy, num_ctas), and 7 common pitfalls with solutions.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},"nvidia","NVIDIA","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fnvidia.png",[12,16,17,20],{"name":13,"slug":14,"type":15},"Performance","performance","tag",{"name":9,"slug":8,"type":15},{"name":18,"slug":19,"type":15},"Engineering","engineering",{"name":21,"slug":22,"type":15},"Debugging","debugging",2473,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fskills","2026-07-14T05:29:51.879789","CC-BY-4.0 AND Apache-2.0",281,[],{"repoUrl":24,"stars":23,"forks":27,"topics":30,"description":31},[],"AI agent skills published by NVIDIA","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fskills\u002Ftree\u002FHEAD\u002Fskills\u002Ftilegym-cutile-autotuning","---\nname: tilegym-cutile-autotuning\ndescription: \"Use when adding, modifying, optimizing, or debugging CuTile autotuning code. Trigger signals: `exhaustive_search` \u002F `replace_hints` \u002F `hints_fn` \u002F `cuda.tile.tune` in code, `autotune` in filenames, or correctness\u002Fperformance issues in autotuned CuTile kernels. Covers: tune-once\u002Fcache\u002Flaunch pattern, per-architecture configs (sm80–sm120), parameter space design (tile sizes, occupancy, num_ctas), and 7 common pitfalls with solutions.\"\nlicense: CC-BY-4.0 AND Apache-2.0\n---\n\n# CuTile Autotuning\n\nAdd autotuning to CuTile kernels using the `exhaustive_search` API with tune-once\u002Fcache\u002Fdirect-launch pattern.\n\n## Instructions\n\nFollow the decision tree to classify the kernel, design a search space, implement the tune-once\u002Fcache\u002Flaunch pattern, and validate performance.\n\n1. **Classify** — use the Decision Tree to determine search dimensions (occupancy-only vs full tile search)\n2. **Design search space** — select the matching template from `references\u002Fkernel-type-templates.md`; prune to ≤ 30 configs in the final code via arch filters (directed exploration probes may temporarily exceed this — see Design Philosophy)\n3. **Implement** — add `exhaustive_search` + cache + `ct.launch` following the Step-by-Step Workflow; handle in-place writes with split-buffer if needed\n4. **Test** — run correctness with autotune enabled and with `DISABLE_AUTOTUNE=1`\n5. **Validate** — A\u002FB benchmark against fixed best-known config; see `references\u002Fsearch-strategies.md`\n6. **Shrink** — prune dead-weight configs that never win, targeting ≤ 8 configs per architecture to minimize compilation cost (Step 10)\n\n## Task Router — Jump to What You Need\n\n| What are you trying to do? | Go to |\n|---|---|\n| Add autotune to a new kernel (most common) | Quick Reference below → Workflow: Adding Autotune → `references\u002Fkernel-type-templates.md` (pick by kernel type: T1=elementwise, T2=in-place, T3=matmul, T4=persistent, T5=FMHA, T6=FP8, T7=grouped GEMM, T8=varlen attention, T9=dual-GEMM fusion) |\n| Debug: data corruption \u002F wrong results after first run | Pitfall #1 (In-Place Kernel) |\n| Debug: autotune taking 5+ minutes | Pitfall #2 (Compilation Timeout) |\n| Debug: search space generator returning zero configs | Pitfall #5 first; also check arch filters, size guards, and `num_ctas` constraints |\n| Optimize an existing autotune config | Workflow: Optimizing an Existing Config |\n\n## Quick Reference — Occupancy-Only Autotune (Tune-Once\u002FCache\u002FLaunch)\n\nMost CuTile kernels (elementwise, reduction, LayerNorm) need only occupancy tuning. Copy this pattern:\n\n```python\nfrom types import SimpleNamespace\nfrom cuda.tile.tune import exhaustive_search\nimport cuda.tile as ct\nimport torch\n\ndef _my_autotune_configs():\n    for occ in [1, 2, 4, 8]:\n        yield SimpleNamespace(occupancy=occ)\n\n# Module-level cache: tune once, launch fast forever after\n_autotune_cache = {}\n\ndef my_op(x, output):\n    stream = torch.cuda.current_stream()\n    NUM_SM = torch.cuda.get_device_properties(x.device).multi_processor_count\n\n    # Cache key: anything that affects optimal config (use str() for device)\n    cache_key = (x.shape, x.dtype, str(x.device))\n\n    if cache_key not in _autotune_cache:\n        configs = list(_my_autotune_configs())\n        result = exhaustive_search(\n            configs,\n            stream,\n            grid_fn=lambda cfg: (min(NUM_SM * cfg.occupancy, M), 1, 1),\n            kernel=my_kernel,\n            args_fn=lambda cfg: (x, output, ...),\n            hints_fn=lambda cfg: {\"occupancy\": cfg.occupancy},\n        )\n        best_cfg = result.best.config\n        tuned_kernel = my_kernel.replace_hints(occupancy=best_cfg.occupancy)\n        _autotune_cache[cache_key] = (best_cfg, tuned_kernel)  # cache BOTH\n\n    cfg, tuned_kernel = _autotune_cache[cache_key]\n    grid = (min(NUM_SM * cfg.occupancy, M), 1, 1)\n    ct.launch(stream, grid, tuned_kernel, (x, output, ...))\n```\n\nKey rules:\n- **Tune once, cache, launch directly** — `exhaustive_search` runs only on first call per shape; subsequent calls use cached config + `ct.launch` with zero overhead\n- For in-place kernels use split-buffer during search (separate input\u002Foutput tensors)\n- Keep ≤ 30 configs in final code (see Design Philosophy for temporary directed probes)\n- `exhaustive_search` requires a `Sequence` (list\u002Ftuple) — convert generators with `list()`\n- **Search space must include the original fixed config** — this guarantees autotuning never makes performance worse\n\n**When to use this pattern**: Kernel has fixed block size (not tile-size tunable). Includes: elementwise (SwiGLU, GeGLU), reduction (RMSNorm, LayerNorm), RoPE, and persistent kernels with heuristic block sizes (grouped GEMM).\n\nFor complex kernels (matmul with tile sizes, FMHA, FP8 with num_ctas), read the full guide below + [`kernel-type-templates.md`](references\u002Fkernel-type-templates.md).\n\n> **⚠️ Three pitfalls catch almost everyone — check before submitting:**\n> - **`replace_hints` on hot path?** → Cache BOTH config AND kernel object from `exhaustive_search`. Calling `replace_hints()` every invocation recompiles (100–500× slower) → Pitfall #7\n> - **In-place kernel** (writes back to input tensor)? → MUST use split-buffer pattern during search → Pitfall #1\n> - **Search space empty?** → Check arch filters and `num_ctas` constraints → Pitfall #5\n\n> **Minimum coverage**: On sm100+, FMHA\u002Fmatmul\u002Fvarlen search spaces must include both `num_ctas=1` and `num_ctas=2`. For core dimensions (tile sizes, occupancy), keep at least 2 distinct values even if unsure which is better — let `exhaustive_search` decide.\n\n> **When to stop tuning**: A mean speedup in [0.98, 1.02] means your *current* search space isn't helping — but doesn't mean no config will help. Before stopping, check whether you've covered the key dimensions for this kernel type (consult `references\u002Fkernel-type-templates.md`). If the search space already covers the template's recommended dimensions and the best result is still noise-floor, then stop — further micro-adjustments won't help. If key dimensions are missing (e.g., never tried `num_ctas=2` for a dual-GEMM kernel), expand the search space rather than giving up.\n>\n> Once correctness tests pass and the autotuned kernel shows speedup over the fixed-config baseline, **stop — do not re-run to \"confirm\".** GPU kernel timing fluctuates ±5–10 % between invocations due to clock scaling and OS scheduling; a subsequent timing dip does not mean your code is wrong.\n>\n> To improve speedup, only modify the autotune search space (configs, tile sizes, occupancy, num_ctas). Do not modify other code (Python wrapper, stream management, etc.) to chase speedup — kernel performance is determined by the config selection, not by host-side code.\n\n## Reading Guide\n\n- **Occupancy-only kernels** (elementwise, reduction, persistent with fixed block sizes): Quick Reference + Pitfall Checklist is sufficient — skip `references\u002F` docs. For in-place kernels, also read Pitfall #1.\n- **Complex kernels** (matmul with tunable tile sizes, FMHA, FP8 with num_ctas): Quick Reference → Decision Tree → API Reference → Step-by-Step Workflow → relevant `references\u002F` docs.\n\n**5-step summary**: Classify kernel → Design search space ([`parameter-space-design.md`](references\u002Fparameter-space-design.md)) → Implement using template ([`kernel-type-templates.md`](references\u002Fkernel-type-templates.md)) → Validate with A\u002FB test → Check Pitfall Checklist.\n\n**Reading references**: Read only the reference relevant to your kernel type — e.g., for FMHA, read the Template 5 section in `references\u002Fkernel-type-templates.md`; for hardware constraints, read only the target architecture's section. Avoid reading all references end-to-end when a targeted lookup suffices.\n\n## Design Philosophy\n\n**Build a small, precise search space bottom-up — not a large space trimmed down.** CuTile compilation is much heavier than Triton (~0.5-1s per config), so the **final code** should contain ≤ 30 configs. The approach is: classify the kernel type first, then construct only the relevant configs for that type and architecture.\n\n**Directed exploration during development**: If the initial template configs yield speedup \u003C 1.0, you may run a *temporary* larger probe (30–100 configs) via `bash + python3 -c` to identify which dimensions matter — but this probe must be **directional**, not a blind cartesian product. Use the kernel type classification to decide *which* dimensions to vary (e.g. for dual-GEMM, probe `num_ctas × occupancy` while fixing tile sizes; for FMHA, probe `TILE_M × num_ctas` while fixing TILE_N). Once the probe identifies the winning region, lock the final code's search space to ≤ 8 top candidates. Do NOT write the large probe into the source file — it is a one-shot diagnostic tool.\n\n## Decision Tree: What Search Dimensions Does This Kernel Need?\n\nAll kernels should have autotuning added. The question is not *whether* to autotune, but *what dimensions* to search:\n\n```\nWhat type of kernel is this?\n├── Compute-bound (matmul, GEMM, FMHA) → Does it have multiple tunable dimensions (tile sizes)?\n│   ├── YES → Is it a fused multi-GEMM kernel (dual-GEMM, e.g. Linear+GLUAct)?\n│   │   ├── YES → Template 9: low occupancy (1–2), conservative tiles (2× SHMEM\u002Fregister pressure)\n│   │   └── NO  → Full search: TILE_M × TILE_N × (TILE_K) × occupancy × num_ctas\n│   │             (see matmul\u002FFMHA templates in kernel-type-templates.md)\n│   └── NO  → Occupancy-only search: [1, 2, 4, 8]\n│             (see Quick Reference above)\n├── Balanced (LayerNorm, reduction + compute) →\n│   Occupancy-only search: [1, 2, 4, 8]\n│   Expected benefit: 2-15%\n└── Memory-bound (CE Loss, pure elementwise) →\n    Occupancy-only search: [1, 2, 4, 8]\n    Expected benefit: 0-15% (varies by kernel; zero-cost after tuning)\n```\n\n**Why memory-bound kernels only search occupancy (not num_ctas or tile sizes)**:\n- **`num_ctas` has zero benefit**: `num_ctas > 1` enables TMA multicast, where multiple CTAs share tile data in shared memory (e.g., matmul A\u002FB tiles reused across CTAs). Memory-bound kernels use per-element `ct.gather`\u002F`ct.scatter` with no tile reuse — multi-CTA cooperation adds overhead with no data sharing benefit.\n- **Tile sizes are pre-determined**: BLOCK_SIZE for memory-bound kernels is determined by offline sweep (e.g., 1024 is globally optimal on B200 across [256, 512, 1024, 2048, 4096, 8192]). This is a constant, not a runtime tunable.\n- **Occupancy is the only effective knob**: Higher occupancy lets the GPU hide memory latency by switching to another CTA while one is stalled on a memory request.\n\n> **Evidence — CE Loss experiment**: A 12-config search (occupancy × num_ctas) on Cross-Entropy Loss yielded only 2.5% gain (0.79x → 0.81x vs Triton). The `num_ctas` dimension contributed nothing; the result was reverted because compilation cost outweighed the marginal benefit. Occupancy-only (4 configs) achieves the same result at 3x less compilation time.\n\n**Note on memory-bound kernels**: Adding occupancy-only autotune is always worthwhile because:\n- The tune-once\u002Fcache\u002Flaunch pattern has zero runtime overhead after the first call\n- The search space is tiny (4 configs, ~2-4s compilation)\n- Even small improvements have value at scale\n\n## Occupancy Selection Guide\n\nOccupancy controls how many CTAs run concurrently per SM. Use this as a starting point when designing the occupancy search space:\n\n| Occupancy Range | Best For | Example Kernels |\n|-----------------|----------|-----------------|\n| 1–4 | Compute-bound (heavy math) | Complex transforms, matmul |\n| 4–8 | Balanced (GEMM, TMA) | Matrix multiply, FMHA |\n| 8–16 | Memory-bound (reductions) | Softmax, LayerNorm |\n| 16–32 | Very light (copies, casts) | Type conversions, elementwise |\n\nUse these ranges to seed your initial search space. For occupancy-only kernels, `[1, 2, 4, 8]` covers most cases — see Quick Reference above.\n\n## exhaustive_search API Reference\n\nSee [references\u002Fapi-reference.md](references\u002Fapi-reference.md) for the full\n`exhaustive_search` API surface — current signature, `TuningResult`, the\ntune-once\u002Fcache\u002Flaunch pattern, `replace_hints`, kernel hints, `search_space`\ndesign, and `grid_fn` patterns.\n\n## Step-by-Step Workflow\n\nSee [references\u002Fworkflow.md](references\u002Fworkflow.md) for the end-to-end\nworkflow — adding autotune to a new kernel, handling existing\nmulti-architecture configs, integration with `torch.autograd.Function`,\ncross-backend config transfer (Triton → CuTile), and optimizing an existing\nconfig.\n\n## Pitfall Checklist\n\nSee [references\u002Fpitfalls.md](references\u002Fpitfalls.md) for the full list of\ncommon pitfalls — in-place data corruption, compilation timeout, cold-cache\nperformance skew, NCU profiling interference, `search_space` generator\nexhaustion, FP8 precision loss, and `replace_hints` recompilation on hot\npaths.\n\n## Scope and Boundaries\n\nThis skill covers *only* autotune configuration: search space design, `exhaustive_search` invocation, caching, and `ct.launch` with tuned hints. It does **not** modify kernel code.\n\n**In scope** (autotune config):\n- Search space generator functions\n- `exhaustive_search()` calls and result handling\n- `kernel.replace_hints()` for applying tuned hints\n- Cache logic (key design, dict management)\n- `ct.launch()` with tuned kernel\n- `DISABLE_AUTOTUNE` fallback path\n\n**Out of scope** (kernel code modifications — do NOT make these changes):\n- Math flags (flush_to_zero, rounding_mode)\n- Performance Hints (slice_hint, buffer_depth, copy_config)\n- Memory access patterns (2D→1D gather\u002Fscatter conversion)\n- Codegen optimizations (safe_offs → padding_value)\n- Algorithm changes (K-loop split, load balancing)\n\n## Further Optimization Suggestions\n\nAfter adding autotuning, the following kernel-level optimizations may yield additional gains. These are *outside the scope of this skill* — mention them to the user as potential next steps, but do not implement them as part of autotuning:\n\n- **Math flags**: `flush_to_zero=True` + `rounding_mode=APPROX` can provide 34-72% improvement for FMHA-class kernels (set via environment variables `TILEIR_ENABLE_FTZ=1 TILEIR_ENABLE_APPROX=1` or in kernel code). *Causal chain*: larger tiles initially *decrease* performance by 18-43% due to subnormal handling overhead; enabling FTZ+APPROX rescues this and flips the result to +34-72%. Math flags are therefore a *prerequisite* for large-tile configs to be effective on FMHA-class kernels.\n- **Performance Hints**: `slice_hint`, `buffer_depth`, `copy_config` — requires modifying kernel IR code\n- **Memory access patterns**: Using TMA loads (`ct.load`) instead of `ct.gather`; removing unnecessary bounds checks (`check_bounds=False` when safe)\n- **Codegen quality**: Using `padding_value` parameter instead of manual `ct.where` masking; removing `safe_offs`\n- **Algorithm restructuring**: K-loop split, load balancing, algebraic simplification\n\n## Differences from Triton Autotune\n\nKey differences: Triton uses `@triton.autotune` decorator with `Config(...)` objects; CuTile uses `exhaustive_search()` with `SimpleNamespace` configs + separate cache + `ct.launch`. CuTile has no `num_warps`\u002F`num_stages` (compiler decides) — only tile sizes + `occupancy` + `num_ctas`. CuTile compilation is heavier (keep ≤30 configs in final code). CuTile cache is user-managed in-memory (no automatic persistence). CuTile separates `args_fn` (kernel args) from `hints_fn` (compiler hints).\n\n## Reference Documents\n\n| Category | Document | Content |\n|----------|----------|---------|\n| **API Reference** | [`api-reference.md`](references\u002Fapi-reference.md) | `exhaustive_search` signature, `TuningResult`, tune-once\u002Fcache\u002Flaunch pattern, `replace_hints`, kernel hints, `search_space` design, `grid_fn` patterns |\n| **Workflow** | [`workflow.md`](references\u002Fworkflow.md) | End-to-end workflow: adding autotune to a new kernel, multi-architecture configs, `torch.autograd.Function` integration, Triton→CuTile transfer, optimizing existing configs |\n| **Pitfalls** | [`pitfalls.md`](references\u002Fpitfalls.md) | Common pitfalls: in-place corruption, compilation timeout, cold-cache skew, NCU interference, `search_space` exhaustion, FP8 precision, `replace_hints` recompilation |\n| **Parameter Design** | [`parameter-space-design.md`](references\u002Fparameter-space-design.md) | Per-kernel-type parameter spaces, cross-arch patterns, grid_fn patterns, pruning rules |\n| **Search Strategies** | [`search-strategies.md`](references\u002Fsearch-strategies.md) | Exhaustive search, A\u002FB test methodology, DISABLE_AUTOTUNE pattern |\n| **Templates** | [`kernel-type-templates.md`](references\u002Fkernel-type-templates.md) | Copy-paste autotune templates for 8 kernel types |\n| **Hardware** | [`hardware-constraints.md`](references\u002Fhardware-constraints.md) | Per-architecture constraints, tile size ranges, num_ctas rules, TMA requirements |\n\n## Source Code References\n\nKey files: `ops\u002Fcutile\u002Fmatmul.py` (matmul autotune), `ops\u002Fcutile\u002Fattention.py` (FMHA autotune), `suites\u002Funsloth\u002Fcutile\u002Fct_ops.py` (shared `autotune_configs()` occupancy=[1,2,4,8]), `suites\u002Funsloth\u002Fcutile\u002Fswiglu.py` (elementwise example), `suites\u002Funsloth\u002Fcutile\u002Frope_embedding.py` (split-buffer pattern), `suites\u002Funsloth\u002Fcutile\u002Fgrouped_gemm.py` (persistent GEMM, occupancy-only).\n\n## Worked Examples\n\nEach example shows the **before → after** pattern: `fixed_launch.py` (hardcoded `ct.launch`) and `autotuned_launch.py` (refactored to tune-once\u002Fcache\u002Flaunch).\n\n| Directory | Kernel | Autotune Pattern | Complexity | Key Teaching Point |\n|-----------|--------|-----------------|------------|-------------------|\n| [`assets\u002Fexamples\u002F01_rmsnorm_occupancy_only\u002F`](assets\u002Fexamples\u002F01_rmsnorm_occupancy_only\u002F) | RMSNorm (reduction) | Occupancy-only `[1,2,4,8]` | Low | Most common pattern — no tile tuning, just find best occupancy. Grid = `NUM_SM * cfg.occupancy`. Not in-place. |\n| [`assets\u002Fexamples\u002F02_matmul_full_search\u002F`](assets\u002Fexamples\u002F02_matmul_full_search\u002F) | GEMM C=A@B | Full: `TILE_M\u002FN\u002FK` + `occupancy` + `num_ctas` (sm90+) | High | Compute-bound kernel with multiple tunable dimensions. `args_fn` passes tile sizes as `ct.Constant[int]`. `grid_fn` depends on `cfg`. ≤30 configs. |\n| [`assets\u002Fexamples\u002F03_rope_inplace_splitbuffer\u002F`](assets\u002Fexamples\u002F03_rope_inplace_splitbuffer\u002F) | RoPE embedding (in-place) | Occupancy-only, with split-buffer | Medium | In-place kernel MUST use split-buffer during search to avoid corruption. Search writes to scratch; final `ct.launch` uses real in-place args. |\n",{"data":35,"body":36},{"name":4,"description":6,"license":26},{"type":37,"children":38},"root",[39,48,63,70,75,176,182,290,296,301,634,639,711,721,738,811,847,906,912,950,982,999,1005,1022,1077,1083,1102,1112,1122,1191,1211,1221,1239,1245,1250,1349,1362,1368,1418,1424,1443,1449,1474,1480,1513,1523,1580,1590,1618,1624,1636,1800,1806,1894,1900,2175,2181,2249,2255,2290,2503],{"type":40,"tag":41,"props":42,"children":44},"element","h1",{"id":43},"cutile-autotuning",[45],{"type":46,"value":47},"text","CuTile Autotuning",{"type":40,"tag":49,"props":50,"children":51},"p",{},[52,54,61],{"type":46,"value":53},"Add autotuning to CuTile kernels using the ",{"type":40,"tag":55,"props":56,"children":58},"code",{"className":57},[],[59],{"type":46,"value":60},"exhaustive_search",{"type":46,"value":62}," API with tune-once\u002Fcache\u002Fdirect-launch pattern.",{"type":40,"tag":64,"props":65,"children":67},"h2",{"id":66},"instructions",[68],{"type":46,"value":69},"Instructions",{"type":40,"tag":49,"props":71,"children":72},{},[73],{"type":46,"value":74},"Follow the decision tree to classify the kernel, design a search space, implement the tune-once\u002Fcache\u002Flaunch pattern, and validate performance.",{"type":40,"tag":76,"props":77,"children":78},"ol",{},[79,91,109,134,150,166],{"type":40,"tag":80,"props":81,"children":82},"li",{},[83,89],{"type":40,"tag":84,"props":85,"children":86},"strong",{},[87],{"type":46,"value":88},"Classify",{"type":46,"value":90}," — use the Decision Tree to determine search dimensions (occupancy-only vs full tile search)",{"type":40,"tag":80,"props":92,"children":93},{},[94,99,101,107],{"type":40,"tag":84,"props":95,"children":96},{},[97],{"type":46,"value":98},"Design search space",{"type":46,"value":100}," — select the matching template from ",{"type":40,"tag":55,"props":102,"children":104},{"className":103},[],[105],{"type":46,"value":106},"references\u002Fkernel-type-templates.md",{"type":46,"value":108},"; prune to ≤ 30 configs in the final code via arch filters (directed exploration probes may temporarily exceed this — see Design Philosophy)",{"type":40,"tag":80,"props":110,"children":111},{},[112,117,119,124,126,132],{"type":40,"tag":84,"props":113,"children":114},{},[115],{"type":46,"value":116},"Implement",{"type":46,"value":118}," — add ",{"type":40,"tag":55,"props":120,"children":122},{"className":121},[],[123],{"type":46,"value":60},{"type":46,"value":125}," + cache + ",{"type":40,"tag":55,"props":127,"children":129},{"className":128},[],[130],{"type":46,"value":131},"ct.launch",{"type":46,"value":133}," following the Step-by-Step Workflow; handle in-place writes with split-buffer if needed",{"type":40,"tag":80,"props":135,"children":136},{},[137,142,144],{"type":40,"tag":84,"props":138,"children":139},{},[140],{"type":46,"value":141},"Test",{"type":46,"value":143}," — run correctness with autotune enabled and with ",{"type":40,"tag":55,"props":145,"children":147},{"className":146},[],[148],{"type":46,"value":149},"DISABLE_AUTOTUNE=1",{"type":40,"tag":80,"props":151,"children":152},{},[153,158,160],{"type":40,"tag":84,"props":154,"children":155},{},[156],{"type":46,"value":157},"Validate",{"type":46,"value":159}," — A\u002FB benchmark against fixed best-known config; see ",{"type":40,"tag":55,"props":161,"children":163},{"className":162},[],[164],{"type":46,"value":165},"references\u002Fsearch-strategies.md",{"type":40,"tag":80,"props":167,"children":168},{},[169,174],{"type":40,"tag":84,"props":170,"children":171},{},[172],{"type":46,"value":173},"Shrink",{"type":46,"value":175}," — prune dead-weight configs that never win, targeting ≤ 8 configs per architecture to minimize compilation cost (Step 10)",{"type":40,"tag":64,"props":177,"children":179},{"id":178},"task-router-jump-to-what-you-need",[180],{"type":46,"value":181},"Task Router — Jump to What You Need",{"type":40,"tag":183,"props":184,"children":185},"table",{},[186,205],{"type":40,"tag":187,"props":188,"children":189},"thead",{},[190],{"type":40,"tag":191,"props":192,"children":193},"tr",{},[194,200],{"type":40,"tag":195,"props":196,"children":197},"th",{},[198],{"type":46,"value":199},"What are you trying to do?",{"type":40,"tag":195,"props":201,"children":202},{},[203],{"type":46,"value":204},"Go to",{"type":40,"tag":206,"props":207,"children":208},"tbody",{},[209,230,243,256,277],{"type":40,"tag":191,"props":210,"children":211},{},[212,218],{"type":40,"tag":213,"props":214,"children":215},"td",{},[216],{"type":46,"value":217},"Add autotune to a new kernel (most common)",{"type":40,"tag":213,"props":219,"children":220},{},[221,223,228],{"type":46,"value":222},"Quick Reference below → Workflow: Adding Autotune → ",{"type":40,"tag":55,"props":224,"children":226},{"className":225},[],[227],{"type":46,"value":106},{"type":46,"value":229}," (pick by kernel type: T1=elementwise, T2=in-place, T3=matmul, T4=persistent, T5=FMHA, T6=FP8, T7=grouped GEMM, T8=varlen attention, T9=dual-GEMM fusion)",{"type":40,"tag":191,"props":231,"children":232},{},[233,238],{"type":40,"tag":213,"props":234,"children":235},{},[236],{"type":46,"value":237},"Debug: data corruption \u002F wrong results after first run",{"type":40,"tag":213,"props":239,"children":240},{},[241],{"type":46,"value":242},"Pitfall #1 (In-Place Kernel)",{"type":40,"tag":191,"props":244,"children":245},{},[246,251],{"type":40,"tag":213,"props":247,"children":248},{},[249],{"type":46,"value":250},"Debug: autotune taking 5+ minutes",{"type":40,"tag":213,"props":252,"children":253},{},[254],{"type":46,"value":255},"Pitfall #2 (Compilation Timeout)",{"type":40,"tag":191,"props":257,"children":258},{},[259,264],{"type":40,"tag":213,"props":260,"children":261},{},[262],{"type":46,"value":263},"Debug: search space generator returning zero configs",{"type":40,"tag":213,"props":265,"children":266},{},[267,269,275],{"type":46,"value":268},"Pitfall #5 first; also check arch filters, size guards, and ",{"type":40,"tag":55,"props":270,"children":272},{"className":271},[],[273],{"type":46,"value":274},"num_ctas",{"type":46,"value":276}," constraints",{"type":40,"tag":191,"props":278,"children":279},{},[280,285],{"type":40,"tag":213,"props":281,"children":282},{},[283],{"type":46,"value":284},"Optimize an existing autotune config",{"type":40,"tag":213,"props":286,"children":287},{},[288],{"type":46,"value":289},"Workflow: Optimizing an Existing Config",{"type":40,"tag":64,"props":291,"children":293},{"id":292},"quick-reference-occupancy-only-autotune-tune-oncecachelaunch",[294],{"type":46,"value":295},"Quick Reference — Occupancy-Only Autotune (Tune-Once\u002FCache\u002FLaunch)",{"type":40,"tag":49,"props":297,"children":298},{},[299],{"type":46,"value":300},"Most CuTile kernels (elementwise, reduction, LayerNorm) need only occupancy tuning. Copy this pattern:",{"type":40,"tag":302,"props":303,"children":308},"pre",{"className":304,"code":305,"language":306,"meta":307,"style":307},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","from types import SimpleNamespace\nfrom cuda.tile.tune import exhaustive_search\nimport cuda.tile as ct\nimport torch\n\ndef _my_autotune_configs():\n    for occ in [1, 2, 4, 8]:\n        yield SimpleNamespace(occupancy=occ)\n\n# Module-level cache: tune once, launch fast forever after\n_autotune_cache = {}\n\ndef my_op(x, output):\n    stream = torch.cuda.current_stream()\n    NUM_SM = torch.cuda.get_device_properties(x.device).multi_processor_count\n\n    # Cache key: anything that affects optimal config (use str() for device)\n    cache_key = (x.shape, x.dtype, str(x.device))\n\n    if cache_key not in _autotune_cache:\n        configs = list(_my_autotune_configs())\n        result = exhaustive_search(\n            configs,\n            stream,\n            grid_fn=lambda cfg: (min(NUM_SM * cfg.occupancy, M), 1, 1),\n            kernel=my_kernel,\n            args_fn=lambda cfg: (x, output, ...),\n            hints_fn=lambda cfg: {\"occupancy\": cfg.occupancy},\n        )\n        best_cfg = result.best.config\n        tuned_kernel = my_kernel.replace_hints(occupancy=best_cfg.occupancy)\n        _autotune_cache[cache_key] = (best_cfg, tuned_kernel)  # cache BOTH\n\n    cfg, tuned_kernel = _autotune_cache[cache_key]\n    grid = (min(NUM_SM * cfg.occupancy, M), 1, 1)\n    ct.launch(stream, grid, tuned_kernel, (x, output, ...))\n","python","",[309],{"type":40,"tag":55,"props":310,"children":311},{"__ignoreMap":307},[312,323,332,341,350,360,369,378,387,395,404,413,421,430,439,448,456,465,474,482,491,500,509,518,527,536,545,554,563,572,581,590,599,607,616,625],{"type":40,"tag":313,"props":314,"children":317},"span",{"class":315,"line":316},"line",1,[318],{"type":40,"tag":313,"props":319,"children":320},{},[321],{"type":46,"value":322},"from types import SimpleNamespace\n",{"type":40,"tag":313,"props":324,"children":326},{"class":315,"line":325},2,[327],{"type":40,"tag":313,"props":328,"children":329},{},[330],{"type":46,"value":331},"from cuda.tile.tune import exhaustive_search\n",{"type":40,"tag":313,"props":333,"children":335},{"class":315,"line":334},3,[336],{"type":40,"tag":313,"props":337,"children":338},{},[339],{"type":46,"value":340},"import cuda.tile as ct\n",{"type":40,"tag":313,"props":342,"children":344},{"class":315,"line":343},4,[345],{"type":40,"tag":313,"props":346,"children":347},{},[348],{"type":46,"value":349},"import torch\n",{"type":40,"tag":313,"props":351,"children":353},{"class":315,"line":352},5,[354],{"type":40,"tag":313,"props":355,"children":357},{"emptyLinePlaceholder":356},true,[358],{"type":46,"value":359},"\n",{"type":40,"tag":313,"props":361,"children":363},{"class":315,"line":362},6,[364],{"type":40,"tag":313,"props":365,"children":366},{},[367],{"type":46,"value":368},"def _my_autotune_configs():\n",{"type":40,"tag":313,"props":370,"children":372},{"class":315,"line":371},7,[373],{"type":40,"tag":313,"props":374,"children":375},{},[376],{"type":46,"value":377},"    for occ in [1, 2, 4, 8]:\n",{"type":40,"tag":313,"props":379,"children":381},{"class":315,"line":380},8,[382],{"type":40,"tag":313,"props":383,"children":384},{},[385],{"type":46,"value":386},"        yield SimpleNamespace(occupancy=occ)\n",{"type":40,"tag":313,"props":388,"children":390},{"class":315,"line":389},9,[391],{"type":40,"tag":313,"props":392,"children":393},{"emptyLinePlaceholder":356},[394],{"type":46,"value":359},{"type":40,"tag":313,"props":396,"children":398},{"class":315,"line":397},10,[399],{"type":40,"tag":313,"props":400,"children":401},{},[402],{"type":46,"value":403},"# Module-level cache: tune once, launch fast forever after\n",{"type":40,"tag":313,"props":405,"children":407},{"class":315,"line":406},11,[408],{"type":40,"tag":313,"props":409,"children":410},{},[411],{"type":46,"value":412},"_autotune_cache = {}\n",{"type":40,"tag":313,"props":414,"children":416},{"class":315,"line":415},12,[417],{"type":40,"tag":313,"props":418,"children":419},{"emptyLinePlaceholder":356},[420],{"type":46,"value":359},{"type":40,"tag":313,"props":422,"children":424},{"class":315,"line":423},13,[425],{"type":40,"tag":313,"props":426,"children":427},{},[428],{"type":46,"value":429},"def my_op(x, output):\n",{"type":40,"tag":313,"props":431,"children":433},{"class":315,"line":432},14,[434],{"type":40,"tag":313,"props":435,"children":436},{},[437],{"type":46,"value":438},"    stream = torch.cuda.current_stream()\n",{"type":40,"tag":313,"props":440,"children":442},{"class":315,"line":441},15,[443],{"type":40,"tag":313,"props":444,"children":445},{},[446],{"type":46,"value":447},"    NUM_SM = torch.cuda.get_device_properties(x.device).multi_processor_count\n",{"type":40,"tag":313,"props":449,"children":451},{"class":315,"line":450},16,[452],{"type":40,"tag":313,"props":453,"children":454},{"emptyLinePlaceholder":356},[455],{"type":46,"value":359},{"type":40,"tag":313,"props":457,"children":459},{"class":315,"line":458},17,[460],{"type":40,"tag":313,"props":461,"children":462},{},[463],{"type":46,"value":464},"    # Cache key: anything that affects optimal config (use str() for device)\n",{"type":40,"tag":313,"props":466,"children":468},{"class":315,"line":467},18,[469],{"type":40,"tag":313,"props":470,"children":471},{},[472],{"type":46,"value":473},"    cache_key = (x.shape, x.dtype, str(x.device))\n",{"type":40,"tag":313,"props":475,"children":477},{"class":315,"line":476},19,[478],{"type":40,"tag":313,"props":479,"children":480},{"emptyLinePlaceholder":356},[481],{"type":46,"value":359},{"type":40,"tag":313,"props":483,"children":485},{"class":315,"line":484},20,[486],{"type":40,"tag":313,"props":487,"children":488},{},[489],{"type":46,"value":490},"    if cache_key not in _autotune_cache:\n",{"type":40,"tag":313,"props":492,"children":494},{"class":315,"line":493},21,[495],{"type":40,"tag":313,"props":496,"children":497},{},[498],{"type":46,"value":499},"        configs = list(_my_autotune_configs())\n",{"type":40,"tag":313,"props":501,"children":503},{"class":315,"line":502},22,[504],{"type":40,"tag":313,"props":505,"children":506},{},[507],{"type":46,"value":508},"        result = exhaustive_search(\n",{"type":40,"tag":313,"props":510,"children":512},{"class":315,"line":511},23,[513],{"type":40,"tag":313,"props":514,"children":515},{},[516],{"type":46,"value":517},"            configs,\n",{"type":40,"tag":313,"props":519,"children":521},{"class":315,"line":520},24,[522],{"type":40,"tag":313,"props":523,"children":524},{},[525],{"type":46,"value":526},"            stream,\n",{"type":40,"tag":313,"props":528,"children":530},{"class":315,"line":529},25,[531],{"type":40,"tag":313,"props":532,"children":533},{},[534],{"type":46,"value":535},"            grid_fn=lambda cfg: (min(NUM_SM * cfg.occupancy, M), 1, 1),\n",{"type":40,"tag":313,"props":537,"children":539},{"class":315,"line":538},26,[540],{"type":40,"tag":313,"props":541,"children":542},{},[543],{"type":46,"value":544},"            kernel=my_kernel,\n",{"type":40,"tag":313,"props":546,"children":548},{"class":315,"line":547},27,[549],{"type":40,"tag":313,"props":550,"children":551},{},[552],{"type":46,"value":553},"            args_fn=lambda cfg: (x, output, ...),\n",{"type":40,"tag":313,"props":555,"children":557},{"class":315,"line":556},28,[558],{"type":40,"tag":313,"props":559,"children":560},{},[561],{"type":46,"value":562},"            hints_fn=lambda cfg: {\"occupancy\": cfg.occupancy},\n",{"type":40,"tag":313,"props":564,"children":566},{"class":315,"line":565},29,[567],{"type":40,"tag":313,"props":568,"children":569},{},[570],{"type":46,"value":571},"        )\n",{"type":40,"tag":313,"props":573,"children":575},{"class":315,"line":574},30,[576],{"type":40,"tag":313,"props":577,"children":578},{},[579],{"type":46,"value":580},"        best_cfg = result.best.config\n",{"type":40,"tag":313,"props":582,"children":584},{"class":315,"line":583},31,[585],{"type":40,"tag":313,"props":586,"children":587},{},[588],{"type":46,"value":589},"        tuned_kernel = my_kernel.replace_hints(occupancy=best_cfg.occupancy)\n",{"type":40,"tag":313,"props":591,"children":593},{"class":315,"line":592},32,[594],{"type":40,"tag":313,"props":595,"children":596},{},[597],{"type":46,"value":598},"        _autotune_cache[cache_key] = (best_cfg, tuned_kernel)  # cache BOTH\n",{"type":40,"tag":313,"props":600,"children":602},{"class":315,"line":601},33,[603],{"type":40,"tag":313,"props":604,"children":605},{"emptyLinePlaceholder":356},[606],{"type":46,"value":359},{"type":40,"tag":313,"props":608,"children":610},{"class":315,"line":609},34,[611],{"type":40,"tag":313,"props":612,"children":613},{},[614],{"type":46,"value":615},"    cfg, tuned_kernel = _autotune_cache[cache_key]\n",{"type":40,"tag":313,"props":617,"children":619},{"class":315,"line":618},35,[620],{"type":40,"tag":313,"props":621,"children":622},{},[623],{"type":46,"value":624},"    grid = (min(NUM_SM * cfg.occupancy, M), 1, 1)\n",{"type":40,"tag":313,"props":626,"children":628},{"class":315,"line":627},36,[629],{"type":40,"tag":313,"props":630,"children":631},{},[632],{"type":46,"value":633},"    ct.launch(stream, grid, tuned_kernel, (x, output, ...))\n",{"type":40,"tag":49,"props":635,"children":636},{},[637],{"type":46,"value":638},"Key rules:",{"type":40,"tag":640,"props":641,"children":642},"ul",{},[643,667,672,677,701],{"type":40,"tag":80,"props":644,"children":645},{},[646,651,653,658,660,665],{"type":40,"tag":84,"props":647,"children":648},{},[649],{"type":46,"value":650},"Tune once, cache, launch directly",{"type":46,"value":652}," — ",{"type":40,"tag":55,"props":654,"children":656},{"className":655},[],[657],{"type":46,"value":60},{"type":46,"value":659}," runs only on first call per shape; subsequent calls use cached config + ",{"type":40,"tag":55,"props":661,"children":663},{"className":662},[],[664],{"type":46,"value":131},{"type":46,"value":666}," with zero overhead",{"type":40,"tag":80,"props":668,"children":669},{},[670],{"type":46,"value":671},"For in-place kernels use split-buffer during search (separate input\u002Foutput tensors)",{"type":40,"tag":80,"props":673,"children":674},{},[675],{"type":46,"value":676},"Keep ≤ 30 configs in final code (see Design Philosophy for temporary directed probes)",{"type":40,"tag":80,"props":678,"children":679},{},[680,685,687,693,695],{"type":40,"tag":55,"props":681,"children":683},{"className":682},[],[684],{"type":46,"value":60},{"type":46,"value":686}," requires a ",{"type":40,"tag":55,"props":688,"children":690},{"className":689},[],[691],{"type":46,"value":692},"Sequence",{"type":46,"value":694}," (list\u002Ftuple) — convert generators with ",{"type":40,"tag":55,"props":696,"children":698},{"className":697},[],[699],{"type":46,"value":700},"list()",{"type":40,"tag":80,"props":702,"children":703},{},[704,709],{"type":40,"tag":84,"props":705,"children":706},{},[707],{"type":46,"value":708},"Search space must include the original fixed config",{"type":46,"value":710}," — this guarantees autotuning never makes performance worse",{"type":40,"tag":49,"props":712,"children":713},{},[714,719],{"type":40,"tag":84,"props":715,"children":716},{},[717],{"type":46,"value":718},"When to use this pattern",{"type":46,"value":720},": Kernel has fixed block size (not tile-size tunable). Includes: elementwise (SwiGLU, GeGLU), reduction (RMSNorm, LayerNorm), RoPE, and persistent kernels with heuristic block sizes (grouped GEMM).",{"type":40,"tag":49,"props":722,"children":723},{},[724,726,736],{"type":46,"value":725},"For complex kernels (matmul with tile sizes, FMHA, FP8 with num_ctas), read the full guide below + ",{"type":40,"tag":727,"props":728,"children":729},"a",{"href":106},[730],{"type":40,"tag":55,"props":731,"children":733},{"className":732},[],[734],{"type":46,"value":735},"kernel-type-templates.md",{"type":46,"value":737},".",{"type":40,"tag":739,"props":740,"children":741},"blockquote",{},[742,750],{"type":40,"tag":49,"props":743,"children":744},{},[745],{"type":40,"tag":84,"props":746,"children":747},{},[748],{"type":46,"value":749},"⚠️ Three pitfalls catch almost everyone — check before submitting:",{"type":40,"tag":640,"props":751,"children":752},{},[753,784,794],{"type":40,"tag":80,"props":754,"children":755},{},[756,767,769,774,776,782],{"type":40,"tag":84,"props":757,"children":758},{},[759,765],{"type":40,"tag":55,"props":760,"children":762},{"className":761},[],[763],{"type":46,"value":764},"replace_hints",{"type":46,"value":766}," on hot path?",{"type":46,"value":768}," → Cache BOTH config AND kernel object from ",{"type":40,"tag":55,"props":770,"children":772},{"className":771},[],[773],{"type":46,"value":60},{"type":46,"value":775},". Calling ",{"type":40,"tag":55,"props":777,"children":779},{"className":778},[],[780],{"type":46,"value":781},"replace_hints()",{"type":46,"value":783}," every invocation recompiles (100–500× slower) → Pitfall #7",{"type":40,"tag":80,"props":785,"children":786},{},[787,792],{"type":40,"tag":84,"props":788,"children":789},{},[790],{"type":46,"value":791},"In-place kernel",{"type":46,"value":793}," (writes back to input tensor)? → MUST use split-buffer pattern during search → Pitfall #1",{"type":40,"tag":80,"props":795,"children":796},{},[797,802,804,809],{"type":40,"tag":84,"props":798,"children":799},{},[800],{"type":46,"value":801},"Search space empty?",{"type":46,"value":803}," → Check arch filters and ",{"type":40,"tag":55,"props":805,"children":807},{"className":806},[],[808],{"type":46,"value":274},{"type":46,"value":810}," constraints → Pitfall #5",{"type":40,"tag":739,"props":812,"children":813},{},[814],{"type":40,"tag":49,"props":815,"children":816},{},[817,822,824,830,832,838,840,845],{"type":40,"tag":84,"props":818,"children":819},{},[820],{"type":46,"value":821},"Minimum coverage",{"type":46,"value":823},": On sm100+, FMHA\u002Fmatmul\u002Fvarlen search spaces must include both ",{"type":40,"tag":55,"props":825,"children":827},{"className":826},[],[828],{"type":46,"value":829},"num_ctas=1",{"type":46,"value":831}," and ",{"type":40,"tag":55,"props":833,"children":835},{"className":834},[],[836],{"type":46,"value":837},"num_ctas=2",{"type":46,"value":839},". For core dimensions (tile sizes, occupancy), keep at least 2 distinct values even if unsure which is better — let ",{"type":40,"tag":55,"props":841,"children":843},{"className":842},[],[844],{"type":46,"value":60},{"type":46,"value":846}," decide.",{"type":40,"tag":739,"props":848,"children":849},{},[850,889,901],{"type":40,"tag":49,"props":851,"children":852},{},[853,858,860,865,867,873,875,880,882,887],{"type":40,"tag":84,"props":854,"children":855},{},[856],{"type":46,"value":857},"When to stop tuning",{"type":46,"value":859},": A mean speedup in ",{"type":40,"tag":313,"props":861,"children":862},{},[863],{"type":46,"value":864},"0.98, 1.02",{"type":46,"value":866}," means your ",{"type":40,"tag":868,"props":869,"children":870},"em",{},[871],{"type":46,"value":872},"current",{"type":46,"value":874}," search space isn't helping — but doesn't mean no config will help. Before stopping, check whether you've covered the key dimensions for this kernel type (consult ",{"type":40,"tag":55,"props":876,"children":878},{"className":877},[],[879],{"type":46,"value":106},{"type":46,"value":881},"). If the search space already covers the template's recommended dimensions and the best result is still noise-floor, then stop — further micro-adjustments won't help. If key dimensions are missing (e.g., never tried ",{"type":40,"tag":55,"props":883,"children":885},{"className":884},[],[886],{"type":46,"value":837},{"type":46,"value":888}," for a dual-GEMM kernel), expand the search space rather than giving up.",{"type":40,"tag":49,"props":890,"children":891},{},[892,894,899],{"type":46,"value":893},"Once correctness tests pass and the autotuned kernel shows speedup over the fixed-config baseline, ",{"type":40,"tag":84,"props":895,"children":896},{},[897],{"type":46,"value":898},"stop — do not re-run to \"confirm\".",{"type":46,"value":900}," GPU kernel timing fluctuates ±5–10 % between invocations due to clock scaling and OS scheduling; a subsequent timing dip does not mean your code is wrong.",{"type":40,"tag":49,"props":902,"children":903},{},[904],{"type":46,"value":905},"To improve speedup, only modify the autotune search space (configs, tile sizes, occupancy, num_ctas). Do not modify other code (Python wrapper, stream management, etc.) to chase speedup — kernel performance is determined by the config selection, not by host-side code.",{"type":40,"tag":64,"props":907,"children":909},{"id":908},"reading-guide",[910],{"type":46,"value":911},"Reading Guide",{"type":40,"tag":640,"props":913,"children":914},{},[915,933],{"type":40,"tag":80,"props":916,"children":917},{},[918,923,925,931],{"type":40,"tag":84,"props":919,"children":920},{},[921],{"type":46,"value":922},"Occupancy-only kernels",{"type":46,"value":924}," (elementwise, reduction, persistent with fixed block sizes): Quick Reference + Pitfall Checklist is sufficient — skip ",{"type":40,"tag":55,"props":926,"children":928},{"className":927},[],[929],{"type":46,"value":930},"references\u002F",{"type":46,"value":932}," docs. For in-place kernels, also read Pitfall #1.",{"type":40,"tag":80,"props":934,"children":935},{},[936,941,943,948],{"type":40,"tag":84,"props":937,"children":938},{},[939],{"type":46,"value":940},"Complex kernels",{"type":46,"value":942}," (matmul with tunable tile sizes, FMHA, FP8 with num_ctas): Quick Reference → Decision Tree → API Reference → Step-by-Step Workflow → relevant ",{"type":40,"tag":55,"props":944,"children":946},{"className":945},[],[947],{"type":46,"value":930},{"type":46,"value":949}," docs.",{"type":40,"tag":49,"props":951,"children":952},{},[953,958,960,970,972,980],{"type":40,"tag":84,"props":954,"children":955},{},[956],{"type":46,"value":957},"5-step summary",{"type":46,"value":959},": Classify kernel → Design search space (",{"type":40,"tag":727,"props":961,"children":963},{"href":962},"references\u002Fparameter-space-design.md",[964],{"type":40,"tag":55,"props":965,"children":967},{"className":966},[],[968],{"type":46,"value":969},"parameter-space-design.md",{"type":46,"value":971},") → Implement using template (",{"type":40,"tag":727,"props":973,"children":974},{"href":106},[975],{"type":40,"tag":55,"props":976,"children":978},{"className":977},[],[979],{"type":46,"value":735},{"type":46,"value":981},") → Validate with A\u002FB test → Check Pitfall Checklist.",{"type":40,"tag":49,"props":983,"children":984},{},[985,990,992,997],{"type":40,"tag":84,"props":986,"children":987},{},[988],{"type":46,"value":989},"Reading references",{"type":46,"value":991},": Read only the reference relevant to your kernel type — e.g., for FMHA, read the Template 5 section in ",{"type":40,"tag":55,"props":993,"children":995},{"className":994},[],[996],{"type":46,"value":106},{"type":46,"value":998},"; for hardware constraints, read only the target architecture's section. Avoid reading all references end-to-end when a targeted lookup suffices.",{"type":40,"tag":64,"props":1000,"children":1002},{"id":1001},"design-philosophy",[1003],{"type":46,"value":1004},"Design Philosophy",{"type":40,"tag":49,"props":1006,"children":1007},{},[1008,1013,1015,1020],{"type":40,"tag":84,"props":1009,"children":1010},{},[1011],{"type":46,"value":1012},"Build a small, precise search space bottom-up — not a large space trimmed down.",{"type":46,"value":1014}," CuTile compilation is much heavier than Triton (~0.5-1s per config), so the ",{"type":40,"tag":84,"props":1016,"children":1017},{},[1018],{"type":46,"value":1019},"final code",{"type":46,"value":1021}," should contain ≤ 30 configs. The approach is: classify the kernel type first, then construct only the relevant configs for that type and architecture.",{"type":40,"tag":49,"props":1023,"children":1024},{},[1025,1030,1032,1037,1039,1045,1047,1052,1054,1059,1061,1067,1069,1075],{"type":40,"tag":84,"props":1026,"children":1027},{},[1028],{"type":46,"value":1029},"Directed exploration during development",{"type":46,"value":1031},": If the initial template configs yield speedup \u003C 1.0, you may run a ",{"type":40,"tag":868,"props":1033,"children":1034},{},[1035],{"type":46,"value":1036},"temporary",{"type":46,"value":1038}," larger probe (30–100 configs) via ",{"type":40,"tag":55,"props":1040,"children":1042},{"className":1041},[],[1043],{"type":46,"value":1044},"bash + python3 -c",{"type":46,"value":1046}," to identify which dimensions matter — but this probe must be ",{"type":40,"tag":84,"props":1048,"children":1049},{},[1050],{"type":46,"value":1051},"directional",{"type":46,"value":1053},", not a blind cartesian product. Use the kernel type classification to decide ",{"type":40,"tag":868,"props":1055,"children":1056},{},[1057],{"type":46,"value":1058},"which",{"type":46,"value":1060}," dimensions to vary (e.g. for dual-GEMM, probe ",{"type":40,"tag":55,"props":1062,"children":1064},{"className":1063},[],[1065],{"type":46,"value":1066},"num_ctas × occupancy",{"type":46,"value":1068}," while fixing tile sizes; for FMHA, probe ",{"type":40,"tag":55,"props":1070,"children":1072},{"className":1071},[],[1073],{"type":46,"value":1074},"TILE_M × num_ctas",{"type":46,"value":1076}," while fixing TILE_N). Once the probe identifies the winning region, lock the final code's search space to ≤ 8 top candidates. Do NOT write the large probe into the source file — it is a one-shot diagnostic tool.",{"type":40,"tag":64,"props":1078,"children":1080},{"id":1079},"decision-tree-what-search-dimensions-does-this-kernel-need",[1081],{"type":46,"value":1082},"Decision Tree: What Search Dimensions Does This Kernel Need?",{"type":40,"tag":49,"props":1084,"children":1085},{},[1086,1088,1093,1095,1100],{"type":46,"value":1087},"All kernels should have autotuning added. The question is not ",{"type":40,"tag":868,"props":1089,"children":1090},{},[1091],{"type":46,"value":1092},"whether",{"type":46,"value":1094}," to autotune, but ",{"type":40,"tag":868,"props":1096,"children":1097},{},[1098],{"type":46,"value":1099},"what dimensions",{"type":46,"value":1101}," to search:",{"type":40,"tag":302,"props":1103,"children":1107},{"className":1104,"code":1106,"language":46},[1105],"language-text","What type of kernel is this?\n├── Compute-bound (matmul, GEMM, FMHA) → Does it have multiple tunable dimensions (tile sizes)?\n│   ├── YES → Is it a fused multi-GEMM kernel (dual-GEMM, e.g. Linear+GLUAct)?\n│   │   ├── YES → Template 9: low occupancy (1–2), conservative tiles (2× SHMEM\u002Fregister pressure)\n│   │   └── NO  → Full search: TILE_M × TILE_N × (TILE_K) × occupancy × num_ctas\n│   │             (see matmul\u002FFMHA templates in kernel-type-templates.md)\n│   └── NO  → Occupancy-only search: [1, 2, 4, 8]\n│             (see Quick Reference above)\n├── Balanced (LayerNorm, reduction + compute) →\n│   Occupancy-only search: [1, 2, 4, 8]\n│   Expected benefit: 2-15%\n└── Memory-bound (CE Loss, pure elementwise) →\n    Occupancy-only search: [1, 2, 4, 8]\n    Expected benefit: 0-15% (varies by kernel; zero-cost after tuning)\n",[1108],{"type":40,"tag":55,"props":1109,"children":1110},{"__ignoreMap":307},[1111],{"type":46,"value":1106},{"type":40,"tag":49,"props":1113,"children":1114},{},[1115,1120],{"type":40,"tag":84,"props":1116,"children":1117},{},[1118],{"type":46,"value":1119},"Why memory-bound kernels only search occupancy (not num_ctas or tile sizes)",{"type":46,"value":1121},":",{"type":40,"tag":640,"props":1123,"children":1124},{},[1125,1164,1181],{"type":40,"tag":80,"props":1126,"children":1127},{},[1128,1138,1140,1146,1148,1154,1156,1162],{"type":40,"tag":84,"props":1129,"children":1130},{},[1131,1136],{"type":40,"tag":55,"props":1132,"children":1134},{"className":1133},[],[1135],{"type":46,"value":274},{"type":46,"value":1137}," has zero benefit",{"type":46,"value":1139},": ",{"type":40,"tag":55,"props":1141,"children":1143},{"className":1142},[],[1144],{"type":46,"value":1145},"num_ctas > 1",{"type":46,"value":1147}," enables TMA multicast, where multiple CTAs share tile data in shared memory (e.g., matmul A\u002FB tiles reused across CTAs). Memory-bound kernels use per-element ",{"type":40,"tag":55,"props":1149,"children":1151},{"className":1150},[],[1152],{"type":46,"value":1153},"ct.gather",{"type":46,"value":1155},"\u002F",{"type":40,"tag":55,"props":1157,"children":1159},{"className":1158},[],[1160],{"type":46,"value":1161},"ct.scatter",{"type":46,"value":1163}," with no tile reuse — multi-CTA cooperation adds overhead with no data sharing benefit.",{"type":40,"tag":80,"props":1165,"children":1166},{},[1167,1172,1174,1179],{"type":40,"tag":84,"props":1168,"children":1169},{},[1170],{"type":46,"value":1171},"Tile sizes are pre-determined",{"type":46,"value":1173},": BLOCK_SIZE for memory-bound kernels is determined by offline sweep (e.g., 1024 is globally optimal on B200 across ",{"type":40,"tag":313,"props":1175,"children":1176},{},[1177],{"type":46,"value":1178},"256, 512, 1024, 2048, 4096, 8192",{"type":46,"value":1180},"). This is a constant, not a runtime tunable.",{"type":40,"tag":80,"props":1182,"children":1183},{},[1184,1189],{"type":40,"tag":84,"props":1185,"children":1186},{},[1187],{"type":46,"value":1188},"Occupancy is the only effective knob",{"type":46,"value":1190},": Higher occupancy lets the GPU hide memory latency by switching to another CTA while one is stalled on a memory request.",{"type":40,"tag":739,"props":1192,"children":1193},{},[1194],{"type":40,"tag":49,"props":1195,"children":1196},{},[1197,1202,1204,1209],{"type":40,"tag":84,"props":1198,"children":1199},{},[1200],{"type":46,"value":1201},"Evidence — CE Loss experiment",{"type":46,"value":1203},": A 12-config search (occupancy × num_ctas) on Cross-Entropy Loss yielded only 2.5% gain (0.79x → 0.81x vs Triton). The ",{"type":40,"tag":55,"props":1205,"children":1207},{"className":1206},[],[1208],{"type":46,"value":274},{"type":46,"value":1210}," dimension contributed nothing; the result was reverted because compilation cost outweighed the marginal benefit. Occupancy-only (4 configs) achieves the same result at 3x less compilation time.",{"type":40,"tag":49,"props":1212,"children":1213},{},[1214,1219],{"type":40,"tag":84,"props":1215,"children":1216},{},[1217],{"type":46,"value":1218},"Note on memory-bound kernels",{"type":46,"value":1220},": Adding occupancy-only autotune is always worthwhile because:",{"type":40,"tag":640,"props":1222,"children":1223},{},[1224,1229,1234],{"type":40,"tag":80,"props":1225,"children":1226},{},[1227],{"type":46,"value":1228},"The tune-once\u002Fcache\u002Flaunch pattern has zero runtime overhead after the first call",{"type":40,"tag":80,"props":1230,"children":1231},{},[1232],{"type":46,"value":1233},"The search space is tiny (4 configs, ~2-4s compilation)",{"type":40,"tag":80,"props":1235,"children":1236},{},[1237],{"type":46,"value":1238},"Even small improvements have value at scale",{"type":40,"tag":64,"props":1240,"children":1242},{"id":1241},"occupancy-selection-guide",[1243],{"type":46,"value":1244},"Occupancy Selection Guide",{"type":40,"tag":49,"props":1246,"children":1247},{},[1248],{"type":46,"value":1249},"Occupancy controls how many CTAs run concurrently per SM. Use this as a starting point when designing the occupancy search space:",{"type":40,"tag":183,"props":1251,"children":1252},{},[1253,1274],{"type":40,"tag":187,"props":1254,"children":1255},{},[1256],{"type":40,"tag":191,"props":1257,"children":1258},{},[1259,1264,1269],{"type":40,"tag":195,"props":1260,"children":1261},{},[1262],{"type":46,"value":1263},"Occupancy Range",{"type":40,"tag":195,"props":1265,"children":1266},{},[1267],{"type":46,"value":1268},"Best For",{"type":40,"tag":195,"props":1270,"children":1271},{},[1272],{"type":46,"value":1273},"Example Kernels",{"type":40,"tag":206,"props":1275,"children":1276},{},[1277,1295,1313,1331],{"type":40,"tag":191,"props":1278,"children":1279},{},[1280,1285,1290],{"type":40,"tag":213,"props":1281,"children":1282},{},[1283],{"type":46,"value":1284},"1–4",{"type":40,"tag":213,"props":1286,"children":1287},{},[1288],{"type":46,"value":1289},"Compute-bound (heavy math)",{"type":40,"tag":213,"props":1291,"children":1292},{},[1293],{"type":46,"value":1294},"Complex transforms, matmul",{"type":40,"tag":191,"props":1296,"children":1297},{},[1298,1303,1308],{"type":40,"tag":213,"props":1299,"children":1300},{},[1301],{"type":46,"value":1302},"4–8",{"type":40,"tag":213,"props":1304,"children":1305},{},[1306],{"type":46,"value":1307},"Balanced (GEMM, TMA)",{"type":40,"tag":213,"props":1309,"children":1310},{},[1311],{"type":46,"value":1312},"Matrix multiply, FMHA",{"type":40,"tag":191,"props":1314,"children":1315},{},[1316,1321,1326],{"type":40,"tag":213,"props":1317,"children":1318},{},[1319],{"type":46,"value":1320},"8–16",{"type":40,"tag":213,"props":1322,"children":1323},{},[1324],{"type":46,"value":1325},"Memory-bound (reductions)",{"type":40,"tag":213,"props":1327,"children":1328},{},[1329],{"type":46,"value":1330},"Softmax, LayerNorm",{"type":40,"tag":191,"props":1332,"children":1333},{},[1334,1339,1344],{"type":40,"tag":213,"props":1335,"children":1336},{},[1337],{"type":46,"value":1338},"16–32",{"type":40,"tag":213,"props":1340,"children":1341},{},[1342],{"type":46,"value":1343},"Very light (copies, casts)",{"type":40,"tag":213,"props":1345,"children":1346},{},[1347],{"type":46,"value":1348},"Type conversions, elementwise",{"type":40,"tag":49,"props":1350,"children":1351},{},[1352,1354,1360],{"type":46,"value":1353},"Use these ranges to seed your initial search space. For occupancy-only kernels, ",{"type":40,"tag":55,"props":1355,"children":1357},{"className":1356},[],[1358],{"type":46,"value":1359},"[1, 2, 4, 8]",{"type":46,"value":1361}," covers most cases — see Quick Reference above.",{"type":40,"tag":64,"props":1363,"children":1365},{"id":1364},"exhaustive_search-api-reference",[1366],{"type":46,"value":1367},"exhaustive_search API Reference",{"type":40,"tag":49,"props":1369,"children":1370},{},[1371,1373,1378,1380,1385,1387,1393,1395,1400,1402,1408,1410,1416],{"type":46,"value":1372},"See ",{"type":40,"tag":727,"props":1374,"children":1376},{"href":1375},"references\u002Fapi-reference.md",[1377],{"type":46,"value":1375},{"type":46,"value":1379}," for the full\n",{"type":40,"tag":55,"props":1381,"children":1383},{"className":1382},[],[1384],{"type":46,"value":60},{"type":46,"value":1386}," API surface — current signature, ",{"type":40,"tag":55,"props":1388,"children":1390},{"className":1389},[],[1391],{"type":46,"value":1392},"TuningResult",{"type":46,"value":1394},", the\ntune-once\u002Fcache\u002Flaunch pattern, ",{"type":40,"tag":55,"props":1396,"children":1398},{"className":1397},[],[1399],{"type":46,"value":764},{"type":46,"value":1401},", kernel hints, ",{"type":40,"tag":55,"props":1403,"children":1405},{"className":1404},[],[1406],{"type":46,"value":1407},"search_space",{"type":46,"value":1409},"\ndesign, and ",{"type":40,"tag":55,"props":1411,"children":1413},{"className":1412},[],[1414],{"type":46,"value":1415},"grid_fn",{"type":46,"value":1417}," patterns.",{"type":40,"tag":64,"props":1419,"children":1421},{"id":1420},"step-by-step-workflow",[1422],{"type":46,"value":1423},"Step-by-Step Workflow",{"type":40,"tag":49,"props":1425,"children":1426},{},[1427,1428,1433,1435,1441],{"type":46,"value":1372},{"type":40,"tag":727,"props":1429,"children":1431},{"href":1430},"references\u002Fworkflow.md",[1432],{"type":46,"value":1430},{"type":46,"value":1434}," for the end-to-end\nworkflow — adding autotune to a new kernel, handling existing\nmulti-architecture configs, integration with ",{"type":40,"tag":55,"props":1436,"children":1438},{"className":1437},[],[1439],{"type":46,"value":1440},"torch.autograd.Function",{"type":46,"value":1442},",\ncross-backend config transfer (Triton → CuTile), and optimizing an existing\nconfig.",{"type":40,"tag":64,"props":1444,"children":1446},{"id":1445},"pitfall-checklist",[1447],{"type":46,"value":1448},"Pitfall Checklist",{"type":40,"tag":49,"props":1450,"children":1451},{},[1452,1453,1458,1460,1465,1467,1472],{"type":46,"value":1372},{"type":40,"tag":727,"props":1454,"children":1456},{"href":1455},"references\u002Fpitfalls.md",[1457],{"type":46,"value":1455},{"type":46,"value":1459}," for the full list of\ncommon pitfalls — in-place data corruption, compilation timeout, cold-cache\nperformance skew, NCU profiling interference, ",{"type":40,"tag":55,"props":1461,"children":1463},{"className":1462},[],[1464],{"type":46,"value":1407},{"type":46,"value":1466}," generator\nexhaustion, FP8 precision loss, and ",{"type":40,"tag":55,"props":1468,"children":1470},{"className":1469},[],[1471],{"type":46,"value":764},{"type":46,"value":1473}," recompilation on hot\npaths.",{"type":40,"tag":64,"props":1475,"children":1477},{"id":1476},"scope-and-boundaries",[1478],{"type":46,"value":1479},"Scope and Boundaries",{"type":40,"tag":49,"props":1481,"children":1482},{},[1483,1485,1490,1492,1497,1499,1504,1506,1511],{"type":46,"value":1484},"This skill covers ",{"type":40,"tag":868,"props":1486,"children":1487},{},[1488],{"type":46,"value":1489},"only",{"type":46,"value":1491}," autotune configuration: search space design, ",{"type":40,"tag":55,"props":1493,"children":1495},{"className":1494},[],[1496],{"type":46,"value":60},{"type":46,"value":1498}," invocation, caching, and ",{"type":40,"tag":55,"props":1500,"children":1502},{"className":1501},[],[1503],{"type":46,"value":131},{"type":46,"value":1505}," with tuned hints. It does ",{"type":40,"tag":84,"props":1507,"children":1508},{},[1509],{"type":46,"value":1510},"not",{"type":46,"value":1512}," modify kernel code.",{"type":40,"tag":49,"props":1514,"children":1515},{},[1516,1521],{"type":40,"tag":84,"props":1517,"children":1518},{},[1519],{"type":46,"value":1520},"In scope",{"type":46,"value":1522}," (autotune config):",{"type":40,"tag":640,"props":1524,"children":1525},{},[1526,1531,1542,1553,1558,1569],{"type":40,"tag":80,"props":1527,"children":1528},{},[1529],{"type":46,"value":1530},"Search space generator functions",{"type":40,"tag":80,"props":1532,"children":1533},{},[1534,1540],{"type":40,"tag":55,"props":1535,"children":1537},{"className":1536},[],[1538],{"type":46,"value":1539},"exhaustive_search()",{"type":46,"value":1541}," calls and result handling",{"type":40,"tag":80,"props":1543,"children":1544},{},[1545,1551],{"type":40,"tag":55,"props":1546,"children":1548},{"className":1547},[],[1549],{"type":46,"value":1550},"kernel.replace_hints()",{"type":46,"value":1552}," for applying tuned hints",{"type":40,"tag":80,"props":1554,"children":1555},{},[1556],{"type":46,"value":1557},"Cache logic (key design, dict management)",{"type":40,"tag":80,"props":1559,"children":1560},{},[1561,1567],{"type":40,"tag":55,"props":1562,"children":1564},{"className":1563},[],[1565],{"type":46,"value":1566},"ct.launch()",{"type":46,"value":1568}," with tuned kernel",{"type":40,"tag":80,"props":1570,"children":1571},{},[1572,1578],{"type":40,"tag":55,"props":1573,"children":1575},{"className":1574},[],[1576],{"type":46,"value":1577},"DISABLE_AUTOTUNE",{"type":46,"value":1579}," fallback path",{"type":40,"tag":49,"props":1581,"children":1582},{},[1583,1588],{"type":40,"tag":84,"props":1584,"children":1585},{},[1586],{"type":46,"value":1587},"Out of scope",{"type":46,"value":1589}," (kernel code modifications — do NOT make these changes):",{"type":40,"tag":640,"props":1591,"children":1592},{},[1593,1598,1603,1608,1613],{"type":40,"tag":80,"props":1594,"children":1595},{},[1596],{"type":46,"value":1597},"Math flags (flush_to_zero, rounding_mode)",{"type":40,"tag":80,"props":1599,"children":1600},{},[1601],{"type":46,"value":1602},"Performance Hints (slice_hint, buffer_depth, copy_config)",{"type":40,"tag":80,"props":1604,"children":1605},{},[1606],{"type":46,"value":1607},"Memory access patterns (2D→1D gather\u002Fscatter conversion)",{"type":40,"tag":80,"props":1609,"children":1610},{},[1611],{"type":46,"value":1612},"Codegen optimizations (safe_offs → padding_value)",{"type":40,"tag":80,"props":1614,"children":1615},{},[1616],{"type":46,"value":1617},"Algorithm changes (K-loop split, load balancing)",{"type":40,"tag":64,"props":1619,"children":1621},{"id":1620},"further-optimization-suggestions",[1622],{"type":46,"value":1623},"Further Optimization Suggestions",{"type":40,"tag":49,"props":1625,"children":1626},{},[1627,1629,1634],{"type":46,"value":1628},"After adding autotuning, the following kernel-level optimizations may yield additional gains. These are ",{"type":40,"tag":868,"props":1630,"children":1631},{},[1632],{"type":46,"value":1633},"outside the scope of this skill",{"type":46,"value":1635}," — mention them to the user as potential next steps, but do not implement them as part of autotuning:",{"type":40,"tag":640,"props":1637,"children":1638},{},[1639,1693,1725,1758,1790],{"type":40,"tag":80,"props":1640,"children":1641},{},[1642,1647,1648,1654,1656,1662,1664,1670,1672,1677,1679,1684,1686,1691],{"type":40,"tag":84,"props":1643,"children":1644},{},[1645],{"type":46,"value":1646},"Math flags",{"type":46,"value":1139},{"type":40,"tag":55,"props":1649,"children":1651},{"className":1650},[],[1652],{"type":46,"value":1653},"flush_to_zero=True",{"type":46,"value":1655}," + ",{"type":40,"tag":55,"props":1657,"children":1659},{"className":1658},[],[1660],{"type":46,"value":1661},"rounding_mode=APPROX",{"type":46,"value":1663}," can provide 34-72% improvement for FMHA-class kernels (set via environment variables ",{"type":40,"tag":55,"props":1665,"children":1667},{"className":1666},[],[1668],{"type":46,"value":1669},"TILEIR_ENABLE_FTZ=1 TILEIR_ENABLE_APPROX=1",{"type":46,"value":1671}," or in kernel code). ",{"type":40,"tag":868,"props":1673,"children":1674},{},[1675],{"type":46,"value":1676},"Causal chain",{"type":46,"value":1678},": larger tiles initially ",{"type":40,"tag":868,"props":1680,"children":1681},{},[1682],{"type":46,"value":1683},"decrease",{"type":46,"value":1685}," performance by 18-43% due to subnormal handling overhead; enabling FTZ+APPROX rescues this and flips the result to +34-72%. Math flags are therefore a ",{"type":40,"tag":868,"props":1687,"children":1688},{},[1689],{"type":46,"value":1690},"prerequisite",{"type":46,"value":1692}," for large-tile configs to be effective on FMHA-class kernels.",{"type":40,"tag":80,"props":1694,"children":1695},{},[1696,1701,1702,1708,1710,1716,1717,1723],{"type":40,"tag":84,"props":1697,"children":1698},{},[1699],{"type":46,"value":1700},"Performance Hints",{"type":46,"value":1139},{"type":40,"tag":55,"props":1703,"children":1705},{"className":1704},[],[1706],{"type":46,"value":1707},"slice_hint",{"type":46,"value":1709},", ",{"type":40,"tag":55,"props":1711,"children":1713},{"className":1712},[],[1714],{"type":46,"value":1715},"buffer_depth",{"type":46,"value":1709},{"type":40,"tag":55,"props":1718,"children":1720},{"className":1719},[],[1721],{"type":46,"value":1722},"copy_config",{"type":46,"value":1724}," — requires modifying kernel IR code",{"type":40,"tag":80,"props":1726,"children":1727},{},[1728,1733,1735,1741,1743,1748,1750,1756],{"type":40,"tag":84,"props":1729,"children":1730},{},[1731],{"type":46,"value":1732},"Memory access patterns",{"type":46,"value":1734},": Using TMA loads (",{"type":40,"tag":55,"props":1736,"children":1738},{"className":1737},[],[1739],{"type":46,"value":1740},"ct.load",{"type":46,"value":1742},") instead of ",{"type":40,"tag":55,"props":1744,"children":1746},{"className":1745},[],[1747],{"type":46,"value":1153},{"type":46,"value":1749},"; removing unnecessary bounds checks (",{"type":40,"tag":55,"props":1751,"children":1753},{"className":1752},[],[1754],{"type":46,"value":1755},"check_bounds=False",{"type":46,"value":1757}," when safe)",{"type":40,"tag":80,"props":1759,"children":1760},{},[1761,1766,1768,1774,1776,1782,1784],{"type":40,"tag":84,"props":1762,"children":1763},{},[1764],{"type":46,"value":1765},"Codegen quality",{"type":46,"value":1767},": Using ",{"type":40,"tag":55,"props":1769,"children":1771},{"className":1770},[],[1772],{"type":46,"value":1773},"padding_value",{"type":46,"value":1775}," parameter instead of manual ",{"type":40,"tag":55,"props":1777,"children":1779},{"className":1778},[],[1780],{"type":46,"value":1781},"ct.where",{"type":46,"value":1783}," masking; removing ",{"type":40,"tag":55,"props":1785,"children":1787},{"className":1786},[],[1788],{"type":46,"value":1789},"safe_offs",{"type":40,"tag":80,"props":1791,"children":1792},{},[1793,1798],{"type":40,"tag":84,"props":1794,"children":1795},{},[1796],{"type":46,"value":1797},"Algorithm restructuring",{"type":46,"value":1799},": K-loop split, load balancing, algebraic simplification",{"type":40,"tag":64,"props":1801,"children":1803},{"id":1802},"differences-from-triton-autotune",[1804],{"type":46,"value":1805},"Differences from Triton Autotune",{"type":40,"tag":49,"props":1807,"children":1808},{},[1809,1811,1817,1819,1825,1827,1832,1834,1840,1842,1847,1849,1855,1856,1862,1864,1870,1871,1876,1878,1884,1886,1892],{"type":46,"value":1810},"Key differences: Triton uses ",{"type":40,"tag":55,"props":1812,"children":1814},{"className":1813},[],[1815],{"type":46,"value":1816},"@triton.autotune",{"type":46,"value":1818}," decorator with ",{"type":40,"tag":55,"props":1820,"children":1822},{"className":1821},[],[1823],{"type":46,"value":1824},"Config(...)",{"type":46,"value":1826}," objects; CuTile uses ",{"type":40,"tag":55,"props":1828,"children":1830},{"className":1829},[],[1831],{"type":46,"value":1539},{"type":46,"value":1833}," with ",{"type":40,"tag":55,"props":1835,"children":1837},{"className":1836},[],[1838],{"type":46,"value":1839},"SimpleNamespace",{"type":46,"value":1841}," configs + separate cache + ",{"type":40,"tag":55,"props":1843,"children":1845},{"className":1844},[],[1846],{"type":46,"value":131},{"type":46,"value":1848},". CuTile has no ",{"type":40,"tag":55,"props":1850,"children":1852},{"className":1851},[],[1853],{"type":46,"value":1854},"num_warps",{"type":46,"value":1155},{"type":40,"tag":55,"props":1857,"children":1859},{"className":1858},[],[1860],{"type":46,"value":1861},"num_stages",{"type":46,"value":1863}," (compiler decides) — only tile sizes + ",{"type":40,"tag":55,"props":1865,"children":1867},{"className":1866},[],[1868],{"type":46,"value":1869},"occupancy",{"type":46,"value":1655},{"type":40,"tag":55,"props":1872,"children":1874},{"className":1873},[],[1875],{"type":46,"value":274},{"type":46,"value":1877},". CuTile compilation is heavier (keep ≤30 configs in final code). CuTile cache is user-managed in-memory (no automatic persistence). CuTile separates ",{"type":40,"tag":55,"props":1879,"children":1881},{"className":1880},[],[1882],{"type":46,"value":1883},"args_fn",{"type":46,"value":1885}," (kernel args) from ",{"type":40,"tag":55,"props":1887,"children":1889},{"className":1888},[],[1890],{"type":46,"value":1891},"hints_fn",{"type":46,"value":1893}," (compiler hints).",{"type":40,"tag":64,"props":1895,"children":1897},{"id":1896},"reference-documents",[1898],{"type":46,"value":1899},"Reference Documents",{"type":40,"tag":183,"props":1901,"children":1902},{},[1903,1924],{"type":40,"tag":187,"props":1904,"children":1905},{},[1906],{"type":40,"tag":191,"props":1907,"children":1908},{},[1909,1914,1919],{"type":40,"tag":195,"props":1910,"children":1911},{},[1912],{"type":46,"value":1913},"Category",{"type":40,"tag":195,"props":1915,"children":1916},{},[1917],{"type":46,"value":1918},"Document",{"type":40,"tag":195,"props":1920,"children":1921},{},[1922],{"type":46,"value":1923},"Content",{"type":40,"tag":206,"props":1925,"children":1926},{},[1927,1987,2022,2064,2091,2119,2146],{"type":40,"tag":191,"props":1928,"children":1929},{},[1930,1938,1950],{"type":40,"tag":213,"props":1931,"children":1932},{},[1933],{"type":40,"tag":84,"props":1934,"children":1935},{},[1936],{"type":46,"value":1937},"API Reference",{"type":40,"tag":213,"props":1939,"children":1940},{},[1941],{"type":40,"tag":727,"props":1942,"children":1943},{"href":1375},[1944],{"type":40,"tag":55,"props":1945,"children":1947},{"className":1946},[],[1948],{"type":46,"value":1949},"api-reference.md",{"type":40,"tag":213,"props":1951,"children":1952},{},[1953,1958,1960,1965,1967,1972,1973,1978,1980,1985],{"type":40,"tag":55,"props":1954,"children":1956},{"className":1955},[],[1957],{"type":46,"value":60},{"type":46,"value":1959}," signature, ",{"type":40,"tag":55,"props":1961,"children":1963},{"className":1962},[],[1964],{"type":46,"value":1392},{"type":46,"value":1966},", tune-once\u002Fcache\u002Flaunch pattern, ",{"type":40,"tag":55,"props":1968,"children":1970},{"className":1969},[],[1971],{"type":46,"value":764},{"type":46,"value":1401},{"type":40,"tag":55,"props":1974,"children":1976},{"className":1975},[],[1977],{"type":46,"value":1407},{"type":46,"value":1979}," design, ",{"type":40,"tag":55,"props":1981,"children":1983},{"className":1982},[],[1984],{"type":46,"value":1415},{"type":46,"value":1986}," patterns",{"type":40,"tag":191,"props":1988,"children":1989},{},[1990,1998,2010],{"type":40,"tag":213,"props":1991,"children":1992},{},[1993],{"type":40,"tag":84,"props":1994,"children":1995},{},[1996],{"type":46,"value":1997},"Workflow",{"type":40,"tag":213,"props":1999,"children":2000},{},[2001],{"type":40,"tag":727,"props":2002,"children":2003},{"href":1430},[2004],{"type":40,"tag":55,"props":2005,"children":2007},{"className":2006},[],[2008],{"type":46,"value":2009},"workflow.md",{"type":40,"tag":213,"props":2011,"children":2012},{},[2013,2015,2020],{"type":46,"value":2014},"End-to-end workflow: adding autotune to a new kernel, multi-architecture configs, ",{"type":40,"tag":55,"props":2016,"children":2018},{"className":2017},[],[2019],{"type":46,"value":1440},{"type":46,"value":2021}," integration, Triton→CuTile transfer, optimizing existing configs",{"type":40,"tag":191,"props":2023,"children":2024},{},[2025,2033,2045],{"type":40,"tag":213,"props":2026,"children":2027},{},[2028],{"type":40,"tag":84,"props":2029,"children":2030},{},[2031],{"type":46,"value":2032},"Pitfalls",{"type":40,"tag":213,"props":2034,"children":2035},{},[2036],{"type":40,"tag":727,"props":2037,"children":2038},{"href":1455},[2039],{"type":40,"tag":55,"props":2040,"children":2042},{"className":2041},[],[2043],{"type":46,"value":2044},"pitfalls.md",{"type":40,"tag":213,"props":2046,"children":2047},{},[2048,2050,2055,2057,2062],{"type":46,"value":2049},"Common pitfalls: in-place corruption, compilation timeout, cold-cache skew, NCU interference, ",{"type":40,"tag":55,"props":2051,"children":2053},{"className":2052},[],[2054],{"type":46,"value":1407},{"type":46,"value":2056}," exhaustion, FP8 precision, ",{"type":40,"tag":55,"props":2058,"children":2060},{"className":2059},[],[2061],{"type":46,"value":764},{"type":46,"value":2063}," recompilation",{"type":40,"tag":191,"props":2065,"children":2066},{},[2067,2075,2086],{"type":40,"tag":213,"props":2068,"children":2069},{},[2070],{"type":40,"tag":84,"props":2071,"children":2072},{},[2073],{"type":46,"value":2074},"Parameter Design",{"type":40,"tag":213,"props":2076,"children":2077},{},[2078],{"type":40,"tag":727,"props":2079,"children":2080},{"href":962},[2081],{"type":40,"tag":55,"props":2082,"children":2084},{"className":2083},[],[2085],{"type":46,"value":969},{"type":40,"tag":213,"props":2087,"children":2088},{},[2089],{"type":46,"value":2090},"Per-kernel-type parameter spaces, cross-arch patterns, grid_fn patterns, pruning rules",{"type":40,"tag":191,"props":2092,"children":2093},{},[2094,2102,2114],{"type":40,"tag":213,"props":2095,"children":2096},{},[2097],{"type":40,"tag":84,"props":2098,"children":2099},{},[2100],{"type":46,"value":2101},"Search Strategies",{"type":40,"tag":213,"props":2103,"children":2104},{},[2105],{"type":40,"tag":727,"props":2106,"children":2107},{"href":165},[2108],{"type":40,"tag":55,"props":2109,"children":2111},{"className":2110},[],[2112],{"type":46,"value":2113},"search-strategies.md",{"type":40,"tag":213,"props":2115,"children":2116},{},[2117],{"type":46,"value":2118},"Exhaustive search, A\u002FB test methodology, DISABLE_AUTOTUNE pattern",{"type":40,"tag":191,"props":2120,"children":2121},{},[2122,2130,2141],{"type":40,"tag":213,"props":2123,"children":2124},{},[2125],{"type":40,"tag":84,"props":2126,"children":2127},{},[2128],{"type":46,"value":2129},"Templates",{"type":40,"tag":213,"props":2131,"children":2132},{},[2133],{"type":40,"tag":727,"props":2134,"children":2135},{"href":106},[2136],{"type":40,"tag":55,"props":2137,"children":2139},{"className":2138},[],[2140],{"type":46,"value":735},{"type":40,"tag":213,"props":2142,"children":2143},{},[2144],{"type":46,"value":2145},"Copy-paste autotune templates for 8 kernel types",{"type":40,"tag":191,"props":2147,"children":2148},{},[2149,2157,2170],{"type":40,"tag":213,"props":2150,"children":2151},{},[2152],{"type":40,"tag":84,"props":2153,"children":2154},{},[2155],{"type":46,"value":2156},"Hardware",{"type":40,"tag":213,"props":2158,"children":2159},{},[2160],{"type":40,"tag":727,"props":2161,"children":2163},{"href":2162},"references\u002Fhardware-constraints.md",[2164],{"type":40,"tag":55,"props":2165,"children":2167},{"className":2166},[],[2168],{"type":46,"value":2169},"hardware-constraints.md",{"type":40,"tag":213,"props":2171,"children":2172},{},[2173],{"type":46,"value":2174},"Per-architecture constraints, tile size ranges, num_ctas rules, TMA requirements",{"type":40,"tag":64,"props":2176,"children":2178},{"id":2177},"source-code-references",[2179],{"type":46,"value":2180},"Source Code References",{"type":40,"tag":49,"props":2182,"children":2183},{},[2184,2186,2192,2194,2200,2202,2208,2210,2216,2218,2223,2225,2231,2233,2239,2241,2247],{"type":46,"value":2185},"Key files: ",{"type":40,"tag":55,"props":2187,"children":2189},{"className":2188},[],[2190],{"type":46,"value":2191},"ops\u002Fcutile\u002Fmatmul.py",{"type":46,"value":2193}," (matmul autotune), ",{"type":40,"tag":55,"props":2195,"children":2197},{"className":2196},[],[2198],{"type":46,"value":2199},"ops\u002Fcutile\u002Fattention.py",{"type":46,"value":2201}," (FMHA autotune), ",{"type":40,"tag":55,"props":2203,"children":2205},{"className":2204},[],[2206],{"type":46,"value":2207},"suites\u002Funsloth\u002Fcutile\u002Fct_ops.py",{"type":46,"value":2209}," (shared ",{"type":40,"tag":55,"props":2211,"children":2213},{"className":2212},[],[2214],{"type":46,"value":2215},"autotune_configs()",{"type":46,"value":2217}," occupancy=",{"type":40,"tag":313,"props":2219,"children":2220},{},[2221],{"type":46,"value":2222},"1,2,4,8",{"type":46,"value":2224},"), ",{"type":40,"tag":55,"props":2226,"children":2228},{"className":2227},[],[2229],{"type":46,"value":2230},"suites\u002Funsloth\u002Fcutile\u002Fswiglu.py",{"type":46,"value":2232}," (elementwise example), ",{"type":40,"tag":55,"props":2234,"children":2236},{"className":2235},[],[2237],{"type":46,"value":2238},"suites\u002Funsloth\u002Fcutile\u002Frope_embedding.py",{"type":46,"value":2240}," (split-buffer pattern), ",{"type":40,"tag":55,"props":2242,"children":2244},{"className":2243},[],[2245],{"type":46,"value":2246},"suites\u002Funsloth\u002Fcutile\u002Fgrouped_gemm.py",{"type":46,"value":2248}," (persistent GEMM, occupancy-only).",{"type":40,"tag":64,"props":2250,"children":2252},{"id":2251},"worked-examples",[2253],{"type":46,"value":2254},"Worked Examples",{"type":40,"tag":49,"props":2256,"children":2257},{},[2258,2260,2265,2267,2273,2275,2280,2282,2288],{"type":46,"value":2259},"Each example shows the ",{"type":40,"tag":84,"props":2261,"children":2262},{},[2263],{"type":46,"value":2264},"before → after",{"type":46,"value":2266}," pattern: ",{"type":40,"tag":55,"props":2268,"children":2270},{"className":2269},[],[2271],{"type":46,"value":2272},"fixed_launch.py",{"type":46,"value":2274}," (hardcoded ",{"type":40,"tag":55,"props":2276,"children":2278},{"className":2277},[],[2279],{"type":46,"value":131},{"type":46,"value":2281},") and ",{"type":40,"tag":55,"props":2283,"children":2285},{"className":2284},[],[2286],{"type":46,"value":2287},"autotuned_launch.py",{"type":46,"value":2289}," (refactored to tune-once\u002Fcache\u002Flaunch).",{"type":40,"tag":183,"props":2291,"children":2292},{},[2293,2324],{"type":40,"tag":187,"props":2294,"children":2295},{},[2296],{"type":40,"tag":191,"props":2297,"children":2298},{},[2299,2304,2309,2314,2319],{"type":40,"tag":195,"props":2300,"children":2301},{},[2302],{"type":46,"value":2303},"Directory",{"type":40,"tag":195,"props":2305,"children":2306},{},[2307],{"type":46,"value":2308},"Kernel",{"type":40,"tag":195,"props":2310,"children":2311},{},[2312],{"type":46,"value":2313},"Autotune Pattern",{"type":40,"tag":195,"props":2315,"children":2316},{},[2317],{"type":46,"value":2318},"Complexity",{"type":40,"tag":195,"props":2320,"children":2321},{},[2322],{"type":46,"value":2323},"Key Teaching Point",{"type":40,"tag":206,"props":2325,"children":2326},{},[2327,2376,2461],{"type":40,"tag":191,"props":2328,"children":2329},{},[2330,2342,2347,2358,2363],{"type":40,"tag":213,"props":2331,"children":2332},{},[2333],{"type":40,"tag":727,"props":2334,"children":2336},{"href":2335},"assets\u002Fexamples\u002F01_rmsnorm_occupancy_only\u002F",[2337],{"type":40,"tag":55,"props":2338,"children":2340},{"className":2339},[],[2341],{"type":46,"value":2335},{"type":40,"tag":213,"props":2343,"children":2344},{},[2345],{"type":46,"value":2346},"RMSNorm (reduction)",{"type":40,"tag":213,"props":2348,"children":2349},{},[2350,2352],{"type":46,"value":2351},"Occupancy-only ",{"type":40,"tag":55,"props":2353,"children":2355},{"className":2354},[],[2356],{"type":46,"value":2357},"[1,2,4,8]",{"type":40,"tag":213,"props":2359,"children":2360},{},[2361],{"type":46,"value":2362},"Low",{"type":40,"tag":213,"props":2364,"children":2365},{},[2366,2368,2374],{"type":46,"value":2367},"Most common pattern — no tile tuning, just find best occupancy. Grid = ",{"type":40,"tag":55,"props":2369,"children":2371},{"className":2370},[],[2372],{"type":46,"value":2373},"NUM_SM * cfg.occupancy",{"type":46,"value":2375},". Not in-place.",{"type":40,"tag":191,"props":2377,"children":2378},{},[2379,2391,2396,2421,2426],{"type":40,"tag":213,"props":2380,"children":2381},{},[2382],{"type":40,"tag":727,"props":2383,"children":2385},{"href":2384},"assets\u002Fexamples\u002F02_matmul_full_search\u002F",[2386],{"type":40,"tag":55,"props":2387,"children":2389},{"className":2388},[],[2390],{"type":46,"value":2384},{"type":40,"tag":213,"props":2392,"children":2393},{},[2394],{"type":46,"value":2395},"GEMM C=A@B",{"type":40,"tag":213,"props":2397,"children":2398},{},[2399,2401,2407,2408,2413,2414,2419],{"type":46,"value":2400},"Full: ",{"type":40,"tag":55,"props":2402,"children":2404},{"className":2403},[],[2405],{"type":46,"value":2406},"TILE_M\u002FN\u002FK",{"type":46,"value":1655},{"type":40,"tag":55,"props":2409,"children":2411},{"className":2410},[],[2412],{"type":46,"value":1869},{"type":46,"value":1655},{"type":40,"tag":55,"props":2415,"children":2417},{"className":2416},[],[2418],{"type":46,"value":274},{"type":46,"value":2420}," (sm90+)",{"type":40,"tag":213,"props":2422,"children":2423},{},[2424],{"type":46,"value":2425},"High",{"type":40,"tag":213,"props":2427,"children":2428},{},[2429,2431,2436,2438,2444,2446,2451,2453,2459],{"type":46,"value":2430},"Compute-bound kernel with multiple tunable dimensions. ",{"type":40,"tag":55,"props":2432,"children":2434},{"className":2433},[],[2435],{"type":46,"value":1883},{"type":46,"value":2437}," passes tile sizes as ",{"type":40,"tag":55,"props":2439,"children":2441},{"className":2440},[],[2442],{"type":46,"value":2443},"ct.Constant[int]",{"type":46,"value":2445},". ",{"type":40,"tag":55,"props":2447,"children":2449},{"className":2448},[],[2450],{"type":46,"value":1415},{"type":46,"value":2452}," depends on ",{"type":40,"tag":55,"props":2454,"children":2456},{"className":2455},[],[2457],{"type":46,"value":2458},"cfg",{"type":46,"value":2460},". ≤30 configs.",{"type":40,"tag":191,"props":2462,"children":2463},{},[2464,2476,2481,2486,2491],{"type":40,"tag":213,"props":2465,"children":2466},{},[2467],{"type":40,"tag":727,"props":2468,"children":2470},{"href":2469},"assets\u002Fexamples\u002F03_rope_inplace_splitbuffer\u002F",[2471],{"type":40,"tag":55,"props":2472,"children":2474},{"className":2473},[],[2475],{"type":46,"value":2469},{"type":40,"tag":213,"props":2477,"children":2478},{},[2479],{"type":46,"value":2480},"RoPE embedding (in-place)",{"type":40,"tag":213,"props":2482,"children":2483},{},[2484],{"type":46,"value":2485},"Occupancy-only, with split-buffer",{"type":40,"tag":213,"props":2487,"children":2488},{},[2489],{"type":46,"value":2490},"Medium",{"type":40,"tag":213,"props":2492,"children":2493},{},[2494,2496,2501],{"type":46,"value":2495},"In-place kernel MUST use split-buffer during search to avoid corruption. Search writes to scratch; final ",{"type":40,"tag":55,"props":2497,"children":2499},{"className":2498},[],[2500],{"type":46,"value":131},{"type":46,"value":2502}," uses real in-place args.",{"type":40,"tag":2504,"props":2505,"children":2506},"style",{},[2507],{"type":46,"value":2508},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":2510,"total":2666},[2511,2529,2546,2557,2569,2581,2594,2608,2621,2632,2646,2655],{"slug":2512,"name":2512,"fn":2513,"description":2514,"org":2515,"tags":2516,"stars":2526,"repoUrl":2527,"updatedAt":2528},"nemoclaw-user-guide","retrieve NemoClaw documentation and configuration","Guides human users' AI agents to the NemoClaw docs MCP server and canonical Fern documentation in Markdown form. Use when users ask how to install, configure, operate, troubleshoot, secure, or learn NemoClaw with an AI coding assistant. Trigger keywords - nemoclaw docs, use nemoclaw with ai agent, nemoclaw mcp docs, nemoclaw install help, nemoclaw quickstart, nemoclaw markdown docs, llms.txt, agent skills.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2517,2520,2523],{"name":2518,"slug":2519,"type":15},"Documentation","documentation",{"name":2521,"slug":2522,"type":15},"MCP","mcp",{"name":2524,"slug":2525,"type":15},"Search","search",21777,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FNemoClaw","2026-07-20T06:00:01.461044",{"slug":2530,"name":2530,"fn":2531,"description":2532,"org":2533,"tags":2534,"stars":2543,"repoUrl":2544,"updatedAt":2545},"mcore-build-and-dependency","manage Megatron-LM development environments","Container-based dev environment setup and dependency management for Megatron-LM. Covers acquiring and launching the CI container, uv package management, and updating uv.lock.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2535,2538,2541],{"name":2536,"slug":2537,"type":15},"Containers","containers",{"name":2539,"slug":2540,"type":15},"Deployment","deployment",{"name":2542,"slug":306,"type":15},"Python",17049,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FMegatron-LM","2026-07-27T06:06:11.249662",{"slug":2547,"name":2547,"fn":2548,"description":2549,"org":2550,"tags":2551,"stars":2543,"repoUrl":2544,"updatedAt":2556},"mcore-bump-base-image","update NVIDIA PyTorch base images","Bump the NVIDIA PyTorch base image (`nvcr.io\u002Fnvidia\u002Fpytorch:YY.MM-py3`) used by Megatron-LM CI. Covers the two pin sites (GitHub CI in `docker\u002F.ngc_version.dev` and GitLab CI in `.gitlab\u002Fstages\u002F01.build.yml`), the post-bump CI loop (re-run functional tests, refresh golden values, mark broken tests), and the gotchas that bit PRs",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2552,2555],{"name":2553,"slug":2554,"type":15},"CI\u002FCD","ci-cd",{"name":2539,"slug":2540,"type":15},"2026-07-14T05:25:59.97109",{"slug":2558,"name":2558,"fn":2559,"description":2560,"org":2561,"tags":2562,"stars":2543,"repoUrl":2544,"updatedAt":2568},"mcore-cicd","manage CI\u002FCD pipelines for Megatron-LM","CI\u002FCD reference for Megatron-LM. Covers CI pipeline structure, PR scope labels, triggering internal GitLab CI (which force-pushes the current branch to a pull-request\u002FBRANCH ref — always dry-run and verify the destination first; never run against shared or protected branches), and CI failure investigation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2563,2564,2565],{"name":2553,"slug":2554,"type":15},{"name":2539,"slug":2540,"type":15},{"name":2566,"slug":2567,"type":15},"GitHub","github","2026-07-27T06:06:12.278222",{"slug":2570,"name":2570,"fn":2571,"description":2572,"org":2573,"tags":2574,"stars":2543,"repoUrl":2544,"updatedAt":2580},"mcore-create-issue","investigate CI failures and create issues","Investigate a failing GitHub Actions run or job and create a GitHub issue for the failure.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2575,2576,2577],{"name":21,"slug":22,"type":15},{"name":2566,"slug":2567,"type":15},{"name":2578,"slug":2579,"type":15},"Triage","triage","2026-07-14T05:25:57.442089",{"slug":2582,"name":2582,"fn":2583,"description":2584,"org":2585,"tags":2586,"stars":2543,"repoUrl":2544,"updatedAt":2593},"mcore-linting-and-formatting","lint and format Megatron-LM code","Linting and formatting for Megatron-LM. Covers running autoformat.sh, tools (ruff, black, isort, pylint, mypy), and code style rules.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2587,2590],{"name":2588,"slug":2589,"type":15},"Best Practices","best-practices",{"name":2591,"slug":2592,"type":15},"Code Analysis","code-analysis","2026-07-14T05:25:56.18433",{"slug":2595,"name":2595,"fn":2596,"description":2597,"org":2598,"tags":2599,"stars":2543,"repoUrl":2544,"updatedAt":2607},"mcore-migrate-gpt-to-hybrid","migrate Megatron-LM models to HybridModel","Migration guide for moving Megatron Core GPTModel checkpoints, model providers, training commands, and layer mappings to HybridModel.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2600,2603,2606],{"name":2601,"slug":2602,"type":15},"Machine Learning","machine-learning",{"name":2604,"slug":2605,"type":15},"Migration","migration",{"name":9,"slug":8,"type":15},"2026-07-17T06:07:11.777011",{"slug":2609,"name":2609,"fn":2610,"description":2611,"org":2612,"tags":2613,"stars":2543,"repoUrl":2544,"updatedAt":2620},"mcore-onboard-gb200-1node-tests","onboard functional tests for GB200","Onboard 1-node GitHub MR functional tests for GB200 from existing mr-scoped 2-node tests.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2614,2617],{"name":2615,"slug":2616,"type":15},"QA","qa",{"name":2618,"slug":2619,"type":15},"Testing","testing","2026-07-14T05:25:53.673039",{"slug":2622,"name":2622,"fn":2623,"description":2624,"org":2625,"tags":2626,"stars":2543,"repoUrl":2544,"updatedAt":2631},"mcore-run-on-slurm","launch distributed training jobs on SLURM","How to launch distributed Megatron-LM training jobs on a SLURM cluster. Covers a minimal sbatch skeleton, environment-variable setup for torch.distributed.run, CUDA_DEVICE_MAX_CONNECTIONS rules across hardware and parallelism modes, container conventions, monitoring, and per-rank failure diagnosis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2627,2628],{"name":2539,"slug":2540,"type":15},{"name":2629,"slug":2630,"type":15},"Infrastructure","infrastructure","2026-07-14T05:25:49.362534",{"slug":2633,"name":2633,"fn":2634,"description":2635,"org":2636,"tags":2637,"stars":2543,"repoUrl":2544,"updatedAt":2645},"mcore-split-pr","split pull requests to reduce review load","Split a PR into multiple PRs to reduce the number of required CODEOWNERS reviewer groups.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2638,2641,2642],{"name":2639,"slug":2640,"type":15},"Code Review","code-review",{"name":2566,"slug":2567,"type":15},{"name":2643,"slug":2644,"type":15},"Pull Requests","pull-requests","2026-07-14T05:26:01.226578",{"slug":2647,"name":2647,"fn":2648,"description":2649,"org":2650,"tags":2651,"stars":2543,"repoUrl":2544,"updatedAt":2654},"mcore-testing","run and manage Megatron-LM tests","Test system for Megatron-LM. Covers test layout, recipe YAML structure, adding and running unit and functional tests, golden values, marker filters, and CI parity.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2652,2653],{"name":2615,"slug":2616,"type":15},{"name":2618,"slug":2619,"type":15},"2026-07-14T05:25:54.928983",{"slug":2656,"name":2656,"fn":2657,"description":2658,"org":2659,"tags":2660,"stars":2543,"repoUrl":2544,"updatedAt":2665},"nightly-sync","manage nightly main-to-dev sync workflows","Domain knowledge for the nightly main-to-dev sync workflow. Covers merge strategy, CI architecture, failure investigation, and known issues.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2661,2664],{"name":2662,"slug":2663,"type":15},"Automation","automation",{"name":2553,"slug":2554,"type":15},"2026-07-30T05:29:03.275638",496,{"items":2668,"total":2762},[2669,2684,2694,2708,2718,2733,2748],{"slug":2670,"name":2670,"fn":2671,"description":2672,"org":2673,"tags":2674,"stars":23,"repoUrl":24,"updatedAt":2683},"accelerated-computing-cudf","accelerate data processing with cuDF","Official NVIDIA-authored guidance for NVIDIA cuDF GPU DataFrames, pandas acceleration, dask-cuDF, ETL, joins, groupby, CSV\u002FParquet I\u002FO, nullable semantics, and multi-GPU DataFrame workloads.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2675,2678,2681,2682],{"name":2676,"slug":2677,"type":15},"Data Analysis","data-analysis",{"name":2679,"slug":2680,"type":15},"Data Engineering","data-engineering",{"name":9,"slug":8,"type":15},{"name":13,"slug":14,"type":15},"2026-07-14T05:28:43.176466",{"slug":2685,"name":2685,"fn":2686,"description":2687,"org":2688,"tags":2689,"stars":23,"repoUrl":24,"updatedAt":2693},"aiq-deploy","deploy and manage NVIDIA AI-Q infrastructure","Use when asked to install, deploy, run, validate, troubleshoot, or stop NVIDIA AI-Q Blueprint infrastructure.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2690,2691,2692],{"name":2539,"slug":2540,"type":15},{"name":2629,"slug":2630,"type":15},{"name":9,"slug":8,"type":15},"2026-07-14T05:29:06.667109",{"slug":2695,"name":2695,"fn":2696,"description":2697,"org":2698,"tags":2699,"stars":23,"repoUrl":24,"updatedAt":2707},"aiq-research","conduct deep research with AI-Q","Use when asked to run deep research or AI-Q research through a reachable NVIDIA AI-Q Blueprint backend.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2700,2703,2704],{"name":2701,"slug":2702,"type":15},"Agents","agents",{"name":9,"slug":8,"type":15},{"name":2705,"slug":2706,"type":15},"Research","research","2026-07-14T05:28:06.816956",{"slug":2709,"name":2709,"fn":2710,"description":2711,"org":2712,"tags":2713,"stars":23,"repoUrl":24,"updatedAt":2717},"amc-run-sample-calibration","run AMC sample dataset calibration","Run end-to-end calibration on the shipped sample dataset (sdg_08_2_sample_data_010926.zip) against a running AMC microservice. Use when user says 'test sample dataset', 'run sample calibration', 'verify AMC install', or 'launch and test'.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2714,2715,2716],{"name":2676,"slug":2677,"type":15},{"name":9,"slug":8,"type":15},{"name":2618,"slug":2619,"type":15},"2026-07-17T05:29:03.913266",{"slug":2719,"name":2719,"fn":2720,"description":2721,"org":2722,"tags":2723,"stars":23,"repoUrl":24,"updatedAt":2732},"amc-run-video-calibration","calibrate video datasets with AutoMagicCalib","Calibrate a new dataset from pre-recorded video files via the AutoMagicCalib REST API. Use when user has local MP4s and says 'calibrate my videos', 'run AMC on these videos', or similar. For RTSP\u002Flive streams, use amc-run-rtsp-calibration instead.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2724,2725,2728,2729],{"name":2662,"slug":2663,"type":15},{"name":2726,"slug":2727,"type":15},"Imaging","imaging",{"name":9,"slug":8,"type":15},{"name":2730,"slug":2731,"type":15},"Video","video","2026-07-17T05:28:53.905004",{"slug":2734,"name":2734,"fn":2735,"description":2736,"org":2737,"tags":2738,"stars":23,"repoUrl":24,"updatedAt":2747},"amc-setup-calibration-stack","deploy AutoMagicCalib microservice with Docker","Launch AutoMagicCalib microservice and web UI from NGC release images via Docker Compose. Use when user says 'deploy auto calibration', 'launch auto calibration', 'launch AMC', 'start MS+UI', or 'set up auto-magic-calib'. Requires NGC API key.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2739,2740,2743,2744],{"name":2539,"slug":2540,"type":15},{"name":2741,"slug":2742,"type":15},"Docker","docker",{"name":9,"slug":8,"type":15},{"name":2745,"slug":2746,"type":15},"Operations","operations","2026-07-17T05:28:56.913999",{"slug":2749,"name":2749,"fn":2750,"description":2751,"org":2752,"tags":2753,"stars":23,"repoUrl":24,"updatedAt":2761},"cudaq-guide","develop quantum applications with CUDA-Q","CUDA-Q onboarding guide for installation, test programs, GPU simulation, QPU hardware, and quantum applications.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2754,2755,2758],{"name":9,"slug":8,"type":15},{"name":2756,"slug":2757,"type":15},"Quantum Computing","quantum-computing",{"name":2759,"slug":2760,"type":15},"Simulation","simulation","2026-07-14T05:26:58.898253",305]