[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-huggingface-rocm-kernels":3,"mdc-q17j5u-key":39,"related-repo-huggingface-rocm-kernels":4815,"related-org-huggingface-rocm-kernels":4865},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":28,"repoUrl":29,"updatedAt":30,"license":31,"forks":32,"topics":33,"repo":34,"sourceUrl":37,"mdContent":38},"rocm-kernels","build optimized Triton kernels for AMD GPUs","Provides guidance for writing and benchmarking optimized Triton kernels for AMD GPUs (MI355X, R9700) on ROCm, targeting HuggingFace diffusers (LTX-Video, SD3, FLUX) and transformers. Core kernels: RMSNorm, RoPE 3D, GEGLU, AdaLN. Includes XCD swizzle, autotune, diffusers integration patterns, and LTX-Video pipeline injection.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"huggingface","Hugging Face","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fhuggingface.png",[12,16,19,22,25],{"name":13,"slug":14,"type":15},"Performance","performance","tag",{"name":17,"slug":18,"type":15},"Triton","triton",{"name":20,"slug":21,"type":15},"Deep Learning","deep-learning",{"name":23,"slug":24,"type":15},"AI Infrastructure","ai-infrastructure",{"name":26,"slug":27,"type":15},"ROCm","rocm",712,"https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fkernels","2026-04-16T05:06:54.179351",null,112,[],{"repoUrl":29,"stars":28,"forks":32,"topics":35,"description":36},[],"Build compute kernels and load them from the Hub.","https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fkernels\u002Ftree\u002FHEAD\u002Fkernel-builder\u002Fskills\u002Frocm-kernels","---\nname: rocm-kernels\ndescription: \"Provides guidance for writing and benchmarking optimized Triton kernels for AMD GPUs (MI355X, R9700) on ROCm, targeting HuggingFace diffusers (LTX-Video, SD3, FLUX) and transformers. Core kernels: RMSNorm, RoPE 3D, GEGLU, AdaLN. Includes XCD swizzle, autotune, diffusers integration patterns, and LTX-Video pipeline injection.\"\ndisable-model-invocation: false\nuser-invocable: true\nallowed-tools: \"Read, Grep, Glob, Bash\"\nargument-hint: \"kernel type: rmsnorm, rope, rope-3d, geglu, adaln, gemm, benchmark, diffusers, transformers, ltx-video, huggingface-kernels, get_kernel, autotune, xcd-swizzle\"\n---\n\n# ROCm Triton Kernels for Diffusers & Transformers\n\nThis skill provides patterns and guidance for developing optimized Triton kernels targeting AMD GPUs (MI355X, R9700) on ROCm, for use with HuggingFace **diffusers** (LTX-Video, SD3, FLUX) and **transformers** libraries.\n\n## Quick Start\n\n### Diffusers (LTX-Video)\n\n**Inject optimized kernels into LTX-Video pipeline:**\n```python\nimport os\nos.environ['TRITON_HIP_USE_BLOCK_PINGPONG'] = '1'\nos.environ['TRITON_HIP_USE_ASYNC_COPY'] = '1'\n\nfrom diffusers import LTXPipeline\npipe = LTXPipeline.from_pretrained(\"Lightricks\u002FLTX-Video\", torch_dtype=torch.bfloat16)\npipe.to(\"cuda\")  # ROCm uses same API via HIP\ninject_optimized_kernels(pipe)  # BEFORE CPU offloading\npipe.enable_model_cpu_offload()\n```\n\n**For a minimal integration example (~150 lines):**\n```bash\npython scripts\u002Fltx_kernel_injection_example.py\n```\n\n### Isolated Kernel Micro-benchmarks\n```bash\n# All 4 kernels: correctness + performance + bandwidth\npython scripts\u002Fbenchmark_kernels.py\n\n# Single kernel\npython scripts\u002Fbenchmark_kernels.py --kernel rmsnorm\npython scripts\u002Fbenchmark_kernels.py --kernel rope\npython scripts\u002Fbenchmark_kernels.py --kernel geglu\npython scripts\u002Fbenchmark_kernels.py --kernel adaln\n```\n\n### End-to-End Pipeline Benchmark\n```bash\n# Compare baseline vs Triton vs torch.compile\npython scripts\u002Fbenchmark_e2e.py --mode all\n\n# Quick test\npython scripts\u002Fbenchmark_e2e.py --mode triton --num-frames 9 --steps 5\n\n# Save results for comparison\npython scripts\u002Fbenchmark_e2e.py --mode all --output-json results.json\n```\n\n## Target Model: LTX-Video\n\n### Architecture Overview\n\n| Component | Class | Has Weight | Count | Kernel |\n|-----------|-------|------------|-------|--------|\n| `transformer_blocks.*.norm1` | RMSNorm | **No** (elementwise_affine=False) | 56 | RMSNorm |\n| `transformer_blocks.*.norm2` | RMSNorm | **No** | 56 | RMSNorm |\n| `transformer_blocks.*.attn1.norm_q` | torch.nn.RMSNorm | Yes | 28 | RMSNorm |\n| `transformer_blocks.*.attn1.norm_k` | torch.nn.RMSNorm | Yes | 28 | RMSNorm |\n| `transformer_blocks.*.ff` | FeedForward | - | 28 | **GELU** (not GEGLU!) |\n| Rotary position encoding | LTXVideoRotaryPosEmbed | - | 1 | RoPE 3D |\n\n**Total RMSNorm modules: 168** (56 with weights, 112 without)\n\n### Target Kernels\n\n| Kernel | Use Case | Input Layout | Key Challenge |\n|--------|----------|-------------|---------------|\n| **RMSNorm** | Normalization | `[..., hidden_size]` | Weight may be None; 168 instances |\n| **RoPE 3D** | Video position encoding | `[batch, t*h*w, heads, head_dim]` | 3D → temporal + spatial decomposition |\n| **GEGLU** | Gated activation (SD3\u002FFLUX) | `[batch, seq, 2*hidden]` → `[batch, seq, hidden]` | Gate\u002Fvalue split |\n| **AdaLN** | Conditioned normalization (DiT) | `norm(x) * weight * (1+scale) + shift` | Fused norm + condition |\n\n## Supported Hardware\n\n| GPU | Architecture | Wave Size | LDS\u002FCU | Mem BW | Key Feature | Verified |\n|-----|-------------|-----------|--------|--------|-------------|:--------:|\n| **MI355X** | CDNA3+ (gfx950) | Wave64 | **160 KB** | 8 TB\u002Fs | 32 XCDs, XCD Swizzle for GEMM | Yes |\n| **R9700** | RDNA4 (gfx1201) | **Wave32** | 64 KB | ~608 GB\u002Fs | 256B cacheline, inference-focused | Yes |\n\n> See [MI355X guide](references\u002Fmi355x-optimization-guide.md) | [R9700 guide](references\u002Fr9700-optimization-guide.md)\n\n## When This Skill Applies\n\nUse this skill when:\n- Writing Triton kernels for **RMSNorm, RoPE, GEGLU, AdaLN** on AMD GPUs\n- Integrating custom kernels with **diffusers** pipelines (LTX-Video, SD3, FLUX)\n- Benchmarking kernel performance against PyTorch baseline on ROCm\n- Optimizing existing kernels for MI355X or R9700 architecture\n- Debugging ROCm\u002FHIP-specific kernel issues\n\n## Critical ROCm Constraints\n\n### Things That DON'T Work on AMD\n\n```python\n# FORBIDDEN - CUDA only, NOT available on ROCm\ntl.libdevice.tanh(x)          # Use manual formula below\ntl.libdevice.log1p(x)         # Use: tl.log(1.0 + x)\ntl.math.tanh(x)               # Also NOT available on ROCm Triton\n\n# Manual tanh (ONLY reliable method on ROCm):\ne2x = tl.exp(2.0 * x)\ntanh_x = (e2x - 1.0) \u002F (e2x + 1.0)\n\n# FORBIDDEN - Triton limitations on ROCm\nbreak \u002F continue               # Use: tl.where()\nmin(a, b) \u002F max(a, b)          # Use: tl.minimum(a, b) \u002F tl.maximum(a, b)\n```\n\n### Mandatory Environment Variables\n\n```python\nimport os\nos.environ['TRITON_HIP_USE_BLOCK_PINGPONG'] = '1'\nos.environ['TRITON_HIP_USE_ASYNC_COPY'] = '1'\n```\n\n## Core Kernel Implementations\n\n### 1. RMSNorm (Core Optimization Target)\n\nRow-wise reduction pattern. **168 instances** in LTX-Video, ~5% of total compute.\n\n**CRITICAL: Do NOT autotune BLOCK_D.** Autotune may pick `BLOCK_D \u003C D`, causing partial row processing and wrong results. Always compute `BLOCK_D = triton.next_power_of_2(D)` in the Python wrapper.\n\n```python\n@triton.jit\ndef rmsnorm_kernel(\n    x_ptr, weight_ptr, out_ptr,\n    stride_x, D,\n    eps: tl.constexpr,\n    HAS_WEIGHT: tl.constexpr,\n    BLOCK_D: tl.constexpr,\n):\n    row = tl.program_id(0)\n    offs = tl.arange(0, BLOCK_D)\n    mask = offs \u003C D\n    x = tl.load(x_ptr + row * stride_x + offs, mask=mask, other=0.0).to(tl.float32)\n\n    variance = tl.sum(x * x, axis=0) \u002F D\n    rms_inv = tl.rsqrt(variance + eps)\n\n    if HAS_WEIGHT:\n        w = tl.load(weight_ptr + offs, mask=mask, other=1.0).to(tl.float32)\n        out = x * rms_inv * w\n    else:\n        out = x * rms_inv\n\n    tl.store(out_ptr + row * stride_x + offs, out.to(x.dtype), mask=mask)\n\n\ndef triton_rmsnorm(x, weight=None, eps=1e-6):\n    x_2d = x.contiguous().view(-1, x.shape[-1])\n    out = torch.empty_like(x_2d)\n    M, D = x_2d.shape\n    has_weight = weight is not None\n    if not has_weight:\n        weight = torch.empty(0, device=x.device)\n\n    BLOCK_D = triton.next_power_of_2(D)\n    num_warps = 4 if BLOCK_D \u003C= 1024 else (8 if BLOCK_D \u003C= 4096 else 16)\n    rmsnorm_kernel[(M,)](\n        x_2d, weight, out, x_2d.stride(0), D, eps, has_weight,\n        BLOCK_D=BLOCK_D, num_warps=num_warps, num_stages=2,\n    )\n    return out.view_as(x)\n```\n\n**LTX-Video pitfall: Weight may be None!**\n```python\nhas_weight = hasattr(module, 'weight') and module.weight is not None\n```\n\n### 2. RoPE 3D (Video Position Encoding)\n\nElement-wise pattern. LTX-Video splits `head_dim` into temporal + spatial components.\n\n**CRITICAL: cos\u002Fsin have shape `[seq_len, head_dim]`.** When grid flattens batch dimension (`batch * seq_len`), use `pid_s % seq_len` to index cos\u002Fsin, otherwise batch > 1 causes OOB GPU crash.\n\n```python\n@triton.jit\ndef rope_3d_kernel(\n    qk_ptr, cos_ptr, sin_ptr, out_ptr,\n    seq_len, num_heads, head_dim,\n    stride_s, stride_h, stride_d,\n    BLOCK_HD: tl.constexpr,\n):\n    pid_s = tl.program_id(0)  # batch * seq_len\n    pid_h = tl.program_id(1)  # head index\n    half_dim = head_dim \u002F\u002F 2\n    offs = tl.arange(0, BLOCK_HD)\n    mask = offs \u003C half_dim\n\n    base = pid_s * stride_s + pid_h * stride_h\n    x0 = tl.load(qk_ptr + base + offs, mask=mask, other=0.0).to(tl.float32)\n    x1 = tl.load(qk_ptr + base + half_dim + offs, mask=mask, other=0.0).to(tl.float32)\n\n    seq_idx = pid_s % seq_len  # wrap for batch > 1\n    cos_val = tl.load(cos_ptr + seq_idx * head_dim + offs, mask=mask, other=1.0).to(tl.float32)\n    sin_val = tl.load(sin_ptr + seq_idx * head_dim + offs, mask=mask, other=0.0).to(tl.float32)\n\n    out0 = x0 * cos_val - x1 * sin_val\n    out1 = x0 * sin_val + x1 * cos_val\n\n    tl.store(out_ptr + base + offs, out0.to(x0.dtype), mask=mask)\n    tl.store(out_ptr + base + half_dim + offs, out1.to(x0.dtype), mask=mask)\n\n\ndef triton_rope_3d(qk, cos, sin):\n    qk = qk.contiguous()\n    out = torch.empty_like(qk)\n    batch, seq_len, num_heads, head_dim = qk.shape\n    qk_flat = qk.view(batch * seq_len, num_heads, head_dim)\n    out_flat = out.view(batch * seq_len, num_heads, head_dim)\n    BLOCK_HD = triton.next_power_of_2(head_dim \u002F\u002F 2)\n    num_warps = 4 if BLOCK_HD \u003C= 64 else 8\n    rope_3d_kernel[(batch * seq_len, num_heads)](\n        qk_flat, cos, sin, out_flat,\n        seq_len, num_heads, head_dim,\n        qk_flat.stride(0), qk_flat.stride(1), qk_flat.stride(2),\n        BLOCK_HD=BLOCK_HD, num_warps=num_warps, num_stages=2,\n    )\n    return out\n```\n\n### 3. GEGLU (For SD3\u002FFLUX, NOT LTX-Video)\n\nElement-wise gated activation. Input `[batch, seq, 2*hidden]` → Output `[batch, seq, hidden]`.\n\n**Same BLOCK_SIZE rule: compute dynamically, do NOT autotune.**\n\n```python\n@triton.jit\ndef geglu_kernel(\n    input_ptr, output_ptr,\n    stride_in, stride_out, hidden_size,\n    BLOCK_H: tl.constexpr,\n):\n    row = tl.program_id(0)\n    offs = tl.arange(0, BLOCK_H)\n    mask = offs \u003C hidden_size\n\n    gate = tl.load(input_ptr + row * stride_in + offs, mask=mask, other=0.0).to(tl.float32)\n    value = tl.load(input_ptr + row * stride_in + hidden_size + offs, mask=mask, other=0.0).to(tl.float32)\n\n    # GELU approx — manual tanh (tl.math.tanh NOT available on ROCm)\n    k = 0.7978845608028654  # sqrt(2\u002Fpi)\n    tanh_arg = k * (gate + 0.044715 * gate * gate * gate)\n    e2x = tl.exp(2.0 * tanh_arg)\n    tanh_val = (e2x - 1.0) \u002F (e2x + 1.0)\n    gate_gelu = 0.5 * gate * (1.0 + tanh_val)\n    result = gate_gelu * value\n\n    tl.store(output_ptr + row * stride_out + offs, result.to(gate.dtype), mask=mask)\n\n\ndef triton_geglu(x):\n    x = x.contiguous()\n    *batch_dims, double_h = x.shape\n    hidden_size = double_h \u002F\u002F 2\n    x_2d = x.view(-1, double_h)\n    M = x_2d.shape[0]\n    out = torch.empty(M, hidden_size, device=x.device, dtype=x.dtype)\n    BLOCK_H = triton.next_power_of_2(hidden_size)\n    num_warps = 4 if BLOCK_H \u003C= 1024 else (8 if BLOCK_H \u003C= 4096 else 16)\n    geglu_kernel[(M,)](\n        x_2d, out, x_2d.stride(0), out.stride(0), hidden_size,\n        BLOCK_H=BLOCK_H, num_warps=num_warps, num_stages=2,\n    )\n    return out.view(*batch_dims, hidden_size)\n```\n\n**Warning: LTX-Video uses GELU, NOT GEGLU.** GEGLU is for SD3\u002FFLUX.\n\n### 4. AdaLN (Adaptive Layer Normalization for DiT)\n\nFused normalization + conditioning: `norm(x) * weight * (1 + scale) + shift`\n\n**Same BLOCK_D rule: compute dynamically.**\n\n```python\n@triton.jit\ndef adaln_kernel(\n    x_ptr, weight_ptr, scale_ptr, shift_ptr, out_ptr,\n    stride_x, stride_cond, D,\n    eps: tl.constexpr,\n    BLOCK_D: tl.constexpr,\n):\n    row = tl.program_id(0)\n    offs = tl.arange(0, BLOCK_D)\n    mask = offs \u003C D\n    x = tl.load(x_ptr + row * stride_x + offs, mask=mask, other=0.0).to(tl.float32)\n\n    variance = tl.sum(x * x, axis=0) \u002F D\n    rms_inv = tl.rsqrt(variance + eps)\n    x_norm = x * rms_inv\n\n    w = tl.load(weight_ptr + offs, mask=mask, other=1.0).to(tl.float32)\n    scale = tl.load(scale_ptr + row * stride_cond + offs, mask=mask, other=0.0).to(tl.float32)\n    shift = tl.load(shift_ptr + row * stride_cond + offs, mask=mask, other=0.0).to(tl.float32)\n\n    out = x_norm * w * (1.0 + scale) + shift\n    tl.store(out_ptr + row * stride_x + offs, out.to(x.dtype), mask=mask)\n\n\ndef triton_adaln(x, weight, scale, shift, eps=1e-6):\n    x_flat = x.contiguous().view(-1, x.shape[-1])\n    scale_flat = scale.contiguous().view(-1, x.shape[-1])\n    shift_flat = shift.contiguous().view(-1, x.shape[-1])\n    out = torch.empty_like(x_flat)\n    M, D = x_flat.shape\n    BLOCK_D = triton.next_power_of_2(D)\n    num_warps = 4 if BLOCK_D \u003C= 1024 else (8 if BLOCK_D \u003C= 4096 else 16)\n    adaln_kernel[(M,)](\n        x_flat, weight, scale_flat, shift_flat, out,\n        x_flat.stride(0), scale_flat.stride(0), D, eps,\n        BLOCK_D=BLOCK_D, num_warps=num_warps, num_stages=2,\n    )\n    return out.view_as(x)\n```\n\n## Diffusers Integration\n\n> **See [diffusers-integration.md](references\u002Fdiffusers-integration.md) for the complete guide.**\n\n### Minimal Integration Pattern\n\n```python\ndef patch_rmsnorm_modules(model):\n    \"\"\"Patch all RMSNorm modules to use custom Triton kernel.\"\"\"\n    for name, module in model.named_modules():\n        if type(module).__name__ == 'RMSNorm':\n            eps = getattr(module, 'eps', 1e-6)\n            has_weight = hasattr(module, 'weight') and module.weight is not None\n            if has_weight:\n                def make_forward(mod, epsilon):\n                    def forward(x):\n                        return triton_rmsnorm(x, mod.weight, eps=epsilon)\n                    return forward\n                module.forward = make_forward(module, eps)\n            else:\n                def make_forward(epsilon):\n                    def forward(x):\n                        w = torch.ones(x.shape[-1], device=x.device, dtype=x.dtype)\n                        return triton_rmsnorm(x, w, eps=epsilon)\n                    return forward\n                module.forward = make_forward(eps)\n\npipe = LTXPipeline.from_pretrained(\"Lightricks\u002FLTX-Video\", torch_dtype=torch.bfloat16)\npipe.to(\"cuda\")\npatch_rmsnorm_modules(pipe.transformer)\npipe.enable_model_cpu_offload()\n```\n\n### Diffusers Critical Pitfalls\n\n1. **RMSNorm weight may be None** — LTX-Video uses `elementwise_affine=False`\n2. **Diffusers RMSNorm != torch.nn.RMSNorm** — Use `type(module).__name__` not `isinstance()`\n3. **LTX-Video uses GELU, not GEGLU** — Don't patch GEGLU for LTX-Video\n4. **Inject BEFORE CPU offloading** — `inject_kernels()` then `enable_model_cpu_offload()`\n\n## Performance Expectations\n\n### Micro-benchmark Results (MI355X, BF16)\n\n| Kernel | Avg Speedup | Best Config Speedup | Status |\n|--------|:-----------:|:-------------------:|:------:|\n| **RMSNorm** | **1.71x** | 2.44x ([4×4096×3072]) | PASS |\n| **RoPE 3D** | **1.21x** | 1.52x ([2×4096×16×128]) | PASS |\n| **GEGLU** | **1.43x** | 2.13x ([4×4096×8192]) | PASS |\n| **AdaLN** | **2.22x** | 2.77x ([4×4096×3072]) | PASS |\n\nRMSNorm bandwidth utilization: 3554 GB\u002Fs (MI355X theoretical: 8 TB\u002Fs, ~44%).\n\n### End-to-End LTX-Video (MI355X, 25 frames, 30 steps)\n\n| Mode | Time (s) | Per Step (s) | Peak Mem (GB) | Speedup |\n|------|:--------:|:------------:|:-------------:|:-------:|\n| baseline | 1.20 | 0.040 | 18.58 | 1.00x |\n| **triton** | **0.98** | **0.033** | **18.58** | **1.22x** |\n| torch.compile | 0.78 | 0.026 | 18.58 | 1.54x |\n\n**Key finding**: MI355X Triton E2E speedup (22%) is significantly higher than H100 CUDA reference (6%), because MI355X's default PyTorch RMSNorm path has more room for optimization.\n\n### Micro-benchmark Results (R9700, BF16)\n\n| Kernel | Avg Speedup | Best Config Speedup | Status |\n|--------|:-----------:|:-------------------:|:------:|\n| **RMSNorm** | **2.90x** | 3.97x ([1×8192×2048]) | PASS |\n| **RoPE 3D** | **2.09x** | 2.38x ([1×1024×16×64]) | PASS |\n| **GEGLU** | **1.69x** | 1.93x ([2×1024×8192]) | PASS |\n| **AdaLN** | **3.00x** | 3.67x ([4×4096×3072]) | PASS |\n\nRMSNorm bandwidth utilization: 483 GB\u002Fs (R9700 theoretical: ~608 GB\u002Fs, **~79%**).\n\nR9700 speedups are higher than MI355X because PyTorch's default RDNA4 backend is less mature, leaving more room for Triton optimization. The bandwidth utilization (79%) is also significantly better than MI355X (44%).\n\n### End-to-End LTX-Video (R9700, 25 frames, 30 steps)\n\n| Mode | Time (s) | Per Step (s) | Peak Mem (GB) | Speedup |\n|------|:--------:|:------------:|:-------------:|:-------:|\n| baseline (mean of 3) | 6.91 | 0.231 | 18.58 | 1.00x |\n| **triton (mean of 3)** | **6.10** | **0.203** | **18.58** | **1.13x** |\n| torch.compile (single run) | 5.05 | 0.168 | 18.58 | 1.37x |\n\nReviewer-facing benchmark files for this comparison live in `examples\u002Fltx-video-benchmark\u002F`, including:\n\n- Summary table with `gen_time_s`, `time_per_step_s`, `peak_memory_gb`, and `speedup`\n- Consolidated JSON results in `examples\u002Fltx-video-benchmark\u002Fbenchmark_results.json`\n- OpenCode run outputs in `examples\u002Fltx-video-benchmark\u002Ftrace\u002Fopencode_live\u002Fresults.json`\n- OpenCode parsed trace in `examples\u002Fltx-video-benchmark\u002Ftrace\u002Fopencode_live\u002Fopencode_trace_result.json`\n\n### R9700 Additional Validation\n\n| Test | Result |\n|------|--------|\n| Transformers injection (TinyLlama 1.1B) | PASS — 45 RMSNorm patched, 99.9 tokens\u002Fs |\n| HuggingFace Kernels Hub integration | PASS — Hub kernel loads and runs on ROCm |\n| Local Triton vs Hub kernel (small shape) | Local **5.92x** vs Hub 1.27x (lower launch overhead) |\n| Local Triton vs Hub kernel (large shape) | Local 3.59x vs Hub 3.57x (comparable) |\n| num_warps sweep (2\u002F4\u002F8\u002F16\u002F32) | Default heuristic (4\u002F8\u002F16) is near-optimal; nw=32 always worst |\n| rocprof kernel fusion analysis | Triton fuses 4 PyTorch kernels (pow+mean+rsqrt+mul) into 1 |\n\n### CUDA Reference (H100, for comparison)\n\n| Shape | Custom (ms) | PyTorch (ms) | Speedup |\n|:---|:---:|:---:|:---:|\n| [1×1024×2048] | 0.019 | 0.065 | **3.37x** |\n| [2×4096×3072] | 0.087 | 0.208 | **2.41x** |\n\nH100 E2E: ~6% (RMSNorm is ~5% of total compute).\n\n### Optimization Targets\n\n| Kernel | MI355X | R9700 | Target | Priority |\n|--------|:------:|:-----:|:------:|:--------:|\n| RMSNorm | 1.71x | 2.90x | >3x (R9700) | P0 — MI355X bandwidth util (44%→60%+) |\n| AdaLN | 2.22x | 3.00x | >3.5x (R9700) | P1 — already strong on both |\n| GEGLU | 1.43x | 1.69x | >2x | P1 — tanh overhead |\n| RoPE 3D | 1.21x | 2.09x | >2.5x (R9700) | P2 — small head_dim launch overhead |\n\n## Common Issues on ROCm\n\n| Issue | Symptom | Fix |\n|-------|---------|-----|\n| **Autotune BLOCK_D** | Wrong results (max_abs 4-8+) | **Never autotune BLOCK_D.** Use `triton.next_power_of_2(D)` |\n| **RoPE batch OOB** | GPU crash (`Memory access fault`) | Use `pid_s % seq_len` for cos\u002Fsin indexing |\n| `tl.libdevice` | Not found on AMD | Use manual math formulas |\n| `tl.tanh` \u002F `tl.math.tanh` | Not on ROCm | Manual: `e2x=exp(2x); (e2x-1)\u002F(e2x+1)` |\n| Python min\u002Fmax | Runtime error | `tl.minimum()`\u002F`tl.maximum()` |\n| LDS overflow | HIP OOM | Reduce num_stages to 2 |\n| Weight is None | AttributeError | Check `elementwise_affine` |\n| isinstance() miss | RMSNorm not patched | Use `type(module).__name__` |\n\n> See [troubleshooting.md](references\u002Ftroubleshooting.md) for all common issues.\n\n## Performance Profiling\n\n```bash\nrocprof --stats python your_kernel.py\nrocprofv3 -i metrics.txt python your_kernel.py\nrocm-bandwidth-test\nrocminfo | grep -E \"Name|Compute Unit|Wavefront\"\n```\n\n## See Also\n\n### Benchmark & Test Scripts\n- [benchmark_kernels.py](scripts\u002Fbenchmark_kernels.py) - Micro-benchmark all 4 kernels (correctness + perf + bandwidth)\n- [benchmark_e2e.py](scripts\u002Fbenchmark_e2e.py) - End-to-end LTX-Video pipeline benchmark (baseline vs Triton vs compile)\n- [sweep_num_warps.py](scripts\u002Fsweep_num_warps.py) - num_warps sweep for R9700 Wave32 optimization\n- [ltx_kernel_injection_example.py](scripts\u002Fltx_kernel_injection_example.py) - Minimal diffusers injection example\n- [transformers_injection_example.py](scripts\u002Ftransformers_injection_example.py) - Minimal transformers injection example\n- [huggingface_kernels_example.py](scripts\u002Fhuggingface_kernels_example.py) - HuggingFace Kernels Hub integration example\n\n### Integration Guides\n- [diffusers-integration.md](references\u002Fdiffusers-integration.md) - LTX-Video pipeline integration\n- [transformers-integration.md](references\u002Ftransformers-integration.md) - LLaMA\u002FMistral\u002FQwen integration\n- [huggingface-kernels-integration.md](references\u002Fhuggingface-kernels-integration.md) - HuggingFace Kernels Hub (`get_kernel`)\n- [kernel-templates.md](references\u002Fkernel-templates.md) - Complete Triton kernel templates (incl. GEMM with XCD Swizzle)\n\n### GPU Optimization Guides\n- [mi355x-optimization-guide.md](references\u002Fmi355x-optimization-guide.md) - MI355X (gfx950) deep dive\n- [r9700-optimization-guide.md](references\u002Fr9700-optimization-guide.md) - R9700 (RDNA4) deep dive\n\n### Reference\n- [troubleshooting.md](references\u002Ftroubleshooting.md) - Common issues and solutions\n- [kernelbench-classification.md](references\u002Fkernelbench-classification.md) - KernelBench operator taxonomy\n- [skill-evaluation-methodology.md](references\u002Fskill-evaluation-methodology.md) - How to evaluate and improve skills\n- [kernel-agent-knowledge-base.md](references\u002Fkernel-agent-knowledge-base.md) - Knowledge from kernel-agent project\n\n### External Resources\n- [Triton Documentation](https:\u002F\u002Ftriton-lang.org\u002F)\n- [ROCm Documentation](https:\u002F\u002Frocm.docs.amd.com\u002F)\n- [HuggingFace Kernels Hub](https:\u002F\u002Fhuggingface.co\u002Fkernels-community)\n- [LTX-Video on HuggingFace](https:\u002F\u002Fhuggingface.co\u002FLightricks\u002FLTX-Video)\n- [HuggingFace Diffusers](https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Fdiffusers\u002Fen\u002Findex)\n",{"data":40,"body":45},{"name":4,"description":6,"disable-model-invocation":41,"user-invocable":42,"allowed-tools":43,"argument-hint":44},false,true,"Read, Grep, Glob, Bash","kernel type: rmsnorm, rope, rope-3d, geglu, adaln, gemm, benchmark, diffusers, transformers, ltx-video, huggingface-kernels, get_kernel, autotune, xcd-swizzle",{"type":46,"children":47},"root",[48,57,78,85,92,100,195,203,226,232,357,363,501,507,513,745,755,761,918,924,1058,1082,1088,1093,1136,1142,1148,1252,1258,1286,1292,1298,1310,1336,1685,1693,1707,1713,1726,1760,2105,2111,2129,2137,2439,2449,2455,2466,2474,2764,2770,2788,2794,2988,2994,3072,3078,3084,3248,3253,3259,3392,3402,3408,3567,3579,3584,3590,3717,3730,3800,3806,3913,3919,4009,4014,4020,4154,4160,4406,4421,4427,4530,4536,4542,4611,4617,4669,4675,4698,4704,4749,4755,4809],{"type":49,"tag":50,"props":51,"children":53},"element","h1",{"id":52},"rocm-triton-kernels-for-diffusers-transformers",[54],{"type":55,"value":56},"text","ROCm Triton Kernels for Diffusers & Transformers",{"type":49,"tag":58,"props":59,"children":60},"p",{},[61,63,69,71,76],{"type":55,"value":62},"This skill provides patterns and guidance for developing optimized Triton kernels targeting AMD GPUs (MI355X, R9700) on ROCm, for use with HuggingFace ",{"type":49,"tag":64,"props":65,"children":66},"strong",{},[67],{"type":55,"value":68},"diffusers",{"type":55,"value":70}," (LTX-Video, SD3, FLUX) and ",{"type":49,"tag":64,"props":72,"children":73},{},[74],{"type":55,"value":75},"transformers",{"type":55,"value":77}," libraries.",{"type":49,"tag":79,"props":80,"children":82},"h2",{"id":81},"quick-start",[83],{"type":55,"value":84},"Quick Start",{"type":49,"tag":86,"props":87,"children":89},"h3",{"id":88},"diffusers-ltx-video",[90],{"type":55,"value":91},"Diffusers (LTX-Video)",{"type":49,"tag":58,"props":93,"children":94},{},[95],{"type":49,"tag":64,"props":96,"children":97},{},[98],{"type":55,"value":99},"Inject optimized kernels into LTX-Video pipeline:",{"type":49,"tag":101,"props":102,"children":107},"pre",{"className":103,"code":104,"language":105,"meta":106,"style":106},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","import os\nos.environ['TRITON_HIP_USE_BLOCK_PINGPONG'] = '1'\nos.environ['TRITON_HIP_USE_ASYNC_COPY'] = '1'\n\nfrom diffusers import LTXPipeline\npipe = LTXPipeline.from_pretrained(\"Lightricks\u002FLTX-Video\", torch_dtype=torch.bfloat16)\npipe.to(\"cuda\")  # ROCm uses same API via HIP\ninject_optimized_kernels(pipe)  # BEFORE CPU offloading\npipe.enable_model_cpu_offload()\n","python","",[108],{"type":49,"tag":109,"props":110,"children":111},"code",{"__ignoreMap":106},[112,123,132,141,150,159,168,177,186],{"type":49,"tag":113,"props":114,"children":117},"span",{"class":115,"line":116},"line",1,[118],{"type":49,"tag":113,"props":119,"children":120},{},[121],{"type":55,"value":122},"import os\n",{"type":49,"tag":113,"props":124,"children":126},{"class":115,"line":125},2,[127],{"type":49,"tag":113,"props":128,"children":129},{},[130],{"type":55,"value":131},"os.environ['TRITON_HIP_USE_BLOCK_PINGPONG'] = '1'\n",{"type":49,"tag":113,"props":133,"children":135},{"class":115,"line":134},3,[136],{"type":49,"tag":113,"props":137,"children":138},{},[139],{"type":55,"value":140},"os.environ['TRITON_HIP_USE_ASYNC_COPY'] = '1'\n",{"type":49,"tag":113,"props":142,"children":144},{"class":115,"line":143},4,[145],{"type":49,"tag":113,"props":146,"children":147},{"emptyLinePlaceholder":42},[148],{"type":55,"value":149},"\n",{"type":49,"tag":113,"props":151,"children":153},{"class":115,"line":152},5,[154],{"type":49,"tag":113,"props":155,"children":156},{},[157],{"type":55,"value":158},"from diffusers import LTXPipeline\n",{"type":49,"tag":113,"props":160,"children":162},{"class":115,"line":161},6,[163],{"type":49,"tag":113,"props":164,"children":165},{},[166],{"type":55,"value":167},"pipe = LTXPipeline.from_pretrained(\"Lightricks\u002FLTX-Video\", torch_dtype=torch.bfloat16)\n",{"type":49,"tag":113,"props":169,"children":171},{"class":115,"line":170},7,[172],{"type":49,"tag":113,"props":173,"children":174},{},[175],{"type":55,"value":176},"pipe.to(\"cuda\")  # ROCm uses same API via HIP\n",{"type":49,"tag":113,"props":178,"children":180},{"class":115,"line":179},8,[181],{"type":49,"tag":113,"props":182,"children":183},{},[184],{"type":55,"value":185},"inject_optimized_kernels(pipe)  # BEFORE CPU offloading\n",{"type":49,"tag":113,"props":187,"children":189},{"class":115,"line":188},9,[190],{"type":49,"tag":113,"props":191,"children":192},{},[193],{"type":55,"value":194},"pipe.enable_model_cpu_offload()\n",{"type":49,"tag":58,"props":196,"children":197},{},[198],{"type":49,"tag":64,"props":199,"children":200},{},[201],{"type":55,"value":202},"For a minimal integration example (~150 lines):",{"type":49,"tag":101,"props":204,"children":208},{"className":205,"code":206,"language":207,"meta":106,"style":106},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","python scripts\u002Fltx_kernel_injection_example.py\n","bash",[209],{"type":49,"tag":109,"props":210,"children":211},{"__ignoreMap":106},[212],{"type":49,"tag":113,"props":213,"children":214},{"class":115,"line":116},[215,220],{"type":49,"tag":113,"props":216,"children":218},{"style":217},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[219],{"type":55,"value":105},{"type":49,"tag":113,"props":221,"children":223},{"style":222},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[224],{"type":55,"value":225}," scripts\u002Fltx_kernel_injection_example.py\n",{"type":49,"tag":86,"props":227,"children":229},{"id":228},"isolated-kernel-micro-benchmarks",[230],{"type":55,"value":231},"Isolated Kernel Micro-benchmarks",{"type":49,"tag":101,"props":233,"children":235},{"className":205,"code":234,"language":207,"meta":106,"style":106},"# All 4 kernels: correctness + performance + bandwidth\npython scripts\u002Fbenchmark_kernels.py\n\n# Single kernel\npython scripts\u002Fbenchmark_kernels.py --kernel rmsnorm\npython scripts\u002Fbenchmark_kernels.py --kernel rope\npython scripts\u002Fbenchmark_kernels.py --kernel geglu\npython scripts\u002Fbenchmark_kernels.py --kernel adaln\n",[236],{"type":49,"tag":109,"props":237,"children":238},{"__ignoreMap":106},[239,248,260,267,275,297,317,337],{"type":49,"tag":113,"props":240,"children":241},{"class":115,"line":116},[242],{"type":49,"tag":113,"props":243,"children":245},{"style":244},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[246],{"type":55,"value":247},"# All 4 kernels: correctness + performance + bandwidth\n",{"type":49,"tag":113,"props":249,"children":250},{"class":115,"line":125},[251,255],{"type":49,"tag":113,"props":252,"children":253},{"style":217},[254],{"type":55,"value":105},{"type":49,"tag":113,"props":256,"children":257},{"style":222},[258],{"type":55,"value":259}," scripts\u002Fbenchmark_kernels.py\n",{"type":49,"tag":113,"props":261,"children":262},{"class":115,"line":134},[263],{"type":49,"tag":113,"props":264,"children":265},{"emptyLinePlaceholder":42},[266],{"type":55,"value":149},{"type":49,"tag":113,"props":268,"children":269},{"class":115,"line":143},[270],{"type":49,"tag":113,"props":271,"children":272},{"style":244},[273],{"type":55,"value":274},"# Single kernel\n",{"type":49,"tag":113,"props":276,"children":277},{"class":115,"line":152},[278,282,287,292],{"type":49,"tag":113,"props":279,"children":280},{"style":217},[281],{"type":55,"value":105},{"type":49,"tag":113,"props":283,"children":284},{"style":222},[285],{"type":55,"value":286}," scripts\u002Fbenchmark_kernels.py",{"type":49,"tag":113,"props":288,"children":289},{"style":222},[290],{"type":55,"value":291}," --kernel",{"type":49,"tag":113,"props":293,"children":294},{"style":222},[295],{"type":55,"value":296}," rmsnorm\n",{"type":49,"tag":113,"props":298,"children":299},{"class":115,"line":161},[300,304,308,312],{"type":49,"tag":113,"props":301,"children":302},{"style":217},[303],{"type":55,"value":105},{"type":49,"tag":113,"props":305,"children":306},{"style":222},[307],{"type":55,"value":286},{"type":49,"tag":113,"props":309,"children":310},{"style":222},[311],{"type":55,"value":291},{"type":49,"tag":113,"props":313,"children":314},{"style":222},[315],{"type":55,"value":316}," rope\n",{"type":49,"tag":113,"props":318,"children":319},{"class":115,"line":170},[320,324,328,332],{"type":49,"tag":113,"props":321,"children":322},{"style":217},[323],{"type":55,"value":105},{"type":49,"tag":113,"props":325,"children":326},{"style":222},[327],{"type":55,"value":286},{"type":49,"tag":113,"props":329,"children":330},{"style":222},[331],{"type":55,"value":291},{"type":49,"tag":113,"props":333,"children":334},{"style":222},[335],{"type":55,"value":336}," geglu\n",{"type":49,"tag":113,"props":338,"children":339},{"class":115,"line":179},[340,344,348,352],{"type":49,"tag":113,"props":341,"children":342},{"style":217},[343],{"type":55,"value":105},{"type":49,"tag":113,"props":345,"children":346},{"style":222},[347],{"type":55,"value":286},{"type":49,"tag":113,"props":349,"children":350},{"style":222},[351],{"type":55,"value":291},{"type":49,"tag":113,"props":353,"children":354},{"style":222},[355],{"type":55,"value":356}," adaln\n",{"type":49,"tag":86,"props":358,"children":360},{"id":359},"end-to-end-pipeline-benchmark",[361],{"type":55,"value":362},"End-to-End Pipeline Benchmark",{"type":49,"tag":101,"props":364,"children":366},{"className":205,"code":365,"language":207,"meta":106,"style":106},"# Compare baseline vs Triton vs torch.compile\npython scripts\u002Fbenchmark_e2e.py --mode all\n\n# Quick test\npython scripts\u002Fbenchmark_e2e.py --mode triton --num-frames 9 --steps 5\n\n# Save results for comparison\npython scripts\u002Fbenchmark_e2e.py --mode all --output-json results.json\n",[367],{"type":49,"tag":109,"props":368,"children":369},{"__ignoreMap":106},[370,378,400,407,415,456,463,471],{"type":49,"tag":113,"props":371,"children":372},{"class":115,"line":116},[373],{"type":49,"tag":113,"props":374,"children":375},{"style":244},[376],{"type":55,"value":377},"# Compare baseline vs Triton vs torch.compile\n",{"type":49,"tag":113,"props":379,"children":380},{"class":115,"line":125},[381,385,390,395],{"type":49,"tag":113,"props":382,"children":383},{"style":217},[384],{"type":55,"value":105},{"type":49,"tag":113,"props":386,"children":387},{"style":222},[388],{"type":55,"value":389}," scripts\u002Fbenchmark_e2e.py",{"type":49,"tag":113,"props":391,"children":392},{"style":222},[393],{"type":55,"value":394}," --mode",{"type":49,"tag":113,"props":396,"children":397},{"style":222},[398],{"type":55,"value":399}," all\n",{"type":49,"tag":113,"props":401,"children":402},{"class":115,"line":134},[403],{"type":49,"tag":113,"props":404,"children":405},{"emptyLinePlaceholder":42},[406],{"type":55,"value":149},{"type":49,"tag":113,"props":408,"children":409},{"class":115,"line":143},[410],{"type":49,"tag":113,"props":411,"children":412},{"style":244},[413],{"type":55,"value":414},"# Quick test\n",{"type":49,"tag":113,"props":416,"children":417},{"class":115,"line":152},[418,422,426,430,435,440,446,451],{"type":49,"tag":113,"props":419,"children":420},{"style":217},[421],{"type":55,"value":105},{"type":49,"tag":113,"props":423,"children":424},{"style":222},[425],{"type":55,"value":389},{"type":49,"tag":113,"props":427,"children":428},{"style":222},[429],{"type":55,"value":394},{"type":49,"tag":113,"props":431,"children":432},{"style":222},[433],{"type":55,"value":434}," triton",{"type":49,"tag":113,"props":436,"children":437},{"style":222},[438],{"type":55,"value":439}," --num-frames",{"type":49,"tag":113,"props":441,"children":443},{"style":442},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[444],{"type":55,"value":445}," 9",{"type":49,"tag":113,"props":447,"children":448},{"style":222},[449],{"type":55,"value":450}," --steps",{"type":49,"tag":113,"props":452,"children":453},{"style":442},[454],{"type":55,"value":455}," 5\n",{"type":49,"tag":113,"props":457,"children":458},{"class":115,"line":161},[459],{"type":49,"tag":113,"props":460,"children":461},{"emptyLinePlaceholder":42},[462],{"type":55,"value":149},{"type":49,"tag":113,"props":464,"children":465},{"class":115,"line":170},[466],{"type":49,"tag":113,"props":467,"children":468},{"style":244},[469],{"type":55,"value":470},"# Save results for comparison\n",{"type":49,"tag":113,"props":472,"children":473},{"class":115,"line":179},[474,478,482,486,491,496],{"type":49,"tag":113,"props":475,"children":476},{"style":217},[477],{"type":55,"value":105},{"type":49,"tag":113,"props":479,"children":480},{"style":222},[481],{"type":55,"value":389},{"type":49,"tag":113,"props":483,"children":484},{"style":222},[485],{"type":55,"value":394},{"type":49,"tag":113,"props":487,"children":488},{"style":222},[489],{"type":55,"value":490}," all",{"type":49,"tag":113,"props":492,"children":493},{"style":222},[494],{"type":55,"value":495}," --output-json",{"type":49,"tag":113,"props":497,"children":498},{"style":222},[499],{"type":55,"value":500}," results.json\n",{"type":49,"tag":79,"props":502,"children":504},{"id":503},"target-model-ltx-video",[505],{"type":55,"value":506},"Target Model: LTX-Video",{"type":49,"tag":86,"props":508,"children":510},{"id":509},"architecture-overview",[511],{"type":55,"value":512},"Architecture Overview",{"type":49,"tag":514,"props":515,"children":516},"table",{},[517,551],{"type":49,"tag":518,"props":519,"children":520},"thead",{},[521],{"type":49,"tag":522,"props":523,"children":524},"tr",{},[525,531,536,541,546],{"type":49,"tag":526,"props":527,"children":528},"th",{},[529],{"type":55,"value":530},"Component",{"type":49,"tag":526,"props":532,"children":533},{},[534],{"type":55,"value":535},"Class",{"type":49,"tag":526,"props":537,"children":538},{},[539],{"type":55,"value":540},"Has Weight",{"type":49,"tag":526,"props":542,"children":543},{},[544],{"type":55,"value":545},"Count",{"type":49,"tag":526,"props":547,"children":548},{},[549],{"type":55,"value":550},"Kernel",{"type":49,"tag":552,"props":553,"children":554},"tbody",{},[555,592,623,654,682,718],{"type":49,"tag":522,"props":556,"children":557},{},[558,568,573,583,588],{"type":49,"tag":559,"props":560,"children":561},"td",{},[562],{"type":49,"tag":109,"props":563,"children":565},{"className":564},[],[566],{"type":55,"value":567},"transformer_blocks.*.norm1",{"type":49,"tag":559,"props":569,"children":570},{},[571],{"type":55,"value":572},"RMSNorm",{"type":49,"tag":559,"props":574,"children":575},{},[576,581],{"type":49,"tag":64,"props":577,"children":578},{},[579],{"type":55,"value":580},"No",{"type":55,"value":582}," (elementwise_affine=False)",{"type":49,"tag":559,"props":584,"children":585},{},[586],{"type":55,"value":587},"56",{"type":49,"tag":559,"props":589,"children":590},{},[591],{"type":55,"value":572},{"type":49,"tag":522,"props":593,"children":594},{},[595,604,608,615,619],{"type":49,"tag":559,"props":596,"children":597},{},[598],{"type":49,"tag":109,"props":599,"children":601},{"className":600},[],[602],{"type":55,"value":603},"transformer_blocks.*.norm2",{"type":49,"tag":559,"props":605,"children":606},{},[607],{"type":55,"value":572},{"type":49,"tag":559,"props":609,"children":610},{},[611],{"type":49,"tag":64,"props":612,"children":613},{},[614],{"type":55,"value":580},{"type":49,"tag":559,"props":616,"children":617},{},[618],{"type":55,"value":587},{"type":49,"tag":559,"props":620,"children":621},{},[622],{"type":55,"value":572},{"type":49,"tag":522,"props":624,"children":625},{},[626,635,640,645,650],{"type":49,"tag":559,"props":627,"children":628},{},[629],{"type":49,"tag":109,"props":630,"children":632},{"className":631},[],[633],{"type":55,"value":634},"transformer_blocks.*.attn1.norm_q",{"type":49,"tag":559,"props":636,"children":637},{},[638],{"type":55,"value":639},"torch.nn.RMSNorm",{"type":49,"tag":559,"props":641,"children":642},{},[643],{"type":55,"value":644},"Yes",{"type":49,"tag":559,"props":646,"children":647},{},[648],{"type":55,"value":649},"28",{"type":49,"tag":559,"props":651,"children":652},{},[653],{"type":55,"value":572},{"type":49,"tag":522,"props":655,"children":656},{},[657,666,670,674,678],{"type":49,"tag":559,"props":658,"children":659},{},[660],{"type":49,"tag":109,"props":661,"children":663},{"className":662},[],[664],{"type":55,"value":665},"transformer_blocks.*.attn1.norm_k",{"type":49,"tag":559,"props":667,"children":668},{},[669],{"type":55,"value":639},{"type":49,"tag":559,"props":671,"children":672},{},[673],{"type":55,"value":644},{"type":49,"tag":559,"props":675,"children":676},{},[677],{"type":55,"value":649},{"type":49,"tag":559,"props":679,"children":680},{},[681],{"type":55,"value":572},{"type":49,"tag":522,"props":683,"children":684},{},[685,694,699,704,708],{"type":49,"tag":559,"props":686,"children":687},{},[688],{"type":49,"tag":109,"props":689,"children":691},{"className":690},[],[692],{"type":55,"value":693},"transformer_blocks.*.ff",{"type":49,"tag":559,"props":695,"children":696},{},[697],{"type":55,"value":698},"FeedForward",{"type":49,"tag":559,"props":700,"children":701},{},[702],{"type":55,"value":703},"-",{"type":49,"tag":559,"props":705,"children":706},{},[707],{"type":55,"value":649},{"type":49,"tag":559,"props":709,"children":710},{},[711,716],{"type":49,"tag":64,"props":712,"children":713},{},[714],{"type":55,"value":715},"GELU",{"type":55,"value":717}," (not GEGLU!)",{"type":49,"tag":522,"props":719,"children":720},{},[721,726,731,735,740],{"type":49,"tag":559,"props":722,"children":723},{},[724],{"type":55,"value":725},"Rotary position encoding",{"type":49,"tag":559,"props":727,"children":728},{},[729],{"type":55,"value":730},"LTXVideoRotaryPosEmbed",{"type":49,"tag":559,"props":732,"children":733},{},[734],{"type":55,"value":703},{"type":49,"tag":559,"props":736,"children":737},{},[738],{"type":55,"value":739},"1",{"type":49,"tag":559,"props":741,"children":742},{},[743],{"type":55,"value":744},"RoPE 3D",{"type":49,"tag":58,"props":746,"children":747},{},[748,753],{"type":49,"tag":64,"props":749,"children":750},{},[751],{"type":55,"value":752},"Total RMSNorm modules: 168",{"type":55,"value":754}," (56 with weights, 112 without)",{"type":49,"tag":86,"props":756,"children":758},{"id":757},"target-kernels",[759],{"type":55,"value":760},"Target Kernels",{"type":49,"tag":514,"props":762,"children":763},{},[764,789],{"type":49,"tag":518,"props":765,"children":766},{},[767],{"type":49,"tag":522,"props":768,"children":769},{},[770,774,779,784],{"type":49,"tag":526,"props":771,"children":772},{},[773],{"type":55,"value":550},{"type":49,"tag":526,"props":775,"children":776},{},[777],{"type":55,"value":778},"Use Case",{"type":49,"tag":526,"props":780,"children":781},{},[782],{"type":55,"value":783},"Input Layout",{"type":49,"tag":526,"props":785,"children":786},{},[787],{"type":55,"value":788},"Key Challenge",{"type":49,"tag":552,"props":790,"children":791},{},[792,821,850,888],{"type":49,"tag":522,"props":793,"children":794},{},[795,802,807,816],{"type":49,"tag":559,"props":796,"children":797},{},[798],{"type":49,"tag":64,"props":799,"children":800},{},[801],{"type":55,"value":572},{"type":49,"tag":559,"props":803,"children":804},{},[805],{"type":55,"value":806},"Normalization",{"type":49,"tag":559,"props":808,"children":809},{},[810],{"type":49,"tag":109,"props":811,"children":813},{"className":812},[],[814],{"type":55,"value":815},"[..., hidden_size]",{"type":49,"tag":559,"props":817,"children":818},{},[819],{"type":55,"value":820},"Weight may be None; 168 instances",{"type":49,"tag":522,"props":822,"children":823},{},[824,831,836,845],{"type":49,"tag":559,"props":825,"children":826},{},[827],{"type":49,"tag":64,"props":828,"children":829},{},[830],{"type":55,"value":744},{"type":49,"tag":559,"props":832,"children":833},{},[834],{"type":55,"value":835},"Video position encoding",{"type":49,"tag":559,"props":837,"children":838},{},[839],{"type":49,"tag":109,"props":840,"children":842},{"className":841},[],[843],{"type":55,"value":844},"[batch, t*h*w, heads, head_dim]",{"type":49,"tag":559,"props":846,"children":847},{},[848],{"type":55,"value":849},"3D → temporal + spatial decomposition",{"type":49,"tag":522,"props":851,"children":852},{},[853,861,866,883],{"type":49,"tag":559,"props":854,"children":855},{},[856],{"type":49,"tag":64,"props":857,"children":858},{},[859],{"type":55,"value":860},"GEGLU",{"type":49,"tag":559,"props":862,"children":863},{},[864],{"type":55,"value":865},"Gated activation (SD3\u002FFLUX)",{"type":49,"tag":559,"props":867,"children":868},{},[869,875,877],{"type":49,"tag":109,"props":870,"children":872},{"className":871},[],[873],{"type":55,"value":874},"[batch, seq, 2*hidden]",{"type":55,"value":876}," → ",{"type":49,"tag":109,"props":878,"children":880},{"className":879},[],[881],{"type":55,"value":882},"[batch, seq, hidden]",{"type":49,"tag":559,"props":884,"children":885},{},[886],{"type":55,"value":887},"Gate\u002Fvalue split",{"type":49,"tag":522,"props":889,"children":890},{},[891,899,904,913],{"type":49,"tag":559,"props":892,"children":893},{},[894],{"type":49,"tag":64,"props":895,"children":896},{},[897],{"type":55,"value":898},"AdaLN",{"type":49,"tag":559,"props":900,"children":901},{},[902],{"type":55,"value":903},"Conditioned normalization (DiT)",{"type":49,"tag":559,"props":905,"children":906},{},[907],{"type":49,"tag":109,"props":908,"children":910},{"className":909},[],[911],{"type":55,"value":912},"norm(x) * weight * (1+scale) + shift",{"type":49,"tag":559,"props":914,"children":915},{},[916],{"type":55,"value":917},"Fused norm + condition",{"type":49,"tag":79,"props":919,"children":921},{"id":920},"supported-hardware",[922],{"type":55,"value":923},"Supported Hardware",{"type":49,"tag":514,"props":925,"children":926},{},[927,969],{"type":49,"tag":518,"props":928,"children":929},{},[930],{"type":49,"tag":522,"props":931,"children":932},{},[933,938,943,948,953,958,963],{"type":49,"tag":526,"props":934,"children":935},{},[936],{"type":55,"value":937},"GPU",{"type":49,"tag":526,"props":939,"children":940},{},[941],{"type":55,"value":942},"Architecture",{"type":49,"tag":526,"props":944,"children":945},{},[946],{"type":55,"value":947},"Wave Size",{"type":49,"tag":526,"props":949,"children":950},{},[951],{"type":55,"value":952},"LDS\u002FCU",{"type":49,"tag":526,"props":954,"children":955},{},[956],{"type":55,"value":957},"Mem BW",{"type":49,"tag":526,"props":959,"children":960},{},[961],{"type":55,"value":962},"Key Feature",{"type":49,"tag":526,"props":964,"children":966},{"align":965},"center",[967],{"type":55,"value":968},"Verified",{"type":49,"tag":552,"props":970,"children":971},{},[972,1015],{"type":49,"tag":522,"props":973,"children":974},{},[975,983,988,993,1001,1006,1011],{"type":49,"tag":559,"props":976,"children":977},{},[978],{"type":49,"tag":64,"props":979,"children":980},{},[981],{"type":55,"value":982},"MI355X",{"type":49,"tag":559,"props":984,"children":985},{},[986],{"type":55,"value":987},"CDNA3+ (gfx950)",{"type":49,"tag":559,"props":989,"children":990},{},[991],{"type":55,"value":992},"Wave64",{"type":49,"tag":559,"props":994,"children":995},{},[996],{"type":49,"tag":64,"props":997,"children":998},{},[999],{"type":55,"value":1000},"160 KB",{"type":49,"tag":559,"props":1002,"children":1003},{},[1004],{"type":55,"value":1005},"8 TB\u002Fs",{"type":49,"tag":559,"props":1007,"children":1008},{},[1009],{"type":55,"value":1010},"32 XCDs, XCD Swizzle for GEMM",{"type":49,"tag":559,"props":1012,"children":1013},{"align":965},[1014],{"type":55,"value":644},{"type":49,"tag":522,"props":1016,"children":1017},{},[1018,1026,1031,1039,1044,1049,1054],{"type":49,"tag":559,"props":1019,"children":1020},{},[1021],{"type":49,"tag":64,"props":1022,"children":1023},{},[1024],{"type":55,"value":1025},"R9700",{"type":49,"tag":559,"props":1027,"children":1028},{},[1029],{"type":55,"value":1030},"RDNA4 (gfx1201)",{"type":49,"tag":559,"props":1032,"children":1033},{},[1034],{"type":49,"tag":64,"props":1035,"children":1036},{},[1037],{"type":55,"value":1038},"Wave32",{"type":49,"tag":559,"props":1040,"children":1041},{},[1042],{"type":55,"value":1043},"64 KB",{"type":49,"tag":559,"props":1045,"children":1046},{},[1047],{"type":55,"value":1048},"~608 GB\u002Fs",{"type":49,"tag":559,"props":1050,"children":1051},{},[1052],{"type":55,"value":1053},"256B cacheline, inference-focused",{"type":49,"tag":559,"props":1055,"children":1056},{"align":965},[1057],{"type":55,"value":644},{"type":49,"tag":1059,"props":1060,"children":1061},"blockquote",{},[1062],{"type":49,"tag":58,"props":1063,"children":1064},{},[1065,1067,1074,1076],{"type":55,"value":1066},"See ",{"type":49,"tag":1068,"props":1069,"children":1071},"a",{"href":1070},"references\u002Fmi355x-optimization-guide.md",[1072],{"type":55,"value":1073},"MI355X guide",{"type":55,"value":1075}," | ",{"type":49,"tag":1068,"props":1077,"children":1079},{"href":1078},"references\u002Fr9700-optimization-guide.md",[1080],{"type":55,"value":1081},"R9700 guide",{"type":49,"tag":79,"props":1083,"children":1085},{"id":1084},"when-this-skill-applies",[1086],{"type":55,"value":1087},"When This Skill Applies",{"type":49,"tag":58,"props":1089,"children":1090},{},[1091],{"type":55,"value":1092},"Use this skill when:",{"type":49,"tag":1094,"props":1095,"children":1096},"ul",{},[1097,1110,1121,1126,1131],{"type":49,"tag":1098,"props":1099,"children":1100},"li",{},[1101,1103,1108],{"type":55,"value":1102},"Writing Triton kernels for ",{"type":49,"tag":64,"props":1104,"children":1105},{},[1106],{"type":55,"value":1107},"RMSNorm, RoPE, GEGLU, AdaLN",{"type":55,"value":1109}," on AMD GPUs",{"type":49,"tag":1098,"props":1111,"children":1112},{},[1113,1115,1119],{"type":55,"value":1114},"Integrating custom kernels with ",{"type":49,"tag":64,"props":1116,"children":1117},{},[1118],{"type":55,"value":68},{"type":55,"value":1120}," pipelines (LTX-Video, SD3, FLUX)",{"type":49,"tag":1098,"props":1122,"children":1123},{},[1124],{"type":55,"value":1125},"Benchmarking kernel performance against PyTorch baseline on ROCm",{"type":49,"tag":1098,"props":1127,"children":1128},{},[1129],{"type":55,"value":1130},"Optimizing existing kernels for MI355X or R9700 architecture",{"type":49,"tag":1098,"props":1132,"children":1133},{},[1134],{"type":55,"value":1135},"Debugging ROCm\u002FHIP-specific kernel issues",{"type":49,"tag":79,"props":1137,"children":1139},{"id":1138},"critical-rocm-constraints",[1140],{"type":55,"value":1141},"Critical ROCm Constraints",{"type":49,"tag":86,"props":1143,"children":1145},{"id":1144},"things-that-dont-work-on-amd",[1146],{"type":55,"value":1147},"Things That DON'T Work on AMD",{"type":49,"tag":101,"props":1149,"children":1151},{"className":103,"code":1150,"language":105,"meta":106,"style":106},"# FORBIDDEN - CUDA only, NOT available on ROCm\ntl.libdevice.tanh(x)          # Use manual formula below\ntl.libdevice.log1p(x)         # Use: tl.log(1.0 + x)\ntl.math.tanh(x)               # Also NOT available on ROCm Triton\n\n# Manual tanh (ONLY reliable method on ROCm):\ne2x = tl.exp(2.0 * x)\ntanh_x = (e2x - 1.0) \u002F (e2x + 1.0)\n\n# FORBIDDEN - Triton limitations on ROCm\nbreak \u002F continue               # Use: tl.where()\nmin(a, b) \u002F max(a, b)          # Use: tl.minimum(a, b) \u002F tl.maximum(a, b)\n",[1152],{"type":49,"tag":109,"props":1153,"children":1154},{"__ignoreMap":106},[1155,1163,1171,1179,1187,1194,1202,1210,1218,1225,1234,1243],{"type":49,"tag":113,"props":1156,"children":1157},{"class":115,"line":116},[1158],{"type":49,"tag":113,"props":1159,"children":1160},{},[1161],{"type":55,"value":1162},"# FORBIDDEN - CUDA only, NOT available on ROCm\n",{"type":49,"tag":113,"props":1164,"children":1165},{"class":115,"line":125},[1166],{"type":49,"tag":113,"props":1167,"children":1168},{},[1169],{"type":55,"value":1170},"tl.libdevice.tanh(x)          # Use manual formula below\n",{"type":49,"tag":113,"props":1172,"children":1173},{"class":115,"line":134},[1174],{"type":49,"tag":113,"props":1175,"children":1176},{},[1177],{"type":55,"value":1178},"tl.libdevice.log1p(x)         # Use: tl.log(1.0 + x)\n",{"type":49,"tag":113,"props":1180,"children":1181},{"class":115,"line":143},[1182],{"type":49,"tag":113,"props":1183,"children":1184},{},[1185],{"type":55,"value":1186},"tl.math.tanh(x)               # Also NOT available on ROCm Triton\n",{"type":49,"tag":113,"props":1188,"children":1189},{"class":115,"line":152},[1190],{"type":49,"tag":113,"props":1191,"children":1192},{"emptyLinePlaceholder":42},[1193],{"type":55,"value":149},{"type":49,"tag":113,"props":1195,"children":1196},{"class":115,"line":161},[1197],{"type":49,"tag":113,"props":1198,"children":1199},{},[1200],{"type":55,"value":1201},"# Manual tanh (ONLY reliable method on ROCm):\n",{"type":49,"tag":113,"props":1203,"children":1204},{"class":115,"line":170},[1205],{"type":49,"tag":113,"props":1206,"children":1207},{},[1208],{"type":55,"value":1209},"e2x = tl.exp(2.0 * x)\n",{"type":49,"tag":113,"props":1211,"children":1212},{"class":115,"line":179},[1213],{"type":49,"tag":113,"props":1214,"children":1215},{},[1216],{"type":55,"value":1217},"tanh_x = (e2x - 1.0) \u002F (e2x + 1.0)\n",{"type":49,"tag":113,"props":1219,"children":1220},{"class":115,"line":188},[1221],{"type":49,"tag":113,"props":1222,"children":1223},{"emptyLinePlaceholder":42},[1224],{"type":55,"value":149},{"type":49,"tag":113,"props":1226,"children":1228},{"class":115,"line":1227},10,[1229],{"type":49,"tag":113,"props":1230,"children":1231},{},[1232],{"type":55,"value":1233},"# FORBIDDEN - Triton limitations on ROCm\n",{"type":49,"tag":113,"props":1235,"children":1237},{"class":115,"line":1236},11,[1238],{"type":49,"tag":113,"props":1239,"children":1240},{},[1241],{"type":55,"value":1242},"break \u002F continue               # Use: tl.where()\n",{"type":49,"tag":113,"props":1244,"children":1246},{"class":115,"line":1245},12,[1247],{"type":49,"tag":113,"props":1248,"children":1249},{},[1250],{"type":55,"value":1251},"min(a, b) \u002F max(a, b)          # Use: tl.minimum(a, b) \u002F tl.maximum(a, b)\n",{"type":49,"tag":86,"props":1253,"children":1255},{"id":1254},"mandatory-environment-variables",[1256],{"type":55,"value":1257},"Mandatory Environment Variables",{"type":49,"tag":101,"props":1259,"children":1261},{"className":103,"code":1260,"language":105,"meta":106,"style":106},"import os\nos.environ['TRITON_HIP_USE_BLOCK_PINGPONG'] = '1'\nos.environ['TRITON_HIP_USE_ASYNC_COPY'] = '1'\n",[1262],{"type":49,"tag":109,"props":1263,"children":1264},{"__ignoreMap":106},[1265,1272,1279],{"type":49,"tag":113,"props":1266,"children":1267},{"class":115,"line":116},[1268],{"type":49,"tag":113,"props":1269,"children":1270},{},[1271],{"type":55,"value":122},{"type":49,"tag":113,"props":1273,"children":1274},{"class":115,"line":125},[1275],{"type":49,"tag":113,"props":1276,"children":1277},{},[1278],{"type":55,"value":131},{"type":49,"tag":113,"props":1280,"children":1281},{"class":115,"line":134},[1282],{"type":49,"tag":113,"props":1283,"children":1284},{},[1285],{"type":55,"value":140},{"type":49,"tag":79,"props":1287,"children":1289},{"id":1288},"core-kernel-implementations",[1290],{"type":55,"value":1291},"Core Kernel Implementations",{"type":49,"tag":86,"props":1293,"children":1295},{"id":1294},"_1-rmsnorm-core-optimization-target",[1296],{"type":55,"value":1297},"1. RMSNorm (Core Optimization Target)",{"type":49,"tag":58,"props":1299,"children":1300},{},[1301,1303,1308],{"type":55,"value":1302},"Row-wise reduction pattern. ",{"type":49,"tag":64,"props":1304,"children":1305},{},[1306],{"type":55,"value":1307},"168 instances",{"type":55,"value":1309}," in LTX-Video, ~5% of total compute.",{"type":49,"tag":58,"props":1311,"children":1312},{},[1313,1318,1320,1326,1328,1334],{"type":49,"tag":64,"props":1314,"children":1315},{},[1316],{"type":55,"value":1317},"CRITICAL: Do NOT autotune BLOCK_D.",{"type":55,"value":1319}," Autotune may pick ",{"type":49,"tag":109,"props":1321,"children":1323},{"className":1322},[],[1324],{"type":55,"value":1325},"BLOCK_D \u003C D",{"type":55,"value":1327},", causing partial row processing and wrong results. Always compute ",{"type":49,"tag":109,"props":1329,"children":1331},{"className":1330},[],[1332],{"type":55,"value":1333},"BLOCK_D = triton.next_power_of_2(D)",{"type":55,"value":1335}," in the Python wrapper.",{"type":49,"tag":101,"props":1337,"children":1339},{"className":103,"code":1338,"language":105,"meta":106,"style":106},"@triton.jit\ndef rmsnorm_kernel(\n    x_ptr, weight_ptr, out_ptr,\n    stride_x, D,\n    eps: tl.constexpr,\n    HAS_WEIGHT: tl.constexpr,\n    BLOCK_D: tl.constexpr,\n):\n    row = tl.program_id(0)\n    offs = tl.arange(0, BLOCK_D)\n    mask = offs \u003C D\n    x = tl.load(x_ptr + row * stride_x + offs, mask=mask, other=0.0).to(tl.float32)\n\n    variance = tl.sum(x * x, axis=0) \u002F D\n    rms_inv = tl.rsqrt(variance + eps)\n\n    if HAS_WEIGHT:\n        w = tl.load(weight_ptr + offs, mask=mask, other=1.0).to(tl.float32)\n        out = x * rms_inv * w\n    else:\n        out = x * rms_inv\n\n    tl.store(out_ptr + row * stride_x + offs, out.to(x.dtype), mask=mask)\n\n\ndef triton_rmsnorm(x, weight=None, eps=1e-6):\n    x_2d = x.contiguous().view(-1, x.shape[-1])\n    out = torch.empty_like(x_2d)\n    M, D = x_2d.shape\n    has_weight = weight is not None\n    if not has_weight:\n        weight = torch.empty(0, device=x.device)\n\n    BLOCK_D = triton.next_power_of_2(D)\n    num_warps = 4 if BLOCK_D \u003C= 1024 else (8 if BLOCK_D \u003C= 4096 else 16)\n    rmsnorm_kernel[(M,)](\n        x_2d, weight, out, x_2d.stride(0), D, eps, has_weight,\n        BLOCK_D=BLOCK_D, num_warps=num_warps, num_stages=2,\n    )\n    return out.view_as(x)\n",[1340],{"type":49,"tag":109,"props":1341,"children":1342},{"__ignoreMap":106},[1343,1351,1359,1367,1375,1383,1391,1399,1407,1415,1423,1431,1439,1447,1456,1465,1473,1482,1491,1500,1509,1518,1526,1535,1543,1551,1560,1569,1578,1587,1596,1605,1614,1622,1631,1640,1649,1658,1667,1676],{"type":49,"tag":113,"props":1344,"children":1345},{"class":115,"line":116},[1346],{"type":49,"tag":113,"props":1347,"children":1348},{},[1349],{"type":55,"value":1350},"@triton.jit\n",{"type":49,"tag":113,"props":1352,"children":1353},{"class":115,"line":125},[1354],{"type":49,"tag":113,"props":1355,"children":1356},{},[1357],{"type":55,"value":1358},"def rmsnorm_kernel(\n",{"type":49,"tag":113,"props":1360,"children":1361},{"class":115,"line":134},[1362],{"type":49,"tag":113,"props":1363,"children":1364},{},[1365],{"type":55,"value":1366},"    x_ptr, weight_ptr, out_ptr,\n",{"type":49,"tag":113,"props":1368,"children":1369},{"class":115,"line":143},[1370],{"type":49,"tag":113,"props":1371,"children":1372},{},[1373],{"type":55,"value":1374},"    stride_x, D,\n",{"type":49,"tag":113,"props":1376,"children":1377},{"class":115,"line":152},[1378],{"type":49,"tag":113,"props":1379,"children":1380},{},[1381],{"type":55,"value":1382},"    eps: tl.constexpr,\n",{"type":49,"tag":113,"props":1384,"children":1385},{"class":115,"line":161},[1386],{"type":49,"tag":113,"props":1387,"children":1388},{},[1389],{"type":55,"value":1390},"    HAS_WEIGHT: tl.constexpr,\n",{"type":49,"tag":113,"props":1392,"children":1393},{"class":115,"line":170},[1394],{"type":49,"tag":113,"props":1395,"children":1396},{},[1397],{"type":55,"value":1398},"    BLOCK_D: tl.constexpr,\n",{"type":49,"tag":113,"props":1400,"children":1401},{"class":115,"line":179},[1402],{"type":49,"tag":113,"props":1403,"children":1404},{},[1405],{"type":55,"value":1406},"):\n",{"type":49,"tag":113,"props":1408,"children":1409},{"class":115,"line":188},[1410],{"type":49,"tag":113,"props":1411,"children":1412},{},[1413],{"type":55,"value":1414},"    row = tl.program_id(0)\n",{"type":49,"tag":113,"props":1416,"children":1417},{"class":115,"line":1227},[1418],{"type":49,"tag":113,"props":1419,"children":1420},{},[1421],{"type":55,"value":1422},"    offs = tl.arange(0, BLOCK_D)\n",{"type":49,"tag":113,"props":1424,"children":1425},{"class":115,"line":1236},[1426],{"type":49,"tag":113,"props":1427,"children":1428},{},[1429],{"type":55,"value":1430},"    mask = offs \u003C D\n",{"type":49,"tag":113,"props":1432,"children":1433},{"class":115,"line":1245},[1434],{"type":49,"tag":113,"props":1435,"children":1436},{},[1437],{"type":55,"value":1438},"    x = tl.load(x_ptr + row * stride_x + offs, mask=mask, other=0.0).to(tl.float32)\n",{"type":49,"tag":113,"props":1440,"children":1442},{"class":115,"line":1441},13,[1443],{"type":49,"tag":113,"props":1444,"children":1445},{"emptyLinePlaceholder":42},[1446],{"type":55,"value":149},{"type":49,"tag":113,"props":1448,"children":1450},{"class":115,"line":1449},14,[1451],{"type":49,"tag":113,"props":1452,"children":1453},{},[1454],{"type":55,"value":1455},"    variance = tl.sum(x * x, axis=0) \u002F D\n",{"type":49,"tag":113,"props":1457,"children":1459},{"class":115,"line":1458},15,[1460],{"type":49,"tag":113,"props":1461,"children":1462},{},[1463],{"type":55,"value":1464},"    rms_inv = tl.rsqrt(variance + eps)\n",{"type":49,"tag":113,"props":1466,"children":1468},{"class":115,"line":1467},16,[1469],{"type":49,"tag":113,"props":1470,"children":1471},{"emptyLinePlaceholder":42},[1472],{"type":55,"value":149},{"type":49,"tag":113,"props":1474,"children":1476},{"class":115,"line":1475},17,[1477],{"type":49,"tag":113,"props":1478,"children":1479},{},[1480],{"type":55,"value":1481},"    if HAS_WEIGHT:\n",{"type":49,"tag":113,"props":1483,"children":1485},{"class":115,"line":1484},18,[1486],{"type":49,"tag":113,"props":1487,"children":1488},{},[1489],{"type":55,"value":1490},"        w = tl.load(weight_ptr + offs, mask=mask, other=1.0).to(tl.float32)\n",{"type":49,"tag":113,"props":1492,"children":1494},{"class":115,"line":1493},19,[1495],{"type":49,"tag":113,"props":1496,"children":1497},{},[1498],{"type":55,"value":1499},"        out = x * rms_inv * w\n",{"type":49,"tag":113,"props":1501,"children":1503},{"class":115,"line":1502},20,[1504],{"type":49,"tag":113,"props":1505,"children":1506},{},[1507],{"type":55,"value":1508},"    else:\n",{"type":49,"tag":113,"props":1510,"children":1512},{"class":115,"line":1511},21,[1513],{"type":49,"tag":113,"props":1514,"children":1515},{},[1516],{"type":55,"value":1517},"        out = x * rms_inv\n",{"type":49,"tag":113,"props":1519,"children":1521},{"class":115,"line":1520},22,[1522],{"type":49,"tag":113,"props":1523,"children":1524},{"emptyLinePlaceholder":42},[1525],{"type":55,"value":149},{"type":49,"tag":113,"props":1527,"children":1529},{"class":115,"line":1528},23,[1530],{"type":49,"tag":113,"props":1531,"children":1532},{},[1533],{"type":55,"value":1534},"    tl.store(out_ptr + row * stride_x + offs, out.to(x.dtype), mask=mask)\n",{"type":49,"tag":113,"props":1536,"children":1538},{"class":115,"line":1537},24,[1539],{"type":49,"tag":113,"props":1540,"children":1541},{"emptyLinePlaceholder":42},[1542],{"type":55,"value":149},{"type":49,"tag":113,"props":1544,"children":1546},{"class":115,"line":1545},25,[1547],{"type":49,"tag":113,"props":1548,"children":1549},{"emptyLinePlaceholder":42},[1550],{"type":55,"value":149},{"type":49,"tag":113,"props":1552,"children":1554},{"class":115,"line":1553},26,[1555],{"type":49,"tag":113,"props":1556,"children":1557},{},[1558],{"type":55,"value":1559},"def triton_rmsnorm(x, weight=None, eps=1e-6):\n",{"type":49,"tag":113,"props":1561,"children":1563},{"class":115,"line":1562},27,[1564],{"type":49,"tag":113,"props":1565,"children":1566},{},[1567],{"type":55,"value":1568},"    x_2d = x.contiguous().view(-1, x.shape[-1])\n",{"type":49,"tag":113,"props":1570,"children":1572},{"class":115,"line":1571},28,[1573],{"type":49,"tag":113,"props":1574,"children":1575},{},[1576],{"type":55,"value":1577},"    out = torch.empty_like(x_2d)\n",{"type":49,"tag":113,"props":1579,"children":1581},{"class":115,"line":1580},29,[1582],{"type":49,"tag":113,"props":1583,"children":1584},{},[1585],{"type":55,"value":1586},"    M, D = x_2d.shape\n",{"type":49,"tag":113,"props":1588,"children":1590},{"class":115,"line":1589},30,[1591],{"type":49,"tag":113,"props":1592,"children":1593},{},[1594],{"type":55,"value":1595},"    has_weight = weight is not None\n",{"type":49,"tag":113,"props":1597,"children":1599},{"class":115,"line":1598},31,[1600],{"type":49,"tag":113,"props":1601,"children":1602},{},[1603],{"type":55,"value":1604},"    if not has_weight:\n",{"type":49,"tag":113,"props":1606,"children":1608},{"class":115,"line":1607},32,[1609],{"type":49,"tag":113,"props":1610,"children":1611},{},[1612],{"type":55,"value":1613},"        weight = torch.empty(0, device=x.device)\n",{"type":49,"tag":113,"props":1615,"children":1617},{"class":115,"line":1616},33,[1618],{"type":49,"tag":113,"props":1619,"children":1620},{"emptyLinePlaceholder":42},[1621],{"type":55,"value":149},{"type":49,"tag":113,"props":1623,"children":1625},{"class":115,"line":1624},34,[1626],{"type":49,"tag":113,"props":1627,"children":1628},{},[1629],{"type":55,"value":1630},"    BLOCK_D = triton.next_power_of_2(D)\n",{"type":49,"tag":113,"props":1632,"children":1634},{"class":115,"line":1633},35,[1635],{"type":49,"tag":113,"props":1636,"children":1637},{},[1638],{"type":55,"value":1639},"    num_warps = 4 if BLOCK_D \u003C= 1024 else (8 if BLOCK_D \u003C= 4096 else 16)\n",{"type":49,"tag":113,"props":1641,"children":1643},{"class":115,"line":1642},36,[1644],{"type":49,"tag":113,"props":1645,"children":1646},{},[1647],{"type":55,"value":1648},"    rmsnorm_kernel[(M,)](\n",{"type":49,"tag":113,"props":1650,"children":1652},{"class":115,"line":1651},37,[1653],{"type":49,"tag":113,"props":1654,"children":1655},{},[1656],{"type":55,"value":1657},"        x_2d, weight, out, x_2d.stride(0), D, eps, has_weight,\n",{"type":49,"tag":113,"props":1659,"children":1661},{"class":115,"line":1660},38,[1662],{"type":49,"tag":113,"props":1663,"children":1664},{},[1665],{"type":55,"value":1666},"        BLOCK_D=BLOCK_D, num_warps=num_warps, num_stages=2,\n",{"type":49,"tag":113,"props":1668,"children":1670},{"class":115,"line":1669},39,[1671],{"type":49,"tag":113,"props":1672,"children":1673},{},[1674],{"type":55,"value":1675},"    )\n",{"type":49,"tag":113,"props":1677,"children":1679},{"class":115,"line":1678},40,[1680],{"type":49,"tag":113,"props":1681,"children":1682},{},[1683],{"type":55,"value":1684},"    return out.view_as(x)\n",{"type":49,"tag":58,"props":1686,"children":1687},{},[1688],{"type":49,"tag":64,"props":1689,"children":1690},{},[1691],{"type":55,"value":1692},"LTX-Video pitfall: Weight may be None!",{"type":49,"tag":101,"props":1694,"children":1696},{"className":103,"code":1695,"language":105,"meta":106,"style":106},"has_weight = hasattr(module, 'weight') and module.weight is not None\n",[1697],{"type":49,"tag":109,"props":1698,"children":1699},{"__ignoreMap":106},[1700],{"type":49,"tag":113,"props":1701,"children":1702},{"class":115,"line":116},[1703],{"type":49,"tag":113,"props":1704,"children":1705},{},[1706],{"type":55,"value":1695},{"type":49,"tag":86,"props":1708,"children":1710},{"id":1709},"_2-rope-3d-video-position-encoding",[1711],{"type":55,"value":1712},"2. RoPE 3D (Video Position Encoding)",{"type":49,"tag":58,"props":1714,"children":1715},{},[1716,1718,1724],{"type":55,"value":1717},"Element-wise pattern. LTX-Video splits ",{"type":49,"tag":109,"props":1719,"children":1721},{"className":1720},[],[1722],{"type":55,"value":1723},"head_dim",{"type":55,"value":1725}," into temporal + spatial components.",{"type":49,"tag":58,"props":1727,"children":1728},{},[1729,1742,1744,1750,1752,1758],{"type":49,"tag":64,"props":1730,"children":1731},{},[1732,1734,1740],{"type":55,"value":1733},"CRITICAL: cos\u002Fsin have shape ",{"type":49,"tag":109,"props":1735,"children":1737},{"className":1736},[],[1738],{"type":55,"value":1739},"[seq_len, head_dim]",{"type":55,"value":1741},".",{"type":55,"value":1743}," When grid flattens batch dimension (",{"type":49,"tag":109,"props":1745,"children":1747},{"className":1746},[],[1748],{"type":55,"value":1749},"batch * seq_len",{"type":55,"value":1751},"), use ",{"type":49,"tag":109,"props":1753,"children":1755},{"className":1754},[],[1756],{"type":55,"value":1757},"pid_s % seq_len",{"type":55,"value":1759}," to index cos\u002Fsin, otherwise batch > 1 causes OOB GPU crash.",{"type":49,"tag":101,"props":1761,"children":1763},{"className":103,"code":1762,"language":105,"meta":106,"style":106},"@triton.jit\ndef rope_3d_kernel(\n    qk_ptr, cos_ptr, sin_ptr, out_ptr,\n    seq_len, num_heads, head_dim,\n    stride_s, stride_h, stride_d,\n    BLOCK_HD: tl.constexpr,\n):\n    pid_s = tl.program_id(0)  # batch * seq_len\n    pid_h = tl.program_id(1)  # head index\n    half_dim = head_dim \u002F\u002F 2\n    offs = tl.arange(0, BLOCK_HD)\n    mask = offs \u003C half_dim\n\n    base = pid_s * stride_s + pid_h * stride_h\n    x0 = tl.load(qk_ptr + base + offs, mask=mask, other=0.0).to(tl.float32)\n    x1 = tl.load(qk_ptr + base + half_dim + offs, mask=mask, other=0.0).to(tl.float32)\n\n    seq_idx = pid_s % seq_len  # wrap for batch > 1\n    cos_val = tl.load(cos_ptr + seq_idx * head_dim + offs, mask=mask, other=1.0).to(tl.float32)\n    sin_val = tl.load(sin_ptr + seq_idx * head_dim + offs, mask=mask, other=0.0).to(tl.float32)\n\n    out0 = x0 * cos_val - x1 * sin_val\n    out1 = x0 * sin_val + x1 * cos_val\n\n    tl.store(out_ptr + base + offs, out0.to(x0.dtype), mask=mask)\n    tl.store(out_ptr + base + half_dim + offs, out1.to(x0.dtype), mask=mask)\n\n\ndef triton_rope_3d(qk, cos, sin):\n    qk = qk.contiguous()\n    out = torch.empty_like(qk)\n    batch, seq_len, num_heads, head_dim = qk.shape\n    qk_flat = qk.view(batch * seq_len, num_heads, head_dim)\n    out_flat = out.view(batch * seq_len, num_heads, head_dim)\n    BLOCK_HD = triton.next_power_of_2(head_dim \u002F\u002F 2)\n    num_warps = 4 if BLOCK_HD \u003C= 64 else 8\n    rope_3d_kernel[(batch * seq_len, num_heads)](\n        qk_flat, cos, sin, out_flat,\n        seq_len, num_heads, head_dim,\n        qk_flat.stride(0), qk_flat.stride(1), qk_flat.stride(2),\n        BLOCK_HD=BLOCK_HD, num_warps=num_warps, num_stages=2,\n    )\n    return out\n",[1764],{"type":49,"tag":109,"props":1765,"children":1766},{"__ignoreMap":106},[1767,1774,1782,1790,1798,1806,1814,1821,1829,1837,1845,1853,1861,1868,1876,1884,1892,1899,1907,1915,1923,1930,1938,1946,1953,1961,1969,1976,1983,1991,1999,2007,2015,2023,2031,2039,2047,2055,2063,2071,2079,2088,2096],{"type":49,"tag":113,"props":1768,"children":1769},{"class":115,"line":116},[1770],{"type":49,"tag":113,"props":1771,"children":1772},{},[1773],{"type":55,"value":1350},{"type":49,"tag":113,"props":1775,"children":1776},{"class":115,"line":125},[1777],{"type":49,"tag":113,"props":1778,"children":1779},{},[1780],{"type":55,"value":1781},"def rope_3d_kernel(\n",{"type":49,"tag":113,"props":1783,"children":1784},{"class":115,"line":134},[1785],{"type":49,"tag":113,"props":1786,"children":1787},{},[1788],{"type":55,"value":1789},"    qk_ptr, cos_ptr, sin_ptr, out_ptr,\n",{"type":49,"tag":113,"props":1791,"children":1792},{"class":115,"line":143},[1793],{"type":49,"tag":113,"props":1794,"children":1795},{},[1796],{"type":55,"value":1797},"    seq_len, num_heads, head_dim,\n",{"type":49,"tag":113,"props":1799,"children":1800},{"class":115,"line":152},[1801],{"type":49,"tag":113,"props":1802,"children":1803},{},[1804],{"type":55,"value":1805},"    stride_s, stride_h, stride_d,\n",{"type":49,"tag":113,"props":1807,"children":1808},{"class":115,"line":161},[1809],{"type":49,"tag":113,"props":1810,"children":1811},{},[1812],{"type":55,"value":1813},"    BLOCK_HD: tl.constexpr,\n",{"type":49,"tag":113,"props":1815,"children":1816},{"class":115,"line":170},[1817],{"type":49,"tag":113,"props":1818,"children":1819},{},[1820],{"type":55,"value":1406},{"type":49,"tag":113,"props":1822,"children":1823},{"class":115,"line":179},[1824],{"type":49,"tag":113,"props":1825,"children":1826},{},[1827],{"type":55,"value":1828},"    pid_s = tl.program_id(0)  # batch * seq_len\n",{"type":49,"tag":113,"props":1830,"children":1831},{"class":115,"line":188},[1832],{"type":49,"tag":113,"props":1833,"children":1834},{},[1835],{"type":55,"value":1836},"    pid_h = tl.program_id(1)  # head index\n",{"type":49,"tag":113,"props":1838,"children":1839},{"class":115,"line":1227},[1840],{"type":49,"tag":113,"props":1841,"children":1842},{},[1843],{"type":55,"value":1844},"    half_dim = head_dim \u002F\u002F 2\n",{"type":49,"tag":113,"props":1846,"children":1847},{"class":115,"line":1236},[1848],{"type":49,"tag":113,"props":1849,"children":1850},{},[1851],{"type":55,"value":1852},"    offs = tl.arange(0, BLOCK_HD)\n",{"type":49,"tag":113,"props":1854,"children":1855},{"class":115,"line":1245},[1856],{"type":49,"tag":113,"props":1857,"children":1858},{},[1859],{"type":55,"value":1860},"    mask = offs \u003C half_dim\n",{"type":49,"tag":113,"props":1862,"children":1863},{"class":115,"line":1441},[1864],{"type":49,"tag":113,"props":1865,"children":1866},{"emptyLinePlaceholder":42},[1867],{"type":55,"value":149},{"type":49,"tag":113,"props":1869,"children":1870},{"class":115,"line":1449},[1871],{"type":49,"tag":113,"props":1872,"children":1873},{},[1874],{"type":55,"value":1875},"    base = pid_s * stride_s + pid_h * stride_h\n",{"type":49,"tag":113,"props":1877,"children":1878},{"class":115,"line":1458},[1879],{"type":49,"tag":113,"props":1880,"children":1881},{},[1882],{"type":55,"value":1883},"    x0 = tl.load(qk_ptr + base + offs, mask=mask, other=0.0).to(tl.float32)\n",{"type":49,"tag":113,"props":1885,"children":1886},{"class":115,"line":1467},[1887],{"type":49,"tag":113,"props":1888,"children":1889},{},[1890],{"type":55,"value":1891},"    x1 = tl.load(qk_ptr + base + half_dim + offs, mask=mask, other=0.0).to(tl.float32)\n",{"type":49,"tag":113,"props":1893,"children":1894},{"class":115,"line":1475},[1895],{"type":49,"tag":113,"props":1896,"children":1897},{"emptyLinePlaceholder":42},[1898],{"type":55,"value":149},{"type":49,"tag":113,"props":1900,"children":1901},{"class":115,"line":1484},[1902],{"type":49,"tag":113,"props":1903,"children":1904},{},[1905],{"type":55,"value":1906},"    seq_idx = pid_s % seq_len  # wrap for batch > 1\n",{"type":49,"tag":113,"props":1908,"children":1909},{"class":115,"line":1493},[1910],{"type":49,"tag":113,"props":1911,"children":1912},{},[1913],{"type":55,"value":1914},"    cos_val = tl.load(cos_ptr + seq_idx * head_dim + offs, mask=mask, other=1.0).to(tl.float32)\n",{"type":49,"tag":113,"props":1916,"children":1917},{"class":115,"line":1502},[1918],{"type":49,"tag":113,"props":1919,"children":1920},{},[1921],{"type":55,"value":1922},"    sin_val = tl.load(sin_ptr + seq_idx * head_dim + offs, mask=mask, other=0.0).to(tl.float32)\n",{"type":49,"tag":113,"props":1924,"children":1925},{"class":115,"line":1511},[1926],{"type":49,"tag":113,"props":1927,"children":1928},{"emptyLinePlaceholder":42},[1929],{"type":55,"value":149},{"type":49,"tag":113,"props":1931,"children":1932},{"class":115,"line":1520},[1933],{"type":49,"tag":113,"props":1934,"children":1935},{},[1936],{"type":55,"value":1937},"    out0 = x0 * cos_val - x1 * sin_val\n",{"type":49,"tag":113,"props":1939,"children":1940},{"class":115,"line":1528},[1941],{"type":49,"tag":113,"props":1942,"children":1943},{},[1944],{"type":55,"value":1945},"    out1 = x0 * sin_val + x1 * cos_val\n",{"type":49,"tag":113,"props":1947,"children":1948},{"class":115,"line":1537},[1949],{"type":49,"tag":113,"props":1950,"children":1951},{"emptyLinePlaceholder":42},[1952],{"type":55,"value":149},{"type":49,"tag":113,"props":1954,"children":1955},{"class":115,"line":1545},[1956],{"type":49,"tag":113,"props":1957,"children":1958},{},[1959],{"type":55,"value":1960},"    tl.store(out_ptr + base + offs, out0.to(x0.dtype), mask=mask)\n",{"type":49,"tag":113,"props":1962,"children":1963},{"class":115,"line":1553},[1964],{"type":49,"tag":113,"props":1965,"children":1966},{},[1967],{"type":55,"value":1968},"    tl.store(out_ptr + base + half_dim + offs, out1.to(x0.dtype), mask=mask)\n",{"type":49,"tag":113,"props":1970,"children":1971},{"class":115,"line":1562},[1972],{"type":49,"tag":113,"props":1973,"children":1974},{"emptyLinePlaceholder":42},[1975],{"type":55,"value":149},{"type":49,"tag":113,"props":1977,"children":1978},{"class":115,"line":1571},[1979],{"type":49,"tag":113,"props":1980,"children":1981},{"emptyLinePlaceholder":42},[1982],{"type":55,"value":149},{"type":49,"tag":113,"props":1984,"children":1985},{"class":115,"line":1580},[1986],{"type":49,"tag":113,"props":1987,"children":1988},{},[1989],{"type":55,"value":1990},"def triton_rope_3d(qk, cos, sin):\n",{"type":49,"tag":113,"props":1992,"children":1993},{"class":115,"line":1589},[1994],{"type":49,"tag":113,"props":1995,"children":1996},{},[1997],{"type":55,"value":1998},"    qk = qk.contiguous()\n",{"type":49,"tag":113,"props":2000,"children":2001},{"class":115,"line":1598},[2002],{"type":49,"tag":113,"props":2003,"children":2004},{},[2005],{"type":55,"value":2006},"    out = torch.empty_like(qk)\n",{"type":49,"tag":113,"props":2008,"children":2009},{"class":115,"line":1607},[2010],{"type":49,"tag":113,"props":2011,"children":2012},{},[2013],{"type":55,"value":2014},"    batch, seq_len, num_heads, head_dim = qk.shape\n",{"type":49,"tag":113,"props":2016,"children":2017},{"class":115,"line":1616},[2018],{"type":49,"tag":113,"props":2019,"children":2020},{},[2021],{"type":55,"value":2022},"    qk_flat = qk.view(batch * seq_len, num_heads, head_dim)\n",{"type":49,"tag":113,"props":2024,"children":2025},{"class":115,"line":1624},[2026],{"type":49,"tag":113,"props":2027,"children":2028},{},[2029],{"type":55,"value":2030},"    out_flat = out.view(batch * seq_len, num_heads, head_dim)\n",{"type":49,"tag":113,"props":2032,"children":2033},{"class":115,"line":1633},[2034],{"type":49,"tag":113,"props":2035,"children":2036},{},[2037],{"type":55,"value":2038},"    BLOCK_HD = triton.next_power_of_2(head_dim \u002F\u002F 2)\n",{"type":49,"tag":113,"props":2040,"children":2041},{"class":115,"line":1642},[2042],{"type":49,"tag":113,"props":2043,"children":2044},{},[2045],{"type":55,"value":2046},"    num_warps = 4 if BLOCK_HD \u003C= 64 else 8\n",{"type":49,"tag":113,"props":2048,"children":2049},{"class":115,"line":1651},[2050],{"type":49,"tag":113,"props":2051,"children":2052},{},[2053],{"type":55,"value":2054},"    rope_3d_kernel[(batch * seq_len, num_heads)](\n",{"type":49,"tag":113,"props":2056,"children":2057},{"class":115,"line":1660},[2058],{"type":49,"tag":113,"props":2059,"children":2060},{},[2061],{"type":55,"value":2062},"        qk_flat, cos, sin, out_flat,\n",{"type":49,"tag":113,"props":2064,"children":2065},{"class":115,"line":1669},[2066],{"type":49,"tag":113,"props":2067,"children":2068},{},[2069],{"type":55,"value":2070},"        seq_len, num_heads, head_dim,\n",{"type":49,"tag":113,"props":2072,"children":2073},{"class":115,"line":1678},[2074],{"type":49,"tag":113,"props":2075,"children":2076},{},[2077],{"type":55,"value":2078},"        qk_flat.stride(0), qk_flat.stride(1), qk_flat.stride(2),\n",{"type":49,"tag":113,"props":2080,"children":2082},{"class":115,"line":2081},41,[2083],{"type":49,"tag":113,"props":2084,"children":2085},{},[2086],{"type":55,"value":2087},"        BLOCK_HD=BLOCK_HD, num_warps=num_warps, num_stages=2,\n",{"type":49,"tag":113,"props":2089,"children":2091},{"class":115,"line":2090},42,[2092],{"type":49,"tag":113,"props":2093,"children":2094},{},[2095],{"type":55,"value":1675},{"type":49,"tag":113,"props":2097,"children":2099},{"class":115,"line":2098},43,[2100],{"type":49,"tag":113,"props":2101,"children":2102},{},[2103],{"type":55,"value":2104},"    return out\n",{"type":49,"tag":86,"props":2106,"children":2108},{"id":2107},"_3-geglu-for-sd3flux-not-ltx-video",[2109],{"type":55,"value":2110},"3. GEGLU (For SD3\u002FFLUX, NOT LTX-Video)",{"type":49,"tag":58,"props":2112,"children":2113},{},[2114,2116,2121,2123,2128],{"type":55,"value":2115},"Element-wise gated activation. Input ",{"type":49,"tag":109,"props":2117,"children":2119},{"className":2118},[],[2120],{"type":55,"value":874},{"type":55,"value":2122}," → Output ",{"type":49,"tag":109,"props":2124,"children":2126},{"className":2125},[],[2127],{"type":55,"value":882},{"type":55,"value":1741},{"type":49,"tag":58,"props":2130,"children":2131},{},[2132],{"type":49,"tag":64,"props":2133,"children":2134},{},[2135],{"type":55,"value":2136},"Same BLOCK_SIZE rule: compute dynamically, do NOT autotune.",{"type":49,"tag":101,"props":2138,"children":2140},{"className":103,"code":2139,"language":105,"meta":106,"style":106},"@triton.jit\ndef geglu_kernel(\n    input_ptr, output_ptr,\n    stride_in, stride_out, hidden_size,\n    BLOCK_H: tl.constexpr,\n):\n    row = tl.program_id(0)\n    offs = tl.arange(0, BLOCK_H)\n    mask = offs \u003C hidden_size\n\n    gate = tl.load(input_ptr + row * stride_in + offs, mask=mask, other=0.0).to(tl.float32)\n    value = tl.load(input_ptr + row * stride_in + hidden_size + offs, mask=mask, other=0.0).to(tl.float32)\n\n    # GELU approx — manual tanh (tl.math.tanh NOT available on ROCm)\n    k = 0.7978845608028654  # sqrt(2\u002Fpi)\n    tanh_arg = k * (gate + 0.044715 * gate * gate * gate)\n    e2x = tl.exp(2.0 * tanh_arg)\n    tanh_val = (e2x - 1.0) \u002F (e2x + 1.0)\n    gate_gelu = 0.5 * gate * (1.0 + tanh_val)\n    result = gate_gelu * value\n\n    tl.store(output_ptr + row * stride_out + offs, result.to(gate.dtype), mask=mask)\n\n\ndef triton_geglu(x):\n    x = x.contiguous()\n    *batch_dims, double_h = x.shape\n    hidden_size = double_h \u002F\u002F 2\n    x_2d = x.view(-1, double_h)\n    M = x_2d.shape[0]\n    out = torch.empty(M, hidden_size, device=x.device, dtype=x.dtype)\n    BLOCK_H = triton.next_power_of_2(hidden_size)\n    num_warps = 4 if BLOCK_H \u003C= 1024 else (8 if BLOCK_H \u003C= 4096 else 16)\n    geglu_kernel[(M,)](\n        x_2d, out, x_2d.stride(0), out.stride(0), hidden_size,\n        BLOCK_H=BLOCK_H, num_warps=num_warps, num_stages=2,\n    )\n    return out.view(*batch_dims, hidden_size)\n",[2141],{"type":49,"tag":109,"props":2142,"children":2143},{"__ignoreMap":106},[2144,2151,2159,2167,2175,2183,2190,2197,2205,2213,2220,2228,2236,2243,2251,2259,2267,2275,2283,2291,2299,2306,2314,2321,2328,2336,2344,2352,2360,2368,2376,2384,2392,2400,2408,2416,2424,2431],{"type":49,"tag":113,"props":2145,"children":2146},{"class":115,"line":116},[2147],{"type":49,"tag":113,"props":2148,"children":2149},{},[2150],{"type":55,"value":1350},{"type":49,"tag":113,"props":2152,"children":2153},{"class":115,"line":125},[2154],{"type":49,"tag":113,"props":2155,"children":2156},{},[2157],{"type":55,"value":2158},"def geglu_kernel(\n",{"type":49,"tag":113,"props":2160,"children":2161},{"class":115,"line":134},[2162],{"type":49,"tag":113,"props":2163,"children":2164},{},[2165],{"type":55,"value":2166},"    input_ptr, output_ptr,\n",{"type":49,"tag":113,"props":2168,"children":2169},{"class":115,"line":143},[2170],{"type":49,"tag":113,"props":2171,"children":2172},{},[2173],{"type":55,"value":2174},"    stride_in, stride_out, hidden_size,\n",{"type":49,"tag":113,"props":2176,"children":2177},{"class":115,"line":152},[2178],{"type":49,"tag":113,"props":2179,"children":2180},{},[2181],{"type":55,"value":2182},"    BLOCK_H: tl.constexpr,\n",{"type":49,"tag":113,"props":2184,"children":2185},{"class":115,"line":161},[2186],{"type":49,"tag":113,"props":2187,"children":2188},{},[2189],{"type":55,"value":1406},{"type":49,"tag":113,"props":2191,"children":2192},{"class":115,"line":170},[2193],{"type":49,"tag":113,"props":2194,"children":2195},{},[2196],{"type":55,"value":1414},{"type":49,"tag":113,"props":2198,"children":2199},{"class":115,"line":179},[2200],{"type":49,"tag":113,"props":2201,"children":2202},{},[2203],{"type":55,"value":2204},"    offs = tl.arange(0, BLOCK_H)\n",{"type":49,"tag":113,"props":2206,"children":2207},{"class":115,"line":188},[2208],{"type":49,"tag":113,"props":2209,"children":2210},{},[2211],{"type":55,"value":2212},"    mask = offs \u003C hidden_size\n",{"type":49,"tag":113,"props":2214,"children":2215},{"class":115,"line":1227},[2216],{"type":49,"tag":113,"props":2217,"children":2218},{"emptyLinePlaceholder":42},[2219],{"type":55,"value":149},{"type":49,"tag":113,"props":2221,"children":2222},{"class":115,"line":1236},[2223],{"type":49,"tag":113,"props":2224,"children":2225},{},[2226],{"type":55,"value":2227},"    gate = tl.load(input_ptr + row * stride_in + offs, mask=mask, other=0.0).to(tl.float32)\n",{"type":49,"tag":113,"props":2229,"children":2230},{"class":115,"line":1245},[2231],{"type":49,"tag":113,"props":2232,"children":2233},{},[2234],{"type":55,"value":2235},"    value = tl.load(input_ptr + row * stride_in + hidden_size + offs, mask=mask, other=0.0).to(tl.float32)\n",{"type":49,"tag":113,"props":2237,"children":2238},{"class":115,"line":1441},[2239],{"type":49,"tag":113,"props":2240,"children":2241},{"emptyLinePlaceholder":42},[2242],{"type":55,"value":149},{"type":49,"tag":113,"props":2244,"children":2245},{"class":115,"line":1449},[2246],{"type":49,"tag":113,"props":2247,"children":2248},{},[2249],{"type":55,"value":2250},"    # GELU approx — manual tanh (tl.math.tanh NOT available on ROCm)\n",{"type":49,"tag":113,"props":2252,"children":2253},{"class":115,"line":1458},[2254],{"type":49,"tag":113,"props":2255,"children":2256},{},[2257],{"type":55,"value":2258},"    k = 0.7978845608028654  # sqrt(2\u002Fpi)\n",{"type":49,"tag":113,"props":2260,"children":2261},{"class":115,"line":1467},[2262],{"type":49,"tag":113,"props":2263,"children":2264},{},[2265],{"type":55,"value":2266},"    tanh_arg = k * (gate + 0.044715 * gate * gate * gate)\n",{"type":49,"tag":113,"props":2268,"children":2269},{"class":115,"line":1475},[2270],{"type":49,"tag":113,"props":2271,"children":2272},{},[2273],{"type":55,"value":2274},"    e2x = tl.exp(2.0 * tanh_arg)\n",{"type":49,"tag":113,"props":2276,"children":2277},{"class":115,"line":1484},[2278],{"type":49,"tag":113,"props":2279,"children":2280},{},[2281],{"type":55,"value":2282},"    tanh_val = (e2x - 1.0) \u002F (e2x + 1.0)\n",{"type":49,"tag":113,"props":2284,"children":2285},{"class":115,"line":1493},[2286],{"type":49,"tag":113,"props":2287,"children":2288},{},[2289],{"type":55,"value":2290},"    gate_gelu = 0.5 * gate * (1.0 + tanh_val)\n",{"type":49,"tag":113,"props":2292,"children":2293},{"class":115,"line":1502},[2294],{"type":49,"tag":113,"props":2295,"children":2296},{},[2297],{"type":55,"value":2298},"    result = gate_gelu * value\n",{"type":49,"tag":113,"props":2300,"children":2301},{"class":115,"line":1511},[2302],{"type":49,"tag":113,"props":2303,"children":2304},{"emptyLinePlaceholder":42},[2305],{"type":55,"value":149},{"type":49,"tag":113,"props":2307,"children":2308},{"class":115,"line":1520},[2309],{"type":49,"tag":113,"props":2310,"children":2311},{},[2312],{"type":55,"value":2313},"    tl.store(output_ptr + row * stride_out + offs, result.to(gate.dtype), mask=mask)\n",{"type":49,"tag":113,"props":2315,"children":2316},{"class":115,"line":1528},[2317],{"type":49,"tag":113,"props":2318,"children":2319},{"emptyLinePlaceholder":42},[2320],{"type":55,"value":149},{"type":49,"tag":113,"props":2322,"children":2323},{"class":115,"line":1537},[2324],{"type":49,"tag":113,"props":2325,"children":2326},{"emptyLinePlaceholder":42},[2327],{"type":55,"value":149},{"type":49,"tag":113,"props":2329,"children":2330},{"class":115,"line":1545},[2331],{"type":49,"tag":113,"props":2332,"children":2333},{},[2334],{"type":55,"value":2335},"def triton_geglu(x):\n",{"type":49,"tag":113,"props":2337,"children":2338},{"class":115,"line":1553},[2339],{"type":49,"tag":113,"props":2340,"children":2341},{},[2342],{"type":55,"value":2343},"    x = x.contiguous()\n",{"type":49,"tag":113,"props":2345,"children":2346},{"class":115,"line":1562},[2347],{"type":49,"tag":113,"props":2348,"children":2349},{},[2350],{"type":55,"value":2351},"    *batch_dims, double_h = x.shape\n",{"type":49,"tag":113,"props":2353,"children":2354},{"class":115,"line":1571},[2355],{"type":49,"tag":113,"props":2356,"children":2357},{},[2358],{"type":55,"value":2359},"    hidden_size = double_h \u002F\u002F 2\n",{"type":49,"tag":113,"props":2361,"children":2362},{"class":115,"line":1580},[2363],{"type":49,"tag":113,"props":2364,"children":2365},{},[2366],{"type":55,"value":2367},"    x_2d = x.view(-1, double_h)\n",{"type":49,"tag":113,"props":2369,"children":2370},{"class":115,"line":1589},[2371],{"type":49,"tag":113,"props":2372,"children":2373},{},[2374],{"type":55,"value":2375},"    M = x_2d.shape[0]\n",{"type":49,"tag":113,"props":2377,"children":2378},{"class":115,"line":1598},[2379],{"type":49,"tag":113,"props":2380,"children":2381},{},[2382],{"type":55,"value":2383},"    out = torch.empty(M, hidden_size, device=x.device, dtype=x.dtype)\n",{"type":49,"tag":113,"props":2385,"children":2386},{"class":115,"line":1607},[2387],{"type":49,"tag":113,"props":2388,"children":2389},{},[2390],{"type":55,"value":2391},"    BLOCK_H = triton.next_power_of_2(hidden_size)\n",{"type":49,"tag":113,"props":2393,"children":2394},{"class":115,"line":1616},[2395],{"type":49,"tag":113,"props":2396,"children":2397},{},[2398],{"type":55,"value":2399},"    num_warps = 4 if BLOCK_H \u003C= 1024 else (8 if BLOCK_H \u003C= 4096 else 16)\n",{"type":49,"tag":113,"props":2401,"children":2402},{"class":115,"line":1624},[2403],{"type":49,"tag":113,"props":2404,"children":2405},{},[2406],{"type":55,"value":2407},"    geglu_kernel[(M,)](\n",{"type":49,"tag":113,"props":2409,"children":2410},{"class":115,"line":1633},[2411],{"type":49,"tag":113,"props":2412,"children":2413},{},[2414],{"type":55,"value":2415},"        x_2d, out, x_2d.stride(0), out.stride(0), hidden_size,\n",{"type":49,"tag":113,"props":2417,"children":2418},{"class":115,"line":1642},[2419],{"type":49,"tag":113,"props":2420,"children":2421},{},[2422],{"type":55,"value":2423},"        BLOCK_H=BLOCK_H, num_warps=num_warps, num_stages=2,\n",{"type":49,"tag":113,"props":2425,"children":2426},{"class":115,"line":1651},[2427],{"type":49,"tag":113,"props":2428,"children":2429},{},[2430],{"type":55,"value":1675},{"type":49,"tag":113,"props":2432,"children":2433},{"class":115,"line":1660},[2434],{"type":49,"tag":113,"props":2435,"children":2436},{},[2437],{"type":55,"value":2438},"    return out.view(*batch_dims, hidden_size)\n",{"type":49,"tag":58,"props":2440,"children":2441},{},[2442,2447],{"type":49,"tag":64,"props":2443,"children":2444},{},[2445],{"type":55,"value":2446},"Warning: LTX-Video uses GELU, NOT GEGLU.",{"type":55,"value":2448}," GEGLU is for SD3\u002FFLUX.",{"type":49,"tag":86,"props":2450,"children":2452},{"id":2451},"_4-adaln-adaptive-layer-normalization-for-dit",[2453],{"type":55,"value":2454},"4. AdaLN (Adaptive Layer Normalization for DiT)",{"type":49,"tag":58,"props":2456,"children":2457},{},[2458,2460],{"type":55,"value":2459},"Fused normalization + conditioning: ",{"type":49,"tag":109,"props":2461,"children":2463},{"className":2462},[],[2464],{"type":55,"value":2465},"norm(x) * weight * (1 + scale) + shift",{"type":49,"tag":58,"props":2467,"children":2468},{},[2469],{"type":49,"tag":64,"props":2470,"children":2471},{},[2472],{"type":55,"value":2473},"Same BLOCK_D rule: compute dynamically.",{"type":49,"tag":101,"props":2475,"children":2477},{"className":103,"code":2476,"language":105,"meta":106,"style":106},"@triton.jit\ndef adaln_kernel(\n    x_ptr, weight_ptr, scale_ptr, shift_ptr, out_ptr,\n    stride_x, stride_cond, D,\n    eps: tl.constexpr,\n    BLOCK_D: tl.constexpr,\n):\n    row = tl.program_id(0)\n    offs = tl.arange(0, BLOCK_D)\n    mask = offs \u003C D\n    x = tl.load(x_ptr + row * stride_x + offs, mask=mask, other=0.0).to(tl.float32)\n\n    variance = tl.sum(x * x, axis=0) \u002F D\n    rms_inv = tl.rsqrt(variance + eps)\n    x_norm = x * rms_inv\n\n    w = tl.load(weight_ptr + offs, mask=mask, other=1.0).to(tl.float32)\n    scale = tl.load(scale_ptr + row * stride_cond + offs, mask=mask, other=0.0).to(tl.float32)\n    shift = tl.load(shift_ptr + row * stride_cond + offs, mask=mask, other=0.0).to(tl.float32)\n\n    out = x_norm * w * (1.0 + scale) + shift\n    tl.store(out_ptr + row * stride_x + offs, out.to(x.dtype), mask=mask)\n\n\ndef triton_adaln(x, weight, scale, shift, eps=1e-6):\n    x_flat = x.contiguous().view(-1, x.shape[-1])\n    scale_flat = scale.contiguous().view(-1, x.shape[-1])\n    shift_flat = shift.contiguous().view(-1, x.shape[-1])\n    out = torch.empty_like(x_flat)\n    M, D = x_flat.shape\n    BLOCK_D = triton.next_power_of_2(D)\n    num_warps = 4 if BLOCK_D \u003C= 1024 else (8 if BLOCK_D \u003C= 4096 else 16)\n    adaln_kernel[(M,)](\n        x_flat, weight, scale_flat, shift_flat, out,\n        x_flat.stride(0), scale_flat.stride(0), D, eps,\n        BLOCK_D=BLOCK_D, num_warps=num_warps, num_stages=2,\n    )\n    return out.view_as(x)\n",[2478],{"type":49,"tag":109,"props":2479,"children":2480},{"__ignoreMap":106},[2481,2488,2496,2504,2512,2519,2526,2533,2540,2547,2554,2561,2568,2575,2582,2590,2597,2605,2613,2621,2628,2636,2643,2650,2657,2665,2673,2681,2689,2697,2705,2712,2719,2727,2735,2743,2750,2757],{"type":49,"tag":113,"props":2482,"children":2483},{"class":115,"line":116},[2484],{"type":49,"tag":113,"props":2485,"children":2486},{},[2487],{"type":55,"value":1350},{"type":49,"tag":113,"props":2489,"children":2490},{"class":115,"line":125},[2491],{"type":49,"tag":113,"props":2492,"children":2493},{},[2494],{"type":55,"value":2495},"def adaln_kernel(\n",{"type":49,"tag":113,"props":2497,"children":2498},{"class":115,"line":134},[2499],{"type":49,"tag":113,"props":2500,"children":2501},{},[2502],{"type":55,"value":2503},"    x_ptr, weight_ptr, scale_ptr, shift_ptr, out_ptr,\n",{"type":49,"tag":113,"props":2505,"children":2506},{"class":115,"line":143},[2507],{"type":49,"tag":113,"props":2508,"children":2509},{},[2510],{"type":55,"value":2511},"    stride_x, stride_cond, D,\n",{"type":49,"tag":113,"props":2513,"children":2514},{"class":115,"line":152},[2515],{"type":49,"tag":113,"props":2516,"children":2517},{},[2518],{"type":55,"value":1382},{"type":49,"tag":113,"props":2520,"children":2521},{"class":115,"line":161},[2522],{"type":49,"tag":113,"props":2523,"children":2524},{},[2525],{"type":55,"value":1398},{"type":49,"tag":113,"props":2527,"children":2528},{"class":115,"line":170},[2529],{"type":49,"tag":113,"props":2530,"children":2531},{},[2532],{"type":55,"value":1406},{"type":49,"tag":113,"props":2534,"children":2535},{"class":115,"line":179},[2536],{"type":49,"tag":113,"props":2537,"children":2538},{},[2539],{"type":55,"value":1414},{"type":49,"tag":113,"props":2541,"children":2542},{"class":115,"line":188},[2543],{"type":49,"tag":113,"props":2544,"children":2545},{},[2546],{"type":55,"value":1422},{"type":49,"tag":113,"props":2548,"children":2549},{"class":115,"line":1227},[2550],{"type":49,"tag":113,"props":2551,"children":2552},{},[2553],{"type":55,"value":1430},{"type":49,"tag":113,"props":2555,"children":2556},{"class":115,"line":1236},[2557],{"type":49,"tag":113,"props":2558,"children":2559},{},[2560],{"type":55,"value":1438},{"type":49,"tag":113,"props":2562,"children":2563},{"class":115,"line":1245},[2564],{"type":49,"tag":113,"props":2565,"children":2566},{"emptyLinePlaceholder":42},[2567],{"type":55,"value":149},{"type":49,"tag":113,"props":2569,"children":2570},{"class":115,"line":1441},[2571],{"type":49,"tag":113,"props":2572,"children":2573},{},[2574],{"type":55,"value":1455},{"type":49,"tag":113,"props":2576,"children":2577},{"class":115,"line":1449},[2578],{"type":49,"tag":113,"props":2579,"children":2580},{},[2581],{"type":55,"value":1464},{"type":49,"tag":113,"props":2583,"children":2584},{"class":115,"line":1458},[2585],{"type":49,"tag":113,"props":2586,"children":2587},{},[2588],{"type":55,"value":2589},"    x_norm = x * rms_inv\n",{"type":49,"tag":113,"props":2591,"children":2592},{"class":115,"line":1467},[2593],{"type":49,"tag":113,"props":2594,"children":2595},{"emptyLinePlaceholder":42},[2596],{"type":55,"value":149},{"type":49,"tag":113,"props":2598,"children":2599},{"class":115,"line":1475},[2600],{"type":49,"tag":113,"props":2601,"children":2602},{},[2603],{"type":55,"value":2604},"    w = tl.load(weight_ptr + offs, mask=mask, other=1.0).to(tl.float32)\n",{"type":49,"tag":113,"props":2606,"children":2607},{"class":115,"line":1484},[2608],{"type":49,"tag":113,"props":2609,"children":2610},{},[2611],{"type":55,"value":2612},"    scale = tl.load(scale_ptr + row * stride_cond + offs, mask=mask, other=0.0).to(tl.float32)\n",{"type":49,"tag":113,"props":2614,"children":2615},{"class":115,"line":1493},[2616],{"type":49,"tag":113,"props":2617,"children":2618},{},[2619],{"type":55,"value":2620},"    shift = tl.load(shift_ptr + row * stride_cond + offs, mask=mask, other=0.0).to(tl.float32)\n",{"type":49,"tag":113,"props":2622,"children":2623},{"class":115,"line":1502},[2624],{"type":49,"tag":113,"props":2625,"children":2626},{"emptyLinePlaceholder":42},[2627],{"type":55,"value":149},{"type":49,"tag":113,"props":2629,"children":2630},{"class":115,"line":1511},[2631],{"type":49,"tag":113,"props":2632,"children":2633},{},[2634],{"type":55,"value":2635},"    out = x_norm * w * (1.0 + scale) + shift\n",{"type":49,"tag":113,"props":2637,"children":2638},{"class":115,"line":1520},[2639],{"type":49,"tag":113,"props":2640,"children":2641},{},[2642],{"type":55,"value":1534},{"type":49,"tag":113,"props":2644,"children":2645},{"class":115,"line":1528},[2646],{"type":49,"tag":113,"props":2647,"children":2648},{"emptyLinePlaceholder":42},[2649],{"type":55,"value":149},{"type":49,"tag":113,"props":2651,"children":2652},{"class":115,"line":1537},[2653],{"type":49,"tag":113,"props":2654,"children":2655},{"emptyLinePlaceholder":42},[2656],{"type":55,"value":149},{"type":49,"tag":113,"props":2658,"children":2659},{"class":115,"line":1545},[2660],{"type":49,"tag":113,"props":2661,"children":2662},{},[2663],{"type":55,"value":2664},"def triton_adaln(x, weight, scale, shift, eps=1e-6):\n",{"type":49,"tag":113,"props":2666,"children":2667},{"class":115,"line":1553},[2668],{"type":49,"tag":113,"props":2669,"children":2670},{},[2671],{"type":55,"value":2672},"    x_flat = x.contiguous().view(-1, x.shape[-1])\n",{"type":49,"tag":113,"props":2674,"children":2675},{"class":115,"line":1562},[2676],{"type":49,"tag":113,"props":2677,"children":2678},{},[2679],{"type":55,"value":2680},"    scale_flat = scale.contiguous().view(-1, x.shape[-1])\n",{"type":49,"tag":113,"props":2682,"children":2683},{"class":115,"line":1571},[2684],{"type":49,"tag":113,"props":2685,"children":2686},{},[2687],{"type":55,"value":2688},"    shift_flat = shift.contiguous().view(-1, x.shape[-1])\n",{"type":49,"tag":113,"props":2690,"children":2691},{"class":115,"line":1580},[2692],{"type":49,"tag":113,"props":2693,"children":2694},{},[2695],{"type":55,"value":2696},"    out = torch.empty_like(x_flat)\n",{"type":49,"tag":113,"props":2698,"children":2699},{"class":115,"line":1589},[2700],{"type":49,"tag":113,"props":2701,"children":2702},{},[2703],{"type":55,"value":2704},"    M, D = x_flat.shape\n",{"type":49,"tag":113,"props":2706,"children":2707},{"class":115,"line":1598},[2708],{"type":49,"tag":113,"props":2709,"children":2710},{},[2711],{"type":55,"value":1630},{"type":49,"tag":113,"props":2713,"children":2714},{"class":115,"line":1607},[2715],{"type":49,"tag":113,"props":2716,"children":2717},{},[2718],{"type":55,"value":1639},{"type":49,"tag":113,"props":2720,"children":2721},{"class":115,"line":1616},[2722],{"type":49,"tag":113,"props":2723,"children":2724},{},[2725],{"type":55,"value":2726},"    adaln_kernel[(M,)](\n",{"type":49,"tag":113,"props":2728,"children":2729},{"class":115,"line":1624},[2730],{"type":49,"tag":113,"props":2731,"children":2732},{},[2733],{"type":55,"value":2734},"        x_flat, weight, scale_flat, shift_flat, out,\n",{"type":49,"tag":113,"props":2736,"children":2737},{"class":115,"line":1633},[2738],{"type":49,"tag":113,"props":2739,"children":2740},{},[2741],{"type":55,"value":2742},"        x_flat.stride(0), scale_flat.stride(0), D, eps,\n",{"type":49,"tag":113,"props":2744,"children":2745},{"class":115,"line":1642},[2746],{"type":49,"tag":113,"props":2747,"children":2748},{},[2749],{"type":55,"value":1666},{"type":49,"tag":113,"props":2751,"children":2752},{"class":115,"line":1651},[2753],{"type":49,"tag":113,"props":2754,"children":2755},{},[2756],{"type":55,"value":1675},{"type":49,"tag":113,"props":2758,"children":2759},{"class":115,"line":1660},[2760],{"type":49,"tag":113,"props":2761,"children":2762},{},[2763],{"type":55,"value":1684},{"type":49,"tag":79,"props":2765,"children":2767},{"id":2766},"diffusers-integration",[2768],{"type":55,"value":2769},"Diffusers Integration",{"type":49,"tag":1059,"props":2771,"children":2772},{},[2773],{"type":49,"tag":58,"props":2774,"children":2775},{},[2776],{"type":49,"tag":64,"props":2777,"children":2778},{},[2779,2780,2786],{"type":55,"value":1066},{"type":49,"tag":1068,"props":2781,"children":2783},{"href":2782},"references\u002Fdiffusers-integration.md",[2784],{"type":55,"value":2785},"diffusers-integration.md",{"type":55,"value":2787}," for the complete guide.",{"type":49,"tag":86,"props":2789,"children":2791},{"id":2790},"minimal-integration-pattern",[2792],{"type":55,"value":2793},"Minimal Integration Pattern",{"type":49,"tag":101,"props":2795,"children":2797},{"className":103,"code":2796,"language":105,"meta":106,"style":106},"def patch_rmsnorm_modules(model):\n    \"\"\"Patch all RMSNorm modules to use custom Triton kernel.\"\"\"\n    for name, module in model.named_modules():\n        if type(module).__name__ == 'RMSNorm':\n            eps = getattr(module, 'eps', 1e-6)\n            has_weight = hasattr(module, 'weight') and module.weight is not None\n            if has_weight:\n                def make_forward(mod, epsilon):\n                    def forward(x):\n                        return triton_rmsnorm(x, mod.weight, eps=epsilon)\n                    return forward\n                module.forward = make_forward(module, eps)\n            else:\n                def make_forward(epsilon):\n                    def forward(x):\n                        w = torch.ones(x.shape[-1], device=x.device, dtype=x.dtype)\n                        return triton_rmsnorm(x, w, eps=epsilon)\n                    return forward\n                module.forward = make_forward(eps)\n\npipe = LTXPipeline.from_pretrained(\"Lightricks\u002FLTX-Video\", torch_dtype=torch.bfloat16)\npipe.to(\"cuda\")\npatch_rmsnorm_modules(pipe.transformer)\npipe.enable_model_cpu_offload()\n",[2798],{"type":49,"tag":109,"props":2799,"children":2800},{"__ignoreMap":106},[2801,2809,2817,2825,2833,2841,2849,2857,2865,2873,2881,2889,2897,2905,2913,2920,2928,2936,2943,2951,2958,2965,2973,2981],{"type":49,"tag":113,"props":2802,"children":2803},{"class":115,"line":116},[2804],{"type":49,"tag":113,"props":2805,"children":2806},{},[2807],{"type":55,"value":2808},"def patch_rmsnorm_modules(model):\n",{"type":49,"tag":113,"props":2810,"children":2811},{"class":115,"line":125},[2812],{"type":49,"tag":113,"props":2813,"children":2814},{},[2815],{"type":55,"value":2816},"    \"\"\"Patch all RMSNorm modules to use custom Triton kernel.\"\"\"\n",{"type":49,"tag":113,"props":2818,"children":2819},{"class":115,"line":134},[2820],{"type":49,"tag":113,"props":2821,"children":2822},{},[2823],{"type":55,"value":2824},"    for name, module in model.named_modules():\n",{"type":49,"tag":113,"props":2826,"children":2827},{"class":115,"line":143},[2828],{"type":49,"tag":113,"props":2829,"children":2830},{},[2831],{"type":55,"value":2832},"        if type(module).__name__ == 'RMSNorm':\n",{"type":49,"tag":113,"props":2834,"children":2835},{"class":115,"line":152},[2836],{"type":49,"tag":113,"props":2837,"children":2838},{},[2839],{"type":55,"value":2840},"            eps = getattr(module, 'eps', 1e-6)\n",{"type":49,"tag":113,"props":2842,"children":2843},{"class":115,"line":161},[2844],{"type":49,"tag":113,"props":2845,"children":2846},{},[2847],{"type":55,"value":2848},"            has_weight = hasattr(module, 'weight') and module.weight is not None\n",{"type":49,"tag":113,"props":2850,"children":2851},{"class":115,"line":170},[2852],{"type":49,"tag":113,"props":2853,"children":2854},{},[2855],{"type":55,"value":2856},"            if has_weight:\n",{"type":49,"tag":113,"props":2858,"children":2859},{"class":115,"line":179},[2860],{"type":49,"tag":113,"props":2861,"children":2862},{},[2863],{"type":55,"value":2864},"                def make_forward(mod, epsilon):\n",{"type":49,"tag":113,"props":2866,"children":2867},{"class":115,"line":188},[2868],{"type":49,"tag":113,"props":2869,"children":2870},{},[2871],{"type":55,"value":2872},"                    def forward(x):\n",{"type":49,"tag":113,"props":2874,"children":2875},{"class":115,"line":1227},[2876],{"type":49,"tag":113,"props":2877,"children":2878},{},[2879],{"type":55,"value":2880},"                        return triton_rmsnorm(x, mod.weight, eps=epsilon)\n",{"type":49,"tag":113,"props":2882,"children":2883},{"class":115,"line":1236},[2884],{"type":49,"tag":113,"props":2885,"children":2886},{},[2887],{"type":55,"value":2888},"                    return forward\n",{"type":49,"tag":113,"props":2890,"children":2891},{"class":115,"line":1245},[2892],{"type":49,"tag":113,"props":2893,"children":2894},{},[2895],{"type":55,"value":2896},"                module.forward = make_forward(module, eps)\n",{"type":49,"tag":113,"props":2898,"children":2899},{"class":115,"line":1441},[2900],{"type":49,"tag":113,"props":2901,"children":2902},{},[2903],{"type":55,"value":2904},"            else:\n",{"type":49,"tag":113,"props":2906,"children":2907},{"class":115,"line":1449},[2908],{"type":49,"tag":113,"props":2909,"children":2910},{},[2911],{"type":55,"value":2912},"                def make_forward(epsilon):\n",{"type":49,"tag":113,"props":2914,"children":2915},{"class":115,"line":1458},[2916],{"type":49,"tag":113,"props":2917,"children":2918},{},[2919],{"type":55,"value":2872},{"type":49,"tag":113,"props":2921,"children":2922},{"class":115,"line":1467},[2923],{"type":49,"tag":113,"props":2924,"children":2925},{},[2926],{"type":55,"value":2927},"                        w = torch.ones(x.shape[-1], device=x.device, dtype=x.dtype)\n",{"type":49,"tag":113,"props":2929,"children":2930},{"class":115,"line":1475},[2931],{"type":49,"tag":113,"props":2932,"children":2933},{},[2934],{"type":55,"value":2935},"                        return triton_rmsnorm(x, w, eps=epsilon)\n",{"type":49,"tag":113,"props":2937,"children":2938},{"class":115,"line":1484},[2939],{"type":49,"tag":113,"props":2940,"children":2941},{},[2942],{"type":55,"value":2888},{"type":49,"tag":113,"props":2944,"children":2945},{"class":115,"line":1493},[2946],{"type":49,"tag":113,"props":2947,"children":2948},{},[2949],{"type":55,"value":2950},"                module.forward = make_forward(eps)\n",{"type":49,"tag":113,"props":2952,"children":2953},{"class":115,"line":1502},[2954],{"type":49,"tag":113,"props":2955,"children":2956},{"emptyLinePlaceholder":42},[2957],{"type":55,"value":149},{"type":49,"tag":113,"props":2959,"children":2960},{"class":115,"line":1511},[2961],{"type":49,"tag":113,"props":2962,"children":2963},{},[2964],{"type":55,"value":167},{"type":49,"tag":113,"props":2966,"children":2967},{"class":115,"line":1520},[2968],{"type":49,"tag":113,"props":2969,"children":2970},{},[2971],{"type":55,"value":2972},"pipe.to(\"cuda\")\n",{"type":49,"tag":113,"props":2974,"children":2975},{"class":115,"line":1528},[2976],{"type":49,"tag":113,"props":2977,"children":2978},{},[2979],{"type":55,"value":2980},"patch_rmsnorm_modules(pipe.transformer)\n",{"type":49,"tag":113,"props":2982,"children":2983},{"class":115,"line":1537},[2984],{"type":49,"tag":113,"props":2985,"children":2986},{},[2987],{"type":55,"value":194},{"type":49,"tag":86,"props":2989,"children":2991},{"id":2990},"diffusers-critical-pitfalls",[2992],{"type":55,"value":2993},"Diffusers Critical Pitfalls",{"type":49,"tag":2995,"props":2996,"children":2997},"ol",{},[2998,3014,3038,3048],{"type":49,"tag":1098,"props":2999,"children":3000},{},[3001,3006,3008],{"type":49,"tag":64,"props":3002,"children":3003},{},[3004],{"type":55,"value":3005},"RMSNorm weight may be None",{"type":55,"value":3007}," — LTX-Video uses ",{"type":49,"tag":109,"props":3009,"children":3011},{"className":3010},[],[3012],{"type":55,"value":3013},"elementwise_affine=False",{"type":49,"tag":1098,"props":3015,"children":3016},{},[3017,3022,3024,3030,3032],{"type":49,"tag":64,"props":3018,"children":3019},{},[3020],{"type":55,"value":3021},"Diffusers RMSNorm != torch.nn.RMSNorm",{"type":55,"value":3023}," — Use ",{"type":49,"tag":109,"props":3025,"children":3027},{"className":3026},[],[3028],{"type":55,"value":3029},"type(module).__name__",{"type":55,"value":3031}," not ",{"type":49,"tag":109,"props":3033,"children":3035},{"className":3034},[],[3036],{"type":55,"value":3037},"isinstance()",{"type":49,"tag":1098,"props":3039,"children":3040},{},[3041,3046],{"type":49,"tag":64,"props":3042,"children":3043},{},[3044],{"type":55,"value":3045},"LTX-Video uses GELU, not GEGLU",{"type":55,"value":3047}," — Don't patch GEGLU for LTX-Video",{"type":49,"tag":1098,"props":3049,"children":3050},{},[3051,3056,3058,3064,3066],{"type":49,"tag":64,"props":3052,"children":3053},{},[3054],{"type":55,"value":3055},"Inject BEFORE CPU offloading",{"type":55,"value":3057}," — ",{"type":49,"tag":109,"props":3059,"children":3061},{"className":3060},[],[3062],{"type":55,"value":3063},"inject_kernels()",{"type":55,"value":3065}," then ",{"type":49,"tag":109,"props":3067,"children":3069},{"className":3068},[],[3070],{"type":55,"value":3071},"enable_model_cpu_offload()",{"type":49,"tag":79,"props":3073,"children":3075},{"id":3074},"performance-expectations",[3076],{"type":55,"value":3077},"Performance Expectations",{"type":49,"tag":86,"props":3079,"children":3081},{"id":3080},"micro-benchmark-results-mi355x-bf16",[3082],{"type":55,"value":3083},"Micro-benchmark Results (MI355X, BF16)",{"type":49,"tag":514,"props":3085,"children":3086},{},[3087,3112],{"type":49,"tag":518,"props":3088,"children":3089},{},[3090],{"type":49,"tag":522,"props":3091,"children":3092},{},[3093,3097,3102,3107],{"type":49,"tag":526,"props":3094,"children":3095},{},[3096],{"type":55,"value":550},{"type":49,"tag":526,"props":3098,"children":3099},{"align":965},[3100],{"type":55,"value":3101},"Avg Speedup",{"type":49,"tag":526,"props":3103,"children":3104},{"align":965},[3105],{"type":55,"value":3106},"Best Config Speedup",{"type":49,"tag":526,"props":3108,"children":3109},{"align":965},[3110],{"type":55,"value":3111},"Status",{"type":49,"tag":552,"props":3113,"children":3114},{},[3115,3150,3183,3216],{"type":49,"tag":522,"props":3116,"children":3117},{},[3118,3125,3133,3145],{"type":49,"tag":559,"props":3119,"children":3120},{},[3121],{"type":49,"tag":64,"props":3122,"children":3123},{},[3124],{"type":55,"value":572},{"type":49,"tag":559,"props":3126,"children":3127},{"align":965},[3128],{"type":49,"tag":64,"props":3129,"children":3130},{},[3131],{"type":55,"value":3132},"1.71x",{"type":49,"tag":559,"props":3134,"children":3135},{"align":965},[3136,3138,3143],{"type":55,"value":3137},"2.44x (",{"type":49,"tag":113,"props":3139,"children":3140},{},[3141],{"type":55,"value":3142},"4×4096×3072",{"type":55,"value":3144},")",{"type":49,"tag":559,"props":3146,"children":3147},{"align":965},[3148],{"type":55,"value":3149},"PASS",{"type":49,"tag":522,"props":3151,"children":3152},{},[3153,3160,3168,3179],{"type":49,"tag":559,"props":3154,"children":3155},{},[3156],{"type":49,"tag":64,"props":3157,"children":3158},{},[3159],{"type":55,"value":744},{"type":49,"tag":559,"props":3161,"children":3162},{"align":965},[3163],{"type":49,"tag":64,"props":3164,"children":3165},{},[3166],{"type":55,"value":3167},"1.21x",{"type":49,"tag":559,"props":3169,"children":3170},{"align":965},[3171,3173,3178],{"type":55,"value":3172},"1.52x (",{"type":49,"tag":113,"props":3174,"children":3175},{},[3176],{"type":55,"value":3177},"2×4096×16×128",{"type":55,"value":3144},{"type":49,"tag":559,"props":3180,"children":3181},{"align":965},[3182],{"type":55,"value":3149},{"type":49,"tag":522,"props":3184,"children":3185},{},[3186,3193,3201,3212],{"type":49,"tag":559,"props":3187,"children":3188},{},[3189],{"type":49,"tag":64,"props":3190,"children":3191},{},[3192],{"type":55,"value":860},{"type":49,"tag":559,"props":3194,"children":3195},{"align":965},[3196],{"type":49,"tag":64,"props":3197,"children":3198},{},[3199],{"type":55,"value":3200},"1.43x",{"type":49,"tag":559,"props":3202,"children":3203},{"align":965},[3204,3206,3211],{"type":55,"value":3205},"2.13x (",{"type":49,"tag":113,"props":3207,"children":3208},{},[3209],{"type":55,"value":3210},"4×4096×8192",{"type":55,"value":3144},{"type":49,"tag":559,"props":3213,"children":3214},{"align":965},[3215],{"type":55,"value":3149},{"type":49,"tag":522,"props":3217,"children":3218},{},[3219,3226,3234,3244],{"type":49,"tag":559,"props":3220,"children":3221},{},[3222],{"type":49,"tag":64,"props":3223,"children":3224},{},[3225],{"type":55,"value":898},{"type":49,"tag":559,"props":3227,"children":3228},{"align":965},[3229],{"type":49,"tag":64,"props":3230,"children":3231},{},[3232],{"type":55,"value":3233},"2.22x",{"type":49,"tag":559,"props":3235,"children":3236},{"align":965},[3237,3239,3243],{"type":55,"value":3238},"2.77x (",{"type":49,"tag":113,"props":3240,"children":3241},{},[3242],{"type":55,"value":3142},{"type":55,"value":3144},{"type":49,"tag":559,"props":3245,"children":3246},{"align":965},[3247],{"type":55,"value":3149},{"type":49,"tag":58,"props":3249,"children":3250},{},[3251],{"type":55,"value":3252},"RMSNorm bandwidth utilization: 3554 GB\u002Fs (MI355X theoretical: 8 TB\u002Fs, ~44%).",{"type":49,"tag":86,"props":3254,"children":3256},{"id":3255},"end-to-end-ltx-video-mi355x-25-frames-30-steps",[3257],{"type":55,"value":3258},"End-to-End LTX-Video (MI355X, 25 frames, 30 steps)",{"type":49,"tag":514,"props":3260,"children":3261},{},[3262,3293],{"type":49,"tag":518,"props":3263,"children":3264},{},[3265],{"type":49,"tag":522,"props":3266,"children":3267},{},[3268,3273,3278,3283,3288],{"type":49,"tag":526,"props":3269,"children":3270},{},[3271],{"type":55,"value":3272},"Mode",{"type":49,"tag":526,"props":3274,"children":3275},{"align":965},[3276],{"type":55,"value":3277},"Time (s)",{"type":49,"tag":526,"props":3279,"children":3280},{"align":965},[3281],{"type":55,"value":3282},"Per Step (s)",{"type":49,"tag":526,"props":3284,"children":3285},{"align":965},[3286],{"type":55,"value":3287},"Peak Mem (GB)",{"type":49,"tag":526,"props":3289,"children":3290},{"align":965},[3291],{"type":55,"value":3292},"Speedup",{"type":49,"tag":552,"props":3294,"children":3295},{},[3296,3324,3365],{"type":49,"tag":522,"props":3297,"children":3298},{},[3299,3304,3309,3314,3319],{"type":49,"tag":559,"props":3300,"children":3301},{},[3302],{"type":55,"value":3303},"baseline",{"type":49,"tag":559,"props":3305,"children":3306},{"align":965},[3307],{"type":55,"value":3308},"1.20",{"type":49,"tag":559,"props":3310,"children":3311},{"align":965},[3312],{"type":55,"value":3313},"0.040",{"type":49,"tag":559,"props":3315,"children":3316},{"align":965},[3317],{"type":55,"value":3318},"18.58",{"type":49,"tag":559,"props":3320,"children":3321},{"align":965},[3322],{"type":55,"value":3323},"1.00x",{"type":49,"tag":522,"props":3325,"children":3326},{},[3327,3334,3342,3350,3357],{"type":49,"tag":559,"props":3328,"children":3329},{},[3330],{"type":49,"tag":64,"props":3331,"children":3332},{},[3333],{"type":55,"value":18},{"type":49,"tag":559,"props":3335,"children":3336},{"align":965},[3337],{"type":49,"tag":64,"props":3338,"children":3339},{},[3340],{"type":55,"value":3341},"0.98",{"type":49,"tag":559,"props":3343,"children":3344},{"align":965},[3345],{"type":49,"tag":64,"props":3346,"children":3347},{},[3348],{"type":55,"value":3349},"0.033",{"type":49,"tag":559,"props":3351,"children":3352},{"align":965},[3353],{"type":49,"tag":64,"props":3354,"children":3355},{},[3356],{"type":55,"value":3318},{"type":49,"tag":559,"props":3358,"children":3359},{"align":965},[3360],{"type":49,"tag":64,"props":3361,"children":3362},{},[3363],{"type":55,"value":3364},"1.22x",{"type":49,"tag":522,"props":3366,"children":3367},{},[3368,3373,3378,3383,3387],{"type":49,"tag":559,"props":3369,"children":3370},{},[3371],{"type":55,"value":3372},"torch.compile",{"type":49,"tag":559,"props":3374,"children":3375},{"align":965},[3376],{"type":55,"value":3377},"0.78",{"type":49,"tag":559,"props":3379,"children":3380},{"align":965},[3381],{"type":55,"value":3382},"0.026",{"type":49,"tag":559,"props":3384,"children":3385},{"align":965},[3386],{"type":55,"value":3318},{"type":49,"tag":559,"props":3388,"children":3389},{"align":965},[3390],{"type":55,"value":3391},"1.54x",{"type":49,"tag":58,"props":3393,"children":3394},{},[3395,3400],{"type":49,"tag":64,"props":3396,"children":3397},{},[3398],{"type":55,"value":3399},"Key finding",{"type":55,"value":3401},": MI355X Triton E2E speedup (22%) is significantly higher than H100 CUDA reference (6%), because MI355X's default PyTorch RMSNorm path has more room for optimization.",{"type":49,"tag":86,"props":3403,"children":3405},{"id":3404},"micro-benchmark-results-r9700-bf16",[3406],{"type":55,"value":3407},"Micro-benchmark Results (R9700, BF16)",{"type":49,"tag":514,"props":3409,"children":3410},{},[3411,3433],{"type":49,"tag":518,"props":3412,"children":3413},{},[3414],{"type":49,"tag":522,"props":3415,"children":3416},{},[3417,3421,3425,3429],{"type":49,"tag":526,"props":3418,"children":3419},{},[3420],{"type":55,"value":550},{"type":49,"tag":526,"props":3422,"children":3423},{"align":965},[3424],{"type":55,"value":3101},{"type":49,"tag":526,"props":3426,"children":3427},{"align":965},[3428],{"type":55,"value":3106},{"type":49,"tag":526,"props":3430,"children":3431},{"align":965},[3432],{"type":55,"value":3111},{"type":49,"tag":552,"props":3434,"children":3435},{},[3436,3469,3502,3535],{"type":49,"tag":522,"props":3437,"children":3438},{},[3439,3446,3454,3465],{"type":49,"tag":559,"props":3440,"children":3441},{},[3442],{"type":49,"tag":64,"props":3443,"children":3444},{},[3445],{"type":55,"value":572},{"type":49,"tag":559,"props":3447,"children":3448},{"align":965},[3449],{"type":49,"tag":64,"props":3450,"children":3451},{},[3452],{"type":55,"value":3453},"2.90x",{"type":49,"tag":559,"props":3455,"children":3456},{"align":965},[3457,3459,3464],{"type":55,"value":3458},"3.97x (",{"type":49,"tag":113,"props":3460,"children":3461},{},[3462],{"type":55,"value":3463},"1×8192×2048",{"type":55,"value":3144},{"type":49,"tag":559,"props":3466,"children":3467},{"align":965},[3468],{"type":55,"value":3149},{"type":49,"tag":522,"props":3470,"children":3471},{},[3472,3479,3487,3498],{"type":49,"tag":559,"props":3473,"children":3474},{},[3475],{"type":49,"tag":64,"props":3476,"children":3477},{},[3478],{"type":55,"value":744},{"type":49,"tag":559,"props":3480,"children":3481},{"align":965},[3482],{"type":49,"tag":64,"props":3483,"children":3484},{},[3485],{"type":55,"value":3486},"2.09x",{"type":49,"tag":559,"props":3488,"children":3489},{"align":965},[3490,3492,3497],{"type":55,"value":3491},"2.38x (",{"type":49,"tag":113,"props":3493,"children":3494},{},[3495],{"type":55,"value":3496},"1×1024×16×64",{"type":55,"value":3144},{"type":49,"tag":559,"props":3499,"children":3500},{"align":965},[3501],{"type":55,"value":3149},{"type":49,"tag":522,"props":3503,"children":3504},{},[3505,3512,3520,3531],{"type":49,"tag":559,"props":3506,"children":3507},{},[3508],{"type":49,"tag":64,"props":3509,"children":3510},{},[3511],{"type":55,"value":860},{"type":49,"tag":559,"props":3513,"children":3514},{"align":965},[3515],{"type":49,"tag":64,"props":3516,"children":3517},{},[3518],{"type":55,"value":3519},"1.69x",{"type":49,"tag":559,"props":3521,"children":3522},{"align":965},[3523,3525,3530],{"type":55,"value":3524},"1.93x (",{"type":49,"tag":113,"props":3526,"children":3527},{},[3528],{"type":55,"value":3529},"2×1024×8192",{"type":55,"value":3144},{"type":49,"tag":559,"props":3532,"children":3533},{"align":965},[3534],{"type":55,"value":3149},{"type":49,"tag":522,"props":3536,"children":3537},{},[3538,3545,3553,3563],{"type":49,"tag":559,"props":3539,"children":3540},{},[3541],{"type":49,"tag":64,"props":3542,"children":3543},{},[3544],{"type":55,"value":898},{"type":49,"tag":559,"props":3546,"children":3547},{"align":965},[3548],{"type":49,"tag":64,"props":3549,"children":3550},{},[3551],{"type":55,"value":3552},"3.00x",{"type":49,"tag":559,"props":3554,"children":3555},{"align":965},[3556,3558,3562],{"type":55,"value":3557},"3.67x (",{"type":49,"tag":113,"props":3559,"children":3560},{},[3561],{"type":55,"value":3142},{"type":55,"value":3144},{"type":49,"tag":559,"props":3564,"children":3565},{"align":965},[3566],{"type":55,"value":3149},{"type":49,"tag":58,"props":3568,"children":3569},{},[3570,3572,3577],{"type":55,"value":3571},"RMSNorm bandwidth utilization: 483 GB\u002Fs (R9700 theoretical: ~608 GB\u002Fs, ",{"type":49,"tag":64,"props":3573,"children":3574},{},[3575],{"type":55,"value":3576},"~79%",{"type":55,"value":3578},").",{"type":49,"tag":58,"props":3580,"children":3581},{},[3582],{"type":55,"value":3583},"R9700 speedups are higher than MI355X because PyTorch's default RDNA4 backend is less mature, leaving more room for Triton optimization. The bandwidth utilization (79%) is also significantly better than MI355X (44%).",{"type":49,"tag":86,"props":3585,"children":3587},{"id":3586},"end-to-end-ltx-video-r9700-25-frames-30-steps",[3588],{"type":55,"value":3589},"End-to-End LTX-Video (R9700, 25 frames, 30 steps)",{"type":49,"tag":514,"props":3591,"children":3592},{},[3593,3619],{"type":49,"tag":518,"props":3594,"children":3595},{},[3596],{"type":49,"tag":522,"props":3597,"children":3598},{},[3599,3603,3607,3611,3615],{"type":49,"tag":526,"props":3600,"children":3601},{},[3602],{"type":55,"value":3272},{"type":49,"tag":526,"props":3604,"children":3605},{"align":965},[3606],{"type":55,"value":3277},{"type":49,"tag":526,"props":3608,"children":3609},{"align":965},[3610],{"type":55,"value":3282},{"type":49,"tag":526,"props":3612,"children":3613},{"align":965},[3614],{"type":55,"value":3287},{"type":49,"tag":526,"props":3616,"children":3617},{"align":965},[3618],{"type":55,"value":3292},{"type":49,"tag":552,"props":3620,"children":3621},{},[3622,3648,3690],{"type":49,"tag":522,"props":3623,"children":3624},{},[3625,3630,3635,3640,3644],{"type":49,"tag":559,"props":3626,"children":3627},{},[3628],{"type":55,"value":3629},"baseline (mean of 3)",{"type":49,"tag":559,"props":3631,"children":3632},{"align":965},[3633],{"type":55,"value":3634},"6.91",{"type":49,"tag":559,"props":3636,"children":3637},{"align":965},[3638],{"type":55,"value":3639},"0.231",{"type":49,"tag":559,"props":3641,"children":3642},{"align":965},[3643],{"type":55,"value":3318},{"type":49,"tag":559,"props":3645,"children":3646},{"align":965},[3647],{"type":55,"value":3323},{"type":49,"tag":522,"props":3649,"children":3650},{},[3651,3659,3667,3675,3682],{"type":49,"tag":559,"props":3652,"children":3653},{},[3654],{"type":49,"tag":64,"props":3655,"children":3656},{},[3657],{"type":55,"value":3658},"triton (mean of 3)",{"type":49,"tag":559,"props":3660,"children":3661},{"align":965},[3662],{"type":49,"tag":64,"props":3663,"children":3664},{},[3665],{"type":55,"value":3666},"6.10",{"type":49,"tag":559,"props":3668,"children":3669},{"align":965},[3670],{"type":49,"tag":64,"props":3671,"children":3672},{},[3673],{"type":55,"value":3674},"0.203",{"type":49,"tag":559,"props":3676,"children":3677},{"align":965},[3678],{"type":49,"tag":64,"props":3679,"children":3680},{},[3681],{"type":55,"value":3318},{"type":49,"tag":559,"props":3683,"children":3684},{"align":965},[3685],{"type":49,"tag":64,"props":3686,"children":3687},{},[3688],{"type":55,"value":3689},"1.13x",{"type":49,"tag":522,"props":3691,"children":3692},{},[3693,3698,3703,3708,3712],{"type":49,"tag":559,"props":3694,"children":3695},{},[3696],{"type":55,"value":3697},"torch.compile (single run)",{"type":49,"tag":559,"props":3699,"children":3700},{"align":965},[3701],{"type":55,"value":3702},"5.05",{"type":49,"tag":559,"props":3704,"children":3705},{"align":965},[3706],{"type":55,"value":3707},"0.168",{"type":49,"tag":559,"props":3709,"children":3710},{"align":965},[3711],{"type":55,"value":3318},{"type":49,"tag":559,"props":3713,"children":3714},{"align":965},[3715],{"type":55,"value":3716},"1.37x",{"type":49,"tag":58,"props":3718,"children":3719},{},[3720,3722,3728],{"type":55,"value":3721},"Reviewer-facing benchmark files for this comparison live in ",{"type":49,"tag":109,"props":3723,"children":3725},{"className":3724},[],[3726],{"type":55,"value":3727},"examples\u002Fltx-video-benchmark\u002F",{"type":55,"value":3729},", including:",{"type":49,"tag":1094,"props":3731,"children":3732},{},[3733,3767,3778,3789],{"type":49,"tag":1098,"props":3734,"children":3735},{},[3736,3738,3744,3746,3752,3753,3759,3761],{"type":55,"value":3737},"Summary table with ",{"type":49,"tag":109,"props":3739,"children":3741},{"className":3740},[],[3742],{"type":55,"value":3743},"gen_time_s",{"type":55,"value":3745},", ",{"type":49,"tag":109,"props":3747,"children":3749},{"className":3748},[],[3750],{"type":55,"value":3751},"time_per_step_s",{"type":55,"value":3745},{"type":49,"tag":109,"props":3754,"children":3756},{"className":3755},[],[3757],{"type":55,"value":3758},"peak_memory_gb",{"type":55,"value":3760},", and ",{"type":49,"tag":109,"props":3762,"children":3764},{"className":3763},[],[3765],{"type":55,"value":3766},"speedup",{"type":49,"tag":1098,"props":3768,"children":3769},{},[3770,3772],{"type":55,"value":3771},"Consolidated JSON results in ",{"type":49,"tag":109,"props":3773,"children":3775},{"className":3774},[],[3776],{"type":55,"value":3777},"examples\u002Fltx-video-benchmark\u002Fbenchmark_results.json",{"type":49,"tag":1098,"props":3779,"children":3780},{},[3781,3783],{"type":55,"value":3782},"OpenCode run outputs in ",{"type":49,"tag":109,"props":3784,"children":3786},{"className":3785},[],[3787],{"type":55,"value":3788},"examples\u002Fltx-video-benchmark\u002Ftrace\u002Fopencode_live\u002Fresults.json",{"type":49,"tag":1098,"props":3790,"children":3791},{},[3792,3794],{"type":55,"value":3793},"OpenCode parsed trace in ",{"type":49,"tag":109,"props":3795,"children":3797},{"className":3796},[],[3798],{"type":55,"value":3799},"examples\u002Fltx-video-benchmark\u002Ftrace\u002Fopencode_live\u002Fopencode_trace_result.json",{"type":49,"tag":86,"props":3801,"children":3803},{"id":3802},"r9700-additional-validation",[3804],{"type":55,"value":3805},"R9700 Additional Validation",{"type":49,"tag":514,"props":3807,"children":3808},{},[3809,3825],{"type":49,"tag":518,"props":3810,"children":3811},{},[3812],{"type":49,"tag":522,"props":3813,"children":3814},{},[3815,3820],{"type":49,"tag":526,"props":3816,"children":3817},{},[3818],{"type":55,"value":3819},"Test",{"type":49,"tag":526,"props":3821,"children":3822},{},[3823],{"type":55,"value":3824},"Result",{"type":49,"tag":552,"props":3826,"children":3827},{},[3828,3841,3854,3874,3887,3900],{"type":49,"tag":522,"props":3829,"children":3830},{},[3831,3836],{"type":49,"tag":559,"props":3832,"children":3833},{},[3834],{"type":55,"value":3835},"Transformers injection (TinyLlama 1.1B)",{"type":49,"tag":559,"props":3837,"children":3838},{},[3839],{"type":55,"value":3840},"PASS — 45 RMSNorm patched, 99.9 tokens\u002Fs",{"type":49,"tag":522,"props":3842,"children":3843},{},[3844,3849],{"type":49,"tag":559,"props":3845,"children":3846},{},[3847],{"type":55,"value":3848},"HuggingFace Kernels Hub integration",{"type":49,"tag":559,"props":3850,"children":3851},{},[3852],{"type":55,"value":3853},"PASS — Hub kernel loads and runs on ROCm",{"type":49,"tag":522,"props":3855,"children":3856},{},[3857,3862],{"type":49,"tag":559,"props":3858,"children":3859},{},[3860],{"type":55,"value":3861},"Local Triton vs Hub kernel (small shape)",{"type":49,"tag":559,"props":3863,"children":3864},{},[3865,3867,3872],{"type":55,"value":3866},"Local ",{"type":49,"tag":64,"props":3868,"children":3869},{},[3870],{"type":55,"value":3871},"5.92x",{"type":55,"value":3873}," vs Hub 1.27x (lower launch overhead)",{"type":49,"tag":522,"props":3875,"children":3876},{},[3877,3882],{"type":49,"tag":559,"props":3878,"children":3879},{},[3880],{"type":55,"value":3881},"Local Triton vs Hub kernel (large shape)",{"type":49,"tag":559,"props":3883,"children":3884},{},[3885],{"type":55,"value":3886},"Local 3.59x vs Hub 3.57x (comparable)",{"type":49,"tag":522,"props":3888,"children":3889},{},[3890,3895],{"type":49,"tag":559,"props":3891,"children":3892},{},[3893],{"type":55,"value":3894},"num_warps sweep (2\u002F4\u002F8\u002F16\u002F32)",{"type":49,"tag":559,"props":3896,"children":3897},{},[3898],{"type":55,"value":3899},"Default heuristic (4\u002F8\u002F16) is near-optimal; nw=32 always worst",{"type":49,"tag":522,"props":3901,"children":3902},{},[3903,3908],{"type":49,"tag":559,"props":3904,"children":3905},{},[3906],{"type":55,"value":3907},"rocprof kernel fusion analysis",{"type":49,"tag":559,"props":3909,"children":3910},{},[3911],{"type":55,"value":3912},"Triton fuses 4 PyTorch kernels (pow+mean+rsqrt+mul) into 1",{"type":49,"tag":86,"props":3914,"children":3916},{"id":3915},"cuda-reference-h100-for-comparison",[3917],{"type":55,"value":3918},"CUDA Reference (H100, for comparison)",{"type":49,"tag":514,"props":3920,"children":3921},{},[3922,3948],{"type":49,"tag":518,"props":3923,"children":3924},{},[3925],{"type":49,"tag":522,"props":3926,"children":3927},{},[3928,3934,3939,3944],{"type":49,"tag":526,"props":3929,"children":3931},{"align":3930},"left",[3932],{"type":55,"value":3933},"Shape",{"type":49,"tag":526,"props":3935,"children":3936},{"align":965},[3937],{"type":55,"value":3938},"Custom (ms)",{"type":49,"tag":526,"props":3940,"children":3941},{"align":965},[3942],{"type":55,"value":3943},"PyTorch (ms)",{"type":49,"tag":526,"props":3945,"children":3946},{"align":965},[3947],{"type":55,"value":3292},{"type":49,"tag":552,"props":3949,"children":3950},{},[3951,3980],{"type":49,"tag":522,"props":3952,"children":3953},{},[3954,3962,3967,3972],{"type":49,"tag":559,"props":3955,"children":3956},{"align":3930},[3957],{"type":49,"tag":113,"props":3958,"children":3959},{},[3960],{"type":55,"value":3961},"1×1024×2048",{"type":49,"tag":559,"props":3963,"children":3964},{"align":965},[3965],{"type":55,"value":3966},"0.019",{"type":49,"tag":559,"props":3968,"children":3969},{"align":965},[3970],{"type":55,"value":3971},"0.065",{"type":49,"tag":559,"props":3973,"children":3974},{"align":965},[3975],{"type":49,"tag":64,"props":3976,"children":3977},{},[3978],{"type":55,"value":3979},"3.37x",{"type":49,"tag":522,"props":3981,"children":3982},{},[3983,3991,3996,4001],{"type":49,"tag":559,"props":3984,"children":3985},{"align":3930},[3986],{"type":49,"tag":113,"props":3987,"children":3988},{},[3989],{"type":55,"value":3990},"2×4096×3072",{"type":49,"tag":559,"props":3992,"children":3993},{"align":965},[3994],{"type":55,"value":3995},"0.087",{"type":49,"tag":559,"props":3997,"children":3998},{"align":965},[3999],{"type":55,"value":4000},"0.208",{"type":49,"tag":559,"props":4002,"children":4003},{"align":965},[4004],{"type":49,"tag":64,"props":4005,"children":4006},{},[4007],{"type":55,"value":4008},"2.41x",{"type":49,"tag":58,"props":4010,"children":4011},{},[4012],{"type":55,"value":4013},"H100 E2E: ~6% (RMSNorm is ~5% of total compute).",{"type":49,"tag":86,"props":4015,"children":4017},{"id":4016},"optimization-targets",[4018],{"type":55,"value":4019},"Optimization Targets",{"type":49,"tag":514,"props":4021,"children":4022},{},[4023,4051],{"type":49,"tag":518,"props":4024,"children":4025},{},[4026],{"type":49,"tag":522,"props":4027,"children":4028},{},[4029,4033,4037,4041,4046],{"type":49,"tag":526,"props":4030,"children":4031},{},[4032],{"type":55,"value":550},{"type":49,"tag":526,"props":4034,"children":4035},{"align":965},[4036],{"type":55,"value":982},{"type":49,"tag":526,"props":4038,"children":4039},{"align":965},[4040],{"type":55,"value":1025},{"type":49,"tag":526,"props":4042,"children":4043},{"align":965},[4044],{"type":55,"value":4045},"Target",{"type":49,"tag":526,"props":4047,"children":4048},{"align":965},[4049],{"type":55,"value":4050},"Priority",{"type":49,"tag":552,"props":4052,"children":4053},{},[4054,4079,4104,4129],{"type":49,"tag":522,"props":4055,"children":4056},{},[4057,4061,4065,4069,4074],{"type":49,"tag":559,"props":4058,"children":4059},{},[4060],{"type":55,"value":572},{"type":49,"tag":559,"props":4062,"children":4063},{"align":965},[4064],{"type":55,"value":3132},{"type":49,"tag":559,"props":4066,"children":4067},{"align":965},[4068],{"type":55,"value":3453},{"type":49,"tag":559,"props":4070,"children":4071},{"align":965},[4072],{"type":55,"value":4073},">3x (R9700)",{"type":49,"tag":559,"props":4075,"children":4076},{"align":965},[4077],{"type":55,"value":4078},"P0 — MI355X bandwidth util (44%→60%+)",{"type":49,"tag":522,"props":4080,"children":4081},{},[4082,4086,4090,4094,4099],{"type":49,"tag":559,"props":4083,"children":4084},{},[4085],{"type":55,"value":898},{"type":49,"tag":559,"props":4087,"children":4088},{"align":965},[4089],{"type":55,"value":3233},{"type":49,"tag":559,"props":4091,"children":4092},{"align":965},[4093],{"type":55,"value":3552},{"type":49,"tag":559,"props":4095,"children":4096},{"align":965},[4097],{"type":55,"value":4098},">3.5x (R9700)",{"type":49,"tag":559,"props":4100,"children":4101},{"align":965},[4102],{"type":55,"value":4103},"P1 — already strong on both",{"type":49,"tag":522,"props":4105,"children":4106},{},[4107,4111,4115,4119,4124],{"type":49,"tag":559,"props":4108,"children":4109},{},[4110],{"type":55,"value":860},{"type":49,"tag":559,"props":4112,"children":4113},{"align":965},[4114],{"type":55,"value":3200},{"type":49,"tag":559,"props":4116,"children":4117},{"align":965},[4118],{"type":55,"value":3519},{"type":49,"tag":559,"props":4120,"children":4121},{"align":965},[4122],{"type":55,"value":4123},">2x",{"type":49,"tag":559,"props":4125,"children":4126},{"align":965},[4127],{"type":55,"value":4128},"P1 — tanh overhead",{"type":49,"tag":522,"props":4130,"children":4131},{},[4132,4136,4140,4144,4149],{"type":49,"tag":559,"props":4133,"children":4134},{},[4135],{"type":55,"value":744},{"type":49,"tag":559,"props":4137,"children":4138},{"align":965},[4139],{"type":55,"value":3167},{"type":49,"tag":559,"props":4141,"children":4142},{"align":965},[4143],{"type":55,"value":3486},{"type":49,"tag":559,"props":4145,"children":4146},{"align":965},[4147],{"type":55,"value":4148},">2.5x (R9700)",{"type":49,"tag":559,"props":4150,"children":4151},{"align":965},[4152],{"type":55,"value":4153},"P2 — small head_dim launch overhead",{"type":49,"tag":79,"props":4155,"children":4157},{"id":4156},"common-issues-on-rocm",[4158],{"type":55,"value":4159},"Common Issues on ROCm",{"type":49,"tag":514,"props":4161,"children":4162},{},[4163,4184],{"type":49,"tag":518,"props":4164,"children":4165},{},[4166],{"type":49,"tag":522,"props":4167,"children":4168},{},[4169,4174,4179],{"type":49,"tag":526,"props":4170,"children":4171},{},[4172],{"type":55,"value":4173},"Issue",{"type":49,"tag":526,"props":4175,"children":4176},{},[4177],{"type":55,"value":4178},"Symptom",{"type":49,"tag":526,"props":4180,"children":4181},{},[4182],{"type":55,"value":4183},"Fix",{"type":49,"tag":552,"props":4185,"children":4186},{},[4187,4219,4254,4276,4312,4342,4360,4384],{"type":49,"tag":522,"props":4188,"children":4189},{},[4190,4198,4203],{"type":49,"tag":559,"props":4191,"children":4192},{},[4193],{"type":49,"tag":64,"props":4194,"children":4195},{},[4196],{"type":55,"value":4197},"Autotune BLOCK_D",{"type":49,"tag":559,"props":4199,"children":4200},{},[4201],{"type":55,"value":4202},"Wrong results (max_abs 4-8+)",{"type":49,"tag":559,"props":4204,"children":4205},{},[4206,4211,4213],{"type":49,"tag":64,"props":4207,"children":4208},{},[4209],{"type":55,"value":4210},"Never autotune BLOCK_D.",{"type":55,"value":4212}," Use ",{"type":49,"tag":109,"props":4214,"children":4216},{"className":4215},[],[4217],{"type":55,"value":4218},"triton.next_power_of_2(D)",{"type":49,"tag":522,"props":4220,"children":4221},{},[4222,4230,4242],{"type":49,"tag":559,"props":4223,"children":4224},{},[4225],{"type":49,"tag":64,"props":4226,"children":4227},{},[4228],{"type":55,"value":4229},"RoPE batch OOB",{"type":49,"tag":559,"props":4231,"children":4232},{},[4233,4235,4241],{"type":55,"value":4234},"GPU crash (",{"type":49,"tag":109,"props":4236,"children":4238},{"className":4237},[],[4239],{"type":55,"value":4240},"Memory access fault",{"type":55,"value":3144},{"type":49,"tag":559,"props":4243,"children":4244},{},[4245,4247,4252],{"type":55,"value":4246},"Use ",{"type":49,"tag":109,"props":4248,"children":4250},{"className":4249},[],[4251],{"type":55,"value":1757},{"type":55,"value":4253}," for cos\u002Fsin indexing",{"type":49,"tag":522,"props":4255,"children":4256},{},[4257,4266,4271],{"type":49,"tag":559,"props":4258,"children":4259},{},[4260],{"type":49,"tag":109,"props":4261,"children":4263},{"className":4262},[],[4264],{"type":55,"value":4265},"tl.libdevice",{"type":49,"tag":559,"props":4267,"children":4268},{},[4269],{"type":55,"value":4270},"Not found on AMD",{"type":49,"tag":559,"props":4272,"children":4273},{},[4274],{"type":55,"value":4275},"Use manual math formulas",{"type":49,"tag":522,"props":4277,"children":4278},{},[4279,4296,4301],{"type":49,"tag":559,"props":4280,"children":4281},{},[4282,4288,4290],{"type":49,"tag":109,"props":4283,"children":4285},{"className":4284},[],[4286],{"type":55,"value":4287},"tl.tanh",{"type":55,"value":4289}," \u002F ",{"type":49,"tag":109,"props":4291,"children":4293},{"className":4292},[],[4294],{"type":55,"value":4295},"tl.math.tanh",{"type":49,"tag":559,"props":4297,"children":4298},{},[4299],{"type":55,"value":4300},"Not on ROCm",{"type":49,"tag":559,"props":4302,"children":4303},{},[4304,4306],{"type":55,"value":4305},"Manual: ",{"type":49,"tag":109,"props":4307,"children":4309},{"className":4308},[],[4310],{"type":55,"value":4311},"e2x=exp(2x); (e2x-1)\u002F(e2x+1)",{"type":49,"tag":522,"props":4313,"children":4314},{},[4315,4320,4325],{"type":49,"tag":559,"props":4316,"children":4317},{},[4318],{"type":55,"value":4319},"Python min\u002Fmax",{"type":49,"tag":559,"props":4321,"children":4322},{},[4323],{"type":55,"value":4324},"Runtime error",{"type":49,"tag":559,"props":4326,"children":4327},{},[4328,4334,4336],{"type":49,"tag":109,"props":4329,"children":4331},{"className":4330},[],[4332],{"type":55,"value":4333},"tl.minimum()",{"type":55,"value":4335},"\u002F",{"type":49,"tag":109,"props":4337,"children":4339},{"className":4338},[],[4340],{"type":55,"value":4341},"tl.maximum()",{"type":49,"tag":522,"props":4343,"children":4344},{},[4345,4350,4355],{"type":49,"tag":559,"props":4346,"children":4347},{},[4348],{"type":55,"value":4349},"LDS overflow",{"type":49,"tag":559,"props":4351,"children":4352},{},[4353],{"type":55,"value":4354},"HIP OOM",{"type":49,"tag":559,"props":4356,"children":4357},{},[4358],{"type":55,"value":4359},"Reduce num_stages to 2",{"type":49,"tag":522,"props":4361,"children":4362},{},[4363,4368,4373],{"type":49,"tag":559,"props":4364,"children":4365},{},[4366],{"type":55,"value":4367},"Weight is None",{"type":49,"tag":559,"props":4369,"children":4370},{},[4371],{"type":55,"value":4372},"AttributeError",{"type":49,"tag":559,"props":4374,"children":4375},{},[4376,4378],{"type":55,"value":4377},"Check ",{"type":49,"tag":109,"props":4379,"children":4381},{"className":4380},[],[4382],{"type":55,"value":4383},"elementwise_affine",{"type":49,"tag":522,"props":4385,"children":4386},{},[4387,4392,4397],{"type":49,"tag":559,"props":4388,"children":4389},{},[4390],{"type":55,"value":4391},"isinstance() miss",{"type":49,"tag":559,"props":4393,"children":4394},{},[4395],{"type":55,"value":4396},"RMSNorm not patched",{"type":49,"tag":559,"props":4398,"children":4399},{},[4400,4401],{"type":55,"value":4246},{"type":49,"tag":109,"props":4402,"children":4404},{"className":4403},[],[4405],{"type":55,"value":3029},{"type":49,"tag":1059,"props":4407,"children":4408},{},[4409],{"type":49,"tag":58,"props":4410,"children":4411},{},[4412,4413,4419],{"type":55,"value":1066},{"type":49,"tag":1068,"props":4414,"children":4416},{"href":4415},"references\u002Ftroubleshooting.md",[4417],{"type":55,"value":4418},"troubleshooting.md",{"type":55,"value":4420}," for all common issues.",{"type":49,"tag":79,"props":4422,"children":4424},{"id":4423},"performance-profiling",[4425],{"type":55,"value":4426},"Performance Profiling",{"type":49,"tag":101,"props":4428,"children":4430},{"className":205,"code":4429,"language":207,"meta":106,"style":106},"rocprof --stats python your_kernel.py\nrocprofv3 -i metrics.txt python your_kernel.py\nrocm-bandwidth-test\nrocminfo | grep -E \"Name|Compute Unit|Wavefront\"\n",[4431],{"type":49,"tag":109,"props":4432,"children":4433},{"__ignoreMap":106},[4434,4457,4483,4491],{"type":49,"tag":113,"props":4435,"children":4436},{"class":115,"line":116},[4437,4442,4447,4452],{"type":49,"tag":113,"props":4438,"children":4439},{"style":217},[4440],{"type":55,"value":4441},"rocprof",{"type":49,"tag":113,"props":4443,"children":4444},{"style":222},[4445],{"type":55,"value":4446}," --stats",{"type":49,"tag":113,"props":4448,"children":4449},{"style":222},[4450],{"type":55,"value":4451}," python",{"type":49,"tag":113,"props":4453,"children":4454},{"style":222},[4455],{"type":55,"value":4456}," your_kernel.py\n",{"type":49,"tag":113,"props":4458,"children":4459},{"class":115,"line":125},[4460,4465,4470,4475,4479],{"type":49,"tag":113,"props":4461,"children":4462},{"style":217},[4463],{"type":55,"value":4464},"rocprofv3",{"type":49,"tag":113,"props":4466,"children":4467},{"style":222},[4468],{"type":55,"value":4469}," -i",{"type":49,"tag":113,"props":4471,"children":4472},{"style":222},[4473],{"type":55,"value":4474}," metrics.txt",{"type":49,"tag":113,"props":4476,"children":4477},{"style":222},[4478],{"type":55,"value":4451},{"type":49,"tag":113,"props":4480,"children":4481},{"style":222},[4482],{"type":55,"value":4456},{"type":49,"tag":113,"props":4484,"children":4485},{"class":115,"line":134},[4486],{"type":49,"tag":113,"props":4487,"children":4488},{"style":217},[4489],{"type":55,"value":4490},"rocm-bandwidth-test\n",{"type":49,"tag":113,"props":4492,"children":4493},{"class":115,"line":143},[4494,4499,4505,4510,4515,4520,4525],{"type":49,"tag":113,"props":4495,"children":4496},{"style":217},[4497],{"type":55,"value":4498},"rocminfo",{"type":49,"tag":113,"props":4500,"children":4502},{"style":4501},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[4503],{"type":55,"value":4504}," |",{"type":49,"tag":113,"props":4506,"children":4507},{"style":217},[4508],{"type":55,"value":4509}," grep",{"type":49,"tag":113,"props":4511,"children":4512},{"style":222},[4513],{"type":55,"value":4514}," -E",{"type":49,"tag":113,"props":4516,"children":4517},{"style":4501},[4518],{"type":55,"value":4519}," \"",{"type":49,"tag":113,"props":4521,"children":4522},{"style":222},[4523],{"type":55,"value":4524},"Name|Compute Unit|Wavefront",{"type":49,"tag":113,"props":4526,"children":4527},{"style":4501},[4528],{"type":55,"value":4529},"\"\n",{"type":49,"tag":79,"props":4531,"children":4533},{"id":4532},"see-also",[4534],{"type":55,"value":4535},"See Also",{"type":49,"tag":86,"props":4537,"children":4539},{"id":4538},"benchmark-test-scripts",[4540],{"type":55,"value":4541},"Benchmark & Test Scripts",{"type":49,"tag":1094,"props":4543,"children":4544},{},[4545,4556,4567,4578,4589,4600],{"type":49,"tag":1098,"props":4546,"children":4547},{},[4548,4554],{"type":49,"tag":1068,"props":4549,"children":4551},{"href":4550},"scripts\u002Fbenchmark_kernels.py",[4552],{"type":55,"value":4553},"benchmark_kernels.py",{"type":55,"value":4555}," - Micro-benchmark all 4 kernels (correctness + perf + bandwidth)",{"type":49,"tag":1098,"props":4557,"children":4558},{},[4559,4565],{"type":49,"tag":1068,"props":4560,"children":4562},{"href":4561},"scripts\u002Fbenchmark_e2e.py",[4563],{"type":55,"value":4564},"benchmark_e2e.py",{"type":55,"value":4566}," - End-to-end LTX-Video pipeline benchmark (baseline vs Triton vs compile)",{"type":49,"tag":1098,"props":4568,"children":4569},{},[4570,4576],{"type":49,"tag":1068,"props":4571,"children":4573},{"href":4572},"scripts\u002Fsweep_num_warps.py",[4574],{"type":55,"value":4575},"sweep_num_warps.py",{"type":55,"value":4577}," - num_warps sweep for R9700 Wave32 optimization",{"type":49,"tag":1098,"props":4579,"children":4580},{},[4581,4587],{"type":49,"tag":1068,"props":4582,"children":4584},{"href":4583},"scripts\u002Fltx_kernel_injection_example.py",[4585],{"type":55,"value":4586},"ltx_kernel_injection_example.py",{"type":55,"value":4588}," - Minimal diffusers injection example",{"type":49,"tag":1098,"props":4590,"children":4591},{},[4592,4598],{"type":49,"tag":1068,"props":4593,"children":4595},{"href":4594},"scripts\u002Ftransformers_injection_example.py",[4596],{"type":55,"value":4597},"transformers_injection_example.py",{"type":55,"value":4599}," - Minimal transformers injection example",{"type":49,"tag":1098,"props":4601,"children":4602},{},[4603,4609],{"type":49,"tag":1068,"props":4604,"children":4606},{"href":4605},"scripts\u002Fhuggingface_kernels_example.py",[4607],{"type":55,"value":4608},"huggingface_kernels_example.py",{"type":55,"value":4610}," - HuggingFace Kernels Hub integration example",{"type":49,"tag":86,"props":4612,"children":4614},{"id":4613},"integration-guides",[4615],{"type":55,"value":4616},"Integration Guides",{"type":49,"tag":1094,"props":4618,"children":4619},{},[4620,4629,4640,4658],{"type":49,"tag":1098,"props":4621,"children":4622},{},[4623,4627],{"type":49,"tag":1068,"props":4624,"children":4625},{"href":2782},[4626],{"type":55,"value":2785},{"type":55,"value":4628}," - LTX-Video pipeline integration",{"type":49,"tag":1098,"props":4630,"children":4631},{},[4632,4638],{"type":49,"tag":1068,"props":4633,"children":4635},{"href":4634},"references\u002Ftransformers-integration.md",[4636],{"type":55,"value":4637},"transformers-integration.md",{"type":55,"value":4639}," - LLaMA\u002FMistral\u002FQwen integration",{"type":49,"tag":1098,"props":4641,"children":4642},{},[4643,4649,4651,4657],{"type":49,"tag":1068,"props":4644,"children":4646},{"href":4645},"references\u002Fhuggingface-kernels-integration.md",[4647],{"type":55,"value":4648},"huggingface-kernels-integration.md",{"type":55,"value":4650}," - HuggingFace Kernels Hub (",{"type":49,"tag":109,"props":4652,"children":4654},{"className":4653},[],[4655],{"type":55,"value":4656},"get_kernel",{"type":55,"value":3144},{"type":49,"tag":1098,"props":4659,"children":4660},{},[4661,4667],{"type":49,"tag":1068,"props":4662,"children":4664},{"href":4663},"references\u002Fkernel-templates.md",[4665],{"type":55,"value":4666},"kernel-templates.md",{"type":55,"value":4668}," - Complete Triton kernel templates (incl. GEMM with XCD Swizzle)",{"type":49,"tag":86,"props":4670,"children":4672},{"id":4671},"gpu-optimization-guides",[4673],{"type":55,"value":4674},"GPU Optimization Guides",{"type":49,"tag":1094,"props":4676,"children":4677},{},[4678,4688],{"type":49,"tag":1098,"props":4679,"children":4680},{},[4681,4686],{"type":49,"tag":1068,"props":4682,"children":4683},{"href":1070},[4684],{"type":55,"value":4685},"mi355x-optimization-guide.md",{"type":55,"value":4687}," - MI355X (gfx950) deep dive",{"type":49,"tag":1098,"props":4689,"children":4690},{},[4691,4696],{"type":49,"tag":1068,"props":4692,"children":4693},{"href":1078},[4694],{"type":55,"value":4695},"r9700-optimization-guide.md",{"type":55,"value":4697}," - R9700 (RDNA4) deep dive",{"type":49,"tag":86,"props":4699,"children":4701},{"id":4700},"reference",[4702],{"type":55,"value":4703},"Reference",{"type":49,"tag":1094,"props":4705,"children":4706},{},[4707,4716,4727,4738],{"type":49,"tag":1098,"props":4708,"children":4709},{},[4710,4714],{"type":49,"tag":1068,"props":4711,"children":4712},{"href":4415},[4713],{"type":55,"value":4418},{"type":55,"value":4715}," - Common issues and solutions",{"type":49,"tag":1098,"props":4717,"children":4718},{},[4719,4725],{"type":49,"tag":1068,"props":4720,"children":4722},{"href":4721},"references\u002Fkernelbench-classification.md",[4723],{"type":55,"value":4724},"kernelbench-classification.md",{"type":55,"value":4726}," - KernelBench operator taxonomy",{"type":49,"tag":1098,"props":4728,"children":4729},{},[4730,4736],{"type":49,"tag":1068,"props":4731,"children":4733},{"href":4732},"references\u002Fskill-evaluation-methodology.md",[4734],{"type":55,"value":4735},"skill-evaluation-methodology.md",{"type":55,"value":4737}," - How to evaluate and improve skills",{"type":49,"tag":1098,"props":4739,"children":4740},{},[4741,4747],{"type":49,"tag":1068,"props":4742,"children":4744},{"href":4743},"references\u002Fkernel-agent-knowledge-base.md",[4745],{"type":55,"value":4746},"kernel-agent-knowledge-base.md",{"type":55,"value":4748}," - Knowledge from kernel-agent project",{"type":49,"tag":86,"props":4750,"children":4752},{"id":4751},"external-resources",[4753],{"type":55,"value":4754},"External Resources",{"type":49,"tag":1094,"props":4756,"children":4757},{},[4758,4769,4779,4789,4799],{"type":49,"tag":1098,"props":4759,"children":4760},{},[4761],{"type":49,"tag":1068,"props":4762,"children":4766},{"href":4763,"rel":4764},"https:\u002F\u002Ftriton-lang.org\u002F",[4765],"nofollow",[4767],{"type":55,"value":4768},"Triton Documentation",{"type":49,"tag":1098,"props":4770,"children":4771},{},[4772],{"type":49,"tag":1068,"props":4773,"children":4776},{"href":4774,"rel":4775},"https:\u002F\u002Frocm.docs.amd.com\u002F",[4765],[4777],{"type":55,"value":4778},"ROCm Documentation",{"type":49,"tag":1098,"props":4780,"children":4781},{},[4782],{"type":49,"tag":1068,"props":4783,"children":4786},{"href":4784,"rel":4785},"https:\u002F\u002Fhuggingface.co\u002Fkernels-community",[4765],[4787],{"type":55,"value":4788},"HuggingFace Kernels Hub",{"type":49,"tag":1098,"props":4790,"children":4791},{},[4792],{"type":49,"tag":1068,"props":4793,"children":4796},{"href":4794,"rel":4795},"https:\u002F\u002Fhuggingface.co\u002FLightricks\u002FLTX-Video",[4765],[4797],{"type":55,"value":4798},"LTX-Video on HuggingFace",{"type":49,"tag":1098,"props":4800,"children":4801},{},[4802],{"type":49,"tag":1068,"props":4803,"children":4806},{"href":4804,"rel":4805},"https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Fdiffusers\u002Fen\u002Findex",[4765],[4807],{"type":55,"value":4808},"HuggingFace Diffusers",{"type":49,"tag":4810,"props":4811,"children":4812},"style",{},[4813],{"type":55,"value":4814},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":4816,"total":143},[4817,4833,4846,4854],{"slug":4818,"name":4818,"fn":4819,"description":4820,"org":4821,"tags":4822,"stars":28,"repoUrl":29,"updatedAt":4832},"cpu-kernels","optimize C++ CPU kernels for Hugging Face","Provides guidance for writing, optimizing, and benchmarking C++ CPU kernels with SIMD intrinsics (AVX2\u002FAVX512) for the Hugging Face kernels ecosystem. Includes a two-phase workflow: Phase 1 correctness (generic → AVX2) and Phase 2 performance exploration (AVX512 with branching trial loop), runtime CPU dispatch, OpenMP threading, and brgemm integration for GEMM-heavy kernels.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4823,4826,4829,4831],{"name":4824,"slug":4825,"type":15},"C#","c",{"name":4827,"slug":4828,"type":15},"Engineering","engineering",{"name":9,"slug":4830,"type":15},"hugging-face",{"name":13,"slug":14,"type":15},"2026-06-09T07:19:37.783356",{"slug":4834,"name":4834,"fn":4835,"description":4836,"org":4837,"tags":4838,"stars":28,"repoUrl":29,"updatedAt":4845},"cuda-kernels","write and benchmark optimized CUDA kernels","Provides guidance for writing and benchmarking optimized CUDA kernels for NVIDIA GPUs (H100, A100, T4) targeting HuggingFace diffusers and transformers libraries. Kernels must be kernel-builder\u002FABI3-compliant: no pybind11, no setup.py, TORCH_LIBRARY_EXPAND bindings only. Supports models like LTX-Video, Stable Diffusion, LLaMA, Mistral, and Qwen. Includes integration with HuggingFace Kernels Hub (get_kernel) for loading pre-compiled kernels. Includes benchmarking scripts to compare kernel performance against baseline implementations.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4839,4840,4841,4842,4843],{"name":23,"slug":24,"type":15},{"name":20,"slug":21,"type":15},{"name":9,"slug":4830,"type":15},{"name":13,"slug":14,"type":15},{"name":4844,"slug":105,"type":15},"Python","2026-05-15T06:18:29.266429",{"slug":4,"name":4,"fn":5,"description":6,"org":4847,"tags":4848,"stars":28,"repoUrl":29,"updatedAt":30},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4849,4850,4851,4852,4853],{"name":23,"slug":24,"type":15},{"name":20,"slug":21,"type":15},{"name":13,"slug":14,"type":15},{"name":26,"slug":27,"type":15},{"name":17,"slug":18,"type":15},{"slug":4855,"name":4855,"fn":4856,"description":4857,"org":4858,"tags":4859,"stars":28,"repoUrl":29,"updatedAt":4864},"xpu-kernels","optimize Triton kernels for Intel XPU","Provides guidance for writing, optimizing, and benchmarking Triton kernels for Intel XPU GPUs (Battlemage\u002FArc Pro B50) using the Xe-Forge optimization framework. Includes an LLM-driven trial-loop workflow (analyze, validate, benchmark, profile, finalize), XPU-specific patterns (tensor descriptors, GRF mode, tile swizzling), KernelBench fused kernels, and Flash Attention.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4860,4861,4862,4863],{"name":23,"slug":24,"type":15},{"name":20,"slug":21,"type":15},{"name":13,"slug":14,"type":15},{"name":17,"slug":18,"type":15},"2026-07-18T05:15:18.767865",{"items":4866,"total":1651},[4867,4885,4899,4916,4930,4941,4954,4969,4983,4993,5006,5018],{"slug":4868,"name":4868,"fn":4869,"description":4870,"org":4871,"tags":4872,"stars":4882,"repoUrl":4883,"updatedAt":4884},"train-sentence-transformers","train sentence-transformers models","Train or fine-tune sentence-transformers models across `SentenceTransformer` (bi-encoder; dense or static embedding model; for retrieval, similarity, clustering, classification, paraphrase mining, dedup, multimodal), `CrossEncoder` (reranker; pair scoring for two-stage retrieval \u002F pair classification), and `SparseEncoder` (SPLADE, sparse embedding model; for learned-sparse retrieval). Covers loss selection, hard-negative mining, evaluators, distillation, LoRA, Matryoshka, and Hugging Face Hub publishing. Use for any sentence-transformers training task.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4873,4874,4875,4878,4879],{"name":20,"slug":21,"type":15},{"name":9,"slug":4830,"type":15},{"name":4876,"slug":4877,"type":15},"LLM","llm",{"name":4844,"slug":105,"type":15},{"name":4880,"slug":4881,"type":15},"Search","search",18914,"https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fsentence-transformers","2026-05-08T05:09:16.820066",{"slug":4886,"name":4886,"fn":4887,"description":4888,"org":4889,"tags":4890,"stars":4896,"repoUrl":4897,"updatedAt":4898},"trl-training","train and fine-tune LLMs with TRL","Train and fine-tune transformer language models using TRL (Transformers Reinforcement Learning). Supports SFT, DPO, GRPO, KTO, RLOO and Reward Model training via CLI commands.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4891,4892,4893,4894,4895],{"name":23,"slug":24,"type":15},{"name":20,"slug":21,"type":15},{"name":9,"slug":4830,"type":15},{"name":4876,"slug":4877,"type":15},{"name":4844,"slug":105,"type":15},18850,"https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftrl","2026-04-06T18:25:32.746828",{"slug":4900,"name":4900,"fn":4901,"description":4902,"org":4903,"tags":4904,"stars":4913,"repoUrl":4914,"updatedAt":4915},"hf-cli","manage Hugging Face Hub resources via CLI","Hugging Face Hub CLI (`hf`) for downloading, uploading, and managing models, datasets, spaces, buckets, repos, papers, jobs, and more on the Hugging Face Hub. Use when: handling authentication; managing local cache; managing Hugging Face Buckets; running or scheduling jobs on Hugging Face infrastructure; managing Hugging Face repos; discussions and pull requests; browsing models, datasets and spaces; reading, searching, or browsing academic papers; managing collections; querying datasets; configuring spaces; setting up webhooks; or deploying and managing HF Inference Endpoints. Make sure to use this skill whenever the user mentions 'hf', 'huggingface', 'Hugging Face', 'huggingface-cli', or 'hugging face cli', or wants to do anything related to the Hugging Face ecosystem and to AI and ML in general. Also use for cloud storage needs like training checkpoints, data pipelines, or agent traces. Use even if the user doesn't explicitly ask for a CLI command. Replaces the deprecated `huggingface-cli`.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4905,4908,4911,4912],{"name":4906,"slug":4907,"type":15},"CLI","cli",{"name":4909,"slug":4910,"type":15},"Datasets","datasets",{"name":9,"slug":4830,"type":15},{"name":4876,"slug":4877,"type":15},10861,"https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fskills","2026-04-06T18:25:34.020855",{"slug":4917,"name":4917,"fn":4918,"description":4919,"org":4920,"tags":4921,"stars":4913,"repoUrl":4914,"updatedAt":4929},"hf-cloud-aws-context-discovery","discover local AWS environment context","Discover the user's local AWS context (active profile, region, account ID, caller identity) at the start of any AWS task. Use this skill before any other AWS work — deploying to SageMaker, creating resources, calling AWS APIs, or anything that touches an AWS account. Use it especially when the user has not specified a region or profile explicitly, when they say things like \"use my AWS account\", \"deploy to AWS\", \"use my profile\", or when about to make any AWS CLI or SDK call. Never guess the region or account ID — always use this skill to read it from the local configuration first.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4922,4925,4926],{"name":4923,"slug":4924,"type":15},"AWS","aws",{"name":4906,"slug":4907,"type":15},{"name":4927,"slug":4928,"type":15},"Configuration","configuration","2026-07-08T05:55:33.716099",{"slug":4931,"name":4931,"fn":4932,"description":4933,"org":4934,"tags":4935,"stars":4913,"repoUrl":4914,"updatedAt":4940},"hf-cloud-python-env-setup","set up Python environments for AWS","Set up an isolated Python environment for SageMaker \u002F AWS work, with the right Python version and current boto3. Use this skill whenever Python code will be executed for a SageMaker deployment, training job, or any AWS automation — including when about to run `pip install`, when about to invoke `boto3`, when creating or activating a virtualenv, or when the user asks to \"set up the environment\". Never use system Python and never `pip install` into it. Always isolate. This skill prevents the most common failure modes: wrong Python version, dependency conflicts, and stale SDKs.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4936,4937,4938,4939],{"name":23,"slug":24,"type":15},{"name":4923,"slug":4924,"type":15},{"name":4827,"slug":4828,"type":15},{"name":4844,"slug":105,"type":15},"2026-07-08T05:55:32.505017",{"slug":4942,"name":4942,"fn":4943,"description":4944,"org":4945,"tags":4946,"stars":4913,"repoUrl":4914,"updatedAt":4953},"hf-cloud-sagemaker-deployment-planner","plan model deployments to Amazon SageMaker","Plan and coordinate the deployment of a model to Amazon SageMaker AI. Use this skill whenever the user wants to deploy, host, serve, or expose a model on SageMaker or AWS — including phrases like \"deploy a model\", \"host this LLM on AWS\", \"serve this embedding model\", \"deploy a reranker\", \"deploy a text-to-image \u002F diffusion model\", \"host this for async inference\", \"create an endpoint\", \"serve my fine-tuned model\", or any request that involves making a model available for inference on AWS. Use this even when the user is vague (e.g. \"I just want to get this running on AWS, you figure it out\"). Works for text-generation LLMs, embedding models, rerankers, classifiers, text-to-image \u002F diffusion models — picks the right serving stack and chooses between real-time and async inference. This is the entry-point skill for SageMaker deployment work — it asks clarifying questions, picks a deployment pathway, and coordinates the other deployment skills.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4947,4948,4949,4952],{"name":23,"slug":24,"type":15},{"name":4923,"slug":4924,"type":15},{"name":4950,"slug":4951,"type":15},"Deployment","deployment",{"name":9,"slug":4830,"type":15},"2026-07-08T05:55:37.387689",{"slug":4955,"name":4955,"fn":4956,"description":4957,"org":4958,"tags":4959,"stars":4913,"repoUrl":4914,"updatedAt":4968},"hf-cloud-sagemaker-iam-preflight","configure SageMaker IAM roles","Ensure a usable SageMaker execution role exists before deploying or training. Use this skill whenever about to create a SageMaker endpoint, model, training job, or any resource that requires an execution role. Use it especially when the user has not provided a role ARN explicitly, when scripts are about to call `iam:CreateRole`, or when an AccessDenied error mentions an IAM action. Never blindly call `iam:CreateRole` — always check for existing roles first. This skill prevents the most common SageMaker deployment failure: trying to create IAM resources from an SSO principal that has no IAM write permissions.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4960,4961,4962,4965],{"name":23,"slug":24,"type":15},{"name":4923,"slug":4924,"type":15},{"name":4963,"slug":4964,"type":15},"Permissions","permissions",{"name":4966,"slug":4967,"type":15},"Security","security","2026-07-08T05:55:34.948771",{"slug":4970,"name":4970,"fn":4971,"description":4972,"org":4973,"tags":4974,"stars":4913,"repoUrl":4914,"updatedAt":4982},"hf-cloud-sagemaker-production-defaults","create production-ready SageMaker endpoints","Create a SageMaker endpoint (real-time or async) with autoscaling, CloudWatch alarms, and tagging enabled by default. Use this skill whenever about to create a SageMaker endpoint, write deployment code that calls `create_endpoint`, or finalize a deployment after the image URI and IAM role are known. Provides deploy.py for real-time endpoints and deploy_async.py for async endpoints (with genuine scale-to-zero support). This is the last step in the SageMaker deployment workflow. Never generate a bare `create_endpoint` call without these defaults — endpoints without autoscaling or alarms are demos, not deployments.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4975,4976,4977,4978,4979],{"name":23,"slug":24,"type":15},{"name":4923,"slug":4924,"type":15},{"name":4950,"slug":4951,"type":15},{"name":9,"slug":4830,"type":15},{"name":4980,"slug":4981,"type":15},"Monitoring","monitoring","2026-07-08T05:55:38.664702",{"slug":4984,"name":4984,"fn":4985,"description":4986,"org":4987,"tags":4988,"stars":4913,"repoUrl":4914,"updatedAt":4992},"hf-cloud-serving-image-selection","select SageMaker serving containers","Pick the right serving container for a SageMaker model deployment and find its current image URI. Use this skill whenever about to deploy a model to a SageMaker endpoint and an image URI needs to be chosen — including when the user says \"deploy this LLM\", \"host this HuggingFace model\", \"serve this fine-tuned model\", \"deploy this embedding model\", \"host a reranker\", \"serve a sentence-transformers model\", or when about to hardcode any container URI in deployment code. HuggingFace-curated Deep Learning Containers are ALWAYS preferred: HuggingFace vLLM (LLMs and generative rerankers), HuggingFace vLLM-Omni (multimodal), TEI (embeddings\u002Fcross-encoder rerankers), HF Inference Toolkit (other transformers). Generic images (AWS vLLM, DJL-LMI, SGLang) are used only when no HuggingFace image is compatible — never merely because they carry a newer version. Never hardcode a container URI from memory and never default to TGI. Prevents stale-image failures and wrong-region URIs.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4989,4990,4991],{"name":23,"slug":24,"type":15},{"name":4923,"slug":4924,"type":15},{"name":4950,"slug":4951,"type":15},"2026-07-08T05:55:36.173465",{"slug":4994,"name":4994,"fn":4995,"description":4996,"org":4997,"tags":4998,"stars":4913,"repoUrl":4914,"updatedAt":5005},"hf-mcp","access Hugging Face Hub via MCP","Use Hugging Face Hub via MCP server tools. Search models, datasets, Spaces, papers. Get repo details, fetch documentation, run compute jobs, and use Gradio Spaces as AI tools. Available when connected to the HF MCP server.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[4999,5000,5001,5002],{"name":4909,"slug":4910,"type":15},{"name":9,"slug":4830,"type":15},{"name":4876,"slug":4877,"type":15},{"name":5003,"slug":5004,"type":15},"MCP","mcp","2026-04-06T18:25:50.364185",{"slug":5007,"name":5007,"fn":5008,"description":5009,"org":5010,"tags":5011,"stars":4913,"repoUrl":4914,"updatedAt":5017},"hf-mem","estimate memory for Hugging Face models","Hugging Face CLI to estimate the required memory to load Safetensors or GGUF model weights for inference from the Hugging Face Hub",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[5012,5013,5014,5015,5016],{"name":23,"slug":24,"type":15},{"name":4906,"slug":4907,"type":15},{"name":9,"slug":4830,"type":15},{"name":4876,"slug":4877,"type":15},{"name":13,"slug":14,"type":15},"2026-06-13T07:23:57.101435",{"slug":5019,"name":5019,"fn":5020,"description":5021,"org":5022,"tags":5023,"stars":4913,"repoUrl":4914,"updatedAt":5032},"huggingface-best","find and compare Hugging Face models","Use when the user asks about finding the best, top, or recommended model for a task, wants to know what AI model to use, or wants to compare models by benchmark scores. Triggers on: \"best model for X\", \"what model should I use for\", \"top models for [task]\", \"which model runs on my laptop\u002Fmachine\u002Fdevice\", \"recommend a model for\", \"what LLM should I use for\", \"compare models for\", \"what's state of the art for\", or any question about choosing an AI model for a specific use case. Always use this skill when the user wants model recommendations or comparisons, even if they don't explicitly mention HuggingFace or benchmarks.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[5024,5027,5028,5029],{"name":5025,"slug":5026,"type":15},"Analytics","analytics",{"name":9,"slug":4830,"type":15},{"name":4876,"slug":4877,"type":15},{"name":5030,"slug":5031,"type":15},"Research","research","2026-04-24T05:09:45.870658"]