[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-huggingface-huggingface-zerogpu":3,"mdc--sdvhnt-key":38,"related-repo-huggingface-huggingface-zerogpu":2859,"related-org-huggingface-huggingface-zerogpu":2956},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":27,"repoUrl":28,"updatedAt":29,"license":30,"forks":31,"topics":32,"repo":33,"sourceUrl":36,"mdContent":37},"huggingface-zerogpu","configure and deploy Hugging Face ZeroGPU spaces","AI demos and GPU compute with Gradio Spaces and Hugging Face Spaces ZeroGPU. Use when writing or reviewing code that uses `@spaces.GPU`, configuring `python_version` or `requirements.txt` for a ZeroGPU Space, or handling ZeroGPU-specific code constraints — pickle-based process isolation, `gr.State` semantics across the worker boundary, no `torch.compile` (use AoTI instead), CUDA wheel-only builds (no `nvcc` at build or runtime), large vs xlarge sizing, and dynamic duration callables. Make sure to use this skill whenever the user mentions ZeroGPU, `@spaces.GPU`, or the `spaces` Python package, or hits ZeroGPU-specific code errors like `PicklingError` across the worker boundary, `illegal duration`, or `flash-attn` wheel-build failures — even when the user does not explicitly ask for ZeroGPU coding guidance. Trigger on `import spaces` or `@spaces.GPU` in code.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"huggingface","Hugging Face","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fhuggingface.png",[12,16,18,21,24],{"name":13,"slug":14,"type":15},"Gradio","gradio","tag",{"name":9,"slug":17,"type":15},"hugging-face",{"name":19,"slug":20,"type":15},"Deployment","deployment",{"name":22,"slug":23,"type":15},"Python","python",{"name":25,"slug":26,"type":15},"AI Infrastructure","ai-infrastructure",10861,"https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fskills","2026-05-21T06:58:55.776031",null,721,[],{"repoUrl":28,"stars":27,"forks":31,"topics":34,"description":35},[],"Give your agents the power of the Hugging Face ecosystem","https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fskills\u002Ftree\u002FHEAD\u002Fskills\u002Fhuggingface-zerogpu","---\nname: huggingface-zerogpu\ndescription: AI demos and GPU compute with Gradio Spaces and Hugging Face Spaces ZeroGPU. Use when writing or reviewing code that uses `@spaces.GPU`, configuring `python_version` or `requirements.txt` for a ZeroGPU Space, or handling ZeroGPU-specific code constraints — pickle-based process isolation, `gr.State` semantics across the worker boundary, no `torch.compile` (use AoTI instead), CUDA wheel-only builds (no `nvcc` at build or runtime), large vs xlarge sizing, and dynamic duration callables. Make sure to use this skill whenever the user mentions ZeroGPU, `@spaces.GPU`, or the `spaces` Python package, or hits ZeroGPU-specific code errors like `PicklingError` across the worker boundary, `illegal duration`, or `flash-attn` wheel-build failures — even when the user does not explicitly ask for ZeroGPU coding guidance. Trigger on `import spaces` or `@spaces.GPU` in code.\n---\n\n# Hugging Face ZeroGPU\n\nRules and patterns for ML demos on Hugging Face Spaces with **ZeroGPU** hardware. Covers `@spaces.GPU`, duration and quota tuning, process isolation, the CUDA availability model, concurrency safety, and CUDA build constraints.\n\n## Scope\n\nThis skill is for **Gradio SDK Spaces using ZeroGPU hardware**. Docker and Static Spaces cannot schedule onto ZeroGPU, and Streamlit apps now run as Docker Spaces — so this skill applies only to Gradio. For general Gradio coding (components, layouts, event listeners), see the `huggingface-gradio` skill in this repo. The authoritative ZeroGPU docs live at https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Fhub\u002Fspaces-zerogpu — refer to them for the current backing GPU, runtime version lists, and tier thresholds, all of which change over time.\n\n## Reference Files\n\n| Reference | When to read |\n|-----------|--------------|\n| `references\u002Fconcurrency.md` | Always read alongside SKILL.md when writing ZeroGPU code — handlers run in parallel by default |\n| `references\u002Fhow-zerogpu-works.md` | When reasoning about cold-starts, worker reuse, why module-scope warmup does not carry to requests, or why returning CUDA tensors hangs |\n| `references\u002Fhow-quota-works.md` | When choosing `duration` values, debugging `illegal duration` vs `quota exceeded` errors, or explaining why default 60s blocks short tasks |\n| `references\u002Fcuda-and-deps.md` | When installing CUDA-dependent packages (e.g. `flash-attn`), pinning torch side-cars, or reading wheel filename tags |\n\n## Hardware\n\nZeroGPU exposes two GPU sizes that map to a fraction of the backing card:\n\n| `size` | Slice of backing GPU | Quota cost |\n|--------|----------------------|------------|\n| `large` *(default)* | Half | 1x |\n| `xlarge` | Full | 2x |\n\nDefault `large` gives half a physical GPU, so memory bandwidth and compute are significantly lower than the full card's specs. Use `xlarge` only when the workload genuinely needs the extra memory or compute.\n\n> **Backing GPU changes without notice.** ZeroGPU has already migrated across GPU generations several times; older write-ups may name A100 or H200, but those are outdated. For the current backing GPU and exact per-size VRAM, always check the [ZeroGPU docs](https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Fhub\u002Fspaces-zerogpu) before sizing workloads.\n\n## Basic Pattern\n\n```python\nimport spaces\nimport torch\nfrom transformers import pipeline\n\npipe = pipeline(\"text-generation\", model=\"...\", device=\"cuda\")\n\n@spaces.GPU\ndef generate(prompt: str) -> str:\n    return pipe(prompt, max_new_tokens=100)[0][\"generated_text\"]\n```\n\nKey rules:\n\n1. **Instantiate models at module scope** and call `.to(\"cuda\")` eagerly. ZeroGPU handles the actual device mapping transparently (see CUDA availability model below).\n2. **Decorate GPU functions with `@spaces.GPU`**. The decorator is a no-op outside ZeroGPU, so it is safe to keep in all environments.\n3. **Set `duration` to match the realistic worst-case workload** (default 60s). The platform pre-checks `requested duration` against the user's `remaining quota` — not against the actual run time — so a 10-second task left at the 60s default fails with `quota exceeded` as soon as the user's remaining quota drops below 60s. Smaller declared `duration` also ranks higher in the node-level queue. See \"Duration and Quota\" below.\n4. **`torch.compile` is NOT supported.** Use PyTorch [ahead-of-time compilation (AoTI)](https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fzerogpu-aoti) (torch 2.8+) instead.\n5. **Use `size=\"xlarge\"` sparingly.** It allocates the full backing GPU, but costs 2x quota and tends to queue longer.\n\n```python\n@spaces.GPU(duration=120)\ndef generate_image(prompt: str):\n    return pipe(prompt).images[0]\n```\n\n## CUDA Availability Model\n\nReal GPU access is **only** available inside `@spaces.GPU`-decorated functions. Outside those functions, the GPU is not attached to the process.\n\nHowever, `import spaces` **monkey-patches `torch`** so that:\n\n- `torch.cuda.is_available()` returns `True` globally.\n- `.to(\"cuda\")` \u002F `device=\"cuda\"` calls at module scope succeed without error.\n\nThis is intentional. Module-scope `model.to(\"cuda\")` calls register tensors with the ZeroGPU backend, which writes them to a disk offload directory at a startup \"pack\" step and frees the corresponding RAM. When a `@spaces.GPU` call lands, a forked GPU worker process streams those weights from disk into VRAM via a pinned-memory pipeline. Warm workers (reused across requests on the same GPU slot) keep weights resident on the GPU and skip the disk → VRAM step. The user-facing rule: write `device=\"cuda\"` at module scope and it works — see `references\u002Fhow-zerogpu-works.md` for the full lifecycle.\n\n| Action | Where | Why |\n|--------|-------|-----|\n| `model.to(\"cuda\")` \u002F `pipe(..., device=\"cuda\")` | **Module scope** | ZeroGPU registers the tensor and manages device migration |\n| Actual CUDA computation (inference, etc.) | **Inside `@spaces.GPU`** | Real GPU is only attached during the decorated call |\n| Branching on `torch.cuda.is_available()` | Avoid relying on it | Always returns `True` due to the monkey-patch |\n\nDo not run inference or CUDA kernels at module scope — the real GPU is not attached, so operations either silently run on CPU or fail.\n\n### Device selection idiom still works\n\nThe standard idiom remains correct under ZeroGPU:\n\n```python\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nmodel = AutoModel.from_pretrained(\"...\").to(device)\n```\n\n- **ZeroGPU** — `is_available()` is `True` (monkey-patched), so the model is registered for automatic device migration.\n- **Dedicated GPU Spaces \u002F local GPU** — `is_available()` is genuinely `True`.\n- **CPU Spaces \u002F local CPU** — resolves to `\"cpu\"`.\n\nDo not hardcode `device=\"cuda\"` — it breaks on CPU-only environments.\n\n### Eager loading is the right default\n\nLoad models at module scope, not lazily on first request. The Space process starts before any user arrives, so cold-start cost is paid once. Lazy loading (`global model; if model is None: ...`, `@lru_cache` wrappers, factory functions instantiating on first call) just pushes that cost onto the first user.\n\n## Local Development: Just Install `spaces`\n\nDo **not** wrap `import spaces` in `try\u002Fexcept` and redefine `spaces.GPU` as a no-op fallback for local runs. Off-ZeroGPU, the `spaces` package is already a true no-op:\n\n- Heavyweight behavior (CUDA monkey-patching, client init, startup hooks) is gated on the `SPACES_ZERO_GPU` env var, set only on ZeroGPU.\n- `@spaces.GPU` returns the undecorated function unchanged off-ZeroGPU.\n- Top-level `import spaces` performs only lightweight imports.\n\nThe Gradio SDK base image installs `spaces` on every hardware tier. So even after duplicating a Space onto a dedicated GPU (T4, L4, A10G, etc.) or CPU basic, no code changes are needed — `import spaces` still succeeds and `@spaces.GPU` becomes a transparent passthrough.\n\n### Anti-pattern\n\n```python\ntry:\n    import spaces\nexcept ImportError:\n    class spaces:  # type: ignore\n        @staticmethod\n        def GPU(func=None, **kwargs):\n            return func if func else (lambda f: f)\n```\n\nProblems:\n\n1. The fallback must mimic every `@spaces.GPU` call shape — bare decorator, `duration=...`, `size=...`, generators, `aoti_*` helpers — and drifts as the `spaces` API grows.\n2. It hides `spaces` from `requirements.txt`, even though the Space needs it at deploy time.\n3. It solves a non-problem: the real package is already a no-op locally.\n\n### Do this instead\n\nAdd `spaces` to dependencies and import it unconditionally:\n\n```python\nimport spaces\n\n@spaces.GPU\ndef generate(prompt: str) -> str:\n    ...\n```\n\n## Duration and Quota\n\nThree things happen when you declare `@spaces.GPU(duration=N)`:\n\n1. **Tier-max check** — each visitor tier has a per-call `duration` cap. Declaring `duration` larger than the cap fails immediately with `ZeroGPU illegal duration`, regardless of remaining quota. (Tier numbers change over time — see the [ZeroGPU docs](https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Fhub\u002Fspaces-zerogpu).)\n2. **Quota pre-check** — the platform compares `requested duration` against the user's `remaining quota`. If `remaining \u003C requested`, the call fails with `ZeroGPU quota exceeded` — even if the actual work would have fit. The error message shows the explicit numbers, e.g. `\"60s requested vs. 30s left\"`. A 10-second task left at the default 60s therefore blocks the user once their remaining quota drops below 60s.\n3. **Queue priority** — the queue is node-level (requests from all Spaces on the same node compete for GPU slots), and shorter declared `duration` ranks higher.\n\nAll three favor declaring the smallest realistic `duration` — including for short tasks. Explicit `@spaces.GPU(duration=15)` on a 10-second task avoids premature `quota exceeded` rejections and ranks higher in the queue.\n\n> **`xlarge` doubles the request.** `requested = N * 2` when `size=\"xlarge\"`, both for the tier-max check and the quota pre-check. So `@spaces.GPU(duration=60, size=\"xlarge\")` is internally a 120s request.\n\n### Dynamic duration for variable workloads\n\nFor workloads whose runtime depends on inputs, pass a callable that estimates per request. A static high `duration` locks out low-tier users (whose tier cap may be smaller than the static value) and unnecessarily reserves quota for light inputs.\n\n```python\ndef estimate_duration(prompt, steps):\n    return int(steps * 3.5)\n\n@spaces.GPU(duration=estimate_duration)\ndef generate(prompt, steps):\n    return pipe(prompt, num_inference_steps=steps).images[0]\n```\n\nFor the full distinction between `illegal duration` vs `quota exceeded`, runs-per-day limits, the 24h quota window, and pay-as-you-go billing, see `references\u002Fhow-quota-works.md`.\n\n## Process Isolation and Pickle\n\n`@spaces.GPU`-decorated functions run in a **separate process** managed by the ZeroGPU scheduler. Arguments and return values cross the process boundary via **pickle serialization**.\n\nConsequences:\n\n- **Only picklable objects** can be passed in or returned. Open file handles, database connections, locks, lambdas, and closures over unpicklable state will raise `PicklingError`.\n- **Do NOT return CUDA tensors directly.** Unpickling a CUDA tensor in the main process triggers `torch.cuda._lazy_init()`, which ZeroGPU blocks. Convert to CPU first: return `tensor.cpu()` or `tensor.cpu().numpy()`.\n- CPU tensors, numpy arrays, PIL Images, and plain Python objects work fine.\n- Large objects incur serialization overhead. Prefer lightweight returns (tensors, arrays, file paths, base64 strings) over complex object graphs.\n\n### `gr.State` semantics across the boundary\n\nBecause handlers run in a separate process, `gr.State` values are **pickled on every yield** — they are NOT shared by reference.\n\n- The generator receives a **copy** of the state (`id()` differs from the caller's).\n- In-place mutations inside the generator are **invisible** to other handlers until the mutated state is explicitly yielded back.\n- Yielding `gr.update()` for a `gr.State` slot **skips the update** — other handlers continue to see the pre-yield value.\n- Each yield that returns the state object creates a **new copy** via pickle.\n\nPractical guidance:\n\n- **Do NOT assume reference semantics for `gr.State`** on ZeroGPU. Code that mutates state in a generator and expects another handler to see those mutations will silently use stale data.\n- **Every yield including a `gr.State` value triggers a full pickle round-trip.** For large state (model sessions, frame buffers), minimize how often you yield it — ideally once at the end. Use `gr.update()` for the state slot on intermediate yields.\n- **CUDA tensors inside state must be moved to CPU before yielding** — same `torch.cuda._lazy_init()` issue as above.\n\n## Concurrency\n\nHandlers run **concurrently by default** on ZeroGPU. This is not opt-in. Code that worked in single-user testing can silently corrupt or leak data in production.\n\nThree rules. Full treatment with examples in `references\u002Fconcurrency.md`.\n\n1. **No mutable global state.** Concurrent requests overwrite each other.\n2. **No fixed file paths for outputs.** Concurrent requests clobber the same file. Use `tempfile` for unique paths.\n3. **Read-only globals are safe.** Model objects, tokenizers, configs loaded once at startup and only read during requests are safe and encouraged.\n\n## Call Granularity\n\nEach entry into a `@spaces.GPU` function carries non-trivial cost — pickle round-trip across the process boundary, worker warm-up, CUDA re-attach, and a fresh pass through the node-level queue. Calling a decorated function from inside a hot loop multiplies these costs and adds a new failure mode: a later iteration may fail to acquire a GPU slot, stalling the whole job mid-way.\n\nDecorate the outer function that owns the loop, not the per-iteration worker:\n\n```python\n# Avoid — N GPU entries for N frames\ndef process_video(frames):\n    return [process_frame(f) for f in frames]\n\n@spaces.GPU(duration=...)\ndef process_frame(frame):\n    ...\n\n# Prefer — one GPU entry for the whole video\n@spaces.GPU(duration=...)\ndef process_video(frames):\n    return [process_frame(f) for f in frames]\n\ndef process_frame(frame):\n    ...\n```\n\nIf the loop mixes heavy CPU work with GPU work, wrapping the whole loop charges that CPU time against the user's quota. When that cost is material, batching the GPU work so CPU pre\u002Fpost-processing stays outside the decorator is a situational optimization — not the default.\n\n## CUDA Build Constraints\n\nHF Spaces builds Docker images in a CPU-only environment. **On ZeroGPU, the build phase has no `nvcc`** because the base image is `python:3.13` (dedicated-GPU Spaces use `nvidia\u002Fcuda:*-devel-*` and have `nvcc` at build time). A CUDA-dependent package whose only distribution is sdist — e.g. bare `flash-attn` — therefore cannot be installed via `requirements.txt` on ZeroGPU. Only pre-built wheels work.\n\nZeroGPU **runtime** does have `nvcc` available, mounted from a CUDA devel image at `\u002Fcuda-image` since 2025-07 (originally added for AoTI support). This is what makes `torch.export` \u002F AoTI workflows possible inside `@spaces.GPU` calls.\n\n**Bottom line**: install every CUDA-dependent package from a pre-built wheel. If no wheel is available on PyPI, build one externally (e.g. host on HF Hub) and pin the URL. For `flash-attn`, the upstream releases page ships a fairly complete wheel matrix covering most Python × CUDA × torch combinations.\n\nFor wheel-tag reading (cxx11 ABI, `cu12torch2.X`, `cp3XX`), torch-family side-car drift, and the kernels-community fallback, see `references\u002Fcuda-and-deps.md`.\n\n## Example Caching\n\n`gr.Examples` behavior is environment-dependent. On ZeroGPU specifically:\n\n- `cache_examples` defaults to `True` (Spaces sets `GRADIO_CACHE_EXAMPLES=true`).\n- `cache_mode` defaults to `\"lazy\"` (Spaces sets `GRADIO_CACHE_MODE=lazy` only on ZeroGPU).\n\nZeroGPU defaults to `lazy` because eager caching pre-runs every example at app startup, but ZeroGPU has **no GPU attached at startup** — only during request handling. Eager caching of GPU-bound examples would fail there.\n\nWhen `cache_examples=True`, the `run_on_click` \u002F `run_examples_on_click` parameter is silently ignored. If your app relies on click-populates-only behavior, set `cache_examples=False` explicitly to preserve it.\n\nTo reproduce ZeroGPU example-caching behavior locally:\n\n```bash\nGRADIO_CACHE_EXAMPLES=true GRADIO_CACHE_MODE=lazy python app.py\n```\n\n## Dependency Management\n\n### `python_version` pin in README frontmatter\n\nPinning `python_version` is **effectively required** for ZeroGPU. The runtime default is currently Python 3.10, so a local environment using 3.11+ will fail to install on the Space without an explicit pin. Pin to a ZeroGPU-supported version (3.12 is a reasonable default); the authoritative supported list lives in the [ZeroGPU docs](https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Fhub\u002Fspaces-zerogpu) — do not hardcode the full list, refer to the docs.\n\n```yaml\n# README.md frontmatter\npython_version: \"3.12\"\n```\n\nBoth `\"3.12\"` and `\"3.12.12\"` forms are accepted.\n\n### Do not pin `spaces` in `requirements.txt`\n\nThe Space platform pins its own `spaces` version. A conflicting pin in `requirements.txt` causes pip resolution to fail at build time.\n\n> **Rule**: Do not include `spaces` in `requirements.txt`.\n\nHow to achieve this depends on your tooling:\n\n- **Hand-written `requirements.txt`**: simply omit `spaces`.\n- **uv** (`pyproject.toml`-managed): declare `spaces` in `pyproject.toml` so uv co-resolves transitive constraints (notably `psutil`, which `spaces` pins), then exclude it from the export:\n  ```bash\n  uv export --no-hashes --no-dev --no-emit-package spaces -o requirements.txt\n  ```\n  Without `spaces` in `pyproject.toml`, uv cannot see its transitive constraints and may resolve incompatible versions at build time.\n- **pip-tools** (`pip-compile`) \u002F **Poetry**: use the equivalent exclude mechanism.\n\n### Pin `torch` to match wheel tags\n\nIf you install a CUDA-dependent wheel via direct URL, the wheel filename encodes the `torch` major.minor it was built against (e.g. `cu12torch2.8`). Pin `torch==X.Y.Z` in `requirements.txt` to match — otherwise pip may resolve `torch` to a different version and the Space fails on first import. Details and the kernels-community alternative are in `references\u002Fcuda-and-deps.md`.\n",{"data":39,"body":40},{"name":4,"description":6},{"type":41,"children":42},"root",[43,52,75,82,112,118,246,252,257,340,359,381,387,480,485,613,644,650,669,694,735,769,883,888,895,900,923,990,1002,1008,1029,1041,1083,1121,1147,1153,1216,1221,1291,1297,1309,1352,1358,1371,1477,1504,1544,1550,1562,1616,1640,1646,1669,1674,1737,1749,1768,1842,1847,1906,1912,1924,1935,1976,1982,1994,1999,2123,2128,2134,2189,2231,2248,2274,2280,2291,2345,2365,2401,2406,2460,2466,2478,2503,2548,2569,2586,2605,2630,2635,2793,2806,2853],{"type":44,"tag":45,"props":46,"children":48},"element","h1",{"id":47},"hugging-face-zerogpu",[49],{"type":50,"value":51},"text","Hugging Face ZeroGPU",{"type":44,"tag":53,"props":54,"children":55},"p",{},[56,58,64,66,73],{"type":50,"value":57},"Rules and patterns for ML demos on Hugging Face Spaces with ",{"type":44,"tag":59,"props":60,"children":61},"strong",{},[62],{"type":50,"value":63},"ZeroGPU",{"type":50,"value":65}," hardware. Covers ",{"type":44,"tag":67,"props":68,"children":70},"code",{"className":69},[],[71],{"type":50,"value":72},"@spaces.GPU",{"type":50,"value":74},", duration and quota tuning, process isolation, the CUDA availability model, concurrency safety, and CUDA build constraints.",{"type":44,"tag":76,"props":77,"children":79},"h2",{"id":78},"scope",[80],{"type":50,"value":81},"Scope",{"type":44,"tag":53,"props":83,"children":84},{},[85,87,92,94,100,102,110],{"type":50,"value":86},"This skill is for ",{"type":44,"tag":59,"props":88,"children":89},{},[90],{"type":50,"value":91},"Gradio SDK Spaces using ZeroGPU hardware",{"type":50,"value":93},". Docker and Static Spaces cannot schedule onto ZeroGPU, and Streamlit apps now run as Docker Spaces — so this skill applies only to Gradio. For general Gradio coding (components, layouts, event listeners), see the ",{"type":44,"tag":67,"props":95,"children":97},{"className":96},[],[98],{"type":50,"value":99},"huggingface-gradio",{"type":50,"value":101}," skill in this repo. The authoritative ZeroGPU docs live at ",{"type":44,"tag":103,"props":104,"children":108},"a",{"href":105,"rel":106},"https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Fhub\u002Fspaces-zerogpu",[107],"nofollow",[109],{"type":50,"value":105},{"type":50,"value":111}," — refer to them for the current backing GPU, runtime version lists, and tier thresholds, all of which change over time.",{"type":44,"tag":76,"props":113,"children":115},{"id":114},"reference-files",[116],{"type":50,"value":117},"Reference Files",{"type":44,"tag":119,"props":120,"children":121},"table",{},[122,141],{"type":44,"tag":123,"props":124,"children":125},"thead",{},[126],{"type":44,"tag":127,"props":128,"children":129},"tr",{},[130,136],{"type":44,"tag":131,"props":132,"children":133},"th",{},[134],{"type":50,"value":135},"Reference",{"type":44,"tag":131,"props":137,"children":138},{},[139],{"type":50,"value":140},"When to read",{"type":44,"tag":142,"props":143,"children":144},"tbody",{},[145,163,180,221],{"type":44,"tag":127,"props":146,"children":147},{},[148,158],{"type":44,"tag":149,"props":150,"children":151},"td",{},[152],{"type":44,"tag":67,"props":153,"children":155},{"className":154},[],[156],{"type":50,"value":157},"references\u002Fconcurrency.md",{"type":44,"tag":149,"props":159,"children":160},{},[161],{"type":50,"value":162},"Always read alongside SKILL.md when writing ZeroGPU code — handlers run in parallel by default",{"type":44,"tag":127,"props":164,"children":165},{},[166,175],{"type":44,"tag":149,"props":167,"children":168},{},[169],{"type":44,"tag":67,"props":170,"children":172},{"className":171},[],[173],{"type":50,"value":174},"references\u002Fhow-zerogpu-works.md",{"type":44,"tag":149,"props":176,"children":177},{},[178],{"type":50,"value":179},"When reasoning about cold-starts, worker reuse, why module-scope warmup does not carry to requests, or why returning CUDA tensors hangs",{"type":44,"tag":127,"props":181,"children":182},{},[183,192],{"type":44,"tag":149,"props":184,"children":185},{},[186],{"type":44,"tag":67,"props":187,"children":189},{"className":188},[],[190],{"type":50,"value":191},"references\u002Fhow-quota-works.md",{"type":44,"tag":149,"props":193,"children":194},{},[195,197,203,205,211,213,219],{"type":50,"value":196},"When choosing ",{"type":44,"tag":67,"props":198,"children":200},{"className":199},[],[201],{"type":50,"value":202},"duration",{"type":50,"value":204}," values, debugging ",{"type":44,"tag":67,"props":206,"children":208},{"className":207},[],[209],{"type":50,"value":210},"illegal duration",{"type":50,"value":212}," vs ",{"type":44,"tag":67,"props":214,"children":216},{"className":215},[],[217],{"type":50,"value":218},"quota exceeded",{"type":50,"value":220}," errors, or explaining why default 60s blocks short tasks",{"type":44,"tag":127,"props":222,"children":223},{},[224,233],{"type":44,"tag":149,"props":225,"children":226},{},[227],{"type":44,"tag":67,"props":228,"children":230},{"className":229},[],[231],{"type":50,"value":232},"references\u002Fcuda-and-deps.md",{"type":44,"tag":149,"props":234,"children":235},{},[236,238,244],{"type":50,"value":237},"When installing CUDA-dependent packages (e.g. ",{"type":44,"tag":67,"props":239,"children":241},{"className":240},[],[242],{"type":50,"value":243},"flash-attn",{"type":50,"value":245},"), pinning torch side-cars, or reading wheel filename tags",{"type":44,"tag":76,"props":247,"children":249},{"id":248},"hardware",[250],{"type":50,"value":251},"Hardware",{"type":44,"tag":53,"props":253,"children":254},{},[255],{"type":50,"value":256},"ZeroGPU exposes two GPU sizes that map to a fraction of the backing card:",{"type":44,"tag":119,"props":258,"children":259},{},[260,285],{"type":44,"tag":123,"props":261,"children":262},{},[263],{"type":44,"tag":127,"props":264,"children":265},{},[266,275,280],{"type":44,"tag":131,"props":267,"children":268},{},[269],{"type":44,"tag":67,"props":270,"children":272},{"className":271},[],[273],{"type":50,"value":274},"size",{"type":44,"tag":131,"props":276,"children":277},{},[278],{"type":50,"value":279},"Slice of backing GPU",{"type":44,"tag":131,"props":281,"children":282},{},[283],{"type":50,"value":284},"Quota cost",{"type":44,"tag":142,"props":286,"children":287},{},[288,318],{"type":44,"tag":127,"props":289,"children":290},{},[291,308,313],{"type":44,"tag":149,"props":292,"children":293},{},[294,300,302],{"type":44,"tag":67,"props":295,"children":297},{"className":296},[],[298],{"type":50,"value":299},"large",{"type":50,"value":301}," ",{"type":44,"tag":303,"props":304,"children":305},"em",{},[306],{"type":50,"value":307},"(default)",{"type":44,"tag":149,"props":309,"children":310},{},[311],{"type":50,"value":312},"Half",{"type":44,"tag":149,"props":314,"children":315},{},[316],{"type":50,"value":317},"1x",{"type":44,"tag":127,"props":319,"children":320},{},[321,330,335],{"type":44,"tag":149,"props":322,"children":323},{},[324],{"type":44,"tag":67,"props":325,"children":327},{"className":326},[],[328],{"type":50,"value":329},"xlarge",{"type":44,"tag":149,"props":331,"children":332},{},[333],{"type":50,"value":334},"Full",{"type":44,"tag":149,"props":336,"children":337},{},[338],{"type":50,"value":339},"2x",{"type":44,"tag":53,"props":341,"children":342},{},[343,345,350,352,357],{"type":50,"value":344},"Default ",{"type":44,"tag":67,"props":346,"children":348},{"className":347},[],[349],{"type":50,"value":299},{"type":50,"value":351}," gives half a physical GPU, so memory bandwidth and compute are significantly lower than the full card's specs. Use ",{"type":44,"tag":67,"props":353,"children":355},{"className":354},[],[356],{"type":50,"value":329},{"type":50,"value":358}," only when the workload genuinely needs the extra memory or compute.",{"type":44,"tag":360,"props":361,"children":362},"blockquote",{},[363],{"type":44,"tag":53,"props":364,"children":365},{},[366,371,373,379],{"type":44,"tag":59,"props":367,"children":368},{},[369],{"type":50,"value":370},"Backing GPU changes without notice.",{"type":50,"value":372}," ZeroGPU has already migrated across GPU generations several times; older write-ups may name A100 or H200, but those are outdated. For the current backing GPU and exact per-size VRAM, always check the ",{"type":44,"tag":103,"props":374,"children":376},{"href":105,"rel":375},[107],[377],{"type":50,"value":378},"ZeroGPU docs",{"type":50,"value":380}," before sizing workloads.",{"type":44,"tag":76,"props":382,"children":384},{"id":383},"basic-pattern",[385],{"type":50,"value":386},"Basic Pattern",{"type":44,"tag":388,"props":389,"children":393},"pre",{"className":390,"code":391,"language":23,"meta":392,"style":392},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","import spaces\nimport torch\nfrom transformers import pipeline\n\npipe = pipeline(\"text-generation\", model=\"...\", device=\"cuda\")\n\n@spaces.GPU\ndef generate(prompt: str) -> str:\n    return pipe(prompt, max_new_tokens=100)[0][\"generated_text\"]\n","",[394],{"type":44,"tag":67,"props":395,"children":396},{"__ignoreMap":392},[397,408,417,426,436,445,453,462,471],{"type":44,"tag":398,"props":399,"children":402},"span",{"class":400,"line":401},"line",1,[403],{"type":44,"tag":398,"props":404,"children":405},{},[406],{"type":50,"value":407},"import spaces\n",{"type":44,"tag":398,"props":409,"children":411},{"class":400,"line":410},2,[412],{"type":44,"tag":398,"props":413,"children":414},{},[415],{"type":50,"value":416},"import torch\n",{"type":44,"tag":398,"props":418,"children":420},{"class":400,"line":419},3,[421],{"type":44,"tag":398,"props":422,"children":423},{},[424],{"type":50,"value":425},"from transformers import pipeline\n",{"type":44,"tag":398,"props":427,"children":429},{"class":400,"line":428},4,[430],{"type":44,"tag":398,"props":431,"children":433},{"emptyLinePlaceholder":432},true,[434],{"type":50,"value":435},"\n",{"type":44,"tag":398,"props":437,"children":439},{"class":400,"line":438},5,[440],{"type":44,"tag":398,"props":441,"children":442},{},[443],{"type":50,"value":444},"pipe = pipeline(\"text-generation\", model=\"...\", device=\"cuda\")\n",{"type":44,"tag":398,"props":446,"children":448},{"class":400,"line":447},6,[449],{"type":44,"tag":398,"props":450,"children":451},{"emptyLinePlaceholder":432},[452],{"type":50,"value":435},{"type":44,"tag":398,"props":454,"children":456},{"class":400,"line":455},7,[457],{"type":44,"tag":398,"props":458,"children":459},{},[460],{"type":50,"value":461},"@spaces.GPU\n",{"type":44,"tag":398,"props":463,"children":465},{"class":400,"line":464},8,[466],{"type":44,"tag":398,"props":467,"children":468},{},[469],{"type":50,"value":470},"def generate(prompt: str) -> str:\n",{"type":44,"tag":398,"props":472,"children":474},{"class":400,"line":473},9,[475],{"type":44,"tag":398,"props":476,"children":477},{},[478],{"type":50,"value":479},"    return pipe(prompt, max_new_tokens=100)[0][\"generated_text\"]\n",{"type":44,"tag":53,"props":481,"children":482},{},[483],{"type":50,"value":484},"Key rules:",{"type":44,"tag":486,"props":487,"children":488},"ol",{},[489,508,523,570,595],{"type":44,"tag":490,"props":491,"children":492},"li",{},[493,498,500,506],{"type":44,"tag":59,"props":494,"children":495},{},[496],{"type":50,"value":497},"Instantiate models at module scope",{"type":50,"value":499}," and call ",{"type":44,"tag":67,"props":501,"children":503},{"className":502},[],[504],{"type":50,"value":505},".to(\"cuda\")",{"type":50,"value":507}," eagerly. ZeroGPU handles the actual device mapping transparently (see CUDA availability model below).",{"type":44,"tag":490,"props":509,"children":510},{},[511,521],{"type":44,"tag":59,"props":512,"children":513},{},[514,516],{"type":50,"value":515},"Decorate GPU functions with ",{"type":44,"tag":67,"props":517,"children":519},{"className":518},[],[520],{"type":50,"value":72},{"type":50,"value":522},". The decorator is a no-op outside ZeroGPU, so it is safe to keep in all environments.",{"type":44,"tag":490,"props":524,"children":525},{},[526,538,540,546,548,554,556,561,563,568],{"type":44,"tag":59,"props":527,"children":528},{},[529,531,536],{"type":50,"value":530},"Set ",{"type":44,"tag":67,"props":532,"children":534},{"className":533},[],[535],{"type":50,"value":202},{"type":50,"value":537}," to match the realistic worst-case workload",{"type":50,"value":539}," (default 60s). The platform pre-checks ",{"type":44,"tag":67,"props":541,"children":543},{"className":542},[],[544],{"type":50,"value":545},"requested duration",{"type":50,"value":547}," against the user's ",{"type":44,"tag":67,"props":549,"children":551},{"className":550},[],[552],{"type":50,"value":553},"remaining quota",{"type":50,"value":555}," — not against the actual run time — so a 10-second task left at the 60s default fails with ",{"type":44,"tag":67,"props":557,"children":559},{"className":558},[],[560],{"type":50,"value":218},{"type":50,"value":562}," as soon as the user's remaining quota drops below 60s. Smaller declared ",{"type":44,"tag":67,"props":564,"children":566},{"className":565},[],[567],{"type":50,"value":202},{"type":50,"value":569}," also ranks higher in the node-level queue. See \"Duration and Quota\" below.",{"type":44,"tag":490,"props":571,"children":572},{},[573,584,586,593],{"type":44,"tag":59,"props":574,"children":575},{},[576,582],{"type":44,"tag":67,"props":577,"children":579},{"className":578},[],[580],{"type":50,"value":581},"torch.compile",{"type":50,"value":583}," is NOT supported.",{"type":50,"value":585}," Use PyTorch ",{"type":44,"tag":103,"props":587,"children":590},{"href":588,"rel":589},"https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fzerogpu-aoti",[107],[591],{"type":50,"value":592},"ahead-of-time compilation (AoTI)",{"type":50,"value":594}," (torch 2.8+) instead.",{"type":44,"tag":490,"props":596,"children":597},{},[598,611],{"type":44,"tag":59,"props":599,"children":600},{},[601,603,609],{"type":50,"value":602},"Use ",{"type":44,"tag":67,"props":604,"children":606},{"className":605},[],[607],{"type":50,"value":608},"size=\"xlarge\"",{"type":50,"value":610}," sparingly.",{"type":50,"value":612}," It allocates the full backing GPU, but costs 2x quota and tends to queue longer.",{"type":44,"tag":388,"props":614,"children":616},{"className":390,"code":615,"language":23,"meta":392,"style":392},"@spaces.GPU(duration=120)\ndef generate_image(prompt: str):\n    return pipe(prompt).images[0]\n",[617],{"type":44,"tag":67,"props":618,"children":619},{"__ignoreMap":392},[620,628,636],{"type":44,"tag":398,"props":621,"children":622},{"class":400,"line":401},[623],{"type":44,"tag":398,"props":624,"children":625},{},[626],{"type":50,"value":627},"@spaces.GPU(duration=120)\n",{"type":44,"tag":398,"props":629,"children":630},{"class":400,"line":410},[631],{"type":44,"tag":398,"props":632,"children":633},{},[634],{"type":50,"value":635},"def generate_image(prompt: str):\n",{"type":44,"tag":398,"props":637,"children":638},{"class":400,"line":419},[639],{"type":44,"tag":398,"props":640,"children":641},{},[642],{"type":50,"value":643},"    return pipe(prompt).images[0]\n",{"type":44,"tag":76,"props":645,"children":647},{"id":646},"cuda-availability-model",[648],{"type":50,"value":649},"CUDA Availability Model",{"type":44,"tag":53,"props":651,"children":652},{},[653,655,660,662,667],{"type":50,"value":654},"Real GPU access is ",{"type":44,"tag":59,"props":656,"children":657},{},[658],{"type":50,"value":659},"only",{"type":50,"value":661}," available inside ",{"type":44,"tag":67,"props":663,"children":665},{"className":664},[],[666],{"type":50,"value":72},{"type":50,"value":668},"-decorated functions. Outside those functions, the GPU is not attached to the process.",{"type":44,"tag":53,"props":670,"children":671},{},[672,674,680,681,692],{"type":50,"value":673},"However, ",{"type":44,"tag":67,"props":675,"children":677},{"className":676},[],[678],{"type":50,"value":679},"import spaces",{"type":50,"value":301},{"type":44,"tag":59,"props":682,"children":683},{},[684,686],{"type":50,"value":685},"monkey-patches ",{"type":44,"tag":67,"props":687,"children":689},{"className":688},[],[690],{"type":50,"value":691},"torch",{"type":50,"value":693}," so that:",{"type":44,"tag":695,"props":696,"children":697},"ul",{},[698,717],{"type":44,"tag":490,"props":699,"children":700},{},[701,707,709,715],{"type":44,"tag":67,"props":702,"children":704},{"className":703},[],[705],{"type":50,"value":706},"torch.cuda.is_available()",{"type":50,"value":708}," returns ",{"type":44,"tag":67,"props":710,"children":712},{"className":711},[],[713],{"type":50,"value":714},"True",{"type":50,"value":716}," globally.",{"type":44,"tag":490,"props":718,"children":719},{},[720,725,727,733],{"type":44,"tag":67,"props":721,"children":723},{"className":722},[],[724],{"type":50,"value":505},{"type":50,"value":726}," \u002F ",{"type":44,"tag":67,"props":728,"children":730},{"className":729},[],[731],{"type":50,"value":732},"device=\"cuda\"",{"type":50,"value":734}," calls at module scope succeed without error.",{"type":44,"tag":53,"props":736,"children":737},{},[738,740,746,748,753,755,760,762,767],{"type":50,"value":739},"This is intentional. Module-scope ",{"type":44,"tag":67,"props":741,"children":743},{"className":742},[],[744],{"type":50,"value":745},"model.to(\"cuda\")",{"type":50,"value":747}," calls register tensors with the ZeroGPU backend, which writes them to a disk offload directory at a startup \"pack\" step and frees the corresponding RAM. When a ",{"type":44,"tag":67,"props":749,"children":751},{"className":750},[],[752],{"type":50,"value":72},{"type":50,"value":754}," call lands, a forked GPU worker process streams those weights from disk into VRAM via a pinned-memory pipeline. Warm workers (reused across requests on the same GPU slot) keep weights resident on the GPU and skip the disk → VRAM step. The user-facing rule: write ",{"type":44,"tag":67,"props":756,"children":758},{"className":757},[],[759],{"type":50,"value":732},{"type":50,"value":761}," at module scope and it works — see ",{"type":44,"tag":67,"props":763,"children":765},{"className":764},[],[766],{"type":50,"value":174},{"type":50,"value":768}," for the full lifecycle.",{"type":44,"tag":119,"props":770,"children":771},{},[772,793],{"type":44,"tag":123,"props":773,"children":774},{},[775],{"type":44,"tag":127,"props":776,"children":777},{},[778,783,788],{"type":44,"tag":131,"props":779,"children":780},{},[781],{"type":50,"value":782},"Action",{"type":44,"tag":131,"props":784,"children":785},{},[786],{"type":50,"value":787},"Where",{"type":44,"tag":131,"props":789,"children":790},{},[791],{"type":50,"value":792},"Why",{"type":44,"tag":142,"props":794,"children":795},{},[796,827,853],{"type":44,"tag":127,"props":797,"children":798},{},[799,814,822],{"type":44,"tag":149,"props":800,"children":801},{},[802,807,808],{"type":44,"tag":67,"props":803,"children":805},{"className":804},[],[806],{"type":50,"value":745},{"type":50,"value":726},{"type":44,"tag":67,"props":809,"children":811},{"className":810},[],[812],{"type":50,"value":813},"pipe(..., device=\"cuda\")",{"type":44,"tag":149,"props":815,"children":816},{},[817],{"type":44,"tag":59,"props":818,"children":819},{},[820],{"type":50,"value":821},"Module scope",{"type":44,"tag":149,"props":823,"children":824},{},[825],{"type":50,"value":826},"ZeroGPU registers the tensor and manages device migration",{"type":44,"tag":127,"props":828,"children":829},{},[830,835,848],{"type":44,"tag":149,"props":831,"children":832},{},[833],{"type":50,"value":834},"Actual CUDA computation (inference, etc.)",{"type":44,"tag":149,"props":836,"children":837},{},[838],{"type":44,"tag":59,"props":839,"children":840},{},[841,843],{"type":50,"value":842},"Inside ",{"type":44,"tag":67,"props":844,"children":846},{"className":845},[],[847],{"type":50,"value":72},{"type":44,"tag":149,"props":849,"children":850},{},[851],{"type":50,"value":852},"Real GPU is only attached during the decorated call",{"type":44,"tag":127,"props":854,"children":855},{},[856,866,871],{"type":44,"tag":149,"props":857,"children":858},{},[859,861],{"type":50,"value":860},"Branching on ",{"type":44,"tag":67,"props":862,"children":864},{"className":863},[],[865],{"type":50,"value":706},{"type":44,"tag":149,"props":867,"children":868},{},[869],{"type":50,"value":870},"Avoid relying on it",{"type":44,"tag":149,"props":872,"children":873},{},[874,876,881],{"type":50,"value":875},"Always returns ",{"type":44,"tag":67,"props":877,"children":879},{"className":878},[],[880],{"type":50,"value":714},{"type":50,"value":882}," due to the monkey-patch",{"type":44,"tag":53,"props":884,"children":885},{},[886],{"type":50,"value":887},"Do not run inference or CUDA kernels at module scope — the real GPU is not attached, so operations either silently run on CPU or fail.",{"type":44,"tag":889,"props":890,"children":892},"h3",{"id":891},"device-selection-idiom-still-works",[893],{"type":50,"value":894},"Device selection idiom still works",{"type":44,"tag":53,"props":896,"children":897},{},[898],{"type":50,"value":899},"The standard idiom remains correct under ZeroGPU:",{"type":44,"tag":388,"props":901,"children":903},{"className":390,"code":902,"language":23,"meta":392,"style":392},"device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nmodel = AutoModel.from_pretrained(\"...\").to(device)\n",[904],{"type":44,"tag":67,"props":905,"children":906},{"__ignoreMap":392},[907,915],{"type":44,"tag":398,"props":908,"children":909},{"class":400,"line":401},[910],{"type":44,"tag":398,"props":911,"children":912},{},[913],{"type":50,"value":914},"device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",{"type":44,"tag":398,"props":916,"children":917},{"class":400,"line":410},[918],{"type":44,"tag":398,"props":919,"children":920},{},[921],{"type":50,"value":922},"model = AutoModel.from_pretrained(\"...\").to(device)\n",{"type":44,"tag":695,"props":924,"children":925},{},[926,950,973],{"type":44,"tag":490,"props":927,"children":928},{},[929,933,935,941,943,948],{"type":44,"tag":59,"props":930,"children":931},{},[932],{"type":50,"value":63},{"type":50,"value":934}," — ",{"type":44,"tag":67,"props":936,"children":938},{"className":937},[],[939],{"type":50,"value":940},"is_available()",{"type":50,"value":942}," is ",{"type":44,"tag":67,"props":944,"children":946},{"className":945},[],[947],{"type":50,"value":714},{"type":50,"value":949}," (monkey-patched), so the model is registered for automatic device migration.",{"type":44,"tag":490,"props":951,"children":952},{},[953,958,959,964,966,971],{"type":44,"tag":59,"props":954,"children":955},{},[956],{"type":50,"value":957},"Dedicated GPU Spaces \u002F local GPU",{"type":50,"value":934},{"type":44,"tag":67,"props":960,"children":962},{"className":961},[],[963],{"type":50,"value":940},{"type":50,"value":965}," is genuinely ",{"type":44,"tag":67,"props":967,"children":969},{"className":968},[],[970],{"type":50,"value":714},{"type":50,"value":972},".",{"type":44,"tag":490,"props":974,"children":975},{},[976,981,983,989],{"type":44,"tag":59,"props":977,"children":978},{},[979],{"type":50,"value":980},"CPU Spaces \u002F local CPU",{"type":50,"value":982}," — resolves to ",{"type":44,"tag":67,"props":984,"children":986},{"className":985},[],[987],{"type":50,"value":988},"\"cpu\"",{"type":50,"value":972},{"type":44,"tag":53,"props":991,"children":992},{},[993,995,1000],{"type":50,"value":994},"Do not hardcode ",{"type":44,"tag":67,"props":996,"children":998},{"className":997},[],[999],{"type":50,"value":732},{"type":50,"value":1001}," — it breaks on CPU-only environments.",{"type":44,"tag":889,"props":1003,"children":1005},{"id":1004},"eager-loading-is-the-right-default",[1006],{"type":50,"value":1007},"Eager loading is the right default",{"type":44,"tag":53,"props":1009,"children":1010},{},[1011,1013,1019,1021,1027],{"type":50,"value":1012},"Load models at module scope, not lazily on first request. The Space process starts before any user arrives, so cold-start cost is paid once. Lazy loading (",{"type":44,"tag":67,"props":1014,"children":1016},{"className":1015},[],[1017],{"type":50,"value":1018},"global model; if model is None: ...",{"type":50,"value":1020},", ",{"type":44,"tag":67,"props":1022,"children":1024},{"className":1023},[],[1025],{"type":50,"value":1026},"@lru_cache",{"type":50,"value":1028}," wrappers, factory functions instantiating on first call) just pushes that cost onto the first user.",{"type":44,"tag":76,"props":1030,"children":1032},{"id":1031},"local-development-just-install-spaces",[1033,1035],{"type":50,"value":1034},"Local Development: Just Install ",{"type":44,"tag":67,"props":1036,"children":1038},{"className":1037},[],[1039],{"type":50,"value":1040},"spaces",{"type":44,"tag":53,"props":1042,"children":1043},{},[1044,1046,1051,1053,1058,1060,1066,1068,1074,1076,1081],{"type":50,"value":1045},"Do ",{"type":44,"tag":59,"props":1047,"children":1048},{},[1049],{"type":50,"value":1050},"not",{"type":50,"value":1052}," wrap ",{"type":44,"tag":67,"props":1054,"children":1056},{"className":1055},[],[1057],{"type":50,"value":679},{"type":50,"value":1059}," in ",{"type":44,"tag":67,"props":1061,"children":1063},{"className":1062},[],[1064],{"type":50,"value":1065},"try\u002Fexcept",{"type":50,"value":1067}," and redefine ",{"type":44,"tag":67,"props":1069,"children":1071},{"className":1070},[],[1072],{"type":50,"value":1073},"spaces.GPU",{"type":50,"value":1075}," as a no-op fallback for local runs. Off-ZeroGPU, the ",{"type":44,"tag":67,"props":1077,"children":1079},{"className":1078},[],[1080],{"type":50,"value":1040},{"type":50,"value":1082}," package is already a true no-op:",{"type":44,"tag":695,"props":1084,"children":1085},{},[1086,1099,1109],{"type":44,"tag":490,"props":1087,"children":1088},{},[1089,1091,1097],{"type":50,"value":1090},"Heavyweight behavior (CUDA monkey-patching, client init, startup hooks) is gated on the ",{"type":44,"tag":67,"props":1092,"children":1094},{"className":1093},[],[1095],{"type":50,"value":1096},"SPACES_ZERO_GPU",{"type":50,"value":1098}," env var, set only on ZeroGPU.",{"type":44,"tag":490,"props":1100,"children":1101},{},[1102,1107],{"type":44,"tag":67,"props":1103,"children":1105},{"className":1104},[],[1106],{"type":50,"value":72},{"type":50,"value":1108}," returns the undecorated function unchanged off-ZeroGPU.",{"type":44,"tag":490,"props":1110,"children":1111},{},[1112,1114,1119],{"type":50,"value":1113},"Top-level ",{"type":44,"tag":67,"props":1115,"children":1117},{"className":1116},[],[1118],{"type":50,"value":679},{"type":50,"value":1120}," performs only lightweight imports.",{"type":44,"tag":53,"props":1122,"children":1123},{},[1124,1126,1131,1133,1138,1140,1145],{"type":50,"value":1125},"The Gradio SDK base image installs ",{"type":44,"tag":67,"props":1127,"children":1129},{"className":1128},[],[1130],{"type":50,"value":1040},{"type":50,"value":1132}," on every hardware tier. So even after duplicating a Space onto a dedicated GPU (T4, L4, A10G, etc.) or CPU basic, no code changes are needed — ",{"type":44,"tag":67,"props":1134,"children":1136},{"className":1135},[],[1137],{"type":50,"value":679},{"type":50,"value":1139}," still succeeds and ",{"type":44,"tag":67,"props":1141,"children":1143},{"className":1142},[],[1144],{"type":50,"value":72},{"type":50,"value":1146}," becomes a transparent passthrough.",{"type":44,"tag":889,"props":1148,"children":1150},{"id":1149},"anti-pattern",[1151],{"type":50,"value":1152},"Anti-pattern",{"type":44,"tag":388,"props":1154,"children":1156},{"className":390,"code":1155,"language":23,"meta":392,"style":392},"try:\n    import spaces\nexcept ImportError:\n    class spaces:  # type: ignore\n        @staticmethod\n        def GPU(func=None, **kwargs):\n            return func if func else (lambda f: f)\n",[1157],{"type":44,"tag":67,"props":1158,"children":1159},{"__ignoreMap":392},[1160,1168,1176,1184,1192,1200,1208],{"type":44,"tag":398,"props":1161,"children":1162},{"class":400,"line":401},[1163],{"type":44,"tag":398,"props":1164,"children":1165},{},[1166],{"type":50,"value":1167},"try:\n",{"type":44,"tag":398,"props":1169,"children":1170},{"class":400,"line":410},[1171],{"type":44,"tag":398,"props":1172,"children":1173},{},[1174],{"type":50,"value":1175},"    import spaces\n",{"type":44,"tag":398,"props":1177,"children":1178},{"class":400,"line":419},[1179],{"type":44,"tag":398,"props":1180,"children":1181},{},[1182],{"type":50,"value":1183},"except ImportError:\n",{"type":44,"tag":398,"props":1185,"children":1186},{"class":400,"line":428},[1187],{"type":44,"tag":398,"props":1188,"children":1189},{},[1190],{"type":50,"value":1191},"    class spaces:  # type: ignore\n",{"type":44,"tag":398,"props":1193,"children":1194},{"class":400,"line":438},[1195],{"type":44,"tag":398,"props":1196,"children":1197},{},[1198],{"type":50,"value":1199},"        @staticmethod\n",{"type":44,"tag":398,"props":1201,"children":1202},{"class":400,"line":447},[1203],{"type":44,"tag":398,"props":1204,"children":1205},{},[1206],{"type":50,"value":1207},"        def GPU(func=None, **kwargs):\n",{"type":44,"tag":398,"props":1209,"children":1210},{"class":400,"line":455},[1211],{"type":44,"tag":398,"props":1212,"children":1213},{},[1214],{"type":50,"value":1215},"            return func if func else (lambda f: f)\n",{"type":44,"tag":53,"props":1217,"children":1218},{},[1219],{"type":50,"value":1220},"Problems:",{"type":44,"tag":486,"props":1222,"children":1223},{},[1224,1266,1286],{"type":44,"tag":490,"props":1225,"children":1226},{},[1227,1229,1234,1236,1242,1243,1249,1251,1257,1259,1264],{"type":50,"value":1228},"The fallback must mimic every ",{"type":44,"tag":67,"props":1230,"children":1232},{"className":1231},[],[1233],{"type":50,"value":72},{"type":50,"value":1235}," call shape — bare decorator, ",{"type":44,"tag":67,"props":1237,"children":1239},{"className":1238},[],[1240],{"type":50,"value":1241},"duration=...",{"type":50,"value":1020},{"type":44,"tag":67,"props":1244,"children":1246},{"className":1245},[],[1247],{"type":50,"value":1248},"size=...",{"type":50,"value":1250},", generators, ",{"type":44,"tag":67,"props":1252,"children":1254},{"className":1253},[],[1255],{"type":50,"value":1256},"aoti_*",{"type":50,"value":1258}," helpers — and drifts as the ",{"type":44,"tag":67,"props":1260,"children":1262},{"className":1261},[],[1263],{"type":50,"value":1040},{"type":50,"value":1265}," API grows.",{"type":44,"tag":490,"props":1267,"children":1268},{},[1269,1271,1276,1278,1284],{"type":50,"value":1270},"It hides ",{"type":44,"tag":67,"props":1272,"children":1274},{"className":1273},[],[1275],{"type":50,"value":1040},{"type":50,"value":1277}," from ",{"type":44,"tag":67,"props":1279,"children":1281},{"className":1280},[],[1282],{"type":50,"value":1283},"requirements.txt",{"type":50,"value":1285},", even though the Space needs it at deploy time.",{"type":44,"tag":490,"props":1287,"children":1288},{},[1289],{"type":50,"value":1290},"It solves a non-problem: the real package is already a no-op locally.",{"type":44,"tag":889,"props":1292,"children":1294},{"id":1293},"do-this-instead",[1295],{"type":50,"value":1296},"Do this instead",{"type":44,"tag":53,"props":1298,"children":1299},{},[1300,1302,1307],{"type":50,"value":1301},"Add ",{"type":44,"tag":67,"props":1303,"children":1305},{"className":1304},[],[1306],{"type":50,"value":1040},{"type":50,"value":1308}," to dependencies and import it unconditionally:",{"type":44,"tag":388,"props":1310,"children":1312},{"className":390,"code":1311,"language":23,"meta":392,"style":392},"import spaces\n\n@spaces.GPU\ndef generate(prompt: str) -> str:\n    ...\n",[1313],{"type":44,"tag":67,"props":1314,"children":1315},{"__ignoreMap":392},[1316,1323,1330,1337,1344],{"type":44,"tag":398,"props":1317,"children":1318},{"class":400,"line":401},[1319],{"type":44,"tag":398,"props":1320,"children":1321},{},[1322],{"type":50,"value":407},{"type":44,"tag":398,"props":1324,"children":1325},{"class":400,"line":410},[1326],{"type":44,"tag":398,"props":1327,"children":1328},{"emptyLinePlaceholder":432},[1329],{"type":50,"value":435},{"type":44,"tag":398,"props":1331,"children":1332},{"class":400,"line":419},[1333],{"type":44,"tag":398,"props":1334,"children":1335},{},[1336],{"type":50,"value":461},{"type":44,"tag":398,"props":1338,"children":1339},{"class":400,"line":428},[1340],{"type":44,"tag":398,"props":1341,"children":1342},{},[1343],{"type":50,"value":470},{"type":44,"tag":398,"props":1345,"children":1346},{"class":400,"line":438},[1347],{"type":44,"tag":398,"props":1348,"children":1349},{},[1350],{"type":50,"value":1351},"    ...\n",{"type":44,"tag":76,"props":1353,"children":1355},{"id":1354},"duration-and-quota",[1356],{"type":50,"value":1357},"Duration and Quota",{"type":44,"tag":53,"props":1359,"children":1360},{},[1361,1363,1369],{"type":50,"value":1362},"Three things happen when you declare ",{"type":44,"tag":67,"props":1364,"children":1366},{"className":1365},[],[1367],{"type":50,"value":1368},"@spaces.GPU(duration=N)",{"type":50,"value":1370},":",{"type":44,"tag":486,"props":1372,"children":1373},{},[1374,1413,1460],{"type":44,"tag":490,"props":1375,"children":1376},{},[1377,1382,1384,1389,1391,1396,1398,1404,1406,1411],{"type":44,"tag":59,"props":1378,"children":1379},{},[1380],{"type":50,"value":1381},"Tier-max check",{"type":50,"value":1383}," — each visitor tier has a per-call ",{"type":44,"tag":67,"props":1385,"children":1387},{"className":1386},[],[1388],{"type":50,"value":202},{"type":50,"value":1390}," cap. Declaring ",{"type":44,"tag":67,"props":1392,"children":1394},{"className":1393},[],[1395],{"type":50,"value":202},{"type":50,"value":1397}," larger than the cap fails immediately with ",{"type":44,"tag":67,"props":1399,"children":1401},{"className":1400},[],[1402],{"type":50,"value":1403},"ZeroGPU illegal duration",{"type":50,"value":1405},", regardless of remaining quota. (Tier numbers change over time — see the ",{"type":44,"tag":103,"props":1407,"children":1409},{"href":105,"rel":1408},[107],[1410],{"type":50,"value":378},{"type":50,"value":1412},".)",{"type":44,"tag":490,"props":1414,"children":1415},{},[1416,1421,1423,1428,1429,1434,1436,1442,1444,1450,1452,1458],{"type":44,"tag":59,"props":1417,"children":1418},{},[1419],{"type":50,"value":1420},"Quota pre-check",{"type":50,"value":1422}," — the platform compares ",{"type":44,"tag":67,"props":1424,"children":1426},{"className":1425},[],[1427],{"type":50,"value":545},{"type":50,"value":547},{"type":44,"tag":67,"props":1430,"children":1432},{"className":1431},[],[1433],{"type":50,"value":553},{"type":50,"value":1435},". If ",{"type":44,"tag":67,"props":1437,"children":1439},{"className":1438},[],[1440],{"type":50,"value":1441},"remaining \u003C requested",{"type":50,"value":1443},", the call fails with ",{"type":44,"tag":67,"props":1445,"children":1447},{"className":1446},[],[1448],{"type":50,"value":1449},"ZeroGPU quota exceeded",{"type":50,"value":1451}," — even if the actual work would have fit. The error message shows the explicit numbers, e.g. ",{"type":44,"tag":67,"props":1453,"children":1455},{"className":1454},[],[1456],{"type":50,"value":1457},"\"60s requested vs. 30s left\"",{"type":50,"value":1459},". A 10-second task left at the default 60s therefore blocks the user once their remaining quota drops below 60s.",{"type":44,"tag":490,"props":1461,"children":1462},{},[1463,1468,1470,1475],{"type":44,"tag":59,"props":1464,"children":1465},{},[1466],{"type":50,"value":1467},"Queue priority",{"type":50,"value":1469}," — the queue is node-level (requests from all Spaces on the same node compete for GPU slots), and shorter declared ",{"type":44,"tag":67,"props":1471,"children":1473},{"className":1472},[],[1474],{"type":50,"value":202},{"type":50,"value":1476}," ranks higher.",{"type":44,"tag":53,"props":1478,"children":1479},{},[1480,1482,1487,1489,1495,1497,1502],{"type":50,"value":1481},"All three favor declaring the smallest realistic ",{"type":44,"tag":67,"props":1483,"children":1485},{"className":1484},[],[1486],{"type":50,"value":202},{"type":50,"value":1488}," — including for short tasks. Explicit ",{"type":44,"tag":67,"props":1490,"children":1492},{"className":1491},[],[1493],{"type":50,"value":1494},"@spaces.GPU(duration=15)",{"type":50,"value":1496}," on a 10-second task avoids premature ",{"type":44,"tag":67,"props":1498,"children":1500},{"className":1499},[],[1501],{"type":50,"value":218},{"type":50,"value":1503}," rejections and ranks higher in the queue.",{"type":44,"tag":360,"props":1505,"children":1506},{},[1507],{"type":44,"tag":53,"props":1508,"children":1509},{},[1510,1520,1521,1527,1529,1534,1536,1542],{"type":44,"tag":59,"props":1511,"children":1512},{},[1513,1518],{"type":44,"tag":67,"props":1514,"children":1516},{"className":1515},[],[1517],{"type":50,"value":329},{"type":50,"value":1519}," doubles the request.",{"type":50,"value":301},{"type":44,"tag":67,"props":1522,"children":1524},{"className":1523},[],[1525],{"type":50,"value":1526},"requested = N * 2",{"type":50,"value":1528}," when ",{"type":44,"tag":67,"props":1530,"children":1532},{"className":1531},[],[1533],{"type":50,"value":608},{"type":50,"value":1535},", both for the tier-max check and the quota pre-check. So ",{"type":44,"tag":67,"props":1537,"children":1539},{"className":1538},[],[1540],{"type":50,"value":1541},"@spaces.GPU(duration=60, size=\"xlarge\")",{"type":50,"value":1543}," is internally a 120s request.",{"type":44,"tag":889,"props":1545,"children":1547},{"id":1546},"dynamic-duration-for-variable-workloads",[1548],{"type":50,"value":1549},"Dynamic duration for variable workloads",{"type":44,"tag":53,"props":1551,"children":1552},{},[1553,1555,1560],{"type":50,"value":1554},"For workloads whose runtime depends on inputs, pass a callable that estimates per request. A static high ",{"type":44,"tag":67,"props":1556,"children":1558},{"className":1557},[],[1559],{"type":50,"value":202},{"type":50,"value":1561}," locks out low-tier users (whose tier cap may be smaller than the static value) and unnecessarily reserves quota for light inputs.",{"type":44,"tag":388,"props":1563,"children":1565},{"className":390,"code":1564,"language":23,"meta":392,"style":392},"def estimate_duration(prompt, steps):\n    return int(steps * 3.5)\n\n@spaces.GPU(duration=estimate_duration)\ndef generate(prompt, steps):\n    return pipe(prompt, num_inference_steps=steps).images[0]\n",[1566],{"type":44,"tag":67,"props":1567,"children":1568},{"__ignoreMap":392},[1569,1577,1585,1592,1600,1608],{"type":44,"tag":398,"props":1570,"children":1571},{"class":400,"line":401},[1572],{"type":44,"tag":398,"props":1573,"children":1574},{},[1575],{"type":50,"value":1576},"def estimate_duration(prompt, steps):\n",{"type":44,"tag":398,"props":1578,"children":1579},{"class":400,"line":410},[1580],{"type":44,"tag":398,"props":1581,"children":1582},{},[1583],{"type":50,"value":1584},"    return int(steps * 3.5)\n",{"type":44,"tag":398,"props":1586,"children":1587},{"class":400,"line":419},[1588],{"type":44,"tag":398,"props":1589,"children":1590},{"emptyLinePlaceholder":432},[1591],{"type":50,"value":435},{"type":44,"tag":398,"props":1593,"children":1594},{"class":400,"line":428},[1595],{"type":44,"tag":398,"props":1596,"children":1597},{},[1598],{"type":50,"value":1599},"@spaces.GPU(duration=estimate_duration)\n",{"type":44,"tag":398,"props":1601,"children":1602},{"class":400,"line":438},[1603],{"type":44,"tag":398,"props":1604,"children":1605},{},[1606],{"type":50,"value":1607},"def generate(prompt, steps):\n",{"type":44,"tag":398,"props":1609,"children":1610},{"class":400,"line":447},[1611],{"type":44,"tag":398,"props":1612,"children":1613},{},[1614],{"type":50,"value":1615},"    return pipe(prompt, num_inference_steps=steps).images[0]\n",{"type":44,"tag":53,"props":1617,"children":1618},{},[1619,1621,1626,1627,1632,1634,1639],{"type":50,"value":1620},"For the full distinction between ",{"type":44,"tag":67,"props":1622,"children":1624},{"className":1623},[],[1625],{"type":50,"value":210},{"type":50,"value":212},{"type":44,"tag":67,"props":1628,"children":1630},{"className":1629},[],[1631],{"type":50,"value":218},{"type":50,"value":1633},", runs-per-day limits, the 24h quota window, and pay-as-you-go billing, see ",{"type":44,"tag":67,"props":1635,"children":1637},{"className":1636},[],[1638],{"type":50,"value":191},{"type":50,"value":972},{"type":44,"tag":76,"props":1641,"children":1643},{"id":1642},"process-isolation-and-pickle",[1644],{"type":50,"value":1645},"Process Isolation and Pickle",{"type":44,"tag":53,"props":1647,"children":1648},{},[1649,1654,1656,1661,1663,1668],{"type":44,"tag":67,"props":1650,"children":1652},{"className":1651},[],[1653],{"type":50,"value":72},{"type":50,"value":1655},"-decorated functions run in a ",{"type":44,"tag":59,"props":1657,"children":1658},{},[1659],{"type":50,"value":1660},"separate process",{"type":50,"value":1662}," managed by the ZeroGPU scheduler. Arguments and return values cross the process boundary via ",{"type":44,"tag":59,"props":1664,"children":1665},{},[1666],{"type":50,"value":1667},"pickle serialization",{"type":50,"value":972},{"type":44,"tag":53,"props":1670,"children":1671},{},[1672],{"type":50,"value":1673},"Consequences:",{"type":44,"tag":695,"props":1675,"children":1676},{},[1677,1694,1727,1732],{"type":44,"tag":490,"props":1678,"children":1679},{},[1680,1685,1687,1693],{"type":44,"tag":59,"props":1681,"children":1682},{},[1683],{"type":50,"value":1684},"Only picklable objects",{"type":50,"value":1686}," can be passed in or returned. Open file handles, database connections, locks, lambdas, and closures over unpicklable state will raise ",{"type":44,"tag":67,"props":1688,"children":1690},{"className":1689},[],[1691],{"type":50,"value":1692},"PicklingError",{"type":50,"value":972},{"type":44,"tag":490,"props":1695,"children":1696},{},[1697,1702,1704,1710,1712,1718,1720,1726],{"type":44,"tag":59,"props":1698,"children":1699},{},[1700],{"type":50,"value":1701},"Do NOT return CUDA tensors directly.",{"type":50,"value":1703}," Unpickling a CUDA tensor in the main process triggers ",{"type":44,"tag":67,"props":1705,"children":1707},{"className":1706},[],[1708],{"type":50,"value":1709},"torch.cuda._lazy_init()",{"type":50,"value":1711},", which ZeroGPU blocks. Convert to CPU first: return ",{"type":44,"tag":67,"props":1713,"children":1715},{"className":1714},[],[1716],{"type":50,"value":1717},"tensor.cpu()",{"type":50,"value":1719}," or ",{"type":44,"tag":67,"props":1721,"children":1723},{"className":1722},[],[1724],{"type":50,"value":1725},"tensor.cpu().numpy()",{"type":50,"value":972},{"type":44,"tag":490,"props":1728,"children":1729},{},[1730],{"type":50,"value":1731},"CPU tensors, numpy arrays, PIL Images, and plain Python objects work fine.",{"type":44,"tag":490,"props":1733,"children":1734},{},[1735],{"type":50,"value":1736},"Large objects incur serialization overhead. Prefer lightweight returns (tensors, arrays, file paths, base64 strings) over complex object graphs.",{"type":44,"tag":889,"props":1738,"children":1740},{"id":1739},"grstate-semantics-across-the-boundary",[1741,1747],{"type":44,"tag":67,"props":1742,"children":1744},{"className":1743},[],[1745],{"type":50,"value":1746},"gr.State",{"type":50,"value":1748}," semantics across the boundary",{"type":44,"tag":53,"props":1750,"children":1751},{},[1752,1754,1759,1761,1766],{"type":50,"value":1753},"Because handlers run in a separate process, ",{"type":44,"tag":67,"props":1755,"children":1757},{"className":1756},[],[1758],{"type":50,"value":1746},{"type":50,"value":1760}," values are ",{"type":44,"tag":59,"props":1762,"children":1763},{},[1764],{"type":50,"value":1765},"pickled on every yield",{"type":50,"value":1767}," — they are NOT shared by reference.",{"type":44,"tag":695,"props":1769,"children":1770},{},[1771,1791,1803,1830],{"type":44,"tag":490,"props":1772,"children":1773},{},[1774,1776,1781,1783,1789],{"type":50,"value":1775},"The generator receives a ",{"type":44,"tag":59,"props":1777,"children":1778},{},[1779],{"type":50,"value":1780},"copy",{"type":50,"value":1782}," of the state (",{"type":44,"tag":67,"props":1784,"children":1786},{"className":1785},[],[1787],{"type":50,"value":1788},"id()",{"type":50,"value":1790}," differs from the caller's).",{"type":44,"tag":490,"props":1792,"children":1793},{},[1794,1796,1801],{"type":50,"value":1795},"In-place mutations inside the generator are ",{"type":44,"tag":59,"props":1797,"children":1798},{},[1799],{"type":50,"value":1800},"invisible",{"type":50,"value":1802}," to other handlers until the mutated state is explicitly yielded back.",{"type":44,"tag":490,"props":1804,"children":1805},{},[1806,1808,1814,1816,1821,1823,1828],{"type":50,"value":1807},"Yielding ",{"type":44,"tag":67,"props":1809,"children":1811},{"className":1810},[],[1812],{"type":50,"value":1813},"gr.update()",{"type":50,"value":1815}," for a ",{"type":44,"tag":67,"props":1817,"children":1819},{"className":1818},[],[1820],{"type":50,"value":1746},{"type":50,"value":1822}," slot ",{"type":44,"tag":59,"props":1824,"children":1825},{},[1826],{"type":50,"value":1827},"skips the update",{"type":50,"value":1829}," — other handlers continue to see the pre-yield value.",{"type":44,"tag":490,"props":1831,"children":1832},{},[1833,1835,1840],{"type":50,"value":1834},"Each yield that returns the state object creates a ",{"type":44,"tag":59,"props":1836,"children":1837},{},[1838],{"type":50,"value":1839},"new copy",{"type":50,"value":1841}," via pickle.",{"type":44,"tag":53,"props":1843,"children":1844},{},[1845],{"type":50,"value":1846},"Practical guidance:",{"type":44,"tag":695,"props":1848,"children":1849},{},[1850,1865,1889],{"type":44,"tag":490,"props":1851,"children":1852},{},[1853,1863],{"type":44,"tag":59,"props":1854,"children":1855},{},[1856,1858],{"type":50,"value":1857},"Do NOT assume reference semantics for ",{"type":44,"tag":67,"props":1859,"children":1861},{"className":1860},[],[1862],{"type":50,"value":1746},{"type":50,"value":1864}," on ZeroGPU. Code that mutates state in a generator and expects another handler to see those mutations will silently use stale data.",{"type":44,"tag":490,"props":1866,"children":1867},{},[1868,1880,1882,1887],{"type":44,"tag":59,"props":1869,"children":1870},{},[1871,1873,1878],{"type":50,"value":1872},"Every yield including a ",{"type":44,"tag":67,"props":1874,"children":1876},{"className":1875},[],[1877],{"type":50,"value":1746},{"type":50,"value":1879}," value triggers a full pickle round-trip.",{"type":50,"value":1881}," For large state (model sessions, frame buffers), minimize how often you yield it — ideally once at the end. Use ",{"type":44,"tag":67,"props":1883,"children":1885},{"className":1884},[],[1886],{"type":50,"value":1813},{"type":50,"value":1888}," for the state slot on intermediate yields.",{"type":44,"tag":490,"props":1890,"children":1891},{},[1892,1897,1899,1904],{"type":44,"tag":59,"props":1893,"children":1894},{},[1895],{"type":50,"value":1896},"CUDA tensors inside state must be moved to CPU before yielding",{"type":50,"value":1898}," — same ",{"type":44,"tag":67,"props":1900,"children":1902},{"className":1901},[],[1903],{"type":50,"value":1709},{"type":50,"value":1905}," issue as above.",{"type":44,"tag":76,"props":1907,"children":1909},{"id":1908},"concurrency",[1910],{"type":50,"value":1911},"Concurrency",{"type":44,"tag":53,"props":1913,"children":1914},{},[1915,1917,1922],{"type":50,"value":1916},"Handlers run ",{"type":44,"tag":59,"props":1918,"children":1919},{},[1920],{"type":50,"value":1921},"concurrently by default",{"type":50,"value":1923}," on ZeroGPU. This is not opt-in. Code that worked in single-user testing can silently corrupt or leak data in production.",{"type":44,"tag":53,"props":1925,"children":1926},{},[1927,1929,1934],{"type":50,"value":1928},"Three rules. Full treatment with examples in ",{"type":44,"tag":67,"props":1930,"children":1932},{"className":1931},[],[1933],{"type":50,"value":157},{"type":50,"value":972},{"type":44,"tag":486,"props":1936,"children":1937},{},[1938,1948,1966],{"type":44,"tag":490,"props":1939,"children":1940},{},[1941,1946],{"type":44,"tag":59,"props":1942,"children":1943},{},[1944],{"type":50,"value":1945},"No mutable global state.",{"type":50,"value":1947}," Concurrent requests overwrite each other.",{"type":44,"tag":490,"props":1949,"children":1950},{},[1951,1956,1958,1964],{"type":44,"tag":59,"props":1952,"children":1953},{},[1954],{"type":50,"value":1955},"No fixed file paths for outputs.",{"type":50,"value":1957}," Concurrent requests clobber the same file. Use ",{"type":44,"tag":67,"props":1959,"children":1961},{"className":1960},[],[1962],{"type":50,"value":1963},"tempfile",{"type":50,"value":1965}," for unique paths.",{"type":44,"tag":490,"props":1967,"children":1968},{},[1969,1974],{"type":44,"tag":59,"props":1970,"children":1971},{},[1972],{"type":50,"value":1973},"Read-only globals are safe.",{"type":50,"value":1975}," Model objects, tokenizers, configs loaded once at startup and only read during requests are safe and encouraged.",{"type":44,"tag":76,"props":1977,"children":1979},{"id":1978},"call-granularity",[1980],{"type":50,"value":1981},"Call Granularity",{"type":44,"tag":53,"props":1983,"children":1984},{},[1985,1987,1992],{"type":50,"value":1986},"Each entry into a ",{"type":44,"tag":67,"props":1988,"children":1990},{"className":1989},[],[1991],{"type":50,"value":72},{"type":50,"value":1993}," function carries non-trivial cost — pickle round-trip across the process boundary, worker warm-up, CUDA re-attach, and a fresh pass through the node-level queue. Calling a decorated function from inside a hot loop multiplies these costs and adds a new failure mode: a later iteration may fail to acquire a GPU slot, stalling the whole job mid-way.",{"type":44,"tag":53,"props":1995,"children":1996},{},[1997],{"type":50,"value":1998},"Decorate the outer function that owns the loop, not the per-iteration worker:",{"type":44,"tag":388,"props":2000,"children":2002},{"className":390,"code":2001,"language":23,"meta":392,"style":392},"# Avoid — N GPU entries for N frames\ndef process_video(frames):\n    return [process_frame(f) for f in frames]\n\n@spaces.GPU(duration=...)\ndef process_frame(frame):\n    ...\n\n# Prefer — one GPU entry for the whole video\n@spaces.GPU(duration=...)\ndef process_video(frames):\n    return [process_frame(f) for f in frames]\n\ndef process_frame(frame):\n    ...\n",[2003],{"type":44,"tag":67,"props":2004,"children":2005},{"__ignoreMap":392},[2006,2014,2022,2030,2037,2045,2053,2060,2067,2075,2083,2091,2099,2107,2115],{"type":44,"tag":398,"props":2007,"children":2008},{"class":400,"line":401},[2009],{"type":44,"tag":398,"props":2010,"children":2011},{},[2012],{"type":50,"value":2013},"# Avoid — N GPU entries for N frames\n",{"type":44,"tag":398,"props":2015,"children":2016},{"class":400,"line":410},[2017],{"type":44,"tag":398,"props":2018,"children":2019},{},[2020],{"type":50,"value":2021},"def process_video(frames):\n",{"type":44,"tag":398,"props":2023,"children":2024},{"class":400,"line":419},[2025],{"type":44,"tag":398,"props":2026,"children":2027},{},[2028],{"type":50,"value":2029},"    return [process_frame(f) for f in frames]\n",{"type":44,"tag":398,"props":2031,"children":2032},{"class":400,"line":428},[2033],{"type":44,"tag":398,"props":2034,"children":2035},{"emptyLinePlaceholder":432},[2036],{"type":50,"value":435},{"type":44,"tag":398,"props":2038,"children":2039},{"class":400,"line":438},[2040],{"type":44,"tag":398,"props":2041,"children":2042},{},[2043],{"type":50,"value":2044},"@spaces.GPU(duration=...)\n",{"type":44,"tag":398,"props":2046,"children":2047},{"class":400,"line":447},[2048],{"type":44,"tag":398,"props":2049,"children":2050},{},[2051],{"type":50,"value":2052},"def process_frame(frame):\n",{"type":44,"tag":398,"props":2054,"children":2055},{"class":400,"line":455},[2056],{"type":44,"tag":398,"props":2057,"children":2058},{},[2059],{"type":50,"value":1351},{"type":44,"tag":398,"props":2061,"children":2062},{"class":400,"line":464},[2063],{"type":44,"tag":398,"props":2064,"children":2065},{"emptyLinePlaceholder":432},[2066],{"type":50,"value":435},{"type":44,"tag":398,"props":2068,"children":2069},{"class":400,"line":473},[2070],{"type":44,"tag":398,"props":2071,"children":2072},{},[2073],{"type":50,"value":2074},"# Prefer — one GPU entry for the whole video\n",{"type":44,"tag":398,"props":2076,"children":2078},{"class":400,"line":2077},10,[2079],{"type":44,"tag":398,"props":2080,"children":2081},{},[2082],{"type":50,"value":2044},{"type":44,"tag":398,"props":2084,"children":2086},{"class":400,"line":2085},11,[2087],{"type":44,"tag":398,"props":2088,"children":2089},{},[2090],{"type":50,"value":2021},{"type":44,"tag":398,"props":2092,"children":2094},{"class":400,"line":2093},12,[2095],{"type":44,"tag":398,"props":2096,"children":2097},{},[2098],{"type":50,"value":2029},{"type":44,"tag":398,"props":2100,"children":2102},{"class":400,"line":2101},13,[2103],{"type":44,"tag":398,"props":2104,"children":2105},{"emptyLinePlaceholder":432},[2106],{"type":50,"value":435},{"type":44,"tag":398,"props":2108,"children":2110},{"class":400,"line":2109},14,[2111],{"type":44,"tag":398,"props":2112,"children":2113},{},[2114],{"type":50,"value":2052},{"type":44,"tag":398,"props":2116,"children":2118},{"class":400,"line":2117},15,[2119],{"type":44,"tag":398,"props":2120,"children":2121},{},[2122],{"type":50,"value":1351},{"type":44,"tag":53,"props":2124,"children":2125},{},[2126],{"type":50,"value":2127},"If the loop mixes heavy CPU work with GPU work, wrapping the whole loop charges that CPU time against the user's quota. When that cost is material, batching the GPU work so CPU pre\u002Fpost-processing stays outside the decorator is a situational optimization — not the default.",{"type":44,"tag":76,"props":2129,"children":2131},{"id":2130},"cuda-build-constraints",[2132],{"type":50,"value":2133},"CUDA Build Constraints",{"type":44,"tag":53,"props":2135,"children":2136},{},[2137,2139,2150,2152,2158,2160,2166,2168,2173,2175,2180,2182,2187],{"type":50,"value":2138},"HF Spaces builds Docker images in a CPU-only environment. ",{"type":44,"tag":59,"props":2140,"children":2141},{},[2142,2144],{"type":50,"value":2143},"On ZeroGPU, the build phase has no ",{"type":44,"tag":67,"props":2145,"children":2147},{"className":2146},[],[2148],{"type":50,"value":2149},"nvcc",{"type":50,"value":2151}," because the base image is ",{"type":44,"tag":67,"props":2153,"children":2155},{"className":2154},[],[2156],{"type":50,"value":2157},"python:3.13",{"type":50,"value":2159}," (dedicated-GPU Spaces use ",{"type":44,"tag":67,"props":2161,"children":2163},{"className":2162},[],[2164],{"type":50,"value":2165},"nvidia\u002Fcuda:*-devel-*",{"type":50,"value":2167}," and have ",{"type":44,"tag":67,"props":2169,"children":2171},{"className":2170},[],[2172],{"type":50,"value":2149},{"type":50,"value":2174}," at build time). A CUDA-dependent package whose only distribution is sdist — e.g. bare ",{"type":44,"tag":67,"props":2176,"children":2178},{"className":2177},[],[2179],{"type":50,"value":243},{"type":50,"value":2181}," — therefore cannot be installed via ",{"type":44,"tag":67,"props":2183,"children":2185},{"className":2184},[],[2186],{"type":50,"value":1283},{"type":50,"value":2188}," on ZeroGPU. Only pre-built wheels work.",{"type":44,"tag":53,"props":2190,"children":2191},{},[2192,2194,2199,2201,2206,2208,2214,2216,2222,2224,2229],{"type":50,"value":2193},"ZeroGPU ",{"type":44,"tag":59,"props":2195,"children":2196},{},[2197],{"type":50,"value":2198},"runtime",{"type":50,"value":2200}," does have ",{"type":44,"tag":67,"props":2202,"children":2204},{"className":2203},[],[2205],{"type":50,"value":2149},{"type":50,"value":2207}," available, mounted from a CUDA devel image at ",{"type":44,"tag":67,"props":2209,"children":2211},{"className":2210},[],[2212],{"type":50,"value":2213},"\u002Fcuda-image",{"type":50,"value":2215}," since 2025-07 (originally added for AoTI support). This is what makes ",{"type":44,"tag":67,"props":2217,"children":2219},{"className":2218},[],[2220],{"type":50,"value":2221},"torch.export",{"type":50,"value":2223}," \u002F AoTI workflows possible inside ",{"type":44,"tag":67,"props":2225,"children":2227},{"className":2226},[],[2228],{"type":50,"value":72},{"type":50,"value":2230}," calls.",{"type":44,"tag":53,"props":2232,"children":2233},{},[2234,2239,2241,2246],{"type":44,"tag":59,"props":2235,"children":2236},{},[2237],{"type":50,"value":2238},"Bottom line",{"type":50,"value":2240},": install every CUDA-dependent package from a pre-built wheel. If no wheel is available on PyPI, build one externally (e.g. host on HF Hub) and pin the URL. For ",{"type":44,"tag":67,"props":2242,"children":2244},{"className":2243},[],[2245],{"type":50,"value":243},{"type":50,"value":2247},", the upstream releases page ships a fairly complete wheel matrix covering most Python × CUDA × torch combinations.",{"type":44,"tag":53,"props":2249,"children":2250},{},[2251,2253,2259,2260,2266,2268,2273],{"type":50,"value":2252},"For wheel-tag reading (cxx11 ABI, ",{"type":44,"tag":67,"props":2254,"children":2256},{"className":2255},[],[2257],{"type":50,"value":2258},"cu12torch2.X",{"type":50,"value":1020},{"type":44,"tag":67,"props":2261,"children":2263},{"className":2262},[],[2264],{"type":50,"value":2265},"cp3XX",{"type":50,"value":2267},"), torch-family side-car drift, and the kernels-community fallback, see ",{"type":44,"tag":67,"props":2269,"children":2271},{"className":2270},[],[2272],{"type":50,"value":232},{"type":50,"value":972},{"type":44,"tag":76,"props":2275,"children":2277},{"id":2276},"example-caching",[2278],{"type":50,"value":2279},"Example Caching",{"type":44,"tag":53,"props":2281,"children":2282},{},[2283,2289],{"type":44,"tag":67,"props":2284,"children":2286},{"className":2285},[],[2287],{"type":50,"value":2288},"gr.Examples",{"type":50,"value":2290}," behavior is environment-dependent. On ZeroGPU specifically:",{"type":44,"tag":695,"props":2292,"children":2293},{},[2294,2320],{"type":44,"tag":490,"props":2295,"children":2296},{},[2297,2303,2305,2310,2312,2318],{"type":44,"tag":67,"props":2298,"children":2300},{"className":2299},[],[2301],{"type":50,"value":2302},"cache_examples",{"type":50,"value":2304}," defaults to ",{"type":44,"tag":67,"props":2306,"children":2308},{"className":2307},[],[2309],{"type":50,"value":714},{"type":50,"value":2311}," (Spaces sets ",{"type":44,"tag":67,"props":2313,"children":2315},{"className":2314},[],[2316],{"type":50,"value":2317},"GRADIO_CACHE_EXAMPLES=true",{"type":50,"value":2319},").",{"type":44,"tag":490,"props":2321,"children":2322},{},[2323,2329,2330,2336,2337,2343],{"type":44,"tag":67,"props":2324,"children":2326},{"className":2325},[],[2327],{"type":50,"value":2328},"cache_mode",{"type":50,"value":2304},{"type":44,"tag":67,"props":2331,"children":2333},{"className":2332},[],[2334],{"type":50,"value":2335},"\"lazy\"",{"type":50,"value":2311},{"type":44,"tag":67,"props":2338,"children":2340},{"className":2339},[],[2341],{"type":50,"value":2342},"GRADIO_CACHE_MODE=lazy",{"type":50,"value":2344}," only on ZeroGPU).",{"type":44,"tag":53,"props":2346,"children":2347},{},[2348,2350,2356,2358,2363],{"type":50,"value":2349},"ZeroGPU defaults to ",{"type":44,"tag":67,"props":2351,"children":2353},{"className":2352},[],[2354],{"type":50,"value":2355},"lazy",{"type":50,"value":2357}," because eager caching pre-runs every example at app startup, but ZeroGPU has ",{"type":44,"tag":59,"props":2359,"children":2360},{},[2361],{"type":50,"value":2362},"no GPU attached at startup",{"type":50,"value":2364}," — only during request handling. Eager caching of GPU-bound examples would fail there.",{"type":44,"tag":53,"props":2366,"children":2367},{},[2368,2370,2376,2378,2384,2385,2391,2393,2399],{"type":50,"value":2369},"When ",{"type":44,"tag":67,"props":2371,"children":2373},{"className":2372},[],[2374],{"type":50,"value":2375},"cache_examples=True",{"type":50,"value":2377},", the ",{"type":44,"tag":67,"props":2379,"children":2381},{"className":2380},[],[2382],{"type":50,"value":2383},"run_on_click",{"type":50,"value":726},{"type":44,"tag":67,"props":2386,"children":2388},{"className":2387},[],[2389],{"type":50,"value":2390},"run_examples_on_click",{"type":50,"value":2392}," parameter is silently ignored. If your app relies on click-populates-only behavior, set ",{"type":44,"tag":67,"props":2394,"children":2396},{"className":2395},[],[2397],{"type":50,"value":2398},"cache_examples=False",{"type":50,"value":2400}," explicitly to preserve it.",{"type":44,"tag":53,"props":2402,"children":2403},{},[2404],{"type":50,"value":2405},"To reproduce ZeroGPU example-caching behavior locally:",{"type":44,"tag":388,"props":2407,"children":2411},{"className":2408,"code":2409,"language":2410,"meta":392,"style":392},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","GRADIO_CACHE_EXAMPLES=true GRADIO_CACHE_MODE=lazy python app.py\n","bash",[2412],{"type":44,"tag":67,"props":2413,"children":2414},{"__ignoreMap":392},[2415],{"type":44,"tag":398,"props":2416,"children":2417},{"class":400,"line":401},[2418,2424,2430,2436,2441,2445,2449,2455],{"type":44,"tag":398,"props":2419,"children":2421},{"style":2420},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[2422],{"type":50,"value":2423},"GRADIO_CACHE_EXAMPLES",{"type":44,"tag":398,"props":2425,"children":2427},{"style":2426},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[2428],{"type":50,"value":2429},"=",{"type":44,"tag":398,"props":2431,"children":2433},{"style":2432},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[2434],{"type":50,"value":2435},"true",{"type":44,"tag":398,"props":2437,"children":2438},{"style":2420},[2439],{"type":50,"value":2440}," GRADIO_CACHE_MODE",{"type":44,"tag":398,"props":2442,"children":2443},{"style":2426},[2444],{"type":50,"value":2429},{"type":44,"tag":398,"props":2446,"children":2447},{"style":2432},[2448],{"type":50,"value":2355},{"type":44,"tag":398,"props":2450,"children":2452},{"style":2451},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[2453],{"type":50,"value":2454}," python",{"type":44,"tag":398,"props":2456,"children":2457},{"style":2432},[2458],{"type":50,"value":2459}," app.py\n",{"type":44,"tag":76,"props":2461,"children":2463},{"id":2462},"dependency-management",[2464],{"type":50,"value":2465},"Dependency Management",{"type":44,"tag":889,"props":2467,"children":2469},{"id":2468},"python_version-pin-in-readme-frontmatter",[2470,2476],{"type":44,"tag":67,"props":2471,"children":2473},{"className":2472},[],[2474],{"type":50,"value":2475},"python_version",{"type":50,"value":2477}," pin in README frontmatter",{"type":44,"tag":53,"props":2479,"children":2480},{},[2481,2483,2488,2489,2494,2496,2501],{"type":50,"value":2482},"Pinning ",{"type":44,"tag":67,"props":2484,"children":2486},{"className":2485},[],[2487],{"type":50,"value":2475},{"type":50,"value":942},{"type":44,"tag":59,"props":2490,"children":2491},{},[2492],{"type":50,"value":2493},"effectively required",{"type":50,"value":2495}," for ZeroGPU. The runtime default is currently Python 3.10, so a local environment using 3.11+ will fail to install on the Space without an explicit pin. Pin to a ZeroGPU-supported version (3.12 is a reasonable default); the authoritative supported list lives in the ",{"type":44,"tag":103,"props":2497,"children":2499},{"href":105,"rel":2498},[107],[2500],{"type":50,"value":378},{"type":50,"value":2502}," — do not hardcode the full list, refer to the docs.",{"type":44,"tag":388,"props":2504,"children":2508},{"className":2505,"code":2506,"language":2507,"meta":392,"style":392},"language-yaml shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","# README.md frontmatter\npython_version: \"3.12\"\n","yaml",[2509],{"type":44,"tag":67,"props":2510,"children":2511},{"__ignoreMap":392},[2512,2521],{"type":44,"tag":398,"props":2513,"children":2514},{"class":400,"line":401},[2515],{"type":44,"tag":398,"props":2516,"children":2518},{"style":2517},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[2519],{"type":50,"value":2520},"# README.md frontmatter\n",{"type":44,"tag":398,"props":2522,"children":2523},{"class":400,"line":410},[2524,2529,2533,2538,2543],{"type":44,"tag":398,"props":2525,"children":2527},{"style":2526},"--shiki-light:#E53935;--shiki-default:#F07178;--shiki-dark:#F07178",[2528],{"type":50,"value":2475},{"type":44,"tag":398,"props":2530,"children":2531},{"style":2426},[2532],{"type":50,"value":1370},{"type":44,"tag":398,"props":2534,"children":2535},{"style":2426},[2536],{"type":50,"value":2537}," \"",{"type":44,"tag":398,"props":2539,"children":2540},{"style":2432},[2541],{"type":50,"value":2542},"3.12",{"type":44,"tag":398,"props":2544,"children":2545},{"style":2426},[2546],{"type":50,"value":2547},"\"\n",{"type":44,"tag":53,"props":2549,"children":2550},{},[2551,2553,2559,2561,2567],{"type":50,"value":2552},"Both ",{"type":44,"tag":67,"props":2554,"children":2556},{"className":2555},[],[2557],{"type":50,"value":2558},"\"3.12\"",{"type":50,"value":2560}," and ",{"type":44,"tag":67,"props":2562,"children":2564},{"className":2563},[],[2565],{"type":50,"value":2566},"\"3.12.12\"",{"type":50,"value":2568}," forms are accepted.",{"type":44,"tag":889,"props":2570,"children":2572},{"id":2571},"do-not-pin-spaces-in-requirementstxt",[2573,2575,2580,2581],{"type":50,"value":2574},"Do not pin ",{"type":44,"tag":67,"props":2576,"children":2578},{"className":2577},[],[2579],{"type":50,"value":1040},{"type":50,"value":1059},{"type":44,"tag":67,"props":2582,"children":2584},{"className":2583},[],[2585],{"type":50,"value":1283},{"type":44,"tag":53,"props":2587,"children":2588},{},[2589,2591,2596,2598,2603],{"type":50,"value":2590},"The Space platform pins its own ",{"type":44,"tag":67,"props":2592,"children":2594},{"className":2593},[],[2595],{"type":50,"value":1040},{"type":50,"value":2597}," version. A conflicting pin in ",{"type":44,"tag":67,"props":2599,"children":2601},{"className":2600},[],[2602],{"type":50,"value":1283},{"type":50,"value":2604}," causes pip resolution to fail at build time.",{"type":44,"tag":360,"props":2606,"children":2607},{},[2608],{"type":44,"tag":53,"props":2609,"children":2610},{},[2611,2616,2618,2623,2624,2629],{"type":44,"tag":59,"props":2612,"children":2613},{},[2614],{"type":50,"value":2615},"Rule",{"type":50,"value":2617},": Do not include ",{"type":44,"tag":67,"props":2619,"children":2621},{"className":2620},[],[2622],{"type":50,"value":1040},{"type":50,"value":1059},{"type":44,"tag":67,"props":2625,"children":2627},{"className":2626},[],[2628],{"type":50,"value":1283},{"type":50,"value":972},{"type":44,"tag":53,"props":2631,"children":2632},{},[2633],{"type":50,"value":2634},"How to achieve this depends on your tooling:",{"type":44,"tag":695,"props":2636,"children":2637},{},[2638,2659,2769],{"type":44,"tag":490,"props":2639,"children":2640},{},[2641,2651,2653,2658],{"type":44,"tag":59,"props":2642,"children":2643},{},[2644,2646],{"type":50,"value":2645},"Hand-written ",{"type":44,"tag":67,"props":2647,"children":2649},{"className":2648},[],[2650],{"type":50,"value":1283},{"type":50,"value":2652},": simply omit ",{"type":44,"tag":67,"props":2654,"children":2656},{"className":2655},[],[2657],{"type":50,"value":1040},{"type":50,"value":972},{"type":44,"tag":490,"props":2660,"children":2661},{},[2662,2667,2669,2675,2677,2682,2683,2688,2690,2696,2698,2703,2705,2754,2756,2761,2762,2767],{"type":44,"tag":59,"props":2663,"children":2664},{},[2665],{"type":50,"value":2666},"uv",{"type":50,"value":2668}," (",{"type":44,"tag":67,"props":2670,"children":2672},{"className":2671},[],[2673],{"type":50,"value":2674},"pyproject.toml",{"type":50,"value":2676},"-managed): declare ",{"type":44,"tag":67,"props":2678,"children":2680},{"className":2679},[],[2681],{"type":50,"value":1040},{"type":50,"value":1059},{"type":44,"tag":67,"props":2684,"children":2686},{"className":2685},[],[2687],{"type":50,"value":2674},{"type":50,"value":2689}," so uv co-resolves transitive constraints (notably ",{"type":44,"tag":67,"props":2691,"children":2693},{"className":2692},[],[2694],{"type":50,"value":2695},"psutil",{"type":50,"value":2697},", which ",{"type":44,"tag":67,"props":2699,"children":2701},{"className":2700},[],[2702],{"type":50,"value":1040},{"type":50,"value":2704}," pins), then exclude it from the export:\n",{"type":44,"tag":388,"props":2706,"children":2708},{"className":2408,"code":2707,"language":2410,"meta":392,"style":392},"uv export --no-hashes --no-dev --no-emit-package spaces -o requirements.txt\n",[2709],{"type":44,"tag":67,"props":2710,"children":2711},{"__ignoreMap":392},[2712],{"type":44,"tag":398,"props":2713,"children":2714},{"class":400,"line":401},[2715,2719,2724,2729,2734,2739,2744,2749],{"type":44,"tag":398,"props":2716,"children":2717},{"style":2451},[2718],{"type":50,"value":2666},{"type":44,"tag":398,"props":2720,"children":2721},{"style":2432},[2722],{"type":50,"value":2723}," export",{"type":44,"tag":398,"props":2725,"children":2726},{"style":2432},[2727],{"type":50,"value":2728}," --no-hashes",{"type":44,"tag":398,"props":2730,"children":2731},{"style":2432},[2732],{"type":50,"value":2733}," --no-dev",{"type":44,"tag":398,"props":2735,"children":2736},{"style":2432},[2737],{"type":50,"value":2738}," --no-emit-package",{"type":44,"tag":398,"props":2740,"children":2741},{"style":2432},[2742],{"type":50,"value":2743}," spaces",{"type":44,"tag":398,"props":2745,"children":2746},{"style":2432},[2747],{"type":50,"value":2748}," -o",{"type":44,"tag":398,"props":2750,"children":2751},{"style":2432},[2752],{"type":50,"value":2753}," requirements.txt\n",{"type":50,"value":2755},"\nWithout ",{"type":44,"tag":67,"props":2757,"children":2759},{"className":2758},[],[2760],{"type":50,"value":1040},{"type":50,"value":1059},{"type":44,"tag":67,"props":2763,"children":2765},{"className":2764},[],[2766],{"type":50,"value":2674},{"type":50,"value":2768},", uv cannot see its transitive constraints and may resolve incompatible versions at build time.",{"type":44,"tag":490,"props":2770,"children":2771},{},[2772,2777,2778,2784,2786,2791],{"type":44,"tag":59,"props":2773,"children":2774},{},[2775],{"type":50,"value":2776},"pip-tools",{"type":50,"value":2668},{"type":44,"tag":67,"props":2779,"children":2781},{"className":2780},[],[2782],{"type":50,"value":2783},"pip-compile",{"type":50,"value":2785},") \u002F ",{"type":44,"tag":59,"props":2787,"children":2788},{},[2789],{"type":50,"value":2790},"Poetry",{"type":50,"value":2792},": use the equivalent exclude mechanism.",{"type":44,"tag":889,"props":2794,"children":2796},{"id":2795},"pin-torch-to-match-wheel-tags",[2797,2799,2804],{"type":50,"value":2798},"Pin ",{"type":44,"tag":67,"props":2800,"children":2802},{"className":2801},[],[2803],{"type":50,"value":691},{"type":50,"value":2805}," to match wheel tags",{"type":44,"tag":53,"props":2807,"children":2808},{},[2809,2811,2816,2818,2824,2826,2832,2833,2838,2840,2845,2847,2852],{"type":50,"value":2810},"If you install a CUDA-dependent wheel via direct URL, the wheel filename encodes the ",{"type":44,"tag":67,"props":2812,"children":2814},{"className":2813},[],[2815],{"type":50,"value":691},{"type":50,"value":2817}," major.minor it was built against (e.g. ",{"type":44,"tag":67,"props":2819,"children":2821},{"className":2820},[],[2822],{"type":50,"value":2823},"cu12torch2.8",{"type":50,"value":2825},"). Pin ",{"type":44,"tag":67,"props":2827,"children":2829},{"className":2828},[],[2830],{"type":50,"value":2831},"torch==X.Y.Z",{"type":50,"value":1059},{"type":44,"tag":67,"props":2834,"children":2836},{"className":2835},[],[2837],{"type":50,"value":1283},{"type":50,"value":2839}," to match — otherwise pip may resolve ",{"type":44,"tag":67,"props":2841,"children":2843},{"className":2842},[],[2844],{"type":50,"value":691},{"type":50,"value":2846}," to a different version and the Space fails on first import. Details and the kernels-community alternative are in ",{"type":44,"tag":67,"props":2848,"children":2850},{"className":2849},[],[2851],{"type":50,"value":232},{"type":50,"value":972},{"type":44,"tag":2854,"props":2855,"children":2856},"style",{},[2857],{"type":50,"value":2858},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":2860,"total":2955},[2861,2878,2892,2905,2916,2931,2945],{"slug":2862,"name":2862,"fn":2863,"description":2864,"org":2865,"tags":2866,"stars":27,"repoUrl":28,"updatedAt":2877},"hf-cli","manage Hugging Face Hub resources via CLI","Hugging Face Hub CLI (`hf`) for downloading, uploading, and managing models, datasets, spaces, buckets, repos, papers, jobs, and more on the Hugging Face Hub. Use when: handling authentication; managing local cache; managing Hugging Face Buckets; running or scheduling jobs on Hugging Face infrastructure; managing Hugging Face repos; discussions and pull requests; browsing models, datasets and spaces; reading, searching, or browsing academic papers; managing collections; querying datasets; configuring spaces; setting up webhooks; or deploying and managing HF Inference Endpoints. Make sure to use this skill whenever the user mentions 'hf', 'huggingface', 'Hugging Face', 'huggingface-cli', or 'hugging face cli', or wants to do anything related to the Hugging Face ecosystem and to AI and ML in general. Also use for cloud storage needs like training checkpoints, data pipelines, or agent traces. Use even if the user doesn't explicitly ask for a CLI command. Replaces the deprecated `huggingface-cli`.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2867,2870,2873,2874],{"name":2868,"slug":2869,"type":15},"CLI","cli",{"name":2871,"slug":2872,"type":15},"Datasets","datasets",{"name":9,"slug":17,"type":15},{"name":2875,"slug":2876,"type":15},"LLM","llm","2026-04-06T18:25:34.020855",{"slug":2879,"name":2879,"fn":2880,"description":2881,"org":2882,"tags":2883,"stars":27,"repoUrl":28,"updatedAt":2891},"hf-cloud-aws-context-discovery","discover local AWS environment context","Discover the user's local AWS context (active profile, region, account ID, caller identity) at the start of any AWS task. Use this skill before any other AWS work — deploying to SageMaker, creating resources, calling AWS APIs, or anything that touches an AWS account. Use it especially when the user has not specified a region or profile explicitly, when they say things like \"use my AWS account\", \"deploy to AWS\", \"use my profile\", or when about to make any AWS CLI or SDK call. Never guess the region or account ID — always use this skill to read it from the local configuration first.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2884,2887,2888],{"name":2885,"slug":2886,"type":15},"AWS","aws",{"name":2868,"slug":2869,"type":15},{"name":2889,"slug":2890,"type":15},"Configuration","configuration","2026-07-08T05:55:33.716099",{"slug":2893,"name":2893,"fn":2894,"description":2895,"org":2896,"tags":2897,"stars":27,"repoUrl":28,"updatedAt":2904},"hf-cloud-python-env-setup","set up Python environments for AWS","Set up an isolated Python environment for SageMaker \u002F AWS work, with the right Python version and current boto3. Use this skill whenever Python code will be executed for a SageMaker deployment, training job, or any AWS automation — including when about to run `pip install`, when about to invoke `boto3`, when creating or activating a virtualenv, or when the user asks to \"set up the environment\". Never use system Python and never `pip install` into it. Always isolate. This skill prevents the most common failure modes: wrong Python version, dependency conflicts, and stale SDKs.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2898,2899,2900,2903],{"name":25,"slug":26,"type":15},{"name":2885,"slug":2886,"type":15},{"name":2901,"slug":2902,"type":15},"Engineering","engineering",{"name":22,"slug":23,"type":15},"2026-07-08T05:55:32.505017",{"slug":2906,"name":2906,"fn":2907,"description":2908,"org":2909,"tags":2910,"stars":27,"repoUrl":28,"updatedAt":2915},"hf-cloud-sagemaker-deployment-planner","plan model deployments to Amazon SageMaker","Plan and coordinate the deployment of a model to Amazon SageMaker AI. Use this skill whenever the user wants to deploy, host, serve, or expose a model on SageMaker or AWS — including phrases like \"deploy a model\", \"host this LLM on AWS\", \"serve this embedding model\", \"deploy a reranker\", \"deploy a text-to-image \u002F diffusion model\", \"host this for async inference\", \"create an endpoint\", \"serve my fine-tuned model\", or any request that involves making a model available for inference on AWS. Use this even when the user is vague (e.g. \"I just want to get this running on AWS, you figure it out\"). Works for text-generation LLMs, embedding models, rerankers, classifiers, text-to-image \u002F diffusion models — picks the right serving stack and chooses between real-time and async inference. This is the entry-point skill for SageMaker deployment work — it asks clarifying questions, picks a deployment pathway, and coordinates the other deployment skills.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2911,2912,2913,2914],{"name":25,"slug":26,"type":15},{"name":2885,"slug":2886,"type":15},{"name":19,"slug":20,"type":15},{"name":9,"slug":17,"type":15},"2026-07-08T05:55:37.387689",{"slug":2917,"name":2917,"fn":2918,"description":2919,"org":2920,"tags":2921,"stars":27,"repoUrl":28,"updatedAt":2930},"hf-cloud-sagemaker-iam-preflight","configure SageMaker IAM roles","Ensure a usable SageMaker execution role exists before deploying or training. Use this skill whenever about to create a SageMaker endpoint, model, training job, or any resource that requires an execution role. Use it especially when the user has not provided a role ARN explicitly, when scripts are about to call `iam:CreateRole`, or when an AccessDenied error mentions an IAM action. Never blindly call `iam:CreateRole` — always check for existing roles first. This skill prevents the most common SageMaker deployment failure: trying to create IAM resources from an SSO principal that has no IAM write permissions.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2922,2923,2924,2927],{"name":25,"slug":26,"type":15},{"name":2885,"slug":2886,"type":15},{"name":2925,"slug":2926,"type":15},"Permissions","permissions",{"name":2928,"slug":2929,"type":15},"Security","security","2026-07-08T05:55:34.948771",{"slug":2932,"name":2932,"fn":2933,"description":2934,"org":2935,"tags":2936,"stars":27,"repoUrl":28,"updatedAt":2944},"hf-cloud-sagemaker-production-defaults","create production-ready SageMaker endpoints","Create a SageMaker endpoint (real-time or async) with autoscaling, CloudWatch alarms, and tagging enabled by default. Use this skill whenever about to create a SageMaker endpoint, write deployment code that calls `create_endpoint`, or finalize a deployment after the image URI and IAM role are known. Provides deploy.py for real-time endpoints and deploy_async.py for async endpoints (with genuine scale-to-zero support). This is the last step in the SageMaker deployment workflow. Never generate a bare `create_endpoint` call without these defaults — endpoints without autoscaling or alarms are demos, not deployments.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2937,2938,2939,2940,2941],{"name":25,"slug":26,"type":15},{"name":2885,"slug":2886,"type":15},{"name":19,"slug":20,"type":15},{"name":9,"slug":17,"type":15},{"name":2942,"slug":2943,"type":15},"Monitoring","monitoring","2026-07-08T05:55:38.664702",{"slug":2946,"name":2946,"fn":2947,"description":2948,"org":2949,"tags":2950,"stars":27,"repoUrl":28,"updatedAt":2954},"hf-cloud-serving-image-selection","select SageMaker serving containers","Pick the right serving container for a SageMaker model deployment and find its current image URI. Use this skill whenever about to deploy a model to a SageMaker endpoint and an image URI needs to be chosen — including when the user says \"deploy this LLM\", \"host this HuggingFace model\", \"serve this fine-tuned model\", \"deploy this embedding model\", \"host a reranker\", \"serve a sentence-transformers model\", or when about to hardcode any container URI in deployment code. HuggingFace-curated Deep Learning Containers are ALWAYS preferred: HuggingFace vLLM (LLMs and generative rerankers), HuggingFace vLLM-Omni (multimodal), TEI (embeddings\u002Fcross-encoder rerankers), HF Inference Toolkit (other transformers). Generic images (AWS vLLM, DJL-LMI, SGLang) are used only when no HuggingFace image is compatible — never merely because they carry a newer version. Never hardcode a container URI from memory and never default to TGI. Prevents stale-image failures and wrong-region URIs.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2951,2952,2953],{"name":25,"slug":26,"type":15},{"name":2885,"slug":2886,"type":15},{"name":19,"slug":20,"type":15},"2026-07-08T05:55:36.173465",24,{"items":2957,"total":3080},[2958,2976,2990,2997,3003,3010,3017,3024,3032,3038,3051,3065],{"slug":2959,"name":2959,"fn":2960,"description":2961,"org":2962,"tags":2963,"stars":2973,"repoUrl":2974,"updatedAt":2975},"train-sentence-transformers","train sentence-transformers models","Train or fine-tune sentence-transformers models across `SentenceTransformer` (bi-encoder; dense or static embedding model; for retrieval, similarity, clustering, classification, paraphrase mining, dedup, multimodal), `CrossEncoder` (reranker; pair scoring for two-stage retrieval \u002F pair classification), and `SparseEncoder` (SPLADE, sparse embedding model; for learned-sparse retrieval). Covers loss selection, hard-negative mining, evaluators, distillation, LoRA, Matryoshka, and Hugging Face Hub publishing. Use for any sentence-transformers training task.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2964,2967,2968,2969,2970],{"name":2965,"slug":2966,"type":15},"Deep Learning","deep-learning",{"name":9,"slug":17,"type":15},{"name":2875,"slug":2876,"type":15},{"name":22,"slug":23,"type":15},{"name":2971,"slug":2972,"type":15},"Search","search",18914,"https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fsentence-transformers","2026-05-08T05:09:16.820066",{"slug":2977,"name":2977,"fn":2978,"description":2979,"org":2980,"tags":2981,"stars":2987,"repoUrl":2988,"updatedAt":2989},"trl-training","train and fine-tune LLMs with TRL","Train and fine-tune transformer language models using TRL (Transformers Reinforcement Learning). Supports SFT, DPO, GRPO, KTO, RLOO and Reward Model training via CLI commands.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2982,2983,2984,2985,2986],{"name":25,"slug":26,"type":15},{"name":2965,"slug":2966,"type":15},{"name":9,"slug":17,"type":15},{"name":2875,"slug":2876,"type":15},{"name":22,"slug":23,"type":15},18850,"https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftrl","2026-04-06T18:25:32.746828",{"slug":2862,"name":2862,"fn":2863,"description":2864,"org":2991,"tags":2992,"stars":27,"repoUrl":28,"updatedAt":2877},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2993,2994,2995,2996],{"name":2868,"slug":2869,"type":15},{"name":2871,"slug":2872,"type":15},{"name":9,"slug":17,"type":15},{"name":2875,"slug":2876,"type":15},{"slug":2879,"name":2879,"fn":2880,"description":2881,"org":2998,"tags":2999,"stars":27,"repoUrl":28,"updatedAt":2891},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3000,3001,3002],{"name":2885,"slug":2886,"type":15},{"name":2868,"slug":2869,"type":15},{"name":2889,"slug":2890,"type":15},{"slug":2893,"name":2893,"fn":2894,"description":2895,"org":3004,"tags":3005,"stars":27,"repoUrl":28,"updatedAt":2904},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3006,3007,3008,3009],{"name":25,"slug":26,"type":15},{"name":2885,"slug":2886,"type":15},{"name":2901,"slug":2902,"type":15},{"name":22,"slug":23,"type":15},{"slug":2906,"name":2906,"fn":2907,"description":2908,"org":3011,"tags":3012,"stars":27,"repoUrl":28,"updatedAt":2915},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3013,3014,3015,3016],{"name":25,"slug":26,"type":15},{"name":2885,"slug":2886,"type":15},{"name":19,"slug":20,"type":15},{"name":9,"slug":17,"type":15},{"slug":2917,"name":2917,"fn":2918,"description":2919,"org":3018,"tags":3019,"stars":27,"repoUrl":28,"updatedAt":2930},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3020,3021,3022,3023],{"name":25,"slug":26,"type":15},{"name":2885,"slug":2886,"type":15},{"name":2925,"slug":2926,"type":15},{"name":2928,"slug":2929,"type":15},{"slug":2932,"name":2932,"fn":2933,"description":2934,"org":3025,"tags":3026,"stars":27,"repoUrl":28,"updatedAt":2944},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3027,3028,3029,3030,3031],{"name":25,"slug":26,"type":15},{"name":2885,"slug":2886,"type":15},{"name":19,"slug":20,"type":15},{"name":9,"slug":17,"type":15},{"name":2942,"slug":2943,"type":15},{"slug":2946,"name":2946,"fn":2947,"description":2948,"org":3033,"tags":3034,"stars":27,"repoUrl":28,"updatedAt":2954},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3035,3036,3037],{"name":25,"slug":26,"type":15},{"name":2885,"slug":2886,"type":15},{"name":19,"slug":20,"type":15},{"slug":3039,"name":3039,"fn":3040,"description":3041,"org":3042,"tags":3043,"stars":27,"repoUrl":28,"updatedAt":3050},"hf-mcp","access Hugging Face Hub via MCP","Use Hugging Face Hub via MCP server tools. Search models, datasets, Spaces, papers. Get repo details, fetch documentation, run compute jobs, and use Gradio Spaces as AI tools. Available when connected to the HF MCP server.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3044,3045,3046,3047],{"name":2871,"slug":2872,"type":15},{"name":9,"slug":17,"type":15},{"name":2875,"slug":2876,"type":15},{"name":3048,"slug":3049,"type":15},"MCP","mcp","2026-04-06T18:25:50.364185",{"slug":3052,"name":3052,"fn":3053,"description":3054,"org":3055,"tags":3056,"stars":27,"repoUrl":28,"updatedAt":3064},"hf-mem","estimate memory for Hugging Face models","Hugging Face CLI to estimate the required memory to load Safetensors or GGUF model weights for inference from the Hugging Face Hub",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3057,3058,3059,3060,3061],{"name":25,"slug":26,"type":15},{"name":2868,"slug":2869,"type":15},{"name":9,"slug":17,"type":15},{"name":2875,"slug":2876,"type":15},{"name":3062,"slug":3063,"type":15},"Performance","performance","2026-06-13T07:23:57.101435",{"slug":3066,"name":3066,"fn":3067,"description":3068,"org":3069,"tags":3070,"stars":27,"repoUrl":28,"updatedAt":3079},"huggingface-best","find and compare Hugging Face models","Use when the user asks about finding the best, top, or recommended model for a task, wants to know what AI model to use, or wants to compare models by benchmark scores. Triggers on: \"best model for X\", \"what model should I use for\", \"top models for [task]\", \"which model runs on my laptop\u002Fmachine\u002Fdevice\", \"recommend a model for\", \"what LLM should I use for\", \"compare models for\", \"what's state of the art for\", or any question about choosing an AI model for a specific use case. Always use this skill when the user wants model recommendations or comparisons, even if they don't explicitly mention HuggingFace or benchmarks.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3071,3074,3075,3076],{"name":3072,"slug":3073,"type":15},"Analytics","analytics",{"name":9,"slug":17,"type":15},{"name":2875,"slug":2876,"type":15},{"name":3077,"slug":3078,"type":15},"Research","research","2026-04-24T05:09:45.870658",37]