[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-openai-huggingface-community-evals":3,"mdc-tmxg3d-key":39,"related-org-openai-huggingface-community-evals":1803,"related-repo-openai-huggingface-community-evals":2010},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":28,"repoUrl":29,"updatedAt":30,"license":31,"forks":32,"topics":33,"repo":34,"sourceUrl":37,"mdContent":38},"huggingface-community-evals","run evaluations for Hugging Face Hub models","Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM \u002F Transformers \u002F accelerate. Not for HF Jobs orchestration, model-card PRs, .eval_results publication, or community-evals automation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"openai","OpenAI","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fopenai.png",[12,16,19,22,25],{"name":13,"slug":14,"type":15},"Hugging Face","hugging-face","tag",{"name":17,"slug":18,"type":15},"LLM","llm",{"name":20,"slug":21,"type":15},"Deep Learning","deep-learning",{"name":23,"slug":24,"type":15},"Evals","evals",{"name":26,"slug":27,"type":15},"Testing","testing",3992,"https:\u002F\u002Fgithub.com\u002Fopenai\u002Fplugins","2026-04-06T18:41:50.46837",null,465,[],{"repoUrl":29,"stars":28,"forks":32,"topics":35,"description":36},[],"OpenAI Plugins","https:\u002F\u002Fgithub.com\u002Fopenai\u002Fplugins\u002Ftree\u002FHEAD\u002Fplugins\u002Fhugging-face\u002Fskills\u002Fcommunity-evals","---\nname: huggingface-community-evals\ndescription: Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM \u002F Transformers \u002F accelerate. Not for HF Jobs orchestration, model-card PRs, .eval_results publication, or community-evals automation.\n---\n\n# Overview\n\nThis skill is for **running evaluations against models on the Hugging Face Hub on local hardware**.\n\nIt covers:\n- `inspect-ai` with local inference\n- `lighteval` with local inference\n- choosing between `vllm`, Hugging Face Transformers, and `accelerate`\n- smoke tests, task selection, and backend fallback strategy\n\nIt does **not** cover:\n- Hugging Face Jobs orchestration\n- model-card or `model-index` edits\n- README table extraction\n- Artificial Analysis imports\n- `.eval_results` generation or publishing\n- PR creation or community-evals automation\n\nIf the user wants to **run the same eval remotely on Hugging Face Jobs**, hand off to the `hugging-face-jobs` skill and pass it one of the local scripts in this skill.\n\nIf the user wants to **publish results into the community evals workflow**, stop after generating the evaluation run and hand off that publishing step to `~\u002Fcode\u002Fcommunity-evals`.\n\n> All paths below are relative to the directory containing this `SKILL.md`.\n\n# When To Use Which Script\n\n| Use case | Script |\n|---|---|\n| Local `inspect-ai` eval on a Hub model via inference providers | `scripts\u002Finspect_eval_uv.py` |\n| Local GPU eval with `inspect-ai` using `vllm` or Transformers | `scripts\u002Finspect_vllm_uv.py` |\n| Local GPU eval with `lighteval` using `vllm` or `accelerate` | `scripts\u002Flighteval_vllm_uv.py` |\n| Extra command patterns | `examples\u002FUSAGE_EXAMPLES.md` |\n\n# Prerequisites\n\n- Prefer `uv run` for local execution.\n- Set `HF_TOKEN` for gated\u002Fprivate models.\n- For local GPU runs, verify GPU access before starting:\n\n```bash\nuv --version\nprintenv HF_TOKEN >\u002Fdev\u002Fnull\nnvidia-smi\n```\n\nIf `nvidia-smi` is unavailable, either:\n- use `scripts\u002Finspect_eval_uv.py` for lighter provider-backed evaluation, or\n- hand off to the `hugging-face-jobs` skill if the user wants remote compute.\n\n# Core Workflow\n\n1. Choose the evaluation framework.\n   - Use `inspect-ai` when you want explicit task control and inspect-native flows.\n   - Use `lighteval` when the benchmark is naturally expressed as a lighteval task string, especially leaderboard-style tasks.\n2. Choose the inference backend.\n   - Prefer `vllm` for throughput on supported architectures.\n   - Use Hugging Face Transformers (`--backend hf`) or `accelerate` as compatibility fallbacks.\n3. Start with a smoke test.\n   - `inspect-ai`: add `--limit 10` or similar.\n   - `lighteval`: add `--max-samples 10`.\n4. Scale up only after the smoke test passes.\n5. If the user wants remote execution, hand off to `hugging-face-jobs` with the same script + args.\n\n# Quick Start\n\n## Option A: inspect-ai with local inference providers path\n\nBest when the model is already supported by Hugging Face Inference Providers and you want the lowest local setup overhead.\n\n```bash\nuv run scripts\u002Finspect_eval_uv.py \\\n  --model meta-llama\u002FLlama-3.2-1B \\\n  --task mmlu \\\n  --limit 20\n```\n\nUse this path when:\n- you want a quick local smoke test\n- you do not need direct GPU control\n- the task already exists in `inspect-evals`\n\n## Option B: inspect-ai on Local GPU\n\nBest when you need to load the Hub model directly, use `vllm`, or fall back to Transformers for unsupported architectures.\n\nLocal GPU:\n\n```bash\nuv run scripts\u002Finspect_vllm_uv.py \\\n  --model meta-llama\u002FLlama-3.2-1B \\\n  --task gsm8k \\\n  --limit 20\n```\n\nTransformers fallback:\n\n```bash\nuv run scripts\u002Finspect_vllm_uv.py \\\n  --model microsoft\u002Fphi-2 \\\n  --task mmlu \\\n  --backend hf \\\n  --trust-remote-code \\\n  --limit 20\n```\n\n## Option C: lighteval on Local GPU\n\nBest when the task is naturally expressed as a `lighteval` task string, especially Open LLM Leaderboard style benchmarks.\n\nLocal GPU:\n\n```bash\nuv run scripts\u002Flighteval_vllm_uv.py \\\n  --model meta-llama\u002FLlama-3.2-3B-Instruct \\\n  --tasks \"leaderboard|mmlu|5,leaderboard|gsm8k|5\" \\\n  --max-samples 20 \\\n  --use-chat-template\n```\n\n`accelerate` fallback:\n\n```bash\nuv run scripts\u002Flighteval_vllm_uv.py \\\n  --model microsoft\u002Fphi-2 \\\n  --tasks \"leaderboard|mmlu|5\" \\\n  --backend accelerate \\\n  --trust-remote-code \\\n  --max-samples 20\n```\n\n# Remote Execution Boundary\n\nThis skill intentionally stops at **local execution and backend selection**.\n\nIf the user wants to:\n- run these scripts on Hugging Face Jobs\n- pick remote hardware\n- pass secrets to remote jobs\n- schedule recurring runs\n- inspect \u002F cancel \u002F monitor jobs\n\nthen switch to the **`hugging-face-jobs`** skill and pass it one of these scripts plus the chosen arguments.\n\n# Task Selection\n\n`inspect-ai` examples:\n- `mmlu`\n- `gsm8k`\n- `hellaswag`\n- `arc_challenge`\n- `truthfulqa`\n- `winogrande`\n- `humaneval`\n\n`lighteval` task strings use `suite|task|num_fewshot`:\n- `leaderboard|mmlu|5`\n- `leaderboard|gsm8k|5`\n- `leaderboard|arc_challenge|25`\n- `lighteval|hellaswag|0`\n\nMultiple `lighteval` tasks can be comma-separated in `--tasks`.\n\n# Backend Selection\n\n- Prefer `inspect_vllm_uv.py --backend vllm` for fast GPU inference on supported architectures.\n- Use `inspect_vllm_uv.py --backend hf` when `vllm` does not support the model.\n- Prefer `lighteval_vllm_uv.py --backend vllm` for throughput on supported models.\n- Use `lighteval_vllm_uv.py --backend accelerate` as the compatibility fallback.\n- Use `inspect_eval_uv.py` when Inference Providers already cover the model and you do not need direct GPU control.\n\n# Hardware Guidance\n\n| Model size | Suggested local hardware |\n|---|---|\n| `\u003C 3B` | consumer GPU \u002F Apple Silicon \u002F small dev GPU |\n| `3B - 13B` | stronger local GPU |\n| `13B+` | high-memory local GPU or hand off to `hugging-face-jobs` |\n\nFor smoke tests, prefer cheaper local runs plus `--limit` or `--max-samples`.\n\n# Troubleshooting\n\n- CUDA or vLLM OOM:\n  - reduce `--batch-size`\n  - reduce `--gpu-memory-utilization`\n  - switch to a smaller model for the smoke test\n  - if necessary, hand off to `hugging-face-jobs`\n- Model unsupported by `vllm`:\n  - switch to `--backend hf` for `inspect-ai`\n  - switch to `--backend accelerate` for `lighteval`\n- Gated\u002Fprivate repo access fails:\n  - verify `HF_TOKEN`\n- Custom model code required:\n  - add `--trust-remote-code`\n\n# Examples\n\nSee:\n- `examples\u002FUSAGE_EXAMPLES.md` for local command patterns\n- `scripts\u002Finspect_eval_uv.py`\n- `scripts\u002Finspect_vllm_uv.py`\n- `scripts\u002Flighteval_vllm_uv.py`\n",{"data":40,"body":41},{"name":4,"description":6},{"type":42,"children":43},"root",[44,53,67,72,123,135,182,202,220,236,242,376,382,416,479,492,519,525,658,664,671,676,755,760,784,790,802,807,876,881,980,986,998,1002,1097,1107,1210,1216,1227,1232,1260,1275,1281,1291,1357,1375,1413,1432,1438,1508,1514,1592,1611,1617,1749,1755,1760,1797],{"type":45,"tag":46,"props":47,"children":49},"element","h1",{"id":48},"overview",[50],{"type":51,"value":52},"text","Overview",{"type":45,"tag":54,"props":55,"children":56},"p",{},[57,59,65],{"type":51,"value":58},"This skill is for ",{"type":45,"tag":60,"props":61,"children":62},"strong",{},[63],{"type":51,"value":64},"running evaluations against models on the Hugging Face Hub on local hardware",{"type":51,"value":66},".",{"type":45,"tag":54,"props":68,"children":69},{},[70],{"type":51,"value":71},"It covers:",{"type":45,"tag":73,"props":74,"children":75},"ul",{},[76,89,99,118],{"type":45,"tag":77,"props":78,"children":79},"li",{},[80,87],{"type":45,"tag":81,"props":82,"children":84},"code",{"className":83},[],[85],{"type":51,"value":86},"inspect-ai",{"type":51,"value":88}," with local inference",{"type":45,"tag":77,"props":90,"children":91},{},[92,98],{"type":45,"tag":81,"props":93,"children":95},{"className":94},[],[96],{"type":51,"value":97},"lighteval",{"type":51,"value":88},{"type":45,"tag":77,"props":100,"children":101},{},[102,104,110,112],{"type":51,"value":103},"choosing between ",{"type":45,"tag":81,"props":105,"children":107},{"className":106},[],[108],{"type":51,"value":109},"vllm",{"type":51,"value":111},", Hugging Face Transformers, and ",{"type":45,"tag":81,"props":113,"children":115},{"className":114},[],[116],{"type":51,"value":117},"accelerate",{"type":45,"tag":77,"props":119,"children":120},{},[121],{"type":51,"value":122},"smoke tests, task selection, and backend fallback strategy",{"type":45,"tag":54,"props":124,"children":125},{},[126,128,133],{"type":51,"value":127},"It does ",{"type":45,"tag":60,"props":129,"children":130},{},[131],{"type":51,"value":132},"not",{"type":51,"value":134}," cover:",{"type":45,"tag":73,"props":136,"children":137},{},[138,143,156,161,166,177],{"type":45,"tag":77,"props":139,"children":140},{},[141],{"type":51,"value":142},"Hugging Face Jobs orchestration",{"type":45,"tag":77,"props":144,"children":145},{},[146,148,154],{"type":51,"value":147},"model-card or ",{"type":45,"tag":81,"props":149,"children":151},{"className":150},[],[152],{"type":51,"value":153},"model-index",{"type":51,"value":155}," edits",{"type":45,"tag":77,"props":157,"children":158},{},[159],{"type":51,"value":160},"README table extraction",{"type":45,"tag":77,"props":162,"children":163},{},[164],{"type":51,"value":165},"Artificial Analysis imports",{"type":45,"tag":77,"props":167,"children":168},{},[169,175],{"type":45,"tag":81,"props":170,"children":172},{"className":171},[],[173],{"type":51,"value":174},".eval_results",{"type":51,"value":176}," generation or publishing",{"type":45,"tag":77,"props":178,"children":179},{},[180],{"type":51,"value":181},"PR creation or community-evals automation",{"type":45,"tag":54,"props":183,"children":184},{},[185,187,192,194,200],{"type":51,"value":186},"If the user wants to ",{"type":45,"tag":60,"props":188,"children":189},{},[190],{"type":51,"value":191},"run the same eval remotely on Hugging Face Jobs",{"type":51,"value":193},", hand off to the ",{"type":45,"tag":81,"props":195,"children":197},{"className":196},[],[198],{"type":51,"value":199},"hugging-face-jobs",{"type":51,"value":201}," skill and pass it one of the local scripts in this skill.",{"type":45,"tag":54,"props":203,"children":204},{},[205,206,211,213,219],{"type":51,"value":186},{"type":45,"tag":60,"props":207,"children":208},{},[209],{"type":51,"value":210},"publish results into the community evals workflow",{"type":51,"value":212},", stop after generating the evaluation run and hand off that publishing step to ",{"type":45,"tag":81,"props":214,"children":216},{"className":215},[],[217],{"type":51,"value":218},"~\u002Fcode\u002Fcommunity-evals",{"type":51,"value":66},{"type":45,"tag":221,"props":222,"children":223},"blockquote",{},[224],{"type":45,"tag":54,"props":225,"children":226},{},[227,229,235],{"type":51,"value":228},"All paths below are relative to the directory containing this ",{"type":45,"tag":81,"props":230,"children":232},{"className":231},[],[233],{"type":51,"value":234},"SKILL.md",{"type":51,"value":66},{"type":45,"tag":46,"props":237,"children":239},{"id":238},"when-to-use-which-script",[240],{"type":51,"value":241},"When To Use Which Script",{"type":45,"tag":243,"props":244,"children":245},"table",{},[246,265],{"type":45,"tag":247,"props":248,"children":249},"thead",{},[250],{"type":45,"tag":251,"props":252,"children":253},"tr",{},[254,260],{"type":45,"tag":255,"props":256,"children":257},"th",{},[258],{"type":51,"value":259},"Use case",{"type":45,"tag":255,"props":261,"children":262},{},[263],{"type":51,"value":264},"Script",{"type":45,"tag":266,"props":267,"children":268},"tbody",{},[269,294,325,359],{"type":45,"tag":251,"props":270,"children":271},{},[272,285],{"type":45,"tag":273,"props":274,"children":275},"td",{},[276,278,283],{"type":51,"value":277},"Local ",{"type":45,"tag":81,"props":279,"children":281},{"className":280},[],[282],{"type":51,"value":86},{"type":51,"value":284}," eval on a Hub model via inference providers",{"type":45,"tag":273,"props":286,"children":287},{},[288],{"type":45,"tag":81,"props":289,"children":291},{"className":290},[],[292],{"type":51,"value":293},"scripts\u002Finspect_eval_uv.py",{"type":45,"tag":251,"props":295,"children":296},{},[297,316],{"type":45,"tag":273,"props":298,"children":299},{},[300,302,307,309,314],{"type":51,"value":301},"Local GPU eval with ",{"type":45,"tag":81,"props":303,"children":305},{"className":304},[],[306],{"type":51,"value":86},{"type":51,"value":308}," using ",{"type":45,"tag":81,"props":310,"children":312},{"className":311},[],[313],{"type":51,"value":109},{"type":51,"value":315}," or Transformers",{"type":45,"tag":273,"props":317,"children":318},{},[319],{"type":45,"tag":81,"props":320,"children":322},{"className":321},[],[323],{"type":51,"value":324},"scripts\u002Finspect_vllm_uv.py",{"type":45,"tag":251,"props":326,"children":327},{},[328,350],{"type":45,"tag":273,"props":329,"children":330},{},[331,332,337,338,343,345],{"type":51,"value":301},{"type":45,"tag":81,"props":333,"children":335},{"className":334},[],[336],{"type":51,"value":97},{"type":51,"value":308},{"type":45,"tag":81,"props":339,"children":341},{"className":340},[],[342],{"type":51,"value":109},{"type":51,"value":344}," or ",{"type":45,"tag":81,"props":346,"children":348},{"className":347},[],[349],{"type":51,"value":117},{"type":45,"tag":273,"props":351,"children":352},{},[353],{"type":45,"tag":81,"props":354,"children":356},{"className":355},[],[357],{"type":51,"value":358},"scripts\u002Flighteval_vllm_uv.py",{"type":45,"tag":251,"props":360,"children":361},{},[362,367],{"type":45,"tag":273,"props":363,"children":364},{},[365],{"type":51,"value":366},"Extra command patterns",{"type":45,"tag":273,"props":368,"children":369},{},[370],{"type":45,"tag":81,"props":371,"children":373},{"className":372},[],[374],{"type":51,"value":375},"examples\u002FUSAGE_EXAMPLES.md",{"type":45,"tag":46,"props":377,"children":379},{"id":378},"prerequisites",[380],{"type":51,"value":381},"Prerequisites",{"type":45,"tag":73,"props":383,"children":384},{},[385,398,411],{"type":45,"tag":77,"props":386,"children":387},{},[388,390,396],{"type":51,"value":389},"Prefer ",{"type":45,"tag":81,"props":391,"children":393},{"className":392},[],[394],{"type":51,"value":395},"uv run",{"type":51,"value":397}," for local execution.",{"type":45,"tag":77,"props":399,"children":400},{},[401,403,409],{"type":51,"value":402},"Set ",{"type":45,"tag":81,"props":404,"children":406},{"className":405},[],[407],{"type":51,"value":408},"HF_TOKEN",{"type":51,"value":410}," for gated\u002Fprivate models.",{"type":45,"tag":77,"props":412,"children":413},{},[414],{"type":51,"value":415},"For local GPU runs, verify GPU access before starting:",{"type":45,"tag":417,"props":418,"children":423},"pre",{"className":419,"code":420,"language":421,"meta":422,"style":422},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","uv --version\nprintenv HF_TOKEN >\u002Fdev\u002Fnull\nnvidia-smi\n","bash","",[424],{"type":45,"tag":81,"props":425,"children":426},{"__ignoreMap":422},[427,445,470],{"type":45,"tag":428,"props":429,"children":432},"span",{"class":430,"line":431},"line",1,[433,439],{"type":45,"tag":428,"props":434,"children":436},{"style":435},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[437],{"type":51,"value":438},"uv",{"type":45,"tag":428,"props":440,"children":442},{"style":441},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[443],{"type":51,"value":444}," --version\n",{"type":45,"tag":428,"props":446,"children":448},{"class":430,"line":447},2,[449,454,459,465],{"type":45,"tag":428,"props":450,"children":451},{"style":435},[452],{"type":51,"value":453},"printenv",{"type":45,"tag":428,"props":455,"children":456},{"style":441},[457],{"type":51,"value":458}," HF_TOKEN",{"type":45,"tag":428,"props":460,"children":462},{"style":461},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[463],{"type":51,"value":464}," >",{"type":45,"tag":428,"props":466,"children":467},{"style":441},[468],{"type":51,"value":469},"\u002Fdev\u002Fnull\n",{"type":45,"tag":428,"props":471,"children":473},{"class":430,"line":472},3,[474],{"type":45,"tag":428,"props":475,"children":476},{"style":435},[477],{"type":51,"value":478},"nvidia-smi\n",{"type":45,"tag":54,"props":480,"children":481},{},[482,484,490],{"type":51,"value":483},"If ",{"type":45,"tag":81,"props":485,"children":487},{"className":486},[],[488],{"type":51,"value":489},"nvidia-smi",{"type":51,"value":491}," is unavailable, either:",{"type":45,"tag":73,"props":493,"children":494},{},[495,507],{"type":45,"tag":77,"props":496,"children":497},{},[498,500,505],{"type":51,"value":499},"use ",{"type":45,"tag":81,"props":501,"children":503},{"className":502},[],[504],{"type":51,"value":293},{"type":51,"value":506}," for lighter provider-backed evaluation, or",{"type":45,"tag":77,"props":508,"children":509},{},[510,512,517],{"type":51,"value":511},"hand off to the ",{"type":45,"tag":81,"props":513,"children":515},{"className":514},[],[516],{"type":51,"value":199},{"type":51,"value":518}," skill if the user wants remote compute.",{"type":45,"tag":46,"props":520,"children":522},{"id":521},"core-workflow",[523],{"type":51,"value":524},"Core Workflow",{"type":45,"tag":526,"props":527,"children":528},"ol",{},[529,560,599,641,646],{"type":45,"tag":77,"props":530,"children":531},{},[532,534],{"type":51,"value":533},"Choose the evaluation framework.\n",{"type":45,"tag":73,"props":535,"children":536},{},[537,549],{"type":45,"tag":77,"props":538,"children":539},{},[540,542,547],{"type":51,"value":541},"Use ",{"type":45,"tag":81,"props":543,"children":545},{"className":544},[],[546],{"type":51,"value":86},{"type":51,"value":548}," when you want explicit task control and inspect-native flows.",{"type":45,"tag":77,"props":550,"children":551},{},[552,553,558],{"type":51,"value":541},{"type":45,"tag":81,"props":554,"children":556},{"className":555},[],[557],{"type":51,"value":97},{"type":51,"value":559}," when the benchmark is naturally expressed as a lighteval task string, especially leaderboard-style tasks.",{"type":45,"tag":77,"props":561,"children":562},{},[563,565],{"type":51,"value":564},"Choose the inference backend.\n",{"type":45,"tag":73,"props":566,"children":567},{},[568,579],{"type":45,"tag":77,"props":569,"children":570},{},[571,572,577],{"type":51,"value":389},{"type":45,"tag":81,"props":573,"children":575},{"className":574},[],[576],{"type":51,"value":109},{"type":51,"value":578}," for throughput on supported architectures.",{"type":45,"tag":77,"props":580,"children":581},{},[582,584,590,592,597],{"type":51,"value":583},"Use Hugging Face Transformers (",{"type":45,"tag":81,"props":585,"children":587},{"className":586},[],[588],{"type":51,"value":589},"--backend hf",{"type":51,"value":591},") or ",{"type":45,"tag":81,"props":593,"children":595},{"className":594},[],[596],{"type":51,"value":117},{"type":51,"value":598}," as compatibility fallbacks.",{"type":45,"tag":77,"props":600,"children":601},{},[602,604],{"type":51,"value":603},"Start with a smoke test.\n",{"type":45,"tag":73,"props":605,"children":606},{},[607,625],{"type":45,"tag":77,"props":608,"children":609},{},[610,615,617,623],{"type":45,"tag":81,"props":611,"children":613},{"className":612},[],[614],{"type":51,"value":86},{"type":51,"value":616},": add ",{"type":45,"tag":81,"props":618,"children":620},{"className":619},[],[621],{"type":51,"value":622},"--limit 10",{"type":51,"value":624}," or similar.",{"type":45,"tag":77,"props":626,"children":627},{},[628,633,634,640],{"type":45,"tag":81,"props":629,"children":631},{"className":630},[],[632],{"type":51,"value":97},{"type":51,"value":616},{"type":45,"tag":81,"props":635,"children":637},{"className":636},[],[638],{"type":51,"value":639},"--max-samples 10",{"type":51,"value":66},{"type":45,"tag":77,"props":642,"children":643},{},[644],{"type":51,"value":645},"Scale up only after the smoke test passes.",{"type":45,"tag":77,"props":647,"children":648},{},[649,651,656],{"type":51,"value":650},"If the user wants remote execution, hand off to ",{"type":45,"tag":81,"props":652,"children":654},{"className":653},[],[655],{"type":51,"value":199},{"type":51,"value":657}," with the same script + args.",{"type":45,"tag":46,"props":659,"children":661},{"id":660},"quick-start",[662],{"type":51,"value":663},"Quick Start",{"type":45,"tag":665,"props":666,"children":668},"h2",{"id":667},"option-a-inspect-ai-with-local-inference-providers-path",[669],{"type":51,"value":670},"Option A: inspect-ai with local inference providers path",{"type":45,"tag":54,"props":672,"children":673},{},[674],{"type":51,"value":675},"Best when the model is already supported by Hugging Face Inference Providers and you want the lowest local setup overhead.",{"type":45,"tag":417,"props":677,"children":679},{"className":419,"code":678,"language":421,"meta":422,"style":422},"uv run scripts\u002Finspect_eval_uv.py \\\n  --model meta-llama\u002FLlama-3.2-1B \\\n  --task mmlu \\\n  --limit 20\n",[680],{"type":45,"tag":81,"props":681,"children":682},{"__ignoreMap":422},[683,706,723,740],{"type":45,"tag":428,"props":684,"children":685},{"class":430,"line":431},[686,690,695,700],{"type":45,"tag":428,"props":687,"children":688},{"style":435},[689],{"type":51,"value":438},{"type":45,"tag":428,"props":691,"children":692},{"style":441},[693],{"type":51,"value":694}," run",{"type":45,"tag":428,"props":696,"children":697},{"style":441},[698],{"type":51,"value":699}," scripts\u002Finspect_eval_uv.py",{"type":45,"tag":428,"props":701,"children":703},{"style":702},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[704],{"type":51,"value":705}," \\\n",{"type":45,"tag":428,"props":707,"children":708},{"class":430,"line":447},[709,714,719],{"type":45,"tag":428,"props":710,"children":711},{"style":441},[712],{"type":51,"value":713},"  --model",{"type":45,"tag":428,"props":715,"children":716},{"style":441},[717],{"type":51,"value":718}," meta-llama\u002FLlama-3.2-1B",{"type":45,"tag":428,"props":720,"children":721},{"style":702},[722],{"type":51,"value":705},{"type":45,"tag":428,"props":724,"children":725},{"class":430,"line":472},[726,731,736],{"type":45,"tag":428,"props":727,"children":728},{"style":441},[729],{"type":51,"value":730},"  --task",{"type":45,"tag":428,"props":732,"children":733},{"style":441},[734],{"type":51,"value":735}," mmlu",{"type":45,"tag":428,"props":737,"children":738},{"style":702},[739],{"type":51,"value":705},{"type":45,"tag":428,"props":741,"children":743},{"class":430,"line":742},4,[744,749],{"type":45,"tag":428,"props":745,"children":746},{"style":441},[747],{"type":51,"value":748},"  --limit",{"type":45,"tag":428,"props":750,"children":752},{"style":751},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[753],{"type":51,"value":754}," 20\n",{"type":45,"tag":54,"props":756,"children":757},{},[758],{"type":51,"value":759},"Use this path when:",{"type":45,"tag":73,"props":761,"children":762},{},[763,768,773],{"type":45,"tag":77,"props":764,"children":765},{},[766],{"type":51,"value":767},"you want a quick local smoke test",{"type":45,"tag":77,"props":769,"children":770},{},[771],{"type":51,"value":772},"you do not need direct GPU control",{"type":45,"tag":77,"props":774,"children":775},{},[776,778],{"type":51,"value":777},"the task already exists in ",{"type":45,"tag":81,"props":779,"children":781},{"className":780},[],[782],{"type":51,"value":783},"inspect-evals",{"type":45,"tag":665,"props":785,"children":787},{"id":786},"option-b-inspect-ai-on-local-gpu",[788],{"type":51,"value":789},"Option B: inspect-ai on Local GPU",{"type":45,"tag":54,"props":791,"children":792},{},[793,795,800],{"type":51,"value":794},"Best when you need to load the Hub model directly, use ",{"type":45,"tag":81,"props":796,"children":798},{"className":797},[],[799],{"type":51,"value":109},{"type":51,"value":801},", or fall back to Transformers for unsupported architectures.",{"type":45,"tag":54,"props":803,"children":804},{},[805],{"type":51,"value":806},"Local GPU:",{"type":45,"tag":417,"props":808,"children":810},{"className":419,"code":809,"language":421,"meta":422,"style":422},"uv run scripts\u002Finspect_vllm_uv.py \\\n  --model meta-llama\u002FLlama-3.2-1B \\\n  --task gsm8k \\\n  --limit 20\n",[811],{"type":45,"tag":81,"props":812,"children":813},{"__ignoreMap":422},[814,834,849,865],{"type":45,"tag":428,"props":815,"children":816},{"class":430,"line":431},[817,821,825,830],{"type":45,"tag":428,"props":818,"children":819},{"style":435},[820],{"type":51,"value":438},{"type":45,"tag":428,"props":822,"children":823},{"style":441},[824],{"type":51,"value":694},{"type":45,"tag":428,"props":826,"children":827},{"style":441},[828],{"type":51,"value":829}," scripts\u002Finspect_vllm_uv.py",{"type":45,"tag":428,"props":831,"children":832},{"style":702},[833],{"type":51,"value":705},{"type":45,"tag":428,"props":835,"children":836},{"class":430,"line":447},[837,841,845],{"type":45,"tag":428,"props":838,"children":839},{"style":441},[840],{"type":51,"value":713},{"type":45,"tag":428,"props":842,"children":843},{"style":441},[844],{"type":51,"value":718},{"type":45,"tag":428,"props":846,"children":847},{"style":702},[848],{"type":51,"value":705},{"type":45,"tag":428,"props":850,"children":851},{"class":430,"line":472},[852,856,861],{"type":45,"tag":428,"props":853,"children":854},{"style":441},[855],{"type":51,"value":730},{"type":45,"tag":428,"props":857,"children":858},{"style":441},[859],{"type":51,"value":860}," gsm8k",{"type":45,"tag":428,"props":862,"children":863},{"style":702},[864],{"type":51,"value":705},{"type":45,"tag":428,"props":866,"children":867},{"class":430,"line":742},[868,872],{"type":45,"tag":428,"props":869,"children":870},{"style":441},[871],{"type":51,"value":748},{"type":45,"tag":428,"props":873,"children":874},{"style":751},[875],{"type":51,"value":754},{"type":45,"tag":54,"props":877,"children":878},{},[879],{"type":51,"value":880},"Transformers fallback:",{"type":45,"tag":417,"props":882,"children":884},{"className":419,"code":883,"language":421,"meta":422,"style":422},"uv run scripts\u002Finspect_vllm_uv.py \\\n  --model microsoft\u002Fphi-2 \\\n  --task mmlu \\\n  --backend hf \\\n  --trust-remote-code \\\n  --limit 20\n",[885],{"type":45,"tag":81,"props":886,"children":887},{"__ignoreMap":422},[888,907,923,938,955,968],{"type":45,"tag":428,"props":889,"children":890},{"class":430,"line":431},[891,895,899,903],{"type":45,"tag":428,"props":892,"children":893},{"style":435},[894],{"type":51,"value":438},{"type":45,"tag":428,"props":896,"children":897},{"style":441},[898],{"type":51,"value":694},{"type":45,"tag":428,"props":900,"children":901},{"style":441},[902],{"type":51,"value":829},{"type":45,"tag":428,"props":904,"children":905},{"style":702},[906],{"type":51,"value":705},{"type":45,"tag":428,"props":908,"children":909},{"class":430,"line":447},[910,914,919],{"type":45,"tag":428,"props":911,"children":912},{"style":441},[913],{"type":51,"value":713},{"type":45,"tag":428,"props":915,"children":916},{"style":441},[917],{"type":51,"value":918}," microsoft\u002Fphi-2",{"type":45,"tag":428,"props":920,"children":921},{"style":702},[922],{"type":51,"value":705},{"type":45,"tag":428,"props":924,"children":925},{"class":430,"line":472},[926,930,934],{"type":45,"tag":428,"props":927,"children":928},{"style":441},[929],{"type":51,"value":730},{"type":45,"tag":428,"props":931,"children":932},{"style":441},[933],{"type":51,"value":735},{"type":45,"tag":428,"props":935,"children":936},{"style":702},[937],{"type":51,"value":705},{"type":45,"tag":428,"props":939,"children":940},{"class":430,"line":742},[941,946,951],{"type":45,"tag":428,"props":942,"children":943},{"style":441},[944],{"type":51,"value":945},"  --backend",{"type":45,"tag":428,"props":947,"children":948},{"style":441},[949],{"type":51,"value":950}," hf",{"type":45,"tag":428,"props":952,"children":953},{"style":702},[954],{"type":51,"value":705},{"type":45,"tag":428,"props":956,"children":958},{"class":430,"line":957},5,[959,964],{"type":45,"tag":428,"props":960,"children":961},{"style":441},[962],{"type":51,"value":963},"  --trust-remote-code",{"type":45,"tag":428,"props":965,"children":966},{"style":702},[967],{"type":51,"value":705},{"type":45,"tag":428,"props":969,"children":971},{"class":430,"line":970},6,[972,976],{"type":45,"tag":428,"props":973,"children":974},{"style":441},[975],{"type":51,"value":748},{"type":45,"tag":428,"props":977,"children":978},{"style":751},[979],{"type":51,"value":754},{"type":45,"tag":665,"props":981,"children":983},{"id":982},"option-c-lighteval-on-local-gpu",[984],{"type":51,"value":985},"Option C: lighteval on Local GPU",{"type":45,"tag":54,"props":987,"children":988},{},[989,991,996],{"type":51,"value":990},"Best when the task is naturally expressed as a ",{"type":45,"tag":81,"props":992,"children":994},{"className":993},[],[995],{"type":51,"value":97},{"type":51,"value":997}," task string, especially Open LLM Leaderboard style benchmarks.",{"type":45,"tag":54,"props":999,"children":1000},{},[1001],{"type":51,"value":806},{"type":45,"tag":417,"props":1003,"children":1005},{"className":419,"code":1004,"language":421,"meta":422,"style":422},"uv run scripts\u002Flighteval_vllm_uv.py \\\n  --model meta-llama\u002FLlama-3.2-3B-Instruct \\\n  --tasks \"leaderboard|mmlu|5,leaderboard|gsm8k|5\" \\\n  --max-samples 20 \\\n  --use-chat-template\n",[1006],{"type":45,"tag":81,"props":1007,"children":1008},{"__ignoreMap":422},[1009,1029,1045,1072,1089],{"type":45,"tag":428,"props":1010,"children":1011},{"class":430,"line":431},[1012,1016,1020,1025],{"type":45,"tag":428,"props":1013,"children":1014},{"style":435},[1015],{"type":51,"value":438},{"type":45,"tag":428,"props":1017,"children":1018},{"style":441},[1019],{"type":51,"value":694},{"type":45,"tag":428,"props":1021,"children":1022},{"style":441},[1023],{"type":51,"value":1024}," scripts\u002Flighteval_vllm_uv.py",{"type":45,"tag":428,"props":1026,"children":1027},{"style":702},[1028],{"type":51,"value":705},{"type":45,"tag":428,"props":1030,"children":1031},{"class":430,"line":447},[1032,1036,1041],{"type":45,"tag":428,"props":1033,"children":1034},{"style":441},[1035],{"type":51,"value":713},{"type":45,"tag":428,"props":1037,"children":1038},{"style":441},[1039],{"type":51,"value":1040}," meta-llama\u002FLlama-3.2-3B-Instruct",{"type":45,"tag":428,"props":1042,"children":1043},{"style":702},[1044],{"type":51,"value":705},{"type":45,"tag":428,"props":1046,"children":1047},{"class":430,"line":472},[1048,1053,1058,1063,1068],{"type":45,"tag":428,"props":1049,"children":1050},{"style":441},[1051],{"type":51,"value":1052},"  --tasks",{"type":45,"tag":428,"props":1054,"children":1055},{"style":461},[1056],{"type":51,"value":1057}," \"",{"type":45,"tag":428,"props":1059,"children":1060},{"style":441},[1061],{"type":51,"value":1062},"leaderboard|mmlu|5,leaderboard|gsm8k|5",{"type":45,"tag":428,"props":1064,"children":1065},{"style":461},[1066],{"type":51,"value":1067},"\"",{"type":45,"tag":428,"props":1069,"children":1070},{"style":702},[1071],{"type":51,"value":705},{"type":45,"tag":428,"props":1073,"children":1074},{"class":430,"line":742},[1075,1080,1085],{"type":45,"tag":428,"props":1076,"children":1077},{"style":441},[1078],{"type":51,"value":1079},"  --max-samples",{"type":45,"tag":428,"props":1081,"children":1082},{"style":751},[1083],{"type":51,"value":1084}," 20",{"type":45,"tag":428,"props":1086,"children":1087},{"style":702},[1088],{"type":51,"value":705},{"type":45,"tag":428,"props":1090,"children":1091},{"class":430,"line":957},[1092],{"type":45,"tag":428,"props":1093,"children":1094},{"style":441},[1095],{"type":51,"value":1096},"  --use-chat-template\n",{"type":45,"tag":54,"props":1098,"children":1099},{},[1100,1105],{"type":45,"tag":81,"props":1101,"children":1103},{"className":1102},[],[1104],{"type":51,"value":117},{"type":51,"value":1106}," fallback:",{"type":45,"tag":417,"props":1108,"children":1110},{"className":419,"code":1109,"language":421,"meta":422,"style":422},"uv run scripts\u002Flighteval_vllm_uv.py \\\n  --model microsoft\u002Fphi-2 \\\n  --tasks \"leaderboard|mmlu|5\" \\\n  --backend accelerate \\\n  --trust-remote-code \\\n  --max-samples 20\n",[1111],{"type":45,"tag":81,"props":1112,"children":1113},{"__ignoreMap":422},[1114,1133,1148,1172,1188,1199],{"type":45,"tag":428,"props":1115,"children":1116},{"class":430,"line":431},[1117,1121,1125,1129],{"type":45,"tag":428,"props":1118,"children":1119},{"style":435},[1120],{"type":51,"value":438},{"type":45,"tag":428,"props":1122,"children":1123},{"style":441},[1124],{"type":51,"value":694},{"type":45,"tag":428,"props":1126,"children":1127},{"style":441},[1128],{"type":51,"value":1024},{"type":45,"tag":428,"props":1130,"children":1131},{"style":702},[1132],{"type":51,"value":705},{"type":45,"tag":428,"props":1134,"children":1135},{"class":430,"line":447},[1136,1140,1144],{"type":45,"tag":428,"props":1137,"children":1138},{"style":441},[1139],{"type":51,"value":713},{"type":45,"tag":428,"props":1141,"children":1142},{"style":441},[1143],{"type":51,"value":918},{"type":45,"tag":428,"props":1145,"children":1146},{"style":702},[1147],{"type":51,"value":705},{"type":45,"tag":428,"props":1149,"children":1150},{"class":430,"line":472},[1151,1155,1159,1164,1168],{"type":45,"tag":428,"props":1152,"children":1153},{"style":441},[1154],{"type":51,"value":1052},{"type":45,"tag":428,"props":1156,"children":1157},{"style":461},[1158],{"type":51,"value":1057},{"type":45,"tag":428,"props":1160,"children":1161},{"style":441},[1162],{"type":51,"value":1163},"leaderboard|mmlu|5",{"type":45,"tag":428,"props":1165,"children":1166},{"style":461},[1167],{"type":51,"value":1067},{"type":45,"tag":428,"props":1169,"children":1170},{"style":702},[1171],{"type":51,"value":705},{"type":45,"tag":428,"props":1173,"children":1174},{"class":430,"line":742},[1175,1179,1184],{"type":45,"tag":428,"props":1176,"children":1177},{"style":441},[1178],{"type":51,"value":945},{"type":45,"tag":428,"props":1180,"children":1181},{"style":441},[1182],{"type":51,"value":1183}," accelerate",{"type":45,"tag":428,"props":1185,"children":1186},{"style":702},[1187],{"type":51,"value":705},{"type":45,"tag":428,"props":1189,"children":1190},{"class":430,"line":957},[1191,1195],{"type":45,"tag":428,"props":1192,"children":1193},{"style":441},[1194],{"type":51,"value":963},{"type":45,"tag":428,"props":1196,"children":1197},{"style":702},[1198],{"type":51,"value":705},{"type":45,"tag":428,"props":1200,"children":1201},{"class":430,"line":970},[1202,1206],{"type":45,"tag":428,"props":1203,"children":1204},{"style":441},[1205],{"type":51,"value":1079},{"type":45,"tag":428,"props":1207,"children":1208},{"style":751},[1209],{"type":51,"value":754},{"type":45,"tag":46,"props":1211,"children":1213},{"id":1212},"remote-execution-boundary",[1214],{"type":51,"value":1215},"Remote Execution Boundary",{"type":45,"tag":54,"props":1217,"children":1218},{},[1219,1221,1226],{"type":51,"value":1220},"This skill intentionally stops at ",{"type":45,"tag":60,"props":1222,"children":1223},{},[1224],{"type":51,"value":1225},"local execution and backend selection",{"type":51,"value":66},{"type":45,"tag":54,"props":1228,"children":1229},{},[1230],{"type":51,"value":1231},"If the user wants to:",{"type":45,"tag":73,"props":1233,"children":1234},{},[1235,1240,1245,1250,1255],{"type":45,"tag":77,"props":1236,"children":1237},{},[1238],{"type":51,"value":1239},"run these scripts on Hugging Face Jobs",{"type":45,"tag":77,"props":1241,"children":1242},{},[1243],{"type":51,"value":1244},"pick remote hardware",{"type":45,"tag":77,"props":1246,"children":1247},{},[1248],{"type":51,"value":1249},"pass secrets to remote jobs",{"type":45,"tag":77,"props":1251,"children":1252},{},[1253],{"type":51,"value":1254},"schedule recurring runs",{"type":45,"tag":77,"props":1256,"children":1257},{},[1258],{"type":51,"value":1259},"inspect \u002F cancel \u002F monitor jobs",{"type":45,"tag":54,"props":1261,"children":1262},{},[1263,1265,1273],{"type":51,"value":1264},"then switch to the ",{"type":45,"tag":60,"props":1266,"children":1267},{},[1268],{"type":45,"tag":81,"props":1269,"children":1271},{"className":1270},[],[1272],{"type":51,"value":199},{"type":51,"value":1274}," skill and pass it one of these scripts plus the chosen arguments.",{"type":45,"tag":46,"props":1276,"children":1278},{"id":1277},"task-selection",[1279],{"type":51,"value":1280},"Task Selection",{"type":45,"tag":54,"props":1282,"children":1283},{},[1284,1289],{"type":45,"tag":81,"props":1285,"children":1287},{"className":1286},[],[1288],{"type":51,"value":86},{"type":51,"value":1290}," examples:",{"type":45,"tag":73,"props":1292,"children":1293},{},[1294,1303,1312,1321,1330,1339,1348],{"type":45,"tag":77,"props":1295,"children":1296},{},[1297],{"type":45,"tag":81,"props":1298,"children":1300},{"className":1299},[],[1301],{"type":51,"value":1302},"mmlu",{"type":45,"tag":77,"props":1304,"children":1305},{},[1306],{"type":45,"tag":81,"props":1307,"children":1309},{"className":1308},[],[1310],{"type":51,"value":1311},"gsm8k",{"type":45,"tag":77,"props":1313,"children":1314},{},[1315],{"type":45,"tag":81,"props":1316,"children":1318},{"className":1317},[],[1319],{"type":51,"value":1320},"hellaswag",{"type":45,"tag":77,"props":1322,"children":1323},{},[1324],{"type":45,"tag":81,"props":1325,"children":1327},{"className":1326},[],[1328],{"type":51,"value":1329},"arc_challenge",{"type":45,"tag":77,"props":1331,"children":1332},{},[1333],{"type":45,"tag":81,"props":1334,"children":1336},{"className":1335},[],[1337],{"type":51,"value":1338},"truthfulqa",{"type":45,"tag":77,"props":1340,"children":1341},{},[1342],{"type":45,"tag":81,"props":1343,"children":1345},{"className":1344},[],[1346],{"type":51,"value":1347},"winogrande",{"type":45,"tag":77,"props":1349,"children":1350},{},[1351],{"type":45,"tag":81,"props":1352,"children":1354},{"className":1353},[],[1355],{"type":51,"value":1356},"humaneval",{"type":45,"tag":54,"props":1358,"children":1359},{},[1360,1365,1367,1373],{"type":45,"tag":81,"props":1361,"children":1363},{"className":1362},[],[1364],{"type":51,"value":97},{"type":51,"value":1366}," task strings use ",{"type":45,"tag":81,"props":1368,"children":1370},{"className":1369},[],[1371],{"type":51,"value":1372},"suite|task|num_fewshot",{"type":51,"value":1374},":",{"type":45,"tag":73,"props":1376,"children":1377},{},[1378,1386,1395,1404],{"type":45,"tag":77,"props":1379,"children":1380},{},[1381],{"type":45,"tag":81,"props":1382,"children":1384},{"className":1383},[],[1385],{"type":51,"value":1163},{"type":45,"tag":77,"props":1387,"children":1388},{},[1389],{"type":45,"tag":81,"props":1390,"children":1392},{"className":1391},[],[1393],{"type":51,"value":1394},"leaderboard|gsm8k|5",{"type":45,"tag":77,"props":1396,"children":1397},{},[1398],{"type":45,"tag":81,"props":1399,"children":1401},{"className":1400},[],[1402],{"type":51,"value":1403},"leaderboard|arc_challenge|25",{"type":45,"tag":77,"props":1405,"children":1406},{},[1407],{"type":45,"tag":81,"props":1408,"children":1410},{"className":1409},[],[1411],{"type":51,"value":1412},"lighteval|hellaswag|0",{"type":45,"tag":54,"props":1414,"children":1415},{},[1416,1418,1423,1425,1431],{"type":51,"value":1417},"Multiple ",{"type":45,"tag":81,"props":1419,"children":1421},{"className":1420},[],[1422],{"type":51,"value":97},{"type":51,"value":1424}," tasks can be comma-separated in ",{"type":45,"tag":81,"props":1426,"children":1428},{"className":1427},[],[1429],{"type":51,"value":1430},"--tasks",{"type":51,"value":66},{"type":45,"tag":46,"props":1433,"children":1435},{"id":1434},"backend-selection",[1436],{"type":51,"value":1437},"Backend Selection",{"type":45,"tag":73,"props":1439,"children":1440},{},[1441,1453,1472,1484,1496],{"type":45,"tag":77,"props":1442,"children":1443},{},[1444,1445,1451],{"type":51,"value":389},{"type":45,"tag":81,"props":1446,"children":1448},{"className":1447},[],[1449],{"type":51,"value":1450},"inspect_vllm_uv.py --backend vllm",{"type":51,"value":1452}," for fast GPU inference on supported architectures.",{"type":45,"tag":77,"props":1454,"children":1455},{},[1456,1457,1463,1465,1470],{"type":51,"value":541},{"type":45,"tag":81,"props":1458,"children":1460},{"className":1459},[],[1461],{"type":51,"value":1462},"inspect_vllm_uv.py --backend hf",{"type":51,"value":1464}," when ",{"type":45,"tag":81,"props":1466,"children":1468},{"className":1467},[],[1469],{"type":51,"value":109},{"type":51,"value":1471}," does not support the model.",{"type":45,"tag":77,"props":1473,"children":1474},{},[1475,1476,1482],{"type":51,"value":389},{"type":45,"tag":81,"props":1477,"children":1479},{"className":1478},[],[1480],{"type":51,"value":1481},"lighteval_vllm_uv.py --backend vllm",{"type":51,"value":1483}," for throughput on supported models.",{"type":45,"tag":77,"props":1485,"children":1486},{},[1487,1488,1494],{"type":51,"value":541},{"type":45,"tag":81,"props":1489,"children":1491},{"className":1490},[],[1492],{"type":51,"value":1493},"lighteval_vllm_uv.py --backend accelerate",{"type":51,"value":1495}," as the compatibility fallback.",{"type":45,"tag":77,"props":1497,"children":1498},{},[1499,1500,1506],{"type":51,"value":541},{"type":45,"tag":81,"props":1501,"children":1503},{"className":1502},[],[1504],{"type":51,"value":1505},"inspect_eval_uv.py",{"type":51,"value":1507}," when Inference Providers already cover the model and you do not need direct GPU control.",{"type":45,"tag":46,"props":1509,"children":1511},{"id":1510},"hardware-guidance",[1512],{"type":51,"value":1513},"Hardware Guidance",{"type":45,"tag":243,"props":1515,"children":1516},{},[1517,1533],{"type":45,"tag":247,"props":1518,"children":1519},{},[1520],{"type":45,"tag":251,"props":1521,"children":1522},{},[1523,1528],{"type":45,"tag":255,"props":1524,"children":1525},{},[1526],{"type":51,"value":1527},"Model size",{"type":45,"tag":255,"props":1529,"children":1530},{},[1531],{"type":51,"value":1532},"Suggested local hardware",{"type":45,"tag":266,"props":1534,"children":1535},{},[1536,1553,1570],{"type":45,"tag":251,"props":1537,"children":1538},{},[1539,1548],{"type":45,"tag":273,"props":1540,"children":1541},{},[1542],{"type":45,"tag":81,"props":1543,"children":1545},{"className":1544},[],[1546],{"type":51,"value":1547},"\u003C 3B",{"type":45,"tag":273,"props":1549,"children":1550},{},[1551],{"type":51,"value":1552},"consumer GPU \u002F Apple Silicon \u002F small dev GPU",{"type":45,"tag":251,"props":1554,"children":1555},{},[1556,1565],{"type":45,"tag":273,"props":1557,"children":1558},{},[1559],{"type":45,"tag":81,"props":1560,"children":1562},{"className":1561},[],[1563],{"type":51,"value":1564},"3B - 13B",{"type":45,"tag":273,"props":1566,"children":1567},{},[1568],{"type":51,"value":1569},"stronger local GPU",{"type":45,"tag":251,"props":1571,"children":1572},{},[1573,1582],{"type":45,"tag":273,"props":1574,"children":1575},{},[1576],{"type":45,"tag":81,"props":1577,"children":1579},{"className":1578},[],[1580],{"type":51,"value":1581},"13B+",{"type":45,"tag":273,"props":1583,"children":1584},{},[1585,1587],{"type":51,"value":1586},"high-memory local GPU or hand off to ",{"type":45,"tag":81,"props":1588,"children":1590},{"className":1589},[],[1591],{"type":51,"value":199},{"type":45,"tag":54,"props":1593,"children":1594},{},[1595,1597,1603,1604,1610],{"type":51,"value":1596},"For smoke tests, prefer cheaper local runs plus ",{"type":45,"tag":81,"props":1598,"children":1600},{"className":1599},[],[1601],{"type":51,"value":1602},"--limit",{"type":51,"value":344},{"type":45,"tag":81,"props":1605,"children":1607},{"className":1606},[],[1608],{"type":51,"value":1609},"--max-samples",{"type":51,"value":66},{"type":45,"tag":46,"props":1612,"children":1614},{"id":1613},"troubleshooting",[1615],{"type":51,"value":1616},"Troubleshooting",{"type":45,"tag":73,"props":1618,"children":1619},{},[1620,1664,1712,1730],{"type":45,"tag":77,"props":1621,"children":1622},{},[1623,1625],{"type":51,"value":1624},"CUDA or vLLM OOM:\n",{"type":45,"tag":73,"props":1626,"children":1627},{},[1628,1639,1649,1654],{"type":45,"tag":77,"props":1629,"children":1630},{},[1631,1633],{"type":51,"value":1632},"reduce ",{"type":45,"tag":81,"props":1634,"children":1636},{"className":1635},[],[1637],{"type":51,"value":1638},"--batch-size",{"type":45,"tag":77,"props":1640,"children":1641},{},[1642,1643],{"type":51,"value":1632},{"type":45,"tag":81,"props":1644,"children":1646},{"className":1645},[],[1647],{"type":51,"value":1648},"--gpu-memory-utilization",{"type":45,"tag":77,"props":1650,"children":1651},{},[1652],{"type":51,"value":1653},"switch to a smaller model for the smoke test",{"type":45,"tag":77,"props":1655,"children":1656},{},[1657,1659],{"type":51,"value":1658},"if necessary, hand off to ",{"type":45,"tag":81,"props":1660,"children":1662},{"className":1661},[],[1663],{"type":51,"value":199},{"type":45,"tag":77,"props":1665,"children":1666},{},[1667,1669,1674,1676],{"type":51,"value":1668},"Model unsupported by ",{"type":45,"tag":81,"props":1670,"children":1672},{"className":1671},[],[1673],{"type":51,"value":109},{"type":51,"value":1675},":\n",{"type":45,"tag":73,"props":1677,"children":1678},{},[1679,1696],{"type":45,"tag":77,"props":1680,"children":1681},{},[1682,1684,1689,1691],{"type":51,"value":1683},"switch to ",{"type":45,"tag":81,"props":1685,"children":1687},{"className":1686},[],[1688],{"type":51,"value":589},{"type":51,"value":1690}," for ",{"type":45,"tag":81,"props":1692,"children":1694},{"className":1693},[],[1695],{"type":51,"value":86},{"type":45,"tag":77,"props":1697,"children":1698},{},[1699,1700,1706,1707],{"type":51,"value":1683},{"type":45,"tag":81,"props":1701,"children":1703},{"className":1702},[],[1704],{"type":51,"value":1705},"--backend accelerate",{"type":51,"value":1690},{"type":45,"tag":81,"props":1708,"children":1710},{"className":1709},[],[1711],{"type":51,"value":97},{"type":45,"tag":77,"props":1713,"children":1714},{},[1715,1717],{"type":51,"value":1716},"Gated\u002Fprivate repo access fails:\n",{"type":45,"tag":73,"props":1718,"children":1719},{},[1720],{"type":45,"tag":77,"props":1721,"children":1722},{},[1723,1725],{"type":51,"value":1724},"verify ",{"type":45,"tag":81,"props":1726,"children":1728},{"className":1727},[],[1729],{"type":51,"value":408},{"type":45,"tag":77,"props":1731,"children":1732},{},[1733,1735],{"type":51,"value":1734},"Custom model code required:\n",{"type":45,"tag":73,"props":1736,"children":1737},{},[1738],{"type":45,"tag":77,"props":1739,"children":1740},{},[1741,1743],{"type":51,"value":1742},"add ",{"type":45,"tag":81,"props":1744,"children":1746},{"className":1745},[],[1747],{"type":51,"value":1748},"--trust-remote-code",{"type":45,"tag":46,"props":1750,"children":1752},{"id":1751},"examples",[1753],{"type":51,"value":1754},"Examples",{"type":45,"tag":54,"props":1756,"children":1757},{},[1758],{"type":51,"value":1759},"See:",{"type":45,"tag":73,"props":1761,"children":1762},{},[1763,1773,1781,1789],{"type":45,"tag":77,"props":1764,"children":1765},{},[1766,1771],{"type":45,"tag":81,"props":1767,"children":1769},{"className":1768},[],[1770],{"type":51,"value":375},{"type":51,"value":1772}," for local command patterns",{"type":45,"tag":77,"props":1774,"children":1775},{},[1776],{"type":45,"tag":81,"props":1777,"children":1779},{"className":1778},[],[1780],{"type":51,"value":293},{"type":45,"tag":77,"props":1782,"children":1783},{},[1784],{"type":45,"tag":81,"props":1785,"children":1787},{"className":1786},[],[1788],{"type":51,"value":324},{"type":45,"tag":77,"props":1790,"children":1791},{},[1792],{"type":45,"tag":81,"props":1793,"children":1795},{"className":1794},[],[1796],{"type":51,"value":358},{"type":45,"tag":1798,"props":1799,"children":1800},"style",{},[1801],{"type":51,"value":1802},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":1804,"total":2009},[1805,1826,1849,1866,1882,1901,1920,1936,1952,1966,1978,1993],{"slug":1806,"name":1806,"fn":1807,"description":1808,"org":1809,"tags":1810,"stars":1823,"repoUrl":1824,"updatedAt":1825},"prior-auth-packet-builder","build healthcare prior authorization packets","Build a concise prior authorization packet from local case files and payer policy docs.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1811,1814,1817,1820],{"name":1812,"slug":1813,"type":15},"Documents","documents",{"name":1815,"slug":1816,"type":15},"Healthcare","healthcare",{"name":1818,"slug":1819,"type":15},"Insurance","insurance",{"name":1821,"slug":1822,"type":15},"Regulatory Compliance","regulatory-compliance",28169,"https:\u002F\u002Fgithub.com\u002Fopenai\u002Fopenai-agents-python","2026-04-16T05:11:39.180399",{"slug":1827,"name":1827,"fn":1828,"description":1829,"org":1830,"tags":1831,"stars":1846,"repoUrl":1847,"updatedAt":1848},"aspnet-core","build ASP.NET Core web applications","Build, review, refactor, or architect ASP.NET Core web applications using current official guidance for .NET web development. Use when working on Blazor Web Apps, Razor Pages, MVC, Minimal APIs, controller-based Web APIs, SignalR, gRPC, middleware, dependency injection, configuration, authentication, authorization, testing, performance, deployment, or ASP.NET Core upgrades.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1832,1835,1837,1840,1843],{"name":1833,"slug":1834,"type":15},".NET","dotnet",{"name":1836,"slug":1827,"type":15},"ASP.NET Core",{"name":1838,"slug":1839,"type":15},"Blazor","blazor",{"name":1841,"slug":1842,"type":15},"C#","csharp",{"name":1844,"slug":1845,"type":15},"Web Development","web-development",23787,"https:\u002F\u002Fgithub.com\u002Fopenai\u002Fskills","2026-04-12T05:07:02.819491",{"slug":1850,"name":1850,"fn":1851,"description":1852,"org":1853,"tags":1854,"stars":1846,"repoUrl":1847,"updatedAt":1865},"chatgpt-apps","build ChatGPT Apps SDK applications","Build, scaffold, refactor, and troubleshoot ChatGPT Apps SDK applications that combine an MCP server and widget UI. Use when Codex needs to design tools, register UI resources, wire the MCP Apps bridge or ChatGPT compatibility APIs, apply Apps SDK metadata or CSP or domain settings, or produce a docs-aligned project scaffold. Prefer a docs-first workflow by invoking the openai-docs skill or OpenAI developer docs MCP tools before generating code.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1855,1858,1861,1864],{"name":1856,"slug":1857,"type":15},"Apps SDK","apps-sdk",{"name":1859,"slug":1860,"type":15},"ChatGPT","chatgpt",{"name":1862,"slug":1863,"type":15},"MCP","mcp",{"name":9,"slug":8,"type":15},"2026-04-12T05:07:05.468097",{"slug":1867,"name":1867,"fn":1868,"description":1869,"org":1870,"tags":1871,"stars":1846,"repoUrl":1847,"updatedAt":1881},"cli-creator","build CLIs from API docs","Build a composable CLI for Codex from API docs, an OpenAPI spec, existing curl examples, an SDK, a web app, an admin tool, or a local script. Use when the user wants Codex to create a command-line tool that can run from any repo, expose composable read\u002Fwrite commands, return stable JSON, manage auth, and pair with a companion skill.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1872,1875,1878],{"name":1873,"slug":1874,"type":15},"API Development","api-development",{"name":1876,"slug":1877,"type":15},"CLI","cli",{"name":1879,"slug":1880,"type":15},"Codex","codex","2026-04-12T05:07:04.132762",{"slug":1883,"name":1883,"fn":1884,"description":1885,"org":1886,"tags":1887,"stars":1846,"repoUrl":1847,"updatedAt":1900},"cloudflare-deploy","deploy projects to Cloudflare","Deploy applications and infrastructure to Cloudflare using Workers, Pages, and related platform services. Use when the user asks to deploy, host, publish, or set up a project on Cloudflare.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1888,1891,1894,1897],{"name":1889,"slug":1890,"type":15},"Cloudflare","cloudflare",{"name":1892,"slug":1893,"type":15},"Cloudflare Pages","cloudflare-pages",{"name":1895,"slug":1896,"type":15},"Cloudflare Workers","cloudflare-workers",{"name":1898,"slug":1899,"type":15},"Deployment","deployment","2026-04-12T05:07:14.275118",{"slug":1902,"name":1902,"fn":1903,"description":1904,"org":1905,"tags":1906,"stars":1846,"repoUrl":1847,"updatedAt":1919},"define-goal","define and set measurable project goals","Help the user define a concrete, measurable goal before starting work, especially when they ask to use the goal tool, create a goal, set an objective, clarify success criteria, or turn a fuzzy intention into a quantitative outcome. Use this skill for goal creation and goal refinement only; it does not manage durable snapshots, decision logs, or long-running execution artifacts.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1907,1910,1913,1916],{"name":1908,"slug":1909,"type":15},"Productivity","productivity",{"name":1911,"slug":1912,"type":15},"Project Management","project-management",{"name":1914,"slug":1915,"type":15},"Strategy","strategy",{"name":1917,"slug":1918,"type":15},"Task Management","task-management","2026-05-23T06:17:16.870838",{"slug":1921,"name":1921,"fn":1922,"description":1923,"org":1924,"tags":1925,"stars":1846,"repoUrl":1847,"updatedAt":1935},"figma","translate Figma designs into code","Use the Figma MCP server to fetch design context, screenshots, variables, and assets from Figma, and to translate Figma nodes into production code. Trigger when a task involves Figma URLs, node IDs, design-to-code implementation, or Figma MCP setup and troubleshooting.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1926,1929,1931,1934],{"name":1927,"slug":1928,"type":15},"Design","design",{"name":1930,"slug":1921,"type":15},"Figma",{"name":1932,"slug":1933,"type":15},"Frontend","frontend",{"name":1862,"slug":1863,"type":15},"2026-04-12T05:06:47.939943",{"slug":1937,"name":1937,"fn":1938,"description":1939,"org":1940,"tags":1941,"stars":1846,"repoUrl":1847,"updatedAt":1951},"figma-code-connect-components","connect Figma designs to code components","Connects Figma design components to code components using Code Connect mapping tools. Use when user says \"code connect\", \"connect this component to code\", \"map this component\", \"link component to code\", \"create code connect mapping\", or wants to establish mappings between Figma designs and code implementations. For canvas writes via `use_figma`, use `figma-use`.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1942,1943,1946,1947,1948],{"name":1927,"slug":1928,"type":15},{"name":1944,"slug":1945,"type":15},"Design System","design-system",{"name":1930,"slug":1921,"type":15},{"name":1932,"slug":1933,"type":15},{"name":1949,"slug":1950,"type":15},"UI Components","ui-components","2026-05-10T05:59:52.971881",{"slug":1953,"name":1953,"fn":1954,"description":1955,"org":1956,"tags":1957,"stars":1846,"repoUrl":1847,"updatedAt":1965},"figma-create-design-system-rules","generate design system rules from Figma","Generates custom design system rules for the user's codebase. Use when user says \"create design system rules\", \"generate rules for my project\", \"set up design rules\", \"customize design system guidelines\", or wants to establish project-specific conventions for Figma-to-code workflows. Requires Figma MCP server connection.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1958,1959,1960,1963,1964],{"name":1927,"slug":1928,"type":15},{"name":1944,"slug":1945,"type":15},{"name":1961,"slug":1962,"type":15},"Documentation","documentation",{"name":1930,"slug":1921,"type":15},{"name":1932,"slug":1933,"type":15},"2026-05-16T06:07:47.821474",{"slug":1967,"name":1967,"fn":1968,"description":1969,"org":1970,"tags":1971,"stars":1846,"repoUrl":1847,"updatedAt":1977},"figma-implement-design","translate Figma designs into application code","Translates Figma designs into production-ready application code with 1:1 visual fidelity. Use when implementing UI code from Figma files, when user mentions \"implement design\", \"generate code\", \"implement component\", provides Figma URLs, or asks to build components matching Figma specs. For Figma canvas writes via `use_figma`, use `figma-use`.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1972,1973,1974,1975,1976],{"name":1927,"slug":1928,"type":15},{"name":1930,"slug":1921,"type":15},{"name":1932,"slug":1933,"type":15},{"name":1949,"slug":1950,"type":15},{"name":1844,"slug":1845,"type":15},"2026-05-16T06:07:40.583615",{"slug":1979,"name":1979,"fn":1980,"description":1981,"org":1982,"tags":1983,"stars":1846,"repoUrl":1847,"updatedAt":1992},"hatch-pet","create animated pets for Codex","Create, repair, validate, visually QA, and package Codex-compatible animated pets and pet spritesheets from character art, generated images, company or prospect brand cues, or visual references. Use when a user wants a lightweight-worker Codex pet workflow, a non-pixel custom pet style, a prospect or company mascot pet, or a full 8x9 animated pet atlas with transparent unused cells, QA contact sheets, and pet.json packaging. This skill composes the installed $imagegen system skill for visual generation and uses bundled scripts for deterministic spritesheet assembly.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1984,1987,1988,1991],{"name":1985,"slug":1986,"type":15},"Animation","animation",{"name":1879,"slug":1880,"type":15},{"name":1989,"slug":1990,"type":15},"Creative","creative",{"name":1927,"slug":1928,"type":15},"2026-05-02T05:31:48.48485",{"slug":1994,"name":1994,"fn":1995,"description":1996,"org":1997,"tags":1998,"stars":1846,"repoUrl":1847,"updatedAt":2008},"imagegen","generate and edit raster images","Generate or edit raster images when the task benefits from AI-created bitmap visuals such as photos, illustrations, textures, sprites, mockups, or transparent-background cutouts. Use when Codex should create a brand-new image, transform an existing image, or derive visual variants from references, and the output should be a bitmap asset rather than repo-native code or vector. Do not use when the task is better handled by editing existing SVG\u002Fvector\u002Fcode-native assets, extending an established icon or logo system, or building the visual directly in HTML\u002FCSS\u002Fcanvas.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1999,2000,2001,2004,2007],{"name":1989,"slug":1990,"type":15},{"name":1927,"slug":1928,"type":15},{"name":2002,"slug":2003,"type":15},"Image Generation","image-generation",{"name":2005,"slug":2006,"type":15},"Images","images",{"name":9,"slug":8,"type":15},"2026-05-15T06:23:24.312127",675,{"items":2011,"total":2121},[2012,2029,2043,2055,2073,2091,2109],{"slug":2013,"name":2013,"fn":2014,"description":2015,"org":2016,"tags":2017,"stars":28,"repoUrl":29,"updatedAt":2028},"accessibility-and-inclusive-visualization","make data visualizations accessible","Make data visualizations accessible and inclusive. Use when the user needs chart or diagram accessibility guidance, text alternatives for complex visuals, color and contrast review, keyboard support, reduced-motion behavior for animation or parallax, or an accessibility QA workflow for exported figures, UML-like diagrams, and dashboards.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2018,2021,2024,2027],{"name":2019,"slug":2020,"type":15},"Accessibility","accessibility",{"name":2022,"slug":2023,"type":15},"Charts","charts",{"name":2025,"slug":2026,"type":15},"Data Visualization","data-visualization",{"name":1927,"slug":1928,"type":15},"2026-06-30T19:00:57.102",{"slug":2030,"name":2030,"fn":2031,"description":2032,"org":2033,"tags":2034,"stars":28,"repoUrl":29,"updatedAt":2042},"agent-browser","automate browser interactions for agents","Browser automation CLI for AI agents. Use when the user needs to interact with websites, verify dev server output, test web apps, navigate pages, fill forms, click buttons, take screenshots, extract data, or automate any browser task. Also triggers when a dev server starts so you can verify it visually.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2035,2038,2041],{"name":2036,"slug":2037,"type":15},"Agents","agents",{"name":2039,"slug":2040,"type":15},"Browser Automation","browser-automation",{"name":26,"slug":27,"type":15},"2026-04-06T18:41:03.44016",{"slug":2044,"name":2044,"fn":2045,"description":2046,"org":2047,"tags":2048,"stars":28,"repoUrl":29,"updatedAt":2054},"agent-browser-verify","verify dev server output with automated browser","Automated browser verification for dev servers. Triggers when a dev server starts to run a visual gut-check with agent-browser — verifies the page loads, checks for console errors, validates key UI elements, and reports pass\u002Ffail before continuing.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2049,2050,2053],{"name":2039,"slug":2040,"type":15},{"name":2051,"slug":2052,"type":15},"Local Development","local-development",{"name":26,"slug":27,"type":15},"2026-04-06T18:41:17.526867",{"slug":2056,"name":2056,"fn":2057,"description":2058,"org":2059,"tags":2060,"stars":28,"repoUrl":29,"updatedAt":2072},"agents-sdk","build AI agents on Cloudflare Workers","Build AI agents on Cloudflare Workers using the Agents SDK. Load when creating stateful agents, durable workflows, real-time WebSocket apps, scheduled tasks, MCP servers, or chat applications. Covers Agent class, state management, callable RPC, Workflows integration, and React hooks. Biases towards retrieval from Cloudflare docs over pre-trained knowledge.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2061,2062,2063,2066,2069],{"name":2036,"slug":2037,"type":15},{"name":1895,"slug":1896,"type":15},{"name":2064,"slug":2065,"type":15},"SDK","sdk",{"name":2067,"slug":2068,"type":15},"Serverless","serverless",{"name":2070,"slug":2071,"type":15},"WebSockets","websockets","2026-04-06T18:39:51.717063",{"slug":2074,"name":2074,"fn":2075,"description":2076,"org":2077,"tags":2078,"stars":28,"repoUrl":29,"updatedAt":2090},"ai-elements","build chat UIs with AI Elements","AI Elements component library guidance — pre-built React components for AI interfaces built on shadcn\u002Fui. Use when building chat UIs, message displays, tool call rendering, streaming responses, reasoning panels, or any AI-native interface with the AI SDK.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2079,2080,2083,2086,2087],{"name":1932,"slug":1933,"type":15},{"name":2081,"slug":2082,"type":15},"React","react",{"name":2084,"slug":2085,"type":15},"shadcn\u002Fui","shadcn-ui",{"name":1949,"slug":1950,"type":15},{"name":2088,"slug":2089,"type":15},"Vercel","vercel","2026-04-06T18:40:59.619419",{"slug":2092,"name":2092,"fn":2093,"description":2094,"org":2095,"tags":2096,"stars":28,"repoUrl":29,"updatedAt":2108},"ai-gateway","configure Vercel AI Gateway","Vercel AI Gateway expert guidance. Use when configuring model routing, provider failover, cost tracking, or managing multiple AI providers through a unified API.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2097,2100,2103,2104,2107],{"name":2098,"slug":2099,"type":15},"AI Infrastructure","ai-infrastructure",{"name":2101,"slug":2102,"type":15},"Cost Optimization","cost-optimization",{"name":17,"slug":18,"type":15},{"name":2105,"slug":2106,"type":15},"Performance","performance",{"name":2088,"slug":2089,"type":15},"2026-04-06T18:40:44.377464",{"slug":2110,"name":2110,"fn":2111,"description":2112,"org":2113,"tags":2114,"stars":28,"repoUrl":29,"updatedAt":2120},"ai-generation-persistence","implement persistence patterns for AI generations","AI generation persistence patterns — unique IDs, addressable URLs, database storage, and cost tracking for every LLM generation",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2115,2116,2119],{"name":2101,"slug":2102,"type":15},{"name":2117,"slug":2118,"type":15},"Database","database",{"name":17,"slug":18,"type":15},"2026-04-06T18:41:08.513425",600]