[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-vllm-vllm-prefix-cache-bench":3,"mdc--wu5xub-key":35,"related-org-vllm-vllm-prefix-cache-bench":1370,"related-repo-vllm-vllm-prefix-cache-bench":1463},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":12,"stars":24,"repoUrl":25,"updatedAt":26,"license":27,"forks":28,"topics":29,"repo":30,"sourceUrl":33,"mdContent":34},"vllm-prefix-cache-bench","benchmark vLLM prefix caching efficiency","This is a skill for benchmarking the efficiency of automatic prefix caching in vLLM using fixed prompts, real-world datasets, or synthetic prefix\u002Fsuffix patterns. Use when the user asks to benchmark prefix caching hit rate, caching efficiency, or repeated-prompt performance in vLLM.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},"vllm","vLLM","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fvllm.png","vllm-project",[13,17,20,23],{"name":14,"slug":15,"type":16},"Performance","performance","tag",{"name":18,"slug":19,"type":16},"Benchmarking","benchmarking",{"name":21,"slug":22,"type":16},"LLM","llm",{"name":9,"slug":8,"type":16},88,"https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm-skills","2026-07-18T05:48:07.243092",null,23,[],{"repoUrl":25,"stars":24,"forks":28,"topics":31,"description":32},[],"Agent skills for vLLM","https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm-skills\u002Ftree\u002FHEAD\u002Fplugins\u002Fvllm-skills\u002Fskills\u002Fvllm-prefix-cache-bench","---\nname: vllm-prefix-cache-bench\ndescription: This is a skill for benchmarking the efficiency of automatic prefix caching in vLLM using fixed prompts, real-world datasets, or synthetic prefix\u002Fsuffix patterns. Use when the user asks to benchmark prefix caching hit rate, caching efficiency, or repeated-prompt performance in vLLM.\n---\n\n# vLLM Prefix Caching Benchmark\n\nBenchmark the efficiency of vLLM's automatic prefix caching (APC) feature. The offline script `benchmarks\u002Fbenchmark_prefix_caching.py` runs directly against the vLLM engine (no server required). For online\u002Fserving tests, use `vllm bench serve` with the `prefix_repetition` dataset.\n\n## When to use\n\n- User wants to measure the performance impact of prefix caching for repeated or partially-shared prompts.\n- User wants to compare throughput\u002Flatency with and without `--enable-prefix-caching`.\n- User wants to test prefix caching using a fixed synthetic prompt, a real dataset (e.g. ShareGPT), or a synthetic prefix\u002Fsuffix repetition pattern.\n\n## Option 1 (default). Fixed Prompt with Prefix Caching\n\nRuns a synthetic benchmark with a fixed prompt repeated multiple times to directly measure cache hit efficiency. No dataset download required.\n\n```bash\npython3 benchmarks\u002Fbenchmark_prefix_caching.py \\\n  --model Qwen\u002FQwen3-8B \\\n  --enable-prefix-caching \\\n  --num-prompts 1 \\\n  --repeat-count 100 \\\n  --input-length-range 128:256\n```\n\nTo compare against the baseline without caching:\n\n```bash\npython3 benchmarks\u002Fbenchmark_prefix_caching.py \\\n  --model Qwen\u002FQwen3-8B \\\n  --no-enable-prefix-caching \\\n  --num-prompts 1 \\\n  --repeat-count 100 \\\n  --input-length-range 128:256\n```\n\n## Option 2. ShareGPT Dataset with Prefix Caching\n\nUses real-world conversational data from ShareGPT to evaluate prefix caching with naturally occurring prompt sharing.\n\nFirst, download the dataset:\n\n```bash\nwget https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fanon8231489123\u002FShareGPT_Vicuna_unfiltered\u002Fresolve\u002Fmain\u002FShareGPT_V3_unfiltered_cleaned_split.json\n```\n\nThen run the benchmark:\n\n```bash\npython3 benchmarks\u002Fbenchmark_prefix_caching.py \\\n  --model Qwen\u002FQwen3-8B \\\n  --dataset-path ShareGPT_V3_unfiltered_cleaned_split.json \\\n  --enable-prefix-caching \\\n  --num-prompts 20 \\\n  --repeat-count 5 \\\n  --input-length-range 128:256\n```\n\n## Option 3. Prefix Repetition Dataset (Online)\n\nUses `vllm bench serve` with the synthetic `prefix_repetition` dataset to test caching via the serving API. This requires a running vLLM server.\n\nFirst, start the server:\n\n```bash\nvllm serve Qwen\u002FQwen3-8B\n```\n\nThen run the benchmark:\n\n```bash\nvllm bench serve \\\n  --backend openai \\\n  --model Qwen\u002FQwen3-8B \\\n  --dataset-name prefix_repetition \\\n  --num-prompts 100 \\\n  --prefix-repetition-prefix-len 512 \\\n  --prefix-repetition-suffix-len 128 \\\n  --prefix-repetition-num-prefixes 5 \\\n  --prefix-repetition-output-len 128\n```\n\nKey parameters for `prefix_repetition`:\n\n| Parameter | Description |\n|---|---|\n| `--prefix-repetition-prefix-len` | Number of tokens in the shared prefix portion |\n| `--prefix-repetition-suffix-len` | Number of tokens in the unique suffix portion |\n| `--prefix-repetition-num-prefixes` | Number of distinct prefixes to cycle through |\n| `--prefix-repetition-output-len` | Number of output tokens to generate per request |\n\n## Notes\n\n- Run all commands from the root of the vLLM repository (`cd vllm`).\n- Keep the default model (`Qwen\u002FQwen3-8B`) unless the user specifies a different one or the model is unavailable; change only `--model`.\n- `--repeat-count` in Option 1 and 2 controls how many times each sampled prompt is replayed; higher values increase cache hit rate.\n- `--input-length-range` accepts a `min:max` token range, e.g. `128:256`.\n- For multi-GPU setups, add `--tensor-parallel-size \u003CN>`.\n- To test different hash algorithms for prefix caching internals, use `--prefix-caching-hash-algo xxhash` (requires `pip install xxhash`).\n\n## Arguments for `benchmark_prefix_caching.py`\n\n| Argument | Required | Description |\n|---|---|---|\n| `--model` | Yes | Model name or path (HuggingFace ID or local path) |\n| `--num-prompts` | Yes | Number of prompts to process |\n| `--input-length-range` | Yes | Token length range for inputs, e.g. `128:256` |\n| `--repeat-count` | No | Number of times each prompt is repeated (default: 1) |\n| `--dataset-path` | No | Path to a dataset file (e.g. ShareGPT JSON). Omit for synthetic fixed-prompt mode |\n| `--prefix-len` | No | Fixed prefix token length to prepend to every prompt |\n| `--output-len` | No | Number of output tokens to generate per request |\n| `--sort` | No | Sort prompts by length before benchmarking |\n| `--enable-prefix-caching` \u002F `--no-enable-prefix-caching` | No | Toggle APC (recommended: enable to test caching) |\n| `--prefix-caching-hash-algo` | No | Hash algorithm: `sha256`, `sha256_cbor`, `xxhash`, `xxhash_cbor` |\n| `--tensor-parallel-size` | No | Number of GPUs for tensor parallelism |\n| `--disable-detokenize` | No | Skip detokenization to reduce overhead |\n\n## Troubleshooting\n\n- If `python3 benchmarks\u002F*.py` reports file not found, locate your local vLLM repository first and run the command from that repo root.\n- If you do not have the repository yet, clone it and continue:\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm\ncd vllm\n```\n\n- If HuggingFace model download fails due to access restrictions, set your token: `export HF_TOKEN=\u003Cyour_token>` or pass `--hf-token \u003Cyour_token>`.\n- If `xxhash` or `cbor2` is not installed and you use those hash algorithms, install them first: `pip install xxhash cbor2`.\n",{"data":36,"body":37},{"name":4,"description":6},{"type":38,"children":39},"root",[40,49,80,87,115,121,126,243,248,338,344,349,354,374,379,488,494,513,518,542,546,702,714,810,816,921,933,1249,1255,1276,1315,1364],{"type":41,"tag":42,"props":43,"children":45},"element","h1",{"id":44},"vllm-prefix-caching-benchmark",[46],{"type":47,"value":48},"text","vLLM Prefix Caching Benchmark",{"type":41,"tag":50,"props":51,"children":52},"p",{},[53,55,62,64,70,72,78],{"type":47,"value":54},"Benchmark the efficiency of vLLM's automatic prefix caching (APC) feature. The offline script ",{"type":41,"tag":56,"props":57,"children":59},"code",{"className":58},[],[60],{"type":47,"value":61},"benchmarks\u002Fbenchmark_prefix_caching.py",{"type":47,"value":63}," runs directly against the vLLM engine (no server required). For online\u002Fserving tests, use ",{"type":41,"tag":56,"props":65,"children":67},{"className":66},[],[68],{"type":47,"value":69},"vllm bench serve",{"type":47,"value":71}," with the ",{"type":41,"tag":56,"props":73,"children":75},{"className":74},[],[76],{"type":47,"value":77},"prefix_repetition",{"type":47,"value":79}," dataset.",{"type":41,"tag":81,"props":82,"children":84},"h2",{"id":83},"when-to-use",[85],{"type":47,"value":86},"When to use",{"type":41,"tag":88,"props":89,"children":90},"ul",{},[91,97,110],{"type":41,"tag":92,"props":93,"children":94},"li",{},[95],{"type":47,"value":96},"User wants to measure the performance impact of prefix caching for repeated or partially-shared prompts.",{"type":41,"tag":92,"props":98,"children":99},{},[100,102,108],{"type":47,"value":101},"User wants to compare throughput\u002Flatency with and without ",{"type":41,"tag":56,"props":103,"children":105},{"className":104},[],[106],{"type":47,"value":107},"--enable-prefix-caching",{"type":47,"value":109},".",{"type":41,"tag":92,"props":111,"children":112},{},[113],{"type":47,"value":114},"User wants to test prefix caching using a fixed synthetic prompt, a real dataset (e.g. ShareGPT), or a synthetic prefix\u002Fsuffix repetition pattern.",{"type":41,"tag":81,"props":116,"children":118},{"id":117},"option-1-default-fixed-prompt-with-prefix-caching",[119],{"type":47,"value":120},"Option 1 (default). Fixed Prompt with Prefix Caching",{"type":41,"tag":50,"props":122,"children":123},{},[124],{"type":47,"value":125},"Runs a synthetic benchmark with a fixed prompt repeated multiple times to directly measure cache hit efficiency. No dataset download required.",{"type":41,"tag":127,"props":128,"children":133},"pre",{"className":129,"code":130,"language":131,"meta":132,"style":132},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","python3 benchmarks\u002Fbenchmark_prefix_caching.py \\\n  --model Qwen\u002FQwen3-8B \\\n  --enable-prefix-caching \\\n  --num-prompts 1 \\\n  --repeat-count 100 \\\n  --input-length-range 128:256\n","bash","",[134],{"type":41,"tag":56,"props":135,"children":136},{"__ignoreMap":132},[137,161,179,192,211,229],{"type":41,"tag":138,"props":139,"children":142},"span",{"class":140,"line":141},"line",1,[143,149,155],{"type":41,"tag":138,"props":144,"children":146},{"style":145},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[147],{"type":47,"value":148},"python3",{"type":41,"tag":138,"props":150,"children":152},{"style":151},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[153],{"type":47,"value":154}," benchmarks\u002Fbenchmark_prefix_caching.py",{"type":41,"tag":138,"props":156,"children":158},{"style":157},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[159],{"type":47,"value":160}," \\\n",{"type":41,"tag":138,"props":162,"children":164},{"class":140,"line":163},2,[165,170,175],{"type":41,"tag":138,"props":166,"children":167},{"style":151},[168],{"type":47,"value":169},"  --model",{"type":41,"tag":138,"props":171,"children":172},{"style":151},[173],{"type":47,"value":174}," Qwen\u002FQwen3-8B",{"type":41,"tag":138,"props":176,"children":177},{"style":157},[178],{"type":47,"value":160},{"type":41,"tag":138,"props":180,"children":182},{"class":140,"line":181},3,[183,188],{"type":41,"tag":138,"props":184,"children":185},{"style":151},[186],{"type":47,"value":187},"  --enable-prefix-caching",{"type":41,"tag":138,"props":189,"children":190},{"style":157},[191],{"type":47,"value":160},{"type":41,"tag":138,"props":193,"children":195},{"class":140,"line":194},4,[196,201,207],{"type":41,"tag":138,"props":197,"children":198},{"style":151},[199],{"type":47,"value":200},"  --num-prompts",{"type":41,"tag":138,"props":202,"children":204},{"style":203},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[205],{"type":47,"value":206}," 1",{"type":41,"tag":138,"props":208,"children":209},{"style":157},[210],{"type":47,"value":160},{"type":41,"tag":138,"props":212,"children":214},{"class":140,"line":213},5,[215,220,225],{"type":41,"tag":138,"props":216,"children":217},{"style":151},[218],{"type":47,"value":219},"  --repeat-count",{"type":41,"tag":138,"props":221,"children":222},{"style":203},[223],{"type":47,"value":224}," 100",{"type":41,"tag":138,"props":226,"children":227},{"style":157},[228],{"type":47,"value":160},{"type":41,"tag":138,"props":230,"children":232},{"class":140,"line":231},6,[233,238],{"type":41,"tag":138,"props":234,"children":235},{"style":151},[236],{"type":47,"value":237},"  --input-length-range",{"type":41,"tag":138,"props":239,"children":240},{"style":151},[241],{"type":47,"value":242}," 128:256\n",{"type":41,"tag":50,"props":244,"children":245},{},[246],{"type":47,"value":247},"To compare against the baseline without caching:",{"type":41,"tag":127,"props":249,"children":251},{"className":129,"code":250,"language":131,"meta":132,"style":132},"python3 benchmarks\u002Fbenchmark_prefix_caching.py \\\n  --model Qwen\u002FQwen3-8B \\\n  --no-enable-prefix-caching \\\n  --num-prompts 1 \\\n  --repeat-count 100 \\\n  --input-length-range 128:256\n",[252],{"type":41,"tag":56,"props":253,"children":254},{"__ignoreMap":132},[255,270,285,297,312,327],{"type":41,"tag":138,"props":256,"children":257},{"class":140,"line":141},[258,262,266],{"type":41,"tag":138,"props":259,"children":260},{"style":145},[261],{"type":47,"value":148},{"type":41,"tag":138,"props":263,"children":264},{"style":151},[265],{"type":47,"value":154},{"type":41,"tag":138,"props":267,"children":268},{"style":157},[269],{"type":47,"value":160},{"type":41,"tag":138,"props":271,"children":272},{"class":140,"line":163},[273,277,281],{"type":41,"tag":138,"props":274,"children":275},{"style":151},[276],{"type":47,"value":169},{"type":41,"tag":138,"props":278,"children":279},{"style":151},[280],{"type":47,"value":174},{"type":41,"tag":138,"props":282,"children":283},{"style":157},[284],{"type":47,"value":160},{"type":41,"tag":138,"props":286,"children":287},{"class":140,"line":181},[288,293],{"type":41,"tag":138,"props":289,"children":290},{"style":151},[291],{"type":47,"value":292},"  --no-enable-prefix-caching",{"type":41,"tag":138,"props":294,"children":295},{"style":157},[296],{"type":47,"value":160},{"type":41,"tag":138,"props":298,"children":299},{"class":140,"line":194},[300,304,308],{"type":41,"tag":138,"props":301,"children":302},{"style":151},[303],{"type":47,"value":200},{"type":41,"tag":138,"props":305,"children":306},{"style":203},[307],{"type":47,"value":206},{"type":41,"tag":138,"props":309,"children":310},{"style":157},[311],{"type":47,"value":160},{"type":41,"tag":138,"props":313,"children":314},{"class":140,"line":213},[315,319,323],{"type":41,"tag":138,"props":316,"children":317},{"style":151},[318],{"type":47,"value":219},{"type":41,"tag":138,"props":320,"children":321},{"style":203},[322],{"type":47,"value":224},{"type":41,"tag":138,"props":324,"children":325},{"style":157},[326],{"type":47,"value":160},{"type":41,"tag":138,"props":328,"children":329},{"class":140,"line":231},[330,334],{"type":41,"tag":138,"props":331,"children":332},{"style":151},[333],{"type":47,"value":237},{"type":41,"tag":138,"props":335,"children":336},{"style":151},[337],{"type":47,"value":242},{"type":41,"tag":81,"props":339,"children":341},{"id":340},"option-2-sharegpt-dataset-with-prefix-caching",[342],{"type":47,"value":343},"Option 2. ShareGPT Dataset with Prefix Caching",{"type":41,"tag":50,"props":345,"children":346},{},[347],{"type":47,"value":348},"Uses real-world conversational data from ShareGPT to evaluate prefix caching with naturally occurring prompt sharing.",{"type":41,"tag":50,"props":350,"children":351},{},[352],{"type":47,"value":353},"First, download the dataset:",{"type":41,"tag":127,"props":355,"children":357},{"className":129,"code":356,"language":131,"meta":132,"style":132},"wget https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fanon8231489123\u002FShareGPT_Vicuna_unfiltered\u002Fresolve\u002Fmain\u002FShareGPT_V3_unfiltered_cleaned_split.json\n",[358],{"type":41,"tag":56,"props":359,"children":360},{"__ignoreMap":132},[361],{"type":41,"tag":138,"props":362,"children":363},{"class":140,"line":141},[364,369],{"type":41,"tag":138,"props":365,"children":366},{"style":145},[367],{"type":47,"value":368},"wget",{"type":41,"tag":138,"props":370,"children":371},{"style":151},[372],{"type":47,"value":373}," https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fanon8231489123\u002FShareGPT_Vicuna_unfiltered\u002Fresolve\u002Fmain\u002FShareGPT_V3_unfiltered_cleaned_split.json\n",{"type":41,"tag":50,"props":375,"children":376},{},[377],{"type":47,"value":378},"Then run the benchmark:",{"type":41,"tag":127,"props":380,"children":382},{"className":129,"code":381,"language":131,"meta":132,"style":132},"python3 benchmarks\u002Fbenchmark_prefix_caching.py \\\n  --model Qwen\u002FQwen3-8B \\\n  --dataset-path ShareGPT_V3_unfiltered_cleaned_split.json \\\n  --enable-prefix-caching \\\n  --num-prompts 20 \\\n  --repeat-count 5 \\\n  --input-length-range 128:256\n",[383],{"type":41,"tag":56,"props":384,"children":385},{"__ignoreMap":132},[386,401,416,433,444,460,476],{"type":41,"tag":138,"props":387,"children":388},{"class":140,"line":141},[389,393,397],{"type":41,"tag":138,"props":390,"children":391},{"style":145},[392],{"type":47,"value":148},{"type":41,"tag":138,"props":394,"children":395},{"style":151},[396],{"type":47,"value":154},{"type":41,"tag":138,"props":398,"children":399},{"style":157},[400],{"type":47,"value":160},{"type":41,"tag":138,"props":402,"children":403},{"class":140,"line":163},[404,408,412],{"type":41,"tag":138,"props":405,"children":406},{"style":151},[407],{"type":47,"value":169},{"type":41,"tag":138,"props":409,"children":410},{"style":151},[411],{"type":47,"value":174},{"type":41,"tag":138,"props":413,"children":414},{"style":157},[415],{"type":47,"value":160},{"type":41,"tag":138,"props":417,"children":418},{"class":140,"line":181},[419,424,429],{"type":41,"tag":138,"props":420,"children":421},{"style":151},[422],{"type":47,"value":423},"  --dataset-path",{"type":41,"tag":138,"props":425,"children":426},{"style":151},[427],{"type":47,"value":428}," ShareGPT_V3_unfiltered_cleaned_split.json",{"type":41,"tag":138,"props":430,"children":431},{"style":157},[432],{"type":47,"value":160},{"type":41,"tag":138,"props":434,"children":435},{"class":140,"line":194},[436,440],{"type":41,"tag":138,"props":437,"children":438},{"style":151},[439],{"type":47,"value":187},{"type":41,"tag":138,"props":441,"children":442},{"style":157},[443],{"type":47,"value":160},{"type":41,"tag":138,"props":445,"children":446},{"class":140,"line":213},[447,451,456],{"type":41,"tag":138,"props":448,"children":449},{"style":151},[450],{"type":47,"value":200},{"type":41,"tag":138,"props":452,"children":453},{"style":203},[454],{"type":47,"value":455}," 20",{"type":41,"tag":138,"props":457,"children":458},{"style":157},[459],{"type":47,"value":160},{"type":41,"tag":138,"props":461,"children":462},{"class":140,"line":231},[463,467,472],{"type":41,"tag":138,"props":464,"children":465},{"style":151},[466],{"type":47,"value":219},{"type":41,"tag":138,"props":468,"children":469},{"style":203},[470],{"type":47,"value":471}," 5",{"type":41,"tag":138,"props":473,"children":474},{"style":157},[475],{"type":47,"value":160},{"type":41,"tag":138,"props":477,"children":479},{"class":140,"line":478},7,[480,484],{"type":41,"tag":138,"props":481,"children":482},{"style":151},[483],{"type":47,"value":237},{"type":41,"tag":138,"props":485,"children":486},{"style":151},[487],{"type":47,"value":242},{"type":41,"tag":81,"props":489,"children":491},{"id":490},"option-3-prefix-repetition-dataset-online",[492],{"type":47,"value":493},"Option 3. Prefix Repetition Dataset (Online)",{"type":41,"tag":50,"props":495,"children":496},{},[497,499,504,506,511],{"type":47,"value":498},"Uses ",{"type":41,"tag":56,"props":500,"children":502},{"className":501},[],[503],{"type":47,"value":69},{"type":47,"value":505}," with the synthetic ",{"type":41,"tag":56,"props":507,"children":509},{"className":508},[],[510],{"type":47,"value":77},{"type":47,"value":512}," dataset to test caching via the serving API. This requires a running vLLM server.",{"type":41,"tag":50,"props":514,"children":515},{},[516],{"type":47,"value":517},"First, start the server:",{"type":41,"tag":127,"props":519,"children":521},{"className":129,"code":520,"language":131,"meta":132,"style":132},"vllm serve Qwen\u002FQwen3-8B\n",[522],{"type":41,"tag":56,"props":523,"children":524},{"__ignoreMap":132},[525],{"type":41,"tag":138,"props":526,"children":527},{"class":140,"line":141},[528,532,537],{"type":41,"tag":138,"props":529,"children":530},{"style":145},[531],{"type":47,"value":8},{"type":41,"tag":138,"props":533,"children":534},{"style":151},[535],{"type":47,"value":536}," serve",{"type":41,"tag":138,"props":538,"children":539},{"style":151},[540],{"type":47,"value":541}," Qwen\u002FQwen3-8B\n",{"type":41,"tag":50,"props":543,"children":544},{},[545],{"type":47,"value":378},{"type":41,"tag":127,"props":547,"children":549},{"className":129,"code":548,"language":131,"meta":132,"style":132},"vllm bench serve \\\n  --backend openai \\\n  --model Qwen\u002FQwen3-8B \\\n  --dataset-name prefix_repetition \\\n  --num-prompts 100 \\\n  --prefix-repetition-prefix-len 512 \\\n  --prefix-repetition-suffix-len 128 \\\n  --prefix-repetition-num-prefixes 5 \\\n  --prefix-repetition-output-len 128\n",[550],{"type":41,"tag":56,"props":551,"children":552},{"__ignoreMap":132},[553,573,590,605,622,637,654,671,688],{"type":41,"tag":138,"props":554,"children":555},{"class":140,"line":141},[556,560,565,569],{"type":41,"tag":138,"props":557,"children":558},{"style":145},[559],{"type":47,"value":8},{"type":41,"tag":138,"props":561,"children":562},{"style":151},[563],{"type":47,"value":564}," bench",{"type":41,"tag":138,"props":566,"children":567},{"style":151},[568],{"type":47,"value":536},{"type":41,"tag":138,"props":570,"children":571},{"style":157},[572],{"type":47,"value":160},{"type":41,"tag":138,"props":574,"children":575},{"class":140,"line":163},[576,581,586],{"type":41,"tag":138,"props":577,"children":578},{"style":151},[579],{"type":47,"value":580},"  --backend",{"type":41,"tag":138,"props":582,"children":583},{"style":151},[584],{"type":47,"value":585}," openai",{"type":41,"tag":138,"props":587,"children":588},{"style":157},[589],{"type":47,"value":160},{"type":41,"tag":138,"props":591,"children":592},{"class":140,"line":181},[593,597,601],{"type":41,"tag":138,"props":594,"children":595},{"style":151},[596],{"type":47,"value":169},{"type":41,"tag":138,"props":598,"children":599},{"style":151},[600],{"type":47,"value":174},{"type":41,"tag":138,"props":602,"children":603},{"style":157},[604],{"type":47,"value":160},{"type":41,"tag":138,"props":606,"children":607},{"class":140,"line":194},[608,613,618],{"type":41,"tag":138,"props":609,"children":610},{"style":151},[611],{"type":47,"value":612},"  --dataset-name",{"type":41,"tag":138,"props":614,"children":615},{"style":151},[616],{"type":47,"value":617}," prefix_repetition",{"type":41,"tag":138,"props":619,"children":620},{"style":157},[621],{"type":47,"value":160},{"type":41,"tag":138,"props":623,"children":624},{"class":140,"line":213},[625,629,633],{"type":41,"tag":138,"props":626,"children":627},{"style":151},[628],{"type":47,"value":200},{"type":41,"tag":138,"props":630,"children":631},{"style":203},[632],{"type":47,"value":224},{"type":41,"tag":138,"props":634,"children":635},{"style":157},[636],{"type":47,"value":160},{"type":41,"tag":138,"props":638,"children":639},{"class":140,"line":231},[640,645,650],{"type":41,"tag":138,"props":641,"children":642},{"style":151},[643],{"type":47,"value":644},"  --prefix-repetition-prefix-len",{"type":41,"tag":138,"props":646,"children":647},{"style":203},[648],{"type":47,"value":649}," 512",{"type":41,"tag":138,"props":651,"children":652},{"style":157},[653],{"type":47,"value":160},{"type":41,"tag":138,"props":655,"children":656},{"class":140,"line":478},[657,662,667],{"type":41,"tag":138,"props":658,"children":659},{"style":151},[660],{"type":47,"value":661},"  --prefix-repetition-suffix-len",{"type":41,"tag":138,"props":663,"children":664},{"style":203},[665],{"type":47,"value":666}," 128",{"type":41,"tag":138,"props":668,"children":669},{"style":157},[670],{"type":47,"value":160},{"type":41,"tag":138,"props":672,"children":674},{"class":140,"line":673},8,[675,680,684],{"type":41,"tag":138,"props":676,"children":677},{"style":151},[678],{"type":47,"value":679},"  --prefix-repetition-num-prefixes",{"type":41,"tag":138,"props":681,"children":682},{"style":203},[683],{"type":47,"value":471},{"type":41,"tag":138,"props":685,"children":686},{"style":157},[687],{"type":47,"value":160},{"type":41,"tag":138,"props":689,"children":691},{"class":140,"line":690},9,[692,697],{"type":41,"tag":138,"props":693,"children":694},{"style":151},[695],{"type":47,"value":696},"  --prefix-repetition-output-len",{"type":41,"tag":138,"props":698,"children":699},{"style":203},[700],{"type":47,"value":701}," 128\n",{"type":41,"tag":50,"props":703,"children":704},{},[705,707,712],{"type":47,"value":706},"Key parameters for ",{"type":41,"tag":56,"props":708,"children":710},{"className":709},[],[711],{"type":47,"value":77},{"type":47,"value":713},":",{"type":41,"tag":715,"props":716,"children":717},"table",{},[718,737],{"type":41,"tag":719,"props":720,"children":721},"thead",{},[722],{"type":41,"tag":723,"props":724,"children":725},"tr",{},[726,732],{"type":41,"tag":727,"props":728,"children":729},"th",{},[730],{"type":47,"value":731},"Parameter",{"type":41,"tag":727,"props":733,"children":734},{},[735],{"type":47,"value":736},"Description",{"type":41,"tag":738,"props":739,"children":740},"tbody",{},[741,759,776,793],{"type":41,"tag":723,"props":742,"children":743},{},[744,754],{"type":41,"tag":745,"props":746,"children":747},"td",{},[748],{"type":41,"tag":56,"props":749,"children":751},{"className":750},[],[752],{"type":47,"value":753},"--prefix-repetition-prefix-len",{"type":41,"tag":745,"props":755,"children":756},{},[757],{"type":47,"value":758},"Number of tokens in the shared prefix portion",{"type":41,"tag":723,"props":760,"children":761},{},[762,771],{"type":41,"tag":745,"props":763,"children":764},{},[765],{"type":41,"tag":56,"props":766,"children":768},{"className":767},[],[769],{"type":47,"value":770},"--prefix-repetition-suffix-len",{"type":41,"tag":745,"props":772,"children":773},{},[774],{"type":47,"value":775},"Number of tokens in the unique suffix portion",{"type":41,"tag":723,"props":777,"children":778},{},[779,788],{"type":41,"tag":745,"props":780,"children":781},{},[782],{"type":41,"tag":56,"props":783,"children":785},{"className":784},[],[786],{"type":47,"value":787},"--prefix-repetition-num-prefixes",{"type":41,"tag":745,"props":789,"children":790},{},[791],{"type":47,"value":792},"Number of distinct prefixes to cycle through",{"type":41,"tag":723,"props":794,"children":795},{},[796,805],{"type":41,"tag":745,"props":797,"children":798},{},[799],{"type":41,"tag":56,"props":800,"children":802},{"className":801},[],[803],{"type":47,"value":804},"--prefix-repetition-output-len",{"type":41,"tag":745,"props":806,"children":807},{},[808],{"type":47,"value":809},"Number of output tokens to generate per request",{"type":41,"tag":81,"props":811,"children":813},{"id":812},"notes",[814],{"type":47,"value":815},"Notes",{"type":41,"tag":88,"props":817,"children":818},{},[819,832,852,863,889,901],{"type":41,"tag":92,"props":820,"children":821},{},[822,824,830],{"type":47,"value":823},"Run all commands from the root of the vLLM repository (",{"type":41,"tag":56,"props":825,"children":827},{"className":826},[],[828],{"type":47,"value":829},"cd vllm",{"type":47,"value":831},").",{"type":41,"tag":92,"props":833,"children":834},{},[835,837,843,845,851],{"type":47,"value":836},"Keep the default model (",{"type":41,"tag":56,"props":838,"children":840},{"className":839},[],[841],{"type":47,"value":842},"Qwen\u002FQwen3-8B",{"type":47,"value":844},") unless the user specifies a different one or the model is unavailable; change only ",{"type":41,"tag":56,"props":846,"children":848},{"className":847},[],[849],{"type":47,"value":850},"--model",{"type":47,"value":109},{"type":41,"tag":92,"props":853,"children":854},{},[855,861],{"type":41,"tag":56,"props":856,"children":858},{"className":857},[],[859],{"type":47,"value":860},"--repeat-count",{"type":47,"value":862}," in Option 1 and 2 controls how many times each sampled prompt is replayed; higher values increase cache hit rate.",{"type":41,"tag":92,"props":864,"children":865},{},[866,872,874,880,882,888],{"type":41,"tag":56,"props":867,"children":869},{"className":868},[],[870],{"type":47,"value":871},"--input-length-range",{"type":47,"value":873}," accepts a ",{"type":41,"tag":56,"props":875,"children":877},{"className":876},[],[878],{"type":47,"value":879},"min:max",{"type":47,"value":881}," token range, e.g. ",{"type":41,"tag":56,"props":883,"children":885},{"className":884},[],[886],{"type":47,"value":887},"128:256",{"type":47,"value":109},{"type":41,"tag":92,"props":890,"children":891},{},[892,894,900],{"type":47,"value":893},"For multi-GPU setups, add ",{"type":41,"tag":56,"props":895,"children":897},{"className":896},[],[898],{"type":47,"value":899},"--tensor-parallel-size \u003CN>",{"type":47,"value":109},{"type":41,"tag":92,"props":902,"children":903},{},[904,906,912,914,920],{"type":47,"value":905},"To test different hash algorithms for prefix caching internals, use ",{"type":41,"tag":56,"props":907,"children":909},{"className":908},[],[910],{"type":47,"value":911},"--prefix-caching-hash-algo xxhash",{"type":47,"value":913}," (requires ",{"type":41,"tag":56,"props":915,"children":917},{"className":916},[],[918],{"type":47,"value":919},"pip install xxhash",{"type":47,"value":831},{"type":41,"tag":81,"props":922,"children":924},{"id":923},"arguments-for-benchmark_prefix_cachingpy",[925,927],{"type":47,"value":926},"Arguments for ",{"type":41,"tag":56,"props":928,"children":930},{"className":929},[],[931],{"type":47,"value":932},"benchmark_prefix_caching.py",{"type":41,"tag":715,"props":934,"children":935},{},[936,956],{"type":41,"tag":719,"props":937,"children":938},{},[939],{"type":41,"tag":723,"props":940,"children":941},{},[942,947,952],{"type":41,"tag":727,"props":943,"children":944},{},[945],{"type":47,"value":946},"Argument",{"type":41,"tag":727,"props":948,"children":949},{},[950],{"type":47,"value":951},"Required",{"type":41,"tag":727,"props":953,"children":954},{},[955],{"type":47,"value":736},{"type":41,"tag":738,"props":957,"children":958},{},[959,980,1001,1026,1047,1068,1089,1109,1130,1158,1207,1228],{"type":41,"tag":723,"props":960,"children":961},{},[962,970,975],{"type":41,"tag":745,"props":963,"children":964},{},[965],{"type":41,"tag":56,"props":966,"children":968},{"className":967},[],[969],{"type":47,"value":850},{"type":41,"tag":745,"props":971,"children":972},{},[973],{"type":47,"value":974},"Yes",{"type":41,"tag":745,"props":976,"children":977},{},[978],{"type":47,"value":979},"Model name or path (HuggingFace ID or local path)",{"type":41,"tag":723,"props":981,"children":982},{},[983,992,996],{"type":41,"tag":745,"props":984,"children":985},{},[986],{"type":41,"tag":56,"props":987,"children":989},{"className":988},[],[990],{"type":47,"value":991},"--num-prompts",{"type":41,"tag":745,"props":993,"children":994},{},[995],{"type":47,"value":974},{"type":41,"tag":745,"props":997,"children":998},{},[999],{"type":47,"value":1000},"Number of prompts to process",{"type":41,"tag":723,"props":1002,"children":1003},{},[1004,1012,1016],{"type":41,"tag":745,"props":1005,"children":1006},{},[1007],{"type":41,"tag":56,"props":1008,"children":1010},{"className":1009},[],[1011],{"type":47,"value":871},{"type":41,"tag":745,"props":1013,"children":1014},{},[1015],{"type":47,"value":974},{"type":41,"tag":745,"props":1017,"children":1018},{},[1019,1021],{"type":47,"value":1020},"Token length range for inputs, e.g. ",{"type":41,"tag":56,"props":1022,"children":1024},{"className":1023},[],[1025],{"type":47,"value":887},{"type":41,"tag":723,"props":1027,"children":1028},{},[1029,1037,1042],{"type":41,"tag":745,"props":1030,"children":1031},{},[1032],{"type":41,"tag":56,"props":1033,"children":1035},{"className":1034},[],[1036],{"type":47,"value":860},{"type":41,"tag":745,"props":1038,"children":1039},{},[1040],{"type":47,"value":1041},"No",{"type":41,"tag":745,"props":1043,"children":1044},{},[1045],{"type":47,"value":1046},"Number of times each prompt is repeated (default: 1)",{"type":41,"tag":723,"props":1048,"children":1049},{},[1050,1059,1063],{"type":41,"tag":745,"props":1051,"children":1052},{},[1053],{"type":41,"tag":56,"props":1054,"children":1056},{"className":1055},[],[1057],{"type":47,"value":1058},"--dataset-path",{"type":41,"tag":745,"props":1060,"children":1061},{},[1062],{"type":47,"value":1041},{"type":41,"tag":745,"props":1064,"children":1065},{},[1066],{"type":47,"value":1067},"Path to a dataset file (e.g. ShareGPT JSON). Omit for synthetic fixed-prompt mode",{"type":41,"tag":723,"props":1069,"children":1070},{},[1071,1080,1084],{"type":41,"tag":745,"props":1072,"children":1073},{},[1074],{"type":41,"tag":56,"props":1075,"children":1077},{"className":1076},[],[1078],{"type":47,"value":1079},"--prefix-len",{"type":41,"tag":745,"props":1081,"children":1082},{},[1083],{"type":47,"value":1041},{"type":41,"tag":745,"props":1085,"children":1086},{},[1087],{"type":47,"value":1088},"Fixed prefix token length to prepend to every prompt",{"type":41,"tag":723,"props":1090,"children":1091},{},[1092,1101,1105],{"type":41,"tag":745,"props":1093,"children":1094},{},[1095],{"type":41,"tag":56,"props":1096,"children":1098},{"className":1097},[],[1099],{"type":47,"value":1100},"--output-len",{"type":41,"tag":745,"props":1102,"children":1103},{},[1104],{"type":47,"value":1041},{"type":41,"tag":745,"props":1106,"children":1107},{},[1108],{"type":47,"value":809},{"type":41,"tag":723,"props":1110,"children":1111},{},[1112,1121,1125],{"type":41,"tag":745,"props":1113,"children":1114},{},[1115],{"type":41,"tag":56,"props":1116,"children":1118},{"className":1117},[],[1119],{"type":47,"value":1120},"--sort",{"type":41,"tag":745,"props":1122,"children":1123},{},[1124],{"type":47,"value":1041},{"type":41,"tag":745,"props":1126,"children":1127},{},[1128],{"type":47,"value":1129},"Sort prompts by length before benchmarking",{"type":41,"tag":723,"props":1131,"children":1132},{},[1133,1149,1153],{"type":41,"tag":745,"props":1134,"children":1135},{},[1136,1141,1143],{"type":41,"tag":56,"props":1137,"children":1139},{"className":1138},[],[1140],{"type":47,"value":107},{"type":47,"value":1142}," \u002F ",{"type":41,"tag":56,"props":1144,"children":1146},{"className":1145},[],[1147],{"type":47,"value":1148},"--no-enable-prefix-caching",{"type":41,"tag":745,"props":1150,"children":1151},{},[1152],{"type":47,"value":1041},{"type":41,"tag":745,"props":1154,"children":1155},{},[1156],{"type":47,"value":1157},"Toggle APC (recommended: enable to test caching)",{"type":41,"tag":723,"props":1159,"children":1160},{},[1161,1170,1174],{"type":41,"tag":745,"props":1162,"children":1163},{},[1164],{"type":41,"tag":56,"props":1165,"children":1167},{"className":1166},[],[1168],{"type":47,"value":1169},"--prefix-caching-hash-algo",{"type":41,"tag":745,"props":1171,"children":1172},{},[1173],{"type":47,"value":1041},{"type":41,"tag":745,"props":1175,"children":1176},{},[1177,1179,1185,1187,1193,1194,1200,1201],{"type":47,"value":1178},"Hash algorithm: ",{"type":41,"tag":56,"props":1180,"children":1182},{"className":1181},[],[1183],{"type":47,"value":1184},"sha256",{"type":47,"value":1186},", ",{"type":41,"tag":56,"props":1188,"children":1190},{"className":1189},[],[1191],{"type":47,"value":1192},"sha256_cbor",{"type":47,"value":1186},{"type":41,"tag":56,"props":1195,"children":1197},{"className":1196},[],[1198],{"type":47,"value":1199},"xxhash",{"type":47,"value":1186},{"type":41,"tag":56,"props":1202,"children":1204},{"className":1203},[],[1205],{"type":47,"value":1206},"xxhash_cbor",{"type":41,"tag":723,"props":1208,"children":1209},{},[1210,1219,1223],{"type":41,"tag":745,"props":1211,"children":1212},{},[1213],{"type":41,"tag":56,"props":1214,"children":1216},{"className":1215},[],[1217],{"type":47,"value":1218},"--tensor-parallel-size",{"type":41,"tag":745,"props":1220,"children":1221},{},[1222],{"type":47,"value":1041},{"type":41,"tag":745,"props":1224,"children":1225},{},[1226],{"type":47,"value":1227},"Number of GPUs for tensor parallelism",{"type":41,"tag":723,"props":1229,"children":1230},{},[1231,1240,1244],{"type":41,"tag":745,"props":1232,"children":1233},{},[1234],{"type":41,"tag":56,"props":1235,"children":1237},{"className":1236},[],[1238],{"type":47,"value":1239},"--disable-detokenize",{"type":41,"tag":745,"props":1241,"children":1242},{},[1243],{"type":47,"value":1041},{"type":41,"tag":745,"props":1245,"children":1246},{},[1247],{"type":47,"value":1248},"Skip detokenization to reduce overhead",{"type":41,"tag":81,"props":1250,"children":1252},{"id":1251},"troubleshooting",[1253],{"type":47,"value":1254},"Troubleshooting",{"type":41,"tag":88,"props":1256,"children":1257},{},[1258,1271],{"type":41,"tag":92,"props":1259,"children":1260},{},[1261,1263,1269],{"type":47,"value":1262},"If ",{"type":41,"tag":56,"props":1264,"children":1266},{"className":1265},[],[1267],{"type":47,"value":1268},"python3 benchmarks\u002F*.py",{"type":47,"value":1270}," reports file not found, locate your local vLLM repository first and run the command from that repo root.",{"type":41,"tag":92,"props":1272,"children":1273},{},[1274],{"type":47,"value":1275},"If you do not have the repository yet, clone it and continue:",{"type":41,"tag":127,"props":1277,"children":1279},{"className":129,"code":1278,"language":131,"meta":132,"style":132},"git clone https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm\ncd vllm\n",[1280],{"type":41,"tag":56,"props":1281,"children":1282},{"__ignoreMap":132},[1283,1301],{"type":41,"tag":138,"props":1284,"children":1285},{"class":140,"line":141},[1286,1291,1296],{"type":41,"tag":138,"props":1287,"children":1288},{"style":145},[1289],{"type":47,"value":1290},"git",{"type":41,"tag":138,"props":1292,"children":1293},{"style":151},[1294],{"type":47,"value":1295}," clone",{"type":41,"tag":138,"props":1297,"children":1298},{"style":151},[1299],{"type":47,"value":1300}," https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm\n",{"type":41,"tag":138,"props":1302,"children":1303},{"class":140,"line":163},[1304,1310],{"type":41,"tag":138,"props":1305,"children":1307},{"style":1306},"--shiki-light:#6182B8;--shiki-default:#82AAFF;--shiki-dark:#82AAFF",[1308],{"type":47,"value":1309},"cd",{"type":41,"tag":138,"props":1311,"children":1312},{"style":151},[1313],{"type":47,"value":1314}," vllm\n",{"type":41,"tag":88,"props":1316,"children":1317},{},[1318,1338],{"type":41,"tag":92,"props":1319,"children":1320},{},[1321,1323,1329,1331,1337],{"type":47,"value":1322},"If HuggingFace model download fails due to access restrictions, set your token: ",{"type":41,"tag":56,"props":1324,"children":1326},{"className":1325},[],[1327],{"type":47,"value":1328},"export HF_TOKEN=\u003Cyour_token>",{"type":47,"value":1330}," or pass ",{"type":41,"tag":56,"props":1332,"children":1334},{"className":1333},[],[1335],{"type":47,"value":1336},"--hf-token \u003Cyour_token>",{"type":47,"value":109},{"type":41,"tag":92,"props":1339,"children":1340},{},[1341,1342,1347,1349,1355,1357,1363],{"type":47,"value":1262},{"type":41,"tag":56,"props":1343,"children":1345},{"className":1344},[],[1346],{"type":47,"value":1199},{"type":47,"value":1348}," or ",{"type":41,"tag":56,"props":1350,"children":1352},{"className":1351},[],[1353],{"type":47,"value":1354},"cbor2",{"type":47,"value":1356}," is not installed and you use those hash algorithms, install them first: ",{"type":41,"tag":56,"props":1358,"children":1360},{"className":1359},[],[1361],{"type":47,"value":1362},"pip install xxhash cbor2",{"type":47,"value":109},{"type":41,"tag":1365,"props":1366,"children":1367},"style",{},[1368],{"type":47,"value":1369},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":1371,"total":478},[1372,1389,1400,1414,1429,1445,1456],{"slug":1373,"name":1373,"fn":1374,"description":1375,"org":1376,"tags":1377,"stars":1386,"repoUrl":1387,"updatedAt":1388},"openclaw-vsr-bridge","integrate vLLM Semantic Router","Install vLLM Semantic Router in agent-safe mode, import supported OpenClaw model providers into canonical VSR config, and rewrite OpenClaw to target VSR.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1378,1381,1382,1385],{"name":1379,"slug":1380,"type":16},"Agents","agents",{"name":21,"slug":22,"type":16},{"name":1383,"slug":1384,"type":16},"Routing","routing",{"name":9,"slug":8,"type":16},4962,"https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fsemantic-router","2026-07-18T05:48:09.091161",{"slug":1390,"name":1390,"fn":1391,"description":1392,"org":1393,"tags":1394,"stars":24,"repoUrl":25,"updatedAt":1399},"vllm-bench-random-synthetic","run vLLM synthetic performance benchmarks","Run vLLM performance benchmark using synthetic random data to measure throughput, TTFT (Time to First Token), TPOT (Time per Output Token), and other key performance metrics. Use when the user wants to quickly test vLLM serving performance without downloading external datasets.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1395,1396,1397,1398],{"name":18,"slug":19,"type":16},{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16},"2026-07-18T05:46:59.036927",{"slug":1401,"name":1401,"fn":1402,"description":1403,"org":1404,"tags":1405,"stars":24,"repoUrl":25,"updatedAt":1413},"vllm-bench-serve","benchmark LLM serving endpoints","Benchmark vLLM or OpenAI-compatible serving endpoints using vllm bench serve. Supports multiple datasets (random, sharegpt, sonnet, HF), backends (openai, openai-chat, vllm-pooling, embeddings), throughput\u002Flatency testing with request-rate control, and result saving. Use when benchmarking LLM serving performance, measuring TTFT\u002FTPOT, or load testing inference APIs.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1406,1409,1410,1411,1412],{"name":1407,"slug":1408,"type":16},"API Development","api-development",{"name":18,"slug":19,"type":16},{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16},"2026-07-18T05:47:02.283164",{"slug":1415,"name":1415,"fn":1416,"description":1417,"org":1418,"tags":1419,"stars":24,"repoUrl":25,"updatedAt":1428},"vllm-deploy-docker","deploy vLLM using Docker containers","Deploy vLLM using Docker (pre-built images or build-from-source) with NVIDIA GPU support and run the OpenAI-compatible server.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1420,1423,1426,1427],{"name":1421,"slug":1422,"type":16},"Deployment","deployment",{"name":1424,"slug":1425,"type":16},"Docker","docker",{"name":21,"slug":22,"type":16},{"name":9,"slug":8,"type":16},"2026-07-18T05:47:01.814419",{"slug":1430,"name":1430,"fn":1431,"description":1432,"org":1433,"tags":1434,"stars":24,"repoUrl":25,"updatedAt":1444},"vllm-deploy-k8s","deploy vLLM to Kubernetes clusters","Deploy vLLM to Kubernetes (K8s) with GPU support, health probes, and OpenAI-compatible API endpoint. Use this skill whenever the user wants to deploy, run, or serve vLLM on a Kubernetes cluster, including creating deployments, services, checking existing deployments, or managing vLLM on K8s.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1435,1438,1439,1442,1443],{"name":1436,"slug":1437,"type":16},"AI Infrastructure","ai-infrastructure",{"name":1421,"slug":1422,"type":16},{"name":1440,"slug":1441,"type":16},"Kubernetes","kubernetes",{"name":21,"slug":22,"type":16},{"name":9,"slug":8,"type":16},"2026-07-18T05:46:58.556637",{"slug":1446,"name":1446,"fn":1447,"description":1448,"org":1449,"tags":1450,"stars":24,"repoUrl":25,"updatedAt":1455},"vllm-deploy-simple","deploy and test vLLM servers","Quick install and deploy vLLM, start serving with a simple LLM, and test OpenAI API.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1451,1452,1453,1454],{"name":1407,"slug":1408,"type":16},{"name":1421,"slug":1422,"type":16},{"name":21,"slug":22,"type":16},{"name":9,"slug":8,"type":16},"2026-07-18T05:47:01.359655",{"slug":4,"name":4,"fn":5,"description":6,"org":1457,"tags":1458,"stars":24,"repoUrl":25,"updatedAt":26},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1459,1460,1461,1462],{"name":18,"slug":19,"type":16},{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16},{"items":1464,"total":231},[1465,1472,1480,1487,1495,1502],{"slug":1390,"name":1390,"fn":1391,"description":1392,"org":1466,"tags":1467,"stars":24,"repoUrl":25,"updatedAt":1399},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1468,1469,1470,1471],{"name":18,"slug":19,"type":16},{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16},{"slug":1401,"name":1401,"fn":1402,"description":1403,"org":1473,"tags":1474,"stars":24,"repoUrl":25,"updatedAt":1413},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1475,1476,1477,1478,1479],{"name":1407,"slug":1408,"type":16},{"name":18,"slug":19,"type":16},{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16},{"slug":1415,"name":1415,"fn":1416,"description":1417,"org":1481,"tags":1482,"stars":24,"repoUrl":25,"updatedAt":1428},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1483,1484,1485,1486],{"name":1421,"slug":1422,"type":16},{"name":1424,"slug":1425,"type":16},{"name":21,"slug":22,"type":16},{"name":9,"slug":8,"type":16},{"slug":1430,"name":1430,"fn":1431,"description":1432,"org":1488,"tags":1489,"stars":24,"repoUrl":25,"updatedAt":1444},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1490,1491,1492,1493,1494],{"name":1436,"slug":1437,"type":16},{"name":1421,"slug":1422,"type":16},{"name":1440,"slug":1441,"type":16},{"name":21,"slug":22,"type":16},{"name":9,"slug":8,"type":16},{"slug":1446,"name":1446,"fn":1447,"description":1448,"org":1496,"tags":1497,"stars":24,"repoUrl":25,"updatedAt":1455},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1498,1499,1500,1501],{"name":1407,"slug":1408,"type":16},{"name":1421,"slug":1422,"type":16},{"name":21,"slug":22,"type":16},{"name":9,"slug":8,"type":16},{"slug":4,"name":4,"fn":5,"description":6,"org":1503,"tags":1504,"stars":24,"repoUrl":25,"updatedAt":26},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1505,1506,1507,1508],{"name":18,"slug":19,"type":16},{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16}]