[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-vllm-vllm-bench-random-synthetic":3,"mdc--urh94q-key":35,"related-org-vllm-vllm-bench-random-synthetic":1401,"related-repo-vllm-vllm-bench-random-synthetic":1494},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":12,"stars":24,"repoUrl":25,"updatedAt":26,"license":27,"forks":28,"topics":29,"repo":30,"sourceUrl":33,"mdContent":34},"vllm-bench-random-synthetic","run vLLM synthetic performance benchmarks","Run vLLM performance benchmark using synthetic random data to measure throughput, TTFT (Time to First Token), TPOT (Time per Output Token), and other key performance metrics. Use when the user wants to quickly test vLLM serving performance without downloading external datasets.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},"vllm","vLLM","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fvllm.png","vllm-project",[13,17,20,23],{"name":14,"slug":15,"type":16},"Performance","performance","tag",{"name":18,"slug":19,"type":16},"Benchmarking","benchmarking",{"name":21,"slug":22,"type":16},"LLM","llm",{"name":9,"slug":8,"type":16},88,"https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm-skills","2026-07-18T05:46:59.036927",null,23,[],{"repoUrl":25,"stars":24,"forks":28,"topics":31,"description":32},[],"Agent skills for vLLM","https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm-skills\u002Ftree\u002FHEAD\u002Fplugins\u002Fvllm-skills\u002Fskills\u002Fvllm-bench-random-synthetic","---\nname: vllm-bench-random-synthetic\ndescription: Run vLLM performance benchmark using synthetic random data to measure throughput, TTFT (Time to First Token), TPOT (Time per Output Token), and other key performance metrics. Use when the user wants to quickly test vLLM serving performance without downloading external datasets.\n---\n\n# vLLM Benchmark with Random Synthetic Data\n\nRun a quick performance benchmark on a vLLM server using synthetic random data. This skill measures core serving metrics including request throughput, token throughput, TTFT (Time to First Token), TPOT (Time per Output Token), and inter-token latency.\n\n## When to use\n\n- User wants to quickly benchmark vLLM serving performance\n- User wants to measure throughput and latency metrics without downloading datasets\n- User wants to test a vLLM deployment with synthetic workload\n- User wants baseline performance numbers for a specific model\n\n## Prerequisites\n\n- vLLM must be installed (`pip install vllm`)\n- A vLLM server must be running (or can be started as part of the benchmark)\n- For GPU models, NVIDIA GPU with appropriate drivers must be available\n\n## Quick Start\n\nThe simplest way to run the benchmark:\n\n```bash\n# Start vLLM server (in background or separate terminal)\nvllm serve Qwen\u002FQwen2.5-1.5B-Instruct\n\n# Run benchmark with random synthetic data\nvllm bench serve \\\n  --backend openai-chat \\\n  --model Qwen\u002FQwen2.5-1.5B-Instruct \\\n  --endpoint \u002Fv1\u002Fchat\u002Fcompletions \\\n  --dataset-name random \\\n  --num-prompts 10\n```\n\n**Note**: \n- Use `--backend openai-chat` with endpoint `\u002Fv1\u002Fchat\u002Fcompletions` for online benchmarks.\n\n## Parameters\n\n| Parameter | Description | Default |\n|-----------|-------------|---------|\n| `--backend` | Backend type: `vllm`, `openai`, `openai-chat` | `vllm` |\n| `--model` | Model name (must match the server) | Required |\n| `--endpoint` | API endpoint path | `\u002Fv1\u002Fcompletions` or `\u002Fv1\u002Fchat\u002Fcompletions` |\n| `--dataset-name` | Dataset to use | `random` (synthetic) |\n| `--num-prompts` | Number of requests to send | `10` |\n| `--port` | Server port | `8000` |\n| `--max-concurrency` | Maximum concurrent requests | Auto |\n| `--save-result` | Save results to file | Off |\n| `--result-dir` | Directory to save results | `.\u002F` |\n\n## Expected Output\n\nWhen successful, you will see output like:\n\n```\n============ Serving Benchmark Result ============\nSuccessful requests:                     10\nBenchmark duration (s):                  5.78\nTotal input tokens:                      1369\nTotal generated tokens:                  2212\nRequest throughput (req\u002Fs):              1.73\nOutput token throughput (tok\u002Fs):         382.89\nTotal token throughput (tok\u002Fs):          619.85\n---------------Time to First Token----------------\nMean TTFT (ms):                          71.54\nMedian TTFT (ms):                        73.88\nP99 TTFT (ms):                           79.49\n-----Time per Output Token (excl. 1st token)------\nMean TPOT (ms):                          7.91\nMedian TPOT (ms):                        7.96\nP99 TPOT (ms):                           8.03\n---------------Inter-token Latency----------------\nMean ITL (ms):                           7.74\nMedian ITL (ms):                         7.70\nP99 ITL (ms):                            8.39\n==================================================\n```\n\n## Advanced Usage\n\n### With more prompts for better statistics\n\n```bash\nvllm bench serve \\\n  --backend openai-chat \\\n  --model Qwen\u002FQwen2.5-1.5B-Instruct \\\n  --endpoint \u002Fv1\u002Fchat\u002Fcompletions \\\n  --dataset-name random \\\n  --num-prompts 100\n```\n\n### Save results to file\n\n```bash\nvllm bench serve \\\n  --backend openai-chat \\\n  --model Qwen\u002FQwen2.5-1.5B-Instruct \\\n  --endpoint \u002Fv1\u002Fchat\u002Fcompletions \\\n  --dataset-name random \\\n  --num-prompts 50 \\\n  --save-result \\\n  --result-dir .\u002Fbenchmark-results\u002F\n```\n\n### Custom port and concurrency\n\n```bash\nvllm bench serve \\\n  --backend openai-chat \\\n  --model meta-llama\u002FLlama-3.1-8B-Instruct \\\n  --endpoint \u002Fv1\u002Fchat\u002Fcompletions \\\n  --dataset-name random \\\n  --num-prompts 100 \\\n  --port 8001 \\\n  --max-concurrency 4\n```\n\n## Model Recommendations\n\nFor quick testing (small models, fast):\n- `Qwen\u002FQwen2.5-1.5B-Instruct` (recommended for quick tests)\n- `facebook\u002Fopt-125m`\n- `facebook\u002Fopt-350m`\n\nFor realistic benchmarks (medium models):\n- `Qwen\u002FQwen2.5-7B-Instruct`\n- `meta-llama\u002FLlama-3.1-8B-Instruct`\n- `mistralai\u002FMistral-7B-Instruct-v0.3`\n\n## Workflow\n\n1. **Check if vLLM is installed**: Run `vllm --version` to verify\n2. **Check if server is already running**: Run `curl http:\u002F\u002Flocalhost:8000\u002Fhealth` to check\n3. **Start vLLM server if needed**: Run `vllm serve \u003Cmodel-name>` (wait for \"Application startup complete\")\n4. **Run benchmark**: Execute `vllm bench serve` with appropriate parameters\n5. **Review results**: Check throughput and latency metrics\n6. **Clean up**: If the agent skill started the vLLM server (not a pre-existing one), stop it after benchmark completion using `kill \u003CPID>`\n\n## Troubleshooting\n\n**Server not responding**:\n- Check if server is running: `curl http:\u002F\u002Flocalhost:8000\u002Fhealth`\n- Verify port matches: Use `--port` flag if server is on different port\n\n**Model not found**:\n- Ensure model name matches exactly between server and benchmark\n- Check HuggingFace access: `export HF_TOKEN=\u003Cyour_token>` if needed\n\n**Out of memory**:\n- Use a smaller model (e.g., Qwen2.5-1.5B-Instruct)\n- Reduce `--num-prompts` or `--max-concurrency`\n\n**Connection refused**:\n- Server may still be starting (wait for \"Application startup complete\")\n- Check firewall or network settings\n\n## Notes\n\n- The `random` dataset generates synthetic prompts automatically\n- Benchmark duration scales with `--num-prompts`\n- For production benchmarking, use at least 100 prompts for stable statistics\n- Results may vary based on hardware, model size, and system load\n- First run may be slower due to model loading and compilation\n- **Important**: If the agent skill starts a vLLM server for benchmarking, it must stop the server after the benchmark completes to free up resources. Do not stop pre-existing servers that were already running before the benchmark.\n",{"data":36,"body":37},{"name":4,"description":6},{"type":38,"children":39},"root",[40,49,55,62,87,93,120,126,131,303,314,338,344,627,633,638,648,654,661,759,764,891,897,1030,1036,1041,1073,1078,1108,1114,1214,1220,1229,1254,1263,1284,1293,1317,1326,1339,1345,1395],{"type":41,"tag":42,"props":43,"children":45},"element","h1",{"id":44},"vllm-benchmark-with-random-synthetic-data",[46],{"type":47,"value":48},"text","vLLM Benchmark with Random Synthetic Data",{"type":41,"tag":50,"props":51,"children":52},"p",{},[53],{"type":47,"value":54},"Run a quick performance benchmark on a vLLM server using synthetic random data. This skill measures core serving metrics including request throughput, token throughput, TTFT (Time to First Token), TPOT (Time per Output Token), and inter-token latency.",{"type":41,"tag":56,"props":57,"children":59},"h2",{"id":58},"when-to-use",[60],{"type":47,"value":61},"When to use",{"type":41,"tag":63,"props":64,"children":65},"ul",{},[66,72,77,82],{"type":41,"tag":67,"props":68,"children":69},"li",{},[70],{"type":47,"value":71},"User wants to quickly benchmark vLLM serving performance",{"type":41,"tag":67,"props":73,"children":74},{},[75],{"type":47,"value":76},"User wants to measure throughput and latency metrics without downloading datasets",{"type":41,"tag":67,"props":78,"children":79},{},[80],{"type":47,"value":81},"User wants to test a vLLM deployment with synthetic workload",{"type":41,"tag":67,"props":83,"children":84},{},[85],{"type":47,"value":86},"User wants baseline performance numbers for a specific model",{"type":41,"tag":56,"props":88,"children":90},{"id":89},"prerequisites",[91],{"type":47,"value":92},"Prerequisites",{"type":41,"tag":63,"props":94,"children":95},{},[96,110,115],{"type":41,"tag":67,"props":97,"children":98},{},[99,101,108],{"type":47,"value":100},"vLLM must be installed (",{"type":41,"tag":102,"props":103,"children":105},"code",{"className":104},[],[106],{"type":47,"value":107},"pip install vllm",{"type":47,"value":109},")",{"type":41,"tag":67,"props":111,"children":112},{},[113],{"type":47,"value":114},"A vLLM server must be running (or can be started as part of the benchmark)",{"type":41,"tag":67,"props":116,"children":117},{},[118],{"type":47,"value":119},"For GPU models, NVIDIA GPU with appropriate drivers must be available",{"type":41,"tag":56,"props":121,"children":123},{"id":122},"quick-start",[124],{"type":47,"value":125},"Quick Start",{"type":41,"tag":50,"props":127,"children":128},{},[129],{"type":47,"value":130},"The simplest way to run the benchmark:",{"type":41,"tag":132,"props":133,"children":138},"pre",{"className":134,"code":135,"language":136,"meta":137,"style":137},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","# Start vLLM server (in background or separate terminal)\nvllm serve Qwen\u002FQwen2.5-1.5B-Instruct\n\n# Run benchmark with random synthetic data\nvllm bench serve \\\n  --backend openai-chat \\\n  --model Qwen\u002FQwen2.5-1.5B-Instruct \\\n  --endpoint \u002Fv1\u002Fchat\u002Fcompletions \\\n  --dataset-name random \\\n  --num-prompts 10\n","bash","",[139],{"type":41,"tag":102,"props":140,"children":141},{"__ignoreMap":137},[142,154,174,184,193,216,234,252,270,288],{"type":41,"tag":143,"props":144,"children":147},"span",{"class":145,"line":146},"line",1,[148],{"type":41,"tag":143,"props":149,"children":151},{"style":150},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[152],{"type":47,"value":153},"# Start vLLM server (in background or separate terminal)\n",{"type":41,"tag":143,"props":155,"children":157},{"class":145,"line":156},2,[158,163,169],{"type":41,"tag":143,"props":159,"children":161},{"style":160},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[162],{"type":47,"value":8},{"type":41,"tag":143,"props":164,"children":166},{"style":165},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[167],{"type":47,"value":168}," serve",{"type":41,"tag":143,"props":170,"children":171},{"style":165},[172],{"type":47,"value":173}," Qwen\u002FQwen2.5-1.5B-Instruct\n",{"type":41,"tag":143,"props":175,"children":177},{"class":145,"line":176},3,[178],{"type":41,"tag":143,"props":179,"children":181},{"emptyLinePlaceholder":180},true,[182],{"type":47,"value":183},"\n",{"type":41,"tag":143,"props":185,"children":187},{"class":145,"line":186},4,[188],{"type":41,"tag":143,"props":189,"children":190},{"style":150},[191],{"type":47,"value":192},"# Run benchmark with random synthetic data\n",{"type":41,"tag":143,"props":194,"children":196},{"class":145,"line":195},5,[197,201,206,210],{"type":41,"tag":143,"props":198,"children":199},{"style":160},[200],{"type":47,"value":8},{"type":41,"tag":143,"props":202,"children":203},{"style":165},[204],{"type":47,"value":205}," bench",{"type":41,"tag":143,"props":207,"children":208},{"style":165},[209],{"type":47,"value":168},{"type":41,"tag":143,"props":211,"children":213},{"style":212},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[214],{"type":47,"value":215}," \\\n",{"type":41,"tag":143,"props":217,"children":219},{"class":145,"line":218},6,[220,225,230],{"type":41,"tag":143,"props":221,"children":222},{"style":165},[223],{"type":47,"value":224},"  --backend",{"type":41,"tag":143,"props":226,"children":227},{"style":165},[228],{"type":47,"value":229}," openai-chat",{"type":41,"tag":143,"props":231,"children":232},{"style":212},[233],{"type":47,"value":215},{"type":41,"tag":143,"props":235,"children":237},{"class":145,"line":236},7,[238,243,248],{"type":41,"tag":143,"props":239,"children":240},{"style":165},[241],{"type":47,"value":242},"  --model",{"type":41,"tag":143,"props":244,"children":245},{"style":165},[246],{"type":47,"value":247}," Qwen\u002FQwen2.5-1.5B-Instruct",{"type":41,"tag":143,"props":249,"children":250},{"style":212},[251],{"type":47,"value":215},{"type":41,"tag":143,"props":253,"children":255},{"class":145,"line":254},8,[256,261,266],{"type":41,"tag":143,"props":257,"children":258},{"style":165},[259],{"type":47,"value":260},"  --endpoint",{"type":41,"tag":143,"props":262,"children":263},{"style":165},[264],{"type":47,"value":265}," \u002Fv1\u002Fchat\u002Fcompletions",{"type":41,"tag":143,"props":267,"children":268},{"style":212},[269],{"type":47,"value":215},{"type":41,"tag":143,"props":271,"children":273},{"class":145,"line":272},9,[274,279,284],{"type":41,"tag":143,"props":275,"children":276},{"style":165},[277],{"type":47,"value":278},"  --dataset-name",{"type":41,"tag":143,"props":280,"children":281},{"style":165},[282],{"type":47,"value":283}," random",{"type":41,"tag":143,"props":285,"children":286},{"style":212},[287],{"type":47,"value":215},{"type":41,"tag":143,"props":289,"children":291},{"class":145,"line":290},10,[292,297],{"type":41,"tag":143,"props":293,"children":294},{"style":165},[295],{"type":47,"value":296},"  --num-prompts",{"type":41,"tag":143,"props":298,"children":300},{"style":299},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[301],{"type":47,"value":302}," 10\n",{"type":41,"tag":50,"props":304,"children":305},{},[306,312],{"type":41,"tag":307,"props":308,"children":309},"strong",{},[310],{"type":47,"value":311},"Note",{"type":47,"value":313},":",{"type":41,"tag":63,"props":315,"children":316},{},[317],{"type":41,"tag":67,"props":318,"children":319},{},[320,322,328,330,336],{"type":47,"value":321},"Use ",{"type":41,"tag":102,"props":323,"children":325},{"className":324},[],[326],{"type":47,"value":327},"--backend openai-chat",{"type":47,"value":329}," with endpoint ",{"type":41,"tag":102,"props":331,"children":333},{"className":332},[],[334],{"type":47,"value":335},"\u002Fv1\u002Fchat\u002Fcompletions",{"type":47,"value":337}," for online benchmarks.",{"type":41,"tag":56,"props":339,"children":341},{"id":340},"parameters",[342],{"type":47,"value":343},"Parameters",{"type":41,"tag":345,"props":346,"children":347},"table",{},[348,372],{"type":41,"tag":349,"props":350,"children":351},"thead",{},[352],{"type":41,"tag":353,"props":354,"children":355},"tr",{},[356,362,367],{"type":41,"tag":357,"props":358,"children":359},"th",{},[360],{"type":47,"value":361},"Parameter",{"type":41,"tag":357,"props":363,"children":364},{},[365],{"type":47,"value":366},"Description",{"type":41,"tag":357,"props":368,"children":369},{},[370],{"type":47,"value":371},"Default",{"type":41,"tag":373,"props":374,"children":375},"tbody",{},[376,422,444,477,505,531,557,579,601],{"type":41,"tag":353,"props":377,"children":378},{},[379,389,414],{"type":41,"tag":380,"props":381,"children":382},"td",{},[383],{"type":41,"tag":102,"props":384,"children":386},{"className":385},[],[387],{"type":47,"value":388},"--backend",{"type":41,"tag":380,"props":390,"children":391},{},[392,394,399,401,407,408],{"type":47,"value":393},"Backend type: ",{"type":41,"tag":102,"props":395,"children":397},{"className":396},[],[398],{"type":47,"value":8},{"type":47,"value":400},", ",{"type":41,"tag":102,"props":402,"children":404},{"className":403},[],[405],{"type":47,"value":406},"openai",{"type":47,"value":400},{"type":41,"tag":102,"props":409,"children":411},{"className":410},[],[412],{"type":47,"value":413},"openai-chat",{"type":41,"tag":380,"props":415,"children":416},{},[417],{"type":41,"tag":102,"props":418,"children":420},{"className":419},[],[421],{"type":47,"value":8},{"type":41,"tag":353,"props":423,"children":424},{},[425,434,439],{"type":41,"tag":380,"props":426,"children":427},{},[428],{"type":41,"tag":102,"props":429,"children":431},{"className":430},[],[432],{"type":47,"value":433},"--model",{"type":41,"tag":380,"props":435,"children":436},{},[437],{"type":47,"value":438},"Model name (must match the server)",{"type":41,"tag":380,"props":440,"children":441},{},[442],{"type":47,"value":443},"Required",{"type":41,"tag":353,"props":445,"children":446},{},[447,456,461],{"type":41,"tag":380,"props":448,"children":449},{},[450],{"type":41,"tag":102,"props":451,"children":453},{"className":452},[],[454],{"type":47,"value":455},"--endpoint",{"type":41,"tag":380,"props":457,"children":458},{},[459],{"type":47,"value":460},"API endpoint path",{"type":41,"tag":380,"props":462,"children":463},{},[464,470,472],{"type":41,"tag":102,"props":465,"children":467},{"className":466},[],[468],{"type":47,"value":469},"\u002Fv1\u002Fcompletions",{"type":47,"value":471}," or ",{"type":41,"tag":102,"props":473,"children":475},{"className":474},[],[476],{"type":47,"value":335},{"type":41,"tag":353,"props":478,"children":479},{},[480,489,494],{"type":41,"tag":380,"props":481,"children":482},{},[483],{"type":41,"tag":102,"props":484,"children":486},{"className":485},[],[487],{"type":47,"value":488},"--dataset-name",{"type":41,"tag":380,"props":490,"children":491},{},[492],{"type":47,"value":493},"Dataset to use",{"type":41,"tag":380,"props":495,"children":496},{},[497,503],{"type":41,"tag":102,"props":498,"children":500},{"className":499},[],[501],{"type":47,"value":502},"random",{"type":47,"value":504}," (synthetic)",{"type":41,"tag":353,"props":506,"children":507},{},[508,517,522],{"type":41,"tag":380,"props":509,"children":510},{},[511],{"type":41,"tag":102,"props":512,"children":514},{"className":513},[],[515],{"type":47,"value":516},"--num-prompts",{"type":41,"tag":380,"props":518,"children":519},{},[520],{"type":47,"value":521},"Number of requests to send",{"type":41,"tag":380,"props":523,"children":524},{},[525],{"type":41,"tag":102,"props":526,"children":528},{"className":527},[],[529],{"type":47,"value":530},"10",{"type":41,"tag":353,"props":532,"children":533},{},[534,543,548],{"type":41,"tag":380,"props":535,"children":536},{},[537],{"type":41,"tag":102,"props":538,"children":540},{"className":539},[],[541],{"type":47,"value":542},"--port",{"type":41,"tag":380,"props":544,"children":545},{},[546],{"type":47,"value":547},"Server port",{"type":41,"tag":380,"props":549,"children":550},{},[551],{"type":41,"tag":102,"props":552,"children":554},{"className":553},[],[555],{"type":47,"value":556},"8000",{"type":41,"tag":353,"props":558,"children":559},{},[560,569,574],{"type":41,"tag":380,"props":561,"children":562},{},[563],{"type":41,"tag":102,"props":564,"children":566},{"className":565},[],[567],{"type":47,"value":568},"--max-concurrency",{"type":41,"tag":380,"props":570,"children":571},{},[572],{"type":47,"value":573},"Maximum concurrent requests",{"type":41,"tag":380,"props":575,"children":576},{},[577],{"type":47,"value":578},"Auto",{"type":41,"tag":353,"props":580,"children":581},{},[582,591,596],{"type":41,"tag":380,"props":583,"children":584},{},[585],{"type":41,"tag":102,"props":586,"children":588},{"className":587},[],[589],{"type":47,"value":590},"--save-result",{"type":41,"tag":380,"props":592,"children":593},{},[594],{"type":47,"value":595},"Save results to file",{"type":41,"tag":380,"props":597,"children":598},{},[599],{"type":47,"value":600},"Off",{"type":41,"tag":353,"props":602,"children":603},{},[604,613,618],{"type":41,"tag":380,"props":605,"children":606},{},[607],{"type":41,"tag":102,"props":608,"children":610},{"className":609},[],[611],{"type":47,"value":612},"--result-dir",{"type":41,"tag":380,"props":614,"children":615},{},[616],{"type":47,"value":617},"Directory to save results",{"type":41,"tag":380,"props":619,"children":620},{},[621],{"type":41,"tag":102,"props":622,"children":624},{"className":623},[],[625],{"type":47,"value":626},".\u002F",{"type":41,"tag":56,"props":628,"children":630},{"id":629},"expected-output",[631],{"type":47,"value":632},"Expected Output",{"type":41,"tag":50,"props":634,"children":635},{},[636],{"type":47,"value":637},"When successful, you will see output like:",{"type":41,"tag":132,"props":639,"children":643},{"className":640,"code":642,"language":47},[641],"language-text","============ Serving Benchmark Result ============\nSuccessful requests:                     10\nBenchmark duration (s):                  5.78\nTotal input tokens:                      1369\nTotal generated tokens:                  2212\nRequest throughput (req\u002Fs):              1.73\nOutput token throughput (tok\u002Fs):         382.89\nTotal token throughput (tok\u002Fs):          619.85\n---------------Time to First Token----------------\nMean TTFT (ms):                          71.54\nMedian TTFT (ms):                        73.88\nP99 TTFT (ms):                           79.49\n-----Time per Output Token (excl. 1st token)------\nMean TPOT (ms):                          7.91\nMedian TPOT (ms):                        7.96\nP99 TPOT (ms):                           8.03\n---------------Inter-token Latency----------------\nMean ITL (ms):                           7.74\nMedian ITL (ms):                         7.70\nP99 ITL (ms):                            8.39\n==================================================\n",[644],{"type":41,"tag":102,"props":645,"children":646},{"__ignoreMap":137},[647],{"type":47,"value":642},{"type":41,"tag":56,"props":649,"children":651},{"id":650},"advanced-usage",[652],{"type":47,"value":653},"Advanced Usage",{"type":41,"tag":655,"props":656,"children":658},"h3",{"id":657},"with-more-prompts-for-better-statistics",[659],{"type":47,"value":660},"With more prompts for better statistics",{"type":41,"tag":132,"props":662,"children":664},{"className":134,"code":663,"language":136,"meta":137,"style":137},"vllm bench serve \\\n  --backend openai-chat \\\n  --model Qwen\u002FQwen2.5-1.5B-Instruct \\\n  --endpoint \u002Fv1\u002Fchat\u002Fcompletions \\\n  --dataset-name random \\\n  --num-prompts 100\n",[665],{"type":41,"tag":102,"props":666,"children":667},{"__ignoreMap":137},[668,687,702,717,732,747],{"type":41,"tag":143,"props":669,"children":670},{"class":145,"line":146},[671,675,679,683],{"type":41,"tag":143,"props":672,"children":673},{"style":160},[674],{"type":47,"value":8},{"type":41,"tag":143,"props":676,"children":677},{"style":165},[678],{"type":47,"value":205},{"type":41,"tag":143,"props":680,"children":681},{"style":165},[682],{"type":47,"value":168},{"type":41,"tag":143,"props":684,"children":685},{"style":212},[686],{"type":47,"value":215},{"type":41,"tag":143,"props":688,"children":689},{"class":145,"line":156},[690,694,698],{"type":41,"tag":143,"props":691,"children":692},{"style":165},[693],{"type":47,"value":224},{"type":41,"tag":143,"props":695,"children":696},{"style":165},[697],{"type":47,"value":229},{"type":41,"tag":143,"props":699,"children":700},{"style":212},[701],{"type":47,"value":215},{"type":41,"tag":143,"props":703,"children":704},{"class":145,"line":176},[705,709,713],{"type":41,"tag":143,"props":706,"children":707},{"style":165},[708],{"type":47,"value":242},{"type":41,"tag":143,"props":710,"children":711},{"style":165},[712],{"type":47,"value":247},{"type":41,"tag":143,"props":714,"children":715},{"style":212},[716],{"type":47,"value":215},{"type":41,"tag":143,"props":718,"children":719},{"class":145,"line":186},[720,724,728],{"type":41,"tag":143,"props":721,"children":722},{"style":165},[723],{"type":47,"value":260},{"type":41,"tag":143,"props":725,"children":726},{"style":165},[727],{"type":47,"value":265},{"type":41,"tag":143,"props":729,"children":730},{"style":212},[731],{"type":47,"value":215},{"type":41,"tag":143,"props":733,"children":734},{"class":145,"line":195},[735,739,743],{"type":41,"tag":143,"props":736,"children":737},{"style":165},[738],{"type":47,"value":278},{"type":41,"tag":143,"props":740,"children":741},{"style":165},[742],{"type":47,"value":283},{"type":41,"tag":143,"props":744,"children":745},{"style":212},[746],{"type":47,"value":215},{"type":41,"tag":143,"props":748,"children":749},{"class":145,"line":218},[750,754],{"type":41,"tag":143,"props":751,"children":752},{"style":165},[753],{"type":47,"value":296},{"type":41,"tag":143,"props":755,"children":756},{"style":299},[757],{"type":47,"value":758}," 100\n",{"type":41,"tag":655,"props":760,"children":762},{"id":761},"save-results-to-file",[763],{"type":47,"value":595},{"type":41,"tag":132,"props":765,"children":767},{"className":134,"code":766,"language":136,"meta":137,"style":137},"vllm bench serve \\\n  --backend openai-chat \\\n  --model Qwen\u002FQwen2.5-1.5B-Instruct \\\n  --endpoint \u002Fv1\u002Fchat\u002Fcompletions \\\n  --dataset-name random \\\n  --num-prompts 50 \\\n  --save-result \\\n  --result-dir .\u002Fbenchmark-results\u002F\n",[768],{"type":41,"tag":102,"props":769,"children":770},{"__ignoreMap":137},[771,790,805,820,835,850,866,878],{"type":41,"tag":143,"props":772,"children":773},{"class":145,"line":146},[774,778,782,786],{"type":41,"tag":143,"props":775,"children":776},{"style":160},[777],{"type":47,"value":8},{"type":41,"tag":143,"props":779,"children":780},{"style":165},[781],{"type":47,"value":205},{"type":41,"tag":143,"props":783,"children":784},{"style":165},[785],{"type":47,"value":168},{"type":41,"tag":143,"props":787,"children":788},{"style":212},[789],{"type":47,"value":215},{"type":41,"tag":143,"props":791,"children":792},{"class":145,"line":156},[793,797,801],{"type":41,"tag":143,"props":794,"children":795},{"style":165},[796],{"type":47,"value":224},{"type":41,"tag":143,"props":798,"children":799},{"style":165},[800],{"type":47,"value":229},{"type":41,"tag":143,"props":802,"children":803},{"style":212},[804],{"type":47,"value":215},{"type":41,"tag":143,"props":806,"children":807},{"class":145,"line":176},[808,812,816],{"type":41,"tag":143,"props":809,"children":810},{"style":165},[811],{"type":47,"value":242},{"type":41,"tag":143,"props":813,"children":814},{"style":165},[815],{"type":47,"value":247},{"type":41,"tag":143,"props":817,"children":818},{"style":212},[819],{"type":47,"value":215},{"type":41,"tag":143,"props":821,"children":822},{"class":145,"line":186},[823,827,831],{"type":41,"tag":143,"props":824,"children":825},{"style":165},[826],{"type":47,"value":260},{"type":41,"tag":143,"props":828,"children":829},{"style":165},[830],{"type":47,"value":265},{"type":41,"tag":143,"props":832,"children":833},{"style":212},[834],{"type":47,"value":215},{"type":41,"tag":143,"props":836,"children":837},{"class":145,"line":195},[838,842,846],{"type":41,"tag":143,"props":839,"children":840},{"style":165},[841],{"type":47,"value":278},{"type":41,"tag":143,"props":843,"children":844},{"style":165},[845],{"type":47,"value":283},{"type":41,"tag":143,"props":847,"children":848},{"style":212},[849],{"type":47,"value":215},{"type":41,"tag":143,"props":851,"children":852},{"class":145,"line":218},[853,857,862],{"type":41,"tag":143,"props":854,"children":855},{"style":165},[856],{"type":47,"value":296},{"type":41,"tag":143,"props":858,"children":859},{"style":299},[860],{"type":47,"value":861}," 50",{"type":41,"tag":143,"props":863,"children":864},{"style":212},[865],{"type":47,"value":215},{"type":41,"tag":143,"props":867,"children":868},{"class":145,"line":236},[869,874],{"type":41,"tag":143,"props":870,"children":871},{"style":165},[872],{"type":47,"value":873},"  --save-result",{"type":41,"tag":143,"props":875,"children":876},{"style":212},[877],{"type":47,"value":215},{"type":41,"tag":143,"props":879,"children":880},{"class":145,"line":254},[881,886],{"type":41,"tag":143,"props":882,"children":883},{"style":165},[884],{"type":47,"value":885},"  --result-dir",{"type":41,"tag":143,"props":887,"children":888},{"style":165},[889],{"type":47,"value":890}," .\u002Fbenchmark-results\u002F\n",{"type":41,"tag":655,"props":892,"children":894},{"id":893},"custom-port-and-concurrency",[895],{"type":47,"value":896},"Custom port and concurrency",{"type":41,"tag":132,"props":898,"children":900},{"className":134,"code":899,"language":136,"meta":137,"style":137},"vllm bench serve \\\n  --backend openai-chat \\\n  --model meta-llama\u002FLlama-3.1-8B-Instruct \\\n  --endpoint \u002Fv1\u002Fchat\u002Fcompletions \\\n  --dataset-name random \\\n  --num-prompts 100 \\\n  --port 8001 \\\n  --max-concurrency 4\n",[901],{"type":41,"tag":102,"props":902,"children":903},{"__ignoreMap":137},[904,923,938,954,969,984,1000,1017],{"type":41,"tag":143,"props":905,"children":906},{"class":145,"line":146},[907,911,915,919],{"type":41,"tag":143,"props":908,"children":909},{"style":160},[910],{"type":47,"value":8},{"type":41,"tag":143,"props":912,"children":913},{"style":165},[914],{"type":47,"value":205},{"type":41,"tag":143,"props":916,"children":917},{"style":165},[918],{"type":47,"value":168},{"type":41,"tag":143,"props":920,"children":921},{"style":212},[922],{"type":47,"value":215},{"type":41,"tag":143,"props":924,"children":925},{"class":145,"line":156},[926,930,934],{"type":41,"tag":143,"props":927,"children":928},{"style":165},[929],{"type":47,"value":224},{"type":41,"tag":143,"props":931,"children":932},{"style":165},[933],{"type":47,"value":229},{"type":41,"tag":143,"props":935,"children":936},{"style":212},[937],{"type":47,"value":215},{"type":41,"tag":143,"props":939,"children":940},{"class":145,"line":176},[941,945,950],{"type":41,"tag":143,"props":942,"children":943},{"style":165},[944],{"type":47,"value":242},{"type":41,"tag":143,"props":946,"children":947},{"style":165},[948],{"type":47,"value":949}," meta-llama\u002FLlama-3.1-8B-Instruct",{"type":41,"tag":143,"props":951,"children":952},{"style":212},[953],{"type":47,"value":215},{"type":41,"tag":143,"props":955,"children":956},{"class":145,"line":186},[957,961,965],{"type":41,"tag":143,"props":958,"children":959},{"style":165},[960],{"type":47,"value":260},{"type":41,"tag":143,"props":962,"children":963},{"style":165},[964],{"type":47,"value":265},{"type":41,"tag":143,"props":966,"children":967},{"style":212},[968],{"type":47,"value":215},{"type":41,"tag":143,"props":970,"children":971},{"class":145,"line":195},[972,976,980],{"type":41,"tag":143,"props":973,"children":974},{"style":165},[975],{"type":47,"value":278},{"type":41,"tag":143,"props":977,"children":978},{"style":165},[979],{"type":47,"value":283},{"type":41,"tag":143,"props":981,"children":982},{"style":212},[983],{"type":47,"value":215},{"type":41,"tag":143,"props":985,"children":986},{"class":145,"line":218},[987,991,996],{"type":41,"tag":143,"props":988,"children":989},{"style":165},[990],{"type":47,"value":296},{"type":41,"tag":143,"props":992,"children":993},{"style":299},[994],{"type":47,"value":995}," 100",{"type":41,"tag":143,"props":997,"children":998},{"style":212},[999],{"type":47,"value":215},{"type":41,"tag":143,"props":1001,"children":1002},{"class":145,"line":236},[1003,1008,1013],{"type":41,"tag":143,"props":1004,"children":1005},{"style":165},[1006],{"type":47,"value":1007},"  --port",{"type":41,"tag":143,"props":1009,"children":1010},{"style":299},[1011],{"type":47,"value":1012}," 8001",{"type":41,"tag":143,"props":1014,"children":1015},{"style":212},[1016],{"type":47,"value":215},{"type":41,"tag":143,"props":1018,"children":1019},{"class":145,"line":254},[1020,1025],{"type":41,"tag":143,"props":1021,"children":1022},{"style":165},[1023],{"type":47,"value":1024},"  --max-concurrency",{"type":41,"tag":143,"props":1026,"children":1027},{"style":299},[1028],{"type":47,"value":1029}," 4\n",{"type":41,"tag":56,"props":1031,"children":1033},{"id":1032},"model-recommendations",[1034],{"type":47,"value":1035},"Model Recommendations",{"type":41,"tag":50,"props":1037,"children":1038},{},[1039],{"type":47,"value":1040},"For quick testing (small models, fast):",{"type":41,"tag":63,"props":1042,"children":1043},{},[1044,1055,1064],{"type":41,"tag":67,"props":1045,"children":1046},{},[1047,1053],{"type":41,"tag":102,"props":1048,"children":1050},{"className":1049},[],[1051],{"type":47,"value":1052},"Qwen\u002FQwen2.5-1.5B-Instruct",{"type":47,"value":1054}," (recommended for quick tests)",{"type":41,"tag":67,"props":1056,"children":1057},{},[1058],{"type":41,"tag":102,"props":1059,"children":1061},{"className":1060},[],[1062],{"type":47,"value":1063},"facebook\u002Fopt-125m",{"type":41,"tag":67,"props":1065,"children":1066},{},[1067],{"type":41,"tag":102,"props":1068,"children":1070},{"className":1069},[],[1071],{"type":47,"value":1072},"facebook\u002Fopt-350m",{"type":41,"tag":50,"props":1074,"children":1075},{},[1076],{"type":47,"value":1077},"For realistic benchmarks (medium models):",{"type":41,"tag":63,"props":1079,"children":1080},{},[1081,1090,1099],{"type":41,"tag":67,"props":1082,"children":1083},{},[1084],{"type":41,"tag":102,"props":1085,"children":1087},{"className":1086},[],[1088],{"type":47,"value":1089},"Qwen\u002FQwen2.5-7B-Instruct",{"type":41,"tag":67,"props":1091,"children":1092},{},[1093],{"type":41,"tag":102,"props":1094,"children":1096},{"className":1095},[],[1097],{"type":47,"value":1098},"meta-llama\u002FLlama-3.1-8B-Instruct",{"type":41,"tag":67,"props":1100,"children":1101},{},[1102],{"type":41,"tag":102,"props":1103,"children":1105},{"className":1104},[],[1106],{"type":47,"value":1107},"mistralai\u002FMistral-7B-Instruct-v0.3",{"type":41,"tag":56,"props":1109,"children":1111},{"id":1110},"workflow",[1112],{"type":47,"value":1113},"Workflow",{"type":41,"tag":1115,"props":1116,"children":1117},"ol",{},[1118,1136,1153,1170,1188,1198],{"type":41,"tag":67,"props":1119,"children":1120},{},[1121,1126,1128,1134],{"type":41,"tag":307,"props":1122,"children":1123},{},[1124],{"type":47,"value":1125},"Check if vLLM is installed",{"type":47,"value":1127},": Run ",{"type":41,"tag":102,"props":1129,"children":1131},{"className":1130},[],[1132],{"type":47,"value":1133},"vllm --version",{"type":47,"value":1135}," to verify",{"type":41,"tag":67,"props":1137,"children":1138},{},[1139,1144,1145,1151],{"type":41,"tag":307,"props":1140,"children":1141},{},[1142],{"type":47,"value":1143},"Check if server is already running",{"type":47,"value":1127},{"type":41,"tag":102,"props":1146,"children":1148},{"className":1147},[],[1149],{"type":47,"value":1150},"curl http:\u002F\u002Flocalhost:8000\u002Fhealth",{"type":47,"value":1152}," to check",{"type":41,"tag":67,"props":1154,"children":1155},{},[1156,1161,1162,1168],{"type":41,"tag":307,"props":1157,"children":1158},{},[1159],{"type":47,"value":1160},"Start vLLM server if needed",{"type":47,"value":1127},{"type":41,"tag":102,"props":1163,"children":1165},{"className":1164},[],[1166],{"type":47,"value":1167},"vllm serve \u003Cmodel-name>",{"type":47,"value":1169}," (wait for \"Application startup complete\")",{"type":41,"tag":67,"props":1171,"children":1172},{},[1173,1178,1180,1186],{"type":41,"tag":307,"props":1174,"children":1175},{},[1176],{"type":47,"value":1177},"Run benchmark",{"type":47,"value":1179},": Execute ",{"type":41,"tag":102,"props":1181,"children":1183},{"className":1182},[],[1184],{"type":47,"value":1185},"vllm bench serve",{"type":47,"value":1187}," with appropriate parameters",{"type":41,"tag":67,"props":1189,"children":1190},{},[1191,1196],{"type":41,"tag":307,"props":1192,"children":1193},{},[1194],{"type":47,"value":1195},"Review results",{"type":47,"value":1197},": Check throughput and latency metrics",{"type":41,"tag":67,"props":1199,"children":1200},{},[1201,1206,1208],{"type":41,"tag":307,"props":1202,"children":1203},{},[1204],{"type":47,"value":1205},"Clean up",{"type":47,"value":1207},": If the agent skill started the vLLM server (not a pre-existing one), stop it after benchmark completion using ",{"type":41,"tag":102,"props":1209,"children":1211},{"className":1210},[],[1212],{"type":47,"value":1213},"kill \u003CPID>",{"type":41,"tag":56,"props":1215,"children":1217},{"id":1216},"troubleshooting",[1218],{"type":47,"value":1219},"Troubleshooting",{"type":41,"tag":50,"props":1221,"children":1222},{},[1223,1228],{"type":41,"tag":307,"props":1224,"children":1225},{},[1226],{"type":47,"value":1227},"Server not responding",{"type":47,"value":313},{"type":41,"tag":63,"props":1230,"children":1231},{},[1232,1242],{"type":41,"tag":67,"props":1233,"children":1234},{},[1235,1237],{"type":47,"value":1236},"Check if server is running: ",{"type":41,"tag":102,"props":1238,"children":1240},{"className":1239},[],[1241],{"type":47,"value":1150},{"type":41,"tag":67,"props":1243,"children":1244},{},[1245,1247,1252],{"type":47,"value":1246},"Verify port matches: Use ",{"type":41,"tag":102,"props":1248,"children":1250},{"className":1249},[],[1251],{"type":47,"value":542},{"type":47,"value":1253}," flag if server is on different port",{"type":41,"tag":50,"props":1255,"children":1256},{},[1257,1262],{"type":41,"tag":307,"props":1258,"children":1259},{},[1260],{"type":47,"value":1261},"Model not found",{"type":47,"value":313},{"type":41,"tag":63,"props":1264,"children":1265},{},[1266,1271],{"type":41,"tag":67,"props":1267,"children":1268},{},[1269],{"type":47,"value":1270},"Ensure model name matches exactly between server and benchmark",{"type":41,"tag":67,"props":1272,"children":1273},{},[1274,1276,1282],{"type":47,"value":1275},"Check HuggingFace access: ",{"type":41,"tag":102,"props":1277,"children":1279},{"className":1278},[],[1280],{"type":47,"value":1281},"export HF_TOKEN=\u003Cyour_token>",{"type":47,"value":1283}," if needed",{"type":41,"tag":50,"props":1285,"children":1286},{},[1287,1292],{"type":41,"tag":307,"props":1288,"children":1289},{},[1290],{"type":47,"value":1291},"Out of memory",{"type":47,"value":313},{"type":41,"tag":63,"props":1294,"children":1295},{},[1296,1301],{"type":41,"tag":67,"props":1297,"children":1298},{},[1299],{"type":47,"value":1300},"Use a smaller model (e.g., Qwen2.5-1.5B-Instruct)",{"type":41,"tag":67,"props":1302,"children":1303},{},[1304,1306,1311,1312],{"type":47,"value":1305},"Reduce ",{"type":41,"tag":102,"props":1307,"children":1309},{"className":1308},[],[1310],{"type":47,"value":516},{"type":47,"value":471},{"type":41,"tag":102,"props":1313,"children":1315},{"className":1314},[],[1316],{"type":47,"value":568},{"type":41,"tag":50,"props":1318,"children":1319},{},[1320,1325],{"type":41,"tag":307,"props":1321,"children":1322},{},[1323],{"type":47,"value":1324},"Connection refused",{"type":47,"value":313},{"type":41,"tag":63,"props":1327,"children":1328},{},[1329,1334],{"type":41,"tag":67,"props":1330,"children":1331},{},[1332],{"type":47,"value":1333},"Server may still be starting (wait for \"Application startup complete\")",{"type":41,"tag":67,"props":1335,"children":1336},{},[1337],{"type":47,"value":1338},"Check firewall or network settings",{"type":41,"tag":56,"props":1340,"children":1342},{"id":1341},"notes",[1343],{"type":47,"value":1344},"Notes",{"type":41,"tag":63,"props":1346,"children":1347},{},[1348,1360,1370,1375,1380,1385],{"type":41,"tag":67,"props":1349,"children":1350},{},[1351,1353,1358],{"type":47,"value":1352},"The ",{"type":41,"tag":102,"props":1354,"children":1356},{"className":1355},[],[1357],{"type":47,"value":502},{"type":47,"value":1359}," dataset generates synthetic prompts automatically",{"type":41,"tag":67,"props":1361,"children":1362},{},[1363,1365],{"type":47,"value":1364},"Benchmark duration scales with ",{"type":41,"tag":102,"props":1366,"children":1368},{"className":1367},[],[1369],{"type":47,"value":516},{"type":41,"tag":67,"props":1371,"children":1372},{},[1373],{"type":47,"value":1374},"For production benchmarking, use at least 100 prompts for stable statistics",{"type":41,"tag":67,"props":1376,"children":1377},{},[1378],{"type":47,"value":1379},"Results may vary based on hardware, model size, and system load",{"type":41,"tag":67,"props":1381,"children":1382},{},[1383],{"type":47,"value":1384},"First run may be slower due to model loading and compilation",{"type":41,"tag":67,"props":1386,"children":1387},{},[1388,1393],{"type":41,"tag":307,"props":1389,"children":1390},{},[1391],{"type":47,"value":1392},"Important",{"type":47,"value":1394},": If the agent skill starts a vLLM server for benchmarking, it must stop the server after the benchmark completes to free up resources. Do not stop pre-existing servers that were already running before the benchmark.",{"type":41,"tag":1396,"props":1397,"children":1398},"style",{},[1399],{"type":47,"value":1400},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":1402,"total":236},[1403,1420,1427,1441,1456,1472,1483],{"slug":1404,"name":1404,"fn":1405,"description":1406,"org":1407,"tags":1408,"stars":1417,"repoUrl":1418,"updatedAt":1419},"openclaw-vsr-bridge","integrate vLLM Semantic Router","Install vLLM Semantic Router in agent-safe mode, import supported OpenClaw model providers into canonical VSR config, and rewrite OpenClaw to target VSR.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1409,1412,1413,1416],{"name":1410,"slug":1411,"type":16},"Agents","agents",{"name":21,"slug":22,"type":16},{"name":1414,"slug":1415,"type":16},"Routing","routing",{"name":9,"slug":8,"type":16},4962,"https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fsemantic-router","2026-07-18T05:48:09.091161",{"slug":4,"name":4,"fn":5,"description":6,"org":1421,"tags":1422,"stars":24,"repoUrl":25,"updatedAt":26},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1423,1424,1425,1426],{"name":18,"slug":19,"type":16},{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16},{"slug":1428,"name":1428,"fn":1429,"description":1430,"org":1431,"tags":1432,"stars":24,"repoUrl":25,"updatedAt":1440},"vllm-bench-serve","benchmark LLM serving endpoints","Benchmark vLLM or OpenAI-compatible serving endpoints using vllm bench serve. Supports multiple datasets (random, sharegpt, sonnet, HF), backends (openai, openai-chat, vllm-pooling, embeddings), throughput\u002Flatency testing with request-rate control, and result saving. Use when benchmarking LLM serving performance, measuring TTFT\u002FTPOT, or load testing inference APIs.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1433,1436,1437,1438,1439],{"name":1434,"slug":1435,"type":16},"API Development","api-development",{"name":18,"slug":19,"type":16},{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16},"2026-07-18T05:47:02.283164",{"slug":1442,"name":1442,"fn":1443,"description":1444,"org":1445,"tags":1446,"stars":24,"repoUrl":25,"updatedAt":1455},"vllm-deploy-docker","deploy vLLM using Docker containers","Deploy vLLM using Docker (pre-built images or build-from-source) with NVIDIA GPU support and run the OpenAI-compatible server.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1447,1450,1453,1454],{"name":1448,"slug":1449,"type":16},"Deployment","deployment",{"name":1451,"slug":1452,"type":16},"Docker","docker",{"name":21,"slug":22,"type":16},{"name":9,"slug":8,"type":16},"2026-07-18T05:47:01.814419",{"slug":1457,"name":1457,"fn":1458,"description":1459,"org":1460,"tags":1461,"stars":24,"repoUrl":25,"updatedAt":1471},"vllm-deploy-k8s","deploy vLLM to Kubernetes clusters","Deploy vLLM to Kubernetes (K8s) with GPU support, health probes, and OpenAI-compatible API endpoint. Use this skill whenever the user wants to deploy, run, or serve vLLM on a Kubernetes cluster, including creating deployments, services, checking existing deployments, or managing vLLM on K8s.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1462,1465,1466,1469,1470],{"name":1463,"slug":1464,"type":16},"AI Infrastructure","ai-infrastructure",{"name":1448,"slug":1449,"type":16},{"name":1467,"slug":1468,"type":16},"Kubernetes","kubernetes",{"name":21,"slug":22,"type":16},{"name":9,"slug":8,"type":16},"2026-07-18T05:46:58.556637",{"slug":1473,"name":1473,"fn":1474,"description":1475,"org":1476,"tags":1477,"stars":24,"repoUrl":25,"updatedAt":1482},"vllm-deploy-simple","deploy and test vLLM servers","Quick install and deploy vLLM, start serving with a simple LLM, and test OpenAI API.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1478,1479,1480,1481],{"name":1434,"slug":1435,"type":16},{"name":1448,"slug":1449,"type":16},{"name":21,"slug":22,"type":16},{"name":9,"slug":8,"type":16},"2026-07-18T05:47:01.359655",{"slug":1484,"name":1484,"fn":1485,"description":1486,"org":1487,"tags":1488,"stars":24,"repoUrl":25,"updatedAt":1493},"vllm-prefix-cache-bench","benchmark vLLM prefix caching efficiency","This is a skill for benchmarking the efficiency of automatic prefix caching in vLLM using fixed prompts, real-world datasets, or synthetic prefix\u002Fsuffix patterns. Use when the user asks to benchmark prefix caching hit rate, caching efficiency, or repeated-prompt performance in vLLM.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1489,1490,1491,1492],{"name":18,"slug":19,"type":16},{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16},"2026-07-18T05:48:07.243092",{"items":1495,"total":218},[1496,1503,1511,1518,1526,1533],{"slug":4,"name":4,"fn":5,"description":6,"org":1497,"tags":1498,"stars":24,"repoUrl":25,"updatedAt":26},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1499,1500,1501,1502],{"name":18,"slug":19,"type":16},{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16},{"slug":1428,"name":1428,"fn":1429,"description":1430,"org":1504,"tags":1505,"stars":24,"repoUrl":25,"updatedAt":1440},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1506,1507,1508,1509,1510],{"name":1434,"slug":1435,"type":16},{"name":18,"slug":19,"type":16},{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16},{"slug":1442,"name":1442,"fn":1443,"description":1444,"org":1512,"tags":1513,"stars":24,"repoUrl":25,"updatedAt":1455},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1514,1515,1516,1517],{"name":1448,"slug":1449,"type":16},{"name":1451,"slug":1452,"type":16},{"name":21,"slug":22,"type":16},{"name":9,"slug":8,"type":16},{"slug":1457,"name":1457,"fn":1458,"description":1459,"org":1519,"tags":1520,"stars":24,"repoUrl":25,"updatedAt":1471},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1521,1522,1523,1524,1525],{"name":1463,"slug":1464,"type":16},{"name":1448,"slug":1449,"type":16},{"name":1467,"slug":1468,"type":16},{"name":21,"slug":22,"type":16},{"name":9,"slug":8,"type":16},{"slug":1473,"name":1473,"fn":1474,"description":1475,"org":1527,"tags":1528,"stars":24,"repoUrl":25,"updatedAt":1482},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1529,1530,1531,1532],{"name":1434,"slug":1435,"type":16},{"name":1448,"slug":1449,"type":16},{"name":21,"slug":22,"type":16},{"name":9,"slug":8,"type":16},{"slug":1484,"name":1484,"fn":1485,"description":1486,"org":1534,"tags":1535,"stars":24,"repoUrl":25,"updatedAt":1493},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1536,1537,1538,1539],{"name":18,"slug":19,"type":16},{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16}]