[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-vllm-vllm-deploy-simple":3,"mdc--ze9h1q-key":35,"related-org-vllm-vllm-deploy-simple":1827,"related-repo-vllm-vllm-deploy-simple":1920},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":12,"stars":24,"repoUrl":25,"updatedAt":26,"license":27,"forks":28,"topics":29,"repo":30,"sourceUrl":33,"mdContent":34},"vllm-deploy-simple","deploy and test vLLM servers","Quick install and deploy vLLM, start serving with a simple LLM, and test OpenAI API.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},"vllm","vLLM","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fvllm.png","vllm-project",[13,17,18,21],{"name":14,"slug":15,"type":16},"LLM","llm","tag",{"name":9,"slug":8,"type":16},{"name":19,"slug":20,"type":16},"Deployment","deployment",{"name":22,"slug":23,"type":16},"API Development","api-development",88,"https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm-skills","2026-07-18T05:47:01.359655",null,23,[],{"repoUrl":25,"stars":24,"forks":28,"topics":31,"description":32},[],"Agent skills for vLLM","https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm-skills\u002Ftree\u002FHEAD\u002Fplugins\u002Fvllm-skills\u002Fskills\u002Fvllm-deploy-simple","---\nname: vllm-deploy-simple\ndescription: Quick install and deploy vLLM, start serving with a simple LLM, and test OpenAI API.\n---\n\n# vLLM Simple Deployment\n\nA simple skill to quickly install vLLM, start a server, and validate the OpenAI-compatible API.\n\n## What this skill does\n\nThis skill provides a streamlined workflow to:\n- Detect hardware backend (NVIDIA CUDA, AMD ROCm, Google TPU, or CPU)\n- Install vLLM with appropriate backend support\n- Start the vLLM server with configurable model and port\n- Test the OpenAI-compatible API endpoint\n- Validate the deployment is working correctly\n- Support virtual environment isolation\n\n## Prerequisites\n\n- Python 3.10+\n- GPU (NVIDIA CUDA, AMD ROCm) (recommended) or TPU or CPU\n- pip or uv package manager\n- curl (for API testing)\n- Virtual environment (optional but recommended)\n\n## Usage\n\n### Create a venv\n\nIf user did not specify the venv path or asked to deploy in the current environment, create a venv using uv with python 3.12 in the current folder. If uv not found, make a folder in this path and use python to create a virtual environment.\n\n### Run the complete workflow (suggested)\n\nIf user did not specify the venv path, model, or port, use default options:\n\n```bash\n# Default deployment options (--venv \".\" --model \"Qwen\u002FQwen2.5-1.5B-Instruct\" --port 8000 --gpu_memory_utilization 0.8)\nscripts\u002Fquickstart.sh\n```\n\nOr with custom options:\n\n```bash\n# Use custom virtual environment\nscripts\u002Fquickstart.sh --venv \u002Fpath\u002Fto\u002Fvenv\n\n# Use custom model and port\nscripts\u002Fquickstart.sh --model \"Qwen\u002FQwen2.5-1.5B-Instruct\" --port 8000\n\n# Use custom GPU memory utilization\nscripts\u002Fquickstart.sh --gpu_memory_utilization 0.6\n\n# Combine all options\nscripts\u002Fquickstart.sh --venv \u002Fpath\u002Fto\u002Fvenv --model \"Qwen\u002FQwen2.5-1.5B-Instruct\" --port 8000 --gpu_memory_utilization 0.8\n```\n\nThis will:\n1. Activate the virtual environment (if specified)\n2. Detect hardware backend (CUDA\u002FROCm\u002FTPU\u002FCPU)\n3. Install vLLM with appropriate backend support\n4. Start the vLLM server in the background\n5. Wait for the server to be ready\n6. Test the API with a sample request\n7. Display the server status\n\n### Run individual commands (for step-by-step usage or troubleshooting)\n\n**Install vLLM:**\n```bash\nscripts\u002Fquickstart.sh install\n# Or with virtual environment\nscripts\u002Fquickstart.sh install --venv \u002Fpath\u002Fto\u002Fvenv\n```\n\n**Start the server:**\n```bash\nscripts\u002Fquickstart.sh start\n# Or with custom options\nscripts\u002Fquickstart.sh start --venv \u002Fpath\u002Fto\u002Fvenv --model \"Qwen\u002FQwen2.5-1.5B-Instruct\" --port 8000 --gpu_memory_utilization 0.8\n```\n\n**Test the API:**\n```bash\nscripts\u002Fquickstart.sh test\n# Or with custom port\nscripts\u002Fquickstart.sh test --port 8000\n```\n\n**Stop the server:**\n```bash\nscripts\u002Fquickstart.sh stop\n# Or with virtual environment\nscripts\u002Fquickstart.sh stop --venv \u002Fpath\u002Fto\u002Fvenv\n```\n\n**Check server status:**\n```bash\nscripts\u002Fquickstart.sh status\n```\n\n**Restart the server:**\n```bash\nscripts\u002Fquickstart.sh restart\n# Or with custom options\nscripts\u002Fquickstart.sh restart --venv \u002Fpath\u002Fto\u002Fvenv --port 8000 --gpu_memory_utilization 0.8\n```\n\n## Configuration\n\nThe script supports the following command-line options:\n\n```bash\nscripts\u002Fquickstart.sh [command] [OPTIONS]\n\nCommands:\n  install  - Install vLLM and dependencies\n  start    - Start the vLLM server\n  stop     - Stop the vLLM server\n  test     - Test the OpenAI-compatible API\n  status   - Show server status\n  restart  - Restart the server\n  all      - Run complete workflow (default)\n\nOptions:\n  --model MODEL                 Model to use (default: Qwen\u002FQwen2.5-1.5B-Instruct)\n  --port PORT                   Port to run server on (default: 8000)\n  --venv VENV_PATH              Virtual environment path (default: .)\n  --gpu_memory_utilization VRAM GPU memory utilization (default: 0.8)\n```\n\n### Hardware Backend Detection\n\nThe script automatically detects your hardware and installs the appropriate vLLM version:\n\n- **NVIDIA CUDA**: Detected via `nvidia-smi` command\n- **AMD ROCm**: Detected via `\u002Fdev\u002Fkfd` and `\u002Fdev\u002Fdri` devices\n- **Google TPU**: Detected via `TPU_NAME` environment variable or `gcloud` command\n- **CPU**: Fallback if no GPU\u002FTPU detected\n\nFor Google TPU, the script installs `vllm-tpu` instead of the standard `vllm` package.\n\n## API Testing\n\nThe test script sends a simple chat completion request:\n\n```bash\ncurl http:\u002F\u002Flocalhost:8000\u002Fv1\u002Fchat\u002Fcompletions \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\n    \"model\": \"Qwen\u002FQwen2.5-1.5B-Instruct\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Say hello!\"}],\n    \"max_tokens\": 50\n  }'\n```\n\n## Troubleshooting\n\n**Virtual environment not found:**\n- Ensure the path provided with `--venv` exists and is a valid virtual environment\n- Check that the activation script exists (`bin\u002Factivate` on Linux\u002FmacOS or `Scripts\u002Factivate` on Windows)\n- Check and install uv, and create a new virtual environment with uv: `uv venv \u002Fpath\u002Fto\u002Fvenv` (suggested); or with pip: `python3 -m venv \u002Fpath\u002Fto\u002Fvenv` \n\n**Server won't start:**\n- Check if the port is already in use: `lsof -i :8000`\n- Verify GPU availability: `nvidia-smi` (for NVIDIA) or `rocm-smi` (for AMD)\n- Check vLLM installation: `python -c \"import vllm; print(vllm.__version__)\"`\n- Review server logs at `$VENV_PATH\u002Ftmp\u002Fvllm-server.log`\n\n**API returns errors:**\n- Wait a few seconds for the model to load\n- Check server logs: `cat $VENV_PATH\u002Ftmp\u002Fvllm-server.log`\n- Verify the server is running: `scripts\u002Fquickstart.sh status`\n\n**Out of memory:**\n- Use a smaller model (e.g., Qwen2.5-0.5B-Instruct)\n- Reduce `--gpu-memory-utilization` parameter\n- Close other GPU-intensive applications\n\n**Wrong backend detected:**\n- For NVIDIA: Ensure `nvidia-smi` is in your PATH\n- For AMD: Check that ROCm drivers are properly installed\n- For TPU: Set `TPU_NAME` environment variable or install `gcloud`\n\n## Notes\n\n- The server runs in the background and logs to `$VENV_PATH\u002Ftmp\u002Fvllm-server.log`\n- The PID is stored in `$VENV_PATH\u002Ftmp\u002Fvllm-server.pid` for easy management\n- First run will download the model (~3GB for Qwen2.5-1.5B-Instruct)\n- Subsequent runs will use the cached model\n- The script automatically detects and uses `uv` if available, otherwise falls back to `pip`\n- Virtual environment support allows isolation from system Python packages\n- Arguments can be specified in any order (e.g., `scripts\u002Fquickstart.sh --port 8080 start --venv \u002Fpath\u002Fto\u002Fvenv`)\n",{"data":36,"body":37},{"name":4,"description":6},{"type":38,"children":39},"root",[40,49,55,62,67,102,108,136,142,149,154,160,165,199,204,400,405,443,449,458,505,513,592,600,647,655,701,709,728,736,798,804,809,1264,1270,1275,1355,1375,1381,1386,1491,1497,1505,1561,1569,1625,1633,1663,1671,1697,1705,1742,1748,1821],{"type":41,"tag":42,"props":43,"children":45},"element","h1",{"id":44},"vllm-simple-deployment",[46],{"type":47,"value":48},"text","vLLM Simple Deployment",{"type":41,"tag":50,"props":51,"children":52},"p",{},[53],{"type":47,"value":54},"A simple skill to quickly install vLLM, start a server, and validate the OpenAI-compatible API.",{"type":41,"tag":56,"props":57,"children":59},"h2",{"id":58},"what-this-skill-does",[60],{"type":47,"value":61},"What this skill does",{"type":41,"tag":50,"props":63,"children":64},{},[65],{"type":47,"value":66},"This skill provides a streamlined workflow to:",{"type":41,"tag":68,"props":69,"children":70},"ul",{},[71,77,82,87,92,97],{"type":41,"tag":72,"props":73,"children":74},"li",{},[75],{"type":47,"value":76},"Detect hardware backend (NVIDIA CUDA, AMD ROCm, Google TPU, or CPU)",{"type":41,"tag":72,"props":78,"children":79},{},[80],{"type":47,"value":81},"Install vLLM with appropriate backend support",{"type":41,"tag":72,"props":83,"children":84},{},[85],{"type":47,"value":86},"Start the vLLM server with configurable model and port",{"type":41,"tag":72,"props":88,"children":89},{},[90],{"type":47,"value":91},"Test the OpenAI-compatible API endpoint",{"type":41,"tag":72,"props":93,"children":94},{},[95],{"type":47,"value":96},"Validate the deployment is working correctly",{"type":41,"tag":72,"props":98,"children":99},{},[100],{"type":47,"value":101},"Support virtual environment isolation",{"type":41,"tag":56,"props":103,"children":105},{"id":104},"prerequisites",[106],{"type":47,"value":107},"Prerequisites",{"type":41,"tag":68,"props":109,"children":110},{},[111,116,121,126,131],{"type":41,"tag":72,"props":112,"children":113},{},[114],{"type":47,"value":115},"Python 3.10+",{"type":41,"tag":72,"props":117,"children":118},{},[119],{"type":47,"value":120},"GPU (NVIDIA CUDA, AMD ROCm) (recommended) or TPU or CPU",{"type":41,"tag":72,"props":122,"children":123},{},[124],{"type":47,"value":125},"pip or uv package manager",{"type":41,"tag":72,"props":127,"children":128},{},[129],{"type":47,"value":130},"curl (for API testing)",{"type":41,"tag":72,"props":132,"children":133},{},[134],{"type":47,"value":135},"Virtual environment (optional but recommended)",{"type":41,"tag":56,"props":137,"children":139},{"id":138},"usage",[140],{"type":47,"value":141},"Usage",{"type":41,"tag":143,"props":144,"children":146},"h3",{"id":145},"create-a-venv",[147],{"type":47,"value":148},"Create a venv",{"type":41,"tag":50,"props":150,"children":151},{},[152],{"type":47,"value":153},"If user did not specify the venv path or asked to deploy in the current environment, create a venv using uv with python 3.12 in the current folder. If uv not found, make a folder in this path and use python to create a virtual environment.",{"type":41,"tag":143,"props":155,"children":157},{"id":156},"run-the-complete-workflow-suggested",[158],{"type":47,"value":159},"Run the complete workflow (suggested)",{"type":41,"tag":50,"props":161,"children":162},{},[163],{"type":47,"value":164},"If user did not specify the venv path, model, or port, use default options:",{"type":41,"tag":166,"props":167,"children":172},"pre",{"className":168,"code":169,"language":170,"meta":171,"style":171},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","# Default deployment options (--venv \".\" --model \"Qwen\u002FQwen2.5-1.5B-Instruct\" --port 8000 --gpu_memory_utilization 0.8)\nscripts\u002Fquickstart.sh\n","bash","",[173],{"type":41,"tag":174,"props":175,"children":176},"code",{"__ignoreMap":171},[177,189],{"type":41,"tag":178,"props":179,"children":182},"span",{"class":180,"line":181},"line",1,[183],{"type":41,"tag":178,"props":184,"children":186},{"style":185},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[187],{"type":47,"value":188},"# Default deployment options (--venv \".\" --model \"Qwen\u002FQwen2.5-1.5B-Instruct\" --port 8000 --gpu_memory_utilization 0.8)\n",{"type":41,"tag":178,"props":190,"children":192},{"class":180,"line":191},2,[193],{"type":41,"tag":178,"props":194,"children":196},{"style":195},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[197],{"type":47,"value":198},"scripts\u002Fquickstart.sh\n",{"type":41,"tag":50,"props":200,"children":201},{},[202],{"type":47,"value":203},"Or with custom options:",{"type":41,"tag":166,"props":205,"children":207},{"className":168,"code":206,"language":170,"meta":171,"style":171},"# Use custom virtual environment\nscripts\u002Fquickstart.sh --venv \u002Fpath\u002Fto\u002Fvenv\n\n# Use custom model and port\nscripts\u002Fquickstart.sh --model \"Qwen\u002FQwen2.5-1.5B-Instruct\" --port 8000\n\n# Use custom GPU memory utilization\nscripts\u002Fquickstart.sh --gpu_memory_utilization 0.6\n\n# Combine all options\nscripts\u002Fquickstart.sh --venv \u002Fpath\u002Fto\u002Fvenv --model \"Qwen\u002FQwen2.5-1.5B-Instruct\" --port 8000 --gpu_memory_utilization 0.8\n",[208],{"type":41,"tag":174,"props":209,"children":210},{"__ignoreMap":171},[211,219,238,248,257,297,305,314,332,340,349],{"type":41,"tag":178,"props":212,"children":213},{"class":180,"line":181},[214],{"type":41,"tag":178,"props":215,"children":216},{"style":185},[217],{"type":47,"value":218},"# Use custom virtual environment\n",{"type":41,"tag":178,"props":220,"children":221},{"class":180,"line":191},[222,227,233],{"type":41,"tag":178,"props":223,"children":224},{"style":195},[225],{"type":47,"value":226},"scripts\u002Fquickstart.sh",{"type":41,"tag":178,"props":228,"children":230},{"style":229},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[231],{"type":47,"value":232}," --venv",{"type":41,"tag":178,"props":234,"children":235},{"style":229},[236],{"type":47,"value":237}," \u002Fpath\u002Fto\u002Fvenv\n",{"type":41,"tag":178,"props":239,"children":241},{"class":180,"line":240},3,[242],{"type":41,"tag":178,"props":243,"children":245},{"emptyLinePlaceholder":244},true,[246],{"type":47,"value":247},"\n",{"type":41,"tag":178,"props":249,"children":251},{"class":180,"line":250},4,[252],{"type":41,"tag":178,"props":253,"children":254},{"style":185},[255],{"type":47,"value":256},"# Use custom model and port\n",{"type":41,"tag":178,"props":258,"children":260},{"class":180,"line":259},5,[261,265,270,276,281,286,291],{"type":41,"tag":178,"props":262,"children":263},{"style":195},[264],{"type":47,"value":226},{"type":41,"tag":178,"props":266,"children":267},{"style":229},[268],{"type":47,"value":269}," --model",{"type":41,"tag":178,"props":271,"children":273},{"style":272},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[274],{"type":47,"value":275}," \"",{"type":41,"tag":178,"props":277,"children":278},{"style":229},[279],{"type":47,"value":280},"Qwen\u002FQwen2.5-1.5B-Instruct",{"type":41,"tag":178,"props":282,"children":283},{"style":272},[284],{"type":47,"value":285},"\"",{"type":41,"tag":178,"props":287,"children":288},{"style":229},[289],{"type":47,"value":290}," --port",{"type":41,"tag":178,"props":292,"children":294},{"style":293},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[295],{"type":47,"value":296}," 8000\n",{"type":41,"tag":178,"props":298,"children":300},{"class":180,"line":299},6,[301],{"type":41,"tag":178,"props":302,"children":303},{"emptyLinePlaceholder":244},[304],{"type":47,"value":247},{"type":41,"tag":178,"props":306,"children":308},{"class":180,"line":307},7,[309],{"type":41,"tag":178,"props":310,"children":311},{"style":185},[312],{"type":47,"value":313},"# Use custom GPU memory utilization\n",{"type":41,"tag":178,"props":315,"children":317},{"class":180,"line":316},8,[318,322,327],{"type":41,"tag":178,"props":319,"children":320},{"style":195},[321],{"type":47,"value":226},{"type":41,"tag":178,"props":323,"children":324},{"style":229},[325],{"type":47,"value":326}," --gpu_memory_utilization",{"type":41,"tag":178,"props":328,"children":329},{"style":293},[330],{"type":47,"value":331}," 0.6\n",{"type":41,"tag":178,"props":333,"children":335},{"class":180,"line":334},9,[336],{"type":41,"tag":178,"props":337,"children":338},{"emptyLinePlaceholder":244},[339],{"type":47,"value":247},{"type":41,"tag":178,"props":341,"children":343},{"class":180,"line":342},10,[344],{"type":41,"tag":178,"props":345,"children":346},{"style":185},[347],{"type":47,"value":348},"# Combine all options\n",{"type":41,"tag":178,"props":350,"children":352},{"class":180,"line":351},11,[353,357,361,366,370,374,378,382,386,391,395],{"type":41,"tag":178,"props":354,"children":355},{"style":195},[356],{"type":47,"value":226},{"type":41,"tag":178,"props":358,"children":359},{"style":229},[360],{"type":47,"value":232},{"type":41,"tag":178,"props":362,"children":363},{"style":229},[364],{"type":47,"value":365}," \u002Fpath\u002Fto\u002Fvenv",{"type":41,"tag":178,"props":367,"children":368},{"style":229},[369],{"type":47,"value":269},{"type":41,"tag":178,"props":371,"children":372},{"style":272},[373],{"type":47,"value":275},{"type":41,"tag":178,"props":375,"children":376},{"style":229},[377],{"type":47,"value":280},{"type":41,"tag":178,"props":379,"children":380},{"style":272},[381],{"type":47,"value":285},{"type":41,"tag":178,"props":383,"children":384},{"style":229},[385],{"type":47,"value":290},{"type":41,"tag":178,"props":387,"children":388},{"style":293},[389],{"type":47,"value":390}," 8000",{"type":41,"tag":178,"props":392,"children":393},{"style":229},[394],{"type":47,"value":326},{"type":41,"tag":178,"props":396,"children":397},{"style":293},[398],{"type":47,"value":399}," 0.8\n",{"type":41,"tag":50,"props":401,"children":402},{},[403],{"type":47,"value":404},"This will:",{"type":41,"tag":406,"props":407,"children":408},"ol",{},[409,414,419,423,428,433,438],{"type":41,"tag":72,"props":410,"children":411},{},[412],{"type":47,"value":413},"Activate the virtual environment (if specified)",{"type":41,"tag":72,"props":415,"children":416},{},[417],{"type":47,"value":418},"Detect hardware backend (CUDA\u002FROCm\u002FTPU\u002FCPU)",{"type":41,"tag":72,"props":420,"children":421},{},[422],{"type":47,"value":81},{"type":41,"tag":72,"props":424,"children":425},{},[426],{"type":47,"value":427},"Start the vLLM server in the background",{"type":41,"tag":72,"props":429,"children":430},{},[431],{"type":47,"value":432},"Wait for the server to be ready",{"type":41,"tag":72,"props":434,"children":435},{},[436],{"type":47,"value":437},"Test the API with a sample request",{"type":41,"tag":72,"props":439,"children":440},{},[441],{"type":47,"value":442},"Display the server status",{"type":41,"tag":143,"props":444,"children":446},{"id":445},"run-individual-commands-for-step-by-step-usage-or-troubleshooting",[447],{"type":47,"value":448},"Run individual commands (for step-by-step usage or troubleshooting)",{"type":41,"tag":50,"props":450,"children":451},{},[452],{"type":41,"tag":453,"props":454,"children":455},"strong",{},[456],{"type":47,"value":457},"Install vLLM:",{"type":41,"tag":166,"props":459,"children":461},{"className":168,"code":460,"language":170,"meta":171,"style":171},"scripts\u002Fquickstart.sh install\n# Or with virtual environment\nscripts\u002Fquickstart.sh install --venv \u002Fpath\u002Fto\u002Fvenv\n",[462],{"type":41,"tag":174,"props":463,"children":464},{"__ignoreMap":171},[465,477,485],{"type":41,"tag":178,"props":466,"children":467},{"class":180,"line":181},[468,472],{"type":41,"tag":178,"props":469,"children":470},{"style":195},[471],{"type":47,"value":226},{"type":41,"tag":178,"props":473,"children":474},{"style":229},[475],{"type":47,"value":476}," install\n",{"type":41,"tag":178,"props":478,"children":479},{"class":180,"line":191},[480],{"type":41,"tag":178,"props":481,"children":482},{"style":185},[483],{"type":47,"value":484},"# Or with virtual environment\n",{"type":41,"tag":178,"props":486,"children":487},{"class":180,"line":240},[488,492,497,501],{"type":41,"tag":178,"props":489,"children":490},{"style":195},[491],{"type":47,"value":226},{"type":41,"tag":178,"props":493,"children":494},{"style":229},[495],{"type":47,"value":496}," install",{"type":41,"tag":178,"props":498,"children":499},{"style":229},[500],{"type":47,"value":232},{"type":41,"tag":178,"props":502,"children":503},{"style":229},[504],{"type":47,"value":237},{"type":41,"tag":50,"props":506,"children":507},{},[508],{"type":41,"tag":453,"props":509,"children":510},{},[511],{"type":47,"value":512},"Start the server:",{"type":41,"tag":166,"props":514,"children":516},{"className":168,"code":515,"language":170,"meta":171,"style":171},"scripts\u002Fquickstart.sh start\n# Or with custom options\nscripts\u002Fquickstart.sh start --venv \u002Fpath\u002Fto\u002Fvenv --model \"Qwen\u002FQwen2.5-1.5B-Instruct\" --port 8000 --gpu_memory_utilization 0.8\n",[517],{"type":41,"tag":174,"props":518,"children":519},{"__ignoreMap":171},[520,532,540],{"type":41,"tag":178,"props":521,"children":522},{"class":180,"line":181},[523,527],{"type":41,"tag":178,"props":524,"children":525},{"style":195},[526],{"type":47,"value":226},{"type":41,"tag":178,"props":528,"children":529},{"style":229},[530],{"type":47,"value":531}," start\n",{"type":41,"tag":178,"props":533,"children":534},{"class":180,"line":191},[535],{"type":41,"tag":178,"props":536,"children":537},{"style":185},[538],{"type":47,"value":539},"# Or with custom options\n",{"type":41,"tag":178,"props":541,"children":542},{"class":180,"line":240},[543,547,552,556,560,564,568,572,576,580,584,588],{"type":41,"tag":178,"props":544,"children":545},{"style":195},[546],{"type":47,"value":226},{"type":41,"tag":178,"props":548,"children":549},{"style":229},[550],{"type":47,"value":551}," start",{"type":41,"tag":178,"props":553,"children":554},{"style":229},[555],{"type":47,"value":232},{"type":41,"tag":178,"props":557,"children":558},{"style":229},[559],{"type":47,"value":365},{"type":41,"tag":178,"props":561,"children":562},{"style":229},[563],{"type":47,"value":269},{"type":41,"tag":178,"props":565,"children":566},{"style":272},[567],{"type":47,"value":275},{"type":41,"tag":178,"props":569,"children":570},{"style":229},[571],{"type":47,"value":280},{"type":41,"tag":178,"props":573,"children":574},{"style":272},[575],{"type":47,"value":285},{"type":41,"tag":178,"props":577,"children":578},{"style":229},[579],{"type":47,"value":290},{"type":41,"tag":178,"props":581,"children":582},{"style":293},[583],{"type":47,"value":390},{"type":41,"tag":178,"props":585,"children":586},{"style":229},[587],{"type":47,"value":326},{"type":41,"tag":178,"props":589,"children":590},{"style":293},[591],{"type":47,"value":399},{"type":41,"tag":50,"props":593,"children":594},{},[595],{"type":41,"tag":453,"props":596,"children":597},{},[598],{"type":47,"value":599},"Test the API:",{"type":41,"tag":166,"props":601,"children":603},{"className":168,"code":602,"language":170,"meta":171,"style":171},"scripts\u002Fquickstart.sh test\n# Or with custom port\nscripts\u002Fquickstart.sh test --port 8000\n",[604],{"type":41,"tag":174,"props":605,"children":606},{"__ignoreMap":171},[607,619,627],{"type":41,"tag":178,"props":608,"children":609},{"class":180,"line":181},[610,614],{"type":41,"tag":178,"props":611,"children":612},{"style":195},[613],{"type":47,"value":226},{"type":41,"tag":178,"props":615,"children":616},{"style":229},[617],{"type":47,"value":618}," test\n",{"type":41,"tag":178,"props":620,"children":621},{"class":180,"line":191},[622],{"type":41,"tag":178,"props":623,"children":624},{"style":185},[625],{"type":47,"value":626},"# Or with custom port\n",{"type":41,"tag":178,"props":628,"children":629},{"class":180,"line":240},[630,634,639,643],{"type":41,"tag":178,"props":631,"children":632},{"style":195},[633],{"type":47,"value":226},{"type":41,"tag":178,"props":635,"children":636},{"style":229},[637],{"type":47,"value":638}," test",{"type":41,"tag":178,"props":640,"children":641},{"style":229},[642],{"type":47,"value":290},{"type":41,"tag":178,"props":644,"children":645},{"style":293},[646],{"type":47,"value":296},{"type":41,"tag":50,"props":648,"children":649},{},[650],{"type":41,"tag":453,"props":651,"children":652},{},[653],{"type":47,"value":654},"Stop the server:",{"type":41,"tag":166,"props":656,"children":658},{"className":168,"code":657,"language":170,"meta":171,"style":171},"scripts\u002Fquickstart.sh stop\n# Or with virtual environment\nscripts\u002Fquickstart.sh stop --venv \u002Fpath\u002Fto\u002Fvenv\n",[659],{"type":41,"tag":174,"props":660,"children":661},{"__ignoreMap":171},[662,674,681],{"type":41,"tag":178,"props":663,"children":664},{"class":180,"line":181},[665,669],{"type":41,"tag":178,"props":666,"children":667},{"style":195},[668],{"type":47,"value":226},{"type":41,"tag":178,"props":670,"children":671},{"style":229},[672],{"type":47,"value":673}," stop\n",{"type":41,"tag":178,"props":675,"children":676},{"class":180,"line":191},[677],{"type":41,"tag":178,"props":678,"children":679},{"style":185},[680],{"type":47,"value":484},{"type":41,"tag":178,"props":682,"children":683},{"class":180,"line":240},[684,688,693,697],{"type":41,"tag":178,"props":685,"children":686},{"style":195},[687],{"type":47,"value":226},{"type":41,"tag":178,"props":689,"children":690},{"style":229},[691],{"type":47,"value":692}," stop",{"type":41,"tag":178,"props":694,"children":695},{"style":229},[696],{"type":47,"value":232},{"type":41,"tag":178,"props":698,"children":699},{"style":229},[700],{"type":47,"value":237},{"type":41,"tag":50,"props":702,"children":703},{},[704],{"type":41,"tag":453,"props":705,"children":706},{},[707],{"type":47,"value":708},"Check server status:",{"type":41,"tag":166,"props":710,"children":712},{"className":168,"code":711,"language":170,"meta":171,"style":171},"scripts\u002Fquickstart.sh status\n",[713],{"type":41,"tag":174,"props":714,"children":715},{"__ignoreMap":171},[716],{"type":41,"tag":178,"props":717,"children":718},{"class":180,"line":181},[719,723],{"type":41,"tag":178,"props":720,"children":721},{"style":195},[722],{"type":47,"value":226},{"type":41,"tag":178,"props":724,"children":725},{"style":229},[726],{"type":47,"value":727}," status\n",{"type":41,"tag":50,"props":729,"children":730},{},[731],{"type":41,"tag":453,"props":732,"children":733},{},[734],{"type":47,"value":735},"Restart the server:",{"type":41,"tag":166,"props":737,"children":739},{"className":168,"code":738,"language":170,"meta":171,"style":171},"scripts\u002Fquickstart.sh restart\n# Or with custom options\nscripts\u002Fquickstart.sh restart --venv \u002Fpath\u002Fto\u002Fvenv --port 8000 --gpu_memory_utilization 0.8\n",[740],{"type":41,"tag":174,"props":741,"children":742},{"__ignoreMap":171},[743,755,762],{"type":41,"tag":178,"props":744,"children":745},{"class":180,"line":181},[746,750],{"type":41,"tag":178,"props":747,"children":748},{"style":195},[749],{"type":47,"value":226},{"type":41,"tag":178,"props":751,"children":752},{"style":229},[753],{"type":47,"value":754}," restart\n",{"type":41,"tag":178,"props":756,"children":757},{"class":180,"line":191},[758],{"type":41,"tag":178,"props":759,"children":760},{"style":185},[761],{"type":47,"value":539},{"type":41,"tag":178,"props":763,"children":764},{"class":180,"line":240},[765,769,774,778,782,786,790,794],{"type":41,"tag":178,"props":766,"children":767},{"style":195},[768],{"type":47,"value":226},{"type":41,"tag":178,"props":770,"children":771},{"style":229},[772],{"type":47,"value":773}," restart",{"type":41,"tag":178,"props":775,"children":776},{"style":229},[777],{"type":47,"value":232},{"type":41,"tag":178,"props":779,"children":780},{"style":229},[781],{"type":47,"value":365},{"type":41,"tag":178,"props":783,"children":784},{"style":229},[785],{"type":47,"value":290},{"type":41,"tag":178,"props":787,"children":788},{"style":293},[789],{"type":47,"value":390},{"type":41,"tag":178,"props":791,"children":792},{"style":229},[793],{"type":47,"value":326},{"type":41,"tag":178,"props":795,"children":796},{"style":293},[797],{"type":47,"value":399},{"type":41,"tag":56,"props":799,"children":801},{"id":800},"configuration",[802],{"type":47,"value":803},"Configuration",{"type":41,"tag":50,"props":805,"children":806},{},[807],{"type":47,"value":808},"The script supports the following command-line options:",{"type":41,"tag":166,"props":810,"children":812},{"className":168,"code":811,"language":170,"meta":171,"style":171},"scripts\u002Fquickstart.sh [command] [OPTIONS]\n\nCommands:\n  install  - Install vLLM and dependencies\n  start    - Start the vLLM server\n  stop     - Stop the vLLM server\n  test     - Test the OpenAI-compatible API\n  status   - Show server status\n  restart  - Restart the server\n  all      - Run complete workflow (default)\n\nOptions:\n  --model MODEL                 Model to use (default: Qwen\u002FQwen2.5-1.5B-Instruct)\n  --port PORT                   Port to run server on (default: 8000)\n  --venv VENV_PATH              Virtual environment path (default: .)\n  --gpu_memory_utilization VRAM GPU memory utilization (default: 0.8)\n",[813],{"type":41,"tag":174,"props":814,"children":815},{"__ignoreMap":171},[816,844,851,859,892,924,954,986,1013,1038,1071,1078,1087,1130,1180,1222],{"type":41,"tag":178,"props":817,"children":818},{"class":180,"line":181},[819,823,829,834,839],{"type":41,"tag":178,"props":820,"children":821},{"style":195},[822],{"type":47,"value":226},{"type":41,"tag":178,"props":824,"children":826},{"style":825},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[827],{"type":47,"value":828}," [command] ",{"type":41,"tag":178,"props":830,"children":831},{"style":272},[832],{"type":47,"value":833},"[",{"type":41,"tag":178,"props":835,"children":836},{"style":825},[837],{"type":47,"value":838},"OPTIONS",{"type":41,"tag":178,"props":840,"children":841},{"style":272},[842],{"type":47,"value":843},"]\n",{"type":41,"tag":178,"props":845,"children":846},{"class":180,"line":191},[847],{"type":41,"tag":178,"props":848,"children":849},{"emptyLinePlaceholder":244},[850],{"type":47,"value":247},{"type":41,"tag":178,"props":852,"children":853},{"class":180,"line":240},[854],{"type":41,"tag":178,"props":855,"children":856},{"style":195},[857],{"type":47,"value":858},"Commands:\n",{"type":41,"tag":178,"props":860,"children":861},{"class":180,"line":250},[862,867,872,877,882,887],{"type":41,"tag":178,"props":863,"children":864},{"style":195},[865],{"type":47,"value":866},"  install",{"type":41,"tag":178,"props":868,"children":869},{"style":229},[870],{"type":47,"value":871},"  -",{"type":41,"tag":178,"props":873,"children":874},{"style":229},[875],{"type":47,"value":876}," Install",{"type":41,"tag":178,"props":878,"children":879},{"style":229},[880],{"type":47,"value":881}," vLLM",{"type":41,"tag":178,"props":883,"children":884},{"style":229},[885],{"type":47,"value":886}," and",{"type":41,"tag":178,"props":888,"children":889},{"style":229},[890],{"type":47,"value":891}," dependencies\n",{"type":41,"tag":178,"props":893,"children":894},{"class":180,"line":259},[895,900,905,910,915,919],{"type":41,"tag":178,"props":896,"children":897},{"style":195},[898],{"type":47,"value":899},"  start",{"type":41,"tag":178,"props":901,"children":902},{"style":229},[903],{"type":47,"value":904},"    -",{"type":41,"tag":178,"props":906,"children":907},{"style":229},[908],{"type":47,"value":909}," Start",{"type":41,"tag":178,"props":911,"children":912},{"style":229},[913],{"type":47,"value":914}," the",{"type":41,"tag":178,"props":916,"children":917},{"style":229},[918],{"type":47,"value":881},{"type":41,"tag":178,"props":920,"children":921},{"style":229},[922],{"type":47,"value":923}," server\n",{"type":41,"tag":178,"props":925,"children":926},{"class":180,"line":299},[927,932,937,942,946,950],{"type":41,"tag":178,"props":928,"children":929},{"style":195},[930],{"type":47,"value":931},"  stop",{"type":41,"tag":178,"props":933,"children":934},{"style":229},[935],{"type":47,"value":936},"     -",{"type":41,"tag":178,"props":938,"children":939},{"style":229},[940],{"type":47,"value":941}," Stop",{"type":41,"tag":178,"props":943,"children":944},{"style":229},[945],{"type":47,"value":914},{"type":41,"tag":178,"props":947,"children":948},{"style":229},[949],{"type":47,"value":881},{"type":41,"tag":178,"props":951,"children":952},{"style":229},[953],{"type":47,"value":923},{"type":41,"tag":178,"props":955,"children":956},{"class":180,"line":307},[957,963,967,972,976,981],{"type":41,"tag":178,"props":958,"children":960},{"style":959},"--shiki-light:#6182B8;--shiki-default:#82AAFF;--shiki-dark:#82AAFF",[961],{"type":47,"value":962},"  test",{"type":41,"tag":178,"props":964,"children":965},{"style":229},[966],{"type":47,"value":936},{"type":41,"tag":178,"props":968,"children":969},{"style":229},[970],{"type":47,"value":971}," Test",{"type":41,"tag":178,"props":973,"children":974},{"style":229},[975],{"type":47,"value":914},{"type":41,"tag":178,"props":977,"children":978},{"style":229},[979],{"type":47,"value":980}," OpenAI-compatible",{"type":41,"tag":178,"props":982,"children":983},{"style":229},[984],{"type":47,"value":985}," API\n",{"type":41,"tag":178,"props":987,"children":988},{"class":180,"line":316},[989,994,999,1004,1009],{"type":41,"tag":178,"props":990,"children":991},{"style":195},[992],{"type":47,"value":993},"  status",{"type":41,"tag":178,"props":995,"children":996},{"style":229},[997],{"type":47,"value":998},"   -",{"type":41,"tag":178,"props":1000,"children":1001},{"style":229},[1002],{"type":47,"value":1003}," Show",{"type":41,"tag":178,"props":1005,"children":1006},{"style":229},[1007],{"type":47,"value":1008}," server",{"type":41,"tag":178,"props":1010,"children":1011},{"style":229},[1012],{"type":47,"value":727},{"type":41,"tag":178,"props":1014,"children":1015},{"class":180,"line":334},[1016,1021,1025,1030,1034],{"type":41,"tag":178,"props":1017,"children":1018},{"style":195},[1019],{"type":47,"value":1020},"  restart",{"type":41,"tag":178,"props":1022,"children":1023},{"style":229},[1024],{"type":47,"value":871},{"type":41,"tag":178,"props":1026,"children":1027},{"style":229},[1028],{"type":47,"value":1029}," Restart",{"type":41,"tag":178,"props":1031,"children":1032},{"style":229},[1033],{"type":47,"value":914},{"type":41,"tag":178,"props":1035,"children":1036},{"style":229},[1037],{"type":47,"value":923},{"type":41,"tag":178,"props":1039,"children":1040},{"class":180,"line":342},[1041,1046,1051,1056,1061,1066],{"type":41,"tag":178,"props":1042,"children":1043},{"style":195},[1044],{"type":47,"value":1045},"  all",{"type":41,"tag":178,"props":1047,"children":1048},{"style":229},[1049],{"type":47,"value":1050},"      -",{"type":41,"tag":178,"props":1052,"children":1053},{"style":229},[1054],{"type":47,"value":1055}," Run",{"type":41,"tag":178,"props":1057,"children":1058},{"style":229},[1059],{"type":47,"value":1060}," complete",{"type":41,"tag":178,"props":1062,"children":1063},{"style":229},[1064],{"type":47,"value":1065}," workflow",{"type":41,"tag":178,"props":1067,"children":1068},{"style":825},[1069],{"type":47,"value":1070}," (default)\n",{"type":41,"tag":178,"props":1072,"children":1073},{"class":180,"line":351},[1074],{"type":41,"tag":178,"props":1075,"children":1076},{"emptyLinePlaceholder":244},[1077],{"type":47,"value":247},{"type":41,"tag":178,"props":1079,"children":1081},{"class":180,"line":1080},12,[1082],{"type":41,"tag":178,"props":1083,"children":1084},{"style":195},[1085],{"type":47,"value":1086},"Options:\n",{"type":41,"tag":178,"props":1088,"children":1090},{"class":180,"line":1089},13,[1091,1096,1101,1106,1111,1116,1121,1125],{"type":41,"tag":178,"props":1092,"children":1093},{"style":195},[1094],{"type":47,"value":1095},"  --model",{"type":41,"tag":178,"props":1097,"children":1098},{"style":229},[1099],{"type":47,"value":1100}," MODEL",{"type":41,"tag":178,"props":1102,"children":1103},{"style":229},[1104],{"type":47,"value":1105},"                 Model",{"type":41,"tag":178,"props":1107,"children":1108},{"style":229},[1109],{"type":47,"value":1110}," to",{"type":41,"tag":178,"props":1112,"children":1113},{"style":229},[1114],{"type":47,"value":1115}," use",{"type":41,"tag":178,"props":1117,"children":1118},{"style":825},[1119],{"type":47,"value":1120}," (default: ",{"type":41,"tag":178,"props":1122,"children":1123},{"style":229},[1124],{"type":47,"value":280},{"type":41,"tag":178,"props":1126,"children":1127},{"style":825},[1128],{"type":47,"value":1129},")\n",{"type":41,"tag":178,"props":1131,"children":1133},{"class":180,"line":1132},14,[1134,1139,1144,1149,1153,1158,1162,1167,1171,1176],{"type":41,"tag":178,"props":1135,"children":1136},{"style":195},[1137],{"type":47,"value":1138},"  --port",{"type":41,"tag":178,"props":1140,"children":1141},{"style":229},[1142],{"type":47,"value":1143}," PORT",{"type":41,"tag":178,"props":1145,"children":1146},{"style":229},[1147],{"type":47,"value":1148},"                   Port",{"type":41,"tag":178,"props":1150,"children":1151},{"style":229},[1152],{"type":47,"value":1110},{"type":41,"tag":178,"props":1154,"children":1155},{"style":229},[1156],{"type":47,"value":1157}," run",{"type":41,"tag":178,"props":1159,"children":1160},{"style":229},[1161],{"type":47,"value":1008},{"type":41,"tag":178,"props":1163,"children":1164},{"style":229},[1165],{"type":47,"value":1166}," on",{"type":41,"tag":178,"props":1168,"children":1169},{"style":825},[1170],{"type":47,"value":1120},{"type":41,"tag":178,"props":1172,"children":1173},{"style":293},[1174],{"type":47,"value":1175},"8000",{"type":41,"tag":178,"props":1177,"children":1178},{"style":825},[1179],{"type":47,"value":1129},{"type":41,"tag":178,"props":1181,"children":1183},{"class":180,"line":1182},15,[1184,1189,1194,1199,1204,1209,1213,1218],{"type":41,"tag":178,"props":1185,"children":1186},{"style":195},[1187],{"type":47,"value":1188},"  --venv",{"type":41,"tag":178,"props":1190,"children":1191},{"style":229},[1192],{"type":47,"value":1193}," VENV_PATH",{"type":41,"tag":178,"props":1195,"children":1196},{"style":229},[1197],{"type":47,"value":1198},"              Virtual",{"type":41,"tag":178,"props":1200,"children":1201},{"style":229},[1202],{"type":47,"value":1203}," environment",{"type":41,"tag":178,"props":1205,"children":1206},{"style":229},[1207],{"type":47,"value":1208}," path",{"type":41,"tag":178,"props":1210,"children":1211},{"style":825},[1212],{"type":47,"value":1120},{"type":41,"tag":178,"props":1214,"children":1215},{"style":229},[1216],{"type":47,"value":1217},".",{"type":41,"tag":178,"props":1219,"children":1220},{"style":825},[1221],{"type":47,"value":1129},{"type":41,"tag":178,"props":1223,"children":1225},{"class":180,"line":1224},16,[1226,1231,1236,1241,1246,1251,1255,1260],{"type":41,"tag":178,"props":1227,"children":1228},{"style":195},[1229],{"type":47,"value":1230},"  --gpu_memory_utilization",{"type":41,"tag":178,"props":1232,"children":1233},{"style":229},[1234],{"type":47,"value":1235}," VRAM",{"type":41,"tag":178,"props":1237,"children":1238},{"style":229},[1239],{"type":47,"value":1240}," GPU",{"type":41,"tag":178,"props":1242,"children":1243},{"style":229},[1244],{"type":47,"value":1245}," memory",{"type":41,"tag":178,"props":1247,"children":1248},{"style":229},[1249],{"type":47,"value":1250}," utilization",{"type":41,"tag":178,"props":1252,"children":1253},{"style":825},[1254],{"type":47,"value":1120},{"type":41,"tag":178,"props":1256,"children":1257},{"style":293},[1258],{"type":47,"value":1259},"0.8",{"type":41,"tag":178,"props":1261,"children":1262},{"style":825},[1263],{"type":47,"value":1129},{"type":41,"tag":143,"props":1265,"children":1267},{"id":1266},"hardware-backend-detection",[1268],{"type":47,"value":1269},"Hardware Backend Detection",{"type":41,"tag":50,"props":1271,"children":1272},{},[1273],{"type":47,"value":1274},"The script automatically detects your hardware and installs the appropriate vLLM version:",{"type":41,"tag":68,"props":1276,"children":1277},{},[1278,1296,1321,1345],{"type":41,"tag":72,"props":1279,"children":1280},{},[1281,1286,1288,1294],{"type":41,"tag":453,"props":1282,"children":1283},{},[1284],{"type":47,"value":1285},"NVIDIA CUDA",{"type":47,"value":1287},": Detected via ",{"type":41,"tag":174,"props":1289,"children":1291},{"className":1290},[],[1292],{"type":47,"value":1293},"nvidia-smi",{"type":47,"value":1295}," command",{"type":41,"tag":72,"props":1297,"children":1298},{},[1299,1304,1305,1311,1313,1319],{"type":41,"tag":453,"props":1300,"children":1301},{},[1302],{"type":47,"value":1303},"AMD ROCm",{"type":47,"value":1287},{"type":41,"tag":174,"props":1306,"children":1308},{"className":1307},[],[1309],{"type":47,"value":1310},"\u002Fdev\u002Fkfd",{"type":47,"value":1312}," and ",{"type":41,"tag":174,"props":1314,"children":1316},{"className":1315},[],[1317],{"type":47,"value":1318},"\u002Fdev\u002Fdri",{"type":47,"value":1320}," devices",{"type":41,"tag":72,"props":1322,"children":1323},{},[1324,1329,1330,1336,1338,1344],{"type":41,"tag":453,"props":1325,"children":1326},{},[1327],{"type":47,"value":1328},"Google TPU",{"type":47,"value":1287},{"type":41,"tag":174,"props":1331,"children":1333},{"className":1332},[],[1334],{"type":47,"value":1335},"TPU_NAME",{"type":47,"value":1337}," environment variable or ",{"type":41,"tag":174,"props":1339,"children":1341},{"className":1340},[],[1342],{"type":47,"value":1343},"gcloud",{"type":47,"value":1295},{"type":41,"tag":72,"props":1346,"children":1347},{},[1348,1353],{"type":41,"tag":453,"props":1349,"children":1350},{},[1351],{"type":47,"value":1352},"CPU",{"type":47,"value":1354},": Fallback if no GPU\u002FTPU detected",{"type":41,"tag":50,"props":1356,"children":1357},{},[1358,1360,1366,1368,1373],{"type":47,"value":1359},"For Google TPU, the script installs ",{"type":41,"tag":174,"props":1361,"children":1363},{"className":1362},[],[1364],{"type":47,"value":1365},"vllm-tpu",{"type":47,"value":1367}," instead of the standard ",{"type":41,"tag":174,"props":1369,"children":1371},{"className":1370},[],[1372],{"type":47,"value":8},{"type":47,"value":1374}," package.",{"type":41,"tag":56,"props":1376,"children":1378},{"id":1377},"api-testing",[1379],{"type":47,"value":1380},"API Testing",{"type":41,"tag":50,"props":1382,"children":1383},{},[1384],{"type":47,"value":1385},"The test script sends a simple chat completion request:",{"type":41,"tag":166,"props":1387,"children":1389},{"className":168,"code":1388,"language":170,"meta":171,"style":171},"curl http:\u002F\u002Flocalhost:8000\u002Fv1\u002Fchat\u002Fcompletions \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\n    \"model\": \"Qwen\u002FQwen2.5-1.5B-Instruct\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Say hello!\"}],\n    \"max_tokens\": 50\n  }'\n",[1390],{"type":41,"tag":174,"props":1391,"children":1392},{"__ignoreMap":171},[1393,1411,1436,1454,1462,1470,1478],{"type":41,"tag":178,"props":1394,"children":1395},{"class":180,"line":181},[1396,1401,1406],{"type":41,"tag":178,"props":1397,"children":1398},{"style":195},[1399],{"type":47,"value":1400},"curl",{"type":41,"tag":178,"props":1402,"children":1403},{"style":229},[1404],{"type":47,"value":1405}," http:\u002F\u002Flocalhost:8000\u002Fv1\u002Fchat\u002Fcompletions",{"type":41,"tag":178,"props":1407,"children":1408},{"style":825},[1409],{"type":47,"value":1410}," \\\n",{"type":41,"tag":178,"props":1412,"children":1413},{"class":180,"line":191},[1414,1419,1423,1428,1432],{"type":41,"tag":178,"props":1415,"children":1416},{"style":229},[1417],{"type":47,"value":1418},"  -H",{"type":41,"tag":178,"props":1420,"children":1421},{"style":272},[1422],{"type":47,"value":275},{"type":41,"tag":178,"props":1424,"children":1425},{"style":229},[1426],{"type":47,"value":1427},"Content-Type: application\u002Fjson",{"type":41,"tag":178,"props":1429,"children":1430},{"style":272},[1431],{"type":47,"value":285},{"type":41,"tag":178,"props":1433,"children":1434},{"style":825},[1435],{"type":47,"value":1410},{"type":41,"tag":178,"props":1437,"children":1438},{"class":180,"line":240},[1439,1444,1449],{"type":41,"tag":178,"props":1440,"children":1441},{"style":229},[1442],{"type":47,"value":1443},"  -d",{"type":41,"tag":178,"props":1445,"children":1446},{"style":272},[1447],{"type":47,"value":1448}," '",{"type":41,"tag":178,"props":1450,"children":1451},{"style":229},[1452],{"type":47,"value":1453},"{\n",{"type":41,"tag":178,"props":1455,"children":1456},{"class":180,"line":250},[1457],{"type":41,"tag":178,"props":1458,"children":1459},{"style":229},[1460],{"type":47,"value":1461},"    \"model\": \"Qwen\u002FQwen2.5-1.5B-Instruct\",\n",{"type":41,"tag":178,"props":1463,"children":1464},{"class":180,"line":259},[1465],{"type":41,"tag":178,"props":1466,"children":1467},{"style":229},[1468],{"type":47,"value":1469},"    \"messages\": [{\"role\": \"user\", \"content\": \"Say hello!\"}],\n",{"type":41,"tag":178,"props":1471,"children":1472},{"class":180,"line":299},[1473],{"type":41,"tag":178,"props":1474,"children":1475},{"style":229},[1476],{"type":47,"value":1477},"    \"max_tokens\": 50\n",{"type":41,"tag":178,"props":1479,"children":1480},{"class":180,"line":307},[1481,1486],{"type":41,"tag":178,"props":1482,"children":1483},{"style":229},[1484],{"type":47,"value":1485},"  }",{"type":41,"tag":178,"props":1487,"children":1488},{"style":272},[1489],{"type":47,"value":1490},"'\n",{"type":41,"tag":56,"props":1492,"children":1494},{"id":1493},"troubleshooting",[1495],{"type":47,"value":1496},"Troubleshooting",{"type":41,"tag":50,"props":1498,"children":1499},{},[1500],{"type":41,"tag":453,"props":1501,"children":1502},{},[1503],{"type":47,"value":1504},"Virtual environment not found:",{"type":41,"tag":68,"props":1506,"children":1507},{},[1508,1521,1542],{"type":41,"tag":72,"props":1509,"children":1510},{},[1511,1513,1519],{"type":47,"value":1512},"Ensure the path provided with ",{"type":41,"tag":174,"props":1514,"children":1516},{"className":1515},[],[1517],{"type":47,"value":1518},"--venv",{"type":47,"value":1520}," exists and is a valid virtual environment",{"type":41,"tag":72,"props":1522,"children":1523},{},[1524,1526,1532,1534,1540],{"type":47,"value":1525},"Check that the activation script exists (",{"type":41,"tag":174,"props":1527,"children":1529},{"className":1528},[],[1530],{"type":47,"value":1531},"bin\u002Factivate",{"type":47,"value":1533}," on Linux\u002FmacOS or ",{"type":41,"tag":174,"props":1535,"children":1537},{"className":1536},[],[1538],{"type":47,"value":1539},"Scripts\u002Factivate",{"type":47,"value":1541}," on Windows)",{"type":41,"tag":72,"props":1543,"children":1544},{},[1545,1547,1553,1555],{"type":47,"value":1546},"Check and install uv, and create a new virtual environment with uv: ",{"type":41,"tag":174,"props":1548,"children":1550},{"className":1549},[],[1551],{"type":47,"value":1552},"uv venv \u002Fpath\u002Fto\u002Fvenv",{"type":47,"value":1554}," (suggested); or with pip: ",{"type":41,"tag":174,"props":1556,"children":1558},{"className":1557},[],[1559],{"type":47,"value":1560},"python3 -m venv \u002Fpath\u002Fto\u002Fvenv",{"type":41,"tag":50,"props":1562,"children":1563},{},[1564],{"type":41,"tag":453,"props":1565,"children":1566},{},[1567],{"type":47,"value":1568},"Server won't start:",{"type":41,"tag":68,"props":1570,"children":1571},{},[1572,1583,1603,1614],{"type":41,"tag":72,"props":1573,"children":1574},{},[1575,1577],{"type":47,"value":1576},"Check if the port is already in use: ",{"type":41,"tag":174,"props":1578,"children":1580},{"className":1579},[],[1581],{"type":47,"value":1582},"lsof -i :8000",{"type":41,"tag":72,"props":1584,"children":1585},{},[1586,1588,1593,1595,1601],{"type":47,"value":1587},"Verify GPU availability: ",{"type":41,"tag":174,"props":1589,"children":1591},{"className":1590},[],[1592],{"type":47,"value":1293},{"type":47,"value":1594}," (for NVIDIA) or ",{"type":41,"tag":174,"props":1596,"children":1598},{"className":1597},[],[1599],{"type":47,"value":1600},"rocm-smi",{"type":47,"value":1602}," (for AMD)",{"type":41,"tag":72,"props":1604,"children":1605},{},[1606,1608],{"type":47,"value":1607},"Check vLLM installation: ",{"type":41,"tag":174,"props":1609,"children":1611},{"className":1610},[],[1612],{"type":47,"value":1613},"python -c \"import vllm; print(vllm.__version__)\"",{"type":41,"tag":72,"props":1615,"children":1616},{},[1617,1619],{"type":47,"value":1618},"Review server logs at ",{"type":41,"tag":174,"props":1620,"children":1622},{"className":1621},[],[1623],{"type":47,"value":1624},"$VENV_PATH\u002Ftmp\u002Fvllm-server.log",{"type":41,"tag":50,"props":1626,"children":1627},{},[1628],{"type":41,"tag":453,"props":1629,"children":1630},{},[1631],{"type":47,"value":1632},"API returns errors:",{"type":41,"tag":68,"props":1634,"children":1635},{},[1636,1641,1652],{"type":41,"tag":72,"props":1637,"children":1638},{},[1639],{"type":47,"value":1640},"Wait a few seconds for the model to load",{"type":41,"tag":72,"props":1642,"children":1643},{},[1644,1646],{"type":47,"value":1645},"Check server logs: ",{"type":41,"tag":174,"props":1647,"children":1649},{"className":1648},[],[1650],{"type":47,"value":1651},"cat $VENV_PATH\u002Ftmp\u002Fvllm-server.log",{"type":41,"tag":72,"props":1653,"children":1654},{},[1655,1657],{"type":47,"value":1656},"Verify the server is running: ",{"type":41,"tag":174,"props":1658,"children":1660},{"className":1659},[],[1661],{"type":47,"value":1662},"scripts\u002Fquickstart.sh status",{"type":41,"tag":50,"props":1664,"children":1665},{},[1666],{"type":41,"tag":453,"props":1667,"children":1668},{},[1669],{"type":47,"value":1670},"Out of memory:",{"type":41,"tag":68,"props":1672,"children":1673},{},[1674,1679,1692],{"type":41,"tag":72,"props":1675,"children":1676},{},[1677],{"type":47,"value":1678},"Use a smaller model (e.g., Qwen2.5-0.5B-Instruct)",{"type":41,"tag":72,"props":1680,"children":1681},{},[1682,1684,1690],{"type":47,"value":1683},"Reduce ",{"type":41,"tag":174,"props":1685,"children":1687},{"className":1686},[],[1688],{"type":47,"value":1689},"--gpu-memory-utilization",{"type":47,"value":1691}," parameter",{"type":41,"tag":72,"props":1693,"children":1694},{},[1695],{"type":47,"value":1696},"Close other GPU-intensive applications",{"type":41,"tag":50,"props":1698,"children":1699},{},[1700],{"type":41,"tag":453,"props":1701,"children":1702},{},[1703],{"type":47,"value":1704},"Wrong backend detected:",{"type":41,"tag":68,"props":1706,"children":1707},{},[1708,1720,1725],{"type":41,"tag":72,"props":1709,"children":1710},{},[1711,1713,1718],{"type":47,"value":1712},"For NVIDIA: Ensure ",{"type":41,"tag":174,"props":1714,"children":1716},{"className":1715},[],[1717],{"type":47,"value":1293},{"type":47,"value":1719}," is in your PATH",{"type":41,"tag":72,"props":1721,"children":1722},{},[1723],{"type":47,"value":1724},"For AMD: Check that ROCm drivers are properly installed",{"type":41,"tag":72,"props":1726,"children":1727},{},[1728,1730,1735,1737],{"type":47,"value":1729},"For TPU: Set ",{"type":41,"tag":174,"props":1731,"children":1733},{"className":1732},[],[1734],{"type":47,"value":1335},{"type":47,"value":1736}," environment variable or install ",{"type":41,"tag":174,"props":1738,"children":1740},{"className":1739},[],[1741],{"type":47,"value":1343},{"type":41,"tag":56,"props":1743,"children":1745},{"id":1744},"notes",[1746],{"type":47,"value":1747},"Notes",{"type":41,"tag":68,"props":1749,"children":1750},{},[1751,1761,1774,1779,1784,1803,1808],{"type":41,"tag":72,"props":1752,"children":1753},{},[1754,1756],{"type":47,"value":1755},"The server runs in the background and logs to ",{"type":41,"tag":174,"props":1757,"children":1759},{"className":1758},[],[1760],{"type":47,"value":1624},{"type":41,"tag":72,"props":1762,"children":1763},{},[1764,1766,1772],{"type":47,"value":1765},"The PID is stored in ",{"type":41,"tag":174,"props":1767,"children":1769},{"className":1768},[],[1770],{"type":47,"value":1771},"$VENV_PATH\u002Ftmp\u002Fvllm-server.pid",{"type":47,"value":1773}," for easy management",{"type":41,"tag":72,"props":1775,"children":1776},{},[1777],{"type":47,"value":1778},"First run will download the model (~3GB for Qwen2.5-1.5B-Instruct)",{"type":41,"tag":72,"props":1780,"children":1781},{},[1782],{"type":47,"value":1783},"Subsequent runs will use the cached model",{"type":41,"tag":72,"props":1785,"children":1786},{},[1787,1789,1795,1797],{"type":47,"value":1788},"The script automatically detects and uses ",{"type":41,"tag":174,"props":1790,"children":1792},{"className":1791},[],[1793],{"type":47,"value":1794},"uv",{"type":47,"value":1796}," if available, otherwise falls back to ",{"type":41,"tag":174,"props":1798,"children":1800},{"className":1799},[],[1801],{"type":47,"value":1802},"pip",{"type":41,"tag":72,"props":1804,"children":1805},{},[1806],{"type":47,"value":1807},"Virtual environment support allows isolation from system Python packages",{"type":41,"tag":72,"props":1809,"children":1810},{},[1811,1813,1819],{"type":47,"value":1812},"Arguments can be specified in any order (e.g., ",{"type":41,"tag":174,"props":1814,"children":1816},{"className":1815},[],[1817],{"type":47,"value":1818},"scripts\u002Fquickstart.sh --port 8080 start --venv \u002Fpath\u002Fto\u002Fvenv",{"type":47,"value":1820},")",{"type":41,"tag":1822,"props":1823,"children":1824},"style",{},[1825],{"type":47,"value":1826},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":1828,"total":307},[1829,1846,1861,1873,1886,1902,1909],{"slug":1830,"name":1830,"fn":1831,"description":1832,"org":1833,"tags":1834,"stars":1843,"repoUrl":1844,"updatedAt":1845},"openclaw-vsr-bridge","integrate vLLM Semantic Router","Install vLLM Semantic Router in agent-safe mode, import supported OpenClaw model providers into canonical VSR config, and rewrite OpenClaw to target VSR.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1835,1838,1839,1842],{"name":1836,"slug":1837,"type":16},"Agents","agents",{"name":14,"slug":15,"type":16},{"name":1840,"slug":1841,"type":16},"Routing","routing",{"name":9,"slug":8,"type":16},4962,"https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fsemantic-router","2026-07-18T05:48:09.091161",{"slug":1847,"name":1847,"fn":1848,"description":1849,"org":1850,"tags":1851,"stars":24,"repoUrl":25,"updatedAt":1860},"vllm-bench-random-synthetic","run vLLM synthetic performance benchmarks","Run vLLM performance benchmark using synthetic random data to measure throughput, TTFT (Time to First Token), TPOT (Time per Output Token), and other key performance metrics. Use when the user wants to quickly test vLLM serving performance without downloading external datasets.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1852,1855,1856,1859],{"name":1853,"slug":1854,"type":16},"Benchmarking","benchmarking",{"name":14,"slug":15,"type":16},{"name":1857,"slug":1858,"type":16},"Performance","performance",{"name":9,"slug":8,"type":16},"2026-07-18T05:46:59.036927",{"slug":1862,"name":1862,"fn":1863,"description":1864,"org":1865,"tags":1866,"stars":24,"repoUrl":25,"updatedAt":1872},"vllm-bench-serve","benchmark LLM serving endpoints","Benchmark vLLM or OpenAI-compatible serving endpoints using vllm bench serve. Supports multiple datasets (random, sharegpt, sonnet, HF), backends (openai, openai-chat, vllm-pooling, embeddings), throughput\u002Flatency testing with request-rate control, and result saving. Use when benchmarking LLM serving performance, measuring TTFT\u002FTPOT, or load testing inference APIs.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1867,1868,1869,1870,1871],{"name":22,"slug":23,"type":16},{"name":1853,"slug":1854,"type":16},{"name":14,"slug":15,"type":16},{"name":1857,"slug":1858,"type":16},{"name":9,"slug":8,"type":16},"2026-07-18T05:47:02.283164",{"slug":1874,"name":1874,"fn":1875,"description":1876,"org":1877,"tags":1878,"stars":24,"repoUrl":25,"updatedAt":1885},"vllm-deploy-docker","deploy vLLM using Docker containers","Deploy vLLM using Docker (pre-built images or build-from-source) with NVIDIA GPU support and run the OpenAI-compatible server.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1879,1880,1883,1884],{"name":19,"slug":20,"type":16},{"name":1881,"slug":1882,"type":16},"Docker","docker",{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16},"2026-07-18T05:47:01.814419",{"slug":1887,"name":1887,"fn":1888,"description":1889,"org":1890,"tags":1891,"stars":24,"repoUrl":25,"updatedAt":1901},"vllm-deploy-k8s","deploy vLLM to Kubernetes clusters","Deploy vLLM to Kubernetes (K8s) with GPU support, health probes, and OpenAI-compatible API endpoint. Use this skill whenever the user wants to deploy, run, or serve vLLM on a Kubernetes cluster, including creating deployments, services, checking existing deployments, or managing vLLM on K8s.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1892,1895,1896,1899,1900],{"name":1893,"slug":1894,"type":16},"AI Infrastructure","ai-infrastructure",{"name":19,"slug":20,"type":16},{"name":1897,"slug":1898,"type":16},"Kubernetes","kubernetes",{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16},"2026-07-18T05:46:58.556637",{"slug":4,"name":4,"fn":5,"description":6,"org":1903,"tags":1904,"stars":24,"repoUrl":25,"updatedAt":26},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1905,1906,1907,1908],{"name":22,"slug":23,"type":16},{"name":19,"slug":20,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16},{"slug":1910,"name":1910,"fn":1911,"description":1912,"org":1913,"tags":1914,"stars":24,"repoUrl":25,"updatedAt":1919},"vllm-prefix-cache-bench","benchmark vLLM prefix caching efficiency","This is a skill for benchmarking the efficiency of automatic prefix caching in vLLM using fixed prompts, real-world datasets, or synthetic prefix\u002Fsuffix patterns. Use when the user asks to benchmark prefix caching hit rate, caching efficiency, or repeated-prompt performance in vLLM.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1915,1916,1917,1918],{"name":1853,"slug":1854,"type":16},{"name":14,"slug":15,"type":16},{"name":1857,"slug":1858,"type":16},{"name":9,"slug":8,"type":16},"2026-07-18T05:48:07.243092",{"items":1921,"total":299},[1922,1929,1937,1944,1952,1959],{"slug":1847,"name":1847,"fn":1848,"description":1849,"org":1923,"tags":1924,"stars":24,"repoUrl":25,"updatedAt":1860},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1925,1926,1927,1928],{"name":1853,"slug":1854,"type":16},{"name":14,"slug":15,"type":16},{"name":1857,"slug":1858,"type":16},{"name":9,"slug":8,"type":16},{"slug":1862,"name":1862,"fn":1863,"description":1864,"org":1930,"tags":1931,"stars":24,"repoUrl":25,"updatedAt":1872},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1932,1933,1934,1935,1936],{"name":22,"slug":23,"type":16},{"name":1853,"slug":1854,"type":16},{"name":14,"slug":15,"type":16},{"name":1857,"slug":1858,"type":16},{"name":9,"slug":8,"type":16},{"slug":1874,"name":1874,"fn":1875,"description":1876,"org":1938,"tags":1939,"stars":24,"repoUrl":25,"updatedAt":1885},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1940,1941,1942,1943],{"name":19,"slug":20,"type":16},{"name":1881,"slug":1882,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16},{"slug":1887,"name":1887,"fn":1888,"description":1889,"org":1945,"tags":1946,"stars":24,"repoUrl":25,"updatedAt":1901},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1947,1948,1949,1950,1951],{"name":1893,"slug":1894,"type":16},{"name":19,"slug":20,"type":16},{"name":1897,"slug":1898,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16},{"slug":4,"name":4,"fn":5,"description":6,"org":1953,"tags":1954,"stars":24,"repoUrl":25,"updatedAt":26},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1955,1956,1957,1958],{"name":22,"slug":23,"type":16},{"name":19,"slug":20,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":8,"type":16},{"slug":1910,"name":1910,"fn":1911,"description":1912,"org":1960,"tags":1961,"stars":24,"repoUrl":25,"updatedAt":1919},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1962,1963,1964,1965],{"name":1853,"slug":1854,"type":16},{"name":14,"slug":15,"type":16},{"name":1857,"slug":1858,"type":16},{"name":9,"slug":8,"type":16}]