{"id":31132,"date":"2026-06-12T10:00:00","date_gmt":"2026-06-12T02:00:00","guid":{"rendered":"https:\/\/cn.hostease.com\/blog\/?p=31132"},"modified":"2026-06-15T16:16:49","modified_gmt":"2026-06-15T08:16:49","slug":"vllm-private-deployment-guide","status":"publish","type":"post","link":"https:\/\/cn.hostease.com\/blog\/guides\/vllm-private-deployment-guide\/","title":{"rendered":"vLLM \u79c1\u6709\u5316\u90e8\u7f72\u6307\u5357\uff1a\u5728\u72ec\u7acb\u670d\u52a1\u5668\u4e0a\u642d\u5efa\u9ad8\u6027\u80fd\u5927\u6a21\u578b\u63a8\u7406\u670d\u52a1"},"content":{"rendered":"

\u5982\u4f55\u5728\u81ea\u5df1\u7684 GPU \u670d\u52a1\u5668\u4e0a\u642d\u5efa\u9ad8\u6027\u80fd\u5927\u6a21\u578b\u63a8\u7406\u670d\u52a1\uff1fvLLM \u662f\u76ee\u524d\u541e\u5410\u91cf\u6700\u9ad8\u7684\u5f00\u6e90\u63a8\u7406\u5f15\u64ce\uff0c\u672c\u6587\u624b\u628a\u624b\u6559\u4f60\u4ece\u96f6\u90e8\u7f72\uff0c\u6db5\u76d6\u786c\u4ef6\u9009\u578b\u3001\u5b89\u88c5\u914d\u7f6e\u3001\u6027\u80fd\u8c03\u4f18\u5168\u6d41\u7a0b\u3002<\/p>\n

\u4e3a\u4ec0\u4e48\u9009\u62e9 vLLM \u4f5c\u4e3a\u63a8\u7406\u5f15\u64ce<\/h2>\n

vLLM \u7531 UC Berkeley \u5f00\u6e90\uff0c\u662f\u5f53\u524d\u6700\u6d41\u884c\u7684\u9ad8\u6027\u80fd\u5927\u6a21\u578b\u63a8\u7406\u5f15\u64ce\u4e4b\u4e00\u3002\u76f8\u6bd4\u76f4\u63a5\u4f7f\u7528 Hugging Face Transformers \u52a0\u8f7d\u6a21\u578b\uff0cvLLM \u6709\u4e09\u4e2a\u6838\u5fc3\u4f18\u52bf\uff1a<\/p>\n

**\u541e\u5410\u91cf\u63d0\u5347 2-4 \u500d**\uff1avLLM \u7684 PagedAttention \u6280\u672f\u5c06 KV Cache \u4ece\u8fde\u7eed\u5185\u5b58\u6539\u4e3a\u5206\u9875\u7ba1\u7406\uff0c\u6d88\u9664\u4e86\u663e\u5b58\u788e\u7247\u5316\u95ee\u9898\u3002\u76f8\u540c\u786c\u4ef6\u6761\u4ef6\u4e0b\uff0c\u6bcf\u79d2\u5904\u7406\u7684 Token \u6570\u91cf\uff08tokens\/s\uff09\u662f\u539f\u751f Transformers \u7684 2-4 \u500d\uff1b**\u663e\u5b58\u5229\u7528\u7387\u63d0\u9ad8 50% \u4ee5\u4e0a**\uff1a\u4f20\u7edf\u63a8\u7406\u6846\u67b6\u4e3a KV Cache \u9884\u5206\u914d\u8fde\u7eed\u663e\u5b58\uff0c\u5bfc\u81f4\u5927\u91cf\u7a7a\u95f4\u6d6a\u8d39\u3002PagedAttention \u7c7b\u4f3c\u64cd\u4f5c\u7cfb\u7edf\u7684\u865a\u62df\u5185\u5b58\u673a\u5236\uff0c\u6309\u9700\u5206\u914d\u7269\u7406\u9875\u9762\uff0c\u663e\u5b58\u5229\u7528\u7387\u4ece\u7ea6 40% \u63d0\u5347\u81f3 90% \u4ee5\u4e0a\uff1b**\u517c\u5bb9 OpenAI API \u683c\u5f0f**\uff1avLLM \u542f\u52a8\u540e\u4f1a\u66b4\u9732\u4e00\u4e2a\u4e0e OpenAI \u5b8c\u5168\u517c\u5bb9\u7684 REST API \u7aef\u70b9\uff0c\u4f60\u7684\u5e94\u7528\u4ee3\u7801\u53ea\u9700\u5c06 `base_url` \u6307\u5411\u81ea\u5df1\u7684\u670d\u52a1\u5668\u5373\u53ef\u65e0\u7f1d\u5207\u6362\u3002<\/p>\n

\u5982\u679c\u4f60\u6b63\u5728\u4f7f\u7528\u72ec\u7acb\u670d\u52a1\u5668<\/a>\u8fd0\u884c AI \u5e94\u7528\uff0cvLLM \u79c1\u6709\u5316\u90e8\u7f72\u80fd\u5728\u4e0d\u589e\u52a0\u786c\u4ef6\u6210\u672c\u7684\u524d\u63d0\u4e0b\uff0c\u5c06\u63a8\u7406\u541e\u5410\u91cf\u63d0\u5347\u6570\u500d\u3002<\/p>\n

\u786c\u4ef6\u8981\u6c42\u4e0e\u670d\u52a1\u5668\u9009\u578b<\/h2>\n

\u90e8\u7f72 vLLM \u524d\uff0c\u9700\u8981\u786e\u8ba4\u670d\u52a1\u5668\u7684 GPU \u914d\u7f6e\u6ee1\u8db3\u76ee\u6807\u6a21\u578b\u7684\u663e\u5b58\u9700\u6c42\u3002\u4ee5\u4e0b\u662f\u6700\u5e38\u89c1\u7684\u6a21\u578b\u4e0e\u786c\u4ef6\u5bf9\u7167\uff1a<\/p>\n

**7B \u53c2\u6570\u6a21\u578b**\uff08\u5982 Llama-3-8B\u3001Qwen2.5-7B\uff09\uff1a\u6700\u5c11\u9700\u8981 1 \u5f20 24GB \u663e\u5b58\u7684 GPU\uff08RTX 4090\u3001RTX 3090\u3001A5000\uff09\uff0cFP16 \u7cbe\u5ea6\u4e0b\u663e\u5b58\u5360\u7528\u7ea6 14GB\uff1b**13B \u53c2\u6570\u6a21\u578b**\uff08\u5982 Llama-2-13B\u3001Qwen2.5-14B\uff09\uff1a\u63a8\u8350 1 \u5f20 48GB \u663e\u5b58\u7684 GPU\uff08A6000\u3001L40\uff09\u6216 2 \u5f20 24GB GPU \u505a\u5f20\u91cf\u5e76\u884c\uff1b**70B \u53c2\u6570\u6a21\u578b**\uff08\u5982 Llama-3-70B\u3001Qwen2.5-72B\uff09\uff1a\u81f3\u5c11\u9700\u8981 2 \u5f20 80GB \u663e\u5b58\u7684 GPU\uff08A100\u3001H100\uff09\uff0c\u4f7f\u7528 4-bit \u91cf\u5316\u540e\u53ef\u5728 2 \u5f20 A100 \u4e0a\u8fd0\u884c\uff1b**MoE \u6a21\u578b**\uff08\u5982 DeepSeek-V3\u3001Qwen-MoE\uff09\uff1a\u63a8\u8350 4-8 \u5f20 A100 \u6216 H100\uff0c\u663e\u5b58\u603b\u5bb9\u91cf\u4e0d\u4f4e\u4e8e 320GB\u3002<\/p>\n

\u5982\u679c\u4f60\u7684\u6a21\u578b\u5728 13B \u4ee5\u4e0b\uff0c\u4e00\u5f20 RTX 4090\uff0824GB \u663e\u5b58\uff09\u5373\u53ef\u6ee1\u8db3\u5927\u90e8\u5206\u9700\u6c42\u3002\u5bf9\u4e8e 70B \u7ea7\u522b\u6a21\u578b\uff0c\u4f7f\u7528 AWQ \u6216 GPTQ 4-bit \u91cf\u5316\u53ef\u4ee5\u5c06\u663e\u5b58\u9700\u6c42\u964d\u4f4e\u5230\u539f\u6765\u7684 1\/4 \u5de6\u53f3\uff0c\u9009\u62e9\u5408\u9002\u7684\u670d\u52a1\u5668\u914d\u7f6e<\/a>\u53ef\u4ee5\u5927\u5e45\u964d\u4f4e\u786c\u4ef6\u95e8\u69db\u3002<\/p>\n

\u73af\u5883\u51c6\u5907\u4e0e vLLM \u5b89\u88c5<\/h2>\n

vLLM \u4f9d\u8d56 NVIDIA CUDA \u73af\u5883\uff0c\u63a8\u8350\u4f7f\u7528 Conda \u6216 Docker \u4e24\u79cd\u65b9\u5f0f\u5b89\u88c5\u3002\u4ee5\u4e0b\u662f\u57fa\u4e8e Ubuntu 22.04 + CUDA 12.x \u7684\u5b8c\u6574\u5b89\u88c5\u6b65\u9aa4\u3002<\/p>\n

\u65b9\u5f0f\u4e00\uff1aConda \u865a\u62df\u73af\u5883\u5b89\u88c5<\/h3>\n
<h2>\u521b\u5efa\u4e13\u7528\u865a\u62df\u73af\u5883<\/h2>\nconda create -n vllm python=3.11 -y\nconda activate vllm\n\n<h2>\u5b89\u88c5 vLLM\uff08\u81ea\u52a8\u5339\u914d CUDA \u7248\u672c\uff09<\/h2>\npip install vllm\n\n<h2>\u9a8c\u8bc1\u5b89\u88c5<\/h2>\npython -c "import vllm; print(vllm.__version__)"\n<\/code><\/pre>\n
\u65b9\u5f0f\u4e8c\uff1aDocker \u5bb9\u5668\u90e8\u7f72<\/h3>\n
<h2>\u62c9\u53d6\u5b98\u65b9\u955c\u50cf\uff08\u5305\u542b CUDA\u3001PyTorch\u3001vLLM \u5168\u5957\u73af\u5883\uff09<\/h2>\ndocker pull vllm\/vllm-openai:latest\n\n<h2>\u9a8c\u8bc1 GPU \u662f\u5426\u53ef\u88ab\u5bb9\u5668\u8bbf\u95ee<\/h2>\ndocker run --rm --gpus all nvidia\/cuda:12.4.0-base-ubuntu22.04 nvidia-smi\n<\/code><\/pre>\n
Docker \u65b9\u5f0f\u7684\u4f18\u52bf\u662f\u73af\u5883\u9694\u79bb\u5f7b\u5e95\uff0c\u4e0d\u4f1a\u6c61\u67d3\u5bbf\u4e3b\u673a\u7684 CUDA \u548c Python \u73af\u5883\u3002\u5982\u679c\u4f60\u5728\u4e91\u670d\u52a1\u5668\uff08\u57fa\u4e8e\u4e91\u8ba1\u7b97\u7684\u865a\u62df\u670d\u52a1\u5668\uff09<\/a>\u4e0a\u540c\u65f6\u8fd0\u884c\u5176\u4ed6 GPU \u670d\u52a1\uff08\u5982 Stable Diffusion\uff09\uff0c\u5f3a\u70c8\u5efa\u8bae\u4f7f\u7528 Docker \u65b9\u5f0f\u3002<\/p>\n
\u786e\u8ba4 CUDA \u548c GPU \u9a71\u52a8\u7248\u672c\u517c\u5bb9\u662f\u5b89\u88c5\u6210\u529f\u7684\u5173\u952e\u3002\u8fd0\u884c nvidia-smi<\/code> \u68c0\u67e5\u9a71\u52a8\u7248\u672c\uff0cvLLM \u8981\u6c42 CUDA 12.1 \u53ca\u4ee5\u4e0a\u3001NVIDIA \u9a71\u52a8 530 \u53ca\u4ee5\u4e0a\u3002<\/p>\n
\u4e0b\u8f7d\u6a21\u578b\u5e76\u542f\u52a8\u63a8\u7406\u670d\u52a1<\/h2>\n
vLLM \u652f\u6301\u4ece Hugging Face \u76f4\u63a5\u52a0\u8f7d\u6a21\u578b\uff0c\u4e5f\u652f\u6301\u52a0\u8f7d\u672c\u5730\u5df2\u4e0b\u8f7d\u7684\u6a21\u578b\u6587\u4ef6\u3002<\/p>\n
\u4ece Hugging Face \u5728\u7ebf\u52a0\u8f7d<\/h3>\n
python -m vllm.entrypoints.openai.api_server \\\n    --model Qwen\/Qwen2.5-7B-Instruct \\\n    --host 0.0.0.0 \\\n    --port 8000 \\\n    --max-model-len 8192\n<\/code><\/pre>\n
\u4f7f\u7528\u672c\u5730\u6a21\u578b\u8def\u5f84<\/h3>\n
<h2>\u5148\u4e0b\u8f7d\u6a21\u578b\u5230\u672c\u5730<\/h2>\nhuggingface-cli download Qwen\/Qwen2.5-7B-Instruct \\\n    --local-dir \/data\/models\/qwen2.5-7b-instruct\n\n<h2>\u4f7f\u7528\u672c\u5730\u8def\u5f84\u542f\u52a8<\/h2>\npython -m vllm.entrypoints.openai.api_server \\\n    --model \/data\/models\/qwen2.5-7b-instruct \\\n    --host 0.0.0.0 \\\n    --port 8000 \\\n    --max-model-len 8192\n<\/code><\/pre>\n
\u9996\u6b21\u542f\u52a8\u65f6 vLLM \u4f1a\u8fdb\u884c\u6a21\u578b\u7f16\u8bd1\u548c\u6743\u91cd\u52a0\u8f7d\uff0c7B \u6a21\u578b\u901a\u5e38\u9700\u8981 30-60 \u79d2\u3002\u542f\u52a8\u6210\u529f\u540e\u7ec8\u7aef\u4f1a\u663e\u793a\u7c7b\u4f3c\u4ee5\u4e0b\u7684\u65e5\u5fd7\uff1a<\/p>\n
INFO:     Started server process [12345]\nINFO:     Waiting for application startup.\nINFO:     Application startup complete.\nINFO:     Uvicorn running on http:\/\/0.0.0.0:8000\n<\/code><\/pre>\n
\u9a8c\u8bc1 API \u670d\u52a1<\/h3>\n
curl http:\/\/localhost:8000\/v1\/chat\/completions \\\n    -H "Content-Type: application\/json" \\\n    -d '{\n        "model": "Qwen\/Qwen2.5-7B-Instruct",\n        "messages": [\n            {"role": "user", "content": "\u4f60\u597d\uff0c\u8bf7\u4ecb\u7ecd\u4e00\u4e0b\u4f60\u81ea\u5df1"}\n        ],\n        "max_tokens": 256\n    }'\n<\/code><\/pre>\n
\u8fd4\u56de JSON \u4e2d\u5305\u542b choices[0].message.content<\/code> \u5b57\u6bb5\u5373\u8868\u793a\u670d\u52a1\u6b63\u5e38\u8fd0\u884c\u3002<\/p>\n
PagedAttention \u5185\u5b58\u4f18\u5316\u8be6\u89e3<\/h2>\n
PagedAttention \u662f vLLM \u7684\u6838\u5fc3\u6280\u672f\u3002\u4f20\u7edf\u63a8\u7406\u6846\u67b6\u7684\u505a\u6cd5\u662f\u4e3a\u6bcf\u4e2a\u8bf7\u6c42\u9884\u5206\u914d\u4e00\u6bb5\u8fde\u7eed\u663e\u5b58\u6765\u5b58\u50a8 KV Cache\uff0c\u8fd9\u5bfc\u81f4\u4e24\u4e2a\u95ee\u9898\uff1a\u4e00\u662f\u6bcf\u4e2a\u8bf7\u6c42\u5206\u914d\u7684\u6700\u5927\u957f\u5ea6\u5fc5\u987b\u6309\u6700\u574f\u60c5\u51b5\u4f30\u7b97\uff0c\u5b9e\u9645\u4f7f\u7528\u65f6\u5927\u91cf\u663e\u5b58\u7a7a\u95f2\uff1b\u4e8c\u662f\u4e0d\u540c\u8bf7\u6c42\u4e4b\u95f4\u7684\u663e\u5b58\u65e0\u6cd5\u5171\u4eab\uff0c\u5373\u4f7f\u5b83\u4eec\u5904\u7406\u7684\u662f\u76f8\u540c\u7684 Prompt \u524d\u7f00\u3002<\/p>\n
PagedAttention \u5c06 KV Cache \u5206\u5272\u6210\u56fa\u5b9a\u5927\u5c0f\u7684”\u9875\u9762”\uff08\u9ed8\u8ba4\u6bcf\u9875 16 \u4e2a Token\uff09\uff0c\u6309\u9700\u52a8\u6001\u5206\u914d\u3002\u5f53\u8bf7\u6c42\u751f\u6210\u5b8c\u4e00\u4e2a Token \u540e\u624d\u5206\u914d\u4e0b\u4e00\u4e2a\u9875\u9762\uff0c\u751f\u6210\u7ed3\u675f\u540e\u7acb\u5373\u91ca\u653e\u3002\u8fd9\u4f7f\u5f97\u663e\u5b58\u5229\u7528\u7387\u4ece\u4f20\u7edf\u65b9\u6848\u7684 40% \u5de6\u53f3\u63d0\u5347\u5230 90% \u4ee5\u4e0a\u3002<\/p>\n
\u5b9e\u9645\u6548\u679c\u4f53\u73b0\u5728\u4e24\u4e2a\u65b9\u9762\uff1a<\/p>\n
**\u5e76\u53d1\u5904\u7406\u80fd\u529b\u63d0\u5347**\uff1a\u5728 24GB \u663e\u5b58\u7684 RTX 4090 \u4e0a\uff0c\u8fd0\u884c Qwen2.5-7B-Instruct\uff0c\u4f20\u7edf\u65b9\u6848\u6700\u591a\u540c\u65f6\u5904\u7406 8 \u4e2a\u8bf7\u6c42\uff0cvLLM \u53ef\u4ee5\u540c\u65f6\u5904\u7406 24-32 \u4e2a\u8bf7\u6c42\uff1b**\u957f\u6587\u672c\u5904\u7406\u66f4\u7a33\u5b9a**\uff1a\u5904\u7406 4096 Token \u4ee5\u4e0a\u7684\u957f\u6587\u672c\u65f6\uff0c\u4f20\u7edf\u65b9\u6848\u5bb9\u6613\u51fa\u73b0 OOM\uff08\u663e\u5b58\u4e0d\u8db3\uff09\u9519\u8bef\uff0cvLLM \u901a\u8fc7\u52a8\u6001\u9875\u9762\u5206\u914d\u53ef\u4ee5\u7a33\u5b9a\u5904\u7406\u3002<\/p>\n
\u901a\u8fc7\u8c03\u6574 --gpu-memory-utilization<\/code> \u53c2\u6570\uff08\u9ed8\u8ba4 0.9\uff09\uff0c\u4f60\u53ef\u4ee5\u63a7\u5236 vLLM \u4f7f\u7528\u591a\u5c11\u6bd4\u4f8b\u7684 GPU \u663e\u5b58\u3002\u5efa\u8bae\u4fdd\u6301\u9ed8\u8ba4\u503c\uff0c\u9884\u7559 10% \u663e\u5b58\u7ed9 CUDA \u5185\u6838\u548c\u7cfb\u7edf\u8fdb\u7a0b\u3002<\/p>\n
\u91cf\u5316\u52a0\u901f\uff1a\u7528 4-bit \u6a21\u578b\u964d\u4f4e\u663e\u5b58\u95e8\u69db<\/h2>\n
\u5f53\u670d\u52a1\u5668\u663e\u5b58\u4e0d\u8db3\u4ee5\u8fd0\u884c\u5b8c\u6574\u7cbe\u5ea6\u6a21\u578b\u65f6\uff0c\u91cf\u5316\u662f\u6700\u76f4\u63a5\u7684\u89e3\u51b3\u65b9\u6848\u3002vLLM \u539f\u751f\u652f\u6301 AWQ \u548c GPTQ \u4e24\u79cd\u91cf\u5316\u683c\u5f0f\uff0c\u53ef\u4ee5\u5c06\u6a21\u578b\u663e\u5b58\u5360\u7528\u964d\u4f4e 60-75%\u3002<\/p>\n
\u4ee5 Qwen2.5-7B \u4e3a\u4f8b\uff0c\u4e0d\u540c\u7cbe\u5ea6\u7684\u663e\u5b58\u5360\u7528\u5bf9\u6bd4\uff1a<\/p>\n
**FP16**\uff08\u9ed8\u8ba4\u7cbe\u5ea6\uff09\uff1a\u7ea6 14GB \u663e\u5b58\uff0c\u63a8\u7406\u7cbe\u5ea6\u6700\u9ad8\uff1b**INT8**\uff088-bit \u91cf\u5316\uff09\uff1a\u7ea6 8GB \u663e\u5b58\uff0c\u7cbe\u5ea6\u635f\u5931\u6781\u5c0f\uff1b**INT4**\uff084-bit \u91cf\u5316\uff09\uff1a\u7ea6 4.5GB \u663e\u5b58\uff0c\u7cbe\u5ea6\u635f\u5931\u53ef\u63a5\u53d7\u3002<\/p>\n
\u4f7f\u7528\u91cf\u5316\u6a21\u578b\u53ea\u9700\u5728\u542f\u52a8\u65f6\u6307\u5b9a\u91cf\u5316\u7248\u672c\u7684\u6a21\u578b\u8def\u5f84\uff1a<\/p>\n
<h2>\u4f7f\u7528 AWQ \u91cf\u5316\u7248\u672c<\/h2>\npython -m vllm.entrypoints.openai.api_server \\\n    --model Qwen\/Qwen2.5-7B-Instruct-AWQ \\\n    --host 0.0.0.0 \\\n    --port 8000 \\\n    --max-model-len 8192\n\n<h2>\u4f7f\u7528 GPTQ \u91cf\u5316\u7248\u672c<\/h2>\npython -m vllm.entrypoints.openai.api_server \\\n    --model Qwen\/Qwen2.5-7B-Instruct-GPTQ-Int4 \\\n    --host 0.0.0.0 \\\n    --port 8000 \\\n    --quantization gptq \\\n    --max-model-len 8192\n<\/code><\/pre>\n
\u9009\u62e9 AWQ \u8fd8\u662f GPTQ \u53d6\u51b3\u4e8e\u4f60\u627e\u5230\u7684\u91cf\u5316\u6a21\u578b\u8d44\u6e90\u3002\u793e\u533a\u666e\u904d\u53cd\u9988 AWQ \u5728 vLLM \u4e0a\u7684\u63a8\u7406\u901f\u5ea6\u7565\u5feb\u4e8e GPTQ\uff0c\u4f46\u5dee\u8ddd\u5728 5% \u4ee5\u5185\u3002\u4f18\u5148\u9009\u62e9\u4e0b\u8f7d\u91cf\u5927\u3001\u793e\u533a\u8bc4\u4ef7\u597d\u7684\u91cf\u5316\u7248\u672c\u3002<\/p>\n
\u5bf9\u4e8e 70B \u7ea7\u522b\u6a21\u578b\uff0c4-bit \u91cf\u5316\u662f\u79c1\u6709\u5316\u90e8\u7f72\u7684\u5173\u952e\u2014\u2014\u5b83\u5c06\u663e\u5b58\u9700\u6c42\u4ece 140GB \u964d\u81f3 35GB\uff0c\u4f7f\u5f97 2 \u5f20 A100\uff0880GB\uff09\u5373\u53ef\u8fd0\u884c\u3002<\/p>\n
\u751f\u4ea7\u73af\u5883\u914d\u7f6e\u4e0e\u6027\u80fd\u8c03\u4f18<\/h2>\n
\u5c06 vLLM \u4ece\u5f00\u53d1\u6d4b\u8bd5\u63a8\u5411\u751f\u4ea7\u73af\u5883\uff0c\u9700\u8981\u5173\u6ce8\u4ee5\u4e0b\u51e0\u4e2a\u5173\u952e\u53c2\u6570\u3002<\/p>\n
\u6838\u5fc3\u542f\u52a8\u53c2\u6570<\/h3>\n
python -m vllm.entrypoints.openai.api_server \\\n    --model \/data\/models\/qwen2.5-7b-instruct \\\n    --host 0.0.0.0 \\\n    --port 8000 \\\n    --max-model-len 8192 \\\n    --tensor-parallel-size 1 \\\n    --gpu-memory-utilization 0.90 \\\n    --max-num-seqs 64 \\\n    --max-num-batched-tokens 8192 \\\n    --disable-log-requests \\\n    --served-model-name qwen2.5-7b\n<\/code><\/pre>\n
\u5404\u53c2\u6570\u7684\u542b\u4e49\uff1a<\/p>\n
`–tensor-parallel-size`\uff1a\u5f20\u91cf\u5e76\u884c\u7684 GPU \u6570\u91cf\uff0c\u5355\u5361\u90e8\u7f72\u8bbe\u4e3a 1\uff0c\u591a\u5361\u90e8\u7f72\u8bbe\u4e3a GPU \u6570\u91cf\uff1b`–max-num-seqs`\uff1a\u6700\u5927\u5e76\u53d1\u8bf7\u6c42\u6570\uff0c\u6839\u636e\u663e\u5b58\u5927\u5c0f\u8c03\u6574\uff0c24GB \u663e\u5b58\u5efa\u8bae 32-64\uff1b`–max-num-batched-tokens`\uff1a\u5355\u6b21\u6279\u5904\u7406\u7684\u6700\u5927 Token \u6570\uff0c\u5f71\u54cd\u541e\u5410\u91cf\u4e0a\u9650\uff1b`–served-model-name`\uff1aAPI \u4e2d\u66b4\u9732\u7684\u6a21\u578b\u540d\u79f0\uff0c\u5ba2\u6237\u7aef\u8bf7\u6c42\u65f6\u4f7f\u7528\u6b64\u540d\u79f0\u3002<\/p>\n
\u6dfb\u52a0 API \u5bc6\u94a5\u8ba4\u8bc1<\/h3>\n
\u751f\u4ea7\u73af\u5883\u4e0d\u5e94\u66b4\u9732\u65e0\u8ba4\u8bc1\u7684 API \u7aef\u70b9\u3002vLLM \u652f\u6301\u901a\u8fc7\u73af\u5883\u53d8\u91cf\u8bbe\u7f6e API \u5bc6\u94a5\uff1a<\/p>\n
export VLLM_API_KEY=your-secret-api-key-here\n\npython -m vllm.entrypoints.openai.api_server \\\n    --model \/data\/models\/qwen2.5-7b-instruct \\\n    --api-key $VLLM_API_KEY \\\n    --host 0.0.0.0 \\\n    --port 8000\n<\/code><\/pre>\n
\u5ba2\u6237\u7aef\u8bf7\u6c42\u65f6\u9700\u5728 Header \u4e2d\u643a\u5e26 Authorization: Bearer your-secret-api-key-here<\/code>\u3002<\/p>\n
\u4f7f\u7528 systemd \u7ba1\u7406\u670d\u52a1<\/h3>\n
\u5c06 vLLM \u6ce8\u518c\u4e3a systemd \u670d\u52a1\uff0c\u786e\u4fdd\u670d\u52a1\u5668\u91cd\u542f\u540e\u81ea\u52a8\u6062\u590d\uff1a<\/p>\n
<h2>\/etc\/systemd\/system\/vllm.service<\/h2>\n[Unit]\nDescription=vLLM Inference Server\nAfter=network.target\n\n[Service]\nType=simple\nUser=vllm\nWorkingDirectory=\/data\/models\nEnvironment="CUDA_VISIBLE_DEVICES=0"\nEnvironment="VLLM_API_KEY=your-secret-key"\nExecStart=\/home\/vllm\/miniconda3\/envs\/vllm\/bin\/python -m vllm.entrypoints.openai.api_server \\\n    --model \/data\/models\/qwen2.5-7b-instruct \\\n    --api-key ${VLLM_API_KEY} \\\n    --host 0.0.0.0 \\\n    --port 8000 \\\n    --max-model-len 8192 \\\n    --gpu-memory-utilization 0.90\nRestart=on-failure\nRestartSec=10\n\n[Install]\nWantedBy=multi-user.target\n<\/code><\/pre>\n
<h2>\u542f\u7528\u5e76\u542f\u52a8\u670d\u52a1<\/h2>\nsudo systemctl daemon-reload\nsudo systemctl enable vllm\nsudo systemctl start vllm\n\n<h2>\u67e5\u770b\u8fd0\u884c\u72b6\u6001<\/h2>\nsudo systemctl status vllm\nsudo journalctl -u vllm -f\n<\/code><\/pre>\n
\u53cd\u5411\u4ee3\u7406\u4e0e HTTPS \u914d\u7f6e<\/h2>\n
\u5982\u679c vLLM \u90e8\u7f72\u5728\u516c\u7f51\u53ef\u8bbf\u95ee\u7684\u670d\u52a1\u5668\u4e0a\uff0c\u5efa\u8bae\u5728\u524d\u7aef\u52a0\u4e00\u5c42 Nginx \u53cd\u5411\u4ee3\u7406\uff0c\u7528\u4e8e HTTPS \u7ec8\u6b62\u3001\u8bf7\u6c42\u9650\u6d41\u548c\u65e5\u5fd7\u8bb0\u5f55\u3002\u5982\u679c\u4f60\u4f7f\u7528\u7684\u662f Hostease \u6258\u7ba1\u65b9\u6848<\/a>\uff0c\u53ef\u4ee5\u914d\u5408\u5176\u63d0\u4f9b\u7684\u514d\u8d39 SSL\uff08Secure Sockets Layer\uff0c\u5b89\u5168\u5957\u63a5\u5c42\uff09 \u8bc1\u4e66\u5feb\u901f\u5b8c\u6210\u914d\u7f6e\uff1a<\/p>\n
<h2>\/etc\/nginx\/sites-available\/vllm-api<\/h2>\nserver {\n    listen 443 ssl;\n    server_name api.yourdomain.com;\n\n    ssl_certificate \/etc\/letsencrypt\/live\/api.yourdomain.com\/fullchain.pem;\n    ssl_certificate_key \/etc\/letsencrypt\/live\/api.yourdomain.com\/privkey.pem;\n\n    location \/v1\/ {\n        proxy_pass http:\/\/127.0.0.1:8000;\n        proxy_set_header Host $host;\n        proxy_set_header X-Real-IP $remote_addr;\n        proxy_read_timeout 300s;\n        proxy_buffering off;\n\n        # \u9650\u6d41\uff1a\u6bcf\u79d2\u6700\u591a 10 \u4e2a\u8bf7\u6c42\n        limit_req zone=api burst=20 nodelay;\n    }\n}\n<\/code><\/pre>\n
proxy_buffering off<\/code> \u662f\u5173\u952e\u914d\u7f6e\u2014\u2014\u5927\u6a21\u578b\u63a8\u7406\u7684\u6d41\u5f0f\u54cd\u5e94\uff08streaming\uff09\u9700\u8981\u5b9e\u65f6\u8f6c\u53d1\u6570\u636e\uff0c\u5982\u679c Nginx \u5f00\u542f\u7f13\u51b2\u4f1a\u5bfc\u81f4\u5ba2\u6237\u7aef\u6536\u4e0d\u5230\u4e2d\u95f4\u7684 Token \u8f93\u51fa\u3002<\/p>\n
\u5e38\u89c1\u95ee\u9898\u6392\u67e5<\/h2>\n
\u90e8\u7f72\u8fc7\u7a0b\u4e2d\u53ef\u80fd\u9047\u5230\u7684\u5178\u578b\u95ee\u9898\u53ca\u89e3\u51b3\u65b9\u6848\uff1a<\/p>\n
**CUDA out of memory**\uff1a\u964d\u4f4e `–max-model-len` \u6216 `–max-num-seqs`\uff0c\u6216\u4f7f\u7528\u91cf\u5316\u6a21\u578b\u51cf\u5c11\u663e\u5b58\u5360\u7528\uff1b**\u6a21\u578b\u52a0\u8f7d\u7f13\u6162**\uff1a\u9996\u6b21\u542f\u52a8\u9700\u8981\u7f16\u8bd1 CUDA \u5185\u6838\uff0c\u540e\u7eed\u542f\u52a8\u4f1a\u4f7f\u7528\u7f13\u5b58\u3002\u786e\u4fdd `~\/.cache\/vllm` \u76ee\u5f55\u6709\u8db3\u591f\u7a7a\u95f4\uff1b**\u63a8\u7406\u901f\u5ea6\u6162**\uff1a\u68c0\u67e5\u662f\u5426\u542f\u7528\u4e86 `–enforce-eager`\uff08\u7981\u7528 CUDA Graph\uff09\uff0c\u5982\u679c\u6709\u5219\u79fb\u9664\u6b64\u53c2\u6570\u4ee5\u542f\u7528 CUDA Graph \u52a0\u901f\uff1b**\u8fde\u63a5\u88ab\u62d2\u7edd**\uff1a\u68c0\u67e5\u9632\u706b\u5899\u89c4\u5219\u662f\u5426\u653e\u884c\u4e86 8000 \u7aef\u53e3\uff0c\u4ee5\u53ca `–host` \u662f\u5426\u8bbe\u7f6e\u4e3a `0.0.0.0`\u3002<\/p>\n
\u603b\u7ed3\u4e0e\u884c\u52a8\u5efa\u8bae<\/h2>\n
\u63a8\u8350\u4ece vLLM \u5b98\u65b9 Docker \u955c\u50cf\u5f00\u59cb\u90e8\u7f72\uff0c\u914d\u5408 4-bit \u91cf\u5316\u53ef\u5728\u6d88\u8d39\u7ea7 GPU \u4e0a\u8fd0\u884c 7B \u6a21\u578b\u3002\u751f\u4ea7\u73af\u5883\u52a1\u5fc5\u914d\u7f6e HTTPS\u3001API \u5bc6\u94a5\u8ba4\u8bc1\u548c systemd \u670d\u52a1\u3002\u5982\u679c\u4f60\u9700\u8981\u4e00\u53f0\u914d\u7f6e GPU \u7684[\u72ec\u7acb\u670d\u52a1\u5668](https:\/\/cn.hostease.com\/dedicated-server\/)\u6765\u8fd0\u884c vLLM\uff0c\u53ef\u4ee5\u67e5\u770b Hostease \u72ec\u7acb\u670d\u52a1\u5668\u65b9\u6848<\/a>\uff0c\u652f\u6301\u81ea\u5b9a\u4e49 GPU \u914d\u7f6e\u3002\u66f4\u591a\u670d\u52a1\u5668\u8fd0\u7ef4\u6280\u5de7\u8bf7\u53c2\u8003 \u670d\u52a1\u5668\u8fd0\u7ef4\u6307\u5357<\/a> \u548c \u4e3b\u673a\u9009\u8d2d\u6307\u5357<\/a>\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"
\u624b\u628a\u624b\u6559\u4f60vLLM\u79c1\u6709\u5316\u90e8\u7f72\uff0c\u4ece\u786c\u4ef6\u9009\u578b\u5230\u751f\u4ea7\u914d\u7f6e\u5168\u6d41\u7a0b\uff0c\u5305\u542b\u91cf\u5316\u52a0\u901f\u548cNginx\u53cd\u4ee3\u914d\u7f6e\u3002<\/p>\n","protected":false},"author":2,"featured_media":31174,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[3822],"tags":[597,4399,3976],"class_list":["post-31132","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-guides","tag-gpu","tag-vllm","tag-category"],"aioseo_notices":[],"jetpack_featured_media_url":"https:\/\/cn.hostease.com\/blog\/wp-content\/uploads\/2026\/06\/31132_vllm-private-deployment-guide-cover.webp","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/cn.hostease.com\/blog\/wp-json\/wp\/v2\/posts\/31132","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cn.hostease.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cn.hostease.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cn.hostease.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/cn.hostease.com\/blog\/wp-json\/wp\/v2\/comments?post=31132"}],"version-history":[{"count":1,"href":"https:\/\/cn.hostease.com\/blog\/wp-json\/wp\/v2\/posts\/31132\/revisions"}],"predecessor-version":[{"id":31154,"href":"https:\/\/cn.hostease.com\/blog\/wp-json\/wp\/v2\/posts\/31132\/revisions\/31154"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cn.hostease.com\/blog\/wp-json\/wp\/v2\/media\/31174"}],"wp:attachment":[{"href":"https:\/\/cn.hostease.com\/blog\/wp-json\/wp\/v2\/media?parent=31132"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cn.hostease.com\/blog\/wp-json\/wp\/v2\/categories?post=31132"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cn.hostease.com\/blog\/wp-json\/wp\/v2\/tags?post=31132"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}