作者吴业亮博客wuyeliang.blog.csdn.net1. 目标本文档用于在三台 GPU 主机上部署一套基于 vLLM 的分布式推理实验环境并结合 Mooncake 实现 PD 分离。部署目标包括1 台 Prefill 节点2 台 Decode 节点1 个 Mooncake Master1 个统一 Proxy 入口1 个 Web Dashboard2. 拓扑规划主机角色如下179.20.4.21Mooncake MastervLLM PrefillProxyDashboard179.20.4.22vLLM Decode 1179.20.4.23vLLM Decode 2端口规划如下50051Mooncake Master8010Prefill8020Decode8000Proxy8088Dashboard3. 环境要求最低要求Ubuntu 22.04Python 3.10NVIDIA GPU已安装 NVIDIA 驱动三台机器之间网络互通root 权限本次实验环境每台机器 1 张NVIDIA GeForce RTX 3080驱动版本示例570.86.104. 软件版本约定建议固定如下版本vllm0.11.0torch2.8.0torchvision0.23.0torchaudio2.8.0mooncake-transfer-engine说明该版本组合用于规避较新torch/vllm与当前驱动不兼容的问题如果你的驱动、CUDA 环境不同请先单机验证兼容性5. 模型约定实验模型Qwen/Qwen2.5-1.5B-Instruct本手册推荐使用“本地模型目录”启动方式而不是每次启动都直接从远端仓库读取模型元数据。原因可减少镜像站接口限流影响更利于复现更适合写成稳定部署手册本地模型目录约定/opt/models/Qwen2.5-1.5B-Instruct6. 目录约定部署目录应用目录/opt/vllm-pd/app虚拟环境/opt/vllm-pd/venv日志目录/var/log/vllm-pd模型目录/opt/models临时部署包目录/root/vllm-pd-bundle7. 网络与防火墙要求部署前需确认三台机器之间可互相访问50051、8010、8020外部访问179.20.4.21时可访问8000、8088建议先在机器内网或公网环境下确认ping179.20.4.21ping179.20.4.22ping179.20.4.23部署成功后建议重点检查179.20.4.21:50051179.20.4.21:8010179.20.4.21:8000179.20.4.21:8088179.20.4.22:8020179.20.4.23:80208. 国内源建议APT 源建议https://mirrors.tuna.tsinghua.edu.cn/ubuntu/PyPI 源建议https://pypi.tuna.tsinghua.edu.cn/simpleHugging Face 镜像建议https://hf-mirror.com注意PyTorch 的部分 CUDA wheel 仍可能需要走官方 wheel 索引模型镜像站可能出现429限流因此手册推荐模型提前下载到本地目录9. 部署文件建议准备以下文件deploy_bundle/install_stack.shdeploy_bundle/mooncake_connector_proxy.pydeploy_bundle/dashboard.pydeploy_bundle/requirements.txt打包命令示例tar-czfdeploy_bundle.tgz-Cdeploy_bundle.上传到远端示例scpdeploy_bundle.tgz root179.20.4.21:/root/deploy_bundle.tgzscpdeploy_bundle.tgz root179.20.4.22:/root/deploy_bundle.tgzscpdeploy_bundle.tgz root179.20.4.23:/root/deploy_bundle.tgz10. 三台主机部署步骤10.1179.20.4.21部署 Prefill / Proxy / Dashboardrm-rf/root/vllm-pd-bundlemkdir-p/root/vllm-pd-bundletar-xzf/root/deploy_bundle.tgz-C/root/vllm-pd-bundlebash/root/vllm-pd-bundle/install_stack.sh\prefill\179.20.4.21\Qwen/Qwen2.5-1.5B-Instruct\179.20.4.21\179.20.4.22\179.20.4.2310.2179.20.4.22部署 Decode 1rm-rf/root/vllm-pd-bundlemkdir-p/root/vllm-pd-bundletar-xzf/root/deploy_bundle.tgz-C/root/vllm-pd-bundlebash/root/vllm-pd-bundle/install_stack.sh\decode\179.20.4.22\Qwen/Qwen2.5-1.5B-Instruct\179.20.4.21\179.20.4.22\179.20.4.2310.3179.20.4.23部署 Decode 2rm-rf/root/vllm-pd-bundlemkdir-p/root/vllm-pd-bundletar-xzf/root/deploy_bundle.tgz-C/root/vllm-pd-bundlebash/root/vllm-pd-bundle/install_stack.sh\decode\179.20.4.23\Qwen/Qwen2.5-1.5B-Instruct\179.20.4.21\179.20.4.22\179.20.4.2311. systemd 服务清单预期服务mooncake-master.servicevllm-prefill.servicevllm-decode.servicevllm-proxy.servicevllm-dashboard.service查看状态systemctl is-active mooncake-master.service systemctl is-active vllm-prefill.service systemctl is-active vllm-decode.service systemctl is-active vllm-proxy.service systemctl is-active vllm-dashboard.service查看详细状态systemctl --no-pager--fullstatus mooncake-master.service systemctl --no-pager--fullstatus vllm-prefill.service systemctl --no-pager--fullstatus vllm-decode.service systemctl --no-pager--fullstatus vllm-proxy.service systemctl --no-pager--fullstatus vllm-dashboard.service12. 部署成功后的监听端口179.20.4.21预期监听50051801080008088检查命令ss-ltnp|egrep:(50051|8010|8000|8088)\b179.20.4.22预期监听8020检查命令ss-ltnp|egrep:(8020)\b179.20.4.23预期监听8020检查命令ss-ltnp|egrep:(8020)\b13. 健康检查179.20.4.21curlhttp://127.0.0.1:8010/healthcurlhttp://127.0.0.1:8000/healthzcurlhttp://127.0.0.1:8088/api/status179.20.4.22curlhttp://127.0.0.1:8020/health179.20.4.23curlhttp://127.0.0.1:8020/health14. 对外访问地址部署成功后对外主要使用Proxyhttp://179.20.4.21:8000Dashboardhttp://179.20.4.21:8088OpenAI 兼容聊天接口http://179.20.4.21:8000/v1/chat/completions15. 请求验证示例curlhttp://179.20.4.21:8000/v1/chat/completions\-HContent-Type: application/json\-HAuthorization: Bearer EMPTY\-d{ model: Qwen/Qwen2.5-1.5B-Instruct, messages: [ {role: user, content: 请用一句话介绍 Mooncake。} ], temperature: 0.2, max_tokens: 64, stream: false }16. 常见问题16.18088打不开先看systemctl is-active vllm-dashboard.service ss-ltnp|grep8088如果没有监听不是防火墙问题而是 Dashboard 没起来16.28000打不开先看systemctl is-active vllm-proxy.service ss-ltnp|grep8000如果 Proxy 已启动但/healthz返回ready:false通常说明Prefill 还没准备好Decode 还没准备好16.38010/8020起不来重点看日志tail-n100/var/log/vllm-pd/vllm-prefill.logtail-n100/var/log/vllm-pd/vllm-decode.log常见原因torch / vllm / CUDA / 驱动不兼容模型远端仓库接口限流模型本地目录不完整16.4 Hugging Face 镜像返回429这是已知风险。建议处理方式不依赖运行时在线列目录先把模型文件落到本地目录再从本地目录启动 vLLM16.5.22主机安装时被unattended-upgrade干扰确认方式pgrep-afunattended-upgrlsof/var/lib/dpkg/lock-frontend dpkg--configure-a不要只凭进程存在就判断系统仍然被锁。17. 日志路径关键日志位置/root/reinstall21.log/root/reinstall22.log/root/reinstall23.log/root/recover21.log/root/recover22.log/root/recover23.log/var/log/vllm-pd/mooncake-master.log/var/log/vllm-pd/vllm-prefill.log/var/log/vllm-pd/vllm-decode.log/var/log/vllm-pd/vllm-proxy.log/var/log/vllm-pd/vllm-dashboard.log18. 回滚与重试停止服务systemctl stop mooncake-master.service systemctl stop vllm-prefill.service systemctl stop vllm-decode.service systemctl stop vllm-proxy.service systemctl stop vllm-dashboard.service清理应用目录前请谨慎确认/opt/vllm-pd/opt/models/Qwen2.5-1.5B-Instruct如果只是重启服务优先不要删除模型目录以免重复下载大文件。重启服务systemctl restart mooncake-master.service systemctl restart vllm-prefill.service systemctl restart vllm-decode.service systemctl restart vllm-proxy.service systemctl restart vllm-dashboard.service19. 验收标准部署完成后至少满足以下条件179.20.4.21上50051/8010/8000/8088均已监听179.20.4.22上8020已监听179.20.4.23上8020已监听http://179.20.4.21:8000/healthz返回ready:truehttp://179.20.4.21:8088/api/status可返回集群状态通过 Proxy 的 OpenAI 兼容接口可以返回模型响应20. 备注本文档基于当前这套三机实验环境编写重点是让部署流程有明确步骤让排障路径足够清晰为后续整理成“最终成功复现版”提供基础如果后续要扩展为正式生产部署手册建议再补充模型文件校验systemd 单元参数解释防火墙与安全组配置项资源监控与日志采集升级和回滚策略