Zoom RTMS 手动 WebSocket 协议实现指南不依赖 SDK 构建实时媒体流服务【免费下载链接】knowledge-work-pluginsOpen source repository of plugins primarily intended for knowledge workers to use in Claude Cowork项目地址: https://gitcode.com/GitHub_Trending/kn/knowledge-work-plugins导读Zoom Realtime Media StreamsRTMS是 Zoom 提供的实时媒体接入服务让后端应用无需机器人参会即可直接通过 WebSocket 接收会议、网络研讨会Webinar与 Video SDK 会话中的音频、视频、屏幕共享、实时转写Transcript和聊天数据。本指南以partner-built/zoom-plugin/skills/rtms/examples/manual-websocket.md为骨架完整讲解不借助zoom/rtmsSDK、从零实现 RTMS 双通道 WebSocket 协议的步骤HMAC-SHA256 签名、Webhook 处理、信令通道握手、媒体通道参数协商、心跳保活、媒体数据处理、断线重连与优雅关闭。读完本文你将具备在任意语言或自定义协议约束下独立实现 RTMS 客户端的完整能力。适用场景为什么需要手动实现 WebSocketRTMS 官方推荐使用 SDKzoom/rtms它封装了握手、心跳、重连等全部复杂度。但以下场景必须走手动实现路线语言没有官方 SDK 支持官方 SDK 目前仅支持部分平台如 darwin-arm64 与 linux-x64Windows、Intel Mac、linux-arm64 等平台需回退到手动实现见 troubleshooting/common-issues.md自定义协议需求需要精确控制每个消息帧、媒体参数或接入私有传输层学习底层协议理解 RTMS 的信令/媒体分离设计与消息语义为基于 SDK 的开发排障打下基础。SDK 与手动实现的核心差异在 SKILL.md 中有明确对照SDK 自动处理 WebSocket 复杂度与自动重连手动实现则拥有全部协议控制权、代码量更大。RTMS 双通道架构信令 WebSocket 与媒体 WebSocketRTMS 协议要求建立两条 WebSocket 连接将控制面与数据面彻底分离这一设计在 connection-architecture.md 中有完整架构图连接职责消息流Signaling WebSocket信令通道控制面认证握手、媒体服务器发现、流启停、心跳、事件通知握手请求(1)/响应(2)、客户端就绪(7)、心跳(12/13)、事件更新(6)Media WebSocket媒体通道数据面音频、视频、转写、聊天、屏幕共享数据媒体握手请求(3)/响应(4)、媒体数据(14-18)、心跳(12/13)两条连接的设计带来四个好处关注点分离控制逻辑不干扰媒体流、独立伸缩信令与媒体服务器可分别扩容、故障隔离媒体重连无需重新认证、以及支持 Split 模式每种媒体类型一条独立连接。Split 模式推荐用于大多数场景Unified 模式单条媒体连接承载所有媒体类型则适用于对音画同步有强需求的实时混流场景。连接流程全景来自 lifecycle-flow.md会议/Webinar/Video SDK 会话启动 ↓ Zoom 发送 webhook 事件meeting/webinar/session.rtms_started ↓ 后端接收 webhook → 立即返回 HTTP 200 ↓ 连接信令 WS → 发送握手请求(msg_type 1) ↓ 收到握手响应(msg_type 2) → 提取 media_server.server_urls.all ↓ 连接媒体 WS → 发送媒体握手请求(msg_type 3) ↓ 收到媒体握手响应(msg_type 4) → 向信令发送 CLIENT_READY_ACK(msg_type 7) ↓ 接收媒体数据音频14/视频15/共享16/转写17/聊天18 ↓ 响应心跳(12→13) ↓ 收到 .rtms_stopped → 关闭两条连接并清理注意会议与 Webinar 的 Webhook 负载都使用meeting_uuid不是webinar_uuidVideo SDK 使用session_id。一旦连接建立三条产品线的信令/媒体协议完全相同仅初始 Webhook 事件名与负载 ID 字段不同。完整实现可运行的 Node.js 手动 WebSocket 客户端以下代码来自 manual-websocket.md依赖ws、crypto与express三个 npm 包是完整可运行的服务端骨架。下文将分模块逐段剖析。1. 环境准备与连接状态管理const WebSocket require(ws); const crypto require(crypto); const express require(express); const app express(); app.use(express.json()); // Configuration const CLIENT_ID process.env.ZOOM_CLIENT_ID; const CLIENT_SECRET process.env.ZOOM_CLIENT_SECRET; const SECRET_TOKEN process.env.ZOOM_SECRET_TOKEN; // Active connections const signalingConnections new Map(); const mediaConnections new Map(); const activeSessions new Map(); const activeVideoUsers new Map();三个环境变量与 SDK 版环境变量见 SKILL.md对应ZOOM_CLIENT_IDOAuth Client ID 或 SDK Key、ZOOM_CLIENT_SECRETOAuth Client Secret 或 SDK Secret、ZOOM_SECRET_TOKENWebhook 验证令牌用于处理endpoint.url_validation挑战。四个 Map 分别维护信令连接、媒体连接、活动会话与开启摄像头的参与者集合——会话跟踪是防止重复连接的关键详见后文错误处理与重连。2. 签名生成HMAC-SHA256// // SIGNATURE GENERATION // Uses meeting_uuid for meetings/webinars, session_id for Video SDK // function generateSignature(clientId, idValue, streamId, clientSecret) { const message ${clientId},${idValue},${streamId}; return crypto.createHmac(sha256, clientSecret) .update(message) .digest(hex); }签名规则HMAC-SHA256(clientSecret, clientId,idValue,streamId)输出hex不是 base64。idValue从 Webhook 负载中提取// meeting_uuid for meetings/webinars, session_id for Video SDK const idValue payload.meeting_uuid || payload.session_id;信令握手与媒体握手都需要该签名。签名错误时握手响应会返回状态码3STATUS_INVALID_SIGNATURE。排查清单来自 troubleshooting/common-issues.md确认使用正确的 clientId不是 App 名称、正确的 clientSecret、消息中无多余空格、输出为 hex。3. Webhook 处理器先回 200再处理业务// // WEBHOOK HANDLER // const RTMS_EVENTS [meeting.rtms_started, webinar.rtms_started, session.rtms_started]; const RTMS_STOP_EVENTS [meeting.rtms_stopped, webinar.rtms_stopped, session.rtms_stopped]; app.post(/webhook, (req, res) { // CRITICAL: Respond 200 IMMEDIATELY before any processing! res.status(200).send(); const { event, payload } req.body; // Handle URL validation challenge if (event endpoint.url_validation) { const hash crypto .createHmac(sha256, SECRET_TOKEN) .update(payload.plainToken) .digest(hex); return res.json({ plainToken: payload.plainToken, encryptedToken: hash }); } // Handle RTMS events (meetings, webinars, and Video SDK) if (RTMS_EVENTS.includes(event)) { handleRTMSStarted(payload.object); } else if (RTMS_STOP_EVENTS.includes(event)) { handleRTMSStopped(payload.object); } });这是整个实现中最重要的约束必须先执行res.status(200).send()再处理业务逻辑。若处理耗时过长Zoom 会判定失败并重发 Webhook重试会建立第二条连接把第一条连接踢下线每个流只允许 1 条连接。正确写法是先返回 200 再异步处理如setImmediate见 webhooks.md。4. RTMS 启动处理器与重复连接防护function handleRTMSStarted(payload) { const { rtms_stream_id, server_urls } payload; // meeting_uuid for meetings/webinars, session_id for Video SDK const idValue payload.meeting_uuid || payload.session_id; // Prevent duplicate connections if (activeSessions.has(rtms_stream_id)) { console.log(Already connected to this stream, ignoring); return; } activeSessions.set(rtms_stream_id, { idValue: idValue, startTime: Date.now() }); connectToSignaling(idValue, rtms_stream_id, server_urls); }activeSessions充当去重闸门同样的rtms_stream_id只会建立一次连接。这与 Webhook 响应时机问题见 troubleshooting/common-issues.md 的Webhook Response Timing小节一起构成重复连接故障的两大根因。5. 信令 WebSocket握手与消息分发function connectToSignaling(idValue, streamId, serverUrl) { console.log(Connecting to signaling:, serverUrl); const signature generateSignature(CLIENT_ID, idValue, streamId, CLIENT_SECRET); const ws new WebSocket(serverUrl); signalingConnections.set(streamId, ws); ws.on(open, () { console.log(Signaling connected, sending handshake); ws.send(JSON.stringify({ msg_type: 1, // SIGNALING_HAND_SHAKE_REQ protocol_version: 1, meeting_uuid: idValue, // Works for both meeting_uuid and session_id rtms_stream_id: streamId, sequence: Math.floor(Math.random() * 1000000), signature: signature, media_type: 9 // AUDIO(1) | TRANSCRIPT(8) })); }); ws.on(message, (data) { const msg JSON.parse(data.toString()); handleSignalingMessage(msg, idValue, streamId); }); ws.on(close, (code, reason) { console.log(Signaling closed:, code, reason.toString()); signalingConnections.delete(streamId); // Implement reconnection logic if needed }); ws.on(error, (error) { console.error(Signaling error:, error); }); }media_type是位掩码Audio1、Video2、Screen Share4、Transcript8、Chat16、All32完整定义见>function handleSignalingMessage(msg, idValue, streamId) { switch (msg.msg_type) { case 2: // SIGNALING_HAND_SHAKE_RESP if (msg.status_code 0) { console.log(Signaling handshake success); // Extract media server URL and connect const mediaUrl msg.media_server.server_urls.all; connectToMedia(idValue, streamId, mediaUrl); } else { console.error(Signaling handshake failed:, msg.status_code); } break; case 6: // EVENT_UPDATE handleEventUpdate(msg, streamId); break; case 8: // STREAM_STATE_UPDATE console.log(Stream state:, msg.state); break; case 9: // SESSION_STATE_UPDATE console.log(Session state:, msg.state); break; case 12: // KEEP_ALIVE_REQ const signalingWs signalingConnections.get(streamId); if (signalingWs) { signalingWs.send(JSON.stringify({ msg_type: 13, // KEEP_ALIVE_RESP timestamp: msg.timestamp })); } break; } }握手响应msg_type 2中携带media_server.server_urls.all这是媒体服务器的连接地址。STREAM_STATE_UPDATE8与SESSION_STATE_UPDATE9携带的state枚举定义见>function handleEventUpdate(msg, streamId) { const eventType msg.event?.event_type ?? msg.event_type; const participants msg.event?.participants ?? []; switch (eventType) { case 2: // ACTIVE_SPEAKER_CHANGE console.log(Active speaker:, msg.user_name); break; case 3: // PARTICIPANT_JOIN console.log(Participant joined:, msg.user_name); break; case 4: // PARTICIPANT_LEAVE console.log(Participant left:, msg.user_name); break; case 5: // SHARING_START console.log(Sharing started by:, msg.user_name); break; case 6: // SHARING_STOP console.log(Sharing stopped); break; case 8: // PARTICIPANT_VIDEO_ON for (const participant of participants) { const set activeVideoUsers.get(streamId) || new Set(); set.add(participant.user_id); activeVideoUsers.set(streamId, set); } break; case 9: // PARTICIPANT_VIDEO_OFF for (const participant of participants) { activeVideoUsers.get(streamId)?.delete(participant.user_id); } break; } }事件类型完整枚举见>function connectToMedia(idValue, streamId, mediaUrl) { console.log(Connecting to media:, mediaUrl); const signature generateSignature(CLIENT_ID, idValue, streamId, CLIENT_SECRET); const ws new WebSocket(mediaUrl); mediaConnections.set(streamId, ws); ws.on(open, () { console.log(Media connected, sending handshake); ws.send(JSON.stringify({ msg_type: 3, // DATA_HAND_SHAKE_REQ protocol_version: 1, meeting_uuid: idValue, // Works for both meeting_uuid and session_id rtms_stream_id: streamId, signature: signature, media_type: 9, // AUDIO(1) | TRANSCRIPT(8) payload_encryption: false, media_params: { audio: { content_type: 2, // RAW_AUDIO sample_rate: 1, // 16kHz channel: 1, // Mono codec: 1, // L16 (PCM) data_opt: 1, // Mixed stream send_rate: 20 // 20ms chunks }, transcript: { content_type: 5, // TEXT src_language: 9, // English enable_lid: false // Fixed language, no auto-switch } } })); }); ws.on(message, (data) { const msg JSON.parse(data.toString()); handleMediaMessage(msg, streamId); }); ws.on(close, (code, reason) { console.log(Media closed:, code, reason.toString()); mediaConnections.delete(streamId); }); ws.on(error, (error) { console.error(Media error:, error); }); }媒体通道的握手请求携带完整的media_params其取值映射全部在>function handleMediaMessage(msg, streamId) { switch (msg.msg_type) { case 4: // DATA_HAND_SHAKE_RESP if (msg.status_code 0) { console.log(Media handshake success, sending client ready); // Tell signaling were ready to receive const signalingWs signalingConnections.get(streamId); if (signalingWs) { signalingWs.send(JSON.stringify({ msg_type: 7, // CLIENT_READY_ACK rtms_stream_id: streamId })); } } else { console.error(Media handshake failed:, msg.status_code); } break; case 12: // KEEP_ALIVE_REQ const mediaWs mediaConnections.get(streamId); if (mediaWs) { mediaWs.send(JSON.stringify({ msg_type: 13, // KEEP_ALIVE_RESP timestamp: msg.timestamp })); } break; case 14: // MEDIA_DATA_AUDIO handleAudioData(msg); break; case 15: // MEDIA_DATA_VIDEO handleVideoData(msg); break; case 16: // MEDIA_DATA_SHARE handleShareData(msg); break; case 17: // MEDIA_DATA_TRANSCRIPT handleTranscriptData(msg); break; case 18: // MEDIA_DATA_CHAT handleChatData(msg); break; } }关键握手时序媒体握手成功msg_type 4、status_code 0后必须向信令通道发送msg_type: 7CLIENT_READY_ACK通知服务端客户端已就绪随后媒体数据才开始下发。若跳过此步骤将收不到任何媒体数据。8. 媒体数据处理函数function handleAudioData(msg) { const audioBuffer Buffer.from(msg.content, base64); console.log(Audio: ${audioBuffer.length} bytes from ${msg.user_name || mixed}); // Process audio: // - Send to transcription service // - Save to file // - Stream to output } function handleVideoData(msg) { const videoBuffer Buffer.from(msg.content, base64); console.log(Video: ${videoBuffer.length} bytes from ${msg.user_name}); // Process video: // - Decode H.264/JPG // - Save frames // - AI analysis } function handleShareData(msg) { const shareBuffer Buffer.from(msg.content, base64); console.log(Share: ${shareBuffer.length} bytes from ${msg.user_name}); } function handleTranscriptData(msg) { console.log([${msg.user_name}]: ${msg.content}); // Save transcript, process with AI, etc. } function handleChatData(msg) { console.log([Chat] ${msg.user_name}: ${msg.content}); }媒体数据统一以base64 编码的content字段下发需用Buffer.from(msg.content, base64)还原二进制。各媒体类型的格式约定见 media-types.md音频PCM 16-bit 采样L16可直接送转写服务或落盘视频H.264 NAL 单元fps5或 JPG/PNG 帧fps≤5可交由 FFmpeg 解码或抽帧做 AI 视觉分析屏幕共享与普通视频是独立消息类型msg_type 16 vs 15且media_type位不同4 vs 2必须单独订阅适合幻灯片场景使用低 FPS 的 JPG如 fps1转写JSON 文本字段结构包含user_id、user_name、text、timestamp、is_finalis_finaltrue表示已定稿的片段聊天JSON 文本。9. RTMS 停止处理器与资源清理function handleRTMSStopped(payload) { const streamId payload.rtms_stream_id; console.log(RTMS stopped:, streamId); // Close connections const signalingWs signalingConnections.get(streamId); const mediaWs mediaConnections.get(streamId); if (signalingWs) signalingWs.close(); if (mediaWs) mediaWs.close(); // Cleanup signalingConnections.delete(streamId); mediaConnections.delete(streamId); activeSessions.delete(streamId); }rtms_stopped类事件触发后应关闭两条连接并从所有 Map 中清理。停止原因枚举RTMS_STOP_REASON完整列表见>const PORT process.env.PORT || 3000; app.listen(PORT, () { console.log(RTMS server running on port ${PORT}); });协议消息类型速查表信令通道消息完整枚举见>{ content_type: 2, // 1RTP, 2RAW_AUDIO sample_rate: 1, // 08kHz, 116kHz, 232kHz, 348kHz channel: 1, // 1Mono, 2Stereo (OPUS only) codec: 1, // 1L16, 2G.711, 3G.722, 4OPUS data_opt: 1, // 1Mixed, 2Multi-streams send_rate: 20 // Chunk size in ms (multiple of 20) }两个重要的工程细节Stereo双声道仅在与 OPUS 编解码器组合时支持L16/PCM 下只能用 Monodata_opt: 1混合流下音频消息不含说话人user_id元数据。若需逐说话人区分改用data_opt: 2Multi-streams或使用ACTIVE_SPEAKER_CHANGE事件事件类型 2做发言人跟踪见 troubleshooting/common-issues.md 的Audio Metadata Missing userId小节。视频参数{ content_type: 3, // 3RAW_VIDEO codec: 7, // 5JPG, 6PNG, 7H.264 resolution: 2, // 1SD, 2HD, 3FHD, 4QHD fps: 25, // 1-30 (JPG/PNG max 5) data_opt: 3 // 3Single active speaker }编码与 FPS 的对应规则见 media-types.mdfps ≤ 5 用 JPG/PNGfps 5 用 H.264。分辨率枚举SD854x480 或 640x360、HD1280x720、FHD1920x1080、QHD2560x1440。屏幕共享参数{ content_type: 3, // 3RAW_VIDEO codec: 5, // 5JPG, 6PNG, 7H.264 resolution: 3, // 1SD, 2HD, 3FHD, 4QHD fps: 1 // 1-30 (JPG/PNG max 1) }静态幻灯片场景推荐 JPG fps1低带宽高清晰度动态演示可提高到 15-30fps 并用 H.264。注意屏幕共享的 JPG/PNG 最大 FPS 上限是 1区别于视频的 5。转写参数{ content_type: 5, // 5TEXT src_language: 9, // 9English enable_lid: false // Fixed language, no auto-switch }enable_lidLanguage Identification控制语言自动识别true或省略时 RTMS 可在转写中自动切换语言可能导致启动阶段转写不稳定、有可感知的延迟见 troubleshooting/common-issues.md 的Transcript Language Delay小节false时锁定src_language固定语言下游转写处理更可预期。状态码速查完整 44 项见>function subscribeToParticipantVideo(streamId, userId) { const signalingWs signalingConnections.get(streamId); if (!signalingWs) return; signalingWs.send(JSON.stringify({ msg_type: 28, // VIDEO_SUBSCRIPTION_REQ user_id: userId, subscribe: true, timestamp: Date.now() })); }主动关闭流function closeStream(streamId) { const signalingWs signalingConnections.get(streamId); if (!signalingWs) return; signalingWs.send(JSON.stringify({ msg_type: 21, // STREAM_CLOSE_REQ rtms_stream_id: streamId })); }STREAM_CLOSE_REQ需在信令通道发送预期先收到STREAM_CLOSE_RESP确认随后是正常的 socket 拆除。完整订阅/选择流程打开VIDEO_SINGLE_INDIVIDUAL_STREAM媒体 socket → 订阅事件 → 选user_id→ 发送订阅请求 → 等待响应见 connection.md 的Single Individual Video Subscription Flow小节。错误处理指数退避重连RTMS不会自动重连重连是客户端自己的责任。断线后应使用指数退避策略// Implement exponential backoff for reconnection let retryDelay 1000; ws.on(close, (code, reason) { console.log(Connection closed:, code, reason); // Dont reconnect if intentionally closed if (code 1000) return; setTimeout(() { reconnect(); }, retryDelay); retryDelay Math.min(retryDelay * 2, 30000); }); ws.on(error, (error) { console.error(WebSocket error:, error); // Connection will close, triggering reconnection });重连窗口约束信令通道约 60 秒、媒体通道约 65 秒媒体保活容忍已提升。正常关闭码 1000 不重连。此外来自 troubleshooting/common-issues.md不要缓存 Webhook 中的服务器 URL可能过期应使用新鲜的 Webhook 负载防火墙需放行 WSS443 端口。实战技巧音频录制断隙填充真实网络环境下音频分片可能丢帧或迟到直接拼接会产生播放卡顿。用静音帧填充 ≥500ms 的间隙可实现连续播放function handleAudioData(msg, streamId) { const now msg.timestamp; const last lastTimestamps.get(streamId) || now; const gap now - last; // Fill gaps 500ms with silence if (gap 500) { const silentFrames Math.floor(gap / 20); console.log(Filling ${silentFrames} silent frames); for (let i 0; i silentFrames; i) { const silentFrame Buffer.alloc(640); // 20ms 16kHz mono writeToFile(silentFrame); } } lastTimestamps.set(streamId, now); const audioBuffer Buffer.from(msg.content, base64); writeToFile(audioBuffer); }640 字节 20ms × 16kHz × 16bitL16 单声道每秒 32000 字节 ÷ 50 帧 640 字节/帧与媒体握手中的send_rate: 20、sample_rate: 1、channel: 1、codec: 1参数严格对应。常见故障排查速览症状可能原因解决方案连接失败签名无效状态码 3核对 clientId/clientSecret/hex 输出见签名章节随机断线、重复连接Webhook 响应过慢触发 Zoom 重试先res.status(200).send()再异步处理收不到数据media_type位掩码错误如9音频转写、32全部约 60 秒被断未响应心跳收到 msg_type 12 立即回 msg_type 13无视频数据个人视频模式下未发送订阅请求订阅PARTICIPANT_VIDEO_ON/OFF并发送VIDEO_SUBSCRIPTION_REQ转写启动慢LID 自动语言识别生效src_languageenable_lid: false收不到屏幕共享共享与视频是独立类型media_type需包含 4DESKSHARE旧 Node.js 段错误版本低于 20.3.0升级 Node.js 至 20.3.024 LTS 推荐下一步SDK 快速入门SDK 自动处理上述全部复杂度是大多数场景的首选AI 集成转写与智能分析的落地模式数据类型参考全部枚举与常量消息类型、事件类型、状态码、媒体参数媒体类型参考各媒体格式与参数配置连接架构双通道设计原理与地区路由生命周期流程从 Webhook 到媒体流的完整时序常见问题连接与数据问题的诊断清单5 分钟运行手册上线前的预检清单。掌握上述协议细节后你可以在任意语言中复刻这套双通道实现也能在基于 SDK 的开发中精准定位握手、心跳、媒体参数协商等环节的问题。【免费下载链接】knowledge-work-pluginsOpen source repository of plugins primarily intended for knowledge workers to use in Claude Cowork项目地址: https://gitcode.com/GitHub_Trending/kn/knowledge-work-plugins创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考