Zoom RTMS Samples Repository

https://github.com/zoom/rtms-samples
Admin
A collection of sample projects demonstrating how to work with Zoom's Realtime Media Streams (RTMS)...
Tokens:123,999
Snippets:465
Trust Score:8.2
License:MIT
Update:8 months ago
Show doc for...
Tokens:
Raw
### Go Example: Handling Media Messages and Starting Stream

Source: https://github.com/zoom/rtms-samples/blob/main/RTMS_CONNECTION_FLOW.md

Example Go code demonstrating how to handle incoming media messages, specifically the media handshake response, and send the start stream command to the signaling socket.

```APIDOC
## Go Example: Start Streaming

### Description
This Go function `handleMediaMessage` processes incoming data from the media connection. Upon successfully unmarshalling a media handshake response (`msg_type: 4`, `status_code: 0`), it sends a start stream command (`msg_type: 7`) to the signaling connection.

### Method
N/A (Server-side Go function)

### Endpoint
N/A (Interacts with WebSocket connections)

### Parameters
#### Function Parameters
- **mediaConn** (*websocket.Conn) - Pointer to the WebSocket connection object for media.
- **signalingConn** (*websocket.Conn) - Pointer to the WebSocket connection object for signaling.
- **data** ([]byte) - The raw byte data received from the media connection.
- **streamID** (string) - The ID of the RTMS stream.

#### Request Body (for signalingConn.WriteJSON)
```json
{
  "msg_type": 7,
  "rtms_stream_id": "<stream_id>"
}
```

### Response
#### Success Response
- No explicit success response is documented, but the command is sent.

#### Response Example (for signalingConn.WriteJSON)
```json
{
  "msg_type": 7,
  "rtms_stream_id": "your_stream_id_here"
}
```
```

--------------------------------

### Start Application (npm start)

Source: https://github.com/zoom/rtms-samples/blob/main/boilerplate/working_sdk/README.md

Starts the RTMS developer preview application using the npm start script defined in the package.json file.

```bash
npm start
```

--------------------------------

### Install Dependencies and Configure Environment

Source: https://github.com/zoom/rtms-samples/blob/main/audio/send_audio_to_deepgram_transcribe_service_js/README.md

Provides the necessary bash commands to install project dependencies using npm and to set up the environment variables by copying an example configuration file.

```bash
npm install
cp .env.example .env   # Fill in your credentials
node index.js
```

--------------------------------

### Install Dependencies and Run Application

Source: https://github.com/zoom/rtms-samples/blob/main/zoom_apps/ai_industry_specific_notetaker_js/readme.md

Installs project dependencies using npm and starts the Node.js application. Ensure you have Node.js installed. The app will then be accessible via specified local URLs.

```bash
npm install
node index.js
```

--------------------------------

### Python Example: Handling Media Messages and Starting Stream

Source: https://github.com/zoom/rtms-samples/blob/main/RTMS_CONNECTION_FLOW.md

Example Python code demonstrating how to handle incoming media messages, specifically the media handshake response, and send the start stream command to the signaling socket.

```APIDOC
## Python Example: Start Streaming

### Description
This Python function `handle_media_message` processes messages received from the media WebSocket. Upon receiving a successful media handshake response (`msg_type: 4`, `status_code: 0`), it sends a start stream command (`msg_type: 7`) to the signaling WebSocket.

### Method
N/A (Server-side Python function)

### Endpoint
N/A (Interacts with WebSocket connections)

### Parameters
#### Function Parameters
- **msg** (dict) - The incoming message dictionary from the media WebSocket.
- **media_ws** (websocket object) - The WebSocket connection object for media.
- **signaling_ws** (websocket object) - The WebSocket connection object for signaling.
- **stream_id** (str) - The ID of the RTMS stream.

#### Request Body (for signaling_ws.send)
```json
{
  "msg_type": 7,
  "rtms_stream_id": "<stream_id>"
}
```

### Response
#### Success Response (Print Statement)
- Prints 'Media streaming started' to the console upon successful command.

#### Response Example (for signaling_ws.send)
```json
{
  "msg_type": 7,
  "rtms_stream_id": "your_stream_id_here"
}
```
```

--------------------------------

### Run Server using Gunicorn

Source: https://github.com/zoom/rtms-samples/blob/main/boilerplate/working_python_wss/README.md

Starts the Python web application using the Gunicorn WSGI HTTP Server. This command binds the application to port 3000, making it accessible for receiving webhook events.

```bash
gunicorn index:app --bind 0.0.0.0:3000
```

--------------------------------

### JavaScript Example: Handling Media Messages and Starting Stream

Source: https://github.com/zoom/rtms-samples/blob/main/RTMS_CONNECTION_FLOW.md

Example JavaScript code demonstrating how to handle incoming media messages, specifically the media handshake response, and send the start stream command to the signaling socket.

```APIDOC
## JavaScript Example: Start Streaming

### Description
This JavaScript function `handleMediaMessage` processes messages received from the media WebSocket. Upon receiving a successful media handshake response (`msg_type: 4`, `status_code: 0`), it sends a start stream command (`msg_type: 7`) to the signaling WebSocket.

### Method
N/A (Client-side JavaScript function)

### Endpoint
N/A (Interacts with WebSocket connections)

### Parameters
#### Function Parameters
- **msg** (object) - The incoming message object from the media WebSocket.
- **mediaWs** (WebSocket) - The WebSocket connection object for media.
- **signalingWs** (WebSocket) - The WebSocket connection object for signaling.
- **streamId** (string) - The ID of the RTMS stream.

#### Request Body (for signalingWs.send)
```json
{
  "msg_type": 7,
  "rtms_stream_id": "<stream_id>"
}
```

### Response
#### Success Response (Console Log)
- Logs 'Media streaming started' to the console upon successful command.

#### Response Example (for signalingWs.send)
```json
{
  "msg_type": 7,
  "rtms_stream_id": "your_stream_id_here"
}
```
```

--------------------------------

### Copy Environment Example

Source: https://github.com/zoom/rtms-samples/blob/main/boilerplate/working_sdk/README.md

Copies the example environment file (.env.example) to a new file named .env. This is a common practice for configuring applications with sensitive credentials.

```bash
cp .env.example .env
```

--------------------------------

### Start ZoomOpenAPI Tool Server (npm)

Source: https://github.com/zoom/rtms-samples/blob/main/rtms_mcp_client/README.md

This command sequence initiates the ZoomOpenAPI tool server. It requires navigating to the 'tools-zoom-openapi--server' directory and running 'npm start'. This server should be started before the LLM Router and MCP client.

```bash
cd tools-zoom-openapi--server
npm start
```

--------------------------------

### Install Go Dependencies

Source: https://github.com/zoom/rtms-samples/blob/main/boilerplate/working_go/README.md

Installs necessary Go packages for the RTMS Media Receiver example. Requires Go 1.18+ and a Go module system.

```bash
go get github.com/gorilla/websocket github.com/joho/godotenv

go mod tidy
```

--------------------------------

### Install GStreamer and Dependencies (Ubuntu)

Source: https://github.com/zoom/rtms-samples/blob/main/streaming/stream_to_aws_kinesis_passthru_js/readme.md

Installs GStreamer and its essential plugins on Ubuntu systems, along with build tools required for compiling the AWS KVS GStreamer plugin. This setup is critical for media processing.

```bash
sudo apt update
sudo apt install -y \
  git cmake build-essential libssl-dev libcurl4-openssl-dev \
  liblog4cplus-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev \
  gstreamer1.0-tools gstreamer1.0-plugins-base gstreamer1.0-plugins-good \
  gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav
```

--------------------------------

### Start MCP Client (npm)

Source: https://github.com/zoom/rtms-samples/blob/main/rtms_mcp_client/README.md

This command initiates the MCP client. Navigate to the 'mcp-client' directory and run 'npm start'. This is the final service to be started and requires both the ZoomOpenAPI tool server and LLM Router to be running.

```bash
cd mcp-client
npm start
```

--------------------------------

### Install Frontend Dependencies

Source: https://github.com/zoom/rtms-samples/blob/main/zoom_apps/prompt_for_user_consent_js/README.md

Installs Node.js dependencies for the frontend service. This command should be run from the 'frontend' directory.

```bash
cd frontend && npm install
```

--------------------------------

### Install Backend Dependencies

Source: https://github.com/zoom/rtms-samples/blob/main/zoom_apps/prompt_for_user_consent_js/README.md

Installs Node.js dependencies for the backend service. This command should be run from the 'backend' directory.

```bash
cd backend && npm install
```

--------------------------------

### Start LLM Router Server (npm)

Source: https://github.com/zoom/rtms-samples/blob/main/rtms_mcp_client/README.md

This command starts the LLM Router server. Navigate to the 'llm-router-server' directory and execute 'npm start'. This service depends on the ZoomOpenAPI tool server being active.

```bash
cd llm-router-server
npm start
```

--------------------------------

### Run Zoom RTMS Media Receiver Server (Go)

Source: https://github.com/zoom/rtms-samples/blob/main/boilerplate/working_go/README.md

Starts the Go server for the Zoom RTMS Media Receiver. This command assumes `main.go` contains the server logic and dependencies are installed.

```bash
go run main.go
```

--------------------------------

### Install Node.js Dependencies

Source: https://github.com/zoom/rtms-samples/blob/main/audio/send_audio_to_whisper_local_transcribe_service_js/README.md

Installs project dependencies using npm. This command should be run in the project's root directory.

```bash
npm install
```

--------------------------------

### Start Local Frontend Development Server

Source: https://github.com/zoom/rtms-samples/blob/main/zoom_apps/prompt_for_user_consent_js/README.md

Starts the frontend development server using npm. This command should be run from the 'frontend' directory.

```bash
cd frontend && npm start
```

--------------------------------

### Start Local Backend Development Server

Source: https://github.com/zoom/rtms-samples/blob/main/zoom_apps/prompt_for_user_consent_js/README.md

Starts the backend development server using npm. This command should be run from the 'backend' directory.

```bash
cd backend && npm run dev
```

--------------------------------

### Build Project with Maven

Source: https://github.com/zoom/rtms-samples/blob/main/boilerplate/working_java/README.md

Command to clean and install project dependencies using Maven.

```bash
mvn clean install
```

--------------------------------

### Create Environment File (Bash)

Source: https://github.com/zoom/rtms-samples/blob/main/zoom_apps/ai_transcript_analysis_js/readme.md

Copies the example environment file to create a new `.env` file for storing sensitive credentials and configuration. Users must then populate this file with their specific API keys and tokens.

```bash
cp .env.example .env
```

--------------------------------

### Install RTMS SDK Dependencies

Source: https://github.com/zoom/rtms-samples/blob/main/boilerplate/working_sdk/README.md

Installs the Node.js dependencies for the RTMS SDK after cloning it. This command should be run from within the cloned rtms directory.

```bash
cd rtms
npm install
```

--------------------------------

### Configure Environment Variables

Source: https://github.com/zoom/rtms-samples/blob/main/audio/send_audio_to_whisper_local_transcribe_service_js/README.md

Copies the example environment file to `.env`. This file should then be populated with necessary credentials and configuration settings.

```bash
cp .env.example .env
```

--------------------------------

### Start Chroma Tool Server

Source: https://github.com/zoom/rtms-samples/blob/main/rtms_mcp_client/README.md

This command navigates to the Chroma tool server directory and starts the server using npm. This server exposes ChromaDB-backed tools for adding documents and performing semantic searches via MCP.

```bash
cd tools-chroma-server
npm start
```

--------------------------------

### Install SDK from GitHub (npm)

Source: https://github.com/zoom/rtms-samples/blob/main/boilerplate/working_sdk/README.md

Installs the Zoom RTMS SDK directly from its GitHub repository using npm. This is the standard method for adding the SDK to a project.

```bash
npm install github:zoom/rtms
```

--------------------------------

### Start and Send Data to KVS Stream

Source: https://github.com/zoom/rtms-samples/blob/main/streaming/stream_to_aws_kinesis_passthru_js/readme-forputmedia.md

Example usage of the kvs_putmedia_producer module to start the streaming process and periodically send audio and video buffers. Includes handling for SIGINT to stop the stream.

```javascript
const { startStream, stopStream, sendAudioBuffer, sendVideoBuffer } = require('./kvs_putmedia_producer');

(async () => {
  await startStream();

  // Example: send audio & video buffers periodically
  setInterval(() => {
    sendAudioBuffer(getPcmAudioChunk());
    sendVideoBuffer(getH264VideoChunk());
  }, 40); // ~25fps

  process.on('SIGINT', stopStream);
})();
```

--------------------------------

### Configure Environment Variables

Source: https://github.com/zoom/rtms-samples/blob/main/boilerplate/working_java/README.md

Example .env file for configuring Zoom and server settings. Sensitive information like client secrets should be kept secure.

```properties
# Zoom Configuration
ZOOM_CLIENT_ID=your_client_id
ZOOM_CLIENT_SECRET=your_client_secret
ZOOM_SECRET_TOKEN=your_secret_token

# Server Configuration
SERVER_PORT=5050

# Application Configuration
APP_WEBHOOK_PATH=/webhook
APP_MODE=webhook
```

--------------------------------

### Clone RTMS Developer Preview App

Source: https://github.com/zoom/rtms-samples/blob/main/boilerplate/working_sdk/README.md

Clones the RTMS developer preview JavaScript application from GitHub. This is the first step in the manual setup process.

```bash
git clone https://github.com/zoom/rtms-developer-preview-js
```

--------------------------------

### Navigate to App Root Directory

Source: https://github.com/zoom/rtms-samples/blob/main/boilerplate/working_sdk/README.md

Navigates back to the root directory of the developer preview application after performing SDK setup steps. This is part of the manual workaround.

```bash
cd ../../../
```

--------------------------------

### Install Node Dependencies

Source: https://github.com/zoom/rtms-samples/blob/main/streaming/stream_to_aws_kinesis_passthru_js/readme.md

Installs the necessary Node.js packages for the project using npm. This is a standard step for any Node.js application.

```bash
npm install
```

--------------------------------

### Start Node.js Application

Source: https://github.com/zoom/rtms-samples/blob/main/audio/send_audio_to_deepgram_transcribe_service_js/readme.md

This command initiates the Express server for the Zoom RTMS and Deepgram integration. It assumes Node.js is installed and dependencies are managed via npm or yarn.

```bash
node index.js 

```

--------------------------------

### FFmpeg Installation Commands

Source: https://github.com/zoom/rtms-samples/blob/main/streaming/stream_audio_and_video_to_youtube_greedy_gap_filler_js/readme.md

Provides commands for installing FFmpeg on different operating systems. FFmpeg is crucial for processing and streaming the captured audio and video data to YouTube.

```bash
# macOS
brew install ffmpeg
```

```bash
# Ubuntu/Debian
sudo apt-get install ffmpeg
```

```bash
# Windows
# Download from https://ffmpeg.org/download.html
```

--------------------------------

### Configure Video Parameters (JavaScript)

Source: https://github.com/zoom/rtms-samples/blob/main/boilerplate/working_sdk/README.md

Sets video parameters for the RTMS client using the JavaScript SDK. This example configures raw video with H.264 codec, HD resolution, and 30 FPS.

```javascript
client.setVideoParams({
  contentType: rtms.VideoContentType.RAW_VIDEO,
  codec: rtms.VideoCodec.H264,
  resolution: rtms.VideoResolution.HD,
  dataOpt: rtms.VideoDataOption.VIDEO_MIXED_GALLERY_VIEW,
  fps: 30
});
```

--------------------------------

### Start Server with Python

Source: https://github.com/zoom/rtms-samples/blob/main/video-sdk/vsdk_working_python/README.md

Starts the Python-based HTTP server on port 3000. This server handles incoming webhook events and WebSocket connections from the Zoom Video SDK.

```python
python3 index:app --bind 0.0.0.0:3000
```

--------------------------------

### Configure Audio Parameters (JavaScript)

Source: https://github.com/zoom/rtms-samples/blob/main/boilerplate/working_sdk/README.md

Sets audio parameters for the RTMS client using the JavaScript SDK. This example configures raw audio with Opus codec, 16kHz sample rate, and mono channel.

```javascript
client.setAudioParameters({
  contentType: rtms.AudioContentType.RAW_AUDIO,
  codec: rtms.AudioCodec.OPUS,
  sampleRate: 16000,
  channel: rtms.AudioChannel.MONO,
  dataOpt: rtms.AudioDataOption.AUDIO_MIXED_STREAM,
  duration: 20,
  frameSize: 320
});
```

--------------------------------

### Index.py - Main Entry Point

Source: https://github.com/zoom/rtms-samples/blob/main/video-sdk/vsdk_working_python/README.md

The main Python file that orchestrates the Zoom Video SDK sample. It handles webhook events, WebSocket connections, media processing, and client broadcasting.

```python
# index.py
# Main entry point that handles webhook events, WebSocket connections, and media processing
```

--------------------------------

### RTMS Connection Flow Examples (Go)

Source: https://github.com/zoom/rtms-samples/blob/main/ARCHITECTURE.md

Demonstrates the complete RTMS protocol specification for establishing and managing WebSocket connections, including manual handshake, authentication, and custom reconnection strategies. This provides a low-level approach for maximum control.

```go
// Example for Native WebSocket implementation (refer to RTMS_CONNECTION_FLOW.md for full details)
package main

import (
	"fmt"
	"log"
	"github.com/gorilla/websocket"
)

func main() {
	conn, _, err := websocket.DefaultDialer.Dial("wss://your-rtms-server.com", nil)
	if err != nil {
		log.Fatal("dial error:", err)
	}
defer conn.Close()

	// Send handshake
	message := []byte(`{"msg_type": 1}`)
	err = conn.WriteMessage(websocket.TextMessage, message)
	if err != nil {
		log.Println("write error:", err)
		return
	}

	for {
		_, message, err := conn.ReadMessage()
		if err != nil {
			log.Println("read error:", err)
			// Implement custom reconnection logic here
			return
		}
		fmt.Printf("Received: %s\n", message)
		// Handle different message types
	}
}
```

--------------------------------

### Quick Start with FastAPI (Python)

Source: https://github.com/zoom/rtms-samples/blob/main/library/python/README.md

Illustrates a FastAPI application setup using RTMSManager for handling media streams. It initializes RTMSManager during application startup, registers event listeners for audio and transcript data, and sets up the webhook integration.

```python
import os
from fastapi import FastAPI
from library.python.rtms_manager import RTMSManager, MediaType
from library.python.webhook_manager import WebhookManager

app = FastAPI()
rtms = None

@app.on_event("startup")
async def startup():
    global rtms
    rtms = await RTMSManager.init({
        'credentials': {
            'meeting': {
                'client_id': os.environ['ZOOM_CLIENT_ID'],
                'client_secret': os.environ['ZOOM_CLIENT_SECRET'],
                'secret_token': os.environ['ZOOM_SECRET_TOKEN'],
            }
        },
        'media_types': MediaType.AUDIO | MediaType.TRANSCRIPT,
    })
    
    rtms.on('audio', lambda data: print(f"Audio: {len(data['buffer'])} bytes"))
    rtms.on('transcript', lambda data: print(f"{data['user_name']}: {data['text']}"))
    
    webhook = WebhookManager(
        webhook_path='/webhook',
        zoom_secret_token=os.environ['ZOOM_SECRET_TOKEN']
    )
    webhook.setup_fastapi(app, rtms)

@app.on_event("shutdown")
async def shutdown():
    if rtms:
        await rtms.stop()
```

--------------------------------

### Quick Start with Flask (Python)

Source: https://github.com/zoom/rtms-samples/blob/main/library/python/README.md

Demonstrates a basic Flask application setup using RTMSManager to handle audio and transcript media streams. It initializes RTMSManager with credentials and media types, sets up event listeners for audio, transcript, and errors, and configures a webhook for validation.

```python
import os
import asyncio
from flask import Flask
from library.python.rtms_manager import RTMSManager, MediaType
from library.python.webhook_manager import WebhookManager

app = Flask(__name__)
rtms = None

async def init_rtms():
    global rtms
    rtms = await RTMSManager.init({
        'credentials': {
            'meeting': {
                'client_id': os.environ['ZOOM_CLIENT_ID'],
                'client_secret': os.environ['ZOOM_CLIENT_SECRET'],
                'secret_token': os.environ['ZOOM_SECRET_TOKEN'],
            }
        },
        'media_types': MediaType.AUDIO | MediaType.TRANSCRIPT,
        'logging': 'info'
    })
    
    # Handle media events
    rtms.on('audio', lambda data: print(f"Audio from {data['user_name']}: {len(data['buffer'])} bytes"))
    rtms.on('transcript', lambda data: print(f"{data['user_name']}: {data['text']}"))
    rtms.on('error', lambda err: print(f"Error: {err}"))

# Initialize on startup
asyncio.get_event_loop().run_until_complete(init_rtms())

# Setup webhook with auto-validation
webhook = WebhookManager(
    webhook_path='/webhook',
    zoom_secret_token=os.environ['ZOOM_SECRET_TOKEN']
)
webhook.setup_flask(app, rtms)

if __name__ == '__main__':
    app.run(port=3000)
```

--------------------------------

### FFmpeg HLS Live Transcoding Setup

Source: https://github.com/zoom/rtms-samples/blob/main/streaming/stream_audio_and_video_to_custom_frontend_passthru_js/readme.md

This example demonstrates how FFmpeg is used to transcode raw audio and video data received via pipes into HLS live streaming segments. It assumes audio and video buffers are available in respective pipes.

```bash
ffmpeg -i <audio_pipe> -i <video_pipe> -c:a aac -b:a 128k -c:v libx264 -preset veryfast -f hls -hls_time 4 -hls_list_size 0 -hls_segment_filename 'public/hls/segment%03d.ts' public/hls/stream.m3u8
```

--------------------------------

### RTMS SDK Client Setup and Event Handling - JavaScript

Source: https://context7.com/zoom/rtms-samples/llms.txt

Sets up an RTMS client using the official Zoom SDK, handling webhook events for stream start and stop. It configures audio, video, and screen share parameters, and registers data handlers for various media types.

```javascript
import rtms from "@zoom/rtms";

let clients = new Map();

// Set up webhook event handler
rtms.onWebhookEvent(({ event, payload }) => {
  const streamId = payload?.rtms_stream_id;

  if (event === "meeting.rtms_stopped") {
    const client = clients.get(streamId);
    if (client) {
      client.leave();
      clients.delete(streamId);
    }
    return;
  }

  if (event !== "meeting.rtms_started") return;

  // Create new RTMS client for the stream
  const client = new rtms.Client();
  clients.set(streamId, client);

  // Configure audio parameters
  client.setAudioParams({
    contentType: rtms.AudioContentType.RAW_AUDIO,
    codec: rtms.AudioCodec.L16,
    channel: rtms.AudioChannel.MONO,
    dataOpt: rtms.AudioDataOption.AUDIO_MIXED_STREAM,
    duration: 100  // 100ms chunks
  });

  // Configure video parameters (720p H.264)
  client.setVideoParams({
    contentType: rtms.VideoContentType.RAW_VIDEO,
    codec: rtms.VideoCodec.H264,
    resolution: rtms.VideoResolution.HD,
    dataOpt: rtms.VideoDataOption.VIDEO_SINGLE_ACTIVE_STREAM,
    fps: 30
  });

  // Configure screen share (JPEG format)
  client.setDeskshareParams({
    contentType: rtms.VideoContentType.RAW_VIDEO,
    codec: rtms.VideoCodec.JPG,
    resolution: rtms.VideoResolution.SD,
    fps: 5
  });

  // Register data handlers
  client.onAudioData((data, size, timestamp, metadata) => {
    console.log(`Audio: ${size} bytes from ${metadata.userName}`);
  });

  client.onVideoData((data, size, timestamp, metadata) => {
    console.log(`Video: ${size} bytes from ${metadata.userName}`);
  });

  client.onDeskshareData((data, size, timestamp, metadata) => {
    console.log(`Screen share: ${size} bytes from ${metadata.userName}`);
  });

  client.onTranscriptData((data, size, timestamp, metadata) => {
    console.log(`${metadata.userName}: ${data}`);
  });

  // Join using webhook payload directly
  client.join(payload);
});

```

--------------------------------

### Install or Upgrade Whisper (Python)

Source: https://github.com/zoom/rtms-samples/blob/main/audio/send_audio_to_whisper_local_transcribe_service_js/README.md

Provides commands to upgrade pip and then install the openai-whisper library. This resolves issues related to Whisper installation failures.

```python
pip install --upgrade pip
pip install openai-whisper
```
Tokens:
Raw