### Local Development Setup with ngrok

Source: https://context7.com/nexmo-se/elevenlabs-agent-ws-connector/llms.txt

This bash script outlines the steps for setting up the connector for local development. It includes cloning the repository, installing dependencies, configuring environment variables, setting up ngrok tunnels, and starting the connector.

```bash
# 1. Clone and install dependencies
git clone https://github.com/nexmo-se/11l-agent-ws-connector.git
cd 11l-agent-ws-connector
npm install

# 2. Configure environment
cp .env-example .env
# Edit .env with your ElevenLabs credentials

# 3. Configure ngrok for multiple tunnels (ngrok.yml)
# version: "2"
# tunnels:
#   connector:
#     addr: 6000
#     proto: http
#   voice-app:
#     addr: 8000
#     proto: http

# 4. Start ngrok tunnels
ngrok start --all

# 5. Start the connector
node elevenlabs-agent-ws-connector.cjs

# Output:
# Connector application listening on port 6000
# >>> Wait around 36 seconds to see the actual streaming timer average ...
# >>> Average streaming timer (should be close to 20 AND under 20.000): 19.85
```

--------------------------------

### Initialize Environment Configuration

Source: https://github.com/nexmo-se/elevenlabs-agent-ws-connector/blob/master/README.md

Create the .env file from the provided example template to store necessary API credentials.

```bash
cp .env.example .env
```

--------------------------------

### Local Development with ngrok

Source: https://context7.com/nexmo-se/elevenlabs-agent-ws-connector/llms.txt

Step-by-step guide for setting up and running the connector locally using ngrok for tunneling.

```APIDOC
## Server Setup and Deployment

### Local Development with ngrok

Complete setup for running the connector locally with ngrok tunneling.

```bash
# 1. Clone and install dependencies
git clone https://github.com/nexmo-se/11l-agent-ws-connector.git
cd 11l-agent-ws-connector
npm install

# 2. Configure environment
cp .env-example .env
# Edit .env with your ElevenLabs credentials

# 3. Configure ngrok for multiple tunnels (ngrok.yml)
# version: "2"
# tunnels:
#   connector:
#     addr: 6000
#     proto: http
#   voice-app:
#     addr: 8000
#     proto: http

# 4. Start ngrok tunnels
ngrok start --all

# 5. Start the connector
node elevenlabs-agent-ws-connector.cjs

# Output:
# Connector application listening on port 6000
# >>> Wait around 36 seconds to see the actual streaming timer average ...
# >>> Average streaming timer (should be close to 20 AND under 20.000): 19.85
```
```

--------------------------------

### Launch Connector Application

Source: https://github.com/nexmo-se/elevenlabs-agent-ws-connector/blob/master/README.md

Start the connector server application using Node.js.

```bash
node elevenlabs-agent-ws-connector.cjs
```

--------------------------------

### Install Dependencies

Source: https://github.com/nexmo-se/elevenlabs-agent-ws-connector/blob/master/README.md

Install the required Node.js modules for the connector application.

```bash
npm install
```

--------------------------------

### Environment Configuration for ElevenLabs Connector

Source: https://context7.com/nexmo-se/elevenlabs-agent-ws-connector/llms.txt

This `.env` file example shows the required and optional environment variables for configuring the connector. It includes ElevenLabs API and Agent IDs, optional Voice ID, and settings for audio recording and server port.

```bash
# .env file configuration

# ElevenLabs API key (required)
ELEVENLABS_API_KEY=your_api_key_here

# ElevenLabs Agent ID (required)
# Found in ElevenLabs Console: Conversational AI > Agents > ... > Copy Agent ID
ELEVENLABS_AGENT_ID=your_agent_id_here

# ElevenLabs Voice ID (optional - overrides agent default)
# See: https://help.elevenlabs.io/hc/en-us/articles/14599760033937
ELEVENLABS_VOICE_ID=9BWtsMINqrJLrRacOk9x

# Enable audio recording for debugging (optional)
RECORD_ALL_AUDIO=false

# Server port (optional, defaults to 6000)
PORT=6000

# VCR port for cloud deployment (optional)
VCR_PORT=6000
```

--------------------------------

### Audio Recording Configuration and Conversion

Source: https://context7.com/nexmo-se/elevenlabs-agent-ws-connector/llms.txt

This section explains how to enable and configure audio recording for debugging purposes by setting `RECORD_ALL_AUDIO=true` in the `.env` file. It also provides an example of how to convert raw PCM audio files to WAV format using ffmpeg.

```javascript
// Set RECORD_ALL_AUDIO=true in .env to enable
// Audio files are saved to ./recordings/ directory

// Recording filenames format:
// - Audio sent to ElevenLabs: {call_uuid}_rec_to_11l_{timestamp}.raw
// - Audio sent to Vonage: {call_uuid}_rec_to_vg_{timestamp}.raw

// Example filenames:
// abc123_rec_to_11l_2025_01_15_14_30_45_123.raw
// abc123_rec_to_vg_2025_01_15_14_30_45_123.raw

// Audio format: Raw PCM, 16-bit, 16kHz, mono
// Convert to WAV using ffmpeg:
ffmpeg -f s16le -ar 16000 -ac 1 -i recording.raw recording.wav
```

--------------------------------

### Express.js Webhook Handler

Source: https://context7.com/nexmo-se/elevenlabs-agent-ws-connector/llms.txt

An example of how to handle incoming webhook events in an Express.js application. It differentiates between user transcripts and agent responses and logs them to the console.

```javascript
app.post('/events', (req, res) => {
  const { type, transcript, response, call_uuid } = req.body;

  if (type === 'user_transcript') {
    console.log(`User [${call_uuid}]: ${transcript}`);
  } else if (type === 'agent_response') {
    console.log(`Agent [${call_uuid}]: ${response}`);
  }

  res.status(200).send('OK');
});
```

--------------------------------

### Webhook Payloads for User Transcript and Agent Response

Source: https://context7.com/nexmo-se/elevenlabs-agent-ws-connector/llms.txt

These are example JSON payloads for the webhook events sent by the connector. The `user_transcript` event contains the user's spoken input, while `agent_response` contains the AI agent's reply.

```json
{
  "type": "user_transcript",
  "transcript": "What are your business hours?",
  "call_uuid": "abc123-def456-ghi789"
}
```

```json
{
  "type": "agent_response",
  "response": "Our business hours are Monday through Friday, 9 AM to 5 PM Eastern Time.",
  "call_uuid": "abc123-def456-ghi789"
}
```

--------------------------------

### Environment Configuration

Source: https://context7.com/nexmo-se/elevenlabs-agent-ws-connector/llms.txt

Required and optional environment variables for configuring the connector, including ElevenLabs credentials and audio recording settings.

```APIDOC
## Environment Configuration

### Required Environment Variables

Configure the connector using environment variables for ElevenLabs credentials and optional audio recording.

```bash
# .env file configuration

# ElevenLabs API key (required)
ELEVENLABS_API_KEY=your_api_key_here

# ElevenLabs Agent ID (required)
# Found in ElevenLabs Console: Conversational AI > Agents > ... > Copy Agent ID
ELEVENLABS_AGENT_ID=your_agent_id_here

# ElevenLabs Voice ID (optional - overrides agent default)
# See: https://help.elevenlabs.io/hc/en-us/articles/14599760033937
ELEVENLABS_VOICE_ID=9BWtsMINqrJLrRacOk9x

# Enable audio recording for debugging (optional)
RECORD_ALL_AUDIO=false

# Server port (optional, defaults to 6000)
PORT=6000

# VCR port for cloud deployment (optional)
VCR_PORT=6000
```
```

--------------------------------

### Initialize ElevenLabs Conversation

Source: https://context7.com/nexmo-se/elevenlabs-agent-ws-connector/llms.txt

Constructs the WebSocket URL and sends the initial configuration message to the ElevenLabs agent.

```javascript
// ElevenLabs WebSocket URL construction
const elevenLabsWsUrl = "wss://api.elevenlabs.io/v1/convai/conversation?agent_id=" + elevenLabsAgentId;

// Connection with API key authentication
const elevenLabsWs = new webSocket(elevenLabsWsUrl, {
  headers: { "xi-api-key": elevenLabsApiKey },
});

// Initialization message sent on connection open
const initMessage = {
  "type": "conversation_initiation_client_data",
  "conversation_config_override": {
    "agent": {
      "prompt": {
        "prompt": "You are a helpful assistant."
      },
      "first_message": "Hi, I'm Aria from ElevenLabs support. How can I help you today?",
      "language": "en"
    },
    "tts": {
      "voice_id": "9BWtsMINqrJLrRacOk9x"  // Aria voice
    }
  }
};

elevenLabsWs.send(JSON.stringify(initMessage));
```

--------------------------------

### Audio Recording Configuration

Source: https://context7.com/nexmo-se/elevenlabs-agent-ws-connector/llms.txt

Instructions on enabling and configuring audio recording for debugging and quality assurance, including file naming conventions and format.

```APIDOC
### Audio Recording Configuration

Enable audio recording for debugging and quality assurance purposes.

```javascript
// Set RECORD_ALL_AUDIO=true in .env to enable
// Audio files are saved to ./recordings/ directory

// Recording filenames format:
// - Audio sent to ElevenLabs: {call_uuid}_rec_to_11l_{timestamp}.raw
// - Audio sent to Vonage: {call_uuid}_rec_to_vg_{timestamp}.raw

// Example filenames:
// abc123_rec_to_11l_2025_01_15_14_30_45_123.raw
// abc123_rec_to_vg_2025_01_15_14_30_45_123.raw

// Audio format: Raw PCM, 16-bit, 16kHz, mono
// Convert to WAV using ffmpeg:
ffmpeg -f s16le -ar 16000 -ac 1 -i recording.raw recording.wav
```
```

--------------------------------

### Stream Audio to ElevenLabs

Source: https://context7.com/nexmo-se/elevenlabs-agent-ws-connector/llms.txt

Forwards binary audio chunks received from the Vonage WebSocket to the ElevenLabs API as base64 encoded strings.

```javascript
// Sending user audio to ElevenLabs
ws.on('message', async (msg) => {
  if (typeof msg !== "string") {
    // Binary audio data from Vonage
    elevenLabsWs.send(JSON.stringify({
      user_audio_chunk: msg.toString('base64')
    }));
  }
});
```

--------------------------------

### Configure Vonage NCCO for WebSocket

Source: https://context7.com/nexmo-se/elevenlabs-agent-ws-connector/llms.txt

Defines the Nexmo Call Control Object to initiate a WebSocket connection to the connector server.

```javascript
// Vonage Voice API NCCO (Nexmo Call Control Object) to connect to this endpoint
const ncco = [
  {
    "action": "connect",
    "endpoint": [
      {
        "type": "websocket",
        "uri": "wss://your-connector-server.com/socket?peer_uuid=call-uuid-123&webhook_url=https://your-app.com/events",
        "content-type": "audio/l16;rate=16000"
      }
    ]
  }
];

// Example WebSocket connection from Vonage platform
// Query parameters:
// - peer_uuid: The UUID of the peer call leg
// - webhook_url: URL to receive transcription and response events
```

--------------------------------

### Transcript and Response Webhooks

Source: https://context7.com/nexmo-se/elevenlabs-agent-ws-connector/llms.txt

The connector sends real-time events to your application via HTTP POST requests to a specified endpoint. This enables logging conversations and triggering custom actions.

```APIDOC
## Transcript and Response Webhooks

The connector sends real-time events to your application via HTTP POST requests, enabling you to log conversations, trigger actions, or build custom integrations.

### User Transcript Webhook

**Method:** POST
**Endpoint:** `https://your-app.com/events`

**Request Body:**
```json
{
  "type": "user_transcript",
  "transcript": "What are your business hours?",
  "call_uuid": "abc123-def456-ghi789"
}
```

### Agent Response Webhook

**Method:** POST
**Endpoint:** `https://your-app.com/events`

**Request Body:**
```json
{
  "type": "agent_response",
  "response": "Our business hours are Monday through Friday, 9 AM to 5 PM Eastern Time.",
  "call_uuid": "abc123-def456-ghi789"
}
```

### Example Webhook Handler (Express.js)

```javascript
app.post('/events', (req, res) => {
  const { type, transcript, response, call_uuid } = req.body;

  if (type === 'user_transcript') {
    console.log(`User [${call_uuid}]: ${transcript}`);
  } else if (type === 'agent_response') {
    console.log(`Agent [${call_uuid}]: ${response}`);
  }

  res.status(200).send('OK');
});
```
```

--------------------------------

### VCR Health Check Endpoint

Source: https://context7.com/nexmo-se/elevenlabs-agent-ws-connector/llms.txt

This bash command demonstrates how to perform a health check on the connector's VCR health endpoint. A successful check returns an HTTP 200 OK with 'Ok' in the body.

```bash
# Health check request
curl -X GET https://your-connector-server.com/_/health

# Response
# HTTP 200 OK
# Body: Ok
```

--------------------------------

### Handle ElevenLabs WebSocket Events

Source: https://context7.com/nexmo-se/elevenlabs-agent-ws-connector/llms.txt

Processes incoming messages from ElevenLabs, including audio payloads, transcriptions, agent responses, interruptions, and keep-alive pings.

```javascript
// Event handling for ElevenLabs messages
elevenLabsWs.on('message', async(msg) => {
  const data = JSON.parse(msg.toString());

  switch(data.type) {
    case 'audio':
      // AI-generated speech audio (base64 encoded PCM 16kHz)
      const audioPayload = Buffer.from(data.audio_event.audio_base_64, 'base64');
      // Buffer and stream to Vonage at proper rate
      payloadToVg = Buffer.concat([payloadToVg, audioPayload]);
      break;

    case 'user_transcript':
      // User's speech transcription
      // { user_transcription_event: { user_transcript: "Hello, how are you?" } }
      await axios.post(webhookUrl, {
        type: 'user_transcript',
        transcript: data.user_transcription_event.user_transcript,
        call_uuid: peerUuid
      });
      break;

    case 'agent_response':
      // AI agent's text response
      // { agent_response_event: { agent_response: "I'm doing well, thank you!" } }
      await axios.post(webhookUrl, {
        type: 'agent_response',
        response: data.agent_response_event.agent_response,
        call_uuid: peerUuid
      });
      break;

    case 'interruption':
      // Barge-in detected - clear audio buffer
      payloadToVg = Buffer.alloc(0);
      streamToVgIndex = 0;
      break;

    case 'ping':
      // Keep-alive ping - must respond with pong
      elevenLabsWs.send(JSON.stringify({
        type: "pong",
        event_id: data.ping_event.event_id
      }));
      break;
  }
});
```

--------------------------------

### Health Check Endpoint

Source: https://context7.com/nexmo-se/elevenlabs-agent-ws-connector/llms.txt

A health check endpoint to verify the status of the Vonage Cloud Runtime (VCR) serverless infrastructure deployment.

```APIDOC
## Health Check Endpoint

### `GET /_/health` - VCR Health Check

Health check endpoint for Vonage Cloud Runtime (VCR) serverless infrastructure deployment.

### Method
GET

### Endpoint
`https://your-connector-server.com/_/health`

### Response
#### Success Response (200)
- **Body**: Ok
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.