Eleven Agents

https://elevenlabs.io/docs/eleven-agents
Admin
Eleven Agents is an AI agent platform by ElevenLabs that enables developers to build, deploy, and...
Tokens:425,885
Snippets:4,309
Trust Score:10
Update:4 hours ago
Show doc for...
Tokens:
Raw
### Minimal working example with ConversationProvider and useConversationControls

Source: https://elevenlabs.io/docs/eleven-agents/libraries/react

A minimal example that connects to an agent and lets the user start and end a voice conversation. Uses ConversationProvider, useConversationControls, and useConversationStatus. The startSession call includes an example agentId.

```jsx
import {
  ConversationProvider,
  useConversationControls,
  useConversationStatus,
} from "@elevenlabs/react";
function App() {
  return (
    <ConversationProvider>
      <Agent />
    </ConversationProvider>
  );
}
function Agent() {
  const { startSession, endSession } = useConversationControls();
  const { status } = useConversationStatus();
  if (status === "connected") {
    return <button onClick={endSession}>End</button>;
  }
  return (
    <button onClick={() => startSession({ agentId: "agent_7101k5zvyjhmfg983brhmhkd98n6" })}>
      Start
    </button>
  );
}
```

--------------------------------

### Clone and set up the connector repository

Source: https://elevenlabs.io/docs/eleven-agents/phone-numbers/telephony/vonage

Commands to clone the connector repository and copy the environment file. Run these in your terminal to get started.

```bash
git clone https://github.com/nexmo-se/elevenlabs-agent-ws-connector.git
cd elevenlabs-agent-ws-connector
cp .env.example .env
```

--------------------------------

### Install VC++ runtime, Media Foundation, and get cert

Source: https://elevenlabs.io/docs/eleven-agents/phone-numbers/microsoft-teams/graph-media-bot

On the VM, installs the VC++ runtime and Media Foundation feature required by NativeMedia.dll, then uses win-acme to obtain a Let's Encrypt certificate for the VM's FQDN and store it as a PFX file. The VM's cloudapp.azure.com FQDN works for the cert.

```powershell
# VC++ runtime + Media Foundation feature (required by NativeMedia.dll)
choco install -y vcredist140
Install-WindowsFeature Server-Media-Foundation
# CA cert for the VM's FQDN via win-acme (HTTP-01), then import to LocalMachine\My
& wacs.exe --target manual --host <vm-fqdn>.cloudapp.azure.com `
  --validation selfhosting --store pfxfile --pfxfilepath C:\bot\certs --accepttos
```

--------------------------------

### Install ElevenLabs CLI via Homebrew

Source: https://elevenlabs.io/docs/eleven-agents/operate/cli

Recommended installation method for macOS using Homebrew. Installs a standalone binary and makes the `elevenlabs` command globally available.

```bash
brew install elevenlabs/tap/elevenlabs
```

--------------------------------

### Navigate to project directory

Source: https://elevenlabs.io/docs/eleven-agents/guides/quickstarts/next-js

Change into the newly created project directory.

```bash
cd my-conversational-agent
```

--------------------------------

### Install Tailwind CSS for Next.js

Source: https://elevenlabs.io/docs/eleven-agents/libraries/web-sockets

Install Tailwind CSS and its dependencies for styling the Next.js project. Follow the official Tailwind setup guide for Next.js after running these commands.

```bash
npm install -D tailwindcss postcss autoprefixer
npx tailwindcss init -p
```

--------------------------------

### Start a conversation with useConversationControls and useConversationStatus

Source: https://elevenlabs.io/docs/eleven-agents/libraries/react-native

Example of starting and ending a conversation using the useConversationControls and useConversationStatus hooks. The startSession method requires an agentId, and the button toggles between Start and End based on the connection status.

```jsx
import { useConversationControls, useConversationStatus } from "@elevenlabs/react-native";
import React from "react";
import { View, Text, Button } from "react-native";
function ConversationComponent() {
  const { startSession, endSession } = useConversationControls();
  const { status } = useConversationStatus();
  const handleStart = async () => {
    await startSession({
      agentId: "agent_7101k5zvyjhmfg983brhmhkd98n6",
    });
  };
  return (
    <View>
      <Text>Status: {status}</Text>
      <Button
        title={status === "connected" ? "End" : "Start"}
        onPress={status === "connected" ? endSession : handleStart}
      />
    </View>
  );
}
```

--------------------------------

### Initialize a new project with elevenlabs agents init

Source: https://elevenlabs.io/docs/eleven-agents/operate/cli

Run this command to create the project structure with configuration directories and registry files.

```bash
elevenlabs agents init
```

--------------------------------

### Create project directory and navigate into it

Source: https://elevenlabs.io/docs/eleven-agents/guides/integrations/raspberry-pi-voice-assistant

Creates a new directory named 'eleven-voice-assistant' and changes into it.

```bash
mkdir eleven-voice-assistant
cd eleven-voice-assistant
```

--------------------------------

### GET /v1/convai/users

Source: https://elevenlabs.io/docs/eleven-agents/api-reference/users/list

Get distinct users from conversations with pagination. Supports filtering by agent, branch, call start time, and user ID, as well as sorting and pagination.

```APIDOC
## GET /v1/convai/users

### Description
Get distinct users from conversations with pagination.

### Method
GET

### Endpoint
https://api.elevenlabs.io/v1/convai/users

### Parameters
#### Headers
- **xi-api-key** (string) - Optional - API key for authentication.

#### Query Parameters
- **agent_id** (string) - Optional - Agent id (agent_…) or speech engine external id (seng_), resolved to the same underlying resource.
- **branch_id** (string) - Optional - Filter conversations by branch ID.
- **call_start_before_unix** (integer) - Optional - Unix timestamp (in seconds) to filter conversations up to this start date.
- **call_start_after_unix** (integer) - Optional - Unix timestamp (in seconds) to filter conversations after to this start date.
- **search** (string) - Optional - Search/filter by user ID (exact match).
- **page_size** (integer) - Optional - How many users to return at maximum. Defaults to 30. Valid range: 1-100.
- **sort_by** (enum) - Optional - The field to sort the results by. Defaults to last_contact_unix_secs. Allowed values: last_contact_unix_secs, conversation_count, average_sentiment_score.
- **sort_direction** (enum) - Optional - The direction to sort the results. Allowed values: asc, desc.
- **cursor** (string) - Optional - Used for fetching next page. Cursor is returned in the response.

### Response
#### Success Response (200)
- **users** (list of objects) - List of users.
- **has_more** (boolean) - Indicates if there are more results.
- **next_cursor** (string) - Optional - Cursor for the next page.

#### Errors
- **422** - Users List Request Unprocessable Entity Error
```

--------------------------------

### Create your first agent with elevenlabs agents add

Source: https://elevenlabs.io/docs/eleven-agents/operate/cli

Creates a new agent configuration using the assistant template.

```bash
elevenlabs agents add "My Assistant" --template assistant
```

--------------------------------

### Initialize npm and install dependencies

Source: https://elevenlabs.io/docs/eleven-agents/guides/quickstarts/java-script

Initializes a new npm project and installs Vite and the ElevenLabs client package.

```bash
npm init -y
npm install vite @elevenlabs/client
```

--------------------------------

### WebSocket handler for agent_chat_response_part

Source: https://elevenlabs.io/docs/eleven-agents/customization/events/client-events

Example WebSocket handler that processes start, delta, and stop parts of the agent_chat_response_part event. It initializes a response buffer on start, appends text on delta, and finalizes the response on stop.

```javascript
// Example handler
websocket.on('agent_chat_response_part', (event) => {
  const { text_response_part } = event;
  const { type: partType, text, response_id } = text_response_part;
  if (partType === 'start') {
    initializeResponseBuffer(response_id);
  } else if (partType === 'delta') {
    appendToResponseBuffer(response_id, text);
  } else if (partType === 'stop') {
    finalizeResponse(response_id);
  }
});
```

--------------------------------

### Start the Frontend Server

Source: https://elevenlabs.io/docs/eleven-agents/guides/quickstarts/java-script

Runs the frontend development server using npm. Ensure 'YOUR_AGENT_ID' is replaced with the actual agent ID before running.

```bash
npm run dev:frontend
```

--------------------------------

### GET conversation response with otlp_traces

Source: https://elevenlabs.io/docs/eleven-agents/customization/opentelemetry-traces

Example JSON response from the GET conversation endpoint with format=opentelemetry. The otlp_traces object is additive; transcript is still returned. Signed URLs in span attributes expire after about 15 minutes.

```JSON
{
  "conversation_id": "conv_9001k1zph3fkeh5s8xg9z90swaqa",
  "agent_id": "agent_7101k5zvyjhmfg983brhmhkd98n6",
  "status": "done",
  "transcript": [],
  "otlp_traces": {
    "resourceSpans": []
  }
}
```

--------------------------------

### Create the HTML Interface (index.html)

Source: https://elevenlabs.io/docs/eleven-agents/guides/quickstarts/java-script

Sets up a simple user interface with start and stop buttons and status displays. Includes a script tag referencing script.js.

```html
<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1.0" />
        <title>ElevenLabs Agents</title>
    </head>
    <body style="font-family: Arial, sans-serif; text-align: center; padding: 50px;">
        <h1>ElevenLabs Agents</h1>
        <div style="margin-bottom: 20px;">
            <button id="startButton" style="padding: 10px 20px; margin: 5px;">Start Conversation</button>
            <button id="stopButton" style="padding: 10px 20px; margin: 5px;" disabled>Stop Conversation</button>
        </div>
        <div style="font-size: 18px;">
            <p>Status: <span id="connectionStatus">Disconnected</span></p>
            <p>Agent is <span id="agentStatus">listening</span></p>
        </div>
        <script type="module" src="../images/script.js"></script>
    </body>
</html>
```

--------------------------------

### agent_chat_response_part start, delta, and stop events

Source: https://elevenlabs.io/docs/eleven-agents/customization/events/client-events

Example JSON structures for the start, delta, and stop messages of the agent_chat_response_part event. These stream the agent's response text as it is generated. The response_id identifies the message and matches the response_id of the later agent_response event.

```json
// Example start event
{
  "type": "agent_chat_response_part",
  "text_response_part": {
    "type": "start",
    "text": "",
    "event_id": 12345,
    "response_id": "f47ac10b-58cc-4372-a567-0e02b2c3d479"
  }
}
```

```json
// Example delta event with text chunk
{
  "type": "agent_chat_response_part",
  "text_response_part": {
    "type": "delta",
    "text": "Hello, how can I",
    "event_id": 12345,
    "response_id": "f47ac10b-58cc-4372-a567-0e02b2c3d479"
  }
}
```

```json
// Example stop event
{
  "type": "agent_chat_response_part",
  "text_response_part": {
    "type": "stop",
    "text": "",
    "event_id": 12345,
    "response_id": "f47ac10b-58cc-4372-a567-0e02b2c3d479"
  }
}
```

--------------------------------

### Configure system tools with ElevenLabs API (Python)

Source: https://elevenlabs.io/docs/eleven-agents/customization/tools/system-tools

Python example showing how to create an agent with built-in system tools (end_call, language_detection) using the ElevenLabs SDK. Each tool is keyed by name under built_in_tools, and leaving description blank uses the default prompt.

```python
from elevenlabs import AgentConfig, ConversationalConfig, ElevenLabs
elevenlabs = ElevenLabs(api_key="YOUR_API_KEY")
# Each system tool is keyed by name under built_in_tools.
# Leave "description" blank to use the default, tool-specific prompt.
response = elevenlabs.conversational_ai.agents.create(
    conversation_config=ConversationalConfig(
        agent=AgentConfig(
            prompt={
                "built_in_tools": {
                    "end_call": {
                        "type": "system",
                        "name": "end_call",
                        "description": "",
                        "params": {"system_tool_type": "end_call"},
                    },
                    "language_detection": {
                        "type": "system",
                        "name": "language_detection",
                        "description": "",
                        "params": {"system_tool_type": "language_detection"},
                    },
                }
            },
        ),
    ),
)
```

--------------------------------

### getId - Get current conversation ID

Source: https://elevenlabs.io/docs/eleven-agents/libraries/react

Returns the current conversation ID. Example logs the ID, e.g., "conv_9001k1zph3fkeh5s8xg9z90swaqa".

```javascript
const { getId } = useConversation();
const conversationId = getId();
console.log(conversationId); // e.g., "conv_9001k1zph3fkeh5s8xg9z90swaqa"
```

--------------------------------

### First message example

Source: https://elevenlabs.io/docs/eleven-agents/quickstart

This is the first message the assistant will speak out loud when a user starts a conversation. Replace <company name> with your actual company name.

```text
Hi, this is Alexis from <company name> support. How can I help you today?
```

--------------------------------

### Install elevenlabs with pyaudio extra

Source: https://elevenlabs.io/docs/eleven-agents/libraries/python

Install the pyaudio extra for default audio input/output. May require additional system dependencies.

```bash
pip install "elevenlabs[pyaudio]"
# or
poetry add "elevenlabs[pyaudio]"
```

--------------------------------

### conversation_initiation_metadata event structure

Source: https://elevenlabs.io/docs/eleven-agents/customization/events/client-events

Example JSON structure for the conversation_initiation_metadata event, automatically sent when starting a conversation. Includes conversation ID and audio format settings.

```json
{
  "type": "conversation_initiation_metadata",
  "conversation_initiation_metadata_event": {
    "conversation_id": "conv_123",
    "agent_output_audio_format": "pcm_44100",  // TTS output format
    "user_input_audio_format": "pcm_16000"    // ASR input format
  }
}
```

--------------------------------

### Create project directory

Source: https://elevenlabs.io/docs/eleven-agents/guides/quickstarts/java-script

Creates a new directory for the project and navigates into it.

```bash
mkdir elevenlabs-conversational-ai
cd elevenlabs-conversational-ai
```

--------------------------------

### Get agent at branch tip with client.conversational_ai.agents.get

Source: https://elevenlabs.io/docs/eleven-agents/operate/versioning

Retrieves an agent at the tip of a branch using the branch_id parameter. The example uses a placeholder branch ID 'agtbrch_xxxx'.

```python
agent = client.conversational_ai.agents.get(
    agent_id="agent_7101k5zvyjhmfg983brhmhkd98n6",
    branch_id="agtbrch_xxxx"
)
```

--------------------------------

### API Implementation for play_keypad_touch_tone (Python)

Source: https://elevenlabs.io/docs/eleven-agents/customization/tools/system-tools/play-keypad-touch-tone

Python example using the ElevenLabs SDK to create an agent with the 'play_keypad_touch_tone' system tool enabled. Includes an optional custom description and the required 'params' with 'system_tool_type'.

```python
from elevenlabs import AgentConfig, ConversationalConfig, ElevenLabs
elevenlabs = ElevenLabs(api_key="YOUR_API_KEY")
response = elevenlabs.conversational_ai.agents.create(
    conversation_config=ConversationalConfig(
        agent=AgentConfig(
            first_message="Hi, I can help you navigate phone systems. How can I assist you today?",
            prompt={
                "prompt": "You are a helpful assistant that can interact with phone systems.",
                "built_in_tools": {
                    "play_keypad_touch_tone": {
                        "type": "system",
                        "name": "play_keypad_touch_tone",
                        # Optional custom description
                        "description": "Play DTMF tones to interact with automated phone systems.",
                        "params": {"system_tool_type": "play_keypad_touch_tone"},
                    }
                },
            },
        ),
    ),
)
```

--------------------------------

### Get agent at specific version with client.conversational_ai.agents.get

Source: https://elevenlabs.io/docs/eleven-agents/operate/versioning

Retrieves an agent at a specific version using the version_id parameter. The example uses a placeholder version ID 'agtvrsn_xxxx'.

```python
agent = client.conversational_ai.agents.get(
    agent_id="agent_7101k5zvyjhmfg983brhmhkd98n6",
    version_id="agtvrsn_xxxx"
)
```

--------------------------------

### Create project file structure

Source: https://elevenlabs.io/docs/eleven-agents/guides/quickstarts/java-script

Shows the required file structure for the project.

```text
elevenlabs-conversational-ai/
├── index.html
├── script.js
├── package-lock.json
├── package.json
└── node_modules
```

--------------------------------

### CI/CD Pipeline Integration with GitHub Actions

Source: https://elevenlabs.io/docs/eleven-agents/operate/cli

Example GitHub Actions workflow step to install the CLI, set the API key, preview changes, deploy, and verify status.

```yaml
# In your GitHub Actions workflow
- name: Deploy ElevenAgents agents
  run: |
    npm install -g @elevenlabs/cli
    export ELEVENLABS_API_KEY=${{ secrets.ELEVENLABS_API_KEY }}
    elevenlabs agents push --dry-run  # Preview changes
    elevenlabs agents push            # Deploy
    elevenlabs agents status          # Verify deployment
```
Tokens:
Raw