### Basic Vision Agent Setup in Python

Source: https://visionagents.ai/guides/event-system

Initializes a basic Vision Agent with core components like edge, LLM, TTS, and STT. This forms the foundation before adding event handling capabilities. It requires Python's asyncio and specific Vision Agents libraries.

```python
import asyncio
import logging
from uuid import uuid4

from dotenv import load_dotenv
from vision_agents.core.edge.types import User
from vision_agents.plugins import elevenlabs, deepgram, openai, getstream
from vision_agents.core import agents

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
load_dotenv()

async def start_agent():
    call_id = str(uuid4())
    
    # Create the agent with basic components
    agent = agents.Agent(
        edge=getstream.Edge(),
        agent_user=User(name="My AI Assistant", id="agent"),
        instructions="You're a helpful voice AI assistant. Keep responses short and conversational.",
        llm=openai.LLM(model="gpt-4o-mini"),
        tts=elevenlabs.TTS(),
        stt=deepgram.STT(),
    )
    
    await agent.create_user()
    
    # Create and join the call
    call = agent.edge.client.video.call("default", call_id)
    await agent.edge.open_demo(call)
    
    with await agent.join(call):
        await agent.finish()
```

--------------------------------

### Python: Initialize Vogent Turn Detection

Source: https://visionagents.ai/integrations/vogent

Shows how to initialize the Vogent TurnDetection class. It includes examples for default settings and custom settings like buffer duration and confidence threshold. The `start()` method initiates detection and downloads models if necessary.

```python
from vision_agents.plugins import vogent

# Default settings
turn_detection = vogent.TurnDetection()

# Custom settings for more aggressive turn detection
turn_detection = vogent.TurnDetection(
    buffer_in_seconds=1.5,
    confidence_threshold=0.7
)

# Start detection (downloads models if needed)
await turn_detection.start()
```

--------------------------------

### Using Logging for Events in VisionAgents (Python)

Source: https://visionagents.ai/guides/event-system

This example demonstrates the importance of appropriate logging within VisionAgents event handlers. Logging key information, such as user transcripts and their confidence levels, aids in debugging and understanding the agent's interaction flow.

```python
@agent.events.subscribe
async def handle_transcript(event: STTTranscriptEvent):
    # Log important events for debugging
    logger.info(f"Transcript: {event.text} (confidence: {event.confidence})")

```

--------------------------------

### Python Example: Vogent Turn Detection with Agent

Source: https://visionagents.ai/integrations/vogent

Demonstrates how to integrate Vogent's turn detection with a Vision Agent. It shows initialization with custom settings and how to subscribe to turn started and ended events. This requires importing necessary classes from the vision_agents library.

```python
from vision_agents.core import Agent
from vision_agents.plugins import vogent
from vision_agents.core.turn_detection.events import TurnStartedEvent, TurnEndedEvent

# Create turn detection with custom settings
turn_detection = vogent.TurnDetection(
    buffer_in_seconds=2.0,
    confidence_threshold=0.5
)

# Use with an agent
agent = Agent(
    turn_detection=turn_detection,
    # ... other agent configuration
)

# Or use standalone
await turn_detection.start()

# Listen for turn events
@turn_detection.events.subscribe
async def on_turn_started(event: TurnStartedEvent):
    print(f"User {event.participant.user_id} started speaking")

@turn_detection.events.subscribe
async def on_turn_ended(event: TurnEndedEvent):
    print(f"User {event.participant.user_id} finished speaking")
    print(f"Confidence: {event.confidence}")

# Stop when finished
await turn_detection.stop()
```

--------------------------------

### Setup Tracing Instrumentation (Python)

Source: https://visionagents.ai/core/telemetry

Configures OpenTelemetry tracing by setting up a TracerProvider, an OTLP gRPC exporter, and a BatchSpanProcessor. This code should be executed before the agent or server starts to ensure all spans are captured.

```python
from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter

resource = Resource.create(
    {
        "service.name": "agents",
    }
)
tp = TracerProvider(resource=resource)
exporter = OTLPSpanExporter(endpoint="localhost:4317", insecure=True)

tp.add_span_processor(BatchSpanProcessor(exporter))
trace.set_tracer_provider(tp)
```

--------------------------------

### Handle Multiple Event Types in Python

Source: https://visionagents.ai/guides/event-system

Demonstrates how to handle multiple distinct event types within a single subscription using `typing.Union`. This example specifically handles participants joining or leaving a call session and logs the name of the participant involved.

```python
from typing import Union
from vision_agents.core.events import CallSessionParticipantJoinedEvent, CallSessionParticipantLeftEvent

@agent.events.subscribe
async def handle_participant_changes(event: Union[CallSessionParticipantJoinedEvent, CallSessionParticipantLeftEvent]):
    if isinstance(event, CallSessionParticipantJoinedEvent):
        logger.info(f"Participant joined: {event.participant.user.name}")
    else:
        logger.info(f"Participant left: {event.participant.user.name}")
```

--------------------------------

### Packaging and Quality Checks

Source: https://visionagents.ai/integrations/create-your-own-plugin

Outlines the steps for packaging and distributing the plugin. This includes installing development dependencies, running tests, and executing pre-commit hooks to ensure code quality before submission.

```bash
uv sync
uv run pre-commit run --all-files
```

--------------------------------

### STT Plugin Workflow Example

Source: https://visionagents.ai/integrations/create-your-own-plugin

Illustrates the typical workflow for an STT plugin, from receiving audio via WebRTC to processing it with a provider and emitting a transcript event.

```text
Call → WebRTC Track → STT.process_audio()
                               ↓
                        calls provider
                               ↓
                Provider transcript  →  self.events.send(events.STTTranscriptEvent(
                                            session_id=self.session_id,
                                            plugin_name=self.provider_name,
                                            text=text,
                                            user_metadata=user_metadata,
                                            confidence=metadata.get("confidence"),
                                            language=metadata.get("language"),
                                            processing_time_ms=metadata.get("processing_time_ms"),
                                            audio_duration_ms=metadata.get("audio_duration_ms"),
                                            model_name=metadata.get("model_name"),
                                            words=metadata.get("words"),
                                        ))
                                                        ↓
                                            Your application consumes the event
```

--------------------------------

### Full Agent Example with Event Handling and Call Integration

Source: https://visionagents.ai/core/agent-core

A complete example demonstrating agent creation with specified LLM, STT, and TTS services. It includes setting up an event handler for participants joining a call and managing the call lifecycle.

```python
import asyncio
from uuid import uuid4
from vision_agents.core import agents
from vision_agents.plugins import openai, deepgram, elevenlabs, getstream
from vision_agents.core.edge.types import User
from vision_agents.core.events import CallSessionParticipantJoinedEvent

async def main():
    # Create agent
    agent = agents.Agent(
        edge=getstream.Edge(),
        agent_user=User(name="AI Assistant", id="agent"),
        instructions="You're a helpful voice AI assistant. Keep responses short and conversational.",
        llm=openai.LLM(model="gpt-4o-mini"),
        stt=deepgram.STT(),
        tts=elevenlabs.TTS(),
        processors=[]
    )

    # Set up event handlers
    @agent.subscribe
    async def on_participant_joined(event: CallSessionParticipantJoinedEvent):
        await agent.simple_response(f"Hello, {event.participant.user.name}!")

    # Create and join call
    await agent.create_user()
    call = agent.edge.client.video.call("default", str(uuid4()))
    await agent.edge.open_demo(call)

    with await agent.join(call):
        await agent.finish()

    await agent.close()

if __name__ == "__main__":
    asyncio.run(main())
```

--------------------------------

### Install OpenTelemetry OTLP Exporter (Bash)

Source: https://visionagents.ai/core/telemetry

Installs the necessary OpenTelemetry SDK and OTLP exporter packages for gRPC. This is a prerequisite for setting up tracing instrumentation.

```bash
# with uv:
uv add --dev opentelemetry-sdk opentelemetry-exporter-otlp

# or with pip:
pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-grpc
```

--------------------------------

### Install Vision Agents with Wizper Plugin

Source: https://visionagents.ai/integrations/wizper

This command installs the Vision Agents SDK with the Wizper plugin, enabling Speech-to-Text and translation capabilities. Ensure you have a package manager like `uv` installed.

```sh
uv add vision-agents[wizper]
```

--------------------------------

### Setup Metrics Instrumentation (Python)

Source: https://visionagents.ai/core/telemetry

Configures OpenTelemetry metrics by setting up a MeterProvider with a PrometheusMetricReader and starting an HTTP server to expose metrics. The Python program must remain running for Prometheus to scrape the metrics endpoint.

```python
from opentelemetry import metrics
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.exporter.prometheus import PrometheusMetricReader
from prometheus_client import start_http_server

resource = Resource.create(
    {
        "service.name": "my-service-name",
    }
)

reader = PrometheusMetricReader()
metrics.set_meter_provider(
    MeterProvider(resource=resource, metric_readers=[reader])
)

start_http_server(port=9464)
```

--------------------------------

### Install ElevenLabs Plugin

Source: https://visionagents.ai/integrations/elevenlabs

Command to install the Stream ElevenLabs plugin using `uv`. This command adds the necessary dependencies for ElevenLabs integration.

```sh
uv add vision-agents[elevenlabs]
```

--------------------------------

### Install Vision Agents with xAI Support

Source: https://visionagents.ai/integrations/xai

Install the Vision Agents library with the xAI plugin using the provided shell command. This command fetches and installs the necessary packages for integrating xAI's Grok models.

```sh
uv add vision-agents[xai]
```

--------------------------------

### Install Vision Agents with Gemini Support (Bash)

Source: https://visionagents.ai/introduction/voice-agents

Installs the Vision Agents library with necessary dependencies for Gemini integration and Python dotenv for environment variable management. Requires Python 3.12+.

```bash
uv init

uv add "vision-agents[getstream, gemini]" python-dotenv
```

--------------------------------

### Plugin Testing with pytest

Source: https://visionagents.ai/integrations/create-your-own-plugin

Provides bash commands to install dependencies, set up a virtual environment using 'uv', and run the test suite with 'pytest'. This ensures the plugin functions correctly.

```bash
# Install dependencies and create virtual environment
uv sync

# Run tests
uv run pytest -v
```

--------------------------------

### Install Prometheus Exporter (Bash)

Source: https://visionagents.ai/core/telemetry

Installs the OpenTelemetry Prometheus exporter and the prometheus-client library. These are required for exposing metrics to Prometheus.

```bash
# with uv:
uv install opentelemetry-exporter-prometheus prometheus-client

# or with pip:
pip install opentelemetry-exporter-prometheus prometheus-client
```

--------------------------------

### Import Core Packages for Vision Agent

Source: https://visionagents.ai/introduction/video-agents

Imports essential libraries and classes for building a Vision Agent in Python. It includes logging, environment variable loading, and components from 'vision_agents.core' and 'vision_agents.plugins'. These are foundational for agent setup and execution.

```python
import logging

from dotenv import load_dotenv

from vision_agents.core import User, Agent, cli
from vision_agents.core.agents import AgentLauncher
from vision_agents.plugins import getstream, openai

logger = logging.getLogger(__name__)

load_dotenv()
```

--------------------------------

### Python YOLOPoseProcessor Setup

Source: https://visionagents.ai/guides/video-processors

Sets up an Agent with the YOLOPoseProcessor for real-time pose detection and annotation of video frames. It integrates with other components like Edge, User, and LLM plugins.

```python
from vision_agents.core import Agent, User
from vision_agents.plugins import getstream, gemini, ultralytics

# In your agent setup
agent = Agent(
    edge=getstream.Edge(),
    agent_user=User(name="AI golf coach"),
    instructions="Read @golf_coach.md",
    llm=gemini.Realtime(fps=10),
    processors=[
        ultralytics.YOLOPoseProcessor(
            model_path="yolo11n-pose.pt",
            conf_threshold=0.5,
            enable_hand_tracking=True
        )
    ],
)

```

--------------------------------

### Install Smart Turn Plugin

Source: https://visionagents.ai/integrations/smart-turn

Installs the Smart Turn plugin for Vision Agents using pip. This command adds the necessary packages for the smart_turn functionality.

```sh
uv add vision-agents[smart_turn]
```

--------------------------------

### Install Fish Audio Plugin for Vision Agents

Source: https://visionagents.ai/integrations/fish

Installs the Fish Audio plugin for Vision Agents using the `uv` package manager. This command adds the necessary dependencies for the Fish Audio integration.

```sh
uv add vision-agents[fish]
```

--------------------------------

### Install Gemini Live Plugin

Source: https://visionagents.ai/integrations/gemini

Command to install the Gemini Live plugin for Vision Agents using `uv`. This command adds the necessary dependencies for Gemini integration.

```shell
uv add vision-agents[gemini]
```

--------------------------------

### Install Dependencies for Vision Agents with OpenAI, Cartesia, Deepgram

Source: https://visionagents.ai/introduction/voice-agents

Installs the necessary plugins for using OpenAI, Cartesia, and Deepgram with the vision-agents framework. This command updates the project's dependencies to include support for these services, enabling custom LLM, TTS, and STT functionalities.

```bash
uv add "vision-agents[openai, cartesia, deepgram]"
```

--------------------------------

### Configure Gemini Live Connection

Source: https://visionagents.ai/integrations/gemini

Demonstrates how to configure the Gemini Live connection using `LiveConnectConfigDict`. This example specifies response modalities and speech configuration, such as language code.

```python
from google.genai.types import LiveConnectConfigDict, Modality, SpeechConfigDict

config = LiveConnectConfigDict(
    response_modalities=[Modality.AUDIO],
    speech_config=SpeechConfigDict(
        language_code="en-US",
    ),
)

realtime = gemini.Realtime(config=config)
```

--------------------------------

### Real-World Example: Agent with MCP Server and Custom Functions (Python)

Source: https://visionagents.ai/guides/mcp-tool-calling

A comprehensive example demonstrating the integration of an MCP server with custom LLM functions. This code sets up an `MCPServerRemote` for GitHub, defines a custom function `get_current_time` registered with the LLM, and then initializes an `Agent` with both the server and the LLM. Finally, it showcases using the agent for a complex query involving external services and custom logic.

```python
import asyncio
from vision_agents.core.agents import Agent
from vision_agents.core.mcp import MCPServerRemote
from vision_agents.plugins import openai, getstream
from vision_agents.core.edge.types import User
import os

async def main():
    # Create MCP servers
    github_server = MCPServerRemote(
        url="https://api.githubcopilot.com/mcp/",
        headers={"Authorization": f"Bearer {os.getenv('GITHUB_PAT')}"},
        timeout=10.0,
        session_timeout=300.0
    )
    
    # Create LLM with custom functions
    llm = openai.LLM(model="gpt-4o-mini")
    
    @llm.register_function(description="Get current time")
    def get_current_time() -> str:
        from datetime import datetime
        return datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    
    # Create agent
    agent = Agent(
        edge=getstream.Edge(),
        llm=llm,
        agent_user=User(name="GitHub Assistant", id="github-assistant"),
        instructions="You are a helpful GitHub assistant with access to repositories, issues, and more.",
        mcp_servers=[github_server]
    )
    
    # Use the agent
    response = await agent.simple_response(
        "What repositories do I have and what time is it?"
    )
    print(response.text)

if __name__ == "__main__":
    asyncio.run(main())
```

--------------------------------

### xAI LLM Streaming Responses Example

Source: https://visionagents.ai/integrations/xai

Illustrates how to enable streaming responses from the xAI LLM plugin for real-time text generation. This is useful for applications requiring immediate feedback.

```python
response = await llm.create_response(
    input="Tell me about quantum computing",
    instructions="You are a helpful science educator.",
    stream=True
)
```

--------------------------------

### Create Agent with Fast-Whisper Speech-to-Text

Source: https://visionagents.ai/integrations/fast-whisper

Example of creating a Vision Agent instance configured with the Fast-Whisper STT plugin. This demonstrates initializing an agent with specific STT, LLM, and TTS plugins.

```python
from vision_agents.plugins import fast_whisper
from vision_agents.core import Agent, User
from vision_agents.plugins import getstream, openai, elevenlabs

# Create agent with Fast-Whisper STT
agent = Agent(
    edge=getstream.Edge(),
    agent_user=User(name="AI Assistant", id="agent"),
    instructions="You are a helpful voice assistant.",
    stt=fast_whisper.STT(model_size="base"),
    llm=openai.LLM(model="gpt-4o-mini"),
    tts=elevenlabs.TTS()
)
```

--------------------------------

### xAI LLM Conversation Memory Example

Source: https://visionagents.ai/integrations/xai

Demonstrates the conversation memory feature of the xAI plugin. The LLM can retain context from previous messages, allowing for natural multi-turn conversations.

```python
llm = xai.LLM(model="grok-4.1")

# First message
await llm.simple_response("My name is Alice and I have 2 cats")

# Second message - the LLM remembers the context
response = await llm.simple_response("How many pets do I have?")
print(response.text)  # Will mention the 2 cats
```

--------------------------------

### Focused Event Handlers in VisionAgents (Python)

Source: https://visionagents.ai/guides/event-system

This example highlights the best practice of keeping VisionAgents event handlers focused on a single responsibility. It contrasts a good example of a handler for a specific event with a bad example of a handler attempting to manage multiple event types, emphasizing modularity and maintainability.

```python
# Good: Single responsibility
@agent.events.subscribe
async def handle_participant_joined(event: CallSessionParticipantJoinedEvent):
    await agent.simple_response(f"Hello {event.participant.user.name}!")

# Avoid: Multiple responsibilities
@agent.events.subscribe
async def handle_everything(event):
    # Don't handle all events in one function
    pass

```

--------------------------------

### Subscribe to Call Session Events in Python

Source: https://visionagents.ai/guides/event-system

Handles events related to the start and end of call sessions. It logs messages when a session starts or ends and can optionally send a response when the session begins. No response is sent when the session ends.

```python
from vision_agents.core.events import CallSessionStartedEvent, CallSessionEndedEvent

# Subscribe to call session started events
@agent.events.subscribe
async def handle_session_started(event: CallSessionStartedEvent):
    logger.info("Call session started")
    await agent.simple_response("The call has started. I'm here to help!")

# Subscribe to call session ended events
@agent.events.subscribe
async def handle_session_ended(event: CallSessionEndedEvent):
    logger.info("Call session ended")
    # No response needed as the call is ending
```

--------------------------------

### Complete Realtime Agent Example

Source: https://visionagents.ai/core/realtime-core

This comprehensive Python script sets up a realtime voice assistant using the `Agent` framework. It initializes an `openai.Realtime` LLM, configures the agent, sets up an event handler for participant joins, creates and joins a call, and manages the agent's lifecycle.

```python
import asyncio
from vision_agents.core.agents import Agent
from vision_agents.plugins import openai, getstream
from vision_agents.core.edge.types import User
from vision_agents.core.events import CallSessionParticipantJoinedEvent

async def main():
    # Create Realtime LLM
    llm = openai.Realtime(
        model="gpt-realtime",
        voice="marin"
    )

    # Create agent with Realtime LLM
    agent = Agent(
        edge=getstream.Edge(),
        agent_user=User(name="AI Assistant", id="agent"),
        instructions="You're a helpful voice assistant. Keep responses conversational and natural.",
        llm=llm,
        processors=[]
    )

    # Set up event handlers
    @agent.subscribe
    async def on_participant_joined(event: CallSessionParticipantJoinedEvent):
        await agent.simple_response(f"Hello {event.participant.user.name}!")

    # Create and join call
    await agent.create_user()
    call = agent.edge.client.video.call("default", "realtime-demo")
    await agent.edge.open_demo(call)

    with await agent.join(call):
        await agent.finish()

    await agent.close()

if __name__ == "__main__":
    asyncio.run(main())
```

--------------------------------

### Run Jaeger Locally (Bash)

Source: https://visionagents.ai/core/telemetry

Starts a Jaeger all-in-one instance using Docker, enabling OTLP collection on port 4317. This allows for easy local verification of the tracing setup.

```bash
docker run --rm -it \
         -e COLLECTOR_OTLP_ENABLED=true \
         -p 16686:16686 -p 4317:4317 -p 4318:4318 \
         jaegertracing/all-in-one:1.51
```

--------------------------------

### Configure OpenAI Realtime LLM with Instructions

Source: https://visionagents.ai/core/realtime-core

Shows an example of configuring the OpenAI `Realtime` LLM with additional parameters, including `model`, `voice`, `fps`, and custom `instructions`. This allows for fine-tuning the AI's behavior and response style for specific use cases. It's useful for tailoring the AI assistant's persona and capabilities.

```python
llm = Realtime(
    model="gpt-realtime",
    voice="marin",
    fps=1,  # Video frames per second
    instructions="You are a helpful assistant"
)
```

--------------------------------

### Create GitHub MCP Server and Agent with Vision Agents

Source: https://visionagents.ai/introduction/voice-agents

This snippet demonstrates how to initialize an MCPServerRemote for GitHub Copilot and configure an Agent with specific instructions and MCP servers. It requires the 'MCPServerRemote' and 'Agent' classes, along with 'edge', 'llm', and 'agent_user' objects. The agent is configured to interact with GitHub via the MCP server.

```python
github_server = MCPServerRemote(
    url="https://api.githubcopilot.com/mcp/",
    headers={"Authorization": f"Bearer {github_pat}"},
    timeout=10.0,  # Shorter connection timeout
    session_timeout=300.0
)

agent = Agent(
    edge=edge,
    llm=llm,
    agent_user=agent_user,
    instructions="You are a helpful AI assistant with access to GitHub via MCP server. You can help with GitHub operations like creating issues, managing pull requests, searching repositories, and more. Keep responses conversational and helpful.",
    mcp_servers=[github_server],
)
```

--------------------------------

### Fast-Whisper GPU Acceleration Configuration

Source: https://visionagents.ai/integrations/fast-whisper

Configuration for enabling GPU acceleration with Fast-Whisper STT. This example sets the device to 'cuda' and compute type to 'float16' for optimal GPU performance. CUDA installation is required.

```python
stt = fast_whisper.STT(
    model_size="medium",
    device="cuda",
    compute_type="float16"  # Use float16 for GPU
)
```

--------------------------------

### Set xAI API Key Environment Variable (Bash)

Source: https://visionagents.ai/integrations/xai

Provides the command to set the `XAI_API_KEY` environment variable, which is necessary for authenticating with the xAI service. This is a crucial step for getting started with the xAI plugin.

```bash
export XAI_API_KEY="your_api_key_here"
```

--------------------------------

### Use @mention for Agent Instructions

Source: https://visionagents.ai/introduction/video-agents

Demonstrates an alternative method for providing agent instructions by referencing an external Markdown file using the '@mention' syntax within the instructions string. This is useful for managing longer or more complex instructions separate from the main code.

```python
instructions="Read @voice-agent-instructions.md"
```

--------------------------------

### Python: Subscribe to Turn Started Event

Source: https://visionagents.ai/integrations/vogent

Provides an example of subscribing to the `TurnStartedEvent` emitted by Vogent. This function is triggered when the system detects a user has begun speaking. It demonstrates accessing participant information and confidence level.

```python
from vision_agents.core.turn_detection.events import TurnStartedEvent

@turn_detection.events.subscribe
async def on_turn_started(event: TurnStartedEvent):
    print(f"Turn started by {event.participant.user_id}")
    print(f"Confidence: {event.confidence}")
```

--------------------------------

### Initialize Vision Agent with xAI Grok LLM

Source: https://visionagents.ai/integrations/xai

Create a Vision Agent instance configured to use xAI's Grok model. This example shows how to set up the agent with a specific Grok model, speech-to-text, text-to-speech, and edge processing.

```python
from vision_agents.plugins import xai
from vision_agents.core import Agent, User
from vision_agents.plugins import getstream, deepgram, elevenlabs

# Create agent with Grok
agent = Agent(
    edge=getstream.Edge(),
    agent_user=User(name="AI Assistant", id="agent"),
    instructions="You are a helpful assistant with access to real-time information.",
    llm=xai.LLM(model="grok-4.1"),
    stt=deepgram.STT(),
    tts=elevenlabs.TTS()
)
```

--------------------------------

### Join Call and Use OpenAI's create_response Method

Source: https://visionagents.ai/introduction/voice-agents

Demonstrates how to join a call and utilize OpenAI's `create_response` method for advanced LLM interactions within the agent. This function handles agent user creation, call setup, and initiates the agent's participation in the call, including sending a text and image prompt to the LLM.

```python
async def join_call(agent: Agent, call_type: str, call_id: str, **kwargs) -> None:
    """Join the call and start the agent."""
    # Ensure the agent user is created
    await agent.create_user()
    # Create a call
    call = await agent.create_call(call_type, call_id)

    logger.info("🤖 Starting OpenAI Agent...")

    # Have the agent join the call/room
    with await agent.join(call):
        logger.info("Joining call")

        await agent.edge.open_demo(call)
        logger.info("LLM ready")

        # Example: use native openAI create response
        await agent.llm.create_response(input=[
            {
                "role": "user",
                "content": [
                    {"type": "input_text", "text": "Tell me a short poem about this image"},
                    {"type": "input_image", "image_url": f"https://images.unsplash.com/photo-1757495361144-0c2bfba62b9e?q=80&w=2340&auto=format&fit=crop&ixlib=rb-4.1.0&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D"},
                ],
            }
        ],)

        # run till the call ends
        await agent.finish()
```

--------------------------------

### Add Event Filtering in Python

Source: https://visionagents.ai/guides/event-system

Implements event filtering to process only specific events that meet certain criteria. This example shows how to handle `STTTranscriptEvent` only if the confidence level is above 0.8 and includes custom logic to respond if the transcript contains the word 'help'.

```python
from vision_agents.core.events import STTTranscriptEvent

@agent.events.subscribe
async def handle_transcript(event: STTTranscriptEvent):
    # Only respond to transcripts with high confidence
    if event.response and event.response.confidence > 0.8:
        logger.info(f"High confidence transcript: {event.text}")
        
        # Add custom logic based on transcript content
        if "help" in event.text.lower():
            await agent.simple_response("I'm here to help! What do you need?")
```

--------------------------------

### LLM Agent Initialization (Python)

Source: https://visionagents.ai/core/llm-core

Demonstrates initializing an Agent with an LLM, including traditional and real-time modes. The traditional mode requires separate STT and TTS services, while the real-time mode uses a dedicated real-time LLM.

```python
from agent import Agent
from providers import openai, deepgram, elevenlabs

# Traditional mode
agent = Agent(
    llm=openai.LLM(model="gpt-4o-mini"),
    stt=deepgram.STT(),
    tts=elevenlabs.TTS()
)

# Realtime mode
agent = Agent(
    llm=openai.Realtime(model="gpt-4o-realtime-preview")
)
```

--------------------------------

### OpenAI Realtime Agent Initialization and Usage

Source: https://visionagents.ai/integrations/openai

Demonstrates how to initialize an Agent with the OpenAI Realtime LLM integration. It sets up a Stream client, creates a user, configures the agent with instructions and the OpenAI Realtime model, and joins a call to interact with the user.

```python
from vision_agents.plugins import openai, getstream
from vision_agents.core.agents import Agent
from getstream import AsyncStream

# Create Stream client and user
client = AsyncStream()
agent_user = await client.create_user(name="AI Assistant")

# Create agent with OpenAI Realtime
agent = Agent(
    edge=getstream.Edge(),
    agent_user=agent_user,
    instructions="You are a helpful voice assistant.",
    llm=openai.Realtime(model="gpt-realtime", voice="marin", fps=1),
    processors=[]
)

# Create and join a call
call = client.video.call("default", call_id)
await call.get_or_create(data={"created_by_id": agent.agent_user.id})

with await agent.join(call):
    # Wait for LLM to be ready
    await agent.llm.simple_response(text="Please greet the user.")
    # Keep running until call ends
    await agent.finish()
```

--------------------------------

### Initialize and Use Smart Turn TurnDetection in Python

Source: https://visionagents.ai/integrations/smart-turn

Demonstrates initializing the TurnDetection class with custom settings and integrating it with a Vision Agents Agent. It also shows how to use it standalone and subscribe to turn events.

```python
from vision_agents.core import Agent
from vision_agents.plugins import smart_turn

# Create turn detection with custom settings
turn_detection = smart_turn.TurnDetection(
    buffer_in_seconds=2.0,
    confidence_threshold=0.5
)

# Use with an agent
agent = Agent(
    turn_detection=turn_detection,
    # ... other agent configuration
)

# Or use standalone
await turn_detection.start()

# Listen for turn events
@turn_detection.events.subscribe
async def on_turn_started(event: smart_turn.TurnStartedEvent):
    print(f"User {event.participant.user_id} started speaking")

@turn_detection.events.subscribe
async def on_turn_ended(event: smart_turn.TurnEndedEvent):
    print(f"User {event.participant.user_id} finished speaking")
    print(f"Confidence: {event.confidence}")

# Stop when finished
await turn_detection.stop()
```

--------------------------------

### Vision Agent with Participant Joined Event Handling in Python

Source: https://visionagents.ai/guides/event-system

Enhances a basic Vision Agent to greet new participants joining a call. It subscribes to the `CallSessionParticipantJoinedEvent` and uses `agent.simple_response` to deliver a welcome message. Prerequisites include the basic agent setup and event handling libraries.

```python
import asyncio
import logging
from uuid import uuid4

from dotenv import load_dotenv
from vision_agents.core.edge.types import User
from vision_agents.plugins import elevenlabs, deepgram, openai, getstream
from vision_agents.core import agents
from vision_agents.core.events import CallSessionParticipantJoinedEvent

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
load_dotenv()

async def start_agent():
    call_id = str(uuid4())
    
    # Create the agent with basic components
    agent = agents.Agent(
        edge=getstream.Edge(),
        agent_user=User(name="My AI Assistant", id="agent"),
        instructions="You're a helpful voice AI assistant. Keep responses short and conversational.",
        llm=openai.LLM(model="gpt-4o-mini"),
        tts=elevenlabs.TTS(),
        stt=deepgram.STT(),
    )
    
    await agent.create_user()
    
    # Subscribe to participant joined events
    @agent.events.subscribe
    async def handle_participant_joined(event: CallSessionParticipantJoinedEvent):
        # Skip if the participant joining is the agent itself
        if event.participant.user.id == "agent":
            return
        
        # Log the event
        logger.info(f"New participant joined: {event.participant.user.name}")
        
        # Greet the new participant
        await agent.simple_response(f"Hello {event.participant.user.name}! Welcome to the call.")
    
    # Create and join the call
    call = agent.edge.client.video.call("default", call_id)
    await agent.edge.open_demo(call)
    
    with await agent.join(call):
        await agent.finish()

if __name__ == "__main__":
    asyncio.run(start_agent())
```

--------------------------------

### Initialize OpenAI Realtime LLM

Source: https://visionagents.ai/core/realtime-core

Shows how to initialize the `Realtime` LLM specifically for the OpenAI provider. It includes importing the `Realtime` class from `vision_agents.plugins.openai` and setting the `model`, `voice`, and `fps` parameters. This is useful for configuring OpenAI-specific real-time audio interactions.

```python
from vision_agents.plugins.openai import Realtime

llm = Realtime(
    model="gpt-realtime",
    voice="marin",
    fps=1
)
```

--------------------------------

### Install Vision Agents with uv (Shell)

Source: https://github.com/GetStream/vision-agents

This command installs the Vision Agents package using the `uv` package manager. `uv` is a fast Python package installer and resolver. Ensure `uv` is installed in your environment before running this command.

```bash
uv add visi
```

--------------------------------

### Install Cartesia Plugin with vision-agents

Source: https://visionagents.ai/integrations/cartesia

This command installs the Cartesia plugin as part of the vision-agents package. Ensure you have 'uv' installed for package management. This command adds the necessary dependencies for Cartesia functionality.

```shell
uv add vision-agents[cartesia]
```

--------------------------------

### Initialize Agent with OpenAI TTS

Source: https://visionagents.ai/integrations/openai

Demonstrates setting up an Agent with OpenAI's Text-to-Speech (TTS) capabilities. This involves creating a Stream client and user, then initializing the Agent with a specified LLM and the `openai.TTS` plugin, including model and voice selection.

```python
from vision_agents.plugins import openai, getstream
from vision_agents.core.agents import Agent
from getstream import AsyncStream

# Create Stream client and user
client = AsyncStream()
agent_user = await client.create_user(name="AI Assistant")

# Assuming 'your_llm' is an already configured LLM object
# your_llm = ... 

# Create agent with OpenAI TTS
agent = Agent(
    edge=getstream.Edge(),
    agent_user=agent_user,
    instructions="You are a helpful assistant.",
    llm=your_llm,  # Any LLM that outputs text
    tts=openai.TTS(model="gpt-4o-mini-tts", voice="alloy"),
)
```

--------------------------------

### Initialize Realtime LLM for OpenAI and Gemini

Source: https://visionagents.ai/core/realtime-core

Demonstrates how to initialize the Realtime LLM for both OpenAI and Google Gemini providers. It shows the necessary imports and instantiation for each, along with their respective model and voice/config parameters. This snippet is useful for setting up real-time communication endpoints.

```python
from vision_agents.plugins import openai, gemini
from vision_agents.core.agents import Agent
from vision_agents.core.edge.types import User

# OpenAI Realtime
llm = openai.Realtime(model="gpt-realtime", voice="marin")

# Gemini Live
llm = gemini.Realtime(model="gemini-2.5-flash-native-audio-preview-09-2025")

# Use with Agent
agent = Agent(
    edge=getstream.Edge(),
    agent_user=User(name="AI Assistant", id="agent"),
    instructions="You're a helpful voice assistant",
    llm=llm,  # Realtime LLM replaces STT/TTS
    processors=[]
)
```

--------------------------------

### Install Vision Agents with Extra Integrations (Python)

Source: https://github.com/GetStream/vision-agents

Installs the Vision Agents library with specified extra integrations such as getstream, openai, elevenlabs, and deepgram. This command allows for a customized installation tailored to the specific needs of the project.

```python
uv add "vision-agents[getstream, openai, elevenlabs, deepgram]"
```

--------------------------------

### Configure Agent with Processors

Source: https://visionagents.ai/guides/video-processors

Demonstrates how to configure an Agent with various processors like ImageCapture and AudioLogger. It utilizes imports from vision_agents.core and vision_agents.plugins.

```python
from vision_agents.core import Agent, User
from vision_agents.core.processors import AudioLogger, ImageCapture
from vision_agents.plugins import getstream, openai

agent = Agent(
    edge=getstream.Edge(),
    agent_user=User(name="AI Assistant"),
    instructions="You're a helpful assistant.",
    llm=openai.LLM(model="gpt-4o-mini"),
    processors=[
        ImageCapture(output_dir="./captures", interval=5),
        AudioLogger(interval=2),
    ],
)
```

--------------------------------

### Campaign Started Event

Source: https://visionagents.ai/reference/events-reference

Indicates the start of a campaign. Includes creation timestamp, custom data, and optional campaign details.

```APIDOC
## CampaignStartedEvent

### Description
This event signifies the initiation of a campaign. It includes the timestamp of creation, custom data associated with the event, and optionally, details about the campaign that has started.

### Event Type
`campaign.started`

### Parameters
- **created_at** (datetime.datetime) - Timestamp when the event occurred.
- **custom** (Dict[str, object]) - Custom data associated with the event.
- **type** (str) - The event type, defaults to 'campaign.started'.
- **received_at** (Optional[datetime.datetime]) - Timestamp when the event was received.
- **campaign** ('Optional[CampaignResponse]') - An object containing details about the started campaign, if available.
```

--------------------------------

### Join Call and Start Agent with Gemini (Python)

Source: https://visionagents.ai/introduction/voice-agents

Handles joining a call and initiating the Gemini Realtime Agent. This function creates the agent user, sets up the call, opens the demo, interacts with the LLM using simple_response, and ensures the agent finishes gracefully.

```python
async def join_call(agent: Agent, call_type: str, call_id: str, **kwargs) -> None:
    """Join the call and start the agent."""
    # Ensure the agent user is created
    await agent.create_user()
    # Create a call
    call = await agent.create_call(call_type, call_id)

    logger.info("🤖 Starting Gemini Realtime Agent...")

    # Have the agent join the call/room
    with await agent.join(call):
        logger.info("Joining call")

        await agent.edge.open_demo(call)
        logger.info("LLM ready")

        # Example 1: standardized simple response
        await agent.llm.simple_response("chat with the user about the weather.")
        
        # run till the call ends
        await agent.finish()


if __name__ == "__main__":
    cli(AgentLauncher(create_agent=create_agent, join_call=join_call))
```

--------------------------------

### Install Decart Plugin

Source: https://visionagents.ai/integrations/decart

Command to install the Decart plugin for Vision Agents using uv. This is a prerequisite for using the plugin's functionalities.

```sh
uv add vision-agents-plugins-decart
```

--------------------------------

### Add Dependencies for Vision Agents with OpenAI

Source: https://visionagents.ai/introduction/video-agents

Installs the necessary packages for building a Vision Agent with Stream and OpenAI integration. It assumes a Python 3.12+ project and uses 'uv' as the package manager. The command adds 'vision-agents' with 'getstream' and 'openai' extras.

```bash
uv init

uv add "vision-agents[getstream, openai]"
```

--------------------------------

### Call Session Started Event

Source: https://visionagents.ai/reference/events-reference

Indicates the start of a new call session. It contains the call ID, creation timestamp, and session ID.

```APIDOC
## CallSessionStartedEvent

### Description
This event signifies the initiation of a new call session. It includes the unique call identifier, the timestamp of creation, and the specific session ID.

### Event Type
`call.session_started`

### Parameters
- **call_cid** (str) - Unique identifier for the call.
- **created_at** (datetime.datetime) - Timestamp when the event occurred.
- **session_id** (str) - Identifier for the call session.
- **call** ('CallResponse') - An object containing details about the call that started.
```

--------------------------------

### Configure API Keys for Deepgram, Cartesia, OpenAI

Source: https://visionagents.ai/introduction/voice-agents

Sets up the necessary API credentials for Deepgram, Cartesia, and OpenAI in the .env file. These keys are essential for authenticating with the respective services to utilize their TTS, STT, and LLM capabilities within the agent framework.

```env
# Deepgram API credentials
DEEPGRAM_API_KEY=
# Cartesia API credentials
CARTESIA_API_KEY=
# OpenAI API credentials
OPENAI_API_KEY=
```

--------------------------------

### Install Fast-Whisper Plugin for Vision Agents

Source: https://visionagents.ai/integrations/fast-whisper

Command to install the Fast-Whisper plugin for Vision Agents using 'uv'. This command ensures that the necessary dependencies for Fast-Whisper are included.

```sh
uv add vision-agents[fast-whisper]
```

--------------------------------

### Initialize Agent with MCP Servers

Source: https://visionagents.ai/core/agent-core

Demonstrates how to initialize an agent and include MCP servers in its configuration. MCP tools are automatically registered with the LLM's function registry.

```python
agent = agents.Agent(
    # ... other parameters
    mcp_servers=[github_server]
)
```

--------------------------------

### Install Deepgram Plugin

Source: https://visionagents.ai/integrations/deepgram

Installs the Deepgram plugin for the Vision Agents SDK. This command adds the necessary dependencies to integrate Deepgram's Speech-to-Text capabilities.

```sh
uv add vision-agents[deepgram]
```

--------------------------------

### Install Kokoro TTS Plugin

Source: https://visionagents.ai/integrations/kokoro

Installs the Kokoro plugin for Stream Agents. This command adds the necessary dependencies to your project, enabling the use of the Kokoro TTS engine.

```shell
uv add tream-agents[kokoro]
```

--------------------------------

### Import Core Modules for Voice Agent (Python)

Source: https://visionagents.ai/introduction/voice-agents

Imports essential modules for building a voice agent, including logging, environment variable loading, core Vision Agents components (User, Agent, cli), AgentLauncher, and specific plugins for Getstream and Gemini.

```python
import logging

from dotenv import load_dotenv

from vision_agents.core import User, Agent, cli
from vision_agents.core.agents import AgentLauncher
from vision_agents.plugins import getstream, gemini

logger = logging.getLogger(__name__)

load_dotenv()
```

--------------------------------

### Install Vogent Plugin

Source: https://visionagents.ai/integrations/vogent

Installs the Vogent plugin for the Vision Agents SDK. This command uses 'uv' to add the necessary package, enabling Vogent's turn detection capabilities.

```sh
uv add vision-agents[vogent]
```

--------------------------------

### Install AWS Polly Plugin for Stream AI SDK

Source: https://visionagents.ai/integrations/aws-polly

Installs the Stream AWS plugin using pip. This command adds the necessary dependencies for AWS integration, including Polly.

```sh
uv add vision-agents[aws]
```