### Clone and Install Dependencies Source: https://docs.livekit.io/agents/multimodality/audio/wakeword.md Clone the hello-wakeword example repository and install its dependencies using uv. ```shell git clone https://github.com/livekit-examples/hello-wakeword cd hello-wakeword uv sync --all-packages ``` -------------------------------- ### Initialize and Start Anam Avatar Session in Python Source: https://docs.livekit.io/agents/models/avatar/plugins/anam.md Example of initializing an AgentServer and an AvatarSession with persona configuration, then starting both the avatar and the agent session. ```python from livekit import agents from livekit.agents import AgentServer, AgentSession from livekit.plugins import anam server = AgentServer() @server.rtc_session(agent_name="my-agent") async def my_agent(ctx: agents.JobContext): session = AgentSession( # ... stt, llm, tts, etc. ) avatar = anam.AvatarSession( persona_config=anam.PersonaConfig( name="...", # Name of the avatar to use. avatarId="...", # ID of the avatar to use. See "Avatar setup" for details. ), ) # Start the avatar and wait for it to join await avatar.start(session, room=ctx.room) # Start your agent session with the user await session.start( # ... room, agent, room_options, etc.... ) ``` -------------------------------- ### Initialize and Start LiveAvatar Session Source: https://docs.livekit.io/agents/models/avatar/plugins/liveavatar.md Example of initializing an AgentServer and an AgentSession, then configuring and starting a LiveAvatar session within the agent. This requires setting up STT, LLM, and TTS components for the AgentSession. ```python from livekit import agents from livekit.agents import AgentServer, AgentSession from livekit.plugins import liveavatar server = AgentServer() @server.rtc_session(agent_name="my-agent") async def my_agent(ctx: agents.JobContext): session = AgentSession( # ... stt, llm, tts, etc. ) avatar = liveavatar.AvatarSession( avatar_id="...", # ID of the LiveAvatar avatar to use ) # Start the avatar and wait for it to join await avatar.start(session, room=ctx.room) # Start your agent session with the user await session.start( # ... room, agent, room_options, etc.... ) ``` -------------------------------- ### Initialize and Start D-ID Avatar Session (Python) Source: https://docs.livekit.io/agents/models/avatar/plugins/did.md Example of initializing an AgentSession and a D-ID AvatarSession, then starting both in Python. Ensure D-ID API key is set in .env. ```python from livekit import agents from livekit.agents import AgentServer, AgentSession from livekit.plugins import did server = AgentServer() @server.rtc_session(agent_name="my-agent") async def my_agent(ctx: agents.JobContext): session = AgentSession( # ... stt, llm, tts, etc. ) avatar = did.AvatarSession( agent_id="...", # ID of the D-ID avatar to use. See "Avatar setup" for details. ) # Start the avatar and wait for it to join await avatar.start(session, room=ctx.room) # Start your agent session with the user await session.start( # ... room, agent, room_options, etc.... ) ``` -------------------------------- ### Initialize AgentSession with Simplismart STT Source: https://docs.livekit.io/agents/models/stt/simplismart.md Initialize an AgentSession using the Simplismart STT plugin. This example shows basic setup; other components like VAD, LLM, and TTS would be configured separately. ```python from livekit.plugins import simplismart session = AgentSession( stt = simplismart.STT(), # ... vad, llm, tts, etc. ) ``` -------------------------------- ### Start AgentSession with Local bitHuman Model Source: https://docs.livekit.io/agents/models/avatar/plugins/bithuman.md Integrate a local bitHuman .imx model into an AgentSession. This example shows how to start the avatar and then the agent session. ```python from livekit.plugins import bithuman session = AgentSession( # ... stt, llm, tts, etc. ) avatar = bithuman.AvatarSession( model_path="./albert_einstein.imx", # This example uses a demo model installed in the current directory ) # Start the avatar and wait for it to join await avatar.start(session, room=ctx.room) # Start your agent session with the user await session.start( room=ctx.room, ) ``` -------------------------------- ### Initialize and Start D-ID Avatar Session (Node.js) Source: https://docs.livekit.io/agents/models/avatar/plugins/did.md Example of initializing an AgentSession and a D-ID AvatarSession, then starting both in Node.js. Ensure D-ID API key is set in .env. ```typescript import { voice } from '@livekit.io/agents'; import * as did from '@livekit.io/agents-plugin-did'; const session = new voice.AgentSession({ // ... stt, llm, tts, etc. }); const avatar = new did.AvatarSession({ agentId: "...", // ID of the D-ID avatar to use. See "Avatar setup" for details. }); // Start the avatar and wait for it to join await avatar.start(session, room); // Start your agent session with the user await session.start( // ... room, agent, room_options, etc. ); ``` -------------------------------- ### Install LiveKit CLI from Source Source: https://docs.livekit.io/agents/integrations/google.md Clone the LiveKit CLI repository and install it from source using make. Ensure Git LFS is installed. ```shell git clone github.com/livekit/livekit-cli make install ``` -------------------------------- ### Initialize AgentSession with Sarvam LLM Source: https://docs.livekit.io/agents/models/llm/sarvam.md Initialize an AgentSession using Sarvam as the LLM provider. This example shows basic setup; other components like TTS, STT, and VAD can be configured as needed. ```python from livekit.plugins import sarvam session = AgentSession( llm=sarvam.LLM( model="sarvam-30b", ), # ... tts, stt, vad, turn_handling, etc. ) ``` -------------------------------- ### Start Agent in Development Mode Source: https://docs.livekit.io/agents/server/startup-modes.md Execute this command to start the agent server in development mode. ```shell pnpm dev ``` -------------------------------- ### Install Simplismart Plugin Source: https://docs.livekit.io/agents/models/stt/plugins/simplismart Install the Simplismart plugin for LiveKit Agents using pip. ```bash uv add "livekit-agents[simplismart]~=1.5" ``` -------------------------------- ### Initialize AgentSession with Soniox TTS Source: https://docs.livekit.io/agents/models/tts/plugins/soniox Initialize an AgentSession with Soniox as the Text-to-Speech provider. This example shows basic setup; other configurations like LLM and STT are omitted. ```python from livekit.plugins import soniox session = AgentSession( tts=soniox.TTS( ), # ... llm, stt, etc. ) ``` -------------------------------- ### Install Groq Plugin for Node.js Source: https://docs.livekit.io/agents/models/llm/plugins/groq Install the Groq plugin for LiveKit Agents in Node.js using pnpm. ```bash pnpm add @livekit.agents-plugin-openai@1.x ``` -------------------------------- ### Install Gradium Plugin Source: https://docs.livekit.io/agents/models/stt/gradium.md Install the livekit-agents package with Gradium support using uv. ```shell uv add "livekit-agents[gradium]~=1.5" ``` -------------------------------- ### Install Gemini Plugin for Node.js Source: https://docs.livekit.io/agents/models/realtime/plugins/gemini.md Install the Google plugin for LiveKit Agents using pnpm. ```shell pnpm add "@livekit/agents-plugin-google@1.x" ``` -------------------------------- ### Install Fish Audio Plugin Source: https://docs.livekit.io/agents/models/tts/fishaudio.md Install the Fish Audio plugin for LiveKit Agents using pip. ```shell uv add "livekit-agents[fishaudio]~=1.6" ``` -------------------------------- ### Install AsyncAI TTS Plugin Source: https://docs.livekit.io/agents/models/tts/asyncai Install the livekit-agents package with AsyncAI support using pip. ```bash uv add "livekit-agents[asyncai]~=1.5" ``` -------------------------------- ### Install Resemble AI Plugin Source: https://docs.livekit.io/agents/models/tts/plugins/resemble Install the Resemble AI plugin for LiveKit Agents using pip. ```bash uv add "livekit-agents[resemble]~=1.5" ``` -------------------------------- ### Install AWS Plugin Source: https://docs.livekit.io/agents/models/realtime/plugins/nova-sonic.md Install the LiveKit AWS plugin with the realtime extra using pip. ```shell uv add "livekit-plugins-aws[realtime]" ``` -------------------------------- ### Initialize and Start Anam Avatar Session in Node.js Source: https://docs.livekit.io/agents/models/avatar/plugins/anam.md Example of initializing an AgentSession and an AvatarSession with persona configuration, then starting both the avatar and the agent session. ```typescript import { voice } from '@livekit.io/agents'; import * as anam from '@livekit.io/agents-plugin-anam'; const session = new voice.AgentSession({ // ... stt, llm, tts, etc. }); const avatar = new anam.AvatarSession({ personaConfig: { name: "...", // Name of the avatar to use. avatarId: "...", // ID of the avatar to use. See "Avatar setup" for details. }, }); // Start the avatar and wait for it to join await avatar.start(session, room); // Start your agent session with the user await session.start( // ... room, agent, room_options, etc. ); ``` -------------------------------- ### Navigate to Agent Directory Source: https://docs.livekit.io/agents/start/voice-ai.md Change into the newly created agent project directory to proceed with setup. ```shell cd my-agent ``` -------------------------------- ### Initialize AgentSession with Qwen 3 TTS Source: https://docs.livekit.io/agents/models/tts/plugins/simplismart Set up an AgentSession to utilize Qwen 3 TTS from Simplismart. Requires a deployed model endpoint URL and specific parameters like language and leading silence. ```python from livekit.plugins import simplismart session = AgentSession( tts=simplismart.TTS( model="qwen-tts", voice="Chelsie", base_url="https:///v1/audio/speech", language="English", leading_silence=True, ) # ... llm, stt, etc. ) ``` -------------------------------- ### Initialize OpenAI LLM with WebSearch Tool (Python) Source: https://docs.livekit.io/agents/models/llm/openai.md Demonstrates initializing an agent with the OpenAI LLM and the WebSearch provider tool in Python. Ensure the `openai` plugin is installed. ```python from livekit.plugins import openai agent = MyAgent( llm=openai.responses.LLM(model="gpt-4.1"), tools=[openai.tools.WebSearch()], # replace with any supported provider tool ) ``` -------------------------------- ### Node.js Session Start with Input/Output Options Source: https://docs.livekit.io/agents/logic/sessions.md Example of starting a session with separate input and output options in Node.js, including noise cancellation and participant identity. ```typescript import { BackgroundVoiceCancellation } from '@livekit/noise-cancellation-node'; // ... session and agentsetup await session.start({ room: ctx.room, agent: myAgent, inputOptions: { textEnabled: true, audioEnabled: true, videoEnabled: true, noiseCancellation: BackgroundVoiceCancellation(), participantIdentity: "user_123", }, outputOptions: { syncTranscription: false, }, }); ``` -------------------------------- ### Initialize LiveKit Gemini Agent Project Source: https://docs.livekit.io/agents/integrations/google.md Use uv to create a new project for your voice agent. Navigate into the created directory. ```shell uv init livekit-gemini-agent --bare cd livekit-gemini-agent ``` -------------------------------- ### Initialize AgentSession with Perplexity LLM Source: https://docs.livekit.io/agents/models/llm/perplexity.md Demonstrates how to initialize an `AgentSession` using Perplexity's LLM. Ensure necessary imports are present. This setup is for Python. ```python from livekit.plugins import perplexity session = AgentSession( llm=perplexity.responses.LLM( model="perplexity/sonar", ), # ... tts, stt, vad, turn_handling, etc. ) ``` -------------------------------- ### Python Session Start with Room Options Source: https://docs.livekit.io/agents/logic/sessions.md Example of starting a session with custom room input and output options in Python, including noise cancellation and participant identity. ```python from livekit.agents import room_io from livekit.plugins import noise_cancellation room_options=room_io.RoomOptions( video_input=True, audio_input=room_io.AudioInputOptions( noise_cancellation=noise_cancellation.BVC(), ), text_output=room_io.TextOutputOptions( sync_transcription=False, ), participant_identity="user_123", ) await session.start( agent=my_agent, room=room, room_options=room_options, ) ``` -------------------------------- ### Install Groq Plugin for Python Source: https://docs.livekit.io/agents/models/llm/groq.md Install the Groq plugin for LiveKit Agents in a Python environment using uv. ```shell uv add "livekit-agents[groq]~=1.5" ``` -------------------------------- ### Initialize and Start Tavus Avatar in Node.js Source: https://docs.livekit.io/agents/models/avatar/plugins/tavus.md Initialize an AvatarSession with replica and persona IDs, then start it within an AgentSession. This example assumes STT, LLM, and TTS components are configured. ```typescript import { voice } from '@livekit/agents'; import * as tavus from '@livekit/agents-plugin-tavus'; const session = new voice.AgentSession({ // Add STT, LLM, TTS, and other components here }); const avatar = new tavus.AvatarSession({ replicaID: '...', // ID of the Tavus replica to use personaID: '...', // ID of the Tavus persona to use (see preceding section for configuration details) }); // Start the avatar and wait for it to join await avatar.start(session, room); // Start your agent session with the user await session.start( // ... room, agent, room_options, etc. ); ``` -------------------------------- ### Initialize and Start Tavus Avatar in Python Source: https://docs.livekit.io/agents/models/avatar/plugins/tavus.md Initialize an AvatarSession with replica and persona IDs, then start it within an AgentSession. This example assumes STT, LLM, and TTS components are configured. ```python from livekit import agents from livekit.agents import AgentServer, AgentSession from livekit.plugins import tavus server = AgentServer() @server.rtc_session(agent_name="my-agent") async def my_agent(ctx: agents.JobContext): session = AgentSession( # ... stt, llm, tts, etc. ) avatar = tavus.AvatarSession( replica_id="...", # ID of the Tavus replica to use persona_id="...", # ID of the Tavus persona to use (see preceding section for configuration details) ) # Start the avatar and wait for it to join await avatar.start(session, room=ctx.room) # Start your agent session with the user await session.start( # ... room, agent, room_options, etc.... ) ``` -------------------------------- ### Initialize OpenAI LLM with WebSearch Tool (Node.js) Source: https://docs.livekit.io/agents/models/llm/openai.md Shows how to set up a LiveKit agent using the OpenAI LLM and the WebSearch provider tool in Node.js. Requires the `@livekit/agents-plugin-openai` package. ```typescript import { voice } from '@livekit/agents'; import * as openai from '@livekit/agents-plugin-openai'; const agent = voice.Agent.create({ instructions: 'You are a helpful assistant.', llm: new openai.responses.LLM({ model: 'gpt-4.1' }), tools: [new openai.WebSearch({ searchContextSize: 'medium' })], }); ``` -------------------------------- ### Initialize AgentSession with xAI STT (Node.js) Source: https://docs.livekit.io/agents/models/stt/xai.md This Node.js example demonstrates how to set up an AgentSession using xAI STT through LiveKit Inference. Ensure you provide the correct model and language. Other agent components like LLM, TTS, VAD, and turn handling can also be configured. ```typescript import { AgentSession, inference } from '@livekit/agents'; const session = new AgentSession({ stt: new inference.STT({ model: "xai/stt-1", language: "en" }), // ... llm, tts, vad, turnHandling, etc. }); ``` -------------------------------- ### Configure Turn Boundary Cooldown in Python Source: https://docs.livekit.io/agents/logic/turns/adaptive-interruption-handling.md Tune the start and end cooldown windows for adaptive interruption detection. This example sets a 0.5-second start cooldown and a 2.0-second end cooldown. ```python turn_handling = { "interruption": {"backchannel_boundary": (0.5, 2.0)}, } ``` -------------------------------- ### Start AgentSession with Image or Avatar ID Source: https://docs.livekit.io/agents/models/avatar/plugins/bithuman.md Initialize and start an AgentSession using either a local image file or a bitHuman avatar ID. This example also configures audio output options. ```python from livekit.agents import room_io from livekit.plugins import bithuman from PIL import Image avatar = bithuman.AvatarSession( avatar_image=Image.open("avatar.jpg").convert("RGB"), # This example uses an image in the current directory. # or: avatar_id="your-avatar-id" # You can also use an existing avatar ID. ) await avatar.start(session, room=ctx.room) await session.start( room=ctx.room, room_options=room_io.RoomOptions(audio_output=False), ) ``` -------------------------------- ### Initialize AgentSession with OpenAI LLM (Node.js) Source: https://docs.livekit.io/agents/models/llm/plugins/openai This Node.js example shows how to set up an AgentSession using an OpenAI model through LiveKit Inference. Import 'AgentSession' and 'inference' from '@livekit/agents'. Model options like 'reasoning_effort' can be configured. ```javascript import { AgentSession, inference } from '@livekit/agents'; const session = new AgentSession({ llm: new inference.LLM({ model: "openai/gpt-5-mini", provider: "openai", modelOptions: { reasoning_effort: "low" } }), // ... tts, stt, vad, turnHandling, etc. }); ``` -------------------------------- ### Initialize AgentSession with Baseten LLM Source: https://docs.livekit.io/agents/models/llm/baseten.md Use the Baseten LLM within an AgentSession by providing the model name. This example shows how to set up the LLM component for your agent. ```python from livekit.plugins import baseten session = AgentSession( llm=baseten.LLM( model="openai/gpt-oss-120b" ), # ... tts, stt, vad, turn_handling, etc. ) ``` -------------------------------- ### Filter MCP Tools with a Predicate Function Source: https://docs.livekit.io/agents/logic/tools/mcp.md Use MCPToolset.filter_tools() for more control over tool filtering. It accepts a predicate function and filters tools in-place after setup. This example filters tools to keep only those with 'search' in their ID after manually calling setup(). ```python from livekit.agents import mcp toolset = mcp.MCPToolset( id="my-api", mcp_server=mcp.MCPServerHTTP("https://your-mcp-server.com/mcp"), ) # When using outside of AgentSession, call setup() manually await toolset.setup() toolset.filter_tools(lambda tool: "search" in tool.id) ``` -------------------------------- ### Swift: Real-time Wakeword Detection with Listener Source: https://docs.livekit.io/agents/multimodality/audio/wakeword.md Use WakeWordListener for hands-free wakeword detection in Swift. This example starts the listener and consumes detections as an async sequence. ```swift import LiveKitWakeWord let classifier = Bundle.main.url(forResource: "hey_livekit", withExtension: "onnx")! let model = try WakeWordModel(models: [classifier], sampleRate: 16_000) let listener = WakeWordListener(model: model, threshold: 0.5, debounce: 2.0) try listener.start() for await detection in listener.detections() { print("Detected \(detection.name) (\(String(format: "%.2f", detection.confidence)))") } ``` -------------------------------- ### Initialize Agent with OpenAI LLM and WebSearch Tool (Python) Source: https://docs.livekit.io/agents/models/llm/plugins/openai Example of initializing a LiveKit Agent in Python with the OpenAI LLM and the WebSearch provider tool. Ensure the correct imports are included. ```python from livekit.plugins import openai agent = MyAgent( llm=openai.responses.LLM(model="gpt-4.1"), tools=[openai.tools.WebSearch()], # replace with any supported provider tool ) ``` -------------------------------- ### Example .env.local file Source: https://docs.livekit.io/agents/server/startup-modes.md This file contains your LiveKit API key, secret, and WebSocket URL. Ensure these are correctly set for authentication. ```dotenv LIVEKIT_API_KEY= LIVEKIT_API_SECRET= LIVEKIT_URL=%{wsURL}% ``` -------------------------------- ### Python Agent Test Setup Source: https://docs.livekit.io/agents/start/testing/test-framework.md Set up an asynchronous test for a LiveKit Agent using pytest. This includes initializing an LLM, creating an AgentSession, and starting the agent. ```python @pytest.mark.asyncio # Or your async testing framework of choice async def test_your_agent() -> None: async with ( # You must create an LLM instance for the `judge` method inference.LLM(model="openai/chat-latest") as llm, # Create a session for the life of this test. # LLM is not required - it will use the agent's LLM if you don't provide one here AgentSession(llm=llm) as session, ): # Start the agent in the session await session.start(Assistant()) # Run a single conversation turn based on the given user input result = await session.run(user_input="Hello") # ...your assertions go here... ``` -------------------------------- ### Create Telnyx LLM in Node.js Source: https://docs.livekit.io/agents/models/llm/telnyx.md Instantiate an AgentSession with Telnyx as the LLM provider using the `withTelnyx` method. This example shows basic setup with a specified model. ```typescript import * as openai from '@livekit/agents-plugin-openai'; const session = new voice.AgentSession({ llm: openai.LLM.withTelnyx({ model: "meta-llama/Meta-Llama-3.1-70B-Instruct", }), // ... tts, stt, vad, turnHandling, etc. }); ``` -------------------------------- ### Initialize AgentSession with Ultravox Source: https://docs.livekit.io/agents/models/realtime/plugins/ultravox.md Initialize an AgentSession using the Ultravox Realtime model. Ensure the `livekit.plugins.ultravox` module is imported. ```python from livekit.plugins import ultravox session = AgentSession( llm=ultravox.realtime.RealtimeModel(), ) ``` -------------------------------- ### Create Telnyx LLM in Python Source: https://docs.livekit.io/agents/models/llm/telnyx.md Instantiate an AgentSession with Telnyx as the LLM provider using the `with_telnyx` method. This example shows basic setup with a specified model. ```python from livekit.plugins import openai session = AgentSession( llm=openai.LLM.with_telnyx( model="meta-llama/Meta-Llama-3.1-70B-Instruct", ), # ... tts, stt, vad, turn_handling, etc. ) ``` -------------------------------- ### Initialize Node.js Agent with CLI Source: https://docs.livekit.io/agents/start/voice-ai.md Use the LiveKit CLI to initialize a new agent project with a Node.js starter template. Ensure your project meets the requirements before running. ```shell lk agent init my-agent --template agent-starter-node ``` -------------------------------- ### Initialize and Execute a TaskGroup in Python Source: https://docs.livekit.io/agents/build/tasks Demonstrates how to create a TaskGroup, add tasks with descriptions and IDs, and execute the group to retrieve results. Results are accessed by task ID. ```python from livekit.agents.beta.workflows import GetEmailTask, TaskGroup # Create and configure TaskGroup with the current agent's chat context chat_ctx = self.chat_ctx task_group = TaskGroup(chat_ctx=chat_ctx) # Add tasks using lambda factories task_group.add( lambda: GetEmailTask(), id="get_email_task", description="Collects the user's email" ) task_group.add( lambda: GetCommuteTask(), id="get_commute_task", description="Records the user's commute flexibility" ) # Execute the task group results = await task_group # Returns TaskGroupResult object task_results = results.task_results # Access results by task ID print(task_results) # Output: { # "get_email_task": GetEmailResult(email="john.doe@gmail.com"), # "get_commute_task": CommuteResult(can_commute=True, commute_method="subway") # } ``` -------------------------------- ### Initialize AgentSession with Default Bedrock LLM Source: https://docs.livekit.io/agents/models/llm/aws.md Initialize an AgentSession using the default Amazon Bedrock LLM model (Amazon Nova 2 Lite). This is a quick way to get started. ```python from livekit.plugins import aws # Use default model (Amazon Nova 2 Lite) session = AgentSession( llm=aws.LLM(), # ... tts, stt, vad, turn_handling, etc. ) ``` -------------------------------- ### Python: Manual Turn Control Setup Source: https://docs.livekit.io/agents/logic/turns.md Configure AgentSession for manual turn detection. Disable audio input initially and use RPC methods to manage user speaking states. ```python session = AgentSession( turn_handling=TurnHandlingOptions( turn_detection="manual", ), # ... stt, tts, llm, etc. ) # Disable audio input at the start session.input.set_audio_enabled(False) # When user starts speaking @ctx.room.local_participant.register_rpc_method("start_turn") async def start_turn(data: rtc.RpcInvocationData): session.interrupt() # Stop any current agent speech session.clear_user_turn() # Clear any previous input session.input.set_audio_enabled(True) # Start listening # When user finishes speaking @ctx.room.local_participant.register_rpc_method("end_turn") async def end_turn(data: rtc.RpcInvocationData): session.input.set_audio_enabled(False) # Stop listening session.commit_user_turn() # Process the input and generate response # When user cancels their turn @ctx.room.local_participant.register_rpc_method("cancel_turn") async def cancel_turn(data: rtc.RpcInvocationData): session.input.set_audio_enabled(False) # Stop listening session.clear_user_turn() # Discard the input ``` -------------------------------- ### Agent Environment Variables Source: https://docs.livekit.io/agents/ops/deployment/custom Configure your agent server with environment variables for secrets like API keys. This example shows minimum required keys for an agent built with the Voice AI quickstart. ```env DEEPGRAM_API_KEY= OPENAI_API_KEY= CARTESIA_API_KEY= LIVEKIT_API_KEY= LIVEKIT_API_SECRET= LIVEKIT_URL= ``` -------------------------------- ### Initialize AgentSession with Realtime Model (Python) Source: https://docs.livekit.io/agents/models/realtime.md Instantiate an AgentSession with a realtime model plugin for direct speech processing. Ensure the 'openai' plugin is installed. ```python from livekit.agents import AgentSession from livekit.plugins import openai session = AgentSession( llm=openai.realtime.RealtimeModel() ) ``` -------------------------------- ### Basic AgentSession with AvatarTalk Source: https://docs.livekit.io/agents/models/avatar/plugins/avatartalk.md Example of setting up an AgentSession and integrating an AvatarTalk avatar. Configure STT, LLM, and TTS as needed. ```python from livekit import agents from livekit.agents import AgentServer, AgentSession from livekit.plugins import avatartalk server = AgentServer() @server.rtc_session(agent_name="my-agent") async def my_agent(ctx: agents.JobContext): session = AgentSession( # ... stt, llm, tts, etc. ) avatar = avatartalk.AvatarSession( avatar="...", # ID of the AvatarTalk avatar to use. See "Avatar setup" for details. ) # Start the avatar and wait for it to join await avatar.start(session, room=ctx.room) # Start your agent session with the user await session.start( # ... room, agent, room_options, etc.... ) ``` -------------------------------- ### Node.js Agent Test Setup with Vitest Source: https://docs.livekit.io/agents/start/testing/test-framework.md Set up a test for a LiveKit Agent using Vitest. This includes initializing the logger, LLM, AgentSession, and starting the agent within Vitest's describe/it blocks. ```typescript import { inference, initializeLogger, voice } from '@livekit/agents'; import { describe, it, beforeAll, afterAll } from 'vitest'; // Import your agent class import { Agent } from './agent'; // Initialize logger to suppress CLI output initializeLogger({ pretty: false, level: 'warn' }); const { AgentSession } = voice; describe('YourAgent', () => { let session: voice.AgentSession; let llm: inference.LLM; beforeAll(async () => { // You must create an LLM instance for the `judge` method llm = new inference.LLM({ model: 'openai/chat-latest' }); // Create a session for the life of this test. // LLM is not required - it will use the agent's LLM if you don't provide one here session = new AgentSession({ llm }); // Start the agent in the session await session.start({ agent: new Agent() }); }); afterAll(async () => { await session?.close(); }); it('should test your agent', async () => { // Run a single conversation turn based on the given user input const result = await session.run({ userInput: 'Hello' }).wait(); // ...your assertions go here... }); }); ``` -------------------------------- ### Initialize Python Agent with CLI Source: https://docs.livekit.io/agents/start/voice-ai.md Use the LiveKit CLI to initialize a new agent project with a Python starter template. Ensure your project meets the requirements before running. ```shell lk agent init my-agent --template agent-starter-python ``` -------------------------------- ### Integrate Virtual Avatar with AgentSession Source: https://docs.livekit.io/agents/models/avatar.md This snippet demonstrates how to add a virtual avatar using the Anam plugin to an AgentSession. It shows the setup for the AgentServer, the RTC session, and the AvatarSession configuration. Ensure you have the necessary API keys and have installed the Anam plugin. ```python from livekit import agents from livekit.agents import AgentServer, AgentSession from livekit.plugins import anam server = AgentServer() @server.rtc_session(agent_name="my-agent") async def my_agent(ctx: agents.JobContext): session = AgentSession( # ... stt, llm, tts, etc. ) avatar = anam.AvatarSession( persona_config=anam.PersonaConfig( name="...", # Name of the avatar to use. avatarId="...", # ID of the avatar to use. ), ) # Start the avatar await avatar.start(session, room=ctx.room) # Wait for the avatar to join and publish its video track await avatar.wait_for_join() # Start your agent session with the user await session.start( # ... room, agent, room_options, etc.... ) ``` -------------------------------- ### Install Node.js Agent Dependencies Source: https://docs.livekit.io/agents/start/voice-ai.md Install the necessary runtime and plugin dependencies for your Node.js agent using 'pnpm install'. ```shell pnpm install ``` -------------------------------- ### Initialize and Start TruGen Avatar in Node.js Source: https://docs.livekit.io/agents/models/avatar/plugins/trugen.md Initialize an AvatarSession with TruGen and start it within an AgentSession. Ensure the avatar is started before the main agent session. ```typescript import { voice } from '@livekit/agents'; import * as trugen from '@livekit.agents-plugin-trugen'; const session = new voice.AgentSession({ // ... stt, llm, tts, etc. }); const avatar = new trugen.AvatarSession({ avatarId: '...', // ID of the TruGen stock avatar to use }); // Start the avatar and wait for it to join await avatar.start(session, room); // Start your agent session with the user await session.start({ // ... room, agent, room_options, etc. }); ``` -------------------------------- ### Initialize and Start TruGen Avatar in Python Source: https://docs.livekit.io/agents/models/avatar/plugins/trugen.md Initialize an AvatarSession with TruGen and start it within an AgentSession. Ensure the avatar is started before the main agent session. ```python from livekit import agents from livekit.agents import AgentServer, AgentSession from livekit.plugins import trugen server = AgentServer() @server.rtc_session() async def my_agent(ctx: agents.JobContext): session = AgentSession( # ... stt, llm, tts, etc. ) avatar = trugen.AvatarSession( avatar_id="...", # ID of the TruGen stock avatar to use ) # Start the avatar and wait for it to join await avatar.start(session, room=ctx.room) # Start your agent session with the user await session.start( # ... room, agent, room_options, etc.... ) ``` -------------------------------- ### Start Agent Server in Production (Node.js) Source: https://docs.livekit.io/agents/server/startup-modes.md Before starting your Node.js agent server in production, build the project using `tsc`. This command then starts the server with production optimizations. ```shell tsc && node agent.js start ``` -------------------------------- ### Node.js Usage Example Source: https://docs.livekit.io/agents/models/stt/plugins/assemblyai Example of how to integrate AssemblyAI STT into an AgentSession in Node.js. ```APIDOC ## Node.js Usage Example This example shows how to set up the AssemblyAI STT service within a LiveKit AgentSession using Node.js. ### Method ```javascript import { voice } from '@livekit/agents'; import * as assemblyai from '@livekit/agents-plugin-assemblyai'; const session = new voice.AgentSession({ stt: new assemblyai.STT({ model: "u3-rt-pro", }), // ... llm, tts, etc. }); ``` ### Parameters - **`model`** (`string`): STT model to use. Accepted options are `u3-rt-pro`, `universal-streaming-english`, and `universal-streaming-multilingual`. ``` -------------------------------- ### Initialize Agent with OpenAI LLM and WebSearch Tool (Node.js) Source: https://docs.livekit.io/agents/models/llm/plugins/openai Example of initializing a LiveKit Agent in Node.js using the OpenAI plugin. This demonstrates setting the LLM and a provider tool like WebSearch with specific parameters. ```typescript import { voice } from '@livekit/agents'; import * as openai from '@livekit/agents-plugin-openai'; const agent = voice.Agent.create({ instructions: 'You are a helpful assistant.', llm: new openai.responses.LLM({ model: 'gpt-4.1' }), tools: [new openai.WebSearch({ searchContextSize: 'medium' })], }); ``` -------------------------------- ### Python Usage Example Source: https://docs.livekit.io/agents/models/stt/plugins/assemblyai Example of how to integrate AssemblyAI STT into an AgentSession in Python. ```APIDOC ## Python Usage Example This example demonstrates how to configure the AssemblyAI STT service within a LiveKit AgentSession using Python. ### Method ```python from livekit.plugins import assemblyai session = AgentSession( stt=assemblyai.STT( model="u3-rt-pro", min_turn_silence=100, max_turn_silence=1000, vad_threshold=0.3, ), vad=silero.VAD.load(activation_threshold=0.3), # ... llm, tts, etc. ) ``` ### Parameters - **`model`** (`string`): STT model to use. Accepted options are `u3-rt-pro`, `universal-streaming-english`, and `universal-streaming-multilingual`. - **`min_turn_silence`** (`int`): The minimum duration of silence (in milliseconds) before the model checks for end of turn. Defaults to `100`. - **`max_turn_silence`** (`int`): The maximum duration of silence (in milliseconds) allowed in a turn before end of turn is triggered. - **`vad_threshold`** (`float`): AssemblyAI's internal Silero VAD onset threshold. Defaults to `0.3` for Universal-3 Pro and `0.4` for Universal-Streaming. ``` -------------------------------- ### Install Vitest for Node.js Source: https://docs.livekit.io/agents/start/testing/test-framework.md Install Vitest as a development dependency for writing tests in Node.js. ```shell pnpm add -D vitest ``` -------------------------------- ### Build and Start Node.js Agent in Production Mode Source: https://docs.livekit.io/agents/start/voice-ai-quickstart Build and start your Node.js agent in production mode using `pnpm` commands. This is typically done in the project directory. ```bash pnpm build pnpm start ```