### Navigate to WebSocket Example Directory

Source: https://github.com/uezo/aiavatarkit/blob/main/examples/websocket/README.md

Change the working directory to the location of the WebSocket example files.

```sh
cd aiavatarkit/examples/websocket
```

--------------------------------

### Setup Admin Panel

Source: https://github.com/uezo/aiavatarkit/blob/main/FEATURES.md

Initializes the admin panel for runtime configuration and control. The adapter can be dynamically registered after setup.

```python
# Setup admin panel
from aiavatar.admin import setup_admin_panel

panel = setup_admin_panel(app, adapter=aiavatar_app)

# Dynamic adapter registration after setup
panel.add_adapter(another_adapter, name="secondary")
```

--------------------------------

### Install Dependencies

Source: https://github.com/uezo/aiavatarkit/blob/main/examples/websocket/README.md

Install the necessary Python packages for the WebSocket server.

```sh
pip install aiavatar silero-vad fastapi uvicorn websockets
```

--------------------------------

### Start AIAvatarKit WebSocket Server

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Script to set up and run the AIAvatarKit WebSocket server using FastAPI. It includes downloading an example UI and optionally setting up an admin panel. Requires a running VOICEVOX instance.

```python
from fastapi import FastAPI
from fastapi.staticfiles import StaticFiles
from aiavatar.adapter.websocket.server import AIAvatarWebSocketServer
from aiavatar.util import download_example

# Download example UI if not exists
download_example("websocket/html")

# Build Speech-to-Speech pipeline with WebSocket adapter
aiavatar_app = AIAvatarWebSocketServer(
    openai_api_key=OPENAI_API_KEY
)

# Build websocket server
app = FastAPI()
router = aiavatar_app.get_websocket_router()
app.include_router(router)
app.mount("/static", StaticFiles(directory="html"), name="static")

# Setup admin panel (Optional)
from aiavatar.admin import setup_admin_panel
setup_admin_panel(app, adapter=aiavatar_app)
```

--------------------------------

### Quick Start - Local Console

Source: https://context7.com/uezo/aiavatarkit/llms.txt

Set up a basic AI avatar for local console use. This snippet initializes the AIAvatar with OpenAI and VOICEVOX, then starts listening for voice input.

```python
import asyncio
from aiavatar import AIAvatar

OPENAI_API_KEY = "your-openai-api-key"

aiavatar_app = AIAvatar(
    openai_api_key=OPENAI_API_KEY,
    openai_model="gpt-4o",
    system_prompt="You are a helpful assistant.",
    voicevox_speaker=46,  # Speaker ID for VOICEVOX TTS
    debug=True
)

# Start listening - conversation begins with wake word "Hello"
asyncio.run(aiavatar_app.start_listening())
```

--------------------------------

### Register Example Tools

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Pre-built tools like Google Search and Web Scraper can be added directly to the application.

```python
# Google Search
from examples.tools.gemini_websearch import GeminiWebSearchTool
aiavatar_app.sts.llm.add_tool(GeminiWebSearchTool(gemini_api_key=GEMINI_API_KEY))

# Web Scraper
from examples.tools.webscraper import WebScraperTool
aiavatar_app.sts.llm.add_tool(WebScraperTool())
```

--------------------------------

### Start WebSocket Server

Source: https://github.com/uezo/aiavatarkit/blob/main/examples/websocket/README.md

Launch the FastAPI application using Uvicorn.

```sh
uvicorn server:app
```

--------------------------------

### Install asyncpg for PostgreSQL Support

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Install the `asyncpg` library, which is required for enabling PostgreSQL support in AIAvatarKit. This is recommended for production environments.

```sh
pip install asyncpg
```

--------------------------------

### Install Azure Speech SDK

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Install the required package for Azure speech recognition.

```sh
pip install azure-cognitiveservices-speech
```

--------------------------------

### Admin Panel Setup

Source: https://github.com/uezo/aiavatarkit/blob/main/FEATURES.md

Provides instructions on setting up and extending the Admin Panel for runtime configuration and monitoring of the AI avatar.

```APIDOC
## Admin Panel Setup

### Description
Set up and configure the web-based Admin Panel for monitoring, controlling, and evaluating the AI avatar.

### Features
- **Metrics**: Real-time STS pipeline performance visualization.
- **Logs**: Conversation history with audio playback.
- **Control**: Send messages and control the avatar directly.
- **Config**: Dynamic adjustment of pipeline, VAD, STT, LLM, TTS, and adapter settings.
- **Evaluation**: Run dialogue evaluation scenarios.
- **Characters**: Manage character information (requires CharacterService).
- **Authentication**: Supports basic authentication and API key (Bearer token).

### Setup Example
```python
from aiavatar.admin import setup_admin_panel

panel = setup_admin_panel(app, adapter=aiavatar_app)

# Dynamic adapter registration after setup
panel.add_adapter(another_adapter, name="secondary")
```
```

--------------------------------

### Configure Recording Started Callback

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Set up a callback to trigger when meaningful speech is detected.

```python
# Option 1: Pass callback in constructor
async def my_recording_started_handler(session_id: str):
    print(f"Recording started for session: {session_id}")
    await stop_ai_speech()

vad = SileroSpeechDetector(
    on_recording_started=my_recording_started_handler,
    on_recording_started_min_duration=1.5,    # Trigger after 1.5 sec of speech (default)
    # other parameters...
)
```

--------------------------------

### Install AIAvatarKit for WebSocket Server

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Install AIAvatarKit along with necessary dependencies for building a WebSocket server.

```sh
pip install aiavatar fastapi uvicorn websockets
```

--------------------------------

### Install AIAvatarKit (Console)

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Install the AIAvatarKit package for local console applications. Consider downgrading if using older technical blogs.

```sh
pip install aiavatar
```

--------------------------------

### WebSocket Server Setup

Source: https://context7.com/uezo/aiavatarkit/llms.txt

Deploy AIAvatarKit as a WebSocket server for browser-based interactions. This example configures the server with VAD on the server side and sets up a FastAPI application.

```python
from fastapi import FastAPI
from fastapi.staticfiles import StaticFiles
from aiavatar.adapter.websocket.server import AIAvatarWebSocketServer
from aiavatar.admin import setup_admin_panel

OPENAI_API_KEY = "your-openai-api-key"

# Create WebSocket server with VAD on server side
aiavatar_app = AIAvatarWebSocketServer(
    openai_api_key=OPENAI_API_KEY,
    volume_db_threshold=-30,
    debug=True
)

# Setup FastAPI app
app = FastAPI()
router = aiavatar_app.get_websocket_router()
app.include_router(router)
app.mount("/static", StaticFiles(directory="html"), name="static")

# Optional: Setup admin panel at /admin
setup_admin_panel(app, adapter=aiavatar_app)

# Run with: uvicorn server:app
```

--------------------------------

### Install Cryptography Package

Source: https://github.com/uezo/aiavatarkit/blob/main/examples/websocket/openclaw.md

Install the cryptography package required for certificate generation.

```sh
pip install cryptography
```

--------------------------------

### Install Langfuse

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Install the Langfuse library to enable integration with AIAvatarKit for enhanced observability.

```sh
pip install langfuse
```

--------------------------------

### Install Amazon Transcribe SDK

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Install the required package for Amazon Transcribe.

```sh
pip install amazon-transcribe
```

--------------------------------

### Install FastMCP Dependency

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Install the required package to enable MCP support.

```sh
pip install fastmcp
```

--------------------------------

### Run AIAvatarKit Locally (Console)

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Basic script to start AIAvatarKit for local console interaction. Requires an OpenAI API key and a running VOICEVOX instance for TTS. Conversation starts with a wake word.

```python
import asyncio
from aiavatar import AIAvatar

aiavatar_app = AIAvatar(
    openai_api_key=OPENAI_API_KEY,
    debug=True
)
asyncio.run(aiavatar_app.start_listening())
```

--------------------------------

### Setup Admin Panel with Basic Authentication

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Secure the admin panel using Basic Authentication by providing a username and password. This adds a layer of protection for accessing the panel.

```python
setup_admin_panel(
    app,
    adapter=aiavatar_app,
    api_key="your-api-key",
    basic_auth_username="admin",
    basic_auth_password="your-password",
)
```

--------------------------------

### Setup Admin Panel with Custom HTML

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Customize the admin panel's appearance by providing your own HTML content. This allows for full control over the user interface.

```python
custom_html = open("my_admin.html").read()

setup_admin_panel(
    app,
    adapter=aiavatar_app,
    html=custom_html,       # Use your own HTML instead of the built-in template
)
```

--------------------------------

### Run AIAvatar Application

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Execute the application script to start the AIAvatar service.

```bash
$ run.py
```

--------------------------------

### PostgreSQL Pool Provider Setup

Source: https://github.com/uezo/aiavatarkit/blob/main/FEATURES.md

Configure and initialize a shared PostgreSQL connection pool for efficient resource management across multiple components. Set min_size and max_size for pool configuration.

```python
from aiavatar.database.postgres import PostgreSQLPoolProvider

# Create shared pool provider
pool_provider = PostgreSQLPoolProvider(
    connection_str="postgresql://user:pass@host:5432/db",
    min_size=5,
    max_size=30
)

# Pass to components - all share the same pool
from aiavatar.sts.context.postgres import PostgreSQLContextManager
from aiavatar.sts.session.postgres import PostgreSQLSessionStateManager
from aiavatar.sts.performance.postgres import PostgreSQLPerformanceRecorder
from aiavatar.character import CharacterService

context_manager = PostgreSQLContextManager(get_pool=pool_provider.get_pool)
session_manager = PostgreSQLSessionStateManager(get_pool=pool_provider.get_pool)
performance_recorder = PostgreSQLPerformanceRecorder(get_pool=pool_provider.get_pool)
character_service = CharacterService(db_pool_provider=pool_provider)

```

--------------------------------

### Define Recording Started Callback

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Use the on_recording_started decorator to execute logic when speech detection begins.

```python
@vad.on_recording_started
async def on_recording_started(session_id):
    await stop_ai_speech()
```

--------------------------------

### Initialize Render Loop

Source: https://github.com/uezo/aiavatarkit/blob/main/examples/websocket/html/vrm.html

Starts the animation frame loop for the 3D scene.

```javascript
function animate() {
	requestAnimationFrame(animate);
	controls.update();
	idle.update(clock.getDelta());
	renderer.render(scene, camera);
}
animate();
```

--------------------------------

### Initialize TTS with Pronunciation Preprocessor

Source: https://github.com/uezo/aiavatarkit/blob/main/FEATURES.md

Set up a `PatternMatchPreprocessor` to handle custom string replacements and pronunciation rules for TTS. This example adds a pattern for 'OpenAI' and enables number-dash replacement.

```python
# Usage example
preprocessor = PatternMatchPreprocessor()
preprocessor.add_pattern("OpenAI", "Open A I")
preprocessor.add_number_dash_pattern()  # Number-dash pattern replacement

tts = VoicevoxSpeechSynthesizer(preprocessors=[preprocessor])
```

--------------------------------

### Launch AIAvatarKit Server

Source: https://github.com/uezo/aiavatarkit/blob/main/examples/websocket/openclaw.md

Command to start the AIAvatarKit server after OpenClaw is running.

```sh
python openclaw.py
```

--------------------------------

### Setup Admin Panel

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Configure the built-in admin panel for monitoring and controlling your AI avatar. Access it at the '/admin' route. Optional parameters include an evaluator, character service, character ID, and API key for authentication.

```python
from aiavatar.admin import setup_admin_panel

setup_admin_panel(
    app,
    adapter=aiavatar_app,
    evaluator=evaluator,                    # Optional: If omitted, the pipeline LLM settings are used
    character_service=character_service,    # Optional: If using CharacterService
    character_id=YOUR_CHARACTER_ID,         # Optional: Required if character_service is set
    api_key="your-api-key"                  # Optional: If omitted, no authentication is required
)
```

--------------------------------

### Setup VRMA Section and File Input

Source: https://github.com/uezo/aiavatarkit/blob/main/examples/websocket/html/vrm.html

Initializes a section for VRMA animations, including a file input for adding animations and a button to trigger it. Handles the file selection and caching of VRMA files.

```javascript
const vrmaSection = document.createElement('div'); vrmaSection.style.cssText = 'margin-top:16px'; const vrmaHeading = document.createElement('div'); vrmaHeading.style.cssText = 'font-size:11px;color:#999;margin-bottom:6px'; vrmaHeading.textContent = 'VRMA'; vrmaSection.appendChild(vrmaHeading); const vrmaFileInput = document.createElement('input'); vrmaFileInput.type = 'file'; vrmaFileInput.accept = '.vrma'; vrmaFileInput.multiple = true; vrmaFileInput.style.display = 'none'; const vrmaAddBtn = document.createElement('button'); vrmaAddBtn.className = 'vrmi-sampler-btn'; vrmaAddBtn.textContent = 'Add animation'; vrmaAddBtn.addEventListener('click', () => vrmaFileInput.click()); vrmaFileInput.addEventListener('change', async (e) => { for (const file of e.target.files) { const animName = file.name.replace(/\.vrma$/i, ''); try { await loadAndCacheVRMA(animName, file); } catch (err) { console.error('Failed to load VRMA:', err); alert(`Failed to load VRMA: ${file.name}`); } } vrmaFileInput.value = ''; refreshVrmaList(); }); vrmaSection.appendChild(vrmaAddBtn); vrmaSection.appendChild(vrmaFileInput);
```

--------------------------------

### Microphone Section UI Setup

Source: https://github.com/uezo/aiavatarkit/blob/main/examples/websocket/html/vrm.html

Initializes the microphone section UI, including a toggle for the microphone glow indicator. Assumes 'makeToggle' and 'saveDisplay' functions are defined elsewhere.

```javascript
const micSection = document.createElement('div'); micSection.style.cssText = 'margin-top:12px;'; micSection.appendChild(sectionHeading('Microphone')); const micToggles = document.createElement('div'); micToggles.style.cssText = 'display:flex;flex-direction:column;gap:10px'; const micGlowToggle = makeToggle('Show indicator', showMicGlow, (v) => { showMicGlow = v; if (!v) document.getElementById('micGlow').classList.remove('active'); saveDisplay(); }); const micMuteToggl
```

--------------------------------

### Setup Ambient and Directional Lighting

Source: https://github.com/uezo/aiavatarkit/blob/main/examples/websocket/html/vrm.html

Initializes ambient and directional lights for the scene. The `applyLighting` function dynamically updates these lights based on configuration values, including intensity, position, and color temperature.

```javascript
const ambientLight = new THREE.AmbientLight(0xffffff, 1.0);
scene.add(ambientLight);
const dirLight = new THREE.DirectionalLight(0xffffff, 1.0);
dirLight.position.set(1, 2, 3);
scene.add(dirLight);

applyLighting = () => {
  ambientLight.intensity = L.ambient / 100;
  dirLight.intensity = L.direct / 100;
  const h = L.hAngle * Math.PI / 180;
  const v = L.vAngle * Math.PI / 180;
  dirLight.position.set(
    Math.cos(v) * Math.sin(h),
    Math.sin(v),
    Math.cos(v) * Math.cos(h)
  );
  const [r, g, b] = kelvinToRGB(L.temp);
  dirLight.color.setRGB(r, g, b);
};
applyLighting();
```

--------------------------------

### Setup FastAPI App with AIAvatar Components

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Integrate AIAvatar's API router into a FastAPI application. Ensure you have FastAPI and uvicorn installed to run the server.

```python
from fastapi import FastAPI

import aiavatar_app

app = FastAPI()
router = aiavatar_app.get_api_router()
app.include_router(router)
```

--------------------------------

### Initialize QuickResponderPro

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Configure the QuickResponderPro instance with a context manager and system prompt.

```python
from aiavatar.sts.quick_responder.pro import QuickResponderPro, DEFAULT_QRP_SYSTEM_PROMPT_JA
from aiavatar.sts.llm.context_manager.postgres import PostgreSQLContextManager
from aiavatar.sts.models import STSRequest

quick_responder_pro = QuickResponderPro(
    api_key="YOUR_OPENAI_API_KEY",
    model="gpt-4.1-nano",
    tts=tts,
    context_manager=PostgreSQLContextManager(get_pool=pool_provider.get_pool),
    language="ja",
    system_prompt=DEFAULT_QRP_SYSTEM_PROMPT_JA + "\n\n# Character\nYour character description here.",
    timeout=1.5,
)

@aiavatar_app.sts.on_before_llm
async def on_before_llm(request: STSRequest):
    await quick_responder_pro.respond(request)
```

--------------------------------

### Configure PostgreSQL for AIAvatar Database

Source: https://context7.com/uezo/aiavatarkit/llms.txt

Set up PostgreSQL as a persistent database for AIAvatar Kit, enabling connection pooling for character services, LLM context management, and session management. Ensure asyncpg is installed.

```python
from aiavatar.database.postgres import PostgreSQLPoolProvider
from aiavatar.character import CharacterService
from aiavatar.sts.llm.chatgpt import ChatGPTService
from aiavatar.sts.llm.context_manager.postgres import PostgreSQLContextManager
from aiavatar.adapter.websocket.server import AIAvatarWebSocketServer

# pip install asyncpg
DB_CONNECTION_STR = "postgresql://user:password@localhost:5432/aiavatar"

# Create shared connection pool
pool_provider = PostgreSQLPoolProvider(
    connection_str=DB_CONNECTION_STR,
    max_size=20,
    min_size=5
)

# Character service with PostgreSQL
character_service = CharacterService(
    openai_api_key=OPENAI_API_KEY,
    db_pool_provider=pool_provider
)

# LLM with PostgreSQL context manager
llm = ChatGPTService(
    openai_api_key=OPENAI_API_KEY,
    context_manager=PostgreSQLContextManager(get_pool=pool_provider.get_pool)
)

# Adapter with PostgreSQL session management
aiavatar_app = AIAvatarWebSocketServer(
    llm=llm,
    db_pool_provider=pool_provider
)
```

--------------------------------

### Python Client for AIAvatar HTTP Streaming API

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

A simple Python client to interact with the AIAvatar HTTP streaming API. This example uses `asyncio` and `AIAvatarHttpClient` to start listening for sessions.

```python
import asyncio
from aiavatar.adapter.http.client import AIAvatarHttpClient

aiavatar_app = AIAvatarHttpClient(
    debug=True
)
asyncio.run(aiavatar_app.start_listening(session_id="http_session", user_id="http_user"))
```

--------------------------------

### Setup Dummy Components for Load Testing

Source: https://github.com/uezo/aiavatarkit/blob/main/FEATURES.md

Initialize dummy components for SpeechRecognizer, LLMService, and SpeechSynthesizer. Configure recognized text, response text, synthesized audio data, and delays for performance testing without external dependencies.

```python
from aiavatar.sts.stt.dummy import SpeechRecognizerDummy
from aiavatar.sts.llm.dummy import LLMServiceDummy
from aiavatar.sts.tts.dummy import SpeechSynthesizerDummy

stt = SpeechRecognizerDummy(recognized_text="Hello", wait_sec=0.1)
llm = LLMServiceDummy(response_text="Hi there!", wait_sec=0.5)
tts = SpeechSynthesizerDummy(synthesized_bytes=b"audio_data", wait_sec=0.2)
```

--------------------------------

### Initialize Built-in Tools

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Register pre-built tools for web searching, scraping, and image generation.

```python
# Web Search
from aiavatar.sts.llm.tools.gemini_websearch import GeminiWebSearchTool
google_search_tool = GeminiWebSearchTool(gemini_api_key=GEMINI_API_KEY)
llm.add_tool(google_search_tool)

from aiavatar.sts.llm.tools.openai_websearch import OpenAIWebSearchTool
web_search_tool = OpenAIWebSearchTool(openai_api_key=OPENAI_API_KEY)
llm.add_tool(web_search_tool)

from aiavatar.sts.llm.tools.grok_search import GrokSearchTool
grok_web_search_tool = GrokSearchTool(xai_api_key=XAI_API_KEY)
llm.add_tool(grok_web_search_tool)

# Web Scraper
from aiavatar.sts.llm.tools.webscraper import WebScraperTool
webscraper_tool = WebScraperTool()
# webscraper_tool = WebScraperTool(openai_api_key=OPENAI_API_KEY, return_summary=True)  # Provides summary instead of full innerText (recommended)
llm.add_tool(webscraper_tool)

# Image Generation
from aiavatar.sts.llm.tools.nanobanana import NanoBananaTool
nanobanana_tool = NanoBananaTool(gemini_api_key=GEMINI_API_KEY)
llm.add_tool(nanobanana_tool)

from aiavatar.sts.llm.tools.nanobanana import NanoBananaSelfieTool
selfie_tool = NanoBananaSelfieTool(gemini_api_key=GEMINI_API_KEY, reference_image=image_bytes_or_image_url_of_file_api)
llm.add_tool(selfie_tool)
```

--------------------------------

### Install AIAvatarKit Dependencies

Source: https://github.com/uezo/aiavatarkit/blob/main/examples/websocket/openclaw.md

Command to install required Python packages for the AIAvatarKit environment.

```sh
pip install aiavatar uvicorn fastapi websockets
```

--------------------------------

### Vue App Setup with Tabs and Configuration

Source: https://github.com/uezo/aiavatarkit/blob/main/aiavatar/admin/static/index.html

Sets up the main Vue application, defining components, reactive state for API keys, active tabs, tab availability, and configuration sections. Includes methods for API fetching and toast notifications.

```javascript
const apiKey = ref('');
const activeTab = ref('metrics');
const tabs = [
 { id: 'metrics', label: 'Metrics' },
 { id: 'logs', label: 'Logs' },
 { id: 'control', label: 'Control' },
 { id: 'character', label: 'Character' },
 { id: 'config', label: 'Config' },
 { id: 'evaluation', label: 'Evaluation' },
];
const tabAvailability = reactive({
 character: 'unknown',
 evaluation: 'unknown',
});
const tabSetupMessages = {
 character: 'Character features are not configured. To use this tab, set up CharacterService and call setup_character_api().',
 evaluation: 'Evaluation features are not configured. To use this tab, set up an Evaluator and call setup_evaluation_api().',
};
const activeConfigSection = ref('pipeline');
const configSections = [
 { id: 'pipeline', label: 'Pipeline' },
 { id: 'vad', label: 'VAD' },
 { id: 'stt', label: 'STT' },
 { id: 'llm', label: 'LLM' },
 { id: 'tts', label: 'TTS' },
 { id: 'adapters', label: 'Adapters' },
];
const configLoading = ref(false);
const saving = ref(false);
const toast = reactive({ show: false, message: '', type: 'success' });
let toastTimer = null;
const showToast = (message, type = 'success') => {
 toast.show = true;
 toast.message = message;
 toast.type = type;
 clearTimeout(toastTimer);
 toastTimer = setTimeout(() => {
 toast.show = false;
 }, 2500);
};
```

--------------------------------

### Set up AIAvatarKit router

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Initialize the API router and include it in the application.

```python
router = aiavatar_app.get_api_router()
app.include_router(router)
```

--------------------------------

### Voice Notification HTTP Example

Source: https://github.com/uezo/aiavatarkit/blob/main/examples/websocket/voice_push_notification_skill.md

A complete HTTP request example for notifying a specific session with a greeting.

```http
POST {base_url}/avatar/perform
Authorization: Bearer {api_key}
Content-Type: application/json

{"text": "[face:joy]Hello! You have a new message.", "user_id": "agent:main:main"}
```

--------------------------------

### Configure Custom Clients

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Use Azure OpenAI or other custom clients by passing them directly to the constructor.

```python
from openai import AsyncAzureOpenAI

quick_responder_pro = QuickResponderPro(
    client=AsyncAzureOpenAI(
        api_key="YOUR_AZURE_API_KEY",
        api_version="2025-01-01-preview",
        azure_endpoint="https://your-resource.openai.azure.com/openai/deployments/your-deployment/chat/completions?api-version=2025-01-01-preview"
    ),
    model="your-deployment-name",
    tts=tts,
    context_manager=context_manager,
)
```

```python
quick_responder_pro = QuickResponderPro(
    api_key="YOUR_ANTHROPIC_API_KEY",
    base_url="https://api.anthropic.com/v1/",
    model="claude-haiku-4-5",
    extra_body={"thinking": {"type": "disabled"}},
    tts=tts,
    context_manager=context_manager,
)
```

--------------------------------

### Initialize VRChatFaceController

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Set up the VRChatFaceController with a mapping of expression names to FaceOSC values. Ensure 'neutral: 0' is always included.

```python
vrc_face_controller = VRChatFaceController(
    faces={
        "neutral": 0,   # always set `neutral: 0`

        # key = the name that LLM can understand the expression
        # value = FaceOSC value that is set to the transition on the FX animator controller
        "joy": 1,
        "angry": 2,
        "sorrow": 3,
        "fun": 4
    }
)
```

--------------------------------

### Initialize Character Service with PostgreSQL

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Instantiate the `CharacterService` and provide the configured `PostgreSQLPoolProvider`. This ensures that character data is managed using PostgreSQL.

```python
# Character
from aiavatar.character import CharacterService
character_service = CharacterService(
    openai_api_key=OPENAI_API_KEY,
    db_pool_provider=pool_provider,     # Creates PostgreSQLCharacterRepository and PostgreSQLActivityRepository internally
)
```

--------------------------------

### VRChat Animation Control Example

Source: https://github.com/uezo/aiavatarkit/blob/main/FEATURES.md

Example of how an LLM response containing an animation tag is translated into an OSC message to trigger VRChat avatar animations.

```python
# LLM response example: "See you! [animation:waving_arm]"
# → Send "/avatar/parameters/VRCEmote" = 3 to VRChat
# → Avatar waves
```

--------------------------------

### VRChat Expression Control Example

Source: https://github.com/uezo/aiavatarkit/blob/main/FEATURES.md

Example of how an LLM response containing a face tag is translated into an OSC message to control VRChat avatar expressions.

```python
# LLM response example: "I'm happy! [face:joy]"
# → Send "/avatar/parameters/FaceOSC" = 1 to VRChat
# → Avatar smiles
```

--------------------------------

### Create LINE Bot Adapter

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Initialize the AIAvatarLineBotServer with necessary credentials and configuration for a LINE Bot. Ensure the webhook URL is registered in the LINE Developers Console.

```python
# NOTE: Register https://{your.domain}/webhook as the "Webhook URL" in LINE Developers Console

# Create LINE Bot adapter
from aiavatar.adapter.linebot.server import AIAvatarLineBotServer
aiavatar_app = AIAvatarLineBotServer(
    openai_model="gpt-5.1",
    system_prompt="You are a cat.",
    openai_api_key=OPENAI_API_KEY,
    channel_access_token=LINEBOT_CHANNEL_ACCESS_TOKEN,
    channel_secret=LINEBOT_CHANNEL_SECRET,
    image_download_url_base="https://{your.domain}",
    debug=True
)

# Create FastAPI app
from fastapi import FastAPI
app = FastAPI()
```

--------------------------------

### Format Few-Shot Messages

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Inject few-shot examples into the main LLM to prevent repetition of quick responses.

```python
@character_loader.format_messages
def format_messages(messages):
    messages.append({"role": "user", "content": quick_responder_pro.prompt_prefix + "\n\nHello!"})
    messages.append({"role": "assistant", "content": f"<think>{quick_responder_pro.think_tag_content}</think><answer>Hello!</answer>"})
    messages.append({"role": "user", "content": quick_responder_pro.request_prefix.format(quick_response_text="Hello!") + "\n\nHello!"})
    messages.append({"role": "assistant", "content": "<think>Respond warmly to the greeting.</think><answer>How can I help you today?</answer>"})
    messages.append({"role": "user", "content": "You repeated 'Hello!' which was already sent. Always continue from where the previous output left off."})
    messages.append({"role": "assistant", "content": "<think>Noted the mistake. Will not repeat already-sent text next time.</think><answer>Got it.</answer>"})
    return messages
```

--------------------------------

### Run Python Client

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Execute the Python client script to initiate voice interactions. Save the client code as `client.py`.

```sh
python client.py
```

--------------------------------

### Message Box UI Setup

Source: https://github.com/uezo/aiavatarkit/blob/main/examples/websocket/html/vrm.html

Sets up the message box section, including opacity and speed sliders, and toggles for speech display and menu visibility. Ensure 'applyMsgBoxOpacity', 'applyAutoHide', 'applyVnMenu', and 'saveDisplay' functions are defined elsewhere.

```javascript
const msgSection = document.createElement('div'); msgSection.appendChild(sectionHeading('Message box')); const grid = document.createElement('div'); grid.className = 'vrmi-sliders'; const label = document.createElement('span'); label.textContent = 'Opacity'; const slider = document.createElement('input'); slider.type = 'range'; slider.min = 0; slider.max = 100; slider.value = msgBoxOpacity; const display = document.createElement('span'); display.textContent = msgBoxOpacity + '%'; slider.addEventListener('input', () => { msgBoxOpacity = parseInt(slider.value); display.textContent = msgBoxOpacity + '%'; applyMsgBoxOpacity(); saveDisplay(); }); displaySlider = { slider, display }; grid.appendChild(label); grid.appendChild(slider); grid.appendChild(display); const speedLabel = document.createElement('span'); speedLabel.textContent = 'Speed'; const speedSlider = document.createElement('input'); speedSlider.type = 'range'; speedSlider.min = 1; speedSlider.max = 100; speedSlider.value = msgSpeed; const speedDisplay = document.createElement('span'); speedDisplay.textContent = msgSpeed; speedSlider.addEventListener('input', () => { msgSpeed = parseInt(speedSlider.value); twSpeed = Math.max(1, 101 - msgSpeed); speedDisplay.textContent = msgSpeed; saveDisplay(); }); speedSliderRef = { slider: speedSlider, display: speedDisplay }; grid.appendChild(speedLabel); grid.appendChild(speedSlider); grid.appendChild(speedDisplay); msgSection.appendChild(grid); const msgToggles = document.createElement('div'); msgToggles.style.cssText = 'margin-top:8px;display:flex;flex-direction:column;gap:10px'; const userTextToggle = makeToggle('Show user speech', showUserText, (v) => { showUserText = v; saveDisplay(); }); const aiTextToggle = makeToggle('Show AI speech', showAIText, (v) => { showAIText = v; saveDisplay(); }); const autoHideToggle = makeToggle('Auto-hide', autoHideMsgBox, (v) => { autoHideMsgBox = v; applyAutoHide(); saveDisplay(); }); const vnMenuToggle = makeToggle('Show menu buttons', showVnMenu, (v) => { showVnMenu = v; applyVnMenu(); saveDisplay(); }); msgToggles.appendChild(userTextToggle.row); msgToggles.appendChild(aiTextToggle.row); msgToggles.appendChild(autoHideToggle.row); msgToggles.appendChild(vnMenuToggle.row); msgSection.appendChild(msgToggles); panel.appendChild(msgSection);
```

--------------------------------

### Automated Cron Configuration

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Example cron jobs for scheduling daily diary and schedule generation tasks.

```bash
# Generate daily schedule at 6:00 AM
0 6 * * * /usr/bin/python3 /path/to/generate_schedule.py

# Generate diary at 11:00 PM
0 23 * * * /usr/bin/python3 /path/to/generate_diary.py
```

--------------------------------

### Initialize Three.js Scene and Renderer

Source: https://github.com/uezo/aiavatarkit/blob/main/examples/websocket/html/vrm.html

Sets up the core Three.js scene, perspective camera, and WebGL renderer. Ensures the renderer matches the window size and pixel density, and sets the color space for output.

```javascript
const canvas = document.getElementById('vrmCanvas');
const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(
  20,
  window.innerWidth / window.innerHeight,
  0.1,
  100
);
camera.position.set(0, 1.35, 2.5);
camera.lookAt(0, 1.35, 0);
const renderer = new THREE.WebGLRenderer({ canvas, alpha: true, antialias: true });
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.setPixelRatio(window.devicePixelRatio);
renderer.outputColorSpace = THREE.SRGBColorSpace;
```

--------------------------------

### Initialize AIAvatarKit HTTP Server

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Set up a basic RESTful API server using FastAPI and AIAvatarHttpServer for hosting AIAvatarKit. This enables independent, context-aware conversations via SSE.

```python
from fastapi import FastAPI
from aiavatar.adapter.http.server import AIAvatarHttpServer

# AIAvatar
aiavatar_app = AIAvatarHttpServer(
    openai_api_key=OPENAI_API_KEY,
    debug=True
)

```

--------------------------------

### Run AIAvatarKit WebSocket Server Command

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Command to start the Uvicorn server for the AIAvatarKit WebSocket application.

```bash
python -m uvicorn ws:app
```

--------------------------------

### AIAvatarKit and Lipsync Initialization

Source: https://github.com/uezo/aiavatarkit/blob/main/examples/websocket/html/mpt.html

Initializes the AIAvatar client and the LipsyncEngine with specific asset paths and audio processing options.

```javascript
// AIAvatar const aiavatar = new AIAvatarClient({ webSocketUrl: "../ws" }); // LipSync const lipsyncEngine = new LipsyncEngine({ elements: { video: document.getElementById("base-video"), mouthCanvas: document.getElementById("mouth-canvas"), stage: document.getElementById("stage") }, assets: { // Miuna video: "motionpngtuber/miuna/miuna_loop_mouthless.mp4", track: "motionpngtuber/miuna/mouth_track.json", mouth_closed: "motionpngtuber/miuna/mouth/closed.png", mouth_open: "motionpngtuber/miuna/mouth/open.png", mouth_half: "motionpngtuber/miuna/mouth/half.png", mouth_e: "motionpngtuber/miuna/mouth/e.png", mouth_u: "motionpngtuber/miuna/mouth/u.png" // Pinkchan // video: "motionpngtuber/assets/assets14/pinkchan_mouthless_h264.mp4", // track: "motionpngtuber/assets/assets14/mouth_track.json", // mouth_closed: "motionpngtuber/assets/assets14/mouth/closed.png", // mouth_open: "motionpngtuber/assets/assets14/mouth/open.png", // mouth_half: "motionpngtuber/assets/assets14/mouth/half.png", // mouth_e: "motionpngtuber/assets/assets14/mouth/e.png", // mouth_u: "motionpngtuber/assets/assets14/mouth/u.png" }, options: { hqAudioEnabled: true, // HQ Audio Mode sensitivity: 0, // Set threshold to max because RMS values are large debug: false } }); aiavatar.onPlaybackAnalyze = ({ rms, centroid01, tSec }) => { lipsyncEngine.processAudioData({ rms: rms * 0.2, // Scale down as values are still too large even at max threshold high: centroid01, low: 1 - centroid01 }); }; aiavatar.onResetFace = () => { lipsyncEngine.resetAudioStats(); }; // Close mouth on speech ends aiavatar.onPlaybackEnd = () => { if (lipsyncEngine) { lipsyncEngine.volume = 0; lipsyncEngine.smoothedHighRatio = 0; lipsyncEngine.setMouthState("closed", true); } }; // Camera const camera = new Camera({ videoElement: document.getElementById("cameraVideo"), canvasElement: document.getElementById("cameraCanvas"), onCapture: (imageDataUrl) => { const imageMessage = { type: "invoke", session_id: ui.sessionId, user_id: ui.userId, files: [{url: imageDataUrl}], allow_merge: false, wait_in_queue: true }; aiavatar.ws.send(JSON.stringify(imageMessage)); } }); // Shared UI const ui = new AvatarUI({ aiavatar: aiavatar, camera: camera, toolLabels: { "send_query_to_openclaw": "Asking OpenClaw🦞", }, onStop: () => { lipsyncEngine.volume = 0; lipsyncEngine.smoothedHighRatio = 0; lipsyncEngine.setMouthState("closed", true); } }); // Custom response handling aiavatar.onResponseReceived = (response) => { // Custom handling logic here // Common handling ui.handleResponse(response); } // Console helper: chat("Hello.") or chat("What's this?", imageDataUrl) window.chat = (text, imageDataUrl) => aiavatar.chat(ui.sessionId, ui.userId, text, imageDataUrl);
```

--------------------------------

### Register Tool via `add_tool`

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Demonstrates an alternative method for registering a tool, using the `add_tool` method with `is_dynamic=True` for dynamic tool mode.

```python
# Difine tool without `is_dynamic` for other use cases
weather_tool = Tool("get_weather", get_weather_spec, get_weather, instruction="...")

# Register tool via `add_tool` with `is_dynamic`
llm.add_tool(weather_tool, is_dynamic=True)

```

--------------------------------

### Control API - Listener Management

Source: https://github.com/uezo/aiavatarkit/blob/main/FEATURES.md

Endpoints for controlling the voice input listener, allowing it to be started, stopped, or its status checked.

```APIDOC
## POST /listener/start

### Description
Start voice listener.

### Method
POST

### Endpoint
/listener/start
```

```APIDOC
## POST /listener/stop

### Description
Stop voice listener.

### Method
POST

### Endpoint
/listener/stop
```

```APIDOC
## GET /listener/status

### Description
Get listener status.

### Method
GET

### Endpoint
/listener/status
```

--------------------------------

### Merge Consecutive Utterances

Source: https://github.com/uezo/aiavatarkit/blob/main/FEATURES.md

Example configuration for merging multiple user utterances into a single request within a specified time threshold.

```python
# Example: Merge consecutive utterances within 3 seconds
```

--------------------------------

### Retrieve OpenClaw Session Data

Source: https://github.com/uezo/aiavatarkit/blob/main/examples/websocket/openclaw.md

Example JSON output from the OpenClaw session list command, showing active session keys and metadata.

```json
{
  "path": "/home/{username}/.openclaw/agents/main/sessions/sessions.json",
  "count": 2,
  "activeMinutes": 60,
  "sessions": [
    {
      "key": "agent:main:discord:channel:XXXXXXXXXXXXXXXXXXX",
      "kind": "group",
      "updatedAt": 1771740483138,
      "ageMs": 4071,
      "sessionId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "systemSent": true,
      "abortedLastRun": false,
      "inputTokens": 114780,
      "outputTokens": 22,
      "totalTokens": 114780,
      "totalTokensFresh": true,
      "model": "gpt-5.1-codex",
      "modelProvider": "openai",
      "contextTokens": 400000
    },
    {
      "key": "agent:main:main",
      "kind": "direct",
      "updatedAt": 1771740336404,
      "ageMs": 150805,
      "sessionId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "systemSent": true,
      "abortedLastRun": false,
      "inputTokens": 56344,
      "outputTokens": 521,
      "totalTokens": 19070,
      "totalTokensFresh": true,
      "model": "gpt-5.1-codex",
      "modelProvider": "openai",
      "contextTokens": 400000
    }
  ]
}
```

--------------------------------

### Configure PostgreSQL session manager

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Initialize a PostgreSQLLineBotSessionManager and pass it to the AIAvatarLineBotServer.

```python
# Create PostgreSQLLineBotSessionManager
from aiavatar.adapter.linebot.session_manager.postgres import PostgreSQLLineBotSessionManager
linebot_session_manager = PostgreSQLLineBotSessionManager(
    host=DB_HOST,
    port=DB_PORT,
    dbname=DB_NAME,
    user=DB_USER,
    password=DB_PASSWORD
)

aiavatar_app = AIAvatarLineBotServer(
    openai_model="gpt-5.1",
    system_prompt="You are a cat.",
    openai_api_key=OPENAI_API_KEY,
    channel_access_token=LINEBOT_CHANNEL_ACCESS_TOKEN,
    channel_secret=LINEBOT_CHANNEL_SECRET,
    image_download_url_base="https://{your.domain}",
    session_manager=linebot_session_manager,    # <- Set PostgresSQL session manager
    debug=True
)
```

--------------------------------

### Initialize PGVector Speaker Registry

Source: https://github.com/uezo/aiavatarkit/blob/main/FEATURES.md

Sets up the speaker registry using the PostgreSQL vector store.

```python
from aiavatar.sts.stt.speaker_registry.pgvector import PGVectorStore

speaker_store = PGVectorStore(get_pool=pool_provider.get_pool)
speaker_registry = SpeakerRegistry(store=speaker_store)
```

--------------------------------

### Configure Stream-based Speech Detection

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Set minimum duration or text length thresholds to trigger recording start events in stream-based detectors.

```python
vad = SileroStreamSpeechDetector(
    speech_recognizer=speech_recognizer,
    on_recording_started_min_duration=1.5,    # Trigger after 1.5 sec of speech
    on_recording_started_min_text_length=2,   # OR trigger when text >= 2 chars
)
```

--------------------------------

### Configuration API (ConfigAPI)

Source: https://github.com/uezo/aiavatarkit/blob/main/FEATURES.md

API endpoints for dynamically getting and updating component settings, including STT, LLM, TTS, and component switching.

```APIDOC
## Configuration API (ConfigAPI)

### Description
Provides HTTP endpoints to get and update component settings dynamically.

### STT Configuration
Allows dynamic adjustment of speech recognition settings.

| Endpoint    | Method | Description              |
|-------------|--------|--------------------------|
| /stt/config | GET    | Get STT configuration    |
| /stt/config | POST   | Update STT configuration |

**Configurable items**: language, alternative_languages, timeout, debug, model, base_url, sample_rate.

### LLM Configuration
Enables real-time changes to AI behavior and model configurations.

| Endpoint    | Method | Description            |
|-------------|--------|------------------------|
| /llm/config | GET    | Get LLM configuration  |
| /llm/config | POST   | Update LLM configuration |

**Configurable items**: system_prompt, model, temperature, split_chars, voice_text_tag, use_dynamic_tools, etc.

### TTS Configuration
Allows dynamic adjustment of voice quality and speed.

| Endpoint    | Method | Description            |
|-------------|--------|------------------------|
| /tts/config | GET    | Get TTS configuration  |
| /tts/config | POST   | Update TTS configuration |

**Configurable items**: style_mapper, timeout, debug, voice, speaker, speed, pitch, volume, etc.

### Component Switching
Dynamically switch between STT, LLM, and TTS providers.

| Endpoint         | Method | Description                   |
|------------------|--------|-------------------------------|
| /sts/component   | GET    | Get available component list  |
| /sts/component   | POST   | Switch active component       |

### System Log
Access system logs for troubleshooting during operation.

| Endpoint        | Method | Description                     |
|-----------------|--------|---------------------------------|
| /system/log     | GET    | Get latest logs (max 1000 lines) |

```

--------------------------------

### Inject Dynamic System Prompt Parameters

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Uses placeholders in system prompts and provides values at runtime via system_prompt_params.

```python
aiavatar_app = AIAvatar(
    openai_api_key="YOUR_OPENAI_API_KEY",
    model="gpt-4o",
    system_prompt="User's name is {name}."
)
```

```python
aiavatar_app.sts.invoke(STSRequest(
    # (other fields omitted)
    system_prompt_params={"name": "Nekochan"}
))
```

--------------------------------

### Restore VRM from Cache on Startup

Source: https://github.com/uezo/aiavatarkit/blob/main/examples/websocket/html/vrm.html

Loads and restores a VRM model from the cache when the application starts. It also restores the saved camera state if available.

```javascript
cacheDB.get('vrm_file').then(async blob => {
  if (blob) {
    const url = URL.createObjectURL(blob);
    await loadVRM(url);
    restoreCameraState();
  }
}).catch(() => {});
```

--------------------------------

### Register MCP Tools

Source: https://github.com/uezo/aiavatarkit/blob/main/README.md

Connect to MCP servers using HTTP or Stdio and register tools to the LLM service.

```python
from aiavatar.sts.llm.chatgpt import ChatGPTService
llm = ChatGPTService(openai_api_key=OPENAI_API_KEY)

from aiavatar.sts.llm.tools.mcp import StreamableHttpMCP, StdioMCP

# MCP Server
mcp1 = StreamableHttpMCP(url=MCP1_URL)
mcp1.for_each_tool = llm.add_tool

# MCP Server with Auth
mcp2 = StreamableHttpMCP(url=MCP2_URL, headers={"Authorization": f"Bearer {MCP_JWT}"})
@mcp2.for_each_tool
def mcp2_tools(tool: Tool):
    # Do something here (e.g. edit schema or func)
    llm.add_tool(tool)

# MCP Server (Std I/O)
mcp3 = StdioMCP(server_script="weather.py") # supports .py and .js
mcp3.for_each_tool = llm.add_tool
```