### Start Node.js Application

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/nodejs/README.md

Starts a Node.js application. This command is used to run the main script of the project after all dependencies have been installed and configurations are set.

```bash
npm start
```

--------------------------------

### Install Node.js Dependencies

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/nodejs/README.md

Installs project dependencies using npm. Ensure you have Node.js and npm installed. This command is typically run after cloning a project and setting up environment variables.

```bash
npm install
```

--------------------------------

### Python FastAPI App Setup

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/python/simple-webchat/README.md

This snippet shows the basic setup of a FastAPI application, including essential imports and the main app instance. It also defines the root endpoint '/' which serves the main HTML template.

```python
import uvicorn
from fastapi import FastAPI, Request, Form, UploadFile, File
from fastapi.responses import HTMLResponse, RedirectResponse
from fastapi.templating import Jinja2Templates
from pathlib import Path

from simple_webchat.config import settings
from simple_webchat.openai_client import OpenAIClient
from simple_webchat.avatartalk_client import AvatarTalkClient

app = FastAPI()
templates = Jinja2Templates(directory="templates")

# Initialize clients
openai_client = OpenAIClient()
avatartalk_client = AvatarTalkClient()

@app.get("/", response_class=HTMLResponse)
async def get(request: Request):
    return templates.TemplateResponse("index.html", {"request": request})

# Add other routes here...

if __name__ == "__main__":
    uvicorn.run(app, host=settings.APP_HOST, port=settings.APP_PORT)
```

--------------------------------

### Bash Uv Server Start

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/python/simple-webchat/README.md

This bash command starts the FastAPI application using the `uv` ASGI server in app mode. It's a simpler way to run the application compared to uvicorn.

```bash
uv run simple-webchat
```

--------------------------------

### Running the LiveKit WebChat App with UV

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/python/livekit-webchat/README.md

This command shows how to run the FastAPI LiveKit WebChat application using the `uv` command-line tool. `uv` is a fast Python package installer and application runner. This is a streamlined way to start the development server. Ensure you have `uv` installed and the `.env` file configured.

```bash
uv run livekit-webchat
```

--------------------------------

### Bash Uvicorn Server Start

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/python/simple-webchat/README.md

This bash command starts the FastAPI application using uvicorn with hot-reloading enabled. It specifies the module, app instance, and port for the server.

```bash
uvicorn simple_webchat.app:app --reload --port 8000
```

--------------------------------

### Clone LiveKit Agents Repository and Navigate to Example

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/python/livekit-agents/README.md

These commands demonstrate how to clone the LiveKit Agents repository from GitHub and navigate into the specific directory for the AvatarTalk examples. This is a prerequisite for setting up and running the integration.

```bash
git clone https://github.com/livekit/agents.git
cd agents
cd examples/avatar_agents/avatartalk/
```

--------------------------------

### Environment Variable Setup (.env example)

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/nodejs/simple-webchat/README.md

This snippet shows the structure of the .env file, which is used to configure API keys and other settings for the application. It highlights required keys like OPENAI_API_KEY and AVATARTALK_API_KEY, along with optional parameters for customization.

```dotenv
OPENAI_API_KEY=sk-...
OPENAI_MODEL=gpt-4o-mini
OPENAI_STT_MODEL=whisper-1
AVATARTALK_API_KEY=at_...
AVATARTALK_API_BASE=https://api.avatartalk.ai
AVATARTALK_AVATAR=european_woman
AVATARTALK_EMOTION=neutral
AVATARTALK_LANGUAGE=en
AVATARTALK_DELAYED=false
APP_HOST=127.0.0.1
APP_PORT=8000
APP_DEBUG=true
```

--------------------------------

### Install Dependencies and Run AvatarTalk Agent Worker

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/python/livekit-agents/README.md

This snippet shows the final steps to run the AvatarTalk integration with LiveKit Agents. It involves installing the necessary Python dependencies using pip and then executing the agent worker script with the 'dev' argument.

```bash
pip install -r requirements.txt
python agent_worker.py dev
```

--------------------------------

### Create LiveKit Session Example Request

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/API.md

An example cURL command to demonstrate how to send a POST request to create a LiveKit session with the specified parameters.

```bash
curl -X POST https://api.avatartalk.ai/livekit/create-session \
  -H "Authorization: Bearer your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "room_name": "demo-room",
    "room_token": "eyJhbGc...",
    "listener_token": "eyJhbGc...",
    "livekit_url": "wss://my-livekit.com",
    "avatar": "japanese_woman",
    "emotion": "happy"
  }'
```

--------------------------------

### Node.js Server Setup for LiveKit WebChat

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/nodejs/livekit-webchat/README.md

Sets up an Express server for the LiveKit WebChat application. It handles environment variable loading, defines routes for the UI, session management, chat, voice input, and transcription. Dependencies include Express, Nunjucks, OpenAI SDK, LiveKit Server SDK, and WebSocket client.

```javascript
const express = require('express');
const nunjucks = require('nunjucks');
const dotenv = require('dotenv');

const {
  getChatCompletion,
  getAudioTranscription,
} = require('./openai_client');
const { mintRoom, mintUserToken } = require('./livekit_client');
const { infer } = require('./avatartalk_ws');

dotenv.config();

const app = express();
const port = process.env.APP_PORT || 8000;

// Configure Nunjucks
nunjucks.configure('templates', {
  autoescape: true,
  express: app,
});

app.use(express.json()); // for parsing application/json
app.use(express.urlencoded({ extended: true })); // for parsing application/x-www-form-urlencoded
app.use(express.static('public'));

// Routes
app.get('/', async (req, res) => {
  // Render the main chat UI
  res.render('index.html');
});

app.post('/session', async (req, res) => {
  // Mint a new LiveKit room and a user token for the frontend
  try {
    const roomName = `webchat-${Date.now()}`;
    const { token: roomToken } = await mintRoom(roomName);
    const { token: userToken } = await mintUserToken(roomName);
    res.json({
      roomName,
      roomToken,
      userToken,
    });
  } catch (error) {
    console.error('Error minting session:', error);
    res.status(500).json({ error: 'Failed to mint session' });
  }
});

app.post('/chat', async (req, res) => {
  // Handle text chat input
  const { roomName, message } = req.body;
  try {
    const completion = await getChatCompletion(message);
    await infer(roomName, completion);
    res.json({ reply: completion });
  } catch (error) {
    console.error('Error processing chat:', error);
    res.status(500).json({ error: 'Failed to process chat' });
  }
});

app.post('/voice', async (req, res) => {
  // Handle voice input (audio data)
  // This endpoint is a placeholder and would typically involve
  // receiving audio data, transcribing it, and then sending to infer.
  // For simplicity, we'll simulate a response.
  const { roomName, audioDataBase64 } = req.body;
  try {
    // In a real scenario, you'd decode audioDataBase64, transcribe it,
    // and then call infer.
    const transcription = 'Simulated transcription from voice input';
    const completion = await getChatCompletion(transcription);
    await infer(roomName, completion);
    res.json({ reply: completion, transcribed: transcription });
  } catch (error) {
    console.error('Error processing voice:', error);
    res.status(500).json({ error: 'Failed to process voice' });
  }
});

app.post('/transcribe', async (req, res) => {
  // Handle audio transcription request
  const { audioDataBase64 } = req.body;
  try {
    const transcription = await getAudioTranscription(Buffer.from(audioDataBase64, 'base64'));
    res.json({ transcription });
  } catch (error) {
    console.error('Error transcribing audio:', error);
    res.status(500).json({ error: 'Failed to transcribe audio' });
  }
});

app.listen(port, () => {
  console.log(`Server running at http://${process.env.APP_HOST || '127.0.0.1'}:${port}`);
});

```

--------------------------------

### Request Video from Text - Example Request

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/API.md

An example cURL command to demonstrate creating a video request from text and receiving a Lightning invoice.

```bash
curl -X POST https://api.avatartalk.ai/lightning/request-video/text \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Welcome to AvatarTalk! This is a demonstration of our text-to-speech technology."
  }'
```

--------------------------------

### Lightning Payment Workflow Example (cURL and jq)

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/API.md

This bash script demonstrates a complete Lightning payment workflow, starting with requesting a video from text, extracting the invoice and amount using jq, and preparing for payment. It uses cURL for API requests.

```bash
# Step 1: Request a video from text
RESPONSE=$(curl -X POST https://api.avatartalk.ai/lightning/request-video/text \
  -H "Content-Type: application/json" \
  -d '{"text": "Hello from AvatarTalk!"}')

# Extract invoice and amount
INVOICE=$(echo $RESPONSE | jq -r '.bolt11_invoice')
AMOUNT=$(echo $RESPONSE | jq -r '.amount')
echo "Invoice: $INVOICE"
echo "Amount: $AMOUNT sats"
```

--------------------------------

### Start Audio Recording (JavaScript)

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/python/simple-webchat/templates/index.html

This function initiates audio recording using the MediaRecorder API. It checks for existing media recorders and streams, stopping them if necessary. It then sets up a new MediaRecorder with the audio stream and defines an 'ondataavailable' handler to process recorded chunks. Event listeners are attached to a push-to-talk button ($ptt) to call this function on 'mousedown' and 'touchstart'. Limitations: requires user permission for microphone access.

```javascript
function startRecording() {
  try {
    if (mediaRecorder && mediaRecorder.state !== 'inactive') {
      mediaRecorder.stop();
    }
    navigator.mediaDevices.getUserMedia({ audio: true }).then(s => {
      stream = s;
      mediaRecorder = new MediaRecorder(stream);
      mediaRecorder.addEventListener('dataavailable', async (e) => {
        const blob = new Blob([e.data], { type: 'audio/webm' });
        const formData = new FormData();
        formData.append('file', blob, 'audio.webm');
        formData.append('avatar', $selAvatar.value);
        formData.append('emotion', $selEmotion.value);
        formData.append('language', $selLanguage.value);
        try {
          const resp = await fetch('/upload', {
            method: 'POST',
            body: formData
          });
          const data = await resp.json();
          if (!resp.ok) throw new Error(data.error || 'Upload failed');
          const assistant = data.assistant_text || '';
          addMsg('assistant', assistant);
          history.push({ role: 'assistant', content: assistant });
          const inf = data.inference || {};
          if (inf.mp4_url) {
            $video.src = inf.mp4_url;
            $videoWrap.classList.remove('hidden');
            try { await $video.play(); } catch { } 
          }
          const links = [];
          if (inf.mp4_url) links.push(`<a href="${inf.mp4_url}" target="_blank" rel="noopener">MP4</a>`);
          if (inf.html_url) links.push(`<a href="${inf.html_url}" target="_blank" rel="noopener">Viewer</a>`);
          $videoLinks.innerHTML = links.length ? `Links: ${links.join(' · ')}` : '';
        } catch (err) {
          addMsg('assistant', `Voice error: ${err.message || err}`);
        } finally {
          $ptt.disabled = false;
        }
      });
      mediaRecorder.start();
      $ptt.textContent = '● Recording...';
      $ptt.style.background = '#8a1b1b';
    }).catch(err => {
      addMsg('assistant', `Mic error: ${err.message || err}`);
    });
  } catch (err) {
    addMsg('assistant', `Mic error: ${err.message || err}`);
  }
}
```

--------------------------------

### Install Project Dependencies

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/python/youtube-rtmp-streamer/README.md

Installs the necessary project dependencies using uv, a fast Python package installer. It synchronizes packages and installs pre-commit hooks for code quality.

```shell
uv sync
pre-commit install
```

--------------------------------

### Copy Environment File

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/nodejs/README.md

Copies an example environment file to a new file for configuration. This is a common practice in Node.js projects to set up API keys and other sensitive information before running the application.

```bash
cp .env.example .env
```

--------------------------------

### LiveKit Web Chat Integration (Node.js)

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/README.md

Node.js LiveKit webchat example where the assistant speaks into a LiveKit room via the /ws/infer endpoint. Requires Node.js 18+ and specific LiveKit and API key configurations in the environment.

```bash
cp .env.example .env
# Set OPENAI_API_KEY, AVATARTALK_API_KEY, LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET in .env
npm install
npm start
# Access at http://127.0.0.1:8000
```

--------------------------------

### Running the LiveKit WebChat App with Uvicorn

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/python/livekit-webchat/README.md

This command demonstrates how to run the FastAPI LiveKit WebChat application using uvicorn, a Python ASGI server. It's useful for development and deployment, allowing for hot-reloading when code changes. Ensure you have `uv` installed and the `.env` file configured.

```bash
uv run uvicorn livekit_webchat.app:app --reload --port 8000
```

--------------------------------

### LiveKit Web Chat Integration (Python)

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/README.md

Python LiveKit webchat example where the assistant speaks into a LiveKit room via the /ws/infer endpoint. Requires Python 3.10+, uv, and specific LiveKit and API key configurations in the environment.

```bash
Create .env with OPENAI_API_KEY, AVATARTALK_API_KEY, LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET
uv run livekit-webchat
# Access at http://127.0.0.1:8000
```

--------------------------------

### API Inference (Streaming Output)

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/README.md

Example of how to call the AvatarTalk.ai inference endpoint with the `stream=true` parameter to receive MP4 video data streamed in real time. This is useful for live applications. Requires an API key for authentication.

```bash
curl -X POST https://api.avatartalk.ai/inference?stream=true \
     -H "Authorization: Bearer {YOUR_API_KEY}" \
     -d '{"text": "Streaming test.", "avatar": "african_woman", "emotion": "happy", "language": "en"}' > output.mp4
```

--------------------------------

### Node.js environment configuration example

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/nodejs/youtube-rtmp-streamer/README.md

Example of environment variable configuration for the Node.js YouTube RTMP streamer. This involves copying a template file and filling in required API keys and stream details.

```env
# Copy .env.example to .env and fill in values
OPENAI_API_KEY=your_openai_api_key
AVATARTALK_API_KEY=your_avatartalk_api_key
YOUTUBE_API_KEY=your_youtube_api_key
YOUTUBE_RTMP_URL=rtmp://a.rtmp.youtube.com/live2
YOUTUBE_STREAM_KEY=your_youtube_stream_key
# Optional keys below
# YOUTUBE_LIVE_ID=your_youtube_live_id
# AVATARTALK_AVATAR=default_avatar
# AVATARTALK_LANGUAGE=en
# AVATARTALK_TOPICS_FILE=topics.txt
# AVATARTALK_MODEL=gpt-4o-mini
```

--------------------------------

### API Inference (JSON Output)

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/README.md

Example of how to call the AvatarTalk.ai inference endpoint to generate a talking avatar video. This endpoint returns a JSON object containing URLs for the MP4 video and an HTML player. Requires an API key for authentication.

```bash
curl -X POST https://api.avatartalk.ai/inference \
     -H "Authorization: Bearer {YOUR_API_KEY}" \
     -d '{"text": "Hello, this is a test.", "avatar": "african_man", "emotion": "neutral", "language": "en"}'
```

--------------------------------

### Handle Voice Input and WebSocket for Audio Streaming (JavaScript)

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/python/livekit-webchat/templates/index.html

This snippet manages microphone input using the MediaRecorder API and streams audio data over a WebSocket connection. It handles starting and stopping recordings, configuring WebSocket parameters, and sending audio chunks. Dependencies include the MediaRecorder API and WebSocket API.

```javascript
async function startRecording() { try { const sessionId = "session-id-placeholder"; // Replace with actual session ID if (document.getElementById('stream-audio').checked) { // Open WS relay and stream chunks if (!sessionId) throw new Error('No session'); const params = new URLSearchParams({ session_id: sessionId, avatar: $selAvatar.value, emotion: $selEmotion.value, language: $selLanguage.value, increase_resolution: String($increase.checked), }); const url = `${wsScheme()}//${location.host}/ws/audio?${params.toString()}`; clearLivekitContainer(); audioWS = new WebSocket(url); audioWS.binaryType = 'arraybuffer'; audioWS.onopen = () => { mediaRecorder.start(250); }; audioWS.onerror = (e) => { addMsg('assistant', `Audio WS error`); }; audioWS.onclose = () => { // no-op }; mediaRecorder.ondataavailable = async (e) => { if (e.data && e.data.size && audioWS && audioWS.readyState === WebSocket.OPEN) { try { const buf = await e.data.arrayBuffer(); audioWS.send(buf); } catch { } } }; } else { mediaRecorder.start(); } $ptt.textContent = '● Recording...'; $ptt.style.background = '#8a1b1b'; } } catch (err) { addMsg('assistant', `Mic error: ${err.message || err}`); } } function stopRecording() { try { if (mediaRecorder && mediaRecorder.state !== 'inactive') { mediaRecorder.stop(); } if (audioWS && audioWS.readyState === WebSocket.OPEN) { audioWS.close(); } } finally { if (micStream) { micStream.getTracks().forEach(t => t.stop()); micStream = null; } $ptt.textContent = '🎙️ Hold to talk'; $ptt.style.background = ''; } } $ptt.addEventListener('mousedown', (e) => { e.preventDefault(); startRecording(); }); $ptt.addEventListener('touchstart', (e) => { e.preventDefault(); startRecording(); }); const stopEvents = ['mouseup', 'mouseleave', 'touchend', 'touchcancel']; stopEvents.forEach(ev => $ptt.addEventListener(ev, (e) => { e.preventDefault(); stopRecording(); })); // Initialize joinLiveKit().catch(err => addMsg('assistant', `Setup error: ${err.message || err}`));
```

--------------------------------

### POST /inference - Regular Video Generation

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/README.md

Generates a talking avatar video from text and returns URLs for the MP4 and HTML. The URLs are trigger URLs that initiate processing on first access and serve cached content subsequently.

```APIDOC
## POST /inference

### Description
Generates a talking avatar video from text and returns URLs for the MP4 and HTML. The URLs are trigger URLs that initiate processing on first access and serve cached content subsequently.

### Method
POST

### Endpoint
https://api.avatartalk.ai/inference

### Headers
- **Authorization** (string) - Required - `Bearer {YOUR_API_KEY}`

### Parameters (body)
- **text** (string) - Required - Text to be spoken by the avatar.
- **avatar** (string) - Required - Identifier for the avatar (e.g., `african_man`).
- **emotion** (string) - Required - Emotion of the avatar (e.g., `neutral`, `happy`).
- **language** (string) - Required - Language of the text (e.g., `en`, `es`, `fr`).
- **delayed** (boolean) - Optional - If true, returns trigger URLs without upfront processing.

### Request Example
```json
{
  "text": "Hello, this is a test.",
  "avatar": "african_man",
  "emotion": "neutral",
  "language": "en",
  "delayed": false
}
```

### Response
#### Success Response (200)
- **mp4_url** (string) - URL to the generated MP4 video.
- **html_url** (string) - URL to the HTML file.

#### Response Example
```json
{
  "mp4_url": "https://example.com/path/to/video.mp4",
  "html_url": "https://example.com/path/to/index.html"
}
```
```

--------------------------------

### Python Environment Configuration

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/python/simple-webchat/README.md

This Python module handles loading environment variables using `python-dotenv` and defines a settings class using Pydantic for validation. It specifies default values for various configuration parameters.

```python
from pydantic_settings import BaseSettings
from pathlib import Path


class Settings(BaseSettings):
    # OpenAI settings
    OPENAI_API_KEY: str
    OPENAI_MODEL: str = "gpt-4o-mini"
    OPENAI_STT_MODEL: str = "whisper-1"

    # AvatarTalk settings
    AVATARTALK_API_KEY: str
    AVATARTALK_API_BASE: str = "https://api.avatartalk.ai"
    AVATARTALK_AVATAR: str = "european_woman"
    AVATARTALK_EMOTION: str = "neutral"
    AVATARTALK_LANGUAGE: str = "en"
    AVATARTALK_DELAYED: bool = False

    # App settings
    APP_HOST: str = "127.0.0.1"
    APP_PORT: int = 8000
    APP_DEBUG: bool = True

    class Config:
        env_file = Path(__file__).parent.parent / ".env"
        env_file_encoding = 'utf-8'


settings = Settings()

```

--------------------------------

### LiveKit Client Connection and Event Handling (JavaScript)

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/python/livekit-webchat/templates/index.html

Establishes a connection to a LiveKit room, subscribes to audio and video tracks, and handles track subscription/unsubscription events. It appends attached media elements to a designated container and cleans them up when unsubscribed. Dependencies include the LivekitClient library.

```javascript
let sessionId = null;
let lkRoom = null;

function removeKind(container, tagNames) {
    const tags = Array.isArray(tagNames) ? tagNames : [tagNames];
    Array.from(container.children).forEach((el) => {
        const t = el.tagName.toLowerCase();
        if (tags.includes(t)) el.remove();
    });
}

function clearLivekitContainer() {
    const c = document.getElementById('livekitContainer');
    if (c) c.replaceChildren();
}

async function joinLiveKit() {
    const resp = await fetch('/session', {
        method: 'POST'
    });
    const data = await resp.json();
    if (!resp.ok) throw new Error(data.error || 'Failed to create session');
    sessionId = data.session_id;
    const token = data.token;
    const url = data.livekit_url;
    lkRoom = new LivekitClient.Room();
    await lkRoom.connect(url, token);

    lkRoom.on(LivekitClient.RoomEvent.TrackSubscribed, (track, publication, participant) => {
        const c = document.getElementById('livekitContainer');
        if (!c) return;
        // Replace previous of the same kind to avoid stacking
        if (track.kind === 'video') removeKind(c, 'video');
        if (track.kind === 'audio') removeKind(c, 'audio');

        const el = track.attach();
        el.setAttribute('playsinline', 'true');
        el.style.width = '100%';
        el.style.height = '100%';
        el.style.objectFit = 'cover';
        c.appendChild(el);
    });

    lkRoom.on(LivekitClient.RoomEvent.TrackUnsubscribed, (track) => {
        track.detach().forEach(el => el.remove());
    });
}
```

--------------------------------

### Python OpenAI Chat Client

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/python/simple-webchat/README.md

This Python class encapsulates interactions with the OpenAI API for chat completions and audio transcription. It uses environment variables for API keys and model configurations.

```python
import os
import openai
from pathlib import Path

from pydantic import BaseModel
from dotenv import load_dotenv

from simple_webchat.config import settings


class OpenAIChatResponse(BaseModel):
    role: str
    content: str


class OpenAIClient:
    def __init__(self):
        load_dotenv()
        openai.api_key = settings.OPENAI_API_KEY

    def chat(self, messages: list[dict]) -> str:
        try:
            response = openai.chat.completions.create(
                model=settings.OPENAI_MODEL,
                messages=messages,
            )
            return response.choices[0].message.content
        except Exception as e:
            print(f"Error calling OpenAI chat: {e}")
            return "Sorry, I encountered an error."

    def transcribe_audio(self, audio_file: Path) -> str:
        try:
            with open(audio_file, "rb") as f:
                transcript = openai.audio.transcriptions.create(
                    model=settings.OPENAI_STT_MODEL,
                    file=f,
                )
            return transcript.text
        except Exception as e:
            print(f"Error transcribing audio: {e}")
            return ""

```

--------------------------------

### Web Chat with Text and Streaming (Python)

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/README.md

Python (FastAPI) implementation of a text-first web chat. It supports optional voice input and streaming playback via a server proxy. Requires Python 3.10+ and uv for package management.

```bash
Create .env with OPENAI_API_KEY and AVATARTALK_API_KEY
uv run simple-webchat
# Access at http://127.0.0.1:8000
```

--------------------------------

### Express Server Setup and Routes (app.js)

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/nodejs/simple-webchat/README.md

This Node.js code defines the Express application, sets up middleware, and registers various API endpoints for handling chat, voice input, transcription, and streaming. It demonstrates how to manage requests and responses for different functionalities.

```javascript
import express from "express";
import path from "path";
import multer from "multer";
import OpenAI from "openai";
import dotenv from "dotenv";
import { v4 as uuidv4 } from "uuid";
import fs from "fs";

dotenv.config();

const app = express();
const port = process.env.APP_PORT || 8000;

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

// --- Middleware ---
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
app.use(express.static("public"));
app.set("view engine", "njk");

// --- In-memory storage for streams ---
const streams = {};

// --- Routes ---
app.get("/", (req, res) => {
  res.render("index");
});

app.get("/healthz", (req, res) => {
  res.status(200).send("OK");
});

// POST /chat - handles text-based chat
app.post("/chat", async (req, res) => {
  try {
    const userInput = req.body.message;
    const assistantReply = await getOpenAIChatCompletion(userInput);
    const avatartalkResponse = await callAvatarTalkAPI(assistantReply);
    res.json(avatartalkResponse);
  } catch (error) {
    console.error("Error in /chat:", error);
    res.status(500).json({ error: "Failed to process chat request" });
  }
});

// POST /voice - handles voice input (upload audio file)
const upload = multer({ dest: "uploads/" });
app.post("/voice", upload.single("audio"), async (req, res) => {
  if (!req.file) {
    return res.status(400).send("No audio file uploaded.");
  }
  try {
    const transcription = await transcribeAudio(req.file.path);
    const assistantReply = await getOpenAIChatCompletion(transcription);
    const avatartalkResponse = await callAvatarTalkAPI(assistantReply, true); // Assume voice input might need streaming
    res.json(avatartalkResponse);
  } catch (error) {
    console.error("Error in /voice:", error);
    res.status(500).json({ error: "Failed to process voice request" });
  } finally {
    // Clean up uploaded file
    fs.unlink(req.file.path, (err) => {
      if (err) console.error("Error deleting temp audio file:", err);
    });
  }
});

// POST /transcribe - direct audio transcription
app.post("/transcribe", upload.single("audio"), async (req, res) => {
  if (!req.file) {
    return res.status(400).send("No audio file uploaded.");
  }
  try {
    const transcription = await transcribeAudio(req.file.path);
    res.json({ transcription });
  } catch (error) {
    console.error("Error in /transcribe:", error);
    res.status(500).json({ error: "Failed to transcribe audio" });
  } finally {
    fs.unlink(req.file.path, (err) => {
      if (err) console.error("Error deleting temp audio file:", err);
    });
  }
});

// POST /chat_stream - handles streaming chat response
app.post("/chat_stream", async (req, res) => {
  try {
    const userInput = req.body.message;
    const assistantReply = await getOpenAIChatCompletion(userInput);
    const streamId = uuidv4();
    streams[streamId] = {
      status: "pending",
      response: null,
      chunks: [],
      createdAt: Date.now()
    };
    callAvatarTalkStreamAPI(assistantReply, streamId, res);
  } catch (error) {
    console.error("Error in /chat_stream:", error);
    res.status(500).json({ error: "Failed to initiate streaming chat" });
  }
});

// GET /stream/{id}.mp4 - serves the streamed MP4 video
app.get("/stream/:id.mp4", async (req, res) => {
  const streamId = req.params.id;
  const streamData = streams[streamId];

  if (!streamData) {
    return res.status(404).send("Stream not found or expired.");
  }

  if (streamData.status === "error") {
      return res.status(500).send("Stream generation failed.");
  }

  // Set appropriate headers for video streaming
  res.setHeader("Content-Type", "video/mp4");
  res.setHeader("Transfer-Encoding", "chunked");
  res.setHeader("Connection", "keep-alive");
  res.setHeader("Cache-Control", "no-cache");

  // Stream the chunks
  streamData.chunks.forEach(chunk => res.write(chunk));

  // Handle stream completion or errors
  const interval = setInterval(() => {
    if (streamData.status === "completed") {
      clearInterval(interval);
      // Optionally clean up stream data after a short delay
      setTimeout(() => delete streams[streamId], 60000); // Clean up after 1 minute
    } else if (streamData.status === "error") {
      clearInterval(interval);
      res.end(); // End the response on error
    }
  }, 100); // Check every 100ms

  // Handle client disconnection
  req.on("close", () => {
    clearInterval(interval);
    delete streams[streamId];
  });
});

// --- Helper Functions ---

async function getOpenAIChatCompletion(prompt) {
  try {
    const completion = await openai.chat.completions.create({
      messages: [{ role: "user", content: prompt }],
      model: process.env.OPENAI_MODEL || "gpt-4o-mini",
    });
    return completion.choices[0]?.message?.content;
  } catch (error) {
    console.error("Error calling OpenAI:", error);
    throw error;
  }
}

async function transcribeAudio(filePath) {
  try {
    const file = fs.createReadStream(filePath);
    const transcription = await openai.audio.transcriptions.create({
      file: file,
      model: process.env.OPENAI_STT_MODEL || "whisper-1",
    });
    return transcription.text;
  } catch (error) {
    console.error("Error transcribing audio:", error);
    throw error;
  }
}

async function callAvatarTalkAPI(text, isVoiceInput = false) {
  const apiUrl = process.env.AVATARTALK_API_BASE || "https://api.avatartalk.ai";
  const apiKey = process.env.AVATARTALK_API_KEY;

  if (!apiKey) {
    throw new Error("AVATARTALK_API_KEY is not set.");
  }

  const response = await fetch(`${apiUrl}/inference`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "Authorization": `Bearer ${apiKey}`
    },
    body: JSON.stringify({
      text: text,
      avatar: process.env.AVATARTALK_AVATAR || "european_woman",
      emotion: process.env.AVATARTALK_EMOTION || "neutral",
      language: process.env.AVATARTALK_LANGUAGE || "en",
      delayed: process.env.AVATARTALK_DELAYED === 'true' || false,
      stream: isVoiceInput ? true : false, // Example: force stream for voice input
    }),
  });

  if (!response.ok) {
    const errorBody = await response.text();
    console.error(`AvatarTalk API Error: ${response.status} - ${errorBody}`);
    throw new Error(`AvatarTalk API request failed: ${response.status}`);
  }

  return response.json();
}

async function callAvatarTalkStreamAPI(text, streamId, res) {
  const apiUrl = process.env.AVATARTALK_API_BASE || "https://api.avatartalk.ai";
  const apiKey = process.env.AVATARTALK_API_KEY;

  if (!apiKey) {
    streams[streamId].status = "error";
    console.error("AVATARTALK_API_KEY is not set.");
    return res.status(500).send("Server configuration error.");
  }

  try {
    const streamResponse = await fetch(`${apiUrl}/inference?stream=true`, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "Authorization": `Bearer ${apiKey}`
      },
      body: JSON.stringify({
        text: text,
        avatar: process.env.AVATARTALK_AVATAR || "european_woman",
        emotion: process.env.AVATARTALK_EMOTION || "neutral",
        language: process.env.AVATARTALK_LANGUAGE || "en",
        delayed: false, // Streaming implies not delayed
      }),
    });

    if (!streamResponse.ok) {
      const errorBody = await streamResponse.text();
      console.error(`AvatarTalk Stream API Error: ${streamResponse.status} - ${errorBody}`);
      streams[streamId].status = "error";
      return res.status(500).send("Failed to start video stream.");
    }

    const reader = streamResponse.body.getReader();
    let videoChunks = [];
    let isFirstChunk = true;

    while (true) {
      const { done, value } = await reader.read();
      if (done) {
        streams[streamId].status = "completed";
        // The response is already sent to the client via res.write, no need to send final JSON here
        break;
      }

      // Assuming the response is chunked MP4 data
      const chunk = Buffer.from(value);
      streams[streamId].chunks.push(chunk);

      // Send the first chunk with appropriate headers if not already sent
      if (isFirstChunk) {
          res.status(200);
          // Headers are set in the GET /stream/:id.mp4 route handler
          isFirstChunk = false;
      }
      res.write(chunk);
    }

  } catch (error) {
    console.error("Error during AvatarTalk streaming:", error);
    streams[streamId].status = "error";
    if (!res.writableEnded) {
        res.status(500).send("Error during video stream generation.");
    }
  }
}

// --- Server Start ---
app.listen(port, () => {
  console.log(`Server running at http://${process.env.APP_HOST || '127.0.0.1'}:${port}`);
});

```

--------------------------------

### Web Chat with Text and Streaming (Node.js)

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/README.md

Node.js (Express) implementation of a text-first web chat. It supports optional voice input and streaming playback via a server proxy. Requires Node.js 18+ and environment variables for API keys.

```bash
cp .env.example .env
# Set OPENAI_API_KEY and AVATARTALK_API_KEY in .env
npm install
npm start
# Access at http://127.0.0.1:8000
```

--------------------------------

### OpenAI Chat and Speech-to-Text Client (Node.js)

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/nodejs/livekit-webchat/README.md

Provides utility functions to interact with the OpenAI API for chat completions and audio transcription. It uses the 'openai' npm package and requires the OPENAI_API_KEY environment variable. The functions abstract the API calls for easier integration into the web chat application.

```javascript
const OpenAI = require('openai');

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

async function getChatCompletion(prompt) {
  // Generates a chat completion response from OpenAI.
  // Uses OPENAI_MODEL environment variable, defaults to 'gpt-4o-mini'.
  // Input: string (user prompt)
  // Output: string (AI's response)
  try {
    const completion = await openai.chat.completions.create({
      messages: [{ role: 'user', content: prompt }],
      model: process.env.OPENAI_MODEL || 'gpt-4o-mini',
    });
    return completion.choices[0].message.content;
  } catch (error) {
    console.error('Error getting chat completion:', error);
    throw error;
  }
}

async function getAudioTranscription(audioBuffer) {
  // Transcribes audio data to text using OpenAI's Whisper model.
  // Uses OPENAI_STT_MODEL environment variable, defaults to 'whisper-1'.
  // Input: Buffer (audio data)
  // Output: string (transcribed text)
  try {
    const transcription = await openai.audio.transcriptions.create({
      file: {
        // The file needs to be readable by the OpenAI SDK. 
        // For simplicity, we assume audioBuffer is directly usable or converted.
        // In a real application, you might need to save to a temporary file or stream.
        name: 'audio.webm', // Or appropriate filename based on MIME type
        blob: audioBuffer, // Directly passing buffer if SDK supports, otherwise needs adaptation
      },
      model: process.env.OPENAI_STT_MODEL || 'whisper-1',
    });
    return transcription.text;
  } catch (error) {
    console.error('Error getting audio transcription:', error);
    throw error;
  }
}

module.exports = {
  getChatCompletion,
  getAudioTranscription,
};

```

--------------------------------

### LiveKit Token Minting Client (Node.js)

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/nodejs/livekit-webchat/README.md

Manages the creation of JWT tokens for LiveKit. It includes functions to mint a token for creating a room and another for joining a room as a user. This is crucial for authenticating clients with the LiveKit server. Requires LiveKit URL, API Key, and API Secret from environment variables.

```javascript
const { AccessToken } = require('livekit-server-sdk');

const LIVEKIT_URL = process.env.LIVEKIT_URL;
const LIVEKIT_API_KEY = process.env.LIVEKIT_API_KEY;
const LIVEKIT_API_SECRET = process.env.LIVEKIT_API_SECRET;
const LIVEKIT_TOKEN_TTL = parseInt(process.env.LIVEKIT_TOKEN_TTL || '3600', 10);

function mintRoom(roomName) {
  // Mints a token that allows creating a LiveKit room.
  // This token is typically used by the server to manage rooms.
  // Input: string (roomName)
  // Output: Promise resolving to { token: string }
  const at = new AccessToken(LIVEKIT_API_KEY, LIVEKIT_API_SECRET, {
    identity: 'server-admin',
    ttl: LIVEKIT_TOKEN_TTL,
  });
  at.addGrant({ roomCreate: true }); // Grant permission to create rooms
  const token = at.toJwt();
  return Promise.resolve({ token });
}

function mintUserToken(roomName) {
  // Mints a token for a user to join a specific LiveKit room.
  // The frontend JavaScript will use this token.
  // Input: string (roomName)
  // Output: Promise resolving to { token: string }
  const at = new AccessToken(LIVEKIT_API_KEY, LIVEKIT_API_SECRET, {
    identity: `user-${Date.now()}`,
    ttl: LIVEKIT_TOKEN_TTL,
  });
  // Grant permissions for joining and subscribing to the room
  at.addGrant({ roomJoin: true, room: roomName });
  const token = at.toJwt();
  return Promise.resolve({ token });
}

module.exports = {
  mintRoom,
  mintUserToken,
};

```

--------------------------------

### Python Simple Web Chat with FastAPI

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/python/README.md

A full-stack web chat application using FastAPI and Jinja2. It supports text chat, push-to-talk transcription, and optional streaming playback. Requires Python 3.10+ and environment variables for API keys.

```python
# Example usage (conceptual, actual code in project)
from fastapi import FastAPI, Request
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates

app = FastAPI()
templates = Jinja2Templates(directory="templates")

@app.get("/", response_class=HTMLResponse)
async def read_root(request: Request):
    return templates.TemplateResponse("index.html", {"request": request})

# Additional endpoints for chat, transcription, etc. would be here.
# Requires .env with OPENAI_API_KEY and AVATARTALK_API_KEY
# Run with: uv run simple-webchat

```

--------------------------------

### Python LiveKit Webchat Integration

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/python/README.md

Integrates AvatarTalk AI with LiveKit for real-time communication. The assistant speaks into a LiveKit room via a WebSocket endpoint. Requires Python 3.10+ and LiveKit/OpenAI API credentials in environment variables.

```python
# Example usage (conceptual, actual code in project)
import asyncio
from fastapi import FastAPI, WebSocket

app = FastAPI()

@app.websocket("/ws/infer")
async def websocket_endpoint(websocket: WebSocket):
    await websocket.accept()
    while True:
        data = await websocket.receive_text()
        # Process data, interact with AvatarTalk AI and LiveKit
        await websocket.send_text(f"Message text was: {data}")

# Requires .env with OPENAI_API_KEY, AVATARTALK_API_KEY, LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET
# Run with: uv run livekit-webchat

```

--------------------------------

### Regular Inference Request Example (JSON Response)

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/API.md

Shows the structure of a successful JSON response from a regular POST /inference request. It includes details like video URLs, status, and consumption metrics.

```json
{
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "status": "success",
  "stream": false,
  "text": "Hello, this is a test message",
  "created_at": "2025-09-29T11:50:26.890669Z",
  "language": "en",
  "credits_consumed": 5,
  "avatar": "african_man",
  "emotion": "neutral",
  "file_size_bytes": 2048576,
  "inference_duration_ms": 3500,
  "video_duration_seconds": 4.2,
  "html_url": "https://api.avatartalk.ai/inference/123e4567-e89b-12d3-a456-426614174000/video.html",
  "mp4_url": "https://api.avatartalk.ai/inference/123e4567-e89b-12d3-a456-426614174000/video.mp4"
}
```

--------------------------------

### POST /inference?stream=true - Real-time Streaming Video

Source: https://github.com/avatartalk-ai/avatartalk-examples/blob/main/README.md

Generates a talking avatar video from text and streams the MP4 video data in real time. This is useful for live applications where immediate video output is required.

```APIDOC
## POST /inference?stream=true

### Description
Generates a talking avatar video from text and streams the MP4 video data in real time. This is useful for live applications where immediate video output is required.

### Method
POST

### Endpoint
https://api.avatartalk.ai/inference?stream=true

### Headers
- **Authorization** (string) - Required - `Bearer {YOUR_API_KEY}`

### Parameters (body)
- **text** (string) - Required - Text to be spoken by the avatar.
- **avatar** (string) - Required - Identifier for the avatar (e.g., `african_man`).
- **emotion** (string) - Required - Emotion of the avatar (e.g., `neutral`, `happy`).
- **language** (string) - Required - Language of the text (e.g., `en`, `es`, `fr`).
- **delayed** (boolean) - Optional - If true, returns trigger URLs without upfront processing.

### Request Example
```json
{
  "text": "Streaming video example.",
  "avatar": "asian_woman",
  "emotion": "happy",
  "language": "en",
  "delayed": false
}
```

### Response
#### Success Response (200)
- **(binary)** - MP4 video data streamed in real time.

#### Response Example
*(Binary stream of MP4 data)*
```