Simli

https://docs.simli.com/
Admin
Simli is a speech-to-video API that enables developers to create lip-synced AI avatar interactions...
Tokens:40,673
Snippets:251
Trust Score:2.8
Update:6 months ago
Show doc for...
Tokens:
Raw
### Install Simli Client

Source: https://docs.simli.com/api-reference/javascript

Install the required package via npm.

```bash
npm install simli-client
```

--------------------------------

### Start Peer Connection and Negotiation

Source: https://docs.simli.com/api-reference/simli-webrtc

Initializes a peer connection, adds audio and video transceivers for receiving, and starts the negotiation process. Requires a 'createPeerConnection' function to be defined elsewhere.

```javascript
function start() {
	startStamp = new Date().getTime();
	document.getElementById("start").style.display = "none";

	pc = createPeerConnection().then((pc) => {
		document.getElementById("media").style.display = "block";
		pc.addTransceiver("audio", {direction: "recvonly"});
		pc.addTransceiver("video", {direction: "recvonly"});
		negotiate();

		document.getElementById("stop").style.display = "inline-block";
	});
}
```

--------------------------------

### Start and Stop Peer Connection

Source: https://docs.simli.com/api-reference/simli-webrtc

Functions to initialize media acquisition, start the negotiation process, and clean up resources when stopping the connection.

```javascript
function start() {
	document.getElementById("start").style.display = "none";

	pc = createPeerConnection();

	var time_start = null;

	const current_stamp = () => {
		if (time_start === null) {
			time_start = new Date().getTime();
			return 0;
		} else {
			return new Date().getTime() - time_start;
		}
	};

	// Build media constraints.

	const constraints = {
		audio: true,
		video: true,
	};

	// Acquire media and start negotiation.

	document.getElementById("media").style.display = "block";
	navigator.mediaDevices.getUserMedia(constraints).then(
		(stream) => {
			stream.getTracks().forEach((track) => {
				pc.addTrack(track, stream);
			});
			return negotiate();
		},
		(err) => {
			alert("Could not acquire media: " + err);
		},
	);
	document.getElementById("stop").style.display = "inline-block";
}

function stop() {
	document.getElementById("stop").style.display = "none";

	// close transceivers
	if (pc.getTransceivers) {
		pc.getTransceivers().forEach((transceiver) => {
			if (transceiver.stop) {
				transceiver.stop();
			}
		});
	}

	// close local audio / video
	pc.getSenders().forEach((sender) => {
		sender.track.stop();
	});

	// close peer connection
	setTimeout(() => {
		pc.close();
	}, 500);
}
```

--------------------------------

### GET /auto/start/{agent_id}/{session_token}

Source: https://docs.simli.com/api-reference/openapi_doc

Starts a Simli Auto session using a specific agent ID and session token.

```APIDOC
## GET /auto/start/{agent_id}/{session_token}

### Description
Start Auto Session from config.

### Method
GET

### Endpoint
/auto/start/{agent_id}/{session_token}

### Parameters
#### Path Parameters
- **agent_id** (string) - Required - Unique identifier for the agent.
- **session_token** (string) - Required - Token for the session.

#### Query Parameters
- **redirect** (boolean) - Optional - Whether to redirect (default: false).

### Response
#### Success Response (200)
- **SessionDetails** (object) - Details of the started session.

#### Response Example
{
  "session_id": "string",
  "status": "active"
}
```

--------------------------------

### Install simli-ai Package

Source: https://docs.simli.com/api-reference/python

Install the Simli AI Python package using pip. Ensure you have Python 3.9 or higher.

```bash
pip install simli-ai
```

--------------------------------

### Install Pipecat dependencies

Source: https://docs.simli.com/api-reference/pipecat

Install the required Pipecat packages and supporting libraries.

```bash
pip install "pipecat-ai[simli]"
pip install "pipecat-ai[deepgram]"
pip install "pipecat-ai[cartesia]"
pip install "pipecat-ai[openai]"
pip install "pipecat-ai[elevenlabs]"
pip install "pipecat-ai[webrtc]"
pip install fastapi dotenv loguru
... etc
```

--------------------------------

### Configure environment variables

Source: https://docs.simli.com/api-reference/pipecat

Example content for the .env file containing necessary API keys and IDs.

```bash
CARTESIA_API_KEY=...
CARTESIA_VOICE_ID=...
DEEPGRAM_API_KEY=...
ELEVENLABS_API_KEY=...
ELEVENLABS_VOICE_ID=...
OPENAI_API_KEY=...
SIMLI_API_KEY=...
SIMLI_FACE_ID=...
```

--------------------------------

### Install Python Dependencies

Source: https://docs.simli.com/api-reference/livekit

Installs the required Python packages for LiveKit agents, OpenAI integration, Simli plugins, and environment variable loading.

```bash
pip install \
"livekit-agents[openai]" \
"livekit-plugins-simli" \
"python-dotenv"
```

--------------------------------

### Implement Pipecat runner

Source: https://docs.simli.com/api-reference/pipecat

A complete example of a Pipecat runner file configured for Simli video services.

```python
import os

from dotenv import load_dotenv
from loguru import logger
from simli import SimliConfig

from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.runner.types import RunnerArguments
from pipecat.runner.utils import create_transport
from pipecat.services.cartesia.tts import CartesiaTTSService
from pipecat.services.deepgram.stt import DeepgramSTTService
from pipecat.services.openai.llm import OpenAILLMService
from pipecat.services.simli.video import SimliVideoService
from pipecat.transports.base_transport import BaseTransport, TransportParams
from pipecat.transports.services.daily import DailyParams

load_dotenv(override=True)

transport_params = {
    "daily": lambda: DailyParams(
        audio_in_enabled=True,
        audio_out_enabled=True,
        video_out_enabled=True,
        video_out_is_live=True,
        video_out_width=512,
        video_out_height=512,
        vad_analyzer=SileroVADAnalyzer(),
    ),
    "webrtc": lambda: TransportParams(
        audio_in_enabled=True,
        audio_out_enabled=True,
        video_out_enabled=True,
        video_out_is_live=True,
        video_out_width=512,
        video_out_height=512,
        vad_analyzer=SileroVADAnalyzer(),
    ),
}


async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
    logger.info(f"Starting bot")

    stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))

    tts = CartesiaTTSService(
        api_key=os.getenv("CARTESIA_API_KEY"),
        voice_id=os.getenv("CARTESIA_VOICE_ID"),
    )
    simli_ai = SimliVideoService(
        SimliConfig(os.getenv("SIMLI_API_KEY"), os.getenv("SIMLI_FACE_ID")),
    )


    llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"), model="gpt-4o-mini")
    print("OPENAI_API_KEY:", os.getenv("OPENAI_API_KEY"))

    messages = [
        {
        "role": "system",
        "content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way."
        },
    ]

    context = OpenAILLMContext(messages)
    context_aggregator = llm.create_context_aggregator(context)

    pipeline = Pipeline(
        [
            transport.input(),
            stt,
            context_aggregator.user(),
            llm,
            tts,
            simli_ai,
            transport.output(),
            context_aggregator.assistant(),
        ]
    )

    task = PipelineTask(
        pipeline,
        params=PipelineParams(
            enable_metrics=True,
            enable_usage_metrics=True,
        ),
        idle_timeout_secs=runner_args.pipeline_idle_timeout_secs,
    )

    @transport.event_handler("on_client_connected")
    async def on_client_connected(transport, client):
        logger.info(f"Client connected")
        await task.queue_frames([context_aggregator.user().get_context_frame()])

    @transport.event_handler("on_client_disconnected")
    async def on_client_disconnected(transport, client):
        logger.info(f"Client disconnected")
        await task.cancel()

    runner = PipelineRunner(handle_sigint=runner_args.handle_sigint)
    await runner.run(task)


async def bot(runner_args: RunnerArguments):
    """Main bot entry point compatible with Pipecat Cloud."""
    transport = await create_transport(runner_args, transport_params)
    await run_bot(transport, runner_args)


if __name__ == "__main__":
    from pipecat.runner.run import main

    main()
```

--------------------------------

### GET /auto/start/{agent_id}/{session_token}

Source: https://docs.simli.com/api-reference/start-auto-session-from-config

Starts an auto session using the provided agent ID and session token. Note that this endpoint is currently marked as deprecated.

```APIDOC
## GET /auto/start/{agent_id}/{session_token}

### Description
Starts an auto session from configuration using the specified agent ID and session token.

### Method
GET

### Endpoint
/auto/start/{agent_id}/{session_token}

### Parameters
#### Path Parameters
- **agent_id** (string) - Required - The unique identifier for the agent.
- **session_token** (string) - Required - The session token for authentication.

#### Query Parameters
- **redirect** (boolean) - Optional - Whether to redirect. Defaults to false.

### Response
#### Success Response (200)
- **roomUrl** (string) - The URL for the session room.
- **sessionId** (string) - The unique identifier for the session.

#### Response Example
{
  "roomUrl": "https://example.com/room",
  "sessionId": "session-123"
}
```

--------------------------------

### POST /auto/start/configurable

Source: https://docs.simli.com/api-reference/start-auto-session

Starts a new Simli Auto Session without a preconfigured agent. This endpoint is currently marked as deprecated.

```APIDOC
## POST /auto/start/configurable

### Description
Start a new Simli Auto Session without a preconfigured agent.

### Method
POST

### Endpoint
https://api.simli.ai/auto/start/configurable

### Request Body
- **faceId** (string) - Optional - Default: tmp9i8bbq7c
- **ttsProvider** (string) - Optional - Default: Cartesia (Options: ElevenLabs, PlayHT, Cartesia, User)
- **ttsAPIKey** (string) - Optional
- **ttsModel** (string) - Optional - Default: sonic-turbo-2025-03-07
- **voiceId** (string) - Optional - Default: a167e0f3-df7e-4d52-a9c3-f949145efdab
- **systemPrompt** (string) - Optional
- **firstMessage** (string) - Optional
- **maxSessionLength** (integer) - Optional - Default: 3600
- **maxIdleTime** (integer) - Optional - Default: 300
- **language** (string) - Optional - Default: en
- **customLLMConfig** (object) - Optional
- **llmConfig** (object) - Optional - Default: { "model": "gpt-4o-mini", "provider": "OpenAI" }
- **createTranscript** (boolean) - Optional - Default: false
- **model** (string) - Optional - Default: artalk (Options: fasttalk, artalk)

### Request Example
{
  "faceId": "tmp9i8bbq7c",
  "ttsProvider": "Cartesia",
  "model": "artalk"
}

### Response
#### Success Response (200)
- **roomUrl** (string) - The URL for the session room
- **sessionId** (string) - The unique identifier for the session

#### Response Example
{
  "roomUrl": "https://example.com/room",
  "sessionId": "session-12345"
}
```

--------------------------------

### Legacy SimliClient Initialization

Source: https://docs.simli.com/api-reference/javascript_upgrade_guide

Previous implementation pattern requiring separate Initialize and start calls.

```tsx
import {SimliClient} from "simli-client";

const simliClient = new SimliClient();

function initialize(face_id: string, videoRef: HTMLVideoElement, audioRef: HTMLAudioElement) {
	const SimliConfig = {
		apiKey: process.env.NEXT_PUBLIC_SIMLI_API_KEY,
		faceID: face_id,
		videoRef: videoRef,
		audioRef: audioRef,
	};

	simliClient.Initialize(SimliConfig as any);
}
async function start() {
	await simliClient.start();
}
```

--------------------------------

### Create Mock FastAPI Streaming Server

Source: https://docs.simli.com/api-reference/endpoint/simli-auto/customLLM

A complete FastAPI server example that streams mock completion chunks to a client.

```python
import time
from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse
import json

app = FastAPI()


async def streamOutput():
    data = {
        "id": "",
        "choices": [
            {
                "delta": {"content": "This is a test. "},
                "index": 0,
            }
        ],
        "model": "gpt-4o-mini-2024-07-18",
        "created": 0,
        "object": "chat.completion.chunk",
    }
    for i in range(5):
        data["id"] = f"id-{i}"
        data["choices"][0]["index"] = i
        data["created"] = int(time.time())
        out = f"data: {json.dumps(data)}\n\n"
        yield out
    yield "data: [DONE]\n\n"


@app.post("/chat/completions")
async def chat_completions(_: Request):
    return StreamingResponse(streamOutput(), media_type="text/event-stream")


if __name__ == "__main__":
    import uvicorn

    uvicorn.run(app, host="0.0.0.0", port=11000)
```

--------------------------------

### Environment Variables Configuration

Source: https://docs.simli.com/api-reference/livekit

Example `.env` file content for storing API keys and connection details for Simli, LiveKit, and OpenAI.

```dotenv
LIVEKIT_URL=wss://...
LIVEKIT_API_KEY=...
LIVEKIT_API_SECRET=...

OPENAI_API_KEY=...

SIMLI_API_KEY=...
SIMLI_FACE_ID=...
```

--------------------------------

### GET /peer-to-peer

Source: https://docs.simli.com/api-reference/websockets/peer_to_peer

Establishes a Peer-to-Peer WebRTC connection via WebSocket.

```APIDOC
## GET /peer-to-peer

### Description
Establishes a Peer-to-Peer WebRTC connection. The client must send an offer within 30 seconds of opening the websocket.

### Method
GET

### Endpoint
/peer-to-peer

### Parameters
#### Query Parameters
- **compose_session_token** (string) - Required - The session token used for authentication.
- **enableSFU** (boolean) - Optional - Set to true to route traffic through the Cloudflare network for improved reliability.

### Response
#### Success Response (200)
- **answer** (object) - The server-generated answer to the client's offer, to be used with setRemoteDescription.
```

--------------------------------

### Run the Simli Agent

Source: https://docs.simli.com/api-reference/livekit

Command to execute the Python script and start the Simli AI agent. Visit LiveKit.io sandbox to interact.

```bash
python3 livekit-simli.py
```

--------------------------------

### Start Compose Session - OpenAPI Spec

Source: https://docs.simli.com/api-reference/endpoint/webrtc/startAudioToVideoSession

Use this OpenAPI specification to understand the structure and parameters for starting a Simli Compose session. It defines the request body, including required fields like faceId and optional parameters for session configuration, as well as the expected response formats for successful token generation and error handling.

```yaml
openapi: 3.1.0
info:
  title: Simli
  version: 1.0.0
servers:
  - url: https://api.simli.ai/
    description: API server
security:
  - ApiKeyAuth: []
paths:
  /compose/token:
    post:
      summary: Compose Session Token
      description: Create a temporary Simli Compose Session Token
      operationId: start_audio_to_video_session_compose_token_post
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/StartStreamingSessionRequest'
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                type: object
                properties:
                  session_token:
                    type: string
        '400':
          description: Invalid Data
          content:
            application/json:
              schema:
                type: object
                properties:
                  session_token:
                    type: string
                    description: FAIL TOKEN
                  detail:
                    type: string
                    description: Fail Reason
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
components:
  schemas:
    StartStreamingSessionRequest:
      properties:
        faceId:
          type: string
          title: Faceid
        apiVersion:
          type: string
          title: Apiversion
          const: v2
        sessionAggregator:
          anyOf:
            - type: string
            - type: 'null'
          title: Sessionaggregator
        handleSilence:
          type: boolean
          title: Handlesilence
          default: true
        maxSessionLength:
          type: integer
          title: Maxsessionlength
          default: 3600
        maxIdleTime:
          type: integer
          title: Maxidletime
          default: 300
        startFrame:
          type: integer
          title: Startframe
          default: 0
        audioInputFormat:
          type: string
          title: Audioinputformat
          default: pcm16
      type: object
      required: 
        - faceId
      title: StartStreamingSessionRequest
    HTTPValidationError:
      properties:
        detail:
          items:
            $ref: '#/components/schemas/ValidationError'
          type: array
          title: Detail
      type: object
      title: HTTPValidationError
    ValidationError:
      properties:
        loc:
          items:
            anyOf:
              - type: string
              - type: integer
          type: array
          title: Location
        msg:
          type: string
          title: Message
        type:
          type: string
          title: Error Type
      type: object
      required:
        - loc
        - msg
        - type
      title: ValidationError
  securitySchemes:
    ApiKeyAuth:
      type: apiKey
      in: header
      name: x-simli-api-key

```

--------------------------------

### GET /faces

Source: https://docs.simli.com/api-reference/openapi_doc

Retrieves a list of all available faces.

```APIDOC
## GET /faces

### Description
Retrieves a list of all faces.

### Method
GET

### Endpoint
/faces

### Response
#### Success Response (200)
- **sessions** (array of FacesResponse) - A list of faces.

#### Response Example
(No example provided in source)
```

--------------------------------

### GET /static/hls/{destination}/{file}

Source: https://docs.simli.com/api-reference/openapi_doc

Retrieves HLS stream segments for generated videos.

```APIDOC
## GET /static/hls/{destination}/{file}

### Description
Get HLS Stream of Generated Video.

### Method
GET

### Endpoint
/static/hls/{destination}/{file}

### Parameters
#### Path Parameters
- **destination** (string) - Required - Destination path
- **file** (string) - Required - File name
```

--------------------------------

### Start Simli Client Connection

Source: https://docs.simli.com/api-reference/javascript

Connects the SimliClient to Simli Servers. This method includes automatic retry logic in case of connection failures.

```typescript
async SimliClientInstance.start()
```

--------------------------------

### Create Virtual Environment and Python File

Source: https://docs.simli.com/api-reference/livekit

Sets up a Python virtual environment and creates the main script file. Activate the virtual environment before running.

```bash
python3 -m venv .venv
source .venv/bin/activate
touch livekit-simli.py
```

--------------------------------

### Start Auto Session OpenAPI Specification

Source: https://docs.simli.com/api-reference/start-auto-session-from-config

Defines the GET endpoint for starting an auto session. Note that this specific operation is marked as deprecated.

```yaml
openapi: 3.1.0
info:
  title: Simli
  version: 1.0.0
servers:
  - url: https://api.simli.ai/
    description: API server
security:
  - ApiKeyAuth: []
paths:
  /auto/start/{agent_id}/{session_token}:
    get:
      summary: Start Auto Session from config
      operationId: startAgentIDQuery_auto_start__agent_id___session_token__post
      parameters:
        - name: agent_id
          in: path
          required: true
          schema:
            type: string
            title: Agent Id
        - name: session_token
          in: path
          required: true
          schema:
            type: string
            title: Session Token
        - name: redirect
          in: query
          required: false
          schema:
            type: boolean
            default: false
            title: Redirect
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/SessionDetails'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
      deprecated: true
      security: []
components:
  schemas:
    SessionDetails:
      properties:
        roomUrl:
          type: string
          title: Roomurl
        sessionId:
          type: string
          title: Sessionid
      type: object
      required:
        - roomUrl
        - sessionId
      title: SessionDetails
    HTTPValidationError:
      properties:
        detail:
          items:
            $ref: '#/components/schemas/ValidationError'
          type: array
          title: Detail
      type: object
      title: HTTPValidationError
    ValidationError:
      properties:
        loc:
          items:
            anyOf:
              - type: string
              - type: integer
          type: array
          title: Location
        msg:
          type: string
          title: Message
        type:
          type: string
          title: Error Type
      type: object
      required:
        - loc
        - msg
        - type
      title: ValidationError
  securitySchemes:
    ApiKeyAuth:
      type: apiKey
      in: header
      name: x-simli-api-key
```

--------------------------------

### Initialize project environment

Source: https://docs.simli.com/api-reference/pipecat

Commands to create a virtual environment and a new Python file.

```bash
python3 -m venv .venv
source .venv/bin/activate
touch pipecat-simli.py
```

--------------------------------

### GET /history/sessions

Source: https://docs.simli.com/api-reference/retrieve-session-history

Retrieves a list of session history records for the authenticated user, with optional filtering by start and end timestamps.

```APIDOC
## GET /history/sessions

### Description
Returns a list of session history records for the authenticated user.

### Method
GET

### Endpoint
/history/sessions

### Parameters
#### Query Parameters
- **start** (integer) - Optional - Filter records from this Unix timestamp
- **end** (integer) - Optional - Filter records until this Unix timestamp

### Response
#### Success Response (200)
- **sessions** (array) - A list of session history records

#### Response Example
{
  "sessions": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "userId": "user_123",
      "sessionTotalTime": 120.5,
      "startTime": 1745750387,
      "endTime": 1745750408
    }
  ]
}
```

--------------------------------

### Create and Initialize SimliClient

Source: https://docs.simli.com/api-reference/python

Create an instance of SimliClient and initialize it with your API key and configuration. Replace placeholders with your actual credentials.

```python
import asyncio
from simli import SimliClient, SimliConfig
from simli.renderers import FileRenderer
async def main():
    connection =  SimliClient(
        api_key="YOUR_SIMLI_API_KEY",  # API Key
        SimliConfig(
            faceId="YOUR_FACE_ID",  # Face ID
            maxSessionLength=20,
            maxIdleTime=10,
        )
    )
    await connection.start()

asyncio.run(main())
```

--------------------------------

### GET /compose/ice

Source: https://docs.simli.com/api-reference/api_migration_guide

Retrieves ICE servers for WebRTC connections, replacing the legacy GET /getIceServers endpoint.

```APIDOC
## GET /compose/ice

### Description
Retrieves ICE servers for WebRTC connections.

### Method
GET

### Endpoint
/compose/ice
```

--------------------------------

### SimliClient Initialization and Lifecycle

Source: https://docs.simli.com/api-reference/python

Methods for initializing the Simli client, establishing the WebRTC connection, and cleaning up resources.

```APIDOC
## SimliClient Constructor

### Description
Initializes the SimliClient instance with configuration parameters.

### Parameters
- **api_key** (str) - Required - API key from Simli dashboard.
- **config** (SimliConfig) - Required - Configuration object.
- **simliURL** (str) - Optional - Backend proxy URL.
- **retry_count** (int) - Optional - Reconnection attempts (default: 3).
- **retry_timeout** (float) - Optional - Time between retries (default: 15.0).
- **enableSFU** (bool) - Optional - Enable cloudflare network (default: True).
- **transport_mode** (TransportMode) - Optional - Livekit or P2P.

## async start()

### Description
Initializes the client and starts the WebRTC connection.

## async close()

### Description
Closes the WebRTC connection and cleans up resources.
```

--------------------------------

### WSS /compose/webrtc/livekit

Source: https://docs.simli.com/api-reference/asyncapi_doc

Establishes a Livekit-backed WebRTC connection. Requires a session token and successful room join within 30 seconds.

```APIDOC
## WSS /compose/webrtc/livekit

### Description
Livekit Backed WebRTC connection. You must successfully join the livekit room within 30 seconds.

### Endpoint
wss://api.simli.ai/compose/webrtc/livekit

### Parameters
#### Query Parameters
- **session_token** (string) - Required - Session token generated by https://api.simli.ai/compose/token
```

--------------------------------

### Get Agent OpenAPI Specification

Source: https://docs.simli.com/api-reference/get-agent

Defines the GET /auto/agent/{id} endpoint and the associated AgentResponse schema. Note that this endpoint is marked as deprecated.

```yaml
openapi: 3.1.0
info:
  title: Simli
  version: 1.0.0
servers:
  - url: https://api.simli.ai/
    description: API server
security:
  - ApiKeyAuth: []
paths:
  /auto/agent/{id}:
    parameters:
      - name: id
        in: path
        required: true
        schema:
          type: string
          format: uuid
    get:
      summary: Get Agent
      responses:
        '200':
          description: A single agent
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/AgentResponse'
      deprecated: true
components:
  schemas:
    AgentResponse:
      type: object
      additionalProperties: false
      properties:
        id:
          type: string
          format: uuid
          description: Unique identifier for the agent, generated by the MySQL service
        face_id:
          type: string
          format: uuid
        name:
          type: string
          default: Untitled Agent
        first_message:
          type: string
        prompt:
          type: string
        voice_provider:
          type: string
          enum:
            - elevenlabs
            - cartesia
        voice_id:
          type: string
          format: uuid
        voice_model:
          type: string
          example: sonic-english
        language:
          type: string
        emotion:
          type: string
          format: uuid
        llm_model:
          type: string
          example: gpt-4o-mini
        llm_endpoint:
          type: string
          description: The URL of the custom LLM to use
          example: https://api.example.com/v1/chat/completions
        max_idle_time:
          type: integer
          default: 300
        max_session_length:
          type: integer
          default: 3600
        owner_id:
          type: string
          description: Unique identifier for the owner of the agent
        llm_provider:
          type: string
          enum:
            - openai
            - google
            - user
          description: The provider of the LLM used by the agent
        simli_version:
          type: integer
          description: Version of the Simli model used for the face
          example: 1
        created_at:
          type: string
          format: date-time
          description: Timestamp when the agent was created
        updated_at:
          type: string
          format: date-time
          description: Timestamp when the agent was last updated
      required:
        - id
        - face_id
        - name
        - first_message
        - prompt
        - voice_provider
        - voice_id
        - voice_model
        - language
        - emotion
        - llm_model
        - llm_endpoint
        - max_idle_time
        - max_session_length
        - owner_id
        - llm_provider
        - created_at
        - updated_at
  securitySchemes:
    ApiKeyAuth:
      type: apiKey
      in: header
      name: x-simli-api-key
```

--------------------------------

### OpenAPI Specification for Get MP4 Video

Source: https://docs.simli.com/api-reference/get-mp4-of-generated-video

Defines the GET endpoint for retrieving generated MP4 files. Requires destination and file path parameters.

```yaml
openapi: 3.1.0
info:
  title: Simli
  version: 1.0.0
servers:
  - url: https://api.simli.ai/
    description: API server
security:
  - ApiKeyAuth: []
paths:
  /static/mp4/{destination}/{file}:
    get:
      summary: Get MP4 of Generated Video
      operationId: getCachedVideoMp4_static_mp4__machineIP___file__get
      parameters:
        - name: destination
          in: path
          required: true
          schema:
            type: string
        - name: file
          in: path
          required: true
          schema:
            type: string
            title: File
      responses:
        '200':
          description: Successful Response
          content:
            video/mp4:
              schema: {}
        '404':
          description: File not found
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
      security: []
components:
  schemas:
    HTTPValidationError:
      properties:
        detail:
          items:
            $ref: '#/components/schemas/ValidationError'
          type: array
          title: Detail
      type: object
      title: HTTPValidationError
    ValidationError:
      properties:
        loc:
          items:
            anyOf:
              - type: string
              - type: integer
          type: array
          title: Location
        msg:
          type: string
          title: Message
        type:
          type: string
          title: Error Type
      type: object
      required:
        - loc
        - msg
        - type
      title: ValidationError
  securitySchemes:
    ApiKeyAuth:
      type: apiKey
      in: header
      name: x-simli-api-key
```

--------------------------------

### Initialize WebSocket Decoder

Source: https://docs.simli.com/api-reference/simli-webrtc

Sets up a WebSocket connection to a server and streams audio data from a remote source.

```javascript
async function initializeWebsocketDecoder() {
	ws = new WebSocket("ws://localhost:8080/");
	ws.onopen = function (event) {
		console.log("connected");
	};
	ws.onmessage = function (event) {
		dc.send(event.data);
	};
	ws.binaryType = "arraybuffer";
	while (dc.readyState !== "open") {
		await setTimeout(() => {}, 100);
	}
	response = await fetch("https://radio.talksport.com/stream");
	if (!response.ok) {
		throw new Error("Network response was not ok");
	}
	const reader = response.body.getReader();
	function read() {
		return reader.read().then(({done, value}) => {
			if (done) {
				console.log("Stream complete");
				return;
			}
			ws.send(value);
			return read();
		});
	}

	return read();
}
```

--------------------------------

### LiveKitInitializationRequest

Source: https://docs.simli.com/api-reference/openapi_doc

Request to initialize a LiveKit session.

```APIDOC
## LiveKitInitializationRequest

### Description
Request to initialize a LiveKit session.

### Properties
- **session_token** (string) - Required - The session token.
- **livekit_token** (string) - Required - The LiveKit token.
- **livekit_url** (string) - Required - The LiveKit URL.
```

--------------------------------

### Get All Agents

Source: https://docs.simli.com/api-reference/openapi_doc

Retrieves a list of all agents. This endpoint is deprecated.

```APIDOC
## GET /auto/agents

### Description
Get All Agents. This endpoint is deprecated.

### Method
GET

### Endpoint
/auto/agents

### Responses
#### Success Response (200)
- Description: List of agents
- Content: application/json schema with items referencing `#/components/schemas/AgentResponse`

#### Response Example
```json
[
  {
    "id": "123e4567-e89b-12d3-a456-426614174000",
    "name": "Agent 1"
  }
]
```
```

--------------------------------

### Get Agent

Source: https://docs.simli.com/api-reference/openapi_doc

Retrieves a specific agent by ID. This endpoint is deprecated.

```APIDOC
## GET /auto/agent/{id}

### Description
Get Agent. This endpoint is deprecated.

### Method
GET

### Endpoint
/auto/agent/{id}

### Parameters
#### Path Parameters
- **id** (string) - Required - The UUID of the agent to retrieve.

### Responses
#### Success Response (200)
- Description: A single agent
- Content: application/json schema referencing `#/components/schemas/AgentResponse`
```

--------------------------------

### POST /static/audio

Source: https://docs.simli.com/api-reference/openapi_doc

Initiates the generation of a static video file based on an provided audio sample.

```APIDOC
## POST /static/audio

### Description
Generate and stream a static video file from a given audio sample.

### Method
POST

### Endpoint
/static/audio

### Response
#### Success Response (200)
- **hls_url** (string) - URL to the HLS stream
- **mp4_url** (string) - URL to the MP4 file
```

--------------------------------

### GET /auto/agents

Source: https://docs.simli.com/api-reference/get-all-agents

Retrieves a list of all agents associated with the authenticated account.

```APIDOC
## GET /auto/agents

### Description
Retrieves a list of all agents. Note that this endpoint is deprecated.

### Method
GET

### Endpoint
/auto/agents

### Response
#### Success Response (200)
- **agents** (array) - A list of AgentResponse objects.

#### Response Example
[
  {
    "id": "uuid",
    "face_id": "uuid",
    "name": "Untitled Agent",
    "first_message": "Hello!",
    "prompt": "You are a helpful assistant.",
    "voice_provider": "elevenlabs",
    "voice_id": "uuid",
    "voice_model": "sonic-english",
    "language": "en",
    "emotion": "uuid",
    "llm_model": "gpt-4o-mini",
    "llm_endpoint": "https://api.example.com/v1/chat/completions",
    "max_idle_time": 300,
    "max_session_length": 3600,
    "owner_id": "uuid",
    "llm_provider": "openai",
    "simli_version": 1,
    "created_at": "2023-10-27T10:00:00Z",
    "updated_at": "2023-10-27T10:00:00Z"
  }
]
```

--------------------------------

### GET /faces/trinity/generation_status

Source: https://docs.simli.com/api-reference/check-trinity-generation-status

Retrieves the current status of a Trinity generation process.

```APIDOC
## GET /faces/trinity/generation_status

### Description
Check the status of a Trinity generation process.

### Method
GET

### Endpoint
/faces/trinity/generation_status

### Parameters
#### Query Parameters
- **face_id** (string) - Optional - The ID of the face to check.

### Response
#### Success Response (200)
- **body** (object) - Successful Response

#### Error Response (422)
- **detail** (array) - Validation Error
```

--------------------------------

### Connect to Remote Peer and Handle WebSocket Messages in JavaScript

Source: https://docs.simli.com/api-reference/simli-webrtc

Establishes a connection to a remote peer, sends metadata to obtain a session token, and sets up a WebSocket connection. Handles incoming WebSocket messages for starting/stopping streams, ping-pong, and receiving SDP answers.

```javascript
async function connectToRemotePeer() {
	var offer = pc.localDescription;
	document.getElementById("offer-sdp").textContent = offer.sdp;

	const faceId = document.getElementById("faceId").value;
	if (faceId === "") {
		alert("Please enter faceId");
		return;
	}

	const metadata = {
		faceId: faceId,
		maxSessionLength: parseInt(document.getElementById("maxSessionLength").value),
		maxIdleTime: parseInt(document.getElementById("maxIdleTime").value),
	};
	console.log(metadata);
	console.log(parseInt(document.getElementById("maxSessionLength").value));

	const sessionPromise = await fetch("/compose/token", {
		method: "POST",
		body: JSON.stringify(metadata),
		headers: {
			"Content-Type": "application/json",
			"x-simli-api-key": document.getElementById("apiKey").value,
		},
	});
	session_token = await sessionPromise.json();
	wsURL = new URL(window.location.origin + "/compose/webrtc/p2p");
	wsURL.searchParams.set("session_token", session_token.session_token);
	const ws = new WebSocket(wsURL);
	wsConnection = ws;
	ws.addEventListener("message", async (evt) => {
		dataChannelLog.textContent += "< " + evt.data + "\n";
		if (evt.data === "START") {
			dcZeroAudio = setTimeout(() => {
				var message = new Uint8Array(64000);
				wsConnection.send(message);
				console.log("SEND");
			}, 100);
			return;
		}
		if (evt.data === "STOP") {
			stop();
			return;
		} else if (evt.data.slice(0, 4) === "pong") {
			console.log("PONG");
			var elapsed_ms = current_stamp() - parseInt(evt.data.substring(5), 10);
			dataChannelLog.textContent += " RTT " + elapsed_ms + " ms\n";
		} else {
			try {
				const message = JSON.parse(evt.data);
				if (message.type !== "answer") {
					return;
				}
				answer = message;

```

--------------------------------

### GET /auto/transcript/{session_id}

Source: https://docs.simli.com/api-reference/openapi_doc

Retrieves the transcript for a specific Simli Auto session.

```APIDOC
## GET /auto/transcript/{session_id}

### Description
Get Simli Auto Transcript.

### Method
GET

### Endpoint
/auto/transcript/{session_id}

### Parameters
#### Query Parameters
- **session_id** (string) - Required - The ID of the session to retrieve the transcript for.

### Response
#### Success Response (200)
- **transcript** (object) - The transcript data.
```

--------------------------------

### GET /static/mp4/{destination}/{file}

Source: https://docs.simli.com/api-reference/openapi_doc

Retrieves the MP4 file for a generated video.

```APIDOC
## GET /static/mp4/{destination}/{file}

### Description
Get MP4 of Generated Video.

### Method
GET

### Endpoint
/static/mp4/{destination}/{file}

### Parameters
#### Path Parameters
- **destination** (string) - Required - Destination path
- **file** (string) - Required - File name
```

--------------------------------

### GET /auto/agent/{id}

Source: https://docs.simli.com/api-reference/get-agent

Retrieves the details of a specific agent by its unique identifier.

```APIDOC
## GET /auto/agent/{id}

### Description
Retrieves the details of a specific agent by its unique identifier. Note: This endpoint is deprecated.

### Method
GET

### Endpoint
/auto/agent/{id}

### Parameters
#### Path Parameters
- **id** (string, uuid) - Required - Unique identifier for the agent.

### Response
#### Success Response (200)
- **id** (string, uuid) - Unique identifier for the agent.
- **face_id** (string, uuid) - Unique identifier for the face.
- **name** (string) - Name of the agent.
- **first_message** (string) - The initial message of the agent.
- **prompt** (string) - The system prompt for the agent.
- **voice_provider** (string) - The voice provider (elevenlabs, cartesia).
- **voice_id** (string, uuid) - Unique identifier for the voice.
- **voice_model** (string) - The voice model used.
- **language** (string) - The language of the agent.
- **emotion** (string, uuid) - The emotion identifier.
- **llm_model** (string) - The LLM model used.
- **llm_endpoint** (string) - The URL of the custom LLM.
- **max_idle_time** (integer) - Maximum idle time in seconds.
- **max_session_length** (integer) - Maximum session length in seconds.
- **owner_id** (string) - Unique identifier for the owner.
- **llm_provider** (string) - The LLM provider (openai, google, user).
- **simli_version** (integer) - Version of the Simli model.
- **created_at** (string, date-time) - Timestamp of creation.
- **updated_at** (string, date-time) - Timestamp of last update.
```

--------------------------------

### SimliClient Initialization and Control

Source: https://docs.simli.com/api-reference/javascript

Methods for initializing the SimliClient instance and managing the lifecycle of the WebRTC connection.

```APIDOC
## SimliClient Initialization

### Description
Initializes the SimliClient with the provided configuration, including video/audio elements and network settings.

### Parameters
- **session_token** (string) - Required - The token generated from the session token endpoint.
- **videoElement** (HTMLVideoElement) - Required - The video element to render the avatar.
- **audioElement** (HTMLAudioElement) - Required - The audio element for avatar output.
- **iceServers** (RTCIceServer[] | null) - Required - Configuration for WebRTC ICE servers.
- **logLevel** (LogLevel) - Optional - Logging verbosity (default: LogLevel.DEBUG).
- **transport_mode** (TransportMode) - Optional - Transport mode (default: "p2p").
- **signaling** (SignalingMode) - Optional - Signaling mode (default: "websockets").
- **SimliWSURL** (string) - Optional - WebSocket URL (default: "wss://api.simli.ai").
- **audioBufferSize** (number) - Optional - Buffer size for audio (default: 3000).

## SimliClientInstance.start()

### Description
Connects the client to Simli Servers with automatic retry on failure.

## SimliClientInstance.stop()

### Description
Closes the WebRTC connection and cleans up resources.
```

--------------------------------

### GET /static/hls/{destination}/{file}

Source: https://docs.simli.com/api-reference/get-hls-segment

Retrieves an HLS stream segment for a generated video.

```APIDOC
## GET /static/hls/{destination}/{file}

### Description
Get HLS Stream of Generated Video

### Method
GET

### Endpoint
/static/hls/{destination}/{file}

### Parameters
#### Path Parameters
- **destination** (string) - Required - The destination identifier for the video segment.
- **file** (string) - Required - The specific file name to retrieve.

### Response
#### Success Response (200)
- **Content** (application/x-mpegURL or video/MP2T) - The requested HLS segment or stream file.

#### Error Response (404)
- **Description** - File not found

#### Error Response (422)
- **detail** (array) - Validation Error details
```

--------------------------------

### Play Immediate with File Data

Source: https://docs.simli.com/api-reference/simli-webrtc

Reads a file, prepends it with the 'PLAY_IMMEDIATE' command, and sends the combined data over the data channel. Updates button text upon sending.

```javascript
function playImmediate() {
	// Add event listener for file input change event

	var file = document.getElementById("fileInput").files[0];
	var reader = new FileReader();
	reader.onload = async function (e) {
		const arrayBuffer = new Uint8Array(e.target.result);
		const asciiStr = "PLAY_IMMEDIATE";
		const encoder = new TextEncoder(); // Default is utf-8
		const strBytes = encoder.encode(asciiStr); // Uint8Array of " World!"

		var uint8Array = new Uint8Array(strBytes.length + arrayBuffer.length);
		uint8Array.set(strBytes, 0);
		uint8Array.set(arrayBuffer, strBytes.length);
		var button = document.getElementById("playImmediate");
		// let chunkSize = parseInt(document.getElementById("chunkSize").value);
		// console.log(uint8Array)
		if (dc && dc.readyState === "open") {
			// for (var x = 0; x < 600; x++)
			wsConnection.send(uint8Array);
			// dc.send(uint8Array);
			// console.log("SUNEN");
			dataChannelLog.textContent += "- Sent file: " + file.name + "\n";
			button.textContent = "Sent";
		}
	};

	reader.readAsArrayBuffer(file);
}
```

--------------------------------

### Initialize Simli Client

Source: https://docs.simli.com/api-reference/javascript

Initializes the SimliClient with essential parameters like session token, video and audio elements, and optional configurations for ICE servers, logging level, transport mode, signaling, and buffer size.

```typescript
new SimliClient(session_token: string, videoElement: HTMLVideoElement, audioElement: HTMLAudioElement, iceServers: RTCIceServer[] | null, logLevel: LogLevel = LogLevel.DEBUG, transport_mode: TransportMode = "p2p", signaling: SignalingMode = "websockets", SimliWSURL: string = "https://api.simli.ai", audioBufferSize: number = 3000,)
```

--------------------------------

### Get Active Session Count

Source: https://docs.simli.com/api-reference/openapi_doc

Retrieves the current count of active sessions for rate limiting purposes.

```APIDOC
## GET /ratelimiter/sessions

### Description
Retrieves the current count of active sessions.

### Method
GET

### Endpoint
/ratelimiter/sessions

### Responses
#### Success Response (200)
- **currentUsage** (integer) - The current number of active sessions.

#### Response Example
```json
{
  "currentUsage": 15
}
```

#### Error Response (401)
- Description: Unauthorized - Invalid API key
- Content: application/json schema referencing `#/components/schemas/Error`
```
Tokens:
Raw