Spatius

https://docs.spatius.ai
Admin
Spatius turns avatar speech audio into real-time motion data, enabling on-device avatar rendering...
Tokens:52,520
Snippets:655
Trust Score:2.5
Update:5 months ago
Show doc for...
Tokens:
Raw
### Run Development Server

Source: https://docs.spatius.ai/quickstarts/web-sdk

Start the development server to run the quickstart demo.

```bash
pnpm dev
```

--------------------------------

### Install Golang SDK

Source: https://docs.spatius.ai/sdk-reference/go-sdk/go-sdk

Install the Spatius Golang SDK using go get.

```bash
go get github.com/spatius-ai/spatius-sdk-go
```

--------------------------------

### Server Setup with AvatarSession

Source: https://docs.spatius.ai/livekit-agents/overview

Configures and starts an AvatarSession integrated with a LiveKit Agent session. This example demonstrates piping TTS audio to Spatius and publishing synchronized audio and motion data.

```python
from livekit.agents import AgentSession, JobContext
from livekit.plugins import spatius

async def entrypoint(ctx: JobContext):
    await ctx.connect()

    session = AgentSession(vad=vad, stt=stt, llm=llm, tts=tts)
    avatar = spatius.AvatarSession()
    await avatar.start(session, room=ctx.room)

    await session.start(agent=YourAgent(), room=ctx.room)
```

--------------------------------

### Complete AvatarPlayer Initialization with LiveKit

Source: https://docs.spatius.ai/livekit-agents/client

A full example demonstrating the initialization of AvatarSDK, loading an avatar, creating an AvatarView, setting up the LiveKitProvider, and connecting to a LiveKit server. This covers the entire client-side setup for real-time avatar streaming.

```typescript
import { AvatarPlayer, LiveKitProvider } from '@spatius/avatarkit-rtc'
import { AvatarSDK, AvatarView, AvatarManager, DrivingServiceMode } from '@spatius/avatarkit'

async function init() {
  // Initialize AvatarKit for LiveKit transport rendering.
  // Note: `region` requires a small type augmentation in @spatius/avatarkit@1.0.0 —
  // see the Direct Mode Web guide for the avatarkit-region.d.ts snippet.
  await AvatarSDK.initialize('your-app-id', {
    region: 'us-west',
    drivingServiceMode: DrivingServiceMode.host,
  })

  // `setSessionToken` is not needed on the RTC Adapter path —
  // `DrivingServiceMode.host` does not open a Motion Server WebSocket from the client,
  // and `AvatarManager.load()` fetches avatar metadata over an App-ID-scoped public endpoint.

  // Load avatar and create view
  const avatar = await AvatarManager.shared.load('character-id')
  const container = document.getElementById('avatar-container')!
  const avatarView = new AvatarView(avatar, container)

  // Create player
  const provider = new LiveKitProvider()
  const player = new AvatarPlayer(provider, avatarView, {
    logLevel: 'info',
    enableJitterBuffer: true,
    maxBufferDelayMs: 80,
  })

  // Listen to events
  player.on('connected', () => console.log('Connected!'))
  player.on('disconnected', () => console.log('Disconnected!'))
  player.on('error', (err) => console.error('Error:', err))
  player.on('stalled', async () => {
    console.log('Stream stalled, reconnecting...')
    await player.reconnect()
  })

  // Connect to LiveKit server
  await player.connect({
    url: 'wss://your-livekit-server.com',
    token: 'your-livekit-token',
    roomName: 'my-room',
  })

  // Start microphone
  await player.startPublishing()
}
```

--------------------------------

### Quick Start: Full Avatar Session Lifecycle

Source: https://docs.spatius.ai/sdk-reference/python-sdk/python-sdk

A complete example demonstrating the initialization, connection, audio sending, and closing of an avatar session. Includes callbacks for frame transport, errors, and session closure.

```python
import asyncio
from datetime import datetime, timedelta, timezone

from spatius import new_avatar_session


async def main():
    session = new_avatar_session(
        api_key="your-api-key",
        app_id="your-app-id",
        avatar_id="your-avatar-id",
        expire_at=datetime.now(timezone.utc) + timedelta(minutes=5),
        transport_frames=lambda frame, last: print(
            f"Received frame: {len(frame)} bytes, last={last}"
        ),
        on_error=lambda err: print(f"Session error: {err}"),
        on_close=lambda: print("Session closed"),
    )

    await session.init()
    connection_id = await session.start()
    print(f"Connected: {connection_id}")

    audio_data = b"..."  # mono PCM s16le audio bytes
    request_id = await session.send_audio(audio_data, end=True)
    print(f"Sent audio request: {request_id}")

    await asyncio.sleep(10)
    await session.close()


if __name__ == "__main__":
    asyncio.run(main())
```

--------------------------------

### Clone and Setup iOS Project

Source: https://docs.spatius.ai/quickstarts/ios-sdk

Clone the Spatius avatar demo repository, navigate to the iOS direct mode client directory, install xcodegen, and generate the Xcode project.

```bash
git clone https://github.com/spatius-ai/spatius-avatar-demo.git
cd spatius-avatar-demo/direct-mode/clients/ios
brew install xcodegen
```

```bash
xcodegen generate
open AvatarDemo.xcodeproj
```

--------------------------------

### Clone Repository and Navigate to Quickstart Folder

Source: https://docs.spatius.ai/livekit-agents/quickstart

Clone the Spatius avatar demo repository and change the directory to the LiveKit agent quickstart folder.

```bash
git clone https://github.com/spatius-ai/spatius-avatar-demo.git
cd spatius-avatar-demo/platform-integrations/livekit-agents-demo/livekit-agent-quickstart
```

--------------------------------

### Create Environment File

Source: https://docs.spatius.ai/quickstarts/web-sdk

Copy the example environment file and fill in your Spatius credentials.

```bash
cp .env.example .env
```

--------------------------------

### Install Dependencies

Source: https://docs.spatius.ai/quickstarts/web-sdk

Install the necessary project dependencies using pnpm.

```bash
pnpm install
```

--------------------------------

### Install Spatius Python SDK

Source: https://docs.spatius.ai/sdk-reference/python-sdk/python-sdk

Install the Spatius Python SDK using pip. This is the base installation.

```bash
pip install spatius
```

--------------------------------

### start()

Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference

Starts the avatar driving service connection. This method initiates the connection to the avatar driving service.

```APIDOC
## start()

### Description
Starts the avatar driving service connection.

### Method
`start()`

### Parameters
None

### Response
None
```

--------------------------------

### Install @spatius/avatarkit-rtc and @spatius/avatarkit

Source: https://docs.spatius.ai/sdk-reference/web-sdk/rtc-adapter

Install the RTC adapter and the core avatar rendering SDK using your preferred package manager.

```bash
pnpm add @spatius/avatarkit-rtc @spatius/avatarkit
```

```bash
npm install @spatius/avatarkit-rtc @spatius/avatarkit
```

```bash
yarn add @spatius/avatarkit-rtc @spatius/avatarkit
```

--------------------------------

### Quick Start Golang Avatar Session

Source: https://docs.spatius.ai/sdk-reference/go-sdk/go-sdk

Initialize and start a Spatius avatar session with basic configuration and send audio data. Handles session initialization, starting the connection, and sending audio chunks.

```go
package main

import (
	"context"
	"log"
	"time"

	spatius "github.com/spatius-ai/spatius-sdk-go"
)

func main() {
	ctx := context.Background()

	session := spatius.NewAvatarSession(
		spatius.WithAPIKey("your-api-key"),
		spatius.WithAppID("your-app-id"),
		spatius.WithAvatarID("your-avatar-id"),
		spatius.WithRegion("us-west"),
		spatius.WithExpireAt(time.Now().Add(5*time.Minute).UTC()),
		spatius.WithTransportFrames(func(data []byte, last bool) {
			// Handle encoded motion frame payloads.
		}),
		spatius.WithOnError(func(err error) {
			// Handle async session errors.
		}),
		spatius.WithOnClose(func() {
			// Handle connection close.
		}),
	)

	if err := session.Init(ctx); err != nil {
		log.Fatal(err)
	}
	defer session.Close()

	connectionID, err := session.Start(ctx)
	if err != nil {
		log.Fatal(err)
	}
	log.Printf("connection id: %s", connectionID)

	audioBytes := []byte{} // Replace with mono PCM audio bytes.
	reqID, err := session.SendAudio(audioBytes, true)
	if err != nil {
		log.Fatal(err)
	}
	log.Printf("request id: %s", reqID)
}
```

--------------------------------

### Install Dependencies

Source: https://docs.spatius.ai/livekit-agents/quickstart

Install backend dependencies using uv and frontend dependencies using pnpm. Ensure that the versions of livekit-agents, livekit-plugins-google, and livekit-plugins-spatius are kept aligned.

```bash
# backend
cd backend
uv sync

# frontend
cd ../frontend
pnpm install
```

--------------------------------

### Install AvatarKit UI with shadcn/ui

Source: https://docs.spatius.ai/sdk-reference/web-sdk/avatarkit-ui

Use these commands to initialize shadcn/ui and add the AvatarKit UI components to your project.

```bash
npx shadcn@latest init
npx shadcn@latest add https://ui.spatius.ai/r/spatius-avatar.json
```

--------------------------------

### AvatarController start() Method

Source: https://docs.spatius.ai/sdk-reference/ios-sdk/api-reference

Starts the avatar driving service connection. Call this method to initiate the connection to the avatar service.

```swift
func start()
```

--------------------------------

### Install LiveKit Agents Spatius Plugin (uv)

Source: https://docs.spatius.ai/livekit-agents/overview

Installs the livekit-plugins-spatius package using uv for Python environments.

```bash
uv add livekit-plugins-spatius
```

--------------------------------

### Clone Repository and Navigate

Source: https://docs.spatius.ai/quickstarts/web-sdk

Clone the Spatius Avatar Demo repository and navigate into the specific web quickstart directory.

```bash
git clone https://github.com/spatius-ai/spatius-avatar-demo.git
cd spatius-avatar-demo/direct-mode/clients/web/speech-to-avatar-quickstart
```

--------------------------------

### Create Environment Files

Source: https://docs.spatius.ai/livekit-agents/quickstart

Copy example environment files for backend and frontend configurations. These files will store your Spatius, LiveKit, and Google AI Studio credentials.

```bash
cp backend/.env.example backend/.env
cp frontend/.env.example frontend/.env
```

--------------------------------

### Install livekit-plugins-spatius

Source: https://docs.spatius.ai/livekit-agents/server

Install the package using pip. Ensure to manage version compatibility with livekit-agents.

```bash
pip install livekit-plugins-spatius
```

--------------------------------

### Install iOS Pods and Run Flutter App

Source: https://docs.spatius.ai/quickstarts/flutter-sdk

For iOS development, install CocoaPods dependencies and then run the Flutter application.

```bash
cd ios
pod install
cd ..
flutter run
```

--------------------------------

### Clone Flutter Direct Mode Demo

Source: https://docs.spatius.ai/quickstarts/flutter-sdk

Clone the Spatius avatar demo repository to get started with the Flutter Direct Mode client.

```bash
git clone https://github.com/spatius-ai/spatius-avatar-demo.git
cd spatius-avatar-demo/direct-mode/clients/flutter
```

--------------------------------

### Install @spatius/avatarkit with npm

Source: https://docs.spatius.ai/direct-mode/web

Use this command to add the Avatarkit SDK to your project when using npm.

```bash
npm install @spatius/avatarkit
```

--------------------------------

### Complete Web SDK Usage Example

Source: https://docs.spatius.ai/sdk-reference/web-sdk/reference

Demonstrates the full lifecycle of initializing, loading, and interacting with an avatar using the Web SDK. Ensure `initializeAudioContext` is called within a user gesture handler.

```typescript
import {
  AvatarSDK,
  AvatarManager,
  AvatarView,
  ConnectionState,
  ConversationState,
} from '@spatius/avatarkit'

class AvatarApp {
  private avatarView: AvatarView | null = null

  async init(appId: string, sessionToken: string, avatarId: string, container: HTMLElement) {
    // Initialize the SDK
    await AvatarSDK.initialize(appId, { region: 'us-west' })
    AvatarSDK.setSessionToken(sessionToken)

    // Load the avatar
    const avatar = await AvatarManager.shared.load(avatarId)
    if (!avatar) throw new Error('Failed to load avatar')

    // Create the view
    this.avatarView = new AvatarView(avatar, container)
    this.avatarView.onFirstRendering = () => {
      console.log('First frame rendered')
    }

    // Wire up handlers
    this.avatarView.controller.onConnectionState = (state) => {
      console.log('Connection:', state)
    }
    this.avatarView.controller.onConversationState = (state) => {
      console.log('Conversation:', state)
    }
    this.avatarView.controller.onError = (error) => {
      console.error('Error:', error)
    }
  }

  // Must be called inside a user-gesture handler
  async start() {
    await this.avatarView?.controller.initializeAudioContext()
    await this.avatarView?.controller.start()
  }

  send(audioData: ArrayBuffer, isEnd: boolean) {
    this.avatarView?.controller.send(audioData, isEnd)
  }

  interrupt() {
    this.avatarView?.controller.interrupt()
  }

  dispose() {
    this.avatarView?.controller.close()
    this.avatarView?.dispose()
    this.avatarView = null
  }
}
```

--------------------------------

### Install Flutter Dependencies

Source: https://docs.spatius.ai/direct-mode/flutter

Run this command in your Flutter project directory to install the project dependencies, including the Spatius package.

```bash
flutter pub get
```

--------------------------------

### Install @spatius/avatarkit with yarn

Source: https://docs.spatius.ai/direct-mode/web

Use this command to add the Avatarkit SDK to your project when using yarn.

```bash
yarn add @spatius/avatarkit
```

--------------------------------

### Run the Stack: Frontend

Source: https://docs.spatius.ai/livekit-agents/quickstart

Start the frontend development server in a third terminal. This provides the user interface for connecting to the LiveKit room and interacting with the voice avatar.

```bash
# Terminal 3: frontend
cd spatius-avatar-demo/platform-integrations/livekit-agents-demo/livekit-agent-quickstart/frontend
pnpm dev
```

--------------------------------

### Install @spatius/avatarkit with pnpm

Source: https://docs.spatius.ai/direct-mode/web

Use this command to add the Avatarkit SDK to your project when using pnpm.

```bash
pnpm add @spatius/avatarkit
```

--------------------------------

### Run the Stack: Token Server

Source: https://docs.spatius.ai/livekit-agents/quickstart

Start the token server in a dedicated terminal. This server is responsible for generating authentication tokens for LiveKit and Spatius.

```bash
# Terminal 1: token server
cd spatius-avatar-demo/platform-integrations/livekit-agents-demo/livekit-agent-quickstart/backend
uv run token_server.py
```

--------------------------------

### Install Spatius Python SDK with Opus Support

Source: https://docs.spatius.ai/sdk-reference/python-sdk/python-sdk

Install the Spatius Python SDK with optional Ogg Opus encoder support for encoding raw PCM before sending. This requires a working libopus runtime.

```bash
pip install "spatius[opus]"
```

--------------------------------

### Session Lifecycle Steps

Source: https://docs.spatius.ai/sdk-reference/python-sdk/python-sdk

Outlines the sequential steps for managing an avatar session: initialization, starting the connection, sending audio, receiving frames, and closing the session.

```python
# 1. Initialize and request a session token
await session.init()

# 2. Start the WebSocket connection
connection_id = await session.start()

# 3. Send audio data
request_id = await session.send_audio(audio_bytes, end=True)

# 4. Receive encoded motion frame payloads through transport_frames

# 5. Close the session
await session.close()
```

--------------------------------

### Start Avatar Driving Service Connection

Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference

Starts the avatar driving service connection. Call this method to initiate communication with the avatar service.

```kotlin
fun start()
```

--------------------------------

### Install AvatarKit RTC and AvatarKit (bun)

Source: https://docs.spatius.ai/livekit-agents/overview

Use this command to add the necessary Spatius AvatarKit packages to your project when using bun.

```bash
bun add @spatius/avatarkit-rtc @spatius/avatarkit
```

--------------------------------

### Connect Player to LiveKit Server

Source: https://docs.spatius.ai/livekit-agents/overview

Establishes a connection to the LiveKit server using the provided URL, token, and room name. Ensure these credentials are correct for your LiveKit setup.

```typescript
await player.connect({
  url: 'wss://your-livekit-server.com',
  token: 'your-livekit-token',
  roomName: 'room-name',
})
```

--------------------------------

### Start AvatarController Connection

Source: https://docs.spatius.ai/sdk-reference/web-sdk/reference

Initiates the connection to the Motion Server using the AvatarController. This is a prerequisite for most controller functionalities.

```typescript
await controller.start()
```

--------------------------------

### Clone Spatius Avatar Demo Repository

Source: https://docs.spatius.ai/resources/demo-projects

Clone the spatius-avatar-demo repository and navigate into the project directory. Refer to individual demo READMEs for specific setup and run instructions.

```bash
git clone https://github.com/spatius-ai/spatius-avatar-demo.git
cd spatius-avatar-demo
```

--------------------------------

### Install AvatarKit RTC and AvatarKit (npm)

Source: https://docs.spatius.ai/livekit-agents/overview

Use this command to add the necessary Spatius AvatarKit packages to your project when using npm.

```bash
npm install @spatius/avatarkit-rtc @spatius/avatarkit
```

--------------------------------

### Install AvatarKit RTC and AvatarKit (yarn)

Source: https://docs.spatius.ai/livekit-agents/overview

Use this command to add the necessary Spatius AvatarKit packages to your project when using yarn.

```bash
yarn add @spatius/avatarkit-rtc @spatius/avatarkit
```

--------------------------------

### Install AvatarKit RTC and AvatarKit (pnpm)

Source: https://docs.spatius.ai/livekit-agents/overview

Use this command to add the necessary Spatius AvatarKit packages to your project when using pnpm.

```bash
pnpm add @spatius/avatarkit-rtc @spatius/avatarkit
```

--------------------------------

### Run the Stack: LiveKit Agent Worker

Source: https://docs.spatius.ai/livekit-agents/quickstart

Start the LiveKit agent worker in a separate terminal. This worker processes real-time audio and video streams and integrates with Spatius for avatar functionalities.

```bash
# Terminal 2: LiveKit agent worker
cd spatius-avatar-demo/platform-integrations/livekit-agents-demo/livekit-agent-quickstart/backend
uv run agent.py dev
```

--------------------------------

### Handle Token and SDK Errors during Session Initialization

Source: https://docs.spatius.ai/sdk-reference/python-sdk/python-sdk

Use a try-except block to catch specific errors during session initialization and startup. This example demonstrates handling SessionTokenError for token-related issues and AvatarSDKError for other SDK errors, printing relevant details for each.

```python
from spatius import AvatarSDKError, SessionTokenError


try:
    await session.init()
    await session.start()
except SessionTokenError as error:
    print("token failed", error.code.value, error.server_detail)
except AvatarSDKError as error:
    print("sdk error", error.code.value, error.phase, error.server_detail)
```

--------------------------------

### Switch Avatars

Source: https://docs.spatius.ai/sdk-reference/web-sdk/reference

Provides a step-by-step guide for disposing of the current avatar view, loading a new avatar, creating a new view, and reinitializing the controller.

```typescript
// 1. Dispose the current view
currentAvatarView.dispose()

// 2. Load a new avatar
const newAvatar = await AvatarManager.shared.load('new-avatar-id')

// 3. Create a new view (reuse the same container)
currentAvatarView = new AvatarView(newAvatar, container)

// 4. Reconnect
await currentAvatarView.controller.initializeAudioContext()
await currentAvatarView.controller.start()
```

--------------------------------

### Session Token Authentication Flow

Source: https://docs.spatius.ai/direct-mode/overview

Illustrates the authentication process for obtaining a Session Token, which is required before starting the SDK. The token has a maximum validity of 24 hours.

```text
Your Client → Your Backend → Spatius Console API → Session Token (24 h max)
```

--------------------------------

### init(region:audioFormat:drivingServiceMode:logLevel:)

Source: https://docs.spatius.ai/sdk-reference/ios-sdk/api-reference

Creates a new configuration with the specified parameters for initializing the SDK.

```APIDOC
## init(region:audioFormat:drivingServiceMode:logLevel:)

### Description
Creates a new configuration with the specified parameters.

### Parameters
#### Path Parameters
* `region` (String) - Required - The region to connect to (currently only `"us-west"`)
* `audioFormat` (AudioFormat) - Required - The audio format configuration
* `drivingServiceMode` (DrivingServiceMode) - Required - The driving service mode
* `logLevel` (LogLevel) - Required - The log level
```

--------------------------------

### AvatarManager Get All Cache Size Method

Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference

Gets the total cache size in bytes for all cached avatars. This is an asynchronous operation.

```kotlin
suspend fun getAllCacheSize(): Long
```

--------------------------------

### AvatarController.start

Source: https://docs.spatius.ai/sdk-reference/web-sdk/reference

Establishes a connection to the Motion Server, enabling runtime communication and playback control.

```APIDOC
## AvatarController.start

### Description
Connect to Motion Server.

### Method
`start(): Promise<void>`
```

--------------------------------

### AvatarPlayer Microphone Control

Source: https://docs.spatius.ai/livekit-agents/client

Methods to start and stop publishing audio from the user's microphone. Starting the microphone automatically requests necessary permissions.

```typescript
// Start microphone (requests permission automatically)
await player.startPublishing()

// Stop microphone
await player.stopPublishing()
```

--------------------------------

### AudioFormat.init(sampleRate:)

Source: https://docs.spatius.ai/sdk-reference/ios-sdk/api-reference

Creates a new audio format with the specified sample rate.

```APIDOC
## AudioFormat.init(sampleRate:)

### Description
Creates a new audio format with the specified sample rate.

### Parameters
#### Path Parameters
* `sampleRate` (Int) - Required - The audio sample rate in Hz
```

--------------------------------

### Initialize AvatarPlayer with LiveKitProvider

Source: https://docs.spatius.ai/sdk-reference/web-sdk/rtc-adapter

Instantiate the LiveKitProvider and then use it to create an AvatarPlayer instance. Ensure AvatarPlayer is imported from '@spatius/avatarkit-rtc'.

```typescript
import { AvatarPlayer, LiveKitProvider } from '@spatius/avatarkit-rtc'

const provider = new LiveKitProvider()
const player = new AvatarPlayer(provider, avatarView, options)
```

--------------------------------

### SpatiusAvatarError

Source: https://docs.spatius.ai/sdk-reference/web-sdk/avatarkit-ui

An error overlay shown when avatar setup or streaming fails. Allows for custom error UI.

```APIDOC
## SpatiusAvatarError

### Description
Error overlay shown when avatar setup or streaming fails.

### Props

- **children** (ReactNode) - Optional - Optional custom error UI. When omitted, the default card displays `error.message`.
- **className** (string) - Optional - Additional classes for the overlay container.
- **...props** (HTMLAttributes<HTMLDivElement>) - Optional - Standard div attributes forwarded to the overlay container.
```

--------------------------------

### Clone and Configure Android Project

Source: https://docs.spatius.ai/quickstarts/android-sdk

Clone the Spatius Avatar Demo repository and configure the Android client's local properties file with your Spatius and OpenAI API keys.

```bash
git clone https://github.com/spatius-ai/spatius-avatar-demo.git
cd spatius-avatar-demo/direct-mode/clients/android
cp local.properties.example local.properties
```

```properties
SPATIUS_APP_ID=<your-app-id>
SPATIUS_AVATAR_ID=<your-avatar-id>
OPENAI_API_KEY=<your-openai-api-key>
OPENAI_USE_PROXY=false
```

--------------------------------

### Create Configuration

Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference

Creates a new Configuration object with specified parameters. Defaults are provided for audio format, driving service mode, and log level.

```kotlin
data class Configuration(
      val region: String,
      val audioFormat: AudioFormat = AudioFormat(16000),
      val drivingServiceMode: DrivingServiceMode = DrivingServiceMode.SDK,
      val logLevel: LogLevel = LogLevel.INFO
  )
```

--------------------------------

### AvatarController Volume Property

Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference

Sets or gets the playback volume, ranging from 0.0 (silent) to 1.0 (maximum volume).

```kotlin
var volume: Float
```

--------------------------------

### AvatarManager Get Cache Size Method

Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference

Retrieves the cache size in bytes for a specific avatar ID. This is an asynchronous operation.

```kotlin
suspend fun getCacheSize(id: String): Long
```

--------------------------------

### Configuration(region, audioFormat, drivingServiceMode, logLevel)

Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference

Creates a new configuration with the specified parameters. This allows customization of the SDK's behavior.

```APIDOC
## Configuration(region, audioFormat, drivingServiceMode, logLevel)

### Description
Creates a new configuration with the specified parameters.

### Method
data class Configuration(
    val region: String,
    val audioFormat: AudioFormat = AudioFormat(16000),
    val drivingServiceMode: DrivingServiceMode = DrivingServiceMode.SDK,
    val logLevel: LogLevel = LogLevel.INFO
)

### Parameters
#### Path Parameters
* `region` (String) - Required - The region to connect to (currently only `"us-west"`)
* `audioFormat` (AudioFormat) - Optional - The audio format configuration
* `drivingServiceMode` (DrivingServiceMode) - Optional - The driving service mode
* `logLevel` (LogLevel) - Optional - The log level

### Response
None
```

--------------------------------

### Create Player with LiveKit Provider

Source: https://docs.spatius.ai/livekit-agents/overview

Sets up an AvatarPlayer using a LiveKitProvider for real-time communication and an AvatarView for rendering. Configure the logLevel as needed.

```typescript
import { AvatarPlayer, LiveKitProvider } from '@spatius/avatarkit-rtc'

const provider = new LiveKitProvider()
const player = new AvatarPlayer(provider, avatarView, {
  logLevel: 'warning',
})
```

--------------------------------

### Configure Golang SDK Authentication (Query)

Source: https://docs.spatius.ai/sdk-reference/go-sdk/go-sdk

Configure the SDK to use web-style query parameter authentication for WebSockets.

```go
session := spatius.NewAvatarSession(
	spatius.WithUseQueryAuth(true),
)
```

--------------------------------

### AvatarController Methods

Source: https://docs.spatius.ai/sdk-reference/ios-sdk/api-reference

Provides methods for controlling avatar behavior, including starting and closing connections, managing rendering, and sending/yielding data.

```APIDOC
## start()

### Description
Starts the avatar driving service connection.

### Method
func start()

### Endpoint
N/A (SDK Method)

### Parameters
None

### Response
None
```

```APIDOC
## close()

### Description
Closes the avatar driving service.

### Method
func close()

### Endpoint
N/A (SDK Method)

### Parameters
None

### Response
None
```

```APIDOC
## pauseRendering()

### Description
Pauses avatar rendering.

### Method
func pauseRendering()

### Endpoint
N/A (SDK Method)

### Parameters
None

### Response
None
```

```APIDOC
## resumeRendering()

### Description
Resumes avatar rendering.

### Method
func resumeRendering()

### Endpoint
N/A (SDK Method)

### Parameters
None

### Response
None
```

```APIDOC
## interrupt()

### Description
Stops playback and terminates the current conversation.

### Method
func interrupt()

### Endpoint
N/A (SDK Method)

### Parameters
None

### Response
None
```

```APIDOC
## send(_:end:)

### Description
Sends audio to the avatar driving service.

### Method
func send(_ data: Data, end: Bool) -> String

### Endpoint
N/A (SDK Method)

### Parameters
#### Path Parameters
* **data** (Data) - Required - The audio data to send
* **end** (Bool) - Required - Whether this is the end of the audio stream

### Response
#### Success Response
* **String** - A conversation ID string.
```

```APIDOC
## yield(_:end:audioFormat:)

### Description
Yields audio from the host.

### Method
func yield(_ data: Data, end: Bool, audioFormat: AudioFormat?) -> String

### Endpoint
N/A (SDK Method)

### Parameters
#### Path Parameters
* **data** (Data) - Required - The audio data
* **end** (Bool) - Required - Whether this is the end of the audio stream
* **audioFormat** (AudioFormat?) - Optional - The audio format (optional)

### Response
#### Success Response
* **String** - A conversation ID string.
```

```APIDOC
## yield(_:conversationID:)

### Description
Yields animations from the host.

### Method
func yield(_ animations: [Data], conversationID: String)

### Endpoint
N/A (SDK Method)

### Parameters
#### Path Parameters
* **animations** (Array<Data>) - Required - Array of encoded motion data frames
* **conversationID** (String) - Required - The conversation identifier

### Response
None
```

--------------------------------

### AvatarSession Class

Source: https://docs.spatius.ai/sdk-reference/python-sdk/python-sdk

The main class responsible for managing avatar sessions, providing methods for initialization, starting, sending data, and closing the session.

```APIDOC
### `AvatarSession`

Main class for managing avatar sessions.

#### Methods

* `async init()` - initialize the session and obtain a token
* `async start() -> str` - start the WebSocket connection and return a `connection_id`
* `async send_audio(audio: bytes, end: bool = False) -> str` - send audio data and return a request ID
* `async interrupt() -> str` - interrupt current audio processing in egress mode
* `async close()` - close the session and clean up resources
* `config -> SessionConfig` - current session configuration
```

--------------------------------

### Web SDK Configuration Interface

Source: https://docs.spatius.ai/sdk-reference/web-sdk/reference

Defines the configuration options for initializing the Web SDK. Includes settings for region, driving service mode, logging level, and audio format.

```typescript
interface Configuration {
  region?: string                            // Defaults to 'us-west' (currently the only region)
  drivingServiceMode?: DrivingServiceMode    // Default: DrivingServiceMode.sdk
  logLevel?: LogLevel                        // Default: LogLevel.off
  audioFormat?: AudioFormat                  // Default: { channelCount: 1, sampleRate: 16000 }
}
```

--------------------------------

### AvatarManager Get Total Cache Size

Source: https://docs.spatius.ai/sdk-reference/ios-sdk/api-reference

Retrieves the total cache size in bytes for all cached avatars. This operation may throw an error.

```swift
func getAllCacheSize() throws -> Int
```

--------------------------------

### Composing Next.js Config Wrappers

Source: https://docs.spatius.ai/direct-mode/web

Combine the `withAvatarkit` wrapper with other Next.js configuration wrappers by nesting them. This example shows how to compose `withAvatarkit` with a hypothetical `withOtherPlugin`.

```javascript
import { withAvatarkit } from '@spatius/avatarkit/next'
import withOtherPlugin from 'other-plugin'

export default withAvatarkit(withOtherPlugin({
  // ...your config
}))
```

--------------------------------

### AvatarSDK Initialize Method

Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference

Initializes the AvatarKit SDK. Requires the application context, your application ID, and a Configuration object.

```kotlin
fun initialize(
    context: Context,
    appId: String,
    configuration: Configuration
)
```

--------------------------------

### Initialize AvatarView

Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference

Creates a new avatar view instance. Requires a Context to be provided.

```kotlin
constructor(context: Context)
```

--------------------------------

### Manage Voice Interaction

Source: https://docs.spatius.ai/livekit-agents/overview

Controls the microphone publishing stream for voice interaction within the avatar player. Includes starting, stopping publishing, and disconnecting the player.

```typescript
// Start microphone publishing
await player.startPublishing()

// Stop microphone
await player.stopPublishing()

// Disconnect when done
await player.disconnect()
```

--------------------------------

### AvatarSDK.initialize

Source: https://docs.spatius.ai/sdk-reference/web-sdk/reference

Initializes the SDK with the provided App ID and configuration. This method must be called before any other SDK operations.

```APIDOC
## AvatarSDK.initialize(appId, configuration)

### Description
Initialize the SDK. Must be called before any other operation.

### Method
`initialize`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
* **appId** (string) - Required - Your Spatius application ID.
* **configuration** (object) - Optional - SDK configuration object.
  * **region** (string) - Optional - The region for the SDK to connect to (e.g., 'us-west').
  * **drivingServiceMode** (DrivingServiceMode) - Optional - The driving service mode. Defaults to `DrivingServiceMode.sdk`.
  * **logLevel** (LogLevel) - Optional - The logging level. Defaults to `LogLevel.off`.
  * **audioFormat** (object) - Optional - Audio format configuration.
    * **channelCount** (number) - The number of audio channels.
    * **sampleRate** (number) - The audio sample rate.

### Request Example
```typescript
await AvatarSDK.initialize('your-app-id', {
  region: 'us-west',
  drivingServiceMode: DrivingServiceMode.sdk,
  logLevel: LogLevel.warning,
  audioFormat: {
    channelCount: 1,
    sampleRate: 16000,
  },
})
```

### Response
#### Success Response
None

#### Response Example
None
```

--------------------------------

### Direct Mode Audio Input

Source: https://docs.spatius.ai/sdk-reference/flutter-sdk/api-reference

For Direct Mode, start the connection and send audio data in PCM chunks. The `end` parameter indicates if this is the last chunk.

```dart
await controller.start();
controller.send(audioBytes, end: isLastChunk);
```

--------------------------------

### Configure Avatar Session with Callbacks and Format

Source: https://docs.spatius.ai/sdk-reference/python-sdk/python-sdk

Configure a new avatar session with various parameters including authentication, expiration, region, audio sample rate, audio format, and event callbacks.

```python
from datetime import datetime, timedelta, timezone

from spatius import AudioFormat, new_avatar_session


session = new_avatar_session(
    avatar_id="avatar-123",
    api_key="your-api-key",
    app_id="your-app-id",
    use_query_auth=False,
    expire_at=datetime.now(timezone.utc) + timedelta(minutes=5),
    region="us-west",
    sample_rate=16000,
    audio_format=AudioFormat.PCM_S16LE,
    transport_frames=on_frame_received,
    on_error=on_error,
    on_close=on_close,
)
```

--------------------------------

### Configure Golang SDK Authentication (Header)

Source: https://docs.spatius.ai/sdk-reference/go-sdk/go-sdk

Configure the SDK to use default header-based authentication for WebSockets.

```go
session := spatius.NewAvatarSession(
	spatius.WithUseQueryAuth(false),
)
```

--------------------------------

### Configure Golang SDK Audio Format (PCM)

Source: https://docs.spatius.ai/sdk-reference/go-sdk/go-sdk

Configure the SDK to use PCM 16-bit little-endian audio format with a specified sample rate.

```go
session := spatius.NewAvatarSession(
	spatius.WithSampleRate(24000),
	spatius.WithAudioFormat(spatius.AudioFormatPCMS16LE),
)
```

--------------------------------

### AvatarManager Get Specific Avatar Cache Size

Source: https://docs.spatius.ai/sdk-reference/ios-sdk/api-reference

Retrieves the cache size in bytes for a specific avatar identified by its ID. This operation may throw an error.

```swift
func getCacheSize(id: String) throws -> Int
```

--------------------------------

### Obtaining MediaStreamTracks for Custom Audio

Source: https://docs.spatius.ai/livekit-agents/client

Illustrates how to obtain MediaStreamTracks from various audio sources for custom audio publishing with AvatarPlayer.

```typescript
| `<audio>` element  | `audioElement.captureStream().getAudioTracks()[0]`                       |
| ------------------ | ------------------------------------------------------------------------ |
| Screen share audio | `getDisplayMedia({ audio: true })`                                       |
| Web Audio API      | `audioContext.createMediaStreamDestination().stream.getAudioTracks()[0]` |
```

--------------------------------

### AvatarManager Get Cache Stats Method

Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference

Retrieves statistics about the avatar cache, including total entries, total size, and maximum size. This is an asynchronous operation.

```kotlin
suspend fun getCacheStats(): CacheStats
```

--------------------------------

### Stall Recovery Example

Source: https://docs.spatius.ai/livekit-agents/client

Implement a handler for the 'stalled' event to automatically attempt reconnection when the data stream stops. This ensures resilience against temporary network issues.

```typescript
player.on('stalled', async () => {
  console.log('Stream stalled, reconnecting...')
  try {
    await player.reconnect()
  } catch (error) {
    console.error('Reconnection failed:', error)
  }
})
```

--------------------------------

### Initialize AvatarSDK and Set Session Token

Source: https://docs.spatius.ai/sdk-reference/flutter-sdk/api-reference

Initialize the AvatarSDK with your application ID and configuration. Then, set the session token for authentication. This should be done once before loading avatars.

```dart
await AvatarSDK.initialize(
  appID: appId,
  configuration: Configuration(
    region: 'us-west',
    audioFormat: const AudioFormat(sampleRate: 16000),
    drivingServiceMode: DrivingServiceMode.sdk,
    logLevel: LogLevel.all,
  ),
);

await AvatarSDK.setSessionToken(sessionToken);
```

--------------------------------

### Initialize AvatarSDK

Source: https://docs.spatius.ai/concepts/lifecycle

Call AvatarSDK.initialize once at app startup. This method configures process-level settings like App ID, region, audio format, driving mode, and log level. Initialization is mandatory before loading avatars or creating views.

```javascript
AvatarSDK.initialize(appId, configuration)
```

--------------------------------

### Core Backend Flow Diagram

Source: https://docs.spatius.ai/backend-mode/overview

Illustrates the data flow from your backend's ASR/LLM/TTS through the Spatius Server SDK to Motion Server and finally to the AvatarKit client for local rendering.

```mermaid
flowchart LR
    A["Your backend<br/>(ASR / LLM / TTS)"] --> |avatar speech audio| B["Spatius Server SDK"]
    B --> |connects to| C["Motion Server"]
    C --> |encoded audio + motion messages| B
    B --> |your chosen transport| D["AvatarKit client"]
    D --> |yieldAudioData / yieldFramesData| E["Local avatar rendering"]
```

--------------------------------

### LiveKitEgressConfig

Source: https://docs.spatius.ai/sdk-reference/python-sdk/python-sdk

Configuration object for setting up egress streaming to a LiveKit room.

```APIDOC
### `LiveKitEgressConfig`

Configuration for streaming to a LiveKit room.

#### Fields

* `url` (str) - LiveKit server URL, for example `wss://livekit.example.com`
* `api_key` (str) - deprecated LiveKit API key
* `api_secret` (str) - deprecated LiveKit API secret
* `api_token` (str) - preferred pre-generated LiveKit access token
* `room_name` (str) - room name to join
* `publisher_id` (str) - publisher identity in the room
* `extra_attributes` (dict[str, str]) - extra participant attributes
* `idle_timeout` (int) - idle timeout in seconds, `0` uses server defaults
```

--------------------------------

### Use livekit-plugins-spatius in an Agent Session

Source: https://docs.spatius.ai/livekit-agents/server

Integrate the Spatius AvatarSession into your LiveKit Agents worker. This setup sends TTS audio to Spatius and publishes synchronized audio/motion streams to the LiveKit room.

```python
from livekit.agents import Agent, AgentSession, JobContext, cli, WorkerOptions
from livekit.plugins import spatius

async def entrypoint(ctx: JobContext):
    await ctx.connect()

    session = AgentSession(vad=vad, stt=stt, llm=llm, tts=tts)
    avatar = spatius.AvatarSession()
    await avatar.start(session, room=ctx.room)

    await session.start(agent=YourAgent(), room=ctx.room)

if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))
```

--------------------------------

### Initialize AvatarKit Configuration

Source: https://docs.spatius.ai/sdk-reference/ios-sdk/api-reference

Creates a new configuration for AvatarKit with specified region, audio format, driving service mode, and log level. Ensure the region is set to 'us-west'.

```swift
init(
    region: String,
    audioFormat: AudioFormat,
    drivingServiceMode: DrivingServiceMode,
    logLevel: LogLevel
)
```

--------------------------------

### AudioFormat(sampleRate:)

Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference

Creates a new audio format with the specified sample rate. This configures the audio processing for the SDK.

```APIDOC
## AudioFormat(sampleRate:)

### Description
Creates a new audio format with the specified sample rate.

### Method
class AudioFormat(val sampleRate: Int)

### Parameters
#### Path Parameters
* `sampleRate` (Int) - Required - The audio sample rate in Hz. Supported sample rates are: 8000, 16000, 22050, 24000, 32000, 44100, 48000.

### Response
None
```

--------------------------------

### Session Token Response

Source: https://docs.spatius.ai/api-reference/api-reference

This is an example of a successful response from the Spatius Console API when requesting a session token. The token will be present under one of the keys: `sessionToken`, `sessionKey`, or `token`, potentially within a `data` object.

```json
{
  "sessionToken": "..."
}
```

--------------------------------

### Connect and Stream Audio

Source: https://docs.spatius.ai/direct-mode/web

Starts the avatar controller connection and streams audio data. It includes logic to resolve when connected or reject if the connection fails. Audio chunks are sent using PCM16 mono format.

```typescript
import { ConnectionState } from '@spatius/avatarkit'

await avatarView.controller.start()

await new Promise<void>((resolve, reject) => {
  avatarView.controller.onConnectionState = (state) => {
    if (state === ConnectionState.connected) resolve()
    else if (state === ConnectionState.failed) reject(new Error('Connection failed'))
  }
})

// Stream PCM16 mono audio at the configured sample rate (default 16 kHz)
avatarView.controller.send(audioChunk, /* end */ false)

// Mark end of the conversation round
avatarView.controller.send(lastChunk, /* end */ true)
```

--------------------------------

### Configure Golang SDK Audio Format (Ogg Opus)

Source: https://docs.spatius.ai/sdk-reference/go-sdk/go-sdk

Configure the SDK to use Ogg Opus audio format with a specified sample rate and bitrate. Allows setting a custom encoder or using the default.

```go
session := spatius.NewAvatarSession(
	spatius.WithAudioFormat(spatius.AudioFormatOggOpus),
	spatius.WithSampleRate(24000),
	spatius.WithBitrate(32000),
	spatius.WithOggOpusEncoder(nil),
)
```

--------------------------------

### Send PCM Audio Data

Source: https://docs.spatius.ai/sdk-reference/python-sdk/python-sdk

Example of sending raw PCM audio data to the avatar session. The audio must be mono 16-bit PCM bytes, and the session must be configured with the correct sample rate and audio format.

```python
from spatius import AudioFormat, new_avatar_session


session = new_avatar_session(
    ...,
    sample_rate=16000,
    audio_format=AudioFormat.PCM_S16LE,
)

with open("audio.pcm", "rb") as f:
    audio_data = f.read()

await session.send_audio(audio_data, end=True)
```

--------------------------------

### Initialize AvatarSDK

Source: https://docs.spatius.ai/sdk-reference/web-sdk/reference

Initialize the SDK with your App ID and configuration options. This must be called before any other SDK operations. The configuration includes region, driving service mode, log level, and audio format.

```typescript
await AvatarSDK.initialize('your-app-id', {
  region: 'us-west',
  drivingServiceMode: DrivingServiceMode.sdk,    // optional, default: sdk
  logLevel: LogLevel.warning,                    // optional, default: off
  audioFormat: {                                  // optional
    channelCount: 1,
    sampleRate: 16000,
  },
})
```

--------------------------------

### Common Web SDK Utility Methods

Source: https://docs.spatius.ai/sdk-reference/web-sdk/reference

Provides methods for controlling playback and volume. Use `pause()` and `resume()` to manage audio and animation, `interrupt()` to stop immediately, and `setVolume()`/`getVolume()` for avatar audio.

```typescript
controller.pause()                     // Pause audio + animation
```

```typescript
controller.resume()                    // Resume playback
```

```typescript
controller.interrupt()                 // Stop current playback
```

```typescript
controller.setVolume(0.5)              // 0.0 to 1.0
```

```typescript
controller.getVolume()                 // returns current volume
```

--------------------------------

### Create AudioFormat with Sample Rate

Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference

Creates a new AudioFormat object with a specified sample rate. A list of supported sample rates is available in the documentation.

```kotlin
class AudioFormat(val sampleRate: Int)
```

--------------------------------

### Import AvatarSDK

Source: https://docs.spatius.ai/sdk-reference/web-sdk/reference

Import the AvatarSDK class from the @spatius/avatarkit package. This is the main entry point for SDK initialization and global configuration.

```typescript
import { AvatarSDK } from '@spatius/avatarkit'
```

--------------------------------

### Configure Golang SDK LiveKit Egress

Source: https://docs.spatius.ai/sdk-reference/go-sdk/go-sdk

Configure the SDK to publish avatar output directly into a LiveKit room using egress. Supports API token for authentication.

```go
session := spatius.NewAvatarSession(
	spatius.WithLiveKitEgress(&spatius.LiveKitEgressConfig{
		URL:         "wss://your-livekit-server.com",
		APIToken:    "livekit-access-token",
		RoomName:    "room-name",
		PublisherID: "avatar-publisher",
		ExtraAttributes: map[string]string{
			"role": "avatar",
		},
		IdleTimeout: 60,
	}),
)
```

--------------------------------

### Backend Mode Integration Flowchart

Source: https://docs.spatius.ai/backend-mode/server-sdk

Illustrates the data flow and connections between your backend, Spatius Server SDK, Motion Server, and client SDK in Backend Mode Integration.

```mermaid
flowchart LR
    A["Your backend<br/>(ASR / LLM / TTS)"] --> |avatar speech audio| B["Spatius Server SDK"]
    B --> |API Key + App ID + Avatar ID| C["Motion Server"]
    C --> |encoded audio + motion messages| B
    B --> |your transport| D["Client SDK"]
```

--------------------------------

### Pin livekit-client version

Source: https://docs.spatius.ai/sdk-reference/web-sdk/rtc-adapter

Ensure compatibility by pinning the 'livekit-client' dependency to version '2.16.1' in your package.json.

```json
{
  "dependencies": {
    "livekit-client": "2.16.1"
  }
}
```
Tokens:
Raw