SpatialReal

https://docs.spatialreal.ai
Admin
SpatialReal is an AI infrastructure platform for real-time, high-fidelity digital avatar...
Tokens:90,386
Snippets:1,225
Trust Score:2.8
Update:5 months ago
Show doc for...
Tokens:
Raw
### Golang SDK Quick Start

Source: https://docs.spatialreal.ai/sdk-reference/go-sdk/go-sdk

A quick start guide to initialize and use the Golang SDK, including session creation, initialization, starting a connection, and sending audio.

```go
import (
	avatarsdkgo "github.com/spatialwalk/avatar-sdk-go"
	"context"
	"time"
)

func main() {
	ctx := context.Background()
	session := avatarsdkgo.NewAvatarSession(
		avatarsdkgo.WithAPIKey("your-api-key"),
		avatarsdkgo.WithAppID("your-app-id"),
		avatarsdkgo.WithAvatarID("your-avatar-id"),
		avatarsdkgo.WithConsoleEndpointURL("https://console.us-west.spatialwalk.cloud/v1/console"),
		avatarsdkgo.WithIngressEndpointURL("wss://api.us-west.spatialwalk.cloud/v2/driveningress"),
		avatarsdkgo.WithExpireAt(time.Now().Add(5 * time.Minute)),
		avatarsdkgo.WithTransportFrames(func(data []byte, last bool) {
			// handle animation frame
		}),
		avatarsdkgo.WithOnError(func(err error) {
			// handle error
		}),
		avatarsdkgo.WithOnClose(func() {
			// handle close
		}),
	)

	if err := session.Init(ctx); err != nil {
		// handle init error
	}

	connectionID, err := session.Start(ctx)
	if err != nil {
		// handle start error
	}
	defer session.Close()

	// Send audio
	audioData := []byte("your audio data") // Replace with actual audio data
	requestID, err := session.SendAudio(audioData, true)
	if err != nil {
		// handle error
	}
}
```

--------------------------------

### Quick Start Example

Source: https://docs.spatialreal.ai/sdk-reference/python-sdk/python-sdk

A comprehensive example demonstrating how to initialize, connect, send audio, and close a session using the Python SDK.

```APIDOC
## Quick Start Example

This example shows the basic usage of the Python SDK to connect to avatar services, send audio, and manage the session lifecycle.

```python
import asyncio
from datetime import datetime, timedelta, timezone

from avatarkit import new_avatar_session


async def main():
    session = new_avatar_session(
        api_key="your-api-key",
        app_id="your-app-id",
        console_endpoint_url="https://console.us-west.spatialwalk.cloud/v1/console",
        ingress_endpoint_url="wss://api.us-west.spatialwalk.cloud/v2/driveningress",
        avatar_id="your-avatar-id",
        expire_at=datetime.now(timezone.utc) + timedelta(minutes=5),
        transport_frames=lambda frame, last: print(f"Received frame: {len(frame)} bytes"),
        on_error=lambda err: print(f"Error: {err}"),
        on_close=lambda: print("Session closed"),
    )

    await session.init()
    connection_id = await session.start()
    print(f"Connected: {connection_id}")

    audio_data = b"..."  # Your PCM audio data
    request_id = await session.send_audio(audio_data, end=True)
    print(f"Sent audio: {request_id}")

    await asyncio.sleep(10)
    await session.close()


if __name__ == "__main__":
    asyncio.run(main())
```
```

--------------------------------

### Install Golang SDK

Source: https://docs.spatialreal.ai/sdk-reference/go-sdk/go-sdk

Install the Golang SDK using the go get command. This command fetches and installs the SDK package.

```bash
go get github.com/spatialwalk/avatar-sdk-go
```

--------------------------------

### Install Dependencies and Run Demo

Source: https://docs.spatialreal.ai/overview/speech-to-avatar

Install project dependencies using pnpm and start the development server. Access the demo at http://localhost:3000.

```bash
pnpm install
pnpm dev
```

--------------------------------

### Clone Repository and Navigate

Source: https://docs.spatialreal.ai/overview/agent

Clone the demo repository and navigate to the quickstart directory. Ensure you have Git installed.

```bash
git clone https://github.com/spatialwalk/avatarkit-voice-agent-demo.git
cd avatarkit-voice-agent-demo/spatialreal-agent-quickstart
```

--------------------------------

### Golang SDK Quick Start

Source: https://docs.spatialreal.ai/llms-full.txt

Initialize an AvatarSession with various configuration options and start sending audio data. Ensure to handle initialization and start errors, and defer closing the session.

```go
import (
	avatarsdkgo "github.com/spatialwalk/avatar-sdk-go"
)

func main() {
	session := avatarsdkgo.NewAvatarSession(
		avatarsdkgo.WithAPIKey("your-api-key"),
		avatarsdkgo.WithAppID("your-app-id"),
		avatarsdkgo.WithAvatarID("your-avatar-id"),
		avatarsdkgo.WithConsoleEndpointURL("https://console.us-west.spatialwalk.cloud/v1/console"),
		avatarsdkgo.WithIngressEndpointURL("wss://api.us-west.spatialwalk.cloud/v2/driveningress"),
		avatarsdkgo.WithExpireAt(time.Now().Add(5 * time.Minute)),
		avatarsdkgo.WithTransportFrames(func(data []byte, last bool) {
			// handle animation frame
		}),
		avatarsdkgo.WithOnError(func(err error) {
			// handle error
		}),
		avatarsdkgo.WithOnClose(func() {
			// handle close
		}),
	)

	if err := session.Init(ctx); err != nil {
		// handle init error
	}

	connectionID, err := session.Start(ctx)
	if err != nil {
		// handle start error
	}
	defer session.Close()

	// Send audio
	requestID, err := session.SendAudio(audioData, true)
	if err != nil {
		// handle error
	}
}
```

--------------------------------

### Install Backend Dependencies

Source: https://docs.spatialreal.ai/llms-full.txt

Navigate to the backend directory and install Python dependencies using uv.

```bash
cd backend
uv sync
```

--------------------------------

### Complete LiveKit RTC Integration Example

Source: https://docs.spatialreal.ai/guide/rtc-livekit-client

This example demonstrates initializing the Avatarkit SDK, loading an avatar, setting up the LiveKit provider, and connecting to a LiveKit server. It includes event listeners for connection status and error handling, as well as starting microphone publishing.

```typescript
import { AvatarPlayer, LiveKitProvider } from '@spatialwalk/avatarkit-rtc'
import { AvatarSDK, AvatarView, AvatarManager, DrivingServiceMode, Environment } from '@spatialwalk/avatarkit'

async function init() {
  // Initialize SDK in host mode
  await AvatarSDK.initialize('your-app-id', {
    environment: Environment.intl,
    drivingServiceMode: DrivingServiceMode.host,
  })
  AvatarSDK.setSessionToken('your-session-token')

  // Load avatar and create view
  const avatar = await AvatarManager.shared.load('character-id')
  const container = document.getElementById('avatar-container')!
  const avatarView = new AvatarView(avatar, container)

  // Create player
  const provider = new LiveKitProvider()
  const player = new AvatarPlayer(provider, avatarView, {
    logLevel: 'info',
    transitionStartFrameCount: 5,
    transitionEndFrameCount: 40,
  })

  // Listen to events
  player.on('connected', () => console.log('Connected!'))
  player.on('disconnected', () => console.log('Disconnected!'))
  player.on('error', (err) => console.error('Error:', err))
  player.on('stalled', async () => {
    console.log('Stream stalled, reconnecting...')
    await player.reconnect()
  })

  // Connect to LiveKit server
  await player.connect({
    url: 'wss://your-livekit-server.com',
    token: 'your-livekit-token',
    roomName: 'my-room',
  })

  // Start microphone
  await player.startPublishing()
}
```

--------------------------------

### Install Frontend Dependencies

Source: https://docs.spatialreal.ai/llms-full.txt

Navigate to the frontend directory and install Node.js dependencies using pnpm.

```bash
cd ../frontend
pnpm install
```

--------------------------------

### Install Dependencies

Source: https://docs.spatialreal.ai/overview/agent

Install backend dependencies using `uv` and frontend dependencies using `pnpm`. Navigate to the respective directories before running the commands.

```bash
# backend
cd backend
uv sync

# frontend
cd ../frontend
pnpm install
```

--------------------------------

### Usage Examples

Source: https://docs.spatialreal.ai/llms-full.txt

Code examples demonstrating how to use the event callbacks in different platforms.

```typescript
const controller = avatarView.controller;

// First frame rendered
avatarView.onFirstRendering = () => {
  console.log('First frame rendered');
};

// Connection state (SDK mode only)
controller.onConnectionState = (state) => {
  console.log('Connection:', state);
};

// Conversation state
controller.onConversationState = (state) => {
  console.log('Conversation:', state);
};

// Error handling
controller.onError = (error) => {
  console.error('Error:', error.code, error.message);
};
```

```swift
let controller = avatarView.controller

// First frame rendered
avatarView.onFirstRendering = {
    print("First frame rendered")
}

// Connection state (SDK mode only)
controller.onConnectionState = { state in
    switch state {
    case .connected:
        print("Connected")
    case .failed(let code, let message):
        print("Failed: \(code) \(message)")
    default:
        break
    }
}

// Conversation state
controller.onConversationState = { state in
    print("Conversation: \(state)")
}

// Error handling
controller.onError = { error in
    print("Error: \(error.localizedDescription)")
}
```

```kotlin
val controller = avatarView.controller

// First frame rendered
avatarView.onFirstRendering = {
    Log.d("Avatar", "First frame rendered")
}

// Connection state (SDK mode only)
controller?.onConnectionState = { state ->
    when (state) {
        is ConnectionState.Connected ->
            Log.d("Avatar", "Connected")
        is ConnectionState.Failed ->
            Log.e("Avatar", "Failed: ${state.message}")
        else -> {}
    }
}

// Conversation state
controller?.onConversationState = { state ->
    Log.d("Avatar", "Conversation: $state")
}

// Error handling
controller?.onError = { error ->
    Log.e("Avatar", "Error: ${error.message}")
}
```

--------------------------------

### Initialize Audio and Start AvatarController (SDK Mode)

Source: https://docs.spatialreal.ai/llms-full.txt

In SDK mode, initialize the audio context within a user gesture handler and then connect to the AvatarKit server. This setup is required before sending audio data.

```typescript
await controller.initializeAudioContext()

// Connect to AvatarKit server
await controller.start()
```

--------------------------------

### Install Python SDK

Source: https://docs.spatialreal.ai/sdk-reference/python-sdk/python-sdk

Install the avatarkit package using pip.

```bash
pip install avatarkit
```

--------------------------------

### Golang SDK Quick Start

Source: https://docs.spatialreal.ai/sdk-reference/go-sdk/go-sdk

Initialize an Avatar SDK session with various configuration options and start a connection. Handles audio data and includes error and close callbacks.

```go
import (
	avatarsdkgo "github.com/spatialwalk/avatar-sdk-go"
	"time"
)

func main() {
	session := avatarsdkgo.NewAvatarSession(
		avatarsdkgo.WithAPIKey("your-api-key"),
		avatarsdkgo.WithAppID("your-app-id"),
		avatarsdkgo.WithAvatarID("your-avatar-id"),
		avatarsdkgo.WithConsoleEndpointURL("https://console.us-west.spatialwalk.cloud/v1/console"),
		avatarsdkgo.WithIngressEndpointURL("wss://api.us-west.spatialwalk.cloud/v2/driveningress"),
		avatarsdkgo.WithExpireAt(time.Now().Add(5 * time.Minute)),
		avatarsdkgo.WithTransportFrames(func(data []byte, last bool) {
			// handle animation frame
		}),
		avatarsdkgo.WithOnError(func(err error) {
			// handle error
		}),
		avatarsdkgo.WithOnClose(func() {
			// handle close
		}),
	)

	if err := session.Init(ctx); err != nil {
		// handle init error
	}

	connectionID, err := session.Start(ctx)
	if err != nil {
		// handle start error
	}
	defer session.Close()

	// Send audio
	requestID, err := session.SendAudio(audioData, true)
	if err != nil {
		// handle error
	}
}
```

--------------------------------

### Clone Repository and Navigate

Source: https://docs.spatialreal.ai/overview/speech-to-avatar

Clone the official repository and navigate to the quickstart directory. This is the first step to setting up the demo.

```bash
git clone https://github.com/spatialwalk/avatarkit-voice-agent-demo.git
cd avatarkit-voice-agent-demo/spatialreal-speech-to-avatar-quickstart
```

--------------------------------

### Initialize and Start Avatar Session

Source: https://docs.spatialreal.ai/llms-full.txt

Demonstrates how to initialize and start a new avatar session using the `new_avatar_session` function and then establish a WebSocket connection.

```APIDOC
## Initialize and Start Avatar Session

### Description
This snippet shows the process of creating a new avatar session with specified configurations and then initiating the WebSocket connection.

### Method
```python
new_avatar_session(...)
await session.init()
await session.start()
```

### Parameters for `new_avatar_session`
- **avatar_id** (string) - Required - The ID of the avatar to use.
- **api_key** (string) - Required - Your API key for authentication.
- **app_id** (string) - Required - Your application ID.
- **use_query_auth** (boolean) - Optional - If true, uses web-style authentication via query parameters.
- **expire_at** (datetime) - Required - The expiration time for the session token.
- **console_endpoint_url** (string) - Required - The URL for the console endpoint.
- **ingress_endpoint_url** (string) - Required - The WebSocket URL for ingress.
- **sample_rate** (integer) - Optional - The audio sample rate.
- **transport_frames** (function) - Optional - Callback for receiving animation frames.
- **on_error** (function) - Optional - Callback for handling errors.
- **on_close** (function) - Optional - Callback for session closure.
- **livekit_egress** (LiveKitEgressConfig) - Optional - Configuration for LiveKit egress mode.

### Session Lifecycle Methods
- **`session.init()`**: Initializes the session and requests a session token.
- **`session.start()`**: Starts the WebSocket connection and returns a connection ID.
- **`session.close()`**: Closes the session.

### Request Example
```python
import asyncio
from datetime import datetime, timedelta, timezone

from avatarkit import new_avatar_session


async def main():
    session = new_avatar_session(
        api_key="your-api-key",
        app_id="your-app-id",
        console_endpoint_url="https://console.us-west.spatialwalk.cloud/v1/console",
        ingress_endpoint_url="wss://api.us-west.spatialwalk.cloud/v2/driveningress",
        avatar_id="your-avatar-id",
        expire_at=datetime.now(timezone.utc) + timedelta(minutes=5),
        transport_frames=lambda frame, last: print(f"Received frame: {len(frame)} bytes"),
        on_error=lambda err: print(f"Error: {err}"),
        on_close=lambda: print("Session closed"),
    )

    await session.init()
    connection_id = await session.start()
    print(f"Connected: {connection_id}")

    # ... rest of the session usage

    await session.close()


if __name__ == "__main__":
    asyncio.run(main())
```
```

--------------------------------

### Install AvatarKit UI with shadcn/ui

Source: https://docs.spatialreal.ai/sdk-reference/web-sdk/avatarkit-ui

Use these commands to initialize shadcn/ui and add the AvatarKit UI components to your project.

```bash
npx shadcn@latest init
npx shadcn@latest add https://ui.spatialreal.ai/r/spatialreal-avatar.json
```

--------------------------------

### Install AvatarKit Dependencies and Components

Source: https://docs.spatialreal.ai/guide/agent-quickstart-walkthrough

Installs necessary AvatarKit and LiveKit UI dependencies and adds SpatialReal avatar components using shadcn.

```bash
pnpm add @spatialwalk/avatarkit @livekit/components-react @livekit/components-styles
npx shadcn@latest add https://ui.spatialreal.ai/r/spatialreal-avatar.json
```

--------------------------------

### Create and Fill Backend Environment File

Source: https://docs.spatialreal.ai/llms-full.txt

Copy the example backend environment file and fill in your SpatialReal, LiveKit, and Google API credentials.

```bash
cp backend/.env.example backend/.env
```

```bash
LIVEKIT_URL=wss://your-project.livekit.cloud # https://cloud.livekit.io
LIVEKIT_API_KEY=your_api_key # https://cloud.livekit.io
LIVEKIT_API_SECRET=your_api_secret # https://cloud.livekit.io

GOOGLE_API_KEY=your_google_api_key # https://aistudio.google.com/api-keys
E2E_GOOGLE_MODEL=gemini-2.5-flash-native-audio-preview-12-2025
E2E_GOOGLE_VOICE=Puck

SPATIALREAL_API_KEY=your_api_key # https://app.spatialreal.ai/apps
SPATIALREAL_APP_ID=your_app_id # https://app.spatialreal.ai/apps
SPATIALREAL_AVATAR_ID=6aed28f9-674c-4ffb-89ee-b447b28aa3ed # https://app.spatialreal.ai/avatars/library
```

--------------------------------

### Create Environment Files

Source: https://docs.spatialreal.ai/overview/agent

Copy example environment files for backend and frontend configuration. These files will store your API keys and other settings.

```bash
cp backend/.env.example backend/.env
cp frontend/.env.example frontend/.env
```

--------------------------------

### Installation

Source: https://docs.spatialreal.ai/sdk-reference/android-sdk/api-reference

Add the AvatarKit SDK dependency to your `build.gradle.kts` file.

```APIDOC
```kotlin title="build.gradle.kts"
dependencies {
    implementation("ai.spatialwalk:avatarkit:1.0.0-beta49")
}
```
```

--------------------------------

### Install LiveKit Client (npm)

Source: https://docs.spatialreal.ai/llms-full.txt

Install the necessary AvatarKit, AvatarKit-RTC, and a compatible version of livekit-client using npm. Ensure livekit-client is version 2.16.1 for compatibility.

```bash
npm install @spatialwalk/avatarkit @spatialwalk/avatarkit-rtc livekit-client@2.16.1
```

--------------------------------

### Install LiveKit Client (yarn)

Source: https://docs.spatialreal.ai/llms-full.txt

Install the necessary AvatarKit, AvatarKit-RTC, and a compatible version of livekit-client using yarn. Ensure livekit-client is version 2.16.1 for compatibility.

```bash
yarn add @spatialwalk/avatarkit @spatialwalk/avatarkit-rtc livekit-client@2.16.1
```

--------------------------------

### Create and Update .env File

Source: https://docs.spatialreal.ai/overview/speech-to-avatar

Copy the example environment file and update it with your SpatialReal credentials. Ensure all required variables are correctly set for the demo to function.

```bash
cp .env.example .env
```

```bash
VITE_SPATIALREAL_APP_ID=your_app_id # https://app.spatialreal.ai/apps
VITE_SPATIALREAL_AVATAR_ID=your_avatar_id # https://app.spatialreal.ai/avatars/library
VITE_SPATIALREAL_SESSION_TOKEN=your_temporary_session_token # https://app.spatialreal.ai/apps -> details -> Generate Temporary Token
```

--------------------------------

### Install LiveKit Client (pnpm)

Source: https://docs.spatialreal.ai/llms-full.txt

Install the necessary AvatarKit, AvatarKit-RTC, and a compatible version of livekit-client using pnpm. Ensure livekit-client is version 2.16.1 for compatibility.

```bash
pnpm add @spatialwalk/avatarkit @spatialwalk/avatarkit-rtc livekit-client@2.16.1
```

--------------------------------

### Run Backend Token Server

Source: https://docs.spatialreal.ai/llms-full.txt

Start the token server in the backend directory. This is typically run in the first terminal.

```bash
cd avatarkit-voice-agent-demo/spatialreal-agent-quickstart/backend
uv run token_server.py
```

--------------------------------

### Stall Recovery Example

Source: https://docs.spatialreal.ai/guide/rtc-livekit-client

This example demonstrates how to handle the 'stalled' event by attempting to reconnect to the LiveKit server. This is useful for automatically recovering from temporary network issues.

```typescript
player.on('stalled', async () => {
  console.log('Stream stalled, reconnecting...')
  try {
    await player.reconnect()
  } catch (error) {
    console.error('Reconnection failed:', error)
  }
})
```

--------------------------------

### Create and Fill Frontend Environment File

Source: https://docs.spatialreal.ai/llms-full.txt

Copy the example frontend environment file and fill in your SpatialReal app ID, avatar ID, and token endpoint.

```bash
cp frontend/.env.example frontend/.env
```

```bash
VITE_SPATIALREAL_APP_ID=your_app_id # https://app.spatialreal.ai/apps
VITE_SPATIALREAL_AVATAR_ID=6aed28f9-674c-4ffb-89ee-b447b28aa3ed # https://app.spatialreal.ai/avatars/library
VITE_TOKEN_ENDPOINT=http://localhost:8080/token
VITE_ROOM_NAME=voice-agent-room
```

--------------------------------

### Install AvatarKit and LiveKit Client

Source: https://docs.spatialreal.ai/guide/rtc-livekit-client

Install the necessary packages for AvatarKit and a compatible version of livekit-client. Ensure you use version 2.16.1 of livekit-client due to compatibility requirements.

```bash
pnpm add @spatialwalk/avatarkit @spatialwalk/avatarkit-rtc livekit-client@2.16.1
```

```bash
npm install @spatialwalk/avatarkit @spatialwalk/avatarkit-rtc livekit-client@2.16.1
```

```bash
yarn add @spatialwalk/avatarkit @spatialwalk/avatarkit-rtc livekit-client@2.16.1
```

--------------------------------

### Run Frontend Development Server

Source: https://docs.spatialreal.ai/overview/agent

Start the frontend development server in the frontend directory. This will launch the user interface for interacting with the voice agent.

```bash
# Terminal 3: frontend
cd avatarkit-voice-agent-demo/spatialreal-agent-quickstart/frontend
pnpm dev
```

--------------------------------

### Run Token Server

Source: https://docs.spatialreal.ai/overview/agent

Start the token server in the backend directory. This server is responsible for generating authentication tokens for the voice agent.

```bash
# Terminal 1: token server
cd avatarkit-voice-agent-demo/spatialreal-agent-quickstart/backend
uv run token_server.py
```

--------------------------------

### Install AvatarKit SDK with yarn

Source: https://docs.spatialreal.ai/guide/sdk-mode-web

Use this command to add the AvatarKit SDK to your project using yarn.

```bash
yarn add @spatialwalk/avatarkit
```

--------------------------------

### Install AvatarKit SDK with npm

Source: https://docs.spatialreal.ai/guide/sdk-mode-web

Use this command to add the AvatarKit SDK to your project using npm.

```bash
npm install @spatialwalk/avatarkit
```

--------------------------------

### Install AvatarKit SDK with pnpm

Source: https://docs.spatialreal.ai/guide/sdk-mode-web

Use this command to add the AvatarKit SDK to your project using pnpm.

```bash
pnpm add @spatialwalk/avatarkit
```

--------------------------------

### Initialize SDK

Source: https://docs.spatialreal.ai/guide/sdk-mode-web

Initializes the AvatarSDK with your App ID and configuration. The session token must be set before calling `start()`.

```APIDOC
## Initialize SDK

### Description
Initializes the AvatarSDK with your App ID and configuration. The session token must be set before calling `start()`.

### Method
`AvatarSDK.initialize(appId: string, configuration: Configuration)`

### Parameters
#### Path Parameters
- `appId` (string) - Required - Your unique application identifier.
- `configuration` (Configuration) - Required - SDK configuration object.
  - `environment` (Environment) - Required - The target environment (`Environment.intl` or `Environment.cn`).
  - `drivingServiceMode` (DrivingServiceMode) - Optional - The mode for the driving service (defaults to `DrivingServiceMode.sdk`).

### Request Example
```typescript
import {
  AvatarSDK,
  Environment,
  DrivingServiceMode,
} from '@spatialwalk/avatarkit'

await AvatarSDK.initialize('your-app-id', {
  environment: Environment.intl,  // or Environment.cn
  drivingServiceMode: DrivingServiceMode.sdk,  // Default
})
```
```

--------------------------------

### Initialize AvatarSDK (Web)

Source: https://docs.spatialreal.ai/concepts/lifecycle

Call AvatarSDK.initialize() once at app startup to set global configuration. This example shows the required parameters for web initialization.

```typescript
await AvatarSDK.initialize({
  appId: 'YOUR_APP_ID',
  audioFormat: { channelCount: 1, sampleRate: 16000 },
  drivingServiceMode: 'sdk',
});
```

--------------------------------

### Obtaining MediaStreamTracks for Custom Audio

Source: https://docs.spatialreal.ai/guide/rtc-livekit-client

Examples of how to obtain `MediaStreamTrack` objects from various audio sources for custom audio publishing.

```typescript
audioElement.captureStream().getAudioTracks()[0]

getDisplayMedia({ audio: true })

audioContext.createMediaStreamDestination().stream.getAudioTracks()[0]
```

--------------------------------

### AvatarSDK Initialization

Source: https://docs.spatialreal.ai/sdk-reference/android-sdk/api-reference

Initialize the AvatarKit SDK with your application context, app ID, and configuration. This is a core setup step for using the SDK.

```kotlin
object AvatarSDK
```

```kotlin
var sessionToken: String
```

```kotlin
var userId: String
```

```kotlin
val version: String
```

```kotlin
fun initialize(
      context: Context,
      appId: String,
      configuration: Configuration
  )
```

```kotlin
suspend fun isDeviceSupported(): Boolean
```

```kotlin
suspend fun deviceScore(): DeviceScore
```

--------------------------------

### Connect and Interact with Controller

Source: https://docs.spatialreal.ai/concepts/lifecycle

Start the WebSocket connection using controller.start(). Send audio data using controller.send() or controller.yield().

```mermaid
sequenceDiagram
      autonumber
      participant C as Controller
      participant S as Connection/Voice State

      C->>S: controller.start()
      Note right of S: onConnectionState: connecting
      S-->>C: onConnectionState: connected
      S-->>C: onConversationState: idle
      
      rect rgb(240, 240, 240)
          Note over C,S: Audio Streaming
          C->>S: send(audio, end: false)
          S-->>C: onConversationState: playing
          C->>S: ... multiple chunks ...
          C->>S: send(audio, end: false)
          S-->>C: onConversationState: playing
          C->>S: send(audio chunk, end: true)
      end

      S-->>C: onConversationState: playing
      S-->>C: onConversationState: idle
```

--------------------------------

### Start Avatar Service

Source: https://docs.spatialreal.ai/sdk-reference/ios-sdk/api-reference

Starts the avatar driving service connection. Ensure this is called before attempting to interact with the avatar.

```swift
func start()
```

--------------------------------

### Session Token Response Example

Source: https://docs.spatialreal.ai/api-reference/api-reference

This is an example of a successful response when obtaining a session token. The response contains the generated `sessionToken`.

```json
{
  "sessionToken": "..."
}
```

--------------------------------

### Install livekit-plugins-spatialreal

Source: https://docs.spatialreal.ai/guide/rtc-livekit-server

Install the SpatialReal plugin for LiveKit Agents using pip. This package enables integration between your agent and SpatialReal for avatar streaming.

```bash
pip install livekit-plugins-spatialreal
```

--------------------------------

### Initialize Configuration

Source: https://docs.spatialreal.ai/sdk-reference/ios-sdk/api-reference

Creates a new configuration with the specified parameters for environment, audio format, driving service mode, and log level.

```swift
init(
      environment: Environment,
      audioFormat: AudioFormat,
      drivingServiceMode: DrivingServiceMode,
      logLevel: LogLevel
  )
```

--------------------------------

### Quick Start: Initialize and Use Avatar Session

Source: https://docs.spatialreal.ai/sdk-reference/python-sdk/python-sdk

Connect to avatar services, send audio, and receive animation frames. Ensure you replace placeholders with your actual credentials and endpoints.

```python
import asyncio
from datetime import datetime, timedelta, timezone

from avatarkit import new_avatar_session


async def main():
    session = new_avatar_session(
        api_key="your-api-key",
        app_id="your-app-id",
        console_endpoint_url="https://console.us-west.spatialwalk.cloud/v1/console",
        ingress_endpoint_url="wss://api.us-west.spatialwalk.cloud/v2/driveningress",
        avatar_id="your-avatar-id",
        expire_at=datetime.now(timezone.utc) + timedelta(minutes=5),
        transport_frames=lambda frame, last: print(f"Received frame: {len(frame)} bytes"),
        on_error=lambda err: print(f"Error: {err}"),
        on_close=lambda: print("Session closed"),
    )

    await session.init()
    connection_id = await session.start()
    print(f"Connected: {connection_id}")

    audio_data = b"..."  # Your PCM audio data
    request_id = await session.send_audio(audio_data, end=True)
    print(f"Sent audio: {request_id}")

    await asyncio.sleep(10)
    await session.close()


if __name__ == "__main__":
    asyncio.run(main())
```

--------------------------------

### Configure Environment Variables for Speech-to-Avatar Demo

Source: https://docs.spatialreal.ai/llms-full.txt

Copy the example environment file and update it with your SpatialReal credentials. Ensure you have your App ID, Avatar ID, and a temporary session token.

```bash
cp .env.example .env

VITE_SPATIALREAL_APP_ID=your_app_id # https://app.spatialreal.ai/apps
VITE_SPATIALREAL_AVATAR_ID=your_avatar_id # https://app.spatialreal.ai/avatars/library
VITE_SPATIALREAL_SESSION_TOKEN=your_temporary_session_token # https://app.spatialreal.ai/apps -> details -> Generate Temporary Token
```

--------------------------------

### Start Avatar Driving Service Connection

Source: https://docs.spatialreal.ai/sdk-reference/android-sdk/api-reference

Starts the avatar driving service connection. Call this method to initiate communication with the avatar service.

```kotlin
fun start()
```

--------------------------------

### Start and Send Audio

Source: https://docs.spatialreal.ai/guide/sdk-mode-web

Starts the WebSocket connection to the AvatarKit server and sends audio data. The connection state must be 'connected' before sending data.

```APIDOC
## Start and Send Audio

### Description
Starts the WebSocket connection to the AvatarKit server and sends audio data. The connection state must be 'connected' before sending data. Use `send(audioData, true)` to mark the end of the current audio input.

### Method
`avatarView.controller.start()`
`avatarView.controller.send(audioData: ArrayBuffer, isLastChunk: boolean)`

### Parameters
#### Path Parameters for `send`
- `audioData` (ArrayBuffer) - Required - The audio data in PCM16 format.
- `isLastChunk` (boolean) - Required - Indicates if this is the last chunk of audio for the current conversation round.

### Request Example
```typescript
// Start WebSocket connection
await avatarView.controller.start()

// Wait for connection to be ready
await new Promise<void>((resolve, reject) => {
  avatarView.controller.onConnectionState = (state) => {
    if (state === ConnectionState.connected) resolve()
    else if (state === ConnectionState.failed) reject(new Error('Connection failed'))
  }
})

// Send audio data
const audioData: ArrayBuffer = /* your PCM16 audio data */
avatarView.controller.send(audioData, false)  // Continue sending

// Mark end of audio input
avatarView.controller.send(lastChunk, true)
```
```

--------------------------------

### Initialize SDK with Audio Format (Web)

Source: https://docs.spatialreal.ai/concepts/audio

Configure the audio sample rate and channel count when initializing the Web SDK. Ensure this matches your audio source.

```typescript
await AvatarSDK.initialize({
  appId: 'YOUR_APP_ID',
  audioFormat: { channelCount: 1, sampleRate: 16000 },
  // ...
});
```

--------------------------------

### Handle Session Initialization and Start Errors

Source: https://docs.spatialreal.ai/sdk-reference/python-sdk/python-sdk

Use a try-except block to catch `SessionTokenError` during token creation and `AvatarSDKError` for other SDK-related errors during session initialization and start.

```python
from avatarkit import AvatarSDKError, SessionTokenError


try:
    await session.init()
    await session.start()
except SessionTokenError as error:
    print("token failed", error.code.value, error.server_detail)
except AvatarSDKError as error:
    print("sdk error", error.code.value, error.phase, error.server_detail)
```

--------------------------------

### Initialize SDK with Audio Format (Android)

Source: https://docs.spatialreal.ai/concepts/audio

Configure the audio sample rate for the Android SDK. Ensure the sample rate aligns with your audio input.

```kotlin
AvatarSDK.initialize(
    context = applicationContext,
    appId = "YOUR_APP_ID",
    configuration = Configuration(
        environment = Environment.Intl,
        audioFormat = AudioFormat(16000),
        drivingServiceMode = DrivingServiceMode.SDK,
        logLevel = LogLevel.INFO
    )
)
```

--------------------------------

### Complete Avatar SDK Usage Example

Source: https://docs.spatialreal.ai/sdk-reference/web-sdk/api-reference

Demonstrates the complete lifecycle of initializing and using the Avatar SDK, including loading avatars, setting up views, managing connection and conversation states, and handling audio data.

```typescript
import {
  AvatarSDK,
  AvatarManager,
  AvatarView,
  Environment,
  ConnectionState,
  ConversationState,
} from '@spatialwalk/avatarkit'

class AvatarApp {
  private avatarView: AvatarView | null = null

  async init(appId: string, sessionToken: string, avatarId: string, container: HTMLElement) {
    // Initialize SDK
    await AvatarSDK.initialize(appId, { environment: Environment.cn })
    AvatarSDK.setSessionToken(sessionToken)

    // Load avatar
    const avatar = await AvatarManager.shared.load(avatarId)
    if (!avatar) throw new Error('Failed to load avatar')

    // Create view
    this.avatarView = new AvatarView(avatar, container)
    this.avatarView.onFirstRendering = () => {
      console.log('First frame rendered')
    }

    // Set up handlers
    this.avatarView.controller.onConnectionState = (state) => {
      console.log('Connection:', state)
    }
    this.avatarView.controller.onConversationState = (state) => {
      console.log('Conversation:', state)
    }
    this.avatarView.controller.onError = (error) => {
      console.error('Error:', error)
    }
  }

  // Must be called in user gesture context
  async start() {
    await this.avatarView?.controller.initializeAudioContext()
    await this.avatarView?.controller.start()
  }

  send(audioData: ArrayBuffer, isEnd: boolean) {
    this.avatarView?.controller.send(audioData, isEnd)
  }

  interrupt() {
    this.avatarView?.controller.interrupt()
  }

  dispose() {
    this.avatarView?.controller.close()
    this.avatarView?.dispose()
    this.avatarView = null
  }
}
```

--------------------------------

### Configure Backend Environment Variables

Source: https://docs.spatialreal.ai/overview/agent

Fill in the backend .env file with your LiveKit, Google API, and SpatialReal credentials. Ensure these values are correct for your services.

```bash
LIVEKIT_URL=wss://your-project.livekit.cloud # https://cloud.livekit.io
LIVEKIT_API_KEY=your_api_key # https://cloud.livekit.io
LIVEKIT_API_SECRET=your_api_secret # https://cloud.livekit.io

GOOGLE_API_KEY=your_google_api_key # https://aistudio.google.com/api-keys
E2E_GOOGLE_MODEL=gemini-2.5-flash-native-audio-preview-12-2025
E2E_GOOGLE_VOICE=Puck

SPATIALREAL_API_KEY=your_api_key # https://app.spatialreal.ai/apps
SPATIALREAL_APP_ID=your_app_id # https://app.spatialreal.ai/apps
SPATIALREAL_AVATAR_ID=6aed28f9-674c-4ffb-89ee-b447b28aa3ed # https://app.spatialreal.ai/avatars/library
```

--------------------------------

### Access AvatarController

Source: https://docs.spatialreal.ai/sdk-reference/web-sdk/api-reference

Get the AvatarController instance from an AvatarView object to manage communication.

```typescript
// Accessed via AvatarView
const controller = avatarView.controller
```

--------------------------------

### Configuration Initializer

Source: https://docs.spatialreal.ai/llms-full.txt

Creates a new configuration with the specified parameters.

```APIDOC
## Configuration(environment, audioFormat, drivingServiceMode, logLevel)

### Description
Creates a new configuration with the specified parameters.

### Parameters
* `environment` (Environment) - The environment to use.
* `audioFormat` (AudioFormat) - The audio format configuration.
* `drivingServiceMode` (DrivingServiceMode) - The driving service mode.
* `logLevel` (LogLevel) - The log level.
```

--------------------------------

### Start Avatar Service Connection

Source: https://docs.spatialreal.ai/sdk-reference/web-sdk/api-reference

Connect to the avatar service. This is only applicable in SDK mode.

```typescript
await controller.start()
```

--------------------------------

### Runtime Sequence for Avatar Integration

Source: https://docs.spatialreal.ai/guide/speech-to-avatar-walkthrough

This sequence outlines the steps involved in initializing the SDK and sending audio data. Use it as a reference for debugging connection or audio send issues.

```text
initialize SDK -> set session token -> load avatar -> create AvatarView -> start controller
-> fetch pcm -> controller.send(audio, true)
```

--------------------------------

### Get AvatarSDK Configuration

Source: https://docs.spatialreal.ai/sdk-reference/ios-sdk/api-reference

Access the current configuration object for AvatarKit. This is a read-only property.

```swift
static var configuration: Configuration { get }
```

--------------------------------

### Get AvatarSDK App ID

Source: https://docs.spatialreal.ai/sdk-reference/ios-sdk/api-reference

Access the application identifier for AvatarKit. This is a read-only property.

```swift
static var appID: String { get }
```

--------------------------------

### Get AvatarKit Version

Source: https://docs.spatialreal.ai/sdk-reference/ios-sdk/api-reference

Retrieve the current version of the AvatarKit SDK. This is a read-only property.

```swift
static var version: String { get }
```

--------------------------------

### Microphone Control

Source: https://docs.spatialreal.ai/llms-full.txt

Control the microphone for voice interaction, including starting and stopping audio publishing.

```APIDOC
## Microphone Control

```typescript
// Start microphone (requests permission automatically)
await player.startPublishing()

// Stop microphone
await player.stopPublishing()
```
```

--------------------------------

### Initialize AudioFormat

Source: https://docs.spatialreal.ai/sdk-reference/ios-sdk/api-reference

Creates a new audio format with the specified sample rate. The channel count is fixed to 1 for mono.

```swift
init(sampleRate: Int)
```

--------------------------------

### Configuration Constructor

Source: https://docs.spatialreal.ai/sdk-reference/android-sdk/api-reference

Creates a new Configuration instance with specified parameters. Default values are provided for audio format, driving service mode, and log level.

```kotlin
data class Configuration(
      val environment: Environment,
      val audioFormat: AudioFormat = AudioFormat(16000),
      val drivingServiceMode: DrivingServiceMode = DrivingServiceMode.SDK,
      val logLevel: LogLevel = LogLevel.INFO
  )
```

--------------------------------

### AudioFormat Initializer

Source: https://docs.spatialreal.ai/llms-full.txt

Creates a new audio format with the specified sample rate.

```APIDOC
## AudioFormat(sampleRate:)

### Description
Creates a new audio format with the specified sample rate.

### Parameters
* `sampleRate` (Int) - The audio sample rate in Hz. Supported sample rates are: 8000, 16000, 22050, 24000, 32000, 44100, 48000.
```

--------------------------------

### Set Session Token

Source: https://docs.spatialreal.ai/guide/sdk-mode-web

Sets the session token for authentication. This must be done before starting the avatar controller.

```APIDOC
## Set Session Token

### Description
Sets the session token for authentication. This must be done before starting the avatar controller. It can be called before or after `initialize()`.

### Method
`AvatarSDK.setSessionToken(token: string)`

### Parameters
#### Path Parameters
- `token` (string) - Required - The session token obtained from the AvatarKit Server.
```

--------------------------------

### AudioFormat Initializer

Source: https://docs.spatialreal.ai/sdk-reference/ios-sdk/api-reference

Creates a new audio format configuration with a specified sample rate.

```APIDOC
## init(sampleRate:)

### Description
Creates a new audio format with the specified sample rate.

### Parameters

- `sampleRate` (Int) - The audio sample rate in Hz
```

--------------------------------

### Clone Demo Project Repository

Source: https://docs.spatialreal.ai/overview/demo-projects

Use this command to clone the demo project repository from GitHub and navigate into the project directory.

```bash
git clone https://github.com/spatialwalk/avatarkit-voice-agent-demo.git
cd avatarkit-voice-agent-demo
```

--------------------------------

### Sending Audio Data

Source: https://docs.spatialreal.ai/sdk-reference/python-sdk/python-sdk

Example of how to read PCM audio data from a file and send it through the established session.

```APIDOC
## Sending Audio Data

Send audio data in mono 16-bit PCM (`s16le`) format.

```python
with open("audio.pcm", "rb") as f:
    audio_data = f.read()

await session.send_audio(audio_data, end=True)
```
```

--------------------------------

### LiveKit Egress Configuration

Source: https://docs.spatialreal.ai/sdk-reference/python-sdk/python-sdk

Demonstrates how to configure the SDK for LiveKit egress, streaming audio and animation data to a LiveKit room.

```APIDOC
## LiveKit Egress Mode

Configure LiveKit egress to stream output to a LiveKit room.

```python
from datetime import datetime, timedelta, timezone

from avatarkit import LiveKitEgressConfig, new_avatar_session


session = new_avatar_session(
    avatar_id="avatar-123",
    api_key="your-api-key",
    app_id="your-app-id",
    console_endpoint_url="https://console.us-west.spatialwalk.cloud/v1/console",
    ingress_endpoint_url="wss://api.us-west.spatialwalk.cloud/v2/driveningress",
    expire_at=datetime.now(timezone.utc) + timedelta(minutes=5),
    livekit_egress=LiveKitEgressConfig(
        url="wss://livekit.example.com",
        api_key="livekit-api-key",
        api_secret="livekit-api-secret",
        room_name="my-room",
        publisher_id="avatar-publisher",
    ),
)
```

When LiveKit egress is enabled:
* the server streams output to the specified LiveKit room
* the `transport_frames` callback is not invoked
* audio and animation data are published under the configured publisher ID
```

--------------------------------

### AvatarSDK Initialization and Configuration

Source: https://docs.spatialreal.ai/llms-full.txt

Main entry point for SDK initialization and global configuration. Must be called before any other operations.

```APIDOC
## AvatarSDK

Main entry point for SDK initialization and global configuration.

### Static Methods

#### initialize(appId, configuration)

Initialize the SDK. Must be called before any other operations.

```typescript
await AvatarSDK.initialize('your-app-id', {
  environment: Environment.cn,  // or Environment.intl
  drivingServiceMode: DrivingServiceMode.sdk,  // Optional, default: sdk
  logLevel: LogLevel.warning,  // Optional, default: off
  audioFormat: {  // Optional
    channelCount: 1,
    sampleRate: 16000,
  },
})
```

#### setSessionToken(token)

Set session token for authentication. **Only required for SDK Mode** (server-driven animation).

```typescript
AvatarSDK.setSessionToken('your-session-token')
```

**Important:**

* **Only needed for SDK Mode** - not required for Host Mode or RTC Mode
* Token must be obtained from your backend server
* Token has max 24 hours validity
* Token must be paired with App ID

#### setUserId(userId)

Set user ID for logging and analytics.

```typescript
AvatarSDK.setUserId('user-123')
```

#### cleanup()

Release all SDK resources. Call when SDK is no longer needed.

```typescript
AvatarSDK.cleanup()
```
```

--------------------------------

### AvatarManager Get All Cache Size

Source: https://docs.spatialreal.ai/llms-full.txt

Retrieves the total cache size in bytes for all cached avatars. This is a suspend function.

```APIDOC
## AvatarManager.getAllCacheSize

### Description
Gets the total cache size for all avatars.

### Method
`suspend fun getAllCacheSize(): Long`

### Returns
* `Long` - The total cache size in bytes.
```

--------------------------------

### Connect to LiveKit Server

Source: https://docs.spatialreal.ai/llms-full.txt

Establish a connection to the LiveKit server using the provided configuration, including URL, token, and room name.

```APIDOC
## Connect to LiveKit Server

```typescript
await player.connect({
  url: 'wss://your-livekit-server.com',
  token: 'your-livekit-token',
  roomName: 'room-name',
})
```
```

--------------------------------

### Get Current Conversation ID

Source: https://docs.spatialreal.ai/sdk-reference/web-sdk/api-reference

Retrieve the ID of the currently active conversation. Returns null if no conversation is active.

```typescript
const conversationId = controller.getCurrentConversationId()
// Returns: string | null
```

--------------------------------

### Configuration Interface

Source: https://docs.spatialreal.ai/sdk-reference/web-sdk/api-reference

Defines the configuration options for initializing the SDK.

```APIDOC
## Configuration

```typescript
interface Configuration {
  environment: Environment
  drivingServiceMode?: DrivingServiceMode  // Default: 'sdk'
  logLevel?: LogLevel                       // Default: 'off'
  audioFormat?: AudioFormat                 // Default: { channelCount: 1, sampleRate: 16000 }
}
```
```

--------------------------------

### AvatarController Instance Methods

Source: https://docs.spatialreal.ai/sdk-reference/web-sdk/api-reference

Methods for managing audio, connection, and playback.

```APIDOC
## initializeAudioContext()

### Description
Initialize audio context. MUST be called in a user gesture context.

### Method
`initializeAudioContext()`

### Usage Example
```typescript
button.addEventListener('click', async () => {
  await controller.initializeAudioContext()
})
```

## start()

### Description
Connect to the avatar service. SDK mode only.

### Method
`start()`

## send(audioData, end)

### Description
Send audio data to the server. SDK mode only.

### Parameters
#### Path Parameters
- **audioData** (ArrayBuffer) - Required - PCM16 (S16LE) mono audio data. Byte length must be even.
- **end** (boolean) - Required - Whether this is the final chunk.

### Returns
- **string** - Conversation ID

### Warning
Audio data byte length must be even (2 bytes per sample). Odd-length data will cause a server-side validation error and WebSocket disconnect.

### Usage Example
```typescript
// Send audio data (continue)
controller.send(audioData, false)

// Send final audio data (end conversation)
controller.send(audioData, true)
```

## close()

### Description
Close the service connection. SDK mode only.

### Method
`close()`
```

--------------------------------

### Microphone Control

Source: https://docs.spatialreal.ai/guide/rtc-livekit-client

Control the microphone for publishing audio. This includes starting and stopping the microphone, which automatically handles permission requests.

```APIDOC
## Microphone Control

### Start microphone
```typescript
// Start microphone (requests permission automatically)
await player.startPublishing()
```

### Stop microphone
```typescript
// Stop microphone
await player.stopPublishing()
```
```

--------------------------------

### getCurrentConversationId

Source: https://docs.spatialreal.ai/llms-full.txt

Get the current active conversation ID. Returns the ID of the conversation currently being processed or null if none is active.

```APIDOC
## getCurrentConversationId()

### Description
Get the current active conversation ID.

### Method
GET

### Endpoint
/conversationId

### Response
#### Success Response (200)
- **conversationId** (string | null) - The ID of the current conversation or null.

### Response Example
```json
{
  "conversationId": "some-conversation-id"
}
```

### Request Example
```typescript
const conversationId = controller.getCurrentConversationId()
```
```

--------------------------------

### Get Current Conversation ID with AvatarController

Source: https://docs.spatialreal.ai/llms-full.txt

Retrieve the ID of the currently active conversation. Returns `null` if no conversation is in progress.

```typescript
// Conversation
controller.getCurrentConversationId()  // string | null
```

--------------------------------

### Get Current Conversation ID

Source: https://docs.spatialreal.ai/llms-full.txt

Retrieve the current active `conversationId` using `getCurrentConversationId()`. This is useful for ensuring message synchronization.

```javascript
const currentConversationId = getCurrentConversationId();
```

--------------------------------

### Get Avatar Point Count

Source: https://docs.spatialreal.ai/sdk-reference/ios-sdk/api-reference

Retrieves the point count of the current avatar. This may be relevant for performance or rendering details.

```swift
var pointCount: Int { get }
```

--------------------------------

### Run Backend LiveKit Agent Worker

Source: https://docs.spatialreal.ai/llms-full.txt

Start the LiveKit agent worker in the backend directory. This is typically run in the second terminal.

```bash
cd avatarkit-voice-agent-demo/spatialreal-agent-quickstart/backend
uv run agent.py dev
```
Tokens:
Raw