### Run Development Server Source: https://docs.spatius.ai/quickstarts/web-sdk Start the development server to run the quickstart demo. ```bash pnpm dev ``` -------------------------------- ### Install Golang SDK Source: https://docs.spatius.ai/sdk-reference/go-sdk/go-sdk Install the Spatius Golang SDK using go get. ```bash go get github.com/spatius-ai/spatius-sdk-go ``` -------------------------------- ### Server Setup with AvatarSession Source: https://docs.spatius.ai/livekit-agents/overview Configures and starts an AvatarSession integrated with a LiveKit Agent session. This example demonstrates piping TTS audio to Spatius and publishing synchronized audio and motion data. ```python from livekit.agents import AgentSession, JobContext from livekit.plugins import spatius async def entrypoint(ctx: JobContext): await ctx.connect() session = AgentSession(vad=vad, stt=stt, llm=llm, tts=tts) avatar = spatius.AvatarSession() await avatar.start(session, room=ctx.room) await session.start(agent=YourAgent(), room=ctx.room) ``` -------------------------------- ### Complete AvatarPlayer Initialization with LiveKit Source: https://docs.spatius.ai/livekit-agents/client A full example demonstrating the initialization of AvatarSDK, loading an avatar, creating an AvatarView, setting up the LiveKitProvider, and connecting to a LiveKit server. This covers the entire client-side setup for real-time avatar streaming. ```typescript import { AvatarPlayer, LiveKitProvider } from '@spatius/avatarkit-rtc' import { AvatarSDK, AvatarView, AvatarManager, DrivingServiceMode } from '@spatius/avatarkit' async function init() { // Initialize AvatarKit for LiveKit transport rendering. // Note: `region` requires a small type augmentation in @spatius/avatarkit@1.0.0 — // see the Direct Mode Web guide for the avatarkit-region.d.ts snippet. await AvatarSDK.initialize('your-app-id', { region: 'us-west', drivingServiceMode: DrivingServiceMode.host, }) // `setSessionToken` is not needed on the RTC Adapter path — // `DrivingServiceMode.host` does not open a Motion Server WebSocket from the client, // and `AvatarManager.load()` fetches avatar metadata over an App-ID-scoped public endpoint. // Load avatar and create view const avatar = await AvatarManager.shared.load('character-id') const container = document.getElementById('avatar-container')! const avatarView = new AvatarView(avatar, container) // Create player const provider = new LiveKitProvider() const player = new AvatarPlayer(provider, avatarView, { logLevel: 'info', enableJitterBuffer: true, maxBufferDelayMs: 80, }) // Listen to events player.on('connected', () => console.log('Connected!')) player.on('disconnected', () => console.log('Disconnected!')) player.on('error', (err) => console.error('Error:', err)) player.on('stalled', async () => { console.log('Stream stalled, reconnecting...') await player.reconnect() }) // Connect to LiveKit server await player.connect({ url: 'wss://your-livekit-server.com', token: 'your-livekit-token', roomName: 'my-room', }) // Start microphone await player.startPublishing() } ``` -------------------------------- ### Quick Start: Full Avatar Session Lifecycle Source: https://docs.spatius.ai/sdk-reference/python-sdk/python-sdk A complete example demonstrating the initialization, connection, audio sending, and closing of an avatar session. Includes callbacks for frame transport, errors, and session closure. ```python import asyncio from datetime import datetime, timedelta, timezone from spatius import new_avatar_session async def main(): session = new_avatar_session( api_key="your-api-key", app_id="your-app-id", avatar_id="your-avatar-id", expire_at=datetime.now(timezone.utc) + timedelta(minutes=5), transport_frames=lambda frame, last: print( f"Received frame: {len(frame)} bytes, last={last}" ), on_error=lambda err: print(f"Session error: {err}"), on_close=lambda: print("Session closed"), ) await session.init() connection_id = await session.start() print(f"Connected: {connection_id}") audio_data = b"..." # mono PCM s16le audio bytes request_id = await session.send_audio(audio_data, end=True) print(f"Sent audio request: {request_id}") await asyncio.sleep(10) await session.close() if __name__ == "__main__": asyncio.run(main()) ``` -------------------------------- ### Clone and Setup iOS Project Source: https://docs.spatius.ai/quickstarts/ios-sdk Clone the Spatius avatar demo repository, navigate to the iOS direct mode client directory, install xcodegen, and generate the Xcode project. ```bash git clone https://github.com/spatius-ai/spatius-avatar-demo.git cd spatius-avatar-demo/direct-mode/clients/ios brew install xcodegen ``` ```bash xcodegen generate open AvatarDemo.xcodeproj ``` -------------------------------- ### Clone Repository and Navigate to Quickstart Folder Source: https://docs.spatius.ai/livekit-agents/quickstart Clone the Spatius avatar demo repository and change the directory to the LiveKit agent quickstart folder. ```bash git clone https://github.com/spatius-ai/spatius-avatar-demo.git cd spatius-avatar-demo/platform-integrations/livekit-agents-demo/livekit-agent-quickstart ``` -------------------------------- ### Create Environment File Source: https://docs.spatius.ai/quickstarts/web-sdk Copy the example environment file and fill in your Spatius credentials. ```bash cp .env.example .env ``` -------------------------------- ### Install Dependencies Source: https://docs.spatius.ai/quickstarts/web-sdk Install the necessary project dependencies using pnpm. ```bash pnpm install ``` -------------------------------- ### Install Spatius Python SDK Source: https://docs.spatius.ai/sdk-reference/python-sdk/python-sdk Install the Spatius Python SDK using pip. This is the base installation. ```bash pip install spatius ``` -------------------------------- ### start() Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference Starts the avatar driving service connection. This method initiates the connection to the avatar driving service. ```APIDOC ## start() ### Description Starts the avatar driving service connection. ### Method `start()` ### Parameters None ### Response None ``` -------------------------------- ### Install @spatius/avatarkit-rtc and @spatius/avatarkit Source: https://docs.spatius.ai/sdk-reference/web-sdk/rtc-adapter Install the RTC adapter and the core avatar rendering SDK using your preferred package manager. ```bash pnpm add @spatius/avatarkit-rtc @spatius/avatarkit ``` ```bash npm install @spatius/avatarkit-rtc @spatius/avatarkit ``` ```bash yarn add @spatius/avatarkit-rtc @spatius/avatarkit ``` -------------------------------- ### Quick Start Golang Avatar Session Source: https://docs.spatius.ai/sdk-reference/go-sdk/go-sdk Initialize and start a Spatius avatar session with basic configuration and send audio data. Handles session initialization, starting the connection, and sending audio chunks. ```go package main import ( "context" "log" "time" spatius "github.com/spatius-ai/spatius-sdk-go" ) func main() { ctx := context.Background() session := spatius.NewAvatarSession( spatius.WithAPIKey("your-api-key"), spatius.WithAppID("your-app-id"), spatius.WithAvatarID("your-avatar-id"), spatius.WithRegion("us-west"), spatius.WithExpireAt(time.Now().Add(5*time.Minute).UTC()), spatius.WithTransportFrames(func(data []byte, last bool) { // Handle encoded motion frame payloads. }), spatius.WithOnError(func(err error) { // Handle async session errors. }), spatius.WithOnClose(func() { // Handle connection close. }), ) if err := session.Init(ctx); err != nil { log.Fatal(err) } defer session.Close() connectionID, err := session.Start(ctx) if err != nil { log.Fatal(err) } log.Printf("connection id: %s", connectionID) audioBytes := []byte{} // Replace with mono PCM audio bytes. reqID, err := session.SendAudio(audioBytes, true) if err != nil { log.Fatal(err) } log.Printf("request id: %s", reqID) } ``` -------------------------------- ### Install Dependencies Source: https://docs.spatius.ai/livekit-agents/quickstart Install backend dependencies using uv and frontend dependencies using pnpm. Ensure that the versions of livekit-agents, livekit-plugins-google, and livekit-plugins-spatius are kept aligned. ```bash # backend cd backend uv sync # frontend cd ../frontend pnpm install ``` -------------------------------- ### Install AvatarKit UI with shadcn/ui Source: https://docs.spatius.ai/sdk-reference/web-sdk/avatarkit-ui Use these commands to initialize shadcn/ui and add the AvatarKit UI components to your project. ```bash npx shadcn@latest init npx shadcn@latest add https://ui.spatius.ai/r/spatius-avatar.json ``` -------------------------------- ### AvatarController start() Method Source: https://docs.spatius.ai/sdk-reference/ios-sdk/api-reference Starts the avatar driving service connection. Call this method to initiate the connection to the avatar service. ```swift func start() ``` -------------------------------- ### Install LiveKit Agents Spatius Plugin (uv) Source: https://docs.spatius.ai/livekit-agents/overview Installs the livekit-plugins-spatius package using uv for Python environments. ```bash uv add livekit-plugins-spatius ``` -------------------------------- ### Clone Repository and Navigate Source: https://docs.spatius.ai/quickstarts/web-sdk Clone the Spatius Avatar Demo repository and navigate into the specific web quickstart directory. ```bash git clone https://github.com/spatius-ai/spatius-avatar-demo.git cd spatius-avatar-demo/direct-mode/clients/web/speech-to-avatar-quickstart ``` -------------------------------- ### Create Environment Files Source: https://docs.spatius.ai/livekit-agents/quickstart Copy example environment files for backend and frontend configurations. These files will store your Spatius, LiveKit, and Google AI Studio credentials. ```bash cp backend/.env.example backend/.env cp frontend/.env.example frontend/.env ``` -------------------------------- ### Install livekit-plugins-spatius Source: https://docs.spatius.ai/livekit-agents/server Install the package using pip. Ensure to manage version compatibility with livekit-agents. ```bash pip install livekit-plugins-spatius ``` -------------------------------- ### Install iOS Pods and Run Flutter App Source: https://docs.spatius.ai/quickstarts/flutter-sdk For iOS development, install CocoaPods dependencies and then run the Flutter application. ```bash cd ios pod install cd .. flutter run ``` -------------------------------- ### Clone Flutter Direct Mode Demo Source: https://docs.spatius.ai/quickstarts/flutter-sdk Clone the Spatius avatar demo repository to get started with the Flutter Direct Mode client. ```bash git clone https://github.com/spatius-ai/spatius-avatar-demo.git cd spatius-avatar-demo/direct-mode/clients/flutter ``` -------------------------------- ### Install @spatius/avatarkit with npm Source: https://docs.spatius.ai/direct-mode/web Use this command to add the Avatarkit SDK to your project when using npm. ```bash npm install @spatius/avatarkit ``` -------------------------------- ### Complete Web SDK Usage Example Source: https://docs.spatius.ai/sdk-reference/web-sdk/reference Demonstrates the full lifecycle of initializing, loading, and interacting with an avatar using the Web SDK. Ensure `initializeAudioContext` is called within a user gesture handler. ```typescript import { AvatarSDK, AvatarManager, AvatarView, ConnectionState, ConversationState, } from '@spatius/avatarkit' class AvatarApp { private avatarView: AvatarView | null = null async init(appId: string, sessionToken: string, avatarId: string, container: HTMLElement) { // Initialize the SDK await AvatarSDK.initialize(appId, { region: 'us-west' }) AvatarSDK.setSessionToken(sessionToken) // Load the avatar const avatar = await AvatarManager.shared.load(avatarId) if (!avatar) throw new Error('Failed to load avatar') // Create the view this.avatarView = new AvatarView(avatar, container) this.avatarView.onFirstRendering = () => { console.log('First frame rendered') } // Wire up handlers this.avatarView.controller.onConnectionState = (state) => { console.log('Connection:', state) } this.avatarView.controller.onConversationState = (state) => { console.log('Conversation:', state) } this.avatarView.controller.onError = (error) => { console.error('Error:', error) } } // Must be called inside a user-gesture handler async start() { await this.avatarView?.controller.initializeAudioContext() await this.avatarView?.controller.start() } send(audioData: ArrayBuffer, isEnd: boolean) { this.avatarView?.controller.send(audioData, isEnd) } interrupt() { this.avatarView?.controller.interrupt() } dispose() { this.avatarView?.controller.close() this.avatarView?.dispose() this.avatarView = null } } ``` -------------------------------- ### Install Flutter Dependencies Source: https://docs.spatius.ai/direct-mode/flutter Run this command in your Flutter project directory to install the project dependencies, including the Spatius package. ```bash flutter pub get ``` -------------------------------- ### Install @spatius/avatarkit with yarn Source: https://docs.spatius.ai/direct-mode/web Use this command to add the Avatarkit SDK to your project when using yarn. ```bash yarn add @spatius/avatarkit ``` -------------------------------- ### Run the Stack: Frontend Source: https://docs.spatius.ai/livekit-agents/quickstart Start the frontend development server in a third terminal. This provides the user interface for connecting to the LiveKit room and interacting with the voice avatar. ```bash # Terminal 3: frontend cd spatius-avatar-demo/platform-integrations/livekit-agents-demo/livekit-agent-quickstart/frontend pnpm dev ``` -------------------------------- ### Install @spatius/avatarkit with pnpm Source: https://docs.spatius.ai/direct-mode/web Use this command to add the Avatarkit SDK to your project when using pnpm. ```bash pnpm add @spatius/avatarkit ``` -------------------------------- ### Run the Stack: Token Server Source: https://docs.spatius.ai/livekit-agents/quickstart Start the token server in a dedicated terminal. This server is responsible for generating authentication tokens for LiveKit and Spatius. ```bash # Terminal 1: token server cd spatius-avatar-demo/platform-integrations/livekit-agents-demo/livekit-agent-quickstart/backend uv run token_server.py ``` -------------------------------- ### Install Spatius Python SDK with Opus Support Source: https://docs.spatius.ai/sdk-reference/python-sdk/python-sdk Install the Spatius Python SDK with optional Ogg Opus encoder support for encoding raw PCM before sending. This requires a working libopus runtime. ```bash pip install "spatius[opus]" ``` -------------------------------- ### Session Lifecycle Steps Source: https://docs.spatius.ai/sdk-reference/python-sdk/python-sdk Outlines the sequential steps for managing an avatar session: initialization, starting the connection, sending audio, receiving frames, and closing the session. ```python # 1. Initialize and request a session token await session.init() # 2. Start the WebSocket connection connection_id = await session.start() # 3. Send audio data request_id = await session.send_audio(audio_bytes, end=True) # 4. Receive encoded motion frame payloads through transport_frames # 5. Close the session await session.close() ``` -------------------------------- ### Start Avatar Driving Service Connection Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference Starts the avatar driving service connection. Call this method to initiate communication with the avatar service. ```kotlin fun start() ``` -------------------------------- ### Install AvatarKit RTC and AvatarKit (bun) Source: https://docs.spatius.ai/livekit-agents/overview Use this command to add the necessary Spatius AvatarKit packages to your project when using bun. ```bash bun add @spatius/avatarkit-rtc @spatius/avatarkit ``` -------------------------------- ### Connect Player to LiveKit Server Source: https://docs.spatius.ai/livekit-agents/overview Establishes a connection to the LiveKit server using the provided URL, token, and room name. Ensure these credentials are correct for your LiveKit setup. ```typescript await player.connect({ url: 'wss://your-livekit-server.com', token: 'your-livekit-token', roomName: 'room-name', }) ``` -------------------------------- ### Start AvatarController Connection Source: https://docs.spatius.ai/sdk-reference/web-sdk/reference Initiates the connection to the Motion Server using the AvatarController. This is a prerequisite for most controller functionalities. ```typescript await controller.start() ``` -------------------------------- ### Clone Spatius Avatar Demo Repository Source: https://docs.spatius.ai/resources/demo-projects Clone the spatius-avatar-demo repository and navigate into the project directory. Refer to individual demo READMEs for specific setup and run instructions. ```bash git clone https://github.com/spatius-ai/spatius-avatar-demo.git cd spatius-avatar-demo ``` -------------------------------- ### Install AvatarKit RTC and AvatarKit (npm) Source: https://docs.spatius.ai/livekit-agents/overview Use this command to add the necessary Spatius AvatarKit packages to your project when using npm. ```bash npm install @spatius/avatarkit-rtc @spatius/avatarkit ``` -------------------------------- ### Install AvatarKit RTC and AvatarKit (yarn) Source: https://docs.spatius.ai/livekit-agents/overview Use this command to add the necessary Spatius AvatarKit packages to your project when using yarn. ```bash yarn add @spatius/avatarkit-rtc @spatius/avatarkit ``` -------------------------------- ### Install AvatarKit RTC and AvatarKit (pnpm) Source: https://docs.spatius.ai/livekit-agents/overview Use this command to add the necessary Spatius AvatarKit packages to your project when using pnpm. ```bash pnpm add @spatius/avatarkit-rtc @spatius/avatarkit ``` -------------------------------- ### Run the Stack: LiveKit Agent Worker Source: https://docs.spatius.ai/livekit-agents/quickstart Start the LiveKit agent worker in a separate terminal. This worker processes real-time audio and video streams and integrates with Spatius for avatar functionalities. ```bash # Terminal 2: LiveKit agent worker cd spatius-avatar-demo/platform-integrations/livekit-agents-demo/livekit-agent-quickstart/backend uv run agent.py dev ``` -------------------------------- ### Handle Token and SDK Errors during Session Initialization Source: https://docs.spatius.ai/sdk-reference/python-sdk/python-sdk Use a try-except block to catch specific errors during session initialization and startup. This example demonstrates handling SessionTokenError for token-related issues and AvatarSDKError for other SDK errors, printing relevant details for each. ```python from spatius import AvatarSDKError, SessionTokenError try: await session.init() await session.start() except SessionTokenError as error: print("token failed", error.code.value, error.server_detail) except AvatarSDKError as error: print("sdk error", error.code.value, error.phase, error.server_detail) ``` -------------------------------- ### Switch Avatars Source: https://docs.spatius.ai/sdk-reference/web-sdk/reference Provides a step-by-step guide for disposing of the current avatar view, loading a new avatar, creating a new view, and reinitializing the controller. ```typescript // 1. Dispose the current view currentAvatarView.dispose() // 2. Load a new avatar const newAvatar = await AvatarManager.shared.load('new-avatar-id') // 3. Create a new view (reuse the same container) currentAvatarView = new AvatarView(newAvatar, container) // 4. Reconnect await currentAvatarView.controller.initializeAudioContext() await currentAvatarView.controller.start() ``` -------------------------------- ### Session Token Authentication Flow Source: https://docs.spatius.ai/direct-mode/overview Illustrates the authentication process for obtaining a Session Token, which is required before starting the SDK. The token has a maximum validity of 24 hours. ```text Your Client → Your Backend → Spatius Console API → Session Token (24 h max) ``` -------------------------------- ### init(region:audioFormat:drivingServiceMode:logLevel:) Source: https://docs.spatius.ai/sdk-reference/ios-sdk/api-reference Creates a new configuration with the specified parameters for initializing the SDK. ```APIDOC ## init(region:audioFormat:drivingServiceMode:logLevel:) ### Description Creates a new configuration with the specified parameters. ### Parameters #### Path Parameters * `region` (String) - Required - The region to connect to (currently only `"us-west"`) * `audioFormat` (AudioFormat) - Required - The audio format configuration * `drivingServiceMode` (DrivingServiceMode) - Required - The driving service mode * `logLevel` (LogLevel) - Required - The log level ``` -------------------------------- ### AvatarManager Get All Cache Size Method Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference Gets the total cache size in bytes for all cached avatars. This is an asynchronous operation. ```kotlin suspend fun getAllCacheSize(): Long ``` -------------------------------- ### AvatarController.start Source: https://docs.spatius.ai/sdk-reference/web-sdk/reference Establishes a connection to the Motion Server, enabling runtime communication and playback control. ```APIDOC ## AvatarController.start ### Description Connect to Motion Server. ### Method `start(): Promise` ``` -------------------------------- ### AvatarPlayer Microphone Control Source: https://docs.spatius.ai/livekit-agents/client Methods to start and stop publishing audio from the user's microphone. Starting the microphone automatically requests necessary permissions. ```typescript // Start microphone (requests permission automatically) await player.startPublishing() // Stop microphone await player.stopPublishing() ``` -------------------------------- ### AudioFormat.init(sampleRate:) Source: https://docs.spatius.ai/sdk-reference/ios-sdk/api-reference Creates a new audio format with the specified sample rate. ```APIDOC ## AudioFormat.init(sampleRate:) ### Description Creates a new audio format with the specified sample rate. ### Parameters #### Path Parameters * `sampleRate` (Int) - Required - The audio sample rate in Hz ``` -------------------------------- ### Initialize AvatarPlayer with LiveKitProvider Source: https://docs.spatius.ai/sdk-reference/web-sdk/rtc-adapter Instantiate the LiveKitProvider and then use it to create an AvatarPlayer instance. Ensure AvatarPlayer is imported from '@spatius/avatarkit-rtc'. ```typescript import { AvatarPlayer, LiveKitProvider } from '@spatius/avatarkit-rtc' const provider = new LiveKitProvider() const player = new AvatarPlayer(provider, avatarView, options) ``` -------------------------------- ### SpatiusAvatarError Source: https://docs.spatius.ai/sdk-reference/web-sdk/avatarkit-ui An error overlay shown when avatar setup or streaming fails. Allows for custom error UI. ```APIDOC ## SpatiusAvatarError ### Description Error overlay shown when avatar setup or streaming fails. ### Props - **children** (ReactNode) - Optional - Optional custom error UI. When omitted, the default card displays `error.message`. - **className** (string) - Optional - Additional classes for the overlay container. - **...props** (HTMLAttributes) - Optional - Standard div attributes forwarded to the overlay container. ``` -------------------------------- ### Clone and Configure Android Project Source: https://docs.spatius.ai/quickstarts/android-sdk Clone the Spatius Avatar Demo repository and configure the Android client's local properties file with your Spatius and OpenAI API keys. ```bash git clone https://github.com/spatius-ai/spatius-avatar-demo.git cd spatius-avatar-demo/direct-mode/clients/android cp local.properties.example local.properties ``` ```properties SPATIUS_APP_ID= SPATIUS_AVATAR_ID= OPENAI_API_KEY= OPENAI_USE_PROXY=false ``` -------------------------------- ### Create Configuration Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference Creates a new Configuration object with specified parameters. Defaults are provided for audio format, driving service mode, and log level. ```kotlin data class Configuration( val region: String, val audioFormat: AudioFormat = AudioFormat(16000), val drivingServiceMode: DrivingServiceMode = DrivingServiceMode.SDK, val logLevel: LogLevel = LogLevel.INFO ) ``` -------------------------------- ### AvatarController Volume Property Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference Sets or gets the playback volume, ranging from 0.0 (silent) to 1.0 (maximum volume). ```kotlin var volume: Float ``` -------------------------------- ### AvatarManager Get Cache Size Method Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference Retrieves the cache size in bytes for a specific avatar ID. This is an asynchronous operation. ```kotlin suspend fun getCacheSize(id: String): Long ``` -------------------------------- ### Configuration(region, audioFormat, drivingServiceMode, logLevel) Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference Creates a new configuration with the specified parameters. This allows customization of the SDK's behavior. ```APIDOC ## Configuration(region, audioFormat, drivingServiceMode, logLevel) ### Description Creates a new configuration with the specified parameters. ### Method data class Configuration( val region: String, val audioFormat: AudioFormat = AudioFormat(16000), val drivingServiceMode: DrivingServiceMode = DrivingServiceMode.SDK, val logLevel: LogLevel = LogLevel.INFO ) ### Parameters #### Path Parameters * `region` (String) - Required - The region to connect to (currently only `"us-west"`) * `audioFormat` (AudioFormat) - Optional - The audio format configuration * `drivingServiceMode` (DrivingServiceMode) - Optional - The driving service mode * `logLevel` (LogLevel) - Optional - The log level ### Response None ``` -------------------------------- ### Create Player with LiveKit Provider Source: https://docs.spatius.ai/livekit-agents/overview Sets up an AvatarPlayer using a LiveKitProvider for real-time communication and an AvatarView for rendering. Configure the logLevel as needed. ```typescript import { AvatarPlayer, LiveKitProvider } from '@spatius/avatarkit-rtc' const provider = new LiveKitProvider() const player = new AvatarPlayer(provider, avatarView, { logLevel: 'warning', }) ``` -------------------------------- ### Configure Golang SDK Authentication (Query) Source: https://docs.spatius.ai/sdk-reference/go-sdk/go-sdk Configure the SDK to use web-style query parameter authentication for WebSockets. ```go session := spatius.NewAvatarSession( spatius.WithUseQueryAuth(true), ) ``` -------------------------------- ### AvatarController Methods Source: https://docs.spatius.ai/sdk-reference/ios-sdk/api-reference Provides methods for controlling avatar behavior, including starting and closing connections, managing rendering, and sending/yielding data. ```APIDOC ## start() ### Description Starts the avatar driving service connection. ### Method func start() ### Endpoint N/A (SDK Method) ### Parameters None ### Response None ``` ```APIDOC ## close() ### Description Closes the avatar driving service. ### Method func close() ### Endpoint N/A (SDK Method) ### Parameters None ### Response None ``` ```APIDOC ## pauseRendering() ### Description Pauses avatar rendering. ### Method func pauseRendering() ### Endpoint N/A (SDK Method) ### Parameters None ### Response None ``` ```APIDOC ## resumeRendering() ### Description Resumes avatar rendering. ### Method func resumeRendering() ### Endpoint N/A (SDK Method) ### Parameters None ### Response None ``` ```APIDOC ## interrupt() ### Description Stops playback and terminates the current conversation. ### Method func interrupt() ### Endpoint N/A (SDK Method) ### Parameters None ### Response None ``` ```APIDOC ## send(_:end:) ### Description Sends audio to the avatar driving service. ### Method func send(_ data: Data, end: Bool) -> String ### Endpoint N/A (SDK Method) ### Parameters #### Path Parameters * **data** (Data) - Required - The audio data to send * **end** (Bool) - Required - Whether this is the end of the audio stream ### Response #### Success Response * **String** - A conversation ID string. ``` ```APIDOC ## yield(_:end:audioFormat:) ### Description Yields audio from the host. ### Method func yield(_ data: Data, end: Bool, audioFormat: AudioFormat?) -> String ### Endpoint N/A (SDK Method) ### Parameters #### Path Parameters * **data** (Data) - Required - The audio data * **end** (Bool) - Required - Whether this is the end of the audio stream * **audioFormat** (AudioFormat?) - Optional - The audio format (optional) ### Response #### Success Response * **String** - A conversation ID string. ``` ```APIDOC ## yield(_:conversationID:) ### Description Yields animations from the host. ### Method func yield(_ animations: [Data], conversationID: String) ### Endpoint N/A (SDK Method) ### Parameters #### Path Parameters * **animations** (Array) - Required - Array of encoded motion data frames * **conversationID** (String) - Required - The conversation identifier ### Response None ``` -------------------------------- ### AvatarSession Class Source: https://docs.spatius.ai/sdk-reference/python-sdk/python-sdk The main class responsible for managing avatar sessions, providing methods for initialization, starting, sending data, and closing the session. ```APIDOC ### `AvatarSession` Main class for managing avatar sessions. #### Methods * `async init()` - initialize the session and obtain a token * `async start() -> str` - start the WebSocket connection and return a `connection_id` * `async send_audio(audio: bytes, end: bool = False) -> str` - send audio data and return a request ID * `async interrupt() -> str` - interrupt current audio processing in egress mode * `async close()` - close the session and clean up resources * `config -> SessionConfig` - current session configuration ``` -------------------------------- ### Web SDK Configuration Interface Source: https://docs.spatius.ai/sdk-reference/web-sdk/reference Defines the configuration options for initializing the Web SDK. Includes settings for region, driving service mode, logging level, and audio format. ```typescript interface Configuration { region?: string // Defaults to 'us-west' (currently the only region) drivingServiceMode?: DrivingServiceMode // Default: DrivingServiceMode.sdk logLevel?: LogLevel // Default: LogLevel.off audioFormat?: AudioFormat // Default: { channelCount: 1, sampleRate: 16000 } } ``` -------------------------------- ### AvatarManager Get Total Cache Size Source: https://docs.spatius.ai/sdk-reference/ios-sdk/api-reference Retrieves the total cache size in bytes for all cached avatars. This operation may throw an error. ```swift func getAllCacheSize() throws -> Int ``` -------------------------------- ### Composing Next.js Config Wrappers Source: https://docs.spatius.ai/direct-mode/web Combine the `withAvatarkit` wrapper with other Next.js configuration wrappers by nesting them. This example shows how to compose `withAvatarkit` with a hypothetical `withOtherPlugin`. ```javascript import { withAvatarkit } from '@spatius/avatarkit/next' import withOtherPlugin from 'other-plugin' export default withAvatarkit(withOtherPlugin({ // ...your config })) ``` -------------------------------- ### AvatarSDK Initialize Method Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference Initializes the AvatarKit SDK. Requires the application context, your application ID, and a Configuration object. ```kotlin fun initialize( context: Context, appId: String, configuration: Configuration ) ``` -------------------------------- ### Initialize AvatarView Source: https://docs.spatius.ai/sdk-reference/android-sdk/api-reference Creates a new avatar view instance. Requires a Context to be provided. ```kotlin constructor(context: Context) ``` -------------------------------- ### Manage Voice Interaction Source: https://docs.spatius.ai/livekit-agents/overview Controls the microphone publishing stream for voice interaction within the avatar player. Includes starting, stopping publishing, and disconnecting the player. ```typescript // Start microphone publishing await player.startPublishing() // Stop microphone await player.stopPublishing() // Disconnect when done await player.disconnect() ``` -------------------------------- ### AvatarSDK.initialize Source: https://docs.spatius.ai/sdk-reference/web-sdk/reference Initializes the SDK with the provided App ID and configuration. This method must be called before any other SDK operations. ```APIDOC ## AvatarSDK.initialize(appId, configuration) ### Description Initialize the SDK. Must be called before any other operation. ### Method `initialize` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters * **appId** (string) - Required - Your Spatius application ID. * **configuration** (object) - Optional - SDK configuration object. * **region** (string) - Optional - The region for the SDK to connect to (e.g., 'us-west'). * **drivingServiceMode** (DrivingServiceMode) - Optional - The driving service mode. Defaults to `DrivingServiceMode.sdk`. * **logLevel** (LogLevel) - Optional - The logging level. Defaults to `LogLevel.off`. * **audioFormat** (object) - Optional - Audio format configuration. * **channelCount** (number) - The number of audio channels. * **sampleRate** (number) - The audio sample rate. ### Request Example ```typescript await AvatarSDK.initialize('your-app-id', { region: 'us-west', drivingServiceMode: DrivingServiceMode.sdk, logLevel: LogLevel.warning, audioFormat: { channelCount: 1, sampleRate: 16000, }, }) ``` ### Response #### Success Response None #### Response Example None ``` -------------------------------- ### Direct Mode Audio Input Source: https://docs.spatius.ai/sdk-reference/flutter-sdk/api-reference For Direct Mode, start the connection and send audio data in PCM chunks. The `end` parameter indicates if this is the last chunk. ```dart await controller.start(); controller.send(audioBytes, end: isLastChunk); ``` -------------------------------- ### Configure Avatar Session with Callbacks and Format Source: https://docs.spatius.ai/sdk-reference/python-sdk/python-sdk Configure a new avatar session with various parameters including authentication, expiration, region, audio sample rate, audio format, and event callbacks. ```python from datetime import datetime, timedelta, timezone from spatius import AudioFormat, new_avatar_session session = new_avatar_session( avatar_id="avatar-123", api_key="your-api-key", app_id="your-app-id", use_query_auth=False, expire_at=datetime.now(timezone.utc) + timedelta(minutes=5), region="us-west", sample_rate=16000, audio_format=AudioFormat.PCM_S16LE, transport_frames=on_frame_received, on_error=on_error, on_close=on_close, ) ``` -------------------------------- ### Configure Golang SDK Authentication (Header) Source: https://docs.spatius.ai/sdk-reference/go-sdk/go-sdk Configure the SDK to use default header-based authentication for WebSockets. ```go session := spatius.NewAvatarSession( spatius.WithUseQueryAuth(false), ) ``` -------------------------------- ### Configure Golang SDK Audio Format (PCM) Source: https://docs.spatius.ai/sdk-reference/go-sdk/go-sdk Configure the SDK to use PCM 16-bit little-endian audio format with a specified sample rate. ```go session := spatius.NewAvatarSession( spatius.WithSampleRate(24000), spatius.WithAudioFormat(spatius.AudioFormatPCMS16LE), ) ``` -------------------------------- ### AvatarManager Get Specific Avatar Cache Size Source: https://docs.spatius.ai/sdk-reference/ios-sdk/api-reference Retrieves the cache size in bytes for a specific avatar identified by its ID. This operation may throw an error. ```swift func getCacheSize(id: String) throws -> Int ``` -------------------------------- ### Obtaining MediaStreamTracks for Custom Audio Source: https://docs.spatius.ai/livekit-agents/client Illustrates how to obtain MediaStreamTracks from various audio sources for custom audio publishing with AvatarPlayer. ```typescript | `