Decart

https://docs.platform.decart.ai
Admin
Decart provides state-of-the-art AI models for transforming videos and images in real-time and via...
Tokens:150,720
Snippets:1,448
Trust Score:9.4
Update:2 months ago
Show doc for...
Tokens:
Raw
### Clone and Setup Project

Source: https://docs.platform.decart.ai/examples/real-time-mobile-app

Commands to clone the repository and install initial dependencies.

```bash
git clone https://github.com/DecartAI/decart-example-expo-realtime.git
cd decart-example-expo-realtime
```

```bash
npm install
```

--------------------------------

### Implement Complete Video Transformation Setup

Source: https://docs.platform.decart.ai/models/realtime/video-restyling

A full implementation example including camera stream acquisition, client connection, event handling, and cleanup.

```tsx
import { createDecartClient, models, type DecartSDKError } from "@decartai/sdk";

async function setupVideoTransform() {
  try {
    const model = models.realtime("lucy-restyle-2");
    
    // Get camera with optimal settings
    const stream = await navigator.mediaDevices.getUserMedia({
      audio: true,
      video: {
        frameRate: model.fps,
        width: model.width,
        height: model.height,
      }
    });

    const client = createDecartClient({
      apiKey: process.env.DECART_API_KEY
    });

    const realtimeClient = await client.realtime.connect(stream, {
      model,
      onRemoteStream: (transformedStream) => {
        const videoElement = document.getElementById("output-video");
        videoElement.srcObject = transformedStream;
      }
    });

    // Set up event handlers
    realtimeClient.on("connectionChange", (state) => {
      updateUIStatus(state);
    });

    realtimeClient.on("error", (error) => {
      console.error("Error:", error);
      showErrorMessage(error.message);
    });

    // Style controls
    document.getElementById("style-input").addEventListener("change", (e) => {
      realtimeClient.setPrompt(e.target.value);
    });

    // Cleanup on page unload
    window.addEventListener("beforeunload", () => {
      realtimeClient.disconnect();
    });

    return realtimeClient;
  } catch (error) {
    console.error("Setup failed:", error);
    throw error;
  }
}
```

--------------------------------

### Install Realtime SDK

Source: https://docs.platform.decart.ai/sdks/python-realtime

Install the required package with the realtime extra to enable the decart.realtime module.

```bash
pip install decart[realtime]
```

--------------------------------

### Install the Decart SDK

Source: https://docs.platform.decart.ai/getting-started/overview

Install the official Node.js SDK via npm.

```bash
npm install @decartai/sdk
```

--------------------------------

### Install LiveKit Packages

Source: https://docs.platform.decart.ai/examples/real-time-mobile-app

Install the required LiveKit and WebRTC dependencies.

```bash
npm install @decartai/sdk livekit-client@2.20.1 \
  @livekit/react-native@2.11.1 \
  @livekit/react-native-webrtc@144.1.1
```

--------------------------------

### Install Decart SDK

Source: https://docs.platform.decart.ai/sdks/python

Install the package using pip.

```bash
pip install decart
```

--------------------------------

### Initialize FileInput Examples

Source: https://docs.platform.decart.ai/sdks/javascript-process

Demonstrates various ways to instantiate inputs for processing tasks.

```typescript
// Browser file input
const file = document.querySelector('input[type="file"]').files[0];

// Blob
const blob = new Blob([arrayBuffer], { type: 'image/jpeg' });

// URL string
const url = "https://example.com/image.jpg";

// URL object
const urlObj = new URL("https://example.com/video.mp4");

// ReadableStream (advanced)
const stream = response.body;
```

--------------------------------

### Install and Configure Expo Plugins

Source: https://docs.platform.decart.ai/examples/real-time-mobile-app

Install necessary Expo plugins and register them in app.json.

```bash
npm install @livekit/react-native-expo-plugin@1.0.2 \
  @config-plugins/react-native-webrtc
```

```json
{
  "expo": {
    "plugins": [
      "@livekit/react-native-expo-plugin",
      "@config-plugins/react-native-webrtc"
    ]
  }
}
```

--------------------------------

### Install Decart adapter and TanStack AI

Source: https://docs.platform.decart.ai/ai-frameworks/tanstack-ai

Install the required packages using your preferred package manager.

```bash
npm install @decartai/tanstack-ai-adapter @tanstack/ai
```

```bash
pnpm add @decartai/tanstack-ai-adapter @tanstack/ai
```

```bash
yarn add @decartai/tanstack-ai-adapter @tanstack/ai
```

--------------------------------

### Complete Video Editing Workflow in Python

Source: https://docs.platform.decart.ai/sdks/python-queue

A full implementation example showing client initialization, job submission, polling, and file saving.

```python
import asyncio
from decart import DecartClient, models

async def edit_video():
    async with DecartClient(api_key="your-api-key") as client:
        print("Submitting video editing job...")

        with open("input.mp4", "rb") as video_file:
            result = await client.queue.submit_and_poll({
                "model": models.video("lucy-2.1"),
                "data": video_file,
                "prompt": "Transform into anime style",
                "on_status_change": lambda job: print(f"  Status: {job.status}"),
            })

        if result.status == "completed":
            with open("output.mp4", "wb") as f:
                f.write(result.data)
            print("Video saved to output.mp4")
        else:
            print(f"Editing failed: {result.error}")

asyncio.run(edit_video())
```

--------------------------------

### Install Decart SDK

Source: https://docs.platform.decart.ai/models/realtime/lucy-2.5

Commands to install the required SDK package for your target platform.

```bash
npm install @decartai/sdk
```

```bash
pip install decart
```

--------------------------------

### Quick Start with Realtime API

Source: https://docs.platform.decart.ai/sdks/swift-realtime

Initializes the Decart client, configures a realtime manager, and establishes a transformed video stream using LiveKit.

```swift
import DecartSDK
import LiveKit

let model = Models.realtime(.lucy-restyle-2)

// Create client
let config = DecartConfiguration(apiKey: "your-api-key-here")
let client = DecartClient(decartConfiguration: config)

// Create realtime manager
let manager = try client.createRealtimeManager(
    options: RealtimeConfiguration(
        model: model,
        initialPrompt: DecartPrompt(text: "Anime style", enrich: true)
    )
)

// Build a LiveKit camera track sized to the model
let captureOptions = CameraCaptureOptions(
    position: .front,
    dimensions: Dimensions(width: Int32(model.height), height: Int32(model.width)),
    fps: model.fps
)
let mirror = MirroringVideoProcessor(mode: .auto)
let videoTrack = LocalVideoTrack.createCameraTrack(name: "video0", options: captureOptions, processor: mirror)

// Connect and get transformed stream
let localStream = RealtimeMediaStream(videoTrack: videoTrack, id: .localStream)
let remoteStream = try await manager.connect(localStream: localStream)

// Change style on the fly (suspends until the server acks)
try await manager.setPrompt(DecartPrompt(text: "Cyberpunk city", enrich: true))

// Disconnect when done
try? await videoTrack.stop()
await manager.disconnect()
```

--------------------------------

### Install Decart AI SDK Provider

Source: https://docs.platform.decart.ai/ai-frameworks/ai-sdk

Install the provider package using your preferred package manager.

```bash
npm install @decartai/ai-sdk-provider
```

```bash
pnpm add @decartai/ai-sdk-provider
```

```bash
yarn add @decartai/ai-sdk-provider
```

--------------------------------

### Wrap connection setup in try/catch

Source: https://docs.platform.decart.ai/models/realtime/streaming-best-practices

Use try/catch blocks during initialization to handle media device access failures and connection errors gracefully.

```typescript
async function startRealtime() {
  try {
    const stream = await navigator.mediaDevices.getUserMedia({
      video: { frameRate: model.fps, width: model.width, height: model.height },
    });
    const realtimeClient = await client.realtime.connect(stream, {
      model,
      onRemoteStream: (s) => { videoElement.srcObject = s; },
    });
    return realtimeClient;
  } catch (error) {
    if (error instanceof DOMException && error.name === "NotAllowedError") {
      showCameraPermissionError();
    } else {
      showConnectionError(error);
    }
  }
}
```

--------------------------------

### Complete image editing workflow

Source: https://docs.platform.decart.ai/sdks/swift-process

A full example demonstrating configuration, image loading, input creation, and processing.

```swift
import DecartSDK
import UIKit

func editImage() async throws {
    let config = DecartConfiguration(apiKey: "your-api-key-here")
    let client = DecartClient(decartConfiguration: config)

    // Load image
    guard let image = UIImage(named: "input"),
          let imageData = image.jpegData(compressionQuality: 0.9) else {
        return
    }

    // Create input
    let input = try ImageToImageInput(
        prompt: "Transform to watercolor style",
        data: .image(data: imageData),
        resolution: .res720p,
        enhancePrompt: true
    )

    // Process
    let processClient = try client.createProcessClient(
        model: .lucy_image_2,
        input: input
    )
    let resultData = try await processClient.process()

    // Use the result
    if let resultImage = UIImage(data: resultData) {
        // Display or save the transformed image
        print("Image transformed successfully")
    }
}
```

--------------------------------

### Initialize Camera Capture

Source: https://docs.platform.decart.ai/sdks/swift-realtime

Create a LocalVideoTrack using camera options and a mirroring processor. Capture starts immediately upon creation.

```swift
import DecartSDK
import LiveKit

let model = Models.realtime(.lucy-restyle-2)

let captureOptions = CameraCaptureOptions(
    position: .front,                                                                  // .front or .back
    dimensions: Dimensions(width: Int32(model.height), height: Int32(model.width)),    // portrait — swap for landscape
    fps: model.fps
)

let mirror = MirroringVideoProcessor(mode: .auto)
let videoTrack = LocalVideoTrack.createCameraTrack(
    name: "video0",
    options: captureOptions,
    processor: mirror
)
```

--------------------------------

### Implement Realtime UI with Jetpack Compose

Source: https://docs.platform.decart.ai/sdks/android-realtime

A full layout example featuring a video view, prompt input, and connection controls.

```kotlin
Column(
    modifier = Modifier.fillMaxSize().padding(16.dp),
    verticalArrangement = Arrangement.spacedBy(12.dp),
) {
    // Remote video
    Box(modifier = Modifier.weight(1f).fillMaxWidth()) {
        LiveKitVideoView(
            track = remoteStream?.videoTrack,
            room = remoteStream?.room ?: localStream?.room,
            modifier = Modifier.fillMaxSize(),
        )
    }

    Text("Status: $connectionState")

    Row(modifier = Modifier.fillMaxWidth()) {
        OutlinedTextField(
            value = prompt,
            onValueChange = { prompt = it },
            label = { Text("Style prompt") },
            modifier = Modifier.weight(1f),
        )
        Spacer(modifier = Modifier.width(8.dp))
        Button(
            onClick = {
                coroutineScope.launch {
                    try {
                        client?.realtime?.setPrompt(prompt, enhance = true)
                    } catch (e: Exception) {
                        Log.e("Decart", "setPrompt failed", e)
                    }
                }
            },
            enabled = isConnected,
        ) { Text("Send") }
    }

    Button(
        onClick = {
            if (isConnected) {
                client?.realtime?.disconnect()
                client?.release()
                client = null
            } else {
                permissionLauncher.launch(Manifest.permission.CAMERA)
            }
        },
        modifier = Modifier.fillMaxWidth(),
    ) { Text(if (isConnected) "Disconnect" else "Connect") }
}
```

--------------------------------

### Initialize and Use Queue API

Source: https://docs.platform.decart.ai/sdks/javascript

Example of initializing the client and submitting a video processing job using the Queue API.

```typescript
import { createDecartClient, models } from "@decartai/sdk";
import { writeFileSync } from "fs";

const client = createDecartClient({
  apiKey: process.env.DECART_API_KEY,
});

// Edit a video using the Queue API
const result = await client.queue.submitAndPoll({
  model: models.video("lucy-2.1"),
  data: videoFile,
  prompt: "Transform into anime style",
  resolution: "720p",
  onStatusChange: (job) => {
    console.log(`Status: ${job.status}`);
  },
});

if (result.status === "completed") {
  const buffer = Buffer.from(await result.data.arrayBuffer());
  writeFileSync("output.mp4", buffer);
  console.log("Video saved!");
}
```

--------------------------------

### Implementing a Complete Realtime Video Application

Source: https://docs.platform.decart.ai/sdks/javascript-realtime

A full implementation example demonstrating camera stream acquisition, client connection, event handling, and dynamic prompt updates.

```typescript
import { createDecartClient, models, type DecartSDKError } from "@decartai/sdk";

async function setupRealtimeVideo() {
  try {
    // Get camera stream with optimal settings
    const model = models.realtime("lucy-restyle-2");
    const stream = await navigator.mediaDevices.getUserMedia({
      audio: true,
      video: {
        frameRate: model.fps,
        width: model.width,
        height: model.height,
      },
    });

    // Display input video
    const inputVideo = document.getElementById("input-video") as HTMLVideoElement;
    inputVideo.srcObject = stream;

    // Create client
    const client = createDecartClient({
      apiKey: process.env.DECART_API_KEY,
    });

    // Connect to Realtime API
    const realtimeClient = await client.realtime.connect(stream, {
      model,
      onRemoteStream: (transformedStream) => {
        const outputVideo = document.getElementById("output-video") as HTMLVideoElement;
        outputVideo.srcObject = transformedStream;
      },
      initialState: {
        prompt: {
          text: "Studio Ghibli animation style",
          enhance: true,
        },
      },
    });

    // Handle connection state changes
    realtimeClient.on("connectionChange", (state) => {
      const statusElement = document.getElementById("status");
      statusElement.textContent = `Status: ${state}`;
      statusElement.className = `status-${state}`;
    });

    // Handle errors
    realtimeClient.on("error", (error: DecartSDKError) => {
      console.error("Realtime error:", error);
      const errorElement = document.getElementById("error");
      errorElement.textContent = error.message;
      errorElement.style.display = "block";
    });

    // Allow user to change styles
    const styleInput = document.getElementById("style-input") as HTMLInputElement;
    styleInput.addEventListener("change", (e) => {
      const prompt = (e.target as HTMLInputElement).value;
      realtimeClient.setPrompt(prompt, { enhance: true });
    });

    // Cleanup on page unload
    window.addEventListener("beforeunload", () => {
      realtimeClient.disconnect();
      stream.getTracks().forEach(track => track.stop());
    });

    return realtimeClient;
  } catch (error) {
    console.error("Failed to setup realtime video:", error);
    throw error;
  }
}

// Initialize
setupRealtimeVideo();
```

--------------------------------

### Implement Lucy-2.5 Realtime Application

Source: https://docs.platform.decart.ai/models/realtime/lucy-2.5

Full application examples for managing video streams, connection lifecycle, and character substitution using the Decart SDK.

```typescript
import { createDecartClient, models, type DecartSDKError } from "@decartai/sdk";

async function setupLucy25() {
  const model = models.realtime("lucy-2.5");

  const stream = await navigator.mediaDevices.getUserMedia({
    video: {
      frameRate: model.fps,
      width: model.width,
      height: model.height,
    },
  });

  document.getElementById("input-video").srcObject = stream;

  const client = createDecartClient({
    apiKey: process.env.DECART_API_KEY,
  });

  const realtimeClient = await client.realtime.connect(stream, {
    model,
    onRemoteStream: (transformedStream) => {
      document.getElementById("output-video").srcObject = transformedStream;
    },
  });

  realtimeClient.on("connectionChange", (state) => {
    document.getElementById("status").textContent = state;
  });

  realtimeClient.on("generationTick", ({ seconds }) => {
    document.getElementById("usage").textContent = `${seconds}s`;
  });

  realtimeClient.on("error", (error: DecartSDKError) => {
    console.error("Lucy 2.5 error:", error.code, error.message);
  });

  document.getElementById("character-input").addEventListener("change", async (e) => {
    const file = (e.target as HTMLInputElement).files[0];
    if (file) {
      await realtimeClient.set({
        prompt: "Substitute the character in the video with the person in the reference image.",
        image: file,
        enhance: true,
      });
    }
  });

  window.addEventListener("beforeunload", () => {
    realtimeClient.disconnect();
    stream.getTracks().forEach((track) => track.stop());
  });

  return realtimeClient;
}

setupLucy25();
```

```python
import asyncio
from decart import DecartClient, models

async def main():
    async with DecartClient(api_key="your-api-key-here") as client:
        model = models.realtime("lucy-2.5")

        realtime = await client.realtime.connect(
            stream=media_stream,
            model=model,
            on_remote_stream=lambda s: display(s),
        )

        @realtime.on("connection_change")
        def on_state(state):
            print(f"Connection: {state}")

        @realtime.on("generation_tick")
        def on_tick(seconds):
            print(f"Usage: {seconds}s")

        @realtime.on("error")
        def on_error(error):
            print(f"Lucy 2.5 error: {error.code} {error.message}")

        with open("character.jpg", "rb") as f:
            await realtime.set(
                prompt="Substitute the character in the video with the person in the reference image.",
                image=f,
                enhance=True,
            )

asyncio.run(main())
```

```kotlin
import ai.decart.sdk.*
import ai.decart.sdk.realtime.*
import org.webrtc.*

val model = RealtimeModels.LUCY_2_5
val client = DecartClient(context, DecartClientConfig(apiKey = "your-api-key"))
client.realtime.initialize(eglBase)

val videoSource = client.realtime.createVideoSource(false)!!
val videoTrack = client.realtime.createVideoTrack("camera", videoSource)!!
val enumerator = Camera2Enumerator(context)
val capturer = enumerator.createCapturer(
    enumerator.deviceNames.first { enumerator.isFrontFacing(it) }, null
)
capturer.initialize(
    SurfaceTextureHelper.create("CaptureThread", client.realtime.getEglBaseContext()),
    context, videoSource.capturerObserver
)
capturer.startCapture(model.width, model.height, model.fps)

client.realtime.connect(
    localVideoTrack = videoTrack,
    options = ConnectOptions(
        model = model,
        onRemoteVideoTrack = { track -> remoteRenderer.addSink(track) }
    )
)

lifecycleScope.launch {
    client.realtime.connectionState.collect { state -> statusText.text = state.name }
}
lifecycleScope.launch {
    client.realtime.generationTicks.collect { tick -> usageText.text = "${tick.seconds}s" }
}
lifecycleScope.launch {
    client.realtime.errors.collect { error ->
        Log.e("Lucy25", "Error: ${error.code} ${error.message}") }
}

fun onCharacterSelected(imageBytes: ByteArray) {
    val base64 = Base64.encodeToString(imageBytes, Base64.NO_WRAP)
    client.realtime.setImage(
        imageBase64 = base64,
        prompt = "Substitute the character in the video with the person in the reference image.",
        enhance = true
    )
}

capturer.stopCapture()
client.realtime.disconnect()
client.release()
eglBase.release()
```

--------------------------------

### Implement Full Video Editing Workflow

Source: https://docs.platform.decart.ai/models/realtime/video-editing

A complete setup including media stream acquisition, client initialization, event handling, and prompt-based editing controls.

```tsx
import { createDecartClient, models, type DecartSDKError } from "@decartai/sdk";

async function setupVideoEditing() {
  try {
    const model = models.realtime("lucy-2.1");
    
    // Get camera with optimal settings
    const stream = await navigator.mediaDevices.getUserMedia({
      audio: true,
      video: {
        frameRate: model.fps,
        width: model.width,
        height: model.height,
      }
    });

    const client = createDecartClient({
      apiKey: process.env.DECART_API_KEY
    });

    const realtimeClient = await client.realtime.connect(stream, {
      model,
      onRemoteStream: (editedStream) => {
        const videoElement = document.getElementById("output-video");
        videoElement.srcObject = editedStream;
      }
    });

    // Set up event handlers
    realtimeClient.on("connectionChange", (state) => {
      updateUIStatus(state);
    });

    realtimeClient.on("error", (error) => {
      console.error("Error:", error);
      showErrorMessage(error.message);
    });

    // Editing controls
    document.getElementById("add-dog").addEventListener("click", () => {
      realtimeClient.setPrompt("Add a small dog running around");
    });

    document.getElementById("change-hair").addEventListener("click", () => {
      realtimeClient.setPrompt("Change the person's hair to curly brown hair");
    });

    document.getElementById("add-sunglasses").addEventListener("click", () => {
      realtimeClient.setPrompt("Add stylish sunglasses to the person");
    });

    // Cleanup on page unload
    window.addEventListener("beforeunload", () => {
      realtimeClient.disconnect();
    });

    return realtimeClient;
  } catch (error) {
    console.error("Setup failed:", error);
    throw error;
  }
}
```

--------------------------------

### Implement WebSocket and WebRTC for Video Editing

Source: https://docs.platform.decart.ai/models/realtime/video-editing

This example demonstrates a complete HTML/JavaScript implementation for establishing a WebRTC connection to the Decart API and sending editing prompts.

```html
<!DOCTYPE html>
<html>
<head>
  <title>Video Editing - WebRTC</title>
</head>
<body>
  <!-- Local and edited video streams -->
  <video id="localVideo" autoplay muted playsinline width="45%"></video>
  <video id="remoteVideo" autoplay playsinline width="45%"></video>
  <br>
  
  <!-- Controls -->
  <button id="startBtn">Start</button>
  <input id="promptInput" placeholder="Enter editing instruction...">
  <button id="sendBtn">Apply Edit</button>
  
  <!-- Quick edit buttons -->
  <br><br>
  <button onclick="quickEdit('Add a small dog in the background')">Add Dog</button>
  <button onclick="quickEdit('Change the shirt color to blue')">Blue Shirt</button>
  <button onclick="quickEdit('Add sunglasses to the person')">Add Sunglasses</button>

  <script>
    // WebSocket connection with API key and model
    const ws = new WebSocket('wss://api3.decart.ai/v1/stream?api_key=YOUR_API_KEY&model=lucy-2.1');
    let peerConnection;

    // Handle server messages
    ws.onmessage = async (event) => {
      const message = JSON.parse(event.data);
      
      if (message.type === 'answer' && peerConnection) {
        await peerConnection.setRemoteDescription({
          type: 'answer', 
          sdp: message.sdp
        });
      }
    };

    // Start WebRTC connection
    async function startConnection() {
      if (ws.readyState !== WebSocket.OPEN) return;

      // Create peer connection
      peerConnection = new RTCPeerConnection({
        iceServers: [{ urls: 'stun:stun.l.google.com:19302' }]
      });

      // Send ICE candidates to server
      peerConnection.onicecandidate = (event) => {
        if (event.candidate) {
          ws.send(JSON.stringify({
            type: 'ice-candidate',
            candidate: event.candidate
          }));
        }
      };

      // Receive edited video stream
      peerConnection.ontrack = (event) => {
        document.getElementById('remoteVideo').srcObject = event.streams[0];
      };

      // Get user camera with optimal settings
      const localStream = await navigator.mediaDevices.getUserMedia({
        video: {
          width: { ideal: 1280 },
          height: { ideal: 704 },
          frameRate: { ideal: 25 }
        },
        audio: false
      });

      // Display local video
      document.getElementById('localVideo').srcObject = localStream;
      
      // Add tracks to peer connection
      localStream.getTracks().forEach(track => {
        peerConnection.addTrack(track, localStream);
      });

      // Create and send offer
      const offer = await peerConnection.createOffer();
      await peerConnection.setLocalDescription(offer);
      
      ws.send(JSON.stringify({
        type: 'offer',
        sdp: offer.sdp
      }));
    }

    // Send editing instruction
    function sendPrompt() {
      const promptInput = document.getElementById('promptInput');
      const prompt = promptInput.value.trim();
      
      if (prompt && ws.readyState === WebSocket.OPEN) {
        ws.send(JSON.stringify({
          type: 'prompt',
          prompt: prompt
        }));
      }
    }

    // Quick edit function
    function quickEdit(prompt) {
      if (ws.readyState === WebSocket.OPEN) {
        ws.send(JSON.stringify({
          type: 'prompt',
          prompt: prompt
        }));
      }
    }

    // Event listeners
    document.getElementById('startBtn').onclick = startConnection;
    document.getElementById('sendBtn').onclick = sendPrompt;
    document.getElementById('promptInput').onkeypress = (e) => {
      if (e.key === 'Enter') sendPrompt();
    };
  </script>
</body>
</html>
```

--------------------------------

### Initialize and Connect to Realtime Session

Source: https://docs.platform.decart.ai/sdks/android

Demonstrates initializing the Decart client, setting up WebRTC, connecting with video tracks, and managing session prompts.

```kotlin
import ai.decart.sdk.DecartClient
import ai.decart.sdk.DecartClientConfig
import ai.decart.sdk.realtime.ConnectOptions
import ai.decart.sdk.realtime.InitialPrompt
import ai.decart.sdk.RealtimeModels

val client = DecartClient(context, DecartClientConfig(apiKey = "your-api-key"))
val realtime = client.realtime

// Initialize WebRTC
realtime.initialize(eglBase)

// Connect with a camera track
realtime.connect(
    localVideoTrack = cameraTrack,
    localAudioTrack = null,
    options = ConnectOptions(
        model = RealtimeModels.LUCY_RESTYLE_2,
        onRemoteVideoTrack = { track ->
            // Display the transformed video
            remoteRenderer.addSink(track)
        },
        initialPrompt = InitialPrompt("a cyberpunk cityscape")
    )
)

// Change prompt during session
realtime.setPrompt("a sunny beach scene", enhance = true)

// Disconnect when done
realtime.disconnect()
client.release()
```

--------------------------------

### Initialize and use Virtual Try-On across platforms

Source: https://docs.platform.decart.ai/models/realtime/virtual-try-on

Examples for connecting to the Lucy VTON model and updating outfits in real-time using JavaScript, Python, Android, and Swift SDKs.

```tsx
import { createDecartClient, models } from "@decartai/sdk";

const model = models.realtime("lucy-vton-latest");

const stream = await navigator.mediaDevices.getUserMedia({
  audio: true,
  video: {
    frameRate: model.fps,
    width: model.width,
    height: model.height,
  },
});

const client = createDecartClient({ apiKey: "your-api-key-here" });

const realtimeClient = await client.realtime.connect(stream, {
  model,
  mirror: "auto",
  onRemoteStream: (editedStream) => {
    document.querySelector("#video-output").srcObject = editedStream;
  },
});

// Try on with a garment image and descriptive prompt
const garmentFile = document.querySelector('input[type="file"]').files[0];
await realtimeClient.set({
  prompt: "Substitute the current top with a navy blue hoodie with a white cross logo on the chest",
  image: garmentFile,
  enhance: false,
});

// Disconnect when done
realtimeClient.disconnect();
```

```python
from decart import models
from decart.realtime import RealtimeClient, RealtimeConnectOptions
from decart.types import ModelState

model = models.realtime("lucy-vton-latest")

realtime_client = await RealtimeClient.connect(
    api_key="your-api-key-here",
    stream=camera_stream,
    options=RealtimeConnectOptions(
        model=model,
        initial_state=ModelState(
            prompt="Substitute the current top with a red leather jacket with a zip front",
        ),
        on_remote_stream=lambda edited_stream: handle_stream(edited_stream),
    ),
)

# Change outfit on the fly
await realtime_client.set_prompt("Substitute the current top with a blue denim jacket with a relaxed fit")

# Disconnect when done
await realtime_client.disconnect()
```

```kotlin
import ai.decart.sdk.DecartClient
import ai.decart.sdk.DecartClientConfig
import ai.decart.sdk.RealtimeModels
import ai.decart.sdk.realtime.ConnectOptions
import ai.decart.sdk.realtime.InitialPrompt

val client = DecartClient(context, DecartClientConfig(apiKey = "your-api-key"))
val realtime = client.realtime

realtime.initialize(eglBase)

realtime.connect(
    localVideoTrack = cameraTrack,
    localAudioTrack = null,
    options = ConnectOptions(
        model = RealtimeModels.LUCY_2_1_VTON,
        onRemoteVideoTrack = { track ->
            remoteRenderer.addSink(track)
        },
        initialPrompt = InitialPrompt("Substitute the current top with a red leather jacket with a zip front")
    )
)

// Change outfit on the fly
realtime.setPrompt("Substitute the current top with a blue denim jacket with a relaxed fit", enhance = true)
```

```swift
import DecartSDK

let config = DecartConfiguration(apiKey: "your-api-key-here")
let client = DecartClient(decartConfiguration: config)

let model = Models.realtime(.lucy2_1_vton)

let realtimeManager = try await client.realtime.connect(
    stream: cameraStream,
    model: model,
    initialState: .init(prompt: "Substitute the current top with a red leather jacket with a zip front")
) { editedStream in
    // Display the edited video
    videoView.stream = editedStream
}

// Change outfit on the fly
try await realtimeManager.setPrompt("Substitute the current top with a blue denim jacket with a relaxed fit")
```

--------------------------------

### Install decart-oasis SDK

Source: https://docs.platform.decart.ai/models/realtime/oasis-3

Install the required Python package for interacting with the Oasis 3 model.

```bash
pip install decart-oasis
```

--------------------------------

### Install Python Dependencies

Source: https://docs.platform.decart.ai/examples/comfyui

Install the necessary Python packages required for the Lucy Edit node.

```bash
cd lucy-edit-comfyui
pip install -r requirements.txt
```

--------------------------------

### Initialize Client and Process Video

Source: https://docs.platform.decart.ai/sdks/swift

Configure the Decart client and use the Queue API to submit and poll for video editing results.

```swift
import Foundation
import DecartSDK

let config = DecartConfiguration(apiKey: "your-api-key-here")
let client = DecartClient(decartConfiguration: config)

// Edit a video using the Queue API
let videoData = try Data(contentsOf: videoURL)
let input = try VideoEditInput(
    prompt: "Transform into anime style",
    data: .video(data: videoData)
)

let result = try await client.queue.submitAndPoll(
    model: .lucy_2,
    input: input
) { status in
    print("Status: \(status.status)")
}

switch result {
case .completed(_, let data):
    try data.write(to: outputURL)
    print("Video saved!")
case .failed(_, let error):
    print("Failed: \(error)")
}
```

--------------------------------

### A2VClient.initialize()

Source: https://docs.platform.decart.ai/models/realtime/oasis-3

Opens a session, authenticates, and negotiates the output format. Returns the streams the server advertises.

```APIDOC
## A2VClient.initialize()

### Description
Opens a session, authenticates, and negotiates the output format. Returns the streams the server advertises (e.g., `left_forward`, `front`, `right_forward`).

### Usage
```python
client = A2VClient()
streams = client.initialize()
```
```

--------------------------------

### Connect to Realtime Service in Swift

Source: https://docs.platform.decart.ai/sdks/swift-realtime

Initializes the DecartClient using an ephemeral key and sets up the local media stream for realtime interaction.

```swift
import DecartSDK

func connectToRealtime() async throws -> DecartRealtimeManager {
    // 1. Fetch ephemeral key from your backend
    let ephemeralKey = try await fetchEphemeralKey()

    // 2. Create client with ephemeral key
    let config = DecartConfiguration(apiKey: ephemeralKey)
    let client = DecartClient(decartConfiguration: config)

    // 3. Set up manager and camera, then connect
    let model = Models.realtime(.lucy-restyle-2)

    let manager = try client.createRealtimeManager(
        options: RealtimeConfiguration(
            model: model,
            initialPrompt: DecartPrompt(text: "Anime style", enrich: true)
        )
    )

    let captureOptions = CameraCaptureOptions(
        position: .front,
        dimensions: Dimensions(width: Int32(model.height), height: Int32(model.width)),
        fps: model.fps
    )
    let mirror = MirroringVideoProcessor(mode: .auto)
    let videoTrack = LocalVideoTrack.createCameraTrack(name: "video0", options: captureOptions, processor: mirror)

    let localStream = RealtimeMediaStream(videoTrack: videoTrack, id: .localStream)
    _ = try await manager.connect(localStream: localStream)

    return manager
}
```

--------------------------------

### Install the Decart skill

Source: https://docs.platform.decart.ai/getting-started/ai-coding-agents

Run this command in your project directory to auto-discover and install the Decart skill for your coding agents.

```bash
npx skills add https://docs.platform.decart.ai
```

--------------------------------

### Install Decart LangChain package

Source: https://docs.platform.decart.ai/ai-frameworks/langchain

Install the package and its required peer dependency using your preferred package manager.

```bash
npm install @decartai/langchain @langchain/core
```

```bash
pnpm add @decartai/langchain @langchain/core
```

```bash
yarn add @decartai/langchain @langchain/core
```

--------------------------------

### Implement Product Catalog Pattern

Source: https://docs.platform.decart.ai/examples/ecommerce-try-on

Demonstrates how to structure a product catalog and trigger the try-on process using pre-written prompts for consistent results.

```typescript
interface Product {
  name: string;
  image: string;
  prompt: string;
  price: number;
}

const products: Product[] = [
  {
    name: "Bomber Jacket",
    image: "/products/bomber.png",
    prompt: "Substitute the current top with a black bomber jacket with a blue logo on the chest and a zip front",
    price: 149,
  },
  {
    name: "Knit Beanie",
    image: "/products/beanie.png",
    prompt: "Add a navy blue knit beanie with a white cross logo on the fold to the person's head",
    price: 35,
  },
  {
    name: "Polo Shirt",
    image: "/products/polo.png",
    prompt: "Substitute the current top with a navy blue polo shirt with a white logo on the chest",
    price: 69,
  },
];

// When user clicks "Try it on"
async function tryOn(product: Product) {
  const garmentBlob = await fetch(product.image).then((r) => r.blob());
  await rtClient.set({
    prompt: product.prompt,
    image: garmentBlob,
    enhance: false, // use your pre-written prompt as-is
  });
}
```

--------------------------------

### Get Job OpenAPI Specification

Source: https://docs.platform.decart.ai/api-reference/get-job

Defines the GET endpoint for retrieving job details by ID, including validation error schemas.

```yaml
openapi: 3.1.0
info:
  title: Decart API
  version: 1.0.0
servers: []
security: []
paths:
  /v1/jobs/{job_id}:
    get:
      summary: Get Job
      description: >-
        Get the status and metadata of a job by its ID. Poll this endpoint to
        check if your job has completed.
      operationId: get_job_v1_jobs__job_id__get
      parameters:
        - name: job_id
          in: path
          required: true
          schema:
            type: string
            title: Job Id
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema: {}
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
components:
  schemas:
    HTTPValidationError:
      properties:
        detail:
          items:
            $ref: '#/components/schemas/ValidationError'
          type: array
          title: Detail
      type: object
      title: HTTPValidationError
    ValidationError:
      properties:
        loc:
          items:
            anyOf:
              - type: string
              - type: integer
          type: array
          title: Location
        msg:
          type: string
          title: Message
        type:
          type: string
          title: Error Type
        input:
          title: Input
        ctx:
          type: object
          title: Context
      type: object
      required:
        - loc
        - msg
        - type
      title: ValidationError
```

--------------------------------

### Install Decart skill to specific or global agents

Source: https://docs.platform.decart.ai/getting-started/ai-coding-agents

Use these commands to target specific agents or install the skill globally across all projects.

```bash
npx skills add https://docs.platform.decart.ai -a claude-code
npx skills add https://docs.platform.decart.ai -a cursor
```

```bash
npx skills add https://docs.platform.decart.ai -g
```

--------------------------------

### Get Realtime Quota OpenAPI Specification

Source: https://docs.platform.decart.ai/api-reference/get-realtime-quota

Defines the GET /v1/realtime/quota endpoint, including required authentication headers and the structure of the RealtimeQuotaResponse schema.

```yaml
openapi: 3.1.0
info:
  title: Decart API
  version: 1.0.0
servers: []
security: []
paths:
  /v1/realtime/quota:
    get:
      summary: Get Realtime Quota
      description: >-
        Current concurrent realtime session quota state for the authenticated
        account.


        For org-scoped keys the count is org-wide (shared across the org's
        keys/users).
      operationId: get_realtime_quota_v1_realtime_quota_get
      parameters:
        - name: x-api-key
          in: header
          required: true
          schema:
            type: string
            description: API key for authentication
            title: X-Api-Key
          description: API key for authentication
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/RealtimeQuotaResponse'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
components:
  schemas:
    RealtimeQuotaResponse:
      properties:
        limit:
          anyOf:
            - type: integer
            - type: 'null'
          title: Limit
        active:
          anyOf:
            - type: integer
            - type: 'null'
          title: Active
        remaining:
          anyOf:
            - type: integer
            - type: 'null'
          title: Remaining
      type: object
      title: RealtimeQuotaResponse
    HTTPValidationError:
      properties:
        detail:
          items:
            $ref: '#/components/schemas/ValidationError'
          type: array
          title: Detail
      type: object
      title: HTTPValidationError
    ValidationError:
      properties:
        loc:
          items:
            anyOf:
              - type: string
              - type: integer
          type: array
          title: Location
        msg:
          type: string
          title: Message
        type:
          type: string
          title: Error Type
        input:
          title: Input
        ctx:
          type: object
          title: Context
      type: object
      required:
        - loc
        - msg
        - type
      title: ValidationError
```

--------------------------------

### Get Job Content OpenAPI Specification

Source: https://docs.platform.decart.ai/api-reference/get-job-content

The OpenAPI definition for the GET /v1/jobs/{job_id}/content endpoint, including required authentication headers and error schemas.

```yaml
openapi: 3.1.0
info:
  title: Decart API
  version: 1.0.0
servers: []
security: []
paths:
  /v1/jobs/{job_id}/content:
    get:
      summary: Get Job Content
      description: |-
        Download the result of a completed job.

        Returns:
            Binary content (image/video) or error if not ready/not found
      operationId: get_job_content_v1_jobs__job_id__content_get
      parameters:
        - name: job_id
          in: path
          required: true
          schema:
            type: string
            title: Job Id
        - name: x-api-key
          in: header
          required: true
          schema:
            type: string
            description: API key for authentication
            title: X-Api-Key
          description: API key for authentication
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema: {}
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
components:
  schemas:
    HTTPValidationError:
      properties:
        detail:
          items:
            $ref: '#/components/schemas/ValidationError'
          type: array
          title: Detail
      type: object
      title: HTTPValidationError
    ValidationError:
      properties:
        loc:
          items:
            anyOf:
              - type: string
              - type: integer
          type: array
          title: Location
        msg:
          type: string
          title: Message
        type:
          type: string
          title: Error Type
        input:
          title: Input
        ctx:
          type: object
          title: Context
      type: object
      required:
        - loc
        - msg
        - type
      title: ValidationError
```

--------------------------------

### Connect with Caller-Owned Preview

Source: https://docs.platform.decart.ai/sdks/android-realtime

Create a local video stream before connecting to drive a preview UI, then pass it to the connect method.

```kotlin
val model = RealtimeModels.LUCY_2_1

// Caller-owned preview — sized from the model's capture resolution.
val localStream = realtime.createLocalVideoStream(
    model = model,
    facing = FacingMode.FRONT,
    mirror = MirrorMode.AUTO,
)

// Later, when the user hits Connect:
realtime.connect(
    options = ConnectOptions(
        model = model,
        publishCamera = true,
        onRemoteStream = { stream -> /* render stream.videoTrack */ },
    ),
    localStream = localStream,
)
```

--------------------------------

### generation_started

Source: https://docs.platform.decart.ai/integrations/signaling-proxy-ws

Notifies that the model has started producing frames.

```APIDOC
## generation_started

### Description
Media will start flowing over the WebRTC connection after this message.

### Payload
- **type** (string) - "generation_started"
```

--------------------------------

### Get connection state

Source: https://docs.platform.decart.ai/integrations/sdk-direct

Retrieves the current connection status string.

```typescript
const state = rt.getConnectionState();
// "connecting" | "connected" | "generating" | "disconnected" | "reconnecting"
```

--------------------------------

### GET /v1/jobs/{job_id}

Source: https://docs.platform.decart.ai/models/realtime/virtual-try-on

Poll the status of a submitted virtual try-on job.

```APIDOC
## GET /v1/jobs/{job_id}

### Description
Retrieve the current status of a specific job by its ID.

### Method
GET

### Endpoint
https://api.decart.ai/v1/jobs/{job_id}

### Parameters
#### Path Parameters
- **job_id** (string) - Required - The unique identifier of the job.
```

--------------------------------

### Video Restyling SDK Usage

Source: https://docs.platform.decart.ai/models/realtime/video-restyling

Demonstrates how to initialize the Decart client, connect a video stream, and update the restyling prompt dynamically.

```APIDOC
## SDK Method: client.realtime.connect

### Description
Establishes a WebRTC connection to the inference server to transform a video stream based on a provided model and prompt.

### Parameters
- **stream** (MediaStream) - Required - The input video stream from the user's camera.
- **options** (Object) - Required
  - **model** (Object) - Required - The model configuration object.
  - **onRemoteStream** (Function) - Required - Callback function triggered when the transformed video stream is received.

### Method: realtimeClient.setPrompt

### Description
Updates the transformation prompt on-the-fly during an active session.

### Parameters
- **prompt** (string) - Required - The new style description for the video transformation.

### Method: realtimeClient.disconnect

### Description
Terminates the active WebRTC connection.
```

--------------------------------

### GET /v1/jobs/{job_id}

Source: https://docs.platform.decart.ai/models/video/video-restyling

Retrieves the current status of a specific restyling job.

```APIDOC
## GET /v1/jobs/{job_id}

### Description
Polls the status of a previously submitted job using its unique job ID.

### Method
GET

### Endpoint
https://api.decart.ai/v1/jobs/{job_id}

### Parameters
#### Path Parameters
- **job_id** (string) - Required - The unique identifier of the job.
```

--------------------------------

### GET /v1/jobs/{job_id}

Source: https://docs.platform.decart.ai/models/video/video-editing

Retrieve the current status of a submitted video editing job.

```APIDOC
## GET /v1/jobs/{job_id}

### Description
Polls the status of a specific job to determine if it is still processing, completed, or failed.

### Method
GET

### Endpoint
https://api.decart.ai/v1/jobs/{job_id}

### Parameters
#### Path Parameters
- **job_id** (string) - Required - The unique identifier of the job.
```
Tokens:
Raw