### Install Dependencies and Build

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/README.md

Installs project dependencies and builds the client-js workspace. Run this before starting development.

```bash
yarn
yarn workspace @pipecat-ai/client-js build
```

--------------------------------

### Quickstart: Connect to Bot with Daily Transport

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/README.md

Initialize the Pipecat client with the Daily transport, configure callbacks for connection events, and start a session with your bot. Ensure your server-side endpoint is set up for authentication and secret management.

```typescript
import { PipecatClient, RTVIEvent, RTVIMessage } from "@pipecat/client-js";
import { DailyTransport } from "@pipecat/daily-transport";

const pcClient = new PipecatClient({
  transport: new DailyTransport(),
  enableMic: true,
  enableCam: false,
  callbacks: {
    onConnected: () => {
      console.log("[CALLBACK] User connected");
    },
    onDisconnected: () => {
      console.log("[CALLBACK] User disconnected");
    },
    onTransportStateChanged: (state: string) => {
      console.log("[CALLBACK] State change:", state);
    },
    onBotConnected: () => {
      console.log("[CALLBACK] Bot connected");
    },
    onBotDisconnected: () => {
      console.log("[CALLBACK] Bot disconnected");
    },
    onBotReady: () => {
      console.log("[CALLBACK] Bot ready to chat!");
    },
  },
});

try {
  await pcClient.startBotAndConnect({ endpoint: "https://your-server-side-url/connect" });
} catch (e) {
  console.error(e.message);
}

// Events
pcClient.on(RTVIEvent.TransportStateChanged, (state) => {
  console.log("[EVENT] Transport state change:", state);
});
pcClient.on(RTVIEvent.BotReady, () => {
  console.log("[EVENT] Bot is ready");
});
pcClient.on(RTVIEvent.Connected, () => {
  console.log("[EVENT] User connected");
});
pcClient.on(RTVIEvent.Disconnected, () => {
  console.log("[EVENT] User disconnected");
});
```

--------------------------------

### PipecatClientOptions Example

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/types.md

Example of initializing PipecatClientOptions with a transport and a callback.

```typescript
const options: PipecatClientOptions = {
  transport: new DailyTransport(),
  enableMic: true,
  callbacks: {
    onBotReady: () => console.log("Ready"),
  },
};
```

--------------------------------

### Install Pipecat Client Packages

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/client-react/README.md

Install the necessary Pipecat client packages for JavaScript and React.

```bash
npm install @pipecat-ai/client-js @pipecat-ai/client-react
```

--------------------------------

### Get and Set Bot Start Parameters

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/transport.md

Get or set the parameters used to start the bot. The setter automatically clones Request objects to avoid body reuse issues.

```typescript
get startBotParams(): APIRequest | undefined
set startBotParams(startBotParams: APIRequest)
```

--------------------------------

### Quick Start: Initialize and Connect Pipecat Client

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/client-js/README.md

Instantiate a PipecatClient, configure transports and callbacks, and start the bot connection. Ensure the endpoint URL is correctly set.

```typescript
import { RTVIEvent, RTVIMessage, PipecatClient } from "@pipecat-ai/client-js";
import { DailyTransport } from "@pipecat-ai/daily-transport";

const pcClient = new PipecatClient({
  transport: new DailyTransport(),
  enableMic: true,
  enableCam: false,
  callbacks: {
    onConnected: () => {
      console.log("[CALLBACK] User connected");
    },
    onDisconnected: () => {
      console.log("[CALLBACK] User disconnected");
    },
    onTransportStateChanged: (state: string) => {
      console.log("[CALLBACK] State change:", state);
    },
    onBotConnected: () => {
      console.log("[CALLBACK] Bot connected");
    },
    onBotDisconnected: () => {
      console.log("[CALLBACK] Bot disconnected");
    },
    onBotReady: () => {
      console.log("[CALLBACK] Bot ready to chat!");
    },
  },
});

try {
  await pcClient.startBotAndConnect({ endpoint: "https://your-connect-end-point-here/connect" });
} catch (e) {
  console.error(e.message);
}

// Events
pcClient.on(RTVIEvent.TransportStateChanged, (state) => {
  console.log("[EVENT] Transport state change:", state);
});
pcClient.on(RTVIEvent.BotReady, () => {
  console.log("[EVENT] Bot is ready");
});
pcClient.on(RTVIEvent.Connected, () => {
  console.log("[EVENT] User connected");
});
pcClient.on(RTVIEvent.Disconnected, () => {
  console.log("[EVENT] User disconnected");
});
```

--------------------------------

### Start Bot and Connect

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/pipecat-client.md

Initiates bot startup and establishes the transport connection in a single step. Use this for a combined start and connect operation.

```typescript
const botReady = await client.startBotAndConnect({
  endpoint: "https://your-server/connect",
  headers: new Headers({ "Authorization": "Bearer token" }),
  requestData: { sessionId: "123" },
});
```

--------------------------------

### Install Pipecat JS Client

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/README.md

Install the core Pipecat JS client library using npm.

```bash
npm install @pipecat/client-js
```

--------------------------------

### Example: Initialize Devices if Needed

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/helpers.md

Demonstrates how to check if device initialization is needed and then call initDevices if necessary.

```typescript
if (client.needsInit()) {
  await client.initDevices();
}
```

--------------------------------

### Handling StartBotError

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/errors.md

Example of how to catch and handle a StartBotError when starting a bot. It demonstrates checking the error type and accessing its properties like message and status.

```typescript
try {
  await client.startBotAndConnect({
    endpoint: "https://api.example.com/start",
    headers: new Headers({ "Authorization": "Bearer invalid" }),
  });
} catch (error) {
  if (error instanceof StartBotError) {
    console.error("Bot startup failed:", error.message);
    console.error("HTTP status:", error.status);
    if (error.status === 401) {
      console.error("Authentication failed");
    }
  }
}
```

--------------------------------

### Handle BotAlreadyStartedError Example

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/errors.md

Demonstrates the correct way to handle starting a bot multiple times. Call disconnect() before re-initiating the bot.

```typescript
// Don't do this:
const client = new PipecatClient({ transport });
await client.startBotAndConnect(params);
await client.startBotAndConnect(params); // Throws!

// Instead:
const client = new PipecatClient({ transport });
await client.startBotAndConnect(params);
// ...
await client.disconnect();
await client.startBotAndConnect(params); // OK now
```

--------------------------------

### Install Pipecat Client JS

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/client-js/README.md

Install the Pipecat Client JavaScript library using Yarn or npm.

```bash
yarn add @pipecat-ai/client-js
# or
npm install @pipecat-ai/client-js
```

--------------------------------

### Install Pipecat React Client

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/README.md

Optionally, install the Pipecat React client library for React applications.

```bash
npm install @pipecat/client-react
```

--------------------------------

### startBotParams

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/transport.md

Get or set the parameters used to start the bot. The setter automatically clones Request objects to avoid body reuse issues.

```APIDOC
## startBotParams

### Description
Get or set the parameters used to start the bot. The setter automatically clones Request objects to avoid body reuse issues.

### Signature
```typescript
get startBotParams(): APIRequest | undefined
set startBotParams(startBotParams: APIRequest)
```
```

--------------------------------

### Example: Creating a Custom Transport

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/transport.md

This example demonstrates how to create a custom transport by extending the `Transport` class and implementing its abstract methods.

```APIDOC
### Example: Creating a Custom Transport

```typescript
export class CustomTransport extends Transport {
  async initDevices(): Promise<void> {
    // Request permissions and initialize devices
  }

  async _connect(params?: TransportConnectionParams): Promise<void> {
    // Establish connection
  }

  async _disconnect(): Promise<void> {
    // Clean up connection
  }

  sendMessage(message: RTVIMessage): void {
    // Send message via your protocol
  }

  // Implement all other abstract methods...
}
```
```

--------------------------------

### Stream UI Snapshots

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/README.md

Start streaming UI snapshots with configurable debouncing and viewport tracking. Includes an example of handling UI commands like 'scroll_to' and sending UI events.

```typescript
client.startUISnapshotStream({ debounceMs: 300, trackViewport: true });

client.on(RTVIEvent.UICommand, (data) => {
  if (data.command === "scroll_to") {
    // Handle scroll command
  }
});

sendUIEvent("user_clicked_button", { buttonId: "submit" });
```

--------------------------------

### Install Daily Transport Layer

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/README.md

Install the Daily transport layer for real-time communication with Pipecat.

```bash
npm install @pipecat/daily-transport
```

--------------------------------

### Start and Connect Pipecat Client

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/OVERVIEW.md

Initializes the Pipecat client with a specified transport and connection settings, then starts the bot and connects to the server. Ensure the endpoint and request data are correctly configured.

```typescript
import { PipecatClient, RTVIEvent } from "@pipecat/client-js";
import { DailyTransport } from "@pipecat/daily-transport";

const client = new PipecatClient({
  transport: new DailyTransport(),
  enableMic: true,
  enableCam: false,
});

try {
  const botReady = await client.startBotAndConnect({
    endpoint: "https://api.example.com/sessions/start",
    requestData: { userId: "user123" },
  });
  console.log("Bot ready, version:", botReady.version);
} catch (error) {
  console.error("Failed to connect:", error);
}
```

--------------------------------

### Start Pipecat Client from Button Click

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/client-react/README.md

A minimal implementation of MyApp to demonstrate starting the Pipecat client's voice functionality, typically triggered by a user interaction like a button click.

```tsx
import { usePipecatClient } from "@pipecat-ai/client-react";

const MyApp = () => {
  const client = usePipecatClient();
  return <button onClick={() => client.start()}>OK Computer</button>;
};
```

--------------------------------

### Handle Bot Started Event

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/events.md

Use the RTVIEvent.BotStarted event to execute code once the bot has successfully started. The event provides the response from the bot's start endpoint.

```typescript
client.on(RTVIEvent.BotStarted, (response) => {
  console.log("Bot started with response:", response);
});
```

--------------------------------

### startBotAndConnect

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/pipecat-client.md

Starts the bot and establishes the transport connection in a single step. This method is a convenience wrapper that first calls `startBot()` and then `connect()`.

```APIDOC
## startBotAndConnect

### Description
Starts the bot via an endpoint and establishes the transport connection in one step. Calls `startBot()` then `connect()`.

### Method
`public async startBotAndConnect(startBotParams: APIRequest): Promise<BotReadyData>`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **startBotParams** (`APIRequest`) - Required - Endpoint and configuration for starting the bot

### Request Example
```typescript
const botReady = await client.startBotAndConnect({
  endpoint: "https://your-server/connect",
  headers: new Headers({ "Authorization": "Bearer token" }),
  requestData: { sessionId: "123" },
});
```

### Response
#### Success Response (`Promise<BotReadyData>`)
Resolves when the bot enters the ready state.

#### Response Example
(BotReadyData structure not provided in source)

### Throws
- `StartBotError`: If bot startup fails.
- `ConnectionTimeoutError`: If the connection times out.
```

--------------------------------

### APIRequest Example

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/types.md

Example of configuring an APIRequest with an endpoint, custom headers, request data, and a timeout.

```typescript
const request: APIRequest = {
  endpoint: "https://api.example.com/sessions",
  headers: new Headers({ "Authorization": "Bearer token" }),
  requestData: { user_id: "123" },
  timeout: 5000,
};
```

--------------------------------

### Start Bot

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/OVERVIEW.md

Initiate the bot connection with an endpoint and authentication data. Transitions state to authenticating and then authenticated.

```typescript
await client.startBot({
  endpoint: "https://api.example.com/sessions",
  requestData: { /* auth data */ }
});
```

--------------------------------

### Start Bot

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/pipecat-client.md

Initiates bot startup by posting to a specified endpoint. This method returns connection parameters for the transport. Ensure the endpoint is correctly configured.

```typescript
const connectionParams = await client.startBot({
  endpoint: new URL("https://api.example.com/start-session"),
  requestData: { userId: "user123" },
});
```

--------------------------------

### needsInit

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/helpers.md

Determines if any enabled device requires initialization. This method is available on the `PipecatClient` instance and helps manage the device setup process.

```APIDOC
## needsInit

### Description
Check if device initialization is still needed. Returns true if any enabled device is "uninitialized".

### Availability
Available on `PipecatClient` instance.

### Function Signature
```typescript
public needsInit(): boolean
```

### Returns
- `boolean` — True if initialization is needed for any enabled device.

### Example
```typescript
if (client.needsInit()) {
  await client.initDevices();
}
```
```

--------------------------------

### Custom Transport Implementation Example

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/transport.md

Example of creating a custom transport by extending the base Transport class. This includes implementing methods for device initialization, connection, disconnection, and message sending.

```typescript
export class CustomTransport extends Transport {
  async initDevices(): Promise<void> {
    // Request permissions and initialize devices
  }

  async _connect(params?: TransportConnectionParams): Promise<void> {
    // Establish connection
  }

  async _disconnect(): Promise<void> {
    // Clean up connection
  }

  sendMessage(message: RTVIMessage): void {
    // Send message via your protocol
  }

  // Implement all other abstract methods...
}
```

--------------------------------

### Initialize Devices

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/pipecat-client.md

Initializes local media devices (microphone and camera). This method manages the state transitions for both devices, starting with 'initializing'. Upon successful initialization, devices with a valid 'deviceId' will transition to 'granted'. Failures are categorized by specific error reasons. The Permissions API is consulted after transport resolution to determine authoritative 'denied' statuses.

```APIDOC
## initDevices

### Description
Initialize local media devices (mic/cam). Drives per-device `MediaState` transitions: both devices move to "initializing" on entry. On success, devices reporting a real `deviceId` move to "granted"; failures are classified into specific error reasons. The Permissions API is queried after transport resolution for authoritative "denied" status.

### Throws
`DeviceError` — When device initialization fails

### Example
```typescript
try {
  await client.initDevices();
} catch (error) {
  if (error instanceof DeviceError) {
    console.error("Devices:", error.devices, "Type:", error.type);
  }
}
```
```

--------------------------------

### Start UI Snapshot Stream

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/pipecat-client.md

Starts streaming accessibility snapshots to the server. Snapshots are emitted on various DOM and user interactions. Configure debounce interval, viewport tracking, and console logging via options.

```typescript
public startUISnapshotStream(options: A11ySnapshotStreamerOptions = {}): void
```

```typescript
client.startUISnapshotStream({
  debounceMs: 500,
  trackViewport: true,
});
```

--------------------------------

### Create Generic Client Message

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/rtvi-message.md

Example of creating a generic client message with a custom command and data.

```typescript
// Generic client message
const msg1 = new RTVIMessage(RTVIMessageType.CLIENT_MESSAGE, {
  t: "my_command",
  d: { param: "value" },
});
```

--------------------------------

### Get Available Media Devices

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/pipecat-client.md

Retrieves lists of available audio and video input/output devices.

```APIDOC
## Get Available Media Devices

### getAllMics

```typescript
public async getAllMics(): Promise<MediaDeviceInfo[]>
```

Get list of available microphone devices.

**Returns**: `Promise<MediaDeviceInfo[]>` — Array of mic devices

### getAllCams

```typescript
public async getAllCams(): Promise<MediaDeviceInfo[]>
```

Get list of available camera devices.

**Returns**: `Promise<MediaDeviceInfo[]>` — Array of camera devices

### getAllSpeakers

```typescript
public async getAllSpeakers(): Promise<MediaDeviceInfo[]>
```

Get list of available speaker devices.

**Returns**: `Promise<MediaDeviceInfo[]>` — Array of speaker devices
```

--------------------------------

### Watch for File Changes

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/README.md

Starts the development server to watch for file changes and automatically rebuild. Useful for active development.

```bash
yarn workspace @pipecat-ai/client-js run dev
```

--------------------------------

### version

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/pipecat-client.md

Retrieves the version string of the client library, for example, '1.12.0'.

```APIDOC
## version

### Description
Get client library version string (e.g., "1.12.0").

### Property Signature
```typescript
public get version(): string
```
```

--------------------------------

### Initialize Local Media Devices

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/pipecat-client.md

Initializes local media devices like microphone and camera. It handles state transitions and error classification. Use this to set up media capture at the start of a session.

```typescript
public async initDevices(): Promise<void>
```

```typescript
try {
  await client.initDevices();
} catch (error) {
  if (error instanceof DeviceError) {
    console.error("Devices:", error.devices, "Type:", error.type);
  }
}
```

--------------------------------

### Initialize Pipecat Client and Provider

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/client-react/README.md

Instantiate a PipecatClient and provide it to your React application using PipecatClientProvider. Includes automatic audio output setup with PipecatClientAudio.

```tsx
import { PipecatClient } from "@pipecat-ai/client-js";
import { PipecatClientAudio, PipecatClientProvider } from "@pipecat-ai/client-react";

const client = new PipecatClient({ transport: myTransportType.create() });

render(
  <PipecatClientProvider client={client}>
    <MyApp />
    <PipecatClientAudio />
  </PipecatClientProvider>
);
```

--------------------------------

### Vanilla JavaScript A11ySnapshotStreamer Example

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/a11y-snapshot-streamer.md

Demonstrates how to instantiate and use the A11ySnapshotStreamer in vanilla JavaScript. It includes setting up a callback for received snapshots and configuring stream options.

```typescript
import { A11ySnapshotStreamer } from "@pipecat-ai/client-js";

const streamer = new A11ySnapshotStreamer(
  (snapshot) => {
    // Send or persist the snapshot
    console.log("Snapshot nodes:", snapshot.children?.length);
    fetch("/api/ui-snapshot", {
      method: "POST",
      body: JSON.stringify(snapshot),
    });
  },
  {
    debounceMs: 500,
    trackViewport: true,
    logSnapshots: false,
  }
);

// Start streaming when ready
streamer.start();

// Stop later
setTimeout(() => streamer.stop(), 60000);
```

--------------------------------

### Get Media Tracks

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/pipecat-client.md

Retrieves the current local and bot audio/video media tracks.

```APIDOC
## tracks

```typescript
public tracks(): Tracks
```

Get current media tracks. Returns an object with local and bot audio/video tracks.

**Returns**: `Tracks` — Object containing `local` and `bot` track properties
```

--------------------------------

### Create Text Input Message

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/rtvi-message.md

Example of creating a message for sending text input to the bot, with an option to run immediately.

```typescript
// Text input
const msg2 = new RTVIMessage(RTVIMessageType.SEND_TEXT, {
  content: "Hello, bot",
  options: { run_immediately: true },
});
```

--------------------------------

### Get Client Version

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/pipecat-client.md

Retrieve the version string of the client library. This is useful for compatibility checks and debugging.

```typescript
public get version(): string
```

--------------------------------

### Initialize Pipecat Client with Daily Transport

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/README.md

Instantiate the Pipecat client using the Daily transport for WebRTC communication. Ensure both `@pipecat-ai/client-js` and `@pipecat-ai/daily-transport` are installed.

```typescript
import { PipecatClient } from "@pipecat-ai/client-js";
import { DailyTransport } from "@pipecat-ai/daily-transport";

const pcClient = new PipecatClient({
  transport: new DailyTransport(),
});
```

--------------------------------

### startUISnapshotStream

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/pipecat-client.md

Starts streaming accessibility snapshots to the server. Snapshots are emitted automatically based on DOM mutations, focus changes, scrolling, resizing, visibility changes, and form input.

```APIDOC
## startUISnapshotStream

### Description
Starts streaming accessibility snapshots to the server as first-class `ui-snapshot` RTVI messages. Snapshots are emitted on DOM mutations, focus changes, scrolling, resizing, visibility changes, and form input.

### Method Signature
```typescript
public startUISnapshotStream(options: A11ySnapshotStreamerOptions = {}): void
```

### Parameters
#### Options
- **options.debounceMs** (`number`) - Optional - Minimum interval between emissions. Defaults to `300`.
- **options.trackViewport** (`boolean`) - Optional - Annotate offscreen nodes. Defaults to `true`.
- **options.logSnapshots** (`boolean`) - Optional - Log snapshots to console. Defaults to `false`.

### Example
```typescript
client.startUISnapshotStream({
  debounceMs: 500,
  trackViewport: true,
});
```
```

--------------------------------

### Create and Connect Pipecat Client

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/README.md

Instantiate the PipecatClient with a chosen transport and connection options. Then, start the bot and connect to the specified endpoint. The `botReady` promise resolves when the connection is established.

```typescript
const client = new PipecatClient({
  transport: new DailyTransport(),
  enableMic: true,
  enableCam: false,
});

const botReady = await client.startBotAndConnect({
  endpoint: "https://api.example.com/sessions",
  requestData: { userId: "user123" },
});
```

--------------------------------

### Initialize Devices

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/OVERVIEW.md

Set up media devices. This is optional and can be called before connecting. It updates the media state.

```typescript
await client.initDevices();
```

--------------------------------

### DeviceStatus Error Handling Example

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/types.md

Demonstrates how to check the 'state' property of a DeviceStatus object and access the 'reason' field if an error has occurred.

```typescript
const mic = client.mediaState.mic;
if (mic.state === "error") {
  console.log("Mic blocked:", mic.reason === "blocked");
}
```

--------------------------------

### initDevices

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/transport.md

Initialize local media devices (microphone and/or camera). Should enumerate and request permissions.

```APIDOC
## initDevices

### Description
Initialize local media devices (microphone and/or camera). Should enumerate and request permissions. May fire `onDeviceError`, `onMicUpdated`, `onCamUpdated` callbacks during execution.

### Method
`initDevices`

### Endpoint
None

### Parameters
None

### Request Example
```typescript
abstract initDevices(): Promise<void>;
```

### Response
#### Success Response (200)
`Promise<void>` - Resolves when devices are initialized.

#### Response Example
None

**Throws**: `DeviceError` — On permission denial, device in-use, or other failures
```

--------------------------------

### Import Pipecat Client and Transports

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/README.md

Import necessary classes from the Pipecat client library and transports. Ensure these are the first steps in your application setup.

```typescript
import { PipecatClient, RTVIEvent, LogLevel } from "@pipecat/client-js";
import { DailyTransport } from "@pipecat/daily-transport";
```

--------------------------------

### Initialize Pipecat Client and Connect

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/INDEX.md

Set up the Pipecat client with a transport, enable microphone, and connect to the session. Ensure the endpoint and request data are correctly configured.

```typescript
import { PipecatClient, RTVIEvent, LogLevel } from "@pipecat-ai/client-js";
import { DailyTransport } from "@pipecat-ai/daily-transport";

const client = new PipecatClient({
  transport: new DailyTransport(),
  enableMic: true,
  enableCam: false,
});

client.setLogLevel(LogLevel.DEBUG);

// See: PipecatClient.startBotAndConnect()
const botReady = await client.startBotAndConnect({
  endpoint: "https://api.example.com/sessions",
  requestData: { userId: "user123" },
});
```

--------------------------------

### Initialize and Manage Devices

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/INDEX.md

Initialize device access, retrieve available microphones, update the selected mic, and enable/disable it. Check the media state for mic readiness or errors.

```typescript
// See: PipecatClient device methods, types.md (MediaState)
await client.initDevices();

const mics = await client.getAllMics();
client.updateMic(mics[0].deviceId);
client.enableMic(true);

const state = client.mediaState;
if (state.mic.state === "granted") {
  console.log("Mic ready");
} else if (state.mic.state === "error") {
  console.error("Mic error:", state.mic.reason);
}

// See: events.md (device events)
client.on(RTVIEvent.DeviceError, (error) => {
  console.error("Device error:", error.type);
});
```

--------------------------------

### Start and Stop UI Snapshot Stream

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/INDEX.md

Begin streaming UI snapshots with optional viewport tracking and stop the stream when no longer needed. Handle UI commands received from the server and send custom UI events.

```typescript
// See: PipecatClient.startUISnapshotStream(), events.md (UICommand, UIJobGroup)
client.startUISnapshotStream({
  debounceMs: 300,
  trackViewport: true,
});

client.on(RTVIEvent.UICommand, (data) => {
  if (data.command === "scroll_to") {
    // Handle scroll command
  }
});

client.sendUIEvent("user_interaction", { type: "click" });

client.stopUISnapshotStream();
```

--------------------------------

### Handle UI Snapshots and Commands

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/OVERVIEW.md

Starts streaming UI accessibility snapshots and handles UI commands from the server, such as scrolling or clicking elements. It also allows sending custom UI events and stopping the snapshot stream.

```typescript
// Start streaming accessibility snapshots
client.startUISnapshotStream({
  debounceMs: 300,
  trackViewport: true,
});

// Handle UI commands from server
client.on(RTVIEvent.UICommand, (data) => {
  if (data.command === "scroll_to") {
    const payload = data.payload as ScrollToPayload;
    const el = document.getElementById(payload.target_id || "");
    el?.scrollIntoView({ behavior: "smooth" });
  }
  if (data.command === "click") {
    const payload = data.payload as ClickPayload;
    const el = document.getElementById(payload.target_id || "");
    el?.click();
  }
});

// Send UI events
client.sendUIEvent("button_clicked", { buttonId: "submit-btn" });

// Later, stop snapshots
client.stopUISnapshotStream();
```

--------------------------------

### Define BotAlreadyStartedError

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/errors.md

This error is thrown when attempting to start a bot that is already running. Ensure disconnect() is called before starting again.

```typescript
class BotAlreadyStartedError extends RTVIError {
  constructor(message?: string | undefined)
}
```

--------------------------------

### Abstract initDevices Method Signature

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/transport.md

Implementations must provide this method to initialize local media devices, including enumerating and requesting permissions. It may fire device-related callbacks during execution and throws a DeviceError on failure.

```typescript
abstract initDevices(): Promise<void>
```

--------------------------------

### initialize

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/transport.md

Called once by PipecatClient at construction to wire up callbacks and message handling.

```APIDOC
## initialize

### Description
Called once by `PipecatClient` at construction to wire up callbacks and message handling.

### Method
`initialize`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
- **options** (`PipecatClientOptions`) - Required - Full client configuration
- **messageHandler** (`(ev: RTVIMessage) => void`) - Required - Callback to invoke on received messages

### Request Example
```typescript
abstract initialize(
  options: PipecatClientOptions,
  messageHandler: (ev: RTVIMessage) => void
): void;
```

### Response
#### Success Response (void)
No explicit return value.

#### Response Example
None
```

--------------------------------

### PipecatClient - Device Management

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/INDEX.md

Methods for initializing and managing audio/video devices.

```APIDOC
## PipecatClient Device Management

### Description
Provides functionality to initialize, retrieve, and update audio and video devices.

### Methods
- `initDevices()`: Initializes available devices.
- `getAllMics()`: Retrieves a list of all available microphones.
- `getAllCams()`: Retrieves a list of all available cameras.
- `getAllSpeakers()`: Retrieves a list of all available speakers.
- `updateMic(deviceId)`: Updates the selected microphone.
- `updateCam(deviceId)`: Updates the selected camera.
- `updateSpeaker(deviceId)`: Updates the selected speaker.
- `enableMic()`: Enables the microphone.
- `enableCam()`: Enables the camera.
- `enableScreenShare()`: Initiates screen sharing.
```

--------------------------------

### File Structure

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/OVERVIEW.md

Illustrates the directory and file organization within the client-js package. Helps in understanding the location of different modules and components.

```bash
client-js/
├── client/
│   ├── client.ts              # PipecatClient class
│   ├── transport.ts           # Transport abstract class
│   ├── A11ySnapshotStreamer.ts
│   ├── dispatcher.ts          # MessageDispatcher
│   ├── logger.ts              # LogLevel enum
│   ├── rest_helpers.ts        # makeRequest, APIRequest
│   ├── utils.ts               # Platform detection
│   ├── decorators.ts          # @transportReady, etc.
│   └── index.ts               # Client exports
├── rtvi/
│   ├── common_types.ts        # TransportState, Participant, etc.
│   ├── events.ts              # RTVIEvent enum and types
│   ├── messages.ts            # RTVIMessage and data types
│   ├── errors.ts              # Error classes
│   ├── ui.ts                  # UI command payloads
│   ├── a11y_walker.ts         # Snapshot generation
│   └── index.ts               # RTVI exports
├── index.ts                   # Root exports
└── package.json
```

--------------------------------

### makeRequest()

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/MANIFEST.md

A function to initiate a request. The full signature and examples are provided.

```APIDOC
## makeRequest()

### Description
Initiates a request. This function is part of the public API and is fully documented with its signature and usage examples.

### Method
N/A (Function)

### Parameters
(Details on parameters are available in the full documentation)

### Request Example
(Examples provided in the full documentation)

### Response
(Details on response are available in the full documentation)
```

--------------------------------

### isAPIRequest()

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/MANIFEST.md

A function to check if a request is an API request. The full signature and examples are provided.

```APIDOC
## isAPIRequest()

### Description
Checks if the given request is an API request. This function is part of the public API and is fully documented with its signature and usage examples.

### Method
N/A (Function)

### Parameters
(Details on parameters are available in the full documentation)

### Request Example
(Examples provided in the full documentation)

### Response
(Details on response are available in the full documentation)
```

--------------------------------

### Create Error Message

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/rtvi-message.md

Example of creating an error message using the static `error` method.

```typescript
// Error message
const errMsg = RTVIMessage.error("Something went wrong", false);
```

--------------------------------

### Device Management

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/INDEX.md

Methods for initializing, enumerating, selecting, and managing audio/video devices.

```APIDOC
## `client.initDevices()`

### Description
Requests necessary permissions for device access.

### Method
`POST` (assumed, as it initiates a process)

### Endpoint
`/initDevices` (assumed)

### Parameters
None

### Request Example
```javascript
client.initDevices();
```

### Response
#### Success Response (200)
Indicates that device initialization has started.

#### Response Example
```json
{
  "status": "initializing"
}
```
```

```APIDOC
## `client.getAllMics()`

### Description
Enumerates all available microphone devices.

### Method
`GET` (assumed, as it retrieves a list)

### Endpoint
`/getAllMics` (assumed)

### Parameters
None

### Request Example
```javascript
const mics = client.getAllMics();
console.log('Available microphones:', mics);
```

### Response
#### Success Response (200)
- **mics** (array) - A list of available microphone devices.

#### Response Example
```json
[
  { "deviceId": "mic-1", "label": "Microphone (Built-in)" },
  { "deviceId": "mic-2", "label": "External USB Mic" }
]
```
```

```APIDOC
## `client.getAllCams()`

### Description
Enumerates all available camera devices.

### Method
`GET` (assumed, as it retrieves a list)

### Endpoint
`/getAllCams` (assumed)

### Parameters
None

### Request Example
```javascript
const cams = client.getAllCams();
console.log('Available cameras:', cams);
```

### Response
#### Success Response (200)
- **cams** (array) - A list of available camera devices.

#### Response Example
```json
[
  { "deviceId": "cam-1", "label": "Webcam (Built-in)" },
  { "deviceId": "cam-2", "label": "External USB Camera" }
]
```
```

```APIDOC
## `client.getAllSpeakers()`

### Description
Enumerates all available speaker devices.

### Method
`GET` (assumed, as it retrieves a list)

### Endpoint
`/getAllSpeakers` (assumed)

### Parameters
None

### Request Example
```javascript
const speakers = client.getAllSpeakers();
console.log('Available speakers:', speakers);
```

### Response
#### Success Response (200)
- **speakers** (array) - A list of available speaker devices.

#### Response Example
```json
[
  { "deviceId": "spk-1", "label": "Speakers (Built-in)" },
  { "deviceId": "spk-2", "label": "Headphones" }
]
```
```

```APIDOC
## `client.updateMic(deviceId)`

### Description
Selects a specific microphone device.

### Method
`POST` (assumed, as it updates a selection)

### Endpoint
`/updateMic` (assumed)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **deviceId** (string) - Required - The ID of the microphone device to select.

### Request Example
```javascript
client.updateMic('mic-2');
```

### Response
#### Success Response (200)
Indicates the microphone has been updated.

#### Response Example
```json
{
  "status": "mic_updated"
}
```
```

```APIDOC
## `client.updateCam(deviceId)`

### Description
Selects a specific camera device.

### Method
`POST` (assumed, as it updates a selection)

### Endpoint
`/updateCam` (assumed)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **deviceId** (string) - Required - The ID of the camera device to select.

### Request Example
```javascript
client.updateCam('cam-2');
```

### Response
#### Success Response (200)
Indicates the camera has been updated.

#### Response Example
```json
{
  "status": "cam_updated"
}
```
```

```APIDOC
## `client.updateSpeaker(deviceId)`

### Description
Selects a specific speaker device.

### Method
`POST` (assumed, as it updates a selection)

### Endpoint
`/updateSpeaker` (assumed)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **deviceId** (string) - Required - The ID of the speaker device to select.

### Request Example
```javascript
client.updateSpeaker('spk-2');
```

### Response
#### Success Response (200)
Indicates the speaker has been updated.

#### Response Example
```json
{
  "status": "speaker_updated"
}
```
```

```APIDOC
## `client.enableMic()`

### Description
Enables the currently selected microphone.

### Method
`POST` (assumed, as it enables a device)

### Endpoint
`/enableMic` (assumed)

### Parameters
None

### Request Example
```javascript
client.enableMic();
```

### Response
#### Success Response (200)
Indicates the microphone has been enabled.

#### Response Example
```json
{
  "status": "mic_enabled"
}
```
```

```APIDOC
## `client.enableCam()`

### Description
Enables the currently selected camera.

### Method
`POST` (assumed, as it enables a device)

### Endpoint
`/enableCam` (assumed)

### Parameters
None

### Request Example
```javascript
client.enableCam();
```

### Response
#### Success Response (200)
Indicates the camera has been enabled.

#### Response Example
```json
{
  "status": "cam_enabled"
}
```
```

```APIDOC
## `client.enableScreenShare()`

### Description
Enables screen sharing.

### Method
`POST` (assumed, as it enables a feature)

### Endpoint
`/enableScreenShare` (assumed)

### Parameters
None

### Request Example
```javascript
client.enableScreenShare();
```

### Response
#### Success Response (200)
Indicates screen sharing has been enabled.

#### Response Example
```json
{
  "status": "screen_share_enabled"
}
```
```

```APIDOC
## `client.selectedMic`

### Description
Gets the currently selected microphone device.

### Method
`GET` (assumed, as it retrieves a selection)

### Endpoint
`/selectedMic` (assumed)

### Parameters
None

### Request Example
```javascript
const selectedMic = client.selectedMic;
console.log('Selected microphone:', selectedMic);
```

### Response
#### Success Response (200)
- **selectedMic** (object | null) - The currently selected microphone device object, or null if none is selected.

#### Response Example
```json
{
  "deviceId": "mic-2",
  "label": "External USB Mic"
}
```
```

```APIDOC
## `client.selectedCam`

### Description
Gets the currently selected camera device.

### Method
`GET` (assumed, as it retrieves a selection)

### Endpoint
`/selectedCam` (assumed)

### Parameters
None

### Request Example
```javascript
const selectedCam = client.selectedCam;
console.log('Selected camera:', selectedCam);
```

### Response
#### Success Response (200)
- **selectedCam** (object | null) - The currently selected camera device object, or null if none is selected.

#### Response Example
```json
{
  "deviceId": "cam-2",
  "label": "External USB Camera"
}
```
```

```APIDOC
## `client.selectedSpeaker`

### Description
Gets the currently selected speaker device.

### Method
`GET` (assumed, as it retrieves a selection)

### Endpoint
`/selectedSpeaker` (assumed)

### Parameters
None

### Request Example
```javascript
const selectedSpeaker = client.selectedSpeaker;
console.log('Selected speaker:', selectedSpeaker);
```

### Response
#### Success Response (200)
- **selectedSpeaker** (object | null) - The currently selected speaker device object, or null if none is selected.

#### Response Example
```json
{
  "deviceId": "spk-2",
  "label": "Headphones"
}
```
```

```APIDOC
## `client.isMicEnabled`

### Description
Checks if the microphone is currently enabled.

### Method
`GET` (assumed, as it checks a status)

### Endpoint
`/isMicEnabled` (assumed)

### Parameters
None

### Request Example
```javascript
const isMicOn = client.isMicEnabled;
console.log('Microphone enabled:', isMicOn);
```

### Response
#### Success Response (200)
- **isMicEnabled** (boolean) - True if the microphone is enabled, false otherwise.

#### Response Example
```json
{
  "isMicEnabled": true
}
```
```

```APIDOC
## `client.isCamEnabled`

### Description
Checks if the camera is currently enabled.

### Method
`GET` (assumed, as it checks a status)

### Endpoint
`/isCamEnabled` (assumed)

### Parameters
None

### Request Example
```javascript
const isCamOn = client.isCamEnabled;
console.log('Camera enabled:', isCamOn);
```

### Response
#### Success Response (200)
- **isCamEnabled** (boolean) - True if the camera is enabled, false otherwise.

#### Response Example
```json
{
  "isCamEnabled": true
}
```
```

```APIDOC
## `client.isSharingScreen`

### Description
Checks if screen sharing is currently active.

### Method
`GET` (assumed, as it checks a status)

### Endpoint
`/isSharingScreen` (assumed)

### Parameters
None

### Request Example
```javascript
const isSharing = client.isSharingScreen;
console.log('Screen sharing active:', isSharing);
```

### Response
#### Success Response (200)
- **isSharingScreen** (boolean) - True if screen sharing is active, false otherwise.

#### Response Example
```json
{
  "isSharingScreen": false
}
```
```

```APIDOC
## `client.mediaState`

### Description
Provides a snapshot of the current media state, including device statuses and track information.

### Method
`GET` (assumed, as it retrieves a state)

### Endpoint
`/mediaState` (assumed)

### Parameters
None

### Request Example
```javascript
const mediaInfo = client.mediaState;
console.log('Media state:', mediaInfo);
```

### Response
#### Success Response (200)
- **mediaState** (object) - An object containing the current media status.

#### Response Example
```json
{
  "micEnabled": true,
  "camEnabled": true,
  "isSharingScreen": false,
  "selectedMic": { "deviceId": "mic-2", "label": "External USB Mic" },
  "selectedCam": { "deviceId": "cam-2", "label": "External USB Camera" },
  "tracks": []
}
```
```

```APIDOC
## `client.needsInit()`

### Description
Checks if the client needs to perform device initialization.

### Method
`GET` (assumed, as it checks a condition)

### Endpoint
`/needsInit` (assumed)

### Parameters
None

### Request Example
```javascript
const needsInitialization = client.needsInit();
console.log('Initialization needed:', needsInitialization);
```

### Response
#### Success Response (200)
- **needsInit** (boolean) - True if initialization is required, false otherwise.

#### Response Example
```json
{
  "needsInit": false
}
```
```

```APIDOC
## `client.tracks()`

### Description
Retrieves the current `MediaStreamTrack` objects being used by the client.

### Method
`GET` (assumed, as it retrieves track information)

### Endpoint
`/tracks` (assumed)

### Parameters
None

### Request Example
```javascript
const streamTracks = client.tracks();
console.log('MediaStreamTracks:', streamTracks);
```

### Response
#### Success Response (200)
- **tracks** (array) - An array of `MediaStreamTrack` objects.

#### Response Example
```json
[
  // MediaStreamTrack objects would be listed here
]
```
```

--------------------------------

### Get Tracks

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/transport.md

Retrieve the current media tracks, including local and bot audio/video tracks.

```APIDOC
## Get Tracks

### Description
Get current media tracks. Returns an object with local and bot audio/video tracks.

### Method
GET

### Endpoint
`/tracks`

### Response
#### Success Response (200)
- **tracks** (Tracks) - Object containing local and bot track collections.
```

--------------------------------

### Create UI Event Message

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/rtvi-message.md

Example of creating a message to report a UI event, such as a form submission.

```typescript
// UI event
const msg3 = new RTVIMessage(RTVIMessageType.UI_EVENT, {
  event: "form_submitted",
  payload: { formId: "contact-form" },
});
```

--------------------------------

### needsInit

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/pipecat-client.md

Determines if device initialization is still required. Returns `true` if any enabled device has a status of 'uninitialized'.

```APIDOC
## needsInit

### Description
Check if device initialization is still needed. Returns `true` if any enabled device is still "uninitialized".

### Method Signature
```typescript
public needsInit(): boolean
```
```

--------------------------------

### startBot

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/api-reference/pipecat-client.md

Initiates the bot startup process by sending a request to a specified endpoint. It returns the necessary connection parameters for the transport layer.

```APIDOC
## startBot

### Description
Initiate bot startup by posting to a specified endpoint. Returns connection parameters for the transport.

### Method
`@transportAlreadyStarted
public async startBot(startBotParams: APIRequest): Promise<unknown>`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **startBotParams** (`APIRequest`) - Required - Endpoint request configuration

### Request Example
```typescript
const connectionParams = await client.startBot({
  endpoint: new URL("https://api.example.com/start-session"),
  requestData: { userId: "user123" },
});
```

### Response
#### Success Response (`Promise<unknown>`)
Bot response containing connection parameters.

#### Response Example
(Response structure not provided in source)

### Throws
- `StartBotError`: On request failure.
```

--------------------------------

### Voice Activity Detection (VAD) Events

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/events.md

Events indicating when the user or bot starts or stops speaking.

```APIDOC
## Voice Activity Detection (VAD) Events

### RTVIEvent.UserStartedSpeaking

**Callback**: `() => void`
**Fired when**: User begins speaking (VAD detected)

### RTVIEvent.UserStoppedSpeaking

**Callback**: `() => void`
**Fired when**: User stops speaking

### RTVIEvent.BotStartedSpeaking

**Callback**: `() => void`
**Fired when**: Bot begins speaking

### RTVIEvent.BotStoppedSpeaking

**Callback**: `() => void`
**Fired when**: Bot stops speaking
```

--------------------------------

### Initialize Pipecat Client

Source: https://github.com/pipecat-ai/pipecat-client-web/blob/main/_autodocs/OVERVIEW.md

Instantiate the PipecatClient with transport and media options. Ensure DailyTransport is imported if used.

```typescript
const client = new PipecatClient({
  transport: new DailyTransport(),
  enableMic: true,
  enableCam: false,
  callbacks: { /* ... */ }
});
```