### Create Project from Example Source: https://github.com/livekit/client-sdk-esp32/blob/main/README.md Use this command to create a new project based on an existing LiveKit example. Replace '' with the desired example's directory name. ```sh idf.py create-project-from-example "livekit/livekit=0.3.7:" ``` -------------------------------- ### Example Output: Room Connected Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/examples/custom_hardware/README.md This message indicates that the ESP32 example has successfully established a network connection and joined a LiveKit room. ```txt I (3200) livekit_example: Room state changed: CONNECTED ``` -------------------------------- ### CMakeLists.txt Boilerplate and Project Setup Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/examples/minimal_video/CMakeLists.txt This snippet includes the essential boilerplate and project setup commands required for the CMake build system to function correctly with the ESP-IDF build system. Ensure these lines are in the exact order specified. ```cmake cmake_minimum_required(VERSION 3.5) set(COMPONENTS main) # Trim build include($ENV{IDF_PATH}/tools/cmake/project.cmake) project(minimal_video) ``` -------------------------------- ### Configure LiveKit Sandbox Credentials Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/examples/minimal/README.md Use a LiveKit Sandbox for quick setup by providing its ID. This option is enabled via menuconfig or sdkconfig. ```ini CONFIG_LK_EXAMPLE_USE_SANDBOX=y CONFIG_LK_EXAMPLE_SANDBOX_ID="my-project-xxxxxx" ``` -------------------------------- ### ESP32 Project Setup with CMake Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/examples/custom_hardware/CMakeLists.txt This snippet configures the minimum CMake version, sets the main component, includes the ESP-IDF project CMake module, and defines the project name. It's essential for building ESP32 projects with custom LiveKit integrations. ```cmake cmake_minimum_required(VERSION 3.5) set(COMPONENTS main) include($ENV{IDF_PATH}/tools/cmake/project.cmake) project(lk_custom_hardware) ``` -------------------------------- ### Connect to LiveKit Room using Sandbox or JWT Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Connects to a LiveKit server using either a LiveKit Sandbox for development or a pre-generated JWT token. Ensure the server URL starts with `wss://` or `ws://`. ```c #include "livekit_sandbox.h" livekit_sandbox_options_t sandbox_opts = { .sandbox_id = "my-project-abc123", .room_name = "test-room", .participant_name = "esp32-device" }; livekit_sandbox_res_t res = {}; if (livekit_sandbox_generate(&sandbox_opts, &res)) { livekit_room_connect(room_handle, res.server_url, res.token); livekit_sandbox_res_free(&res); } ``` ```c livekit_err_t err = livekit_room_connect( room_handle, "wss://my-project.livekit.cloud", "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..." // server-generated token ); if (err != LIVEKIT_ERR_NONE) { ESP_LOGE("app", "Connect failed: %d", err); } // Expected log output once connected: // I (19508) livekit_example: Room state: Connected ``` -------------------------------- ### Run Protocol Update Script Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/protocol/README.md Execute the Python script to update Protocol Buffer bindings. Ensure Nanopb is installed and available in your system's PATH. ```sh python update.py ``` -------------------------------- ### LiveKit Room Connection Status Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/examples/minimal/README.md Example log message indicating a successful connection to the LiveKit room. This message appears after the application establishes a network connection and connects to the room. ```txt I (19508) livekit_example: Room state: Connected ``` -------------------------------- ### livekit_system_init Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Initializes the LiveKit system once. This function must be called before creating any room objects. It sets up the internal media library thread scheduler. Returns `LIVEKIT_ERR_NONE` on success. ```APIDOC ## livekit_system_init ### Description One-time system initialization. Must be called once early in `app_main` before any room is created. Initializes the internal media library thread scheduler. Returns `LIVEKIT_ERR_NONE` on success. ### Function Signature ```c livekit_err_t livekit_system_init(void); ``` ### Example Usage ```c #include "livekit.h" void app_main(void) { // Initialize LiveKit system subsystems first livekit_err_t err = livekit_system_init(); if (err != LIVEKIT_ERR_NONE) { ESP_LOGE("app", "LiveKit system init failed: %d", err); return; } // ... board_init(), media_init(), network_connect(), etc. } ``` ``` -------------------------------- ### Initialize LiveKit System Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Must be called once early in app_main before any room is created. Initializes the internal media library thread scheduler. Returns LIVEKIT_ERR_NONE on success. ```c #include "livekit.h" void app_main(void) { // Initialize LiveKit system subsystems first livekit_err_t err = livekit_system_init(); if (err != LIVEKIT_ERR_NONE) { ESP_LOGE("app", "LiveKit system init failed: %d", err); return; } // ... board_init(), media_init(), network_connect(), etc. } ``` -------------------------------- ### Initialize LiveKit System Source: https://github.com/livekit/client-sdk-esp32/blob/main/README.md Include the LiveKit header and call `livekit_system_init()` early in your application's main function before any other LiveKit calls. ```c #include "livekit.h" void app_main(void) { livekit_system_init(); // Your application code... } ``` -------------------------------- ### Configure Development Board Type Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/examples/minimal_video/README.md Set the development board type using `CONFIG_LK_EXAMPLE_CODEC_BOARD_TYPE`. Ensure the board is supported and listed in the `board_cfg.txt` file. ```ini CONFIG_LK_EXAMPLE_CODEC_BOARD_TYPE="ESP32_P4_DEV_V14" ``` -------------------------------- ### Build, Flash, and Monitor ESP32 Application Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/examples/minimal/README.md Command to build the application, flash it to the ESP32 board, and monitor serial output for debugging and status messages. ```sh idf.py flash monitor ``` -------------------------------- ### Create LiveKit Room Object Source: https://github.com/livekit/client-sdk-esp32/blob/main/README.md Initializes a LiveKit room object with specified audio capture and subscription options, along with event handlers. Ensure all necessary fields are configured as per the API reference. ```c static livekit_room_handle_t room_handle = NULL; livekit_room_options_t room_options = { .publish = { .kind = LIVEKIT_MEDIA_TYPE_AUDIO, .audio_encode = { .codec = LIVEKIT_AUDIO_CODEC_OPUS, .sample_rate = 16000, .channel_count = 1 }, .capturer = my_capturer }, .subscribe = { .kind = LIVEKIT_MEDIA_TYPE_AUDIO, .renderer = my_renderer }, .on_state_changed = on_state_changed, .on_participant_info = on_participant_info }; if (livekit_room_create(&room_handle, &room_options) != LIVEKIT_ERR_NONE) { ESP_LOGE(TAG, "Failed to create room object"); } ``` -------------------------------- ### Build and Run Test App with IDF Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/test_app/README.md Use these commands to set the target board, build the test application, and run the tests using pytest. Pytest will handle flashing the application to the device. ```sh idf.py set-target esp32[s3|p4] idf.py build pytest ``` -------------------------------- ### livekit_room_connect Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Asynchronously connects the room to a LiveKit server. The server_url must begin with wss:// or ws://. Connection result is delivered via the on_state_changed callback registered in livekit_room_create. ```APIDOC ## livekit_room_connect ### Description Connects to a LiveKit server using either a sandbox environment or a pre-generated JWT token. ### Method `livekit_room_connect(room_handle, server_url, token)` ### Parameters - **room_handle**: Handle to the room. - **server_url** (string): The URL of the LiveKit server (must start with `wss://` or `ws://`). - **token** (string): The JWT token for authentication. ### Request Example ```c // Option A: connect using a LiveKit Sandbox #include "livekit_sandbox.h" livekit_sandbox_options_t sandbox_opts = { .sandbox_id = "my-project-abc123", .room_name = "test-room", .participant_name = "esp32-device" }; livekit_sandbox_res_t res = {}; if (livekit_sandbox_generate(&sandbox_opts, &res)) { livekit_room_connect(room_handle, res.server_url, res.token); livekit_sandbox_res_free(&res); } // Option B: connect using a pre-generated JWT token livekit_err_t err = livekit_room_connect( room_handle, "wss://my-project.livekit.cloud", "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..." // server-generated token ); if (err != LIVEKIT_ERR_NONE) { ESP_LOGE("app", "Connect failed: %d", err); } ``` ### Response Returns `LIVEKIT_ERR_NONE` on success, or an error code on failure. ``` -------------------------------- ### Configure Pre-generated Token Credentials Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/examples/minimal/README.md Specify a server URL and a pre-generated JWT token for authentication. This is an alternative to using a LiveKit Sandbox. ```ini CONFIG_LK_EXAMPLE_USE_PREGENERATED=y CONFIG_LK_EXAMPLE_TOKEN="your-jwt-token" CONFIG_LK_EXAMPLE_SERVER_URL="ws://localhost:7880" ``` -------------------------------- ### Connect to LiveKit Room Source: https://github.com/livekit/client-sdk-esp32/blob/main/README.md Establishes a connection to a LiveKit room using the room handle, server URL, and authentication token. Connection status is managed asynchronously via the `on_state_changed` handler. ```c livekit_room_connect(room_handle, "", ""); ``` -------------------------------- ### Configure WiFi Network Connection Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/examples/minimal/README.md Set up the WiFi credentials (SSID and password) to connect the ESP32 device to a network. ```ini CONFIG_LK_EXAMPLE_USE_WIFI=y CONFIG_LK_EXAMPLE_WIFI_SSID="" CONFIG_LK_EXAMPLE_WIFI_PASSWORD="" ``` -------------------------------- ### livekit_sandbox_generate Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Fetches a temporary server URL and JWT access token from a LiveKit Sandbox endpoint for development purposes. The token is valid for 15 minutes. Remember to call `livekit_sandbox_res_free` to release allocated strings. ```APIDOC ## livekit_sandbox_generate — Fetch a token from a LiveKit Sandbox (development only) Contacts the LiveKit Cloud sandbox endpoint to obtain a temporary server URL and JWT access token. The token is valid for 15 minutes. Call `livekit_sandbox_res_free` to release the allocated strings. Requires the `livekit/sandbox_token` component. ### Function Signature ```c livekit_sandbox_res_t livekit_sandbox_generate(const livekit_sandbox_options_t *opts, livekit_sandbox_res_t *res) ``` ### Parameters #### `opts` - **sandbox_id** (string) - Required - The ID of the sandbox project from cloud.livekit.io. - **room_name** (string) - Optional - The name of the room. If NULL, a room name is auto-generated. - **participant_name** (string) - Optional - The name of the participant. If NULL, a participant name is auto-generated. #### `res` - **server_url** (string) - Output - The server URL for the sandbox room. - **token** (string) - Output - The JWT access token for authentication. - **room_name** (string) - Output - The actual room name used (can be auto-generated). - **participant_name** (string) - Output - The actual participant name used (can be auto-generated). ### Return Value - `true` if the token generation was successful, `false` otherwise. ### Example Usage ```c #include "livekit_sandbox.h" livekit_sandbox_options_t opts = { .sandbox_id = "my-project-abc123", // from cloud.livekit.io .room_name = "demo-room", // NULL = auto-generated .participant_name = "esp32-device" // NULL = auto-generated }; livekit_sandbox_res_t res = {}; if (!livekit_sandbox_generate(&opts, &res)) { ESP_LOGE("app", "Sandbox token generation failed"); return; } ESP_LOGI("app", "Server: %s", res.server_url); ESP_LOGI("app", "Room: %s", res.room_name); livekit_room_connect(room_handle, res.server_url, res.token); livekit_sandbox_res_free(&res); // free server_url, token, room_name, participant_name ``` ``` -------------------------------- ### Kconfig - Compile-time Configuration Source: https://context7.com/livekit/client-sdk-esp32/llms.txt The SDK exposes tunable parameters via ESP-IDF's `menuconfig`. Set them in `sdkconfig` or `sdkconfig.defaults` without code changes. ```ini # Maximum connection retry attempts (default: 7) CONFIG_LK_MAX_RETRIES=10 # Engine event queue depth (default: 32) CONFIG_LK_ENGINE_QUEUE_SIZE=64 # Engine task stack size in bytes (default: 8192) CONFIG_LK_ENGINE_TASK_STACK_SIZE=12288 # AV capture/publish interval in milliseconds (default: 20 ms → 50 fps) CONFIG_LK_PUB_INTERVAL_MS=20 # Published track names shown to remote participants CONFIG_LK_PUB_AUDIO_TRACK_NAME="Microphone" CONFIG_LK_PUB_VIDEO_TRACK_NAME="Camera" # Maximum concurrent incoming / outgoing data streams (default: 4 each) CONFIG_LK_MAX_DATA_STREAM_READERS=8 CONFIG_LK_MAX_DATA_STREAM_WRITERS=4 ``` -------------------------------- ### livekit_room_create Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Creates and configures a room object. This object is used to manage connections and media streams. The room is not connected upon creation. Call `livekit_room_destroy` when finished with the room. Returns `LIVEKIT_ERR_NONE` on success. ```APIDOC ## livekit_room_create ### Description Create a room object. Allocates and configures a room object with publish/subscribe media options and event callbacks. The room is not yet connected after creation. Returns `LIVEKIT_ERR_NONE` on success; call `livekit_room_destroy` when done. ### Function Signature ```c livekit_err_t livekit_room_create(livekit_room_handle_t *room_handle, const livekit_room_options_t *options); ``` ### Parameters #### Path Parameters - **room_handle** (*livekit_room_handle_t**) - **options** (*const livekit_room_options_t**) ### Example Usage (Audio-only) ```c #include "livekit.h" static livekit_room_handle_t room_handle = NULL; static void on_state_changed(livekit_connection_state_t state, void *ctx) { ESP_LOGI("app", "Room state: %s", livekit_connection_state_str(state)); if (state == LIVEKIT_CONNECTION_STATE_FAILED) { livekit_failure_reason_t reason = livekit_room_get_failure_reason(room_handle); ESP_LOGE("app", "Failure: %s", livekit_failure_reason_str(reason)); } } // Audio-only bidirectional room livekit_room_options_t options = { .publish = { .kind = LIVEKIT_MEDIA_TYPE_AUDIO, .audio_encode = { .codec = LIVEKIT_AUDIO_CODEC_OPUS, .sample_rate = 16000, .channel_count = 1 }, .capturer = media_get_capturer() // esp_capture_handle_t }, .subscribe = { .kind = LIVEKIT_MEDIA_TYPE_AUDIO, .renderer = media_get_renderer() // av_render_handle_t }, .on_state_changed = on_state_changed, .ctx = NULL }; livekit_err_t err = livekit_room_create(&room_handle, &options); if (err != LIVEKIT_ERR_NONE) { ESP_LOGE("app", "Failed to create room: %d", err); } ``` ### Example Usage (Audio + Video) ```c livekit_room_options_t av_options = { .publish = { .kind = LIVEKIT_MEDIA_TYPE_BOTH, // audio + video .audio_encode = { .codec = LIVEKIT_AUDIO_CODEC_OPUS, .sample_rate = 16000, .channel_count = 1 }, .video_encode = { .codec = LIVEKIT_VIDEO_CODEC_H264, .width = 1280, .height = 720, .fps = 30 }, .capturer = media_get_capturer() }, .subscribe = { .kind = LIVEKIT_MEDIA_TYPE_AUDIO, .renderer = media_get_renderer() }, .on_state_changed = on_state_changed }; livekit_room_handle_t av_room = NULL; if (livekit_room_create(&av_room, &av_options) != LIVEKIT_ERR_NONE) { ESP_LOGE("app", "Failed to create AV room"); } ``` ``` -------------------------------- ### Create Audio+Video Room Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Configures a room to publish both audio (Opus) and video (H.264) while subscribing to audio. Requires a state change callback. ```c livekit_room_options_t av_options = { .publish = { .kind = LIVEKIT_MEDIA_TYPE_BOTH, // audio + video .audio_encode = { .codec = LIVEKIT_AUDIO_CODEC_OPUS, .sample_rate = 16000, .channel_count = 1 }, .video_encode = { .codec = LIVEKIT_VIDEO_CODEC_H264, .width = 1280, .height = 720, .fps = 30 }, .capturer = media_get_capturer() }, .subscribe = { .kind = LIVEKIT_MEDIA_TYPE_AUDIO, .renderer = media_get_renderer() }, .on_state_changed = on_state_changed }; livekit_room_handle_t av_room = NULL; if (livekit_room_create(&av_room, &av_options) != LIVEKIT_ERR_NONE) { ESP_LOGE("app", "Failed to create AV room"); } ``` -------------------------------- ### Send Text Data Stream Source: https://github.com/livekit/client-sdk-esp32/blob/main/README.md Opens a text data stream on the 'lk.chat' topic, writes 'hello world' to it, and then closes the stream. Ensure the topic is correctly specified and the stream is properly closed after use. ```c livekit_data_stream_options_t opts = { .topic = "lk.chat", .is_text = true }; livekit_data_stream_handle_t stream; livekit_room_data_stream_open(room_handle, &opts, &stream); livekit_room_data_stream_write(room_handle, stream, (const uint8_t*)"hello world", 11); livekit_room_data_stream_close(room_handle, stream); ``` -------------------------------- ### Create Room Source: https://github.com/livekit/client-sdk-esp32/blob/main/README.md Creates a room object with specified options for publishing and subscribing to media, and handlers for room events. Refer to the API reference for a comprehensive list of options. ```APIDOC ## Create Room ### Description Creates a room object, specifying your capturer, renderer, and handlers for room events. ### Method `livekit_room_create` ### Parameters - `room_handle` (pointer to `livekit_room_handle_t`): Output parameter for the created room handle. - `room_options` (pointer to `livekit_room_options_t`): Configuration options for the room. ### Request Example ```c static livekit_room_handle_t room_handle = NULL; livekit_room_options_t room_options = { .publish = { .kind = LIVEKIT_MEDIA_TYPE_AUDIO, .audio_encode = { .codec = LIVEKIT_AUDIO_CODEC_OPUS, .sample_rate = 16000, .channel_count = 1 }, .capturer = my_capturer }, .subscribe = { .kind = LIVEKIT_MEDIA_TYPE_AUDIO, .renderer = my_renderer }, .on_state_changed = on_state_changed, .on_participant_info = on_participant_info }; if (livekit_room_create(&room_handle, &room_options) != LIVEKIT_ERR_NONE) { ESP_LOGE(TAG, "Failed to create room object"); } ``` ### Response - Returns `LIVEKIT_ERR_NONE` on success, or an error code on failure. ``` -------------------------------- ### Essential CMakeLists.txt Boilerplate Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/examples/voice_agent/CMakeLists.txt These lines must be included in your project's CMakeLists.txt in the specified order for the build system to function correctly. They set up the minimum CMake version, define project components, and include the necessary IDF build tools. ```cmake cmake_minimum_required(VERSION 3.5) set(COMPONENTS main) # Trim build include($ENV{IDF_PATH}/tools/cmake/project.cmake) project(voice_agent) ``` -------------------------------- ### Configure ESP-IDF Project Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/test_app/CMakeLists.txt Sets up the CMake build system for an ESP-IDF project. Includes necessary components and defines the project name. ```cmake cmake_minimum_required(VERSION 3.16) set(COMPONENTS main) set(EXTRA_COMPONENT_DIRS "$ENV{IDF_PATH}/tools/unit-test-app/components") include($ENV{IDF_PATH}/tools/cmake/project.cmake) project(test_app) ``` -------------------------------- ### Configure Nanopb Build Definitions Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/third_party/nanopb/CMakeLists.txt Sets compile definitions for the Nanopb component library to enable specific features like buffer-only mode, UTF-8 validation, and dynamic memory allocation. ```cmake target_compile_definitions(${COMPONENT_LIB} PRIVATE PB_BUFFER_ONLY=1) ``` ```cmake target_compile_definitions(${COMPONENT_LIB} PRIVATE PB_VALIDATE_UTF8=1) ``` ```cmake target_compile_definitions(${COMPONENT_LIB} PRIVATE PB_ENABLE_MALLOC=1) ``` -------------------------------- ### Connect Room Source: https://github.com/livekit/client-sdk-esp32/blob/main/README.md Connects to a LiveKit server using the provided room handle, server URL, and token. Connection status is reported via the `on_state_changed` handler. ```APIDOC ## Connect Room ### Description Connects to a LiveKit server with a given room handle, server URL, and token. ### Method `livekit_room_connect` ### Parameters - `room_handle` (livekit_room_handle_t): Handle to the room object. - `server_url` (const char*): The URL of the LiveKit server. - `token` (const char*): The authentication token for connecting to the server. ### Request Example ```c livekit_room_connect(room_handle, "", ""); ``` ### Notes This method is asynchronous. Use the `on_state_changed` handler provided during room creation to monitor connection status. ``` -------------------------------- ### Add LiveKit Dependency Source: https://github.com/livekit/client-sdk-esp32/blob/main/README.md Add the LiveKit SDK as a dependency to your existing ESP32 application using the idf.py tool. It is recommended to pin to a specific version. ```sh idf.py add-dependency "livekit/livekit=0.3.7" ``` -------------------------------- ### livekit_room_get_state / livekit_connection_state_str Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Queries the current connection state of the room and converts it to a human-readable string. ```APIDOC ## livekit_room_get_state / livekit_connection_state_str ### Description Polls the current connection state and converts it to a human-readable string. States include `DISCONNECTED`, `CONNECTING`, `CONNECTED`, `RECONNECTING`, `FAILED`. ### Method - `livekit_room_get_state(room_handle)` - `livekit_connection_state_str(state)` ### Parameters - **room_handle**: Handle to the room. - **state** (`livekit_connection_state_t`): The connection state to convert to a string. ### Request Example ```c livekit_connection_state_t state = livekit_room_get_state(room_handle); ESP_LOGI("app", "Current state: %s", livekit_connection_state_str(state)); // Inspect failure details when state == LIVEKIT_CONNECTION_STATE_FAILED if (state == LIVEKIT_CONNECTION_STATE_FAILED) { livekit_failure_reason_t reason = livekit_room_get_failure_reason(room_handle); ESP_LOGE("app", "Reason: %s", livekit_failure_reason_str(reason)); } ``` ### Response - **`livekit_room_get_state`**: Returns a `livekit_connection_state_t` enum value representing the current connection state. - **`livekit_connection_state_str`**: Returns a string representation of the connection state. ``` -------------------------------- ### Publish User Packet with Raw Data Source: https://github.com/livekit/client-sdk-esp32/blob/main/README.md Publish a user packet containing a raw data payload under a specific topic. Ensure the `room_handle` is valid and the `options` struct is correctly populated. ```c const char* command = "G5 I0 J3 P0 Q-3 X2 Y3"; livekit_payload_t payload = { .bytes = (uint8_t*)command, .size = strlen(command) }; livekit_data_publish_options_t options = { .payload = &payload, .topic = "gcode", .lossy = false, .destination_identities = (char*[]){ "printer-1" }, .destination_identities_count = 1 }; livekit_room_publish_data(room_handle, &options); ``` -------------------------------- ### RPC Registration and Handling Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Register named methods for remote invocation and handle their execution. Handlers must use `livekit_rpc_return_ok` or `livekit_rpc_return_error` to respond. Unregister methods using `livekit_room_rpc_unregister`. ```APIDOC ## `livekit_room_rpc_register` / `livekit_rpc_return_ok` / `livekit_rpc_return_error` — Register and handle RPC methods Registers a named method that remote participants (e.g., AI agents) can invoke. The handler receives an `livekit_rpc_invocation_t` and must call `livekit_rpc_return_ok` or `livekit_rpc_return_error` before returning. Unregister with `livekit_room_rpc_unregister`. ```c #include "cJSON.h" #include "livekit.h" // Handler: set an on-board LED color and state from a JSON payload static void set_led_state(const livekit_rpc_invocation_t *invocation, void *ctx) { if (invocation->payload == NULL) { livekit_rpc_return_error("Missing payload"); return; } cJSON *root = cJSON_Parse(invocation->payload); if (!root) { livekit_rpc_return_error("Invalid JSON"); return; } // Expected payload: {"color": "red", "state": true} const cJSON *color = cJSON_GetObjectItemCaseSensitive(root, "color"); const cJSON *state = cJSON_GetObjectItemCaseSensitive(root, "state"); if (!cJSON_IsString(color) || !cJSON_IsBool(state)) { cJSON_Delete(root); livekit_rpc_return_error("Unexpected JSON format"); return; } ESP_LOGI("app", "Set LED '%s' to %s", color->valuestring, cJSON_IsTrue(state) ? "ON" : "OFF"); cJSON_Delete(root); livekit_rpc_return_ok(NULL); } // Handler: return CPU temperature as a string static void get_cpu_temp(const livekit_rpc_invocation_t *invocation, void *ctx) { float temp = board_get_temp(); char buf[16]; snprintf(buf, sizeof(buf), "%.2f", temp); livekit_rpc_return_ok(buf); // caller receives e.g. "42.50" } // Register both handlers after creating the room livekit_room_rpc_register(room_handle, "set_led_state", set_led_state); livekit_room_rpc_register(room_handle, "get_cpu_temp", get_cpu_temp); // Unregister when no longer needed livekit_room_rpc_unregister(room_handle, "get_cpu_temp"); ``` ``` -------------------------------- ### Register Nanopb Component Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/third_party/nanopb/CMakeLists.txt Registers the Nanopb source and include directories for the component library. ```cmake idf_component_register(SRC_DIRS ./src INCLUDE_DIRS ./include) ``` -------------------------------- ### Configure Development Board Type Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/examples/minimal/README.md Specify the development board type to ensure correct peripheral access for media capture and rendering. Supported boards are defined in the codec_board component. ```ini CONFIG_LK_EXAMPLE_CODEC_BOARD_TYPE="ESP32_S3_BOX_3" ``` -------------------------------- ### Configure Development Board Type Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/examples/voice_agent/README.md Specify the codec board type if using a board other than the default ESP32-S3-Korvo-2. This uses the Espressif codec_board component. ```ini CONFIG_LK_EXAMPLE_CODEC_BOARD_TYPE="S3_Korvo_V2" ``` -------------------------------- ### Send Ordered Data Streams with LiveKit ESP32 SDK Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Open an outgoing data stream, write data in chunks (auto-chunked at 15000-byte boundaries), and then close the stream. Set `is_text` to `true` for UTF-8 content and `false` for binary data. For binary streams, specify `total_length` if known. ```c // Send a text message (e.g., chat) livekit_data_stream_options_t text_opts = { .topic = "lk.chat", .is_text = true }; livekit_data_stream_handle_t text_stream; livekit_room_data_stream_open(room_handle, &text_opts, &text_stream); livekit_room_data_stream_write(room_handle, text_stream, (const uint8_t *)"hello ", 6); livekit_room_data_stream_write(room_handle, text_stream, (const uint8_t *)"world", 5); livekit_room_data_stream_close(room_handle, text_stream); // Send a binary payload (e.g., a JPEG image) with known total length uint8_t image_data[4096]; size_t image_size = read_image(image_data, sizeof(image_data)); livekit_data_stream_options_t bin_opts = { .topic = "image", .is_text = false, .total_length = image_size, .has_total_length = true }; livekit_data_stream_handle_t bin_stream; if (livekit_room_data_stream_open(room_handle, &bin_opts, &bin_stream) == LIVEKIT_ERR_NONE) { livekit_room_data_stream_write(room_handle, bin_stream, image_data, image_size); livekit_room_data_stream_close(room_handle, bin_stream); } ``` -------------------------------- ### Create Audio-Only Room Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Allocates and configures a room object for audio-only bidirectional communication. The room is not yet connected after creation. Call livekit_room_destroy when done. Requires a state change callback. ```c #include "livekit.h" static livekit_room_handle_t room_handle = NULL; static void on_state_changed(livekit_connection_state_t state, void *ctx) { ESP_LOGI("app", "Room state: %s", livekit_connection_state_str(state)); if (state == LIVEKIT_CONNECTION_STATE_FAILED) { livekit_failure_reason_t reason = livekit_room_get_failure_reason(room_handle); ESP_LOGE("app", "Failure: %s", livekit_failure_reason_str(reason)); } } // Audio-only bidirectional room livekit_room_options_t options = { .publish = { .kind = LIVEKIT_MEDIA_TYPE_AUDIO, .audio_encode = { .codec = LIVEKIT_AUDIO_CODEC_OPUS, .sample_rate = 16000, .channel_count = 1 }, .capturer = media_get_capturer() // esp_capture_handle_t }, .subscribe = { .kind = LIVEKIT_MEDIA_TYPE_AUDIO, .renderer = media_get_renderer() // av_render_handle_t }, .on_state_changed = on_state_changed, .ctx = NULL }; livekit_err_t err = livekit_room_create(&room_handle, &options); if (err != LIVEKIT_ERR_NONE) { ESP_LOGE("app", "Failed to create room: %d", err); } ``` -------------------------------- ### Query LiveKit Connection State Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Retrieves the current connection state of the room and converts it into a human-readable string. If the state is `FAILED`, it also retrieves and logs the failure reason. ```c livekit_connection_state_t state = livekit_room_get_state(room_handle); ESP_LOGI("app", "Current state: %s", livekit_connection_state_str(state)); // Output: "Current state: Connected" // Inspect failure details when state == LIVEKIT_CONNECTION_STATE_FAILED if (state == LIVEKIT_CONNECTION_STATE_FAILED) { livekit_failure_reason_t reason = livekit_room_get_failure_reason(room_handle); ESP_LOGE("app", "Reason: %s", livekit_failure_reason_str(reason)); // Output examples: "Bad Token", "Unreachable", "Max Retries" } ``` -------------------------------- ### Register and Handle RPC Methods with LiveKit ESP32 SDK Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Register named methods for remote invocation and define handlers to process incoming RPC calls. Ensure to call `livekit_rpc_return_ok` or `livekit_rpc_return_error` within the handler. Unregister methods using `livekit_room_rpc_unregister` when no longer needed. ```c #include "cJSON.h" #include "livekit.h" // Handler: set an on-board LED color and state from a JSON payload static void set_led_state(const livekit_rpc_invocation_t *invocation, void *ctx) { if (invocation->payload == NULL) { livekit_rpc_return_error("Missing payload"); return; } cJSON *root = cJSON_Parse(invocation->payload); if (!root) { livekit_rpc_return_error("Invalid JSON"); return; } // Expected payload: {"color": "red", "state": true} const cJSON *color = cJSON_GetObjectItemCaseSensitive(root, "color"); const cJSON *state = cJSON_GetObjectItemCaseSensitive(root, "state"); if (!cJSON_IsString(color) || !cJSON_IsBool(state)) { cJSON_Delete(root); livekit_rpc_return_error("Unexpected JSON format"); return; } ESP_LOGI("app", "Set LED '%s' to %s", color->valuestring, cJSON_IsTrue(state) ? "ON" : "OFF"); cJSON_Delete(root); livekit_rpc_return_ok(NULL); } // Handler: return CPU temperature as a string static void get_cpu_temp(const livekit_rpc_invocation_t *invocation, void *ctx) { float temp = board_get_temp(); char buf[16]; snprintf(buf, sizeof(buf), "%.2f", temp); livekit_rpc_return_ok(buf); // caller receives e.g. "42.50" } // Register both handlers after creating the room livekit_room_rpc_register(room_handle, "set_led_state", set_led_state); livekit_room_rpc_register(room_handle, "get_cpu_temp", get_cpu_temp); // Unregister when no longer needed livekit_room_rpc_unregister(room_handle, "get_cpu_temp"); ``` -------------------------------- ### CMakeLists.txt Boilerplate for ESP-IDF Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/livekit/examples/minimal/CMakeLists.txt This boilerplate must be included in your project's CMakeLists.txt in the specified order for the build system to function correctly with ESP-IDF. ```cmake cmake_minimum_required(VERSION 3.5) set(COMPONENTS main) # Trim build include($ENV{IDF_PATH}/tools/cmake/project.cmake) project(minimal) ``` -------------------------------- ### Participant Info Callback Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Set `on_participant_info` in `livekit_room_options_t` to be notified when any participant joins, updates, or leaves. Participants are identified by kind and state. ```c static void on_participant_info(const livekit_participant_info_t *info, void *ctx) { // Detect when an AI agent becomes active if (info->kind == LIVEKIT_PARTICIPANT_KIND_AGENT && info->state == LIVEKIT_PARTICIPANT_STATE_ACTIVE) { ESP_LOGI("app", "Agent \'%s\' (sid=%s) joined the room", info->name, info->sid); } // Detect a specific named participant if (strncmp(info->name, "operator", 8) == 0 && info->state == LIVEKIT_PARTICIPANT_STATE_DISCONNECTED) { ESP_LOGW("app", "Operator disconnected"); } } livekit_room_options_t options = { // ... publish/subscribe ... .on_participant_info = on_participant_info, .ctx = NULL }; ``` -------------------------------- ### Fetch LiveKit Sandbox Token Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Contacts the LiveKit Cloud sandbox endpoint to obtain a temporary server URL and JWT access token. The token is valid for 15 minutes. Call `livekit_sandbox_res_free` to release the allocated strings. Requires the `livekit/sandbox_token` component. ```c #include "livekit_sandbox.h" livekit_sandbox_options_t opts = { .sandbox_id = "my-project-abc123", // from cloud.livekit.io .room_name = "demo-room", // NULL = auto-generated .participant_name = "esp32-device" // NULL = auto-generated }; livekit_sandbox_res_t res = {}; if (!livekit_sandbox_generate(&opts, &res)) { ESP_LOGE("app", "Sandbox token generation failed"); return; } ESP_LOGI("app", "Server: %s", res.server_url); ESP_LOGI("app", "Room: %s", res.room_name); livekit_room_connect(room_handle, res.server_url, res.token); livekit_sandbox_res_free(&res); // free server_url, token, room_name, participant_name ``` -------------------------------- ### Configure ESP-IDF Component Requirements Source: https://github.com/livekit/client-sdk-esp32/blob/main/components/example_utils/CMakeLists.txt Defines the base requirements for the component and conditionally adds Ethernet support for ESP32P4 targets. ```cmake set(component_requires esp_netif nvs_flash esp_wifi) if("${IDF_TARGET}" STREQUAL "esp32p4") list(APPEND component_requires ethernet_init esp_eth) endif() idf_component_register( SRC_DIRS ./src INCLUDE_DIRS ./include PRIV_REQUIRES ${component_requires} ) ``` -------------------------------- ### livekit_room_publish_data Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Publishes a low-level data packet to participants in the room with optional reliable or lossy delivery. ```APIDOC ## livekit_room_publish_data ### Description Sends a raw byte payload to one or more (or all) participants in the room. Supports reliable or lossy delivery. Useful for high-frequency telemetry, control commands, or any custom binary protocol. ### Method `livekit_room_publish_data(room_handle, opts)` ### Parameters - **room_handle**: Handle to the room. - **opts** (`livekit_data_publish_options_t*`): Configuration options for publishing data. - **payload** (`livekit_data_payload_t*`): The data payload to send. - **bytes** (`uint8_t*`): Pointer to the data buffer. - **size** (size_t): Size of the data buffer in bytes. - **topic** (string): The topic of the data packet. - **lossy** (bool): Whether to use lossy delivery (true) or reliable delivery (false). - **destination_identities** (char**): Array of participant identities to send the data to. If NULL or empty, sends to all participants. - **destination_identities_count** (size_t): The number of destinations in `destination_identities`. ### Request Example ```c // Send a G-code command to a specific participant ("printer-1") const char *command = "G5 I0 J3 P0 Q-3 X2 Y3"; livekit_data_payload_t payload = { .bytes = (uint8_t *)command, .size = strlen(command) }; livekit_data_publish_options_t opts = { .payload = &payload, .topic = "gcode", .lossy = false, // reliable delivery .destination_identities = (char *[]){ "printer-1" }, .destination_identities_count = 1 }; livekit_err_t err = livekit_room_publish_data(room_handle, &opts); if (err != LIVEKIT_ERR_NONE) { ESP_LOGE("app", "Failed to publish data: %d", err); } // Receive data packets — set on_data_received in room options: static void on_data_received(const livekit_data_received_t *data, void *ctx) { ESP_LOGI("app", "Received %zu bytes on topic '%s' from '%s'", data->payload.size, data->topic, data->sender_identity); } ``` ### Response Returns `LIVEKIT_ERR_NONE` on success, or an error code on failure. ``` -------------------------------- ### on_participant_info callback Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Callback function to detect when participants join, update, or leave a room. It provides information about the participant's kind and state. ```APIDOC ## on_participant_info callback — Detect room participants and agents Set `on_participant_info` in `livekit_room_options_t` to be notified when any participant joins, updates, or leaves. Participants are identified by kind (`STANDARD`, `AGENT`, `INGRESS`, `EGRESS`, `SIP`) and state (`JOINING`, `JOINED`, `ACTIVE`, `DISCONNECTED`). ### Callback Signature ```c void (*on_participant_info)(const livekit_participant_info_t *info, void *ctx) ``` ### Parameters #### `info` - **sid** (string) - The unique identifier of the participant. - **name** (string) - The display name of the participant. - **kind** (livekit_participant_kind_t) - The type of participant (e.g., `LIVEKIT_PARTICIPANT_KIND_STANDARD`, `LIVEKIT_PARTICIPANT_KIND_AGENT`). - **state** (livekit_participant_state_t) - The current state of the participant (e.g., `LIVEKIT_PARTICIPANT_STATE_JOINED`, `LIVEKIT_PARTICIPANT_STATE_DISCONNECTED`). #### `ctx` - A user-defined context pointer passed during room connection. ### Example Usage ```c static void on_participant_info(const livekit_participant_info_t *info, void *ctx) { // Detect when an AI agent becomes active if (info->kind == LIVEKIT_PARTICIPANT_KIND_AGENT && info->state == LIVEKIT_PARTICIPANT_STATE_ACTIVE) { ESP_LOGI("app", "Agent \'%s\' (sid=%s) joined the room", info->name, info->sid); } // Detect a specific named participant if (strncmp(info->name, "operator", 8) == 0 && info->state == LIVEKIT_PARTICIPANT_STATE_DISCONNECTED) { ESP_LOGW("app", "Operator disconnected"); } } livekit_room_options_t options = { // ... publish/subscribe ... .on_participant_info = on_participant_info, .ctx = NULL }; ``` ``` -------------------------------- ### livekit_room_close / livekit_room_destroy Source: https://context7.com/livekit/client-sdk-esp32/llms.txt Initiates an asynchronous disconnect from the room and frees all associated resources. ```APIDOC ## livekit_room_close / livekit_room_destroy ### Description `livekit_room_close` initiates an asynchronous disconnect. `livekit_room_destroy` frees all resources; call `close` first for a clean shutdown. ### Method - `livekit_room_close(room_handle)` - `livekit_room_destroy(room_handle)` ### Parameters - **room_handle**: Handle to the room. ### Request Example ```c void leave_room(void) { if (room_handle == NULL) return; livekit_err_t err = livekit_room_close(room_handle); if (err != LIVEKIT_ERR_NONE) { ESP_LOGE("app", "Close failed: %d", err); } err = livekit_room_destroy(room_handle); if (err != LIVEKIT_ERR_NONE) { ESP_LOGE("app", "Destroy failed: %d", err); return; } room_handle = NULL; } ``` ### Response Returns `LIVEKIT_ERR_NONE` on success, or an error code on failure. ```