DepthAI

https://docs.luxonis.com/software
Admin
DepthAI is a software ecosystem and API for Luxonis devices, enabling the development and deployment...
Tokens:14,570
Snippets:141
Trust Score:9.2
Update:8 months ago
Show doc for...
Tokens:
Raw
### Run DepthAI v3 Python Examples

Source: https://docs.luxonis.com/software/-v3/depthai

Executes example Python scripts for DepthAI v3 after installation. This includes running a detection network example (YoloV6) and a script to display all camera streams.

```bash
cd examples/python
# Run YoloV6 detection example
python3 DetectionNetwork/detection_network.py
# Display all camera streams
python3 Camera/camera_all.py
```

--------------------------------

### oakapp.toml Configuration Example

Source: https://docs.luxonis.com/software/-v3/oak-apps

This TOML snippet demonstrates the structure and common options within an oakapp.toml file. It includes required fields like 'identifier' and 'entrypoint', as well as optional sections for 'prepare_container' to install runtime dependencies and 'prepare_build_container' for build-time dependencies.

```toml
# (Required) App Identifier
identifier = "com.luxonis.python_demo"
# (Required) App Entrypoint
entrypoint = ["bash", "-c", "python3 /app/main.py"]

# (Optional) Prepare container commands
# Here is the place where you can install all the dependencies that are needed at run-time
prepare_container = [
    { type = "COPY", source = "requirements.txt", target = "requirements.txt" },
    { type = "RUN", command = "apt-get update" },
    { type = "RUN", command = "apt-get install -y python3-pip" },
    { type = "RUN", command = "pip3 install -r /app/requirements.txt --break-system-packages" },
]

# (Optional) Prepare build dependencies
# Here is the place where you can install all the dependencies that are needed at build-time
prepare_build_container = [
    # Example: npm, gcc, ...
]

# (Optional) Additional commands after all the app files are copied to the container
build_steps = []

```

--------------------------------

### Prepare Container with Dependencies (TOML)

Source: https://docs.luxonis.com/software/-v3/oak-apps/base-image

Specifies the steps to prepare the application's container environment. This includes copying necessary files like `requirements.txt` and installing Python dependencies using pip. These steps ensure that all required packages are available before the application starts.

```toml
1prepare_container = [
2    { type = "COPY", source = "./requirements.txt", target = "./requirements.txt" },
3    { type = "RUN", command = "python3.12 -m pip install -r /app/requirements.txt --break-system-packages"}
4]
```

--------------------------------

### Example oakapp.toml Configuration

Source: https://docs.luxonis.com/software/-v3/oak-apps/configuration

This TOML file defines the metadata, entrypoint, and preparation steps for a DepthAI application. It includes sections for installing dependencies and running build commands within a containerized environment.

```toml
1# (Required) App Identifier
2identifier = "com.luxonis.python_demo"
3# (Required) App Entrypoint
4entrypoint = ["bash", "-c", "python3 /app/main.py"]
5
6# (Optional) Prepare container commands
7# Here is the place where you can install all the dependencies that are needed at run-time
8prepare_container = [
9    { type = "COPY", source = "requirements.txt", target = "requirements.txt" },
10    { type = "RUN", command = "apt-get update" },
11    { type = "RUN", command = "apt-get install -y python3-pip" },
12    { type = "RUN", command = "pip3 install -r /app/requirements.txt --break-system-packages" },
13]
14
15# (Optional) Prepare build dependencies
16# Here is the place where you can install all the dependencies that are needed at build-time
17prepare_build_container = [
18    # Example: npm, gcc, ...
19]
20
21# (Optional) Additional commands after all the app files are copied to the container
22build_steps = []

```

--------------------------------

### TOML Configuration for Frontend Build

Source: https://docs.luxonis.com/software/-v3/oak-apps/base-image

This TOML snippet defines the build process for a static frontend application. It specifies steps to install Node.js using nvm, set up service directories, copy backend run scripts, and build the React.js frontend using npm. The build process includes installing dependencies and creating a production build.

```toml
1prepare_build_container = [{type = "RUN", command = "bash -c 'curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash && . \"$HOME/.nvm/nvm.sh\" && nvm install 24'"}]
2build_steps = [
    "mkdir -p /etc/service/backend",
    "cp /app/backend-run.sh /etc/service/backend/run",
    "chmod +x /etc/service/backend/run",
]
7
8[static_frontend]
9dist_path = "./frontend/dist"
10
11[static_frontend.build]
12source_path = "./frontend"
13steps = ["bash -c 'cd /app/frontend/src && export NODE_OPTIONS=--max-old-space-size=4096 && . $HOME/.nvm/nvm.sh && npm install && npm run build'"]

```

--------------------------------

### Install DepthAI Core Packages and Template App

Source: https://docs.luxonis.com/software/-v3/template

Installs the core DepthAI Python package and clones the template application repository. It then navigates into the cloned directory and installs the necessary Python requirements for the template app.

```bash
pip install depthai --force-reinstall
git clone https://github.com/luxonis/oak-template.git
cd oak-template
pip install -r requirements.txt
```

--------------------------------

### Install pip on Luxonis OS

Source: https://docs.luxonis.com/software/-v3/sw-stack/luxonis-os

This command installs the 'pip' package manager on the Luxonis OS if it is not already present. It uses the python3 module to ensure pip is available.

```bash
python3 -m ensurepip
```

--------------------------------

### Build and Run OAK Apps with oakctl

Source: https://docs.luxonis.com/software/-v3/oak-apps

Demonstrates how to use the `oakctl` tool to run an app directly from source, build it into a distributable package (`.oakapp`), and install the built package onto a device.

```bash
# Run an app directly from source
oakctl app run ./my-oak-app

# Build the app into a distributable package
oakctl app build ./my-oak-app

# Install a built app
oakctl app install my-oak-app.oakapp

```

--------------------------------

### Manage OAK Apps with oakctl

Source: https://docs.luxonis.com/software/-v3/oak-apps

Provides essential `oakctl` commands for managing installed OAK Apps, including listing, starting, stopping, viewing logs, and enabling auto-start functionality.

```bash
# List installed apps
oakctl app list

# Start an app
oakctl app start <container-id>

# View app logs
oakctl app logs <container-id>

# Stop an app
oakctl app stop <container-id>

# Enable auto-start
oakctl app enable <container-id>

```

--------------------------------

### Configure Default App Settings (TOML)

Source: https://docs.luxonis.com/software/-v3/oak-apps/base-image

Defines the core settings for an OAK4 application, including its unique identifier, the command to execute on startup (entrypoint), the application version, and whether to assign a frontend port. The entrypoint typically starts a service manager that handles internal services and application scripts.

```toml
1identifier = "com.example.default"
2entrypoint = ["bash", "-c", "/usr/bin/runsvdir -P /etc/service"]
3app_version = "1.0.0"
4assign_frontend_port = true
```

--------------------------------

### Install DepthAI v3 on Linux/MacOS

Source: https://docs.luxonis.com/software/-v3/depthai

Installs the DepthAI v3 library and its requirements on Linux and macOS systems using Python virtual environments and pip. It clones the repository, sets up a virtual environment, activates it, and then installs the necessary packages.

```bash
1git clone https://github.com/luxonis/depthai-core.git && cd depthai-core
2python3 -m venv venv
3source venv/bin/activate
4# Installs library and requirements
5python3 examples/python/install_requirements.py
```

```bash
pip install depthai --force-reinstall
```

--------------------------------

### Template App Configuration File

Source: https://docs.luxonis.com/software/-v3/template

This TOML file defines the build process for the application's container. It specifies package installations, build steps, DepthAI model configurations, and the entrypoint command for running the application.

```toml
identifier = "custom.oakapp"
app_version = "1.0.0"

prepare_container = [
    { type = "RUN", command = "apt-get update" },
    { type = "RUN", command = "apt-get install -y python3 python3-pip wget git" },
]

prepare_build_container = []

build_steps = ["pip3 install -r /app/requirements.txt --break-system-packages"]

depthai_models = { yaml_path = "./depthai_models" }

entrypoint = ["bash", "-c", "python3 /app/main.py"]
```

--------------------------------

### Define Runtime Container Commands (TOML)

Source: https://docs.luxonis.com/software/-v3/oak-apps/configuration

Specifies commands to be executed within the container to prepare the runtime environment. Supports 'RUN' commands for package installation and updates.

```toml
prepare_container = [
    { type = "RUN", command = "apt-get update" },
    { type = "RUN", command = "apt-get install -y python3-pip" },
]
```

--------------------------------

### Default Camera + NN + Hub Pipeline in Python

Source: https://docs.luxonis.com/software/-v3/template

This Python script sets up a DepthAI pipeline that includes a camera input, a neural network inference using a specified model, and a visualizer for streaming output. It demonstrates how to create nodes, link them, and start the pipeline for real-time processing and visualization.

```python
import os
from pathlib import Path

import depthai as dai
from utils.snaps_producer import SnapsProducer
from depthai_nodes.node import ParsingNeuralNetwork


visualizer = dai.RemoteConnection(httpPort=8082)
device = dai.Device()

with dai.Pipeline(device) as pipeline:
    print("Creating pipeline...")
    platform = device.getPlatform()
    model_description = dai.NNModelDescription.fromYamlFile(
        f"yolov6_nano_r2_coco.{platform.name}.yaml"
    )
    nn_archive = dai.NNArchive(dai.getModelFromZoo(modelDescription=model_description))

    input_node = pipeline.create(dai.node.Camera).build()

    nn_with_parser = pipeline.create(ParsingNeuralNetwork).build(input_node, nn_archive)

    visualizer.addTopic("Video", nn_with_parser.passthrough, "images")
    visualizer.addTopic("Visualizations", nn_with_parser.out, "images")

    snaps_producer = pipeline.create(SnapsProducer).build(
        nn_with_parser.passthrough,
        nn_with_parser.out,
        label_map=nn_archive.getConfigV1().model.heads[0].metadata.classes,
    )

    print("Pipeline created.")

    pipeline.start()
    visualizer.registerPipeline(pipeline)

    while pipeline.isRunning():
        key = visualizer.waitKey(1)
        if key == ord("q"):
            print("Got q key from the remote connection!")
            break
```

--------------------------------

### Install oakctl CLI (Linux/MacOS)

Source: https://docs.luxonis.com/software/-v3/oak-apps

Installs the `oakctl` command-line tool on 64-bit Linux or MacOS systems using a curl script. This tool is essential for managing OAK Apps.

```bash
bash -c "$(curl -fsSL https://oakctl-releases.luxonis.com/oakctl-installer.sh)"
```

--------------------------------

### Check Luxonis OS Version

Source: https://docs.luxonis.com/software/-v3/sw-stack/luxonis-os

This command line snippet allows you to check the current version of Luxonis OS installed on your device. It requires connecting to the device via SSH or ADB.

```bash
cat /etc/os-release
```

--------------------------------

### Avoid High CPU Usage with tryGet() in Tight Loops (Python)

Source: https://docs.luxonis.com/software/-v3/troubleshooting/my-app-is-slow

Demonstrates how to prevent excessive CPU usage when using the non-blocking `tryGet()` method in a tight loop. It provides two solutions: adding a small sleep interval or using the blocking `get()` method instead. This is crucial for efficient resource management in DepthAI applications.

```Python
1while True:
2    queue.tryGet()
```

```Python
1#!/usr/bin/env python3
2import depthai as dai
3import time
4with dai.Pipeline() as pipeline:
5    cam = pipeline.create(dai.node.Camera).build(
6        dai.CameraBoardSocket.CAM_A, sensorFps=19.0
7    )
8    rawQueue = cam.raw.createOutputQueue()
9    pipeline.start()
10    while pipeline.isRunning():
11        rawFrame = rawQueue.tryGet()
12        if rawFrame is not None:
13            print("Got a raw frame")
14        time.sleep(0.001)  # prevents 100% CPU usage
```

```Python
1#!/usr/bin/env python3
2import depthai as dai
3import time
4with dai.Pipeline() as pipeline:
5    cam = pipeline.create(dai.node.Camera).build(
6        dai.CameraBoardSocket.CAM_A, sensorFps=19.0
7    )
8    rawQueue = cam.raw.createOutputQueue()
9    pipeline.start()
10    while pipeline.isRunning():
11        rawFrame = rawQueue.get()
12        print("Got a raw frame")
```

--------------------------------

### Define Backend Service Build Steps (TOML)

Source: https://docs.luxonis.com/software/-v3/oak-apps/base-image

Configures the build steps for registering a backend service. This involves creating a directory for the service, copying the service's run script (e.g., `backend-run.sh`), and ensuring the script has execute permissions. This script is responsible for launching the application's backend.

```toml
1build_steps = [
2    "mkdir -p /etc/service/backend",
3    "cp /app/backend-run.sh /etc/service/backend/run",
4    "chmod +x /etc/service/backend/run",
5]
```

--------------------------------

### Run Docker Commands for Local Registry (Bash)

Source: https://docs.luxonis.com/software/-v3/oak-apps/configuration

Commands to set up and use a local Docker registry for building and testing OAK Apps. This includes running the Zot registry, disabling TLS, and building a custom base image.

```bash
docker run -d -p 5000:5000 --name zot ghcr.io/project-zot/zot-linux-amd64:latest

# Only for local testing with insecure registry
regctl registry set --tls=disabled localhost:5000
docker buildx create --use --driver docker-container --driver-opt network=host

# Create a simple base image
echo "FROM debian:bookworm-slim" > ./Dockerfile

# set IP address of your local registry if not running on localhost
docker buildx build -f ./Dockerfile --platform=linux/arm64 -t 127.0.0.1:5000/my-base:local --push .
```

--------------------------------

### Define Application Build Steps (TOML)

Source: https://docs.luxonis.com/software/-v3/oak-apps/configuration

Lists the commands to be executed within the container after the application files are copied. These are typically build commands for compiling or preparing the application.

```toml
build_steps = [
    "g++ -o /app/my_app /app/main.cpp",
    "chmod +x /app/my_app",
]
```

--------------------------------

### Configure Static Frontend Build (TOML)

Source: https://docs.luxonis.com/software/-v3/oak-apps/configuration

Sets up the configuration for building and serving a static web frontend alongside the application. It includes the path to the built files and optional build configurations like source path and build steps.

```toml
[static_frontend]
dist_path = "./frontend/dist" # path to the built frontend files

[static_frontend.build]
source_path = "./frontend" # path to the frontend source files
steps = ["bash -c 'cd /app/frontend/src && npm install && npm run build'"]
```

--------------------------------

### Configure Additional Mounts with Options (TOML)

Source: https://docs.luxonis.com/software/-v3/oak-apps/configuration

Allows for detailed configuration of additional mounts, specifying source, target, type, options (like 'rbind', 'rw', 'bind', 'ro'), and whether the mount is required. This provides fine-grained control over how directories and devices are linked into the container.

```toml
additional_mounts = [
    { source = "/run/user/1000/pulse", target = "/run/user/1000/pulse", type = "none", options = [
        "rbind",
        "rw",
    ], required = true },
    { source = "/home/root/.config/pulse/cookie", target = "/root/.config/pulse/cookie", type = "none", options = [
        "bind",
        "ro",
    ], required = true },
]
```

--------------------------------

### Configure Optional Mounts (TOML)

Source: https://docs.luxonis.com/software/-v3/oak-apps/configuration

Defines directories or devices that are optional for an application. If these mounts are not present on container startup, they are simply ignored. The syntax is `source[:target[:options]]`.

```toml
optional_mounts = [
    "/data/optional-storage:/app/optional:rw",
]
```

--------------------------------

### Configure Required Mounts (TOML)

Source: https://docs.luxonis.com/software/-v3/oak-apps/configuration

Specifies directories or devices that must be available both at build and run time for an application. If these mounts are not present on container startup, the app will stop with an error. The syntax is `source[:target[:options]]`.

```toml
required_mounts = [
    "/data/my-storage:/app/storage:rw",
]
```

--------------------------------

### Deploy Template App Standalone

Source: https://docs.luxonis.com/software/-v3/template

Command to deploy the template application to an OAK device in standalone mode. This command containerizes the application and deploys it, making results viewable via a web visualizer.

```bash
oakctl app run .
```

--------------------------------

### Run App with oakctl (Shell)

Source: https://docs.luxonis.com/software/-v3/oak-apps/apikey-good-practices

Illustrates the command to run a local application using the oakctl tool. This command assumes that the necessary environment variables, such as DEPTHAI_HUB_API_KEY, are correctly configured, potentially loaded from a .env file.

```bash
1oakctl app run .
```

--------------------------------

### Run Template App in Peripheral Mode

Source: https://docs.luxonis.com/software/-v3/template

Command to run the template application directly from the host machine in peripheral mode. This bypasses containerization but requires a stable connection to the OAK device. Results are viewable via a web visualizer.

```bash
python3 main.py
```

--------------------------------

### Define Runtime Copy Commands (TOML)

Source: https://docs.luxonis.com/software/-v3/oak-apps/configuration

Specifies commands to copy files from the host application directory into the container during the runtime preparation phase. Useful for dependency files like requirements.txt.

```toml
prepare_container = [
    { type = "COPY", source = "requirements.txt", target = "requirements.txt" },
]
```

--------------------------------

### Specify Base Docker Image (TOML)

Source: https://docs.luxonis.com/software/-v3/oak-apps/base-image

Defines the base Docker image for the OAK4 application. It includes the registry API URL, service name, OAuth token URL, authentication type, repository name, and image tag. Using the recommended `luxonis/oakapp-base` image ensures compatibility and includes necessary dependencies.

```toml
1[base_image]
2api_url = "https://registry-1.docker.io"
3service = "registry.docker.io"
4oauth_url = "https://auth.docker.io/token"
5auth_type = "repository"
6auth_name = "luxonis/oakapp-base"
7image_name = "luxonis/oakapp-base"
8image_tag = "1.2.5"
```

--------------------------------

### Manage OAK Devices with oakctl

Source: https://docs.luxonis.com/software/-v3/oak-apps

Shows how to use `oakctl` to discover connected OAK devices on the network and establish a connection to a specific device using its IP address.

```bash
# List connected devices
oakctl list

# Connect to a specific device
oakctl connect <device-ip>

```

--------------------------------

### Enable Frontend Port Assignment (TOML)

Source: https://docs.luxonis.com/software/-v3/oak-apps/configuration

Configures the system to automatically assign an available port to the `OAKAPP_STATIC_FRONTEND_PORT` environment variable within the container. This is useful for accessing the static frontend during development or runtime.

```toml
assign_frontend_port = true
```

--------------------------------

### Configure Base Docker Image (TOML)

Source: https://docs.luxonis.com/software/-v3/oak-apps/configuration

Specifies the base Docker image for the OAK App container. This includes registry details, image name, and tag. It can be configured to use Docker Hub or a local registry.

```toml
[base_image]
api_url = "https://registry-1.docker.io" # Service https address
service = "registry.docker.io" # Name of Image Registry Service
oauth_url = "https://auth.docker.io/token" # address to oauth 2.0 token for this service
auth_type = "repository" # scope type
auth_name = "library/debian" # name of resource

image_name = "library/debian"
image_tag = "bookworm-slim"
```

```toml
[base_image]
image_name = "library/debian"
image_tag = "bookworm-slim"
```

```toml
[base_image]
 api_url = "http://localhost:5000" # Service http address visible from the device
 image_name = "my-base"
 image_tag = "local"
```

--------------------------------

### Update Luxonis OS using oakctl (Host)

Source: https://docs.luxonis.com/software/-v3/sw-stack/luxonis-os

These commands update the oakctl tool itself and then initiate an OS update on the connected device from your host machine. This method preserves user data and does not require SSH/ADB connection during the update process. Compatibility is limited to OAK4 devices with OS 1.11 and later when updating via Ethernet.

```bash
oakctl self-update
oakctl device update
```

--------------------------------

### Configure Environment Variables (TOML)

Source: https://docs.luxonis.com/software/-v3/oak-apps/configuration

Sets environment variables that will be available to the application at runtime. This is a standard way to configure application behavior without modifying the code.

```toml
[env]
MY_ENV_VAR = "some_value"
ANOTHER_ENV_VAR = "another_value"
```

--------------------------------

### Set DEPTHAI_HUB_API_KEY in .env file (Shell)

Source: https://docs.luxonis.com/software/-v3/oak-apps/apikey-good-practices

Provides shell commands to create a .env file containing the DEPTHAI_HUB_API_KEY and add '.env' to the .gitignore file. This ensures the API key is stored locally and not committed to version control. Replace placeholders with your actual path and API key.

```bash
cd <INSERT_PATH_TO_PROJECT>
echo "DEPTHAI_HUB_API_KEY=<INSERT_YOUR_DEPTHAI_HUB_API_KEY>" > .env
echo ".env" >> .gitignore
```

--------------------------------

### Specify Files to Ignore (Plain Text)

Source: https://docs.luxonis.com/software/-v3/oak-apps/configuration

Defines a list of files or directories to be ignored during the container build process. The syntax is similar to .gitignore.

```plaintext
# Ignore all .log files
*.log
# Ingore node_modules/ directory
node_modules/
```

--------------------------------

### OAK App Project Structure

Source: https://docs.luxonis.com/software/-v3/oak-apps

Defines the basic file structure for an OAK App project. Includes the main Python script, dependency list, optional ignore file, and the required TOML configuration file.

```text
1my-oak-app/
2├── .oakappignore      # (Optional) Files to ignore when building the app on the device
3├── main.py            # Your primary application code
4├── requirements.txt   # Python dependencies
5└── oakapp.toml        # App configuration

```

--------------------------------

### Configure Allowed Devices (TOML)

Source: https://docs.luxonis.com/software/-v3/oak-apps/configuration

Specifies which devices the application is permitted to access within the container. This is a security measure to restrict device access.

```toml
allowed_devices = [{ allow = true, access = "rwm" }]
```

--------------------------------

### Load Environment Variables with dotenv (Python)

Source: https://docs.luxonis.com/software/-v3/oak-apps/apikey-good-practices

Demonstrates how to load environment variables, including the DEPTHAI_HUB_API_KEY, from a .env file using the dotenv library in Python. This is useful for overriding default configurations or when the API key is not set globally. Ensure the .env file is excluded from version control.

```python
import os
from dotenv import load_dotenv

load_dotenv(override=True)

print(os.environ["DEPTHAI_HUB_API_KEY"])
```

--------------------------------

### Deploy Custom NN Archive Model with DepthAI v3

Source: https://docs.luxonis.com/software/-v3/depthai

Sets up the DepthAI v3 pipeline to deploy a custom AI model packaged in an `archive.tar.xz` file. It configures a camera node to request specific input size and type, then links it to a NeuralNetwork node using the NN archive.

```python
cam = pipeline.create(dai.node.Camera).build(socket)
# If your nn model requires 640x640 input size (BGR):
cam_out = cam.requestOutput((640, 640), dai.ImgFrame.Type.BGR888p)

nn_archive = dai.NNArchive('./my_nn_archive.tar.xz')
nn = pipeline.create(dai.node.NeuralNetwork).build(cam_out, nn_archive)
```

--------------------------------

### Configure DepthAI Models Cache (TOML)

Source: https://docs.luxonis.com/software/-v3/oak-apps/configuration

Specifies a folder containing DepthAI models in YAML format to be cached within the container at build time. This allows for offline operation by preventing runtime model downloads. It also includes an option to exclusively use cached models.

```toml
depthai_models = { yaml_path = "./backend/src/depthai_models" }
```

--------------------------------

### Enable SSH on Luxonis OS (1.18+)

Source: https://docs.luxonis.com/software/-v3/sw-stack/luxonis-os

This sequence enables SSH access on newer Luxonis OS versions (1.18 and later) where it's disabled by default. It involves pushing a configuration file and rebooting the device. Password-based SSH login can be optionally disabled after connection.

```bash
oakctl adb root && oakctl adb push sshd_config /etc/ssh/sshd_config
```

--------------------------------

### Deploy Custom DLC Model with DepthAI v3

Source: https://docs.luxonis.com/software/-v3/depthai

Configures the DepthAI v3 pipeline to deploy a custom AI model in `.dlc` format using the SNPE backend. It specifies the model path, backend, and backend properties for runtime and performance.

```python
nn = pipeline.create(dai.node.NeuralNetwork)
nn.setModelPath('my_model.dlc')
nn.setBackend("snpe") # Specify SNPE NN backend. This usually gets set under the hood
# Specify SNPE (RVC4) specific settings, like DSP runtime and NN performance profile
nn.setBackendProperties({"runtime": "dsp", "performance_profile": "default"})
```
Tokens:
Raw