RBLN SDK

https://docs.rbln.ai/latest/
Admin
RBLN SDK is a software stack that simplifies running deep learning workloads on Rebellions' neural...
Tokens:267,126
Snippets:2,745
Trust Score:2.5
Update:1 month ago
Show doc for...
Tokens:
Raw
### Example Log Configuration

Source: https://docs.rbln.ai/latest/software/system_management/kubernetes/install_npu_operator.html

A simple numeric list example.

```text
1
2
3
4
```

--------------------------------

### Run Development Setup Script

Source: https://docs.rbln.ai/latest/software/rbln_pytorch/installation.html

Execute the provided setup script to configure the environment.

```bash
./tools/dev-setup.sh pypi
```

--------------------------------

### Install optimum-rbln from source

Source: https://docs.rbln.ai/latest/software/optimum/installation.html

Clone the repository and use the provided uv-sync script to install the package and its dependencies.

```bash
$ git clone https://github.com/RBLN-SW/optimum-rbln.git
$ cd optimum-rbln
$ ./scripts/uv-sync.sh
```

--------------------------------

### Example Placeholder

Source: https://docs.rbln.ai/latest/software/optimum/model_api/transformers/text_model/common.html

A simple numeric example provided in the documentation.

```text
1
```

--------------------------------

### Install Helm

Source: https://docs.rbln.ai/latest/software/system_management/kubernetes/sandboxed_workloads.html

Download and install the Helm CLI tool using the official script.

```bash
$ curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 \
     && chmod 700 get_helm.sh \
     && ./get_helm.sh
```

--------------------------------

### Install Required Packages

Source: https://docs.rbln.ai/latest/software/model_serving/rayserve/tutorial/resnet50.html

Install Ray Serve, requests, and PyTorch dependencies.

```bash
pip install -U ray[serve] requests torch --extra-index-url https://download.pytorch.org/whl/cpu
```

--------------------------------

### Verify Install Plan

Source: https://docs.rbln.ai/latest/software/system_management/openshift/about_openshift.html

Command to check the status of the operator installation plan.

```bash
$ oc get installplan -n rbln-system
```

--------------------------------

### Start VM

Source: https://docs.rbln.ai/latest/software/system_management/virtualization/virtualization_methods/pci_passthrough.html

Launch the VM after the NPU device has been attached.

```bash
$ virsh start vm-name
```

--------------------------------

### Install RBLN SDK Components

Source: https://docs.rbln.ai/latest/supports/release_note.html

Use this command to install the required RBLN SDK packages on Debian-based Linux systems. Ensure the environment is configured according to the Installation Guide for non-default setups.

```bash
pip3 install --extra-index-url https://pypi.rbln.ai/simple --extra-index-url https://download.pytorch.org/whl/cpu --extra-index-url https://wheels.vllm.ai/0.22.0/cpu rebel-compiler==0.11.0.post1 optimum-rbln==0.11.0.post1 vllm-rbln==0.11.0
```

--------------------------------

### POST /start_profile

Source: https://docs.rbln.ai/latest/software/model_serving/vllm_support/features/profiler.html

Starts the profiling process for the vLLM server.

```APIDOC
## POST /start_profile

### Description
Starts the profiling process to begin recording performance metrics.

### Method
POST

### Endpoint
/start_profile

### Request Example
curl -X POST http://localhost:8000/start_profile
```

--------------------------------

### Basic sequence example

Source: https://docs.rbln.ai/latest/software/api/installation.html

A simple numerical sequence example provided in the documentation.

```text
1
2
3
```

--------------------------------

### Install Ray Serve and dependencies

Source: https://docs.rbln.ai/latest/software/model_serving/rayserve/index.html

Use pip to install Ray Serve along with necessary machine learning libraries.

```bash
$ pip3 install "ray[serve]" transformers requests torch --extra-index-url https://download.pytorch.org/whl/cpu
```

--------------------------------

### Manual Build

Source: https://docs.rbln.ai/latest/software/rbln_pytorch/installation.html

Manually sync dependencies and install the project in editable mode.

```bash
uv sync --no-install-project
# export TORCH_RBLN_BUILD_TYPE=Debug   # optional
uv pip install -e . --no-build-isolation
```

--------------------------------

### Build with CMake

Source: https://docs.rbln.ai/latest/software/api/language_binding/c/tutorial/image_classification.html

Commands to compile the C++ example project.

```bash
$ mkdir ${SAMPLE_PATH}/build  
$ cd ${SAMPLE_PATH}/build  
$ cmake ..  
$ make
```

--------------------------------

### Install RBLN SDK 2024.11.27

Source: https://docs.rbln.ai/latest/supports/release_note.html

Installation command for the 2024.11.27.0 SDK release.

```bash
pip3 install -i https://pypi.rbln.ai/simple rebel-compiler==0.6.1 optimum-rbln==0.1.13 vllm-rbln==0.1.2
```

--------------------------------

### Install RBLN SDK 2024.12.27

Source: https://docs.rbln.ai/latest/supports/release_note.html

Installation command for the 2024.12.27.0 SDK release.

```bash
pip3 install -i https://pypi.rbln.ai/simple rebel-compiler==0.6.2 optimum-rbln==0.1.15 vllm-rbln==0.1.3
```

--------------------------------

### Start TorchServe

Source: https://docs.rbln.ai/latest/software/model_serving/torchserve/tutorial/resnet50.html

Launches the TorchServe service with the specified configuration and model store.

```bash
$ torchserve --start --ncs \
        --ts-config ./config.properties \
        --model-store ./model_store \
        --models resnet50=resnet50.mar \
        --disable-token-auth
```

--------------------------------

### Install RBLN Packages

Source: https://docs.rbln.ai/latest/software/optimum/tutorial/llama3-8B.html

Install the required compiler and optimum-rbln packages from the RBLN and PyTorch repositories.

```bash
pip install \
  --extra-index-url https://pypi.rbln.ai/simple \
  rebel-compiler==0.11.2
pip install \
  --extra-index-url https://download.pytorch.org/whl/cpu \
  optimum-rbln==0.11.2
```

--------------------------------

### Installation Command

Source: https://docs.rbln.ai/latest/supports/release_note.html

Use this command to install the RBLN SDK components on Debian-based Linux systems.

```bash
pip3 install --extra-index-url https://pypi.rbln.ai/simple --extra-index-url https://download.pytorch.org/whl/cpu --extra-index-url https://wheels.vllm.ai/0.13.0/cpu rebel-compiler==0.10.2 optimum-rbln==0.10.2 vllm-rbln==0.10.2.post2
```

--------------------------------

### Installation Command

Source: https://docs.rbln.ai/latest/supports/release_note.html

Use this command to install the required Rebellions AI packages from the specified index URLs.

```bash
1
```

```bash
pip3 install --extra-index-url https://pypi.rbln.ai/simple --extra-index-url https://download.pytorch.org/whl/cpu --extra-index-url https://wheels.vllm.ai/0.18.0/cpu rebel-compiler==0.10.3 optimum-rbln==0.10.3 vllm-rbln==0.10.3.post1
```

--------------------------------

### Install TorchServe Dependencies

Source: https://docs.rbln.ai/latest/software/model_serving/torchserve/index.html

Commands to clone the TorchServe repository and install the required dependencies and packages.

```bash
$ git clone https://github.com/pytorch/serve.git
$ cd serve
$ python3 ./ts_scripts/install_dependencies.py
$ pip3 install torchserve torch-model-archiver torch-workflow-archiver
```

--------------------------------

### Deploy the NPU Pod

Source: https://docs.rbln.ai/latest/software/system_management/kubernetes/install_npu_operator.html

Apply the manifest to the cluster to start the pod.

```bash
$ kubectl apply -f npu-demo-pod.yaml
```

--------------------------------

### Install RBLN Device Plugin via Helm

Source: https://docs.rbln.ai/latest/software/system_management/kubernetes/device_plugin.html

Clone the repository and install the device plugin using Helm in the kube-system namespace.

```bash
$ git clone https://github.com/RBLN-SW/k8s-device-plugin
$ cd k8s-device-plugin/deployments/helm/rbln-device-plugin

$ helm install rbln-device-plugin . -n kube-system
```

--------------------------------

### Install optimum-rbln via PyPI

Source: https://docs.rbln.ai/latest/software/optimum/installation.html

Use this command to install the latest release of optimum-rbln from PyPI with the required PyTorch CPU index.

```bash
$ pip install \
  --extra-index-url https://download.pytorch.org/whl/cpu \
  optimum-rbln==0.11.2
```

--------------------------------

### Install RBLN SDK Components

Source: https://docs.rbln.ai/latest/supports/release_note.html

Use this command to install the rebel-compiler, optimum-rbln, and vllm-rbln packages from the RBLN repository.

```bash
pip3 install --extra-index-url https://pypi.rbln.ai/simple rebel-compiler==0.9.4 optimum-rbln==0.9.4 vllm-rbln==0.9.4
```

--------------------------------

### Install RBLN SDK Components

Source: https://docs.rbln.ai/latest/supports/release_note.html

Command to install the RBLN compiler and associated libraries from the RBLN package index.

```bash
pip3 install --extra-index-url https://pypi.rbln.ai/simple rebel-compiler==0.10.1 optimum-rbln==0.10.1 vllm-rbln==0.10.1
```

--------------------------------

### Install RBLN SDK via pip

Source: https://docs.rbln.ai/latest/supports/release_note.html

Install the RBLN SDK components using the specified package repositories.

```bash
pip3 install --extra-index-url https://pypi.rbln.ai/simple --extra-index-url https://download.pytorch.org/whl/cpu --extra-index-url https://wheels.vllm.ai/0.22.0/cpu rebel-compiler==0.11.1.post1 optimum-rbln==0.11.1 vllm-rbln==0.11.1.post1
```

--------------------------------

### Install RBLN Compiler

Source: https://docs.rbln.ai/latest/supports/release_note.html

Commands to install specific versions of the RBLN Compiler compatible with different driver versions.

```bash
pip install -i https://pypi.rbln.ai/simple rebel-compiler==0.3.11
```

```bash
pip install -i https://pypi.rbln.ai/simple rebel-compiler==0.5.0
```

```bash
pip install -i https://pypi.rbln.ai/simple rebel-compiler==0.4.0
```

--------------------------------

### Initialize Qwen2.5-VL-7B with vLLM

Source: https://docs.rbln.ai/latest/software/model_serving/vllm_support/tutorial/vllm_qwen2.5-vl-7B.html

Sets up the LLM instance with specific block sizes, sequence lengths, and RBLN hardware configuration for multi-device support.

```python
def main():
    model_id = "Qwen/Qwen2.5-VL-7B-Instruct"
    os.environ["VLLM_RBLN_NUM_DEVICES_PER_LOCAL_RANK"] = "8"
    llm = LLM(
        model=model_id,
        block_size=16_384,
        max_model_len=114_688,
        max_num_seqs=2,
        additional_config={
            "rbln_config": {
                "visual": {
                    "max_seq_lens": 6400,
                    "device": 0,
                },
                "decoder_batch_sizes": [2, 1],
                "device": [0, 1, 2, 3, 4, 5, 6, 7],
            }
        }
    )

    tokenizer = AutoTokenizer.from_pretrained(model_id)
    # ... (rest of main() unchanged)

if __name__ == "__main__":
    main()
```

--------------------------------

### Install RBLN SDK 2024.10.30

Source: https://docs.rbln.ai/latest/supports/release_note.html

Use this command to install the specific versions of the compiler, optimum-rbln, and vllm-rbln for the 2024.10.30 release.

```bash
pip3 install -i https://pypi.rbln.ai/simple rebel-compiler==0.5.12 optimum-rbln==0.1.12 vllm-rbln==0.1.0
```

--------------------------------

### Install RBLN SDK Components

Source: https://docs.rbln.ai/latest/supports/release_note.html

Use this command to install the RBLN compiler, Optimum RBLN, and vLLM RBLN packages from the official repositories.

```bash
pip3 install --extra-index-url https://pypi.rbln.ai/simple --extra-index-url https://download.pytorch.org/whl/cpu --extra-index-url https://wheels.vllm.ai/0.18.0/cpu rebel-compiler==0.10.4.post1 optimum-rbln==0.10.4 vllm-rbln==0.10.4
```

--------------------------------

### Define RBLNDriver Custom Resource

Source: https://docs.rbln.ai/latest/software/system_management/kubernetes/upgrade_driver.html

Example configuration for the RBLNDriver CRD to manage NPU driver installation and versioning.

```yaml
apiVersion: rebellions.ai/v1alpha1
kind: RBLNDriver
metadata:
  labels:
    app.kubernetes.io/name: rbln-driver
  name: rbln-driver
spec:
  registry: repo.rebellions.ai
  image: rebellions/rbln-driver
  version: "3.2.2"
  imagePullPolicy: IfNotPresent
  imagePullSecrets:
    - drivercred
  resources:
    requests:
      cpu: 250m
      memory: 64Mi
    limits:
      cpu: 500m
      memory: 128Mi
  manager:
    registry: docker.io
    image: rebellions/rbln-k8s-driver-manager
    version: v0.2.2
    imagePullPolicy: IfNotPresent
  smd:
    registry: repo.rebellions.ai
    image: rebellions/rbln-smd
```

--------------------------------

### Install RBLN software packages

Source: https://docs.rbln.ai/latest/supports/release_note.html

Use this command to install the required RBLN compiler and runtime packages from the custom index.

```bash
pip3 install --extra-index-url https://pypi.rbln.ai/simple rebel-compiler==0.10.0 optimum-rbln==0.10.0.post2 vllm-rbln==0.10.0
```

--------------------------------

### Install RBLN SDK 2024.09.27

Source: https://docs.rbln.ai/latest/supports/release_note.html

Use this command to install the specific versions of the compiler, optimum-rbln, and vllm-rbln for the 2024.09.27 release.

```bash
pip3 install -i https://pypi.rbln.ai/simple rebel-compiler==0.5.10 optimum-rbln==0.1.11 vllm-rbln==0.0.7
```

--------------------------------

### Install RBLN SDK 2025.03.28

Source: https://docs.rbln.ai/latest/supports/release_note.html

Use this command to install the specific versions of the RBLN compiler and libraries for the 2025.03.28 release.

```bash
pip3 install --extra-index-url https://pypi.rbln.ai/simple rebel-compiler==0.7.3 optimum-rbln==0.7.3.post2 vllm-rbln==0.7.3
```

--------------------------------

### Install RBLN SDK 2025.04.30

Source: https://docs.rbln.ai/latest/supports/release_note.html

Use this command to install the specific versions of the RBLN compiler and libraries for the 2025.04.30 release.

```bash
pip3 install --extra-index-url https://pypi.rbln.ai/simple rebel-compiler==0.7.4 optimum-rbln==0.7.4 vllm-rbln==0.7.4
```

--------------------------------

### Clone RBLN Model Zoo

Source: https://docs.rbln.ai/latest/getting_started/installation_guide.html

Use this command to clone the repository containing pre-trained model examples and implementation guides.

```bash
$ git clone --recursive https://github.com/RBLN-SW/rbln-model-zoo.git
```

--------------------------------

### Verify Installation with rbln-smi

Source: https://docs.rbln.ai/latest/getting_started/rbln_driver_installation_guide.html

Confirm the driver is loaded and NPUs are accessible.

```bash
$ rbln-smi
```

--------------------------------

### Install PyTorch RBLN Extension

Source: https://docs.rbln.ai/latest/getting_started/installation_guide.html

Install the required PyTorch version and the torch-rbln extension. Ensure rebel-compiler is installed as a prerequisite.

```bash
$ pip install \
  --index-url https://download.pytorch.org/whl/cpu \
  torch==2.11.0+cpu
$ pip install \
  torch-rbln==0.4.0
```

--------------------------------

### Install RBLN Packages

Source: https://docs.rbln.ai/latest/supports/release_note.html

Use this command to install the specific versions of the RBLN compiler, Optimum, and vLLM packages from the RBLN repository.

```bash
pip3 install --extra-index-url https://pypi.rbln.ai/simple rebel-compiler==0.9.2.post2 optimum-rbln==0.9.2 vllm-rbln==0.9.2.post1
```

--------------------------------

### Install RBLN SDK Components

Source: https://docs.rbln.ai/latest/supports/release_note.html

Use pip to install specific versions of the RBLN compiler and associated libraries from the private RBLN repository.

```bash
pip3 install -i https://pypi.rbln.ai/simple rebel-compiler==0.5.9 optimum-rbln==0.1.9 vllm-rbln==0.0.6
```

```bash
pip3 install -i https://pypi.rbln.ai/simple rebel-compiler==0.5.8 optimum-rbln==0.1.8 vllm-rbln==0.0.4
```

```bash
pip3 install -i https://pypi.rbln.ai/simple rebel-compiler==0.5.7 optimum-rbln==0.1.7 vllm-rbln==0.0.3
```

--------------------------------

### Install RBLN SDK via pip

Source: https://docs.rbln.ai/latest/supports/release_note.html

Use this command to install the required RBLN packages on Debian-based Linux distributions.

```bash
pip3 install --extra-index-url https://pypi.rbln.ai/simple --extra-index-url https://download.pytorch.org/whl/cpu --extra-index-url https://wheels.vllm.ai/0.24.0/cpu rebel-compiler==0.11.2 optimum-rbln==0.11.2 vllm-rbln==0.11.2.post1
```

--------------------------------

### Clone and Setup Repository

Source: https://docs.rbln.ai/latest/software/rbln_pytorch/installation.html

Clone the torch-rbln repository and initialize a virtual environment using uv.

```bash
git clone https://github.com/RBLN-SW/torch-rbln.git
cd torch-rbln
uv venv .venv && source .venv/bin/activate
uv pip install -U pip
```

--------------------------------

### Verify Installation

Source: https://docs.rbln.ai/latest/software/rbln_pytorch/installation.html

Check the installed version of torch_rbln.

```bash
python -c "import torch; import torch_rbln; print(torch_rbln.__version__)"
```

--------------------------------

### Install vLLM-RBLN

Source: https://docs.rbln.ai/latest/software/model_serving/vllm_support/index.html

Installs the vLLM-RBLN package in editable mode.

```bash
$ cd vllm-rbln
$ pip install -e .
$ cd ..
```

--------------------------------

### Verify Toolkit Setup

Source: https://docs.rbln.ai/latest/software/system_management/container_toolkit.html

Commands to list discovered CDI resources, view system information, and test NPU tools inside a container.

```bash
# Check what was discovered on the host
$ rbln-ctk cdi list

# View detected runtime and configuration
$ rbln-ctk info

# Use NPU tools inside a container
$ docker run --device rebellions.ai/npu=all -it ubuntu:22.04 rbln-smi
```

--------------------------------

### Install vLLM from Source

Source: https://docs.rbln.ai/latest/software/model_serving/vllm_support/index.html

Builds vLLM from source with the target device set to empty.

```bash
$ cd vllm/
$ VLLM_TARGET_DEVICE=empty pip install -e .
$ cd ..
```

--------------------------------

### Install Rebel Compiler

Source: https://docs.rbln.ai/latest/software/model_serving/vllm_support/tutorial/vllm_custom_kernel.html

Install the required rebel-compiler package from the RBLN repository.

```bash
pip install \
  --extra-index-url https://pypi.rbln.ai/simple \
  rebel-compiler==0.11.2
```

--------------------------------

### Start vLLM API server

Source: https://docs.rbln.ai/latest/software/model_serving/vllm_support/tutorial/openai_api_server.html

Execute this command to initialize the API server with a specific model.

```bash
vllm serve ./rbln-Llama-3-8B-Instruct
```

--------------------------------

### Verify Server Startup

Source: https://docs.rbln.ai/latest/software/model_serving/nvidia_triton_inference_server/tutorial/llama3.1-8B_flash_attention.html

Expected output messages indicating successful initialization of GRPC, HTTP, and Metrics services.

```text
Started GRPCInferenceService at 0.0.0.0:8001
Started HTTPService at 0.0.0.0:8000
Started Metrics Service at 0.0.0.0:8002
```

--------------------------------

### Check RBLN Driver Version

Source: https://docs.rbln.ai/latest/supports/release_note.html

Use this command to verify the currently installed RBLN Driver version.

```bash
rbln-stat -j | grep KMD_version
```

--------------------------------

### Install RBLN Container Toolkit on Ubuntu/Debian

Source: https://docs.rbln.ai/latest/software/system_management/container_toolkit.html

Updates the package index and installs the toolkit.

```bash
$ sudo apt-get update
$ sudo apt-get install rbln-container-toolkit
```

--------------------------------

### Launch Ray Serve Deployment

Source: https://docs.rbln.ai/latest/software/model_serving/rayserve/tutorial/yolov8.html

Use the Ray Serve CLI to start the YOLOv8 application.

```bash
$ serve run yolov8:app --name "yolov8"
```

--------------------------------

### Configure and run RBLN Container Toolkit

Source: https://docs.rbln.ai/latest/software/system_management/container.html

Basic configuration steps for initializing the RBLN Container Toolkit after installation.

```bash
1
2
3
4
5
6
7
8
9
```

--------------------------------

### __init__(cls_name=None, create_runtimes=None, device=None, device_map=None, activate_profiler=None, npu=None, num_devices=None, timeout=None, optimum_rbln_version=None, dtype=None, _compile_cfgs=None, *, optimize_host_memory=None, **kwargs)

Source: https://docs.rbln.ai/latest/software/optimum/model_api/transformers/common/common.html

Initializes a RBLN model configuration with runtime options and compile configurations.

```APIDOC
## __init__

### Description
Initialize a RBLN model configuration with runtime options and compile configurations.

### Parameters
- **cls_name** (str | None) - Optional - The class name of the configuration. Defaults to the current class name.
- **create_runtimes** (bool | None) - Optional - Whether to create RBLN runtimes. Defaults to True.
- **device** (int | list[int] | None) - Optional - The device(s) to load the model onto.
- **device_map** (dict[str, int | list[int]] | None) - Optional - Mapping from compiled model names to device IDs.
- **activate_profiler** (bool | None) - Optional - Whether to activate the profiler for performance analysis.
- **npu** (str | None) - Optional - The NPU device name to use for compilation.
- **num_devices** (int | None) - Optional - Number of devices to distribute the model across.
- **timeout** (int | None) - Optional - The timeout for the runtime in seconds. Defaults to 60.
- **optimum_rbln_version** (str | None) - Optional - The optimum-rbln version used for this configuration.
- **dtype** (str | dtype | None) - Optional - The data type to use for the model.
- **_compile_cfgs** (list[RBLNCompileConfig]) - Optional - List of compilation configurations for the model.
- **kwargs** (Any) - Optional - Additional keyword arguments.

### Errors
- **ValueError** - If unexpected keyword arguments are provided.
```

--------------------------------

### Install Llama3-8B Dependencies

Source: https://docs.rbln.ai/latest/software/profiler/examples/text_generation.html

Navigate to the model directory and install the required Python packages.

```bash
$ cd rbln-model-zoo/huggingface/transformers/text2text-generation/llama/llama3-8b
$ pip3 install -r requirements.txt
```

--------------------------------

### Install Kernel Headers

Source: https://docs.rbln.ai/latest/getting_started/rbln_driver_installation_guide.html

Install required kernel headers for DKMS-based driver builds.

```bash
$ sudo apt-get update
$ sudo apt-get install -y linux-headers-$(uname -r)
```

```bash
$ sudo dnf install -y kernel-headers-$(uname -r) kernel-devel-$(uname -r)
```

--------------------------------

### Start Triton Server

Source: https://docs.rbln.ai/latest/software/model_serving/nvidia_triton_inference_server/tutorial/llama3.1-8B_flash_attention.html

Launch the Triton Inference Server pointing to the model repository.

```bash
$ tritonserver --model-repository /PATH/TO/YOUR/vllm_backend/samples/model_repository
```

```bash
$ tritonserver --model-repository /opt/tritonserver/vllm_backend/samples/model_repository
```

--------------------------------

### Configure and Load RBLN Qwen3 Model

Source: https://docs.rbln.ai/latest/software/optimum/model_api/transformers/text_model/qwen3.html

Demonstrates how to initialize a configuration object and load a pre-trained Qwen3 model for RBLN NPU inference.

```python
from optimum.rbln import RBLNQwen3ForCausalLM, RBLNQwen3ForCausalLMConfig

# Create a configuration object
config = RBLNQwen3ForCausalLMConfig(
    batch_size=1,
    max_seq_len=40960,
    num_devices=4,
    kvcache_partition_len=16384
)

# Use the configuration with from_pretrained
model = RBLNQwen3ForCausalLM.from_pretrained(
    "Qwen/Qwen3-4B",
    export=True,
    rbln_config=config
)
```

--------------------------------

### Install RBLN SDK and dependencies

Source: https://docs.rbln.ai/latest/getting_started/quick_start.html

Install the rebel-compiler and vllm-rbln packages from the RBLN repository.

```bash
$ pip install \
  --extra-index-url https://pypi.rbln.ai/simple \
  rebel-compiler==0.11.2
$ pip install \
  --extra-index-url https://wheels.vllm.ai/0.24.0/cpu \
  --extra-index-url https://download.pytorch.org/whl/cpu \
  vllm-rbln==0.11.2.post1
```

--------------------------------

### Install RBLN SDK

Source: https://docs.rbln.ai/latest/software/api/language_binding/c/installation.html

Installs the rbln-sdk package using the apt-get package manager.

```bash
$ sudo apt-get install rbln-sdk
```

--------------------------------

### RBLN Runtime Usage Example

Source: https://docs.rbln.ai/latest/software/api/language_binding/c/api.html

Demonstrates model creation, input buffer management, synchronous inference, and resource cleanup.

```cpp
#include <rbln/rbln.h>
#include <vector>

/*
 * Alternatively, you can directly update the input buffer as shown below
 */
void UpdateBufferByIndex(RBLNRuntime *rt, int idx,
                         const std::vector<char> &buf) {
  // Gets a pointer to the input buffer at the specified index
  void *input_ptr = rbln_get_input(rt, idx);

  // Gets the layout info. of the input buffer at the specified index
  const RBLNTensorLayout *layout = rbln_get_input_layout(rt, idx);
  // Gets the number of bytes occupied for RBLNTensorLayout
  auto buf_size = rbln_get_layout_nbytes(layout);

  // If the input buffer has a shape of [1, 3, 340, 340] and a float
  // type, the buf_size will be 1 * 3 * 340 * 340 * 4 bytes.
  memcpy(input_ptr, buf.data(), buf_size);
}

void UpdateBufferByName(RBLNRuntime *rt, int idx,
                        const std::vector<char> &buf) {
  // Gets a name to the input buffer at the specified index
  const char *name = rbln_get_input_name(rt, idx);

  // Assigns input data to the input buffer at the specific name
  rbln_set_input_by_name(rt, name, buf.data());
}

/**
 * Alternatively, you can get the Output buffer as shown below
 */
std::vector<float> GetOutputBuffer(RBLNRuntime *rt, int idx) {
  auto buf_size = rbln_get_layout_nbytes(rbln_get_output_layout(rt, idx));
  std::vector<float> buf(buf_size / sizeof(float));

  void *data = rbln_get_output(rt, idx);
  memcpy(buf.data(), data, buf_size);
  return buf;
}

int main() {
  std::string model_path = "${MODEL_PATH}";

  // A sample array of a single dimension
  std::vector<char> sample_input(128, 0);

  // Creates an RBLN compiled model instance from a RBLN compiled model file
  RBLNModel *mod = rbln_create_model(model_path.c_str());

  // Creates an RBLN synchronous runtime from a compiled RBLN model
  RBLNRuntime *rt = rbln_create_runtime(mod, "default", 0, 0);

  // Gets the number of input buffers for the specified RBLN Runtime
  uint32_t n_in = rbln_get_num_inputs(rt);

  // Assigns input data to the input buffer at the specific index
  for (int idx = 0; idx < n_in; idx++) {
    rbln_set_input(rt, idx, sample_input.data());
    /**
     * Alternatively, you can update the input buffer with below
     * functions:
     * UpdateBufferByIndex(rt, idx, sample_input);
     * UpdateBufferByName(rt, idx, sample_input);
     */
  }

  // Triggers a synchronous inference for the RBLN Runtime
  rbln_run(rt);

  // Gets the number of output buffers for the specified RBLN Runtime
  uint32_t n_out = rbln_get_num_outputs(rt);

  // Gets a pointer to the output buffer at the specified index
  for (int idx = 0; idx < n_in; idx++) {
    void *outputs = rbln_get_output(rt, idx);

    /**
     * Alternatively, you can get the Output buffer with below function:
     * std::vector<float> out = GetOutputBuffer(rt, idx);
     */
  }

  // Destroys the specified RBLN Runtime handle
  rbln_destroy_runtime(rt);

  // Destroys the specified RBLN model handle.
  rbln_destroy_model(mod);

  return 0;
}

int main_async() {
  std::string model_path = "${MODEL_PATH}";

  // A sample array of a single dimension
  std::vector<float> sample_input(128, 0);

  // Creates an RBLN compiled model instance from a RBLN compiled model file
  RBLNModel *mod = rbln_create_model(model_path.c_str());

  // Creates an RBLN asynchronous runtime from a compiled RBLN model
  RBLNRuntime *rt = rbln_create_async_runtime(mod, "default", 0, 0);
```

--------------------------------

### Install Helm

Source: https://docs.rbln.ai/latest/software/system_management/kubernetes/install_npu_operator.html

Use this script to install Helm 3 if it is not already present or is below version 3.8.

```bash
$ curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 \
  && chmod 700 get_helm.sh \
  && ./get_helm.sh
```
Tokens:
Raw