Ryzen AI Software

https://ryzenai.docs.amd.com/en/latest/
Admin
Ryzen AI Software provides tools and runtime libraries for optimizing and deploying AI inference on...
Tokens:49,843
Snippets:648
Trust Score:9.9
Update:5 months ago
Show doc for...
Tokens:
Raw
### Example Output of WinML Setup Checker

Source: https://ryzenai.docs.amd.com/en/latest/winml/installation.html

This is an example output from the `check_winml_setup.py` script, showing the status of Python, WASDK packages, and the Windows App SDK Runtime. It helps identify any version mismatches.

```text
============================================================
WinML Setup Checker
============================================================
Python: 3.11.0 (<path_to_python_installation>\python.exe)
WASDK Python Packages:
----------------------------------------
  [✓] wasdk-ML: 2.0.0.dev4
  [✓] wasdk-Bootstrap: 2.0.0.dev4
Windows App SDK Runtime:
----------------------------------------
[✓] Windows App SDK: 2.0-experimental5 (internal: 0.770.2319.0)
Installed runtimes (newest first):
    * 2.0-experimental5 (internal: 0.770.2319.0)
    - 2.0-experimental4 (internal: 0.738.2207.0)
    - 1.8 (internal: 8000.642.119.0)
    - 1.8 (internal: 8000.675.1142.0)
    - 1.8-experimental (internal: 8000.589.1529.0)
    - 1.8-preview (internal: 8000.591.1127.0)
    * Active runtime used by this checker
Expected SDK: 2.0.0-experimental4
============================================================
Status: All components installed. Please, ensure matching Windows App SDK version is Installed.

```

--------------------------------

### Ryzen AI Installation Test Output

Source: https://ryzenai.docs.amd.com/en/latest/inst.html

Example output from the `quicktest.py` script, indicating successful initialization of the ONNX Runtime session.

```text
[I:onnxruntime:, session_state_utils.cc:243 onnxruntime::session_state_utils::SaveInitializedTensors] Saving initialized tensors.
[I:onnxruntime:, session_state_utils.cc:438 onnxruntime::session_state_utils::SaveInitializedTensors] Done saving initialized tensors
[I:onnxruntime:, inference_session.cc:2532 onnxruntime::InferenceSession::Initialize] Session successfully initialized.
Test Finished

```

--------------------------------

### Clone RyzenAI-SW Repository and Install Dependencies

Source: https://ryzenai.docs.amd.com/en/latest/winml/installation.html

Clones the RyzenAI-SW repository and installs the necessary Python packages listed in 'requirements.txt' for the ResNet example. Ensure you are in the correct directory before running.

```bash
git clone https://github.com/amd/RyzenAI-SW.git
cd <RyzenAI-SW>\WinML\CNN\ResNet
pip install --pre -r .\requirements.txt
```

--------------------------------

### Install OpenCV from Source for C++ Deployment

Source: https://ryzenai.docs.amd.com/en/latest/getstartex.html

Build and install OpenCV from source code, ensuring a static build and specifying the installation prefix. This is a prerequisite for C++ deployment examples.

```bash
git clone https://github.com/opencv/opencv.git -b 4.6.0
cd opencv
cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DBUILD_SHARED_LIBS=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_CONFIGURATION_TYPES=Release -A x64 -T host=x64 -G "Visual Studio 17 2022" "-DCMAKE_INSTALL_PREFIX=C:\opencv" "-DCMAKE_PREFIX_PATH=C:\opencv" -DCMAKE_BUILD_TYPE=Release -DBUILD_opencv_python2=OFF -DBUILD_opencv_python3=OFF -DBUILD_WITH_STATIC_CRT=OFF -B build
cmake --build build --config Release
cmake --install build --config Release

```

--------------------------------

### Set Up Conda Environment and Install Requirements

Source: https://ryzenai.docs.amd.com/en/latest/getstartex.html

Create and activate a new conda environment cloned from the base Ryzen AI installation, then install the necessary Python packages using the requirements.txt file.

```bash
conda create --name resnet_env --clone ryzen-ai-<version>
conda activate resnet_env
python -m pip install -r requirements.txt
```

--------------------------------

### Specify VitisAIExecutionProvider for Model Execution (Python)

Source: https://ryzenai.docs.amd.com/en/latest/winml/winml_ep.html

This Python example shows how to get available execution devices, filter for VitisAI NPU devices, and configure session options to use the VitisAIExecutionProvider with a specified configuration file.

```python
# 1. Enumerate and filter EP devices
ep_devices = ort.get_ep_devices()
selected_ep_devices = [
    d for d in ep_devices
    if d.ep_name == "VitisAIExecutionProvider"
    and d.device.type == ort.OrtHardwareDeviceType.NPU]

if not selected_ep_devices:
    raise RuntimeError("VitisAIExecutionProvider is not available on this system.")

# 2. Configure provider-specific options in "vaiml_config.json" file
session_options = ort.SessionOptions()
provider_options = {'config_file':'vaiml_config.json'}
session_options.add_provider_for_devices([selected_ep_devices[0]], provider_options)
```

--------------------------------

### Activate Virtual Environment and Display Installation Path

Source: https://ryzenai.docs.amd.com/en/latest/llm_linux.html

Activates the Python virtual environment and prints the Ryzen AI installation path. Replace <TARGET-PATH> with your actual installation directory.

```bash
# Activate the virtual environment created in Linux Installation step
source <TARGET-PATH>/venv/bin/activate

echo $RYZEN_AI_INSTALLATION_PATH
```

--------------------------------

### Environment Setup

Source: https://ryzenai.docs.amd.com/en/latest/llm/high_level_python.html

Commands to create and activate a Python environment for the Ryzen AI LLM SDK.

```APIDOC
conda create -n ryzenai-llm python=3.12
conda activate ryzenai-llm
pip install lemonade-sdk[dev,oga-ryzenai] --extra-index-url=https://pypi.amd.com/simple
```

--------------------------------

### Download and Install Windows App SDK Runtime

Source: https://ryzenai.docs.amd.com/en/latest/winml/installation.html

Downloads a specific version of the Windows App SDK runtime installer and installs it quietly. This step is crucial for the Windows ML execution providers to function correctly.

```bash
curl -L -o windowsappruntimeinstall-x86.exe "https://aka.ms/windowsappsdk/2.0/2.0.0-experimental4/windowsappruntimeinstall-x86.exe"
windowsappruntimeinstall-x86.exe --quiet
```

--------------------------------

### Install and Activate Ryzen AI Virtual Environment

Source: https://ryzenai.docs.amd.com/en/latest/linux.html

Installs the Ryzen AI package to a target path and activates the created Python virtual environment. Replace <TARGET-PATH> with your desired installation directory.

```bash
./install_ryzen_ai.sh -a yes -p <TARGET-PATH>/venv
source <TARGET-PATH>/venv/bin/activate
```

--------------------------------

### Run Windows ML Setup Checker Script

Source: https://ryzenai.docs.amd.com/en/latest/winml/installation.html

Executes a Python script to verify the Windows ML installation and check for compatibility issues between the installed Python packages and the Windows App SDK runtime.

```bash
cd <RyzenAI-SW>\WinML
python check_winml_setup.py
```

--------------------------------

### Standalone OGA Model Export Environment Setup

Source: https://ryzenai.docs.amd.com/en/latest/oga_model_prepare.html

Commands to set up a conda environment and install necessary packages for exporting OGA models.

```bash
conda create --name oga_builder_env python=3.12
```

```bash
conda activate oga_builder_env
```

```bash
pip install onnxruntime-genai==0.11.2
```

```bash
# pip install other necessary packages
pip install ....
```

```bash
python3 -m onnxruntime_genai.models.builder -m <input quantized model> -o <output OGA model> -p int4 -e dml
```

--------------------------------

### Build Sample Applications (Windows)

Source: https://ryzenai.docs.amd.com/en/latest/ryzen_ai_libraries.html

Builds the Ryzen AI CVML sample applications using CMake on Windows, specifying the OpenCV installation path.

```bash
mkdir build
cmake -S %CD% -B %CD%\build -DOPENCV_INSTALL_ROOT=%OPENCV_INSTALL_ROOT%
cmake --build %CD%\build --config Release
```

--------------------------------

### Run Ryzen AI Installation Test

Source: https://ryzenai.docs.amd.com/en/latest/inst.html

Navigate to the quicktest subfolder and run the Python script to verify the Ryzen AI Software installation. This test is expected to work for Strix (STX) or newer devices.

```bash
cd %RYZEN_AI_INSTALLATION_PATH%/quicktest
python quicktest.py
```

--------------------------------

### Clone RyzenAI-SW Repository

Source: https://ryzenai.docs.amd.com/en/latest/winml/winml_example.html

Clone the RyzenAI-SW repository to access the example source code. This is the initial step for setting up the project.

```bash
git clone https://github.com/amd/RyzenAI-SW.git
cd RyzenAI-SW/WinML/CNN/ResNet/python
```

--------------------------------

### Build Sample Applications (Linux)

Source: https://ryzenai.docs.amd.com/en/latest/ryzen_ai_libraries.html

Builds the Ryzen AI CVML sample applications using CMake on Linux, specifying the OpenCV installation path.

```bash
mkdir build
cmake -S $PWD -B $PWD/build -DOPENCV_INSTALL_ROOT=$OPENCV_INSTALL_ROOT
cmake --build $PWD/build --config Release
```

--------------------------------

### Install AI Analyzer Wheel

Source: https://ryzenai.docs.amd.com/en/latest/ai_analyzer.html

Install the AI Analyzer wheel file if you manually installed the Ryzen AI software. Ensure the RYZEN_AI_INSTALLATION_PATH and version are correctly specified.

```bash
python -m pip install %RYZEN_AI_INSTALLATION_PATH%\aianalyzer-<version>.whl

```

--------------------------------

### Set up Ryzen AI LLM Environment

Source: https://ryzenai.docs.amd.com/en/latest/llm/high_level_python.html

Create and activate a Conda environment for Ryzen AI LLM development. Installs the necessary lemonade-sdk with specific backends.

```bash
conda create -n ryzenai-llm python=3.12
conda activate ryzenai-llm
pip install lemonade-sdk[dev,oga-ryzenai] --extra-index-url=https://pypi.amd.com/simple

```

--------------------------------

### Load BF16 Model with Config File (Python)

Source: https://ryzenai.docs.amd.com/en/latest/modelrun.html

Python example demonstrating how to load a BF16 model using the VitisAI EP and specifying a configuration file via `provider_options`.

```python
import onnxruntime

vai_ep_options = {
    'config_file': 'vai_ep_config.json',
}

session = onnxruntime.InferenceSession(
    "resnet50.onnx",
    providers=['VitisAIExecutionProvider'],
    provider_options=[vai_ep_options]
)


```

--------------------------------

### Install Model Generation Package

Source: https://ryzenai.docs.amd.com/en/latest/llm_linux.html

Installs the 'model-generate' package with version 1.7.1 from the specified AMD Ryzen AI PyPI index. Use '--force-reinstall' to ensure a clean installation.

```bash
pip install model-generate==1.7.1 --force-reinstall --no-deps --extra-index-url https://pypi.amd.com/ryzenai_llm/1.7.1/linux/simple/
```

--------------------------------

### Install ONNX Runtime and Model Generate Wheels

Source: https://ryzenai.docs.amd.com/en/latest/hybrid_oga.html

Install the necessary wheel files for onnxruntime-genai and model-generate using pip, specifying the AMD PyPI index URL.

```bash
pip install onnxruntime-genai-directml-ryzenai==0.11.2 --extra-index-url https://pypi.amd.com/ryzenai_llm/1.7.1/windows/simple/
pip install model-generate==1.7.1 --extra-index-url https://pypi.amd.com/ryzenai_llm/1.7.1/windows/simple/
```

--------------------------------

### Validation Tools Example

Source: https://ryzenai.docs.amd.com/en/latest/llm/high_level_python.html

Command-line example to prompt an LLM using the validation tools.

```APIDOC
lemonade -i amd/Llama-3.2-1B-Instruct-awq-g128-int4-asym-fp16-onnx-hybrid oga-load --device hybrid --dtype int4 llm-prompt --max-new-tokens 64 -p PROMPT
```

--------------------------------

### Install Other Dependencies

Source: https://ryzenai.docs.amd.com/en/latest/oga_model_prepare.html

Install necessary Python libraries for model quantization, including datasets, transformers, accelerate, evaluate, and nltk. Ensure you use the specified version for transformers.

```bash
pip install datasets
pip install transformers==4.57.6
pip install accelerate
pip install evaluate
pip install nltk
```

--------------------------------

### Install NPU Drivers on Linux

Source: https://ryzenai.docs.amd.com/en/latest/linux.html

Installs the XRT package and NPU driver for Ryzen AI on Linux systems. Ensure the .deb files are in the current directory.

```bash
sudo apt install --fix-broken -y ./xrt_202610.2.21.75_24.04-amd64-base.deb
sudo apt install --fix-broken -y ./xrt_202610.2.21.75_24.04-amd64-base-dev.deb
sudo apt install --fix-broken -y ./xrt_202610.2.21.75_24.04-amd64-npu.deb
sudo apt install --fix-broken -y ./xrt_plugin.2.21.260102.53.release_24.04-amd64-amdxdna.deb
```

--------------------------------

### Install AMD Quark

Source: https://ryzenai.docs.amd.com/en/latest/oga_model_prepare.html

Install the AMD Quark library by navigating to the extracted archive directory and running the pip install command. Ensure you replace `<extracted amd quark-version>` and `<version>` with the correct paths and version numbers.

```bash
cd <extracted amd quark-version>
pip install amd_quark-<version>+<>.whl
```

--------------------------------

### Run Ryzen AI Quick Test

Source: https://ryzenai.docs.amd.com/en/latest/linux.html

Navigates to the quicktest directory within the activated virtual environment and runs the quicktest.py script to verify the NPU installation.

```bash
cd <TARGET-PATH>/venv/quicktest
python quicktest.py
```

--------------------------------

### Load BF16 Model with Config File (C++)

Source: https://ryzenai.docs.amd.com/en/latest/modelrun.html

C++ example showing how to load a BF16 model with the VitisAI EP, specifying a configuration file using `AppendExecutionProvider_VitisAI`.

```cpp
#include <onnxruntime_cxx_api.h>

auto onnx_model = "resnet50.onnx"
Ort::Env env(ORT_LOGGING_LEVEL_WARNING, "resnet50_bf16");
auto session_options = Ort::SessionOptions();
auto vai_ep_options = std::unorderd_map<std::string,std::string>({});
vai_ep_options["config_file"] = "vai_ep_config.json";
session_options.AppendExecutionProvider_VitisAI(vai_ep_options);
auto session = Ort::Session(
    env,
    std::basic_string<ORTCHAR_T>(onnx_model.begin(), onnx_model.end()).c_str(),
    session_options);


```

--------------------------------

### Build ResNet C++ Example with CMake

Source: https://ryzenai.docs.amd.com/en/latest/getstartex.html

Use this CMake command to configure and generate build files for the ResNet C++ sample. Ensure OpenCV_DIR is set to your OpenCV build directory.

```bash
cd getting_started_resnet/cpp
cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DBUILD_SHARED_LIBS=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_CONFIGURATION_TYPES=Release -A x64 -T host=x64 -DCMAKE_INSTALL_PREFIX=. -DCMAKE_PREFIX_PATH=. -B build -S resnet_cifar -DOpenCV_DIR="C:/opencv/build" -G "Visual Studio 17 2022"
```

--------------------------------

### Set Up Working Directory and Copy Files for C++ Program

Source: https://ryzenai.docs.amd.com/en/latest/hybrid_oga.html

Prepare a working directory for the C++ program by creating a directory, copying the model benchmark executable, prompt file, and required DLLs.

```batch
mkdir llm_run
cd llm_run

:: Copy the sample C++ executable
xcopy /Y "%RYZEN_AI_INSTALLATION_PATH%\LLM\example\model_benchmark.exe" .

:: Copy the sample prompt file
xcopy /Y "%RYZEN_AI_INSTALLATION_PATH%\LLM\example\amd_genai_prompt.txt" .

:: Copy required DLLs
xcopy /Y "%RYZEN_AI_INSTALLATION_PATH%\deployment\." .

```

--------------------------------

### Clone RyzenAI-SW Repository

Source: https://ryzenai.docs.amd.com/en/latest/getstartex.html

Clone the RyzenAI-SW repository and navigate to the getting_started_resnet/int8 directory to begin the tutorial.

```bash
git clone https://github.com/amd/RyzenAI-SW.git
cd RyzenAI-SW/CNN-examples/getting_started_resnet/int8
```

--------------------------------

### Successful Test Output Verification

Source: https://ryzenai.docs.amd.com/en/latest/inst.html

This is an example of the expected output after a successful test run. It confirms that all nodes were placed on the VitisAIExecutionProvider, indicating the Ryzen AI Software is installed and functioning correctly.

```text
[V:onnxruntime:, session_state.cc:1296 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Node placements
[V:onnxruntime:, session_state.cc:1299 onnxruntime::VerifyEachNodeIsAssignedToAnEp]  All nodes placed on [VitisAIExecutionProvider]. Number of nodes: 3
Test Finished

```

--------------------------------

### Copy Necessary Deployment Files

Source: https://ryzenai.docs.amd.com/en/latest/llm_linux.html

Copies the 'deployment' folder, 'model_benchmark' script, and 'amd_genai_prompt.txt' file to your current working directory. Adjust <TARGET-PATH> as needed.

```bash
# Navigate to <TARGET-PATH>/venv and copy the "deployment" folder
cp -r <TARGET-PATH>/venv/deployment .

# Navigate to <TARGET-PATH>/venv/LLM/examples/ and copy "model_benchmark" file.
cp <TARGET-PATH>/venv/LLM/examples/model_benchmark .

# Navigate to <TARGET-PATH>/venv/LLM/examples/ and copy "amd_genai_prompt.txt" file.
cp <TARGET-PATH>/venv/LLM/examples/amd_genai_prompt.txt .
```

--------------------------------

### Validate Ryzen AI Installation Path

Source: https://ryzenai.docs.amd.com/en/latest/linux.html

Prints the RYZEN_AI_INSTALLATION_PATH environment variable to confirm the installation location.

```bash
echo $RYZEN_AI_INSTALLATION_PATH
```

--------------------------------

### Clone Ryzen AI CVML Samples (Windows)

Source: https://ryzenai.docs.amd.com/en/latest/ryzen_ai_libraries.html

Clones the Ryzen AI SW repository and navigates to the CVML library samples folder on Windows.

```bash
git clone https://github.com/amd/RyzenAI-SW.git -b main --depth-1
cd RyzenAI-SW\Ryzen-AI-CVML-Library\samples
```

--------------------------------

### Install Python and Dependencies on Ubuntu

Source: https://ryzenai.docs.amd.com/en/latest/linux.html

Installs Python 3.12.x and essential libraries required for Ryzen AI on Ubuntu.

```bash
sudo apt update
sudo apt install python3.12
sudo apt install python3.12-venv
sudo apt install libboost-filesystem1.74.0
```

--------------------------------

### Python API Example

Source: https://ryzenai.docs.amd.com/en/latest/llm/high_level_python.html

Example Python script demonstrating the high-level Lemonade API for loading a model and generating text.

```APIDOC
from lemonade.api import from_pretrained

model, tokenizer = from_pretrained(
    "amd/Llama-3.2-1B-Instruct-awq-g128-int4-asym-fp16-onnx-hybrid", recipe="oga-hybrid"
)

input_ids = tokenizer("This is my prompt", return_tensors="pt").input_ids
response = model.generate(input_ids, max_new_tokens=30)

print(tokenizer.decode(response[0]))
```

--------------------------------

### Verify NPU Driver Installation

Source: https://ryzenai.docs.amd.com/en/latest/linux.html

Uses the 'xrt-smi examine' command to verify that the NPU drivers are installed and the NPU device is detected.

```bash
xrt-smi examine
```

--------------------------------

### Clone Ryzen AI CVML Samples (Linux)

Source: https://ryzenai.docs.amd.com/en/latest/ryzen_ai_libraries.html

Clones the Ryzen AI SW repository and navigates to the CVML library samples folder on Linux.

```bash
git clone https://github.com/amd/RyzenAI-SW.git -b main --depth-1
cd RyzenAI-SW/Ryzen-AI-CVML-Library/samples
```

--------------------------------

### Activate Conda Environment

Source: https://ryzenai.docs.amd.com/en/latest/inst.html

Activate the Conda environment created by the Ryzen AI installer. Replace `<version>` with the installed version number.

```bash
conda activate ryzen-ai-<version>
```

--------------------------------

### Activate Conda Environment and Install PyTorch

Source: https://ryzenai.docs.amd.com/en/latest/hybrid_oga.html

Activate the Ryzen AI Conda environment and install the PyTorch library, which is required for running Python scripts.

```bash
conda activate ryzen-ai-<version>
pip install torch==2.7.1

```

--------------------------------

### Platform Information Sample Output

Source: https://ryzenai.docs.amd.com/en/latest/xrt_smi.html

Sample command line output for 'xrt-smi examine --report platform', showing NPU name, power mode, and estimated power consumption.

```text
--------------------------
[00c5:00:01.1] : NPU Strix
--------------------------
Platform
  Name                   : NPU Strix
  Power Mode             : Default

Estimated Power          : 1.277 Watts


```

--------------------------------

### Clone NPU-Optimized LLM Model

Source: https://ryzenai.docs.amd.com/en/latest/llm_linux.html

Installs git-lfs, initializes it, and clones a specific NPU-optimized LLM model from Hugging Face. Ensure git-lfs is installed before proceeding.

```bash
# Make sure git-lfs is installed (https://git-lfs.com)
sudo apt install git-lfs
git lfs install
git clone https://huggingface.co/amd/Phi-3.5-mini-instruct_rai_1.7.1_npu_4K
```

--------------------------------

### Specify VitisAIExecutionProvider for Model Execution (C++)

Source: https://ryzenai.docs.amd.com/en/latest/winml/winml_ep.html

This C++ example demonstrates how to enumerate available execution devices, filter for VitisAI NPU devices, and configure session options to exclusively use the VitisAIExecutionProvider.

```cpp
#include <iostream>
#include <iomanip>
#include <vector>
#include <stdexcept>
#include <win_onnxruntime_cxx_api.h>

// Assuming you have an Ort::Env named 'env'
// 1. Enumerate EP devices
std::vector<Ort::ConstEpDevice> ep_devices = env.GetEpDevices();

// 2. Collect only ReplaceWithExecutionProvider NPU devices
std::vector<Ort::ConstEpDevice> selected_ep_devices;
for (const auto& d : ep_devices) {
    if (std::string(d.EpName()) == "VitisAIExecutionProvider"
        && d.HardwareDevice().Type() == OrtHardwareDeviceType_NPU) {
        selected_ep_devices.push_back(d);
    }
}
if (selected_ep_devices.empty()) {
    throw std::runtime_error("VitisAIExecutionProvider is not available on this system.");
}

// 3. Configure provider-specific options (varies based on EP)
// and append the EP with the correct devices (varies based on EP)
Ort::SessionOptions session_options;
Ort::KeyValuePairs ep_options;
ep_options.Add("optimize_level", "1");
session_options.AppendExecutionProvider_V2(env, { selected_ep_devices.front() }, ep_options);
```

--------------------------------

### Create and Navigate to LLM Directory

Source: https://ryzenai.docs.amd.com/en/latest/llm_linux.html

Use these commands to create a new directory for your LLM project and navigate into it.

```bash
mkdir run_llm
cd run_llm
```

--------------------------------

### Activate Linux Python Virtual Environment

Source: https://ryzenai.docs.amd.com/en/latest/linux.html

Activates the Python virtual environment for running Ryzen AI examples on Linux. This is a prerequisite for running provided examples.

```bash
source <TARGET-PATH>/venv/bin/activate
```

--------------------------------

### NPU Partitions Sample Output

Source: https://ryzenai.docs.amd.com/en/latest/xrt_smi.html

Sample command line output for 'xrt-smi examine --report aie-partitions', detailing NPU partitions, column occupancy, and hardware context information.

```text
--------------------------
[00c5:00:01.1] : NPU Strix
--------------------------
AIE Partitions
  Partition Index: 0
    Columns: [0, 1, 2, 3]
    HW Contexts:
      |PID    |Ctx ID  |Status  |Instr BO  |Sub  |Compl  |Migr  |Err  |Suspensions  |Prio     |GOPS   |EGOPS  |FPS  |Latency  |
      |-------|--------|--------|----------|-----|-------|------|-----|-------------|---------|-------|-------|-----|---------|
      |20696  |0       |Active  |64 KB     |57   |56     |0     |0    |0            | Normal  |0      |0      |0    |0        |


```

--------------------------------

### Install model-generate Package

Source: https://ryzenai.docs.amd.com/en/latest/oga_model_prepare.html

Install the `model-generate` package with version 1.7.1, forcing reinstallation and without dependencies, using the provided extra index URL. This is required for the postprocessing step on Windows.

```bash
pip install model-generate==1.7.1 --force-reinstall --no-deps --extra-index-url https://pypi.amd.com/ryzenai_llm/1.7.1/windows/simple/
```

--------------------------------

### Configure Image-to-Image Demo with ControlNet and Resolution

Source: https://ryzenai.docs.amd.com/en/latest/sd_demo.html

Run the image-to-image demo with custom resolution (1024x1024), a specific ControlNet type (Canny), a custom prompt, and control image path. The '-n 50' argument sets the number of inference steps.

```python
python run_sd3.py -C canny --model_id "stabilityai/stable-diffusion-3-medium-amdnpu" --prompt "Anime style illustration of a girl wearing a suit. A moon in sky. In the background we see a big rain approaching. text 'InstantX' on image" -H 1024 -W 1024 --control_image_path .\ref\canny.jpg -n 50
```

```python
python run_sd3.py -C pose --model_id "stabilityai/stable-diffusion-3-medium-amdnpu" --prompt "Anime style illustration of a girl wearing a suit. A moon in sky. In the background we see a big rain approaching. text 'InstantX' on image" -H 1024 -W 1024 --control_image_path .\ref\pose.jpg -n 50
```

```python
python run_sd3.py -C tile --model_id "stabilityai/stable-diffusion-3-medium-amdnpu" --prompt "Anime style illustration of a girl wearing a suit. A moon in sky. In the background we see a big rain approaching. text 'InstantX' on image" -H 1024 -W 1024 --control_image_path .\ref\tile.jpg -n 50
```

```python
python run_sd3.py -C depth --model_id "stabilityai/stable-diffusion-3-medium-amdnpu" -H 1024 -W 1024 --control_image_path .\assets\depth.jpeg -n 50
```

--------------------------------

### Update PyTorch for ROCm on AMD GPUs

Source: https://ryzenai.docs.amd.com/en/latest/oga_model_prepare.html

Install or update PyTorch to a version compatible with ROCm 6.1 for AMD GPUs. Verify the installation by checking if CUDA is available. This step is crucial for utilizing AMD GPU acceleration.

```bash
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.1
python -c "import torch; print(torch.cuda.is_available())" # Must return `True`
```

--------------------------------

### Create and Configure XRT Drivers

Source: https://ryzenai.docs.amd.com/en/latest/llm_linux.html

Creates an 'xrt.ini' file, adds specific debug configurations, and sets the XRT_INI_PATH environment variable to point to this file.

```bash
# vi xrt.ini            (Creates a new file)

# Add below lines to the file and save it
# [Debug]
# num_heap_pages = 8

# Set XRT_INI_PATH to point to this file
export XRT_INI_PATH=$PWD/xrt.ini
```

--------------------------------

### Set OpenCV Install Root (Linux)

Source: https://ryzenai.docs.amd.com/en/latest/ryzen_ai_libraries.html

Sets the OPENCV_INSTALL_ROOT environment variable on Linux to locate OpenCV libraries.

```bash
export OPENCV_INSTALL_ROOT=<location of OpenCV libraries>
```
Tokens:
Raw