ort

https://ort.pyke.io/
Admin
ort is an open-source Rust binding for ONNX Runtime, making it easy to deploy machine learning...
Tokens:21,122
Snippets:193
Trust Score:5.9
Update:3 weeks ago
Show doc for...
Tokens:
Raw
### Initialize ort with custom dylib path in main.rs

Source: https://ort.pyke.io/setup/linking

Rust example showing how to find the ONNX Runtime dylib path and initialize ort with it using ort::init_from. Must be called before any other ort usage; returns an error if the dylib fails to load.

```rust
fn find_onnxruntime_dylib() -> anyhow::Result<PathBuf> {
    // Find our custom ONNX Runtime dylib path somehow (i.e. resolving it from the root of our program's install folder)
    // The path should point to the `libonnxruntime` binary, which looks like:
    // - on Windows: C:\Program Files\...\onnxruntime.dll
    // - on Linux: /etc/.../libonnxruntime.so
    // - on macOS: /.../libonnxruntime.dylib
    unimplemented!()
}
 
fn main() -> anyhow::Result<()> {
    let dylib_path = find_onnxruntime_dylib()?;
 
    // Initialize ort with the path to the dylib. This **must** be called before any other usage of `ort`!
    // `init_from` returns a `Result<EnvironmentBuilder>` which you can use to further configure the environment
    // before `.commit()`ing; see the Environment docs for more information on what you can configure.
    // `init_from` will return an `Err` if it fails to load the dylib.
    ort::init_from(dylib_path)?.commit();
 
    // Now we can use `ort`!
    let mut session = Session::builder()?
        .commit_from_file("model.onnx")?;
 
    Ok(())
}

```

--------------------------------

### Self-host ort-web with Dist

Source: https://ort.pyke.io/backends/web

Configures ort-web to use a self-hosted build by specifying the dist URL and script name. The example loads the WebGPU build from a CDN. Requires the ort and ort_web crates.

```rust
use ort::session::Session;
use ort_web::Dist;
 
async fn init_model() -> anyhow::Result<Session> {
	let dist = Dist::new("https://cdn.jsdelivr.net/npm/onnxruntime-web@1.27.0/dist/")
		// we want to load the WebGPU build
		.with_script_name("ort.webgpu.min.js");
	ort::set_api(ort_web::api(dist).await?);
}
```

--------------------------------

### Install ort-candle in Cargo.toml

Source: https://ort.pyke.io/backends/candle

Add the `ort-candle` crate as a dependency in your Cargo.toml. The version string includes both the crate version and the ort version it targets.

```toml
[dependencies]
ort-candle = "0.4.0+0.11.0"
...
```

--------------------------------

### Initialize tracing_subscriber in main.rs

Source: https://ort.pyke.io/troubleshooting/logging

Initializes the tracing subscriber in the main function to start logging ort messages.

```rust
fn main() {
    tracing_subscriber::fmt::init();
 
    // ... rest of code
}
```

--------------------------------

### Create a Map from key and value tensors with Map::new_kv

Source: https://ort.pyke.io/fundamentals/value

Creates a Map from two tensors containing keys and values using Map::new_kv. The keys tensor must be of type i64 and the values tensor of type f32 in this example.

```rust
let keys = Tensor::<i64>::from_array(([4], vec![0, 1, 2, 3]))?;
let values = Tensor::<f32>::from_array(([4], vec![1., 2., 3., 4.]))?;
 
let map = Map::new_kv(keys, values)?;
```

--------------------------------

### Create a Sequence from tensors with Sequence::new

Source: https://ort.pyke.io/fundamentals/value

Creates a Sequence from any iterator yielding a Value subtype. The example uses two tensors created with Tensor::new and an allocator, then wraps them in a Sequence::new.

```rust
let tensor1 = Tensor::<f32>::new(&allocator, [1, 128, 128, 3])?;
let tensor2 = Tensor::<f32>::new(&allocator, [1, 224, 224, 3])?;
 
let sequence = Sequence::new(vec![tensor1, tensor2])?;
```

--------------------------------

### Create a Map from an iterator with Map::new

Source: https://ort.pyke.io/fundamentals/value

Creates a Map from any iterator yielding (K, V) tuples, where K and V are tensor element types. The example uses a HashMap<String, f32>.

```rust
let mut map = HashMap::<String, f32>::new();
map.insert("one".to_string(), 1.0);
map.insert("two".to_string(), 2.0);
map.insert("three".to_string(), 3.0);
 
let map = Map::<String, f32>::new(map)?;
```

--------------------------------

### Register multiple execution providers with fallback

Source: https://ort.pyke.io/perf/execution-providers

Register multiple EPs in order of preference. ort falls back to the next successfully registered EP if an operator is unsupported, and to CPU if all fail. The example uses `#[cfg]` attributes to conditionally include TensorRT, CUDA, DirectML, and CoreML.

```rust
use ort::{ep, session::Session};
 
fn main() -> anyhow::Result<()> {
    let session = Session::builder()?
        .with_execution_providers([
            // Prefer TensorRT over CUDA.
            #[cfg(feature = "tensorrt")]
            ep::TensorRT::default().build(),
            #[cfg(feature = "cuda")]
            ep::CUDA::default().build(),
            // Use DirectML on Windows if NVIDIA EPs are not available
            #[cfg(feature = "directml")]
            ep::DirectML::default().build(),
            // Or use ANE on Apple platforms
            #[cfg(feature = "coreml")]
            ep::CoreML::default().build()
        ])?
        .commit_from_file("model.onnx")?;
 
    Ok(())
}
```

--------------------------------

### Diffusion pipeline with I/O binding (text encoder and UNet)

Source: https://ort.pyke.io/perf/io-binding

A complete example showing a text-to-image diffusion loop. It builds a text encoder and a UNet sessions on CUDA, binds inputs and outputs, prepares pinned-memory allocators, and repeatedly runs the UNet with a constant condition tensor. Avoids CPU copies, copies the condition once, and reuses pre-allocated output tensor in pinned memory.

```rust
let mut text_encoder = Session::builder()?
	.with_execution_providers([ep::CUDA::default().build()])?
	.commit_from_file("text_encoder.onnx")?;
let mut unet = Session::builder()?
	.with_execution_providers([ep::CUDA::default().build()])?
	.commit_from_file("unet.onnx")?;
 
let text_condition = {
    let mut binding = text_encoder.create_binding()?;
    binding.bind_input("tokens", &Tensor::<i64>::from_array((
        vec![1, 22],
        vec![49, 272, 503, 286, 1396, 353, 9653, 284, 1234, 287, 616, 2438, 11, 7926, 13, 3423, 338, 3362, 25, 12520, 238, 242]
    ))?)?;
    binding.bind_output_to_device("output0", &text_encoder.allocator().memory_info())?;
    text_encoder.run_binding(&binding)?.remove("output0").unwrap()
};
 
let input_allocator = Allocator::new(
	&unet,
	MemoryInfo::new(AllocationDevice::CUDA_PINNED, 0, AllocatorType::Device, MemoryType::CPUInput)?
)?;
let mut latents = Tensor::<f32>::new(&input_allocator, [1, 4, 64, 64])?;
 
let mut io_binding = unet.create_binding()?;
io_binding.bind_input("condition", &text_condition)?;
 
let output_allocator = Allocator::new(
	&unet,
	MemoryInfo::new(AllocationDevice::CUDA_PINNED, 0, AllocatorType::Device, MemoryType::CPUOutput)?
)?;
io_binding.bind_output("noise_pred", Tensor::<f32>::new(&output_allocator, [1, 4, 64, 64])?)?;
 
for _ in 0..20 {
	io_binding.bind_input("latents", &latents)?;
	let noise_pred = unet.run_binding(&io_binding)?.remove("noise_pred").unwrap();
 
	let mut latents = latents.extract_array_mut();
	latents += &noise_pred.try_extract_array::<f32>()?;
}

```

--------------------------------

### Add ort-web dependency to Cargo.toml

Source: https://ort.pyke.io/backends/web

Declares the ort-web crate version in the [dependencies] section to install the package.

```toml
[dependencies]
ort-web = "0.3.0+1.27"
...
```

--------------------------------

### Initialize the ort-web backend (lib.rs)

Source: https://ort.pyke.io/backends/web

Runs before any other ort API usage to set up the WebGPU execution provider backend.

```rust
use ort_web::FEATURE_WEBGPU;
use wasm_bindgen::JsError;
 
async fn init() -> Result<(), JsError> {
    // This should always be run before you use any other `ort` API.
    ort::set_api(ort_web::api(FEATURE_WEBGPU).await?);
 
    ...
}

```

--------------------------------

### Create a session from a URL (lib.rs)

Source: https://ort.pyke.io/backends/web

Builds a session with the WebGPU execution provider and commits a model from a URL, which must be awaited on the web.

```rust
use ort::{ep, session::Session};
use ort_web::FEATURE_WEBGPU;
use wasm_bindgen::JsError;
 
async fn init() -> Result<(), JsError> {
    ort::set_api(ort_web::api(FEATURE_WEBGPU).await?);
 
    let mut session = Session::builder()?
        .with_execution_providers([
            // only available with FEATURE_WEBGPU
            ep::WebGPU::default().build()
        ])?
        .commit_from_url("./model.onnx")
        .await?; // <- note we must .await on the web
}

```

--------------------------------

### Verify prebuilt binary attestation with gh attestation verify

Source: https://ort.pyke.io/misc/prebuilt-binaries

Verifies the attestation of a prebuilt ONNX Runtime binary archive using GitHub's `gh` CLI. The command checks the file's provenance against the `pykeio` owner and prints the loaded digest and verification success message.

```bash
$ gh attestation verify --owner pykeio ./x86_64-pc-windows-msvc+cu13.tar.lzma2
Loaded digest sha256:e96616510082108be228ad6ea026246a31650b7d446b330c6b9671fcb9ae6267 for file://./x86_64-pc-windows-msvc+cu13.tar.lzma2
Loaded 1 attestation from GitHub API
 
The following policy criteria will be enforced:
- OIDC Issuer must match:................... https://token.actions.githubusercontent.com
- Source Repository Owner URI must match:... https://github.com/pykeio
- Predicate type must match:................ https://slsa.dev/provenance/v1
- Subject Alternative Name must match regex: (?i)^https://github.com/pykeio/
 
✓ Verification succeeded!
 
sha256:e96616510082108be228ad6ea026246a31650b7d446b330c6b9671fcb9ae6267 was attested by:
REPO                  PREDICATE_TYPE                  WORKFLOW
pykeio/ort-artifacts  https://slsa.dev/provenance/v1  .github/workflows/build-runner.yml@refs/heads/main
```

--------------------------------

### Create IoBinding with Session::create_binding

Source: https://ort.pyke.io/perf/io-binding

Creates an IoBinding instance for a session. One binding should be created per request.

```rust
let mut binding = session.create_binding()?;
```

--------------------------------

### Initialize ort-candle backend in main.rs

Source: https://ort.pyke.io/backends/candle

Initialize the `ort-candle` backend by calling `ort::set_api` with `ort_candle::api()`. This should be done as early as possible in your application, before any use of `ort`.

```rust
fn main() {
    // This should run as early in your application as possible - before you ever use `ort`!
    ort::set_api(ort_candle::api());
}
```

--------------------------------

### Configure CoreML EP with subgraphs and compute units

Source: https://ort.pyke.io/perf/execution-providers

Configures a CoreML execution provider with subgraph support and CPUAndNeuralEngine compute units. Requires the `coreml` feature and the `ort` crate.

```rust
use ort::{ep, session::Session};
 
fn main() -> anyhow::Result<()> {
    let session = Session::builder()?
        .with_execution_providers([
            #[cfg(feature = "coreml")]
            ep::CoreML::default()
                // this model uses control flow operators, so enable CoreML on subgraphs too
                .with_subgraphs()
                .with_compute_units(ep::coreml::ComputeUnits::CPUAndNeuralEngine)
                .build()
        ])?
        .commit_from_file("model.onnx")?;
 
    Ok(())
}
```

--------------------------------

### Pin ort version in Cargo.toml with tilde requirement

Source: https://ort.pyke.io/misc/version-mapping

In your Cargo.toml, use a tilde requirement to pin ort to a specific minor version. This ensures you get the exact ONNX Runtime version you need, since ort may update ONNX Runtime on minor versions.

```toml
[dependencies]
ort = { version = "~2.0", ... }
```

--------------------------------

### Initialize alternative backend with ort::set_api

Source: https://ort.pyke.io/backends

Calls ort::set_api with the backend's API implementation, replacing ort_tract with the chosen backend crate. Should run early before any ort usage.

```rust
fn main() {
    // This should run as early in your application as possible - before you ever use `ort`!
    ort::set_api(ort_tract::api());
}
```

--------------------------------

### Configure global execution providers with ort::init().with_execution_providers

Source: https://ort.pyke.io/perf/execution-providers

Configure a global execution provider (CUDA) for all sessions by setting environment EPs before creating any sessions. The configuration must be committed before any session is created, otherwise it will not take effect. Per-session EPs take precedence over environment EPs but do not replace them.

```rust
use ort::{ep, session::Session};
 
fn main() -> anyhow::Result<()> {
    ort::init()
        .with_execution_providers([
            #[cfg(feature = "cuda")]
            ep::CUDA::default().build()
        ])
        .commit();
 
    let session = Session::builder()?.commit_from_file("model.onnx")?;
    // The session will attempt to register the CUDA EP
    // since we configured environment EPs.
 
    Ok(())
}
```

--------------------------------

### Toggle multiple execution provider features

Source: https://ort.pyke.io/backends/web

Combines FEATURE_WEBGL and FEATURE_WEBGPU with a bitwise OR to enable both backends at once.

```rust
use ort_web::{FEATURE_WEBGL, FEATURE_WEBGPU};
ort::set_api(ort_web::api(FEATURE_WEBGL | FEATURE_WEBGPU).await?);

```

--------------------------------

### Enable alternative-backend feature in Cargo.toml

Source: https://ort.pyke.io/backends/web

Configures the ort crate to use the alternative backend, disabling default features like download-binaries.

```toml
[dependencies.ort]
version = "=2.0.0-rc.13"
default-features = false # Disables the `download-binaries` feature since we don't need it
features = [
    "std",
    "ndarray",
    "api-27",
    "alternative-backend"
]

```

--------------------------------

### Bind output to device with IoBinding::bind_output_to_device

Source: https://ort.pyke.io/perf/io-binding

Binds an output to a device without preallocating a tensor, useful when output dimensions are unknown ahead of time.

```rust
let allocator = Allocator::default();
binding.bind_output_to_device("action", &allocator.memory_info())?;
```

--------------------------------

### Build with ORT_LIB_PATH for static linking

Source: https://ort.pyke.io/setup/linking

Set the ORT_LIB_PATH environment variable to the directory containing the static libraries (e.g., onnxruntime/build/Linux/Release) and run cargo build. ort will prefer static libraries if both static and dynamic are present.

```bash
$ ORT_LIB_PATH=~/onnxruntime/build/Linux/Release cargo build
```

--------------------------------

### Pass values to session.run with ort::inputs!

Source: https://ort.pyke.io/fundamentals/value

Shows how to pass tensors to a session's run function by value, by reference, and by view using the ort::inputs! macro.

```rust
let latents = Tensor::<f32>::new(&allocator, [1, 128, 128, 3])?;
let text_embedding = Tensor::<f32>::new(&allocator, [1, 48, 256])?;
let timestep = Tensor::<f32>::new(&allocator, [1])?;

let outputs = session.run(ort::inputs![
    "timestep" => timestep,
    "latents" => &latents,
    "text_embedding" => text_embedding.view()
])?;
```

--------------------------------

### Load an ONNX model with Session::builder

Source: https://ort.pyke.io/

Creates an ort Session by building it with optimization level Level3 and 4 intra threads, then commits from an ONNX file. The ? operator propagates errors, so this code is typically used inside a function that returns a Result.

```rust
use ort::session::{builder::GraphOptimizationLevel, Session};
 
let mut model = Session::builder()?
    .with_optimization_level(GraphOptimizationLevel::Level3)?
    .with_intra_threads(4)?
    .commit_from_file("yolov8m.onnx")?;
```

--------------------------------

### Bind input with IoBinding::bind_input

Source: https://ort.pyke.io/perf/io-binding

Queues input data to be copied to the session's device. The data is not synchronized until the binding is run or synchronize_inputs is called.

```rust
let style_embedding: Tensor<f32> = Tensor::from_array(...)?;
 
binding.bind_input("style_embd", &style_embedding)?;
```

--------------------------------

### Call run_binding() to execute a bound session

Source: https://ort.pyke.io/perf/io-binding

Calls `run_binding()` on a session with a previously created `IoBinding`. The returned `outputs` object behaves like the results from `Session::run`, but exposes only the bound outputs.

```rust
let outputs = session.run_binding(&binding)?;

```

--------------------------------

### Force input synchronization with IoBinding::synchronize_inputs

Source: https://ort.pyke.io/perf/io-binding

Forces all bound inputs to synchronize immediately. After this call, all inputs are synchronized.

```rust
binding.synchronize_inputs()?;
// all inputs are now synchronized
```

--------------------------------

### Check CoreML availability with is_available before registering

Source: https://ort.pyke.io/perf/execution-providers

Checks whether a custom-linked ONNX Runtime build includes CoreML support using `is_available`, then registers the provider. Registration can still fail even if `is_available` returns true.

```rust
use ort::{
    ep::{self, ExecutionProvider},
    session::Session
};
 
fn main() -> anyhow::Result<()> {
    let builder = Session::builder()?;
 
    let coreml = ep::CoreML::default();
    if !coreml.is_available() {
        eprintln!("Please compile ONNX Runtime with CoreML!");
        std::process::exit(1);
    }
 
    // Note that even though ONNX Runtime was compiled with CoreML, registration could still fail!
    coreml.register(&builder)?;
 
    let session = builder.commit_from_file("model.onnx")?;
 
    Ok(())
}
```

--------------------------------

### Enable rpath in Cargo.toml for compile-time dynamic linking

Source: https://ort.pyke.io/setup/linking

Cargo.toml configuration to enable rpath for dev and release profiles, allowing the executable to find dylibs in its parent folder.

```toml
[profile.dev]
rpath = true
 
[profile.release]
rpath = true
 
# do this for any other profiles

```

--------------------------------

### Acquiring a TensorRef view with .view()

Source: https://ort.pyke.io/fundamentals/value

Shows how to obtain a shared view (TensorRef) from a Tensor value using the .view() method. The view is a borrowed reference and does not take ownership.

```rust
let my_tensor: ort::value::Tensor<f32> = Tensor::new(...)?;
 
let tensor_view: ort::value::TensorRef<'_, f32> = my_tensor.view();
```

--------------------------------

### Manually register CUDA EP with custom error handling

Source: https://ort.pyke.io/perf/execution-providers

Manually registers a CUDA execution provider via `ExecutionProvider::register` and exits with an error message if registration fails. Requires the `cuda` feature.

```rust
use ort::{
    ep::{self, ExecutionProvider},
    session::Session
};
 
fn main() -> anyhow::Result<()> {
    let builder = Session::builder()?;
 
    #[cfg(feature = "cuda")]
    {
        let cuda = ep::CUDA::default();
        if cuda.register(&builder).is_err() {
            eprintln!("Failed to register CUDA!");
            std::process::exit(1);
        }
    }
 
    let session = builder.commit_from_file("model.onnx")?;
 
    Ok(())
}
```

--------------------------------

### Enable alternative-backend feature in Cargo.toml

Source: https://ort.pyke.io/backends/candle

Configure the `ort` dependency to enable the `alternative-backend` feature and disable default features so ort does not download or link to ONNX Runtime. The `std` and `ndarray` features are also enabled.

```toml
[dependencies.ort]
version = "=2.0.0-rc.13"
default-features = false # Disables the `download-binaries` feature since we don't need it
features = [
    "std",
    "ndarray",
    "alternative-backend"
]
```

--------------------------------

### Cargo.toml: Set minimum API version to 20

Source: https://ort.pyke.io/setup/multiversion

Disables default features and enables the api-20 feature to target ONNX Runtime 1.20, while still downloading the latest binaries. Only features available since 1.20 are accessible.

```toml
[dependencies.ort]
version = "=2.0.0-rc.13"
default-features = false
features = [
    "std",
    "ndarray",
    "download-binaries",
 
    # We use the `Adapter` API, which is only available since ONNX Runtime 1.20,
    # so set our minimum API version to 20.
    "api-20"
]
```

--------------------------------

### Run inference with ort::inputs! and extract output

Source: https://ort.pyke.io/

Runs inference on a loaded Session using the ort::inputs! macro to pass an input named "image". The output is accessed by name and extracted as a 32-bit float array. The ellipsis indicates that further processing of the predictions is omitted.

```rust
let outputs = model.run(ort::inputs!["image" => image])?;
let predictions = outputs["output0"].try_extract_array::<f32>()?;
...
```

--------------------------------

### Bind output with preallocated tensor using IoBinding::bind_output

Source: https://ort.pyke.io/perf/io-binding

Binds an output to a preallocated tensor. The output will be placed into the provided memory allocation when the binding is run.

```rust
binding.bind_output("action", Tensor::<f32>::new(&Allocator::default(), [1, 32])?)?;
```

--------------------------------

### Extract dynamic output with try_extract_array

Source: https://ort.pyke.io/fundamentals/value

Shows how to extract a DynValue session output by downcasting to an ndarray::ArrayViewD using try_extract_array, panicking if the type is incompatible.

```rust
let outputs = session.run(ort::inputs![TensorRef::from_array_view(&input)?])?;

let Ok(tensor_output): ort::Result<ndarray::ArrayViewD<f32>> = outputs[0].try_extract_array() else {
    panic!("First output was not a Tensor<f32>!");
}
```

--------------------------------

### Register CUDA execution provider for a session

Source: https://ort.pyke.io/perf/execution-providers

Use `SessionBuilder::with_execution_providers` to register the CUDA EP. The `#[cfg(feature = "cuda")]` attribute ensures the EP is only included when the feature is enabled.

```rust
use ort::{ep, session::Session};
 
fn main() -> anyhow::Result<()> {
    let session = Session::builder()?
        .with_execution_providers([
            #[cfg(feature = "cuda")]
            ep::CUDA::default().build()
        ])?
        .commit_from_file("model.onnx")?;
 
    Ok(())
}
```

--------------------------------

### Sync a session output tensor with SyncDirection::Rust

Source: https://ort.pyke.io/backends/web

Shows how to sync a single session output tensor to Rust after run_async, using TensorExt::sync with SyncDirection::Rust, then extract the data as f32. After syncing, the tensor is backed by a Rust buffer; updates from Rust won't reflect in ONNX Runtime until synced with SyncDirection::Runtime.

```rust
use ort_web::{TensorExt, SyncDirection};
 
let mut outputs = session.run_async(ort::inputs![...]).await?;
 
let mut bounding_boxes = outputs.remove("bounding_boxes").unwrap();
bounding_boxes.sync(SyncDirection::Rust).await?;
 
// now we can use the data
let data = bounding_boxes.try_extract_tensor::<f32>()?;
```

--------------------------------

### Set ORT_DYLIB_PATH environment variable via shell

Source: https://ort.pyke.io/setup/linking

Shell command to set the ORT_DYLIB_PATH environment variable to the path of the ONNX Runtime library before running the executable. The path is relative to the executable.

```shell
$ ORT_DYLIB_PATH=../onnxruntime-build/linux-x64/libonnxruntime.so ./mirai

```

--------------------------------

### Set RUST_LOG for ort debug on Windows Command Prompt

Source: https://ort.pyke.io/troubleshooting/logging

Sets the RUST_LOG environment variable to 'ort=debug' and runs the application to see debug messages from ort.

```cmd
set RUST_LOG=ort=debug
cargo run
```

--------------------------------

### Fail on CUDA EP registration error with error_on_failure

Source: https://ort.pyke.io/perf/execution-providers

Registers a CUDA execution provider and makes registration failure fatal. If CUDA fails to register, the program exits with an error at `with_execution_providers`. Requires the `cuda` feature.

```rust
use ort::{ep, session::Session};
 
fn main() -> anyhow::Result<()> {
    let session = Session::builder()?
        .with_execution_providers([
            #[cfg(feature = "cuda")]
            ep::CUDA::default()
                .build()
                .error_on_failure()
        ])?
        .commit_from_file("model.onnx")?;
 
    Ok(())
}
```

--------------------------------

### Enable load-dynamic feature in Cargo.toml

Source: https://ort.pyke.io/setup/linking

Add the load-dynamic feature to the ort dependency in Cargo.toml to enable runtime loading of the ONNX Runtime library.

```toml
[dependencies]
ort = { version = "2", features = [ "load-dynamic" ] }

```

--------------------------------

### Add executable origin to rpath for Linux

Source: https://ort.pyke.io/setup/linking

Cargo config for Linux targets to add the executable origin to rpath, enabling the executable to find dylibs in its parent folder.

```toml
[target.x86_64-unknown-linux-gnu]
rustflags = [ "-Clink-args=-Wl,-rpath,\$ORIGIN" ]
 
# do this for any other Linux targets as well

```

--------------------------------

### Downcast DynValue to TensorRef using downcast_ref

Source: https://ort.pyke.io/fundamentals/value

Shows two equivalent ways to downcast a DynValue to a TensorRef<f32>: using .downcast_ref() directly, or using .view().downcast().

```rust
let tensor_view: ort::value::TensorRef<'_, f32> = dyn_value.downcast_ref()?;
// is equivalent to
let tensor_view: ort::value::TensorRef<'_, f32> = dyn_value.view().downcast()?;
```

--------------------------------

### Configure CUDA feature in Cargo.toml

Source: https://ort.pyke.io/perf/execution-providers

Add the `cuda` feature to the `ort` dependency in Cargo.toml to enable the CUDA execution provider.

```toml
[dependencies]
ort = { version = "2.0", features = [ "cuda" ] }
```
Tokens:
Raw