Candle

https://huggingface.github.io/candle/
Admin
Candle is a minimalist machine learning framework for Rust with a focus on performance, ease of use,...
Tokens:19,778
Snippets:112
Trust Score:9.7
Update:1 month ago
Show doc for...
Tokens:
Raw
### Run the example with cargo run --release

Source: https://huggingface.github.io/candle/guide/hello_world.html

Command to compile and run the Rust example in release mode.

```bash
cargo run --release
```

--------------------------------

### Install hf-hub with async support

Source: https://huggingface.github.io/candle/inference/hub.html

Add the hf-hub dependency with the tokio feature for async operations.

```bash
cargo add hf-hub --features tokio
```

--------------------------------

### Create new Cargo project for CUDA setup

Source: https://huggingface.github.io/candle/guide/installation.html

Commands to create a new Cargo project named 'myapp' and navigate into it, as the first step for adding candle-core with CUDA support.

```bash
cargo new myapp
cd myapp
```

--------------------------------

### Complete example with custom Linear layer

Source: https://huggingface.github.io/candle/guide/hello_world.html

Full program using the custom Linear layer defined earlier. It creates a model with random weights and biases, runs a dummy image, and prints the output. Uses Device::cuda_if_available(0)? to automatically select GPU if available, otherwise CPU.

```rust
extern crate candle_core;
use candle_core::{Device, Result, Tensor};
struct Linear{
    weight: Tensor,
    bias: Tensor,
}
impl Linear{
    fn forward(&self, x: &Tensor) -> Result<Tensor> {
        let x = x.matmul(&self.weight)?;
        x.broadcast_add(&self.bias)
    }
}

struct Model {
    first: Linear,
    second: Linear,
}

impl Model {
    fn forward(&self, image: &Tensor) -> Result<Tensor> {
        let x = self.first.forward(image)?;
        let x = x.relu()?;
        self.second.forward(&x)
    }
}
fn main() -> Result<()> {
    // Use Device::new_cuda(0)?; to use the GPU.
    // Use Device::Cpu; to use the CPU.
    let device = Device::cuda_if_available(0)?;

    // Creating a dummy model
    let weight = Tensor::randn(0f32, 1.0, (784, 100), &device)?;
    let bias = Tensor::randn(0f32, 1.0, (100, ), &device)?;
    let first = Linear{weight, bias};
    let weight = Tensor::randn(0f32, 1.0, (100, 10), &device)?;
    let bias = Tensor::randn(0f32, 1.0, (10, ), &device)?;
    let second = Linear{weight, bias};
    let model = Model { first, second };

    let dummy_image = Tensor::randn(0f32, 1.0, (1, 784), &device)?;

    // Inference on the model
    let digit = model.forward(&dummy_image)?;
    println!("Digit {digit:?} digit");
    Ok(())
}
```

--------------------------------

### Install safetensors crate

Source: https://huggingface.github.io/candle/inference/hub.html

Add the safetensors dependency for direct tensor manipulation.

```bash
cargo add safetensors
```

--------------------------------

### Parquet file inspection output

Source: https://huggingface.github.io/candle/training/training.html

Example output from inspecting the parquet file, showing rows with label and image columns where image is stored as bytes.

```text
Column id 1, name label, value 6
Column id 0, name image, value {bytes: [137, ....]
Column id 1, name label, value 8
Column id 0, name image, value {bytes: [137, ....]
```

--------------------------------

### Verify Cuda installation with nvcc and nvidia-smi

Source: https://huggingface.github.io/candle/print.html

Check that the Cuda compiler driver and GPU compute capability are available before enabling Cuda support. The second command prints a CSV header followed by the compute capability value, e.g. 8.9.

```bash
nvcc --version
```

```bash
nvidia-smi --query-gpu=compute_cap --format=csv
```

--------------------------------

### MNIST model with custom Linear layer

Source: https://huggingface.github.io/candle/print.html

Full example using the custom Linear layer with bias. Uses Device::cuda_if_available(0) to select GPU if present, otherwise CPU. Prints a dummy digit output.

```rust
extern crate candle_core;
use candle_core::{Device, Result, Tensor};
struct Linear{
    weight: Tensor,
    bias: Tensor,
}
impl Linear{
    fn forward(&self, x: &Tensor) -> Result<Tensor> {
        let x = x.matmul(&self.weight)?;
        x.broadcast_add(&self.bias)
    }
}

struct Model {
    first: Linear,
    second: Linear,
}

impl Model {
    fn forward(&self, image: &Tensor) -> Result<Tensor> {
        let x = self.first.forward(image)?;
        let x = x.relu()?;
        self.second.forward(&x)
    }
}
fn main() -> Result<()> {
    // Use Device::new_cuda(0)?; to use the GPU.
    // Use Device::Cpu; to use the CPU.
    let device = Device::cuda_if_available(0)?;

    // Creating a dummy model
    let weight = Tensor::randn(0f32, 1.0, (784, 100), &device)?;
    let bias = Tensor::randn(0f32, 1.0, (100, ), &device)?;
    let first = Linear{weight, bias};
    let weight = Tensor::randn(0f32, 1.0, (100, 10), &device)?;
    let bias = Tensor::randn(0f32, 1.0, (10, ), &device)?;
    let second = Linear{weight, bias};
    let model = Model { first, second };

    let dummy_image = Tensor::randn(0f32, 1.0, (1, 784), &device)?;

    // Inference on the model
    let digit = model.forward(&dummy_image)?;
    println!("Digit {digit:?} digit");
    Ok(())
}

```

--------------------------------

### Check CUDA compute capability

Source: https://huggingface.github.io/candle/guide/installation.html

Command to print the GPU's compute capability, which is needed to verify CUDA installation. The output shows a table with the compute capability value, e.g., 8.9.

```bash
nvidia-smi --query-gpu=compute_cap --format=csv
```

--------------------------------

### MNIST model using candle-nn Linear

Source: https://huggingface.github.io/candle/guide/hello_world.html

Rewrites the example using the built-in candle_nn::Linear layer. Note that the weight tensor dimensions are transposed compared to the custom version: (100, 784) instead of (784, 100). The model uses Linear::new with an optional bias.

```rust
extern crate candle_core;
extern crate candle_nn;
use candle_core::{Device, Result, Tensor};
use candle_nn::{Linear, Module};

struct Model {
    first: Linear,
    second: Linear,
}

impl Model {
    fn forward(&self, image: &Tensor) -> Result<Tensor> {
        let x = self.first.forward(image)?;
        let x = x.relu()?;
        self.second.forward(&x)
    }
}

fn main() -> Result<()> {
    // Use Device::new_cuda(0)?; to use the GPU.
    let device = Device::Cpu;

    // This has changed (784, 100) -> (100, 784) !
    let weight = Tensor::randn(0f32, 1.0, (100, 784), &device)?;
    let bias = Tensor::randn(0f32, 1.0, (100, ), &device)?;
    let first = Linear::new(weight, Some(bias));
    let weight = Tensor::randn(0f32, 1.0, (10, 100), &device)?;
    let bias = Tensor::randn(0f32, 1.0, (10, ), &device)?;
    let second = Linear::new(weight, Some(bias));
    let model = Model { first, second };

    let dummy_image = Tensor::randn(0f32, 1.0, (1, 784), &device)?;

    let digit = model.forward(&dummy_image)?;
    println!("Digit {digit:?} digit");
    Ok(())
}
```

--------------------------------

### Download and load model weights synchronously

Source: https://huggingface.github.io/candle/inference/hub.html

Download a model from the Hub and load its weights into a Candle device.

```rust
#![allow(unused)]
fn main() {
extern crate candle_core;
extern crate hf_hub;
use hf_hub::api::sync::Api;
use candle_core::Device;

let api = Api::new().unwrap();
let repo = api.model("bert-base-uncased".to_string());

let weights = repo.get("model.safetensors").unwrap();

let weights = candle_core::safetensors::load(weights, &Device::Cpu);
}
```

--------------------------------

### Build project with CUDA support

Source: https://huggingface.github.io/candle/guide/installation.html

Builds the project to ensure everything compiles correctly with CUDA support.

```bash
cargo build
```

--------------------------------

### Create Cargo app with candle-core cuda feature

Source: https://huggingface.github.io/candle/print.html

Create a new Cargo app, add candle-core with the cuda feature, and build it. The cuda feature enables Cuda support; run cargo build to verify the build succeeds.

```bash
cargo new myapp
cd myapp
```

```bash
cargo add --git https://github.com/huggingface/candle.git candle-core --features "cuda"
```

```bash
cargo build
```

--------------------------------

### Load tensor from file

Source: https://huggingface.github.io/candle/guide/cheatsheet.html

Comparison of loading tensors from disk.

```python
weights = torch.load("model.bin")
```

```rust
candle::safetensors::load("model.safetensors", &device)
```

--------------------------------

### Efficient loading with memory mapping

Source: https://huggingface.github.io/candle/inference/hub.html

Use memmap2 to map the model file into memory for more efficient loading.

```rust
use candle::Device;
use hf_hub::api::sync::Api;
use memmap2::Mmap;
use std::fs;

let api = Api::new().unwrap();
let repo = api.model("bert-base-uncased".to_string());
let weights_filename = repo.get("model.safetensors").unwrap();

let file = fs::File::open(weights_filename).unwrap();
let mmap = unsafe { Mmap::map(&file).unwrap() };
let weights = candle::safetensors::load_buffer(&mmap[..], &Device::Cpu).unwrap();
```

--------------------------------

### Create new Cargo project without CUDA

Source: https://huggingface.github.io/candle/guide/installation.html

Commands to create a new Cargo project and add candle-core without CUDA support.

```bash
cargo new myapp
cd myapp
cargo add --git https://github.com/huggingface/candle.git candle-core
```

--------------------------------

### Download and load model weights asynchronously

Source: https://huggingface.github.io/candle/inference/hub.html

Use the async API to download and load model weights.

```rust
use candle::Device;
use hf_hub::api::tokio::Api;

let api = Api::new().unwrap();
let repo = api.model("bert-base-uncased".to_string());

let weights_filename = repo.get("model.safetensors").await.unwrap();

let weights = candle::safetensors::load(weights_filename, &Device::Cpu).unwrap();
```

--------------------------------

### Async download and load with hf-hub tokio API

Source: https://huggingface.github.io/candle/print.html

Downloads and loads model weights asynchronously using the tokio-based hf-hub API. Requires the tokio feature.

```rust
use candle::Device;
use hf_hub::api::tokio::Api;

let api = Api::new().unwrap();
let repo = api.model("bert-base-uncased".to_string());

let weights_filename = repo.get("model.safetensors").await.unwrap();

let weights = candle::safetensors::load(weights_filename, &Device::Cpu).unwrap();

```

--------------------------------

### Save tensor to file

Source: https://huggingface.github.io/candle/guide/cheatsheet.html

Comparison of saving tensors to disk.

```python
torch.save({"A": A}, "model.bin")
```

```rust
candle::safetensors::save(&HashMap::from([("A", A)]), "model.safetensors")?
```

--------------------------------

### Add candle-core with CUDA feature

Source: https://huggingface.github.io/candle/guide/installation.html

Adds the candle-core crate from the GitHub repository with the 'cuda' feature enabled.

```bash
cargo add --git https://github.com/huggingface/candle.git candle-core --features "cuda"
```

--------------------------------

### Add candle-nn dependency

Source: https://huggingface.github.io/candle/guide/hello_world.html

Command to add the candle-nn crate from the Hugging Face Candle repository as a dependency.

```bash
cargo add --git https://github.com/huggingface/candle.git candle-nn
```

--------------------------------

### Download and load model weights with hf-hub sync API

Source: https://huggingface.github.io/candle/print.html

Downloads the model.safetensors file for the bert-base-uncased model using the synchronous hf-hub API and loads it into a Candle tensor map.

```rust
#![allow(unused)]
fn main() {
extern crate candle_core;
extern crate hf_hub;
use hf_hub::api::sync::Api;
use candle_core::Device;

let api = Api::new().unwrap();
let repo = api.model("bert-base-uncased".to_string());

let weights = repo.get("model.safetensors").unwrap();

let weights = candle_core::safetensors::load(weights, &Device::Cpu);
}

```

--------------------------------

### Runtime error output with RUST_BACKTRACE=1

Source: https://huggingface.github.io/candle/print.html

Runtime output with RUST_BACKTRACE=1, showing the backtrace that locates the error in myapp::main at line 29. The backtrace includes the candle library frames and the Rust standard library internals.

```text
Error: WithBacktrace { inner: ShapeMismatchBinaryOp { lhs: [1, 784], rhs: [1, 784], op: "matmul" }, backtrace: Backtrace [{ fn: "candle::error::Error::bt", file: "/home/nicolas/.cargo/git/checkouts/candle-5bb8ef7e0626d693/f291065/candle-core/src/error.rs", line: 200 }, { fn: "candle::tensor::Tensor::matmul", file: "/home/nicolas/.cargo/git/checkouts/candle-5bb8ef7e0626d693/f291065/candle-core/src/tensor.rs", line: 816 }, { fn: "myapp::main", file: "./src/main.rs", line: 29 }, { fn: "core::ops::function::FnOnce::call_once", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/ops/function.rs", line: 250 }, { fn: "std::sys_common::backtrace::__rust_begin_short_backtrace", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/sys_common/backtrace.rs", line: 135 }, { fn: "std::rt::lang_start::{{closure}}", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/rt.rs", line: 166 }, { fn: "core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/ops/function.rs", line: 284 }, { fn: "std::panicking::try::do_call", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs", line: 500 }, { fn: "std::panicking::try", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs", line: 464 }, { fn: "std::panic::catch_unwind", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panic.rs", line: 142 }, { fn: "std::rt::lang_start_internal::{{closure}}", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/rt.rs", line: 148 }, { fn: "std::rt::lang_start", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/rt.rs", line: 165 }, { fn: "main" }, { fn: "__libc_start_main" }, { fn: "_start" }] }
```

--------------------------------

### Backtrace output with RUST_BACKTRACE=1

Source: https://huggingface.github.io/candle/error_manage.html

The error output after setting RUST_BACKTRACE=1. It wraps the original error in WithBacktrace and includes a full backtrace, showing the error occurred at myapp::main in ./src/main.rs line 29.

```text
Error: WithBacktrace { inner: ShapeMismatchBinaryOp { lhs: [1, 784], rhs: [1, 784], op: "matmul" }, backtrace: Backtrace [{ fn: "candle::error::Error::bt", file: "/home/nicolas/.cargo/git/checkouts/candle-5bb8ef7e0626d693/f291065/candle-core/src/error.rs", line: 200 }, { fn: "candle::tensor::Tensor::matmul", file: "/home/nicolas/.cargo/git/checkouts/candle-5bb8ef7e0626d693/f291065/candle-core/src/tensor.rs", line: 816 }, { fn: "myapp::main", file: "./src/main.rs", line: 29 }, { fn: "core::ops::function::FnOnce::call_once", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/ops/function.rs", line: 250 }, { fn: "std::sys_common::backtrace::__rust_begin_short_backtrace", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/sys_common/backtrace.rs", line: 135 }, { fn: "std::rt::lang_start::{{closure}}", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/rt.rs", line: 166 }, { fn: "core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/ops/function.rs", line: 284 }, { fn: "std::panicking::try::do_call", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs", line: 500 }, { fn: "std::panicking::try", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs", line: 464 }, { fn: "std::panic::catch_unwind", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panic.rs", line: 142 }, { fn: "std::rt::lang_start_internal::{{closure}}", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/rt.rs", line: 148 }, { fn: "std::panicking::try::do_call", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs", line: 500 }, { fn: "std::panicking::try", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs", line: 464 }, { fn: "std::panic::catch_unwind", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panic.rs", line: 142 }, { fn: "std::rt::lang_start_internal", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/rt.rs", line: 148 }, { fn: "std::rt::lang_start", file: "/rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/rt.rs", line: 165 }, { fn: "main" }, { fn: "__libc_start_main" }, { fn: "_start" }] }
```

--------------------------------

### Index a tensor

Source: https://huggingface.github.io/candle/guide/cheatsheet.html

Comparison of tensor slicing syntax.

```python
tensor[:, :4]
```

```rust
tensor.i((.., ..4))?
```

--------------------------------

### Create a zero-filled tensor

Source: https://huggingface.github.io/candle/guide/cheatsheet.html

Comparison of creating a 2x2 tensor filled with zeros.

```python
torch.zeros((2, 2))
```

```rust
Tensor::zeros((2, 2), DType::F32, &Device::Cpu)?
```

--------------------------------

### Load weights with memmap2

Source: https://huggingface.github.io/candle/print.html

Loads model weights using memory mapping (memmap2) for more efficient loading. Note: this operation is unsafe and may cause issues on Windows, WSL, or network-mounted disks.

```rust
use candle::Device;
use hf_hub::api::sync::Api;
use memmap2::Mmap;
use std::fs;

let api = Api::new().unwrap();
let repo = api.model("bert-base-uncased".to_string());
let weights_filename = repo.get("model.safetensors").unwrap();

let file = fs::File::open(weights_filename).unwrap();
let mmap = unsafe { Mmap::map(&file).unwrap() };
let weights = candle::safetensors::load_buffer(&mmap[..], &Device::Cpu).unwrap();

```

--------------------------------

### Add hf-hub dependency

Source: https://huggingface.github.io/candle/training/training.html

Adds the hf-hub crate to the project, required for downloading the MNIST dataset from the Hugging Face Hub.

```bash
cargo add hf-hub
```

--------------------------------

### Perform arithmetic addition

Source: https://huggingface.github.io/candle/guide/cheatsheet.html

Comparison of tensor addition syntax.

```python
a + b
```

```rust
&a + &b
```

--------------------------------

### Move tensor to device

Source: https://huggingface.github.io/candle/guide/cheatsheet.html

Comparison of moving tensors between devices.

```python
tensor.to(device="cuda")
```

```rust
tensor.to_device(&Device::new_cuda(0)?)?
```

--------------------------------

### Custom Linear layer with bias

Source: https://huggingface.github.io/candle/print.html

Custom Linear layer with weight and bias, using broadcast_add for bias. Shows how to build a layer manually before using candle_nn.

```rust
#![allow(unused)]
fn main() {
extern crate candle_core;
use candle_core::{Device, Result, Tensor};
struct Linear{
    weight: Tensor,
    bias: Tensor,
}
impl Linear{
    fn forward(&self, x: &Tensor) -> Result<Tensor> {
        let x = x.matmul(&self.weight)?;
        x.broadcast_add(&self.bias)
    }
}

struct Model {
    first: Linear,
    second: Linear,
}

impl Model {
    fn forward(&self, image: &Tensor) -> Result<Tensor> {
        let x = self.first.forward(image)?;
        let x = x.relu()?;
        self.second.forward(&x)
    }
}
}

```

--------------------------------

### Neural Network Training Logs

Source: https://huggingface.github.io/candle/training/simplified.html

Shows the training loss and test accuracy over five epochs, followed by real-life votes and a final prediction result.

```text
__
Trying to train neural network.
Epoch:   1 Train loss:  4.42555 Test accuracy:  0.00%
Epoch:   2 Train loss:  0.84677 Test accuracy: 33.33%
Epoch:   3 Train loss:  2.54335 Test accuracy: 33.33%
Epoch:   4 Train loss:  0.37806 Test accuracy: 33.33%
Epoch:   5 Train loss:  0.36647 Test accuracy: 100.00%
real_life_votes: [13, 22]
neural_network_prediction_result: 0.0
```

--------------------------------

### Candle MLP training and prediction in Rust

Source: https://huggingface.github.io/candle/print.html

These Rust snippets define a multi-layer perceptron for binary classification using the Candle library. The first snippet defines constants, a Dataset struct, and the MultiLevelPerceptron model with two hidden layers and ReLU activations. The second snippet implements the training loop with SGD, logging loss and test accuracy, and returns an error if accuracy is below 100%. The third snippet is a tokio test that prepares data tensors, trains the model, and makes a prediction on new votes, printing the real votes and the predicted result.

```rust
const VOTE_DIM: usize = 2;
const RESULTS: usize = 1;
const EPOCHS: usize = 10;
const LAYER1_OUT_SIZE: usize = 4;
const LAYER2_OUT_SIZE: usize = 2;
const LEARNING_RATE: f64 = 0.05;

#[derive(Clone)]
pub struct Dataset {
    pub train_votes: Tensor,
    pub train_results: Tensor,
    pub test_votes: Tensor,
    pub test_results: Tensor,
}

struct MultiLevelPerceptron {
    ln1: Linear,
    ln2: Linear,
    ln3: Linear,
}

impl MultiLevelPerceptron {
    fn new(vs: VarBuilder) -> Result<Self> {
        let ln1 = candle_nn::linear(VOTE_DIM, LAYER1_OUT_SIZE, vs.pp("ln1"))?;
        let ln2 = candle_nn::linear(LAYER1_OUT_SIZE, LAYER2_OUT_SIZE, vs.pp("ln2"))?;
        let ln3 = candle_nn::linear(LAYER2_OUT_SIZE, RESULTS + 1, vs.pp("ln3"))?;
        Ok(Self { ln1, ln2, ln3 })
    }

    fn forward(&self, xs: &Tensor) -> Result<Tensor> {
        let xs = self.ln1.forward(xs)?;
        let xs = xs.relu()?;
        let xs = self.ln2.forward(&xs)?;
        let xs = xs.relu()?;
        self.ln3.forward(&xs)
    }
}

```

```rust
fn train(m: Dataset, dev: &Device) -> anyhow::Result<MultiLevelPerceptron> {
    let train_results = m.train_results.to_device(dev)?;
    let train_votes = m.train_votes.to_device(dev)?;
    let varmap = VarMap::new();
    let vs = VarBuilder::from_varmap(&varmap, DType::F32, dev);
    let model = MultiLevelPerceptron::new(vs.clone())?;
    let mut sgd = candle_nn::SGD::new(varmap.all_vars(), LEARNING_RATE)?;
    let test_votes = m.test_votes.to_device(dev)?;
    let test_results = m.test_results.to_device(dev)?;
    let mut final_accuracy: f32 = 0.0;
    for epoch in 1..EPOCHS + 1 {
        let logits = model.forward(&train_votes)?;
        let log_sm = ops::log_softmax(&logits, D::Minus1)?;
        let loss = loss::nll(&log_sm, &train_results)?;
        sgd.backward_step(&loss)?;

        let test_logits = model.forward(&test_votes)?;
        let sum_ok = test_logits
            .argmax(D::Minus1)?
            .eq(&test_results)?
            .to_dtype(DType::F32)?
            .sum_all()?
            .to_scalar::<f32>()?;
        let test_accuracy = sum_ok / test_results.dims1()? as f32;
        final_accuracy = 100. * test_accuracy;
        println!("Epoch: {epoch:3} Train loss: {:8.5} Test accuracy: {:5.2}%",
                 loss.to_scalar::<f32>()?,
                 final_accuracy
        );
        if final_accuracy == 100.0 {
            break;
        }
    }
    if final_accuracy < 100.0 {
        Err(anyhow::Error::msg("The model is not trained well enough."))
    } else {
        Ok(model)
    }
}

```

```rust
#[tokio::test]
async fn simplified() -> anyhow::Result<()> {

    let dev = Device::cuda_if_available(0)?;

    let train_votes_vec: Vec<u32> = vec![
        15, 10,
        10, 15,
        5, 12,
        30, 20,
        16, 12,
        13, 25,
        6, 14,
        31, 21,
    ];
    let train_votes_tensor = Tensor::from_vec(train_votes_vec.clone(), (train_votes_vec.len() / VOTE_DIM, VOTE_DIM), &dev)?.to_dtype(DType::F32)?;

    let train_results_vec: Vec<u32> = vec![
        1,
        0,
        0,
        1,
        1,
        0,
        0,
        1,
    ];
    let train_results_tensor = Tensor::from_vec(train_results_vec, train_votes_vec.len() / VOTE_DIM, &dev)?;

    let test_votes_vec: Vec<u32> = vec![
        13, 9,
        8, 14,
        3, 10,
    ];
    let test_votes_tensor = Tensor::from_vec(test_votes_vec.clone(), (test_votes_vec.len() / VOTE_DIM, VOTE_DIM), &dev)?.to_dtype(DType::F32)?;

    let test_results_vec: Vec<u32> = vec![
        1,
        0,
        0,
    ];
    let test_results_tensor = Tensor::from_vec(test_results_vec.clone(), test_results_vec.len(), &dev)?;

    let m = Dataset {
        train_votes: train_votes_tensor,
        train_results: train_results_tensor,
        test_votes: test_votes_tensor,
        test_results: test_results_tensor,
    };

    let trained_model: MultiLevelPerceptron;
    loop {
        println!("Trying to train neural network.");
        match train(m.clone(), &dev) {
            Ok(model) => {
                trained_model = model;
                break;
            },
            Err(e) => {
                println!("Error: {}", e);
                continue;
            }
        }

    }

    let real_world_votes: Vec<u32> = vec![
        13, 22,
    ];

    let tensor_test_votes = Tensor::from_vec(real_world_votes.clone(), (1, VOTE_DIM), &dev)?.to_dtype(DType::F32)?;

    let final_result = trained_model.forward(&tensor_test_votes)?;

    let result = final_result
        .argmax(D::Minus1)?
        .to_dtype(DType::F32)?
        .get(0).map(|x| x.to_scalar::<f32>())??;
    println!("real_life_votes: {:?}", real_world_votes);
    println!("neural_network_prediction_result: {:?}", result);

    Ok(())

}

```

--------------------------------

### Perform matrix multiplication

Source: https://huggingface.github.io/candle/guide/cheatsheet.html

Comparison of matrix multiplication syntax.

```python
a.matmul(b)
```

```rust
a.matmul(&b)?
```

--------------------------------

### Basic MNIST model with raw tensors

Source: https://huggingface.github.io/candle/guide/hello_world.html

Defines a simple two-layer model using raw tensors and manual matmul/relu operations. The main function creates random tensors, runs a dummy image through the model, and prints the output. Use Device::new_cuda(0)? to use the GPU instead of CPU.

```rust
extern crate candle_core;
use candle_core::{Device, Result, Tensor};

struct Model {
    first: Tensor,
    second: Tensor,
}

impl Model {
    fn forward(&self, image: &Tensor) -> Result<Tensor> {
        let x = image.matmul(&self.first)?;
        let x = x.relu()?;
        x.matmul(&self.second)
    }
}

fn main() -> Result<()> {
    // Use Device::new_cuda(0)?; to use the GPU.
    let device = Device::Cpu;

    let first = Tensor::randn(0f32, 1.0, (784, 100), &device)?;
    let second = Tensor::randn(0f32, 1.0, (100, 10), &device)?;
    let model = Model { first, second };

    let dummy_image = Tensor::randn(0f32, 1.0, (1, 784), &device)?;

    let digit = model.forward(&dummy_image)?;
    println!("Digit {digit:?} digit");
    Ok(())
}
```

--------------------------------

### Use loaded weights in a linear layer

Source: https://huggingface.github.io/candle/print.html

Loads a specific weight and bias tensor from the downloaded model and uses them to create a linear layer, then runs a forward pass on a zero tensor.

```rust
#![allow(unused)]
fn main() {
extern crate candle_core;
extern crate candle_nn;
extern crate hf_hub;
use hf_hub::api::sync::Api;

let api = Api::new().unwrap();
let repo = api.model("bert-base-uncased".to_string());

let weights = repo.get("model.safetensors").unwrap();
use candle_core::{Device, Tensor, DType};
use candle_nn::{Linear, Module};

let weights = candle_core::safetensors::load(weights, &Device::Cpu).unwrap();

let weight = weights.get("bert.encoder.layer.0.attention.self.query.weight").unwrap();
let bias = weights.get("bert.encoder.layer.0.attention.self.query.bias").unwrap();

let linear = Linear::new(weight.clone(), Some(bias.clone()));

let input_ids = Tensor::zeros((3, 768), DType::F32, &Device::Cpu).unwrap();
let output = linear.forward(&input_ids).unwrap();
}

```

--------------------------------

### Train the MLP model with SGD and early stopping

Source: https://huggingface.github.io/candle/training/simplified.html

Trains the model using stochastic gradient descent with a fixed learning rate. It computes cross-entropy loss, evaluates test accuracy each epoch, and stops early if accuracy reaches 100%. Returns an error if the model does not reach 100% accuracy.

```rust
fn train(m: Dataset, dev: &Device) -> anyhow::Result<MultiLevelPerceptron> {
    let train_results = m.train_results.to_device(dev)?;
    let train_votes = m.train_votes.to_device(dev)?;
    let varmap = VarMap::new();
    let vs = VarBuilder::from_varmap(&varmap, DType::F32, dev);
    let model = MultiLevelPerceptron::new(vs.clone())?;
    let mut sgd = candle_nn::SGD::new(varmap.all_vars(), LEARNING_RATE)?;
    let test_votes = m.test_votes.to_device(dev)?;
    let test_results = m.test_results.to_device(dev)?;
    let mut final_accuracy: f32 = 0.0;
    for epoch in 1..EPOCHS + 1 {
        let logits = model.forward(&train_votes)?;
        let log_sm = ops::log_softmax(&logits, D::Minus1)?;
        let loss = loss::nll(&log_sm, &train_results)?;
        sgd.backward_step(&loss)?;

        let test_logits = model.forward(&test_votes)?;
        let sum_ok = test_logits
            .argmax(D::Minus1)?
            .eq(&test_results)?
            .to_dtype(DType::F32)?
            .sum_all()?
            .to_scalar::<f32>()?;
        let test_accuracy = sum_ok / test_results.dims1()? as f32;
        final_accuracy = 100. * test_accuracy;
        println!("Epoch: {epoch:3} Train loss: {:8.5} Test accuracy: {:5.2}%",
                 loss.to_scalar::<f32>()?,
                 final_accuracy
        );
        if final_accuracy == 100.0 {
            break;
        }
    }
    if final_accuracy < 100.0 {
        Err(anyhow::Error::msg("The model is not trained well enough."))
    } else {
        Ok(model)
    }
}

```

--------------------------------

### Create a tensor from data

Source: https://huggingface.github.io/candle/guide/cheatsheet.html

Comparison of tensor creation from nested arrays.

```python
torch.Tensor([[1, 2], [3, 4]])
```

```rust
Tensor::new(&[[1f32, 2.], [3., 4.]], &Device::Cpu)?
```
Tokens:
Raw