### F# TorchSharp Setup

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial7.ipynb

Initial setup for TorchSharp in F# using nuget package and opening necessary namespaces.

```F#
#r "nuget: TorchSharp-cpu"

open TorchSharp
open type TorchSharp.torch
open type TorchSharp.TensorExtensionMethods
open type TorchSharp.torch.distributions
```

--------------------------------

### Install TorchSharp NuGet Package

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial1.ipynb

Use this directive to install the TorchSharp-cpu NuGet package. This package provides the CPU backend for TorchSharp and works across Windows, Linux, and macOS.

```python
#r "nuget: TorchSharp-cpu"
```

--------------------------------

### Install TorchSharp CPU Package

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial1.ipynb

Use this directive to install the TorchSharp CPU package via NuGet. This package provides the .NET bindings and the CPU backend for TorchSharp.

```fsharp
#r "nuget:TorchSharp-cpu"
```

--------------------------------

### Load TorchSharp NuGet Package

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/synthetic_data.ipynb

Installs the TorchSharp-cpu NuGet package. This is required to use TorchSharp functionalities.

```fsharp
#r "nuget: TorchSharp-cpu"

open TorchSharp
open type TorchSharp.TensorExtensionMethods
```

--------------------------------

### Import TorchSharp

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/synthetic_data.ipynb

Imports the necessary TorchSharp library for CPU operations. Ensure TorchSharp-cpu is installed via NuGet.

```C#
#r "nuget: TorchSharp-cpu"

using TorchSharp;
using static TorchSharp.TensorExtensionMethods;
```

--------------------------------

### Broadcasting Example

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial3.ipynb

Demonstrates broadcasting by adding a tensor with a compatible shape to a larger tensor.

```F#
let a = torch.ones(3,4)
(a + torch.ones(4)).print()
a + torch.ones(1,4)
```

--------------------------------

### Create and Load a Sequential Model in F#

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial6.ipynb

Define a sequential model with linear layers and a ReLU activation, load its weights from a file, and perform a forward pass to get predictions.

```F#
let seq = nn.Sequential(("lin1", nn.Linear(1000L, 100L) :> nn.Module<Tensor,Tensor>), ("relu", nn.ReLU() :> nn.Module<Tensor,Tensor>), ("lin2", nn.Linear(100L, 10L) :> nn.Module<Tensor,Tensor>))
seq.load("tutorial6.model.bin")
predMax = seq.forward(dataBatch).argmax(1L)
refMax.eq(predMax).sum() / predMax.numel().ToScalar()
```

--------------------------------

### Bernoulli Distribution Sample

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial4.ipynb

Samples from a Bernoulli distribution, which generates binary (0 or 1) outcomes. The probability of getting '1' is determined by the input tensor.

```F#
let bern = Bernoulli(torch.tensor(0.5f))
bern.sample().item<single>()
```

--------------------------------

### Broadcasting Tensor Addition

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial3.ipynb

Illustrates TorchSharp's broadcasting capability, where tensors with compatible shapes (even different dimensions) can be used in arithmetic operations without explicit reshaping. This example shows adding a row vector to a matrix.

```C#
a = torch.ones(3,4);
(a + torch.ones(4)).print();
a + torch.ones(1,4)
```

--------------------------------

### Get Total Number of Elements

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial2.ipynb

Call .numel() to get the total count of elements within a tensor.

```python
// The total number of elements held in the tensor:
t.numel()
```

--------------------------------

### Create and Load Model Weights (TorchSharp)

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Utilize the static `nn.Module.Create` method to both instantiate a model and load its weights from a file in a single step. This is convenient for models with parameterless constructors.

```csharp
Trivial model2 = (Trivial)nn.Module.Create<Trivial>("tutorial6.model.bin");
predMax = model2.forward(dataBatch).argmax(1);
(refMax == predMax).sum() / predMax.numel()
```

--------------------------------

### Create and Sample Uniform Distributions

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial4.ipynb

Shows how to create a Uniform distribution parameterized by low and high boundaries, and sample from it.

```F#
let uni = Uniform(torch.tensor(10.0f), torch.tensor(17.0f))
foo uni
```

--------------------------------

### Initialize and Use Tensorboard SummaryWriter (F#)

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial6.ipynb

Set up a SummaryWriter for Tensorboard logging. Specify a directory and whether to create a unique run name. This is used to log scalar values during training.

```fsharp
let writer = torch.utils.tensorboard.SummaryWriter("runs/trivial", createRunName=true)

let model = Trivial()
let optimizer = torch.optim.SGD(model.parameters(), learning_rate)

for epoch = 1 to 50 do
    for itrt = 1 to 20 do
        // Compute the loss
        let pred = model.forward(dataBatch)
        let output = loss pred resultBatch

        // Clear the gradients before doing the back-propagation
        model.zero_grad()

        // Do back-propagation, which computes all the gradients.
        output.backward()

        optimizer.step() |> ignore

    let pred = model.forward(dataBatch)
    let l = (loss pred resultBatch).item<single>()
    writer.add_scalar("fsharp/loss", l, epoch)
```

--------------------------------

### Initialize Tensors and Perform Basic Arithmetic

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial3.ipynb

Demonstrates creating tensors with ones and zeros, and performing element-wise multiplication and addition. Uses broadcasting for scalar multiplication.

```C#
var a = torch.ones(3,4);
var b = torch.zeros(3,4);
var c = torch.tensor(5);
a * c + b
```

--------------------------------

### Get Tensor Shape

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial2.ipynb

Access the .shape property to retrieve the dimensions of a tensor.

```python
t = torch.arange(3.0f, 5.0f, step: 0.1f).reshape(2,2,5);

// The overall shape of the tensor:
t.shape
```

--------------------------------

### Create and Sample Normal Distributions

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial4.ipynb

Demonstrates creating Normal distributions with specified mean and standard deviation, and sampling from them.

```F#
let foo (dist:Distribution) = dist.sample(4L,4L)

let norm1 = Normal(torch.tensor(0.5f), torch.tensor(0.125f))
let norm2 = Normal(torch.tensor(0.15f), torch.tensor(0.025f))

(foo norm1).print()
(foo norm2).print()
```

--------------------------------

### Get Number of Dimensions

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial2.ipynb

Use the .ndim property to find out how many dimensions a tensor has.

```python
// The number of dimensions:
t.ndim
```

--------------------------------

### Initialize Model, Loss, Optimizer, and Scheduler

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/synthetic_data.ipynb

Initializes a 'Net2' model instance, Mean Squared Error loss function, and an Rprop optimizer with a specified learning rate. An ExponentialLR scheduler is also set up.

```C#
var model = new Net2(6);

var loss = torch.nn.MSELoss();

var learning_rate = 0.01f;
var optimizer = torch.optim.Rprop(model.parameters(), learning_rate);
var scheduler = torch.optim.lr_scheduler.ExponentialLR(optimizer);
```

--------------------------------

### Instantiate Model with Custom Constructor (TorchSharp)

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Create a model instance using a custom constructor that directly loads weights from a specified file path. This simplifies the process of initializing a model with pre-trained weights.

```csharp
var model3 = new Trivial("tutorial6.model.bin");
predMax = model3.forward(dataBatch).argmax(1);
(refMax == predMax).sum() / predMax.numel()
```

--------------------------------

### Initialize Model, Loss, Optimizer, and Scheduler

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/synthetic_data.ipynb

Initializes the neural network model, specifies the loss function (Mean Squared Error), and sets up the optimizer (Rprop) and learning rate scheduler (ExponentialLR).

```F#
let model = new Net2(6)

let loss = torch.nn.MSELoss()

let learning_rate = 0.01
let optimizer = torch.optim.Rprop(model.parameters(), learning_rate)
let scheduler = torch.optim.lr_scheduler.ExponentialLR(optimizer)
```

--------------------------------

### Get Transpose of a Tensor

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial2.ipynb

Obtain the transpose of a tensor using the .T property. Note that transposing is different from reshaping.

```python
// Getting the transpose of a matrix:
torch.arange(3.0f, 5.0f, step: 0.1f).reshape(4,5).print();
torch.arange(3.0f, 5.0f, step: 0.1f).reshape(4,5).T.print();
```

```python
torch.arange(3.0f, 5.0f, step: 0.1f).reshape(5,4).print(fltFormat: "0.00");
torch.arange(3.0f, 5.0f, step: 0.1f).reshape(4,5).T.print(fltFormat: "0.00");
```

--------------------------------

### Get Number of Dimensions

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial2.ipynb

Determine the number of dimensions in a tensor using the `.ndim` property. This indicates the rank of the tensor.

```python
t.ndim
```

--------------------------------

### Sample from a Distribution using a Specific Generator

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial4.ipynb

Create a distribution instance and pass a specific generator object during initialization. Sampling from this distribution will use the provided generator's state.

```C#
norm1 = Normal(torch.tensor(0.5f), torch.tensor(0.125f), generator: gen2);
norm1.sample(10)
```

--------------------------------

### Sample from a Normal Distribution with a Generator

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial4.ipynb

Shows how to create a Normal distribution instance that is parameterized by a specific generator object.

```F#
let norm1 = Normal(torch.tensor(0.5f), torch.tensor(0.125f), generator=gen2)
norm1.sample(10L)
```

--------------------------------

### Get Max Index for Ground Truth

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial6.ipynb

Extracts the index of the maximum value from the result batch, representing the ground truth labels.

```F#
let refMax = resultBatch.argmax(1L)
refMax
```

--------------------------------

### Get Ground Truth Maximum Index

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Extracts the index of the maximum value from the ground truth batch. This is used as a reference for comparison.

```csharp
var refMax = resultBatch.argmax(1);
refMax
```

--------------------------------

### Basic Training Loop without LR Scheduler

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial7.ipynb

Sets up a basic training loop using a Trivial model, MSELoss, and SGD optimizer. This serves as a baseline before introducing learning rate schedulers.

```csharp
var learning_rate = 0.01f;
var model = new Trivial();
var loss = nn.MSELoss();

var data = Enumerable.Range(0,16).Select(_ => rand(32,1000)).ToList<torch.Tensor>();  // Our pretend input data
var results = Enumerable.Range(0,16).Select(_ => rand(32,10)).ToList<torch.Tensor>();  // Our pretend ground truth.

var optimizer = torch.optim.SGD(model.parameters(), learning_rate);

for (int i = 0; i < 300; i++) {

    for (int idx = 0; idx < data.Count; i++) {
        // Compute the loss
        using var output = loss.forward(model.forward(data[idx]), results[idx]);

        // Clear the gradients before doing the back-propagation
        model.zero_grad();

        // Do back-propagation, which computes all the gradients.
        output.backward();

        optimizer.step();
    }
}

loss.forward(model.forward(data[0]), results[0]).item<float>()
```

--------------------------------

### Get Total Number of Elements

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial2.ipynb

Calculate the total number of elements in a tensor by calling the `.numel()` method. This is the product of all dimensions' sizes.

```python
t.numel()
```

--------------------------------

### Setting up TorchSharp NuGet Package

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

This code block is used in environments like .NET Interactive notebooks to load the TorchSharp-cpu NuGet package. It also includes necessary using directives for TorchSharp functionalities.

```csharp
#r "nuget: TorchSharp-cpu"

using TorchSharp;
using static TorchSharp.torch;
using static TorchSharp.TensorExtensionMethods;
using static TorchSharp.torch.distributions;
```

--------------------------------

### Create and Sample from Normal and Uniform Distributions

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial4.ipynb

Instantiate Normal and Uniform distributions with specified parameters and sample from them using a provided function. Ensure the distribution parameters are tensors.

```C#
torch.Tensor foo(Distribution dist) { return dist.sample(4,4);}

var norm1 = Normal(torch.tensor(0.5f), torch.tensor(0.125f));
var norm2 = Normal(torch.tensor(0.15f), torch.tensor(0.025f));

foo(norm1).print();
foo(norm2).print();
```

```C#
var uni = Uniform(torch.tensor(10.0f), torch.tensor(17.0f));
foo(uni)
```

--------------------------------

### Get Tensor Shape

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial2.ipynb

Access the overall shape of a tensor using the `.shape` property. This returns a tuple representing the size of each dimension.

```python
let t = torch.arange(3.0f, 5.0f, step=0.1f).reshape(2,2,5);
t.shape
```

--------------------------------

### Initialize TorchSharp

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial3.ipynb

Initializes TorchSharp and imports necessary modules for tensor operations.

```F#
#r "nuget: TorchSharp-cpu"

open TorchSharp
open type TorchSharp.torch
open type TorchSharp.TensorExtensionMethods
```

--------------------------------

### Get Predicted Maximum Index

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Calculates the index of the maximum value from the model's predictions for the data batch. This represents the model's output.

```csharp
var predMax = model.forward(dataBatch).argmax(1);
predMax
```

--------------------------------

### Moving Model Parameters to GPU

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial5.ipynb

Illustrates how to move a model's parameters to the GPU using the 'to()' method. This is essential for training models on GPU hardware.

```F#
let model = ...
model.``to``(torch.CUDA)
```

--------------------------------

### Get Max Index for Model Predictions

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial6.ipynb

Calculates the index of the maximum value from the model's forward pass output, representing the model's predictions.

```F#
let predMax = model.forward(dataBatch).argmax(1L)
predMax
```

--------------------------------

### Get Transpose of a Matrix

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial2.ipynb

Obtain the transpose of a tensor using the `.T` property. Note that transposing is different from reshaping, as it swaps dimensions rather than just changing the layout.

```python
torch.arange(3.0f, 5.0f, step=0.1f).reshape(4,5).print()
torch.arange(3.0f, 5.0f, step=0.1f).reshape(4,5).T
```

```python
torch.arange(3.0f, 5.0f, step=0.1f).reshape(5L,4L).print()
torch.arange(3.0f, 5.0f, step=0.1f).reshape(4L,5L).T
```

--------------------------------

### Model Training with SGD Optimizer in F#

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial6.ipynb

Shows how to use the SGD optimizer for a single training step, simplifying weight updates. This is a common approach for model training.

```fsharp
let learning_rate = 0.001

let optimizer = torch.optim.SGD(model.parameters(), learning_rate)

// Compute the loss
let pred = model.forward(dataBatch)
let output = loss pred resultBatch

// Clear the gradients before doing the back-propagation
model.zero_grad()

// Do back-propagation, which computes all the gradients.
output.backward()

optimizer.step()

(loss pred resultBatch).item<single>()
```

--------------------------------

### Create Tensor with Normal Distribution

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial2.ipynb

Generates a tensor with random numbers drawn from a normal distribution. No specific setup is required beyond importing the TorchSharp library.

```python
torch.randn(3,4)
```

--------------------------------

### Training Loop with StepLR Scheduler

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial7.ipynb

Demonstrates training a Trivial model using an SGD optimizer and a StepLR learning rate scheduler. The scheduler reduces the learning rate by 5% every 25 epochs.

```csharp
var learning_rate = 0.01f;
var model = new Trivial();
var loss = nn.MSELoss();

var data = Enumerable.Range(0,16).Select(_ => rand(32,1000)).ToList<torch.Tensor>();  // Our pretend input data
var results = Enumerable.Range(0,16).Select(_ => rand(32,10)).ToList<torch.Tensor>();  // Our pretend ground truth.

var optimizer = torch.optim.SGD(model.parameters(), learning_rate);
var scheduler = torch.optim.lr_scheduler.StepLR(optimizer, 25, 0.95);

for (int i = 0; i < 300; i++) {

    for (int idx = 0; idx < data.Count; i++) {
        // Compute the loss
        using var output = loss.forward(model.forward(data[idx]), results[idx]);

        // Clear the gradients before doing the back-propagation
        model.zero_grad();

        // Do back-propagation, which computes all the gradients.
        output.backward();

        optimizer.step();
    }

    scheduler.step();
}

loss.forward(model.forward(data[0]), results[0]).item<float>()
```

--------------------------------

### Custom Model Constructor with Weight Loading (TorchSharp)

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Define a custom constructor for your model that takes a file path. Ensure the `load` call occurs after `RegisterComponents` to correctly initialize the model's state.

```csharp
private class Trivial : nn.Module<Tensor,Tensor>
{
    public Trivial(String path) : this()
    {
        this.load(path);
    }

    public Trivial() : base(nameof(Trivial))
    {
        RegisterComponents();
    }

    public override Tensor forward(Tensor input)
    {
        using var x = lin1.forward(input);
        using var y = nn.functional.relu(x);
        return lin2.forward(y);
    }

    private nn.Module<Tensor,Tensor> lin1 = nn.Linear(1000, 100);
    private nn.Module<Tensor,Tensor> lin2 = nn.Linear(100, 10);
}
```

--------------------------------

### Get Imaginary Part of Complex Tensor

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial2.ipynb

Access the imaginary components of a complex tensor using the `.imag` property accessor. This returns a new tensor containing only the imaginary parts.

```python
ct.imag
```

--------------------------------

### Model Training with SGD and Higher Learning Rate in F#

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial6.ipynb

Illustrates using the SGD optimizer with a larger learning rate. Adjusting the learning rate can affect convergence speed and stability.

```fsharp
let learning_rate = 0.01

let optimizer = torch.optim.SGD(model.parameters(), learning_rate)

// Compute the loss
let pred = model.forward(dataBatch)
let output = loss pred resultBatch

// Clear the gradients before doing the back-propagation
model.zero_grad()

// Do back-propagation, which computes all the gradients.
output.backward()

optimizer.step()

(loss pred resultBatch).item<single>()
```

--------------------------------

### Get Real Part of Complex Tensor

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial2.ipynb

Access the real components of a complex tensor using the `.real` property accessor. This returns a new tensor containing only the real parts.

```python
ct.real
```

--------------------------------

### Create 1D Tensor with Integer Range

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial2.ipynb

Generates a 1D tensor containing a sequence of integers from a start value (inclusive) to an end value (exclusive). The default step is 1.

```python
torch.arange(3L,14L)
```

--------------------------------

### Manual Model Training Step in F#

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial6.ipynb

Demonstrates a single step of model training by manually updating weights after computing gradients. Use this for understanding the underlying mechanics of training.

```fsharp
let learning_rate = 0.001f.ToScalar()

// Compute the loss
let pred = model.forward(dataBatch)
let output = loss pred resultBatch

// Clear the gradients before doing the back-propagation
model.zero_grad()

// Do back-propagation, which computes all the gradients.
output.backward()

using(torch.no_grad()) (fun _ -> 
    for param in model.parameters() do
        let grad = param.grad()
        match grad with
        | null -> ()
        | _ -> 
            let update = grad.mul(learning_rate)
            param.sub_(update) |> ignore
)
(loss pred resultBatch).item<single>()
```

--------------------------------

### Create DataLoader for Training

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/synthetic_data.ipynb

Instantiate a DataLoader using a custom dataset for batching and shuffling training data. This prepares data for the training loop.

```csharp
var training_data = new SyntheticDataset("train.dat");
var train = new torch.utils.data.DataLoader(training_data, 200, shuffle: true);
```

--------------------------------

### Load Model Weights and Predict (TorchSharp)

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Instantiate a model with random weights and then load the saved weights before making predictions. This demonstrates the effect of loaded weights on model output.

```csharp
var model1 = new Trivial();
predMax = model1.forward(dataBatch).argmax(1);
(refMax == predMax).sum() / predMax.numel()
```

```csharp
model1.load("tutorial6.model.bin");
predMax = model1.forward(dataBatch).argmax(1);
(refMax == predMax).sum() / predMax.numel()
```

--------------------------------

### Moving Model Parameters to GPU

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial5.ipynb

Illustrates how to move a model's parameters to the GPU using the `to()` method. This is a common step in preparing models for training or inference on GPU hardware.

```C#
var model = ...;
model.to(torch.CUDA);
```

--------------------------------

### Model Training with Adam Optimizer in F#

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial6.ipynb

Demonstrates using the Adam optimizer for model training. Adam is an adaptive learning rate optimization algorithm.

```fsharp
let learning_rate = 0.01

let optimizer = torch.optim.Adam(model.parameters())

// Compute the loss
let pred = model.forward(dataBatch)
let output = loss pred resultBatch

// Clear the gradients before doing the back-propagation
model.zero_grad()

// Do back-propagation, which computes all the gradients.
output.backward()

optimizer.step()

(loss pred resultBatch).item<single>()
```

--------------------------------

### Training Loop with StepLR Scheduler in F#

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial7.ipynb

Demonstrates training a neural network using the StepLR learning rate scheduler in F#. The scheduler adjusts the learning rate every 25 epochs by a factor of 0.95.

```F#
let learning_rate = 0.01
let model = Trivial()

let data = [for i = 1 to 16 do rand(32,1000)]  // Our pretend input data
let result = [for i = 1 to 16 do rand(32,10)]  // Our pretend ground truth.

let loss x y = nn.functional.mse_loss(x,y)

let optimizer = torch.optim.SGD(model.parameters(), learning_rate)
let scheduler = torch.optim.lr_scheduler.StepLR(optimizer, 25, 0.95, verbose=true)

for epoch = 1 to 300 do

    for idx = 0 to data.Length-1 do
        // Compute the loss
        let pred = model.forward(data.[idx])
        let output = loss pred result.[idx]

        // Clear the gradients before doing the back-propagation
        model.zero_grad()

        // Do back-propagation, which computes all the gradients.
        output.backward()

        optimizer.step() |> ignore

    scheduler.step() |> ignore

let pred = model.forward(data.[0])
(loss pred result.[0]).item<single>()
```

--------------------------------

### Display Tensor with Default Formatting

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial1.ipynb

This code snippet demonstrates creating a tensor and displaying it using the default formatting settings in .NET Interactive. TorchSharp now handles this automatically.

```csharp
torch.ones(2,3,3)
```

--------------------------------

### Open TorchSharp Namespaces

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial1.ipynb

Include these 'open' directives to conveniently access TorchSharp functionalities. For tutorials, explicit qualification is often preferred for clarity.

```fsharp
open TorchSharp

open type TorchSharp.torch
open type TorchSharp.torch.nn
open type TorchSharp.torch.nn.functional
open type TorchSharp.TensorExtensionMethods
```

--------------------------------

### Categorical Distribution Sampling

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial4.ipynb

Shows how to sample from a Categorical distribution, which supports N different categories. The probabilities for each category are provided as a tensor. The length of this tensor determines the number of categories.

```C#
var cat = Categorical(torch.tensor(new float[]{0.1f, 0.7f, 0.1f, 0.1f}));
cat.sample(4)
```

--------------------------------

### Model Training with SGD Optimizer and Increased Learning Rate

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Demonstrates training with the SGD optimizer using a higher learning rate. Adjusting the learning rate can affect convergence speed and stability. Gradients are cleared before back-propagation.

```C#
var learning_rate = 0.01f;

var optimizer = torch.optim.SGD(model.parameters(), learning_rate);

// Compute the loss
var output = loss.forward(model.forward(dataBatch), resultBatch);

// Clear the gradients before doing the back-propagation
model.zero_grad();

// Do back-propagation, which computes all the gradients.
output.backward();

optimizer.step();

loss.forward(model.forward(dataBatch), resultBatch).item<float>()

```

--------------------------------

### Instantiate and Use Generative Network

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/synthetic_data.ipynb

Creates an instance of the 'Net' generative network with 6 input features and then uses the 'create_data_file' function to generate training and testing datasets.

```fsharp
let net = new Net(6)
```

```fsharp
create_data_file(net, 6, "train.dat", 2000);
create_data_file(net, 6, "test.dat", 400);
```

--------------------------------

### Standard Training Loop

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/synthetic_data.ipynb

Performs a standard training loop for 10000 epochs on a single batch of data. It prints the initial and final loss and updates the learning rate scheduler every 100 epochs.

```C#
Console.WriteLine(" initial loss = " + loss.forward(model.forward(X_train), y_train).item<float>().ToString());

for (int i = 0; i < 10000; i++) {

    // Compute the loss
    using var output = loss.forward(model.forward(X_train), y_train);

    // Clear the gradients before doing the back-propagation
    model.zero_grad();

    // Do back-propagation, which computes all the gradients.
    output.backward();

    optimizer.step();
    
    if (i % 100 == 99) {
        scheduler.step();
    }
}

Console.WriteLine(" final loss   = " + loss.forward(model.forward(X_train), y_train).item<float>());
```

--------------------------------

### Training Loop with Batches

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/synthetic_data.ipynb

Implements a training loop that iterates over batches of data for 5000 epochs. It calculates loss, performs back-propagation, and updates the model weights for each batch, adjusting the learning rate scheduler after each epoch.

```C#
Console.WriteLine(" initial loss = " + loss.forward(model.forward(X_train), y_train).item<float>().ToString());

for (int i = 0; i < 5000; i++) {

    for (var j = 0; j < X_batch.Length; j++) {
        // Compute the loss
        using var output = loss.forward(model.forward(X_batch[j]), y_batch[j]);

        // Clear the gradients before doing the back-propagation
        model.zero_grad();

        // Do back-propagation, which computes all the gradients.
        output.backward();

        optimizer.step();
    }
    
    scheduler.step();
}

Console.WriteLine(" final loss   = " + loss.forward(model.forward(X_train), y_train).item<float>());
```

--------------------------------

### Binomial Distribution Sampling

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial4.ipynb

Demonstrates sampling from a Binomial distribution, which counts the number of successes in a fixed number of independent Bernoulli trials. The first argument is the number of trials, and the second is the probability of success.

```C#
var bin = Binomial(torch.tensor(100), torch.tensor(0.25f));
bin.sample().item<float>()
```

```C#
bin.sample(3,4)
```

--------------------------------

### Create DataLoader for Training in F#

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/synthetic_data.ipynb

Instantiate a DataLoader with the custom SyntheticDataset. This allows for batching and shuffling of training data, essential for effective model training.

```F#
let training_data = new SyntheticDataset("train.dat")
let train = new torch.utils.data.DataLoader(training_data, 200, shuffle=true)
```

--------------------------------

### Create a Sequential Model in Python

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Construct a simple sequential model by providing a sequence of named layer tuples. Weights can be loaded into such models if layer definitions and names match.

```python
var seq = nn.Sequential(("lin1", nn.Linear(1000, 100)), ("relu1", nn.ReLU()), ("lin2", nn.Linear(100, 10)));
seq.load("tutorial6.model.bin");
predMax = model3.forward(dataBatch).argmax(1);
(refMax == predMax).sum() / predMax.numel()
```

--------------------------------

### Prepare Data for Model Training in TorchSharp

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Generate synthetic input and ground truth data for training a TorchSharp model. This involves creating tensors with appropriate dimensions for batch processing.

```C#
var dataBatch = rand(32,1000);  // Our pretend input data
var resultBatch = rand(32,10);  // Our pretend ground truth.
dataBatch.ToString()
```

--------------------------------

### Manual Model Training Step in TorchSharp

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Performs a single training step by computing loss, back-propagating gradients, and manually updating model weights. Ensure gradients are cleared before back-propagation.

```C#
var learning_rate = 0.001f;

// Compute the loss
var output = loss.forward(model.forward(dataBatch), resultBatch);

// Clear the gradients before doing the back-propagation
model.zero_grad();

// Do back-propagation, which computes all the gradients.
output.backward();

// Adjust the weights using the gradients.
using (torch.no_grad()) {
    foreach (var param in model.parameters()) {
        var grad = param.grad();
        if (grad is not null) {
            var update = grad.mul(learning_rate);
            param.sub_(update);
        }
    }
}

loss.forward(model.forward(dataBatch), resultBatch).item<float>()

```

--------------------------------

### Log Training Loss with Tensorboard

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Use SummaryWriter to log scalar values, such as training loss, during model training. Ensure gradients are cleared before back-propagation and the optimizer step is called after. The logging directory defaults to 'runs' if not specified.

```C#
var writer = torch.utils.tensorboard.SummaryWriter("runs/trivial", createRunName:true);

var model = new Trivial();

var optimizer = torch.optim.SGD(model.parameters(), learning_rate);

for (int i = 0; i < 50; i++) {
    for (int j = 0; j < 20; j++) {
        // Compute the loss
        using var output = loss.forward(model.forward(dataBatch), resultBatch);

        // Clear the gradients before doing the back-propagation
        model.zero_grad();

        // Do back-propagation, which computes all the gradients.
        output.backward();

        optimizer.step();
    }

    var l = loss.forward(model.forward(dataBatch), resultBatch).item<float>();

    writer.add_scalar("csharp/loss", l, i);
}
```

--------------------------------

### Multi-dimensional Batch Input for TorchSharp

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Shows how to input data with multiple preceding dimensions into a TorchSharp model. While possible, not all operators support more than one preceding dimension.

```C#
model.forward(rand(2,3,1000))
```

--------------------------------

### Tensor Operations on CUDA Devices

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial5.ipynb

Demonstrates tensor addition on the same CUDA device and highlights the exception that occurs when attempting to add tensors from different devices (CUDA and CPU).

```C#
var a = torch.ones(3,4, device: torch.CUDA);
var b = torch.ones(3,4, device: torch.CUDA);
var c = torch.ones(3,4, device: torch.CPU);

(a + b).print();
(a + c).print();
```

--------------------------------

### Batch Input for TorchSharp Linear Layer

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Demonstrates feeding batched data into a TorchSharp linear layer. The layer's input size refers to the last dimension, allowing for batch dimensions.

```C#
model.forward(rand(3,1000))
```

--------------------------------

### Invoke a TorchSharp Model with Random Data

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Instantiate and run a TorchSharp model with a single tensor input. Ensure the input tensor's size matches the model's expected input dimensions.

```C#
var input = rand(1000);
var model = new Trivial();
model.forward(input)
```

--------------------------------

### Instantiate Generative Network

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/synthetic_data.ipynb

Creates an instance of the generative network with 6 input features.

```C#
var net = new Net(6);
```

--------------------------------

### Train Neural Network (Single Batch)

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/synthetic_data.ipynb

Performs a standard training loop for a fixed number of epochs (1000). It calculates the loss, performs back-propagation, and updates model weights. The learning rate is adjusted using a scheduler every 100 epochs.

```F#
printf " initial loss = %s\n" (loss.forward(model.forward(X_train), y_train).item<float32>().ToString())

for epoch = 1 to 1000 do

    let output = loss.forward(model.forward(X_train), y_train)
    
    // Clear the gradients before doing the back-propagation
    model.zero_grad()

    // Do back-propagation, which computes all the gradients.
    output.backward()

    optimizer.step() |> ignore

    if epoch % 100 = 99 then
        scheduler.step()

printf " final loss   = %s\n" (loss.forward(model.forward(X_train), y_train).item<float32>().ToString())
```

--------------------------------

### Basic Random Number Generation in TorchSharp

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial4.ipynb

Demonstrates the usage of basic random number generation functions: rand(), randn(), randint(), and randperm(). These functions are straightforward to use for generating tensors with random values.

```C#
torch.rand(10).print();
torch.randn(10).print();
torch.randint(100,10).print();
torch.randperm(25).print();
```

--------------------------------

### Create Synthetic Data File

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/synthetic_data.ipynb

Generates a synthetic dataset using the provided generative network and saves it to binary files. Includes adding noise and ensuring non-negative output values.

```C#
void CreateDataFile(Net net, int n_in, string fileName, int n_items)
{

    var x_lo = -1.0;
    var x_hi = 1.0;

    var X = (x_hi - x_lo) * torch.rand(new long[] {n_items, n_in}) + x_lo;

    torch.Tensor y;

    using (torch.no_grad()) {
        y = net.call(X);
    }

    // Add some noise in order not to make it too easy to train...
    y += torch.randn(y.shape) * 0.01;

    // Make sure that the output isn't negative.
    y = torch.where(y < 0.0, y + 0.01 * torch.randn(y.shape) + 0.01, y);

    // Save the data in two separate, binary files.
    X.save(fileName + ".x");
    y.save(fileName + ".y");
}
```

--------------------------------

### Using DisposeScope to Manage Tensor Lifetimes

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial5.ipynb

Demonstrates how DisposeScope automatically manages tensor lifetimes. Compare tensor counts with and without the 'use d = ...' line to observe the effect on memory management.

```F#
printf "%d\n" torch.Tensor.TotalCount
do 
    use d = torch.NewDisposeScope()
    let t3 = (a + b) * (a + c.cuda())
    t3.print() |> ignore

printf "%d\n" torch.Tensor.TotalCount
```

--------------------------------

### Print Tensors Using the `print()` Extension Method

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial2.ipynb

Uses the convenient `print()` extension method for displaying tensors in notebooks. This is a shorthand for console output.

```C#
torch.zeros(4,4).print();
torch.ones(4,4).print();
```

--------------------------------

### Load Synthetic Data File

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/synthetic_data.ipynb

Loads a previously saved synthetic dataset from binary files. Assumes data is saved in '.x' and '.y' files.

```C#
(torch.Tensor X, torch.Tensor y) LoadDataFile(string fileName)
{
    return (torch.Tensor.load(fileName + ".x"), torch.Tensor.load(fileName + ".y"));
}
```

--------------------------------

### Invoke a Trivial Model with Random Data

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial6.ipynb

Demonstrates how to instantiate and call the `Trivial` model with a random input tensor. The input tensor must match the expected input size of the model's first layer.

```F#
let input = rand(1000)
let model = Trivial()
model.forward(input)
```

--------------------------------

### Print Tensors Using Console Output

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial2.ipynb

Demonstrates printing tensors to the console using `Console.WriteLine` and `Console.Write` with a specific tensor string style. This is an alternative to the default notebook display.

```C#
var style = torch.TensorStringStyle;
Console.WriteLine(torch.zeros(4,4).ToString(style));
Console.Write(torch.ones(4,4).ToString(style));
```

--------------------------------

### Basic Training Loop in F#

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial7.ipynb

A basic F# training loop for a neural network without a learning rate scheduler. It iterates through epochs and batches, computes loss, and updates model weights.

```F#
let learning_rate = 0.01
let model = Trivial()

let data = [for i = 1 to 16 do rand(32,1000)]  // Our pretend input data
let result = [for i = 1 to 16 do rand(32,10)]  // Our pretend ground truth.

let loss x y = nn.functional.mse_loss(x,y)

let optimizer = torch.optim.SGD(model.parameters(), learning_rate)

for epoch = 1 to 300 do

    for idx = 0 to data.Length-1 do
        // Compute the loss
        let pred = model.forward(data.[idx])
        let output = loss pred result.[idx]

        // Clear the gradients before doing the back-propagation
        model.zero_grad()

        // Do back-propagation, which computes all the gradients.
        output.backward()

        optimizer.step() |> ignore

let pred = model.forward(data.[0])
(loss pred result.[0]).item<single>()
```

--------------------------------

### Perform Operations on CUDA Tensors

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial5.ipynb

Demonstrates creating and performing addition operations on tensors residing on the CUDA device. Mixing CPU and CUDA tensors in an operation will result in an exception.

```F#
let a = torch.ones(3,4, device=torch.CUDA)
let b = torch.ones(3,4, device=torch.CUDA)
let c = torch.ones(3,4, device=torch.CPU)

(a + b).print()
(a + c).print()
```

--------------------------------

### Training Loop with DataLoader in F#

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/synthetic_data.ipynb

Iterate through the DataLoader within the training loop to process data in batches. This snippet includes gradient zeroing, back-propagation, optimizer step, and scheduler updates for each epoch.

```F#
printf " initial loss = %s\n" (loss.forward(model.forward(X_train), y_train).item<float32>().ToString())

for epoch = 1 to 1000 do

    for data in train do

        let output = loss.forward(model.forward(data["data"]), data["label"])
        
        // Clear the gradients before doing the back-propagation
        model.zero_grad()

        // Do back-propagation, which computes all the gradients.
        output.backward()

        optimizer.step() |> ignore

    scheduler.step()

printf " final loss   = %s\n" (loss.forward(model.forward(X_train), y_train).item<float32>().ToString())
```

--------------------------------

### Model Training with Adam Optimizer in TorchSharp

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Employs the Adam optimizer for model training, which often provides adaptive learning rates. This is an alternative to SGD. Gradients are cleared before back-propagation.

```C#
var optimizer = torch.optim.Adam(model.parameters());

// Compute the loss
var output = loss.forward(model.forward(dataBatch), resultBatch);

// Clear the gradients before doing the back-propagation
model.zero_grad();

// Do back-propagation, which computes all the gradients.
output.backward();

optimizer.step();

loss.forward(model.forward(dataBatch), resultBatch).item<float>()

```

--------------------------------

### Training Loop with SGD Optimizer in TorchSharp

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Implements a training loop for multiple epochs using the SGD optimizer. This structure is typical for deep learning training, iterating over the dataset multiple times. Ensure gradients are cleared before back-propagation in each iteration.

```C#
var learning_rate = 0.01f;
model = new Trivial();

var optimizer = torch.optim.SGD(model.parameters(), learning_rate);

for (int i = 0; i < 1000; i++) {
    // Compute the loss
    using var output = loss.forward(model.forward(dataBatch), resultBatch);

    // Clear the gradients before doing the back-propagation
    model.zero_grad();

    // Do back-propagation, which computes all the gradients.
    output.backward();

    optimizer.step();
}

loss.forward(model.forward(dataBatch), resultBatch).item<float>()

```

--------------------------------

### Training Loop with DataLoader

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/synthetic_data.ipynb

Iterate through the DataLoader within the training loop to process data in batches. This includes computing loss, back-propagation, and optimizer steps.

```csharp
Console.WriteLine(" initial loss = " + loss.forward(model.forward(X_train), y_train).item<float>().ToString());

for (int i = 0; i < 1000; i++) {

    foreach (var data in train)
    {
        // Compute the loss
        using var output = loss.forward(model.forward(data["data"]), data["label"]);

        // Clear the gradients before doing the back-propagation
        model.zero_grad();

        // Do back-propagation, which computes all the gradients.
        output.backward();

        optimizer.step();
    }
    
    scheduler.step();
}

Console.WriteLine(" final loss   = " + loss.forward(model.forward(X_train), y_train).item<float>());
```

--------------------------------

### Generate Training and Testing Data

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/synthetic_data.ipynb

Generates synthetic training and testing datasets using the 'CreateDataFile' method. Training data has 2000 items, and testing data has 400 items, both with 6 input features.

```C#
CreateDataFile(net, 6, "train.dat", 2000);
CreateDataFile(net, 6, "test.dat", 400);
```

--------------------------------

### Binomial Distribution Tensor Sample

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial4.ipynb

Generates a tensor of samples from a Binomial distribution with specified dimensions. The `sample()` method takes the desired dimensions as arguments.

```F#
bin.sample(3L,4L)
```

--------------------------------

### Load Model Weights (F#)

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial6.ipynb

Load previously saved model weights into a new instance of the model. Ensure the model structure matches the saved weights.

```fsharp
let model2 = nn.Module.Create<Trivial>("tutorial6.model.bin") :?> Trivial
let predMax = model2.forward(dataBatch).argmax(1L)
refMax.eq(predMax).sum() / predMax.numel().ToScalar()
```

--------------------------------

### Train Neural Network (With Batches)

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/synthetic_data.ipynb

Trains the neural network using mini-batches of data. The training loop iterates through epochs and then through each batch, performing forward and backward passes, and updating weights. The scheduler is stepped once per epoch.

```F#
printf " initial loss = %s\n" (loss.forward(model.forward(X_train), y_train).item<float32>().ToString())

for epoch = 1 to 1000 do

    for j = 0 to X_batch.Length-1 do

        let output = loss.forward(model.forward(X_batch[j]), y_batch[j])
        
        // Clear the gradients before doing the back-propagation
        model.zero_grad()

        // Do back-propagation, which computes all the gradients.
        output.backward()

        optimizer.step() |> ignore

    scheduler.step()

printf " final loss   = %s\n" (loss.forward(model.forward(X_train), y_train).item<float32>().ToString())
```

--------------------------------

### Model Training Loop with Adam Optimizer in F#

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial6.ipynb

Presents a typical training loop iterating over multiple epochs using the Adam optimizer. This structure is common for training deep learning models.

```fsharp
let learning_rate = 0.01
let model = Trivial()
let optimizer = torch.optim.Adam(model.parameters())

for epoch = 1 to 500 do
    // Compute the loss
    let pred = model.forward(dataBatch)
    let output = loss pred resultBatch

    // Clear the gradients before doing the back-propagation
    model.zero_grad()

    // Do back-propagation, which computes all the gradients.
    output.backward()

    optimizer.step() |> ignore

(loss pred resultBatch).item<single>()
```

--------------------------------

### Save Model Weights (Python)

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial6.ipynb

Use this method to save the trained weights of your TorchSharp model to a binary file.

```python
model.save("tutorial6.model.bin")
```

--------------------------------

### Save Model Weights in TorchSharp

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Use the `save` method to store the model's weights on disk. This is essential for using a trained model in a separate process.

```python
model.save("tutorial6.model.bin");
```

--------------------------------

### Configure Tensor Formatting (Legacy)

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial1.ipynb

Before version 0.100.3, this code was necessary to set preferred MIME types and register a custom formatter for torch.Tensor objects in .NET Interactive. It ensures tensors are displayed in a readable format.

```csharp
using Microsoft.DotNet.Interactive.Formatting;

Formatter.SetPreferredMimeTypesFor(typeof(torch.Tensor), "text/plain");
Formatter.Register<torch.Tensor>((torch.Tensor x) => x.ToString(TorchSharp.TensorStringStyle.Default, fltFormat:"G7"));
```

--------------------------------

### Load Synthetic Data

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/synthetic_data.ipynb

Loads training and testing datasets from specified files. Ensure the 'train.dat' and 'test.dat' files exist in the correct directory.

```F#
let X_train,y_train = load_data_file("train.dat")
let X_test, y_test =  load_data_file("test.dat")
```

--------------------------------

### Load Training and Test Data

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/synthetic_data.ipynb

Loads training and testing data from specified files. Ensure the 'LoadDataFile' function is accessible in your environment.

```C#
var (X_train, y_train) = LoadDataFile("train.dat");
var (X_test, y_test) = LoadDataFile("test.dat");
```

--------------------------------

### Print Multiple Tensors Using Console.Write

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/FSharp/tutorial2.ipynb

Shows how to print multiple tensors to the console using the ToString(true) method. This is useful when more than one value needs to be displayed in a cell.

```F#
Console.Write(torch.zeros(4,4).ToString(true));
Console.Write(torch.ones(4,4).ToString(true));
```

--------------------------------

### ResNet Constructor in C#

Source: https://github.com/dotnet/torchsharpexamples/blob/main/tutorials/CSharp/tutorial6.ipynb

Defines the ResNet constructor, utilizing Sequential to build the model from a list of named modules. This approach is useful for models with repeating blocks of layers.

```csharp
public ResNet(string name, Func<string, int,int,int,Module> block, int expansion, IList<int> num_blocks, int numClasses, Device device = null) : base(name)
{
    if (planes.Length != strides.Length) throw new ArgumentException("'planes' and 'strides' must have the same length.");

    var modules = new List<(string, Module)>();

    modules.Add(("conv2d-first", Conv2d(3, 64, kernelSize: 3, stride: 1, padding: 1, bias: false)));
    modules.Add(("bnrm2d-first", BatchNorm2d(64)));
    modules.Add(("relu-first", ReLU(inPlace:true)));
    MakeLayer(modules, block, expansion, 64, num_blocks[0], 1);
    MakeLayer(modules, block, expansion, 128, num_blocks[1], 2);
    MakeLayer(modules, block, expansion, 256, num_blocks[2], 2);
    MakeLayer(modules, block, expansion, 512, num_blocks[3], 2);
    modules.Add(("avgpool", AvgPool2d(new long[] { 4, 4 })));
    modules.Add(("flatten", Flatten()));
    modules.Add(($"linear", Linear(512 * expansion, numClasses)));

    layers = Sequential(modules);

    RegisterComponents();
}
```