### Install Dependencies for TorchSharp on Ubuntu

Source: https://github.com/dotnet/torchsharp/blob/main/DEVGUIDE.md

Example commands to install the necessary dependencies for building TorchSharp on Ubuntu 16. This includes LLVM, git, and cmake.

```bash
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
sudo apt-add-repository "deb https://apt.llvm.org/xenial/ llvm-toolchain-xenial-6.0 main"
sudo apt-get -y update
sudo apt-get -y install clang-6.0 git cmake libunwind8 curl libomp-dev
```

--------------------------------

### Verify .NET SDK Installation

Source: https://github.com/dotnet/torchsharp/blob/main/test/notebooks/NativeCudaLoadLinux.ipynb

Confirms that the .NET SDK has been successfully installed by checking its version.

```bash
!dotnet --version
```

--------------------------------

### Install Library and Symbols in CMake

Source: https://github.com/dotnet/torchsharp/blob/main/src/Native/CMakeLists.txt

Defines a CMake function `install_library_and_symbols` to install the target library and its associated symbol files. It calls `strip_symbols` and handles platform-specific installation paths for PDB files on Windows.

```cmake
function(install_library_and_symbols targetName)
    strip_symbols(${targetName} strip_destination_file)

    install (TARGETS ${targetName} DESTINATION .)

    if(WIN32)
        install(FILES ${CMAKE_CURRENT_BINARY_DIR}/${CMAKE_BUILD_TYPE}/${targetName}.pdb DESTINATION .)
    else()
        install(FILES ${strip_destination_file} DESTINATION .)
    endif()
endfunction()
```

--------------------------------

### Loading Optimizer State in TorchSharp

Source: https://github.com/dotnet/torchsharp/wiki/Sharing-Model-Data-between-PyTorch-and-TorchSharp.org

Example showing how to load optimizer state in TorchSharp. Ensure the parameter groups and optimizer type exactly match the PyTorch export.

```csharp
var pgs = new Adam.ParamGroup[] {new (lin1.parameters()), new (lin2.parameters())};

var optimizer = torch.optim.Adam(pgs, 0.00004f);
optimizer.load_state_dict("adam_state.dat");
```

--------------------------------

### Install .NET SDK on Google Colab

Source: https://github.com/dotnet/torchsharp/blob/main/test/notebooks/NativeCudaLoadLinux.ipynb

Installs the .NET SDK version 5.0 on the Ubuntu 18.04 based Google Colab VM. This is a prerequisite for running F# code that utilizes TorchSharp.

```bash
# Install dotnet
!wget https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && sudo dpkg -i packages-microsoft-prod.deb && sudo apt-get update && sudo apt-get install -y apt-transport-https && sudo apt-get update && sudo apt-get install -y dotnet-sdk-5.0
```

--------------------------------

### Build a Neural Network with TorchSharp

Source: https://github.com/dotnet/torchsharp/blob/main/README.md

This example demonstrates building and training a simple sequential neural network using TorchSharp. It includes defining layers, a loss function, and an optimizer, followed by a training loop. Ensure necessary imports are included.

```csharp
using TorchSharp;
using static TorchSharp.torch.nn;

var lin1 = Linear(1000, 100);
var lin2 = Linear(100, 10);
var seq = Sequential(("lin1", lin1), ("relu1", ReLU()), ("drop1", Dropout(0.1)), ("lin2", lin2));

using var x = torch.randn(64, 1000);
using var y = torch.randn(64, 10);

var optimizer = torch.optim.Adam(seq.parameters());

for (int i = 0; i < 10; i++) {
    using var eval = seq.forward(x);
    using var output = functional.mse_loss(eval, y, Reduction.Sum);

    optimizer.zero_grad();

    output.backward();

    optimizer.step();
}
```

--------------------------------

### Start Visual Studio from Command Line

Source: https://github.com/dotnet/torchsharp/blob/main/DEVGUIDE.md

Launches Visual Studio with the correct environment variables and PATH set for developing native C++ code. This ensures that CMake-configured projects are found and loaded correctly.

```bash
devenv TorchSharp.sln
```

--------------------------------

### Exporting Adam Optimizer State in PyTorch

Source: https://github.com/dotnet/torchsharp/wiki/Sharing-Model-Data-between-PyTorch-and-TorchSharp.org

Example demonstrating how to create an Adam optimizer with multiple parameter groups in PyTorch and then export its state to a file.

```python
optim = torch.optim.Adam(lin1.parameters(), lr=0.001, betas=(0.8, 0.9))
optim.add_param_group({'params': lin2.parameters(), 'lr': 0.01, 'betas' : (0.7, 0.79), 'amsgrad': True})

...

f = open("adam_state.dat", "wb")
exportsd.save_adam(optim, f)
f.close()
```

--------------------------------

### Build TorchSharp on Mac

Source: https://github.com/dotnet/torchsharp/blob/main/DEVGUIDE.md

Standard build and test commands for TorchSharp on macOS, assuming the required Clang/LLVM, git, .NET SDK, and CMake versions are installed.

```bash
dotnet build
```

```bash
dotnet test
```

--------------------------------

### Build Native and Managed Code (No CUDA)

Source: https://github.com/dotnet/torchsharp/blob/main/DEVGUIDE.md

Builds the native and managed code for TorchSharp, skipping CUDA components. This is useful for initial setup or when only CPU support is needed.

```bash
dotnet build /p:SkipCuda=true
```

--------------------------------

### Load Native Library

Source: https://github.com/dotnet/torchsharp/blob/main/README.md

Shows how to explicitly load the LibTorch native library using `NativeLibrary.Load`. This is useful when PyTorch is installed via a Python package manager.

```csharp
using System.Runtime.InteropServices;
NativeLibrary.Load("/home/gunes/anaconda3/lib/python3.8/site-packages/torch/lib/libtorch.so")

```

--------------------------------

### Inspect libtorch-cpu Dependencies

Source: https://github.com/dotnet/torchsharp/blob/main/test/notebooks/NativeCudaLoadLinux.ipynb

Examines the installed libtorch-cpu package and its native dependencies, including the C library version and the contents of the TorchSharp and libtorch-cpu native directories. It also checks the LD_LIBRARY_PATH and uses ldd to inspect the libLibTorchSharp.so shared library.

```bash
# Look around packages and dependencies  (libtorch-cpu)
! /lib/x86_64-linux-gnu/libc.so.6 --version
!ls /root/.nuget/packages/libtorch-cpu/1.9.0.10/runtimes/linux-x64/native
!ls /root/.nuget/packages/torchsharp/0.92.52515/runtimes/linux-x64/native/
!echo LD_LIBRARY_PATH=$LD_LIBRARY_PATH
!ldd --version
!ldd  /root/.nuget/packages/torchsharp/0.92.52515/runtimes/linux-x64/native/libLibTorchSharp.so
```

--------------------------------

### Returning a Fresh Tensor Alias

Source: https://github.com/dotnet/torchsharp/blob/main/docfx/articles/memory.md

This C# example demonstrates how to return a fresh tensor alias. Use `alias()` to avoid cloning the tensor while still returning a new reference. This is crucial for proper memory management when the caller needs to know whether to dispose of the returned tensor.

```C#
Tensor flatten(Tensor input) {
    if (input.shape.Length == 1)
        return input.alias();
    else
        return input.reshape(input.numel());
}
```

--------------------------------

### Implement MobileNet with Sequential

Source: https://github.com/dotnet/torchsharp/blob/main/docfx/articles/modules.md

Demonstrates building a MobileNet model by composing layers using Sequential within a custom Module. Useful for creating complex, layered architectures.

```C#
    class MobileNet : Module
    {
        private readonly long[] planes = new long[] { 64, 128, 128, 256, 256, 512, 512, 512, 512, 512, 512, 1024, 1024 };
        private readonly long[] strides = new long[] { 1, 2, 1, 2, 1, 2, 1, 1, 1, 1, 1, 2, 1 };

        private readonly Module layers;

        public MobileNet(string name, int numClasses, Device device = null) : base(name)
        {
            var modules = new List<(string, Module)>();

            modules.Add(("conv2d-first", 
                Conv2d(3, 32, kernel_size: 3, stride: 1, padding: 1, bias: false)));
            modules.Add(("bnrm2d-first", 
                BatchNorm2d(32)));
            modules.Add(("relu-first",   
                ReLU()));
            MakeLayers(modules, 32);
            modules.Add(("avgpool",      
                AvgPool2d(new long[] { 2, 2 })));
            modules.Add(("flatten",      
                Flatten()));
            modules.Add(("linear",       
                Linear(planes[^1], numClasses)));

            layers = Sequential(modules);

            RegisterComponents();
        }

        private void MakeLayers(List<(string, Module)> modules, long in_planes)
        {

            for (var i = 0; i < strides.Length; i++) {
                var out_planes = planes[i];
                var stride = strides[i];

                modules.Add(($"conv2d-{i}a", 
                    Conv2d(in_planes, in_planes, kernel_size: 3, stride: stride, padding: 1, groups: in_planes, bias: false)));
                modules.Add(($"bnrm2d-{i}a", 
                    BatchNorm2d(in_planes)));
                modules.Add(($"relu-{i}a",   
                    ReLU()));
                modules.Add(($"conv2d-{i}b", 
                    Conv2d(in_planes, out_planes, kernel_size: 1L, stride: 1L, padding: 0L, bias: false)));
                modules.Add(($"bnrm2d-{i}b", 
                    BatchNorm2d(out_planes)));
                modules.Add(($"relu-{i}b",   
                    ReLU()));

                in_planes = out_planes;
            }
        }

        public override Tensor forward(Tensor input)
        {
            return layers.forward(input);
        }
    }
```

--------------------------------

### Inspect libtorch-cuda Dependencies

Source: https://github.com/dotnet/torchsharp/blob/main/test/notebooks/NativeCudaLoadLinux.ipynb

Inspect the installed libtorch-cuda packages and their native library dependencies on Linux. This helps verify the correct installation and identify potential linking issues.

```shell
! /lib/x86_64-linux-gnu/libc.so.6 --version
```

```shell
!ls /root/.nuget/packages/libtorch-cuda-11.7-linux-x64/1.9.0.10/
```

```shell
!echo LD_LIBRARY_PATH=$LD_LIBRARY_PATH
```

```shell
!ldd --version
```

```shell
!ls /root/.nuget/packages/torchsharp/0.92.52515/runtimes/linux-x64/native/
```

```shell
!ls /root/.nuget/packages/torchsharp/0.92.52515/lib/net6.0/cuda-11.7/
```

```shell
!ldd /root/.nuget/packages/torchsharp/0.92.52515/lib/net6.0/cuda-11.7/libLibTorchSharp.so
```

--------------------------------

### Build TorchSharp on Windows

Source: https://github.com/dotnet/torchsharp/blob/main/DEVGUIDE.md

Use these commands in a VS 'x64 Native Tools' command prompt to build TorchSharp on Windows. Set SkipNative=true to exclude native components.

```bash
dotnet build /p:SkipNative=true
```

```bash
dotnet build  # for cuda support on Windows and Linux
```

```bash
dotnet test
```

```bash
dotnet pack
```

--------------------------------

### Create a Sequential Module

Source: https://github.com/dotnet/torchsharp/blob/main/docfx/articles/modules.md

Use the `Sequential` class to chain together modules for basic network architectures. It takes a list of named modules.

```C#
var seq = Sequential(("lin1", Linear(100, 10)), ("lin2", Linear(10, 5)));
...
seq.forward(some_data);
```

--------------------------------

### Load Model State Dictionary in Python

Source: https://github.com/dotnet/torchsharp/blob/main/docfx/articles/saveload.md

Create a model instance and load the state dictionary from a file. Ensure the model structure matches the saved state.

```Python
model = [...]
model.load_state_dict(torch.load('model_weights.pth'))
```

--------------------------------

### Build and Pack TorchSharp Package

Source: https://github.com/dotnet/torchsharp/blob/main/DEVGUIDE.md

Commands to build the TorchSharp project and then pack it into a NuGet package. These commands are used both locally and as part of the CI process.

```bash
dotnet build
```

```bash
dotnet pack
```

--------------------------------

### Wrap Complex Expressions with TorchSharp Dispose Scope

Source: https://github.com/dotnet/torchsharp/blob/main/docfx/articles/memory.md

Use `WrappedTensorDisposeScope` to simplify memory management for complex expressions with multiple temporary tensors, avoiding explicit scope setup. This is especially useful in concise functions and properties.

```C#
public static Tensor WrappedTensorDisposeScope(Func<Tensor> expr)
{
    using var scope = torch.NewDisposeScope();
    var result = expr();
    return result.MoveToOuterDisposeScope();
}
```

```C#
public override Tensor entropy() => torch.WrappedTensorDisposeScope(() => ((scale / alpha).log() + (1 + alpha.reciprocal())));
```

--------------------------------

### Fragmenting a Binary File for NuGet Packages

Source: https://github.com/dotnet/torchsharp/blob/main/DEVGUIDE.md

This XML snippet demonstrates how to split a large binary file (torch_cuda_cu.dll) into multiple parts for NuGet packaging. Each part is assigned a unique PackageSuffix and FileUnstitch parameters define its role and size. The last fragment uses FileUnstitchSize='-1' to indicate it contains the remaining data.

```xml
<File Include= "libtorch\lib\torch_cuda_cu.dll"  PackageSuffix="part9-primary" FileUnstitchIndex="0" FileUnstitchStart="0" FileUnstitchSize="250000000" />
<File Include= "libtorch\lib\torch_cuda_cu.dll"  PackageSuffix="part9-fragment1" FileUnstitchIndex="1" FileUnstitchStart="250000000" FileUnstitchSize="250000000" />
<File Include= "libtorch\lib\torch_cuda_cu.dll"  PackageSuffix="part9-fragment2" FileUnstitchIndex="2" FileUnstitchStart="500000000" FileUnstitchSize="250000000" />
<File Include= "libtorch\lib\torch_cuda_cu.dll"  PackageSuffix="part9-fragment3" FileUnstitchIndex="3" FileUnstitchStart="750000000" FileUnstitchSize="-1" />
```

--------------------------------

### Update SHA Hashes for LibTorch Downloads (Linux/Mac)

Source: https://github.com/dotnet/torchsharp/blob/main/DEVGUIDE.md

Run these dotnet build commands on Linux or Mac to test downloads and update SHA hashes for various LibTorch configurations.

```bash
dotnet build src/Redist/libtorch-cpu/libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=linux /p:Configuration=Release /t:Build /p:IncludeLibTorchCpuPackages=true
dotnet build src/Redist/libtorch-cpu/libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=mac /p:TargetArchitecture=arm64 /p:Configuration=Release /t:Build /p:IncludeLibTorchCpuPackages=true
dotnet build src/Redist/libtorch-cpu/libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=windows /p:Configuration=Release /t:Build /p:IncludeLibTorchCpuPackages=true
dotnet build src/Redist/libtorch-cpu/libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=windows /p:Configuration=Debug /t:Build /p:IncludeLibTorchCpuPackages=true

dotnet build src/Redist/libtorch-cuda-12.8/libtorch-cuda-12.8.proj /p:UpdateSHA=true /p:TargetOS=linux /p:Configuration=Release /t:Build /p:IncludeLibTorchCudaPackages=true
dotnet build src/Redist/libtorch-cuda-12.8/libtorch-cuda-12.8.proj /p:UpdateSHA=true /p:TargetOS=windows /p:Configuration=Release /t:Build /p:IncludeLibTorchCudaPackages=true
dotnet build src/Redist/libtorch-cuda-12.8/libtorch-cuda-12.8.proj /p:UpdateSHA=true /p:TargetOS=windows /p:Configuration=Debug /t:Build /p:IncludeLibTorchCudaPackages=true
```

--------------------------------

### TorchSharp Dataset Tensor Disposal Issue Example

Source: https://github.com/dotnet/torchsharp/blob/main/docfx/articles/memory.md

Demonstrates a common issue where tensors created within a custom `Dataset`'s `GetTensor` method are automatically disposed by the `DataLoader` after collation, leading to invalid tensors in subsequent iterations. This occurs because the tensors are attached to the data loader's dispose scope.

```C#
using TorchSharp;

using var dataLoader = torch.utils.data.DataLoader(new MyDataset(), batchSize: 1);
foreach (var _ in dataLoader) ;
// System.InvalidOperationException:
// Tensor invalid -- empty handle.

class MyDataset : torch.utils.data.Dataset
{
    private torch.Tensor tensor = torch.zeros([]);
    public override Dictionary<string, torch.Tensor> GetTensor(long index)
    {
        tensor = tensor + 1;
        // The new tensor is attached to the dispose scope in the data loader,
        // and it will be disposed after collation,
        // so in the next iteration it becomes invalid.
        return new() { ["tensor"] = tensor };
    }

    public override long Count => 3;
}
```

--------------------------------

### F# In-place Operations with 'use'

Source: https://github.com/dotnet/torchsharp/wiki/Memory-Management

Shows how to manage temporaries created by in-place operations (`add_`) using the `use` keyword in F#. Even for in-place mutations, temporaries are created and should be managed.

```F#
let myTensorFunction5(go: bool, input: Tensor) =
    if go then 
        use tmp1 = input.add_(tensor(1))  // NOTE: even for in-place mutations
        use tmp2 = input.add_(tensor(1))  // NOTE: even for in-place mutations
        tmp2.add(tensor(1))
    else
        input.alias()
```

--------------------------------

### Update SHA Hashes for LibTorch Downloads (Windows)

Source: https://github.com/dotnet/torchsharp/blob/main/DEVGUIDE.md

Run these dotnet build commands on Windows to test downloads and update SHA hashes for various LibTorch configurations.

```bash
dotnet build src\Redist\libtorch-cpu\libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=linux /p:Configuration=Release /t:Build /p:IncludeLibTorchCpuPackages=true
dotnet build src\Redist\libtorch-cpu\libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=mac /p:TargetArchitecture=arm64 /p:Configuration=Release /t:Build /p:IncludeLibTorchCpuPackages=true
dotnet build src\Redist\libtorch-cpu\libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=windows /p:Configuration=Release /t:Build /p:IncludeLibTorchCpuPackages=true
dotnet build src\Redist\libtorch-cpu\libtorch-cpu.proj /p:UpdateSHA=true /p:TargetOS=windows /p:Configuration=Debug /t:Build /p:IncludeLibTorchCpuPackages=true

dotnet build src\Redist\libtorch-cuda-12.8\libtorch-cuda-12.8.proj /p:UpdateSHA=true /p:TargetOS=linux /p:Configuration=Release /t:Build /p:IncludeLibTorchCudaPackages=true
dotnet build src\Redist\libtorch-cuda-12.8\libtorch-cuda-12.8.proj /p:UpdateSHA=true /p:TargetOS=windows /p:Configuration=Release /t:Build /p:IncludeLibTorchCudaPackages=true
dotnet build src\Redist\libtorch-cuda-12.8\libtorch-cuda-12.8.proj /p:UpdateSHA=true /p:TargetOS=windows /p:Configuration=Debug /t:Build /p:IncludeLibTorchCudaPackages=true
```

--------------------------------

### Instantiate Conv1d Module

Source: https://github.com/dotnet/torchsharp/blob/main/README.md

Demonstrates how to instantiate a Conv1d module using the factory method. This creates an instance of Modules.Conv1d, which inherits from torch.Module.

```csharp
Module conv1 = Conv1d(...);

```

--------------------------------

### Clean and Reset Project

Source: https://github.com/dotnet/torchsharp/blob/main/DEVGUIDE.md

Before updating, ensure your project is clean and reset to the main branch.

```bash
git checkout main
git clean -xfd .
```

--------------------------------

### Load Model State Dictionary from TorchSharp Format (Python)

Source: https://github.com/dotnet/torchsharp/blob/main/docfx/articles/saveload.md

Load a model's state dictionary from a file saved in the TorchSharp compatible format using a helper script. The file is opened in binary read mode.

```Python
f = open("model_weights.dat", "rb")
model.load_state_dict(importsd.load_state_dict(f))
f.close()
```

--------------------------------

### Create F# Script for TorchSharp CUDA Check

Source: https://github.com/dotnet/torchsharp/blob/main/test/notebooks/NativeCudaLoadLinux.ipynb

This script sets up the necessary NuGet packages and checks for CUDA availability using TorchSharp. It's designed to run on Linux.

```fsharp
printfn "phase0"
```

```fsharp
#r "nuget: TorchSharp, 0.92.52515"
```

```fsharp
#r "nuget: libtorch-cuda-11.7-linux-x64, 1.9.0.10"
```

```fsharp
printfn "phase2"
```

```fsharp
TorchSharp.Torch.IsCudaAvailable() |> printfn "%A"
```

--------------------------------

### Load Model in TorchSharp Format (C#)

Source: https://github.com/dotnet/torchsharp/blob/main/docfx/articles/saveload.md

Load the model's state from a file saved in the TorchSharp format. It is critical that all submodules and buffers have the same names as in the original model.

```C#
model = [...];
model.load("model_weights.dat");
```

--------------------------------

### Investigate GLIB and GLIBCXX Dependencies

Source: https://github.com/dotnet/torchsharp/blob/main/test/notebooks/NativeCudaLoadLinux.ipynb

Use these commands to check the GLIB and GLIBCXX symbol versions available on the system. This helps diagnose compatibility issues when native binaries are built on a newer Linux version than the target system.

```bash
# Investigate GLIB and GLIBCXX dependencies available on this system
!ldd --version
!/sbin/ldconfig -p | grep stdc++
!strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep LIBCXX
```

--------------------------------

### Run TorchSharp with CPU Binaries

Source: https://github.com/dotnet/torchsharp/blob/main/test/notebooks/NativeCudaLoadLinux.ipynb

Executes F# code that loads the TorchSharp and libtorch-cpu packages and checks if CUDA is available. This is expected to return 'false' as only CPU binaries are loaded.

```fsharp
# Run some code with workaround  (libtorch-cpu)
!echo "printfn \"phase0\"" > foo.fsx
!echo "#r \"nuget: TorchSharp, 0.92.52515\"" >> foo.fsx
!echo "#r \"nuget: libtorch-cpu, 1.9.0.10\"" >> foo.fsx
#!echo "printfn \"phase1\"" >> foo.fsx
#!echo "open System.Runtime.InteropServices" >> foo.fsx
#!echo "NativeLibrary.Load(\"/root/.nuget/packages/libtorch-cpu/1.9.0.10/runtimes/linux-x64/native/libtorch.so\") |> printfn \"%A\"" >> foo.fsx
!echo "printfn \"phase2\"" >> foo.fsx
!echo "TorchSharp.Torch.IsCudaAvailable() |> printfn \"%A\"" >> foo.fsx
!cat foo.fsx
!dotnet fsi foo.fsx
```

--------------------------------

### Build Native Code with CUDA

Source: https://github.com/dotnet/torchsharp/blob/main/DEVGUIDE.md

Builds the native and managed code for TorchSharp, including CUDA support. This command should be run after ensuring all prerequisites and configurations are in place.

```bash
dotnet build
```

--------------------------------

### Implement Custom Linear Layer with Parameters in TorchSharp

Source: https://github.com/dotnet/torchsharp/wiki/Creating-Your-Own-TorchSharp-Modules

Create a custom 'Linear' layer by defining trainable 'weights' and 'bias' as Parameters. The forward pass uses torch operators for matrix multiplication and addition.

```C#
        private class MyLinear : Module<Tensor,Tensor>
        {
            public MyLinear(long input_size, long output_size)
                : base("MyLinear")
            {
                weights = Parameter(torch.randn(input_size, output_size));
                bias = Parameter(torch.zeros(output_size));
                RegisterComponents();
            }

            public override Tensor forward(Tensor input)
            {
                var mm = torch.matmul(input,weights);
                return mm.add_(bias);
            }

            private Parameter weights;
            private Parameter bias;
        }
```

--------------------------------

### Implement Custom Module with ModuleList

Source: https://github.com/dotnet/torchsharp/blob/main/docfx/articles/modules.md

Shows how to use ModuleList to dynamically manage a collection of submodules within a custom module. The forward pass must explicitly iterate through the ModuleList.

```C#
        private class TestModule1 : Module<Tensor,Tensor>
        {
            public TestModule1()
                : base("TestModule1")
            {
                RegisterComponents();
            }

            public override Tensor forward(Tensor input)
            {
                for (int i = 0; i < submodules.Count; i++) {      // Using 'for' instead of 'foreach' can be useful for debugging.
                    input = submodules[i].forward(input); 
                }
            }

            private ModuleList submodules = new ModuleList(Linear(100, 10), Linear(10, 5));
        }
```

--------------------------------

### Save Model State Dictionary to TorchSharp Format (Python)

Source: https://github.com/dotnet/torchsharp/blob/main/docfx/articles/saveload.md

Save a model's state dictionary to a file in the TorchSharp compatible format using a helper script. The model is moved to CPU before saving.

```Python
f = open("model_weights.dat", "wb")
exportsd.save_state_dict(model.to("cpu").state_dict(), f)
f.close()
```

--------------------------------

### Create TorchScript Module in Python

Source: https://github.com/dotnet/torchsharp/wiki/TorchScript

Defines a PyTorch nn.Module and converts it to a TorchScript module using `torch.jit.script`. The module is then saved to a file. Note the use of type annotations and `@torch.jit.export` for callable methods.

```python
from typing import Tuple
import torch
from torch import nn
from torch import Tensor

class MyModule(nn.Module):
  def __init__(self):
    super().__init__()
    self.p = nn.Parameter(torch.rand(10))

  def forward(self, x: Tensor, y: Tensor) -> Tuple[Tensor, Tensor]:
    return x + y, x - y
  
  @torch.jit.export
  def predict(self, x: Tensor) -> Tensor:
    return x + self.p

  @torch.jit.export
  def add_scalar(self, x: Tensor, i: int) -> Tensor:
    return x + i

m = MyModule()

m = torch.jit.script(m)
m.save("exported.method.dat")
```

--------------------------------

### Load TorchScript Module in C#

Source: https://github.com/dotnet/torchsharp/wiki/TorchScript

Loads a TorchScript module from a file using `torch.jit.load`. The loaded module can be used for training or inference and is initially placed on the CPU.

```csharp
var m = torch.jit.load("file-name");
```

--------------------------------

### F# Multiple Temporaries with 'use'

Source: https://github.com/dotnet/torchsharp/wiki/Memory-Management

Demonstrates managing multiple temporary tensors using the `use` keyword in F# for explicit memory management.

```F#
let myTensorFunction4(input: Tensor) =
    use tmp1 = input.add(tensor(1))
    use tmp2 = input.add(tensor(1))
    tmp2.add(tensor(1))
```

--------------------------------

### Save Model in TorchSharp Format (C#)

Source: https://github.com/dotnet/torchsharp/blob/main/docfx/articles/saveload.md

Save the model's state using the native TorchSharp serialization format. Ensure the model is created on the CPU before loading weights for efficient memory management.

```C#
model.save("model_weights.dat");
```

--------------------------------

### Build TorchSharp on Linux

Source: https://github.com/dotnet/torchsharp/blob/main/DEVGUIDE.md

Commands to build TorchSharp on Linux after fulfilling the requirements. The 'dotnet build' command without '/p:SkipNative=true' is used for CUDA support.

```bash
dotnet build /p:SkipNative=true
```

```bash
dotnet build  # for cuda support on Windows and Linux
```

```bash
dotnet test
```

--------------------------------

### Tensor Creation from .NET Arrays in TorchSharp

Source: https://github.com/dotnet/torchsharp/blob/main/RELEASENOTES.md

Understand the different methods for creating tensors from .NET arrays, noting whether they copy data or share storage. This is crucial for performance as copying can be significantly slower.

```C#
// Never copy:
public static Tensor from_array(Array input)

// Copy only if dtype or device arguments require it:
public static Tensor frombuffer(Array input, ScalarType dtype, long count = -1, long offset = 0, bool requiresGrad = false, Device? device = null)
public static Tensor as_tensor(Array input,  ScalarType? dtype = null, Device? device = null)
public static Tensor as_tensor(Tensor input, ScalarType? dtype = null, Device? device = null)

// Always copy:
public static Tensor as_tensor(IList<<VARIOUS TYPES>> input,  ScalarType? dtype = null, Device? device = null)
public static Tensor tensor(<<VARIOUS TYPES>> input, torch.Device? device = null, bool requiresGrad = false)
```

--------------------------------

### Using Sequential with Named Children

Source: https://github.com/dotnet/torchsharp/wiki/Creating-Your-Own-TorchSharp-Modules

Construct a Sequential module from the named children of an existing module. The order of children matters for correct execution.

```C#
TestModule1 mod1 = ...
TestModule2 mod2 = ...
var seq1 = nn.Sequential(mod1.named_children());
seq1.forward(t);                 // Does the same as mod1.forward(t)
var seq2 = nn.Sequential(mod2.named_children());
seq2.forward(t);                 // This probably blows up.
```

--------------------------------

### Implement Custom Linear Layer with Parameters

Source: https://github.com/dotnet/torchsharp/blob/main/docfx/articles/modules.md

Define custom trainable parameters like weights and bias directly within a module. Use the Parameter class to ensure they are registered with the TorchSharp runtime.

```C#
private class MyLinear : Module<Tensor,Tensor>
{
    public MyLinear(long input_size, long output_size)
        : base("MyLinear")
    {
        weights = Parameter(torch.randn(input_size, output_size));
        bias = Parameter(torch.zeros(output_size));
        RegisterComponents();
    }

    public override Tensor forward(Tensor input)
    {
        var mm = torch.matmul(input,weights);
        return mm.add_(bias);
    }

    private Parameter weights;
    private Parameter bias;
}
```

--------------------------------

### Pack TorchSharp Packages (Local Build)

Source: https://github.com/dotnet/torchsharp/blob/main/DEVGUIDE.md

Packs TorchSharp packages locally, including CPU and CUDA variants. This command is used to generate distributable packages for local testing or deployment. Ensure `BuildLibTorchPackages` is set in `azure-pipelines.yml` if building new libtorch packages.

```bash
<LibTorchPackageVersion>2.7.1.0</LibTorchPackageVersion>

dotnet pack -c Release -v:n /p:SkipNative=true /p:SkipTests=true /p:IncludeTorchSharpPackage=true /p:IncludeLibTorchCpuPackages=true /p:IncludeLibTorchCudaPackages=true
dotnet pack -c Release -v:n /p:SkipNative=true /p:SkipTests=true /p:TargetOS=linux /p:IncludeTorchSharpPackage=true /p:IncludeLibTorchCpuPackages=true /p:IncludeLibTorchCudaPackages=true
```

--------------------------------

### Verify LibTorch Files

Source: https://github.com/dotnet/torchsharp/blob/main/DEVGUIDE.md

After downloading, check the contents of the libtorch archive to ensure files are present and correctly named. This is crucial for the next step of updating file includes.

```bash
dir bin\obj\x64.Release\libtorch-cpu\libtorch-cxx11-abi-shared-with-deps-2.7.1cpu\libtorch\lib\*.so*
```

--------------------------------

### Restore Packages (libtorch-cuda)

Source: https://github.com/dotnet/torchsharp/blob/main/test/notebooks/NativeCudaLoadLinux.ipynb

Use this F# script to restore NuGet packages, including TorchSharp and the specific libtorch-cuda version for Linux. Ensure the NuGet feed is correctly configured.

```fsharp
printfn "phase0"
#i "nuget: https://donsyme.pkgs.visualstudio.com/TorchSharp/_packaging/packages2%40Local/nuget/v3/index.json"
#r "nuget: TorchSharp, 0.92.52515";;
#r "nuget: libtorch-cuda-11.7-linux-x64, 1.9.0.10";;
printfn "done"
```

```shell
!cat foo.fsx
```

```shell
!dotnet fsi foo.fsx
```

--------------------------------

### Invoke TorchScript Module Methods in C#

Source: https://github.com/dotnet/torchsharp/wiki/TorchScript

Demonstrates calling the `forward` method and other exported methods (like `predict` and `add_scalar`) on a loaded TorchScript module. Non-`forward` methods require their names to be passed as strings.

```csharp
var x = torch.rand(10);
var y = torch.rand(10);
var t0 = m.forward(x,y);
var t1 = m.invoke("predict", x);
var t2 = m.invoke("add_scalar", x, 3.14);
```

--------------------------------

### Define a Custom TorchSharp Module

Source: https://github.com/dotnet/torchsharp/wiki/Creating-Your-Own-TorchSharp-Modules

Derive a class from Module<T..., TResult> to create a custom module. Ensure submodules are disposed and components are registered.

```C#
        private class TestModule1 : Module<Tensor,Tensor>
        {
            public TestModule1()
                : base("TestModule1")
            {
                lin1 = Linear(100, 10);
                lin2 = Linear(10, 5);
                RegisterComponents();
            }

            public override Tensor forward(Tensor input)
            {
                using (var x = lin1.forward(input))
                return lin2.forward(x);
            }

            protected override void Dispose(bool disposing)
            {
                if (disposing) {
                    lin1.Dispose();
                    lin2.Dispose();
                }
                base.Dispose(disposing);
            }

            private Module lin1;
            private Module lin2;
        }
```

--------------------------------

### Add SHA Files for LibTorch

Source: https://github.com/dotnet/torchsharp/blob/main/DEVGUIDE.md

Adds the SHA files for libtorch packages to the Git repository. Ensure these files are present before proceeding with builds.

```bash
git add src\Redist\libtorch-cpu\*.sha
git add src\Redist\libtorch-cuda-12.8\*.sha
```

--------------------------------

### Restore TorchSharp and libtorch-cpu Packages

Source: https://github.com/dotnet/torchsharp/blob/main/test/notebooks/NativeCudaLoadLinux.ipynb

Restores the TorchSharp and libtorch-cpu NuGet packages using a simple F# script. This step is necessary to acquire the libraries needed for CPU-based TorchSharp operations.

```fsharp
# Restore packages (libtorch-cpu)
!echo "printfn \"phase0\"" > foo.fsx
!echo "#r \"nuget: TorchSharp, 0.92.52515\"" >> foo.fsx
!echo "#r \"nuget: libtorch-cpu, 1.9.0.10\"" >> foo.fsx
!echo "printfn \"done\"" >> foo.fsx
!cat foo.fsx
!echo "-------"
!dotnet fsi foo.fsx
```

--------------------------------

### Define a Custom Module

Source: https://github.com/dotnet/torchsharp/blob/main/docfx/articles/modules.md

Create a custom module by inheriting from `Module<Tensor, Tensor>`. Ensure submodules are private fields and call `RegisterComponents()`.

```C#
private class TestModule1 : Module<Tensor,Tensor>
{
    public TestModule1()
        : base("TestModule1")
    {
        lin1 = Linear(100, 10);
        lin2 = Linear(10, 5);
        RegisterComponents();
    }

    public override Tensor forward(Tensor input)
    {
        using (var x = lin1.forward(input))
        return lin2.forward(x);
    }

    private Module lin1;
    private Module lin2;
}
```

--------------------------------

### TorchSharp DataLoader with `disposeBatch: false`

Source: https://github.com/dotnet/torchsharp/blob/main/docfx/articles/memory.md

Illustrates how to configure a `DataLoader` with `disposeBatch: false` to prevent automatic disposal of batches. This is useful when tensors need to persist across iterations, but requires manual memory management to avoid leaks.

```C#
using TorchSharp;

using var dataset = torch.utils.data.TensorDataset(torch.zeros([3]));

using var dataLoader1 = torch.utils.data.DataLoader(dataset, batchSize: 1);
using var dataLoader2 = torch.utils.data.DataLoader(dataset, batchSize: 1, disposeBatch: false);

Console.WriteLine(dataLoader1.First()[0].IsInvalid); // True
Console.WriteLine(dataLoader2.First()[0].IsInvalid); // False
```

--------------------------------

### Module Declaration Order

Source: https://github.com/dotnet/torchsharp/blob/main/docfx/articles/modules.md

Demonstrates correct and incorrect declaration orders for submodules within a custom module. Incorrect ordering can lead to runtime errors when constructing sequential models from named children.

```C#
private class TestModule1 : Module<Tensor,Tensor>
{
    public TestModule1()
        : base("TestModule1")
    {
        lin1 = Linear(100, 10);
        lin2 = Linear(10, 5);
        RegisterComponents();
    }

    public override Tensor forward(Tensor input)
    {
        using (var x = lin1.forward(input))
        return lin2.forward(x);
    }

    // Correct -- the layers are declared in the same order they are invoked.
    private Module lin1;
    private Module lin2;
}

private class TestModule2 : Module<Tensor,Tensor>
{
    public TestModule2()
        : base("TestModule2")
    {
        lin1 = Linear(100, 10);
        lin2 = Linear(10, 5);
        RegisterComponents();
    }

    public override Tensor forward(Tensor input)
    {
        using (var x = lin1.forward(input))
        return lin2.forward(x);
    }

    // Incorrect -- the layers are not declared in the same order they are invoked.
    private Module lin2;
    private Module lin1;
}

... 
TestModule1 mod1 = ...
TestModule2 mod2 = ...
var seq1 = nn.Sequential(mod1.named_children());
seq1.forward(t);                 // Does the same as mod1.forward(t)
var seq2 = nn.Sequential(mod2.named_children());
seq2.forward(t);                 // This probably blows up.
```

--------------------------------

### Run TorchSharp Tests

Source: https://github.com/dotnet/torchsharp/blob/main/DEVGUIDE.md

Executes the test suite for TorchSharp in both Debug and Release configurations. Note that CUDA tests may need to be run manually on Linux or Windows.

```bash
dotnet test -c Debug
dotnet test -c Release
```