### Install Js-PyTorch using npm

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/index.html

Installs the Js-PyTorch package locally using npm. It also shows how to install a specific older version of the package.

```bash
npm install js-pytorch
nvm install js-pytorch@0.1.0
```

--------------------------------

### Initial Setup and Interval Update in JavaScript

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/assets/demo/demo.html

This JavaScript code performs initial setup by calling an 'addBox' function and then sets up an interval to repeatedly call the 'changeMNIST' function every 1200 milliseconds.

```javascript
addBox();
setInterval(changeMNIST, 1200);
```

--------------------------------

### Install Node.js and NVM on MacOS/Linux

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/index.html

Installs Node Version Manager (nvm) and a specific Node.js version (v20) using bash commands. It also verifies the installed versions of Node.js and npm.

```bash
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
nvm install 20
node -v
npm -v
```

--------------------------------

### Install JS-PyTorch with npm

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/README.md

This command installs the JS-PyTorch library locally in your project using npm. Ensure you have Node.js and npm installed.

```bash
npm install js-pytorch
```

--------------------------------

### Install js-pytorch via npm

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/README.md

Installs the js-pytorch library using npm, suitable for MacOS, Windows, and Ubuntu environments. Ensure Node.js is installed.

```bash
npm install js-pytorch
```

--------------------------------

### Initialize JS-PyTorch Scope and Utilities

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/404.html

Sets up the root scope, a hash function for strings, and utility functions for getting and setting data in local storage, scoped to the current path. This is foundational for managing application state and preferences.

```javascript
var media,input,key,value,palette=__md_get("__palette");if(palette&&palette.color){"(prefers-color-scheme)"===palette.color.media&&(media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']"),palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent"));for([key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}
```

--------------------------------

### Define Loss Function and Optimizer in JavaScript

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/tutorials/index.html

Demonstrates the setup of a CrossEntropyLoss function and an Adam optimizer for the neural network, passing model parameters and a learning rate.

```javascript
// Define loss function and optimizer:
let loss_func = new nn.CrossEntropyLoss();
let optimizer = new optim.Adam(model.parameters(), 3e-3);
```

--------------------------------

### JS-PyTorch Browser Installation Script

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/index.html

This HTML snippet shows how to include the JS-PyTorch library in a web page using a CDN link. It ensures the library is available for use in the browser's JavaScript environment.

```HTML
<script src="https://cdnjs.cloudflare.com/ajax/libs/js-pytorch/0.7.2/js-pytorch-browser.js"
        integrity="sha512-l22t7GnqXvHBMCBvPUBdFO2TEYxnb1ziCGcDQcpTB2un16IPA4FE5SIZ8bUR+RwoDZGikQkWisO+fhnakXt9rg=="
        crossorigin="anonymous"
        referrerpolicy="no-referrer"></script>
```

--------------------------------

### js-pytorch Simple Autograd Example

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/README.md

A simple example demonstrating automatic differentiation (autograd) in js-pytorch. It covers tensor creation, basic arithmetic operations, and gradient computation.

```typescript
// Require the Library if running in node (not necessary in the browser):
const { torch } = require("js-pytorch");

// Pass device as an argument to a Tensor or nn.Module (same as PyTorch):
const device = 'gpu';

// Instantiate Tensors:
let x = torch.randn([8, 4, 5]);
let w = torch.randn([8, 5, 4], true, device);
let b = torch.tensor([0.2, 0.5, 0.1, 0.0], true);

// Make calculations:
let out = torch.matmul(x, w);
out = torch.add(out, b);

// Compute gradients on whole graph:
out.backward();

// Get gradients from specific Tensors:
console.log(w.grad);
console.log(b.grad);
```

--------------------------------

### JS-PyTorch Basic Neural Network Example

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/index.html

This JavaScript code demonstrates a basic usage of JS-PyTorch in a browser environment. It creates a random tensor, initializes a linear layer, and performs a forward pass.

```JavaScript
let x = torch.randn([10,5]);
let linear = new torch.nn.Linear(5,1,'gpu',true);
let z = linear.forward(x);
console.log(z.data);
```

--------------------------------

### Local Storage Utility Functions

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/demo/index.html

Provides utility functions for getting and setting data in local storage, scoped to the current path.

```javascript
var _scope = new URL("..", location), _hash = e => [...e].reduce((e, _) => (e << 5) - e + _.charCodeAt(0), 0), _get = (e, _ = localStorage, t = _scope) => JSON.parse(_.getItem(t.pathname + "." + e)), _set = (e, _, t = localStorage, a = _scope) => {
  try {
    t.setItem(a.pathname + "." + e, JSON.stringify(_));
  } catch (e) {}
};
```

--------------------------------

### Initialize and Control Training Loop

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/assets/demo/demo.html

Sets up a flag to start the training loop and initiates the `trainLoop` function. The `trainLoop` recursively calls `trainStep` with a small delay.

```JavaScript
let in_loop = true;

function trainLoopInitializer() {
  in_loop = true;
  trainLoop();
};

function trainLoop() {
  if (!in_loop) return;
  trainStep();
  setTimeout(trainLoop, 0.01);
};
```

--------------------------------

### js-pytorch Complex Autograd Transformer Example

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/README.md

An advanced example showcasing the implementation of a Transformer model using js-pytorch. It includes defining layers, a custom `nn.Module`, loss function, optimizer, and a training loop.

```typescript
// Require the Library if running in node (not necessary in the browser):
const { torch } = require("js-pytorch");
const nn = torch.nn;
const optim = torch.optim;

const device = 'gpu';

// Define training hyperparameters:
const vocab_size = 52;
const hidden_size = 32;
const n_timesteps = 16;
const n_heads = 4;
const dropout_p = 0;
const batch_size = 8;

// Create Transformer decoder Module:
class Transformer extends nn.Module {
  constructor(vocab_size, hidden_size, n_timesteps, n_heads, dropout_p, device) {
    super();
    // Instantiate Transformer's Layers:
    this.embed = new nn.Embedding(vocab_size, hidden_size);
    this.pos_embed = new nn.PositionalEmbedding(n_timesteps, hidden_size);
    this.b1 = new nn.Block(hidden_size, hidden_size, n_heads, n_timesteps, dropout_p, device);
    this.b2 = new nn.Block(hidden_size, hidden_size, n_heads, n_timesteps, dropout_p, device);
    this.ln = new nn.LayerNorm(hidden_size);
    this.linear = new nn.Linear(hidden_size, vocab_size, device);
  }

  forward(x) {
    let z;
    z = torch.add(this.embed.forward(x), this.pos_embed.forward(x));
    z = this.b1.forward(z);
    z = this.b2.forward(z);
    z = this.ln.forward(z);
    z = this.linear.forward(z);
    return z;
  }
}

// Instantiate your custom nn.Module:
const model = new Transformer(vocab_size, hidden_size, n_timesteps, n_heads, dropout_p, device);

// Define loss function and optimizer:
const loss_func = new nn.CrossEntropyLoss();
const optimizer = new optim.Adam(model.parameters(), (lr = 5e-3), (reg = 0));

// Instantiate sample input and output:
let x = torch.randint(0, vocab_size, [batch_size, n_timesteps, 1]);
let y = torch.randint(0, vocab_size, [batch_size, n_timesteps]);
let loss;

// Training Loop:
for (let i = 0; i < 40; i++) {
  // Forward pass through the Transformer:
  let z = model.forward(x);

  // Get loss:
  loss = loss_func.forward(z, y);

  // Backpropagate the loss using torch.tensor's backward() method:
  loss.backward();

  // Update the weights:
  optimizer.step();

  // Reset the gradients to zero after each training step:
  optimizer.zero_grad();

  // Print loss at every iteration:
  console.log(`Iter ${i} - Loss ${loss.data[0].toFixed(4)}`)
}
```

--------------------------------

### Initialize and Run Training Loop in JavaScript

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/demo/index.html

Functions to initialize and manage the training loop for a PyTorch model. `trainLoopInitializer` sets a flag to start the loop, and `trainLoop` recursively calls `trainStep` using `setTimeout` for asynchronous execution.

```javascript
function trainLoopInitializer() {
  in_loop = true;
  trainLoop();
}

function trainLoop() {
  if (!in_loop) return;
  trainStep();
  setTimeout(trainLoop, 0.01);
}
```

--------------------------------

### Clear Training Data and Reset

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/demo/index.html

Clears all training data and resets the training state. It initializes the graph with a starting value and redraws it.

```javascript
function clearTraining() {
  resetTraining();
  data = [[-Math.log(0.1)]];
  plotGraph()
};
```

--------------------------------

### JavaScript: Select and Check HTML Element by Hash

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/operations/index.html

This snippet selects an HTML element based on the URL's hash, checks if it has a 'name' attribute, and sets its 'checked' property if the name starts with '__tabbed_'. It's a common pattern for interactive web pages.

```JavaScript
var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))
```

--------------------------------

### Handle Tabbed Content Check

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/404.html

This JavaScript code snippet targets elements based on the URL's hash. It specifically checks if an element's name starts with '__tabbed_' and, if so, sets its 'checked' property accordingly. This is likely used for managing tabbed interfaces within the documentation.

```javascript
var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))
```

--------------------------------

### Theme Initialization from Local Storage

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/demo/index.html

Initializes the website's theme (color scheme, primary and accent colors) based on local storage settings, with support for dark/light mode preference.

```javascript
var media, input, key, value, palette = _get("__palette");
if (palette && palette.color) {
  "(prefers-color-scheme)" === palette.color.media && (media = matchMedia("(prefers-color-scheme: light)"), input = document.querySelector(media.matches ? "[data-md-color-media='(prefers-color-scheme: light)']" : "[data-md-color-media='(prefers-color-scheme: dark)']"), palette.color.media = input.getAttribute("data-md-color-media"), palette.color.scheme = input.getAttribute("data-md-color-scheme"), palette.color.primary = input.getAttribute("data-md-color-primary"), palette.color.accent = input.getAttribute("data-md-color-accent"));
  for ([key, value] of Object.entries(palette.color)) {
    document.body.setAttribute("data-md-color-" + key, value);
  }
}
```

--------------------------------

### Instantiate Neural Network Model in JavaScript

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/tutorials/index.html

Shows how to instantiate the NeuralNet class with specified input, hidden, and output sizes, along with batch size.

```javascript
// Instantiate Model:
let in_size = 16;
let hidden_size = 32;
let out_size = 10;
let batch_size = 16;

let model = new NeuralNet(in_size,hidden_size,out_size);
```

--------------------------------

### Generate Dummy Data for Training in JavaScript

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/tutorials/index.html

Illustrates the creation of random input (x) and target (y) tensors for training the neural network, using specified batch size and dimensions.

```javascript
// Instantiate input and output:
let x = torch.randn([batch_size, in_size]);
let y = torch.randint(0, out_size, [batch_size]);
let loss;
```

--------------------------------

### Get Number of Tensor Dimensions

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/tensor/index.html

Returns the number of dimensions (or rank) of a tensor. This helps in understanding the tensor's dimensionality.

```javascript
let a = torch.randn([3,2,1,4], true, 'gpu');
console.log(a.ndims);
```

--------------------------------

### Execute a Single Training Step

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/assets/demo/demo.html

Manages the training process, including setting the training flag, updating button styles, and initializing the neural network model with specified layers and device.

```JavaScript
let training = false;

function trainStep() {
  if (!training) {
    training = true;
    buttons = document.getElementsByClassName('layer-button');
    for (button of buttons) {
      button.style.backgroundColor = '#0056b3';
    };
    let _cpu = document.getElementById('cpu-trigger');
    let _gpu = document.getElementById('gpu-trigger');
    if (device === 'cpu') {
      _cpu.style.backgroundColor = '#3e3e3e';
    } else {
      _gpu.style.backgroundColor = '#3e3e3e';
    }

    class NeuralNet extends torch.nn.Module {
      constructor() {
        super();
        this.wIn = new torch.nn.Linear(784, boxCount[0], device);
        this.reluIn = new torch.nn.ReLU();
        if (boxCount.length > 1) {
          this.w1 = new torch.nn.Linear(boxCount[0], boxCount[1], device);
          this.relu1 = new torch.nn.ReLU();
        }
        if (boxCount.length > 2) {
          this.w2 = new torch.nn.Linear(boxCount[1], boxCount[2], device);
          this.relu2 = new torch.nn.ReLU();
        }
        if (boxCount.length > 3) {
          this.w3 = new torch.nn.Linear(boxCount[2], boxCount[3], device);
          this.relu3 = new torch.nn.ReLU();
        }
        if (boxCount.length > 4) {
          this.w4 = new torch.nn.Linear(boxCount[3]

```

--------------------------------

### Get Tensor Length

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/tensor/index.html

Returns the length of a tensor, which corresponds to the size of its first dimension. This is useful for understanding the tensor's shape.

```javascript
let a = torch.randn([3,2], true, 'gpu');
console.log(a.length);
```

--------------------------------

### Define and Train Neural Network in JavaScript with js-pytorch

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/tutorials/index.html

This snippet demonstrates the full implementation of a neural network using js-pytorch. It includes the class declaration for the network, instantiation of the model, definition of the loss function and optimizer, generation of dummy data, and a training loop that iterates, calculates loss, backpropagates, updates weights, and resets gradients.

```javascript
const torch = require("js-pytorch");
const nn = torch.nn;
const optim = torch.optim;

// Implement Module class:
class NeuralNet extends nn.Module {
  constructor(in_size, hidden_size, out_size) {
    super();
    // Instantiate Neural Network's Layers:
    this.w1 = new nn.Linear(in_size, hidden_size);
    this.relu1 = new nn.ReLU();
    this.w2 = new nn.Linear(hidden_size, hidden_size);
    this.relu2 = new nn.ReLU();
    this.w3 = new nn.Linear(hidden_size, out_size);
  };

  forward(x) {
    let z;
    z = this.w1.forward(x);
    z = this.relu1.forward(z);
    z = this.w2.forward(z);
    z = this.relu2.forward(z);
    z = this.w3.forward(z);
    return z;
  };
};

// Instantiate Model:
let in_size = 16;
let hidden_size = 32;
let out_size = 10;
let batch_size = 16;

let model = new NeuralNet(in_size,hidden_size,out_size);

// Define loss function and optimizer:
let loss_func = new nn.CrossEntropyLoss();
let optimizer = new optim.Adam(model.parameters(), 3e-3);

// Instantiate input and output:
let x = torch.randn([batch_size, in_size]);
let y = torch.randint(0, out_size, [batch_size]);
let loss;

// Training Loop:
for (let i = 0; i < 256; i++) {
  let z = model.forward(x);

  // Get loss:
  loss = loss_func.forward(z, y);

  // Backpropagate the loss using torch.tensor's backward() method:
  loss.backward();

  // Update the weights:
  optimizer.step();

  // Reset the gradients to zero after each training step:
  optimizer.zero_grad();

  // Print current loss:
  console.log(`Iter: ${i} - Loss: ${loss.data}`);
}
```

--------------------------------

### Train Transformer Model in JavaScript

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/tutorials/index.html

This snippet demonstrates the training loop for the Transformer model. It includes defining hyperparameters, instantiating the model, setting up the loss function and optimizer, and iterating through training steps with forward pass, loss calculation, backpropagation, and weight updates.

```javascript
// Define training hyperparameters:
const vocab_size = 52;
const hidden_size = 32;
const n_timesteps = 16;
const n_heads = 4;
const dropout_p = 0;
const batch_size = 8;

// Instantiate your custom nn.Module:
const model = new Transformer(vocab_size, hidden_size, n_timesteps, n_heads, dropout_p, device);

// Define loss function and optimizer:
const loss_func = new nn.CrossEntropyLoss();
const optimizer = new optim.Adam(model.parameters(), (lr = 5e-3), (reg = 0));

// Instantiate sample input and output:
let x = torch.randint(0, vocab_size, [batch_size, n_timesteps, 1]);
let y = torch.randint(0, vocab_size, [batch_size, n_timesteps]);
let loss;

// Training Loop:
for (let i = 0; i < 256; i++) {
  // Forward pass through the Transformer:
  let z = model.forward(x);

  // Get loss:
  loss = loss_func.forward(z, y);

  // Backpropagate the loss using torch.tensor's backward() method:
  loss.backward();

  // Update the weights:
  optimizer.step();

  // Reset the gradients to zero after each training step:
  optimizer.zero_grad();

  // Print loss at every iteration:
  console.log(`Iter ${i} - Loss ${loss.data[0].toFixed(4)}`)
}
```

--------------------------------

### JS-PyTorch: Define and Train a Neural Network

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/tutorials/index.html

Shows how to define a neural network using JS-PyTorch's `nn.Module`. It includes setting up layers (Linear, ReLU) in the constructor and defining the forward pass.

```javascript
const torch = require("js-pytorch");
const nn = torch.nn;
const optim = torch.optim;

// Implement Module class:
// (The actual class implementation is missing in the provided text, but this indicates where it would go)
```

--------------------------------

### Run Performance Benchmarks (npm)

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/README.md

Executes all performance benchmarks located in the 'tests/benchmarks/' directory. This helps in evaluating and improving the project's efficiency.

```Shell
npm run bench
```

--------------------------------

### Initialize Syntax Highlighting

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/search.html

Initializes syntax highlighting for all code blocks on the page using highlight.js. This is typically called after the DOM is loaded.

```javascript
hljs.highlightAll();
```

--------------------------------

### Reset Training State

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/demo/index.html

Resets the training state to its initial values. This includes stopping the loop, resetting accuracy, iteration count, and total visited examples. It also clears the graph data and resets button styles.

```javascript
function resetTraining() {
  in_loop = false;
  training = false;
  smoothAcc = 0.1;
  iter = 0;
  total_visited = 0;
  data.push([]);
  plotGraph();
  buttons = document.getElementsByClassName('layer-button');
  for (button of buttons) {
    button.style.backgroundColor = '';
  }
  buttons = document.getElementsByClassName('device-button');
  for (button of buttons) {
    button.style.backgroundColor = '';
  }
};
```

--------------------------------

### Run Tests (npm)

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/README.md

Executes the test suite for the project. This is crucial for verifying the functionality and stability of the codebase.

```Shell
npm test
```

--------------------------------

### Build for Distribution (npm)

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/README.md

Builds the project for distribution, outputting CJS and ESM modules along with TypeScript definitions to the 'dist/' folder. This command is essential for preparing the package for deployment.

```Shell
npm run build
```

--------------------------------

### Initialize Data Layer for Google Analytics

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/assets/demo/demo.html

Sets up the dataLayer for Google Analytics and configures the gtag.js tracking snippet with the provided measurement ID.

```JavaScript
window.dataLayer = window.dataLayer || [];
function gtag(){
  dataLayer.push(arguments);
}

gtag('js', new Date());
gtag('config', 'G-LY5RQBNBBW');
```

--------------------------------

### Neural Network Training Loop

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/demo/index.html

This snippet demonstrates a typical neural network training loop. It includes backpropagation using `backward()`, weight updates with `optimizer.step()`, and resetting gradients with `optimizer.zero_grad()`. It also handles test data processing, loss calculation, and accuracy evaluation, with logic for handling potential overflows.

```javascript
ss = loss_func.forward(z, y_batch) // Backpropagate the loss using neuralforge.tensor's backward() method: loss.backward() // Update the weights: optimizer.step() // Reset the gradients to zero after each training step: optimizer.zero_grad() // Now, get a batch of test data: let [x_test, y_test] = getBatch(test, batch_size) // Run the batch of test data through the model: let z_test = model.forward(x_test) // Get the test loss and accuracy: acc_test = getAccuracy(z_test, y_test) loss_test = loss_func.forward(z_test, y_test) smoothAcc = acc_test * 0.02 + smoothAcc * 0.98 // If loss went to infinity (model way too large for training size), represent that in the graph: if (isNaN(loss_test.data[0])) { overFlow = overFlow * 1.5; data[data.length - 1].push(overFlow + (Math.random() - 0.5) * 15); // If not, just keep adding the loss to the graph: } else {data[data.length - 1].push(loss_test.data[0])} // Display iteration and loss on the screen: document.getElementById('iter').innerHTML = `<b>Iteration:</b> ${iter}`; document.getElementById('total-visited').innerHTML = `<b>Total Training Examples:</b> ${total_visited}`; document.getElementById('loss').innerHTML = `<b>Validation Loss:</b> ${loss_test.data[0].toFixed(3)}`; document.getElementById('device-showcase').innerHTML = `<b>Device:</b> ${device}` document.getElementById('epoch').innerHTML = `<b>Epoch:</b> ${Math.floor(total_visited/train.length)}`; document.getElementById('acc').innerHTML = `<b>Validation Accuracy:</b> ${smoothAcc.toFixed(3)}`; iter += 1; total_visited += batch_size; plotGraph() optimizer.zero_grad() };
```

--------------------------------

### Load Transformer Model in JavaScript

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/tutorials/index.html

This snippet demonstrates how to load a saved Transformer model from a JSON file using `torch.load()`. It involves creating a placeholder model with the same architecture and then loading the weights into it.

```javascript
// To load, instantiate placeHolder using the original model's architecture:
const placeHolder = new Transformer(vocab_size, hidden_size, n_timesteps, n_heads, dropout_p);

// Load weights into placeHolder:
const newModel = torch.load(placeHolder, 'model.json')
```

--------------------------------

### Enable Sphinx Read the Docs Theme Navigation

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/search.html

Enables navigation features for the Sphinx Read the Docs theme. This JavaScript code is often used in documentation sites built with Sphinx.

```javascript
var base_url = ".";
jQuery(function () {
    SphinxRtdTheme.Navigation.enable(true);
});
```

--------------------------------

### Basic js-pytorch Browser Usage

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/README.md

Demonstrates basic tensor operations and neural network usage in the browser after including the js-pytorch script. It shows tensor creation, linear layer definition, and forward pass.

```html
<head>
    <title>My Project</title>
    <!-- New script goes here -->
    <script src="https://cdnjs.cloudflare.com/ajax/libs/js-pytorch/0.7.2/js-pytorch-browser.js" 
            integrity="sha512-l22t7GnqXvHBMCBvPUBdFO2TEYxnb1ziCGcDQcpTB2un16IPA4FE5SIZ8bUR+RwoDZGikQkWisO+fhnakXt9rg=="
            crossorigin="anonymous" 
            referrerpolicy="no-referrer">
    </script>
    <!---->
</head>
<body>
    <script>
        let x = torch.randn([10,5])
        let linear = new torch.nn.Linear(5,1,'gpu',true)
        let z = linear.forward(x)
        console.log(z.data)
    </script>
</body>
```

--------------------------------

### Import and Export PyTorch Components

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/assets/demo/demo.html

Imports necessary components (torch, train, test) from local modules and exports them for global access in the browser environment.

```JavaScript
import { torch } from './index.js';
import { train, test } from './data/data.js';

window.train = train;
window.test = test;
window.torch = torch;
```

--------------------------------

### JS-PyTorch: Compute Gradients with Tensors

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/tutorials/index.html

Demonstrates how to compute gradients for parameter tensors in JS-PyTorch. It involves creating input and parameter tensors, performing calculations, and then using `tensor.backward()` to obtain gradients.

```javascript
const { torch } = require("js-pytorch");

// Instantiate Input Tensor:
let x = torch.randn([8, 4, 5], false);

// Instantiate Parameter Tensors:
let w = torch.randn([8, 5, 4], true);
let b = torch.tensor([0.2, 0.5, 0.1, 0.0], true);

// Make calculations:
let y = torch.matmul(x, w);
y = torch.add(out, b);

// Compute gradients on whole graph:
y.backward();

// Get gradients from specific Tensors:
console.log(w.grad);
console.log(b.grad);
```

--------------------------------

### Neural Network Training Loop in JavaScript

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/tutorials/index.html

Details the core training loop for the neural network. Each iteration involves a forward pass, loss calculation, backpropagation, optimizer step, gradient reset, and logging the loss.

```javascript
// Training Loop:
for (let i = 0; i < 256; i++) {
  let z = model.forward(x);

  // Get loss:
  loss = loss_func.forward(z, y);

  // Backpropagate the loss using torch.tensor's backward() method:
  loss.backward();

  // Update the weights:
  optimizer.step();

  // Reset the gradients to zero after each training step:
  optimizer.zero_grad();

  // Print current loss:
  console.log(`Iter: ${i} - Loss: ${loss.data}`);
}
```

--------------------------------

### Format Code (Prettier)

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/README.md

Improves code formatting using Prettier. This command ensures a consistent and readable code style throughout the project.

```Shell
npm run prettier
```

--------------------------------

### Require Js-PyTorch in JavaScript

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/index.html

Demonstrates how to import and use the Js-PyTorch library in a JavaScript file, accessing its core modules like torch, nn, and optim.

```javascript
const { torch } = require("js-pytorch");
const nn = torch.nn;
const optim = torch.optim;
```

--------------------------------

### Create Batches for Training

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/assets/demo/demo.html

Generates batches of data for training by randomly selecting instances from the provided dataset and converting them into tensors.

```JavaScript
function getBatch(data, batch_size) {
  let x_batch = [];
  let y_batch = [];
  for (let i = 0; i < batch_size; i++) {
    p = Math.floor(Math.random() * data.length);
    x_batch.push(Object.entries(data[p])[0][1])
    y_batch.push(Number(Object.entries(data[p])[0][0]))
  }
  return [torch.tensor(x_batch), torch.tensor(y_batch)];
};
```

--------------------------------

### Initialize Data Layer and Google Analytics

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/demo/index.html

Initializes the data layer for analytics and configures Google Analytics with a specific tracking ID.

```javascript
window.dataLayer = window.dataLayer || [];
function gtag() {
  dataLayer.push(arguments);
}
gtag('js', new Date());
gtag('config', 'G-7QRVV2J34B');
```

--------------------------------

### Implement Transformer Model in JavaScript

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/tutorials/index.html

This snippet shows the full implementation of a Transformer model using js-pytorch. It defines the model architecture, including embedding, positional embedding, transformer blocks, layer normalization, and a linear layer. It also demonstrates the forward pass.

```javascript
const { torch } = require("js-pytorch");
const nn = torch.nn;
const optim = torch.optim;
const device = 'gpu';

// Create Transformer decoder Module:
class Transformer extends nn.Module {
  constructor(vocab_size, hidden_size, n_timesteps, n_heads, dropout_p, device) {
    super();
    // Instantiate Transformer's Layers:
    this.embed = new nn.Embedding(vocab_size, hidden_size);
    this.pos_embed = new nn.PositionalEmbedding(n_timesteps, hidden_size);
    this.b1 = new nn.Block(hidden_size, hidden_size, n_heads, n_timesteps, dropout_p, device);
    this.b2 = new nn.Block(hidden_size, hidden_size, n_heads, n_timesteps, dropout_p, device);
    this.ln = new nn.LayerNorm(hidden_size);
    this.linear = new nn.Linear(hidden_size, vocab_size, device);
  }

  forward(x) {
    let z;
    z = torch.add(this.embed.forward(x), this.pos_embed.forward(x));
    z = this.b1.forward(z);
    z = this.b2.forward(z);
    z = this.ln.forward(z);
    z = this.linear.forward(z);
    return z;
  }
}
```

--------------------------------

### PyTorch Training Loop and Optimization in JavaScript

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/assets/demo/demo.html

Implements a training loop for the neural network using PyTorch in JavaScript. It includes setting up the loss function, optimizer (Adam), and iterating through training data. It also handles calculating loss, backpropagation, weight updates, and gradient resets.

```javascript
// Define loss function and optimizer:
globalThis.loss_func = new torch.nn.CrossEntropyLoss();

// Get live learning rate and regularization values.
let batch_size = Number(document.getElementById('batch-size').value)
let lr = Number(document.getElementById('learning-rate').value)
let reg = Number(document.getElementById('regularization').value )
let beta1 = Number(document.getElementById('beta1').value )
let beta2 = Number(document.getElementById('beta2').value )
let eps = Number(0.000001)

// Build optimizer:
let optimizer = new torch.optim.Adam(model.parameters(), lr=lr, reg=reg, betas=[beta1, beta2], eps=eps)

let loss;
let loss_test;

// Training Loop:
for(let i = 0 ; i < 1 ; i++) {
    let [x_batch, y_batch] = getBatch(train, batch_size)
    let z = model.forward(x_batch)

    // Get loss:
    loss = loss_func.forward(z, y_batch)

    // Backpropagate the loss using neuralforge.tensor's backward() method:
    loss.backward()

    // Update the weights:
    optimizer.step()

    // Reset the gradients to zero after each training step:
    optimizer.zero_grad()

    // Now, get a batch of test data:
    let [x_test, y_test] = getBatch(test, batch_size)

    // Run the batch of test data through the model:
    let z_test = model.forward(x_test)

    // Get the test loss and accuracy:
    acc_test = getAccuracy(z_test, y_test)
    loss_test = loss_func.forward(z_test, y_batch)

    smoothAcc = acc_test * 0.02 + smoothAcc * 0.98

    // If loss went to infinity (model way too large for training size), represent that in the graph:
    if (isNaN(loss_test.data[0])) {
        overFlow = overFlow * 1.5;
        data.push(overFlow + (Math.random() - 0.5) * 15);
    // If not, just keep adding the loss to the graph:
    } else {
        data.push(loss_test.data)
    }

    // Display iteration and loss on the screen:
    document.getElementById('iter').innerHTML = `<b>Iteration:</b> ${iter}`;
    document.getElementById('total-visited').innerHTML = `<b>Total Training Examples:</b> ${total_visited}`;
    document.getElementById('loss').innerHTML = `<b>Validation Loss:</b> ${loss_test.data[0].toFixed(3)}`;
    document.getElementById('device-showcase').innerHTML = `<b>Device:</b> ${device}`
    document.getElementById('epoch').innerHTML = `<b>Epoch:</b> ${Math.floor(total_visited/train.length)}`;
    document.getElementById('acc').innerHTML = `<b>Validation Accuracy:</b> ${smoothAcc.toFixed(3)}`;

    iter += 1;
    total_visited += batch_size;
    plotGraph();
    optimizer.zero_grad()
};
```

--------------------------------

### Update Performance Benchmarks (npm)

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/README.md

Saves new performance benchmarks. This command is used when updating or adding new performance tests to the project.

```Shell
npm run bench:update
```

--------------------------------

### Batch Data Preparation in JavaScript

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/demo/index.html

A function to create batches of input (x) and output (y) data from a provided dataset. It randomly selects instances and converts them into PyTorch tensors.

```javascript
function getBatch(data, batch_size) {
  let x_batch = [];
  let y_batch = [];
  for (let i=0 ; i < batch_size ; i++) {
    p = Math.floor(Math.random() * data.length);
    x_batch.push(Object.entries(data[p])[0][1])
    y_batch.push(Number(Object.entries(data[p])[0][0]))
  };
  return [torch.tensor(x_batch), torch.tensor(y_batch)];
}
```

--------------------------------

### Create Tensor with Data - JS-PyTorch

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/tensor/index.html

Creates a tensor filled with provided data. Supports specifying whether to track gradients and the device (CPU/GPU). Useful for initializing tensors with specific values.

```JavaScript
let a = torch.tensor([[1,2,3],[4,5,6]], false, 'gpu');
console.log(a.data);
```

--------------------------------

### Control Device Selection (CPU/GPU)

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/assets/demo/demo.html

Functions to handle user interaction for selecting the computation device (CPU or GPU). It updates the visual style of the selection buttons.

```JavaScript
let device = 'cpu';

function GPUCaller() {
  if (!training) {
    device = 'gpu';
    let _cpu = document.getElementById('cpu-trigger');
    let _gpu = document.getElementById('gpu-trigger');
    _gpu.style.backgroundColor = '#3e3e3e';
    _cpu.style.backgroundColor = '';
  }
}

function CPUCaller() {
  if (!training) {
    device = 'cpu';
    let _cpu = document.getElementById('cpu-trigger');
    let _gpu = document.getElementById('gpu-trigger');
    _cpu.style.backgroundColor = '#3e3e3e';
    _gpu.style.backgroundColor = '';
  }
}
```

--------------------------------

### Test Transformer Model in JavaScript

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/tutorials/index.html

This snippet illustrates how to test a trained Transformer model. It involves passing test data through the model's forward method and then calculating the loss or accuracy using a loss function.

```javascript
// Load weights into placeHolder:
let z = model.forward(x);

// Then, use a **loss function** or a custom function to calculate your loss or accuracy in comparaison with the **target**:
let loss = nn.CrossEntropyLoss(z,y);
```

--------------------------------

### Device Selection (CPU/GPU) in JavaScript

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/demo/index.html

Functions to switch the computation device between CPU and GPU. These functions update the background color of UI elements to indicate the active device and set a global 'device' variable.

```javascript
function GPUCaller() {
  if (!training) {
    device = 'gpu';
    let _cpu = document.getElementById('cpu-trigger');
    let _gpu = document.getElementById('gpu-trigger');
    _gpu.style.backgroundColor = '#3e3e3e';
    _cpu.style.backgroundColor = '';
  }
}

function CPUCaller() {
  if (!training) {
    device = 'cpu';
    let _cpu = document.getElementById('cpu-trigger');
    let _gpu = document.getElementById('gpu-trigger');
    _cpu.style.backgroundColor = '#3e3e3e';
    _gpu.style.backgroundColor = '';
  }
}
```

--------------------------------

### Import JS-PyTorch Modules

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/demo/index.html

Imports the torch object and train/test data functions from local modules for use in the browser.

```javascript
import { torch } from './index.js';
import { train, test } from './data/data.js';
window.train = train;
window.test = test;
window.torch = torch;
```

--------------------------------

### nn.MultiHeadSelfAttention Layer Initialization

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/layers/index.html

Describes the initialization of the nn.MultiHeadSelfAttention layer, used for applying self-attention. It takes parameters such as input size, output size, number of heads, timesteps, dropout probability, and device.

```javascript
new nn.MultiHeadSelfAttention(in_size,
                          out_size,
                          n_heads,
                          n_timesteps,
                          dropout_prob,
                          device)
```

--------------------------------

### FullyConnected Initialization and Forward Pass

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/layers/index.html

Initializes and performs a forward pass with the FullyConnected module. This module applies a fully-connected layer, ReLU activation, another fully-connected layer, and dropout. It takes input size, output size, dropout probability, device, and bias as parameters.

```JavaScript
const fc = new nn.FullyConnected(10, 15, 0.2, 'gpu');
let x = torch.randn([100,50,10], true, 'gpu');
let y = fc.forward(x);
console.log(y.shape);
```

--------------------------------

### Block Initialization and Forward Pass

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/layers/index.html

Initializes and performs a forward pass with the Block module, which applies a transformer block layer. This includes Layer Normalization, Self Attention, and a Fully Connected layer. It requires input size, output size, number of heads, number of timesteps, dropout probability, and device.

```JavaScript
let z = x.add(this.att.forward(this.ln1.forward(x)));
z = z.add(this.fcc.forward(this.ln2.forward(z)));
return z;
```

--------------------------------

### PyTorch Neural Network Definition and Training Step in JavaScript

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/demo/index.html

Defines a neural network model (`NeuralNet`) extending `torch.nn.Module` with configurable layers based on `boxCount`. It also includes the `trainStep` function which performs a forward pass, calculates loss, and updates model parameters using an Adam optimizer.

```javascript
function trainStep() {
  if (!training) {
    training = true;
    buttons = document.getElementsByClassName('layer-button');
    for (button of buttons) {
      button.style.backgroundColor = '#0056b3';
    };
    let _cpu = document.getElementById('cpu-trigger');
    let _gpu = document.getElementById('gpu-trigger');
    if (device === 'cpu') {
      _cpu.style.backgroundColor = '#3e3e3e';
    } else {
      _gpu.style.backgroundColor = '#3e3e3e';
    }

    class NeuralNet extends torch.nn.Module {
      constructor() {
        super();
        this.wIn = new torch.nn.Linear(784,boxCount[0],device);
        this.reluIn = new torch.nn.ReLU();
        if (boxCount.length > 1) {
          this.w1 = new torch.nn.Linear(boxCount[0],boxCount[1],device);
          this.relu1 = new torch.nn.ReLU();
        }
        if (boxCount.length > 2) {
          this.w2 = new torch.nn.Linear(boxCount[1],boxCount[2],device);
          this.relu2 = new torch.nn.ReLU();
        }
        if (boxCount.length > 3) {
          this.w3 = new torch.nn.Linear(boxCount[2],boxCount[3],device);
          this.relu3 = new torch.nn.ReLU();
        }
        if (boxCount.length > 4) {
          this.w4 = new torch.nn.Linear(boxCount[3],boxCount[4],device);
          this.relu4 = new torch.nn.ReLU();
        };
        this.wOut = new torch.nn.Linear(boxCount[boxCount.length-1], 10, device);
      };
      forward(x) {
        let z;
        z = this.wIn.forward(x);
        z = this.reluIn.forward(z);
        if (boxCount.length > 1) {
          z = this.w1.forward(z);
          z = this.relu1.forward(z);
        }
        if (boxCount.length > 2) {
          z = this.w2.forward(z);
          z = this.relu2.forward(z);
        }
        if (boxCount.length > 3) {
          z = this.w3.forward(z);
          z = this.relu3.forward(z);
        }
        if (boxCount.length > 4) {
          z = this.w4.forward(z);
          z = this.relu4.forward(z);
        };
        z = this.wOut.forward(z);
        z = z.div(100);
        return z;
      };
    };
    globalThis.model = new NeuralNet();
    globalThis.loss_func = new torch.nn.CrossEntropyLoss();
  };

  let batch_size = Number(document.getElementById('batch-size').value)
  let lr = Number(document.getElementById('learning-rate').value)
  let reg = Number(document.getElementById('regularization').value )
  let beta1 = Number(document.getElementById('beta1').value )
  let beta2 = Number(document.getElementById('beta2').value )
  let eps = Number(0.000001)

  let optimizer = new torch.optim.Adam(model.parameters(), lr=lr, reg=reg, betas=[beta1, beta2], eps=eps)
  let loss;
  let loss_test;

  for(let i=0 ; i < 1 ; i++) {
    let [x_batch, y_batch] = getBatch(train, batch_size)
    let z = model.forward(x_batch)
    loss = loss_func(z, y_batch);
    optimizer.zero_grad();
    loss.backward();
    optimizer.step();
  };
}
```

--------------------------------

### nn.Linear Layer Initialization and Usage

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/layers/index.html

Initializes and demonstrates the usage of the nn.Linear layer, which applies a linear transformation to the input tensor. It includes parameters for input size, output size, device, bias, and Xavier initialization.

```javascript
const linear = new nn.Linear(10,15,'gpu');
let x = torch.randn([100,50,10], true, 'gpu');
let y = linear.forward(x);
console.log(y.shape);
```

--------------------------------

### Include js-pytorch in HTML for Browser

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/README.md

Includes the js-pytorch library in an HTML file for browser-based usage. This script tag should be placed within the `<head>` section of your HTML.

```html
<script src="https://cdnjs.cloudflare.com/ajax/libs/js-pytorch/0.7.2/js-pytorch-browser.js"
        integrity="sha512-l22t7GnqXvHBMCBvPUBdFO2TEYxnb1ziCGcDQcpTB2un16IPA4FE5SIZ8bUR+RwoDZGikQkWisO+fhnakXt9rg=="
        crossorigin="anonymous"
        referrerpolicy="no-referrer"></script>
```

--------------------------------

### MultiHeadSelfAttention Initialization and Forward Pass

Source: https://github.com/eduardoleao052/js-pytorch/blob/main/site/layers/index.html

Initializes and performs a forward pass with the MultiHeadSelfAttention module. It takes input size, output size, number of heads, and other parameters. The forward pass processes an input tensor and returns an output tensor.

```JavaScript
const att = new nn.MultiHeadSelfAttention(10, 15, 2, 32, 0.2, 'gpu');
let x = torch.randn([100,50,10], true, 'gpu');
let y = att.forward(x);
console.log(y.shape);
```