### Configure Training Setup

Source: https://github.com/boschresearch/torchphysics/blob/main/experiments/burgers_equation.ipynb

Initializes the training setup with specified variables, training conditions, validation conditions, solution dimensions, and number of iterations.

```python
setup = Setting(variables=(x, t),
                train_conditions={'pde': train_cond},
                val_conditions={},
                solution_dims={'u': 2},
                n_iterations=5)
```

--------------------------------

### Install TorchPhysics from Source

Source: https://github.com/boschresearch/torchphysics/blob/main/README.rst

Install TorchPhysics from a local clone of the repository, including all optional dependencies. This is recommended if you plan to modify the code.

```bash
git clone https://github.com/boschresearch/torchphysics 
cd path_to_torchphysics_folder
pip install .[all]
```

--------------------------------

### Install TorchPhysics from Source

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/readme.md

Install TorchPhysics from its source code. This method is recommended if you plan to modify the library's code.

```bash
git clone https://github.com/boschresearch/torchphysics
cd path_to_torchphysics_folder
pip install .[all]
```

--------------------------------

### Start Training with Trainer

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/tutorial/solve_pde.md

Initiates the training process using the configured trainer and solver.

```python
trainer.fit(solver) # start training
```

--------------------------------

### Install TorchPhysics

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Exercise2_1.ipynb

Installs the TorchPhysics library. Recommended to run in a GPU-enabled environment like Google Colab.

```python
!pip install torchphysics
```

--------------------------------

### Configure and Start Training

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Exercise2_2.ipynb

Sets up the training conditions, optimizer settings, and PyTorch Lightning trainer to initiate the model training process for solving the PDE.

```python
# Start the training
training_conditions = [pde_condition, initial_condition, boundary_condition]
optim = tp.OptimizerSetting(optimizer_class=torch.optim.Adam, lr=learning_rate)
solver = tp.solver.Solver(train_conditions=training_conditions, optimizer_setting=optim)

trainer = pl.Trainer(devices=1, accelerator="gpu",
                     num_sanity_val_steps=0,
                     benchmark=True,
                     max_steps=train_iterations, 
                     logger=False, 
                     enable_checkpointing=False)

trainer.fit(solver) # run the training loop
```

--------------------------------

### Configure Solver and Trainer

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Exercise3_2.ipynb

Sets up the optimizer and solver, then configures and starts the training process using PyTorch Lightning's Trainer.

```python
### Start training
optim = tp.OptimizerSetting(optimizer_class=torch.optim.Adam, lr=learning_rate) 
solver = tp.solver.Solver([data_condition], optimizer_setting=optim)


trainer = pl.Trainer(devices=1, accelerator="gpu",
                     num_sanity_val_steps=0,
                     benchmark=True,
                     max_steps=train_iterations, 
                     logger=False, 
                     enable_checkpointing=False)

trainer.fit(solver)
```

--------------------------------

### Setup Adam Optimizer and Solver

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/deepritz/corner_pde.ipynb

Configures the Adam optimizer with a learning rate and initializes the solver with the defined boundary and PDE conditions.

```python
optim = tp.OptimizerSetting(optimizer_class=torch.optim.Adam, lr=0.001)
solver = tp.solver.Solver(train_conditions=[bound_cond, pde_cond], optimizer_setting=optim)
```

--------------------------------

### torchphysics.solver.Solver.on_train_start

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/api/modules.md

Callback executed at the beginning of the training process. Used for any setup or initialization required before training commences.

```APIDOC
## Solver.on_train_start()

### Description
Callback executed at the beginning of the training process. Used for any setup or initialization required before training commences.

### Method
(Not specified in source)

### Endpoint
(Not specified in source)

### Parameters
(No parameters explicitly documented)

### Request Example
(No request example provided in source)

### Response
(No response details provided in source)
```

--------------------------------

### Import Libraries and Setup

Source: https://github.com/boschresearch/torchphysics/blob/main/experiments/burgers_equation.ipynb

Imports necessary libraries for PyTorch, NumPy, PyTorch Lightning, and torchphysics modules. Sets up CUDA visibility.

```python
import torch
import numpy as np
import pytorch_lightning as pl
from timeit import default_timer as timer

from torchphysics.problem import Variable
from torchphysics.setting import Setting
from torchphysics.problem.domain import (
    Rectangle,
    Interval,
    Circle)
from torchphysics.problem.condition import (
    DirichletCondition,
    DiffEqCondition,
    DataCondition)
from torchphysics.models.fcn import SimpleFCN
from torchphysics import PINNModule, AdaptiveWeightModule
from torchphysics.utils import laplacian, jac, convective
from torchphysics.utils.fdm import FDM, create_validation_data
from torchphysics.utils.plot import Plotter
from torchphysics.utils.evaluation import (
    get_min_max_inside,
    get_min_max_boundary)
from torchphysics.setting import Setting

os.environ["CUDA_VISIBLE_DEVICES"] = "0" # select GPUs to use

#pl.seed_everything(43) # set a global seed
torch.cuda.is_available()
```

--------------------------------

### Initialize PINN Module Solver

Source: https://github.com/boschresearch/torchphysics/blob/main/experiments/burgers_equation.ipynb

Sets up the PINN solver using a SimpleFCN model, Adam optimizer, and an exponential learning rate scheduler. This is a commented-out example.

```python
"""
solver = PINNModule(model=SimpleFCN(variable_dims=setup.variable_dims,
                    solution_dims=setup.solution_dims,
                    depth=4,
                    width=20),
    optimizer=torch.optim.Adam,
    lr=1e-2,
    scheduler = {'class': torch.optim.lr_scheduler.ExponentialLR,'args': {'gamma': 0.99}, 
                 'interval': 'step'}
    #log_plotter=plotter
    )
"""
```

--------------------------------

### Initialize and Run PyTorch Lightning Trainer

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/tutorial/Introduction_Tutorial_PINNs.ipynb

Set up a PyTorch Lightning Trainer with specified parameters like device, number of steps, and benchmark settings, then start the training using the solver.

```python
# Start the training
accelerator = "gpu" if torch.cuda.is_available() else "cpu"

trainer = pl.Trainer(devices=1, accelerator=accelerator, # what to use to solve problem and how many devices
                     num_sanity_val_steps=0,
                     benchmark=True,
                     max_steps=5000, # number of training steps
                     logger=False,
                     enable_checkpointing=False)

trainer.fit(solver) # start training
```

--------------------------------

### Setup Training Environment and Components

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Sol1_2.ipynb

Moves the model and training data to the GPU if available. Initializes the Mean Squared Error loss function and the Adam optimizer for training.

```python
### Move data to GPU
model.to("cuda")
input_training = input_training.to("cuda")
output_training = output_training.to("cuda")

### For the loss, we take the mean squared error and Adam for optimization. 
loss_fn = torch.nn.MSELoss() 
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
```

--------------------------------

### Install TorchPhysics with pip

Source: https://github.com/boschresearch/torchphysics/blob/main/README.rst

Install TorchPhysics using pip for general use. This command installs the core library and its essential dependencies.

```bash
pip install torchphysics
```

--------------------------------

### Setup Solver with Adam Optimizer

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/tutorial/Tutorial_PINNs_Parameter_Dependency.ipynb

Initializes the TorchPhysics solver with a list of training conditions and an Adam optimizer configuration specifying learning rate.

```python
optim = tp.OptimizerSetting(optimizer_class=torch.optim.Adam, lr=0.025)

solver = tp.solver.Solver(train_conditions=training_conditions, optimizer_setting=optim)
```

--------------------------------

### Generate and Plot Example Data

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Exercise3_2.ipynb

Generates the dataset using the `data_create_fn` and plots example D(t) and u(t) functions to visualize the generated data.

```python
### Here we create the data
t_tensor, u_tensor, D_tensor = data_create_fn(N_t, data_batch, H)

### Show an example plot
import matplotlib.pyplot as plt
plt.figure(figsize=(16, 4))
plt.subplot(1, 2, 1)
plt.plot(t_tensor[::2], D_tensor[0])
plt.plot(t_tensor[::2], D_tensor[4000])
plt.plot(t_tensor[::2], D_tensor[8000])
plt.ylabel("D")
plt.xlabel("t")
plt.grid()
plt.subplot(1, 2, 2)
plt.plot(t_tensor, u_tensor[0])
plt.plot(t_tensor, u_tensor[4000])
plt.plot(t_tensor, u_tensor[8000])
plt.ylabel("u")
plt.xlabel("t")
plt.title("Solution for corresponding D")
plt.grid()
```

--------------------------------

### Start Model Training

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/pinn/poisson-with-input-params.ipynb

Initiates the training process for the TorchPhysics solver using the configured PyTorch Lightning trainer.

```python
trainer.fit(solver)
```

--------------------------------

### Plot Example Data

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/operator_learning/Stokes_2D.ipynb

Visualizes one example from the training data, showing the input domain geometry and the corresponding solution for the x-component of the velocity ($u_x$).

```python
# Plot one example of the data
f, axarr = plt.subplots(1,2, figsize=(6, 10))
plot_idx = 12
axarr[0].imshow(train_input[plot_idx, :, :, 0], origin='lower')
axarr[0].title.set_text(r"Example domain")
axarr[1].imshow(train_output[plot_idx, :, :, 0], origin='lower')
axarr[1].title.set_text(r"Example solution $u_x$")
```

--------------------------------

### FDM Training Loop Setup

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Sol1_3.ipynb

Initializes the model, tensors, and optimizer for a Finite Difference Method (FDM) approach. Moves data to CUDA for accelerated computation.

```python
# ## Implementation for FDM:

t = torch.linspace(t_min, t_max, 500).reshape(500, 1)
t_zero = torch.tensor([0.0])


model.to("cuda")
t = t.to("cuda")
t_zero = t_zero.to("cuda")


dT = t[1] - t[0] # step width
t_delta = torch.tensor([dt], device="cuda")


### For the loss, we take the mean squared error and Adam for optimization.
loss_fn_ode = torch.nn.MSELoss()
loss_fn_initial_position = torch.nn.MSELoss()
loss_fn_initial_speed = torch.nn.MSELoss()

optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)

### Training loop
for k in range(train_iterations):
    ### Loss for the differential equation: u_tt = D*(u_t)^2 - g
    u = model(t)
    u_ghost_node = torch.cat((u[1:2], u), dim=0)
    u_t = (u_ghost_node[2:] - u_ghost_node[:-2])/(2.0 * dt)
    u_tt = (u_ghost_node[2:] - 2*u_ghost_node[1:-1] + u_ghost_node[:-2]) / dt**2

    ode = u_tt - D*u_t**2 + g

    loss_ode = loss_fn_ode(ode, torch.zeros_like(ode))

    ### Loss for initial condition: u(0) = H
    u_zero = model(t_zero)
    loss_initial_position = loss_fn_initial_position(u_zero - H, torch.zeros_like(u_zero))

    ### Loss for the initial velocity: u_t(0) = 0
    u_delta_t = model(t_delta)
    loss_initial_speed = loss_fn_initial_speed((u_delta_t - u_zero), torch.zeros_like(u_zero))

    ### Add all loss terms
    total_loss = loss_ode + loss_initial_position + loss_initial_speed

    ### Shows current loss every 250 iterations:
    if k % 250 == 0 or k == train_iterations - 1:
        print("Loss at iteration %i / %i is %f" %(k, train_iterations, total_loss.item()))

    ### Optimization step
    optimizer.zero_grad()
    total_loss.backward()
    optimizer.step()
```

--------------------------------

### Setup L-BFGS Optimizer and Solver

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/deepritz/corner_pde.ipynb

Configures the L-BFGS optimizer with specific arguments and makes the samplers static before re-initializing the solver.

```python
optim = tp.OptimizerSetting(optimizer_class=torch.optim.LBFGS, lr=0.05, 
                            optimizer_args={'max_iter': 2, 'history_size': 100})

pde_cond.sampler = pde_cond.sampler.make_static() 
bound_cond.sampler = bound_cond.sampler.make_static() 
solver = tp.solver.Solver(train_conditions=[bound_cond, pde_cond], optimizer_setting=optim)
```

--------------------------------

### Finite Difference Method Setup

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/pinn/heat-equation.ipynb

Computes the solution to the heat equation using a finite difference scheme for comparison. Requires importing FDM and transform_to_points.

```python
import sys
sys.path.append('..')

from fdm_heat_equation import FDM, transform_to_points

fdm_domain, fdm_time_domains, fdm_solution = FDM([0, w, 0, h], 2*[2e-1], [0,5], [1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0], f)
fdm_inp, fdm_out = transform_to_points(fdm_domain, fdm_time_domains, fdm_solution, [1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0], True)
```

--------------------------------

### Custom Autograd Function - Setup Context (Usage 2)

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/api/torchphysics.models.md

Sets up the context object for a custom autograd function when using the separate forward and setup_context approach.

```python
@staticmethod
def setup_context(ctx: Any, inputs: Tuple[Any, ...], output: Any) -> None:
    pass
```

--------------------------------

### Define Example Function and Input Tensors

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/tutorial/differentialoperators.md

Define a sample function and input tensors for evaluating derivatives. Ensure requires_grad=True for tensors to enable gradient computation.

```python
import torch
# Define some example function:
def f(x, t):
  return torch.sin(t * x[:, :1]) + x[:, 1:]**2

# Define some points where to evaluate the function
x = torch.tensor([[1.0, 1.0], [0, 1], [1, 0]], requires_grad=True)
t = torch.tensor([[1], [0], [2.0]], requires_grad=True)
# requires_grad=True is needed, so PyTorch knows to create a backwards graph.
# These tensors could be seen as a batch with three data points.
```

--------------------------------

### Configure Physics-Informed Neural Network (PINN) Setup

Source: https://github.com/boschresearch/torchphysics/blob/main/experiments/geometry/heat_equation_bearing_orig.ipynb

Assembles the defined variables and conditions into a `Setting` object for the PINN solver. It also initializes a `Plotter` for visualization and defines the neural network architecture (`SimpleFCN`) and the solver (`PINNModule`).

```python
# Put everything together
from torchphysics.utils.plot import Plotter

setup = Setting(variables=(x, t, D),
                train_conditions={'pde': train_cond},
                val_conditions={},
                n_iterations=100)

plotter = Plotter(plot_variables=setup.variables['x'],
                  points=400,
                  dic_for_other_variables={'t': 1.0, 'D': 15.0},
                  all_variables=setup.variables,
                  log_interval=10)

model = SimpleFCN(input_dim=4, width=50)

solver = PINNModule(model=model,
                    optimizer=torch.optim.Adam,
                    lr=1e-3)
```

--------------------------------

### Setup Solver with LBFGS Optimizer

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/tutorial/Tutorial_PINNs_Parameter_Dependency.ipynb

Reconfigures the solver with an LBFGS optimizer, specifying learning rate and optimizer arguments. Note that the PDE condition sampler must be made static for LBFGS.

```python
optim = tp.OptimizerSetting(optimizer_class=torch.optim.LBFGS, lr=0.05, 
                            optimizer_args={'max_iter': 2, 'history_size': 100})

pde_condition.sampler = pde_condition.sampler.make_static() # LBFGS can not work with varying points!
solver = tp.solver.Solver(train_conditions=training_conditions, optimizer_setting=optim)
```

--------------------------------

### Setting up Animation Sampler (Commented Out)

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/pinn/inverse-heat-equation-D-function.ipynb

Initializes an `AnimationSampler` for creating animations. The code for generating and saving the animation is commented out.

```python
anim_sampler = tp.samplers.AnimationSampler(domain_x, domain_t, 200, n_points=760)
#fig, anim = tp.utils.animate(model_u, lambda u: u, anim_sampler, ani_speed=10, ani_type='contour_surface')
#anim.save('inverse-heat-eq.gif')
```

--------------------------------

### Set Up and Train the Solver

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Sol2_1.ipynb

Configures the optimizer and the solver with all defined conditions. It then sets up and runs the PyTorch Lightning trainer to fit the solver to the data.

```python
### Syntax for the training is already implemented:

optim = tp.OptimizerSetting(optimizer_class=torch.optim.Adam, lr=learning_rate) 
solver = tp.solver.Solver([ode_condition, initial_position_condition, initial_velocity_condition],
                          optimizer_setting=optim)

trainer = pl.Trainer(devices=1, accelerator="gpu",
                     num_sanity_val_steps=0,
                     benchmark=True,
                     max_steps=train_iterations, 
                     logger=False, 
                     enable_checkpointing=False)

trainer.fit(solver)
```

--------------------------------

### Optimizer Setup for Inverse Problems

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Exercise2_3.ipynb

Use LBFGS optimizer for better results in inverse problems. This snippet shows the setup for the optimizer and the solver.

```python
optim = tp.OptimizerSetting(optimizer_class=torch.optim.LBFGS, lr=0.5, optimizer_args={'max_iter': 2})
pde_condition.sampler = pde_condition.sampler.make_static()
solver = tp.solver.Solver([pde_condition, data_condition], optimizer_setting=optim)

trainer = pl.Trainer(devices=1, accelerator="gpu",
                     num_sanity_val_steps=0,
                     benchmark=True,
                     max_steps=2500, 
                     logger=False, 
                     enable_checkpointing=False)
trainer.fit(solver)
```

--------------------------------

### Example Usage of nn.Linear

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/api/torchphysics.models.deeponet.md

Demonstrates the basic usage of a linear layer, showing input and output tensor sizes. This is a standard PyTorch example often used for comparison or as a base.

```python
>>> m = nn.Linear(20, 30)
>>> input = torch.randn(128, 20)
>>> output = m(input)
>>> print(output.size())
torch.Size([128, 30])
```

--------------------------------

### Setting up and Training the Solver

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/pinn/inverse-heat-equation-D-function.ipynb

Configures the sampler, solver, and trainer for fitting the model. The trainer is set up with specific parameters for GPU acceleration and maximum steps. The `trainer.fit(solver)` command initiates the training process.

```python
pde_condition.sampler = pde_condition.sampler.make_static()
solver = tp.solver.Solver([pde_condition, data_condition], optimizer_setting=optim)

trainer = pl.Trainer(devices=1, accelerator="gpu",
                     num_sanity_val_steps=0,
                     benchmark=True,
                     max_steps=1000, 
                     logger=False, 
                     enable_checkpointing=False)
                     
trainer.fit(solver)
```

--------------------------------

### torchphysics.models.activation_fn.relu_n.forward

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/api/torchphysics.models.md

Defines the forward pass of a custom autograd Function. This function can be implemented in two ways: combined forward and context setup, or separate forward and context setup functions.

```APIDOC
## static forward(ctx, x, n)

### Description
Define the forward of the custom autograd Function.

This function is to be overridden by all subclasses.
There are two ways to define forward:

Usage 1 (Combined forward and ctx):
```default
@staticmethod
def forward(ctx: Any, *args: Any, **kwargs: Any) -> Any:
    pass
```

- It must accept a context ctx as the first argument, followed by any
  number of arguments (tensors or other types).
- See combining-forward-context for more details

Usage 2 (Separate forward and ctx):

```default
@staticmethod
def forward(*args: Any, **kwargs: Any) -> Any:
    pass


@staticmethod
def setup_context(ctx: Any, inputs: Tuple[Any, ...], output: Any) -> None:
    pass
```

- The forward no longer accepts a ctx argument.
- Instead, you must also override the `torch.autograd.Function.setup_context()`
  staticmethod to handle setting up the `ctx` object.
  `output` is the output of the forward, `inputs` are a Tuple of inputs
  to the forward.
- See extending-autograd for more details

The context can be used to store arbitrary data that can be then
retrieved during the backward pass. Tensors should not be stored
directly on ctx (though this is not currently enforced for
backward compatibility). Instead, tensors should be saved either with
`ctx.save_for_backward()` if they are intended to be used in
`backward` (equivalently, `vjp`) or `ctx.save_for_forward()`
if they are intended to be used for in `jvp`.
```

--------------------------------

### Custom Autograd Function: Separate Forward and Context Setup

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/api/torchphysics.models.deeponet.md

Defines the forward pass and context setup for a custom autograd Function separately. The forward method does not accept a context argument; context is managed by setup_context.

```python
import torch
from typing import Any, Tuple

class MyFunction(torch.autograd.Function):
    @staticmethod
    def forward(*args: Any, **kwargs: Any) -> Any:
        # Implementation here
        pass

    @staticmethod
    def setup_context(ctx: Any, inputs: Tuple[Any, ...], output: Any) -> None:
        # Context setup implementation here
```

--------------------------------

### Configure Optimizer and Solver, then Train

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/pinn/inverse-heat-equation-D-function.ipynb

Sets up the optimizer, the solver with both the PDE and data conditions, and the PyTorch Lightning trainer. The trainer is configured with specific parameters for GPU acceleration and training steps before initiating the training process.

```python
optim = tp.OptimizerSetting(optimizer_class=torch.optim.Adam, lr=0.001)

solver = tp.solver.Solver([pde_condition, data_condition])

trainer = pl.Trainer(devices=1, accelerator="gpu",
                     num_sanity_val_steps=0,
                     benchmark=True,
                     max_steps=8000,
                     logger=False,
                     enable_checkpointing=False)
                     
trainer.fit(solver)
```

--------------------------------

### scatter

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/api/torchphysics.utils.plotting.md

Shows an example of the created points of the sampler.

```APIDOC
## scatter

### Description
Shows one batch of used points in the training. If the sampler is static, the shown points will be the points for the training. If not, the points may vary depending on the sampler.

### Parameters
* **subspace** (*torchphysics.problem.Space*) – The (sub-)space of which the points should be plotted. Only plotting for dimensions <= 3 is possible.
* **\*samplers** (*torchphysics.problem.Samplers*) – The different samplers for which the points should be plotted. The plot for each sampler will be created in the order they were passed in.

### Returns
**fig** – The figure handle of the plot.

### Return type
matplotlib.pyplot.figure
```

--------------------------------

### Training Loss Output (Periodic)

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Sol1_2.ipynb

Example output showing the loss at regular intervals during training.

```text
Loss at iteration 249 / 5000 is 27.958805
Loss at iteration 499 / 5000 is 27.958416
Loss at iteration 749 / 5000 is 2.177030
Loss at iteration 999 / 5000 is 0.083997
Loss at iteration 1249 / 5000 is 0.017167
Loss at iteration 1499 / 5000 is 0.007438
Loss at iteration 1749 / 5000 is 0.004969
Loss at iteration 1999 / 5000 is 0.003936
Loss at iteration 2249 / 5000 is 0.003399
Loss at iteration 2499 / 5000 is 0.003055
Loss at iteration 2749 / 5000 is 0.002793
Loss at iteration 2999 / 5000 is 0.002571
Loss at iteration 3249 / 5000 is 0.002376
Loss at iteration 3499 / 5000 is 0.002196
Loss at iteration 3749 / 5000 is 0.002039
Loss at iteration 3999 / 5000 is 0.001882
Loss at iteration 4249 / 5000 is 0.001732
Loss at iteration 4499 / 5000 is 0.001333
Loss at iteration 4749 / 5000 is 0.001198
Loss at iteration 4999 / 5000 is 0.001086

```

--------------------------------

### Set Up Solver and Trainer

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Exercise2_1.ipynb

Configures the optimizer, solver, and PyTorch Lightning trainer for the neural network training process.

```python
### Syntax for the training is already implemented:
optim = tp.OptimizerSetting(optimizer_class=torch.optim.Adam, lr=learning_rate) 
solver = tp.solver.Solver([ode_condition, initial_position_condition, initial_velocity_condition],
                          optimizer_setting=optim)

trainer = pl.Trainer(devices=1, accelerator="gpu",
                     num_sanity_val_steps=0,
                     benchmark=True,
                     max_steps=train_iterations, 
                     logger=False, 
                     enable_checkpointing=False)

trainer.fit(solver)
```

--------------------------------

### FunctionSetTransform

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/api/torchphysics.problem.domains.functionsets.md

A wrapper class for a function set that modifies its created functions, for example, by clamping values.

```APIDOC
### class torchphysics.problem.domains.functionsets.functionset_operations.FunctionSetTransform(function_set: [FunctionSet](#torchphysics.problem.domains.functionsets.functionset.FunctionSet), transformation)

Bases: [`FunctionSet`](#torchphysics.problem.domains.functionsets.functionset.FunctionSet)

A class that acts as a wrapper of a different function set to further modify
the created functions. E.g., clamping the values produced by a different
function set.

* **Parameters:**
  * **function_set** (*tp.domains.FunctionSet*) – The function set that should be transformed.
  * **transformation** (*callable*) – The function that carries out the transformation. This transformation will
    be carried out “pointwise”.

#### create_functions(device='cpu')

Creates the functions for the function set and stores them.
The created functions can then be retrieved by the get_function method.

* **Parameters:**
  **device** (*str*) – The device on which the functions should be stored.
```

--------------------------------

### Get Space Dimensionality

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/tutorial/tutorial_spaces_and_points.md

Retrieve the total number of dimensions defined within a Space object.

```python
>>> G.dim
3
```

--------------------------------

### Training Loss Output (First Iteration)

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Sol1_2.ipynb

Example output showing the loss at the very first iteration of training.

```text
Loss at iteration 0 / 5000 is 27.970970

```

--------------------------------

### Set Up Optimizer, Solver, and Trainer

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/pinn/hard-constrains.ipynb

Configures the optimizer (Adam), solver, and PyTorch Lightning trainer with specified parameters for training the model.

```python
optim = tp.OptimizerSetting(optimizer_class=torch.optim.Adam, lr=0.001)

solver = tp.solver.Solver([pde_condition, dirichlet_condition, neumann_condition], optimizer_setting=optim)

trainer = pl.Trainer(gpus=1,
                     max_steps=5000,
                     logger=False,
                     benchmark=True,
                     checkpoint_callback=False)
                     
trainer.fit(solver)
```

--------------------------------

### Get Domain Boundaries

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/tutorial/tutorial_domain_basics.md

Retrieves the boundary of existing domains R (unit square) and C (unit circle).

```python
R_bound = R.boundary
C_bound = C.boundary
```

--------------------------------

### Initialize Solver and Trainer

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/deepritz/poisson-equation.ipynb

Configures the PyTorch Lightning Solver with the defined PDE and boundary conditions, and sets up the PyTorch Lightning Trainer with specific training parameters like max steps and device.

```python
solver = tp.solver.Solver([pde_condition,
                           boundary_condition])

trainer = pl.Trainer(devices=1, accelerator="gpu",
                     num_sanity_val_steps=0,
                     benchmark=True,
                     max_steps=2000, 
                     logger=False, 
                     enable_checkpointing=False)
trainer.fit(solver)
```

--------------------------------

### Initialize PyTorch Lightning Trainer

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/tutorial/solve_pde.md

Sets up the PyTorch Lightning Trainer for the training process. Configure GPU usage, max steps, and logging.

```python
import pytorch_lightning as pl

trainer = pl.Trainer(gpus=1, # or None if CPU is used
                     max_steps=4000, # number of training steps
                     logger=False,
                     benchmark=True,
                     checkpoint_callback=False)
```

--------------------------------

### Plotting Inner and Boundary Points

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/deepritz/poisson-equation.ipynb

Visualizes the inner and boundary points sampled from a domain using Matplotlib. Ensure you have matplotlib installed.

```python
inner_points = next(inner_sampler)
boundary_points = next(boundary_sampler)

from matplotlib import pyplot as plt

plt.scatter(inner_points.as_tensor[:,0],
            inner_points.as_tensor[:,1])

plt.scatter(boundary_points.as_tensor[:,0],
            boundary_points.as_tensor[:,1])
```

--------------------------------

### Define Boundary Condition Function

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/pinn/poisson-equation.ipynb

Defines the function `f(x)` that specifies the values on the boundary. This example uses a cosine function.

```python
def f(x):
    return torch.cos(x[:, :1])*torch.cos(x[:, 1:])
```

--------------------------------

### Sample and Visualize Product Domain

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/tutorial/domain_creation.ipynb

Samples points from the boundary of a circle and an interval, then visualizes their Cartesian product. This demonstrates sampling in a product space.

```python
product_sampler = tp.samplers.GridSampler(C.boundary, n_points=50) * tp.samplers.GridSampler(I, n_points=20)
tp.utils.scatter(X*T, product_sampler)
```

--------------------------------

### Set up PyTorch Lightning Trainer

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/pinn/moving-heat-equation.ipynb

Initializes the PyTorch Lightning Trainer with specific configurations for GPU usage, maximum steps, and disabling logging and checkpointing.

```python
import pytorch_lightning as pl
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "0"

trainer = pl.Trainer(devices=1, accelerator="gpu",
                     num_sanity_val_steps=0,
                     benchmark=True,
                     max_steps=5000, 
                     logger=False, 
                     enable_checkpointing=False)
trainer.fit(solver)
```

--------------------------------

### Create Domains

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/pinn/poisson-equation.ipynb

Constructs the computational domain for the problem. This example defines a circular domain and subtracts a smaller square region from its center.

```python
A = tp.domains.Circle(X, center=[0, 0], radius=1.0)
B = tp.domains.Parallelogram(X, [-0.25, -0.25], [-0.25, 0.25], [0.25, -0.25])
```

--------------------------------

### Fit Solver with Trainer

Source: https://github.com/boschresearch/torchphysics/blob/main/experiments/burgers_equation.ipynb

Initiates the training process by fitting the solver module to the setup using the configured PyTorch Lightning trainer.

```python
trainer.fit(solver, setup)
```

--------------------------------

### Setting up Solver and Trainer for DeepONet

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Sol3_2.ipynb

Configures the training process by defining the optimizer settings and creating a Solver instance with the data condition. A PyTorch Lightning Trainer is then set up with specified devices, accelerators, and training iterations.

```python
optim = tp.OptimizerSetting(optimizer_class=torch.optim.Adam, lr=learning_rate) 
solver = tp.solver.Solver([data_condition], optimizer_setting=optim)

trainer = pl.Trainer(devices=1, accelerator="gpu",
                     num_sanity_val_steps=0,
                     benchmark=True,
                     max_steps=train_iterations, 
                     logger=False, 
                     enable_checkpointing=False)
```

--------------------------------

### Initialize Time Grid, Initial Time, and Neural Network Model

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Exercise1_3.ipynb

Sets up the time discretization, a tensor for the initial time point, and a sequential neural network model with Tanh activation functions. Requires PyTorch.

```python
t = torch.linspace(t_min, t_max, N_t, requires_grad=True).reshape(N_t, 1)
t_zero = torch.tensor([0.0], requires_grad=True)

model = torch.nn.Sequential(
    torch.nn.Linear(1, 20), torch.nn.Tanh(), 
    torch.nn.Linear(20, 20), torch.nn.Tanh(), 
    torch.nn.Linear(20, 1)
)
```

--------------------------------

### Create Boundary Samplers

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/tutorial/sampler_tutorial.md

Shows how to create samplers that specifically generate points on the boundary of defined domains. This is useful for applying boundary conditions.

```python
random_R_bound = tp.samplers.RandomUniformSampler(R.boundary, n_points=50)
grid_C_bound = tp.samplers.GridSampler(C.boundary, density=10)
```

--------------------------------

### Create Initial Condition

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/operator_learning/physics_informed/pino_integrator_1D.ipynb

Sets up a physics-informed condition for the initial state. This condition is crucial for ensuring the solution starts correctly in time.

```python
initial_condition = tp.conditions.PIOperatorCondition(module=model, 
                                                      input_function_sampler=fn_sampler, 
                                                      residual_fn=initial_residual, 
                                                      weight=100.0)
```

--------------------------------

### Create TorchPhysics Solver

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/tutorial/Introduction_Tutorial_PINNs.ipynb

Instantiate the TorchPhysics Solver, passing the list of training conditions and the optimizer settings.

```python
solver = tp.solver.Solver(train_conditions=training_conditions, optimizer_setting=optim)
```

--------------------------------

### Model Evaluation and Plotting Output

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Sol1_2.ipynb

Example output from the model evaluation and plotting snippet, showing the calculated relative error and the value of 'D'.

```text
Relative error on the test data is: tensor(0.0040, grad_fn=<DivBackward0>)
Showing D value: 0.8410727977752686

```

--------------------------------

### Configuring Optimizer and Solver

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/pinn/hard-constrains.ipynb

Sets up the optimizer (Adam with a learning rate of 0.001) and the solver. The solver is initialized with all defined conditions (PDE, Dirichlet, Neumann) and the optimizer settings.

```python
optim = tp.OptimizerSetting(optimizer_class=torch.optim.Adam, lr=0.001)

solver = tp.solver.Solver([pde_condition, dirichlet_condition, neumann_condition], optimizer_setting=optim)
```

--------------------------------

### Set up optimizer and solver, then train the model

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/pinn/exp-function-with-param.ipynb

Configures the Adam optimizer with a learning rate and sets up the TorchPhysics solver with the defined conditions. The PyTorch Lightning trainer is then configured and used to fit the solver to the data.

```python
optim = tp.OptimizerSetting(optimizer_class=torch.optim.Adam, lr=0.001)

solver = tp.solver.Solver([pde_condition, boundary_condition], optimizer_setting=optim)

trainer = pl.Trainer(devices=1, accelerator="gpu",
                     num_sanity_val_steps=0,
                     benchmark=True,
                     max_steps=8000, 
                     logger=False, 
                     enable_checkpointing=False)
                     
trainer.fit(solver)
```

--------------------------------

### Plotting Function Example for Plotter

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/api/torchphysics.utils.plotting.md

Define a callable function to extract specific outputs or derivatives from the model for plotting. This function is passed to the Plotter.

```python
def plot_func(u):
    return u[:, 0]
```

--------------------------------

### Initialize and Make Grid Sampler Static

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/tutorial/solve_pde.ipynb

Initializes a GridSampler for boundary points and makes it static to ensure consistent sampling across computations.

```python
bound_sampler = tp.samplers.GridSampler(square.boundary, n_points=500)
bound_sampler = bound_sampler.make_static()
```

--------------------------------

### Plotting Derivative Example for Plotter

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/api/torchphysics.utils.plotting.md

Define a callable function to compute and return derivatives of the model's output with respect to input coordinates for plotting.

```python
def plot_func(u, x):
    return grad(u, x)
```

--------------------------------

### Define ODE Parameters and Training Settings

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Exercise1_3.ipynb

Sets up constants, time domain, and training hyperparameters for an ODE problem. Requires PyTorch.

```python
import torch 

# Here all parameters are defined:
t_min, t_max = 0.0, 3.0
D = 0.02
g, H = 9.81, 50.0

# number of time points 
N_t = 50

train_iterations = 5000
learning_rate = 1.e-3
```

--------------------------------

### Initialize Time Tensor, Initial Condition, and Neural Network

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Sol1_3.ipynb

Sets up the time grid, a tensor for the initial time point, and a sequential neural network model with Tanh activation functions. The model is then moved to the GPU.

```python
t = torch.linspace(t_min, t_max, N_t, requires_grad=True).reshape(N_t, 1)
t_zero = torch.tensor([0.0], requires_grad=True)

model = torch.nn.Sequential(
    torch.nn.Linear(1, 20), torch.nn.Tanh(), 
    torch.nn.Linear(20, 20), torch.nn.Tanh(), 
    torch.nn.Linear(20, 1)
) 

### Move everything to the GPU
model.to("cuda")
t = t.to("cuda")
t_zero = t_zero.to("cuda")
```

--------------------------------

### Create Samplers

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/deepritz/poisson-equation.ipynb

Initializes samplers for the interior and boundary of the defined domain. These samplers will generate points for the optimization process.

```python
inner_sampler = tp.samplers.RandomUniformSampler(D, density=1e4)
boundary_sampler = tp.samplers.RandomUniformSampler(D.boundary, density=1e4)
```

--------------------------------

### Create Random and Grid Samplers

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/tutorial/sampler_tutorial.md

Demonstrates creating random uniform and grid samplers for different domains. Ensure you have imported torchphysics and defined your spaces and domains.

```python
import torchphysics as tp
X = tp.spaces.R2('x') # the space of the object
R = tp.domains.Parallelogram(X, [0, 0], [1, 0], [0, 1]) # unit square
C = tp.domains.Circle(X, [0, 0], 1) # unit circle
random_R = tp.samplers.RandomUniformSampler(R, n_points=50) # 50 random points in R
grid_C = tp.samplers.GridSampler(C, density=10) # grid points with density 10 in C
```

--------------------------------

### Define Input Spaces and Domain

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/operator_learning/physics_informed/PI_DeepONet_integrator.ipynb

Sets up the input spaces for time (t), solution (u), and branch input (f), and defines the domain for time as an interval [0, 1]. It also creates a grid sampler for time points.

```python
import torchphysics as tp 
import torch
import pytorch_lightning as pl

T = tp.spaces.R1("t") # trunk input
U = tp.spaces.R1("u") # solution
F = tp.spaces.R1("f") # branch input

branch_space = tp.spaces.FunctionSpace(T, F)

# Given that DeepONet are models that both get a functional and variable
# input, we also need to define an interval from which we will sample time points.
domain = tp.domains.Interval(T, 0, 1)
# We will sample 10 points in the interval [0, 1] and use them as the discretization
# points for the branch input.
time_grid = tp.samplers.GridSampler(domain, 10).sample_points()
```

--------------------------------

### Add Dirichlet Boundary Condition to Training

Source: https://github.com/boschresearch/torchphysics/blob/main/experiments/burgers_equation.ipynb

Configures and adds a Dirichlet boundary condition to the training setup. `whole_batch=False` allows for point-wise defined boundary functions.

```python
x.add_train_condition(DirichletCondition(dirichlet_fun=x_dirichlet_fun,
                                         solution_name=u,
                                         whole_batch=False, # this enables us to use point-wise defined dirichlet_fun
                                         name='dirichlet',
                                         sampling_strategy='grid',
                                         boundary_sampling_strategy='grid',
                                         norm=norm,
                                         weight=1.0,
                                         dataset_size={'x': 200, 't': 10},
                                         data_plot_variables=('x','t')))
```

--------------------------------

### Configure Optimizer and Solver

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/pinn/poisson-with-input-params.ipynb

Sets up the optimizer using Adam with a learning rate of 0.0001 and initializes the TorchPhysics solver with the defined conditions and optimizer settings.

```python
optim = tp.OptimizerSetting(optimizer_class=torch.optim.Adam, lr=0.0001)

solver = tp.solver.Solver([dirich_cond, pde_cond_1, pde_cond_2], optimizer_setting=optim)
```

--------------------------------

### Define Spaces, Domains, and Samplers

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/workshop/Exercise3_1.ipynb

Sets up the computational space and domains for the problem. Defines a 1D time space (R1) and a 1D solution space (R1). Creates an interval domain for time and samplers for training and initial conditions.

```python
### Spaces, Domains and Sampler like yesterday:
T = tp.spaces.R1('t')
U = tp.spaces.R1('u')
 
int_t = tp.domains.Interval(T, t_min, t_max)

ode_sampler = tp.samplers.RandomUniformSampler(int_t, n_points=N_t)
initial_sampler = tp.samplers.RandomUniformSampler(int_t.boundary_left, n_points=N_initial)
```

--------------------------------

### Configure Git Hooks for Automatic Signing

Source: https://github.com/boschresearch/torchphysics/blob/main/CONTRIBUTING.md

Set up git hooks to automatically add the 'Signed-off-by' line to commit messages. This is a one-time setup in your working directory.

```bash
git config core.hooksPath .githooks
```

--------------------------------

### Sample and Visualize Domains and Boundaries

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/tutorial/domain_creation.ipynb

Creates grid samplers for a domain and its boundary, then visualizes the sampled points. Use this to inspect the geometry of defined domains.

```python
R_sampler = tp.samplers.GridSampler(R, n_points=100) + tp.samplers.GridSampler(R.boundary, n_points=100)
C_sampler = tp.samplers.GridSampler(C, n_points=100) + tp.samplers.GridSampler(C.boundary, n_points=100)

tp.utils.scatter(X, R_sampler)
tp.utils.scatter(X, C_sampler)
```

--------------------------------

### Validation Sanity Check Output

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/pinn/duffing_nonlinear.ipynb

Indicates the start of the validation sanity check phase in the training process. This output appears before the main validation loop begins.

```text
Result:
Validation sanity check: 0it [00:00, ?it/s]
```

--------------------------------

### Point Class

Source: https://github.com/boschresearch/torchphysics/blob/main/docs/api/torchphysics.problem.domains.md

Represents a 0-dimensional domain (a single point). Provides methods for checking if a point is within the domain, sampling points, and getting the bounding box.

```APIDOC
## Point Class

### Description
Represents a 0-dimensional domain, essentially a single point. It offers methods to determine if a given point lies within this domain, to sample points (both randomly and on a grid), and to retrieve the domain's bounding box.

### Methods

#### `__call__(self, x)`
Checks if a point `x` is within the domain.

#### `bounding_box(self)`
Returns the bounding box of the domain.

#### `sample_grid(self, n)`
Samples `n` points on a grid within the domain.

#### `sample_random_uniform(self, n)`
Samples `n` points uniformly at random within the domain.
```

--------------------------------

### Configure and Train Solver for Signorini Equation

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/pinn/signorini-equation.ipynb

Sets up static samplers for various conditions, defines the solver with these conditions, and initiates the training process using PyTorch Lightning's Trainer.

```python
pde_condition.sampler = pde_condition.sampler.make_static()
dirichlet_condition.sampler = dirichlet_condition.sampler.make_static()
obstacle_condition.sampler = obstacle_condition.sampler.make_static()
neumann_condition.sampler = neumann_condition.sampler.make_static()
complementary_condition.sampler = obstacle_condition.sampler
sign_condition.sampler = obstacle_condition.sampler

solver = tp.solver.Solver([pde_condition, dirichlet_condition, obstacle_condition,
                           neumann_condition, complementary_condition, sign_condition],
                          optimizer_setting=optim)

trainer = pl.Trainer(devices=1, accelerator="gpu",
                     num_sanity_val_steps=0,
                     benchmark=True,
                     max_steps=2500, 
                     logger=False, 
                     enable_checkpointing=False)
                     
trainer.fit(solver)
```

--------------------------------

### Configure GPU and PyTorch Lightning

Source: https://github.com/boschresearch/torchphysics/blob/main/examples/tutorial/Tutorial_PINNs_Parameter_Dependency.ipynb

Sets the CUDA visible devices for PyTorch and prints GPU availability. Ensure PyTorch is installed with CUDA support if a GPU is to be used.

```python
import pytorch_lightning as pl
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "1" if torch.cuda.is_available() else "0"
print ("GPU available: " + str(torch.cuda.is_available()))
```