### Set Device for GPU Acceleration

Source: https://github.com/iancovert/neural-gc/blob/master/clstm_lorenz_demo.ipynb

Configures PyTorch to use the GPU for accelerated computation if available. Ensure CUDA is properly installed and configured.

```python
# For GPU acceleration
device = torch.device('cuda')
```

--------------------------------

### Set up cMLP Model

Source: https://github.com/iancovert/neural-gc/blob/master/cmlp_lagged_var_demo.ipynb

Initializes a cMLP model with specified input dimensions, lag, and hidden layer sizes, and moves it to the GPU.

```python
# Set up model
cmlp = cMLP(X.shape[-1], lag=5, hidden=[100]).cuda(device=device)
```

--------------------------------

### Initialize cRNN Model

Source: https://github.com/iancovert/neural-gc/blob/master/crnn_lorenz_demo.ipynb

Sets up the cRNN model with the specified input dimension and hidden layer size, ensuring it is on the GPU.

```python
# Set up model
crnn = cRNN(X.shape[-1], hidden=100).cuda(device=device)

```

--------------------------------

### Initialize cLSTM Model

Source: https://github.com/iancovert/neural-gc/blob/master/clstm_lorenz_demo.ipynb

Sets up the cLSTM model with the specified input dimension and hidden layer size, moving it to the configured CUDA device for training.

```python
# Set up model
clstm = cLSTM(X.shape[-1], hidden=100).cuda(device=device)
```

--------------------------------

### Import Libraries

Source: https://github.com/iancovert/neural-gc/blob/master/cmlp_lagged_var_demo.ipynb

Imports necessary libraries for the cMLP model, data simulation, and plotting.

```python
import torch
import numpy as np
import matplotlib.pyplot as plt
from synthetic import simulate_var
from models.cmlp import cMLP, cMLPSparse, train_model_ista, train_unregularized
```

--------------------------------

### Create Debiased cMLP Model

Source: https://github.com/iancovert/neural-gc/blob/master/cmlp_lagged_var_demo.ipynb

Initializes a sparse cMLP model using a debiased approach. Requires the input data shape, sparsity mask, and desired lag.

```python
# Create a debiased model
sparsity = cmlp.GC().bool()
cmlp_sparse = cMLPSparse(X.shape[-1], sparsity, lag=5, hidden=[100]).cuda(device=device)
```

--------------------------------

### Calculate and Plot Forecasts

Source: https://github.com/iancovert/neural-gc/blob/master/cmlp_lagged_var_demo.ipynb

Calculates optimal forecasts using VAR parameters and then forecasts using a debiased cMLP. It then plots the actual data against both forecasted series for comparison. Ensure 'X', 'beta', and 'cmlp_sparse' are defined.

```python
# Get optimal forecasts using VAR parameters
X_optimal_forecast = np.zeros((X.shape[-1], 1000-3))
for t in range(1000-3):
    X_optimal_forecast[:, t] = np.dot(beta, X_np.T[:, t:(t+3)].flatten(order='F'))
X_optimal_forecast = X_optimal_forecast.T

# Forecast using debiased cMLP
X_pred = cmlp_sparse(X)

# Plot actual data and forecasts
num_points = 25

for i in range(X.shape[-1]):
    plt.figure(figsize=(10, 5))
    plt.plot(X[0, 3:num_points+3, i].cpu().data.numpy(), label='Actual')
    plt.plot(X_pred[0, :num_points, i].cpu().data.numpy(), label='cMLP forecasting')
    plt.plot(X_optimal_forecast[:num_points, i], label='Optimal forecasting')
    plt.legend(loc='upper right')
    plt.title('Series %d forecasting' % (i + 1))
    plt.show()
```

--------------------------------

### Train CMLP Sparse Model

Source: https://github.com/iancovert/neural-gc/blob/master/cmlp_lagged_var_demo.ipynb

Initiates the training of a CMLP sparse model. This snippet is used to observe the training progress and loss reduction over a specified number of iterations.

```python
train_loss_list = train_unregularized(cmlp_sparse, X, lr=1e-3, max_iter=10000,
                                      check_every=100, verbose=1)
```

--------------------------------

### Import Libraries for cLSTM Lorenz Demo

Source: https://github.com/iancovert/neural-gc/blob/master/clstm_lorenz_demo.ipynb

Imports necessary libraries including PyTorch, NumPy, Matplotlib, and custom modules for data simulation and the cLSTM model.

```python
import torch
import numpy as np
import matplotlib.pyplot as plt
from synthetic import simulate_lorenz_96
from models.clstm import cLSTM, train_model_ista
```

--------------------------------

### Import Libraries

Source: https://github.com/iancovert/neural-gc/blob/master/crnn_lorenz_demo.ipynb

Imports necessary libraries for PyTorch, NumPy, Matplotlib, and custom modules for data simulation and cRNN models.

```python
import torch
import numpy as np
import matplotlib.pyplot as plt
from synthetic import simulate_lorenz_96
from models.crnn import cRNN, train_model_ista
```

--------------------------------

### Set Device for GPU Acceleration

Source: https://github.com/iancovert/neural-gc/blob/master/crnn_lorenz_demo.ipynb

Configures PyTorch to use the GPU for accelerated computation if available.

```python
# For GPU acceleration
device = torch.device('cuda')

```

--------------------------------

### ISTA Training Progress Output

Source: https://github.com/iancovert/neural-gc/blob/master/crnn_lorenz_demo.ipynb

Displays the training progress of the ISTA algorithm, showing iteration number, loss, and variable usage percentage at specified intervals.

```text
----------Iter = 50----------
Loss = 23.054893
Variable usage = 100.00%
----------Iter = 100----------
Loss = 13.910990
Variable usage = 25.00%
----------Iter = 150----------
Loss = 12.809491
Variable usage = 24.00%
----------Iter = 200----------
Loss = 12.026926
Variable usage = 28.00%
----------Iter = 250----------
Loss = 11.401364
Variable usage = 28.00%
----------Iter = 300----------
Loss = 10.884741
Variable usage = 29.00%
----------Iter = 350----------
Loss = 10.453658
Variable usage = 29.00%
----------Iter = 400----------
Loss = 10.093219
Variable usage = 30.00%
----------Iter = 450----------
Loss = 9.789865
Variable usage = 32.00%
----------Iter = 500----------
Loss = 9.531705
Variable usage = 33.00%
----------Iter = 550----------
Loss = 9.309165
Variable usage = 33.00%
----------Iter = 600----------
Loss = 9.115885
Variable usage = 34.00%
----------Iter = 650----------
Loss = 8.946098
Variable usage = 34.00%
----------Iter = 700----------
Loss = 8.795321
Variable usage = 36.00%
----------Iter = 750----------
Loss = 8.660369
Variable usage = 37.00%
----------Iter = 800----------
Loss = 8.538839
Variable usage = 37.00%
----------Iter = 850----------
Loss = 8.428749
Variable usage = 37.00%
----------Iter = 900----------
Loss = 8.328047
Variable usage = 38.00%
----------Iter = 950----------
Loss = 8.235439
Variable usage = 38.00%
----------Iter = 1000----------
Loss = 8.149655
Variable usage = 39.00%
----------Iter = 1050----------
Loss = 8.069839
Variable usage = 40.00%
----------Iter = 1100----------
Loss = 7.995345
Variable usage = 41.00%
----------Iter = 1150----------
Loss = 7.925638
Variable usage = 41.00%
----------Iter = 1200----------
Loss = 7.878078
Variable usage = 43.00%
----------Iter = 1250----------
Loss = 7.800751
Variable usage = 41.00%
----------Iter = 1300----------
Loss = 7.742177
Variable usage = 42.00%
----------Iter = 1350----------
Loss = 7.687673
Variable usage = 43.00%
----------Iter = 1400----------
Loss = 7.648279
Variable usage = 43.00%
----------Iter = 1450----------
Loss = 7.590488
Variable usage = 42.00%
----------Iter = 1500----------
Loss = 7.542428
Variable usage = 42.00%
----------Iter = 1550----------
Loss = 7.506766
Variable usage = 43.00%
----------Iter = 1600----------
Loss = 7.460866
Variable usage = 42.00%
----------Iter = 1650----------
Loss = 7.420418
Variable usage = 42.00%
----------Iter = 1700----------
Loss = 7.388299
Variable usage = 43.00%
----------Iter = 1750----------
Loss = 7.345255
Variable usage = 42.00%
----------Iter = 1800----------
Loss = 7.315914
Variable usage = 42.00%
----------Iter = 1850----------
Loss = 7.277280
Variable usage = 42.00%
----------Iter = 1900----------
Loss = 8.088763
Variable usage = 51.00%
----------Iter = 1950----------
Loss = 7.226752
Variable usage = 40.00%
----------Iter = 2000----------
Loss = 7.211196
Variable usage = 45.00%
----------Iter = 2050----------
Loss = 7.343300
Variable usage = 45.00%
----------Iter = 2100----------
Loss = 7.154907
Variable usage = 42.00%
----------Iter = 2150----------
Loss = 7.115057
Variable usage = 42.00%
----------Iter = 2200----------
Loss = 7.098578
Variable usage = 42.00%
----------Iter = 2250----------
Loss = 7.128511
Variable usage = 46.00%
----------Iter = 2300----------
Loss = 7.099002
Variable usage = 44.00%
----------Iter = 2350----------
Loss = 7.022751
Variable usage = 39.00%
----------Iter = 2400----------
Loss = 7.035083
Variable usage = 44.00%
----------Iter = 2450----------
Loss = 6.970329
Variable usage = 41.00%
----------Iter = 2500----------
Loss = 6.950571
Variable usage = 42.00%
----------Iter = 2550----------
Loss = 7.803427
Variable usage = 48.00%
----------Iter = 2600----------
Loss = 6.944563
Variable usage = 42.00%
----------Iter = 2650----------
Loss = 6.885013
Variable usage = 42.00%
----------Iter = 2700----------
Loss = 6.970696
Variable usage = 46.00%
----------Iter = 2750----------
Loss = 6.856977
Variable usage = 43.00%
----------Iter = 2800----------
Loss = 6.840479
Variable usage = 41.00%
----------Iter = 2850----------
Loss = 6.824272
Variable usage = 40.00%
----------Iter = 2900----------
Loss = 6.966159
Variable usage = 49.00%
----------Iter = 2950----------
Loss = 6.911489
Variable usage = 48.00%
----------Iter = 3000----------
Loss = 6.901711
Variable usage = 45.00%
----------Iter = 3050----------
Loss = 6.771117
Variable usage = 40.00%
----------Iter = 3100----------
Loss = 6.746310
Variable usage = 43.00%
----------Iter = 3150----------
Loss = 6.748783
Variable usage = 42.00%
----------Iter = 3200----------
Loss = 6.773919
Variable usage = 43.00%
----------Iter = 3250----------
Loss = 8.153480
Variable usage = 46.00%
----------Iter = 3300----------
Loss = 6.689789
Variable usage = 43.00%
----------Iter = 3350----------
Loss = 6.768144
Variable usage = 42.00%

```

--------------------------------

### Train CMLP Model with ISTA

Source: https://github.com/iancovert/neural-gc/blob/master/cmlp_lagged_var_demo.ipynb

Initiates the training of a CMLP model using the ISTA algorithm. Specify parameters like lambda, lambda_ridge, learning rate, penalty type, maximum iterations, and the check frequency.

```python
train_loss_list = train_model_ista(
    cmlp, X, lam=0.002, lam_ridge=1e-2, lr=5e-2, penalty='H', max_iter=50000,
    check_every=100
)
```

--------------------------------

### Simulate VAR Data

Source: https://github.com/iancovert/neural-gc/blob/master/cmlp_lagged_var_demo.ipynb

Generates synthetic VAR data using the simulate_var function and converts it to a PyTorch tensor on the specified device.

```python
# Simulate data
X_np, beta, GC = simulate_var(p=10, T=1000, lag=3)
X = torch.tensor(X_np[np.newaxis], dtype=torch.float32, device=device)
```

--------------------------------

### Train CRNN with ISTA

Source: https://github.com/iancovert/neural-gc/blob/master/crnn_lorenz_demo.ipynb

Initiates the training of a CRNN model using the ISTA algorithm. Requires the model, input data, and several hyperparameters for training.

```python
train_loss_list = train_model_ista(
    crnn, X, context=10, lam=10.0, lam_ridge=1e-2, lr=1e-3, max_iter=20000,
    check_every=50)

```

--------------------------------

### Verify Lag Selection

Source: https://github.com/iancovert/neural-gc/blob/master/cmlp_lagged_var_demo.ipynb

Analyzes and visualizes the lag selection for Granger causality for each series. It compares the true GC lag with the estimated GC lag.

```python
# Verify lag selection
for i in range(len(GC_est)):
    # Get true GC
    GC_lag = np.zeros((5, len(GC_est)))
    GC_lag[:3, GC[i].astype(bool)] = 1.0

    # Get estimated GC
    GC_est_lag = cmlp.GC(ignore_lag=False, threshold=False)[i].cpu().data.numpy().T[::-1]

    # Make figures
    fig, axarr = plt.subplots(1, 2, figsize=(16, 5))
    axarr[0].imshow(GC_lag, cmap='Blues', extent=(0, len(GC_est), 5, 0))
    axarr[0].set_title('Series %d true GC' % (i + 1))
    axarr[0].set_ylabel('Lag')
    axarr[0].set_xlabel('Series')
    axarr[0].set_xticks(np.arange(len(GC_est)) + 0.5)
    axarr[0].set_xticklabels(range(len(GC_est)))
    axarr[0].set_yticks(np.arange(5) + 0.5)
    axarr[0].set_yticklabels(range(1, 5 + 1))
    axarr[0].tick_params(axis='both', length=0)

    axarr[1].imshow(GC_est_lag, cmap='Blues', extent=(0, len(GC_est), 5, 0))
    axarr[1].set_title('Series %d estimated GC' % (i + 1))
    axarr[1].set_ylabel('Lag')
    axarr[1].set_xlabel('Series')
    axarr[1].set_xticks(np.arange(len(GC_est)) + 0.5)
    axarr[1].set_xticklabels(range(len(GC_est)))
    axarr[1].set_yticks(np.arange(5) + 0.5)
    axarr[1].set_yticklabels(range(1, 5 + 1))
    axarr[1].tick_params(axis='both', length=0)

    # Mark nonzeros
    for i in range(len(GC_est)):
        for j in range(5):
            if GC_est_lag[j, i] > 0.0:
                rect = plt.Rectangle((i, j), 1, 1, facecolor='none', edgecolor='green', linewidth=1.0)
                axarr[1].add_patch(rect)

    plt.show()
```

--------------------------------

### Verify and Visualize Learned Granger Causality

Source: https://github.com/iancovert/neural-gc/blob/master/crnn_lorenz_demo.ipynb

Compares the estimated Granger causality (GC) with the actual GC and visualizes the results. This snippet requires the CRNN model to be loaded and the true GC matrix to be available.

```python
# Verify learned Granger causality
GC_est = crnn.GC().cpu().data.numpy()

print('True variable usage = %.2f%%' % (100 * np.mean(GC)))
print('Estimated variable usage = %.2f%%' % (100 * np.mean(GC_est)))
print('Accuracy = %.2f%%' % (100 * np.mean(GC == GC_est)))

# Make figures
fig, axarr = plt.subplots(1, 2, figsize=(10, 5))
axarr[0].imshow(GC, cmap='Blues')
axarr[0].set_title('GC actual')
axarr[0].set_ylabel('Affected series')
axarr[0].set_xlabel('Causal series')
axarr[0].set_xticks([])
axarr[0].set_yticks([])

axarr[1].imshow(GC_est, cmap='Blues', vmin=0, vmax=1, extent=(0, len(GC_est), len(GC_est), 0))
axarr[1].set_ylabel('Affected series')
axarr[1].set_xlabel('Causal series')
axarr[1].set_xticks([])
axarr[1].set_yticks([])

# Mark disagreements
for i in range(len(GC_est)):
    for j in range(len(GC_est)):
        if GC[i, j] != GC_est[i, j]:
            rect = plt.Rectangle((j, i-0.05), 1, 1, facecolor='none', edgecolor='red', linewidth=1)
            axarr[1].add_patch(rect)

plt.show()
```

--------------------------------

### Train CLSTM Model with ISTA

Source: https://github.com/iancovert/neural-gc/blob/master/clstm_lorenz_demo.ipynb

Initiates the training of a CLSTM model using the ISTA algorithm. Requires the model, input data, and training parameters such as context window size, regularization strengths (lam, lam_ridge), learning rate, and maximum iterations. The function returns a list of training losses.

```python
train_loss_list = train_model_ista(
    clstm, X, context=10, lam=10.0, lam_ridge=1e-2, lr=1e-3, max_iter=20000,
    check_every=50
)
```

--------------------------------

### Verify Learned Granger Causality

Source: https://github.com/iancovert/neural-gc/blob/master/cmlp_lagged_var_demo.ipynb

Compares the estimated Granger causality (GC) with the true GC and visualizes the results. It also highlights disagreements between the true and estimated GC.

```python
# Verify learned Granger causality
GC_est = cmlp.GC().cpu().data.numpy()

print('True variable usage = %.2f%%' % (100 * np.mean(GC)))
print('Estimated variable usage = %.2f%%' % (100 * np.mean(GC_est)))
print('Accuracy = %.2f%%' % (100 * np.mean(GC == GC_est)))

# Make figures
fig, axarr = plt.subplots(1, 2, figsize=(16, 5))
axarr[0].imshow(GC, cmap='Blues')
axarr[0].set_title('GC actual')
axarr[0].set_ylabel('Affected series')
axarr[0].set_xlabel('Causal series')
axarr[0].set_xticks([])
axarr[0].set_yticks([])

axarr[1].imshow(GC_est, cmap='Blues', vmin=0, vmax=1, extent=(0, len(GC_est), len(GC_est), 0))
axarr[1].set_title('GC estimated')
axarr[1].set_ylabel('Affected series')
axarr[1].set_xlabel('Causal series')
axarr[1].set_xticks([])
axarr[1].set_yticks([])

# Mark disagreements
for i in range(len(GC_est)):
    for j in range(len(GC_est)):
        if GC[i, j] != GC_est[i, j]:
            rect = plt.Rectangle((j, i-0.05), 1, 1, facecolor='none', edgecolor='red', linewidth=1)
            axarr[1].add_patch(rect)

plt.show()
```

--------------------------------

### Simulate Lorenz-96 Data

Source: https://github.com/iancovert/neural-gc/blob/master/clstm_lorenz_demo.ipynb

Generates time series data from the Lorenz-96 system and converts it into a PyTorch tensor on the specified device. The simulation parameters (p, F, T) can be adjusted.

```python
# Simulate data
X_np, GC = simulate_lorenz_96(p=10, F=10, T=1000)
X = torch.tensor(X_np[np.newaxis], dtype=torch.float32, device=device)
```

--------------------------------

### Evaluate and Visualize Granger Causality

Source: https://github.com/iancovert/neural-gc/blob/master/clstm_lorenz_demo.ipynb

Checks the learned Granger Causality (GC) by comparing estimated vs. actual values and visualizes the results. Requires `clstm.GC()` to be available and `GC` to be defined.

```python
# Check learned Granger causality
GC_est = clstm.GC().cpu().data.numpy()

print('True variable usage = %.2f%%' % (100 * np.mean(GC)))
print('Estimated variable usage = %.2f%%' % (100 * np.mean(GC_est)))
print('Accuracy = %.2f%%' % (100 * np.mean(GC == GC_est)))

# Make figures
fig, axarr = plt.subplots(1, 2, figsize=(10, 5))
axarr[0].imshow(GC, cmap='Blues')
axarr[0].set_title('GC actual')
axarr[0].set_ylabel('Affected series')
axarr[0].set_xlabel('Causal series')
axarr[0].set_xticks([])
axarr[0].set_yticks([])

axarr[1].imshow(GC_est, cmap='Blues', vmin=0, vmax=1, extent=(0, len(GC_est), len(GC_est), 0))
axarr[1].set_ylabel('Affected series')
axarr[1].set_xlabel('Causal series')
axarr[1].set_xticks([])
axarr[1].set_yticks([])

# Mark disagreements
for i in range(len(GC_est)):
    for j in range(len(GC_est)):
        if GC[i, j] != GC_est[i, j]:
            rect = plt.Rectangle((j, i-0.05), 1, 1, facecolor='none', edgecolor='red', linewidth=1)
            axarr[1].add_patch(rect)

plt.show()
```

--------------------------------

### Plot Training Loss

Source: https://github.com/iancovert/neural-gc/blob/master/cmlp_lagged_var_demo.ipynb

Plots the training loss over time. Ensure 'train_loss_list' is populated.

```python
# Plot loss function
plt.figure(figsize=(10, 5))
plt.title('Debiased model training')
plt.ylabel('Loss')
plt.xlabel('Training steps')
plt.plot(100 * np.arange(len(train_loss_list)), train_loss_list)
plt.show()
```

--------------------------------

### Plot Training Loss

Source: https://github.com/iancovert/neural-gc/blob/master/cmlp_lagged_var_demo.ipynb

Plots the training loss over time. Ensure 'train_loss_list' is populated.

```python
plt.figure(figsize=(8, 5))
plt.plot(50 * np.arange(len(train_loss_list)), train_loss_list)
plt.title('cMLP training')
plt.ylabel('Loss')
plt.xlabel('Training steps')
plt.tight_layout()
plt.show()
```

--------------------------------

### Plot Lorenz-96 Time Series Data

Source: https://github.com/iancovert/neural-gc/blob/master/clstm_lorenz_demo.ipynb

Visualizes the simulated Lorenz-96 time series data, showing both the entire series and the initial segment. This helps in understanding the data's characteristics.

```python
# Plot data
fig, axarr = plt.subplots(1, 2, figsize=(16, 5))
axarr[0].plot(X_np)
axarr[0].set_xlabel('T')
axarr[0].set_title('Entire time series')
axarr[1].plot(X_np[:50])
axarr[1].set_xlabel('T')
axarr[1].set_title('First 50 time points')
plt.tight_layout()
plt.show()
```

--------------------------------

### Plot CRNN Training Loss

Source: https://github.com/iancovert/neural-gc/blob/master/crnn_lorenz_demo.ipynb

Plots the training loss of the CRNN model over training steps. Ensure 'train_loss_list' is populated and matplotlib is imported as 'plt'.

```python
# Loss function plot
plt.figure(figsize=(8, 5))
plt.plot(50 * np.arange(len(train_loss_list)), train_loss_list)
plt.title('cRNN training')
plt.ylabel('Loss')
plt.xlabel('Training steps')
plt.tight_layout()
plt.show()
```

--------------------------------

### Plot CLSTM Training Loss

Source: https://github.com/iancovert/neural-gc/blob/master/clstm_lorenz_demo.ipynb

Plots the training loss over time for the CLSTM model. Ensure `train_loss_list` is populated.

```python
plt.figure(figsize=(8, 5))
plt.plot(50 * np.arange(len(train_loss_list)), train_loss_list)
plt.title('cLSTM training')
plt.ylabel('Loss')
plt.xlabel('Training steps')
plt.tight_layout()
plt.show()
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.