### Install fvgp

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/NonEuclideanInputSpaces.ipynb

Installs the newest version of the fvgp library. This is a prerequisite for running the examples.

```python
#install the newest version of fvgp
#!pip install fvgp~=4.7.2
```

--------------------------------

### Install fvgp and imate

Source: https://github.com/lbl-camera/fvgp/blob/master/examples/gp2ScaleTest.ipynb

Install the latest version of fvgp and imate. This is a prerequisite for using gp2Scale.

```python
##first install the newest version of fvgp
#!pip install fvgp~=4.7.9
#!pip install imate
```

--------------------------------

### Install fvgp Library

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/SingleTaskTest.ipynb

Installs the fvgp library with a specific version. Ensure this is run before importing.

```python
#!pip install fvgp~=4.7.3
```

--------------------------------

### Set Up Local Development Environment

Source: https://github.com/lbl-camera/fvgp/blob/master/CONTRIBUTING.rst

Install the project locally into a virtual environment using setup.py develop.

```shell
mkvirtualenv fvgp
cd fvgp/
python setup.py develop
```

--------------------------------

### Install fvgp and imate

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/gp2ScaleTest.ipynb

Installs the necessary libraries for fvgp and related tools. Ensure you have the latest compatible version of fvgp.

```python
##first install the newest version of fvgp
#!pip install fvgp~=4.7.3
#!pip install imate
```

--------------------------------

### Install fvGP and Plotly

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/MultiTaskTest.ipynb

Installs the specified version of the fvGP library and Plotly for visualization. Ensure you have the correct version of fvGP installed.

```python
##First, install the newest version of fvgp
#!pip install fvgp~=4.7.3
#!pip install plotly
```

--------------------------------

### Install fvGP and Plotly

Source: https://github.com/lbl-camera/fvgp/blob/master/examples/MultiTaskTest.ipynb

Installs the necessary libraries for fvGP and plotting. Ensure you have the correct version of fvGP.

```python
##First, install the newest version of fvgp
#!pip install fvgp~=4.7.9
#!pip install plotly
```

--------------------------------

### Install FVGP Library

Source: https://github.com/lbl-camera/fvgp/blob/master/examples/SingleTaskTest.ipynb

Installs a specific version of the fvgp library. Use this command in your environment to ensure compatibility.

```python
#install the right fvgp version
#!pip install fvgp~=4.7.9
```

--------------------------------

### Import Libraries and Setup

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/MultiTaskTest.ipynb

Imports necessary libraries including NumPy, Matplotlib, fvGP, and Plotly. It also sets up Plotly renderers and enables autoreload for development.

```python
import numpy as np
import matplotlib.pyplot as plt
from fvgp import GP
import plotly.graph_objects as go
from itertools import product
import plotly.io as pio
pio.renderers.default = "png"
%load_ext autoreload
%autoreload 2
```

--------------------------------

### Initiate Asynchronous Training

Source: https://github.com/lbl-camera/fvgp/blob/master/examples/SingleTaskTest.ipynb

Start an asynchronous training job using a Dask client. Training will run in the background, and progress can be monitored by updating hyperparameters.

```python
print(my_gp1.hyperparameters)
opt_obj = my_gp1.train(hyperparameter_bounds=hps_bounds, dask_client=client, asynchronous=True, method='hgdl')
```

--------------------------------

### Start and Monitor Asynchronous Training

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/SingleTaskTest.ipynb

Initiates asynchronous training with specified hyperparameter bounds and a Dask client. The training runs in the background, and hyperparameters can be updated periodically.

```python
print(my_gp1.hyperparameters)
opt_obj = my_gp1.train_async(hyperparameter_bounds=hps_bounds, dask_client=client)
```

```python
# The result won't change much (or at all) since this is such a simple optimization
for i in range(20):
    time.sleep(0.1)
    print("iteration ", i)
    my_gp1.update_hyperparameters(opt_obj)
    print(my_gp1.hyperparameters)
```

--------------------------------

### Default Multi-Task GP Training

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/MultiTaskTest.ipynb

Initialize and train a multi-task Gaussian process model using default settings. This is a minimal setup for basic functionality.

```python
from fvgp import fvGP

my_gp2 = fvGP(x_data3,y_data3)
print("Global Training in progress")
my_gp2.train(max_iter = 2)
```

--------------------------------

### Set Hyperparameters for Asynchronous Training

Source: https://github.com/lbl-camera/fvgp/blob/master/examples/SingleTaskTest.ipynb

Set the initial hyperparameters before starting an asynchronous training process. This is a prerequisite for initiating training.

```python
my_gp1.set_hyperparameters(np.array([1,1]))
```

--------------------------------

### GP Model Warnings

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/MultiTaskTest.ipynb

Displays warnings generated during the GP model setup, related to hyperparameter initialization and noise variance settings.

```text
/home/marcus/Coding/fvGP/fvgp/gp.py:265: UserWarning:

Hyperparameters initialized to a vector of ones.

/home/marcus/Coding/fvGP/fvgp/gp.py:299: UserWarning:

No noise function or measurement noise provided. Noise variances will be set to (0.01 * mean(|y_data|))^2.

/home/marcus/Coding/fvGP/fvgp/gp.py:581: UserWarning:

Default hyperparameter_bounds initialized because none were provided. This will fail for custom kernel, mean, or noise functions
```

--------------------------------

### Set Hyperparameters for Asynchronous Training

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/SingleTaskTest.ipynb

Sets the initial hyperparameters for the Gaussian Process model before starting asynchronous training. Ensure hyperparameters are within valid bounds.

```python
my_gp1.set_hyperparameters(np.array([1,1,1,1]))
```

--------------------------------

### Initialize and Train Multi-Task GP Model

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/MultiTaskTest.ipynb

Initializes a Gaussian Process model with the custom deep multi-task kernel and trains it using MCMC. This requires pre-defined data (`x_data3`, `y_data3`) and the custom kernel function. Hyperparameter bounds are set to guide the training process.

```python
my_gp2 = fvGP(x_data3,y_data3,
              init_hyperparameters=np.ones((n.number_of_hps+2))*0.1, kernel_function=deep_multi_task_kernel
             )
print("Global Training in progress")


bounds = np.zeros((n.number_of_hps+2,2))
bounds[0] = np.array([0.01,1.])
bounds[1] = np.array([0.1,1.])
bounds[2:] = np.array([-1,1])
my_gp2.train(hyperparameter_bounds=bounds,max_iter = 1000, method = "mcmc")
```

--------------------------------

### Initialize Single-Task GP with Custom Kernel and Noise

Source: https://context7.com/lbl-camera/fvgp/llms.txt

Demonstrates initializing a `GP` object with custom kernel and noise functions. Ensure custom functions correctly handle input shapes and hyperparameters.

```python
import numpy as np
from fvgp import GP
from fvgp.kernels import get_distance_matrix, matern_kernel_diff1

# Generate noisy training data from a known function
np.random.seed(42)
x_data = np.random.rand(200).reshape(-1, 1)           # 200 points in [0,1]
y_data = np.sin(5 * x_data[:, 0]) + np.random.randn(200) * 0.05

# Define a custom stationary kernel: amplitude * Matern(diff=1)
def my_kernel(x1, x2, hps):
    d = get_distance_matrix(x1, x2)
    return hps[0] * matern_kernel_diff1(d, hps[1])

# Define a custom noise function
def my_noise(x, hps):
    return np.full(len(x), hps[2])

# Initialize GP with custom kernel and noise
gp = GP(
    x_data, y_data,
    init_hyperparameters=np.array([1.0, 0.1, 0.01]),
    kernel_function=my_kernel,
    noise_function=my_noise,
    compute_device="cpu",
    calc_inv=False,
    ram_economy=False,
)
print("Initialized GP. x_data shape:", gp.x_data.shape)
# Initialized GP. x_data shape: (200, 1)
```

--------------------------------

### Initialize Dask Client and GP Environment

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/gp2ScaleTest.ipynb

Sets up a Dask distributed client for parallel computation and configures the plotting environment. It's recommended to wait for workers to be ready before proceeding.

```python
import numpy as np
import matplotlib.pyplot as plt
from fvgp import GP
from dask.distributed import Client
import sys
%load_ext autoreload
%autoreload 2

#further control plotting 
from loguru import logger
logger.disable("fvgp")


client = Client() ##this is the client you can make locally like this or 
#your HPC team can provide a script to get it. We included an example to get gp2Scale going
#on Perlmutter


#It's good practice to make sure to wait for all the workers to be ready
client.wait_for_workers(4)
```

--------------------------------

### Get Shape of Training Data

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/SingleTaskTest.ipynb

Retrieves and prints the shape of the input training data (x_data) used by the Gaussian Process model.

```python
my_gp1.x_data.shape
```

--------------------------------

### Deploy Project

Source: https://github.com/lbl-camera/fvgp/blob/master/CONTRIBUTING.rst

Deploy the project by committing changes, updating history, and using bump2version. Travis CI will handle PyPI deployment upon successful tests.

```shell
bump2version patch # possible: major / minor / patch
git push
git push --tags
```

--------------------------------

### Initialize and Train Gaussian Process Model

Source: https://github.com/lbl-camera/fvgp/blob/master/examples/SingleTaskTest.ipynb

Initializes a GP model with the generated data and custom settings. It then demonstrates training the model using various methods: Standard (MCMC), ADAM, Global, Local, and HGDL, printing the resulting hyperparameters and training times.

```python
st = time.time()
from loguru import logger
logger.disable("fvgp")
my_gp1 = GP(x_data,y_data,
            init_hyperparameters = np.ones((2))*10.,  # we need enough of those for kernel, noise, and prior mean functions
            noise_variances=np.ones(y_data.shape) * 0.1, # providing noise variances and a noise function will raise a warning 
            compute_device='cpu', 
            #kernel_function=skernel,
            kernel_function_grad=None, 
            #prior_mean_function=meanf,
            prior_mean_function_grad=None,
            #noise_function=my_noise,
            gp2Scale = False,
            calc_inv=False, 
            ram_economy=True,
            )


hps_bounds = np.array([[0.01,100.], #signal variance for the kernel
                       [0.01,100.], #length scale for the kernel
                       #[0.001,0.1],  #noise
                       #[0.01,1.]  #mean
                      ])
#my_gp1.update_gp_data(x_data, y_data, noise_variances_new=np.ones(y_data.shape) * 0.05) #this is just for testing, not needed
print("Standard Training (MCMC)")
hps = my_gp1.train(hyperparameter_bounds=hps_bounds, info = False)
print("Result=", hps, "after ", time.time() - st, " seconds")
print("")

print("ADAM")
hps = my_gp1.train(hyperparameter_bounds=hps_bounds, info = True, max_iter = 100, method="adam")
print("Result=", hps, "after ", time.time() - st, " seconds")
print("")

print("Global Training")
hps = my_gp1.train(hyperparameter_bounds=hps_bounds, method='global', max_iter = 20)
print("Result=", hps, "after ", time.time() - st, " seconds")
print(my_gp1.log_likelihood())
print("")

print("Local Training")
hps = my_gp1.train(hyperparameter_bounds=hps_bounds, method='local')
print("Result=", hps, "after ", time.time() - st, " seconds")
print("")

print("HGDL Training")
hps = my_gp1.train(hyperparameter_bounds=hps_bounds, method='hgdl', max_iter=2, dask_client=client)
print("Result=", hps, "after ", time.time() - st, " seconds")
print("")
```

--------------------------------

### Initialize and Train Gaussian Process

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/SingleTaskTest.ipynb

Initializes a GP model with custom kernel, noise, and mean functions. Demonstrates various training methods: Standard (MCMC), Global, Local, and HGDL (with Dask client). Hyperparameter bounds are specified for optimization.

```python
st = time.time()
from loguru import logger
logger.disable("fvgp")
my_gp1 = GP(x_data,y_data,
            init_hyperparameters = np.ones((4))/10.,  # we need enough of those for kernel, noise, and prior mean functions
            noise_variances=np.ones(y_data.shape) * 0.1, # providing noise variances and a noise function will raise a warning 
            compute_device='cpu', 
            kernel_function=skernel, 
            kernel_function_grad=None, 
            #prior_mean_function=meanf,
            prior_mean_function_grad=None,
            #noise_function=my_noise,
            gp2Scale = False,
            calc_inv=False, 
            ram_economy=False,
            )


hps_bounds = np.array([[0.01,10.], #signal variance for the kernel
                       [0.01,10.], #length scale for the kernel
                       [0.001,0.1],  #noise
                       [0.01,1.]  #mean
                      ])
#my_gp1.update_gp_data(x_data, y_data, noise_variances_new=np.ones(y_data.shape) * 0.05) #this is just for testing, not needed
print("Standard Training (MCMC)")
my_gp1.train(hyperparameter_bounds=hps_bounds, info = True, max_iter = 100)
print(time.time() - st)
print("Global Training")
my_gp1.train(hyperparameter_bounds=hps_bounds, method='global', max_iter = 20)
print("hps: ", my_gp1.hyperparameters)
print("Local Training")
my_gp1.train(hyperparameter_bounds=hps_bounds, method='local')
print(my_gp1.hyperparameters)
print("HGDL Training")
my_gp1.train(hyperparameter_bounds=hps_bounds, method='hgdl', max_iter=2, dask_client=client)
print(my_gp1.hyperparameters)
```

--------------------------------

### Prepare for Prediction

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/SingleTaskTest.ipynb

Sets up an array of x-values for making predictions with the trained model. This typically involves defining a range or specific points of interest.

```python
#let's make a prediction
x_pred = np.linspace(0,1,1000)
```

--------------------------------

### Initialize and Train GP with gp2Scale

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/gp2ScaleTest.ipynb

Initializes a Gaussian Process model using the wendland_anisotropic_gp2Scale_cpu kernel and enables gp2Scale for efficient computation. The model is then trained with specified hyperparameter bounds.

```python
from fvgp.kernels import wendland_anisotropic_gp2Scale_cpu
def kernel(x1,x2,hps):
    return wendland_anisotropic_gp2Scale_cpu(x1,x2,hps)


init_hps = np.array([0.73118673, 0.13813191])
my_gp2S = GP(x_data,y_data, kernel_function=kernel, 
             init_hyperparameters = init_hps, #compute_device = 'gpu', #you can use gpus here
             gp2Scale = True, gp2Scale_batch_size= 1000, 
             gp2Scale_dask_client = client, gp2Scale_linalg_mode = "Chol")

my_gp2S.train(hyperparameter_bounds = hps_bounds, max_iter = 5)
```

--------------------------------

### Predict with Multi-Task GP

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/NonEuclideanInputSpaces.ipynb

Performs predictions using the trained multi-task GP model for given input strings. It shows how to get predictions with and without specifying output indices.

```python
x_pred = ["dwed","dwe"]
my_gp2.posterior_mean(x_pred, x_out = np.array([0,1,2,3]))
my_gp2.posterior_mean(x_pred)
```

--------------------------------

### Optimize GP Hyperparameters using Different Methods

Source: https://context7.com/lbl-camera/fvgp/llms.txt

Shows how to train a GP by optimizing hyperparameters using 'mcmc', 'global', and 'local' methods. Hyperparameter bounds are crucial for successful optimization.

```python
import numpy as np
from fvgp import GP
from fvgp.kernels import get_distance_matrix, matern_kernel_diff1

np.random.seed(0)
x_data = np.random.rand(150).reshape(-1, 1)
y_data = np.sin(10 * x_data[:, 0]) + np.random.randn(150) * 0.1

def my_kernel(x1, x2, hps):
    d = get_distance_matrix(x1, x2)
    return hps[0] * matern_kernel_diff1(d, hps[1])

gp = GP(x_data, y_data,
        init_hyperparameters=np.ones(3),
        kernel_function=my_kernel)

hps_bounds = np.array([
    [0.01, 10.0],   # signal variance
    [0.01, 5.0],    # length scale
    [1e-4, 1.0],    # noise
])

# MCMC training (default)
hps = gp.train(hyperparameter_bounds=hps_bounds, method="mcmc",
               max_iter=200, info=True)
print("MCMC hyperparameters:", hps)

# Global optimizer (differential evolution)
hps = gp.train(hyperparameter_bounds=hps_bounds, method="global", max_iter=50)
print("Global hyperparameters:", hps)

# Local optimizer (L-BFGS-B) – fast refinement
hps = gp.train(hyperparameter_bounds=hps_bounds, method="local")
print("Local hyperparameters:", gp.hyperparameters)

# Asynchronous HGDL training (requires dask client)
# from distributed import Client
# client = Client()
# opt_obj = gp.train(hyperparameter_bounds=hps_bounds, method="hgdl",
#                    dask_client=client, asynchronous=True)
# ... do other work ...
# gp.update_hyperparameters(opt_obj)
# gp.stop_training(opt_obj)
```

--------------------------------

### Define Custom Noise Function

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/SingleTaskTest.ipynb

Defines a custom noise function for the Gaussian Process. This example implements a simple constant noise, but can be extended to use hyperparameters for more complex noise models.

```python
from fvgp.kernels import *
from scipy import sparse
def my_noise(x,hps):
    #This is a simple noise function, but can be arbitrarily complex using many hyperparameters.
    #The noise can be a vector, a matrix, or a sparse matrix in case gp2Scale is used.  
    return np.zeros(len(x)) + hps[2]
```

--------------------------------

### Distributed Large-Scale GPs with gp2Scale

Source: https://context7.com/lbl-camera/fvgp/llms.txt

Enables distributed sparse GP computation for millions of data points using Dask workers. Requires `imate` to be installed. Uses a compactly-supported anisotropic Wendland kernel by default.

```python
import numpy as np
from fvgp import GP
from distributed import Client

# Start a local Dask cluster (on HPC, provide a pre-configured client)
client = Client()

# Large-scale dataset
N = 50000
x_data = np.random.rand(N, 2)
y_data = np.sin(x_data[:, 0] * 5) + np.cos(x_data[:, 1] * 3)

gp = GP(
    x_data, y_data,
    init_hyperparameters=np.array([1.0, 0.1, 0.1]),
    gp2Scale=True,
    gp2Scale_dask_client=client,
    gp2Scale_batch_size=10000,          # covariance blocks per worker batch
    gp2Scale_linalg_mode="sparseCG",    # sparse linear solver
    compute_device="cpu",
)

# Estimate execution time before training
est_time = gp.get_gp2Scale_exec_time(
    time_per_worker_execution=0.5,  # seconds per block
    number_of_workers=4
)
print(f"Estimated training time: {est_time:.1f}s")

# Train (gp2Scale forces method='mcmc' internally)
bounds = np.array([[0.01, 5.0], [0.01, 1.0], [0.01, 1.0]])
gp.train(hyperparameter_bounds=bounds, max_iter=50)
print("Trained hps:", gp.hyperparameters)

# Predict on a small grid
x_pred = np.random.rand(500, 2)
mean = gp.posterior_mean(x_pred)["m(x)"]
print("Prediction shape:", mean.shape)  # (500,)

client.close()
```

--------------------------------

### Enable fvGP Logging

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/api/logging.md

Use this snippet to enable logging specifically for the 'fvgp' namespace. This is typically the first step before configuring other logging behaviors.

```python
from loguru import logger
logger.enable("fvgp")
```

--------------------------------

### Initialize and Train a Gaussian Process

Source: https://github.com/lbl-camera/fvgp/blob/master/examples/SingleTaskTest.ipynb

Initializes a Gaussian Process with specified hyperparameters and noise variances, then trains it using MCMC with defined hyperparameter bounds. Suitable for scenarios requiring parallel GP computations.

```python
#duplicate data: in practice, this would be different data in every column
y_data = np.broadcast_to(y_data[:, None], (y_data.size, 10))

my_gp1 = GP(x_data,y_data,
            init_hyperparameters = np.ones((2))/10.,  # we need enough of those for kernel, noise, and prior mean functions
            noise_variances=np.ones(y_data.shape[0]) * 0.1, # providing noise variances and a noise function will raise a warning 
            compute_device='cpu',
            )


hps_bounds = np.array([[0.01,10.], #signal variance for the kernel
                       [0.01,10.], #length scale for the kernel
                      ])

print("Standard Training (MCMC)")
hps = my_gp1.train(hyperparameter_bounds=hps_bounds, info = True, max_iter = 100)
print("Result=", hps, "after ", time.time() - st, " seconds")
print("")

```

--------------------------------

### Prepare Multi-Task Data and Define Kernel

Source: https://github.com/lbl-camera/fvgp/blob/master/examples/NonEuclideanInputSpaces.ipynb

Sets up multi-task data where each input (string) has multiple output dimensions. It reuses the string distance metric and defines a kernel suitable for multi-task learning.

```python
x_data = ['frf','ferfe','ferf','febhn']
y_data = np.zeros((len(x_data),5))


y_data[:,0] = np.random.rand(len(x_data))
y_data[:,1] = np.random.rand(len(x_data))
y_data[:,2] = np.random.rand(len(x_data))
y_data[:,3] = np.random.rand(len(x_data))
y_data[:,4] = np.random.rand(len(x_data))

#it is good practice to check the format of the data
print(len(x_data))
print(y_data.shape)

def string_distance(string1, string2):
    difference = abs(len(string1) - len(string2))
    common_length = min(len(string1),len(string2))
    string1 = string1[0:common_length]
    string2 = string2[0:common_length]
    
    for i in range(len(string1)):
        if string1[i] != string2[i]:
            difference += 1.

    return difference


from fvgp.kernels import matern_kernel_diff1
def kernel(x1,x2,hps):
    d = np.zeros((len(x1),len(x2)))
    count1 = 0
    for entry in x1:
        string1 = entry[0]
        count2 = 0
        for entry2 in x2:
            string2 = entry2[0]
            d[count1,count2] = string_distance(string1,string2)
            count2 += 1
        count1 += 1
    return hps[0] * matern_kernel_diff1(d,hps[1])
    

bounds = np.array([[0.001,100.],[0.001,100]])

```

--------------------------------

### Clone fvGP Repository

Source: https://github.com/lbl-camera/fvgp/blob/master/CONTRIBUTING.rst

Clone your forked repository locally to begin development.

```shell
git clone git@github.com:your_name_here/fvgp.git
```

--------------------------------

### Import Libraries and Initialize Dask Client

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/SingleTaskTest.ipynb

Imports necessary libraries including numpy, matplotlib, fvgp, time, and distributed for parallel computing. Initializes a Dask client for distributed training.

```python
import numpy as np
import matplotlib.pyplot as plt
from fvgp import GP
import time
from distributed import Client
client = Client()

%load_ext autoreload
%autoreload 2
```

--------------------------------

### Custom MCMC Sampler with Proposal Distributions

Source: https://context7.com/lbl-camera/fvgp/llms.txt

Shows how to use `gpMCMC` for custom MCMC sampling, including defining a log-prior function and custom proposal distributions for specific hyperparameters.

```python
import numpy as np
from fvgp import gpMCMC, ProposalDistribution
from fvgp import GP

np.random.seed(42)
x_data = np.random.rand(60).reshape(-1, 1)
y_data = np.sin(5 * x_data[:, 0]) + np.random.randn(60) * 0.05

gp = GP(x_data, y_data, init_hyperparameters=np.array([1.0, 0.2]))
gp.train(method="local")   # get a good starting point

bounds = np.array([[0.01, 5.0], [0.01, 2.0]])

# Define a uniform prior within bounds
def log_prior(x, args):
    b = args["bounds"]
    if np.any(x < b[:, 0]) or np.any(x > b[:, 1]):
        return -np.inf
    return 0.0   # log(1) for uniform

# Custom proposal for parameter 0 only (univariate normal)
prop0 = ProposalDistribution(
    indices=np.array([0]),
    proposal_dist="normal",
    init_prop_Sigma=np.array([[0.1]]),
    adapt_callable="normal",
    r_opt=0.234,
)
prop1 = ProposalDistribution(
    indices=np.array([1]),
    proposal_dist="normal",
    init_prop_Sigma=np.array([[0.05]]),
)

sampler = gpMCMC(
    log_likelihood_function=gp.log_likelihood,
    prior_function=log_prior,
    proposal_distributions=[prop0, prop1],
    args={"bounds": bounds},
)

result = sampler.run_mcmc(
    x0=gp.hyperparameters,
    n_updates=2000,
    info=False,
    break_condition="default",
)

print("MAP hyperparameters:", result["max x"])
print("Posterior mean:     ", result["mean(x)"])
print("Posterior variance: ", result["var(x)"])
# MAP hyperparameters: [0.94  0.19]
# Posterior mean:      [0.91  0.20]
```

--------------------------------

### Initialize and Train Multi-Task GP

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/MultiTaskTest.ipynb

Initializes a multi-task Gaussian Process model using fvGP with the generated data. The model is then trained for a specified number of iterations.

```python
from fvgp import fvGP

my_gp2 = fvGP(x_data.reshape(len(x_data),1), np.column_stack([y_data1, y_data2]))
print("Global Training in progress")
my_gp2.train(max_iter = 20, info=True)
```

--------------------------------

### Run Tests and Linting

Source: https://github.com/lbl-camera/fvgp/blob/master/CONTRIBUTING.rst

Ensure your changes pass linting with flake8 and all tests, including cross-version testing with tox.

```shell
flake8 fvgp tests
python setup.py test or pytest
tox
```

--------------------------------

### Set up MCMC for Hyperparameter Optimization

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/gp2ScaleTest.ipynb

Configures a Markov Chain Monte Carlo (MCMC) sampler to optimize the GP hyperparameters. This involves defining the objective function, prior, and proposal distribution.

```python
from fvgp import ProposalDistribution
init_s = (np.diag(hps_bounds[:,1]-hps_bounds[:,0])/100.)**2


def obj_func(hps,args):
    return my_gp2S.log_likelihood(hyperparameters=hps[0:2])

from fvgp import gpMCMC
def proposal_distribution(x0, hps, obj):
    cov = obj.prop_args["prop_Sigma"]
    proposal_hps = np.zeros((len(x0)))
    proposal_hps = np.random.multivariate_normal(
        mean = x0, cov = cov, size = 1).reshape(len(x0))
    return proposal_hps

def in_bounds(v,bounds):
    if any(v<bounds[:,0]) or any(v>bounds[:,1]): return False
    return True
def prior_function(theta,args):
    bounds = args["bounds"]
    if in_bounds(theta, bounds): 
        return 0. + np.sum(np.log(theta)/2.)
    else: 
        return -np.inf
pd = ProposalDistribution([0,1] ,proposal_dist=proposal_distribution,
                        init_prop_Sigma = init_s, adapt_callable="normal")

my_mcmc = gpMCMC(obj_func, prior_function, [pd],
                args={"bounds":hps_bounds})
logger.disable("fvgp")
hps = np.random.uniform(
                        low = hps_bounds[:,0], 
                        high = hps_bounds[:,1], 
                        size = len(hps_bounds))
mcmc_result = my_mcmc.run_mcmc(x0=hps, n_updates=110, break_condition="default", info = True)

my_gp2S.set_hyperparameters(mcmc_result["x"][-1])
```

--------------------------------

### Initialize and Train Multi-Task GP

Source: https://github.com/lbl-camera/fvgp/blob/master/examples/MultiTaskTest.ipynb

Initializes a multi-task Gaussian Process with the custom deep kernel and trains it using MCMC. Hyperparameter bounds are specified for the optimization process.

```python
my_gp2 = fvGP(x_data3,y_data3,
              init_hyperparameters=np.ones((n.number_of_hps+2))*0.1, kernel_function=deep_multi_task_kernel
             )

print("MCMC Training in progress")
bounds = np.zeros((n.number_of_hps+2,2))
bounds[0] = np.array([0.01,1.])
bounds[1] = np.array([0.1,1.])
bounds[2:] = np.array([-1,1])
my_gp2.train(hyperparameter_bounds=bounds,max_iter = 1000, method = "mcmc")
```

--------------------------------

### Custom Kernel Functions: Composite, Non-Stationary, Wendland

Source: https://context7.com/lbl-camera/fvgp/llms.txt

Illustrates how to define and use custom kernel functions, including composite kernels (sum of SE and periodic), non-stationary kernels, and Wendland kernels for sparse applications.

```python
import numpy as np
from fvgp import GP
from fvgp.kernels import (
    get_distance_matrix, matern_kernel_diff1, matern_kernel_diff2,
    squared_exponential_kernel, periodic_kernel,
    wendland_anisotropic, non_stat_kernel
)

x_data = np.random.rand(80).reshape(-1, 1)
y_data = np.sin(6 * x_data[:, 0])

# 1. Composite: SE + periodic (sum kernel)
def composite_kernel(x1, x2, hps):
    d = get_distance_matrix(x1, x2)
    se   = hps[0] * squared_exponential_kernel(d, hps[1])
    per  = hps[2] * periodic_kernel(d, hps[3], hps[4])
    return se + per

gp1 = GP(x_data, y_data,
         init_hyperparameters=np.array([1.0, 0.3, 0.5, 0.2, 0.5]))
bounds = np.array([[0.01,5],[0.01,2],[0.01,5],[0.01,2],[0.01,1.5]])
gp1.train(hyperparameter_bounds=bounds, method="global", max_iter=30)

# 2. Non-stationary kernel with basis function locations
x0 = np.linspace(0, 1, 5).reshape(-1, 1)   # basis function centers
def nonstat_kernel_custom(x1, x2, hps): # Renamed to avoid conflict
    w = hps[:5]    # weights for 5 basis functions
    l = hps[5]     # width
    return non_stat_kernel(x1, x2, x0, w, l)

gp2 = GP(x_data, y_data,
         init_hyperparameters=np.ones(6) * 0.5)

# 3. Wendland kernel (sparse, for large datasets)
def wendland_kernel(x1, x2, hps):
    return wendland_anisotropic(x1, x2, hps)

gp3 = GP(x_data, y_data,
         init_hyperparameters=np.array([1.0, 0.5]))

x_pred = np.linspace(0, 1, 100).reshape(-1, 1)
print("SE+Periodic mean:", gp1.posterior_mean(x_pred)["m(x)"][:3])
```

--------------------------------

### Initialize Multi-Task Gaussian Process (fvGP)

Source: https://context7.com/lbl-camera/fvgp/llms.txt

Initializes a multi-task Gaussian Process model. The `y_data` should have shape (V, No) where V is the number of observations and No is the number of outputs/tasks. Missing observations should be marked with `np.nan`.

```python
import numpy as np
from fvgp import fvGP
from fvgp.kernels import get_distance_matrix, matern_kernel_diff1

# Two correlated 1D tasks
def f1(x): return 0.5 * x
def f2(x): return -0.25 * x - 1.0

np.random.seed(42)
x_data = np.random.rand(30).reshape(-1, 1)
y_data = np.column_stack([
    f1(x_data[:, 0]) + np.random.randn(30) * 0.01,
    f2(x_data[:, 0]) + np.random.randn(30) * 0.01,
])

# Example initialization (actual fvGP instantiation would follow)
```

--------------------------------

### MCMC for Hyperparameter Optimization

Source: https://github.com/lbl-camera/fvgp/blob/master/examples/gp2ScaleTest.ipynb

Sets up and runs Markov Chain Monte Carlo (MCMC) sampling to optimize hyperparameters. Includes defining objective, proposal, prior functions, and running the MCMC chain.

```python
from fvgp import ProposalDistribution
init_s = (np.diag(hps_bounds[:,1]-hps_bounds[:,0])/100.)**2


def obj_func(hps,args):
    return my_gp2S.log_likelihood(hyperparameters=hps[0:2])

from fvgp import gpMCMC
def proposal_distribution(x0, hps, obj):
    cov = obj.prop_args["prop_Sigma"]
    proposal_hps = np.zeros((len(x0)))
    proposal_hps = np.random.multivariate_normal(
        mean = x0, cov = cov, size = 1).reshape(len(x0))
    return proposal_hps

def in_bounds(v,bounds):
    if any(v<bounds[:,0]) or any(v>bounds[:,1]): return False
    return True

def prior_function(theta,args):
    bounds = args["bounds"]
    if in_bounds(theta, bounds): 
        return 0.
    else: 
        return -np.inf
        
pd = ProposalDistribution([0,1] ,proposal_dist=proposal_distribution,
                        init_prop_Sigma = init_s, adapt_callable="normal")

my_mcmc = gpMCMC(obj_func, prior_function, [pd],
                args={\"bounds\":hps_bounds})
logger.disable("fvgp")
hps = np.random.uniform(
                        low = hps_bounds[:,0], 
                        high = hps_bounds[:,1], 
                        size = len(hps_bounds))
mcmc_result = my_mcmc.run_mcmc(x0=hps, n_updates=110, break_condition="default", info = True)

my_gp2S.set_hyperparameters(mcmc_result["x"][-1])
```

--------------------------------

### Import Libraries for Multi-Task GP on Words

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/NonEuclideanInputSpaces.ipynb

Imports necessary libraries for multi-task Gaussian Process modeling on non-Euclidean input spaces, including fvgp, numpy, matplotlib, and plotly.

```python
import numpy as np
import matplotlib.pyplot as plt
from fvgp import GP
import plotly.graph_objects as go
from itertools import product

%load_ext autoreload
%autoreload 2
```

--------------------------------

### Import Libraries for GP on Words

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/NonEuclideanInputSpaces.ipynb

Imports necessary libraries including numpy, matplotlib, and fvgp components for Gaussian Process modeling. It also loads IPython extensions for code reloading.

```python
import numpy as np
import matplotlib.pyplot as plt
from fvgp import GP
from dask.distributed import Client
%load_ext autoreload
%autoreload 2
```

--------------------------------

### Define and Plot Sample Functions

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/MultiTaskTest.ipynb

Defines two simple linear functions (f1 and f2) and generates synthetic data with added noise for each. This data is then plotted along with the ground truth functions.

```python
def f1(x): return 0.5 * x
def f2(x): return (-.25 * x) - 1.

x_pred1d = np.linspace(0,1,50)
plt.plot(x_pred1d,f1(x_pred1d))
plt.plot(x_pred1d,f2(x_pred1d))
x_data = np.random.rand(10)
y_data1 = f1(x_data) + np.random.uniform(low = -0.01, high = 0.01, size =len(x_data))
y_data2 = f2(x_data) + np.random.uniform(low = -0.01, high = 0.01, size =len(x_data))
plt.scatter(x_data,y_data1) 
plt.scatter(x_data,y_data2) 
plt.show()
```

--------------------------------

### Configure Logging Level and Output

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/api/logging.md

Configure the logging level to INFO and direct logs to standard output, filtering for messages from the 'fvgp' namespace. Refer to Python's logging levels for more information.

```python
import sys
logger.add(sys.stdout, filter="fvgp", level="INFO")
```

--------------------------------

### Multi-task GP Training and Prediction

Source: https://context7.com/lbl-camera/fvgp/llms.txt

Demonstrates training a multi-task Gaussian Process with missing data and noise variances, followed by prediction and evaluation.

```python
import numpy as np
from fvgp import fvGP

# Assume x_data, y_data, f1, f2 are defined elsewhere
# Example placeholder definitions:
x_data = np.linspace(0, 1, 10).reshape(-1, 1)
y_data = np.column_stack([np.sin(2 * np.pi * x_data[:, 0]), np.cos(2 * np.pi * x_data[:, 0])])
def f1(x): return np.sin(6 * x)
def f2(x): return np.cos(6 * x)

noise_variances = np.full(y_data.shape, 0.01)
y_data[5, 0]         = np.nan   # task 0 missing at point 5
noise_variances[5, 0] = np.nan

my_gp = fvGP(x_data, y_data, noise_variances=noise_variances,
             init_hyperparameters=np.ones(3))

hps_bounds = np.array([[0.01, 5.0], [0.01, 5.0], [1e-4, 0.5]])
my_gp.train(hyperparameter_bounds=hps_bounds, max_iter=100, method="mcmc")
print("Trained hyperparameters:", my_gp.hyperparameters)

# Predict both tasks simultaneously
x_pred = np.linspace(0, 1, 50).reshape(-1, 1)
mean = my_gp.posterior_mean(x_pred)["m(x)"]         # shape (50, 2)
std  = np.sqrt(my_gp.posterior_covariance(
    x_pred, x_out=np.array([0, 1]))["v(x)"])         # shape (50, 2)

print("Mean task 1 at x=0.5:", mean[25, 0])
print("Mean task 2 at x=0.5:", mean[25, 1])

# Multi-task validation metrics
x_test = np.linspace(0, 1, 100).reshape(-1, 1)
y_test = np.column_stack([f1(x_test[:, 0]), f2(x_test[:, 0])])
print("RMSE:", my_gp.rmse(x_test, y_test))
print("R2:  ", my_gp.r2(x_test, y_test))
```

--------------------------------

### Prepare Multi-Task Data for GP

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/NonEuclideanInputSpaces.ipynb

Prepares multi-task data where each input is a string and the output is a numpy array representing multiple tasks. It prints the dimensions of the prepared data.

```python
x_data = ['frf','ferfe','ferf','febhn']
y_data = np.zeros((len(x_data),5))


y_data[:,0] = np.random.rand(len(x_data))
y_data[:,1] = np.random.rand(len(x_data))
y_data[:,2] = np.random.rand(len(x_data))
y_data[:,3] = np.random.rand(len(x_data))
y_data[:,4] = np.random.rand(len(x_data))

#it is good practice to check the format of the data
print(len(x_data))
print(y_data.shape)
```

--------------------------------

### Training Data Output

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/MultiTaskTest.ipynb

Displays the formatted training data, showing the input features and the corresponding target values for each task.

```text
[[0.21969632 0.        ]
 [0.67822974 0.        ]
 [0.39338162 0.        ]
 [0.53768463 0.        ]
 [0.90107951 0.        ]
 [0.3552601  0.        ]
 [0.04446481 0.        ]
 [0.53254495 0.        ]
 [0.59485203 0.        ]
 [0.06274843 0.        ]
 [0.21969632 1.        ]
 [0.67822974 1.        ]
 [0.39338162 1.        ]
 [0.53768463 1.        ]
 [0.90107951 1.        ]
 [0.3552601  1.        ]
 [0.04446481 1.        ]
 [0.53254495 1.        ]
 [0.59485203 1.        ]
 [0.06274843 1.        ]]
[ 0.10278164  0.33148297  0.19577228  0.26939934  0.4443221   0.18696836
  0.01852347  0.26807251  0.30028226  0.02225798 -1.04592364 -1.17354344
 -1.10818716 -1.13253365 -1.2153672  -1.08057904 -1.01004836 -1.12581116
 -1.14926989 -1.01366415]
```

--------------------------------

### Load and Sparsify Data for GP

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/MultiTaskTest.ipynb

Load simulation data and sparsify it for use with Gaussian processes. It is good practice to check the shape of the processed data.

```python
data = np.load("./data/sim_variable_mod.npy")
sparsification = 4

x_data3 = data[:,5:][::sparsification]
y_data3 = data[:,0:2][::sparsification]

#it is good practice to check the format of the data
print(x_data3.shape)
print(y_data3.shape)
```

--------------------------------

### Display Training Progress

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/MultiTaskTest.ipynb

Shows the output from the GP training process, indicating the current iteration and the function value.

```text
Global Training in progress
Finished  10  out of  20  MCMC iterations. f(x)=  43.93801926227358
```

--------------------------------

### Create a New Branch

Source: https://github.com/lbl-camera/fvgp/blob/master/CONTRIBUTING.rst

Create a new branch for your bugfix or feature development.

```shell
git checkout -b name-of-your-bugfix-or-feature
```

--------------------------------

### Manually Control GP Hyperparameters

Source: https://context7.com/lbl-camera/fvgp/llms.txt

Allows direct setting and reading of GP hyperparameters without retraining. Useful for loading saved parameters or testing specific configurations.

```python
import numpy as np
from fvgp import GP

x_data = np.random.rand(60).reshape(-1, 1)
y_data = np.sin(4 * x_data[:, 0])

gp = GP(x_data, y_data, init_hyperparameters=np.array([1.0, 0.3]))

# Read current hyperparameters
print("Current hps:", gp.hyperparameters)

# Manually set hyperparameters (e.g. loaded from a previous run)
saved_hps = np.array([0.85, 0.21])
gp.set_hyperparameters(saved_hps)
print("Updated hps:", gp.hyperparameters)
```

--------------------------------

### Run a Subset of Unit Tests

Source: https://github.com/lbl-camera/fvgp/blob/master/CONTRIBUTING.rst

Execute a specific subset of unit tests using the unittest module.

```python
python -m unittest tests.test_fvgp
```

--------------------------------

### Train Multi-Task GP Model

Source: https://github.com/lbl-camera/fvgp/blob/master/examples/NonEuclideanInputSpaces.ipynb

Initializes and trains a multi-task Gaussian Process model using the custom string kernel. Training involves optimizing hyperparameters within specified bounds.

```python
from fvgp import fvGP
my_gp2 = fvGP(x_data,y_data,init_hyperparameters=np.ones((2)),
              kernel_function=kernel
             )
print("Global Training in progress")
my_gp2.train(hyperparameter_bounds=bounds, max_iter = 20)

```

--------------------------------

### Log to a File

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/api/logging.md

Configure Loguru to write log messages to a file. The filename will include a timestamp, allowing for time-based log organization. Loguru offers many additional configuration options.

```python
logger.add("file_{time}.log")
```

--------------------------------

### GP.set_hyperparameters / GP.hyperparameters

Source: https://context7.com/lbl-camera/fvgp/llms.txt

Provides direct control over GP hyperparameters, allowing manual setting and retrieval without retraining. Useful for loading saved states or testing specific configurations.

```APIDOC
## `GP.set_hyperparameters` / `GP.hyperparameters` — Manual Hyperparameter Control

Allows direct setting of hyperparameters without re-running training, which is useful for loading saved parameters or testing specific configurations. The GP's prior covariance and likelihood are updated immediately.

```python
import numpy as np
from fvgp import GP

x_data = np.random.rand(60).reshape(-1, 1)
y_data = np.sin(4 * x_data[:, 0])

gp = GP(x_data, y_data, init_hyperparameters=np.array([1.0, 0.3]))

# Read current hyperparameters
print("Current hps:", gp.hyperparameters)

# Manually set hyperparameters (e.g. loaded from a previous run)
saved_hps = np.array([0.85, 0.21])
gp.set_hyperparameters(saved_hps)
print("Updated hps:", gp.hyperparameters)
```
```

--------------------------------

### Train Multi-Task GP Model

Source: https://github.com/lbl-camera/fvgp/blob/master/docs/source/examples/NonEuclideanInputSpaces.ipynb

Initializes and trains a multi-task Gaussian Process (fvGP) model using the custom string kernel and specified hyperparameter bounds. Training is limited to a maximum of 20 iterations.

```python
from fvgp import fvGP
my_gp2 = fvGP(x_data,y_data,init_hyperparameters=np.ones((2)),
              kernel_function=kernel
             )
print("Global Training in progress")
my_gp2.train(hyperparameter_bounds=bounds, max_iter = 20)
```