Kernel Methods Library

https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/
Admin
This project provides documentation and examples for a library focused on kernel methods, covering...
Tokens:30,266
Snippets:337
Trust Score:9.7
Update:4 months ago
Show doc for...
Tokens:
Raw
### Example: SqExponentialKernel with ScaleTransform

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Demonstrates composing a SqExponentialKernel with a ScaleTransform and evaluating it.

```julia
SqExponentialKernel() ∘ ScaleTransform(0.5)
(SqExponentialKernel() ∘ ScaleTransform(0.5))(0, 2) == exp(-0.5)
```

--------------------------------

### Example: Chained Transformations

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Demonstrates composing a kernel with multiple ScaleTransforms using the prefix form of the composition operator.

```julia
∘(ExponentialKernel(), ScaleTransform(2), ScaleTransform(0.5))
∘(ExponentialKernel(), ScaleTransform(2), ScaleTransform(0.5))(1, 2) == exp(-1)
```

--------------------------------

### Initial Prediction Visualization

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/train-kernel-parameters

Visualizes the initial prediction of the model using starting parameters before optimization. This helps to see the initial state of the regression.

```julia
p0 = [1.1, 0.1, 0.01, 0.001]
θ = log.(p0)
ŷ = f(x_test, x_train, y_train, θ)
scatter(x_train, y_train; label="data")
plot!(x_test, sinc; label="true function")
plot!(x_test, ŷ; label="prediction")
```

--------------------------------

### Calculate Initial Loss Value

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/train-kernel-parameters

Computes and displays the initial value of the loss function using the starting parameters.

```julia
loss(θ)
```

--------------------------------

### RowVecs Example Usage

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/api

Demonstrates the usage of RowVecs, showing how it wraps a matrix and allows access to individual rows as vectors. The length of RowVecs corresponds to the number of rows.

```julia
X = randn(5, 2)

x = RowVecs(X)

length(x) == 5

X[3, :] == x[3]
```

--------------------------------

### Nested Composite Kernels

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/userguide

Shows an example of a highly nested composite kernel structure involving sums, products, and transformations.

```julia
k = (
    0.5 * SqExponentialKernel() * Matern12Kernel() +
    0.2 * (LinearKernel() ∘ ScaleTransform(2.0) + PolynomialKernel())
) ∘ ARDTransform([0.1, 0.5])
```

--------------------------------

### ColVecs Example Usage

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/api

Illustrates how to use ColVecs to wrap a matrix and access its properties. Shows that the length of ColVecs corresponds to the number of columns and individual elements can be accessed like vectors.

```julia
X = randn(2, 5)

x = ColVecs(X)

length(x) == 5

X[:, 3] == x[3]
```

--------------------------------

### Define SimpleKernel with Custom Metric

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/create_kernel

Use SimpleKernel for kernels where the function depends on a metric (e.g., k(x, y) = f(d(x, y))). This example redefines SqExponentialKernel.

```julia
struct MyKernel <: KernelFunctions.SimpleKernel end

KernelFunctions.kappa(::MyKernel, d2::Real) = exp(-d2)
KernelFunctions.metric(::MyKernel) = SqEuclidean()
```

--------------------------------

### Compose Transforms using ∘ Operator for ChainTransform Example

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/transform

Demonstrates composing two transforms, ScaleTransform and LinearTransform, using the ∘ operator, achieving the same result as ChainTransform.

```julia
map(t2 ∘ t1, ColVecs(X)) == ColVecs(A * (l .* X))
```

--------------------------------

### Define Kernel with Subset of Trainable Parameters

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/create_kernel

Implement Functors.functor to explicitly define which fields are trainable when only a subset of fields should be modified. This example makes 'a' trainable while keeping 'n' fixed.

```julia
import Functors

struct MyKernel{T} <: KernelFunctions.Kernel
    n::Int
    a::Vector{T}
end

function Functors.functor(::Type{<:MyKernel}, x::MyKernel)
    function reconstruct_mykernel(xs)
        # keep field `n` of the original kernel and set `a` to (possibly different) `xs.a`
        return MyKernel(x.n, xs.a)
    end
    return (a = x.a,), reconstruct_mykernel
end
```

--------------------------------

### Overfitting Example with High Polynomial Degree

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/kernel-ridge-regression

Demonstrates overfitting by fitting a model with a high polynomial degree (20) without regularization. This highlights the need for regularization techniques.

```julia
featurized_fit_and_plot(20)
```

--------------------------------

### Define Complex Kernel with Custom Evaluation

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/create_kernel

For kernels that do not fit the SimpleKernel structure, inherit from Kernel and define the (k::MyKernel)(x, y) function. This example recreates NeuralNetworkKernel.

```julia
struct MyKernel <: KernelFunctions.Kernel end

(::MyKernel)(x, y) = asin(dot(x, y) / sqrt((1 + sum(abs2, x)) * (1 + sum(abs2, y))))
```

--------------------------------

### Load Packages and Set Up

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/gaussian-process-priors

Loads necessary packages for Gaussian Processes and plotting, and sets a random seed for reproducibility.

```julia
# Load required packages
using KernelFunctions, LinearAlgebra
using Plots, Plots.PlotMeasures
default(; lw=1.0, legendfontsize=8.0)
using Random: seed!
seed!(42); # reproducibility

```

--------------------------------

### Create KernelProduct using '*' operator

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Demonstrates creating a KernelProduct using the overloaded '*' operator and verifies its equivalence to the constructor. Also shows how to compute the kernel matrix for a product kernel.

```julia
julia> k1 = SqExponentialKernel(); k2 = LinearKernel(); X = rand(5);

julia> (k = k1 * k2) == KernelProduct(k1, k2)
true

julia> kernelmatrix(k1 * k2, X) == kernelmatrix(k1, X) .* kernelmatrix(k2, X)
true

julia> kernelmatrix(k, X) == kernelmatrix(k1 * k2, X)
true
```

--------------------------------

### Initialize and Optimize Parameters

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/train-kernel-parameters

Initializes the parameters and the optimizer, then trains the kernel parameters by iteratively calculating gradients and updating the parameters using the specified optimizer.

```julia
θ = log.([1.1, 0.1, 0.01, 0.001]) # Initial vector
loss(θ)

opt = Optimise.AdaGrad(0.5)

for i in 1:15
    grads = only((Zygote.gradient(loss, θ)))
    Optimise.update!(opt, θ, grads)
end

loss(θ)
```

--------------------------------

### Create KernelTensorProduct using constructor with arguments

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Shows how to construct a KernelTensorProduct by providing kernels as individual arguments, demonstrating their equivalence to the operator-based creation.

```julia
julia> KernelTensorProduct(k1, k2) == k1 ⊗ k2
true
```

--------------------------------

### Creating Composite Kernels

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/userguide

Demonstrates how to create composite kernels by summing and multiplying existing kernels using operators or dedicated functions.

```julia
k1 = SqExponentialKernel()
k2 = Matern32Kernel()
k = 0.5 * k1 + 0.2 * k2 # KernelSum
k = k1 * k2 # KernelProduct
```

--------------------------------

### Create KernelProduct using Tuple or Vector

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Shows how to construct a KernelProduct by providing kernels as a Tuple or a Vector, demonstrating their equivalence to the operator-based creation.

```julia
julia> KernelProduct((k1, k2)) == k1 * k2
true

julia> KernelProduct([k1, k2]) == KernelProduct((k1, k2)) == k1 * k2
true
```

--------------------------------

### Package Information

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/gaussian-process-priors

Displays the package status and versions used in the environment. This is helpful for reproducibility.

```julia
Status `~/work/KernelFunctions.jl/KernelFunctions.jl/examples/gaussian-process-priors/Project.toml`
  [31c24e10] Distributions v0.25.123
  [ec8451be] KernelFunctions v0.11.0 `/home/runner/work/KernelFunctions.jl/KernelFunctions.jl#ce18b29`
  [98b081ad] Literate v2.21.0
  [91a5bcdd] Plots v1.41.6
  [37e2e46d] LinearAlgebra v1.12.0
  [9a3f8284] Random v1.11.0
```

--------------------------------

### Create KernelTensorProduct using 'tensor' operator

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Demonstrates creating a KernelTensorProduct using the overloaded 'tensor' operator (⊗) and verifies its equivalence to the constructor. Also shows how to compute the kernel matrix for a tensor product kernel.

```julia
julia> k1 = SqExponentialKernel(); k2 = LinearKernel(); X = rand(5, 2);

julia> kernelmatrix(k1 ⊗ k2, RowVecs(X)) == kernelmatrix(k1, X[:, 1]) .* kernelmatrix(k2, X[:, 2])
true
```

--------------------------------

### Create KernelTensorProduct using Tuple or Vector

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Shows how to construct a KernelTensorProduct by providing kernels as a Tuple or a Vector, demonstrating their equivalence to the operator-based creation.

```julia
julia> KernelTensorProduct((k1, k2)) == k1 ⊗ k2
true

julia> KernelTensorProduct([k1, k2]) == k1 ⊗ k2
true
```

--------------------------------

### Load Required Packages

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/kernel-ridge-regression

Loads necessary packages for kernel ridge regression, linear algebra, distributions, plotting, and random number generation.

```julia
using KernelFunctions
using LinearAlgebra
using Distributions

using Plots;
default(; lw=2.0, legendfontsize=11.0, ylims=(-150, 500));

using Random: seed!
seed!(42);
```

--------------------------------

### Accessing and Optimizing Kernel Parameters with Flux.jl

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/userguide

Illustrates how to access trainable kernel parameters using Functors.jl and compute gradients using Flux.jl for optimization.

```julia
using Flux
kernelparams = Flux.params(k)
Flux.gradient(kernelparams) do
    # ... some loss function on the kernel ....
end
```

--------------------------------

### Load Packages for Kernel Training

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/train-kernel-parameters

Loads necessary packages for kernel functions, linear algebra, distributions, plotting, benchmarking, optimization, and automatic differentiation. Ensure Zygote is compatible with your Julia version.

```julia
using KernelFunctions
using LinearAlgebra
using Distributions
using Plots
using BenchmarkTools
using Flux
using Flux: Optimise
using Zygote
using Random: seed!
seed!(42);
```

--------------------------------

### System Information

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/support-vector-machine

Provides details about the Julia version, operating system, CPU, and build information. Essential for debugging and ensuring consistent execution environments.

```text
Julia Version 1.12.5
Commit 5fe89b8ddc1 (2026-02-09 16:05 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, icelake-server)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 4 virtual cores)
Environment:
  JULIA_LOAD_PATH = :/home/runner/.julia/packages/JuliaGPsDocs/7M86H/src


```

--------------------------------

### Initialize Positive Kernel Parameters

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/train-kernel-parameters

Initializes kernel parameters with positivity constraints using ParameterHandling.jl. This sets up named tuples where each parameter is wrapped in `positive()` to enforce positivity. The `value_flatten` function prepares these parameters for optimization by creating a flat vector and a function to unflatten them.

```julia
using ParameterHandling

raw_initial_θ = (
    k1=positive(1.1), k2=positive(0.1), k3=positive(0.01), noise_var=positive(0.001)
)

flat_θ, unflatten = ParameterHandling.value_flatten(raw_initial_θ)
```

--------------------------------

### System Information

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/gaussian-process-priors

Provides details about the Julia version, build information, and platform. This information is crucial for debugging and ensuring compatibility.

```julia
Julia Version 1.12.5
Commit 5fe89b8ddc1 (2026-02-09 16:05 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, icelake-server)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 4 virtual cores)
Environment:
  JULIA_LOAD_PATH = :/home/runner/.julia/packages/JuliaGPsDocs/7M86H/src
```

--------------------------------

### KernelSum Constructor with Tuple

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Creates a KernelSum by providing a Tuple of kernels. Recommended for fewer components.

```julia
k1 = SqExponentialKernel()
k2 = LinearKernel()
KernelSum((k1, k2)) == k1 + k2
```

--------------------------------

### Instantiate Polynomial Kernel

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Creates a PolynomialKernel with a specified degree and constant offset 'c'. The kernel is defined as (x' * x + c)^degree.

```julia
PolynomialKernel(; degree::Int=2, c::Real=0.0)
```

--------------------------------

### Kernel Matrix Computation with KernelSum (Matrix Input)

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/design

Illustrates how to compute the kernel matrix for a `KernelSum` when inputs are provided as an `AbstractMatrix`. It requires an `obsdim` argument to specify the observation dimension.

```julia
function kernelmatrix(k::KernelSum, x::AbstractMatrix; obsdim=1)
    return kernelmatrix(k.kernels[1], x; obsdim=obsdim) +
        kernelmatrix(k.kernels[2], x; obsdim=obsdim)
end
```

--------------------------------

### Instantiate Periodic Kernel

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Creates a PeriodicKernel with a specified parameter vector 'r'. The kernel models periodic behavior.

```julia
PeriodicKernel(; r::AbstractVector=ones(Float64, 1))
```

--------------------------------

### Instantiate Linear Kernel

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Creates a LinearKernel with an optional constant offset 'c'. The linear kernel is defined as x' * x + c.

```julia
LinearKernel(; c::Real=0.0)
```

--------------------------------

### RationalQuadraticKernel Constructor

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Constructs a Rational-Quadratic kernel with a specified shape parameter alpha and metric. The Euclidean metric is used by default.

```julia
RationalQuadraticKernel(; α::Real=2.0, metric=Euclidean())
```

--------------------------------

### RationalKernel Constructor

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Constructs a Rational kernel with a specified shape parameter alpha and metric. The Euclidean metric is used by default.

```julia
RationalKernel(; α::Real=2.0, metric=Euclidean())
```

--------------------------------

### Visualize and Compare Multiple Kernels

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/gaussian-process-priors

Defines a list of various Gaussian Process kernels and visualizes each one. This is useful for understanding the behavior and differences between kernels.

```julia
kernels = [
    Matern12Kernel(),
    Matern32Kernel(),
    Matern52Kernel(),
    SqExponentialKernel(),
    WhiteKernel(),
    ConstantKernel(),
    LinearKernel(),
    compose(PeriodicKernel(), ScaleTransform(0.2)),
    NeuralNetworkKernel(),
    GibbsKernel(; lengthscale=x -> sum(exp ∘ sin, x)),
]
plot(
    [visualize(k) for k in kernels]...;
    layout=(length(kernels), 1),
    size=(800, 220 * length(kernels) + 100),
)
```

--------------------------------

### KernelSum Constructor with Vector

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Creates a KernelSum by providing a Vector of kernels. Recommended for a large number of components.

```julia
k1 = SqExponentialKernel()
k2 = LinearKernel()
KernelSum([k1, k2]) == KernelSum((k1, k2)) == k1 + k2
```

--------------------------------

### Train Kernel Parameters with Optimization Loop

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/train-kernel-parameters

Trains the kernel parameters over 15 iterations using Zygote for gradients and AdaGrad for optimization. It visualizes the prediction at each step and saves the animation.

```julia
θ = log.(p0) # Initial vector
opt = Optimise.AdaGrad(0.5)

anim = Animation()
for i in 1:15
    grads = only((Zygote.gradient(loss, θ)))
    Optimise.update!(opt, θ, grads)
    scatter(
        x_train, y_train; lab="data", title="i = $(i), Loss = $(round(loss(θ), digits = 4))"
    )
    plot!(x_test, sinc; lab="true function")
    plot!(x_test, f(x_test, x_train, y_train, θ); lab="Prediction", lw=3.0)
    frame(anim)
end
gif(anim, "train-kernel-param.gif"; show_msg=false, fps=15);
```

--------------------------------

### Package Information

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/support-vector-machine

Lists the status of Julia packages used in the project, including versions and local paths. Useful for environment reproducibility.

```text
Status `~/work/KernelFunctions.jl/KernelFunctions.jl/examples/support-vector-machine/Project.toml`
  [31c24e10] Distributions v0.25.123
  [ec8451be] KernelFunctions v0.11.0 `/home/runner/work/KernelFunctions.jl/KernelFunctions.jl#ce18b29`
  [b1bec4e5] LIBSVM v0.8.1
  [98b081ad] Literate v2.21.0
  [91a5bcdd] Plots v1.41.6
  [37e2e46d] LinearAlgebra v1.12.0


```

--------------------------------

### Import necessary libraries

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/support-vector-machine

Imports required packages for data manipulation, kernel functions, SVM, linear algebra, plotting, and random number generation.

```julia
using Distributions
using KernelFunctions
using LIBSVM
using LinearAlgebra
using Plots
using Random

# Set seed
Random.seed!(1234);
```

--------------------------------

### GammaRationalKernel Constructor

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Constructs a Gamma-Rational kernel with shape parameters alpha and gamma, and a specified metric. The Euclidean metric is used by default.

```julia
GammaRationalKernel(; α::Real=2.0, γ::Real=1.0, metric=Euclidean())
```

--------------------------------

### with_lengthscale (single)

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/transform

Constructs a transformed kernel with a specified single lengthscale.

```APIDOC
## `with_lengthscale` (single)

### Description
Constructs a transformed kernel with a specified single lengthscale.

### Function Signature
```julia
with_lengthscale(kernel::Kernel, lengthscale::Real)
```

### Examples
```julia
kernel = with_lengthscale(SqExponentialKernel(), 2.5)

x = rand(2)
y = rand(2)

kernel(x, y) ≈ (SqExponentialKernel() ∘ ScaleTransform(0.4))(x, y)
```
```

--------------------------------

### Prepare Heterotopic Multi-Output Data

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/api

Formats inputs, observations, and output indices for multi-output kernels when only a subset of outputs are observed at each feature.

```julia
julia> x = [1.0, 2.0, 3.0];

julia> y = [-1.0, 0.0, 1.0];

julia> output_indices = [3, 2, 1];

julia> inputs, outputs = prepare_heterotopic_multi_output_data(x, y, output_indices);

julia> inputs
3-element Vector{Tuple{Float64, Int64}}:
 (1.0, 3)
 (2.0, 2)
 (3.0, 1)

julia> outputs
3-element Vector{Float64}:
 -1.0
  0.0
  1.0
```

--------------------------------

### Generate Toy Dataset for Regression

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/train-kernel-parameters

Generates a 1-dimensional toy dataset with specified bounds and number of samples, adding Gaussian noise to a sinc function.

```julia
xmin, xmax = -3, 3  # Bounds of the data
N = 50 # Number of samples
x_train = rand(Uniform(xmin, xmax), N)  # sample the inputs
σ = 0.1
y_train = sinc.(x_train) + randn(N) * σ  # evaluate a function and add some noise
x_test = range(xmin - 0.1, xmax + 0.1; length=300)
```

--------------------------------

### Instantiate Neural Network Kernel

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Creates a NeuralNetworkKernel, which is the limit of a Bayesian neural network with a single hidden layer as the number of units goes to infinity.

```julia
NeuralNetworkKernel()
```

--------------------------------

### Kernel Matrix Computation with KernelSum (Vector Input)

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/design

Shows a simplified implementation for computing the kernel matrix for a `KernelSum` when inputs are represented as an `AbstractVector`. This version is cleaner and less prone to bugs as it doesn't require an `obsdim` argument.

```julia
function kernelmatrix(k::KernelSum, x::AbstractVector)
    return kernelmatrix(k.kernels[1], x) + kernelmatrix(k.kernels[2], x)
end
```

--------------------------------

### Create a Squared Exponential Kernel

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/userguide

Instantiates a default Squared Exponential kernel. This is a common choice for smooth functions.

```julia
k = SqExponentialKernel()
```

--------------------------------

### Instantiate Piecewise Polynomial Kernel

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Creates a PiecewisePolynomialKernel with a specified dimension, degree, and metric. The kernel is defined on the unit ball with respect to the given metric.

```julia
PiecewisePolynomialKernel(; dim::Int, degree::Int=0, metric=Euclidean())
```

--------------------------------

### Multi-Output Kernel Equivalence

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/design

Shows the mathematical equivalence between a matrix-valued kernel and a standard kernel on an extended input domain. This is the core concept for handling multi-output kernels in the library.

```mathematica
k((x, p), (y, q)) = k_mat(x, y)[p, q]
```

--------------------------------

### ScaledKernel Constructor

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Constructs a scaled kernel by multiplying a base kernel with a variance factor.

```julia
ScaledKernel(k::Kernel, σ²::Real=1.0)
```

--------------------------------

### Create KroneckerPower Matrix

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/api

Constructs a `KroneckerPower` matrix on a D-dimensional input grid using the Kronecker product of a kernel with itself `dims` times. Requires Kronecker.jl and `iskroncompatible(κ)` to be true.

```julia
kernelkronmat(κ::Kernel, X::AbstractVector{<:Real}, dims::Int) -> KroneckerPower

```

--------------------------------

### WienerKernel Constructor

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Constructs the i-times integrated Wiener process kernel function. The 'i' parameter specifies the degree of integration.

```julia
WienerKernel(; i::Int=0)
WienerKernel{i}()
```

--------------------------------

### Generate half-moon dataset

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/support-vector-machine

Generates a dataset with two classes shaped like half-moons, similar to SciKit-Learn's make_moons function. Sets the number of samples per class to 50.

```julia
n1 = n2 = 50;
```

```julia
angle1 = range(0, π; length=n1)
angle2 = range(0, π; length=n2)
X1 = [cos.(angle1) sin.(angle1)] .+ 0.1 .* randn.()
X2 = [1 .- cos.(angle2) 1 .- sin.(angle2) .- 0.5] .+ 0.1 .* randn.()
X = [X1; X2]
x_train = RowVecs(X)
y_train = vcat(fill(-1, n1), fill(1, n2));
```

--------------------------------

### RationalKernel

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

The Rational kernel is defined for inputs x, x' and a metric d(⋅,⋅) with a shape parameter α > 0. It is defined as k(x, x'; α) = (1 + d(x, x')/α)^(-α). The Euclidean metric is used by default. The ExponentialKernel is recovered as α approaches infinity.

```APIDOC
## RationalKernel

### Description
Constructs a Rational kernel with a shape parameter `α` and a specified `metric`. The kernel is defined as `k(x,x′;α)=(1+d(x,x′)/α)−α`.

### Constructor
`RationalKernel(; α::Real=2.0, metric=Euclidean())

### Parameters
* **α** (Real) - Required - Shape parameter (α > 0).
* **metric** (Metric) - Optional - The distance metric to use (defaults to `Euclidean`).

### See Also
* `GammaRationalKernel`
```

--------------------------------

### Create KroneckerProduct Matrix from Vector Collection

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/api

Returns a `KroneckerProduct` matrix on a grid built from a collection of vectors. Requires Kronecker.jl and `iskroncompatible(κ)` to be true.

```julia
kernelkronmat(κ::Kernel, X::AbstractVector{<:AbstractVector}) -> KroneckerProduct

```

--------------------------------

### Create Periodic Kernel with Type and Dimensions

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Creates a PeriodicKernel with a specified element type and number of dimensions for the parameter 'r'. Defaults to Float64 and 1 dimension.

```julia
PeriodicKernel([T=Float64, dims::Int=1])
```

--------------------------------

### Check Kernel Matrix Size

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/api

Demonstrates the relationship between the input vector size and the kernel matrix size. This property holds true for any Kernel k and AbstractVector x.

```julia
size(kernelmatrix(k, x)) == (length(x), length(x))
```

--------------------------------

### Prepare Data for Isotopic Multi-Output GPs

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/api

A helper function to prepare data for multi-output Gaussian processes, transforming common data formats into the required AbstractVector{<:Tuple{T, Int}} format.

```julia
prepare_isotopic_multi_output_data(x::AbstractVector, y::ColVecs)
```

--------------------------------

### Train Model with AdaGrad

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/train-kernel-parameters

Trains the model for 15 iterations using the AdaGrad optimizer. In each iteration, gradients are computed using Zygote and then used to update the flattened parameters.

```julia
opt = Optimise.AdaGrad(0.5)
for i in 1:15
    grads = (Zygote.gradient(loss ∘ unflatten, flat_θ))[1]
    Optimise.update!(opt, flat_θ, grads)
end
```

--------------------------------

### Benchmark One Optimization Step

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/train-kernel-parameters

Benchmarks the computational cost of a single optimization step, including gradient calculation and parameter update using AdaGrad.

```julia
@benchmark let
    θ = log.(p0)
    opt = Optimise.AdaGrad(0.5)
    grads = only((Zygote.gradient(loss, θ)))
    Optimise.update!(opt, θ, grads)
end
```

--------------------------------

### Handle Multi-dimensional Inputs with obsdim=2

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/userguide

Use the obsdim=2 keyword argument as an alternative to ColVecs for specifying input orientation.

```julia
X = rand(10, 5)
k = SqExponentialKernel()
kernelmatrix(k, X; obsdim=2) # same as ColVecs(X)
```

--------------------------------

### Compose Transforms using ∘ Operator

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/transform

Demonstrates composing two transforms, ScaleTransform and ScaleTransform, using the ∘ operator. This is an alternative to using ChainTransform.

```julia
LowRankTransform(rand(10, 5)) ∘ ScaleTransform(2.0)
```

--------------------------------

### GammaRationalKernel

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

The Gamma-Rational kernel is defined for inputs x, x' and a metric d(⋅,⋅) with shape parameters α > 0 and γ ∈ (0, 2]. It is defined as k(x, x'; α, γ) = (1 + d(x, x')^γ / α)^(-α). The Euclidean metric is used by default. The GammaExponentialKernel is recovered as α approaches infinity.

```APIDOC
## GammaRationalKernel

### Description
Constructs a Gamma-Rational kernel with shape parameters `α` and `γ`, and a specified `metric`. The kernel is defined as `k(x,x′;α,γ)=(1+d(x,x′)^γ/α)−α`.

### Constructor
`GammaRationalKernel(; α::Real=2.0, γ::Real=1.0, metric=Euclidean())

### Parameters
* **α** (Real) - Required - Shape parameter (α > 0).
* **γ** (Real) - Required - Shape parameter (γ ∈ (0, 2]).
* **metric** (Metric) - Optional - The distance metric to use (defaults to `Euclidean`).

### See Also
* `RationalKernel`
* `RationalQuadraticKernel`
```

--------------------------------

### KernelSum Constructor using '+' operator

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Creates a sum of kernels using the overloaded '+' operator. This is equivalent to using the KernelSum constructor directly.

```julia
k1 = SqExponentialKernel()
k2 = LinearKernel()
X = rand(5)
(k = k1 + k2) == KernelSum(k1, k2)
kernelmatrix(k1 + k2, X) == kernelmatrix(k1, X) .+ kernelmatrix(k2, X)
kernelmatrix(k, X) == kernelmatrix(k1 + k2, X)
```

--------------------------------

### Define Input Points for GP Evaluation

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/gaussian-process-priors

Defines the number of input points and the range for evaluating the Gaussian Process. This creates a fine grid to approximate the real line for visualization.

```julia
num_inputs = 101
xlim = (-5, 5)
X = range(xlim...; length=num_inputs);

```

--------------------------------

### Visualize Kernel and GP Samples

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/gaussian-process-priors

Defines a function to visualize a kernel's heatmap, its cross-kernel values at specific points, and samples from a Gaussian Process defined by the kernel. Requires `heatmap` and `plot` functions.

```julia
function visualize(k::Kernel)
    K = kernelmatrix(k, X)
    f = mvn_sample(K)

    p_kernel_2d = heatmap(
        X,
        X,
        K;
        yflip=true,
        colorbar=false,
        ylabel=string(nameof(typeof(k))),
        ylim=xlim,
        yticks=([xlim[1], 0, xlim[end]], ["\u22125", raw"$x'வுகளை", "5"]),
        vlim=(0, 1),
        title=raw"$k(x, x')$",
        aspect_ratio=:equal,
        left_margin=5mm,
    )

    p_kernel_cut = plot(
        X,
        k.(X, 0.0);
        title=string(raw"$k(x, x_\mathrm{ref})$"),
        label=raw"$x_\mathrm{ref}=0.0$",
        legend=:topleft,
        foreground_color_legend=nothing,
    )
    plot!(X, k.(X, 1.5); label=raw"$x_\mathrm{ref}=1.5$")

    p_samples = plot(X, f; c="blue", title=raw"$f(x)$", ylim=(-3, 3), label=nothing)

    return plot(
        p_kernel_2d,
        p_kernel_cut,
        p_samples;
        layout=(1, 3),
        xlabel=raw"$xவுகளை",
        xlim=xlim,
        xticks=collect(xlim),
    )
end;

```

--------------------------------

### prepare_heterotopic_multi_output_data

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/api

Converts inputs `x`, observations `y`, and `output_indices` into a format suitable for multi-output kernels, handling cases where only a subset of outputs are observed at each feature. `y[n]` is the observed value for output `output_indices[n]` at feature `x[n]`.

```APIDOC
## prepare_heterotopic_multi_output_data

### Description
Utility functionality to convert a collection of inputs `x`, observations `y`, and `output_indices` into a format suitable for use with multi-output kernels. Handles the situation in which only one (or a subset) of outputs are observed at each feature. Ensures that all arguments are compatible with one another, and returns a vector of inputs and a vector of outputs. `y[n]` should be the observed value associated with output `output_indices[n]` at feature `x[n]`.

### Method Signature
```julia
prepare_heterotopic_multi_output_data(x::AbstractVector, y::AbstractVector{<:Real}, output_indices::AbstractVector{Int})
```

### Example
```julia
x = [1.0, 2.0, 3.0]
y = [-1.0, 0.0, 1.0]
output_indices = [3, 2, 1]
inputs, outputs = prepare_heterotopic_multi_output_data(x, y, output_indices)

# Expected output for inputs:
# 3-element Vector{Tuple{Float64, Int64}}:
#  (1.0, 3)
#  (2.0, 2)
#  (3.0, 1)

# Expected output for outputs:
# 3-element Vector{Float64}:
#  -1.0
#  0.0
#  1.0
```
```

--------------------------------

### Plot Training Data and True Function

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/train-kernel-parameters

Visualizes the generated training data points and the true underlying function (sinc) over a specified range.

```julia
scatter(x_train, y_train; label="data")
plot!(x_test, sinc; label="true function")
```

--------------------------------

### Compose Kernel with Transform

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Composes a kernel with a transformation of its inputs. This is the preferred method for creating kernels with input transformations.

```julia
kernel ∘ transform
∘(kernel, transform)
compose(kernel, transform)
```

--------------------------------

### Create Test Grid for Prediction

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/support-vector-machine

Generates a 100x100 2D grid covering the extent of the training data for prediction evaluation. This is useful for visualizing classification boundaries.

```julia
test_range = range(floor(Int, minimum(X)), ceil(Int, maximum(X))); length=100)
x_test = ColVecs(mapreduce(collect, hcat, Iterators.product(test_range, test_range)));
```

--------------------------------

### Handle Multi-dimensional Inputs with obsdim=1

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/userguide

Use the obsdim=1 keyword argument as an alternative to RowVecs for specifying input orientation.

```julia
X = rand(10, 5)
k = SqExponentialKernel()
kernelmatrix(k, X; obsdim=1) # same as RowVecs(X)
```

--------------------------------

### Calculate Initial Loss

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/train-kernel-parameters

Calculates the loss of the model using the initial parameter values. This involves unflattening the parameters from their flat representation back into the structured named tuple format before passing them to the loss function.

```julia
(loss ∘ unflatten)(flat_θ)
```

--------------------------------

### Prepare Isotopic Multi-Output Data (Column Vectors)

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/api

Converts a collection of inputs and a vector-of-vectors (ColVecs) into a format suitable for multi-output kernels, where outputs are organized by features.

```julia
julia> x = [1.0, 2.0, 3.0];

julia> Y = [1.1 2.1 3.1; 1.2 2.2 3.2]
2×3 Matrix{Float64}:
 1.1  2.1  3.1
 1.2  2.2  3.2

julia> inputs, outputs = prepare_isotopic_multi_output_data(x, ColVecs(Y));

julia> inputs
6-element KernelFunctions.MOInputIsotopicByFeatures{Float64, Vector{Float64}, Int64}:
 (1.0, 1)
 (1.0, 2)
 (2.0, 1)
 (2.0, 2)
 (3.0, 1)
 (3.0, 2)

julia> outputs
6-element Vector{Float64}:
 1.1
 1.2
 2.1
 2.2
 3.1
 3.2
```

--------------------------------

### Plotting a Specific Kernel Visualization

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/gaussian-process-priors

Generates and displays a visualization for the SqExponentialKernel. This involves calling the `visualize` function and setting plot dimensions and margins.

```julia
plot(visualize(SqExponentialKernel()); size=(800, 210), bottommargin=5mm, topmargin=5mm)

```

--------------------------------

### Prepare Isotopic Multi-Output Data (Row Vectors)

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/api

Converts a collection of inputs and output vectors (RowVecs) into a format suitable for multi-output kernels, where outputs are organized by outputs.

```julia
julia> x = [1.0, 2.0, 3.0];

julia> Y = [1.1 1.2; 2.1 2.2; 3.1 3.2]
3×2 Matrix{Float64}:
 1.1  1.2
 2.1  2.2
 3.1  3.2

julia> inputs, outputs = prepare_isotopic_multi_output_data(x, RowVecs(Y));

julia> inputs
6-element KernelFunctions.MOInputIsotopicByOutputs{Float64, Vector{Float64}, Int64}:
 (1.0, 1)
 (2.0, 1)
 (3.0, 1)
 (1.0, 2)
 (2.0, 2)
 (3.0, 2)

julia> outputs
6-element Vector{Float64}:
 1.1
 2.1
 3.1
 1.2
 2.2
 3.2
```

--------------------------------

### Create Mahalanobis-like Kernel

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/userguide

Replicates a Mahalanobis kernel by combining a Squared Exponential kernel with a LinearTransform of the inputs, using a pre-defined matrix P.

```julia
k = SqExponentialKernel() ∘ LinearTransform(sqrt(2) .* Q)
```

--------------------------------

### Implement a Custom Delta Metric

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/metrics

Create a new custom metric, `Delta`, by inheriting from `Distances.PreMetric` and implementing the `_evaluate` function. This metric is useful for kernels that do not require standard distance properties.

```julia
struct Delta <: Distances.PreMetric
end

@inline function Distances._evaluate(::Delta,a::AbstractVector{T},b::AbstractVector{T}) where {T}
    @boundscheck if length(a) != length(b)
        throw(DimensionMismatch("first array has length $(length(a)) which does not match the length of the second, $(length(b))."))
    end
    return a==b
end

@inline (dist::Delta)(a::AbstractArray,b::AbstractArray) = Distances._evaluate(dist,a,b)
@inline (dist::Delta)(a::Number,b::Number) = a==b
```

--------------------------------

### Check Kronecker Compatibility for a Kernel

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/api

Determines if a given kernel `k` is compatible with Kronecker constructions. Returns `false` by default. A kernel is compatible if it satisfies the product condition for all inputs.

```julia
iskroncompatible(k::Kernel)

```

--------------------------------

### Instantiate Piecewise Polynomial Kernel with Degree Type

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Creates a PiecewisePolynomialKernel specifying the degree as a type parameter. Requires dimension and metric.

```julia
PiecewisePolynomialKernel{degree}(; dim::Int, metric=Euclidean())
```

--------------------------------

### prepare_isotopic_multi_output_data (RowVecs)

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/api

Converts a collection of inputs `x` and output vectors `y` (represented by `RowVecs`) into a format suitable for multi-output kernels. `y[n]` is the vector-valued output corresponding to input `x[n]`. Requires `length(x) == length(y)`.

```APIDOC
## prepare_isotopic_multi_output_data (RowVecs)

### Description
Utility functionality to convert a collection of `N = length(x)` inputs `x` and output vectors `y` (efficiently represented by a `RowVecs`) into a format suitable for use with multi-output kernels. `y[n]` is the vector-valued output corresponding to the input `x[n]`. Consequently, it is necessary that `length(x) == length(y)`.

### Method Signature
```julia
prepare_isotopic_multi_output_data(x::AbstractVector, y::RowVecs)
```

### Example
```julia
x = [1.0, 2.0, 3.0]
Y = [1.1 1.2; 2.1 2.2; 3.1 3.2]
inputs, outputs = prepare_isotopic_multi_output_data(x, RowVecs(Y))

# Expected output for inputs:
# 6-element KernelFunctions.MOInputIsotopicByOutputs{Float64, Vector{Float64}, Int64}:
#  (1.0, 1)
#  (2.0, 1)
#  (3.0, 1)
#  (1.0, 2)
#  (2.0, 2)
#  (3.0, 2)

# Expected output for outputs:
# 6-element Vector{Float64}:
#  1.1
#  2.1
#  3.1
#  1.2
#  2.2
#  3.2
```
```

--------------------------------

### kernelkronmat

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/api

Constructs Kronecker product matrices using `Kronecker.jl`. It can create a `KroneckerPower` or a `KroneckerProduct` based on the input dimensions.

```APIDOC
## kernelkronmat

### Description
Constructs Kronecker product matrices.

### Function Signature 1
```
kernelkronmat(κ::Kernel, X::AbstractVector{<:Real}, dims::Int) -> KroneckerPower
```

### Details 1
Return a `KroneckerPower` matrix on the `D`-dimensional input grid constructed by ⊗i=1DX\otimes_{i=1}^D X⊗i=1DX, where `D` is given by `dims`.

### Requirements 1
Requires `Kronecker.jl` and for `iskroncompatible(κ)` to return `true`.

### Function Signature 2
```
kernelkronmat(κ::Kernel, X::AbstractVector{<:AbstractVector}) -> KroneckerProduct
```

### Details 2
Returns a `KroneckerProduct` matrix on the grid built with the collection of vectors {Xi}i=1D\{X_i\}_{i=1}^D{Xi}i=1D: ⊗i=1DXi\otimes_{i=1}^D X_i⊗i=1DXi.

### Requirements 2
Requires `Kronecker.jl` and for `iskroncompatible(κ)` to return `true`.
```

--------------------------------

### Gabor Kernel Construction

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/kernels

Constructs a Gabor kernel by applying transformations to inputs of underlying squared exponential and cosine kernels. Useful for capturing periodic patterns.

```julia
gaborkernel(;
    sqexponential_transform=IdentityTransform(), cosine_tranform=IdentityTransform()
)
```

--------------------------------

### Implement Linear Regression

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/kernel-ridge-regression

Implements the linear regression model using the least-squares estimator for training inputs and predicts values at test inputs.

```julia
function linear_regression(X, y, Xstar)
    weights = (X' * X) \ (X' * y)
    return Xstar * weights
end;
```

--------------------------------

### Construct MOInput Manually

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/api

Manually constructs an MOInput type for multi-dimensional output data, representing inputs for each output dimension.

```julia
julia> x = [1, 2, 3];

julia> MOInput(x, 2)
6-element KernelFunctions.MOInputIsotopicByOutputs{Int64, Vector{Int64}, Int64}:
 (1, 1)
 (2, 1)
 (3, 1)
 (1, 2)
 (2, 2)
 (3, 2)
```

--------------------------------

### Create a Kernel Matrix

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/userguide

Generate a kernel matrix for a collection of Real-valued inputs using the kernelmatrix function.

```julia
k = SqExponentialKernel()
x = rand(10)
kernelmatrix(k, x) # 10x10 matrix
```

--------------------------------

### Visualize Linear Regression Fit

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/kernel-ridge-regression

Applies the linear regression function to the toy data and visualizes the fit against the original observations and the true function.

```julia
y_pred = linear_regression(x_train, y_train, x_test)
scatter(x_train, y_train; label="observations")
plot!(x_test, y_pred; label="linear fit")
```

--------------------------------

### Apply PeriodicTransform to a Single Input

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/transform

Applies a PeriodicTransform to a single input `x`, mapping it onto the unit circle using sine and cosine functions with a given frequency `f`.

```julia
f = rand(); t = PeriodicTransform(f); x = rand();

t(x) == [sinpi(2 * f * x), cospi(2 * f * x)]
```

--------------------------------

### iskroncompatible

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/api

Checks if a given kernel `k` is compatible with Kronecker constructions. A kernel is compatible if it satisfies the product property for all inputs.

```APIDOC
## iskroncompatible

### Description
Determine whether kernel `k` is compatible with Kronecker constructions such as `kernelkronmat`.

### Function Signature
```
iskroncompatible(k::Kernel)
```

### Details
The function returns `false` by default. If `k` is compatible, it must satisfy for all ``x, x' \in \mathbb{R}^D`: k(x,x′)=∏i=1Dk(xi,xi′).
```

--------------------------------

### Benchmark Optimization Step

Source: https://juliagaussianprocesses.github.io/KernelFunctions.jl/stable/examples/train-kernel-parameters

Benchmarks a single step of the optimization process using AdaGrad. This measures the time and memory allocation for calculating gradients and updating the parameters, providing insights into the computational cost per step.

```julia
@benchmark let
    θ = flat_θ[:]
    opt = Optimise.AdaGrad(0.5)
    grads = (Zygote.gradient(loss ∘ unflatten, θ))[1]
    Optimise.update!(opt, θ, grads)
end
```
Tokens:
Raw