CliffordLayers

https://microsoft.github.io/cliffordlayers/
Admin
CliffordLayers is a deep learning framework providing neural network layers and operations inspired...
Tokens:51,162
Snippets:149
Trust Score:9.7
Update:1 month ago
Show doc for...
Tokens:
Raw
### Make an editable install

Source: https://microsoft.github.io/cliffordlayers/install

Installs the package in editable mode for development.

```bash
pip install -e .

```

--------------------------------

### Install cliffordlayers with pip

Source: https://microsoft.github.io/cliffordlayers/install

Installs the cliffordlayers package from PyPI using pip.

```bash
pip install cliffordlayers

```

--------------------------------

### Create and activate conda environment

Source: https://microsoft.github.io/cliffordlayers/install

Creates a conda environment from the provided environment.yml file and activates it.

```bash
cd cliffordlayers
conda env create --file docker/environment.yml
conda activate cliffordlayers

```

--------------------------------

### get_octonion_kernel(w) - construct octonion kernel

Source: https://microsoft.github.io/cliffordlayers/reference/misc

Builds an octonion kernel from a weight tuple of length 8. The input must have shape (8, d_input, d_output, ...) and the output has shape (d_output * 8, d_input * 8, ...). Asserts that the weight list has exactly 8 elements.

```python
def get_octonion_kernel(w: Union[tuple, list, torch.Tensor, nn.Parameter, nn.ParameterList]) -> torch.Tensor:
    """Octonion kernels.

    Args:
        w (Union[tuple, list, torch.Tensor, nn.Parameter, nn.ParameterList]): Weight input of shape `(8, d~input~, d~output~, ...)`.


    Returns:
        (torch.Tensor):  Weight output of shape `(d~output~ * 8, d~input~ * 8, ...)`.
    """
    w = _w_assert(w)
    assert len(w) == 8
    k0 = torch.cat([w[0], -w[1], -w[2], -w[3], -w[4], -w[5], -w[6], w[7]], dim=1)
    k1 = torch.cat([w[1], w[0], w[4], w[5], -w[2], -w[3], -w[7], -w[6]], dim=1)
    k2 = torch.cat([w[2], -w[4], w[0], w[6], w[1], w[7], -w[3], w[5]], dim=1)
    k3 = torch.cat([w[3], -w[5], -w[6], w[0], -w[7], w[1], w[2], -w[4]], dim=1)
    k4 = torch.cat([w[4], w[2], -w[1], -w[7], w[0], w[6], -w[5], -w[3]], dim=1)
    k5 = torch.cat([w[5], w[3], w[7], -w[1], -w[6], w[0], w[4], w[2]], dim=1)
    k6 = torch.cat([w[6], -w[7], w[3], -w[2], w[5], -w[4], w[0], -w[1]], dim=1)
    k7 = torch.cat([w[7], w[6], -w[5], w[4], w[3], -w[2], w[1], w[0]], dim=1)
    k = torch.cat([k0, k1, k2, k3, k4, k5, k6, k7], dim=0)
    return k
```

--------------------------------

### Initialize a 3D Clifford FNO model

Source: https://microsoft.github.io/cliffordlayers/reference/models

Initializes a 3D Clifford FNO model using CliffordMaxwellNet3d with a CliffordFourierBasicBlock3d block. The model uses 4 input channels, 1 output channel, 16 hidden channels, GELU activation, and no normalization.

```python
import torch.nn.functional as F

from cliffordlayers.models.models_3d import (
    CliffordMaxwellNet3d,
    CliffordFourierBasicBlock3d,
)
model = CliffordMaxwellNet3d(
        g = [1, 1, 1],
        block = CliffordFourierBasicBlock3d,
        num_blocks = [1, 1, 1, 1],
        in_channels = 4,
        out_channels = 1,
        hidden_channels = 16,
        activation = F.gelu,
        norm = False,
    )
```

--------------------------------

### Build Docker container

Source: https://microsoft.github.io/cliffordlayers/install

Builds a Docker image named cliffordlayers from the docker directory.

```bash
cd cliffordlayers/docker
docker build -t cliffordlayers .
cd ..

```

--------------------------------

### Initialize 2D Clifford FNO with CliffordFourierBasicBlock2d

Source: https://microsoft.github.io/cliffordlayers/reference/models

Creates a 2D Clifford Fourier Neural Operator using CliffordFluidNet2d with CliffordFourierBasicBlock2d blocks configured via partialclass. Uses signature g = [-1, -1], 32 hidden channels, GELU activation, normalization disabled, and rotation disabled.

```python
import torch.nn.functional as F

from cliffordlayers.models.utils import partialclass
from cliffordlayers.models.basic.twod import (
    CliffordFluidNet2d,
    CliffordFourierBasicBlock2d,
)

model = CliffordFluidNet2d(
        g = [-1, -1],
        block = partialclass(
                "CliffordFourierBasicBlock2d", CliffordFourierBasicBlock2d, modes1=32, modes2=32
            ),
        num_blocks = [1, 1, 1, 1],
        in_channels = in_channels,
        out_channels = out_channels,
        hidden_channels = 32,
        activation = F.gelu,
        norm = False,
        rotation = False,
    )
```

--------------------------------

### Initialize 2D Clifford ResNet with CliffordFluidNet2d

Source: https://microsoft.github.io/cliffordlayers/reference/models

Creates a 2D Clifford ResNet using CliffordFluidNet2d with CliffordBasicBlock2d blocks. Uses signature g = [-1, -1], 32 hidden channels, GELU activation, normalization enabled, and rotation disabled.

```python
import torch.nn.functional as F

from cliffordlayers.models.basic.twod import (
    CliffordFluidNet2d,
    CliffordBasicBlock2d,
)

model = CliffordFluidNet2d(
        g = [-1, -1],
        block = CliffordBasicBlock2d,
        num_blocks = [2, 2, 2, 2],
        in_channels = in_channels,
        out_channels = out_channels,
        hidden_channels = 32,
        activation = F.gelu,
        norm = True,
        rotation = False,
    )
```

--------------------------------

### Run Docker container

Source: https://microsoft.github.io/cliffordlayers/install

Runs the cliffordlayers Docker container with GPU support, mounting the current directory and setting PYTHONPATH.

```bash
docker run --gpus all -it --rm --user $(id -u):$(id -g) \
    -v $(pwd):/code  --workdir /code -e PYTHONPATH=/code \
    cliffordlayers:latest

```

--------------------------------

### get_quaternion_rotation_kernel(w)

Source: https://microsoft.github.io/cliffordlayers/reference/misc

Builds a quaternion rotation kernel from a weight input containing quaternion components, a scaling tensor, and a zero kernel tensor, returning a weight tensor of dimension (d_output * 3, d_input * 4, ...).

```APIDOC
## get_quaternion_rotation_kernel(w)

### Description
Builds a quaternion rotation kernel from a weight input containing quaternion components, a scaling tensor, and a zero kernel tensor, returning a weight tensor of dimension `(d_output * 3, d_input * 4, ...)`.

### Method
Function call

### Endpoint
`get_quaternion_rotation_kernel(w)`

### Parameters
#### Input Parameters
- **w** (`Union[tuple, list, Tensor, Parameter, ParameterList]`) - Required - Weight input of shape `(6, d_input, d_output, ...)`. `w[0]`, `w[1]`, `w[2]`, `w[3]` are the quaternion w; tensors; `w[4]` is the scaling tensor; `w[5]` is the zero kernel tensor.

### Returns
- **Tensor** - Quaternion weight output of dimension `(d_output * 3, d_input * 4, ...)`.

### Source Code
```python
def get_quaternion_rotation_kernel(w: Union[tuple, list, torch.Tensor, nn.Parameter, nn.ParameterList]) -> torch.Tensor:
    """Quaternion rotation, taken mostly from <https://github.com/Orkis-Research/Pytorch-Quaternion-Neural-Networks>

    Args:
        w (Union[tuple, list, torch.Tensor, nn.Parameter, nn.ParameterList]): Weight input of shape `(6, d~input~, d~output~, ...)`.
                    `w[0]`, `w[1]`, `w[2]`, `w[3]` are the quaternion w;
                    tensors; `w[4]` is the scaling tensor; `w[5]` is the zero kernel tensor.

    Returns:
        (torch.Tensor): Quaternion weight output of dimension `(d~output * 3, d~input * 4, ...)`.
    """
    w = _w_assert(w)
    assert len(w) == 6
    square_1 = w[0] * w[0]
    square_2 = w[1] * w[1]
    square_3 = w[2] * w[2]
    square_4 = w[3] * w[3]

    norm = torch.sqrt(square_1 + square_2 + square_3 + square_4 + 0.0001)

    w1_n = w[0] / norm
    w2_n = w[1] / norm
    w3_n = w[2] / norm
    w4_n = w[3] / norm

    norm_factor = 2.0
    square_2 = norm_factor * (w2_n * w2_n)
    square_3 = norm_factor * (w3_n * w3_n)
    square_4 = norm_factor * (w4_n * w4_n)

    rot12 = norm_factor * w1_n * w2_n
    rot13 = norm_factor * w1_n * w3_n
    rot14 = norm_factor * w1_n * w4_n
    rot23 = norm_factor * w2_n * w3_n
    rot24 = norm_factor * w2_n * w4_n
    rot34 = norm_factor * w3_n * w4_n

    scale = w[4]
    zero_kernel = w[5]

    rot_kernel2 = torch.cat(
        [
            zero_kernel,
            scale * (1.0 - (square_3 + square_4)),
            scale * (rot23 - rot14),
            scale * (rot24 + rot13),
        ],
        dim=1,
    )
    rot_kernel3 = torch.cat(
        [
            zero_kernel,
            scale * (rot23 + rot14),
            scale * (1.0 - (square_2 + square_4)),
            scale * (rot34 - rot12),
        ],
        dim=1,
    )
    rot_kernel4 = torch.cat(
        [
            zero_kernel,
            scale * (rot24 - rot13),
            scale * (rot34 + rot12),
            scale * (1.0 - (square_2 + square_3)),
        ],
        dim=1,
    )

    k = torch.cat([rot_kernel2, rot_kernel3, rot_kernel4], dim=0)
    return k
```
```

--------------------------------

### CliffordMaxwellNet3d forward pass with padding

Source: https://microsoft.github.io/cliffordlayers/reference/models

Shows the forward method of CliffordMaxwellNet3d. It handles non-periodic boundaries by padding the input, applies residual layers, then decodes. The padding is removed before the final output.

```python
# Embed for non-periodic boundaries.
        if self.padding > 0:
            B_dim, C_dim, *D_dims, I_dim = range(len(x.shape))
            x = x.permute(B_dim, I_dim, C_dim, *D_dims)
            x = F.pad(x, [0, self.padding, 0, self.padding, 0, self.padding])
            B_dim, I_dim, C_dim, *D_dims = range(len(x.shape))
            x = x.permute(B_dim, C_dim, *D_dims, I_dim)

        # Apply residual layers.
        for layer in self.layers:
            x = layer(x)

        # Decoding layer.
        if self.padding > 0:
            B_dim, C_dim, *D_dims, I_dim = range(len(x.shape))
            x = x.permute(B_dim, I_dim, C_dim, *D_dims)
            x = x[..., : -self.padding, : -self.padding, : -self.padding]
            B_dim, I_dim, C_dim, *D_dims = range(len(x.shape))
            x = x.permute(B_dim, C_dim, *D_dims, I_dim)

        # Output layer.
        x = self.decoder(x)
        return x
```

--------------------------------

### get_quaternion_rotation_kernel(w) - construct quaternion rotation kernel

Source: https://microsoft.github.io/cliffordlayers/reference/misc

Builds a quaternion rotation kernel from a weight input of shape `(6, d~input~, d~output~, ...)`. Expects `w[0]`-`w[3]` as quaternion tensors, `w[4]` as scaling tensor, and `w[5]` as zero kernel tensor; asserts the input has exactly 6 components. Returns a quaternion weight output of dimension `(d~output * 3, d~input * 4, ...)`. The implementation is adapted from Pytorch-Quaternion-Neural-Networks.

```python
def get_quaternion_rotation_kernel(w: Union[tuple, list, torch.Tensor, nn.Parameter, nn.ParameterList]) -> torch.Tensor:
    """Quaternion rotation, taken mostly from <https://github.com/Orkis-Research/Pytorch-Quaternion-Neural-Networks>

    Args:
        w (Union[tuple, list, torch.Tensor, nn.Parameter, nn.ParameterList]): Weight input of shape `(6, d~input~, d~output~, ...)`.
                    `w[0]`, `w[1]`, `w[2]`, `w[3]` are the quaternion w;
                    tensors; `w[4]` is the scaling tensor; `w[5]` is the zero kernel tensor.

    Returns:
        (torch.Tensor): Quaternion weight output of dimension `(d~output * 3, d~input * 4, ...)`.
    """
    w = _w_assert(w)
    assert len(w) == 6
    square_1 = w[0] * w[0]
    square_2 = w[1] * w[1]
    square_3 = w[2] * w[2]
    square_4 = w[3] * w[3]

    norm = torch.sqrt(square_1 + square_2 + square_3 + square_4 + 0.0001)

    w1_n = w[0] / norm
    w2_n = w[1] / norm
    w3_n = w[2] / norm
    w4_n = w[3] / norm

    norm_factor = 2.0
    square_2 = norm_factor * (w2_n * w2_n)
    square_3 = norm_factor * (w3_n * w3_n)
    square_4 = norm_factor * (w4_n * w4_n)

    rot12 = norm_factor * w1_n * w2_n
    rot13 = norm_factor * w1_n * w3_n
    rot14 = norm_factor * w1_n * w4_n
    rot23 = norm_factor * w2_n * w3_n
    rot24 = norm_factor * w2_n * w4_n
    rot34 = norm_factor * w3_n * w4_n

    scale = w[4]
    zero_kernel = w[5]

    rot_kernel2 = torch.cat(
        [
            zero_kernel,
            scale * (1.0 - (square_3 + square_4)),
            scale * (rot23 - rot14),
            scale * (rot24 + rot13),
        ],
        dim=1,
    )
    rot_kernel3 = torch.cat(
        [
            zero_kernel,
            scale * (rot23 + rot14),
            scale * (1.0 - (square_2 + square_4)),
            scale * (rot34 - rot12),
        ],
        dim=1,
    )
    rot_kernel4 = torch.cat(
        [
            zero_kernel,
            scale * (rot24 - rot13),
            scale * (rot34 + rot12),
            scale * (1.0 - (square_2 + square_3)),
        ],
        dim=1,
    )

    k = torch.cat([rot_kernel2, rot_kernel3, rot_kernel4], dim=0)
    return k

```

--------------------------------

### CliffordG3SumVSiLU: vector SiLU with sum aggregation

Source: https://microsoft.github.io/cliffordlayers/reference/modules

Applies vector SiLU using the sum aggregation to vectors in G3. The forward method computes the sigmoid of the sum over the last dimension and multiplies it by the input.

```python
class CliffordG3SumVSiLU(nn.Module):
    """
    A module that applies the vector SiLU using vector sum to vectors in G3.
    """

    def __init__(self):
        super().__init__()

    def forward(self, input):
        return torch.sigmoid(input.sum(-1, keepdim=True)) * input

```

--------------------------------

### get_complex_kernel(w) - construct complex kernel

Source: https://microsoft.github.io/cliffordlayers/reference/misc

Builds a complex-valued kernel from a weight tuple of length 2. The input must have shape (2, d_input, d_output, ...) and the output has shape (d_output * 2, d_input * 2, ...). Asserts that the weight list has exactly 2 elements.

```python
def get_complex_kernel(w: Union[tuple, list, torch.Tensor, nn.Parameter, nn.ParameterList]) -> torch.Tensor:
    """Complex kernel.

    Args:
        w (Union[tuple, list, torch.Tensor, nn.Parameter, nn.ParameterList]): Weight input of shape `(2, d~input~, d~output~, ...)`.

    Returns:
        (torch.Tensor):  Weight output of shape `(d~output~ * 2, d~input~ * 2, ...)`.
    """
    w = _w_assert(w)
    assert len(w) == 2
    k0 = torch.cat([w[0], -w[1]], dim=1)
    k1 = torch.cat([w[1], w[0]], dim=1)
    k = torch.cat([k0, k1], dim=0)
    return k
```

--------------------------------

### CliffordG3UNet2d forward pass and upsampling path

Source: https://microsoft.github.io/cliffordlayers/reference/models

Shows the construction of the upsampling path and the forward method for the CliffordG3UNet2d model. The upsampling path builds a sequence of CliffordG3UpBlock modules with skip connections and CliffordUpsample layers. The forward method expects a 5D tensor, applies the downsampling path, middle block, and then the upsampling path with skip connections, followed by activation, normalization, and a final convolution.

```python
# Increasing resolution
        up = []
        # Number of channels
        in_channels = out_channels
        # For each resolution
        for i in reversed(range(n_resolutions)):
            # `n_blocks` at the same resolution
            out_channels = in_channels
            for _ in range(n_blocks):
                up.append(
                    CliffordG3UpBlock(
                        in_channels,
                        out_channels,
                        activation=activation,
                        norm=norm,
                        prenorm=prenorm,
                        num_groups=num_groups,
                    )
                )
            # Final block to reduce the number of channels
            out_channels = in_channels // ch_mults[i]
            up.append(
                CliffordG3UpBlock(
                    in_channels,
                    out_channels,
                    activation=activation,
                    norm=norm,
                    prenorm=prenorm,
                )
            )
            in_channels = out_channels
            # Up sample at all resolutions except last
            if i > 0:
                up.append(
                    CliffordUpsample(
                        in_channels,
                    )
                )

        # Combine the set of modules
        self.up = nn.ModuleList(up)

        self.activation = get_activation(activation, out_channels)

        if norm:
            self.norm = CliffordG3GroupNorm(num_groups, out_channels, 3)
        else:
            self.norm = nn.Identity()

        # Output layers
        self.conv2 = CliffordG3Conv2d(
            in_channels,
            self.out_channels,
            kernel_size=3,
            padding=1,
        )

    def forward(self, x: torch.Tensor):
        assert x.dim() == 5

        x = self.conv1(x)

        h = [x]
        for m in self.down:
            x = m(x)
            h.append(x)

        x = self.middle(x)

        for m in self.up:
            if isinstance(m, CliffordUpsample):
                x = m(x)
            else:
                # Get the skip connection from first half of U-Net and concatenate
                s = h.pop()
                x = torch.cat((x, s), dim=1)
                x = m(x)

        x = self.activation(self.norm(x))
        x = self.conv2(x)
        return x
```

--------------------------------

### Inverse FFT and spatial-domain reconstruction in CliffordG3Conv2d

Source: https://microsoft.github.io/cliffordlayers/reference/modules

Converts the frequency-domain output into four dual multivectors, applies an inverse FFT to each, and stacks the real and imaginary parts to reconstruct the spatial-domain multivector. The pairing order (0,7), (1,6), (2,5), (3,4) is required to match the Clifford algebra basis ordering.

```python
out_dual_1 = torch.view_as_complex(torch.stack((out_ft[..., 0], out_ft[..., 7]), dim=-1))
        out_dual_2 = torch.view_as_complex(torch.stack((out_ft[..., 1], out_ft[..., 6]), dim=-1))
        out_dual_3 = torch.view_as_complex(torch.stack((out_ft[..., 2], out_ft[..., 5]), dim=-1))
        out_dual_4 = torch.view_as_complex(torch.stack((out_ft[..., 3], out_ft[..., 4]), dim=-1))
        dual_1_ifft = torch.fft.ifftn(out_dual_1, s=(out_dual_1.size(-3), out_dual_1.size(-2), out_dual_1.size(-1)))
        dual_2_ifft = torch.fft.ifftn(out_dual_2, s=(out_dual_2.size(-3), out_dual_2.size(-2), out_dual_2.size(-1)))
        dual_3_ifft = torch.fft.ifftn(out_dual_3, s=(out_dual_3.size(-3), out_dual_3.size(-2), out_dual_3.size(-1)))
        dual_4_ifft = torch.fft.ifftn(out_dual_4, s=(out_dual_4.size(-3), out_dual_4.size(-2), out_dual_4.size(-1)))

        # Finally, return to the multivector in the spatial domain.
        output = torch.stack(
            (
                dual_1_ifft.real,
                dual_2_ifft.real,
                dual_3_ifft.real,
                dual_4_ifft.real,
                dual_4_ifft.imag,
                dual_3_ifft.imag,
                dual_2_ifft.imag,
                dual_1_ifft.imag,
            ),
            dim=-1,
        )

        return output
```

--------------------------------

### get_quaternion_kernel(w) - construct quaternion kernel

Source: https://microsoft.github.io/cliffordlayers/reference/misc

Builds a quaternion kernel from a weight input of shape `(4, d~input~, d~output~, ...)`. Asserts the input has exactly 4 components and returns a tensor of shape `(d~output~ * 4, d~input~ * 4, ...)`. The kernel is constructed by concatenating four block matrices that encode quaternion multiplication.

```python
def get_quaternion_kernel(w: Union[tuple, list, torch.Tensor, nn.Parameter, nn.ParameterList]) -> torch.Tensor:
    """Quaternion kernel.

    Args:
        w (Union[tuple, list, torch.Tensor, nn.Parameter, nn.ParameterList]): Weight input of shape `(4, d~input~, d~output~, ...)`.


    Returns:
        (torch.Tensor):  Weight output of shape `(d~output~ * 4, d~input~ * 4, ...)`.
    """
    w = _w_assert(w)
    assert len(w) == 4
    k0 = torch.cat([w[0], -w[1], -w[2], -w[3]], dim=1)
    k1 = torch.cat([w[1], w[0], w[3], -w[2]], dim=1)
    k2 = torch.cat([w[2], -w[3], w[0], w[1]], dim=1)
    k3 = torch.cat([w[3], w[2], -w[1], w[0]], dim=1)
    k = torch.cat([k0, k1, k2, k3], dim=0)
    return k

```

--------------------------------

### get_3d_clifford_kernel(w, g) - construct 3D Clifford kernel

Source: https://microsoft.github.io/cliffordlayers/reference/misc

Builds the 3D Clifford kernel weight tensor from an 8-element weight list and a 3-element signature tensor. Requires the weight input to have exactly 8 elements and the signature to have 3 elements; raises ValueError on wrong encoding/decoding options. Returns the number of output blades (8) and the concatenated kernel tensor of shape (d_output * 8, d_input * 8, ...).

```python
def get_3d_clifford_kernel(
    w: Union[tuple, list, torch.Tensor, nn.Parameter, nn.ParameterList], g: torch.Tensor
) -> Tuple[int, torch.Tensor]:
    """Clifford kernel for 3d Clifford algebras, g = [-1, -1, -1] corresponds to an octonion kernel.

    Args:
        w (Union[tuple, list, torch.Tensor, nn.Parameter, nn.ParameterList]): Weight input of shape `(8, d~input~, d~output~, ...)`.
        g (torch.Tensor): Signature of Clifford algebra.

    Raises:
        ValueError: Wrong encoding/decoding options provided.

    Returns:
        (Tuple[int, torch.Tensor]): Number of output blades, weight output of dimension `(d~output~ * 8, d~input~ * 8, ...)`.
    """
    assert isinstance(g, torch.Tensor)
    assert g.numel() == 3
    w = _w_assert(w)
    assert len(w) == 8

    k0 = torch.cat(
        [
            w[0],
            w[1] * g[0],
            w[2] * g[1],
            w[3] * g[2],
            -w[4] * g[0] * g[1],
            -w[5] * g[0] * g[2],
            -w[6] * g[1] * g[2],
            -w[7] * g[0] * g[1] * g[2],
        ],
        dim=1,
    )
    k1 = torch.cat(
        [w[1], w[0], -w[4] * g[1], -w[5] * g[2], w[2] * g[1], w[3] * g[2], -w[7] * g[1] * g[2], -w[6] * g[2] * g[1]],
        dim=1,
    )
    k2 = torch.cat(
        [w[2], w[4] * g[0], w[0], -w[6] * g[2], -w[1] * g[0], w[7] * g[0] * g[2], w[3] * g[2], w[5] * g[2] * g[0]],
        dim=1,
    )
    k3 = torch.cat(
        [w[3], w[5] * g[0], w[6] * g[1], w[0], -w[7] * g[0] * g[1], -w[1] * g[0], -w[2] * g[1], -w[4] * g[0] * g[1]],
        dim=1,
    )
    k4 = torch.cat([w[4], w[2], -w[1], g[2] * w[7], w[0], -w[6] * g[2], w[5] * g[2], w[3] * g[2]], dim=1)
    k5 = torch.cat([w[5], w[3], -w[7] * g[1], -w[1], w[6] * g[1], w[0], -w[4] * g[1], -w[2] * g[1]], dim=1)
    k6 = torch.cat([w[6], w[7] * g[0], w[3], -w[2], -w[5] * g[0], w[4] * g[0], w[0], w[1] * g[0]], dim=1)
    k7 = torch.cat([w[7], w[6], -w[5], w[4], w[3], -w[2], w[1], w[0]], dim=1)
    k = torch.cat([k0, k1, k2, k3, k4, k5, k6, k7], dim=0)
    return 8, k
```

--------------------------------

### get_quaternion_kernel(w)

Source: https://microsoft.github.io/cliffordlayers/reference/misc

Constructs a quaternion kernel from a weight input of shape (4, d_input, d_output, ...) and returns a rearranged tensor of shape (d_output * 4, d_input * 4, ...).

```APIDOC
## get_quaternion_kernel(w)

### Description
Constructs a quaternion kernel by rearranging the four quaternion components into a block matrix, producing a weight tensor of shape `(d_output * 4, d_input * 4, ...)`.

### Method
Function call

### Endpoint
`get_quaternion_kernel(w)`

### Parameters
#### Input Parameters
- **w** (`Union[tuple, list, Tensor, Parameter, ParameterList]`) - Required - Weight input of shape `(4, d_input, d_output, ...)`.

### Returns
- **Tensor** - Weight output of shape `(d_output * 4, d_input * 4, ...)`.

### Source Code
```python
def get_quaternion_kernel(w: Union[tuple, list, torch.Tensor, nn.Parameter, nn.ParameterList]) -> torch.Tensor:
    """Quaternion kernel.

    Args:
        w (Union[tuple, list, torch.Tensor, nn.Parameter, nn.ParameterList]): Weight input of shape `(4, d~input~, d~output~, ...)`.


    Returns:
        (torch.Tensor):  Weight output of shape `(d~output~ * 4, d~input~ * 4, ...)`.
    """
    w = _w_assert(w)
    assert len(w) == 4
    k0 = torch.cat([w[0], -w[1], -w[2], -w[3]], dim=1)
    k1 = torch.cat([w[1], w[0], w[3], -w[2]], dim=1)
    k2 = torch.cat([w[2], -w[3], w[0], w[1]], dim=1)
    k3 = torch.cat([w[3], w[2], -w[1], w[0]], dim=1)
    k = torch.cat([k0, k1, k2, k3], dim=0)
    return k
```
```

--------------------------------

### Initialize 2D rotational Clifford ResNet with CliffordNet2d

Source: https://microsoft.github.io/cliffordlayers/reference/models

Creates a 2D rotational Clifford ResNet using CliffordNet2d with CliffordBasicBlock2d blocks. Uses signature g = [-1, -1], 32 hidden channels, GELU activation, normalization enabled, and rotation enabled.

```python
import torch.nn.functional as F

from cliffordlayers.models.basic.twod import (
    CliffordFluidNet2d,
    CliffordBasicBlock2d,
)

model = CliffordNet2d(
        g = [-1, -1],
        block = CliffordBasicBlock2d,
        num_blocks = [2, 2, 2, 2],
        in_channels = in_channels,
        out_channels = out_channels,
        hidden_channels = 32,
        activation = F.gelu,
        norm = True,
        rotation = True,
    )
```

--------------------------------

### CliffordG3MeanVSiLU: vector SiLU with mean aggregation

Source: https://microsoft.github.io/cliffordlayers/reference/modules

Applies vector SiLU using the mean aggregation to vectors in G3. The forward method computes the sigmoid of the mean over the last dimension and multiplies it by the input.

```python
class CliffordG3MeanVSiLU(nn.Module):
    """
    A module that applies the vector SiLU using vector mean to vectors in G3.
    """

    def __init__(self):
        super().__init__()

    def forward(self, input):
        return torch.sigmoid(input.mean(-1, keepdim=True)) * input

```

--------------------------------

### CliffordBasicBlock2d class definition and forward method

Source: https://microsoft.github.io/cliffordlayers/reference/models

Defines a 2D basic residual block using two CliffordConv2d layers with optional Clifford group normalization and rotation. The forward method applies activation and normalization before each convolution and adds the input via a residual connection.

```python
    def __init__(
        self,
        g: Union[tuple, list, torch.Tensor],
        in_channels: int,
        out_channels: int,
        activation: Callable = F.gelu,
        kernel_size: int = 3,
        stride: int = 1,
        padding: int = 1,
        rotation: bool = False,
        norm: bool = False,
        num_groups: int = 1,
    ) -> None:
        super().__init__()
        self.conv1 = CliffordConv2d(
            g,
            in_channels,
            out_channels,
            kernel_size=kernel_size,
            stride=stride,
            padding=padding,
            bias=True,
            rotation=rotation,
        )
        self.conv2 = CliffordConv2d(
            g,
            out_channels,
            out_channels,
            kernel_size=kernel_size,
            stride=stride,
            padding=padding,
            bias=True,
            rotation=rotation,
        )
        self.norm1 = CliffordGroupNorm2d(g, num_groups, in_channels) if norm else nn.Identity()
        self.norm2 = CliffordGroupNorm2d(g, num_groups, out_channels) if norm else nn.Identity()
        self.activation = activation

    def __repr__(self):
        return "CliffordBasicBlock2d"

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        out = self.conv1(self.activation(self.norm1(x)))
        out = self.conv2(self.activation(self.norm2(out)))
        return out + x
```

--------------------------------

### MultiVectorAct: multivector activation module

Source: https://microsoft.github.io/cliffordlayers/reference/modules

Applies multivector activations to the input. Supports aggregation methods "linear", "sum", and "mean"; defaults to "linear". Raises ValueError for unsupported aggregation methods.

```python
class MultiVectorAct(nn.Module):
    """
    A module to apply multivector activations to the input.

    Args:
        channels (int): Number of channels in the input.
        algebra: The algebra object that defines the geometric product.
        input_blades (list, tuple): The nonnegative input blades.
        kernel_blades (list, tuple, optional): The blades that will be used to compute the activation. Defaults to all input blades.
        agg (str, optional): The aggregation method to be used. Options include "linear", "sum", and "mean". Defaults to "linear".
    """

    def __init__(self, channels, algebra, input_blades, kernel_blades=None, agg="linear"):
        super().__init__()
        self.algebra = algebra
        self.input_blades = tuple(input_blades)
        if kernel_blades is not None:
            self.kernel_blades = tuple(kernel_blades)
        else:
            self.kernel_blades = self.input_blades

        if agg == "linear":
            self.conv = nn.Conv1d(channels, channels, kernel_size=len(self.kernel_blades), groups=channels)
        self.agg = agg

    def forward(self, input):
        v = self.algebra.embed(input, self.input_blades)
        if self.agg == "linear":
            v = v * torch.sigmoid(self.conv(v[..., self.kernel_blades]))
        elif self.agg == "sum":
            v = v * torch.sigmoid(v[..., self.kernel_blades].sum(dim=-1, keepdim=True))
        elif self.agg == "mean":
            v = v * torch.sigmoid(v[..., self.kernel_blades].mean(dim=-1, keepdim=True))
        else:
            raise ValueError(f"Aggregation {self.agg} not implemented.")
        v = self.algebra.get(v, self.input_blades)
        return v

```

--------------------------------

### Clone the cliffordlayers repository

Source: https://microsoft.github.io/cliffordlayers/install

Clones the cliffordlayers repository from GitHub for development.

```bash
git clone https://github.com/microsoft/cliffordlayers

```

--------------------------------

### CliffordLinear class implementation

Source: https://microsoft.github.io/cliffordlayers/reference/modules

Defines the CliffordLinear module. The constructor validates the Clifford signature dimension (1, 2, or 3) and raises NotImplementedError for other dimensions. The forward method checks that the input has the expected number of blades and raises ValueError otherwise. Weight initialization uses Kaiming uniform with bounds adjusted by the number of blades.

```python
class CliffordLinear(nn.Module):
    """Clifford linear layer.

    Args:
        g (Union[List, Tuple]): Clifford signature tensor.
        in_channels (int): Number of input channels.
        out_channels (int): Number of output channels.
        bias (bool, optional): If True, adds a learnable bias to the output. Defaults to True.

    """

    def __init__(
        self,
        g,
        in_channels: int,
        out_channels: int,
        bias: bool = True,
    ) -> None:
        super().__init__()
        sig = CliffordSignature(g)

        self.register_buffer("g", sig.g)
        self.dim = sig.dim
        self.n_blades = sig.n_blades

        if self.dim == 1:
            self._get_kernel = get_1d_clifford_kernel
        elif self.dim == 2:
            self._get_kernel = get_2d_clifford_kernel
        elif self.dim == 3:
            self._get_kernel = get_3d_clifford_kernel
        else:
            raise NotImplementedError(
                f"Clifford linear layers are not implemented for {self.dim} dimensions. Wrong Clifford signature."
            )

        self.in_channels = in_channels
        self.out_channels = out_channels
        self.weight = nn.Parameter(torch.empty(self.n_blades, out_channels, in_channels))

        if bias:
            self.bias = nn.Parameter(torch.empty(self.n_blades, out_channels))
        else:
            self.register_parameter("bias", None)

        self.reset_parameters()

    def reset_parameters(self):
        # Initialization of the Clifford linear weight and bias tensors.
        # The number of blades is taken into account when calculated the bounds of Kaiming uniform.
        nn.init.kaiming_uniform_(
            self.weight.view(self.out_channels, self.in_channels * self.n_blades),
            a=math.sqrt(5),
        )
        if self.bias is not None:
            fan_in, _ = nn.init._calculate_fan_in_and_fan_out(
                self.weight.view(self.out_channels, self.in_channels * self.n_blades)
            )
            bound = 1 / math.sqrt(fan_in)
            nn.init.uniform_(self.bias, -bound, bound)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        # Reshape x such that the Clifford kernel can be applied.
        B, _, I = x.shape
        if not (I == self.n_blades):
            raise ValueError(f"Input has {I} blades, but Clifford layer expects {self.n_blades}.")
        B_dim, C_dim, I_dim = range(len(x.shape))
        x = x.permute(B_dim, -1, C_dim)
        x = x.reshape(B, -1)
        # Get Clifford kernel, apply it.
        _, weight = self._get_kernel(self.weight, self.g)
        output = F.linear(x, weight, self.bias.view(-1))
        # Reshape back.
        output = output.view(B, I, -1)
        B_dim, I_dim, C_dim = range(len(output.shape))
        output = output.permute(B_dim, C_dim, I_dim)
        return output
```

--------------------------------

### CliffordSpectralConv3d class definition

Source: https://microsoft.github.io/cliffordlayers/reference/modules

Defines the 3D Clifford Fourier layer as a PyTorch module. The constructor validates that the Clifford algebra signature is 3D, stores the number of blades, and initializes weight parameters only when multiply is True. The forward method checks that the input has the expected number of blades and raises a ValueError otherwise.

```python
class CliffordSpectralConv3d(nn.Module):
    """3d Clifford Fourier layer.
    Performs following three steps:
        1. Clifford Fourier transform over the multivector of 3d Clifford algebras, based on complex Fourier transforms using [pytorch.fft.fftn](https://pytorch.org/docs/stable/generated/torch.fft.fftn.html#torch.fft.fftn).
        2. Weight multiplication in the Clifford Fourier space using the geometric product.
        3. Inverse Clifford Fourier transform, based on inverse complex Fourier transforms using [pytorch.fft.ifftn](https://pytorch.org/docs/stable/generated/torch.fft.fftn.html#torch.fft.ifftn).

    Args:
        g ((Union[tuple, list, torch.Tensor]): Signature of Clifford algebra.
        in_channels (int): Number of input channels.
        out_channels (int): Number of output channels.
        modes1 (int): Number of non-zero Fourier modes in the first dimension.
        modes2 (int): Number of non-zero Fourier modes in the second dimension.
        modes3 (int): Number of non-zero Fourier modes in the second dimension.
        multiply (bool): Multipliation in the Fourier space. If set to False this class only crops high-frequency modes.

    """

    def __init__(
        self,
        g: Union[tuple, list, torch.Tensor],
        in_channels: int,
        out_channels: int,
        modes1: int,
        modes2: int,
        modes3: int,
        multiply: bool = True,
    ) -> None:
        super().__init__()
        sig = CliffordSignature(g)
        self.g = sig.g
        self.dim = sig.dim
        if self.dim != 3:
            raise ValueError("g must be a 3D Clifford algebra")
        self.n_blades = sig.n_blades

        self.in_channels = in_channels
        self.out_channels = out_channels
        self.modes1 = modes1
        self.modes2 = modes2
        self.modes3 = modes3
        self.multiply = multiply

        # Initialize weight parameters.
        if self.multiply:
            scale = 1 / (in_channels * out_channels)
            self.weights = nn.Parameter(
                scale
                * torch.rand(
                    8,
                    out_channels,
                    in_channels,
                    self.modes1 * 2,
                    self.modes2 * 2,
                    self.modes3 * 2,
                    dtype=torch.float32,
                )
            )

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        # Reshape x such that FFT can be applied to dual pairs.
        B, _, *D, I = x.shape
        *_, I = x.shape
        if not (I == self.n_blades):
            raise ValueError(f"Input has {I} blades, but Clifford layer expects {self.n_blades}.")
```

--------------------------------

### CliffordSpectralConv3d

Source: https://microsoft.github.io/cliffordlayers/reference/modules

Documentation for the CliffordSpectralConv3d class, a 3D Clifford Fourier layer that performs Clifford Fourier transform, weight multiplication in Fourier space, and inverse Clifford Fourier transform.

```APIDOC
## CliffordSpectralConv3d

### Description
3d Clifford Fourier layer. Performs the following three steps: 1. Clifford Fourier transform over the multivector of 3d Clifford algebras, based on complex Fourier transforms using pytorch.fft.fftn. 2. Weight multiplication in the Clifford Fourier space using the geometric product. 3. Inverse Clifford Fourier transform, based on inverse complex Fourier transforms using pytorch.fft.ifftn.

### Constructor

#### Parameters
- **g** (Union[tuple, list, Tensor]) - Required - Signature of Clifford algebra.
- **in_channels** (int) - Required - Number of input channels.
- **out_channels** (int) - Required - Number of output channels.
- **modes1** (int) - Required - Number of non-zero Fourier modes in the first dimension.
- **modes2** (int) - Required - Number of non-zero Fourier modes in the second dimension.
- **modes3** (int) - Required - Number of non-zero Fourier modes in the second dimension.
- **multiply** (bool) - Optional, default: True - Multiplication in the Fourier space. If set to False this class only crops high-frequency modes.

### Methods

#### forward(x: torch.Tensor) -> torch.Tensor
Performs the forward pass of the Clifford Fourier layer.

##### Parameters
- **x** (torch.Tensor) - Required - Input tensor.

##### Returns
- **torch.Tensor** - Output tensor.

##### Raises
- **ValueError** - If the input has a number of blades that does not match the expected number of blades for the Clifford algebra.
```
Tokens:
Raw