### Clone Repository and Setup

Source: https://zeta.apac.ai/en/latest/index

General steps to clone the Zeta repository and set up the Python environment for installation.

```shell
git clone <repository_url>
cd zeta
python3 -m venv venv
source venv/bin/activate
```

```shell
git clone <repository_url>
cd zeta
poetry install
```

--------------------------------

### Install Zeta with Poetry

Source: https://zeta.apac.ai/en/latest/index

Instructions for installing Zeta using Poetry, covering project setup and package installation.

```shell
poetry install zeta
```

```shell
poetry install zeta"desktop"
```

--------------------------------

### Install PyTorch

Source: https://zeta.apac.ai/en/latest/zeta/quant/niva

This command installs the PyTorch library, which is a prerequisite for using the Niva module. Ensure you have a compatible version installed.

```Shell
pip install torch torchvision torchaudio
```

--------------------------------

### MkDocs Serve Command Output Example

Source: https://zeta.apac.ai/en/latest/contributing

An example of the typical output when running the `mkdocs serve` command. It shows build progress, directory watching, and the local server address.

```log
INFO     -  Building documentation...
INFO     -  Cleaning site directory
INFO     -  Documentation built in 0.19 seconds
INFO     -  [09:28:33] Watching paths for changes: 'docs', 'mkdocs.yml'
INFO     -  [09:28:33] Serving on http://127.0.0.1:8000/
INFO     -  [09:28:37] Browser connected: http://127.0.0.1:8000/
```

--------------------------------

### SentencePieceTokenizer Initialization Example

Source: https://zeta.apac.ai/en/latest/zeta/tokenizers/sentencepiece

Example of how to initialize the SentencePieceTokenizer with a model path.

```python
from zeta.tokenizers import SentencePieceTokenizer

# Assuming 'path/to/your/model.model' is a valid SentencePiece model file
tokenizer = SentencePieceTokenizer(model_path='path/to/your/model.model')
```

--------------------------------

### Usage Examples for SimpleFeedForward (Python)

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/simple_feedback

Provides examples of how to use the `SimpleFeedForward` module. Demonstrates basic instantiation and usage with sample data, as well as integration into custom `nn.Module` architectures. Requires PyTorch and the zeta library.

```python
import torch
import torch.nn as nn
from zeta.nn.modules import SimpleFeedForward

# Basic Usage
model = SimpleFeedForward(768, 2048, 0.1)
x = torch.randn(1, 768)
output = model(x)
print(output.shape)  # torch.Size([1, 768])

# Integrating with Other Architectures
class CustomModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.ff = SimpleFeedForward(768, 2048, 0.1)
        self.final_layer = nn.Linear(768, 10)  # Example output layer

    def forward(self, x):
        x = self.ff(x)
        x = self.final_layer(x)
        return x
```

--------------------------------

### Install Zeta with Pip

Source: https://zeta.apac.ai/en/latest/index

Instructions for installing the Zeta Python package using pip, including options for headless and desktop installations.

```shell
pip install zeta
```

```shell
pip install zeta[desktop]
```

--------------------------------

### Install MkDocs Dependencies

Source: https://zeta.apac.ai/en/latest/contributing

Installs all required Python packages for the Zeta documentation project. This step is necessary before running MkDocs commands locally.

```shell
pip install -r requirements.txt
```

--------------------------------

### Example: Creating a CausalConv3d Module

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/video_autoencoder

Demonstrates how to instantiate the CausalConv3d class in Python using PyTorch. This example shows the typical parameters required for initialization, such as input channels, output channels, and kernel size.

```python
import torch
from zeta.nn.modules.video_autoencoder import CausalConv3d

# Define input and output channels, and kernel size
input_channels = 64
output_channels = 128
kernel_size = 3

# Create an instance of CausalConv3d
causal_conv_layer = CausalConv3d(
    chan_in=input_channels,
    chan_out=output_channels,
    kernel_size=kernel_size,
    pad_mode='zeros' # Example padding mode
)

print(causal_conv_layer)
```

--------------------------------

### Install PyTorch for zeta.ops

Source: https://zeta.apac.ai/en/latest/zeta/ops/multi_dim_cat

Before using zeta.ops, ensure PyTorch is installed in your environment. This is a prerequisite for leveraging the tensor manipulation capabilities of the zeta library.

```bash
pip install torch
```

--------------------------------

### Basic Zeta FSDP Wrapper Example

Source: https://zeta.apac.ai/en/latest/zeta/training/fsdp

This example demonstrates how to use the fsdp function to wrap a simple PyTorch Sequential model. It initializes a basic model and then applies the fsdp wrapper with default settings, preparing it for distributed training.

```python
import torch.nn as nn

# Define your PyTorch model
model = nn.Sequential(
    nn.Linear(784, 256),
    nn.ReLU(),
    nn.Linear(256, 10),
)

# Wrap the model with FSDP using default settings (no sharding, fp32 precision)
fsdp_model = fsdp(model)
```

--------------------------------

### VisualExpert Usage Examples

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/visual_expert

Illustrates how to use the Visual Expert module in deep learning projects. Includes examples for creating a module instance, performing a forward pass with data, and customizing its behavior for specific tasks.

```python
# Example 1: Creating a Visual Expert Module (Conceptual)
# Assuming 'pretrained_lm_config' holds configuration for a language model
# and 'image_encoder' processes image inputs.

# from zeta.nn.modules.visual_expert import VisualExpert
# from transformers import AutoConfig, AutoModel

# lm_config = AutoConfig.from_pretrained("bert-base-uncased")
# lm_model = AutoModel.from_pretrained("bert-base-uncased")

# visual_expert = VisualExpert(
#     lm_config=lm_config,
#     lm_model=lm_model,
#     image_feature_dim=512 # Dimension of image features
# )

# Example 2: Forward Pass (Conceptual)
# Assuming 'image_features' is a tensor of image embeddings
# and 'text_embeddings' are token embeddings from a language model.

# output_features = visual_expert(image_features, text_embeddings)

# Example 3: Customizing Visual Expert (Conceptual)
# Customization might involve modifying layer configurations or initialization strategies.
# This is highly dependent on the specific implementation details of the VisualExpert class.
```

--------------------------------

### Install and Import Zeta Utils (Python)

Source: https://zeta.apac.ai/en/latest/zeta/utils/cast_tuple

Shows how to install the Zeta package using pip and import the utils module. This is the first step to using the Zeta Utils library in your Python projects.

```python
pip install zeta

from zeta import utils
```

--------------------------------

### Python pick_and_pop Usage Examples

Source: https://zeta.apac.ai/en/latest/zeta/utils/pick_and_pop

Demonstrates how to use the `pick_and_pop` function with different dictionary inputs and key lists. The examples show the expected output when specific keys are removed and returned.

```python
d = {"name": "John", "age": 30, "city": "New York"}
keys = ["name", "city"]

result = pick_and_pop(keys, d)
print(result)  # Returns: {'name': 'John', 'city': 'New York'}
```

```python
d = {1: "apple", 2: "banana", 3: "cherry", 4: "date"}
keys = [2, 4]

result = pick_and_pop(keys, d)
print(result)  # Returns: {2: 'banana', 4: 'date'}
```

--------------------------------

### Install zeta and Import Layer

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/polymorphic_activation

Instructions for installing the zetascale library and importing the PolymorphicNeuronLayer class into your Python environment.

```shell
pip install zetascale
```

```python
from zeta.nn import PolymorphicNeuronLayer
```

--------------------------------

### LanguageTokenizerGPTX Usage Example

Source: https://zeta.apac.ai/en/latest/zeta/tokenizers/language_tokenizer

Example demonstrating how to initialize and use the LanguageTokenizerGPTX class for tokenizing text, decoding token IDs, and checking vocabulary size.

```python
import torch
from zeta import LanguageTokenizerGPTX

# Initialize the tokenizer
tokenizer = LanguageTokenizerGPTX()

# Example 1: Tokenize a single text
text = "Hello, world!"
tokenized_text = tokenizer.tokenize_texts(text)
print(tokenized_text)

# Example 2: Decode a tokenized text
decoded_text = tokenizer.decode(tokenized_text)
print(decoded_text)

# Example 3: Get the number of tokens in the tokenizer's vocabulary
num_tokens = len(tokenizer)
print(f"The tokenizer has {num_tokens} tokens.")
```

--------------------------------

### SELU-Softmax Usage Example: Basic

Source: https://zeta.apac.ai/en/latest/zeta/ops/selu_softmax

Demonstrates the fundamental usage of the selu_softmax function. This example covers prerequisites and provides a complete code snippet for basic application.

```python
# Example 1: Basic Usage
# Prerequisites: Ensure necessary libraries (e.g., PyTorch, TensorFlow) are installed.
# Full Code Example:
# import torch
# import torch.nn.functional as F
# 
# def selu_softmax(x):
#     return F.softmax(F.selu(x), dim=-1)
# 
# # Sample input tensor
# input_tensor = torch.randn(5, 10) # Batch size 5, 10 classes
# output = selu_softmax(input_tensor)
# print(output)
# print(torch.sum(output, dim=-1)) # Should sum to 1 for each item in batch
```

--------------------------------

### SelectEOSAndProject Class Usage Example

Source: https://zeta.apac.ai/en/latest/zeta/utils/main

This example demonstrates the instantiation and usage of the `SelectEOSAndProject` class from the `zeta.utils.main` module. It covers initializing the class with a projection module and performing a forward pass with sample input tensors and sequence lengths.

```python
import torch
import torch.nn as nn

from zeta.utils.main import SelectEOSAndProject

proj_module = nn.Linear(256, 128)
select_and_project = SelectEOSAndProject(proj=proj_module)

x = torch.randn(1, 16, 256)  # Input tensor
seq_len = torch.tensor([10])  # Sequence length
output = select_and_project(x, seq_len)

print(output.shape)
```

--------------------------------

### SophiaG Optimizer Basic Usage Example (Python)

Source: https://zeta.apac.ai/en/latest/zeta/training/optimizers/sophia

A complete, runnable example showing the import of necessary libraries (torch, nn, SophiaG), definition of a simple linear model, and instantiation of the SophiaG optimizer with default parameters.

```Python
import torch
import torch.nn as nn
from zeta import SophiaG

model = nn.Linear(10, 1)
optimizer = SophiaG(model.parameters(), lr=0.01)
```

--------------------------------

### img_compose_bw Example Usage

Source: https://zeta.apac.ai/en/latest/zeta/ops/img_compose_bw

Demonstrates how to import and use the img_compose_bw function with PyTorch. It includes necessary imports and setup for running the example.

```python
import torch
from zeta.ops import img_compose_bw
```

--------------------------------

### Python Example: Using the `once` Decorator

Source: https://zeta.apac.ai/en/latest/zeta/utils/once

This example demonstrates how to use the `once` decorator. The `setup` function, decorated with `@once`, is called twice. The output shows that the function's body executes only during the first call, while subsequent calls are silently ignored.

```Python
@once
def setup():
    print("Setting up...")

# The setup() function is invoked twice.
setup()  # Prints: 'Setting up...'
setup()  # Doesn't print anything.
```

--------------------------------

### Create CausalConv3d Module (Python)

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/video_autoencoder

Demonstrates how to instantiate the `CausalConv3d` class from the Zeta library. This example shows the basic setup with input and output channel configurations and kernel size.

```python
causal_conv = CausalConv3d(chan_in=64, chan_out=128, kernel_size=3)
```

--------------------------------

### External Resources for Initialization

Source: https://zeta.apac.ai/en/latest/zeta/utils/init_zero_

References for understanding PyTorch initialization and weight initialization techniques in neural networks.

```APIDOC
External Resources:
  1. PyTorch Documentation: [torch.nn.init.constant_](https://pytorch.org/docs/stable/nn.init.html#torch.nn.init.constant_)
  2. Blog Post: [Weight Initialization in Neural Networks: A Journey From the Basics to Kaiming](https://towardsdatascience.com/weight-initialization-in-neural-networks-a-journey-from-the-basics-to-kaiming-954fb9b47c79)
```

--------------------------------

### Instantiating and Using MyModel

Source: https://zeta.apac.ai/en/latest/zeta/models/basemodel

Shows how to create an instance of a custom model, MyModel, by passing necessary arguments to its constructor. It also demonstrates calling the forward method on the instantiated object.

```Python
my_model = MyModel(10)
my_model.forward()
```

--------------------------------

### Get Max Negative Value for Tensor dtype

Source: https://zeta.apac.ai/en/latest/zeta/utils/main

Retrieves the maximum negative value for a given tensor's data type. This utility function takes a tensor and returns a float representing the largest negative number representable by its dtype. The example shows how to get this value for a tensor's dtype.

```python
import torch
from zeta.utils.main import max_neg_values

tensor = torch.tensor([1.0, 2.0, 3.0])
max_neg = max_neg_values(tensor.dtype)

print(max_neg)
```

--------------------------------

### Basic NFNStem Usage with PyTorch

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/nfnstem

Demonstrates how to instantiate the NFNStem module and perform a forward pass with a random input tensor. This example shows the fundamental setup for using the NFNStem class within a PyTorch workflow, including input tensor creation and output shape printing.

```python
import torch
from zeta.nn import NFNStem

# Create a random tensor with the shape of (1, 3, 224, 224)
x = torch.randn(1, 3, 224, 224)

# Instantiate the NFNStem module
model = NFNStem()

# Forward pass
out = model(x)
print(out.shape)
# Output: torch.Size([1, 128, 28, 28])
```

--------------------------------

### Initialize and Use NaViT Model

Source: https://zeta.apac.ai/en/latest/zeta/models/navit

Demonstrates initializing the NaViT model with specified parameters (image size, patch size, dimensions, etc.), creating a sample input tensor for images, and performing a forward pass to obtain class probabilities. The output shape is printed to verify the model's behavior.

```python
import torch
from zeta.models import NaViT

# initialize the model
model = NaViT(
    image_size=32,
    patch_size=4,
    num_classes=10,
    dim=512,
    depth=6,
    heads=8,
    mlp_dim=1024
)

# random tensor representing a batch of 10 images, with 3 color channels, each 32x32 pixels
x = torch.randn(10, 3, 32, 32)

# the forward function returns the output of the model, which represents class probabilities for each image.
output = model.forward(x)
print(output.shape)  # prints: torch.Size([10, 10])
```

--------------------------------

### Basic DPO Setup and Usage with PyTorch

Source: https://zeta.apac.ai/en/latest/zeta/rl/dpo

Demonstrates initializing a `PolicyModel` and the `DPO` class from the zeta.rl library. It shows how to compute the DPO loss using sample preferred and unpreferred sequences with PyTorch tensors.

```python
import torch
from torch import nn

from zeta.rl import DPO


# Define a simple policy model
class PolicyModel(nn.Module):
    def __init__(self, dim, output_dim):
        super().__init__()
        self.fc = nn.Linear(dim, output_dim)

    def forward(self, x):
        return self.fc(x)


dim = 10
output_dim = 5
policy_model = PolicyModel(dim, output_dim)

# Initialize DPO with the policy model
dpo_model = DPO(model=policy_model, beta=0.1)

# Sample preferred and unpreferred sequences
preferred_seq = torch.randn(1, 10, 10)
unpreferred_seq = torch.randn(1, 10, 10)

# Compute loss
loss = dpo_model(preferred_seq, unpreferred_seq)
print(loss)
```

--------------------------------

### Install Zeta with Conda

Source: https://zeta.apac.ai/en/latest/index

Steps to install Zeta using Anaconda, involving environment creation and package installation.

```shell
conda create -n zeta_env python=3.10
conda activate zeta_env
pip install zeta
```

```shell
conda create -n zeta_env python=3.10
conda activate zeta_env
pip install zeta[desktop]
```

--------------------------------

### ContrastiveTopK Example

Source: https://zeta.apac.ai/en/latest/zeta/utils/main

Demonstrates the usage of the ContrastiveTopK utility for calculating a contrastive loss. It shows how to initialize the utility and apply it to PyTorch tensors representing logits.

```python
import torch
from zeta.utils.main import ContrastiveTopK

contrastive = ContrastiveTopK(alpha=0.5, k=3)

logits_exp = torch.tensor([1.0, 2.0, 3.0])
logits_ama = torch.tensor([4.0, 5.0, 6.0])

loss = contrastive(logits_exp, logits_ama)
print(loss)
```

--------------------------------

### Serve Zeta Docs Locally with MkDocs

Source: https://zeta.apac.ai/en/latest/contributing

Starts the MkDocs development server to preview documentation changes locally. This command watches for file changes and rebuilds the documentation automatically.

```shell
mkdocs serve
```

--------------------------------

### Install Zetascale Package

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/custom_mlp

Installs the zetascale library using pip, a prerequisite for using CustomMLP. Ensure PyTorch is installed separately.

```shell
pip install zetascale
```

--------------------------------

### SELU-Softmax Usage Example: Neural Network Integration

Source: https://zeta.apac.ai/en/latest/zeta/ops/selu_softmax

Illustrates how to integrate the selu_softmax function within a neural network architecture. This example includes prerequisites and a full code example for practical application.

```python
# Example 2: Using selu_softmax in a Neural Network
# Prerequisites: A defined neural network model structure.
# Full Code Example:
# import torch
# import torch.nn as nn
# import torch.nn.functional as F
# 
# class MyModel(nn.Module):
#     def __init__(self):
#         super().__init__()
#         self.linear = nn.Linear(784, 10)
# 
#     def forward(self, x):
#         x = x.view(x.size(0), -1) # Flatten input
#         x = self.linear(x)
#         # Apply selu_softmax
#         return F.softmax(F.selu(x), dim=-1)
# 
# model = MyModel()
# sample_input = torch.randn(1, 1, 28, 28) # Batch size 1, 1 channel, 28x28 image
# output = model(sample_input)
# print(output)
```

--------------------------------

### SentencePieceTokenizer Encode Example

Source: https://zeta.apac.ai/en/latest/zeta/tokenizers/sentencepiece

Example of encoding a string into token IDs using the SentencePieceTokenizer.

```python
from zeta.tokenizers import SentencePieceTokenizer

tokenizer = SentencePieceTokenizer(model_path='path/to/your/model.model')

text = "Hello, world!"
encoded_tokens = tokenizer.encode(text, bos=True, eos=True)
print(f"Encoded: {encoded_tokens}")
```

--------------------------------

### Instantiate and Run the Model

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/stochasticskipblock

Demonstrates how to create an instance of the 'MyModel' and pass a random input tensor through it. This shows the basic execution flow of the model.

```python
model = MyModel()
input = torch.randn(32, 10)
output = model(input)
```

--------------------------------

### SentencePieceTokenizer Decode Example

Source: https://zeta.apac.ai/en/latest/zeta/tokenizers/sentencepiece

Example of decoding token IDs back into a string using the SentencePieceTokenizer.

```python
from zeta.tokenizers import SentencePieceTokenizer

tokenizer = SentencePieceTokenizer(model_path='path/to/your/model.model')

# Assuming encoded_tokens is a list of integers obtained from tokenizer.encode()
encoded_tokens = [1, 5, 10, 2, 3] # Example token IDs
decoded_text = tokenizer.decode(encoded_tokens)
print(f"Decoded: {decoded_text}")
```

--------------------------------

### SentencePieceTokenizer Infilling Encode Example

Source: https://zeta.apac.ai/en/latest/zeta/tokenizers/sentencepiece

Example of using encode_infilling for specific tokenization needs, such as infilling tasks.

```python
from zeta.tokenizers import SentencePieceTokenizer

tokenizer = SentencePieceTokenizer(model_path='path/to/your/model.model')

infilling_text = "This is a test."
infilling_tokens = tokenizer.encode_infilling(infilling_text)
print(f"Infilling Encoded: {infilling_tokens}")
```

--------------------------------

### Initialize SigLipLoss with Defaults

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/siglip

Demonstrates how to initialize the SigLipLoss module using its default parameters. This is the most basic setup for the module.

```python
from zeta.nn.modules. import SigLipLoss

# Initialize SigLipLoss module
loss = SigLipLoss()
```

--------------------------------

### Initialize and Run GPT4 Model

Source: https://zeta.apac.ai/en/latest/zeta/models/gpt4

Demonstrates initializing the GPT4 model with default parameters and performing inference by passing a sample input tensor. This example requires PyTorch and the zeta library.

```python
import torch
from torch import nn

from zeta.models import GPT4

# Initialize with default parameters
model = GPT4()

# Representing 3 sequences of the maximum length of 8192
input = torch.randint(0, 50432, (3, 8192))

# Pass the input to the model's forward method
output = model.forward(input)
```

--------------------------------

### Initialize MixtureOfSoftmaxes Module

Source: https://zeta.apac.ai/en/latest/zeta/ops/mos

Demonstrates how to initialize the MixtureOfSoftmaxes module with required parameters: number of mixtures, input size, and number of output classes.

```python
mos = MixtureOfSoftmaxes(num_mixtures=5, input_size=128, num_classes=10)
```

--------------------------------

### Python Example: FlexiConv Layer Usage

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/flexiconv

Demonstrates how to instantiate and use the FlexiConv layer from the zeta.nn module. It shows the import statement, initialization with common parameters, and applying the layer to a sample input tensor, including checking the output shape.

```Python
import torch
from zeta.nn import FlexiConv

flexi_conv = FlexiConv(
    in_channels=3, out_channels=64, kernel_size=3, stride=1, padding=1
)
input_tensor = torch.randn(1, 3, 224, 224)  # Example input batch
output = flexi_conv(input_tensor)
output.shape
```

--------------------------------

### Python VisualExpert Module Initialization

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/visual_expert

Demonstrates how to create an instance of the VisualExpert module with specified dimensions, dropout rate, and attention heads.

```python
import torch
from zeta.nn import VisualExpert

# Create a Visual Expert module
visual_expert = VisualExpert(dim=1024, hidden_dim=2048, dropout=0.1, heads=16)
```

--------------------------------

### Basic Usage of Adapt Method

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/adaptive

Demonstrates the basic usage of the 'adapt' method. It shows how to define an adaptation function, create an AdaptiveParameterList, and apply the adaptation using a dictionary mapping parameter indices to functions.

```Python
import torch
import torch.nn as nn
from AdaptiveParameterList import AdaptiveParameterList
from shapeless import x  # Placeholder, as actual import statement was not provided

# Define an adaptation function
def adaptation_function(param):
    return param * 0.9

adaptive_params = AdaptiveParameterList([nn.Parameter(torch.randn(10, 10))])

# Create a dictionary with adaptation functions for the desired indices
adapt_funcs = {0: adaptation_function}

adaptive_params.adapt(adapt_funcs)
```

--------------------------------

### SELU-Softmax Usage Example: Image Classification

Source: https://zeta.apac.ai/en/latest/zeta/ops/selu_softmax

Details the application of selu_softmax in a multi-class image classification context. This example outlines prerequisites and provides a complete code snippet.

```python
# Example 3: Application in a Multi-Class Image Classification
# Prerequisites: A dataset suitable for image classification and a model architecture.
# Full Code Example:
# import torch
# import torch.nn as nn
# import torch.nn.functional as F
# 
# class ImageClassifier(nn.Module):
#     def __init__(self):
#         super().__init__()
#         self.conv1 = nn.Conv2d(3, 32, kernel_size=3, padding=1)
#         self.pool = nn.MaxPool2d(2, 2)
#         self.conv2 = nn.Conv2d(32, 64, kernel_size=3, padding=1)
#         self.fc1 = nn.Linear(64 * 7 * 7, 128) # Assuming input image size 28x28
#         self.fc2 = nn.Linear(128, 10) # 10 classes
# 
#     def forward(self, x):
#         x = self.pool(F.relu(self.conv1(x)))
#         x = self.pool(F.relu(self.conv2(x)))
#         x = x.view(-1, 64 * 7 * 7)
#         x = F.relu(self.fc1(x))
#         # Apply selu_softmax for final classification output
#         return F.softmax(F.selu(self.fc2(x)), dim=-1)
# 
# model = ImageClassifier()
# sample_image = torch.randn(1, 3, 28, 28) # Batch size 1, 3 color channels, 28x28 image
# probabilities = model(sample_image)
# print(probabilities)
```

--------------------------------

### Example Usage of TopNGating in Python

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/topngating

Demonstrates how to instantiate and use the TopNGating module. It shows creating a model instance with specified dimensions and number of gates, and then passing a sample input tensor through the forward method.

```Python
import torch
from zeta.nn import TopNGating

x = torch.randn(1, 2, 3)
model = TopNGating(3, 4)
(out, _, _, _) = model(x)
print(out.shape)
```

--------------------------------

### Using the YarnEmbedding Class Example

Source: https://zeta.apac.ai/en/latest/zeta/nn/embeddings/yarn

Demonstrates how to instantiate and use the YarnEmbedding class with sample input data. This example shows a typical forward pass.

```python
import torch
from zeta.nn.embeddings.yarn import YarnEmbedding

# Example usage:
seq_len = 100
batch_size = 2
embedding_dim = 128

# Initialize YarnEmbedding
yarn_emb = YarnEmbedding(dim=embedding_dim, base=10000, scale=0.5, interpolation='linear')

# Create dummy input tensor (batch_size, seq_len, embedding_dim)
input_tensor = torch.randn(batch_size, seq_len, embedding_dim)

# Apply YarnEmbedding
output_tensor = yarn_emb(input_tensor, seq_len=seq_len)

print(f"Input tensor shape: {input_tensor.shape}")
print(f"Output tensor shape: {output_tensor.shape}")
```

--------------------------------

### Tokenizing Texts Example

Source: https://zeta.apac.ai/en/latest/zeta/tokenizers/multi_modal_tokenizer

This example demonstrates how to use the MultiModalTokenizer to tokenize a list of text strings. It shows the instantiation of the tokenizer and the subsequent call to the tokenize_texts method, printing the results.

```python
import torch

from zeta import MultiModalTokenizer

tokenizer = MultiModalTokenizer()
texts = ["Hello World", "Zeta Library is great!"]
tokenized_texts, only_texts = tokenizer.tokenize_texts(texts)
print(tokenized_texts)
print(only_texts)
```

--------------------------------

### Initialize SigLipLoss Module

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/siglip

Demonstrates how to initialize the SigLipLoss module. Optional parameters like cache_labels, rank, world_size, bidir, and use_horovod can be configured during instantiation to tailor the loss computation for distributed training scenarios.

```python
from zeta.nn.modules import SigLipLoss

# Initialize SigLipLoss module
loss = SigLipLoss(
    cache_labels=False, rank=0, world_size=1, bidir=True, use_horovod=False
)
```

--------------------------------

### Zeta Trainer Basic Usage

Source: https://zeta.apac.ai/en/latest/zeta/training/train

Demonstrates the basic initialization of the Zeta Trainer with essential parameters like batch size, sequence length, learning rate, and output directory. This serves as a starting point for training custom models.

```python
from zeta import Trainer

model = ...  # Your model definition here
Trainer(
    gradient_accumulate_every=2,
    batch_size=32,
    seq_len=128,
    model=model,
    learning_rate=0.001,
    seed=42,
    output_dir="./models/"
)
```

--------------------------------

### HighwayLayer Usage Example 2 (Python)

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/highwaylayer

Illustrates a model incorporating multiple HighwayLayers. This example showcases how to stack HighwayLayers sequentially within a PyTorch model.

```python
import torch
import torch.nn as nn
from zeta.nn.modules.highwaylayer import HighwayLayer

class MultiHighwayModel(nn.Module):
    def __init__(self, dim, num_layers):
        super().__init__()
        self.layers = nn.ModuleList([
            HighwayLayer(dim=dim) for _ in range(num_layers)
        ])

    def forward(self, x):
        for layer in self.layers:
            x = layer(x)
        return x

# Example Usage:
dim = 128
num_layers = 3
model = MultiHighwayModel(dim=dim, num_layers=num_layers)
input_tensor = torch.randn(1, dim)
output_tensor = model(input_tensor)
print(f"Input shape: {input_tensor.shape}")
print(f"Output shape: {output_tensor.shape}")
```

--------------------------------

### Apply QuickGELUActivation in PyTorch

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/quickgeluactivation

This example shows how to instantiate `QuickGELUActivation` from `zeta.nn` and apply it to a PyTorch tensor. It covers creating a tensor, applying the activation function, and printing the output. The description also notes its potential advantages in speed over standard GELU and its suitability for specific model architectures.

```python
import torch
from torch import nn
from zeta.nn import QuickGELUActivation

# create an instance of QuickGELUActivation
activation = QuickGELUActivation()

# create a tensor
x = torch.rand(3)

# apply GELU activation
output = activation(x)

print(output)
```

--------------------------------

### Tokenizing Images Example

Source: https://zeta.apac.ai/en/latest/zeta/tokenizers/multi_modal_tokenizer

This example illustrates the process of tokenizing image data using the MultiModalTokenizer. It involves creating a dummy tensor representing images and passing it to the tokenize_images method, then printing the output.

```python
import torch

from zeta import MultiModalTokenizer

tokenizer = MultiModalTokenizer()
images = torch.randn(2, 3, 224, 224)  # Assuming 2 random images of shape 3x224x224
tokenized_images = tokenizer.tokenize_images(images)
print(tokenized_images)
```

--------------------------------

### GatedResidualBlock Class Documentation

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/gatedresidualblock

Documentation for the GatedResidualBlock class, detailing its constructor arguments, method signatures, return values, and providing a usage example.

```APIDOC
zeta.nn.modules.gatedresidualblock.GatedResidualBlock

Class Definition:
  __init__(self, dim, ff_glu_hidden_dim, dropout_rate=0.0)
    Initializes the GatedResidualBlock.

    Arguments:
      dim (int): The input and output dimension of the block.
      ff_glu_hidden_dim (int): The hidden dimension for the GLU (Gated Linear Unit) feed-forward component.
      dropout_rate (float, optional): The dropout rate to apply. Defaults to 0.0.

Method Definition:
  forward(self, x)
    Performs the forward pass of the GatedResidualBlock.

    Arguments:
      x (torch.Tensor): The input tensor.

    Returns:
      torch.Tensor: The output tensor after applying the gated residual block.

Example: Usage of GatedResidualBlock
  import torch
  from zeta.nn.modules.gatedresidualblock import GatedResidualBlock

  # Example usage
  dim = 512
  ff_glu_hidden_dim = 1024
  dropout_rate = 0.1

  gated_residual_block = GatedResidualBlock(dim=dim, ff_glu_hidden_dim=ff_glu_hidden_dim, dropout_rate=dropout_rate)

  # Create a dummy input tensor
  input_tensor = torch.randn(1, 10, dim) # (batch_size, sequence_length, dimension)

  # Pass the input tensor through the block
  output_tensor = gated_residual_block(input_tensor)

  print("Input tensor shape:", input_tensor.shape)
  print("Output tensor shape:", output_tensor.shape)

Note:
  This block is designed to incorporate gated mechanisms within a residual connection, often used in advanced neural network architectures.
```

--------------------------------

### QUIK Forward Pass Example

Source: https://zeta.apac.ai/en/latest/zeta/quant/quik

Demonstrates a basic forward pass using the QUIK layer with pre-quantized data. This example shows the core operation of applying the QUIK layer to input data.

```python
output = quik(quantized_data)
```

--------------------------------

### zeta.utils.main API Documentation

Source: https://zeta.apac.ai/en/latest/zeta/utils/main

Comprehensive API documentation for utility functions and classes within the zeta.utils.main module. This includes functions for checking existence, providing defaults, memoization, evaluation decorators, tuple casting, conditional execution, and helper classes for comparisons and initialization.

```APIDOC
Function: exists(val)
  Description: Checks if a value is not None or an empty collection.
  Parameters:
    val: The value to check.
  Returns:
    bool: True if the value exists, False otherwise.
  Example:
    exists(5) -> True
    exists(None) -> False
    exists([]) -> False
```

```APIDOC
Function: default(val, d)
  Description: Returns the value if it exists, otherwise returns a default value.
  Parameters:
    val: The value to check.
    d: The default value to return if val is considered non-existent.
  Returns:
    The original value or the default value.
  Example:
    default(5, 0) -> 5
    default(None, 0) -> 0
```

```APIDOC
Function: once(fn)
  Description: Decorator that ensures a function is only called once, returning the cached result on subsequent calls.
  Parameters:
    fn: The function to be memoized.
  Returns:
    The result of the first call to the function.
  Example:
    @once
    def expensive_computation():
        # ...
        return result
    expensive_computation() # runs
    expensive_computation() # returns cached result
```

```APIDOC
Function: eval_decorator(fn)
  Description: Decorator that evaluates a function and returns its result.
  Parameters:
    fn: The function to evaluate.
  Returns:
    The result of the function call.
  Example:
    @eval_decorator
    def get_value():
        return 10
    get_value() -> 10
```

```APIDOC
Function: cast_tuple(val, depth)
  Description: Recursively casts a value into a tuple up to a specified depth.
  Parameters:
    val: The value to cast.
    depth: The maximum depth of recursion for casting.
  Returns:
    The value cast into a tuple structure.
  Example:
    cast_tuple([1, [2, 3]], 1) -> ([1], ([2, 3],))
    cast_tuple([1, [2, 3]], 2) -> ([1], ([2], [3]))
```

```APIDOC
Function: maybe(fn)
  Description: Decorator that returns None if the decorated function raises an exception.
  Parameters:
    fn: The function to wrap.
  Returns:
    The result of the function or None if an exception occurred.
  Example:
    @maybe
    def might_fail():
        raise ValueError('Failed')
    might_fail() -> None
```

```APIDOC
Class: always
  Description: A class that always returns True for comparison operations.
  Parameters:
    value: The value to compare against (not used in logic).
  Methods:
    __call__(self, *args, **kwargs): Returns True.
    __eq__(self, other): Returns True.
    __ne__(self, other): Returns False.
    __lt__(self, other): Returns False.
    __le__(self, other): Returns False.
    __gt__(self, other): Returns False.
    __ge__(self, other): Returns False.
  Example:
    always(5) == 10 -> True
```

```APIDOC
Class: not_equals
  Description: A class that checks for inequality against a given value.
  Parameters:
    value: The value to compare against.
  Methods:
    __call__(self, other): Returns True if self.value != other, False otherwise.
    __eq__(self, other): Returns False.
    __ne__(self, other): Returns True if self.value != other, False otherwise.
  Example:
    not_equals(5)(10) -> True
    not_equals(5)(5) -> False
```

```APIDOC
Class: equals
  Description: A class that checks for equality against a given value.
  Parameters:
    value: The value to compare against.
  Methods:
    __call__(self, other): Returns True if self.value == other, False otherwise.
    __eq__(self, other): Returns True if self.value == other, False otherwise.
    __ne__(self, other): Returns False.
  Example:
    equals(5)(5) -> True
    equals(5)(10) -> False
```

```APIDOC
Function: init_zero_(layer)
  Description: Initializes the weights of a given layer to zero.
  Parameters:
    layer: The layer object to initialize.
  Example:
    init_zero_(my_layer)
```

```APIDOC
Function: pick_and_pop(keys, d)
  Description: Extracts specified keys from a dictionary and returns them along with the modified dictionary.
  Parameters:
    keys: A list or tuple of keys to extract.
    d: The dictionary to extract from.
  Returns:
    A tuple containing the extracted values and the remaining dictionary.
  Example:
    pick_and_pop(['a', 'b'], {'a': 1, 'b': 2, 'c': 3}) -> ({'a': 1, 'b': 2}, {'c': 3})
```

```APIDOC
Function: group_dict_by_key(cond, d)
  Description: Groups items in a dictionary based on a condition applied to their keys.
  Parameters:
    cond: A callable that takes a key and returns a group identifier.
    d: The dictionary to group.
  Returns:
    A dictionary where keys are group identifiers and values are lists of items belonging to that group.
  Example:
    group_dict_by_key(lambda k: k % 2, {'a': 1, 'b': 2, 'c': 3}) -> {0: ['b'], 1: ['a', 'c']}
```

--------------------------------

### zeta.ops: Main Entry Point

Source: https://zeta.apac.ai/en/latest/zeta/ops/reshape_audio_to_text

Documentation for the main entry point or execution function within the zeta.ops module.

```APIDOC
zeta.ops.main

Description:
  Likely serves as the primary execution or orchestration function for operations within the zeta.ops module.

Note: Specific arguments and return values would be detailed in the full documentation.
```

--------------------------------

### zeta.ops: Reshape Audio to Text

Source: https://zeta.apac.ai/en/latest/zeta/ops/reshape_audio_to_text

Details the functionality, parameters, and usage examples for reshaping audio data into a format suitable for text processing. Includes integration examples and collaborative filtering concepts.

```APIDOC
zeta.ops.reshape_audio_to_text

Description:
  Reshapes audio data for text processing tasks.

Sections:
  - Introduction to zeta.ops
  - Purpose of reshape_audio_to_text
  - How reshape_audio_to_text Works
  - Function Definition
    - Parameters and Return Types
    - Functionality and Usage Examples
      - Example 1: Basic Usage
      - Example 2: Integrating with a Model
      - Example 3: Collaborative Filtering between Modalities
  - Additional Information and Tips
  - References and Further Learning

Note: Specific function signature, parameters, and return types are detailed within the linked documentation.
```

--------------------------------

### TopNGating: Initialize and Use with Top N Parameter

Source: https://zeta.apac.ai/en/latest/zeta/nn/modules/topngating

Demonstrates initializing the `TopNGating` module with a specific `top_n` value and performing a forward pass with noise. This example shows how to set up the module for top-k gating and apply it to input data.

```python
import torch
from zeta.nn import TopNGating

x = torch.randn(2, 3, 4)
model = TopNGating(4, 3, top_n=3)
out, _, _, _ = model(x, noise_gates=True, noise_mult=0.7)
print(out.shape)
```

--------------------------------

### Python: logit_scaled_softmax Usage Examples

Source: https://zeta.apac.ai/en/latest/zeta/ops/logit_scaled_softmax

Demonstrates the usage of the `logit_scaled_softmax` function with PyTorch. Includes examples for basic application with default scaling and for adjusting output sharpness by providing a custom scale factor.

```python
import torch
from zeta.ops import logit_scaled_softmax

# Example 1: Basic Usage
logits = torch.tensor([1.0, 2.0, 3.0])
softmax_probs = logit_scaled_softmax(logits)
print(softmax_probs)

# Example 2: Adjusting Sharpness with Scale
scale = 2.0
sharper_softmax_probs = logit_scaled_softmax(logits, scale)
print(sharper_softmax_probs)
```

--------------------------------

### Example 1: Basic Matrix Root Calculation

Source: https://zeta.apac.ai/en/latest/zeta/ops/_matrix_root_eigen

Demonstrates a basic usage of the _matrix_root_eigen function to compute the square root of a matrix. This example highlights the core functionality without advanced options.

```python
import torch
from zeta.ops import _matrix_root_eigen

# Define a sample symmetric positive definite matrix
A = torch.tensor([[2.0, 1.0], [1.0, 2.0]])

# Compute the square root (alpha = 0.5)
sqrt_A = _matrix_root_eigen(A, alpha=0.5)

print("Original Matrix A:")
print(A)
print("\nSquare Root of A:")
print(sqrt_A)

# Verify by multiplying sqrt_A by itself
print("\nVerification (sqrt_A @ sqrt_A):")
print(torch.matmul(sqrt_A, sqrt_A))
```