KerasHub

https://keras.io/keras_hub/
Admin
KerasHub is a pretrained modeling library providing Keras 3 implementations of popular model...
Tokens:603,301
Snippets:4,343
Trust Score:10
Update:2 months ago
Show doc for...
Tokens:
Raw
### Usage Examples for Qwen3MoeBackbone

Source: https://keras.io/keras_hub/api/models/qwen3_moe/qwen3_moe_backbone

Examples demonstrating how to load a pre-trained model and how to initialize a custom model with input data.

```python
input_data = {
    "token_ids": np.ones(shape=(1, 12), dtype="int32"),
    "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]]),
}
```

```python
model = keras_hub.models.Qwen3MoeBackbone.from_preset("qwen3_moe_a2_7b")
model(input_data)
```

```python
model = keras_hub.models.Qwen3MoeBackbone(
    vocabulary_size=151936,
    num_layers=28,
    num_query_heads=16,
    num_key_value_heads=8,
    hidden_dim=2048,
    intermediate_dim=4096,
    moe_intermediate_dim=128,
    num_experts=60,
    top_k=4,
    head_dim=128,
    max_sequence_length=4096,
)
model(input_data)
```

--------------------------------

### Usage examples for QwenMoeBackbone

Source: https://keras.io/keras_hub/api/models/qwen_moe/qwen_moe_backbone

Examples showing how to load a pre-trained model and how to initialize a custom model with specific hyperparameters.

```python
input_data = {
    "token_ids": np.ones(shape=(1, 12), dtype="int32"),
    "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]]),
}

model = keras_hub.models.QwenMoeBackbone.from_preset("qwen_moe_a2_7b")
model(input_data)
```

```python
model = keras_hub.models.QwenMoeBackbone(
    vocabulary_size=151936,
    num_layers=28,
    num_query_heads=16,
    num_key_value_heads=8,
    hidden_dim=2048,
    intermediate_dim=4096,
    moe_intermediate_dim=128,
    shared_expert_intermediate_dim=4096,
    num_experts=60,
    top_k=4,
    head_dim=128,
    max_sequence_length=4096,
)
model(input_data)
```

--------------------------------

### StartEndPacker usage examples

Source: https://keras.io/keras_hub/api/preprocessing_layers/start_end_packer

Various configurations for processing integer and string inputs, including batched and unbatched data and multiple start tokens.

```python
>>> inputs = [5, 6, 7]
>>> start_end_packer = keras_hub.layers.StartEndPacker(
...     sequence_length=7, start_value=1, end_value=2,
... )
>>> outputs = start_end_packer(inputs)
>>> np.array(outputs)
array([1, 5, 6, 7, 2, 0, 0], dtype=int32)
```

```python
>>> inputs = [[5, 6, 7], [8, 9, 10, 11, 12, 13, 14]]
>>> start_end_packer = keras_hub.layers.StartEndPacker(
...     sequence_length=6, start_value=1, end_value=2,
... )
>>> outputs = start_end_packer(inputs)
>>> np.array(outputs)
array([[ 1,  5,  6,  7,  2,  0],
       [ 1,  8,  9, 10, 11,  2]], dtype=int32)
```

```python
>>> inputs = ["this", "is", "fun"]
>>> start_end_packer = keras_hub.layers.StartEndPacker(
...     sequence_length=6, start_value="<s>", end_value="</s>",
...     pad_value="<pad>"
... )
>>> outputs = start_end_packer(inputs)
>>> np.array(outputs).astype("U")
array(['<s>', 'this', 'is', 'fun', '</s>', '<pad>'], dtype='<U5')
```

```python
>>> inputs = [["this", "is", "fun"], ["awesome"]]
>>> start_end_packer = keras_hub.layers.StartEndPacker(
...     sequence_length=6, start_value="<s>", end_value="</s>",
...     pad_value="<pad>"
... )
>>> outputs = start_end_packer(inputs)
>>> np.array(outputs).astype("U")
array([['<s>', 'this', 'is', 'fun', '</s>', '<pad>'],
       ['<s>', 'awesome', '</s>', '<pad>', '<pad>', '<pad>']], dtype='<U7')
```

```python
>>> inputs = [["this", "is", "fun"], ["awesome"]]
>>> start_end_packer = keras_hub.layers.StartEndPacker(
...     sequence_length=6, start_value=["</s>", "<s>"], end_value="</s>",
...     pad_value="<pad>"
... )
>>> outputs = start_end_packer(inputs)
>>> np.array(outputs).astype("U")
array([['</s>', '<s>', 'this', 'is', 'fun', '</s>'],
       ['</s>', '<s>', 'awesome', '</s>', '<pad>', '<pad>']], dtype='<U7')
```

--------------------------------

### Usage Examples for DINOV2Backbone

Source: https://keras.io/keras_hub/api/models/dinov2/dinov2_backbone

Examples demonstrating how to load pre-trained models, customize image shapes, and initialize a model with custom configuration.

```python
# Pretrained DINOV2 model.
input_data = {
    "images": np.ones(shape=(1, 518, 518, 3), dtype="float32"),
}
model = keras_hub.models.DINOV2Backbone.from_preset(
    "dinov2_base"
)
model(input_data)

# Pretrained DINOV2 model with custom image shape.
input_data = {
    "images": np.ones(shape=(1, 224, 224, 3), dtype="float32"),
}
model = keras_hub.models.DINOV2Backbone.from_preset(
    "dinov2_base", image_shape=(224, 224, 3)
)
model(input_data)

# Randomly initialized DINOV2 model with custom config.
model = keras_hub.models.DINOV2Backbone(
    patch_size=14,
    num_layers=2,
    hidden_dim=32,
    num_heads=2,
    intermediate_dim=128,
    image_shape=(224, 224, 3),
    position_embedding_shape=(518, 518),
)
model(input_data)
```

--------------------------------

### XLMRobertaBackbone usage examples

Source: https://keras.io/keras_hub/api/models/xlm_roberta/xlm_roberta_backbone

Examples demonstrating how to load a pre-trained XLM-R model and how to initialize a custom model with specific configuration parameters.

```python
input_data = {
    "token_ids": np.ones(shape=(1, 12), dtype="int32"),
    "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]]),
}

# Pretrained XLM-R encoder.
model = keras_hub.models.XLMRobertaBackbone.from_preset(
    "xlm_roberta_base_multi",
)
model(input_data)

# Randomly initialized XLM-R model with custom config.
model = keras_hub.models.XLMRobertaBackbone(
    vocabulary_size=250002,
    num_layers=4,
    num_heads=4,
    hidden_dim=256,
    intermediate_dim=512,
    max_sequence_length=128
)
model(input_data)
```

--------------------------------

### Examples of from_preset Usage

Source: https://keras.io/keras_hub/api/base_classes/backbone

Demonstrates loading models with and without pre-trained weights.

```python
# Load a Gemma backbone with pre-trained weights.
model = keras_hub.models.Backbone.from_preset(
    "gemma_2b_en",
)

# Load a Bert backbone with a pre-trained config and random weights.
model = keras_hub.models.Backbone.from_preset(
    "bert_base_en",
    load_weights=False,
)
```

--------------------------------

### User Request Example

Source: https://keras.io/keras_hub/guides/function_calling_with_keras_hub

Example of a user prompt requesting a flight booking based on specific criteria.

```text
Book the most comfortable flight from Linz to London on the 24th of July 2025, but only if it costs less than 20€ as of the latest exchange rate.
```

--------------------------------

### Use DenseNetBackbone with Pretrained and Custom Configs

Source: https://keras.io/keras_hub/api/models/densenet/densenet_backbone

Examples of initializing a DenseNetBackbone from a preset or with a custom configuration.

```python
input_data = np.ones(shape=(8, 224, 224, 3))

# Pretrained backbone
model = keras_hub.models.DenseNetBackbone.from_preset(
    "densenet_121_imagenet"
)
model(input_data)

# Randomly initialized backbone with a custom config
model = keras_hub.models.DenseNetBackbone(
    stackwise_num_repeats=[6, 12, 24, 16],
)
model(input_data)
```

--------------------------------

### Install KerasHub and Keras 3

Source: https://keras.io/keras_hub/guides/classification_with_keras_hub

Install the necessary packages to get started with KerasHub and Keras 3.

```bash
!!pip install -q --upgrade keras-hub
!!pip install -q --upgrade keras  # Upgrade to Keras 3.
```

--------------------------------

### DistilBertBackbone Usage Examples

Source: https://keras.io/keras_hub/api/models/distil_bert/distil_bert_backbone

Demonstrates loading a pre-trained DistilBERT model and initializing a custom configuration.

```python
input_data = {
    "token_ids": np.ones(shape=(1, 12), dtype="int32"),
    "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]]),
}

# Pretrained DistilBERT encoder.
model = keras_hub.models.DistilBertBackbone.from_preset(
    "distil_bert_base_en_uncased"
)
model(input_data)

# Randomly initialized DistilBERT encoder with custom config.
model = keras_hub.models.DistilBertBackbone(
    vocabulary_size=30552,
    num_layers=4,
    num_heads=4,
    hidden_dim=256,
    intermediate_dim=512,
    max_sequence_length=128,
)
model(input_data)
```

--------------------------------

### Use FalconBackbone Examples

Source: https://keras.io/keras_hub/api/models/falcon/falcon_backbone

Demonstrates loading a pretrained Falcon model and initializing a custom configuration from scratch.

```python
input_data = {
    "token_ids": np.ones(shape=(1, 12), dtype="int32"),
    "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]]),
}

# Pretrained Falcon decoder.
# TODO: Update the preset.
model = keras_hub.models.FalconBackbone.from_preset("falcon_preset")
model(input_data)

# Randomly initialized Falcon decoder with a custom config.
model = keras_hub.models.FalconBackbone(
    vocabulary_size=10,
    num_layers=2,
    num_attention_heads=2,
    hidden_dim=32,
    intermediate_dim=32*4,
    layer_norm_epsilon=1e-5,
    attention_dropout_rate=0,
    feedforward_dropout_rate=0,
    dtype="float32",
)
model(input_data)
```

--------------------------------

### Pip Update Notice

Source: https://keras.io/keras_hub/guides/classification_with_keras_hub

Example output from the pip installation process indicating a new version is available.

```text
['',
 '\x1b[1m[\x1b[0m\x1b[34;49mnotice\x1b[0m\x1b[1;39;49m]\x1b[0m\x1b[39;49m A new release of pip is available: \x1b[0m\x1b[31;49m23.0.1\x1b[0m\x1b[39;49m -> \x1b[0m\x1b[32;49m24.2\x1b[0m',
 '\x1b[1m[\x1b[0m\x1b[34;49mnotice\x1b[0m\x1b[1;39;49m]\x1b[0m\x1b[39;49m To update, run: \x1b[0m\x1b[32;49mpip install --upgrade pip\x1b[0m']
```

--------------------------------

### Initialize GPT2CausalLM from preset

Source: https://keras.io/keras_hub/api/models/gpt2/gpt2_causal_lm

Instantiate a GPT2CausalLM model using a preset configuration. This is the recommended way to get started with the model.

```python
gpt2_lm = keras_hub.models.GPT2CausalLM.from_preset("gpt2_base_en")
```

--------------------------------

### Example Flight Data

Source: https://keras.io/keras_hub/guides/function_calling_with_keras_hub

A JSON representation of flight data used for tool interaction.

```json
[{"id": 1, "price": "USD 220", "stops": 2, "duration": 4.5}, {"id": 2, "price": "USD 22", "stops": 1, "duration": 2.0}, {"id": 3, "price": "USD 240", "stops": 2, "duration": 13.2}]
```

--------------------------------

### Instantiate T5Gemma2Backbone

Source: https://keras.io/keras_hub/api/models/t5gemma2/t5gemma2_backbone

Instantiates the T5Gemma2 backbone model with specified encoder and decoder configurations. This example shows a basic setup for text-only input.

```python
import numpy as np
from keras_hub.models import T5Gemma2Backbone

input_data = {
    "encoder_token_ids": np.ones(shape=(1, 12), dtype="int32"),
    "encoder_padding_mask": np.array(
        [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]], dtype="int32"
    ),
    "decoder_token_ids": np.ones(shape=(1, 8), dtype="int32"),
    "decoder_padding_mask": np.array(
        [[1, 1, 1, 1, 1, 1, 1, 1]], dtype="int32"
    ),
}

model = T5Gemma2Backbone(
    vocabulary_size=32000,
    encoder_hidden_dim=256,
    encoder_intermediate_dim=512,
    encoder_num_layers=4,
    encoder_num_attention_heads=4,
    encoder_num_key_value_heads=2,
    encoder_head_dim=64,
```

--------------------------------

### Use ViTDetBackbone with presets and custom configurations

Source: https://keras.io/keras_hub/api/models/vit_det/ViTDetBackbone

Examples showing how to load a pretrained ViTDetBackbone or initialize one with custom parameters.

```python
input_data = np.ones((2, 224, 224, 3), dtype="float32")

# Pretrained ViTDetBackbone backbone.
model = keras_hub.models.ViTDetBackbone.from_preset("vit_det")
model(input_data)

# Randomly initialized ViTDetBackbone backbone with a custom config.
model = keras_hub.models.ViTDetBackbone(
        image_shape = (16, 16, 3),
        patch_size = 2,
        hidden_size = 4,
        num_layers = 2,
        global_attention_layer_indices = [2, 5, 8, 11],
        intermediate_dim = 4 * 4,
        num_heads = 2,
        num_output_channels = 2,
        window_size = 2,
)
model(input_data)
```

--------------------------------

### Use Qwen3Backbone with Presets and Custom Config

Source: https://keras.io/keras_hub/api/models/qwen3/qwen3_backbone

Demonstrates loading a pre-trained Qwen3 model and initializing a custom model from scratch.

```python
input_data = {
    "token_ids": np.ones(shape=(1, 12), dtype="int32"),
    "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]]),
}

# Pretrained Qwen3 decoder.
model = keras_hub.models.Qwen3Backbone.from_preset("qwen32.5_0.5b_en")
model(input_data)

# Randomly initialized Qwen3 decoder with custom config.
model = keras_hub.models.Qwen3Backbone(
    vocabulary_size=10,
    hidden_dim=512,
    num_layers=2,
    num_query_heads=32,
    num_key_value_heads=8,
    intermediate_dim=1024,
    layer_norm_epsilon=1e-6,
    dtype="float32"
)
model(input_data)
```

--------------------------------

### Instantiate FalconCausalLM from Preset

Source: https://keras.io/keras_hub/api/models/falcon/falcon_causal_lm

Instantiates an end-to-end Falcon model for causal language modeling using a preset configuration. This is the recommended way to get started.

```python
falcon_lm = keras_hub.models.FalconCausalLM.from_preset(
    "falcon_refinedweb_1b_en"
)
```

--------------------------------

### Usage Example for Qwen3_5MoeBackbone

Source: https://keras.io/keras_hub/api/models/qwen3_5_moe/qwen3_5_moe_backbone

Demonstrates initializing the backbone with specific hyperparameters and performing a forward pass with dummy input data.

```python
input_data = {
    "token_ids": np.ones(shape=(1, 12), dtype="int32"),
    "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]]),
}

model = keras_hub.models.Qwen3_5MoeBackbone(
    vocabulary_size=248320,
    num_layers=4,
    num_query_heads=16,
    num_key_value_heads=2,
    head_dim=256,
    hidden_dim=2048,
    moe_intermediate_dim=512,
    shared_expert_intermediate_size=512,
    num_experts=8,
    top_k=2,
)
model(input_data)
```

--------------------------------

### Automated Tool Calling Execution

Source: https://keras.io/keras_hub/guides/function_calling_with_keras_hub

Executes the automated tool calling example to demonstrate the model's ability to invoke functions.

```python
print("Running automated tool calling example:")
automated_tool_calling_example()
```

--------------------------------

### Instantiate BASNetImageSegmenter Model

Source: https://keras.io/keras_hub/api/models/basnet/basnet_image_segmenter

Instantiate the BASNetImageSegmenter model with a specified backbone and optional preprocessor. Use this for image segmentation tasks. The example demonstrates model evaluation and training setup.

```python
import keras_hub

images = np.ones(shape=(1, 288, 288, 3))
labels = np.zeros(shape=(1, 288, 288, 1))

image_encoder = keras_hub.models.ResNetBackbone.from_preset(
    "resnet_18_imagenet",
    load_weights=False
)
backbone = keras_hub.models.BASNetBackbone(
    image_encoder,
    num_classes=1,
    image_shape=[288, 288, 3]
)
model = keras_hub.models.BASNetImageSegmenter(backbone)

# Evaluate the model
pred_labels = model(images)

# Train the model
model.compile(
    optimizer="adam",
    loss=keras.losses.BinaryCrossentropy(from_logits=False),
    metrics=["accuracy"],
)
model.fit(images, labels, epochs=3)

```

--------------------------------

### SwinTransformerBackbone usage examples

Source: https://keras.io/keras_hub/api/models/swin_transformer/swin_transformer_backbone

Demonstrates loading a pretrained model from a preset and initializing a custom model from scratch.

```python
# Pretrained Swin Transformer backbone.
model = keras_hub.models.SwinTransformerBackbone.from_preset(
    "swin_tiny_224"
)
model(np.ones((1, 224, 224, 3)))

# Randomly initialized Swin Transformer with custom config.
model = keras_hub.models.SwinTransformerBackbone(
    image_shape=(224, 224, 3),
    embed_dim=96,
    depths=(2, 2, 6, 2),
    num_heads=(3, 6, 12, 24),
    window_size=7,
)
model(np.ones((1, 224, 224, 3)))
```

--------------------------------

### Instantiate and run SAM3PromptableConceptBackbone

Source: https://keras.io/keras_hub/api/models/sam3/sam3_pc_backbone

Example showing the initialization of all required sub-components and a forward pass with dummy input data.

```python
import numpy as np
import keras_hub

vision_encoder = keras_hub.layers.SAM3VisionEncoder(
    image_shape=(224, 224, 3),
    patch_size=14,
    num_layers=2,
    hidden_dim=32,
    intermediate_dim=128,
    num_heads=2,
    fpn_hidden_dim=32,
    fpn_scale_factors=[4.0, 2.0, 1.0, 0.5],
    pretrain_image_shape=(112, 112, 3),
    window_size=2,
    global_attn_indexes=[1, 2],
)
text_encoder = keras_hub.layers.SAM3TextEncoder(
    vocabulary_size=1024,
    embedding_dim=32,
    hidden_dim=32,
    num_layers=2,
    num_heads=2,
    intermediate_dim=128,
)
geometry_encoder = keras_hub.layers.SAM3GeometryEncoder(
    num_layers=3,
    hidden_dim=32,
    intermediate_dim=128,
    num_heads=2,
    roi_size=7,
)
detr_encoder = keras_hub.layers.SAM3DetrEncoder(
    num_layers=3,
    hidden_dim=32,
    intermediate_dim=128,
    num_heads=2,
)
detr_decoder = keras_hub.layers.SAM3DetrDecoder(
    image_shape=(224, 224, 3),
    patch_size=14,
    num_layers=2,
    hidden_dim=32,
    intermediate_dim=128,
    num_heads=2,
    num_queries=100,
)
mask_decoder = keras_hub.layers.SAM3MaskDecoder(
    num_upsampling_stages=3,
    hidden_dim=32,
    num_heads=2,
)
backbone = keras_hub.models.SAM3PromptableConceptBackbone(
    vision_encoder=vision_encoder,
    text_encoder=text_encoder,
    geometry_encoder=geometry_encoder,
    detr_encoder=detr_encoder,
    detr_decoder=detr_decoder,
    mask_decoder=mask_decoder,
)
input_data = {
    "pixel_values": np.ones((2, 224, 224, 3), dtype="float32"),
    "token_ids": np.ones((2, 32), dtype="int32"),
    "padding_mask": np.ones((2, 32), dtype="bool"),
    "boxes": np.zeros((2, 1, 5), dtype="float32"),
    "box_labels": np.zeros((2, 1), dtype="int32"),
}
outputs = backbone(input_data)
```

--------------------------------

### Instantiate and Use TransformerDecoder Layer

Source: https://keras.io/keras_hub/api/modeling_layers/transformer_decoder

Demonstrates how to create a TransformerDecoder layer, build a Keras model with it, and call the model with sample input data. This example shows an encoder-decoder setup.

```python
# Create a single transformer decoder layer.
declared_decoder = keras_hub.layers.TransformerDecoder(
    intermediate_dim=64, num_heads=8)

# Create a simple model containing the decoder.
declared_decoder_input = keras.Input(shape=(10, 64))
encoder_input = keras.Input(shape=(10, 64))
declared_output = declared_decoder(declared_decoder_input, encoder_input)
model = keras.Model(
    inputs=(declared_decoder_input, encoder_input),
    outputs=declared_output,
)

# Call decoder on the inputs.
declared_decoder_input_data = np.random.uniform(size=(2, 10, 64))
encoder_input_data = np.random.uniform(size=(2, 10, 64))
declared_decoder_output = model((declared_decoder_input_data, encoder_input_data))

```

--------------------------------

### Gemma3nCausalLMPreprocessor usage examples

Source: https://keras.io/keras_hub/api/models/gemma3n/gemma3n_causal_lm_preprocessor

Demonstrates loading the preprocessor from a preset and handling various input modalities including text, images, and audio for both training and generation tasks.

```python
# === Language ===
# Load the preprocessor from a preset.
preprocessor = keras_hub.models.Gemma3nCausalLMPreprocessor.from_preset(
    "gemma3n_2b_it"
)

# Unbatched inputs.
preprocessor(
    {
        "prompts": "What is the capital of India?",
        "responses": "New Delhi",
    }
)

# Batched inputs.
preprocessor(
    {
        "prompts": [
            "What is the capital of India?",
            "What is the capital of Spain?"
        ],
        "responses": ["New Delhi", "Madrid"],
    }
)

# Apply preprocessing to a [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset).
features = {
    "prompts": [
        "What is the capital of India?",
        "What is the capital of Spain?"
    ],
    "responses": ["New Delhi", "Madrid"],
}

ds = tf.data.Dataset.from_tensor_slices(features)
ds = ds.map(preprocessor, num_parallel_calls=tf.data.AUTOTUNE)

# Prepare tokens for generation (no end token).
preprocessor.generate_preprocess(["The quick brown fox jumped."])

# Map generation outputs back to strings.
preprocessor.generate_postprocess({
    'token_ids': np.array([[2, 818, 3823, 8864, 37423, 32694, 236761, 0]]),
    'padding_mask': np.array([[ 1, 1, 1, 1, 1, 1, 1, 0]]),
})

# === Vision and Language ===
# Load the preprocessor from a preset.
preprocessor = keras_hub.models.Gemma3nCausalLMPreprocessor.from_preset(
    "gemma3n_2b_it"
)

# Text-only inputs (unbatched).
preprocessor(
    {
        "prompts": "What is the capital of India?",
        "responses": "New Delhi",
    }
)

# Text-only inputs (batched).
preprocessor(
    {
        "prompts": [
            "What is the capital of India?",
            "What is the capital of Spain?"
        ],
        "responses": ["New Delhi", "Madrid"],
    }
)

# Unbatched inputs, with one image.
preprocessor(
    {
        "prompts": "this is a lily ",
        "responses": "pristine!",
        "images": np.ones((768, 768, 3), dtype="float32")
    }
)

# Unbatched inputs, with two images.
preprocessor(
    {
        "prompts": "lily: , sunflower: ",
        "responses": "pristine!",
        "images": [
            np.ones((768, 768, 3), dtype="float32"),
            np.ones((768, 768, 3), dtype="float32")
        ],
    }
)

# Batched inputs, one image per prompt.
preprocessor(
    {
        "prompts": [
            "this is a lily: ",
            "this is a sunflower: "
        ],
        "responses": ["pristine!", "radiant!"],
        "images": [
            np.ones((768, 768, 3), dtype="float32"),
            np.ones((768, 768, 3), dtype="float32")
        ]
    }
)

# === Audio and Language ===
# Unbatched inputs, with one audio clip.
preprocessor(
    {
        "prompts": "transcribe this: <start_of_audio>",
        "responses": "hello world",
        "audios": np.ones((16000,), dtype="float32")
    }
)

# === Vision, Audio and Language ===
# Unbatched inputs, with one image and one audio.
preprocessor(
    {
        "prompts": "image: , audio: <start_of_audio>",
        "responses": "multimodal!",
        "images": np.ones((768, 768, 3), dtype="float32"),
        "audios": np.ones((16000,), dtype="float32")
    }
)
```

--------------------------------

### Instantiate SigLIPBackbone with Pretrained Weights

Source: https://keras.io/keras_hub/api/models/siglip/siglip_backbone

Load a pretrained SigLIP base model using the from_preset constructor. This is useful for quickly getting started with a SigLIP model with established weights.

```python
input_data = {
    "images": np.ones(shape=(1, 224, 224, 3), dtype="float32"),
    "token_ids": np.ones(shape=(1, 64), dtype="int32"),
}

# Pretrained SigLIP model.
model = keras_hub.models.SigLIPBackbone.from_preset(
    "siglip_base_patch16_224"
)
model(input_data)
```

--------------------------------

### Install Dependencies

Source: https://keras.io/keras_hub/guides/semantic_segmentation_deeplab_v3

Install the required KerasHub and Keras packages.

```bash
!pip install -q --upgrade keras-hub
!pip install -q --upgrade keras
```

--------------------------------

### Initialize Gemma3nBackbone with custom configuration

Source: https://keras.io/keras_hub/api/models/gemma3n/gemma3n_backbone

Demonstrates how to configure the vision encoder, audio encoder, and backbone components, then perform a forward pass with dummy input data.

```python
import numpy as np
from keras_hub.src.models.gemma3n.gemma3n_audio_encoder import (
    Gemma3nAudioEncoder,
)
from keras_hub.src.models.gemma3n.gemma3n_backbone import Gemma3nBackbone
from keras_hub.src.models.mobilenetv5.mobilenetv5_backbone import (
    MobileNetV5Backbone,
)
from keras_hub.src.models.mobilenetv5.mobilenetv5_builder import (
    convert_arch_def_to_stackwise,
)

# Vision encoder config.
vision_arch_def = [["er_r1_k3_s1_e1_c16"]]
stackwise_params = convert_arch_def_to_stackwise(vision_arch_def)
vision_encoder = MobileNetV5Backbone(
    **stackwise_params,
    num_features=4,
    image_shape=(224, 224, 3),
    use_msfa=False,
)

# Audio encoder config.
audio_encoder = Gemma3nAudioEncoder(
    hidden_size=8,
    input_feat_size=32,
    sscp_conv_channel_size=[4, 8],
    sscp_conv_kernel_size=[(3, 3), (3, 3)],
    sscp_conv_stride_size=[(2, 2), (2, 2)],
    sscp_conv_group_norm_eps=1e-5,
    conf_num_hidden_layers=1,
    rms_norm_eps=1e-6,
    gradient_clipping=1.0,
    conf_residual_weight=0.5,
    conf_num_attention_heads=1,
    conf_attention_chunk_size=4,
    conf_attention_context_right=5,
    conf_attention_context_left=5,
    conf_attention_logit_cap=50.0,
    conf_conv_kernel_size=5,
    conf_reduction_factor=1,
)

# Backbone config.
backbone = Gemma3nBackbone(
    text_vocab_size=50,
    text_hidden_size=8,
    num_hidden_layers=1,
    pad_token_id=0,
    num_attention_heads=1,
    num_key_value_heads=1,
    head_dim=8,
    intermediate_size=[16],
    hidden_activation="gelu_approximate",
    layer_types=["full_attention"],
    sliding_window=4,
    rope_theta=10000.0,
    max_position_embeddings=16,
    vocab_size_per_layer_input=50,
    hidden_size_per_layer_input=2,
    altup_num_inputs=2,
    laurel_rank=1,
    vision_encoder_config=vision_encoder.get_config(),
    vision_hidden_size=16,
    audio_encoder_config=audio_encoder.get_config(),
    audio_hidden_size=8,
)

# Create dummy inputs.
input_data = {
    "token_ids": np.random.randint(0, 50, size=(1, 16), dtype="int32"),
    "attention_mask": np.ones((1, 1, 16, 16), dtype=bool),
    "images": np.random.rand(1, 1, 224, 224, 3).astype("float32"),
    "input_features": np.random.rand(1, 16, 32).astype("float32"),
    "input_features_mask": np.zeros((1, 16), dtype=bool),
}

# Forward pass.
outputs = backbone(input_data)
```

--------------------------------

### Examples of from_preset usage

Source: https://keras.io/keras_hub/api/models/blip2/blip2_image_converter

Shows how to load converters from different presets and apply them to image batches.

```python
batch = np.random.randint(0, 256, size=(2, 512, 512, 3))

# Resize images for "pali_gemma_3b_224".
converter = keras_hub.layers.ImageConverter.from_preset(
    "pali_gemma_3b_224"
)
converter(batch) # # Output shape (2, 224, 224, 3)

# Resize images for "pali_gemma_3b_448" without cropping.
converter = keras_hub.layers.ImageConverter.from_preset(
    "pali_gemma_3b_448",
    crop_to_aspect_ratio=False,
)
converter(batch) # # Output shape (2, 448, 448, 3)
```

--------------------------------

### Install Grounding DINO

Source: https://keras.io/keras_hub/guides/segment_anything_in_keras_hub

Install the Grounding DINO package from the official repository.

```bash
pip install -U git+https://github.com/IDEA-Research/GroundingDINO.git
```

--------------------------------

### Install Dependencies

Source: https://keras.io/keras_hub/guides/object_detection_retinanet

Install the necessary packages to run the object detection tutorial.

```bash
!pip install -q --upgrade keras-hub
!pip install -q --upgrade keras
!pip install -q opencv-python
```

--------------------------------

### Use RWKV7CausalLM for generation

Source: https://keras.io/keras_hub/api/models/rwkv7/rwkv7_causal_lm

Example showing how to initialize the model, configure the preprocessor, and generate text using a greedy sampler.

```python
# Initialize the tokenizer and load assets from a local path.
tokenizer = RWKVTokenizer()
tokenizer.load_assets(rwkv_path)

# Create a preprocessor with a sequence length of 8.
preprocessor = RWKV7CausalLMPreprocessor(tokenizer, sequence_length=8)

# Initialize the model with a backbone and preprocessor.
causal_lm = RWKV7CausalLM(backbone, preprocessor)

# you also can load model by from_preset
rwkv_path = "RWKV7_G1a_0.1B"
tokenizer = RWKVTokenizer.from_preset(rwkv_path)
causal_lm = RWKV7CausalLM.from_preset(rwkv_path)

prompts = ["Bubble sort\n\n```python", "Hello World"]

    causal_lm.compile(sampler="greedy")

    outputs = causal_lm.generate(prompts, max_length=128)
    for out in outputs:
        print(out)
        print("-" * 100)
```

--------------------------------

### Install KerasHub

Source: https://keras.io/keras_hub

Commands to install the stable or nightly versions of the KerasHub library.

```bash
pip install --upgrade keras-hub
```

```bash
pip install --upgrade keras-hub-nightly
```

--------------------------------

### Initialize and use RWKV7CausalLMPreprocessor

Source: https://keras.io/keras_hub/api/models/rwkv7/rwkv7_causal_lm_preprocessor

Demonstrates initializing the preprocessor with a tokenizer and processing input strings for training or generation.

```python
# Initialize the tokenizer and load assets from a local path.
tokenizer = RWKVTokenizer()
tokenizer.load_assets(rwkv_path)

# Create a preprocessor with a sequence length of 8.
preprocessor = RWKV7CausalLMPreprocessor(tokenizer, sequence_length=8)

# Tokenize and pack a batch of sentences.
preprocessor(["Bubble sort", "Hello World"])

# Preprocess inputs for generation with a maximum generation length of 16.
preprocessor.generate_preprocess(
    ["Bubble sort", "Hello World"], 16
)
```

--------------------------------

### Install Required Packages

Source: https://keras.io/keras_hub/guides/gemma4_multimodal_and_agentic_workflows

Install the necessary Keras, KerasHub, and media processing libraries.

```bash
!pip install -q -U keras keras-hub
!pip install -q -U soundfile scipy requests pillow matplotlib av
```

--------------------------------

### Instantiate and Use QwenBackbone

Source: https://keras.io/keras_hub/api/models/qwen/qwen_backbone

Demonstrates how to instantiate and use the QwenBackbone model. Shows loading a preset model and initializing a custom model with specific configurations. Requires numpy and keras_hub.

```python
input_data = {
    "token_ids": np.ones(shape=(1, 12), dtype="int32"),
    "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]])
}

# Pretrained Qwen decoder.
model = keras_hub.models.QwenBackbone.from_preset("qwen2.5_0.5b_en")
model(input_data)

# Randomly initialized Qwen decoder with custom config.
model = keras_hub.models.QwenBackbone(
    vocabulary_size=10,
    hidden_dim=512,
    num_layers=2,
    num_query_heads=32,
    num_key_value_heads=8,
    intermediate_dim=1024,
    layer_norm_epsilon=1e-6,
    dtype="float32"
)
model(input_data)

```

--------------------------------

### Install required packages

Source: https://keras.io/keras_hub/guides/function_gemma_with_keras

Install the necessary libraries for KerasHub and external tool integrations.

```bash
pip install -U keras-hub
pip install yfinance ddgs psutil pytz
```

--------------------------------

### VGGImageClassifier Usage Examples

Source: https://keras.io/keras_hub/api/models/vgg/vgg_image_classifier

Examples demonstrating how to load, predict with, and train the VGGImageClassifier model.

```APIDOC
### Examples

#### Call `predict()` to run inference.

```python
# Load preset and train
images = np.random.randint(0, 256, size=(2, 224, 224, 3))
classifier = keras_hub.models.VGGImageClassifier.from_preset(
    "vgg_16_imagenet"
)
classifier.predict(images)
```

#### Call `fit()` on a single batch.

```python
# Load preset and train
images = np.random.randint(0, 256, size=(2, 224, 224, 3))
labels = [0, 3]
classifier = keras_hub.models.VGGImageClassifier.from_preset(
    "vgg_16_imagenet"
)
classifier.fit(x=images, y=labels, batch_size=2)
```

#### Call `fit()` with custom loss, optimizer and backbone.

```python
classifier = keras_hub.models.VGGImageClassifier.from_preset(
    "vgg_16_imagenet"
)
classifier.compile(
    loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    optimizer=keras.optimizers.Adam(5e-5),
)
classifier.backbone.trainable = False
classifier.fit(x=images, y=labels, batch_size=2)
```

#### Custom backbone.

```python
images = np.random.randint(0, 256, size=(2, 224, 224, 3))
labels = [0, 3]
backbone = keras_hub.models.VGGBackbone(
    stackwise_num_repeats = [2, 2, 3, 3, 3],
    stackwise_num_filters = [64, 128, 256, 512, 512],
    image_shape = (224, 224, 3),
)
classifier = keras_hub.models.VGGImageClassifier(
    backbone=backbone,
    num_classes=4,
)
classifier.fit(x=images, y=labels, batch_size=2)
```
```

--------------------------------

### Use ElectraBackbone with Presets and Custom Initialization

Source: https://keras.io/keras_hub/api/models/electra/electra_backbone

Demonstrates loading a pre-trained ELECTRA model and initializing a custom backbone from scratch.

```python
input_data = {
"token_ids": np.ones(shape=(1, 12), dtype="int32"),
"segment_ids": np.array([[0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0]]),
"padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]]),
}

# Pre-trained ELECTRA encoder.
model = keras_hub.models.ElectraBackbone.from_preset(
    "electra_base_discriminator_en"
)
model(input_data)

# Randomly initialized Electra encoder
backbone = keras_hub.models.ElectraBackbone(
    vocabulary_size=1000,
    num_layers=2,
    num_heads=2,
    hidden_dim=32,
    intermediate_dim=64,
    dropout=0.1,
    max_sequence_length=512,
    )
# Returns sequence and pooled outputs.
sequence_output, pooled_output = backbone(input_data)
```

--------------------------------

### WhitespaceSplitterTokenizer Example

Source: https://keras.io/keras_hub/api/tokenizers/tokenizer

Example of subclassing Tokenizer to create a simple whitespace splitter. Implements tokenize and detokenize methods.

```python
class WhitespaceSplitterTokenizer(keras_hub.tokenizers.Tokenizer):
    def tokenize(self, inputs):
        return tf.strings.split(inputs)

    def detokenize(self, inputs):
        return tf.strings.reduce_join(inputs, separator=" ", axis=-1)

tokenizer = WhitespaceSplitterTokenizer()

# Tokenize some inputs.
tokenizer.tokenize("This is a test")

# Shorthard for `tokenize()`.
tokenizer("This is a test")

# Detokenize some outputs.
tokenizer.detokenize(["This", "is", "a", "test"])
```

--------------------------------

### Instantiating a MobileNetBackbone Model

Source: https://keras.io/keras_hub/api/models/mobilenet/mobilenet_backbone

Example showing how to initialize a MobileNetBackbone with a custom configuration and pass input data through it.

```python
input_data = tf.ones(shape=(8, 224, 224, 3))

# Randomly initialized backbone with a custom config
model = MobileNetBackbone(
    stackwise_expansion=[
            [40, 56],
            [64, 144, 144],
            [72, 72],
            [144, 288, 288],
        ],
        stackwise_num_blocks=[2, 3, 2, 3],
        stackwise_num_filters=[
            [16, 16],
            [24, 24, 24],
            [24, 24],
            [48, 48, 48],
        ],
        stackwise_kernel_size=[[3, 3], [5, 5, 5], [5, 5], [5, 5, 5]],
        stackwise_num_strides=[[2, 1], [2, 1, 1], [1, 1], [2, 1, 1]],
        stackwise_se_ratio=[
            [None, None],
            [0.25, 0.25, 0.25],
            [0.3, 0.3],
            [0.3, 0.25, 0.25],
        ],
        stackwise_activation=[
            ["relu", "relu"],
            ["hard_swish", "hard_swish", "hard_swish"],
            ["hard_swish", "hard_swish"],
            ["hard_swish", "hard_swish", "hard_swish"],
        ],
        output_num_filters=288,
        input_activation="hard_swish",
        output_activation="hard_swish",
        input_num_filters=16,
        image_shape=(224, 224, 3),
        depthwise_filters=8,
        squeeze_and_excite=0.5,

)
output = model(input_data)
```

--------------------------------

### Initialize and use MoonshineAudioToTextPreprocessor

Source: https://keras.io/keras_hub/api/models/moonshine/moonshine_audio_to_text_preprocessor

Demonstrates creating a preprocessor instance with an audio converter and tokenizer, then processing inputs for training and generation.

```python
import keras
from keras_hub.layers import MoonshineAudioConverter
from keras_hub.models import MoonshineTokenizer

# Create audio converter and tokenizer instances.
audio_converter = MoonshineAudioConverter()
tokenizer = MoonshineTokenizer.from_preset("moonshine_base")

# Initialize the preprocessor.
preprocessor = keras_hub.models.MoonshineAudioToTextPreprocessor(
    audio_converter=audio_converter,
    tokenizer=tokenizer,
    decoder_sequence_length=8
)

# Prepare input data (audio tensor and text).
inputs = {
    "audio": keras.random.normal((1, 16000)),
    "text": ["the quick brown fox"]
}

# Process the inputs for training.
x, y, sample_weight = preprocessor(inputs)

# Check output keys and shapes (shapes depend on padding/truncation).
print(x.keys())
# dict_keys(['encoder_input_values', 'encoder_padding_mask',
# 'decoder_token_ids', 'decoder_padding_mask']).
print(x["encoder_input_values"].shape) # e.g., (1, 16000, 1) / padded length
print(x["encoder_padding_mask"].shape) # e.g., (1, 16000) or padded length
print(x["decoder_token_ids"].shape) # (1, 8)
print(x["decoder_padding_mask"].shape) # (1, 8)
print(y.shape) # (1, 8) - Labels
print(sample_weight.shape) # (1, 8) - Sample weights

# Process inputs for generation.
gen_inputs = preprocessor.generate_preprocess(inputs)
print(gen_inputs.keys())
# dict_keys(['encoder_input_values', 'encoder_padding_mask',
# 'decoder_token_ids', 'decoder_padding_mask']).
```

--------------------------------

### BartTokenizer Usage Examples

Source: https://keras.io/keras_hub/api/models/bart/bart_tokenizer

Examples demonstrating unbatched and batched input tokenization, detokenization, and custom vocabulary initialization.

```python
# Unbatched input.
tokenizer = keras_hub.models.BartTokenizer.from_preset(
    "bart_base_en",
)
tokenizer("The quick brown fox jumped.")

# Batched input.
tokenizer(["The quick brown fox jumped.", "The fox slept."])

# Detokenization.
tokenizer.detokenize(tokenizer("The quick brown fox jumped."))

# Custom vocabulary.
vocab = {"<s>": 0, "<pad>": 1, "</s>": 2, "<mask>": 3}
vocab = {**vocab, "a": 4, "Ġquick": 5, "Ġfox": 6}
merges = ["Ġ q", "u i", "c k", "ui ck", "Ġq uick"]
merges += ["Ġ f", "o x", "Ġf ox"]
tokenizer = keras_hub.models.BartTokenizer(
    vocabulary=vocab,
    merges=merges,
)
tokenizer("The quick brown fox jumped.")
```

--------------------------------

### Use Seq2SeqLMPreprocessor for Preprocessing

Source: https://keras.io/keras_hub/api/base_classes/seq_2_seq_lm_preprocessor

Demonstrates how to load a preprocessor from a preset, process text inputs, and use the generate methods.

```python
preprocessor = keras_hub.models.Seq2SeqLMPreprocessor.from_preset(
    "bart_base_en",
    encoder_sequence_length=256,
    decoder_sequence_length=256,
)

# Tokenize, mask and pack a single sentence.
x = {
    "encoder_text": "The fox was sleeping.",
    "decoder_text": "The fox was awake.",
}
x, y, sample_weight = preprocessor(x)

# Tokenize and pad/truncate a batch of labeled sentences.
x = {
    "encoder_text": ["The fox was sleeping."],
    "decoder_text": ["The fox was awake."],
x, y, sample_weight = preprocessor(x)

# With a [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset).
ds = tf.data.Dataset.from_tensor_slices(x)
ds = ds.map(preprocessor, num_parallel_calls=tf.data.AUTOTUNE)

# Generate preprocess and postprocess.
x = preprocessor.generate_preprocess(x)  # Tokenized numeric inputs.
x = preprocessor.generate_postprocess(x)  # Detokenized string outputs.
```
Tokens:
Raw