EmbeddingGemma-300M

https://github.com/zsxkib/cog-google-embeddinggemma-300m
Admin
This project provides text embeddings using Google's GEMMA model, generating 768-dimensional vectors...
Tokens:6,819
Snippets:28
Trust Score:9.3
License:MIT
Update:5 months ago
Show doc for...
Tokens:
Raw
### Generate Embeddings for Various Tasks

Source: https://context7.com/zsxkib/cog-google-embeddinggemma-300m/llms.txt

Generates embeddings for different text types and tasks using the predictor. It iterates through a dictionary of tasks and corresponding example texts, generating embeddings and printing their dimensions. This demonstrates the model's versatility across different use cases.

```python
tasks_examples = {
    "retrieval_query": "How to train neural networks?",
    "retrieval_document": "Neural networks are trained using backpropagation and gradient descent.",
    "classification": "This movie was absolutely fantastic!",
    "clustering": "Apple announces new iPhone with improved camera",
    "question_answering": "What is the capital of France?",
    "fact_verification": "The Earth orbits around the Sun",
    "semantic_similarity": "Dogs are loyal pets",
    "code_retrieval": "def calculate_mean(numbers): return sum(numbers) / len(numbers)"
}

# Generate embeddings for each task type
embeddings = {}
for task, text in tasks_examples.items():
    emb = predictor.predict(
        text=text,
        task=task,
        output_format="array",
        embedding_dim=256
    )
    embeddings[task] = emb
    print(f"{task}: {len(emb)} dimensions")
```

--------------------------------

### Generate Text Embeddings with Custom Dimensions (JavaScript)

Source: https://github.com/zsxkib/cog-google-embeddinggemma-300m/blob/main/REPLICATE_README.md

This JavaScript example demonstrates generating text embeddings with a specified dimension (e.g., 512) using the EmbeddingGemma-300M model on Replicate. By setting the `embedding_dim` parameter, the output is truncated to the desired size. The code converts the resulting base64 string into a Float32Array.

```javascript
const b64_512 = await replicate.run("zsxkib/embedding-gemma-300m", { input: { text: "Hello world", embedding_dim: 512 } });
const embedding_512 = new Float32Array(Buffer.from(b64_512, 'base64').buffer);
console.log(embedding_512.length);  // 512
```

--------------------------------

### Generate Embeddings with Predictor.predict() (Python)

Source: https://context7.com/zsxkib/cog-google-embeddinggemma-300m/llms.txt

Provides examples of using the Predictor.predict() method to generate text embeddings with various configurations. Demonstrates generating document and query embeddings, classification embeddings, and semantic similarity calculations, showcasing options for task type, embedding dimension, output format, and normalization.

```python
from predict import Predictor
import base64
import numpy as np

predictor = Predictor()
predictor.setup()

# Example 1: Document embedding for retrieval
doc_embedding = predictor.predict(
    text="Python is a high-level programming language known for its simplicity and readability.",
    task="retrieval_document",
    embedding_dim=768,
    output_format="base64",
    normalize=True
)

# Decode and use
doc_vec = np.frombuffer(base64.b64decode(doc_embedding), dtype=np.float32)
print(f"Document embedding norm: {np.linalg.norm(doc_vec):.4f}")  # ~1.0 (normalized)

# Example 2: Query embedding with smaller dimensions
query_embedding = predictor.predict(
    text="What is Python used for?",
    task="retrieval_query",
    embedding_dim=256,
    output_format="array",
    normalize=True
)

print(f"Query embedding length: {len(query_embedding)}")  # 256

# Example 3: Classification task
class_embedding = predictor.predict(
    text="This product exceeded my expectations! Highly recommended.",
    task="classification",
    embedding_dim=512,
    output_format="array"
)

print(f"Classification embedding: {len(class_embedding)} dims")

# Example 4: Semantic similarity
text1_emb = predictor.predict(
    text="The cat sat on the mat",
    task="semantic_similarity",
    output_format="array"
)

text2_emb = predictor.predict(
    text="A feline rested on the rug",
    task="semantic_similarity",
    output_format="array"
)

# Calculate cosine similarity
similarity = np.dot(text1_emb, text2_emb)
print(f"Similarity score: {similarity:.4f}")  # High similarity expected
```

--------------------------------

### Generate Text Embeddings with Custom Dimensions (Python)

Source: https://github.com/zsxkib/cog-google-embeddinggemma-300m/blob/main/REPLICATE_README.md

This Python snippet illustrates how to generate text embeddings with specific dimensions (e.g., 256) using the EmbeddingGemma-300M model on Replicate. It utilizes the `embedding_dim` parameter to truncate the output from the full 768 dimensions, optimizing for storage while maintaining performance. The example decodes the base64 output into a NumPy array.

```python
import replicate, base64, numpy as np

# Get 256-dimensional embedding
b64_256 = replicate.run("zsxkib/embedding-gemma-300m", input={"text": "Hello world", "embedding_dim": 256})

# Decode to numpy array
vec_256 = np.frombuffer(base64.b64decode(b64_256), dtype=np.float32)
print(vec_256.shape)  # (256,)
```

--------------------------------

### Predictor.setup() Method

Source: https://context7.com/zsxkib/cog-google-embeddinggemma-300m/llms.txt

This method initializes the EmbeddingGemma-300M model, downloading weights and loading the model for predictions. It handles GPU acceleration detection.

```APIDOC
## Predictor.setup()

### Description
Initializes the EmbeddingGemma-300M model by downloading weights and loading the SentenceTransformer model. Detects CUDA for GPU acceleration.

### Method
`setup()`

### Endpoint
N/A (Local method call)

### Parameters
None

### Request Example
```python
from predict import Predictor
import os

predictor = Predictor()
os.environ["HF_HOME"] = "model_cache"
predictor.setup()
```

### Response
None (Initializes the model in place)

```

--------------------------------

### Initialize Predictor Model (Python)

Source: https://context7.com/zsxkib/cog-google-embeddinggemma-300m/llms.txt

Shows how to set up the Predictor class for the EmbeddingGemma-300M model. This includes initializing the predictor instance, optionally configuring a cache directory for model weights, and loading the SentenceTransformer model, which automatically detects CUDA for GPU acceleration.

```python
from predict import Predictor
import os

# Setup predictor instance
predictor = Predictor()

# Configure cache directory (optional)
os.environ["HF_HOME"] = "model_cache"

# Initialize model (downloads weights if not cached)
predictor.setup()

# Model is now ready for predictions
print(f"Model loaded on device: {predictor.model.device}")
print(f"Max tokens: {predictor.MAX_TOKENS}")  # 2048
```

--------------------------------

### Predict Embeddings Locally with Cog CLI

Source: https://github.com/zsxkib/cog-google-embeddinggemma-300m/blob/main/README.md

This command demonstrates how to locally predict text embeddings using the Cog CLI. It targets the zsxkib/embedding-gemma-300m model and specifies the input text for embedding. This is useful for local development and testing before deploying.

```bash
cog predict -i text="Hello world"
```

--------------------------------

### Run EmbeddingGemma-300M Model with Python

Source: https://github.com/zsxkib/cog-google-embeddinggemma-300m/blob/main/README.md

This Python snippet demonstrates how to use the replicate library to generate text embeddings with the EmbeddingGemma-300M model. It shows how to handle the default base64 output and decode it into a NumPy float32 array. The function takes a text input and optionally task and output format parameters.

```python
import replicate, base64, numpy as np

# Returns a base64 string by default
b64 = replicate.run(
    "zsxkib/embedding-gemma-300m",
    input={"text": "Your text here"}
)

# Decode base64 -> float32 vector
embedding = np.frombuffer(base64.b64decode(b64), dtype=np.float32)
print(embedding.shape)  # (768,)
```

--------------------------------

### Python Semantic Search System with EmbeddingGemma-300M

Source: https://context7.com/zsxkib/cog-google-embeddinggemma-300m/llms.txt

Implements a semantic search system using the EmbeddingGemma-300M model. It allows adding documents with their embeddings and searching for the most relevant documents based on a query's embedding. Requires the 'replicate' and 'numpy' libraries.

```python
import replicate
import base64
import numpy as np
from typing import List, Tuple

class SemanticSearch:
    def __init__(self):
        self.documents = []
        self.embeddings = []

    def add_documents(self, docs: List[str]):
        """Add documents to the search index"""
        for doc in docs:
            emb_b64 = replicate.run(
                "zsxkib/embedding-gemma-300m",
                input={
                    "text": doc,
                    "task": "retrieval_document",
                    "embedding_dim": 384,
                    "normalize": True
                }
            )
            emb = np.frombuffer(base64.b64decode(emb_b64), dtype=np.float32)
            self.documents.append(doc)
            self.embeddings.append(emb)

        self.embeddings = np.vstack(self.embeddings)
        print(f"Indexed {len(self.documents)} documents")

    def search(self, query: str, top_k: int = 3) -> List[Tuple[str, float]]:
        """Search for most relevant documents"""
        query_b64 = replicate.run(
            "zsxkib/embedding-gemma-300m",
            input={
                "text": query,
                "task": "retrieval_query",
                "embedding_dim": 384,
                "normalize": True
            }
        )
        query_emb = np.frombuffer(base64.b64decode(query_b64), dtype=np.float32)

        # Compute cosine similarities
        similarities = self.embeddings @ query_emb

        # Get top-k results
        top_indices = np.argsort(similarities)[-top_k:][::-1]
        results = [(self.documents[i], float(similarities[i])) for i in top_indices]

        return results

# Usage example
search_engine = SemanticSearch()

# Index documents
docs = [
    "Python is a versatile programming language used for web development, data science, and automation.",
    "Machine learning algorithms can identify patterns in large datasets.",
    "The Renaissance was a period of cultural rebirth in Europe.",
    "Neural networks are inspired by biological brain structure.",
    "Climate change is causing rising sea levels and extreme weather.",
]

search_engine.add_documents(docs)

# Search
query = "What programming languages are good for AI?"
results = search_engine.search(query, top_k=3)

print(f"Query: {query}\n")
for rank, (doc, score) in enumerate(results, 1):
    print(f"{rank}. [Score: {score:.4f}] {doc}")

```

--------------------------------

### Generate Text Embeddings with Replicate API (Python)

Source: https://context7.com/zsxkib/cog-google-embeddinggemma-300m/llms.txt

Demonstrates how to generate text embeddings using the EmbeddingGemma-300M Replicate API endpoint. Includes basic usage with default base64 output and decoding to a NumPy array, as well as advanced usage with configurable parameters like task type, embedding dimension, output format, and normalization.

```python
import replicate
import base64
import numpy as np

# Basic usage with base64 output (default, most efficient)
b64_embedding = replicate.run(
    "zsxkib/embedding-gemma-300m",
    input={"text": "Machine learning is transforming artificial intelligence"}
)

# Decode base64 to numpy array
embedding = np.frombuffer(base64.b64decode(b64_embedding), dtype=np.float32)
print(f"Embedding shape: {embedding.shape}")  # (768,)
print(f"First 5 values: {embedding[:5]}")

# Advanced usage with all parameters
result = replicate.run(
    "zsxkib/embedding-gemma-300m",
    input={
        "text": "What are the benefits of renewable energy?",
        "task": "retrieval_query",
        "embedding_dim": 256,
        "output_format": "array",
        "normalize": True
    }
)
print(f"Embedding dimension: {len(result)}")  # 256
print(f"Embedding values (first 3): {result[:3]}")
```

--------------------------------

### Predictor.predict() Method

Source: https://context7.com/zsxkib/cog-google-embeddinggemma-300m/llms.txt

Generates embeddings for input text with various configuration options including task type, output format, and dimensionality.

```APIDOC
## Predictor.predict()

### Description
Generates embeddings for input text, supporting configurable task types, output formats, and dimensionality. Handles text truncation and task-specific prompting.

### Method
`predict(
    text: str, 
    task: str = "retrieval_document", 
    embedding_dim: int = 768, 
    output_format: str = "base64", 
    normalize: bool = False
)`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **text** (string) - Required - The input text to embed.
- **task** (string) - Optional - The task type to optimize embeddings for. Supported: `retrieval_query`, `retrieval_document`, `classification`, `semantic_similarity`, `generation_query`, `generation_document`, `translation_query`, `translation_document`. Defaults to `retrieval_document`.
- **embedding_dim** (integer) - Optional - The desired dimension of the output embedding. Supported: 128, 256, 512, 768. Defaults to 768.
- **output_format** (string) - Optional - The format for the output embedding. Supported: `base64`, `array`. Defaults to `base64`.
- **normalize** (boolean) - Optional - Whether to normalize the output embeddings. Defaults to `false`.

### Request Example
```python
from predict import Predictor
import base64
import numpy as np

predictor = Predictor()
predictor.setup()

# Document embedding for retrieval
doc_embedding = predictor.predict(
    text="Python is a high-level programming language known for its simplicity and readability.",
    task="retrieval_document",
    embedding_dim=768,
    output_format="base64",
    normalize=True
)

# Query embedding with smaller dimensions
query_embedding = predictor.predict(
    text="What is Python used for?",
    task="retrieval_query",
    embedding_dim=256,
    output_format="array",
    normalize=True
)

# Classification task
class_embedding = predictor.predict(
    text="This product exceeded my expectations! Highly recommended.",
    task="classification",
    embedding_dim=512,
    output_format="array"
)
```

### Response
#### Success Response (200)
- **embedding** (string or array) - The generated text embedding. Format depends on `output_format` parameter.

#### Response Example (base64 output)
```
"ASNFZ+PPAgEAAAAAAAAAAAAA..."
```

#### Response Example (array output)
```json
[
  0.1234,
  -0.5678,
  ... 
]
```

```

--------------------------------

### Base64 Encoding and Decoding for Embeddings

Source: https://context7.com/zsxkib/cog-google-embeddinggemma-300m/llms.txt

Illustrates the use of Base64 encoding for efficient embedding transmission. It shows how to encode a numpy array to a Base64 string and decode a Base64 string from an API response back into a numpy array. This section also compares the storage efficiency of Base64 over raw array representation.

```python
import base64
import numpy as np
import replicate

# Encode embeddings to base64 (automatic in API)
embedding_array = np.random.randn(768).astype(np.float32)
base64_string = base64.b64encode(embedding_array.tobytes()).decode('utf-8')
print(f"Base64 length: {len(base64_string)} characters")

# Decode base64 from API response
api_response = replicate.run(
    "zsxkib/embedding-gemma-300m",
    input={"text": "Example text", "output_format": "base64"}
)

decoded_embedding = np.frombuffer(
    base64.b64decode(api_response),
    dtype=np.float32
)

print(f"Decoded shape: {decoded_embedding.shape}")
print(f"Data type: {decoded_embedding.dtype}")

# Storage comparison
array_size = len(str([float(x) for x in decoded_embedding]))
base64_size = len(api_response)
print(f"Base64 efficiency: {(1 - base64_size/array_size) * 100:.1f}% smaller")
```

--------------------------------

### EmbeddingGemma-300M Replicate API

Source: https://context7.com/zsxkib/cog-google-embeddinggemma-300m/llms.txt

This section details how to use the EmbeddingGemma-300M model through the Replicate API endpoint to generate text embeddings.

```APIDOC
## POST /predictions

### Description
Generates text embeddings using the EmbeddingGemma-300M model via the Replicate API.

### Method
POST

### Endpoint
`https://api.replicate.com/v1/predictions`

### Parameters
#### Request Body
- **version** (string) - Required - The model version. Use `zsxkib/embedding-gemma-300m:59636724f635e022c13a24196c39a7f1185243990042988d6098e4f180784a63` for this model.
- **input** (object) - Required - Input parameters for the model.
  - **text** (string) - Required - The input text to embed.
  - **task** (string) - Optional - The task type to optimize embeddings for. Supported: `retrieval_query`, `retrieval_document`, `classification`, `semantic_similarity`, `generation_query`, `generation_document`, `translation_query`, `translation_document`. Defaults to `retrieval_document`.
  - **embedding_dim** (integer) - Optional - The desired dimension of the output embedding. Supported: 128, 256, 512, 768. Defaults to 768.
  - **output_format** (string) - Optional - The format for the output embedding. Supported: `base64`, `array`. Defaults to `base64`.
  - **normalize** (boolean) - Optional - Whether to normalize the output embeddings. Defaults to `false`.

### Request Example
```json
{
  "version": "zsxkib/embedding-gemma-300m:59636724f635e022c13a24196c39a7f1185243990042988d6098e4f180784a63",
  "input": {
    "text": "Machine learning is transforming artificial intelligence",
    "task": "retrieval_query",
    "embedding_dim": 256,
    "output_format": "array",
    "normalize": true
  }
}
```

### Response
#### Success Response (200)
- **output** (string or array) - The generated text embedding. Format depends on `output_format` parameter.

#### Response Example (base64 output)
```
"ASNFZ+PPAgEAAAAAAAAAAAAA..."
```

#### Response Example (array output)
```json
[
  0.1234,
  -0.5678,
  ... 
]
```

```

--------------------------------

### Batch Embedding Generation with Error Handling

Source: https://context7.com/zsxkib/cog-google-embeddinggemma-300m/llms.txt

Provides a robust method for processing multiple texts in batches, including error handling and validation. It iterates through a list of documents, attempting to generate embeddings and catching potential errors like empty strings or excessively long texts. The output includes a summary of processed documents, errors encountered, and the final embedding matrix.

```python
from predict import Predictor
import base64
import numpy as np

predictor = Predictor()
predictor.setup()

documents = [
    "Climate change is affecting global weather patterns.",
    "Machine learning models require large amounts of training data.",
    "",  # Empty string - will raise error
    "The stock market experienced significant volatility today.",
    "A" * 3000,  # Very long text - will be truncated
]

embeddings = []
errors = []

for idx, doc in enumerate(documents):
    try:
        emb = predictor.predict(
            text=doc,
            task="retrieval_document",
            embedding_dim=256,
            output_format="base64",
            normalize=True
        )

        # Convert to numpy for storage
        vec = np.frombuffer(base64.b64decode(emb), dtype=np.float32)
        embeddings.append(vec)
        print(f"Document {idx}: Successfully embedded ({len(vec)} dims)")

    except ValueError as e:
        errors.append((idx, str(e)))
        print(f"Document {idx}: Error - {e}")
        embeddings.append(None)

# Results summary
print(f"\nProcessed: {len([e for e in embeddings if e is not None])}/{len(documents)}")
print(f"Errors: {len(errors)}")
for idx, error in errors:
    print(f"  Document {idx}: {error}")

# Create embedding matrix (excluding errors)
valid_embeddings = [e for e in embeddings if e is not None]
embedding_matrix = np.vstack(valid_embeddings)
print(f"\nFinal embedding matrix shape: {embedding_matrix.shape}")
```

--------------------------------

### Matryoshka Embedding Dimension Reduction

Source: https://context7.com/zsxkib/cog-google-embeddinggemma-300m/llms.txt

Demonstrates efficient embedding dimension reduction using Matryoshka representation learning. It generates embeddings at various specified dimensions and compares their storage efficiency. The code verifies that smaller dimensions are compatible prefixes of larger ones, highlighting flexible trade-offs between storage and accuracy.

```python
from predict import Predictor
import numpy as np

predictor = Predictor()
predictor.setup()

text = "Artificial intelligence is revolutionizing healthcare and medical diagnostics."

# Generate embeddings at different dimensions
dimensions = [128, 256, 512, 768]
embeddings = {}

for dim in dimensions:
    emb = predictor.predict(
        text=text,
        task="retrieval_document",
        embedding_dim=dim,
        output_format="array"
    )
    embeddings[dim] = emb
    print(f"{dim}D embedding - Storage: {len(emb) * 4} bytes")

# Compare storage efficiency
print(f"\nStorage savings:")
print(f"128D vs 768D: {(1 - 128/768) * 100:.1f}% reduction")
print(f"256D vs 768D: {(1 - 256/768) * 100:.1f}% reduction")

# All dimensions are compatible (smaller dims are prefixes of larger ones)
print(f"\nFirst 128 values match: {np.allclose(embeddings[768][:128], embeddings[128])}")
```

--------------------------------

### Generate Base64 Text Embeddings (Python)

Source: https://github.com/zsxkib/cog-google-embeddinggemma-300m/blob/main/REPLICATE_README.md

This Python snippet demonstrates how to generate a base64 encoded text embedding using the zsxkib/embedding-gemma-300m model from Replicate. It shows how to decode the base64 output into a NumPy array for further processing. The default output dimension is 768.

```python
import replicate, base64, numpy as np

# Get base64 embedding (default)
b64 = replicate.run("zsxkib/embedding-gemma-300m", input={"text": "Hello world"})

# Decode to numpy array
vec = np.frombuffer(base64.b64decode(b64), dtype=np.float32)
print(vec.shape)  # (768,)
```

--------------------------------

### Generate Base64 Text Embeddings (JavaScript)

Source: https://github.com/zsxkib/cog-google-embeddinggemma-300m/blob/main/REPLICATE_README.md

This JavaScript snippet shows how to generate a base64 encoded text embedding using the zsxkib/embedding-gemma-300m model via Replicate. It then converts the base64 string into a Float32Array, representing the 768-dimensional vector. This is useful for web applications that need to process embeddings client-side.

```javascript
const b64 = await replicate.run("zsxkib/embedding-gemma-300m", { input: { text: "Hello world" } });
const embedding = new Float32Array(Buffer.from(b64, 'base64').buffer);
console.log(embedding.length);  // 768
```
Tokens:
Raw