### Install PyLate-RS and FastPlaid

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

Install the required libraries using pip. Ensure you have Python and pip installed.

```bash
pip install pylate-rs fast-plaid
```

--------------------------------

### Python Quick Start: ColBERT Model Inference

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

Demonstrates initializing a ColBERT model, encoding queries and documents, and calculating similarity scores. Includes an example of using hierarchical pooling to reduce embedding size.

```python
from pylate_rs import models

# Initialize the model for your target device ("cpu", "cuda", or "mps")
model = models.ColBERT(
    model_name_or_path="lightonai/GTE-ModernColBERT-v1",
    device="cpu"
)

# Encode queries and documents
queries_embeddings = model.encode(
    sentences=["What is the capital of France?", "How big is the sun?"],
    is_query=True
)

documents_embeddings = model.encode(
    sentences=["Paris is the capital of France.", "The sun is a star."],
    is_query=False
)

# Calculate similarity scores
similarities = model.similarity(queries_embeddings, documents_embeddings)

print(f"Similarity scores:\n{similarities}")

# Use hierarchical pooling to reduce document embedding size and speed up downstream tasks
pooled_documents_embeddings = model.encode(
    sentences=["Paris is the capital of France.", "The sun is a star."],
    is_query=False,
    pool_factor=2, # Halves the number of token embeddings
)

similarities_pooled = model.similarity(queries_embeddings, pooled_documents_embeddings)

print(f"Similarity scores with pooling:\n{similarities_pooled}")
```

--------------------------------

### Install pylate-rs Locally from Source for GPU Support (Python)

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

Clone the repository and install pylate-rs locally from source to enable GPU support. This method is useful for development or when pre-built wheels are not available.

```sh
git clone https://github.com/lightonai/pylate-rs.git
cd pylate-rs
pip install .
```

--------------------------------

### Install pylate-rs for Intel CPU (MKL) (Python)

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

Use this command to install the pylate-rs version optimized for Intel CPUs with MKL support.

```sh
pip install pylate-rs-mkl
```

--------------------------------

### Install pylate-rs for Standard CPU (Python)

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

Use this command to install the standard CPU version of pylate-rs for Python.

```sh
pip install pylate-rs
```

--------------------------------

### Install Pylate-RS npm Package

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

Install the WebAssembly package for JavaScript and TypeScript projects using npm.

```bash
npm install pylate-rs
```

--------------------------------

### Install pylate-rs from Source for GPU Support (Python)

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

Install pylate-rs from source to enable GPU support. This is necessary if pre-built CUDA wheels are not available.

```sh
pip install git+https://github.com/lightonai/pylate-rs.git
```

--------------------------------

### Application Initialization

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/index.html

Initializes the application when the DOM is fully loaded. This is the entry point for the application's setup.

```javascript
document.addEventListener('DOMContentLoaded', () => App.init());
```

--------------------------------

### Python Quick Start: Initialize Model and Encode

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/pkg/README.md

Initialize a ColBERT model for a specified device and encode queries and documents. Ensure the model name or path is correct.

```python
from pylate_rs import models

# Initialize the model for your target device ("cpu", "cuda", or "mps")
model = models.ColBERT(
    model_name_or_path="lightonai/GTE-ModernColBERT-v1",
    device="cpu"
)

# Encode queries and documents
queries_embeddings = model.encode(
    sentences=["What is the capital of France?", "How big is the sun?"],
    is_query=True
)

documents_embeddings = model.encode(
    sentences=["Paris is the capital of France.", "The sun is a star."],
    is_query=False
)
```

--------------------------------

### Initialize ColBERT Model with Defaults (Rust)

Source: https://context7.com/lightonai/pylate-rs/llms.txt

Initialize a ColBERT model with default settings, typically using the CPU. This is a simple way to get started.

```rust
use anyhow::Result;
use candle_core::Device;
use pylate_rs::ColBERT;

fn main() -> Result<()> {
    // Simple initialization with defaults (CPU)
    let mut model: ColBERT = ColBERT::from("lightonai/GTE-ModernColBERT-v1")
        .try_into()?;

    Ok(())
}
```

--------------------------------

### Install pylate-rs for Apple CPU (Python)

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

Use this command to install the pylate-rs version optimized for Apple CPUs (macOS).

```sh
pip install pylate-rs-accelerate
```

--------------------------------

### Install pylate-rs for Apple GPU (M1/M2/M3) (Python)

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

Use this command to install the pylate-rs version optimized for Apple GPUs (M1/M2/M3).

```sh
pip install pylate-rs-metal
```

--------------------------------

### Python Quick Start: Calculate Similarity Scores

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/pkg/README.md

Calculate similarity scores between encoded queries and documents using the model's similarity method. This follows the encoding step.

```python
# Calculate similarity scores
similarities = model.similarity(queries_embeddings, documents_embeddings)

print(f"Similarity scores:\n{similarities}")
```

--------------------------------

### Push Model to Hugging Face Hub in PyLate Format

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

This Python snippet shows how to push a model to the Hugging Face Hub in PyLate format using the 'pylate' library. Ensure you have 'pylate' installed and replace placeholders with your actual model path, username, model name, and Hugging Face token.

```bash
pip install pylate
```

```python
from pylate import models

# Load your model
model = models.ColBERT(model_name_or_path="your-base-model-on-hf")

# Push in PyLate format
model.push_to_hub(
    repo_id="YourUsername/YourModelName",
    private=False,
    token="YOUR_HUGGINGFACE_TOKEN",
)
```

--------------------------------

### Initialize ColBERT Model on CUDA GPU (Rust)

Source: https://context7.com/lightonai/pylate-rs/llms.txt

Initialize a ColBERT model on a CUDA-enabled GPU for faster inference. Ensure CUDA drivers and toolkit are correctly installed.

```rust
use anyhow::Result;
use candle_core::Device;
use pylate_rs::ColBERT;

fn main() -> Result<()> {
    // CUDA GPU initialization
    let cuda_device = Device::new_cuda(0)?;
    let mut gpu_model: ColBERT = ColBERT::from("lightonai/GTE-ModernColBERT-v1")
        .with_device(cuda_device)
        .try_into()?;

    Ok(())
}
```

--------------------------------

### Complete Retrieval Pipeline with FastPlaid

Source: https://context7.com/lightonai/pylate-rs/llms.txt

Integrate pylate-rs with FastPlaid for an end-to-end multi-vector search pipeline, including indexing and retrieval. Ensure torch is installed.

```python
import torch
from fast_plaid import search
from pylate_rs import models

# Initialize model
model = models.ColBERT(
    model_name_or_path="lightonai/GTE-ModernColBERT-v1",
    device="cpu"  # or "mps", "cuda"
)

# Document corpus
documents = [
    "1st Arrondissement: Louvre, Tuileries Garden, Palais Royal.",
    "5th Arrondissement: Latin Quarter, Sorbonne, Panthéon.",
    "7th Arrondissement: Eiffel Tower, Musée d'Orsay, Les Invalides.",
    "8th Arrondissement: Champs-Élysées, Arc de Triomphe, luxury shopping.",
    "18th Arrondissement: Montmartre, Sacré-Cœur, Moulin Rouge."
]

# Encode documents with hierarchical pooling
documents_embeddings = model.encode(
    sentences=documents,
    is_query=False,
    pool_factor=2
)

# Create FastPlaid index
fast_plaid = search.FastPlaid(index="paris_index")
fast_plaid.create(
    documents_embeddings=[torch.tensor(emb) for emb in documents_embeddings]
)

# Search queries
queries = [
    "arrondissement with the Eiffel Tower",
    "Latin Quarter and Sorbonne University",
    "arrondissement with Sacré-Cœur"
]

queries_embeddings = model.encode(sentences=queries, is_query=True)

# Retrieve top-k results
results = fast_plaid.search(
    queries_embeddings=torch.tensor(queries_embeddings),
    top_k=3
)
print(results)
```

--------------------------------

### Python Quick Start: Hierarchical Pooling for Embeddings

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/pkg/README.md

Encode documents using hierarchical pooling to reduce embedding size and potentially speed up downstream tasks. The pool_factor controls the reduction.

```python
# Use hierarchical pooling to reduce document embedding size and speed up downstream tasks
pooled_documents_embeddings = model.encode(
    sentences=["Paris is the capital of France.", "The sun is a star."],
    is_query=False,
    pool_factor=2, # Halves the number of token embeddings
)

similarities_pooled = model.similarity(queries_embeddings, pooled_documents_embeddings)

print(f"Similarity scores with pooling:\n{similarities_pooled}")
```

--------------------------------

### Initialize PyLate-RS

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/index.html

Imports necessary components from the pylate_rs.js module. Ensure the module is correctly placed.

```javascript
import init, { ColBERT, hierarchical_pooling } from "./pkg/pylate_rs.js";
```

--------------------------------

### Initialize Application and Event Listeners in JavaScript

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/index.html

Initializes the application, sets up event listeners for UI elements, and loads the initial model. Handles potential errors during WASM module initialization.

```javascript
async init() {
    this.initEventListeners();
    this.populateModelSelector();
    this.initCopyButtons();
    hljs.highlightAll();
    try {
      await init();
      this.loadModel(this.state.currentModelName);
    } catch (e) {
      console.error("Fatal: Failed to initialize WASM module.", e);
      this.updateAllStatuses(`Error: Could not initialize the application.`, 'error');
    }
  }
```

--------------------------------

### Initialize ColBERT Model for Apple Metal (MPS)

Source: https://context7.com/lightonai/pylate-rs/llms.txt

Initialize a ColBERT model for Apple Metal GPU (M1/M2/M3) inference. Requires specifying 'mps' as the device. Query and document lengths can be configured.

```python
from pylate_rs import models

# Initialize model for Apple Metal GPU (M1/M2/M3)
model_mps = models.ColBERT(
    model_name_or_path="lightonai/GTE-ModernColBERT-v1",
    device="mps",
    query_length=32,
    document_length=180
)
```

--------------------------------

### Run All Interactive Demos

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/index.html

A utility function to sequentially call all available interactive demo handlers.

```javascript
runAllDemos() {
  this.handleSimilarityDemo();
  this.handleMatrixDemo();
  this.handlePoolingDemo();
}
```

--------------------------------

### Initialize ColBERT Model and Load Status Updates

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/index.html

Handles the loading of a ColBERT model from fetched files, updating UI statuses throughout the process. Includes error handling for download and initialization phases. Sets `this.state.colbertModel` and calls `runAllDemos()` on success.

```javascript
try {
  let modelFiles;
  try {
    this.updateAllStatuses(`⏳ Downloading ${modelRepo} from Hugging Face Hub...`, 'loading');
    modelFiles = await fetchAllFiles(`https://huggingface.co/${modelRepo}/resolve/main`)
  } catch (e) {
    console.warn(`Unable to download the model.`, e);
  }
  this.updateAllStatuses(`Initializing ${modelRepo}...`, 'loading');
  const [tokenizer, model, config, stConfig, dense, denseConfig, tokensConfig] = modelFiles;
  this.state.colbertModel = new ColBERT(model, dense, tokenizer, config, stConfig, denseConfig, tokensConfig, 32);
  this.updateAllStatuses(`✅ ${modelRepo} loaded successfully.`, 'success');
  this.runAllDemos();
} catch (error) {
  console.error("Model Loading Error:", error);
  const msg = `Error: Could not load model ${modelRepo}.`;
  this.updateAllStatuses(msg, 'error');
} finally {
  this.state.isLoading = false;
}
```

--------------------------------

### Main Application Object

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/index.html

Initializes the main application object, defining its state and selectors for UI elements. This object manages the application's logic and interactions.

```javascript
const App = {
  // --- Application State ---
  state: {
    colbertModel: null,
    currentModelName: CONSTANTS.DEFAULT_MODEL,
    showMaxSimOnly: false,
    isLoading: false,
  },
  // --- UI Element Selectors ---
  ui: {
    // Model & Status
    modelSelector: document.getElementById('model-selector'),
    statusText: document.getElementById('status-text'),
    // Similarity Demo
    queryInput: document.getElementById('query-input'),
    documentInput: document.getElementById('document-input'),
    resultsList: document.getElementById('results-list'),
    // Matrix Demo
    matrixContainer: document.getElementById('token-matrix-container'),
    matrixQueryInput: document.getElementById('matrix-query-input'),
    matrixDocInput: document.getElementById('matrix-doc-input'),
    maxSimToggle: document.getElementById('max-sim-toggle'),
    matrixStatusText: document.getElementById('matrix-status-text'),
    // Token Importance Demo
    tokenImportanceContainer: document.getElementById('token-importance-container'),
    // Pooling Demo
    poolingDocInput: document.getElementById('pooling-doc-input'),
    poolingQueryInput: document.getElementById('pooling-query-input'),
    poolFactorSlider: document.getElementById('pool-factor-slider'),
    poolFactorValue: document.getElementById('pool-factor-value')
  },
};
```

--------------------------------

### Initialize ColBERT Model for CPU

Source: https://context7.com/lightonai/pylate-rs/llms.txt

Initialize a ColBERT model for CPU inference. Ensure the model name or path is correct.

```python
from pylate_rs import models

# Initialize model for CPU inference
model = models.ColBERT(
    model_name_or_path="lightonai/GTE-ModernColBERT-v1",
    device="cpu"
)
```

--------------------------------

### Fetch Model Files from Hugging Face Hub

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/index.html

Asynchronously fetches required model files (tokenizer, model, config, etc.) from a specified Hugging Face Hub repository. Throws an error if any file is not found. Requires `CONSTANTS.REQUIRED_FILES` to be defined.

```javascript
const fetchAllFiles = async (basePath) => {
  const responses = await Promise.all(CONSTANTS.REQUIRED_FILES.map(file => fetch(`${basePath}/${file}`)))
  for (const response of responses) {
    if (!response.ok) throw new Error(`File not found: ${response.url}`);
  }
  return Promise.all(responses.map(res => res.arrayBuffer().then(b => new Uint8Array(b))))
}
```

--------------------------------

### Configure ColBERT Model with Builder Pattern (Rust)

Source: https://context7.com/lightonai/pylate-rs/llms.txt

Configure a ColBERT model using the fluent builder pattern for fine-grained control over parameters like device, lengths, batch size, and prefix tokens. Ensure candle-core is available.

```rust
use anyhow::Result;
use candle_core::Device;
use pylate_rs::ColBERT;

fn main() -> Result<()> {
    // Full configuration with builder pattern
    let mut model_configured: ColBERT = ColBERT::from("lightonai/GTE-ModernColBERT-v1")
        .with_device(Device::Cpu)
        .with_query_length(32)
        .with_document_length(180)
        .with_batch_size(64)
        .with_do_query_expansion(true)
        .with_attend_to_expansion_tokens(false)
        .with_query_prefix("[Q]".to_string())
        .with_document_prefix("[D]".to_string())
        .with_mask_token("[MASK]".to_string())
        .try_into()?;

    Ok(())
}
```

--------------------------------

### Handle Matrix Demo

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/index.html

Orchestrates the fetching and rendering of interaction matrices and token importance visualizations. Requires `this.state.colbertModel` and valid query/document inputs. Handles cases with no valid tokens.

```javascript
handleMatrixDemo() {
  if (!this.state.colbertModel) return;
  const query = this.ui.matrixQueryInput.value;
  const doc = this.ui.matrixDocInput.value;
  if (!query.trim() || !doc.trim()) {
    this.ui.matrixContainer.innerHTML = `<p class="text-gray-500 text-center p-4">Please provide a query and a document.</p>`;
    this.ui.tokenImportanceContainer.innerHTML = `<p class="text-gray-500 text-center text-sm">Enter a query and document to begin.</p>`;
    return;
  }
  try {
    const rawResult = this.state.colbertModel.raw_similarity_matrix({ queries: [query], documents: [doc] });
    const { matrix, queryTokens, docTokens, maxSimIndicesByQuery, maxSimDocIndices } = this.processMatrixData(rawResult);
    if (queryTokens.length === 0 || docTokens.length === 0) {
      this.ui.matrixContainer.innerHTML = `<p class="text-gray-500 text-center p-4">No valid tokens to display.</p>`;
      return;
    }
    this.renderMatrix(matrix, queryTokens, docTokens, maxSimIndicesByQuery);
    this.renderTokenImportance(matrix, docTokens, maxSimDocIndices);
  } catch (error) {
    this.ui.matrixContainer.innerHTML = `<p class="text-red-600 text-center p-4">Failed to generate visualizations.</p>`;
    console.error("Visualization Error:", error);
  }
}
```

--------------------------------

### Initialize Copy-to-Clipboard Buttons in JavaScript

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/index.html

Initializes all copy-to-clipboard buttons within code blocks. Provides visual feedback upon successful copy or error.

```javascript
initCopyButtons() {
    const copyButtons = document.querySelectorAll('.copy-button');
    if (!copyButtons.length) return;
    const successIcon = `<svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-check"><polyline points="20 6 9 17 4 12"></polyline></svg>
`;
    copyButtons.forEach(button => {
      const iconSpan = button.querySelector('.copy-icon');
      const textSpan = button.querySelector('.copy-text');
      const originalIcon = iconSpan.innerHTML;
      button.addEventListener('click', () => {
        const container = button.closest('.code-block-container');
        const codeBlock = container ? container.querySelector('pre code') : null;
        if (codeBlock) {
          navigator.clipboard.writeText(codeBlock.innerText).then(() => {
            textSpan.textContent = 'Copied!';
            iconSpan.innerHTML = successIcon;
            setTimeout(() => {
              textSpan.textContent = 'Copy';
              iconSpan.innerHTML = originalIcon;
            }, 2000);
          }).catch(err => {
            console.error('Failed to copy text: ', err);
            textSpan.textContent = 'Error';
            setTimeout(() => {
              textSpan.textContent = 'Copy';
            }, 2000);
          });
        }
      });
    });
  }
```

--------------------------------

### Save Model Locally in PyLate Format

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

This Python code snippet demonstrates how to save a model in PyLate format to a local directory using the 'pylate' library. Replace 'your-base-model-on-hf' with the path to your base model and specify your desired local save path.

```python
from pylate import models

# Load your model
model = models.ColBERT(model_name_or_path="your-base-model-on-hf")

# Save in PyLate format
model.save_pretrained("path/to/save/GTE-ModernColBERT-v1-pylate")
```

--------------------------------

### Search Documents using FastPlaid Index

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

This snippet shows how to load an existing FastPlaid index, encode search queries, and then perform a search to retrieve the top K most relevant documents. Ensure the `model` object is available from the previous step.

```python
import torch
from fast_plaid import search
from pylate_rs import models

fast_plaid = search.FastPlaid(index="index")

queries = [
    "arrondissement with the Eiffel Tower and Musée d'Orsay",
    "Latin Quarter and Sorbonne University",
    "arrondissement with Sacré-Cœur and Moulin Rouge",
    "arrondissement with the Louvre and Tuileries Garden",
    "arrondissement with Notre-Dame Cathedral and the Marais",
]

queries_embeddings = model.encode(
    sentences=queries,
    is_query=True,
)

scores = fast_plaid.search(
    queries_embeddings=torch.tensor(queries_embeddings),
    top_k=3,
)

print(scores)
```

--------------------------------

### Load ColBERT Model in JavaScript

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

Load a ColBERT model by fetching required files from a local path or Hugging Face Hub. Ensure all REQUIRED_FILES are accessible.

```javascript
import { ColBERT } from "pylate-rs";

const REQUIRED_FILES = [
  "tokenizer.json",
  "model.safetensors",
  "config.json",
  "config_sentence_transformers.json",
  "1_Dense/model.safetensors",
  "1_Dense/config.json",
  "special_tokens_map.json",
];

async function loadModel(modelRepo) {
  const fetchAllFiles = async (basePath) => {
    const responses = await Promise.all(
      REQUIRED_FILES.map((file) => fetch(`${basePath}/${file}`))
    );
    for (const response of responses) {
      if (!response.ok) throw new Error(`File not found: ${response.url}`);
    }
    return Promise.all(
      responses.map((res) => res.arrayBuffer().then((b) => new Uint8Array(b)))
    );
  };

  try {
    let modelFiles;
    try {
      // Attempt to load from a local `models` directory first
      modelFiles = await fetchAllFiles(`models/${modelRepo}`);
    } catch (e) {
      console.warn(
        `Local model not found, falling back to Hugging Face Hub.`, 
        e
      );
      // Fallback to fetching directly from the Hugging Face Hub
      modelFiles = await fetchAllFiles(
        `https://huggingface.co/${modelRepo}/resolve/main`
      );
    }

    const [
      tokenizer,
      model,
      config,
      stConfig,
      dense,
      denseConfig,
      tokensConfig,
    ] = modelFiles;

    // Instantiate the model with the loaded files
    const colbertModel = new ColBERT(
      model,
      dense,
      tokenizer,
      config,
      stConfig,
      denseConfig,
      tokensConfig,
      32
    );

    // You can now use `colbertModel` for encoding
    console.log("Model loaded successfully!");
    return colbertModel;
  } catch (error) {
    console.error("Model Loading Error:", error);
  }
}
```

--------------------------------

### Encode and Compare Text Embeddings with pylate-rs

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

Demonstrates encoding queries and documents, calculating similarity, and applying hierarchical pooling for enhanced results using the pylate-rs library in Rust. Ensure the 'candle-core' and 'pylate-rs' crates are added to your Cargo.toml.

```rust
use anyhow::Result;
use candle_core::Device;
use pylate_rs::{hierarchical_pooling, ColBERT};

fn main() -> Result<()> {
    // Set the device (e.g., Cpu, Cuda, Metal)
    let device = Device::Cpu;

    // Initialize the model
    let mut model: ColBERT = ColBERT::from("lightonai/GTE-ModernColBERT-v1")
        .with_device(device)
        .try_into()?;

    // Encode queries and documents
    let queries = vec!["What is the capital of France?".to_string()];
    let documents = vec!["Paris is the capital of France.".to_string()];

    let query_embeddings = model.encode(&queries, true)?;
    let document_embeddings = model.encode(&documents, false)?;

    // Calculate similarity
    let similarities = model.similarity(&query_embeddings, &document_embeddings)?;
    println!("Similarity score: {}", similarities.data[0][0]);

    // Use hierarchical pooling
    let pooled_document_embeddings = hierarchical_pooling(&document_embeddings, 2)?;
    let pooled_similarities = model.similarity(&query_embeddings, &pooled_document_embeddings)?;
    println!("Similarity score after hierarchical pooling: {}", pooled_similarities.data[0][0]);

    Ok(())
}
```

--------------------------------

### Add pylate-rs to Cargo.toml with MKL Feature (Rust)

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

Add the pylate-rs crate to your Rust project's Cargo.toml file, enabling the 'mkl' feature for optimized performance on Intel CPUs with MKL support.

```toml
cargo add pylate-rs --features mkl
```

--------------------------------

### Add pylate-rs to Cargo.toml with Metal Feature (Rust)

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

Add the pylate-rs crate to your Rust project's Cargo.toml file, enabling the 'metal' feature for optimized performance on Apple GPUs (M1/M2/M3).

```toml
cargo add pylate-rs --features metal
```

--------------------------------

### Add pylate-rs to Cargo.toml with Apple Accelerate Feature (Rust)

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

Add the pylate-rs crate to your Rust project's Cargo.toml file, enabling the 'accelerate' feature for optimized performance on Apple CPUs (macOS).

```toml
cargo add pylate-rs --features accelerate
```

--------------------------------

### Initialize ColBERT Model for CUDA GPU

Source: https://context7.com/lightonai/pylate-rs/llms.txt

Initialize a ColBERT model for CUDA GPU inference. Specify the device as 'cuda' or a specific GPU index. Configurable parameters include query length, document length, batch size, and attention to expansion tokens.

```python
from pylate_rs import models

# Initialize model for CUDA GPU inference
model_cuda = models.ColBERT(
    model_name_or_path="lightonai/GTE-ModernColBERT-v1",
    device="cuda",  # or "cuda:0", "cuda:1" for specific GPU
    query_length=32,
    document_length=180,
    batch_size=64,
    attend_to_expansion_tokens=False
)
```

--------------------------------

### Handle Similarity Demo

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/index.html

Calculates and displays similarity scores between a query and multiple documents. Renders results in a list, sorting by score and showing a progress bar for score visualization. Requires `this.state.colbertModel` to be initialized.

```javascript
handleSimilarityDemo() {
  if (!this.state.colbertModel) return;
  const query = this.ui.queryInput.value;
  const documents = this.ui.documentInput.value.split('\n').filter(doc => doc.trim());
  this.ui.resultsList.innerHTML = '';
  if (!query.trim() || documents.length === 0) {
    this.ui.resultsList.innerHTML = `<p class="text-gray-500 p-2">Enter a query and at least one document.</p>`;
    return;
  }
  try {
    const { data } = this.state.colbertModel.similarity({ queries: [query], documents });
    const scores = data[0];
    const results = documents.map((doc, i) => ({ text: doc, score: scores[i] }))
      .sort((a, b) => b.score - a.score);
    const maxScore = results.length > 0 && results[0].score > 0 ? results[0].score : 1;
    const fragment = document.createDocumentFragment();
    results.forEach(result => {
      const scorePercentage = (result.score / maxScore) * 100;
      const resultEl = document.createElement('div');
      resultEl.className = 'bg-white p-3 rounded-md border border-gray-200';
      resultEl.innerHTML = ` <p class="text-gray-800 mb-2 text-base">${result.text}</p>
        <div class="flex items-center gap-2">
          <span class="text-xs font-semibold text-indigo-700 w-24">Score: ${result.score.toFixed(3)}</span>
          <div class="w-full bg-indigo-100 rounded-full h-2">
            <div class="bg-indigo-500 h-2 rounded-full" style="width: ${scorePercentage}%"></div>
          </div>
        </div>`;
      fragment.appendChild(resultEl);
    });
    this.ui.resultsList.appendChild(fragment);
  } catch (error) {
    this.ui.resultsList.innerHTML = `<p class="text-red-600">Error during calculation. See console.</p>`;
    console.error("Similarity Calculation Error:", error);
  }
}
```

--------------------------------

### Create FastPlaid Index with ColBERT Embeddings

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/index.html

This code snippet initializes a ColBERT model, encodes a list of documents into embeddings, and then creates a FastPlaid index. Ensure the model is loaded to the correct device (cpu, mps, or cuda). The pool_factor can be adjusted to control the number of embeddings.

```python
import torch
from fast_plaid import search
from pylate_rs import models

model = models.ColBERT(
    model_name_or_path="lightonai/GTE-ModernColBERT-v1",
    device="cpu", # mps or cuda
)


documents = [
    "1st Arrondissement: Louvre, Tuileries Garden, Palais Royal, historic, tourist.",
    "2nd Arrondissement: Bourse, financial, covered passages, Sentier, business.",
    "3rd Arrondissement: Marais, Musée Picasso, galleries, trendy, historic.",
    "4th Arrondissement: Notre-Dame, Marais, Hôtel de Ville, LGBTQ+.",
    "5th Arrondissement: Latin Quarter, Sorbonne, Panthéon, student, intellectual.",
    "6th Arrondissement: Saint-Germain-des-Prés, Luxembourg Gardens, chic, artistic, cafés.",
    "7th Arrondissement: Eiffel Tower, Musée d'Orsay, Les Invalides, affluent, prestigious.",
    "8th Arrondissement: Champs-Élysées, Arc de Triomphe, luxury, shopping, Élysée.",
    "9th Arrondissement: Palais Garnier, department stores, shopping, theaters.",
    "10th Arrondissement: Gare du Nord, Gare de l'Est, Canal Saint-Martin.",
    "11th Arrondissement: Bastille, nightlife, Oberkampf, revolutionary, hip.",
    "12th Arrondissement: Bois de Vincennes, Opéra Bastille, Bercy, residential.",
    "13th Arrondissement: Chinatown, Bibliothèque Nationale, modern, diverse, street-art.",
    "14th Arrondissement: Montparnasse, Catacombs, residential, artistic, quiet.",
    "15th Arrondissement: Residential, family, populous, Parc André Citroën.",
    "16th Arrondissement: Trocadéro, Bois de Boulogne, affluent, elegant, embassies.",
    "17th Arrondissement: Diverse, Palais des Congrès, residential, Batignolles.",
    "18th Arrondissement: Montmartre, Sacré-Cœur, Moulin Rouge, artistic, historic.",
    "19th Arrondissement: Parc de la Villette, Cité des Sciences, canals, diverse.",
    "20th Arrondissement: Père Lachaise, Belleville, cosmopolitan, artistic, historic.",
]

# Encoding documents
documents_embeddings = model.encode(
    sentences=documents,
    is_query=False,
    pool_factor=2, # Let's divide the number of embeddings by 2.
)

# Creating the FastPlaid index
fast_plaid = search.FastPlaid(index="index")


fast_plaid.create(
    documents_embeddings=[torch.tensor(embedding) for embedding in documents_embeddings]
)
```

--------------------------------

### Load ColBERT Model from Local Directory (Rust)

Source: https://context7.com/lightonai/pylate-rs/llms.txt

Load a pre-trained ColBERT model from a local directory. This is useful for offline use or custom model deployments.

```rust
use anyhow::Result;
use candle_core::Device;
use pylate_rs::ColBERT;

fn main() -> Result<()> {
    // Load from local directory
    let mut local_model: ColBERT = ColBERT::from("./models/my-colbert-model")
        .with_device(Device::Cpu)
        .try_into()?;

    Ok(())
}
```

--------------------------------

### Render Token-to-Token Similarity Matrix

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/index.html

Renders a visual grid representing the token-to-token similarity matrix. It dynamically sets grid dimensions and cell background colors based on similarity scores, optionally highlighting only the maximum similarity for each query token.

```javascript
renderMatrix(matrix, queryTokens, docTokens, maxSimIndicesByQuery) {
    const flatScores = matrix.flat();
    const minScore = Math.min(...flatScores),
        maxScore = Math.max(...flatScores);
    const grid = document.createElement('div');
    grid.className = 'matrix';
    grid.style.gridTemplateColumns = `auto repeat(${docTokens.length}, minmax(${CONSTANTS.CELL_SIZE}, auto))`;
    grid.style.gridTemplateRows = `auto repeat(${queryTokens.length}, ${CONSTANTS.CELL_SIZE})`;
    // Top-left empty cell
    grid.appendChild(document.createElement('div'));
    // Document token headers
    docTokens.forEach(token => {
        const label = document.createElement('div');
        label.className = 'token-label';
        label.textContent = token;
        grid.appendChild(label);
    });
    // Query tokens and score cells
    queryTokens.forEach((queryToken, i) => {
        const queryLabel = document.createElement('div');
        queryLabel.className = 'token-label query-token-label';
        queryLabel.textContent = queryToken;
        grid.appendChild(queryLabel);
        const maxScoreDocIndex = maxSimIndicesByQuery[i];
        docTokens.forEach((_, j) => {
            const cell = document.createElement('div');
            cell.className = 'matrix-cell';
            const score = matrix[i][j];
            const intensity = (maxScore > minScore) ? (score - minScore) / (maxScore - minScore) : 0;
            const color = Utils.getColorForIntensity(intensity);
            if (this.state.showMaxSimOnly) {
                cell.style.backgroundColor = (j === maxScoreDocIndex) ? color : '#f9fafb';
                if (j === maxScoreDocIndex)
                    cell.style.border = '2px solid #111827';
            } else {
                cell.style.backgroundColor = color;
            }
            cell.innerHTML = `<span class="tooltip">${score.toFixed(2)}</span>`;
            grid.appendChild(cell);
        });
    });
    this.ui.matrixContainer.innerHTML = '';
    this.ui.matrixContainer.appendChild(grid);
}
```

--------------------------------

### Add pylate-rs to Cargo.toml with CUDA Feature (Rust)

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

Add the pylate-rs crate to your Rust project's Cargo.toml file, enabling the 'cuda' feature for optimized performance on NVIDIA GPUs.

```toml
cargo add pylate-rs --features cuda
```

--------------------------------

### Add pylate-rs to Cargo.toml with Default CPU Feature (Rust)

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

Add the pylate-rs crate to your Rust project's Cargo.toml file. This configuration uses the default feature flag for standard CPU support.

```toml
cargo add pylate-rs
```

--------------------------------

### Encode Documents and Create FastPlaid Index

Source: https://github.com/lightonai/pylate-rs/blob/main/README.md

This snippet demonstrates how to initialize a ColBERT model, encode a list of documents, and then create a FastPlaid index with these embeddings. The `pool_factor` can be adjusted to control the number of embeddings per document.

```python
import torch
from fast_plaid import search
from pylate_rs import models

model = models.ColBERT(
    model_name_or_path="lightonai/GTE-ModernColBERT-v1",
    device="cpu", # mps or cuda
)

documents = [
    "1st Arrondissement: Louvre, Tuileries Garden, Palais Royal, historic, tourist.",
    "2nd Arrondissement: Bourse, financial, covered passages, Sentier, business.",
    "3rd Arrondissement: Marais, Musée Picasso, galleries, trendy, historic.",
    "4th Arrondissement: Notre-Dame, Marais, Hôtel de Ville, LGBTQ+.",
    "5th Arrondissement: Latin Quarter, Sorbonne, Panthéon, student, intellectual.",
    "6th Arrondissement: Saint-Germain-des-Prés, Luxembourg Gardens, chic, artistic, cafés.",
    "7th Arrondissement: Eiffel Tower, Musée d'Orsay, Les Invalides, affluent, prestigious.",
    "8th Arrondissement: Champs-Élysées, Arc de Triomphe, luxury, shopping, Élysée.",
    "9th Arrondissement: Palais Garnier, department stores, shopping, theaters.",
    "10th Arrondissement: Gare du Nord, Gare de l'Est, Canal Saint-Martin.",
    "11th Arrondissement: Bastille, nightlife, Oberkampf, revolutionary, hip.",
    "12th Arrondissement: Bois de Vincennes, Opéra Bastille, Bercy, residential.",
    "13th Arrondissement: Chinatown, Bibliothèque Nationale, modern, diverse, street-art.",
    "14th Arrondissement: Montparnasse, Catacombs, residential, artistic, quiet.",
    "15th Arrondissement: Residential, family, populous, Parc André Citroën.",
    "16th Arrondissement: Trocadéro, Bois de Boulogne, affluent, elegant, embassies.",
    "17th Arrondissement: Diverse, Palais des Congrès, residential, Batignolles.",
    "18th Arrondissement: Montmartre, Sacré-Cœur, Moulin Rouge, artistic, historic.",
    "19th Arrondissement: Parc de la Villette, Cité des Sciences, canals, diverse.",
    "20th Arrondissement: Père Lachaise, Belleville, cosmopolitan, artistic, historic.",
]

# Encoding documents
documents_embeddings = model.encode(
    sentences=documents,
    is_query=False,
    pool_factor=2, # Let's divide the number of embeddings by 2.
)

# Creating the FastPlaid index
fast_plaid = search.FastPlaid(index="index")


fast_plaid.create(
    documents_embeddings=[torch.tensor(embedding) for embedding in documents_embeddings]
)
```

--------------------------------

### Application Constants

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/index.html

Defines constants used throughout the application, including debounce delays, default and available models, required files for models, and cell size.

```javascript
const CONSTANTS = {
  DEBOUNCE_DELAY_INPUT: 400,
  DEBOUNCE_DELAY_MATRIX: 500,
  DEBOUNCE_DELAY_SLIDER: 50,
  DEFAULT_MODEL: "lightonai/answerai-colbert-small-v1",
  AVAILABLE_MODELS: [
    "lightonai/answerai-colbert-small-v1",
    "lightonai/GTE-ModernColBERT-v1",
    "lightonai/colbertv2.0",
    "lightonai/Reason-ModernColBERT",
  ],
  REQUIRED_FILES: [
    'tokenizer.json',
    'model.safetensors',
    'config.json',
    'config_sentence_transformers.json',
    '1_Dense/model.safetensors',
    '1_Dense/config.json',
    "special_tokens_map.json"
  ],
  CELL_SIZE: "3.5rem"
};
```

--------------------------------

### Encode Sentences and Compute Similarity in Rust

Source: https://context7.com/lightonai/pylate-rs/llms.txt

Use the native Rust API with Candle tensors to encode queries and documents, then compute their similarity scores. Ensure the model is loaded and device is specified.

```rust
use anyhow::Result;
use candle_core::Device;
use pylate_rs::ColBERT;

fn main() -> Result<()> {
    let device = Device::Cpu;

    let mut model: ColBERT = ColBERT::from("lightonai/GTE-ModernColBERT-v1")
        .with_device(device)
        .try_into()?;

    // Prepare queries and documents
    let queries = vec![
        "What is the capital of France?".to_string(),
        "How big is the sun?".to_string(),
    ];
    let documents = vec![
        "Paris is the capital of France.".to_string(),
        "The sun is a star at the center of the Solar System.".to_string(),
        "Berlin is the capital of Germany.".to_string(),
    ];

    // Encode queries (is_query=true uses fixed-length [MASK] padding)
    let query_embeddings = model.encode(&queries, true)?;
    println!("Query embeddings dims: {:?}", query_embeddings.dims());

    // Encode documents (is_query=false uses variable-length encoding)
    let document_embeddings = model.encode(&documents, false)?;
    println!("Document embeddings dims: {:?}", document_embeddings.dims());

    // Calculate similarity scores
    let similarities = model.similarity(&query_embeddings, &document_embeddings)?;

    // Access results
    for (i, query) in queries.iter().enumerate() {
        println!("\nQuery: {}", query);
        for (j, doc) in documents.iter().enumerate() {
            println!("  Score {:.4}: {}", similarities.data[i][j], doc);
        }
    }

    Ok(())
}

```

--------------------------------

### Pooling Demo: Calculate and Display Scores

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/index.html

Demonstrates the effect of hierarchical pooling on embeddings. It calculates scores for original vs. pooled embeddings, rendering visualizations of embedding block lengths and updating status messages.

```javascript
handlePoolingDemo() {
    if (!this.state.colbertModel) return;
    this.ui.poolingStatus.textContent = '⚙️ Processing...';
    const docText = this.ui.poolingDocInput.value;
    const queryText = this.ui.poolingQueryInput.value;
    const poolFactor = parseInt(this.ui.poolFactorSlider.value, 10);
    this.ui.originalScore.textContent = '-';
    this.ui.pooledScore.textContent = '-';
    if (!docText.trim()) {
        this.ui.poolingStatus.textContent = 'Please enter a document.';
        this.renderEmbeddingBlocks(this.ui.originalEmbeddingsVis, 0, this.ui.originalTokenCount);
        this.renderEmbeddingBlocks(this.ui.pooledEmbeddingsVis, 0, this.ui.pooledTokenCount);
        return;
    }
    try {
        const originalDocResult = this.state.colbertModel.encode({ sentences: [docText] }, false);
        const originalDocEmbeddings = originalDocResult.embeddings[0] || [];
        this.renderEmbeddingBlocks(this.ui.originalEmbeddingsVis, originalDocEmbeddings.length, this.ui.originalTokenCount);
        const pooledResult = hierarchical_pooling({ embeddings: [originalDocEmbeddings], pool_factor: poolFactor });
        const pooledDocEmbeddings = pooledResult.embeddings[0] || [];
        this.renderEmbeddingBlocks(this.ui.pooledEmbeddingsVis, pooledDocEmbeddings.length, this.ui.pooledTokenCount);
        if (queryText.trim()) {
            const queryResult = this.state.colbertModel.encode({ sentences: [queryText] }, false);
            const queryEmbeddings = queryResult.embeddings[0] || [];
            const calculateScore = (qVe
```

--------------------------------

### Encode Queries and Documents with ColBERT

Source: https://context7.com/lightonai/pylate-rs/llms.txt

Encode queries and documents using the ColBERT model. This is a prerequisite for calculating similarity or performing retrieval.

```python
query_embeddings = model.encode(sentences=queries, is_query=True)
document_embeddings = model.encode(sentences=documents, is_query=False)
```

--------------------------------

### Populate Model Selector Dropdown in JavaScript

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/index.html

Populates the model selector dropdown with available ColBERT models. Sets the initial selected model based on the current state.

```javascript
populateModelSelector() {
    CONSTANTS.AVAILABLE_MODELS.forEach(modelName => {
      const option = document.createElement('option');
      option.value = modelName;
      option.textContent = modelName;
      this.ui.modelSelector.appendChild(option);
    });
    this.ui.modelSelector.value = this.state.currentModelName;
  }
```

--------------------------------

### Render Aggregated Token Importance

Source: https://github.com/lightonai/pylate-rs/blob/main/docs/index.html

Visualizes aggregated token importance by calculating column sums from the similarity matrix. It highlights tokens based on their normalized importance score, adjusting text color for readability.

```javascript
renderTokenImportance(matrix, docTokens, maxSimDocIndices) {
    const columnSums = new Array(docTokens.length).fill(0);
    matrix.forEach(row => row.forEach((score, j) => columnSums[j] += score));
    const minSum = Math.min(...columnSums),
        maxSum = Math.max(...columnSums);
    const range = maxSum - minSum;
    this.ui.tokenImportanceContainer.innerHTML = '';
    const fragment = document.createDocumentFragment();
    docTokens.forEach((token, i) => {
        const tokenSpan = document.createElement('span');
        if (maxSimDocIndices.has(i)) {
            const normScore = range > 0 ? (columnSums[i] - minSum) / range : 0;
            tokenSpan.className = 'highlighted-token';
            tokenSpan.style.backgroundColor = Utils.getColorForIntensity(normScore);
            tokenSpan.style.color = normScore > 0.6 ? '#f8fafc' : '#1f2937';
            tokenSpan.textContent = token;
        } else {
            tokenSpan.textContent = token + ' ';
            tokenSpan.style.color = '#6b7280';
        }
        fragment.appendChild(tokenSpan);
    });
    this.ui.tokenImportanceContainer.appendChild(fragment);
}
```

--------------------------------

### Encode and Calculate Similarity in WebAssembly

Source: https://context7.com/lightonai/pylate-rs/llms.txt

Encodes queries and documents into embeddings and computes similarity scores using WASM bindings. Hierarchical pooling can be applied to reduce embedding size. Assumes the model is already loaded.

```javascript
import { ColBERT, hierarchical_pooling } from "pylate-rs";

// Assume model is already loaded (see previous example)
const model = await loadModel("lightonai/GTE-ModernColBERT-v1");

// Encode queries
const queryResult = model.encode(
    { sentences: ["What is the capital of France?", "How big is the sun?"] },
    true  // is_query=true
);
console.log("Query embeddings:", queryResult.embeddings.length);

// Encode documents
const documentResult = model.encode(
    {
        sentences: [
            "Paris is the capital of France.",
            "The sun is a star at the center of the Solar System.",
            "Berlin is the capital of Germany."
        ],
        batch_size: 16  // Optional: override batch size
    },
    false  // is_query=false
);
console.log("Document embeddings:", documentResult.embeddings.length);

// Calculate similarity scores directly from text
const similarityResult = model.similarity({
    queries: ["What is the capital of France?"],
    documents: ["Paris is the capital of France.", "Berlin is the capital of Germany."]
});
console.log("Similarity scores:", similarityResult.data);
// Output: [[25.123, 18.456]]

// Get raw similarity matrix with token information
const rawResult = model.raw_similarity_matrix({
    queries: ["capital of France"],
    documents: ["Paris is the capital of France."]
});
console.log("Query tokens:", rawResult.query_tokens);
console.log("Document tokens:", rawResult.document_tokens);
console.log("Matrix shape:", rawResult.similarity_matrix[0][0].length);

// Apply hierarchical pooling to reduce embedding size
const pooledResult = hierarchical_pooling({
    embeddings: documentResult.embeddings,
    pool_factor: 2
});
console.log("Pooled embeddings:", pooledResult.embeddings.length);
```