FastEmbed

https://qdrant.github.io/fastembed/
Admin
FastEmbed is a lightweight and fast Python library built for embedding generation using ONNX Runtime...
Tokens:38,598
Snippets:314
Trust Score:9.7
Update:4 weeks ago
Show doc for...
Tokens:
Raw
### Install dependencies

Source: https://qdrant.github.io/fastembed/examples/Hybrid_Search

Installs required packages: qdrant-client, fastembed, datasets, transformers.

```bash
!pip install -qU qdrant-client fastembed datasets transformers
```

--------------------------------

### Install FastEmbed with pip

Source: https://qdrant.github.io/fastembed

Install the FastEmbed library using pip.

```bash
pip install fastembed

```

--------------------------------

### Prepare example documents

Source: https://qdrant.github.io/fastembed/qdrant/Usage_With_Qdrant

Defines a list of example documents about Maharana Pratap. These documents will be embedded and added to the Qdrant collection.

```python
# Example list of documents
documents: list[str] = [
    "Maharana Pratap was a Rajput warrior king from Mewar",
    "He fought against the Mughal Empire led by Akbar",
    "The Battle of Haldighati in 1576 was his most famous battle",
    "He refused to submit to Akbar and continued guerrilla warfare",
    "His capital was Chittorgarh, which he lost to the Mughals",
    "He died in 1597 at the age of 57",
    "Maharana Pratap is considered a symbol of Rajput resistance against foreign rule",
    "His legacy is celebrated in Rajasthan through festivals and monuments",
    "He had 11 wives and 17 sons, including Amar Singh I who succeeded him as ruler of Mewar",
    "His life has been depicted in various films, TV shows, and books",
]

```

--------------------------------

### Install dependencies with pip

Source: https://qdrant.github.io/fastembed/examples/FastEmbed_vs_HF_Comparison

Installs matplotlib, transformers, and torch using pip with the -qq flag to suppress output.

```bash
!pip install matplotlib transformers torch -qq

```

--------------------------------

### Install dependencies with pip

Source: https://qdrant.github.io/fastembed/examples/Hindi_Tamil_RAG_with_Navarasa7B

Install required packages for the notebook: fastembed, datasets, qdrant-client, peft, transformers, accelerate, and bitsandbytes. Run this in a Colab or local environment before proceeding.

```bash
!pip install -U fastembed datasets qdrant-client peft transformers accelerate bitsandbytes -qq

```

--------------------------------

### Install FastEmbed

Source: https://qdrant.github.io/fastembed/qdrant/Retrieval_with_FastEmbed

Install the fastembed package to create embeddings and perform retrieval.

```bash
# !pip install fastembed --quiet --upgrade

```

--------------------------------

### Install onnxruntime-gpu for CUDA 11.x

Source: https://qdrant.github.io/fastembed/examples/FastEmbed_GPU

Install the latest onnxruntime-gpu with CUDA 11.x support. Ensure CuDNN 9.x is installed.

```bash
!pip install onnxruntime-gpu -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-11/pypi/simple/ -qq

```

--------------------------------

### Install fastembed-gpu

Source: https://qdrant.github.io/fastembed/examples/FastEmbed_GPU

Install the fastembed-gpu package, which substitutes onnxruntime with onnxruntime-gpu. Note that onnxruntime-gpu and onnxruntime cannot coexist in the same environment.

```bash
!pip install fastembed-gpu

```

--------------------------------

### Install Qdrant Client with FastEmbed

Source: https://qdrant.github.io/fastembed

Install the Qdrant client with FastEmbed support using pip. On zsh, you may need to quote the package name.

```bash
pip install qdrant-client[fastembed]

```

--------------------------------

### Install dependencies with pip

Source: https://qdrant.github.io/fastembed/qdrant/Binary_Quantization_with_Qdrant

Installs the required Python packages: qdrant-client, pandas, and dataset. The --quiet and --upgrade flags suppress output and upgrade existing packages.

```bash
!pip install qdrant-client pandas dataset --quiet --upgrade

```

--------------------------------

### Install CUDA 12.x on Ubuntu 22.04

Source: https://qdrant.github.io/fastembed/examples/FastEmbed_GPU

Download and install the CUDA toolkit for Ubuntu 22.04 x86_64. After installation, set LD_LIBRARY_PATH to /usr/local/cuda/lib64.

```bash
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install cuda

```

--------------------------------

### Install fastembed with pip

Source: https://qdrant.github.io/fastembed/Getting%20Started

Install the fastembed package using pip with the -Uqq flags for quiet upgrade.

```bash
!pip install -Uqq fastembed

```

--------------------------------

### Install dependencies for binary quantization

Source: https://qdrant.github.io/fastembed/experimental/Binary%20Quantization%20from%20Scratch

Installs required Python packages: matplotlib, tqdm, pandas, numpy, and datasets.

```bash
!pip install matplotlib tqdm pandas numpy datasets --quiet --upgrade

```

--------------------------------

### Use FastEmbed with Qdrant

Source: https://qdrant.github.io/fastembed

Initialize an in-process Qdrant client, add documents with metadata and IDs, and query the collection using text. The example shows how to add documents and perform a text search.

```python
from qdrant_client import QdrantClient

# Initialize the client
client = QdrantClient(":memory:")  # Using an in-process Qdrant

# Prepare your documents, metadata, and IDs
docs = ["Qdrant has Langchain integrations", "Qdrant also has Llama Index integrations"]
metadata = [
    {"source": "Langchain-docs"},
    {"source": "Llama-index-docs"},
]
ids = [42, 2]

client.add(
    collection_name="demo_collection",
    documents=docs,
    metadata=metadata,
    ids=ids
)

search_result = client.query(
    collection_name="demo_collection",
    query_text="This is a query document"
)
print(search_result)

```

--------------------------------

### Error: CuDNN library not installed

Source: https://qdrant.github.io/fastembed/examples/FastEmbed_GPU

Common error when CuDNN library is missing: libcudnn.so.x cannot be found.

```text
FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcudnn.so.x: cannot open shared object file: No such file or directory

```

--------------------------------

### Error: CUDA library not installed

Source: https://qdrant.github.io/fastembed/examples/FastEmbed_GPU

Common error when CUDA library is missing: libcublasLt.so.x cannot be found.

```text
FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.x: cannot open shared object file: No such file or directory

```

--------------------------------

### Load and filter the ESCI dataset

Source: https://qdrant.github.io/fastembed/examples/Hybrid_Search

Loads the tasksource/esci dataset, selects the first 1000 examples, and filters to product_locale == 'us', resulting in 919 rows.

```python
dataset = load_dataset("tasksource/esci", split="train")
# We'll select the first 1000 examples for this demo
dataset = dataset.select(range(1000))
dataset = dataset.filter(lambda x: x["product_locale"] == "us")
dataset
```

```python
Dataset({
    features: ['example_id', 'query', 'query_id', 'product_id', 'product_locale', 'esci_label', 'small_version', 'large_version', 'product_title', 'product_description', 'product_bullet_point', 'product_brand', 'product_color', 'product_text'],
    num_rows: 919
})
```

--------------------------------

### Install fastembed-gpu with onnxruntime-gpu 1.18.0 for CuDNN 8

Source: https://qdrant.github.io/fastembed/examples/FastEmbed_GPU

Lock onnxruntime-gpu to 1.18.0 with CUDA 12.x to work with CuDNN 8, then install fastembed-gpu.

```bash
!pip install onnxruntime-gpu==1.18.0 -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/ -qq
!pip install fastembed-gpu -qqq

```

--------------------------------

### Import libraries and set random seeds

Source: https://qdrant.github.io/fastembed/qdrant/Binary_Quantization_with_Qdrant

Imports the necessary libraries and sets random seeds for reproducibility. The time module is imported but not used in the subsequent examples.

```python
import os
import random
import time

import datasets
import numpy as np
import pandas as pd
from qdrant_client import QdrantClient, models

random.seed(37)
np.random.seed(37)

```

--------------------------------

### Initialize FastEmbed TextEmbedding with default model

Source: https://qdrant.github.io/fastembed/examples/FastEmbed_vs_HF_Comparison

Initializes a FastEmbed TextEmbedding instance using the same model ID as the HuggingFace example. The output shows a warning about tokenizers parallelism and a progress bar for fetching model files.

```python
embedding_model = TextEmbedding(model_name=model_id)
```

```text
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
    - Avoid using `tokenizers` before the fork if possible
    - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
```

```text
Fetching 9 files:   0%|          | 0/9 [00:00<?, ?it/s]
```

--------------------------------

### Example output of decoded SPLADE tokens and weights

Source: https://qdrant.github.io/fastembed/examples/SPLADE_with_FastEmbed

Shows the JSON output of the decoded tokens and weights for a sample SPLADE embedding, with tokens like 'chandra' and 'third' having the highest weights.

```json
{
    "chandra": 1.7163975238800049,
    "third": 1.5655333995819092,
    "##ya": 1.535199522972107,
    "india": 1.5310232639312744,
    "3": 1.385086178779602,
    "mission": 1.3676567077636719,
    "lunar": 1.3591278791427612,
    "moon": 1.2580504417419434,
    "indian": 1.1001816987991333,
    "##an": 1.015198826789856,
    "3rd": 0.7661278247833252,
    "was": 0.7177659273147583,
    "spacecraft": 0.7071069478988647,
    "space": 0.5978556871414185,
    "flight": 0.4988254904747009,
    "satellite": 0.4826201796531677,
    "first": 0.46230843663215637,
    "expedition": 0.4515230059623718,
    "three": 0.4467709958553314,
    "fourth": 0.44249090552330017,
    "vehicle": 0.390580952167511,
    "iii": 0.3862902522087097,
    "2": 0.36459630727767944,
    "##3": 0.2941221296787262,
    "planet": 0.27236196398735046,
    "second": 0.26897504925727844,
    "missions": 0.2608523368835449,
    "launched": 0.15740394592285156,
    "had": 0.12667948007583618,
    "largest": 0.09955651313066483,
    "leader": 0.09747757017612457,
    ",": 0.05297207832336426,
    "study": 0.02079751156270504,
    "-": 0.01963476650416851
}

```

--------------------------------

### Set up Qdrant and insert points

Source: https://qdrant.github.io/fastembed/examples/Hindi_Tamil_RAG_with_Navarasa7B

Create an in-memory Qdrant client, create a collection with cosine distance, and upsert the context points into it.

```python
search_client = QdrantClient(":memory:")

search_client.create_collection(
    collection_name="hindi_tamil_contexts",
    vectors_config=VectorParams(size=len(context_points[0].vector), distance=Distance.COSINE),
)
search_client.upsert(collection_name="hindi_tamil_contexts", points=context_points)

```

--------------------------------

### Initialize in-memory Qdrant client

Source: https://qdrant.github.io/fastembed/examples/Hybrid_Search

Initializes an in-memory Qdrant client using Numpy under the hood for demonstration purposes. In production, use Docker or Cloud for full database support.

```python
client = QdrantClient(":memory:")
```

--------------------------------

### Initialize TextEmbedding with CPU (default)

Source: https://qdrant.github.io/fastembed/examples/FastEmbed_GPU

Creates a TextEmbedding model without specifying providers, which defaults to CPU execution. The model downloads files and prints the available providers.

```python
embedding_model_cpu = TextEmbedding(model_name="BAAI/bge-small-en-v1.5")
embedding_model_cpu.model.model.get_providers()

```

```text
Fetching 5 files:   0%|          | 0/5 [00:00<?, ?it/s]

```

```text
['CPUExecutionProvider']

```

--------------------------------

### Install CuDNN 9.x on Ubuntu 22.04

Source: https://qdrant.github.io/fastembed/examples/FastEmbed_GPU

Download and install CuDNN 9.x for Ubuntu 22.04 x86_64. You can choose a specific version like cudnn-cuda-11 or cudnn-cuda-12.

```bash
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install cudnn

```

--------------------------------

### Model ready confirmation output

Source: https://qdrant.github.io/fastembed/Getting%20Started

Output confirming the default model is ready after initialization.

```text
The model BAAI/bge-small-en-v1.5 is ready to use.

```

--------------------------------

### Install fastembed-gpu with CuDNN 9 for CUDA 12.x

Source: https://qdrant.github.io/fastembed/examples/FastEmbed_GPU

For CUDA 12.x with onnxruntime-gpu 1.19.0+, install CuDNN 9 and then fastembed-gpu. Google Colab defaults to CUDA 12.x and CuDNN 8.x.

```bash
!sudo apt install cudnn9
!pip install fastembed-gpu -qqq

```

--------------------------------

### Import libraries

Source: https://qdrant.github.io/fastembed/examples/Hybrid_Search

Imports necessary modules including Qdrant client, models, datasets, transformers, and fastembed.

```python
import json

import numpy as np
import pandas as pd
from datasets import load_dataset
from qdrant_client import QdrantClient
from qdrant_client.models import (
    Distance,
    NamedSparseVector,
    NamedVector,
    SparseVector,
    PointStruct,
    SearchRequest,
    SparseIndexParams,
    SparseVectorParams,
    VectorParams,
    ScoredPoint,
)
from transformers import AutoTokenizer

import fastembed
from fastembed import SparseEmbedding, SparseTextEmbedding, TextEmbedding
```

--------------------------------

### Sparse embedding output example

Source: https://qdrant.github.io/fastembed/examples/Hybrid_Search

Example output of a sparse embedding, showing a JSON object mapping tokens (including subword tokens like "##bed") to their importance scores. The scores are floating-point numbers, and the object is typically used for hybrid search.

```json
{
    "fast": 2.5722568035125732,
    "##bed": 2.1076643466949463,
    "##em": 1.9767478704452515,
    "text": 1.8717442750930786,
    "em": 1.6925323009490967,
    "library": 1.4840331077575684,
    "##ding": 1.2125115394592285,
    "bed": 1.2047640085220337,
    "good": 0.9082595109939575,
    "librarian": 0.9047954082489014,
    "is": 0.8048465847969055,
    "software": 0.7071243524551392,
    "format": 0.6728256344795227,
    "great": 0.613731324672699,
    "texts": 0.5162659287452698,
    "quick": 0.49152234196662903,
    "device": 0.4451214075088501,
    "file": 0.44369080662727356,
    "for": 0.4135623872280121,
    "best": 0.38512513041496277,
    "technique": 0.36704862117767334,
    "facility": 0.2912774682044983,
    "method": 0.26381176710128784,
    "ideal": 0.26357391476631165,
    "perfect": 0.2541669011116028,
    "##bing": 0.24062614142894745,
    "material": 0.23214206099510193,
    "storage": 0.21676106750965118,
    "tool": 0.21145488321781158,
    "nice": 0.20749981701374054,
    "web": 0.19741220772266388,
    "architecture": 0.1926720291376114,
    "##b": 0.18546289205551147,
    "book": 0.18313883244991302,
    "a": 0.17295610904693604,
    "speed": 0.17008088529109955,
    "##am": 0.1678074449300766,
    "##ization": 0.16315481066703796,
    "browser": 0.11985089629888535,
    "##ogen": 0.10030396282672882,
    "database": 0.09790635108947754,
    "connection": 0.09682106971740723,
    "excellent": 0.0670650377869606,
    "computer": 0.06487759202718735,
    "java": 0.055845409631729126,
    "algorithm": 0.051508933305740356,
    "program": 0.04257776960730553,
    "wonderful": 0.00856015458703041
}
```

--------------------------------

### Initialize SPLADE model

Source: https://qdrant.github.io/fastembed/examples/SPLADE_with_FastEmbed

Create a SparseTextEmbedding instance with the model name 'prithvida/Splade_PP_en_v1'. This triggers the model download, as shown by the progress output.

```python
model_name = "prithvida/Splade_PP_en_v1"
# This triggers the model download
model = SparseTextEmbedding(model_name=model_name)

```

```text
Fetching 9 files:   0%|          | 0/9 [00:00<?, ?it/s]

```

--------------------------------

### Get DataFrame length

Source: https://qdrant.github.io/fastembed/examples/Hybrid_Search

Returns the number of rows in the cleaned DataFrame.

```python
len(df)
```

--------------------------------

### Get embedding dimension

Source: https://qdrant.github.io/fastembed/experimental/Binary%20Quantization%20from%20Scratch

Stores the number of dimensions of the embedding vectors.

```python
n_dim = openai_vectors.shape[1]
n_dim

```

--------------------------------

### Initialize TextEmbedding with CUDAExecutionProvider

Source: https://qdrant.github.io/fastembed/examples/FastEmbed_GPU

Creates a TextEmbedding model with GPU acceleration by specifying providers=['CUDAExecutionProvider']. The model downloads files from Hugging Face Hub and prints the available providers.

```python
import numpy as np

from fastembed import TextEmbedding

embedding_model_gpu = TextEmbedding(
    model_name="BAAI/bge-small-en-v1.5", providers=["CUDAExecutionProvider"]
)
embedding_model_gpu.model.model.get_providers()

```

```text
/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:88: UserWarning: 
The secret `HF_TOKEN` does not exist in your Colab secrets.
To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.
You will be able to reuse this secret in all of your notebooks.
Please note that authentication is recommended but still optional to access public models or datasets.
  warnings.warn(


```

```text
Fetching 5 files:   0%|          | 0/5 [00:00<?, ?it/s]

```

```text
tokenizer_config.json:   0%|          | 0.00/1.24k [00:00<?, ?B/s]

```

```text
config.json:   0%|          | 0.00/706 [00:00<?, ?B/s]

```

```text
special_tokens_map.json:   0%|          | 0.00/695 [00:00<?, ?B/s]

```

```text
tokenizer.json:   0%|          | 0.00/711k [00:00<?, ?B/s]

```

```text
model_optimized.onnx:   0%|          | 0.00/66.5M [00:00<?, ?B/s]

```

```text
['CUDAExecutionProvider', 'CPUExecutionProvider']

```

--------------------------------

### Import libraries for binary quantization

Source: https://qdrant.github.io/fastembed/experimental/Binary%20Quantization%20from%20Scratch

Imports numpy, pandas, datasets, and tqdm for data handling and progress bars.

```python
import numpy as np
import pandas as pd
from datasets import load_dataset
from tqdm import tqdm

```

--------------------------------

### Import required libraries

Source: https://qdrant.github.io/fastembed/examples/Hindi_Tamil_RAG_with_Navarasa7B

Import all necessary libraries: numpy, datasets, peft, qdrant-client, transformers, and fastembed. These are used for data loading, model loading, vector search, and embedding generation.

```python
import numpy as np
from datasets import load_dataset
from peft import AutoPeftModelForCausalLM
from qdrant_client import QdrantClient
from qdrant_client.models import PointStruct, VectorParams, Distance
from transformers import AutoTokenizer

from fastembed import TextEmbedding

```

--------------------------------

### Get length of RRF rank list

Source: https://qdrant.github.io/fastembed/examples/Hybrid_Search

Prints the number of items in the RRF-ranked list, which is 11.

```python
len(rrf_rank_list)

```

--------------------------------

### Create Qdrant collection with binary quantization

Source: https://qdrant.github.io/fastembed/qdrant/Binary_Quantization_with_Qdrant

Creates a Qdrant client and a collection named 'binary-quantization' with binary quantization enabled. The collection uses DOT distance, on-disk vectors, and the binary quantization config with always_ram=True. The output True indicates successful creation.

```python
client = QdrantClient(  # assumes Qdrant is launched at localhost:6333
    prefer_grpc=True,
)

collection_name = "binary-quantization"

client.create_collection(
    collection_name=collection_name,
    vectors_config=models.VectorParams(
        size=n_dim,
        distance=models.Distance.DOT,
        on_disk=True,
    ),
    quantization_config=models.BinaryQuantization(
        binary=models.BinaryQuantizationConfig(always_ram=True),
    ),
)

```

```text
True

```

--------------------------------

### Get top context text

Source: https://qdrant.github.io/fastembed/examples/Hindi_Tamil_RAG_with_Navarasa7B

Extract the text of the top search result and check its length.

```python
search_context_text = search_context[0].payload["text"]
len(search_context_text)

```

--------------------------------

### Get ground truth answer

Source: https://qdrant.github.io/fastembed/examples/Hindi_Tamil_RAG_with_Navarasa7B

Retrieve the ground truth answer for the selected question to compare with the generated answer.

```python
ds[idx]["answer"]

```

--------------------------------

### Initialize Sparse and Dense Embedding Models

Source: https://qdrant.github.io/fastembed/examples/Hybrid_Search

Initializes the sparse and dense embedding models. The model download is triggered automatically upon instantiation. The sparse model uses the SPLADE++ model, and the dense model uses BGE-large-en-v1.5.

```python
sparse_model_name = "prithvida/Splade_PP_en_v1"
dense_model_name = "BAAI/bge-large-en-v1.5"
# This triggers the model download
sparse_model = SparseTextEmbedding(model_name=sparse_model_name, batch_size=32)
dense_model = TextEmbedding(model_name=dense_model_name, batch_size=32)

```

--------------------------------

### Reciprocal Rank Fusion (RRF) implementation and example output

Source: https://qdrant.github.io/fastembed/examples/Hybrid_Search

Implements Reciprocal Rank Fusion (RRF) using NumPy for combining multiple rank lists. The function takes a list of rank lists (each a list of (item, rank) tuples) and returns a sorted list of items by their RRF scores. The example usage with three rank lists produces the output shown, demonstrating the fusion result.

```python
def rrf(rank_lists, alpha=60, default_rank=1000):
    """
    Optimized Reciprocal Rank Fusion (RRF) using NumPy for large rank lists.

    :param rank_lists: A list of rank lists. Each rank list should be a list of (item, rank) tuples.
    :param alpha: The parameter alpha used in the RRF formula. Default is 60.
    :param default_rank: The default rank assigned to items not present in a rank list. Default is 1000.
    :return: Sorted list of items based on their RRF scores.
    """
    # Consolidate all unique items from all rank lists
    all_items = set(item for rank_list in rank_lists for item, _ in rank_list)

    # Create a mapping of items to indices
    item_to_index = {item: idx for idx, item in enumerate(all_items)}

    # Initialize a matrix to hold the ranks, filled with the default rank
    rank_matrix = np.full((len(all_items), len(rank_lists)), default_rank)

    # Fill in the actual ranks from the rank lists
    for list_idx, rank_list in enumerate(rank_lists):
        for item, rank in rank_list:
            rank_matrix[item_to_index[item], list_idx] = rank

    # Calculate RRF scores using NumPy operations
    rrf_scores = np.sum(1.0 / (alpha + rank_matrix), axis=1)

    # Sort items based on RRF scores
    sorted_indices = np.argsort(-rrf_scores)  # Negative for descending order

    # Retrieve sorted items
    sorted_items = [(list(item_to_index.keys())[idx], rrf_scores[idx]) for idx in sorted_indices]

    return sorted_items


# Example usage
rank_list1 = [("A", 1), ("B", 2), ("C", 3)]
rank_list2 = [("B", 1), ("C", 2), ("D", 3)]
rank_list3 = [("A", 2), ("D", 1), ("E", 3)]

# Combine the rank lists
sorted_items = rrf([rank_list1, rank_list2, rank_list3])
sorted_items
```

```python
[('A', 0.033465871107430434),
 ('B', 0.033465871107430434),
 ('D', 0.03320985472238179),
 ('C', 0.03294544435749548),
 ('E', 0.01775980832584606)]
```

--------------------------------

### Define sample documents list

Source: https://qdrant.github.io/fastembed/examples/FastEmbed_vs_HF_Comparison

Defines a list of strings containing sample documents about the Chandrayaan-3 mission.

```python
documents: list[str] = [
    "Chandrayaan-3 is India's third lunar mission",
    "It aimed to land a rover on the Moon's surface - joining the US, China and Russia",
    "The mission is a follow-up to Chandrayaan-2, which had partial success",
    "Chandrayaan-3 will be launched by the Indian Space Research Organisation (ISRO)",
    "The estimated cost of the mission is around $35 million",
    "It will carry instruments to study the lunar surface and atmosphere",
    "Chandrayaan-3 landed on the Moon's surface on 23rd August 2023",
    "It consists of a lander named Vikram and a rover named Pragyan similar to Chandrayaan-2. Its propulsion module would act like an orbiter.",
    "The propulsion module carries the lander and rover configuration until the spacecraft is in a 100-kilometre (62 mi) lunar orbit",
    "The mission used GSLV Mk III rocket for its launch",
    "Chandrayaan-3 was launched from the Satish Dhawan Space Centre in Sriharikota",
    "Chandrayaan-3 was launched earlier in the year 2023",
]
len(documents)

```

--------------------------------

### Tokenizers parallelism warning

Source: https://qdrant.github.io/fastembed/examples/Hindi_Tamil_RAG_with_Navarasa7B

Warning message from the tokenizers library after the pip install command. It indicates that parallelism was disabled to avoid deadlocks and suggests setting TOKENIZERS_PARALLELISM to true or false.

```text
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
    - Avoid using `tokenizers` before the fork if possible
    - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)

```

--------------------------------

### Import libraries for comparison

Source: https://qdrant.github.io/fastembed/examples/FastEmbed_vs_HF_Comparison

Imports time, typing, matplotlib, torch, transformers, and fastembed modules needed for the comparison.

```python
import time
from typing import Callable

import matplotlib.pyplot as plt
import torch.nn.functional as F
from torch import Tensor
from transformers import AutoModel, AutoTokenizer

from fastembed import TextEmbedding

```

--------------------------------

### Retrieval Embedding Generation with FastEmbed

Source: https://qdrant.github.io/fastembed

Generate embeddings for a list of documents using the TextEmbedding class. The example shows how to embed passages and queries, and the output is a list of numpy arrays.

```python
from fastembed import TextEmbedding

documents: list[str] = [
    "passage: Hello, World!",
    "query: Hello, World!",
    "passage: This is an example passage.",
    "fastembed is supported by and maintained by Qdrant."
]
embedding_model = TextEmbedding()
embeddings: list[np.ndarray] = embedding_model.embed(documents)

```

--------------------------------

### Import QdrantClient

Source: https://qdrant.github.io/fastembed/qdrant/Usage_With_Qdrant

Imports the QdrantClient class from the qdrant_client package, which is used to interact with the Qdrant database.

```python
from qdrant_client import QdrantClient

```

--------------------------------

### Model download progress output

Source: https://qdrant.github.io/fastembed/Getting%20Started

Output showing the model download progress when initializing TextEmbedding for the first time.

```text
Fetching 9 files:   0%|          | 0/9 [00:00<?, ?it/s]

```

--------------------------------

### Compute relevance scores for example query

Source: https://qdrant.github.io/fastembed/examples/ColBERT_with_FastEmbed

Calls the compute_relevance_scores function with the first query embedding and document embeddings, requesting top 3 documents. Prints the sorted document indices.

```python
sorted_indices = compute_relevance_scores(
    np.array(query_embeddings[0]), np.array(document_embeddings), k=3
)
print("Sorted document indices:", sorted_indices)

```

--------------------------------

### Import Libraries

Source: https://qdrant.github.io/fastembed/qdrant/Retrieval_with_FastEmbed

Import numpy and the TextEmbedding class from fastembed.

```python
import numpy as np
from fastembed import TextEmbedding

```

--------------------------------

### Sample query indices from dataset

Source: https://qdrant.github.io/fastembed/qdrant/Binary_Quantization_with_Qdrant

Samples 100 random indices from the dataset and creates a query dataset from those vectors. The final expression prints the list of sampled indices.

```python
query_indices = random.sample(range(len(dataset)), 100)
query_dataset = dataset[query_indices]
query_indices
```

--------------------------------

### Retrieve points by ID from RRF rank list

Source: https://qdrant.github.io/fastembed/examples/Hybrid_Search

Retrieves the full points from the collection using the IDs from the RRF-ranked list. The function takes the Qdrant client, collection name, and the rrf_rank_list, then calls client.retrieve with the list of IDs. The example calls it with the previously computed rrf_rank_list.

```python
def find_point_by_id(
    client: QdrantClient, collection_name: str, rrf_rank_list: list[tuple[int, float]]
):
    return client.retrieve(
        collection_name=collection_name, ids=[item[0] for item in rrf_rank_list]
    )


find_point_by_id(client, collection_name, rrf_rank_list)
```

--------------------------------

### Load SPLADE tokenizer from supported models

Source: https://qdrant.github.io/fastembed/examples/SPLADE_with_FastEmbed

Initializes a tokenizer from the first supported SPLADE model's Hugging Face source. Requires the transformers library and SparseTextEmbedding from fastembed.

```python
import json
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(
    SparseTextEmbedding.list_supported_models()[0]["sources"]["hf"]
)

```

--------------------------------

### Convert search results to rank lists and apply RRF

Source: https://qdrant.github.io/fastembed/examples/Hybrid_Search

Converts the dense and sparse search results into rank lists, then applies the RRF function to combine them. The rank_list function enumerates the search results starting at rank 1. The combined rrf_rank_list is displayed, showing the fused ranking of point IDs with their RRF scores.

```python
def rank_list(search_result: list[ScoredPoint]):
    return [(point.id, rank + 1) for rank, point in enumerate(search_result)]


dense_rank_list, sparse_rank_list = rank_list(search_results[0]), rank_list(search_results[1])
```

```python
rrf_rank_list = rrf([dense_rank_list, sparse_rank_list])
```

```python
rrf_rank_list
```

```python
[(3, 0.032018442622950824),
 (8, 0.03149801587301587),
 (1, 0.03131881575727918),
 (13, 0.030834914611005692),
 (15, 0.030536130536130537),
 (9, 0.030309988518943745),
 (12, 0.030158730158730156),
 (14, 0.029437229437229435),
 (11, 0.028985507246376812),
 (2, 0.01707242848447961),
 (4, 0.01564927857935627)]
```

--------------------------------

### Create Qdrant points

Source: https://qdrant.github.io/fastembed/examples/Hindi_Tamil_RAG_with_Navarasa7B

Create a list of PointStruct objects for Qdrant, each containing an id, the embedding vector, and the original text as payload.

```python
context_points = [
    PointStruct(id=idx, vector=emb, payload={"text": text})
    for idx, (emb, text) in enumerate(zip(context_embeddings, contexts))
]

```

--------------------------------

### List supported image embedding models

Source: https://qdrant.github.io/fastembed/examples/Image_Embedding

Calls ImageEmbedding.list_supported_models() to retrieve a list of supported models with details like dimension, description, size, and source. The output shows two models: Qdrant/clip-ViT-B-32-vision and Qdrant/resnet50-onnx.

```python
ImageEmbedding.list_supported_models()
```

--------------------------------

### Add documents to Qdrant collection

Source: https://qdrant.github.io/fastembed/qdrant/Usage_With_Qdrant

Creates an in-memory Qdrant client and adds the documents to a collection named 'demo_collection'. The add method creates the collection if it does not exist.

```python
client = QdrantClient(":memory:")
client.add(collection_name="test_collection", documents=documents)

```

--------------------------------

### Dense Model Download Progress

Source: https://qdrant.github.io/fastembed/examples/Hybrid_Search

Output from the model download for the dense model, showing progress of fetching 7 files.

```text
Fetching 7 files:   0%|          | 0/7 [00:00<?, ?it/s]

```

--------------------------------

### Prepare documents, metadata, and IDs for adding

Source: https://qdrant.github.io/fastembed/qdrant/Usage_With_Qdrant

Defines documents, metadata, and IDs to be added to the collection. The add method uses FastEmbed to create passage embeddings and upserts the documents with metadata as Points.

```python
# Prepare your documents, metadata, and IDs
docs = ["Qdrant has Langchain integrations", "Qdrant also has Llama Index integrations"]
metadata = [
    {"source": "Langchain-docs"},
    {"source": "Linkedin-docs"},
]
ids = [42, 2]

# Use the new add method
client.add(collection_name="demo_collection", documents=docs, metadata=metadata, ids=ids)

```

--------------------------------

### Load dataset

Source: https://qdrant.github.io/fastembed/examples/Hindi_Tamil_RAG_with_Navarasa7B

Load the Hindi and Tamil question-answering dataset from Hugging Face. The dataset contains questions, contexts, and answers.

```python
ds = load_dataset("nirantk/chaii-hindi-and-tamil-question-answering", split="train")

```

--------------------------------

### Embed documents with TextEmbedding

Source: https://qdrant.github.io/fastembed/Getting%20Started

Initialize the TextEmbedding class with the default model BAAI/bge-small-en-v1.5, embed a list of documents, and check the vector dimension (384). The model download is triggered on initialization.

```python
import numpy as np

from fastembed import TextEmbedding


# Example list of documents
documents: list[str] = [
    "This is built to be faster and lighter than other embedding libraries e.g. Transformers, Sentence-Transformers, etc.",
    "fastembed is supported by and maintained by Qdrant.",
]

# This will trigger the model download and initialization
embedding_model = TextEmbedding()
print("The model BAAI/bge-small-en-v1.5 is ready to use.")

embeddings_generator = embedding_model.embed(documents)
embeddings_list = list(embeddings_generator)
len(embeddings_list[0])  # Vector of 384 dimensions

```

--------------------------------

### List supported sparse embedding models

Source: https://qdrant.github.io/fastembed/examples/SPLADE_with_FastEmbed

Call the list_supported_models() method to see available sparse embedding models. The output shows the SPLADE_PP_en_v1 model details.

```python
SparseTextEmbedding.list_supported_models()

```

```text
[{'model': 'prithvida/Splade_PP_en_v1',
  'vocab_size': 30522,
  'description': 'Misspelled version of the model. Retained for backward compatibility. Independent Implementation of SPLADE++ Model for English',
  'size_in_GB': 0.532,
  'sources': {'hf': 'Qdrant/SPLADE_PP_en_v1'}},
 {'model': 'prithivida/Splade_PP_en_v1',
  'vocab_size': 30522,
  'description': 'Independent Implementation of SPLADE++ Model for English',
  'size_in_GB': 0.532,
  'sources': {'hf': 'Qdrant/SPLADE_PP_en_v1'}}]

```

--------------------------------

### Generate answer with model

Source: https://qdrant.github.io/fastembed/examples/Hindi_Tamil_RAG_with_Navarasa7B

Construct the prompt with the question and context, tokenize it, generate an answer with the model (max 50 new tokens), and decode the response.

```python
input_prompt = """
Answer the following question based on the context given after it in the same language as the question:
### Question:
{}

### Context:
{}

### Answer:
{}"""

input_text = input_prompt.format(
    questions[idx],  # question
    search_context_text[:2000],  # context
    "",  # output - leave this blank for generation!
)

inputs = tokenizer([input_text], return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=50, use_cache=True)
response = tokenizer.batch_decode(outputs)[0]

```

--------------------------------

### Prepare Documents and Generate Embeddings

Source: https://qdrant.github.io/fastembed/qdrant/Retrieval_with_FastEmbed

Initialize the embedding model and generate embeddings for the documents using passage_embed. The documents are a list of strings about Maharana Pratap.

```python
# Example list of documents
documents: list[str] = [
    "Maharana Pratap was a Rajput warrior king from Mewar",
    "He fought against the Mughal Empire led by Akbar",
    "The Battle of Haldighati in 1576 was his most famous battle",
    "He refused to submit to Akbar and continued guerrilla warfare",
    "His capital was Chittorgarh, which he lost to the Mughals",
    "He died in 1597 at the age of 57",
    "Maharana Pratap is considered a symbol of Rajput resistance against foreign rule",
    "His legacy is celebrated in Rajasthan through festivals and monuments",
    "He had 11 wives and 17 sons, including Amar Singh I who succeeded him as ruler of Mewar",
    "His life has been depicted in various films, TV shows, and books",
]
# Initialize the DefaultEmbedding class with the desired parameters
embedding_model = TextEmbedding(model_name="BAAI/bge-small")

# We'll use the passage_embed method to get the embeddings for the documents
embeddings: list[np.ndarray] = list(
    embedding_model.passage_embed(documents)
)  # notice that we are casting the generator to a list

print(embeddings[0].shape, len(embeddings))

```

--------------------------------

### Import SparseTextEmbedding and SparseEmbedding

Source: https://qdrant.github.io/fastembed/examples/SPLADE_with_FastEmbed

Import the necessary classes from fastembed for sparse embedding generation.

```python
from fastembed import SparseTextEmbedding, SparseEmbedding

```
Tokens:
Raw