### Install BCEmbedding from Source

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Install BCEmbedding from its source code for development or advanced usage. This is the recommended installation method.

```bash
git clone git@github.com:netease-youdao/BCEmbedding.git
cd BCEmbedding
pip install -v -e .
```

--------------------------------

### Install BCEmbedding Package

Source: https://github.com/netease-youdao/bcembedding/blob/master/README_zh.md

Installation methods for the BCEmbedding package via pip or source.

```bash
pip install BCEmbedding==0.1.5
```

```bash
git clone git@github.com:netease-youdao/BCEmbedding.git
cd BCEmbedding
pip install -v -e .
```

--------------------------------

### Install BCEmbedding (Minimal)

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Install the BCEmbedding package for minimal usage. It's recommended to manually install a compatible PyTorch version first to avoid CUDA conflicts.

```bash
pip install BCEmbedding==0.1.5
```

--------------------------------

### Install LlamaIndex

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Install the llama-index library to use BCEmbedding tools with it. Ensure you are using version 0.9.42.post2 or compatible.

```bash
pip install llama-index==0.9.42.post2
```

--------------------------------

### Install BCEmbedding Environment

Source: https://github.com/netease-youdao/bcembedding/blob/master/README_zh.md

Commands to set up a conda environment for the project.

```bash
conda create --name bce python=3.10 -y
conda activate bce
```

--------------------------------

### Install Langchain and Related Packages

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Install the necessary Langchain packages for integrating with BCEmbedding tools. Ensure you are using compatible versions as specified.

```bash
pip install langchain==0.1.0
pip install langchain-community==0.0.9
pip install langchain-core==0.1.7
pip install langsmith==0.0.77
```

--------------------------------

### Install LlamaIndex Dependencies

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Installs the required versions of transformers and llama-index for RAG evaluation.

```bash
pip install transformers==4.36.0

pip install llama-index==0.9.22
```

--------------------------------

### Install MTEB for Evaluation

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Install the MTEB library to evaluate embedding and reranker models. Ensure you are using version 1.1.1 or compatible.

```bash
pip install mteb==1.1.1
```

--------------------------------

### Test NPU Performance and Accuracy

Source: https://github.com/netease-youdao/bcembedding/blob/master/Docs/RyzenAI/ReadMe.md

Benchmark the NPU's performance and accuracy against the CPU using a public dataset. This requires installing PyTorch, setuptools, and c_mteb.

```bash
pip install torch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cpu
pip install setuptools==75.1.0
pip install c_mteb==1.1.1
pip install huggingface_hub[hf_xet]

python .\test_perf_accuray.py
```

--------------------------------

### Langchain Integration with BCERerank

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Demonstrates integrating BCEmbedding's `BCERerank` tool with Langchain for RAG frameworks. This example shows initializing embedding models, rerankers, loading documents, splitting texts, and setting up a `ContextualCompressionRetriever`.

```python
# We provide the advanced preproc tokenization for reranking.
from BCEmbedding.tools.langchain import BCERerank

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import PyPDFLoader
from langchain_community.vectorstores import FAISS

from langchain.embeddings import HuggingFaceEmbeddings
from langchain_community.vectorstores.utils import DistanceStrategy
from langchain.retrievers import ContextualCompressionRetriever


# init embedding model
embedding_model_name = 'maidalun1020/bce-embedding-base_v1'
embedding_model_kwargs = {'device': 'cuda:0'}
embedding_encode_kwargs = {'batch_size': 32, 'normalize_embeddings': True, 'show_progress_bar': False}

embed_model = HuggingFaceEmbeddings(
  model_name=embedding_model_name,
  model_kwargs=embedding_model_kwargs,
  encode_kwargs=embedding_encode_kwargs
)

reranker_args = {'model': 'maidalun1020/bce-reranker-base_v1', 'top_n': 5, 'device': 'cuda:1'}
reranker = BCERerank(**reranker_args)

# init documents
documents = PyPDFLoader("BCEmbedding/tools/eval_rag/eval_pdfs/Comp_en_llama2.pdf").load()
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1500, chunk_overlap=200)
texts = text_splitter.split_documents(documents)
```

--------------------------------

### Activate Ryzen AI Conda Environment

Source: https://github.com/netease-youdao/bcembedding/blob/master/Docs/RyzenAI/ReadMe.md

Activate the necessary conda environment for Ryzen AI 1.x. Ensure you have installed the Ryzen AI MSI with the relative NPU driver.

```bash
conda activate ryzen-ai-1.x
```

--------------------------------

### LlamaIndex Integration with BCERerank

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Integrates BCERerank for reranking retrieved nodes in LlamaIndex RAG pipelines. Requires setup of embedding model, reranker, LLM, and service context, followed by document loading and indexing. Supports both manual two-stage retrieval and integrated query engine usage.

```python
from BCEmbedding.tools.llama_index import BCERerank

import os
from llama_index.embeddings import HuggingFaceEmbedding
from llama_index import VectorStoreIndex, ServiceContext, SimpleDirectoryReader
from llama_index.node_parser import SimpleNodeParser
from llama_index.llms import OpenAI
from llama_index.retrievers import VectorIndexRetriever

# Initialize BCE embedding model
embed_model = HuggingFaceEmbedding(
    model_name="maidalun1020/bce-embedding-base_v1",
    max_length=512,
    embed_batch_size=32,
    device="cuda:0"
)

# Initialize BCE reranker
reranker = BCERerank(
    model="maidalun1020/bce-reranker-base_v1",
    top_n=5,          # Return top 5 reranked nodes
    device="cuda:1"
)

# Setup LLM and service context
llm = OpenAI(
    model="gpt-3.5-turbo",
    api_key=os.environ.get("OPENAI_API_KEY")
)
service_context = ServiceContext.from_defaults(
    llm=llm,
    embed_model=embed_model
)

# Load and index documents
documents = SimpleDirectoryReader(input_files=["document.pdf"]).load_data()
node_parser = SimpleNodeParser.from_defaults(chunk_size=400, chunk_overlap=80)
nodes = node_parser.get_nodes_from_documents(documents)
index = VectorStoreIndex(nodes, service_context=service_context)

# Method 1: Manual two-stage retrieval
query = "What is the main conclusion?"

vector_retriever = VectorIndexRetriever(
    index=index,
    similarity_top_k=10,
    service_context=service_context
)

# First stage: embedding-based retrieval
retrieval_by_embedding = vector_retriever.retrieve(query)

# Second stage: reranking
retrieval_by_reranker = reranker.postprocess_nodes(
    retrieval_by_embedding,
    query_str=query
)

for node in retrieval_by_reranker:
    print(f"Score: {node.score:.3f}")
    print(f"Text: {node.node.get_content()[:200]}...")
    print("---")

# Method 2: Integrated query engine with reranking
query_engine = index.as_query_engine(
    node_postprocessors=[reranker]
)
response = query_engine.query(query)
print(response)
```

--------------------------------

### Load Documents and Create Vector Index

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Load documents from a PDF file and parse them into nodes. Then, create a VectorStoreIndex for efficient retrieval. Adjust chunk size and overlap as needed.

```python
documents = SimpleDirectoryReader(input_files=["BCEmbedding/tools/eval_rag/eval_pdfs/Comp_en_llama2.pdf"]).load_data()
node_parser = SimpleNodeParser.from_defaults(chunk_size=400, chunk_overlap=80)
nodes = node_parser.get_nodes_from_documents(documents[0:36])
index = VectorStoreIndex(nodes, service_context=service_context)
```

--------------------------------

### Initialize EmbeddingModel

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Configure the EmbeddingModel with specific pooling strategies, FP16 precision, and device settings.

```python
from BCEmbedding import EmbeddingModel

# Initialize with default BCE embedding model
model = EmbeddingModel(
    model_name_or_path="maidalun1020/bce-embedding-base_v1",
    pooler="cls",        # 'cls' (recommended) or 'mean' pooling
    use_fp16=False,      # Enable FP16 for memory efficiency
    device=None          # Auto-detect: 'cuda', 'cpu', 'cuda:0', or '0'
)

# For specific GPU or FP16 mode
model_fp16 = EmbeddingModel(
    model_name_or_path="maidalun1020/bce-embedding-base_v1",
    pooler="cls",
    use_fp16=True,
    device="cuda:0"
)
```

--------------------------------

### Initialize LlamaIndex Service Context

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Set up the service context for LlamaIndex, including the language model and embedding model. Ensure your OpenAI API key and base URL are set in environment variables.

```python
llm = OpenAI(model='gpt-3.5-turbo-0613', api_key=os.environ.get('OPENAI_API_KEY'), api_base=os.environ.get('OPENAI_BASE_URL'))
service_context = ServiceContext.from_defaults(llm=llm, embed_model=embed_model)
```

--------------------------------

### Reproduce LlamaIndex RAG Results

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Runs the reproduction script for LlamaIndex RAG benchmarks and summarizes the output.

```bash
# There should be two GPUs available at least.
CUDA_VISIBLE_DEVICES=0,1 python BCEmbedding/tools/eval_rag/eval_llamaindex_reproduce.py
```

```bash
python BCEmbedding/tools/eval_rag/summarize_eval_results.py --results_dir BCEmbedding/results/rag_reproduce_results
```

--------------------------------

### Initialize Embedding and Reranker Models

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Initialize the embedding and reranker models with specified parameters. Ensure CUDA devices are available if 'cuda:0' or 'cuda:1' are used.

```python
# init embedding model and reranker model
embed_args = {'model_name': 'maidalun1020/bce-embedding-base_v1', 'max_length': 512, 'embed_batch_size': 32, 'device': 'cuda:0'}
embed_model = HuggingFaceEmbedding(**embed_args)

reranker_args = {'model': 'maidalun1020/bce-reranker-base_v1', 'top_n': 5, 'device': 'cuda:1'}
reranker_model = BCERerank(**reranker_args)
```

--------------------------------

### Configure API Environment Variables

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Sets the necessary environment variables for OpenAI and Cohere API access.

```bash
export OPENAI_BASE_URL={openai_base_url}  # https://api.openai.com/v1
export OPENAI_API_KEY={your_openai_api_key}
export COHERE_APPKEY={your_cohere_api_key}
```

--------------------------------

### Evaluate Reranker Models with MTEB

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Evaluate reranker models using the MTEB benchmark. Specify the model path. The script will automatically run the relevant reranking datasets.

```bash
python BCEmbedding/tools/eval_mteb/eval_reranker_mteb.py --model_name_or_path maidalun1020/bce-reranker-base_v1
```

--------------------------------

### Export Model to ONNX

Source: https://github.com/netease-youdao/bcembedding/blob/master/Docs/RyzenAI/ReadMe.md

Export the specified model to ONNX format using the provided script. This step requires the 'accelerate' library and configures model inputs and outputs.

```bash
pip install accelerate
python export_to_onnx.py --model maidalun1020/bce-embedding-base_v1 --token_length 512 --output_dir bce_onnx_export --opset 17 --model_inputs "input_ids,attention_mask" --model_outputs "token_embeddings,sentence_embedding"
move bce_onnx_export\bce-embedding-base_v1.onnx .\
```

--------------------------------

### Direct Reranker Usage with Transformers

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Demonstrates how to use BCEmbedding reranker models directly with Hugging Face's AutoModelForSequenceClassification. This allows for pairwise scoring of query-passage pairs, including cross-lingual scenarios.

```python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

# Initialize model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("maidalun1020/bce-reranker-base_v1")
model = AutoModelForSequenceClassification.from_pretrained("maidalun1020/bce-reranker-base_v1")

device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)
model.eval()

# Prepare query-passage pairs
sentence_pairs = [
    ["What is deep learning?", "Deep learning is a type of machine learning using neural networks."],
    ["What is deep learning?", "The ocean contains many species of fish."],
    ["什么是自然语言处理?", "NLP是让计算机理解人类语言的技术。"],  # Cross-lingual
]


```

--------------------------------

### RerankerModel with Transformers

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Utilize the `transformers` library for the RerankerModel. This includes initializing the tokenizer and model, moving the model to the correct device, preparing input tensors for sentence pairs, and calculating sigmoid-activated scores.

```python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

# init model and tokenizer
tokenizer = AutoTokenizer.from_pretrained('maidalun1020/bce-reranker-base_v1')
model = AutoModelForSequenceClassification.from_pretrained('maidalun1020/bce-reranker-base_v1')

device = 'cuda'  # if no GPU, set "cpu"
model.to(device)

# get inputs
inputs = tokenizer(sentence_pairs, padding=True, truncation=True, max_length=512, return_tensors="pt")
inputs_on_device = {k: v.to(device) for k, v in inputs.items()}

# calculate scores
scores = model(**inputs_on_device, return_dict=True).logits.view(-1,).float()
scores = torch.sigmoid(scores)
```

--------------------------------

### Run NPU-Only Inference

Source: https://github.com/netease-youdao/bcembedding/blob/master/Docs/RyzenAI/ReadMe.md

Execute NPU-only inference using the generated ONNX context cache file for sanity testing.

```bash
python .\run_npu_only.py --input_model .\bce-embedding-base_v1_ctx.onnx
```

--------------------------------

### Compile Model and Generate EP Context Cache

Source: https://github.com/netease-youdao/bcembedding/blob/master/Docs/RyzenAI/ReadMe.md

Compile the exported ONNX model to generate an EP Context Cache for NPU compatibility. This process offloads operators to the AMD Ryzen AI NPU.

```bash
python .\compile.py  .\bce-embedding-base_v1.onnx
```

--------------------------------

### Evaluate Embedding Models with MTEB

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Evaluate embedding models using the MTEB benchmark. Specify the model path and pooling method. Use '--trust_remote_code' if required by the model.

```bash
python BCEmbedding/tools/eval_mteb/eval_embedding_mteb.py --model_name_or_path maidalun1020/bce-embedding-base_v1 --pooler cls
```

```bash
python BCEmbedding/tools/eval_mteb/eval_embedding_mteb.py --model_name_or_path {mean_pooler_models} --pooler mean
```

```bash
python BCEmbedding/tools/eval_mteb/eval_embedding_mteb.py --model_name_or_path jinaai/jina-embeddings-v2-base-en --pooler mean --trust_remote_code
```

--------------------------------

### Score Sentence Pairs with Transformers

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Use the BCEmbedding model with the transformers library to score sentence pairs. Ensure the model and device are correctly set up.

```python
from transformers import AutoTokenizer, AutoModel
import torch

model_name = "maidalun1020/bce-reranker-base_v1"
device = "cuda" if torch.cuda.is_available() else "cpu"
model = AutoModel.from_pretrained(model_name).to(device)
tokenizer = AutoTokenizer.from_pretrained(model_name)

sentence_pairs = [
    ["What is NLP?", "Natural language processing enables computers to understand human language."],
    ["What is NLP?", "Basketball is a popular sport worldwide."],
]

with torch.no_grad():
    inputs = tokenizer(
        sentence_pairs,
        padding=True,
        truncation=True,
        max_length=512,
        return_tensors="pt"
    )
    inputs = {k: v.to(device) for k, v in inputs.items()}

    logits = model(**inputs, return_dict=True).logits.view(-1).float()
    scores = torch.sigmoid(logits)

print(scores.tolist())  # [0.94, 0.08, 0.89]
```

--------------------------------

### Query with Embedding and Reranker using Query Engine

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Create a query engine that automatically applies the reranker model as a postprocessor. This simplifies the RAG pipeline for querying.

```python
# example #2.2. query with EmbeddingModel and RerankerModel
query_engine = index.as_query_engine(node_postprocessors=[reranker_model])
query_response = query_engine.query(query)
```

--------------------------------

### LangChain Integration: BCERerank Document Compressor

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Integrate BCEmbedding's reranking capabilities into LangChain using the `BCERerank` class. This compressor automatically reranks retrieved documents and adds relevance scores to metadata, enhancing RAG pipelines.

```python
from BCEmbedding.tools.langchain import BCERerank

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import PyPDFLoader
from langchain_community.vectorstores import FAISS
from langchain.embeddings import HuggingFaceEmbeddings
from langchain_community.vectorstores.utils import DistanceStrategy
from langchain.retrievers import ContextualCompressionRetriever

# Initialize BCE embedding model via HuggingFace
embed_model = HuggingFaceEmbeddings(
    model_name="maidalun1020/bce-embedding-base_v1",
    model_kwargs={"device": "cuda:0"},
    encode_kwargs={
        "batch_size": 32,
        "normalize_embeddings": True,
        "show_progress_bar": False
    }
)

# Initialize BCE reranker
reranker = BCERerank(
    model="maidalun1020/bce-reranker-base_v1",
    top_n=5,          # Return top 5 reranked documents
    device="cuda:1"   # Use separate GPU for reranker
)

# Load and split documents
documents = PyPDFLoader("research_paper.pdf").load()
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1500,
    chunk_overlap=200
)
texts = text_splitter.split_documents(documents)

# Create vector store and retriever
retriever = FAISS.from_documents(
    texts,
    embed_model,
    distance_strategy=DistanceStrategy.MAX_INNER_PRODUCT
).as_retriever(
    search_type="similarity",
    search_kwargs={"score_threshold": 0.3, "k": 10}
)

# Create compression retriever with reranking
compression_retriever = ContextualCompressionRetriever(
    base_compressor=reranker,
    base_retriever=retriever
)

# Query with two-stage retrieval
query = "What are the main findings of the research?"
results = compression_retriever.get_relevant_documents(query)

for doc in results:
    print(f"Score: {doc.metadata['relevance_score']:.3f}")
    print(f"Content: {doc.page_content[:200]}...")
    print("---")
```

--------------------------------

### Initialize RerankerModel

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Configure the RerankerModel for cross-encoder scoring, including support for long passage chunking.

```python
from BCEmbedding import RerankerModel

# Initialize with default BCE reranker model
model = RerankerModel(
    model_name_or_path="maidalun1020/bce-reranker-base_v1",
    use_fp16=False,      # Enable FP16 for memory efficiency
    device=None          # Auto-detect: 'cuda', 'cpu', 'cuda:0', or '0'
)

# With custom chunking for long passages
model_long = RerankerModel(
    model_name_or_path="maidalun1020/bce-reranker-base_v1",
    use_fp16=True,
    device="cuda:0",
    max_length=512,       # Max tokens per chunk
    overlap_tokens=80     # Token overlap between chunks
)
```

--------------------------------

### Initialize and Use EmbeddingModel

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Initialize the EmbeddingModel with a specified model name and use it to encode sentences into embeddings. Ensure the model is correctly loaded before encoding.

```python
from BCEmbedding import EmbeddingModel

# init embedding model
model = EmbeddingModel(model_name_or_path="maidalun1020/bce-embedding-base_v1")

# extract embeddings
embeddings = model.encode(sentences)
```

--------------------------------

### Evaluate Multiple Domains

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Evaluates embedding and reranker models on the CrosslingualMultiDomainsDataset and summarizes the results.

```bash
# There should be two GPUs available at least.
CUDA_VISIBLE_DEVICES=0,1 python BCEmbedding/tools/eval_rag/eval_llamaindex_multiple_domains.py
```

```bash
python BCEmbedding/tools/eval_rag/summarize_eval_results.py --results_dir BCEmbedding/results/rag_results
```

--------------------------------

### EmbeddingModel - Initialize Embedding Model

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Initializes the EmbeddingModel for generating dense semantic embeddings. Supports automatic GPU detection, FP16 inference, and different pooling strategies.

```APIDOC
## EmbeddingModel - Initialize Embedding Model

### Description
Initializes the `EmbeddingModel` class to generate dense semantic embeddings from text using a pretrained transformer model. It supports automatic GPU detection, FP16 inference for memory efficiency, and multiple pooling strategies (cls or mean) for different use cases.

### Method
`EmbeddingModel(...)`

### Parameters
- **model_name_or_path** (str) - Required - The name or path of the pretrained transformer model.
- **pooler** (str) - Optional - The pooling strategy to use. Options: 'cls' (recommended) or 'mean'. Defaults to 'cls'.
- **use_fp16** (bool) - Optional - Whether to enable FP16 for memory efficiency. Defaults to False.
- **device** (str or None) - Optional - The device to use for inference. Auto-detects if None. Options: 'cuda', 'cpu', 'cuda:0', or '0'.

### Request Example
```python
from BCEmbedding import EmbeddingModel

# Initialize with default BCE embedding model
model = EmbeddingModel(
    model_name_or_path="maidalun1020/bce-embedding-base_v1",
    pooler="cls",        # 'cls' (recommended) or 'mean' pooling
    use_fp16=False,      # Enable FP16 for memory efficiency
    device=None          # Auto-detect: 'cuda', 'cpu', 'cuda:0', or '0'
)

# For specific GPU or FP16 mode
model_fp16 = EmbeddingModel(
    model_name_or_path="maidalun1020/bce-embedding-base_v1",
    pooler="cls",
    use_fp16=True,
    device="cuda:0"
)
```
```

--------------------------------

### EmbeddingModel with Sentence-Transformers

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Use the `sentence_transformers` library to initialize and use the EmbeddingModel. This method allows for direct encoding of sentences and normalization of the resulting embeddings.

```python
from sentence_transformers import SentenceTransformer

# list of sentences
sentences = ['sentence_0', 'sentence_1', ...]

# init embedding model
## New update for sentence-trnasformers. So clean up your "`SENTENCE_TRANSFORMERS_HOME`/maidalun1020_bce-embedding-base_v1" or "~/.cache/torch/sentence_transformers/maidalun1020_bce-embedding-base_v1" first for downloading new version.
model = SentenceTransformer("maidalun1020/bce-embedding-base_v1")

# extract embeddings
embeddings = model.encode(sentences, normalize_embeddings=True)
```

--------------------------------

### RerankerModel - Initialize Reranker Model

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Initializes the RerankerModel for performing cross-encoder reranking. Supports multiple languages and long passages with automatic chunking.

```APIDOC
## RerankerModel - Initialize Reranker Model

### Description
The `RerankerModel` class performs cross-encoder reranking to refine search results. It supports Chinese, English, Japanese, and Korean, handles long passages with automatic chunking, and provides meaningful relevance scores between 0 and 1.

### Method
`RerankerModel(...)`

### Parameters
- **model_name_or_path** (str) - Required - The name or path of the pretrained reranker model.
- **use_fp16** (bool) - Optional - Whether to enable FP16 for memory efficiency. Defaults to False.
- **device** (str or None) - Optional - The device to use for inference. Auto-detects if None. Options: 'cuda', 'cpu', 'cuda:0', or '0'.
- **max_length** (int) - Optional - Maximum tokens per chunk for long passages. Defaults to 512.
- **overlap_tokens** (int) - Optional - Token overlap between chunks for long passages. Defaults to 80.

### Request Example
```python
from BCEmbedding import RerankerModel

# Initialize with default BCE reranker model
model = RerankerModel(
    model_name_or_path="maidalun1020/bce-reranker-base_v1",
    use_fp16=False,      # Enable FP16 for memory efficiency
    device=None          # Auto-detect: 'cuda', 'cpu', 'cuda:0', or '0'
)

# With custom chunking for long passages
model_long = RerankerModel(
    model_name_or_path="maidalun1020/bce-reranker-base_v1",
    use_fp16=True,
    device="cuda:0",
    max_length=512,       # Max tokens per chunk
    overlap_tokens=80     # Token overlap between chunks
)
```
```

--------------------------------

### Summarize Evaluation Results

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Summarizes evaluation results for embedding or reranker models from a specified directory.

```bash
python BCEmbedding/tools/eval_mteb/summarize_eval_results.py --results_dir {your_embedding_results_dir | your_reranker_results_dir}
```

--------------------------------

### Initialize EmbeddingModel

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Initialize the EmbeddingModel for sentence embedding. The 'cls' pooler is used by default. This model supports Chinese and English.

```python
from BCEmbedding import EmbeddingModel

# list of sentences
sentences = ['sentence_0', 'sentence_1']

```

--------------------------------

### Score Multiple Query-Passage Pairs with BCEmbedding

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Efficiently compute scores for multiple query-passage pairs using `compute_score` with a list of pairs. Supports batch processing and progress bars for large datasets. Handles cross-lingual pairs.

```python
sentence_pairs = [
    ["What is Python?", "Python is a high-level programming language known for its simplicity."],
    ["What is Python?", "Snakes are reptiles found on every continent except Antarctica."],
    ["什么是深度学习?", "深度学习是机器学习的一个子领域,使用多层神经网络。"],
    ["What is deep learning?", "深度学习使用神经网络进行复杂模式识别。"]  # Cross-lingual
]

scores = model.compute_score(
    sentence_pairs,
    batch_size=256,       # Batch size for inference
    max_length=512,       # Max token length
    enable_tqdm=True      # Show progress bar
)
# Output: [0.94, 0.12, 0.91, 0.87]
```

--------------------------------

### Direct Embedding Generation with Transformers

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Utilizes Hugging Face transformers to directly load and use BCEmbedding models for generating sentence embeddings. This method allows for fine-grained control over the embedding process, including device placement and batching.

```python
import torch
from transformers import AutoModel, AutoTokenizer

# Initialize model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("maidalun1020/bce-embedding-base_v1")
model = AutoModel.from_pretrained("maidalun1020/bce-embedding-base_v1")

device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)
model.eval()

# Prepare sentences
sentences = [
    "Machine learning is a subset of artificial intelligence.",
    "机器学习是人工智能的子集。",
    "Deep learning uses neural networks with many layers."
]

# Tokenize and encode
with torch.no_grad():
    inputs = tokenizer(
        sentences,
        padding=True,
        truncation=True,
        max_length=512,
        return_tensors="pt"
    )
    inputs = {k: v.to(device) for k, v in inputs.items()}

    # Get embeddings using CLS pooling
    outputs = model(**inputs, return_dict=True)
    embeddings = outputs.last_hidden_state[:, 0]  # CLS token

    # Normalize embeddings
    embeddings = embeddings / embeddings.norm(dim=1, keepdim=True)

print(embeddings.shape)  # torch.Size([3, 768])

# Calculate pairwise similarities
similarities = torch.mm(embeddings, embeddings.T)
print(similarities)
# tensor([[1.0000, 0.9234, 0.8567],
#         [0.9234, 1.0000, 0.8123],
#         [0.8567, 0.8123, 1.0000]])
```

--------------------------------

### Use RerankerModel with BCEmbedding

Source: https://github.com/netease-youdao/bcembedding/blob/master/README_zh.md

Calculate semantic scores or rerank passages using the RerankerModel class.

```python
from BCEmbedding import RerankerModel

# your query and corresponding passages
query = 'input_query'
passages = ['passage_0', 'passage_1']

# construct sentence pairs
sentence_pairs = [[query, passage] for passage in passages]

# init reranker model
model = RerankerModel(model_name_or_path="maidalun1020/bce-reranker-base_v1")

# method 0: calculate scores of sentence pairs
scores = model.compute_score(sentence_pairs)

# method 1: rerank passages
rerank_results = model.rerank(query, passages)
```

--------------------------------

### Use RerankerModel with Transformers

Source: https://github.com/netease-youdao/bcembedding/blob/master/README_zh.md

Directly use the reranker model via the Hugging Face transformers library.

```python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

# init model and tokenizer
tokenizer = AutoTokenizer.from_pretrained('maidalun1020/bce-reranker-base_v1')
model = AutoModelForSequenceClassification.from_pretrained('maidalun1020/bce-reranker-base_v1')

device = 'cuda'  # if no GPU, set "cpu"
model.to(device)

# get inputs
inputs = tokenizer(sentence_pairs, padding=True, truncation=True, max_length=512, return_tensors="pt")
inputs_on_device = {k: v.to(device) for k, v in inputs.items()}
```

--------------------------------

### RerankerModel with Sentence-Transformers

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Initialize the RerankerModel using `sentence_transformers.CrossEncoder`. This is a straightforward way to predict scores for sentence pairs, with a specified maximum sequence length.

```python
from sentence_transformers import CrossEncoder

# init reranker model
model = CrossEncoder('maidalun1020/bce-reranker-base_v1', max_length=512)

# calculate scores of sentence pairs
scores = model.predict(sentence_pairs)
```

--------------------------------

### Extract Embeddings for Query and Passages

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Generate embeddings for a given query and a list of passages using the initialized embedding model. This is a fundamental step for semantic search.

```python
# example #1. extract embeddings
query = 'apples'
passages = [
        'I like apples', 
        'I like oranges',
        'Apples and oranges are fruits'
    ]
query_embedding = embed_model.get_query_embedding(query)
passages_embeddings = embed_model.get_text_embedding_batch(passages)
```

--------------------------------

### Generate Normalized Embeddings with Sentence Transformers

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Utilize the Sentence Transformers library to generate normalized embeddings for a list of sentences. Ensure the SentenceTransformer model is loaded correctly.

```python
from sentence_transformers import SentenceTransformer, CrossEncoder

embed_model = SentenceTransformer("maidalun1020/bce-embedding-base_v1")

sentences = [
    "Artificial intelligence is transforming industries.",
    "人工智能正在改变各行各业。",
    "The weather forecast predicts rain tomorrow."
]

embeddings = embed_model.encode(
    sentences,
    normalize_embeddings=True,
    show_progress_bar=True
)
print(embeddings.shape)  # (3, 768)
```

--------------------------------

### Retrieve Documents using Embedding and Reranker

Source: https://github.com/netease-youdao/bcembedding/blob/master/README.md

Perform retrieval using a vector retriever and then post-process the results with the reranker model. This enhances the relevance of retrieved documents.

```python
# example #2.1. retrieval with EmbeddingModel and RerankerModel
vector_retriever = VectorIndexRetriever(index=index, similarity_top_k=10, service_context=service_context)
retrieval_by_embedding = vector_retriever.retrieve(query)
retrieval_by_reranker = reranker_model.postprocess_nodes(retrieval_by_embedding, query_str=query)
```

--------------------------------

### Use EmbeddingModel with BCEmbedding

Source: https://github.com/netease-youdao/bcembedding/blob/master/README_zh.md

Extract embeddings using the high-level EmbeddingModel class.

```python
from BCEmbedding import EmbeddingModel

# list of sentences
sentences = ['sentence_0', 'sentence_1']

# init embedding model
model = EmbeddingModel(model_name_or_path="maidalun1020/bce-embedding-base_v1")

# extract embeddings
embeddings = model.encode(sentences)
```

--------------------------------

### Use EmbeddingModel with Transformers

Source: https://github.com/netease-youdao/bcembedding/blob/master/README_zh.md

Directly use the embedding model via the Hugging Face transformers library.

```python
from transformers import AutoModel, AutoTokenizer

# list of sentences
sentences = ['sentence_0', 'sentence_1']

# init model and tokenizer
tokenizer = AutoTokenizer.from_pretrained('maidalun1020/bce-embedding-base_v1')
model = AutoModel.from_pretrained('maidalun1020/bce-embedding-base_v1')

device = 'cuda'  # if no GPU, set "cpu"
model.to(device)

# get inputs
inputs = tokenizer(sentences, padding=True, truncation=True, max_length=512, return_tensors="pt")
inputs_on_device = {k: v.to(device) for k, v in inputs.items()}

# get embeddings
outputs = model(**inputs_on_device, return_dict=True)
embeddings = outputs.last_hidden_state[:, 0]  # cls pooler
embeddings = embeddings / embeddings.norm(dim=1, keepdim=True)  # normalize
```

--------------------------------

### Score a Single Query-Passage Pair with BCEmbedding

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Use `compute_score` for a quick relevance check between one query and one passage. Ensure the model is initialized before calling this method.

```python
query = "What are the benefits of exercise?"
passage = "Regular physical activity improves cardiovascular health and mental well-being."
sentence_pair = [query, passage]

score = model.compute_score(sentence_pair)
# Output: 0.92 (high relevance)
```

--------------------------------

### Calculate Relevance Scores with RerankerModel.compute_score

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Compute relevance scores for query-passage pairs using the reranker model.

```python
from BCEmbedding import RerankerModel

model = RerankerModel(model_name_or_path="maidalun1020/bce-reranker-base_v1")
```

--------------------------------

### Generate Text Embeddings with EmbeddingModel.encode

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Convert text into dense vector representations and compute cosine similarity between queries and documents.

```python
from BCEmbedding import EmbeddingModel

model = EmbeddingModel(model_name_or_path="maidalun1020/bce-embedding-base_v1")

# Single sentence embedding
sentences = ["What is machine learning?"]
embeddings = model.encode(sentences)
# Output: numpy array of shape (1, 768)

# Batch embedding with configuration
sentences = [
    "The quick brown fox jumps over the lazy dog.",
    "机器学习是人工智能的一个重要分支。",
    "Natural language processing enables computers to understand text.",
    "深度学习模型可以自动提取特征。"
]

embeddings = model.encode(
    sentences,
    batch_size=256,           # Batch size for inference
    max_length=512,           # Max token length
    normalize_to_unit=True,   # L2 normalize embeddings
    return_numpy=True,        # Return numpy array (False for torch.Tensor)
    enable_tqdm=True          # Show progress bar
)
# Output: numpy array of shape (4, 768)

# Calculate similarity between query and documents
import numpy as np

query = "What is artificial intelligence?"
documents = [
    "AI is the simulation of human intelligence by machines.",
    "The weather today is sunny and warm.",
    "人工智能正在改变各个行业。"  # Chinese: AI is changing various industries
]

query_embedding = model.encode([query])
doc_embeddings = model.encode(documents)

# Cosine similarity (embeddings are already normalized)
similarities = np.dot(doc_embeddings, query_embedding.T).flatten()
# Output: [0.89, 0.23, 0.78] - AI-related docs score higher
```

--------------------------------

### Rerank Passages by Relevance using RerankerModel

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

The `rerank` method sorts a list of passages by their relevance to a given query. It handles long passages by chunking and merging scores, making it suitable for RAG.

```python
from BCEmbedding import RerankerModel

model = RerankerModel(model_name_or_path="maidalun1020/bce-reranker-base_v1")

# Rerank search results
query = "How does photosynthesis work?"

passages = [
    "Photosynthesis is the process by which plants convert sunlight into energy.",
    "The water cycle involves evaporation, condensation, and precipitation.",
    "Plants use chlorophyll to capture light energy and convert CO2 and water into glucose.",
    "Climate change is affecting ecosystems worldwide.",
    "光合作用是植物将太阳能转化为化学能的过程。"  # Chinese passage about photosynthesis
]

result = model.rerank(
    query=query,
    passages=passages,
    batch_size=256
)

# Result structure
print(result['rerank_passages'])  # Passages sorted by relevance
# [
#   "Plants use chlorophyll to capture light energy...",
#   "Photosynthesis is the process by which plants convert...",
#   "光合作用是植物将太阳能转化为化学能的过程。",
#   "The water cycle involves evaporation...",
#   "Climate change is affecting ecosystems worldwide."
# ]

print(result['rerank_scores'])    # Corresponding relevance scores
# [0.94, 0.91, 0.85, 0.18, 0.11]

print(result['rerank_ids'])       # Original indices in sorted order
# [2, 0, 4, 1, 3]

# Filter by relevance threshold
threshold = 0.5
relevant_passages = [
    (passage, score)
    for passage, score in zip(result['rerank_passages'], result['rerank_scores'])
    if score >= threshold
]
# Returns only passages with score >= 0.5
```

--------------------------------

### Rerank Sentence Pairs with Sentence Transformers CrossEncoder

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Employ the CrossEncoder from Sentence Transformers for reranking sentence pairs. Set the max_length parameter appropriately for the model.

```python
from sentence_transformers import SentenceTransformer, CrossEncoder

reranker = CrossEncoder("maidalun1020/bce-reranker-base_v1", max_length=512)

sentence_pairs = [
    ["What is NLP?", "Natural language processing enables computers to understand human language."],
    ["What is NLP?", "Basketball is a popular sport worldwide."],
]

scores = reranker.predict(sentence_pairs)
print(scores)  # [0.93, 0.05]
```

--------------------------------

### EmbeddingModel.encode - Generate Text Embeddings

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Encodes a list of sentences into dense vector representations. Supports batch processing, configurable max length, L2 normalization, and different return types.

```APIDOC
## EmbeddingModel.encode - Generate Text Embeddings

### Description
The `encode` method converts a list of sentences into dense vector representations. It supports batch processing, configurable max length, optional L2 normalization, and can return either numpy arrays or PyTorch tensors.

### Method
`model.encode(sentences, batch_size=32, max_length=512, normalize_to_unit=False, return_numpy=True, enable_tqdm=False)`

### Parameters
- **sentences** (list[str]) - Required - A list of sentences to encode.
- **batch_size** (int) - Optional - Batch size for inference. Defaults to 32.
- **max_length** (int) - Optional - Maximum token length for each sentence. Defaults to 512.
- **normalize_to_unit** (bool) - Optional - Whether to L2 normalize the embeddings. Defaults to False.
- **return_numpy** (bool) - Optional - Whether to return numpy arrays (True) or torch.Tensor (False). Defaults to True.
- **enable_tqdm** (bool) - Optional - Whether to show a progress bar. Defaults to False.

### Request Example
```python
from BCEmbedding import EmbeddingModel

model = EmbeddingModel(model_name_or_path="maidalun1020/bce-embedding-base_v1")

# Single sentence embedding
sentences = ["What is machine learning?"]
embeddings = model.encode(sentences)
# Output: numpy array of shape (1, 768)

# Batch embedding with configuration
sentences = [
    "The quick brown fox jumps over the lazy dog.",
    "机器学习是人工智能的一个重要分支。",
    "Natural language processing enables computers to understand text.",
    "深度学习模型可以自动提取特征。"
]

embeddings = model.encode(
    sentences,
    batch_size=256,           # Batch size for inference
    max_length=512,           # Max token length
    normalize_to_unit=True,   # L2 normalize embeddings
    return_numpy=True,        # Return numpy array (False for torch.Tensor)
    enable_tqdm=True          # Show progress bar
)
# Output: numpy array of shape (4, 768)

# Calculate similarity between query and documents
import numpy as np

query = "What is artificial intelligence?"
documents = [
    "AI is the simulation of human intelligence by machines.",
    "The weather today is sunny and warm.",
    "人工智能正在改变各个行业。"  # Chinese: AI is changing various industries
]

query_embedding = model.encode([query])
doc_embeddings = model.encode(documents)

# Cosine similarity (embeddings are already normalized)
similarities = np.dot(doc_embeddings, query_embedding.T).flatten()
# Output: [0.89, 0.23, 0.78] - AI-related docs score higher
```

### Response
#### Success Response (200)
- **embeddings** (numpy.ndarray or torch.Tensor) - A list of dense vector representations for the input sentences.
```

--------------------------------

### RerankerModel.compute_score - Calculate Relevance Scores

Source: https://context7.com/netease-youdao/bcembedding/llms.txt

Computes relevance scores for query-passage pairs using the reranker model. Returns sigmoid-normalized scores between 0 and 1.

```APIDOC
## RerankerModel.compute_score - Calculate Relevance Scores

### Description
The `compute_score` method calculates relevance scores for query-passage pairs. It returns sigmoid-normalized scores between 0 and 1, where higher scores indicate stronger semantic relevance.

### Method
`model.compute_score(query, passages)`

### Parameters
- **query** (str) - Required - The query string.
- **passages** (list[str]) - Required - A list of passage strings to score against the query.

### Request Example
```python
from BCEmbedding import RerankerModel

model = RerankerModel(model_name_or_path="maidalun1020/bce-reranker-base_v1")

# Example usage would follow here, but is not provided in the source text.
# For instance:
# query = "What is the capital of France?"
# passages = ["Paris is the capital of France.", "The Eiffel Tower is in Paris.", "France is a country in Europe."]
# scores = model.compute_score(query, passages)
# print(scores) # Expected output: [0.9, 0.7, 0.6] (example scores)
```

### Response
#### Success Response (200)
- **scores** (list[float]) - A list of relevance scores, each between 0 and 1, corresponding to the input passages.
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.