Sentence Transformer

https://www.sbert.net/docs/sentence_transformer/
Admin
Sentence Transformer is a Python framework for computing dense vector representations for sentences,...
Tokens:86,927
Snippets:1,116
Trust Score:9.4
Update:8 months ago
Show doc for...
Tokens:
Raw
### Install MTEB Benchmark

Source: https://www.sbert.net/docs/sentence_transformer/usage/mteb_evaluation

Installs the MTEB library and its required dependencies. This is a prerequisite for using MTEB for evaluating SentenceTransformer models.

```bash
pip install mteb>=2.0.0
```

--------------------------------

### Image Search Installation (Python)

Source: https://www.sbert.net/docs/sentence_transformer/loss_overview

Provides instructions on how to install the necessary libraries for performing image search using Sentence-Transformers. This typically involves installing additional dependencies like CLIP.

```bash
# Install sentence-transformers if you haven't already
pip install -U sentence-transformers

# Install CLIP and other dependencies for image search
pip install ftfy regex tqdm Pillow 
# For CLIP model:
pip install git+https://github.com/openai/CLIP.git
```

--------------------------------

### Load and Inspect Natural Questions Dataset (Python)

Source: https://www.sbert.net/docs/sentence_transformer/dataset_overview

Loads the 'sentence-transformers/natural-questions' dataset from the Hugging Face Hub and prints the first training example. This example requires the 'datasets' library to be installed.

```python
from datasets import load_dataset

# Indicate the dataset id from the Hub
dataset_id = "sentence-transformers/natural-questions"
dataset = load_dataset(dataset_id, split="train")

print(dataset[0])
```

--------------------------------

### Install Sentence Transformers from Source

Source: https://www.sbert.net/docs/sentence_transformer/loss_overview

Installs the Sentence-Transformers library directly from its source code. This is useful for development or when needing the latest unreleased features.

```bash
git clone https://github.com/UKPLab/sentence-transformers.git
cd sentence-transformers
pip install .
```

--------------------------------

### Quickstart: Initialize and Use Sentence Transformer Model (Python)

Source: https://www.sbert.net/docs/sentence_transformer/loss_overview

Demonstrates the basic usage of Sentence-Transformers to initialize a model and compute embeddings for sentences. This is the fundamental step for most downstream tasks.

```python
from sentence_transformers import SentenceTransformer

# Load a pre-trained model
model = SentenceTransformer('all-MiniLM-L6-v2')

# Sentences to embed
sentences = ['This is an example sentence', 'Each sentence is converted']

# Compute embeddings
embeddings = model.encode(sentences)

# Print the embeddings
print(embeddings)
```

--------------------------------

### Install Sentence Transformers with Pip

Source: https://www.sbert.net/docs/sentence_transformer/loss_overview

Installs the Sentence-Transformers library using pip. This is the most common installation method for Python projects.

```bash
pip install -U sentence-transformers
```

--------------------------------

### Install Sentence Transformers with OpenVINO

Source: https://www.sbert.net/docs/sentence_transformer/usage/efficiency

Installs the Sentence Transformers library with the necessary extra dependencies for OpenVINO support. This is a prerequisite for using the OpenVINO backend.

```bash
pip install sentence-transformers[openvino]
```

--------------------------------

### Quickstart: Initialize and Use Sparse Encoder Model (Python)

Source: https://www.sbert.net/docs/sentence_transformer/loss_overview

Demonstrates the initialization and usage of a Sparse Encoder model. Sparse encoders are useful for tasks like keyword matching and information retrieval where term frequency is important.

```python
from sentence_transformers import SentenceTransformer

# Load a pre-trained sparse encoder model
# Example: 'prithivida/splade_cocondenser_all-MiniLM-L6-v2'
model = SentenceTransformer('prithivida/splade_cocondenser_all-MiniLM-L6-v2')

# Sentences to encode
sentences = ['This is an example sentence', 'Each sentence is converted']

# Compute sparse embeddings (term importance scores)
sparse_embeddings = model.encode(sentences)

# The output is a dictionary containing token importance scores
print(sparse_embeddings)
```

--------------------------------

### Load Dataset from Hugging Face Hub using Datasets library

Source: https://www.sbert.net/docs/sentence_transformer/dataset_overview

This code snippet demonstrates how to load a dataset from the Hugging Face Hub using the `datasets` library. Ensure you have the library installed (`pip install datasets`). The `load_dataset()` function is the primary tool for this operation.

```python
from datasets import load_dataset
```

--------------------------------

### Quickstart: Initialize and Use Cross Encoder Model (Python)

Source: https://www.sbert.net/docs/sentence_transformer/loss_overview

Shows how to initialize and use a Cross-Encoder model for tasks like semantic similarity scoring. Cross-encoders process pairs of sentences and are generally more accurate but slower than bi-encoders.

```python
from sentence_transformers import CrossEncoder

# Load a pre-trained cross-encoder model
model = CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')

# Pairs of sentences to score
# The first sentence is the query, the second is a potential match
# The model outputs a score between 0 and 1, where 1 means high similarity
scores = model.predict([('What is the capital of France?', 'The capital of France is Paris.'),
                        ('What is the capital of France?', 'The weather is nice today.')])

print(scores)
```

--------------------------------

### Install PyTorch with CUDA Support

Source: https://www.sbert.net/docs/sentence_transformer/loss_overview

Installs PyTorch, a deep learning framework, with CUDA support. This is a prerequisite for Sentence-Transformers to leverage GPU acceleration for faster computations.

```bash
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
```

--------------------------------

### config_sentence_transformers.json Example

Source: https://www.sbert.net/docs/sentence_transformer/usage/custom_models

Presents an example of a `config_sentence_transformers.json` file, containing metadata about the Sentence Transformer model. This includes version information for the used libraries and settings for prompts and similarity functions.

```json
{
  "__version__": {
    "sentence_transformers": "3.0.1",
    "transformers": "4.43.4",
    "pytorch": "2.5.0"
  },
  "prompts": {},
  "default_prompt_name": null,
  "similarity_fn_name": null
}
```

--------------------------------

### Initialize Sentence Transformer Model

Source: https://www.sbert.net/docs/sentence_transformer/training_overview

Loads a pre-trained Sentence Transformer model. This is useful for direct usage or as a starting point for finetuning. It relies on the `sentence_transformers` library.

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
```

--------------------------------

### Initialize SentenceTransformer with Dot Product Similarity

Source: https://www.sbert.net/docs/sentence_transformer/usage/semantic_textual_similarity

Initializes a SentenceTransformer model using a pretrained model and explicitly sets the similarity function to DOT_PRODUCT during initialization. This is an efficient way to set the similarity metric from the start.

```python
from sentence_transformers import SentenceTransformer, SimilarityFunction

model = SentenceTransformer("all-MiniLM-L6-v2", similarity_fn_name=SimilarityFunction.DOT_PRODUCT)
```

--------------------------------

### Using Prompt Templates for Encoding (Python)

Source: https://www.sbert.net/docs/sentence_transformer/loss_overview

Illustrates how to use prompt templates with Sentence Transformers. Prompts can guide the model to generate embeddings tailored for specific tasks or contexts, improving performance.

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer('all-MiniLM-L6-v2')

sentences = ['A man is eating food.', 'A man is eating a piece of bread.']

# Define a prompt template
# The prompt guides the model's understanding of the task
prompt_template = "This text is about: {sentence}"

# Apply the prompt template before encoding
# Note: Not all models are trained to benefit from prompts. Check model documentation.
# This example assumes a model that might respond to such prompts.
# For models explicitly trained with prompts, the usage might be more direct.

# A common way is to format the sentences first if the model expects it:
formatted_sentences = [prompt_template.format(sentence=s) for s in sentences]
embeddings = model.encode(formatted_sentences)

print('Embeddings computed using prompt template.')
```

--------------------------------

### Manual Implementation of Semantic Search (Python)

Source: https://www.sbert.net/docs/sentence_transformer/loss_overview

Provides a basic example of implementing semantic search using Sentence Transformers. It involves encoding a corpus and a query, then finding the most similar documents using cosine similarity.

```python
from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity

model = SentenceTransformer('all-MiniLM-L6-v2')

corpus = [
    "A man is eating food.",
    "A man is eating a piece of bread.",
    "The weather is nice today.",
    "The soccer game with the expensive shoes will be stopped.",
    "A man is eating pasta."
]

corpus_embeddings = model.encode(corpus)

query = "A man is eating something."
query_embedding = model.encode([query])

# Calculate cosine similarities between query and all corpus documents
cosine_scores = cosine_similarity(query_embedding, corpus_embeddings)[0]

# Get the top 3 most similar documents
results = sorted(range(len(cosine_scores)), key=lambda k: cosine_scores[k], reverse=True)[:3]

print(f"Query: {query}")
print("Top 3 results:")
for idx in results:
    print(f"- {corpus[idx]} (Score: {cosine_scores[idx]:.4f})")
```

--------------------------------

### Transformer Module Configuration Example

Source: https://www.sbert.net/docs/sentence_transformer/usage/custom_models

Illustrates a sample `sentence_bert_config.json` file, which stores default keyword arguments for the Transformer module. This configuration is used when initializing the module during model loading.

```json
{
  "max_seq_length": 4096,
  "do_lower_case": false
}
```

--------------------------------

### Image Search Usage Example (Python)

Source: https://www.sbert.net/docs/sentence_transformer/loss_overview

Demonstrates how to use Sentence Transformers with multimodal models (like CLIP) to perform image search. This involves encoding both images and text queries into a shared embedding space.

```python
from sentence_transformers import SentenceTransformer
from PIL import Image
import requests
import numpy as np

# Load a CLIP model (or another multimodal model)
model = SentenceTransformer('clip-ViT-B-32')

# Example: Encode an image
# You can load images from local files or URLs
img_url = 'https://upload.wikimedia.org/wikipedia/commons/thumb/9/92/Cat_in_a_box.jpg/1024px-Cat_in_a_box.jpg'
image = Image.open(requests.get(img_url, stream=True).raw)
image_embedding = model.encode([image])

# Example: Encode text queries
text_queries = [
    "A photo of a cat",
    "A dog playing fetch",
    "A landscape painting"
]
txt_embeddings = model.encode(text_queries)

# Calculate similarity between image and text embeddings
from sklearn.metrics.pairwise import cosine_similarity

similarities = cosine_similarity(image_embedding, txt_embeddings)

print("Similarity scores between image and text queries:")
for i, query in enumerate(text_queries):
    print(f"- {query}: {similarities[0][i]:.4f}")

# The query with the highest similarity is the best match.
```

--------------------------------

### Install Sentence Transformers with Conda

Source: https://www.sbert.net/docs/sentence_transformer/loss_overview

Installs the Sentence-Transformers library using Conda. This is useful for managing environments and dependencies, especially when working with scientific packages.

```bash
conda install -c conda-forge sentence-transformers
```

--------------------------------

### Run Training Script with DDP using torchrun

Source: https://www.sbert.net/docs/sentence_transformer/training/distributed

This command initiates distributed training using the torchrun launcher. It specifies the number of processes per node (GPUs) and the training script to execute. Ensure your script is structured with a `main` function for proper execution across processes.

```bash
torchrun --nproc_per_node=4 train_script.py
```

--------------------------------

### Run Training Script with DDP using accelerate

Source: https://www.sbert.net/docs/sentence_transformer/training/distributed

This command launches distributed training using the accelerate library. It specifies the total number of processes to use for training and the training script. Similar to torchrun, your script should be wrapped in a `main` function.

```bash
accelerate launch --num_processes 4 train_script.py
```

--------------------------------

### Initialize SentenceTransformerTrainingArguments

Source: https://www.sbert.net/docs/sentence_transformer/training_overview

Shows how to initialize `SentenceTransformerTrainingArguments` to configure training parameters such as output directory, epochs, batch size, learning rate, and mixed precision. It also covers tracking and debugging parameters like evaluation strategy and saving checkpoints.

```python
from sentence_transformers.training_args import SentenceTransformerTrainingArguments
from sentence_transformers.training_args import BatchSamplers

args = SentenceTransformerTrainingArguments(
    # Required parameter:
    output_dir="models/mpnet-base-all-nli-triplet",
    # Optional training parameters:
    num_train_epochs=1,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    learning_rate=2e-5,
    warmup_ratio=0.1,
    fp16=True,  # Set to False if you get an error that your GPU can't run on FP16
    bf16=False,  # Set to True if you have a GPU that supports BF16
    batch_sampler=BatchSamplers.NO_DUPLICATES,  # losses that use "in-batch negatives" benefit from no duplicates
    # Optional tracking/debugging parameters:
    eval_strategy="steps",
    eval_steps=100,
    save_strategy="steps",
    save_steps=100,
    save_total_limit=2,
    logging_steps=100,
    run_name="mpnet-base-all-nli-triplet",  # Will be used in W&B if `wandb` is installed
)


```

--------------------------------

### Loading a Pretrained Adapter (Python)

Source: https://www.sbert.net/docs/sentence_transformer/loss_overview

Demonstrates how to load a pre-trained adapter for sentence transformer models. This is a common first step before fine-tuning or using a model for inference.

```python
from sentence_transformers import SentenceTransformer

# Load a pre-trained model
model = SentenceTransformer('all-MiniLM-L6-v2')
```

--------------------------------

### Editable Install of Sentence Transformers

Source: https://www.sbert.net/docs/sentence_transformer/loss_overview

Installs the Sentence-Transformers library in an editable mode. Changes made to the source code will be immediately reflected without needing to reinstall.

```bash
git clone https://github.com/UKPLab/sentence-transformers.git
cd sentence-transformers
pip install -e .
```

--------------------------------

### Multi-Dataset Training with Sentence Transformers

Source: https://www.sbert.net/docs/sentence_transformer/training_overview

Demonstrates loading multiple datasets and defining a dictionary of losses for multi-dataset training. This example utilizes various datasets from the Hugging Face datasets library and prepares them for training with `SentenceTransformerTrainer`. It shows how to combine datasets and losses into dictionaries for flexible training configurations.

```python
from datasets import load_dataset
from sentence_transformers import SentenceTransformer, SentenceTransformerTrainer
from sentence_transformers.losses import CoSENTLoss, MultipleNegativesRankingLoss, SoftmaxLoss

# 1. Load a model to finetune
model = SentenceTransformer("bert-base-uncased")

# 2. Load several Datasets to train with
# (anchor, positive)
all_nli_pair_train = load_dataset("sentence-transformers/all-nli", "pair", split="train[:10000]")
# (premise, hypothesis) + label
all_nli_pair_class_train = load_dataset("sentence-transformers/all-nli", "pair-class", split="train[:10000]")
# (sentence1, sentence2) + score
all_nli_pair_score_train = load_dataset("sentence-transformers/all-nli", "pair-score", split="train[:10000]")
# (anchor, positive, negative)
all_nli_triplet_train = load_dataset("sentence-transformers/all-nli", "triplet", split="train[:10000]")
# (sentence1, sentence2) + score
stsb_pair_score_train = load_dataset("sentence-transformers/stsb", split="train[:10000]")
# (anchor, positive)
quora_pair_train = load_dataset("sentence-transformers/quora-duplicates", "pair", split="train[:10000]")
# (query, answer)
natural_questions_train = load_dataset("sentence-transformers/natural-questions", split="train[:10000]")

# We can combine all datasets into a dictionary with dataset names to datasets
train_dataset = {
    "all-nli-pair": all_nli_pair_train,
    "all-nli-pair-class": all_nli_pair_class_train,
    "all-nli-pair-score": all_nli_pair_score_train,
    "all-nli-triplet": all_nli_triplet_train,
    "stsb": stsb_pair_score_train,
    "quora": quora_pair_train,
    "natural-questions": natural_questions_train,
}

# 3. Load several Datasets to evaluate with
# (anchor, positive, negative)
all_nli_triplet_dev = load_dataset("sentence-transformers/all-nli", "triplet", split="dev")
# (sentence1, sentence2, score)
stsb_pair_score_dev = load_dataset("sentence-transformers/stsb", split="validation")
# (anchor, positive)
quora_pair_dev = load_dataset("sentence-transformers/quora-duplicates", "pair", split="train[10000:11000]")
# (query, answer)
natural_questions_dev = load_dataset("sentence-transformers/natural-questions", split="train[10000:11000]")

# We can use a dictionary for the evaluation dataset too, but we don't have to. We could also just use
# no evaluation dataset, or one dataset.
eval_dataset = {
    "all-nli-triplet": all_nli_triplet_dev,
    "stsb": stsb_pair_score_dev,
    "quora": quora_pair_dev,
    "natural-questions": natural_questions_dev,
}

# 4. Load several loss functions to train with
# Example: Define losses for specific datasets if needed
# losses = {
#     "all-nli-pair": CoSENTLoss(model=model),
#     "stsb": SoftmaxLoss(model=model, sentence_features=[model]) # Example, adjust as needed
# }

# 5. Initialize SentenceTransformerTrainer (details omitted for brevity, requires TrainingArguments)
# trainer = SentenceTransformerTrainer(
#     model=model,
#     train_dataset=train_dataset,
#     eval_dataset=eval_dataset,
#     # Add other necessary arguments like train_loss, compute_metrics, etc.
#     # If using multiple losses, you might pass a dictionary or a custom loss function
# )

# trainer.train()

```

--------------------------------

### TSDAE Training Example - Sentence Transformers

Source: https://www.sbert.net/docs/sentence_transformer/dataset_overview

An example of training a SentenceTransformer model using the TSDAE (Transformer-based Denoising Autoencoder) objective. This method is effective for unsupervised learning of sentence embeddings.

```python
from sentence_transformers import SentenceTransformer, InputExample, losses
from torch.utils.data import DataLoader

# Define your dataset
# sentences = [...] 
# train_examples = [InputExample(texts=[sentence, sentence], label=1.0) for sentence in sentences]

# Load a pre-trained model
model = SentenceTransformer('distilroberta-base')

# Define the TSDAE loss function
train_loss = losses.TSDAELoss(model=model, sentence_chunk_size=128, vocab_size=30000, ...) # Add other parameters as needed

# Create a DataLoader
train_dataloader = DataLoader(train_examples, shuffle=True, batch_size=16)

# Train the model
model.fit(train_objectives=[(train_dataloader, train_loss)], epochs=1, warmup_steps=100)
```

--------------------------------

### Install ONNX Runtime for Sentence Transformers

Source: https://www.sbert.net/docs/sentence_transformer/usage/efficiency

Provides the pip commands to install Sentence Transformers with the necessary extras for ONNX Runtime acceleration. Use `onnx-gpu` for GPU support or `onnx` for CPU support.

```bash
pip install sentence-transformers[onnx-gpu]
# or
pip install sentence-transformers[onnx]
```

--------------------------------

### Initialize CoSENTLoss with SentenceTransformer Model

Source: https://www.sbert.net/docs/sentence_transformer/training_overview

Demonstrates how to initialize the `CoSENTLoss` function using a pre-trained `SentenceTransformer` model. This loss function requires pairs of sentences and a float similarity score as labels. It also shows loading a compatible dataset for training.

```python
from datasets import load_dataset
from sentence_transformers import SentenceTransformer
from sentence_transformers.losses import CoSENTLoss

# Load a model to train/finetune
model = SentenceTransformer("xlm-roberta-base")

# Initialize the CoSENTLoss
# This loss requires pairs of text and a float similarity score as a label
loss = CoSENTLoss(model)

# Load an example training dataset that works with our loss function:
train_dataset = load_dataset("sentence-transformers/all-nli", "pair-score", split="train")
"""
Dataset({
    features: ['sentence1', 'sentence2', 'label'],
    num_rows: 942069
})
"""

```

--------------------------------

### Training a Sentence Transformer Model (Python)

Source: https://www.sbert.net/docs/sentence_transformer/usage/efficiency

Provides a basic example of how to train (fine-tune) a Sentence Transformer model using a dataset of sentence pairs. This involves defining training examples, loss function, and training arguments.

```python
from sentence_transformers import SentenceTransformer, InputExample, losses
from torch.utils.data import DataLoader

# 1. Define Training Data
train_examples = [
    InputExample(texts=['Sentence A1', 'Sentence B1'], label=1.0),
    InputExample(texts=['Sentence A2', 'Sentence B2'], label=0.0),
    InputExample(texts=['Sentence A3', 'Sentence B3'], label=1.0),
    InputExample(texts=['Sentence A4', 'Sentence B4'], label=0.0),
]

# Create a DataLoader
train_dataloader = DataLoader(train_examples, shuffle=True, batch_size=16)

# 2. Define Loss Function
# MultipleNegativesRankingLoss is suitable for training with sentence pairs where positive pairs have label 1 and negative pairs have label 0.
train_loss = losses.MultipleNegativesRankingLoss(model=None) # Model will be set during fit

# 3. Load a Pre-trained Model
model = SentenceTransformer('all-MiniLM-L6-v2')

# 4. Define Training Arguments (optional, defaults are often good)
num_epochs = 1
warmup_steps = int(len(train_dataloader) * num_epochs * 0.1) # 10% of training steps

# 5. Train the Model
model.fit(train_object=(train_dataloader, train_loss),
          epochs=num_epochs,
          warmup_steps=warmup_steps,
          output_path='training_output')

print("Model training complete. Model saved to 'training_output'.")
```

--------------------------------

### BatchAllTripletLoss Example

Source: https://www.sbert.net/docs/sentence_transformer/usage/custom_models

Demonstrates the usage of BatchAllTripletLoss for training sentence embeddings. This loss function considers all possible triplets within a batch.

```python
from sentence_transformers import SentenceTransformer, losses
from torch.utils.data import DataLoader

# Load a pre-trained model
model = SentenceTransformer('all-MiniLM-L6-v2')

# Prepare training data (list of sentences and labels)
train_data = [
    (['Anchor 1', 'Positive 1'], 1.0), # Anchor and positive pair
    (['Anchor 2', 'Negative 1'], 0.0)  # Anchor and negative pair
]

# Convert to InputExample format if needed, or use a DataLoader that provides triplets
# For simplicity, let's assume a DataLoader yields (anchor, positive, negative) or similar structure
# Example structure: DataLoader yields (sentences, labels)
# where sentences might be ['anchor', 'positive', 'negative']

# Let's create a dummy DataLoader for demonstration
class TripletDataLoader:
    def __init__(self, data):
        self.data = data
        self.batch_size = 16

    def __iter__(self):
        # In a real scenario, this would yield batches of triplets
        # For BatchAllTripletLoss, we need anchor, positive, and negative samples
        # Let's simulate a batch with anchor, positive, negative sentences
        batch_anchors = [d[0][0] for d in self.data[:self.batch_size]]
        batch_positives = [d[0][1] for d in self.data[:self.batch_size]]
        # Assuming negatives are also provided or derived
        batch_negatives = [d[1][0] for d in self.data[:self.batch_size]] # Placeholder for actual negatives
        yield (batch_anchors, batch_positives, batch_negatives)

    def __len__(self):
        return len(self.data) // self.batch_size

# Dummy data for demonstration
triplet_data = [
    (['Sentence A', 'Sentence B'], ['Sentence C']),
    (['Sentence D', 'Sentence E'], ['Sentence F'])
]

data_loader = TripletDataLoader(triplet_data)

# Define the BatchAllTripletLoss
train_loss = losses.BatchAllTripletLoss(model=model, margin=0.5)

# Train the model
model.fit(train_objectives=[(data_loader, train_loss)], epochs=1, warmup_steps=100)
```

--------------------------------

### Initialize Sentence Transformer with Transformer Shortcut

Source: https://www.sbert.net/docs/sentence_transformer/training_overview

A simplified way to initialize a Sentence Transformer model using a Transformer checkpoint directly. This leverages the default pooling mechanism. Requires the `sentence_transformers` library.

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("google-bert/bert-base-uncased")
```

--------------------------------

### Cross-Encoder Usage Example

Source: https://www.sbert.net/docs/sentence_transformer/usage/custom_models

Demonstrates the basic usage of a Cross-Encoder model for scoring sentence pairs. Cross-encoders are typically used for re-ranking.

```python
from sentence_transformers import CrossEncoder

# Load a pre-trained Cross-Encoder model
model = CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')

# Define sentence pairs
sentences1 = ["What is the capital of France?", "Who is the president of the USA?"]
sentences2 = ["Paris is the capital of France.", "Joe Biden is the president of the USA."]

# Predict scores
scores = model.predict(zip(sentences1, sentences2))

print(scores)
```
Tokens:
Raw