### Install BLEURT from Source

Source: https://context7.com/google-research/bleurt/llms.txt

Install BLEURT and verify the installation using unit tests. Ensure pip is up-to-date before cloning the repository.

```bash
pip install --upgrade pip
git clone https://github.com/google-research/bleurt.git
cd bleurt
pip install .

# Verify installation with unit tests
python -m unittest bleurt.score_test
python -m unittest bleurt.score_not_eager_test
python -m unittest bleurt.finetune_test
python -m unittest bleurt.score_files_test
```

--------------------------------

### Install BLEURT and Dependencies

Source: https://github.com/google-research/bleurt/blob/master/README.md

Installs BLEURT and its dependencies using pip and git clone. Ensure pip is up-to-date before installation.

```bash
pip install --upgrade pip  # ensures that pip is current
git clone https://github.com/google-research/bleurt.git
cd bleurt
pip install .
```

--------------------------------

### Download and Use BLEURT Checkpoint via Command Line

Source: https://github.com/google-research/bleurt/blob/master/README.md

This command-line example shows how to download the BLEURT-20 checkpoint, unzip it, and then use it to score candidate and reference files. It specifies the candidate file, reference file, and the path to the BLEURT checkpoint.

```bash
wget https://storage.googleapis.com/bleurt-oss-21/BLEURT-20.zip .
unzip BLEURT-20.zip
python -m bleurt.score_files \
  -candidate_file=bleurt/test_data/candidates \
  -reference_file=bleurt/test_data/references \
  -bleurt_checkpoint=BLEURT-20
```

--------------------------------

### Verify BLEURT Installation

Source: https://github.com/google-research/bleurt/blob/master/README.md

Runs unit tests to verify the BLEURT installation. These tests cover scoring, non-eager scoring, fine-tuning, and file scoring functionalities.

```bash
python -m unittest bleurt.score_test
python -m unittest bleurt.score_not_eager_test
python -m unittest bleurt.finetune_test
python -m unittest bleurt.score_files_test
```

--------------------------------

### Enable Length-Based Batching with Python API

Source: https://github.com/google-research/bleurt/blob/master/README.md

This demonstrates how to enable length-based batching when using the Python API for BLEURT scoring. This optimization groups examples of similar lengths to improve computational efficiency.

```python
from bleurt.score import LengthBatchingBleurtScorer

# Instantiate LengthBatchingBleurtScorer instead of BleurtScorer
```

--------------------------------

### Integrate BLEURT in TensorFlow Computation Graph

Source: https://github.com/google-research/bleurt/blob/master/README.md

Shows an example of how BLEURT can be embedded within a TensorFlow computation graph, useful for visualization with Tensorboard during model training.

```python
import tensorflow as tf
```

--------------------------------

### List Fine-tuning Script Parameters

Source: https://github.com/google-research/bleurt/blob/master/checkpoints.md

Execute this command to view all available parameters for the fine-tuning script, including batch size, learning rate, and sequence length.

```bash
python finetune.py -helpfull
```

--------------------------------

### Visualize Training Progress with Tensorboard

Source: https://github.com/google-research/bleurt/blob/master/checkpoints.md

Launch Tensorboard to visualize the training progress of your fine-tuned BLEURT model. The log directory should match the model directory used during fine-tuning.

```bash
tensorboard --logdir my_new_bleurt_checkpoint
```

--------------------------------

### List BLEURT Command-Line Options

Source: https://github.com/google-research/bleurt/blob/master/README.md

Displays a short help message listing all available command-line options for the `bleurt.score_files` module.

```bash
python -m bleurt.score_files -helpshort
```

--------------------------------

### Fine-Tuning: Adapt BLEURT with custom ratings data (Bash)

Source: https://context7.com/google-research/bleurt/llms.txt

Adapt an existing BLEURT checkpoint to custom human ratings data using `bleurt.finetune`. The training data should be in JSONL format with 'reference', 'candidate', and 'score' fields. Monitor training with Tensorboard.

```bash
# ratings_train.jsonl format (one JSON record per line):
# {"reference": "The cat sat.", "candidate": "A cat was sitting.", "score": 0.8}
# {"reference": "Good morning.", "candidate": "Good evening.", "score": 0.3}

python -m bleurt.finetune \
  -init_bleurt_checkpoint=bleurt/test_checkpoint \
  -model_dir=my_finetuned_bleurt \
  -train_set=bleurt/test_data/ratings_train.jsonl \
  -dev_set=bleurt/test_data/ratings_dev.jsonl \
  -num_train_steps=500

# Monitor training with Tensorboard
tensorboard --logdir my_finetuned_bleurt

# The fine-tuned model is exported as a BLEURT checkpoint and can be used like any other:
from bleurt import score
scorer = score.BleurtScorer("my_finetuned_bleurt/export/bleurt_best/<timestamp>")
```

--------------------------------

### Download BLEURT Checkpoint and Run Scoring

Source: https://github.com/google-research/bleurt/blob/master/README.md

Downloads the recommended BLEURT-20 checkpoint and runs BLEURT scoring on candidate and reference files. Input files should contain one sentence per line.

```bash
# Downloads the BLEURT-base checkpoint.
wget https://storage.googleapis.com/bleurt-oss-21/BLEURT-20.zip .
unzip BLEURT-20.zip

# Runs the scoring.
python -m bleurt.score_files \
  -candidate_file=bleurt/test_data/candidates \
  -reference_file=bleurt/test_data/references \
  -bleurt_checkpoint=BLEURT-20
```

--------------------------------

### Download BLEURT Checkpoints

Source: https://context7.com/google-research/bleurt/llms.txt

Download pre-trained BLEURT checkpoints, including the recommended BLEURT-20 and its distilled variants for faster inference.

```bash
# Recommended checkpoint: BLEURT-20 (full model)
wget https://storage.googleapis.com/bleurt-oss-21/BLEURT-20.zip .
unzip BLEURT-20.zip

# Distilled variants (faster, smaller, slightly less accurate):
# BLEURT-20-D12: 167M params, ~3.5x smaller
wget https://storage.googleapis.com/bleurt-oss-21/BLEURT-20-D12.zip .
unzip BLEURT-20-D12.zip

# BLEURT-20-D6: 45M params
wget https://storage.googleapis.com/bleurt-oss-21/BLEURT-20-D6.zip .
unzip BLEURT-20-D6.zip

# BLEURT-20-D3: 30M params (fastest)
wget https://storage.googleapis.com/bleurt-oss-21/BLEURT-20-D3.zip .
unzip BLEURT-20-D3.zip
```

--------------------------------

### Command-Line Interface: Speed Optimizations

Source: https://context7.com/google-research/bleurt/llms.txt

Achieve up to 20x speedup by combining larger batch sizes, length-based batching, and using a distilled checkpoint.

```APIDOC
## Speed Optimizations

### Description
Combine batch size tuning, length-based batching, and a distilled checkpoint for up to 20x speedup.

### Method
```bash
python -m bleurt.score_files
```

### Parameters
#### Path Parameters
- `-candidate_file` (string) - Required - Path to the file containing candidate sentences.
- `-reference_file` (string) - Required - Path to the file containing reference sentences.
- `-bleurt_checkpoint` (string) - Required - Path or name of the BLEURT checkpoint to use (e.g., `BLEURT-20-D12`).
- `-scores_file` (string) - Optional - Path to save the output scores.

#### Query Parameters
- `-bleurt_batch_size` (integer) - Optional - Larger batch size for optimization (default: 16).
- `-batch_same_length` (boolean) - Optional - Enable length-based batching for speedup (default: `False`).

### Request Example
```bash
# Download distilled 12-layer model (~3.5x smaller than BLEURT-20)
wget https://storage.googleapis.com/bleurt-oss-21/BLEURT-20-D12.zip .
unzip BLEURT-20-D12.zip

python -m bleurt.score_files \
  -candidate_file=bleurt/test_data/candidates \
  -reference_file=bleurt/test_data/references \
  -bleurt_batch_size=100 \
  -batch_same_length=True \
  -bleurt_checkpoint=BLEURT-20-D12
```
```

--------------------------------

### Score Files with Distilled Model

Source: https://github.com/google-research/bleurt/blob/master/README.md

Illustrates combining techniques to speed up BLEURT scoring. Downloads a distilled model and uses it to score candidate and reference files with optimized batching.

```bash
# Downloads the 12-layer distilled model, which is ~3.5X smaller.
wget https://storage.googleapis.com/bleurt-oss-21/BLEURT-20-D12.zip .
unzip BLEURT-20-D12.zip

python -m bleurt.score_files \
  -candidate_file=bleurt/test_data/candidates \
  -reference_file=bleurt/test_data/references \
  -bleurt_batch_size=100 \
  -batch_same_length=True \
  -bleurt_checkpoint=BLEURT-20-D12
```

--------------------------------

### Fine-Tuning: From an Existing BLEURT Checkpoint

Source: https://context7.com/google-research/bleurt/llms.txt

Adapt an existing BLEURT checkpoint to custom data using `bleurt.finetune` with a JSONL file containing reference, candidate, and score fields.

```APIDOC
## Fine-Tuning: From an Existing BLEURT Checkpoint

Use `bleurt.finetune` to adapt an existing BLEURT checkpoint to custom human ratings data provided as a JSONL file with `reference`, `candidate`, and `score` fields.

```bash
# ratings_train.jsonl format (one JSON record per line):
# {"reference": "The cat sat.", "candidate": "A cat was sitting.", "score": 0.8}
# {"reference": "Good morning.", "candidate": "Good evening.", "score": 0.3}

python -m bleurt.finetune \
  -init_bleurt_checkpoint=bleurt/test_checkpoint \
  -model_dir=my_finetuned_bleurt \
  -train_set=bleurt/test_data/ratings_train.jsonl \
  -dev_set=bleurt/test_data/ratings_dev.jsonl \
  -num_train_steps=500

# Monitor training with Tensorboard
tensorboard --logdir my_finetuned_bleurt

# The fine-tuned model is exported as a BLEURT checkpoint and can be used like any other:
from bleurt import score
scorer = score.BleurtScorer("my_finetuned_bleurt/export/bleurt_best/<timestamp>")
```
```

--------------------------------

### Use BLEURT Python API for Scoring

Source: https://github.com/google-research/bleurt/blob/master/README.md

Demonstrates how to use the BLEURT Python API to score a list of references against a list of candidates. Requires specifying a checkpoint path.

```python
from bleurt import score

checkpoint = "bleurt/test_checkpoint"
references = ["This is a test."]
candidates = ["This is the test."]

scorer = score.BleurtScorer(checkpoint)
scores = scorer.score(references=references, candidates=candidates)
assert isinstance(scores, list) and len(scores) == 1
print(scores)
```

--------------------------------

### Reproduce WMT Benchmark Results

Source: https://context7.com/google-research/bleurt/llms.txt

Use `bleurt.wmt.benchmark` to train and evaluate BLEURT from scratch on WMT ratings, reproducing ACL paper results. Specify training and testing years, model directory, and checkpoint paths. Results are saved in a JSON file.

```bash
BERT_DIR=bleurt/test_checkpoint
BERT_CKPT=variables/variables

# Reproduce Table 2 from the ACL paper (train on 2015-2016, test on 2017)
python -m bleurt.wmt.benchmark \
  -train_years="2015 2016" \
  -test_years="2017" \
  -dev_ratio=0.1 \
  -model_dir=bleurt_model \
  -results_json=results.json \
  -init_checkpoint=${BERT_DIR}/${BERT_CKPT} \
  -bert_config_file=${BERT_DIR}/bert_config.json \
  -vocab_file=${BERT_DIR}/vocab.txt \
  -do_lower_case=True \
  -num_train_steps=20000
```

--------------------------------

### Fine-tune BLEURT from BERT Checkpoint

Source: https://github.com/google-research/bleurt/blob/master/checkpoints.md

Train a new metric from a fresh BERT checkpoint by specifying the BERT directory, config file, and vocabulary file. This method allows training from scratch.

```bash
BERT_DIR=bleurt/test_checkpoint
BERT_CKPT=variables/variables
python -m bleurt.finetune \
  -init_checkpoint=${BERT_DIR}/${BERT_CKPT} \
  -bert_config_file=${BERT_DIR}/bert_config.json \
  -vocab_file=${BERT_DIR}/vocab.txt \
  -model_dir=my_new_bleurt_checkpoint \
  -train_set=bleurt/test_data/ratings_train.jsonl \
  -dev_set=bleurt/test_data/ratings_dev.jsonl \
  -num_train_steps=500
```

--------------------------------

### Score Files (Command-line)

Source: https://github.com/google-research/bleurt/blob/master/README.md

Use the command-line interface to score sentences from files. Supports both plain text files and JSONL format.

```APIDOC
## Score Files (Command-line)

### Description
Use the command-line interface to score sentences from files. Supports both plain text files and JSONL format.

### Method
`python -m bleurt.score_files`

### Parameters
#### Path Parameters
None

#### Query Parameters
- **-candidate_file** (string) - Required - Path to the file containing candidate sentences, one per line.
- **-reference_file** (string) - Required - Path to the file containing reference sentences, one per line.
- **-sentence_pairs_file** (string) - Required (if not using candidate/reference files) - Path to a JSONL file containing sentence pairs.
- **-bleurt_checkpoint** (string) - Optional - Path to the BLEURT checkpoint to use. Defaults to a light but inaccurate BERT-Tiny checkpoint.
- **-scores_file** (string) - Optional - Path to save the output scores. Defaults to standard output.

### Request Example
```bash
python -m bleurt.score_files \
  -candidate_file=bleurt/test_data/candidates \
  -reference_file=bleurt/test_data/references \
  -bleurt_checkpoint=BLEURT-20
```

```bash
python -m bleurt.score_files \
  -sentence_pairs_file=bleurt/test_data/sentence_pairs.jsonl \
  -bleurt_checkpoint=bleurt/test_checkpoint
```

### Response
#### Success Response (200)
Output is typically printed to standard output or saved to a specified file, with one BLEURT score per sentence pair.

### Other Commands
To list all command-line options:
```bash
python -m bleurt.score_files -helpshort
```
```

--------------------------------

### Encode Reference-Candidate Pairs

Source: https://context7.com/google-research/bleurt/llms.txt

Use `encode_example` for a single pair and `encode_batch` for a list of pairs to tokenize and encode them into BERT-format integer tensors. Requires loaded config and tokenizer.

```python
from bleurt import encoding
from bleurt.lib import tokenizers
from bleurt import checkpoint as checkpoint_lib

# Load config and create tokenizer
config = checkpoint_lib.read_bleurt_config("bleurt/test_checkpoint")
tokenizer = tokenizers.create_tokenizer(
    vocab_file=config["vocab_file"],
    do_lower_case=config["do_lower_case"],
    sp_model=config["sp_model"]
)
max_seq_length = config["max_seq_length"]

# Encode a single pair
input_ids, input_mask, segment_ids = encoding.encode_example(
    reference="The cat sat on the mat.",
    candidate="A cat was sitting on the mat.",
    tokenizer=tokenizer,
    max_seq_length=max_seq_length
)

# Encode a batch of pairs (returns numpy arrays of shape [n, max_seq_length])
import numpy as np
references = ["The cat sat.", "Good morning."]
candidates = ["A cat was sitting.", "Good evening."]
ids, mask, segs = encoding.encode_batch(references, candidates, tokenizer, max_seq_length)
assert ids.shape == (2, max_seq_length)
assert mask.shape == (2, max_seq_length)
assert segs.shape == (2, max_seq_length)
```

--------------------------------

### Reproduce ACL Paper Benchmark Results

Source: https://github.com/google-research/bleurt/blob/master/wmt_experiments.md

This script can be used to re-train BLEURT checkpoints from scratch using WMT data. It downloads ratings, postprocesses them, trains a checkpoint, and computes correlations with human ratings. Ensure BERT_DIR points to your pre-trained BERT checkpoint.

```bash
BERT_DIR=bleurt/test_checkpoint
BERT_CKPT=variables/variables
python -m bleurt.wmt.benchmark \
 -train_years="2015 2016" \
 -test_years="2017" \
 -dev_ratio=0.1 \
 -model_dir=bleurt_model \
 -results_json=results.json \
 -init_checkpoint=${BERT_DIR}/${BERT_CKPT} \
 -bert_config_file=${BERT_DIR}/bert_config.json \
 -vocab_file=${BERT_DIR}/vocab.txt \
 -do_lower_case=True \
 -num_train_steps=20000
```

--------------------------------

### Fine-Tuning: Train BLEURT from raw BERT checkpoint (Bash)

Source: https://context7.com/google-research/bleurt/llms.txt

Train a new BLEURT metric from a raw BERT checkpoint for full initialization control. This requires specifying the BERT directory, config file, and vocabulary file. Use `sentence_piece_model` for RemBERT checkpoints.

```bash
BERT_DIR=bleurt/test_checkpoint
BERT_CKPT=variables/variables

python -m bleurt.finetune \
  -init_checkpoint=${BERT_DIR}/${BERT_CKPT} \
  -bert_config_file=${BERT_DIR}/bert_config.json \
  -vocab_file=${BERT_DIR}/vocab.txt \
  -model_dir=my_new_bleurt_checkpoint \
  -train_set=bleurt/test_data/ratings_train.jsonl \
  -dev_set=bleurt/test_data/ratings_dev.jsonl \
  -num_train_steps=500

# For RemBERT-based checkpoints, use sentence_piece_model instead of vocab_file:
# -sentence_piece_model=path/to/sentencepiece_model

# List all fine-tuning flags:
python -m bleurt.finetune -helpfull
```

--------------------------------

### Fine-Tuning: From a Raw BERT Checkpoint

Source: https://context7.com/google-research/bleurt/llms.txt

Train a new BLEURT metric from a raw BERT checkpoint, offering full control over initialization.

```APIDOC
## Fine-Tuning: From a Raw BERT Checkpoint

Train a new BLEURT metric from a fresh BERT checkpoint (not pre-fine-tuned on ratings), allowing full control over initialization.

```bash
BERT_DIR=bleurt/test_checkpoint
BERT_CKPT=variables/variables

python -m bleurt.finetune \
  -init_checkpoint=${BERT_DIR}/${BERT_CKPT} \
  -bert_config_file=${BERT_DIR}/bert_config.json \
  -vocab_file=${BERT_DIR}/vocab.txt \
  -model_dir=my_new_bleurt_checkpoint \
  -train_set=bleurt/test_data/ratings_train.jsonl \
  -dev_set=bleurt/test_data/ratings_dev.jsonl \
  -num_train_steps=500

# For RemBERT-based checkpoints, use sentence_piece_model instead of vocab_file:
# -sentence_piece_model=path/to/sentencepiece_model

# List all fine-tuning flags:
python -m bleurt.finetune -helpfull
```
```

--------------------------------

### Run BLEURT Scoring with Output File

Source: https://github.com/google-research/bleurt/blob/master/README.md

Uses the command-line interface to score sentence pairs from files and save the results to a specified scores file. Supports both plain text and JSONL formats.

```bash
python -m bleurt.score_files \
  -candidate_file=bleurt/test_data/candidates \
  -reference_file=bleurt/test_data/references \
  -bleurt_checkpoint=bleurt/test_checkpoint \
  -scores_file=scores
```

```bash
python -m bleurt.score_files \
  -sentence_pairs_file=bleurt/test_data/sentence_pairs.jsonl \
  -bleurt_checkpoint=bleurt/test_checkpoint
```

--------------------------------

### Download and Aggregate WMT Human Ratings

Source: https://context7.com/google-research/bleurt/llms.txt

Use `bleurt.wmt.db_builder` to download WMT Metrics Shared Task human ratings and aggregate them into a JSONL file. Specify target language and rating years. Optionally, average duplicates and set a development ratio.

```bash
python -m bleurt.wmt.db_builder \
  -target_language="en" \
  -rating_years="2015 2016" \
  -target_file=wmt.jsonl
```

```bash
python -m bleurt.wmt.db_builder \
  -target_language="*" \
  -rating_years="2017 2018 2019" \
  -average_duplicates=True \
  -dev_ratio=0.1 \
  -target_file=wmt_all.jsonl
```

--------------------------------

### Score Sentence Pairs from Text Files using CLI

Source: https://context7.com/google-research/bleurt/llms.txt

Use the command-line interface to score sentence pairs from separate reference and candidate files, or from a JSONL file. Specify the checkpoint and output file.

```bash
# Score using separate reference and candidate files
python -m bleurt.score_files \
  -candidate_file=bleurt/test_data/candidates \
  -reference_file=bleurt/test_data/references \
  -bleurt_checkpoint=BLEURT-20 \
  -scores_file=scores.txt

# Score using a JSONL file with {"reference": "...", "candidate": "..."} records
python -m bleurt.score_files \
  -sentence_pairs_file=bleurt/test_data/sentence_pairs.jsonl \
  -bleurt_checkpoint=BLEURT-20

# List all available command-line flags
python -m bleurt.score_files -helpshort

# Example output in scores.txt (one score per line):
# 0.8731
# 0.4219
# 0.9103
```

--------------------------------

### Download and Aggregate WMT Ratings

Source: https://github.com/google-research/bleurt/blob/master/wmt_experiments.md

Use this command to download all necessary WMT ratings archives and aggregate them into a single JSONL file. Specify the target language and the years for which to download ratings.

```bash
python -m bleurt.wmt.db_builder \
  -target_language="en" \
  -rating_years="2015 2016" \
  -target_file=wmt.jsonl
```

--------------------------------

### Python API

Source: https://github.com/google-research/bleurt/blob/master/README.md

Utilize BLEURT as a Python library for scoring sentence pairs.

```APIDOC
## Python API

### Description
Utilize BLEURT as a Python library for scoring sentence pairs. The library supports both eager mode (default in TF 2.0) and TF 1.0 sessions.

### Method
`from bleurt import score`
`scorer = score.BleurtScorer(checkpoint)`
`scores = scorer.score(references=references, candidates=candidates)`

### Parameters
#### `BleurtScorer` Constructor
- **checkpoint** (string) - Required - Path to the BLEURT checkpoint to use. Defaults to `BERT-Tiny` if not specified.

#### `score` Method
- **references** (list of strings) - Required - A list of reference sentences.
- **candidates** (list of strings) - Required - A list of candidate sentences.

### Request Example
```python
from bleurt import score

checkpoint = "bleurt/test_checkpoint"
references = ["This is a test."]
candidates = ["This is the test."]

scorer = score.BleurtScorer(checkpoint)
scores = scorer.score(references=references, candidates=candidates)
assert isinstance(scores, list) and len(scores) == 1
print(scores)
```

### Response
#### Success Response (200)
- **scores** (list of floats) - A list of BLEURT scores, one for each sentence pair.
```

--------------------------------

### Python API: LengthBatchingBleurtScorer

Source: https://context7.com/google-research/bleurt/llms.txt

`LengthBatchingBleurtScorer` optimizes scoring by grouping sentences by length, offering significant speedups for large datasets.

```APIDOC
## Python API: `LengthBatchingBleurtScorer`

`LengthBatchingBleurtScorer` is a drop-in replacement for `BleurtScorer` that groups sentences by length before batching, eliminating wasted padding computation for a typical 2–10x speedup on large datasets.

```python
from bleurt import score

# Requires a checkpoint with dynamic_seq_length support (e.g., BLEURT-20)
scorer = score.LengthBatchingBleurtScorer("BLEURT-20")

references = ["The quick brown fox jumps over the lazy dog."] * 500
candidates = ["A fast fox leaped above a sleeping dog."] * 500

# Internally sorts by token length, batches similar-length examples,
# and crops padding — resulting in much faster throughput
scores = scorer.score(references=references, candidates=candidates, batch_size=100)

print(f"Scored {len(scores)} pairs")
print(f"Mean score: {sum(scores)/len(scores):.4f}")
# Expected output:
# Scored 500 pairs
# Mean score: 0.7843
```
```

--------------------------------

### Command-Line Interface: Score from Text Files

Source: https://context7.com/google-research/bleurt/llms.txt

Score sentence pairs from separate reference and candidate files, or from a JSONL file. This CLI tool allows for batch processing of text files for evaluation.

```APIDOC
## Score from Text Files

### Description
Score sentence pairs from two plain-text files (one sentence per line) or a JSONL file.

### Method
```bash
python -m bleurt.score_files
```

### Parameters
#### Path Parameters
- `-candidate_file` (string) - Required - Path to the file containing candidate sentences.
- `-reference_file` (string) - Required - Path to the file containing reference sentences.
- `-sentence_pairs_file` (string) - Required - Path to the JSONL file with `{"reference": "...", "candidate": "..."}` records.
- `-bleurt_checkpoint` (string) - Required - Path or name of the BLEURT checkpoint to use.
- `-scores_file` (string) - Optional - Path to save the output scores.

### Request Example
```bash
# Score using separate reference and candidate files
python -m bleurt.score_files \
  -candidate_file=bleurt/test_data/candidates \
  -reference_file=bleurt/test_data/references \
  -bleurt_checkpoint=BLEURT-20 \
  -scores_file=scores.txt

# Score using a JSONL file with {"reference": "...", "candidate": "..."} records
python -m bleurt.score_files \
  -sentence_pairs_file=bleurt/test_data/sentence_pairs.jsonl \
  -bleurt_checkpoint=BLEURT-20
```

### Response
#### Success Response (200)
- `scores.txt` (file) - Contains one score per line corresponding to each sentence pair.

### Response Example
```text
0.8731
0.4219
0.9103
```
```

--------------------------------

### Python API: LengthBatchingBleurtScorer for faster scoring

Source: https://context7.com/google-research/bleurt/llms.txt

Use LengthBatchingBleurtScorer for a significant speedup on large datasets by grouping sentences by length before batching. Requires a checkpoint with dynamic sequence length support.

```python
from bleurt import score

# Requires a checkpoint with dynamic_seq_length support (e.g., BLEURT-20)
scorer = score.LengthBatchingBleurtScorer("BLEURT-20")

references = ["The quick brown fox jumps over the lazy dog."] * 500
candidates = ["A fast fox leaped above a sleeping dog."] * 500

# Internally sorts by token length, batches similar-length examples,
# and crops padding — resulting in much faster throughput
scores = scorer.score(references=references, candidates=candidates, batch_size=100)

print(f"Scored {len(scores)} pairs")
print(f"Mean score: {sum(scores)/len(scores):.4f}")
```

--------------------------------

### Create and Use BLEURT Ops in TensorFlow

Source: https://github.com/google-research/bleurt/blob/master/README.md

This snippet demonstrates how to create and use BLEURT operations within a TensorFlow environment. Ensure `tf.enable_eager_execution()` is set if using TensorFlow 1.x.

```python
from bleurt import score

references = tf.constant(["This is a test."])
candidates = tf.constant(["This is the test."])

bleurt_ops = score.create_bleurt_ops()
bleurt_out = bleurt_ops(references=references, candidates=candidates)

assert bleurt_out["predictions"].shape == (1,)
print(bleurt_out["predictions"])
```

--------------------------------

### Read BLEURT Checkpoint Configuration

Source: https://context7.com/google-research/bleurt/llms.txt

The `read_bleurt_config` function reads and validates a `bleurt_config.json` file from a checkpoint directory, resolving relative paths to absolute ones. It returns a dictionary containing configuration details.

```python
from bleurt import checkpoint as checkpoint_lib

# Read config from an existing checkpoint
config = checkpoint_lib.read_bleurt_config("BLEURT-20")

print(config["name"])
print(config["max_seq_length"])
print(config["vocab_file"])
print(config["sp_model"])
print(config["do_lower_case"])
print(config["dynamic_seq_length"])
print(config["tf_checkpoint_variables"])
```

--------------------------------

### Fine-tune BLEURT from Existing Checkpoint

Source: https://github.com/google-research/bleurt/blob/master/checkpoints.md

Use this command to fine-tune a BLEURT checkpoint on a custom set of ratings. Ensure the train and dev sets are correctly formatted JSONL files.

```bash
python -m bleurt.finetune \
  -init_bleurt_checkpoint=bleurt/test_checkpoint \
  -model_dir=my_new_bleurt_checkpoint \
  -train_set=bleurt/test_data/ratings_train.jsonl \
  -dev_set=bleurt/test_data/ratings_dev.jsonl \
  -num_train_steps=500
```

--------------------------------

### Python API: `BleurtScorer`

Source: https://context7.com/google-research/bleurt/llms.txt

The `BleurtScorer` class provides a programmatic way to score sentence pairs using BLEURT. It loads the model once and supports efficient batched scoring.

```APIDOC
## Python API: `BleurtScorer`

### Description
`BleurtScorer` is the primary Python class for scoring reference-candidate sentence pairs programmatically; it loads the model once and supports batched scoring.

### Method
```python
from bleurt import score

scorer = score.BleurtScorer(checkpoint_path)
scores = scorer.score(references, candidates, batch_size=None)
```

### Parameters
#### Initialization Parameters
- `checkpoint_path` (string) - Required - Path or name of the BLEURT checkpoint to load (e.g., "BLEURT-20").

#### `score` Method Parameters
- `references` (list of strings) - Required - A list of reference sentences.
- `candidates` (list of strings) - Required - A list of candidate sentences.
- `batch_size` (integer) - Optional - Custom batch size for scoring (default is 16). Adjust for memory/speed tradeoff.

### Request Example
```python
from bleurt import score

# Initialize scorer with the recommended checkpoint
scorer = score.BleurtScorer("BLEURT-20")

# Score multiple sentence pairs at once
references = [
    "The cat sat on the mat.",
    "It is a great day outside.",
    "She ordered a coffee.",
]
candidates = [
    "A cat was sitting on a mat.",        # paraphrase
    "The weather is absolutely terrible.", # opposite meaning
    "She asked for a cup of coffee.",      # near-paraphrase
]

scores = scorer.score(references=references, candidates=candidates)
print(scores)

# Custom batch size for memory/speed tradeoff
scores_batched = scorer.score(
    references=references,
    candidates=candidates,
    batch_size=32  # default is 16
)
```

### Response
#### Success Response (200)
- `scores` (list of floats) - A list of BLEURT scores, one for each sentence pair provided.

### Response Example
```python
[0.873, -0.241, 0.791]
```
```

--------------------------------

### Optimize BLEURT Scoring Speed

Source: https://context7.com/google-research/bleurt/llms.txt

Achieve up to 20x speedup by combining a larger batch size, length-based batching, and a distilled checkpoint. Download the distilled model first.

```bash
# Download distilled 12-layer model (~3.5x smaller than BLEURT-20)
wget https://storage.googleapis.com/bleurt-oss-21/BLEURT-20-D12.zip .
unzip BLEURT-20-D12.zip

python -m bleurt.score_files \
  -candidate_file=bleurt/test_data/candidates \
  -reference_file=bleurt/test_data/references \
  -bleurt_batch_size=100 \
  -batch_same_length=True \
  -bleurt_checkpoint=BLEURT-20-D12
```

--------------------------------

### Tensorflow API

Source: https://github.com/google-research/bleurt/blob/master/README.md

Integrate BLEURT into a TensorFlow computation graph for advanced use cases like Tensorboard visualization during model training.

```APIDOC
## Tensorflow API

### Description
BLEURT may be embedded in a TF computation graph, e.g., to visualize it on the Tensorboard while training a model.

### Method
Requires importing TensorFlow and using BLEURT's TF operations.

### Parameters
Specific parameters depend on the TensorFlow operations exposed by BLEURT for graph embedding.

### Request Example
```python
import tensorflow as tf
# Example usage would involve defining BLEURT operations within a TF graph.
# Specific code depends on the exact TF API provided by BLEURT.
```

### Response
Response structure depends on the specific TensorFlow operations used and the context of the computation graph.
```

--------------------------------

### Python API: SavedModelBleurtScorer

Source: https://context7.com/google-research/bleurt/llms.txt

`SavedModelBleurtScorer` loads a BLEURT SavedModel with in-graph preprocessing, suitable for scenarios where tokenization must occur within the TF graph.

```APIDOC
## Python API: `SavedModelBleurtScorer`

`SavedModelBleurtScorer` loads a BLEURT SavedModel that includes in-graph string preprocessing, useful when the tokenization must happen inside the TF graph (e.g., for serving).

```python
from bleurt import score

# Load a SavedModel-based BLEURT (with in-graph preprocessing)
scorer = score.SavedModelBleurtScorer(saved_model="path/to/bleurt_saved_model")

references = ["This is a test sentence.", "Another reference sentence."]
candidates = ["This is a test.", "Another candidate sentence here."]

scores = scorer.score(references=references, candidates=candidates, batch_size=16)
print(scores)  # [0.8203, 0.7654]

# With raw tf.Example input (serialize_input=False):
scorer_raw = score.SavedModelBleurtScorer(
    saved_model="path/to/bleurt_saved_model",
    serialize_input=False  # pass references/candidates as tf.string tensors
)
scores_raw = scorer_raw.score(references=references, candidates=candidates)
```
```

--------------------------------

### TensorFlow API: create_bleurt_ops

Source: https://context7.com/google-research/bleurt/llms.txt

`create_bleurt_ops` integrates BLEURT scoring directly into TensorFlow graphs for use in training loops and Tensorboard.

```APIDOC
## TensorFlow API: `create_bleurt_ops`

`create_bleurt_ops` embeds BLEURT scoring directly into a TensorFlow computation graph, enabling integration with model training loops and Tensorboard visualization.

```python
import tensorflow as tf
# For TF 1.x, call: tf.enable_eager_execution()

from bleurt import score

# Create TF ops (loads the checkpoint once)
bleurt_ops = score.create_bleurt_ops(checkpoint="BLEURT-20")

# Use TF string tensors as inputs
references = tf.constant(["The cat sat on the mat.", "It is sunny today."])
candidates = tf.constant(["A cat was on the mat.", "The weather is great."])

# Run scoring within the TF graph
bleurt_out = bleurt_ops(references=references, candidates=candidates)

# Output tensor shape: (n_pairs,)
assert bleurt_out["predictions"].shape == (2,)
print(bleurt_out["predictions"])
# Example output: tf.Tensor([0.8412 0.7631], shape=(2,), dtype=float32)

# Integration example: use in a training loop metric
def compute_bleurt_loss(ref_batch, cand_batch):
    out = bleurt_ops(references=ref_batch, candidates=cand_batch)
    return 1.0 - tf.reduce_mean(out["predictions"])
```
```

--------------------------------

### Score Sentence Pairs Programmatically with Python API

Source: https://context7.com/google-research/bleurt/llms.txt

Use the `BleurtScorer` class to programmatically score multiple sentence pairs. Initialize the scorer with a checkpoint and call the `score` method.

```python
from bleurt import score

# Initialize scorer with the recommended checkpoint
scorer = score.BleurtScorer("BLEURT-20")

# Score multiple sentence pairs at once
references = [
    "The cat sat on the mat.",
    "It is a great day outside.",
    "She ordered a coffee.",
]
candidates = [
    "A cat was sitting on a mat.",        # paraphrase
    "The weather is absolutely terrible.", # opposite meaning
    "She asked for a cup of coffee.",      # near-paraphrase
]

scores = scorer.score(references=references, candidates=candidates)
# scores is a list of floats, one per pair
print(scores)
# Example output: [0.873, -0.241, 0.791]

# Custom batch size for memory/speed tradeoff
scores_batched = scorer.score(
    references=references,
    candidates=candidates,
    batch_size=32  # default is 16
)

```

--------------------------------

### Python API: SavedModelBleurtScorer for serving

Source: https://context7.com/google-research/bleurt/llms.txt

Load a BLEURT SavedModel with in-graph preprocessing using SavedModelBleurtScorer. This is useful when tokenization must occur within the TensorFlow graph, such as for serving. The serialize_input flag controls input format.

```python
from bleurt import score

# Load a SavedModel-based BLEURT (with in-graph preprocessing)
scorer = score.SavedModelBleurtScorer(saved_model="path/to/bleurt_saved_model")

references = ["This is a test sentence.", "Another reference sentence."]
candidates = ["This is a test.", "Another candidate sentence here."]

scores = scorer.score(references=references, candidates=candidates, batch_size=16)
print(scores)

# With raw tf.Example input (serialize_input=False):
scorer_raw = score.SavedModelBleurtScorer(
    saved_model="path/to/bleurt_saved_model",
    serialize_input=False  # pass references/candidates as tf.string tensors
)
scores_raw = scorer_raw.score(references=references, candidates=candidates)
```

--------------------------------

### TensorFlow API: create_bleurt_ops for graph integration

Source: https://context7.com/google-research/bleurt/llms.txt

Embed BLEURT scoring directly into a TensorFlow computation graph using create_bleurt_ops. This enables integration with model training loops and Tensorboard visualization. Ensure eager execution is enabled for TF 1.x.

```python
import tensorflow as tf
# For TF 1.x, call: tf.enable_eager_execution()

from bleurt import score

# Create TF ops (loads the checkpoint once)
bleurt_ops = score.create_bleurt_ops(checkpoint="BLEURT-20")

# Use TF string tensors as inputs
references = tf.constant(["The cat sat on the mat.", "It is sunny today."])
candidates = tf.constant(["A cat was on the mat.", "The weather is great."])

# Run scoring within the TF graph
bleurt_out = bleurt_ops(references=references, candidates=candidates)

# Output tensor shape: (n_pairs,)
assert bleurt_out["predictions"].shape == (2,)
print(bleurt_out["predictions"])

# Integration example: use in a training loop metric
def compute_bleurt_loss(ref_batch, cand_batch):
    out = bleurt_ops(references=ref_batch, candidates=cand_batch)
    return 1.0 - tf.reduce_mean(out["predictions"])
```

--------------------------------

### BLEURT EMNLP Paper Citation

Source: https://github.com/google-research/bleurt/blob/master/README.md

Citation for the follow-up paper on learning compact metrics for MT, which led to BLEURT-20.

```bibtex
@inproceedings{pu2021learning,
  title = {Learning compact metrics for MT},
  author = {Pu, Amy and Chung, Hyung Won and Parikh, Ankur P and Gehrmann, Sebastian and Sellam, Thibault},
  booktitle = {Proceedings of EMNLP},
  year = {2021}
}
```

--------------------------------

### BLEURT ACL Paper Citation

Source: https://github.com/google-research/bleurt/blob/master/README.md

Citation for the BLEURT paper presented at ACL 2020.

```bibtex
@inproceedings{sellam2020bleurt,
  title = {BLEURT: Learning Robust Metrics for Text Generation},
  author = {Thibault Sellam and Dipanjan Das and Ankur P Parikh},
  year = {2020},
  booktitle = {Proceedings of ACL}
}
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.