### Running the Example Notebook (Shell)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/SentimentAnalysis/README.md

Command to launch the provided Jupyter Notebook containing the sentiment analysis example. This requires Jupyter Notebook to be installed and the notebook file 'SentimentAnalysis.ipynb' to be accessible.

```Shell
jupyter notebook SentimentAnalysis.ipynb
```

--------------------------------

### Running LASER Docker Image with Port Mapping (Example)

Source: https://github.com/facebookresearch/laser/blob/main/docker/README.md

Example command to run the LASER Docker image, mapping local port 8081 to the container's port 80 and starting the Flask REST server via `python app.py`.

```Docker
docker run -it -p 8081:80 laser python app.py
```

--------------------------------

### Run P-xSIM Example Script (bash)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/pxsim/README.md

Executes the example bash script to demonstrate P-xSIM calculation. This script downloads sample data (flores200) and encoders (laser2, LaBSE) and then performs the P-xSIM evaluation using laser2 for k-nearest-neighbors and LaBSE as the auxiliary model.

```bash
bash ./eval.sh
```

--------------------------------

### Installing laser_encoders via pip

Source: https://github.com/facebookresearch/laser/blob/main/laser_encoders/README.md

Installs the `laser_encoders` Python package from the Python Package Index (PyPI) using the pip package manager. This is the standard way to install the library for general use.

```Shell
pip install laser_encoders
```

--------------------------------

### Install and Run CCMatrix Download Script

Source: https://github.com/facebookresearch/laser/blob/main/tasks/CCMatrix/README.md

These commands install the required cc_net library and then execute the main Python script to download the CCMatrix parallel sentence data. This allows users to reproduce the data generation process described in the paper.

```bash
pip3 install cc_net
python3 dl_cc_matrix.py
```

--------------------------------

### Configuring HuggingFace Encoder for xSIM++ Evaluation

Source: https://github.com/facebookresearch/laser/blob/main/tasks/xsimplusplus/README.md

These command-line arguments are used to configure the xSIM++ evaluation script to utilize a sentence-transformer model hosted on HuggingFace, such as LaBSE. They specify the encoder name, enable the HuggingFace integration flag, and set the expected embedding dimension.

```Shell
--src-encoder LaBSE
--use-hugging-face
--embedding-dimension 768
```

--------------------------------

### Installing laser_encoders in editable mode

Source: https://github.com/facebookresearch/laser/blob/main/laser_encoders/README.md

Installs the `laser_encoders` package from a local clone of the repository in "editable" mode. This is useful for development, allowing changes in the source code to be reflected without re-installation.

```Shell
pip install . -e
```

--------------------------------

### Sending Request to LASER REST Server (Python)

Source: https://github.com/facebookresearch/laser/blob/main/docker/README.md

Python code snippet demonstrating how to send a GET request to the running LASER REST server to obtain sentence embeddings. It uses the `requests` library and prints the embedding from the JSON response.

```Python
import requests
import numpy as np
url = "http://127.0.0.1:[CHANGEME_LOCAL_PORT]/vectorize"
params = {"q": "Hey, how are you?\nI'm OK and you?", "lang": "en"}
resp = requests.get(url=url, params=params).json()
print(resp["embedding"])
```

--------------------------------

### Running MLDoc Classification Script (Bash)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/mldoc/README.md

This bash command executes the script responsible for calculating multilingual sentence embeddings for all languages in the MLDoc corpus and subsequently training the document classifier. It is a necessary step in the installation and setup process.

```bash
bash ./mldoc.sh
```

--------------------------------

### Embed Text with Specific LASER3 Encoder - Bash

Source: https://github.com/facebookresearch/laser/blob/main/nllb/README.md

Provides an example command to embed text from an input file ([INFILE]) into an output file ([OUTFILE]) using a specific LASER3 encoder (e.g., 'wol_Latn'). Replace placeholders with actual file paths and language code.

```bash
./LASER/tasks/embed/embed.sh [INFILE] [OUTFILE] wol_Latn
```

--------------------------------

### Calculate xSIM from Embeddings (Python)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/xsim/README.md

Shows two ways to calculate xSIM error and total examples (`err`, `nbex`) using the `xSIM` function. Method A takes numpy arrays `x` and `y` directly. Method B takes paths to binary embedding files `x` and `y`, requiring `dim` and optionally `fp16` parameters.

```python
# A: numpy arrays x and y

err, nbex = xSIM(x, y)

# B: binary embedding files x and y

fp16_flag = False     # set true if embeddings are saved in 16 bit
embedding_dim = 1024  # set dimension of saved embeddings
err, nbex = xSIM(
  x,
  y,
  dim=embedding_dim,
  fp16=fp16_flag
)
```

--------------------------------

### Building LASER Docker Image (Basic)

Source: https://github.com/facebookresearch/laser/blob/main/docker/README.md

Command to build the LASER Docker image using the default Dockerfile. This requires executing the command from the root directory of the LASER project.

```Docker
docker build --tag laser -f docker/Dockerfile .
```

--------------------------------

### Running LASER Docker Image (Basic)

Source: https://github.com/facebookresearch/laser/blob/main/docker/README.md

Command to run the built LASER Docker image in interactive mode. This uses the default entrypoint defined in the Dockerfile.

```Docker
docker run -it laser
```

--------------------------------

### Initializing and using tokenizer/encoder separately (Python)

Source: https://github.com/facebookresearch/laser/blob/main/laser_encoders/README.md

Shows how to initialize the tokenizer and encoder components of LASER separately using `initialize_tokenizer` and `initialize_encoder`. This provides more granular control over the tokenization and encoding process before generating embeddings. Requires the `laser_encoders` package.

```Python
from laser_encoders import initialize_encoder, initialize_tokenizer

# Initialize the LASER tokenizer
tokenizer = initialize_tokenizer(lang="igbo")
tokenized_sentence = tokenizer.tokenize("nnọọ, kedu ka ị mere")

# Initialize the LASER sentence encoder
encoder = initialize_encoder(lang="igbo")

# Encode tokenized sentences into embeddings
embeddings = encoder.encode_sentences([tokenized_sentence])
```

--------------------------------

### Running LASER Docker Image with Port Mapping (Placeholder)

Source: https://github.com/facebookresearch/laser/blob/main/docker/README.md

Command to run the LASER Docker image, mapping a local port to the container's port 80 and overriding the default entrypoint to run the Flask REST server. Replace `[CHANGEME_LOCAL_PORT]` with the desired local port.

```Docker
docker run -it -p [CHANGEME_LOCAL_PORT]:80 laser python app.py
```

--------------------------------

### Using LaserTokenizer and SentenceEncoder with local paths (Python)

Source: https://github.com/facebookresearch/laser/blob/main/laser_encoders/README.md

Shows how to initialize `LaserTokenizer` and `SentenceEncoder` separately using local file paths for the SPM model, model file, and vocabulary. This allows for manual tokenization before encoding, using locally managed model files. Requires `laser_encoders.laser_tokenizer`, `laser_encoders.models`, and `pathlib`.

```Python
from laser_encoders.laser_tokenizer import LaserTokenizer
from laser_encoders.models import SentenceEncoder
from pathlib import Path

tokenizer = LaserTokenizer(spm_model=Path(path/to/spm_model))

tokenized_sentence = tokenizer.tokenize("This is a test sentence.")

encoder = SentenceEncoder(model_path=path/to/downloaded/model, spm_vocab=path/to/cvocab)
embeddings = encoder.encode_sentences([tokenized_sentence])
```

--------------------------------

### Using SentenceEncoder with local model paths (Python)

Source: https://github.com/facebookresearch/laser/blob/main/laser_encoders/README.md

Demonstrates how to initialize the `SentenceEncoder` class directly by providing explicit file paths to the downloaded model, SPM model, and vocabulary files. This is used when models are managed manually rather than through the pipeline or initialization functions. Requires the `laser_encoders.models` module and `pathlib`.

```Python
from laser_encoders.models import SentenceEncoder
from pathlib import Path

encoder = SentenceEncoder(model_path=path/to/downloaded/model, spm_model=Path(path/to/spm_model), spm_vocab=path/to/cvocab)
embeddings = encoder("This is a test sentence.")
```

--------------------------------

### Tokenizing a file using LaserTokenizer (Python)

Source: https://github.com/facebookresearch/laser/blob/main/laser_encoders/README.md

Demonstrates how to use the `tokenize_file` method of the `LaserTokenizer` class to process an entire input file, tokenize its content, and write the tokenized output to a specified output file. Requires `laser_encoders.laser_tokenizer` and `pathlib`.

```Python
tokenized_sentence = tokenizer.tokenize_file(inp_fname=Path(path/to/input_file.txt), out_fname=Path(path/to/output_file.txt))
```

--------------------------------

### Download All LASER Encoders - Bash

Source: https://github.com/facebookresearch/laser/blob/main/nllb/README.md

Executes the download script to fetch all available LASER2 and LASER3 encoders by default. Note that downloading all LASER3 encoders may require significant disk space.

```bash
bash ./download_models.sh
```

--------------------------------

### Building LASER Docker Image with Specific Languages

Source: https://github.com/facebookresearch/laser/blob/main/docker/README.md

Command to build the LASER Docker image while pre-downloading models for specified languages. The `langs` build argument accepts a space-separated list of language codes (e.g., "eng_Latn fra_Latn").

```Docker
docker build --build-arg langs="eng_Latn fra_Latn" -t laser -f docker/Dockerfile .
```

--------------------------------

### Encoding sentences using LaserEncoderPipeline (Python)

Source: https://github.com/facebookresearch/laser/blob/main/laser_encoders/README.md

Demonstrates how to use the `LaserEncoderPipeline` class to initialize the encoder for a specific language and encode a list of sentences into embeddings. It also shows how to obtain L2-normalized embeddings. Requires the `laser_encoders` package.

```Python
from laser_encoders import LaserEncoderPipeline

# Initialize the LASER encoder pipeline
encoder = LaserEncoderPipeline(lang="igbo")

# Encode sentences into embeddings
embeddings = encoder.encode_sentences(["nnọọ, kedu ka ị mere"])
# If you want the output embeddings to be L2-normalized, set normalize_embeddings to True
normalized_embeddings = encoder.encode_sentences(["nnọọ, kedu ka ị mere"], normalize_embeddings=True)
```

--------------------------------

### Configure xSIM with HuggingFace Encoder (Bash)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/xsim/README.md

Shows the command-line arguments to add or modify in the `eval.sh` script to use a HuggingFace sentence-transformer model (like LaBSE) instead of the default LASER encoder. Specifies the encoder name, enables the HuggingFace flag, and sets the embedding dimension.

```bash
--src-encoder LaBSE
--use-hugging-face
--embedding-dimension 768
```

--------------------------------

### Download LASER WMT '22 Encoders (bash)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/wmt22/README.md

This command executes the `download_models.sh` script located in the current directory. The script downloads the pre-trained LASER sentence encoders required for the WMT '22 shared task for all 24 supported languages and places them in the `$LASER/models/wmt22` directory.

```bash
bash ./download_models.sh
```

--------------------------------

### Downloading LASER models to a custom directory (Shell)

Source: https://github.com/facebookresearch/laser/blob/main/laser_encoders/README.md

Executes the `download_models` script to download pre-trained LASER models, specifying a custom directory path where the models should be saved instead of the default cache location. Replace `path/to/model/directory` with the desired path.

```Shell
python -m laser_encoders.download_models --model-dir=path/to/model/directory
```

--------------------------------

### Downloading LASER models for a specific language (Shell)

Source: https://github.com/facebookresearch/laser/blob/main/laser_encoders/README.md

Executes the `download_models` script provided by the `laser_encoders` package to download the pre-trained LASER model and associated files for a specified language. Replace `your_prefered_language` with the desired language code.

```Shell
python -m laser_encoders.download_models --lang=your_prefered_language  # e.g., --lang="igbo""
```

--------------------------------

### Import xSIM Function (Python)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/xsim/README.md

Imports the main `xSIM` function from the `xsim` library, making it available for use in a Python script. This is the first step to using the xSIM functionality programmatically.

```python
from xsim import xSIM
```

--------------------------------

### Run WMT Data Processing Script - Bash

Source: https://github.com/facebookresearch/laser/blob/main/tasks/similarity/README.md

Executes the wmt.sh script to automate the process of downloading the newstest2012 dataset, calculating sentence embeddings using LASER, and computing the similarity search error rate for each language pair within the corpus.

```Bash
bash ./wmt.sh
```

--------------------------------

### Run XNLI Cross-lingual NLI Script (Bash)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/xnli/README.md

This bash script automates the process of setting up the XNLI and MultiNLI corpora, calculating multilingual sentence embeddings, training the NLI classifier, and displaying the results for cross-lingual transfer.

```Bash
bash ./xnli.sh
```

--------------------------------

### Run BUCC Bitext Mining Script

Source: https://github.com/facebookresearch/laser/blob/main/tasks/bucc/README.md

Executes the main script (`bucc.sh`) to perform bitext mining on the downloaded BUCC shared task data. This script orchestrates the process of embedding sentences, calculating distances, and extracting parallel pairs based on a threshold.

```Bash
./bucc.sh
```

--------------------------------

### Run LASER Embedding Script (Generic)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/embed/README.md

Executes the `embed.sh` script to generate sentence embeddings. It takes an input text file, an output file path for the binary embeddings, and an optional language code.

```bash
bash ./embed.sh INPUT-FILE OUTPUT-FILE [LANGUAGE]
```

--------------------------------

### Select xSIM Margin Scoring Type (Python)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/xsim/README.md

Demonstrates how to specify the margin-based scoring method for xSIM. Method A shows the default 'ratio' margin. Method B explicitly sets the margin to 'distance'. Method C explicitly sets the margin to 'absolute'.

```python
# A: ratio (default)
err, nbex = xSIM(x, y)

# B: distance
err, nbex = xSIM(x, y, margin='distance')

# C: absolute
err, nbex = xSIM(x, y, margin='absolute')
```

--------------------------------

### Run LASER Embedding Script (LASER2 Default)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/embed/README.md

Executes the `embed.sh` script without specifying a language, defaulting to the LASER2 model which supports 93 languages.

```bash
./embed.sh input_file output_file
```

--------------------------------

### Run LASER Embedding Script (Specific Language - Wolof)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/embed/README.md

Executes the `embed.sh` script specifying the language code 'wol_Latn' to use a language-specific LASER3 encoder if available.

```bash
./embed.sh input_file output_file wol_Latn
```

--------------------------------

### Run LASER Embedding Script (Specific Language - Hausa)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/embed/README.md

Executes the `embed.sh` script specifying the language code 'hau_Latn' to use a language-specific LASER3 encoder if available.

```bash
./embed.sh input_file output_file hau_Latn
```

--------------------------------

### Download Specific Language Pair Bitext (Bash)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/WikiMatrix/README.md

Downloads a gzipped TSV file containing parallel sentences for a specific language pair from the WikiMatrix dataset. The language pair codes (e.g., 'en-fr') must be replaced with the desired pair, ensuring alphabetical order. This command downloads a single file.

```bash
wget https://dl.fbaipublicfiles.com/laser/WikiMatrix/v1/WikiMatrix.en-fr.tsv.gz
```

--------------------------------

### Using laser_encoders for Sentence Embedding (Python)

Source: https://github.com/facebookresearch/laser/blob/main/README.md

This snippet demonstrates how to use the `laser_encoders` package to generate sentence embeddings. It initializes the pipeline for a specific language (English), encodes a list of sentences, and prints the shape of the resulting embedding tensor.

```Python
from laser_encoders import LaserEncoderPipeline
encoder = LaserEncoderPipeline(lang="eng_Latn")
embeddings = encoder.encode_sentences(["Hi!", "This is a sentence encoder."])
print(embeddings.shape)  # (2, 1024)
```

--------------------------------

### Download All Language Pairs Tarball (Bash)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/WikiMatrix/README.md

Downloads a large tar file (approximately 65GB) containing all 1620 language pairs of the WikiMatrix parallel sentence dataset. This is an alternative to downloading individual language pair files.

```bash
wget https://dl.fbaipublicfiles.com/laser/WikiMatrix/WikiMatrix.v1.1620_language_pairs.tar
```

--------------------------------

### Run LASER Embedding Script (Specific Language - Irish)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/embed/README.md

Executes the `embed.sh` script specifying the language code 'gle_Latn' to use a language-specific LASER3 encoder if available.

```bash
./embed.sh input_file output_file gle_Latn
```

--------------------------------

### Specify xSIM Error Calculation Type (Python)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/xsim/README.md

Illustrates how to choose between textual-based and index-based error calculation for xSIM. Method A uses `eval_text` with a path to the target text file for textual-based error (allowing duplicates). Method B shows the default index-based error calculation without the `eval_text` parameter.

```python
# A: textual-based error (allows for duplicates)

tgt_text = "/path/to/target-text-file"
err, nbex = xSIM(x, y, eval_text=tgt_text)

# B: index-based error (default)

err, nbex = xSIM(x, y)
```

--------------------------------

### Read LASER Binary Embeddings (Python)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/embed/README.md

Reads the raw binary output file generated by `embed.sh` into a NumPy array. Assumes the embeddings are float32 and have a dimension of 1024.

```python
import numpy as np
dim = 1024
X = np.fromfile("my_embeddings.bin", dtype=np.float32, count=-1)
X.resize(X.shape[0] // dim, dim)
```

--------------------------------

### Extract Parallel Text from TSV (Bash)

Source: https://github.com/facebookresearch/laser/blob/main/tasks/WikiMatrix/README.md

Executes the `extract.py` Python script to process a gzipped TSV file containing mined bitext. It extracts parallel sentences based on a specified margin threshold, writing the output to a plain text file. Requires the `extract.py` script and the input TSV file.

```bash
python3 extract.py \
  --tsv WikiMatrix.en-fr.tsv.gz \
  --bitext WikiMatrix.en-fr.txt \
  --src-lang en --trg-lang fr \
  --threshold 1.04
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.