### Setup Conda Virtual Environment

Source: https://github.com/qdata/textattack/blob/master/docs/0_get_started/installation.md

Example commands to create and activate a dedicated Python environment for TextAttack using Conda.

```bash
conda create -n textattackenv python=3.8
conda activate textattackenv
conda env list
```

--------------------------------

### Install TextAttack from Source

Source: https://github.com/qdata/textattack/blob/master/docs/0_get_started/installation.md

Commands to clone the repository and install the package in editable mode for development purposes.

```bash
git clone https://github.com/QData/TextAttack.git
cd TextAttack
pip install -e .[dev]
```

--------------------------------

### Initialize Dataset and Attack Parameters

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/1_Introduction_and_Transformations.ipynb

Sets up a custom dataset with labeled examples and defines the attack configuration, including the number of examples to process. It then initializes the dataset and attacker objects.

```python
custom_dataset = [\n    ("Malaria deaths in Africa fall by 5% from last year", 0),\n    ("Washington Nationals defeat the Houston Astros to win the World Series", 1),\n    ("Exxon Mobil hires a new CEO", 2),\n    ("Microsoft invests $1 billion in OpenAI", 3),\n]\n\nattack_args = AttackArgs(num_examples=4)\n\ndataset = Dataset(custom_dataset)
```

--------------------------------

### Install Optional Dependencies

Source: https://github.com/qdata/textattack/blob/master/docs/0_get_started/installation.md

Commands to install additional packages required for specific attacks or features within the framework.

```bash
pip install textattack[tensorflow]
pip install textattack[optional]
pip install textattack[tensorflow,optional]
```

--------------------------------

### Install Dependencies

Source: https://github.com/qdata/textattack/blob/master/tests/badcharacters2021/badcharacters.ipynb

Command to install required Python packages from a requirements file.

```bash
!pip install -r requirements.txt
```

--------------------------------

### Automated Model Setup and Dependency Configuration

Source: https://github.com/qdata/textattack/blob/master/tests/badcharacters2021/badcharacters.ipynb

Scripts to download the IBM MAX Toxic model assets, install required dependencies, and modify configuration files to point to local directories.

```python
def run(cmd):
    print(f"Running: {cmd}")
    subprocess.run(cmd, shell=True, check=True)

os.makedirs("temp/toxic", exist_ok=True)
url = "https://codait-cos-max.s3.us.cloud-object-storage.appdomain.cloud/max-toxic-comment-classifier/1.0.0/assets.tar.gz"
response = requests.get(url)
with open("temp/toxic/assets/assets.tar.gz", "wb") as f:
    f.write(response.content)
run("git clone https://github.com/IBM/MAX-Toxic-Comment-Classifier.git")
```

--------------------------------

### Install TextAttack with TensorFlow Support

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/3_Augmentations.ipynb

This command installs the TextAttack library with TensorFlow support, which is necessary for running the tutorial code in a notebook environment.

```bash
pip3 install textattack[tensorflow]
```

--------------------------------

### Install TextAttack and Captum

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/Example_5_Explain_BERT.ipynb

Installs the necessary libraries for TextAttack with TensorFlow support and the Captum library for model explanation. It's recommended to run this in your notebook environment.

```bash
pip3 install textattack[tensorflow]
pip3 install captum
```

--------------------------------

### Install Dependencies with Pip

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/Example_1_sklearn.ipynb

Installs necessary libraries including datasets, nltk, and sklearn for text analysis and model training. This command ensures all required packages are available in the current environment.

```bash
!pip install datasets nltk sklearn
```

--------------------------------

### Setup TextAttack Model, Tokenizer, and Dataset in Python

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/2_Constraints.ipynb

Initializes a Hugging Face model and tokenizer for sequence classification, wraps them in a TextAttack model wrapper, and creates a dataset instance. This setup is necessary for defining and testing TextAttack goals and constraints. Dependencies include `transformers` and `textattack.models.wrappers.HuggingFaceModelWrapper`, `textattack.goal_functions.UntargetedClassification`, and `textattack.datasets.HuggingFaceDataset`.

```python
# Import the model
import transformers
from textattack.models.wrappers import HuggingFaceModelWrapper

model = transformers.AutoModelForSequenceClassification.from_pretrained(
    "textattack/albert-base-v2-ag-news"
)
tokenizer = transformers.AutoTokenizer.from_pretrained(
    "textattack/albert-base-v2-ag-news"
)

model_wrapper = HuggingFaceModelWrapper(model, tokenizer)

# Create the goal function using the model
from textattack.goal_functions import UntargetedClassification

goal_function = UntargetedClassification(model_wrapper)

# Import the dataset
from textattack.datasets import HuggingFaceDataset

dataset = HuggingFaceDataset("ag_news", None, "test")
```

--------------------------------

### Install TextAttack via Pip

Source: https://github.com/qdata/textattack/blob/master/docs/0_get_started/installation.md

Standard installation command for TextAttack including support for TensorFlow dependencies.

```bash
pip install textattack[tensorflow]
```

--------------------------------

### Configure and Initialize Attack

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/1_Introduction_and_Transformations.ipynb

Sets up an attack by combining a transformation, constraints, and a search method. This example uses GreedySearch and common modification constraints.

```python
from textattack.search_methods import GreedySearch
from textattack.constraints.pre_transformation import RepeatModification, StopwordModification
from textattack import Attack

transformation = BananaWordSwap()
constraints = [RepeatModification(), StopwordModification()]
search_method = GreedySearch()
attack = Attack(goal_function, constraints, transformation, search_method)
```

--------------------------------

### Install TextAttack Development Environment

Source: https://github.com/qdata/textattack/blob/master/CONTRIBUTING.md

Commands to set up a local development environment for TextAttack. This installs the package in editable mode and includes development dependencies like linters and testing tools.

```bash
cd TextAttack
pip install -e .[dev]
pip install black docformatter isort pytest pytest-xdist
```

--------------------------------

### Build and Preview Documentation

Source: https://github.com/qdata/textattack/blob/master/CONTRIBUTING.md

Commands to build the project documentation using Sphinx. 'make docs' builds the documentation once, while 'make docs-auto' starts a live-reloading server for continuous previewing during development.

```bash
make docs
# or for live preview
make docs-auto
```

--------------------------------

### Install and Configure TextAttack Environment

Source: https://github.com/qdata/textattack/blob/master/docs/1start/FAQ.md

Commands to install the TextAttack library, including optional dependencies and setting up a dedicated Conda environment for development.

```bash
pip install --force-reinstall textattack
pip install textattack[tensorflow,optional]
conda create -n textattackenv python=3.8
conda activate textattackenv
conda env list
git clone https://github.com/QData/TextAttack.git
cd TextAttack
pip install .[dev]
```

--------------------------------

### Configure Search Methods for Adversarial Attacks

Source: https://context7.com/qdata/textattack/llms.txt

Demonstrates how to initialize various search algorithms like GreedyWordSwapWIR, BeamSearch, and GeneticAlgorithm to explore transformation spaces for adversarial examples.

```python
from textattack.search_methods import (GreedyWordSwapWIR, BeamSearch, GeneticAlgorithm, ParticleSwarmOptimization)

search_greedy_wir = GreedyWordSwapWIR(wir_method="delete")
search_beam = BeamSearch(beam_width=4)
search_genetic = GeneticAlgorithm(pop_size=60, max_iters=20, post_crossover_check=False)
search_pso = ParticleSwarmOptimization(pop_size=60, max_iters=20)
```

--------------------------------

### Initializing Attack Components

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/1_Introduction_and_Transformations.ipynb

This section covers the setup of the model wrapper and goal function required to execute an adversarial attack.

```APIDOC
## Setup: Goal Function and Model Wrapper

### Description
To perform an attack, you must wrap a HuggingFace model and define an untargeted classification goal function.

### Implementation
```python
import transformers
from textattack.models.wrappers import HuggingFaceModelWrapper
from textattack.goal_functions import UntargetedClassification

model = transformers.AutoModelForSequenceClassification.from_pretrained("textattack/bert-base-uncased-ag-news")
tokenizer = transformers.AutoTokenizer.from_pretrained("textattack/bert-base-uncased-ag-news")

model_wrapper = HuggingFaceModelWrapper(model, tokenizer)
goal_function = UntargetedClassification(model_wrapper)
```
```

--------------------------------

### Installing Project Requirements

Source: https://github.com/qdata/textattack/blob/master/tests/badcharacters2021/badcharacters.ipynb

This command installs the necessary Python packages listed in the requirements.txt file for the toxic comment classifier project. It uses pip, the Python package installer, and shows output indicating already satisfied requirements like torch and pytorch-pretrained-bert.

```Shell
pip install -r temp/toxic/toxic/requirements.txt
```

--------------------------------

### Execute Adversarial Attacks via Command Line

Source: https://github.com/qdata/textattack/blob/master/README.md

Examples of running pre-defined adversarial attack recipes against machine learning models using the TextAttack CLI. These commands specify the target model, the attack recipe, and the number of examples to evaluate.

```bash
textattack attack --model bert-base-uncased-sst2 --recipe textfooler --num-examples 10
textattack attack --model t5-en-de --recipe seq2sick --num-examples 100
```

--------------------------------

### Prepare Model and Calculate Outputs

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/Example_5_Explain_BERT.ipynb

Prepares a deep copy of the model, moves it to the appropriate device (CPU or GPU), and defines a function to calculate model outputs given tokenized inputs. This setup is crucial for attribution methods.

```python
sel = 2
batch_encoded = model.tokenizer(
    [dataset[i][0]["text"] for i in range(sel)], padding=True, return_tensors="pt"
)
batch_encoded.to(device)
labels = [dataset[i][1] for i in range(sel)]

clone = deepcopy(model)
clone.model.to(device)


def calculate(input_ids, token_type_ids, attention_mask):
    # convert back to list of text
    return clone.model(input_ids, token_type_ids, attention_mask)[0]


# x = calculate(**batch_encoded)

lig = LayerIntegratedGradients(calculate, clone.model.bert.embeddings)
```

--------------------------------

### Train Models via Command Line Interface

Source: https://context7.com/qdata/textattack/llms.txt

Provides examples of using the textattack CLI to train models on standard datasets or perform adversarial training.

```bash
textattack train --model-name-or-path lstm --dataset yelp_polarity --epochs 50 --learning-rate 1e-5
textattack train --model-name-or-path bert-base-uncased --dataset glue^cola --per-device-train-batch-size 8 --epochs 5
textattack train --model-name-or-path bert-base-uncased --dataset imdb --attack deepwordbug --num-train-adv-examples 1000 --epochs 3
```

--------------------------------

### Using HuggingFace Models and Datasets

Source: https://github.com/qdata/textattack/blob/master/README.md

Example of loading and attacking a model and dataset directly from HuggingFace.

```APIDOC
## Using HuggingFace Models and Datasets

### Description
This command shows how to leverage TextAttack's integration with HuggingFace. It allows you to specify a model and dataset directly from the HuggingFace Hub for your attacks.

### Method
CLI Command

### Endpoint
N/A

### Parameters
#### Command Line Arguments
- **--model-from-huggingface** (string) - Required - The name of the model from HuggingFace Hub.
- **--dataset-from-huggingface** (string) - Required - The name of the dataset from HuggingFace Hub.
- **--recipe** (string) - Required - The attack recipe to apply.
- **--num-examples** (integer) - Optional - The number of examples to run the attack on.

### Request Example
```bash
textattack attack --model-from-huggingface distilbert-base-uncased-finetuned-sst-2-english --dataset-from-huggingface glue^sst2 --recipe deepwordbug --num-examples 10
```

### Response
N/A (This is a command-line execution example)

```

--------------------------------

### Model Training with `textattack train`

Source: https://github.com/qdata/textattack/blob/master/README.md

Provides examples of training and fine-tuning models using the `textattack train` command-line interface.

```APIDOC
## Model Training

### Description
Train or fine-tune models (LSTMs, CNNs, transformers) using the `textattack train` command. Datasets are automatically loaded via the `datasets` package.

### Method
Command-Line Interface

### Endpoint
N/A (Command-Line Tool)

### Parameters
N/A (Command-Line Arguments)

### Request Example
```bash
# Train default LSTM on Yelp Polarity dataset for 50 epochs
textattack train --model-name-or-path lstm --dataset yelp_polarity --epochs 50 --learning-rate 1e-5

# Fine-tune bert-base on CoLA dataset for 5 epochs
textattack train --model-name-or-path bert-base-uncased --dataset glue^cola --per-device-train-batch-size 8 --epochs 5
```

### Response
N/A (Command-line output)

#### Response Example
N/A
```

--------------------------------

### Initialize and Execute TextAttack Adversarial Attack

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/Example_6_Chinese_Attack.ipynb

This snippet shows the setup of an Attack object with goal functions, constraints, and search methods. It then initializes an Attacker with specific arguments and runs the attack process on a provided dataset.

```python
attack = Attack(goal_function, constraints, transformation, search_method)
attack_args = AttackArgs(num_examples=20)
attacker = Attacker(attack, dataset, attack_args)
attack_results = attacker.attack_dataset()
```

--------------------------------

### TextAttack Command-Line Usage

Source: https://github.com/qdata/textattack/blob/master/README.md

Examples of how to use the TextAttack command-line interface to run attack recipes.

```APIDOC
## TextFooler Attack Example

### Description
Runs the TextFooler attack against a BERT model fine-tuned on SST-2.

### Method
Command-line execution

### Endpoint
N/A

### Parameters
- `--model` (string) - Required - The pre-trained model to attack.
- `--recipe` (string) - Required - The attack recipe to use (e.g., `textfooler`).
- `--num-examples` (integer) - Required - The number of examples to run the attack on.

### Request Example
```bash
textattack attack --model bert-base-uncased-sst2 --recipe textfooler --num-examples 10
```

### Response
N/A

## Seq2Sick Attack Example

### Description
Runs the seq2sick (black-box) attack against a T5 model fine-tuned for English-German translation.

### Method
Command-line execution

### Endpoint
N/A

### Parameters
- `--model` (string) - Required - The pre-trained model to attack.
- `--recipe` (string) - Required - The attack recipe to use (e.g., `seq2sick`).
- `--num-examples` (integer) - Required - The number of examples to run the attack on.

### Request Example
```bash
textattack attack --model t5-en-de --recipe seq2sick --num-examples 100
```

### Response
N/A
```

--------------------------------

### Using Built-in Models and Datasets

Source: https://github.com/qdata/textattack/blob/master/README.md

Example of how to run an attack using a built-in model and dataset. TextAttack automatically matches the correct dataset to the model.

```APIDOC
## Using Built-in Models and Datasets

### Description
This command demonstrates how to initiate an attack using a pre-trained model (e.g., 'roberta-base-sst2') and a corresponding built-in dataset. TextAttack handles the automatic loading and matching of the appropriate dataset for the specified model.

### Method
CLI Command

### Endpoint
N/A

### Parameters
#### Command Line Arguments
- **--model** (string) - Required - The name of the pre-trained model to use.
- **--recipe** (string) - Required - The attack recipe to apply.
- **--num-examples** (integer) - Optional - The number of examples to run the attack on.

### Request Example
```bash
textattack attack --model roberta-base-sst2 --recipe textfooler --num-examples 10
```

### Response
N/A (This is a command-line execution example)

```

--------------------------------

### Run Seq2Sick Attack with TextAttack CLI

Source: https://github.com/qdata/textattack/blob/master/docs/3recipes/attack_recipes_cmd.md

This command demonstrates running the seq2sick (black-box) attack against a T5 model fine-tuned for English-German translation. It includes the model, recipe, and the number of examples.

```bash
textattack attack --model t5-en-de --recipe seq2sick --num-examples 100
```

--------------------------------

### TextAttack MNLI Attack Setup and Execution

Source: https://github.com/qdata/textattack/blob/master/tests/badcharacters2021/badcharacters.ipynb

This Python script demonstrates setting up and running an adversarial attack using TextAttack on the MultiNLI dataset with a Fairseq MNLI model. It includes downloading the dataset, creating a dataset object, building an attack recipe, and configuring the attacker.

```python
import os
import requests
import zipfile
import json
import shutil
from io import BytesIO
import textattack

# Assuming FairseqMnliWrapper and args are defined elsewhere
# model = torch.hub.load('pytorch/fairseq', 'roberta.large.mnli').eval()
# model_wrapper = FairseqMnliWrapper(model)

url = "https://cims.nyu.edu/~sbowman/multinli/multinli_1.0.zip"
os.makedirs("temp/mnli", exist_ok=True)
response = requests.get(url)
response.raise_for_status()
with zipfile.ZipFile(BytesIO(response.content)) as zip_ref:
    zip_ref.extractall("temp/mnli")
label_map = {'contradiction': 0, 'neutral': 1, 'entailment': 2}
pairs = []
with open("temp/mnli/multinli_1.0/multinli_1.0_dev_matched.jsonl", 'r') as f:
    for line in f:
        sample = json.loads(line)
        if sample['gold_label'] not in label_map:
            continue

        premise = sample['sentence1']
        hypothesis = sample['sentence2']
        label = label_map[sample['gold_label']]
        item = ((premise, hypothesis), label)
        pairs.append(item)
dataset = textattack.datasets.Dataset(pairs, input_columns=["premise", "hypothesis"])
print(dataset[0])
attack = textattack.attack_recipes.BadCharacters2021.build(
    model_wrapper, 
    goal_function_type="targeted_strict", 
    perturbation_type=args.perturbation_type
)
attack_args = textattack.AttackArgs(
    num_examples=10,
    log_to_csv="results/mnli/log.csv"
)
attacker = textattack.Attacker(attack, dataset, attack_args)
attacker.attack_dataset()
if args.store_temp_files == False:
    if os.path.isdir("temp/mnli"):
        shutil.rmtree("temp/mnli")
if args.store_results == False:
    if os.path.isdir("results/mnli"):
        shutil.rmtree("results/mnli")
```

--------------------------------

### Initialize and Run TextAttack

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/2_Constraints.ipynb

Sets up an attack object with a goal function, constraints, transformation, and search method. It then configures attack arguments for logging to CSV and specifies the number of successful examples to achieve. Finally, it initializes an attacker with the defined attack, dataset, and arguments, and executes the attack on the dataset.

```python
from textattack.loggers import CSVLogger
from textattack.attack_results import SuccessfulAttackResult
from textattack import Attacker, AttackArgs

attack_args = AttackArgs(
    num_successful_examples=5, log_to_csv="results.csv", csv_coloring_style="html"
)
attacker = Attacker(attack, dataset, attack_args)

attack_results = attacker.attack_dataset()
```

--------------------------------

### Perform HotFlip Attack with Advanced Metrics

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/0_End_to_End.ipynb

This command executes a HotFlip attack against an LSTM model on the MR dataset. It targets 4 examples, starting from the 3rd example (offset), and enables advanced metrics for detailed evaluation.

```bash
!textattack attack --model lstm-mr --recipe hotflip --num-examples 4 --num-examples-offset 3 --enable-advance-metrics
```

--------------------------------

### GET /metrics/quality

Source: https://github.com/qdata/textattack/blob/master/docs/apidoc/textattack.metrics.rst

Retrieves the collection of quality-specific metrics used to evaluate the semantic similarity and linguistic quality of adversarial examples.

```APIDOC
## GET /metrics/quality

### Description
Provides access to the quality_metrics subpackage, which contains implementations for measuring output quality.

### Method
GET

### Endpoint
/metrics/quality

### Parameters
#### Query Parameters
- **metric_type** (string) - Optional - Filter by specific quality metric type.

### Request Example
GET /metrics/quality?metric_type=semantic

### Response
#### Success Response (200)
- **metrics** (array) - List of available quality metrics.

#### Response Example
{
  "metrics": ["bert_score", "use_score"]
}
```

--------------------------------

### Clone and Set Up TextAttack Repository

Source: https://github.com/qdata/textattack/blob/master/CONTRIBUTING.md

Instructions for cloning the TextAttack repository, adding the base repository as a remote, and creating a new branch for development. This ensures your local copy is up-to-date and development is done on a separate branch.

```bash
git clone git@github.com:<your Github handle>/TextAttack.git
cd TextAttack
git remote add upstream https://github.com/QData/TextAttack
git checkout -b a-descriptive-name-for-my-changes
```

--------------------------------

### Initialize and Run TextAttack Recipe

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/Example_5_Explain_BERT.ipynb

Demonstrates how to build a predefined attack recipe (PWWSRen2019) and execute it against a dataset using the Attacker class.

```python
from textattack.attack_recipes import PWWSRen2019
from textattack import Attacker

attack = PWWSRen2019.build(model)
attacker = Attacker(attack, dataset)
attacker.attack_dataset()
```

--------------------------------

### Attack CNN Model from File with Offset

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/0_End_to_End.ipynb

This command attacks a CNN model on the IMDB dataset using a custom attack defined in a file. It targets 2 examples, starting from the 18th example (offset), and uses the '--attack-n' flag.

```bash
! textattack attack --model cnn-imdb --attack-from-file tests/sample_inputs/attack_from_file.py^Attack --num-examples 2  --num-examples-offset 18 --attack-n
```

--------------------------------

### TextAttack Translation Attack Example

Source: https://github.com/qdata/textattack/blob/master/tests/badcharacters2021/badcharacters.ipynb

Example of setting up and running a translation attack using TextAttack.

```APIDOC
## TextAttack Translation Attack Example

### Description
This example demonstrates how to configure and execute a translation attack using the TextAttack framework, specifically targeting a Fairseq translation model.

### Setup
1. Load a Fairseq translation model using `torch.hub`.
2. Instantiate the `FairseqTranslationWrapper` with the loaded model.
3. Download and load the WMT14 English-French dataset.
4. Build an attack recipe (e.g., `BadCharacters2021`).
5. Configure `AttackArgs` for the attack.
6. Initialize the `Attacker`.
7. Run the attack.
8. Optionally clean up temporary files and results.

### Parameters
#### `model_wrapper`
- **model_wrapper** (FairseqTranslationWrapper) - Required - An instance of the translation model wrapper.

#### `dataset`
- **dataset** (textattack.datasets.Dataset) - Required - The dataset to use for the attack.

#### `attack_args`
- **num_examples** (int) - Optional - Number of examples to attack.
- **log_to_csv** (str) - Optional - Path to save attack logs as CSV.

### Request Body
(Not applicable)

### Response
(Attack results are logged to CSV and console output)

### Request Example
```python
import textattack
import torch
import os
import shutil

# Assume FairseqTranslationWrapper, download_en_fr_dataset, load_en_fr_dataset are defined
# Assume args object with perturbation_type, store_temp_files, store_results is available

# 1. Load model and wrapper
model = torch.hub.load(
    'pytorch/fairseq',
    'transformer.wmt14.en-fr',
    tokenizer='moses',
    bpe='subword_nmt',
    verbose=False
).eval()
model_wrapper = FairseqTranslationWrapper(model)

# 2. Download and load dataset
download_en_fr_dataset()
dataset = load_en_fr_dataset()

# 3. Build attack recipe
attack = textattack.attack_recipes.BadCharacters2021.build(
    model_wrapper,
    goal_function_type="maximize_levenshtein",
    perturbation_type=args.perturbation_type,
    target_distance=0.1
)

# 4. Configure attack arguments
attack_args = textattack.AttackArgs(
    num_examples=1,
    log_to_csv="results/translation/log.csv"
)

# 5. Initialize and run attacker
attacker = textattack.Attacker(attack, dataset, attack_args)
attacker.attack_dataset()

# 6. Clean up (optional)
if not args.store_temp_files:
    if os.path.isdir("temp/translation"):
        shutil.rmtree("temp/translation")
if not args.store_results:
    if os.path.isdir("results/translation"):
        # Consider removing or archiving results directory if needed
        pass
```

### Response Example
```
# Console output will show attack progress and results.
# Example log entry in results/translation/log.csv might look like:
# "result_id","original_text","perturbed_text","original_score","perturbed_score","ground_truth","attack_num_queries","attack_time_seconds","attack_success","attack_recipe_name","goal_function_type","perturbation_type","target_distance","model_name"
# "1","The United States remained the only superpower...","The United States remained the only superpower...","0.9","0.7","Les États-Unis sont restés...","10","0.5","True","BadCharacters2021","maximize_levenshtein","delete","0.1","fairseq-transformer-wmt14enfr"
```
```

--------------------------------

### Integrate Custom Models and Datasets

Source: https://github.com/qdata/textattack/blob/master/docs/1start/FAQ.md

Demonstrates how to load custom models and datasets from Python files to be used with the TextAttack framework.

```python
# my_model.py
model = load_your_model_with_custom_code()
tokenizer = load_your_tokenizer_with_custom_code()

# my_dataset.py
dataset = [('Today was....', 1), ('This movie is...', 0)]

# Loading custom dataset via TextAttack API
import textattack
my_dataset = [("text", label)]
new_dataset = textattack.datasets.Dataset(my_dataset)
```

--------------------------------

### Bash: List TextAttack Components

Source: https://github.com/qdata/textattack/blob/master/README.md

Demonstrates how to use the `textattack list` command to enumerate available components within TextAttack, such as pretrained models or search methods.

```bash
textattack list models
```

```bash
textattack list search-methods
```

--------------------------------

### Initialize and Run Attack Recipe in Python

Source: https://github.com/qdata/textattack/blob/master/docs/3recipes/attack_recipes.rst

Programmatically initialize an attack recipe by building it with a model wrapper, then execute it against a dataset.

```python
attack = InputReductionFeng2018.build(model)
attack.attack_dataset(dataset)
```

--------------------------------

### Keras Model Training Output Example

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/Example_3_Keras.ipynb

Example output from training a Keras model, showing the loss and accuracy for each epoch on both the training and validation datasets. This output is typically generated by `model.fit()` and can be logged or analyzed.

```text
Epoch 1/18
44/44 [==============================] - 0s 9ms/step - loss: 0.9584 - accuracy: 0.4987 - val_loss: 0.7314 - val_accuracy: 0.5056
Epoch 2/18
44/44 [==============================] - 0s 6ms/step - loss: 0.9078 - accuracy: 0.5064 - val_loss: 0.7149 - val_accuracy: 0.5332
Epoch 3/18
44/44 [==============================] - 0s 6ms/step - loss: 0.8743 - accuracy: 0.5264 - val_loss: 0.7000 - val_accuracy: 0.5600
Epoch 4/18
44/44 [==============================] - 0s 6ms/step - loss: 0.8534 - accuracy: 0.5385 - val_loss: 0.6840 - val_accuracy: 0.5904
Epoch 5/18
44/44 [==============================] - 0s 6ms/step - loss: 0.8329 - accuracy: 0.5564 - val_loss: 0.6754 - val_accuracy: 0.6064
Epoch 6/18
44/44 [==============================] - 0s 6ms/step - loss: 0.8168 - accuracy: 0.5615 - val_loss: 0.6637 - val_accuracy: 0.6348
Epoch 7/18
44/44 [==============================] - 0s 6ms/step - loss: 0.7942 - accuracy: 0.5767 - val_loss: 0.6568 - val_accuracy: 0.6460
Epoch 8/18
44/44 [==============================] - 0s 6ms/step - loss: 0.7798 - accuracy: 0.5895 - val_loss: 0.6464 - val_accuracy: 0.6632
Epoch 9/18
44/44 [==============================] - 0s 6ms/step - loss: 0.7624 - accuracy: 0.6000 - val_loss: 0.6357 - val_accuracy: 0.6772
Epoch 10/18
44/44 [==============================] - 0s 6ms/step - loss: 0.7523 - accuracy: 0.6096 - val_loss: 0.6275 - val_accuracy: 0.6932
Epoch 11/18
44/44 [==============================] - 0s 6ms/step - loss: 0.7391 - accuracy: 0.6185 - val_loss: 0.6196 - val_accuracy: 0.6996
Epoch 12/18
44/44 [==============================] - 0s 6ms/step - loss: 0.7265 - accuracy: 0.6324 - val_loss: 0.6126 - val_accuracy: 0.7076
Epoch 13/18
44/44 [==============================] - 0s 6ms/step - loss: 0.7140 - accuracy: 0.6408 - val_loss: 0.6047 - val_accuracy: 0.7196
Epoch 14/18
44/44 [==============================] - 0s 6ms/step - loss: 0.7042 - accuracy: 0.6476 - val_loss: 0.5981 - val_accuracy: 0.7268
Epoch 15/18
44/44 [==============================] - 0s 6ms/step - loss: 0.6944 - accuracy: 0.6586 - val_loss: 0.5906 - val_accuracy: 0.7340
Epoch 16/18
44/44 [==============================] - 0s 6ms/step - loss: 0.6798 - accuracy: 0.6677 - val_loss: 0.5826 - val_accuracy: 0.7432
Epoch 17/18
44/44 [==============================] - 0s 6ms/step - loss: 0.6702 - accuracy: 0.6766 - val_loss: 0.5741 - val_accuracy: 0.7524
Epoch 18/18
44/44 [==============================] - 0s 6ms/step - loss: 0.6643 - accuracy: 0.6834 - val_loss: 0.5667 - val_accuracy: 0.7580
```

--------------------------------

### Run TextAttack Attack on Custom Dataset

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/4_Custom_Datasets_Word_Embedding.ipynb

This code iterates through a custom dataset, applying the configured attack to each example and printing the results. It demonstrates how to execute an attack and display the output, including skipped examples.

```python
# Assuming 'attack' and 'custom_dataset' are defined
# custom_dataset = [...] 

for example, label in custom_dataset:
    result = attack.attack(example, label)
    print(result.__str__(color_method="ansi"))
```

--------------------------------

### GET /metrics/words-perturbed

Source: https://github.com/qdata/textattack/blob/master/docs/apidoc/textattack.metrics.attack_metrics.rst

Retrieves statistics regarding the number of words perturbed during the attack.

```APIDOC
## GET /metrics/words-perturbed

### Description
Returns the average or total count of words modified in the input text to achieve the adversarial goal.

### Method
GET

### Endpoint
/metrics/words-perturbed

### Parameters
#### Query Parameters
- **attack_id** (string) - Required - The unique identifier for the completed attack session.

### Request Example
GET /metrics/words-perturbed?attack_id=12345

### Response
#### Success Response (200)
- **avg_perturbed_words** (float) - The average number of words changed per successful attack.

#### Response Example
{
  "avg_perturbed_words": 3.2
}
```

--------------------------------

### Using Pre-built Augmentation Recipes (CheckListAugmenter)

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/3_Augmentations.ipynb

This snippet shows how to use a pre-built augmentation recipe, specifically the `CheckListAugmenter`, which applies transformations based on CheckList methodology.

```APIDOC
## Using Pre-built Augmentation Recipes (CheckListAugmenter)

### Description
This section demonstrates the usage of pre-built augmentation recipes provided by TextAttack. The example uses `CheckListAugmenter`, which employs transformations like Name Replacement, Location Replacement, and Number Alteration.

### Method
Instantiate the `CheckListAugmenter` class and call its augmentation method.

### Endpoint
N/A (Python Class Instantiation)

### Parameters
#### CheckListAugmenter Parameters
- **pct_words_to_swap** (float) - Optional: Percentage of words to swap per augmented example. Default is 0.1.
- **transformations_per_example** (int) - Optional: Maximum number of augmentations per input. Default is 1.

### Request Example
```python
# import the CheckListAugmenter
from textattack.augmentation import CheckListAugmenter

# Alter default values if desired
augmenter = CheckListAugmenter(pct_words_to_swap=0.2, transformations_per_example=5)
s = "I'd love to go to Japan but the tickets are 500 dollars"
aumentations = augmenter.augment(s)
print(aumentations)
```

### Response
#### Success Response (List of augmented strings)
- **augmented_strings** (list of str) - A list containing the generated augmented text using CheckList transformations.

#### Response Example
```json
[
  "I'd love to go to China but the tickets are 500 dollars",
  "I'd love to go to Japan but the tickets are 500 dollars",
  "I'd love to go to Japan but the tickets are 500 dollars",
  "I'd love to go to Japan but the tickets are 500 dollars",
  "I'd love to go to Japan but the tickets are 500 dollars"
]
```
```

--------------------------------

### GET /metrics/attack-queries

Source: https://github.com/qdata/textattack/blob/master/docs/apidoc/textattack.metrics.attack_metrics.rst

Retrieves the number of queries performed during an adversarial attack process.

```APIDOC
## GET /metrics/attack-queries

### Description
Calculates and returns the total number of queries made to the target model during the attack execution.

### Method
GET

### Endpoint
/metrics/attack-queries

### Parameters
#### Query Parameters
- **attack_id** (string) - Required - The unique identifier for the completed attack session.

### Request Example
GET /metrics/attack-queries?attack_id=12345

### Response
#### Success Response (200)
- **query_count** (integer) - The total number of queries performed.

#### Response Example
{
  "query_count": 150
}
```

--------------------------------

### GET /metrics/attack-success-rate

Source: https://github.com/qdata/textattack/blob/master/docs/apidoc/textattack.metrics.attack_metrics.rst

Calculates the success rate of the adversarial attack based on the provided results.

```APIDOC
## GET /metrics/attack-success-rate

### Description
Computes the percentage of successful adversarial examples generated relative to the total number of attempts.

### Method
GET

### Endpoint
/metrics/attack-success-rate

### Parameters
#### Query Parameters
- **attack_id** (string) - Required - The unique identifier for the completed attack session.

### Request Example
GET /metrics/attack-success-rate?attack_id=12345

### Response
#### Success Response (200)
- **success_rate** (float) - The calculated success rate as a decimal (0.0 to 1.0).

#### Response Example
{
  "success_rate": 0.85
}
```

--------------------------------

### Execute Dataset Attack with TextAttack

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/Example_3_Keras.ipynb

Demonstrates how to initialize a model wrapper, load a dataset, and run an attack using the Attacker class and a specific recipe.

```python
from textattack import AttackArgs
from textattack.datasets import Dataset
from textattack import Attacker

model_wrapper = CustomKerasModelWrapper(model)
dataset = HuggingFaceDataset("rotten_tomatoes", None, "test", shuffle=True)

attack = PWWSRen2019.build(model_wrapper)

attack_args = AttackArgs(num_examples=10, checkpoint_dir="checkpoints")

attacker = Attacker(attack, dataset, attack_args)

attacker.attack_dataset()
```

--------------------------------

### GET /api/v1/attack/results

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/Example_5_Explain_BERT.ipynb

Retrieves the summary and detailed results of a completed adversarial attack execution.

```APIDOC
## GET /api/v1/attack/results

### Description
Fetches the statistical summary and individual attack result objects for a specific attack job.

### Method
GET

### Endpoint
/api/v1/attack/results

### Parameters
#### Query Parameters
- **job_id** (string) - Required - The unique identifier for the attack job.

### Request Example
GET /api/v1/attack/results?job_id=12345

### Response
#### Success Response (200)
- **successful_attacks** (integer) - Number of successful adversarial attacks.
- **failed_attacks** (integer) - Number of failed adversarial attacks.
- **attack_success_rate** (float) - The percentage of successful attacks.
- **avg_perturbed_word_pct** (float) - Average percentage of words perturbed per input.
- **avg_num_queries** (float) - Average number of queries performed per attack.

#### Response Example
{
  "successful_attacks": 10,
  "failed_attacks": 0,
  "attack_success_rate": 100.0,
  "avg_perturbed_word_pct": 16.33,
  "avg_num_queries": 331.8
}
```

--------------------------------

### Download OpenHowNet Resources

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/Example_6_Chinese_Attack.ipynb

Downloads the necessary resources for the OpenHowNet library, which is used for synonym replacement in Chinese text transformations.

```python
import OpenHowNet

OpenHowNet.download()
```

--------------------------------

### Perform Text Augmentation with WordNetAugmenter

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/3_Augmentations.ipynb

Demonstrates how to initialize the WordNetAugmenter with specific parameters like swap percentage and high yield mode. It also shows how to enable advanced metrics to evaluate the quality of the generated augmentations.

```python
from textattack.augmentation import WordNetAugmenter

augmenter = WordNetAugmenter(
    pct_words_to_swap=0.4,
    transformations_per_example=5,
    high_yield=True,
    enable_advanced_metrics=True,
)
s = "I'd love to go to Japan but the tickets are 500 dollars"
results = augmenter.augment(s)
print(f"Average Original Perplexity Score: {results[1]['avg_original_perplexity']}\n")
print(f"Average Augment Perplexity Score: {results[1]['avg_attack_perplexity']}\n")
print(f"Average Augment USE Score: {results[2]['avg_attack_use_score']}\n")
print(f"Augmentations:")
results[0]
```

--------------------------------

### GET /api/attack/results

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/Example_2_allennlp.ipynb

Retrieves the summary statistics and individual result objects for a completed adversarial attack execution.

```APIDOC
## GET /api/attack/results

### Description
Fetches the performance metrics of an adversarial attack, including success rates, accuracy degradation, and query counts.

### Method
GET

### Endpoint
/api/attack/results

### Parameters
#### Query Parameters
- **job_id** (string) - Required - The unique identifier for the attack execution job.

### Request Example
GET /api/attack/results?job_id=12345

### Response
#### Success Response (200)
- **successful_attacks** (integer) - Number of successful adversarial perturbations.
- **failed_attacks** (integer) - Number of failed adversarial attempts.
- **skipped_attacks** (integer) - Number of inputs skipped during the attack.
- **original_accuracy** (float) - Accuracy of the model on the original dataset.
- **attack_success_rate** (float) - Percentage of successful attacks.

#### Response Example
{
  "successful_attacks": 3,
  "failed_attacks": 6,
  "skipped_attacks": 1,
  "original_accuracy": 0.9,
  "attack_success_rate": 0.3333
}
```

--------------------------------

### GET /metrics/attack

Source: https://github.com/qdata/textattack/blob/master/docs/apidoc/textattack.metrics.rst

Retrieves the collection of attack-specific metrics used to evaluate the performance and success rate of adversarial attacks.

```APIDOC
## GET /metrics/attack

### Description
Provides access to the attack_metrics subpackage, which contains implementations for measuring attack effectiveness.

### Method
GET

### Endpoint
/metrics/attack

### Parameters
#### Query Parameters
- **format** (string) - Optional - The response format (json or xml).

### Request Example
GET /metrics/attack?format=json

### Response
#### Success Response (200)
- **metrics** (array) - List of available attack metrics.

#### Response Example
{
  "metrics": ["success_rate", "avg_num_queries"]
}
```

--------------------------------

### Create Custom Datasets in TextAttack

Source: https://context7.com/qdata/textattack/llms.txt

Demonstrates how to initialize a TextAttack Dataset from a list of tuples for simple classification or multi-input tasks like entailment.

```python
import textattack

# Simple classification dataset
custom_data = [("This movie is great!", 1), ("Terrible film, waste of time.", 0)]
custom_dataset = textattack.datasets.Dataset(custom_data)

# Multi-input dataset for entailment
entailment_data = [({"premise": "A man is playing guitar.", "hypothesis": "Someone is making music."}, 0)]
entailment_dataset = textattack.datasets.Dataset(
    entailment_data,
    input_columns=("premise", "hypothesis"),
    label_names=["entailment", "neutral", "contradiction"]
)
```

--------------------------------

### Loading and Running Attack Recipes

Source: https://context7.com/qdata/textattack/llms.txt

Initialize a model wrapper using HuggingFace transformers and instantiate various pre-built attack recipes. This allows for rapid testing of adversarial robustness against different attack strategies.

```python
import textattack
import transformers

model = transformers.AutoModelForSequenceClassification.from_pretrained("textattack/bert-base-uncased-mr")
tokenizer = transformers.AutoTokenizer.from_pretrained("textattack/bert-base-uncased-mr")
model_wrapper = textattack.models.wrappers.HuggingFaceModelWrapper(model, tokenizer)

attack_textfooler = textattack.attack_recipes.TextFoolerJin2019.build(model_wrapper)
text = "This movie is absolutely fantastic and I loved every minute of it."
result = attack_textfooler.attack(text, 1)
print(result)
```

--------------------------------

### Execute Attack on Dataset

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/1_Introduction_and_Transformations.ipynb

Uses the Attacker class to run an attack against a dataset for a specified number of examples. It returns a collection of attack results.

```python
from textattack import Attacker, AttackArgs

attack_args = AttackArgs(num_examples=10)
attacker = Attacker(attack, dataset, attack_args)
attack_results = attacker.attack_dataset()
```

--------------------------------

### Perform Adversarial Model Training

Source: https://context7.com/qdata/textattack/llms.txt

Configures and executes a training pipeline that incorporates adversarial examples generated by an attack recipe during the training process.

```python
import textattack, transformers

model = transformers.AutoModelForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=2)
tokenizer = transformers.AutoTokenizer.from_pretrained("bert-base-uncased")
model_wrapper = textattack.models.wrappers.HuggingFaceModelWrapper(model, tokenizer)
attack = textattack.attack_recipes.DeepWordBugGao2018.build(model_wrapper)

training_args = textattack.TrainingArgs(num_epochs=3, num_clean_epochs=1, num_train_adv_examples=1000, output_dir="./training_output")
trainer = textattack.Trainer(model_wrapper, task_type="classification", attack=attack, train_dataset=train_dataset, eval_dataset=eval_dataset, training_args=training_args)
trainer.train()
```

--------------------------------

### Loading Custom Model from File

Source: https://github.com/qdata/textattack/blob/master/README.md

Instructions on how to load a custom model and tokenizer from a Python file.

```APIDOC
## Loading Custom Model from File

### Description
This section explains how to use your own trained models with TextAttack by providing a Python file that defines the `model` and `tokenizer` variables. The tokenizer must have an `encode()` method, and the model must be callable via its `__call__` method.

### Method
CLI Command

### Endpoint
N/A

### Parameters
#### Command Line Arguments
- **--model-from-file** (string) - Required - Path to the Python file containing the `model` and `tokenizer`.

### Request Example
Assume you have a file named `my_model.py` with the following content:
```python
model = load_your_model_with_custom_code() # replace this line with your model loading code
tokenizer = load_your_tokenizer_with_custom_code() # replace this line with your tokenizer loading code
```

Then, run the attack using:
```bash
textattack attack --model-from-file my_model.py --recipe <your_recipe> --num-examples 10
```

### Response
N/A (This is a command-line execution example)

```

--------------------------------

### Run TextFooler Attack on BERT

Source: https://github.com/qdata/textattack/blob/master/docs/3recipes/attack_recipes_cmd.md

Executes the TextFooler attack recipe against a BERT model fine-tuned on the MR sentiment classification dataset. It targets 100 examples from the dataset.

```bash
textattack attack --recipe textfooler --model bert-base-uncased-mr --num-examples 100
```

--------------------------------

### TextAttack CLI: Running Attacks and Listing Resources

Source: https://context7.com/qdata/textattack/llms.txt

This section demonstrates how to use the TextAttack command-line interface (CLI) to perform various tasks. It covers running different types of adversarial attacks on specified models and datasets, listing available attack recipes and models, and peeking at dataset contents. The CLI offers flexibility in configuring attacks with custom parameters.

```bash
textattack --help

textattack attack --recipe textfooler --model bert-base-uncased-sst2 --num-examples 100

textattack attack --model distilbert-base-uncased-cola --recipe deepwordbug --num-examples 100

textattack attack --model lstm-mr --num-examples 20 \
  --search-method beam-search^beam_width=4 --transformation word-swap-embedding \
  --constraints repeat stopword max-words-perturbed^max_num_words=2 embedding^min_cos_sim=0.8 part-of-speech \
  --goal-function untargeted-classification

textattack attack --model-from-huggingface distilbert-base-uncased-finetuned-sst-2-english \
  --dataset-from-huggingface glue^sst2 --recipe deepwordbug --num-examples 10

textattack attack --recipe textfooler --model bert-base-uncased-mr --interactive

textattack list attack-recipes

textattack list models

textattack peek-dataset --dataset-from-huggingface snli
```

--------------------------------

### Run TextAttack Augmentation via Command Line

Source: https://github.com/qdata/textattack/blob/master/docs/2notebook/3_Augmentations.ipynb

Shows the equivalent command-line interface command for performing text augmentation using the checklist recipe.

```bash
textattack augment --recipe checklist --pct-words-to-swap .1 --transformations-per-example 5 --exclude-original --interactive
```