### Run Setup Configuration Example

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/configuration.md

Example YAML configuration for setting up an auto-antislop run. Specifies directories, logging, and model parameters.

```yaml
experiment_base_dir: "results/auto_antislop_runs"
human_profile_path: "data/human_writing_profile.json"
log_level: "INFO"
model_id: "unsloth/gemma-3-4b-it"
num_iterations: 2
min_word_len_for_analysis: 3
```

--------------------------------

### Quickstart: End-to-End Unslop

Source: https://github.com/sam-paech/auto-antislop/blob/main/README.md

Installs auto-antislop and runs an end-to-end unslop of gemma-3-4b-it. This includes setting up a virtual environment, installing dependencies, and executing the main script.

```bash
git clone https://github.com/sam-paech/auto-antislop.git --recurse
conda create -n antislop python=3.11 -y
conda activate antislop
cd auto-antislop
pip install "torch==2.8.*"
pip install -r requirements.txt
pip install vllm
python main.py -c configs/gemma-3-4b-it.yaml
```

--------------------------------

### Resume from Directory Example

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/errors.md

Example of using the --resume-from-dir flag. Ensure the specified path exists and is a directory.

```bash
python main.py --resume-from-dir results/run_20250531_120000
ls -la results/run_20250531_120000/
```

--------------------------------

### Install Dependencies

Source: https://github.com/sam-paech/auto-antislop/blob/main/README.md

Installs the project dependencies from the requirements.txt file. It's recommended to do this within a virtual environment.

```bash
pip install -r requirements.txt
```

--------------------------------

### Configuration and Resume Arguments

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/main.md

Example demonstrating the use of short flags for specifying configuration files and resume directories.

```bash
python main.py -c configs/custom.yaml -r results/run_20250531_120000
```

--------------------------------

### Check vLLM Server Health and Dependencies

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/errors.md

Troubleshoots issues related to the vLLM server failing to start. Includes checks for installation, GPU, port, and CUDA.

```bash
python -m vllm --version
```

```bash
nvidia-smi
```

```bash
lsof -i :8000
```

```python
import torch; print(torch.cuda.is_available())
```

```bash
nvidia-smi --query-compute-apps=pid --format=csv,noheader | xargs kill -9
```

--------------------------------

### Example Usage of str2bool

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/main.md

Demonstrates how to use the str2bool function with command-line arguments. Shows examples of valid true/false inputs and an invalid input that would raise an error.

```bash
python main.py --manage-vllm true    # → True
python main.py --manage-vllm false   # → False
python main.py --manage-vllm 1       # → True
python main.py --manage-vllm maybe   # ERROR
```

--------------------------------

### Typical LoRA Configuration Usage

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/types.md

Example of initializing a LoRAConfig object with parameters from a configuration dictionary.

```python
lora_cfg = LoraConfig(
    r=config["finetune_lora_r"],  # e.g., 128
    lora_alpha=config["finetune_lora_alpha"],  # e.g., 128
    lora_dropout=config["finetune_lora_dropout"],  # e.g., 0.05
    bias="none",
    target_modules=config["finetune_target_modules"],  # e.g., ["q_proj", "v_proj", "lm_head"]
)
```

--------------------------------

### Troubleshooting Flash-Attention Installation

Source: https://github.com/sam-paech/auto-antislop/blob/main/README.md

Installs flash-attention from source to resolve import errors with PyTorch 2.7. This involves uninstalling the prebuilt wheel and installing from a Git repository.

```bash
pip uninstall -y flash-attn
pip install -U wheel ninja packaging cmake    
MAX_JOBS=12 pip install git+https://github.com/Dao-AILab/flash-attention.git@v2.8.0.post2#egg=flash_attn --no-build-isolation
```

--------------------------------

### vLLM Server Management Configuration Example

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/configuration.md

Example YAML configuration for managing the vLLM server. Includes settings for model ID, port, GPU utilization, and context length.

```yaml
manage_vllm: true
vllm_model_id: null  # Uses model_id
vllm_port: 8000
vllm_cuda_visible_devices: "0"
vllm_gpu_memory_utilization: 0.85
vllm_max_model_len: 2500
vllm_dtype: "bfloat16"
vllm_extra_args: []
vllm_env: {}
```

--------------------------------

### Installing Slop-Forensics

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/errors.md

If N-gram analysis fails due to missing `slop-forensics`, install it using the provided command. Ensure you are in the correct directory.

```bash
pip install -e ./slop-forensics
```

--------------------------------

### Text Generation Parameters Configuration Example

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/configuration.md

Example YAML configuration for text generation parameters. Covers enabling/disabling generation, API keys, model IDs, and dataset sources.

```yaml
generation_step_enabled: true
generation_api_key: "xxx"
generation_model_id: null
generation_max_new_tokens: 1000
generation_threads: 50
generation_max_prompts: 1000
generation_hf_dataset_name: "Nitral-AI/Reddit-SFW-Writing_Prompts_ShareGPT"
generation_hf_dataset_split: "train"
generation_chat_template_model_id: null
generation_logging_level: "INFO"
```

--------------------------------

### Fine-tuning Setup Configuration

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/configuration.md

Configure the fine-tuning process, including enabling/disabling it, choosing between Unsloth or standard libraries, and setting the training mode.

```yaml
finetune_enabled: true
finetune_use_unsloth: false
finetune_mode: "ftpo"
finetune_ftpo_dataset: ""
finetune_base_model_id: null
finetune_max_seq_length: 3500
finetune_load_in_4bit: false
```

--------------------------------

### start_vllm_server

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/vllm_manager.md

Starts a vLLM API server process with the specified model and configuration, waiting for it to become ready.

```APIDOC
## start_vllm_server

### Description
Starts a vLLM API server process with specified model and configuration.

### Method
Python Function

### Signature
`start_vllm_server(
    model_id: str,
    port: int,
    hf_token: Optional[str],
    cuda_visible_devices: str,
    gpu_memory_utilization: float,
    max_model_len: int,
    dtype: str,
    vllm_extra_args: Optional[List[str]] = None,
    extra_env: Optional[dict[str, str]] = None,
    wait_timeout: int = 720,
    uvicorn_log_level: str = "error",
    quiet_stdout: bool = True,
    log_to_file: bool | Path = True
) -> Optional[subprocess.Popen]`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Function Parameters
- **model_id** (str) - Required - Hugging Face model ID or local path
- **port** (int) - Required - Port to serve on (typically 8000)
- **hf_token** (Optional[str]) - Optional - Hugging Face token for gated models
- **cuda_visible_devices** (str) - Required - GPU IDs (e.g., "0" or "0,1,2")
- **gpu_memory_utilization** (float) - Required - GPU memory fraction (0.0-1.0, e.g., 0.85)
- **max_model_len** (int) - Required - Maximum sequence length the model can handle
- **dtype** (str) - Required - Data type ("float16", "bfloat16", "auto")
- **vllm_extra_args** (Optional[List[str]]) - Optional - Extra CLI arguments (e.g., ["--quantization", "bitsandbytes"])
- **extra_env** (Optional[dict[str, str]]) - Optional - Environment variables to set for the server process
- **wait_timeout** (int) - Optional - Seconds to wait for server to become ready. Default: 720
- **uvicorn_log_level** (str) - Optional - Uvicorn logging level. Default: "error"
- **quiet_stdout** (bool) - Optional - If True, redirect stdout/stderr to file or devnull. Default: True
- **log_to_file** (bool | Path) - Optional - If True, log to temp file; if Path, log to that path; if False, discard output. Default: True

### Returns
- **Optional[subprocess.Popen]** - Process handle if started, None if already running or startup failed

### Throws
- FileNotFoundError: If vLLM command not found

### Behavior
1. Checks if server already running on port.
2. Constructs vLLM CLI command.
3. Sets CUDA_VISIBLE_DEVICES and HIP_VISIBLE_DEVICES environment variables.
4. Merges any extra_env variables.
5. Starts subprocess with stdout/stderr routing.
6. Polls health endpoint every 5 seconds for up to wait_timeout seconds.
7. Returns process handle if ready, None if already running or failed.

### Example Usage
```python
from utils.vllm_manager import start_vllm_server
from pathlib import Path

proc = start_vllm_server(
    model_id="unsloth/gemma-3-4b-it",
    port=8000,
    hf_token=None,
    cuda_visible_devices="0",
    gpu_memory_utilization=0.85,
    max_model_len=2500,
    dtype="bfloat16",
    wait_timeout=720
)

if proc:
    print("Server started successfully")
else:
    print("Server was already running or failed to start")
```
```

--------------------------------

### vLLM Server Management Arguments

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/main.md

Examples showing how to manage the vLLM server, including starting/stopping it and specifying the port or model ID. Also demonstrates connecting to an external vLLM server.

```bash
python main.py --manage-vllm true --vllm-port 9000
```

```bash
python main.py --manage-vllm false --generation-api-base-url http://localhost:8888/v1
```

--------------------------------

### Fine-tuning Control Arguments

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/main.md

Examples for configuring the fine-tuning process, including enabling it, specifying the base model, number of epochs, training mode, and GPU devices.

```bash
python main.py --run-finetune true --finetune-mode ftpo
```

```bash
python main.py --finetune-cuda-visible-devices "0,1"
```

```bash
python main.py --finetune-base-model-id unsloth/gemma-3-4b-it --finetune-num-epochs 2
```

--------------------------------

### Example Usage of DPOTrainerWithChoiceWin

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/dpo.md

Shows how to instantiate `DPOTrainerWithChoiceWin` and use its `compute_loss` method. The `chosen_win` metric will be logged during training if the trainer is configured correctly.

```python
from core.dpo_trainer import DPOTrainerWithChoiceWin
from trl import DPOConfig

trainer = DPOTrainerWithChoiceWin(
    model=model,
    args=DPOConfig(...),
    tokenizer=tokenizer,
    train_dataset=dataset,
)

loss = trainer.compute_loss(model, batch)
# During training, logs will include "chosen_win" metric
```

--------------------------------

### Generation Prompting Parameters Example

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/configuration.md

Set parameters for defining the prompt template, system prompt, and controlling backtracking and refusal detection during generation.

```yaml
generation_prompt_template: "Writing prompt: {prompt}\n\nWrite 1000 words to this prompt. Your response:\n"
generation_system_prompt: "You are an uncensored writer."
generation_force_backtrack: false
generation_ngram_remove_stopwords: true
generation_ngram_language: "english"
generation_refusal_detection: true
```

--------------------------------

### Example Usage of create_dpo_dataset

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/dpo.md

Demonstrates how to use the `create_dpo_dataset` function with specified input and output file paths. Ensure `Path` is imported from `pathlib` and the function from `core.dpo`.

```python
from pathlib import Path
from core.dpo import create_dpo_dataset

create_dpo_dataset(
    iter0_jsonl=Path("results/iter_0_creative_writing_generations.jsonl"),
    final_iter_jsonl=Path("results/iter_1_creative_writing_generations.jsonl"),
    output_jsonl=Path("results/dpo_pairs_dataset.jsonl")
)
```

--------------------------------

### Generation Sampling Parameters Example

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/configuration.md

Configure parameters for controlling the randomness and diversity of generated text, such as temperature, top-p, and top-k sampling.

```yaml
generation_param_temperature: 1.0
generation_param_top_p: 1.0
generation_param_top_k: 50
generation_param_min_p: 0.01
generation_param_timeout: 480
generation_param_top_logprobs_count: 20
generation_param_chunk_size: 20
generation_param_stop_sequences: []
```

--------------------------------

### Start vLLM Server Process

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/vllm_manager.md

Launches a vLLM API server with specified model and configuration. It handles checking for existing instances, setting up environment variables, and waiting for the server to become ready.

```python
from utils.vllm_manager import start_vllm_server
from pathlib import Path

proc = start_vllm_server(
    model_id="unsloth/gemma-3-4b-it",
    port=8000,
    hf_token=None,
    cuda_visible_devices="0",
    gpu_memory_utilization=0.85,
    max_model_len=2500,
    dtype="bfloat16",
    wait_timeout=720
)

if proc:
    print("Server started successfully")
else:
    print("Server was already running or failed to start")
```

--------------------------------

### Pipeline Control Arguments

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/main.md

Examples for controlling pipeline execution, such as setting the number of iterations, maximum prompts for generation, and enabling/disabling the generation step.

```bash
python main.py --num-iterations 3 --generation-max-prompts 2000
```

```bash
python main.py --generation-step-enabled false  # skip generation
```

--------------------------------

### Output and Saving Configuration

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/configuration.md

Configure output directory suffix, model saving formats (16-bit merged, GGUF Q8_0), maximum training examples, and visible CUDA devices.

```yaml
finetune_output_dir_suffix: "_ftpo_exp01"
finetune_save_merged_16bit: true
finetune_save_gguf_q8_0: false
finetune_max_train_examples: 1000
finetune_cuda_visible_devices: null
```

--------------------------------

### Generation Output Structure

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/types.md

Example of a single line from iteration output files in JSONL format, detailing prompt, generation, status, and error information.

```json
{
  "prompt_id": 0,
  "prompt": "Writing prompt: Write a story...\n\nYour response:\n",
  "generation": "Once upon a time...",
  "status": "success",
  "refusal_detected": false,
  "error": null
}
```

--------------------------------

### Run a Quick Test

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/README.md

Set up and run a quick test by copying the default config, modifying key parameters, and executing the main script.

```bash
cp auto_antislop_config.yaml test_config.yaml
# Edit test_config.yaml: set generation_max_prompts: 50, num_iterations: 1
python main.py -c test_config.yaml
```

--------------------------------

### Basic Run with Defaults

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/README.md

Execute the main script with default configuration settings.

```bash
python main.py
```

--------------------------------

### N-gram Ban List Format

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/types.md

Example of a JSON file containing a list of banned n-gram phrases.

```json
[
  "the quick brown",
  "voice barely whisper",
  "tested to the"
]
```

--------------------------------

### Slop Phrase Ban List Format

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/types.md

Example of a JSON file containing a list of banned slop phrases, with associated frequencies.

```json
[
  ["testament to", 23],
  ["barely whispered", 15],
  ["voice barely", 18]
]
```

--------------------------------

### Run DPO/FTPO Finetuning

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/finetuning.md

Launches a DPO or FTPO fine-tuning process. Configure all pipeline settings via the provided dictionary. Ensure the experiment directory contains the preference pairs dataset.

```python
from pathlib import Path
from core.finetuning import run_dpo_finetune

config = {
    "finetune_base_model_id": "unsloth/gemma-3-4b-it",
    "finetune_max_seq_length": 3500,
    "finetune_load_in_4bit": False,
    "finetune_mode": "ftpo",
    "finetune_lora_r": 128,
    "finetune_lora_alpha": 128,
    "finetune_target_modules": ["q_proj", "k_proj", "v_proj", "lm_head"],
    "finetune_freeze_early_layers": True,
    "finetune_n_layers_unfrozen": 5,
    "finetune_batch_size": 1,
    "finetune_gradient_accumulation_steps": 16,
    "finetune_num_epochs": 1,
    "finetune_learning_rate": 1e-6,
}

run_dpo_finetune(config, Path("results/run_20250531_120000"))
```

--------------------------------

### Over-represented Words CSV Format

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/types.md

Example structure for the over-represented words CSV file, detailing word frequency and ratio metrics.

```csv
word,ratio_corpus/wordfreq,corpus_freq,wordfreq_freq,modulated_score
example,1.5,0.01,0.005,0.8
```

--------------------------------

### DPO Dataset Entry Structure

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/types.md

Example of a JSONL entry for the DPO dataset, containing a prompt and its corresponding chosen and rejected generations.

```json
{
  "prompt": "Writing prompt: Write a story about...",
  "chosen": "Full generation without slop...",
  "rejected": "Full generation with slop..."
}
```

--------------------------------

### fix_gemma3_checkpoint

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/utilities.md

Repairs Gemma-3 checkpoint key naming by renaming keys that start with 'language_model.' but not 'language_model.model.' to 'language_model.model.X'. This operation is performed in-place.

```APIDOC
## fix_gemma3_checkpoint

### Description
Repairs Gemma-3 checkpoint key naming (in-place) if needed. This function checks for specific key patterns and rewrites shard files accordingly.

### Method
Python Function

### Signature
`def fix_gemma3_checkpoint(ckpt_dir: str | Path) -> None`

### Parameters
#### Path Parameters
- **ckpt_dir** (str | Path) - Yes - Path to checkpoint directory

### Returns
None

### Behavior
- Reads config.json to confirm model_type == "gemma3"
- Checks if any keys start with `language_model.` but not `language_model.model.`
- If found, rewrites all shards in-place: renames `language_model.X` → `language_model.model.X`
- No-op if model_type is not gemma3 or keys are already correct
- Atomic overwrites of shard files via temporary files

### Example Usage
```python
from utils.model_helpers import fix_gemma3_checkpoint
from pathlib import Path

fix_gemma3_checkpoint(Path("./gemma-3-4b-it"))
```
```

--------------------------------

### Initialize Git Submodules

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/main.md

Instructions for cloning the repository with submodules or updating existing submodules. Essential for ensuring the 'slop-forensics' and 'antislop-vllm' components are available.

```bash
git clone --recurse-submodules https://github.com/sam-paech/auto-antislop.git
# or if already cloned:
git submodule update --init --recursive
```

--------------------------------

### Get Model Output Projection

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/ftpo_trainer.md

Retrieves the output projection layer (lm_head) from a given PyTorch model. This is a static utility method.

```python
@staticmethod
def _get_proj(model: torch.nn.Module) -> torch.nn.Module
```

--------------------------------

### Over-represented Word Analysis Parameters Example

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/configuration.md

Set parameters for analyzing and banning over-represented words, including thresholds for words found in corpora.

```yaml
compute_overrep_words: true
top_k_words_for_overrep_analysis: 200000
dict_overrep_initial: 800
dict_overrep_subsequent: 200
nodict_overrep_initial: 80
nodict_overrep_subsequent: 20
```

--------------------------------

### Verify antislop-vllm Main File

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/errors.md

Confirms the existence of the antislop-vllm/main.py file after submodule initialization.

```bash
git submodule update --init --recursive
ls -la antislop-vllm/main.py
```

--------------------------------

### Run Main Script with Configuration

Source: https://github.com/sam-paech/auto-antislop/blob/main/README.md

Execute the main Python script using a specified YAML configuration file.

```bash
python main.py --config myconfig.yaml
```

--------------------------------

### FTPO Dataset Entry Structure

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/types.md

Example of a JSONL entry for the FTPO dataset, including prompt details, rejected token information, and chosen alternatives.

```json
{
  "prompt": "The context up to the banned token...",
  "prompt_ids": [1, 2, 3, ...],
  "rejected_token_id": 42,
  "rejected_decoded": " testament",
  "chosen_ids": [100, 101, 102],
  "multi_chosen_decoded": [" said", " wrote", " noted"],
  "ftpo_pairs": [
    {"rejected": 42, "chosen": 100},
    {"rejected": 42, "chosen": 101},
    {"rejected": 42, "chosen": 102}
  ]
}
```

--------------------------------

### Accessing Configuration Values

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/types.md

Demonstrates common patterns for accessing values from the configuration dictionary, including providing default values or accessing required keys.

```python
config.get("vllm_port", 8000)  # with default
config["generation_max_prompts"]  # required key
```

--------------------------------

### Reducing Training Batch Size

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/errors.md

To mitigate Out of Memory (OOM) errors during training, reduce the `finetune_batch_size`. A value of 1 is often a good starting point.

```yaml
finetune_batch_size: 1
```

--------------------------------

### Orchestrate Full Anti-Slop Pipeline

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/orchestration.md

Executes the complete iterative pipeline for identifying and banning slop patterns. Use this to start or resume the entire process.

```python
def orchestrate_pipeline(
    config: Dict[str, Any],
    experiment_dir: Path,
    resume_mode: bool
) -> None:
    pass
```

```python
from pathlib import Path
from utils.config_loader import load_pipeline_config, merge_config_with_cli_args
from core.orchestration import orchestrate_pipeline

config = load_pipeline_config(Path("config.yaml"))
experiment_dir = Path("results/run_20250531_120000")

orchestrate_pipeline(config, experiment_dir, resume_mode=False)
```

--------------------------------

### FTPOTrainer.__init__

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/ftpo_trainer.md

Initializes the FTPO trainer, inheriting from trl.DPOTrainer. It customizes the trainer by setting remove_unused_columns to False and overriding the data_collator with ftpo_collator.

```APIDOC
## FTPOTrainer.__init__

### Description
Initialize FTPO trainer with required arguments. This method calls the parent DPOTrainer's initialization, sets `remove_unused_columns = False`, and overrides the `data_collator` with `ftpo_collator`.

### Method
__init__

### Parameters
* **(all DPOTrainer args)** - Standard DPO trainer arguments

### Returns
None

### Example Usage
```python
from core.ftpo_trainer import FTPOTrainer
from trl import DPOConfig
from transformers import AutoTokenizer, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("model_id")
tokenizer = AutoTokenizer.from_pretrained("model_id")

training_args = DPOConfig(
    output_dir="./output",
    learning_rate=1e-6,
    per_device_train_batch_size=1,
    max_length=3500,
    num_train_epochs=1,
    bf16=True,
)

trainer = FTPOTrainer(
    model=model,
    args=training_args,
    tokenizer=tokenizer,
    train_dataset=dataset,
    peft_config=lora_config,
)

trainer.train()
```
```

--------------------------------

### N-gram Analysis & Banning Parameters Example

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/configuration.md

Configure parameters for enabling and controlling n-gram analysis and banning, including thresholds for bigrams and trigrams found in corpora.

```yaml
enable_ngram_ban: true
top_k_bigrams: 5000
top_k_trigrams: 5000
dict_bigrams_initial: 400
dict_bigrams_subsequent: 70
nodict_bigrams_initial: 800
nodict_bigrams_subsequent: 100
dict_trigrams_initial: 300
dict_trigrams_subsequent: 50
nodict_trigrams_initial: 800
nodict_trigrams_subsequent: 100
extra_ngrams_to_ban: []
```

--------------------------------

### Skip Generation, Enable Fine-tuning

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/README.md

Configure the run to skip the generation phase and proceed directly to fine-tuning.

```bash
python main.py --generation-step-enabled false --run-finetune true
```

--------------------------------

### Training Parameters Configuration

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/configuration.md

Set training parameters like batch size, gradient accumulation, learning rate, and number of epochs. Enable auto-learning rate adjustment if needed.

```yaml
finetune_batch_size: 1
finetune_gradient_accumulation_steps: 16
finetune_warmup_ratio: 0.1
finetune_num_epochs: 1
finetune_learning_rate: 0.000001
finetune_auto_learning_rate: true
finetune_auto_learning_rate_adjustment_scaling: 0.15
finetune_gradient_checkpointing: "unsloth"
finetune_chat_template: ""
```

--------------------------------

### Use Custom Ban Lists and Whitelist

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/README.md

Configure custom n-grams, slop phrases, and whitelist strings within a YAML configuration.

```yaml
extra_ngrams_to_ban:
  - "voice barely whisper"
  - "testament to"

extra_slop_phrases_to_ban:
  - "nodded slowly"
  - "eyes widened"

whitelist_strings:
  - "think"
  - "thought"
```

--------------------------------

### Compute FTPO Loss

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/ftpo_trainer.md

Calculates the FTPO loss, which includes preference loss and optional MSE regularization on logits. Use this method to get the scalar loss for backpropagation.

```python
def compute_loss(
    self,
    model: torch.nn.Module,
    inputs: Dict[str, torch.Tensor],
    return_outputs: bool = False,
    **_
) -> Union[float, Tuple[float, Dict]]
```

```python
loss = trainer.compute_loss(model, batch)
# loss is a scalar tensor ready for backward()

# With outputs:
loss, outputs = trainer.compute_loss(model, batch, return_outputs=True)
```

--------------------------------

### Custom Configuration and Overrides

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/main.md

Run the pipeline using a custom configuration file and override specific parameters like the number of iterations and maximum prompts for generation.

```bash
python main.py --config-file configs/gemma-3-4b-it.yaml \
  --num-iterations 3 \
  --generation-max-prompts 2000
```

--------------------------------

### Check vLLM Server Logs and Model Accessibility

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/errors.md

Resolves vLLM server startup timeouts by checking logs and model accessibility. Consider increasing wait_timeout in config.

```bash
tail -f /tmp/vllm_*.log
```

```bash
huggingface-cli scan-cache
```

--------------------------------

### Verify Required Configuration Keys

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/errors.md

Ensures all necessary keys are present in the YAML configuration file or provided via CLI arguments.

```bash
cat auto_antislop_config.yaml | grep required_key
```

--------------------------------

### Fix Gemma-3 Checkpoint Key Naming

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/utilities.md

Repairs Gemma-3 checkpoint key naming in-place if keys start with 'language_model.' but not 'language_model.model.'. This function is a no-op if the model type is not gemma3 or keys are already correct. It performs atomic overwrites of shard files using temporary files.

```python
def fix_gemma3_checkpoint(ckpt_dir: str | Path) -> None:
    pass
```

```python
from utils.model_helpers import fix_gemma3_checkpoint
from pathlib import Path

fix_gemma3_checkpoint(Path("./gemma-3-4b-it"))
```

--------------------------------

### Command-Line Arguments with argparse

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/types.md

Demonstrates standard argparse types used in the main script for command-line argument parsing.

```python
parser.add_argument("--config-file", type=Path)
parser.add_argument("--num-iterations", type=int)
parser.add_argument("--vllm-port", type=int)
parser.add_argument("--manage-vllm", type=str2bool)
parser.add_argument("--finetune-cuda-visible-devices", type=str)
parser.add_argument("--finetune-mode", choices=["dpo", "ftpo"])
```

--------------------------------

### run_dpo_finetune

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/finetuning.md

Main entry point for fine-tuning the model using DPO or FTPO preference optimization. It handles model loading, dataset preparation, LoRA configuration, and trainer execution.

```APIDOC
## run_dpo_finetune

### Description
Main entry point for fine-tuning the model using DPO or FTPO preference optimization. It handles model loading, dataset preparation, LoRA configuration, and trainer execution.

### Method
```python
def run_dpo_finetune(config: dict, experiment_run_dir: Path) -> None
```

### Parameters
#### Path Parameters
- **config** (dict) - Required - Complete pipeline config with finetune_* keys
- **experiment_run_dir** (Path) - Required - Experiment directory containing preference pairs dataset

### Request Example
```python
from pathlib import Path
from core.finetuning import run_dpo_finetune

config = {
    "finetune_base_model_id": "unsloth/gemma-3-4b-it",
    "finetune_max_seq_length": 3500,
    "finetune_load_in_4bit": False,
    "finetune_mode": "ftpo",
    "finetune_lora_r": 128,
    "finetune_lora_alpha": 128,
    "finetune_target_modules": ["q_proj", "k_proj", "v_proj", "lm_head"],
    "finetune_freeze_early_layers": True,
    "finetune_n_layers_unfrozen": 5,
    "finetune_batch_size": 1,
    "finetune_gradient_accumulation_steps": 16,
    "finetune_num_epochs": 1,
    "finetune_learning_rate": 1e-6,
}

run_dpo_finetune(config, Path("results/run_20250531_120000"))
```

### Returns
None

### Source
`core/finetuning.py:362`
```

--------------------------------

### Minimal Pipeline Configuration

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/README.md

This YAML configuration provides the essential settings to run the Auto-Antislop pipeline. It includes experiment directories, model IDs, iteration counts, and vLLM server parameters.

```yaml
experiment_base_dir: "results/auto_antislop_runs"
human_profile_path: "data/human_writing_profile.json"
model_id: "unsloth/gemma-3-4b-it"
num_iterations: 2
log_level: "INFO"

manage_vllm: true
vllm_port: 8000
vllm_cuda_visible_devices: "0"
vllm_max_model_len: 2500
vllm_dtype: "bfloat16"

generation_api_key: "dummy"
generation_max_prompts: 1000
generation_hf_dataset_name: "Nitral-AI/Reddit-SFW-Writing_Prompts_ShareGPT"

finetune_enabled: true
finetune_mode: "ftpo"
finetune_lora_r: 128
finetune_batch_size: 1
finetune_learning_rate: 0.000001
```

--------------------------------

### Verifying Model Information

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/errors.md

Before loading a model, verify its existence and accessibility using the `huggingface-cli model-info` command. This is crucial for identifying issues with invalid model IDs or gated models.

```bash
huggingface-cli model-info <model_id>
```

--------------------------------

### Log in to Hugging Face CLI

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/README.md

Authenticate with the Hugging Face Hub using the CLI, necessary for accessing gated models.

```bash
huggingface-cli login
```

--------------------------------

### Initialize Git Submodules

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/errors.md

Ensures required git submodules are present. Run this if you encounter FileNotFoundError for submodules.

```bash
git clone --recurse-submodules https://github.com/sam-paech/auto-antislop.git
```

```bash
git submodule update --init --recursive
```

--------------------------------

### Create or Resume Experiment Directory

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/utilities.md

Use to create a new timestamped experiment directory or to validate and use an existing directory for resuming a run. Parent directories are created as needed.

```python
from pathlib import Path
from utils.fs_helpers import create_experiment_dir

exp_dir = create_experiment_dir(Path("results"), resume_dir=None)
# 	→ results/run_20250531_120000

# Resume existing run
exp_dir = create_experiment_dir(Path("results"), Path("results/run_20250530_100000"))
# 	→ results/run_20250530_100000 (if it exists)
```

--------------------------------

### Custom Configuration and Overrides

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/README.md

Run the script using a custom YAML configuration file and override specific parameters via command-line arguments.

```bash
python main.py -c configs/gemma-3-4b-it.yaml \
  --num-iterations 3 \
  --generation-max-prompts 2000
```

--------------------------------

### Resume Existing Run

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/main.md

Continue a previous auto-antislop run from a specified directory, preserving the experiment state and configuration.

```bash
python main.py --config-file configs/gemma-3-4b-it.yaml \
  --resume-from-dir results/auto_antislop_runs/run_20250531_120000
```

--------------------------------

### Specify Custom API Base URL

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/errors.md

Run the main script with a custom API base URL to connect to a remote vLLM server. This is a resolution for ConnectionError.

```bash
python main.py --generation-api-base-url http://remote-host:8888/v1
```

--------------------------------

### Load Pipeline Configuration

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/config_loader.md

Loads the main configuration from a specified YAML file path. Returns an empty dictionary if the file is not found or cannot be parsed. Logs successful loads and errors.

```python
from pathlib import Path
from utils.config_loader import load_pipeline_config

config = load_pipeline_config(Path("auto_antislop_config.yaml"))
print(f"Loaded {len(config)} config keys")
```

--------------------------------

### Main Function Signature

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/main.md

Defines the main entry point for the auto-antislop pipeline. It returns None and orchestrates the entire process.

```python
def main() -> None
```

--------------------------------

### Load Finetuning Libraries

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/finetuning.md

Lazily loads necessary transformer libraries and Unsloth if specified. Configures logging for quiet operation. Call this before other finetuning functions.

```python
from core.finetuning import load_imports

load_imports(use_unsloth=False)
```

--------------------------------

### Use External vLLM Server

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/README.md

Configure the script to connect to an externally managed vLLM server instead of managing it internally.

```bash
python main.py --manage-vllm false \
  --generation-api-base-url http://remote:8888/v1
```

--------------------------------

### Resume from Checkpoint

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/README.md

Continue a previous run from a saved checkpoint directory.

```bash
python main.py -r results/run_20250531_120000
```

--------------------------------

### Resume from Specific Directory

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/README.md

Resume a previous run by specifying the directory containing the run's results and checkpoints.

```bash
python main.py --resume-from-dir results/auto_antislop_runs/run_20250531_120000
```

--------------------------------

### WhitelistBuilder.write

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/utilities.md

Writes the provided whitelist entries to a JSON file. It handles directory creation and ensures the output is sorted and UTF-8 encoded.

```APIDOC
## WhitelistBuilder.write

### Description
Writes whitelist to a JSON file.

### Method Signature
```python
@classmethod
def WhitelistBuilder.write(cls, file_path: Path, whitelist: Iterable[str]) -> None
```

### Parameters
#### Path Parameters
- **file_path** (Path) - Required - Output JSON file path
- **whitelist** (Iterable[str]) - Required - Whitelist entries

### Returns
- **None**

### Behavior
- Creates parent directories as needed
- Writes sorted whitelist as JSON with 2-space indent
- Uses UTF-8 encoding
```

--------------------------------

### create_dpo_dataset

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/dpo.md

Creates a standard DPO dataset by pairing baseline and final-iteration generations. It matches rows by prompt_id, pairs generations, and writes the results to a JSONL file.

```APIDOC
## create_dpo_dataset

### Description
Creates a standard DPO (Direct Preference Optimization) dataset by pairing baseline and final-iteration generations.

### Method Signature
```python
def create_dpo_dataset(
    iter0_jsonl: Path,
    final_iter_jsonl: Path,
    output_jsonl: Path
) -> None
```

### Parameters
#### Path Parameters
- **iter0_jsonl** (Path) - Yes - Path to iteration 0 generation output
- **final_iter_jsonl** (Path) - Yes - Path to final iteration generation output
- **output_jsonl** (Path) - Yes - Path where DPO pairs will be written

### Response
#### Success Response
None

### Output Format
JSONL with one pair per line
```json
{
  "prompt": "Writing prompt: ...",
  "chosen": "Generation from final iteration (without slop)",
  "rejected": "Generation from iteration 0 (baseline with slop)"
}
```

### Example Usage
```python
from pathlib import Path
from core.dpo import create_dpo_dataset

create_dpo_dataset(
    iter0_jsonl=Path("results/iter_0_creative_writing_generations.jsonl"),
    final_iter_jsonl=Path("results/iter_1_creative_writing_generations.jsonl"),
    output_jsonl=Path("results/dpo_pairs_dataset.jsonl")
)
```
```

--------------------------------

### DPO/FTPO Beta Configuration

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/configuration.md

Set the global scale for preference loss. The `ftpo_beta` key is an alias for `finetune_beta` in the FTPO context.

```yaml
finetune_beta: 0.1
```

--------------------------------

### Debug with Verbose Logging

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/README.md

Enable detailed debug logging for troubleshooting.

```bash
python main.py --log-level DEBUG
```

--------------------------------

### FTPO Hyper-parameters Configuration

Source: https://github.com/sam-paech/auto-antislop/blob/main/README.md

This snippet shows the tunable hyper-parameters for FTPO training that can be set in the configuration file. It includes parameters for the preference loss scale, MSE loss terms for target and non-target tokens, and clipping epsilon for logits.

```yaml
# ── FTPO-specific hyper-parameters ─────────────────────────────────────────
# Leave any of these out (or set to null) to fall back to FTPOTrainer defaults.
ftpo_beta: 0.1                  # Global scale on pref loss (higher = steeper sigmoid).

# MSE loss term 1: light mse loss applied tokenwise on target tokens
ftpo_lambda_mse_target_tokenwise: 0.05  # Strength of MSE loss tether on the individual logits in the
                                        #   chosen+rejected set vs reference.
ftpo_tau_mse_target_tokenwise: 0.5      # Grace bandwidth (logits) before the above MSE loss kicks in.

# MSE loss term 2: stronger mse term applied to remaining (non-target) vocab
ftpo_lambda_mse: 0.4

ftpo_clip_epsilon_logits: 2     # For a chosen token: "after winning vs rejected token by this margin, preference loss turns off"

```

--------------------------------

### Check vLLM Server Logs

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/main.md

If the vLLM server is managed by the project, its logs can be monitored in real-time using the tail -f command on the log file.

```bash
tail -f /tmp/vllm_*.log
```

--------------------------------

### Ensure Antislop vLLM Configuration Exists

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/utilities.md

Copies the antislop-vllm configuration template to 'config.yaml' if 'config.yaml' does not already exist. This operation is non-fatal if the copy fails.

```python
from pathlib import Path
from utils.fs_helpers import ensure_antislop_vllm_config_exists

ensure_antislop_vllm_config_exists(Path("path/to/antislop-vllm"))
```

--------------------------------

### Skip Generation, Only Fine-tune

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/README.md

Disable the text generation step and proceed directly to the fine-tuning process.

```bash
python main.py --generation-step-enabled false
```

--------------------------------

### create_experiment_dir

Source: https://github.com/sam-paech/auto-antislop/blob/main/_autodocs/api-reference/utilities.md

Creates a timestamped experiment directory or validates an existing one for resume mode. It handles directory creation and validation for experiment tracking.

```APIDOC
## create_experiment_dir

### Description
Creates a timestamped experiment directory or validates an existing one for resume mode.

### Method
```python
def create_experiment_dir(base_dir_path: Path, resume_dir: Path | None = None) -> Path
```

### Parameters
#### Path Parameters
- **base_dir_path** (Path) - Yes - Base directory for timestamped runs
- **resume_dir** (Path | None) - No - None - If set, must be an existing directory to resume

### Returns
Path — experiment directory to use

### Throws
FileNotFoundError if resume_dir is provided but doesn't exist

### Behavior
- If resume_dir provided: validates it exists, returns it (no timestamp)
- If resume_dir is None: creates new `run_YYYYMMDD_HHMMSS` directory under base_dir_path
- Creates parent directories as needed
- Logs directory creation

### Example Usage
```python
from pathlib import Path
from utils.fs_helpers import create_experiment_dir

exp_dir = create_experiment_dir(Path("results"), resume_dir=None)
# → results/run_20250531_120000

# Resume existing run
exp_dir = create_experiment_dir(Path("results"), Path("results/run_20250530_100000"))
# → results/run_20250530_100000 (if it exists)
```
```