### Install Python Dependencies

Source: https://github.com/eric-mitchell/detect-gpt/blob/main/README.md

Installs the necessary Python packages for the project. Ensure you are in a virtual environment before running.

```bash
python3 -m venv env
source env/bin/activate
pip install -r requirements.txt
```

--------------------------------

### Run Basic Detection Experiment

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

Execute a standard DetectGPT experiment using GPT-2 XL as the base model and T5-3b for perturbations on the XSum dataset. Configure sample count, perturbation levels, and word masking.

```bash
python run.py \
    --base_model_name gpt2-xl \
    --mask_filling_model_name t5-3b \
    --dataset xsum \
    --dataset_key document \
    --n_samples 200 \
    --n_perturbation_list 1,10,100 \
    --pct_words_masked 0.3 \
    --span_length 2 \
    --batch_size 50
```

--------------------------------

### Run Detection on WritingPrompts Dataset

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

Execute a DetectGPT experiment using the WritingPrompts dataset. Ensure the dataset is downloaded and placed in the `data/writingPrompts/` directory. Uses GPT-2 XL and T5-3b.

```bash
python run.py \
    --base_model_name gpt2-xl \
    --mask_filling_model_name t5-3b \
    --dataset writing \
    --n_samples 500 \
    --n_perturbation_list 1,10,100 \
    --pct_words_masked 0.3 \
    --span_length 2
```

--------------------------------

### Run DetectGPT Experiments via CLI

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

Execute DetectGPT experiments using the run.py script with various configuration flags.

```bash
python run.py \
    --base_model_name gpt2-xl \
    --mask_filling_model_name t5-3b \
    --dataset xsum \
    --n_samples 500 \
    --skip_baselines
```

```bash
# GPT-NeoX-20B with T5-11b (requires significant GPU memory)
python run.py \
    --base_model_name EleutherAI/gpt-neox-20b \
    --mask_filling_model_name t5-11b \
    --dataset xsum \
    --n_samples 500 \
    --n_perturbation_list 1,10,100 \
    --pct_words_masked 0.3 \
    --span_length 2 \
    --batch_size 20

# Enable 8-bit quantization for memory efficiency
python run.py \
    --base_model_name EleutherAI/gpt-neox-20b \
    --mask_filling_model_name t5-11b \
    --dataset xsum \
    --n_samples 500 \
    --int8 \
    --batch_size 20

# Half precision (fp16) inference
python run.py \
    --base_model_name EleutherAI/gpt-j-6B \
    --mask_filling_model_name t5-3b \
    --dataset xsum \
    --n_samples 500 \
    --half \
    --batch_size 50
```

```bash
# Generate with GPT-2 XL, score with GPT-Neo-2.7B
python run.py \
    --base_model_name gpt2-xl \
    --scoring_model_name EleutherAI/gpt-neo-2.7B \
    --mask_filling_model_name t5-3b \
    --dataset xsum \
    --n_samples 500 \
    --n_perturbation_list 1,10,100 \
    --pct_words_masked 0.3 \
    --span_length 2 \
    --output_name cross_model
```

--------------------------------

### Run Detection with Top-k Sampling

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

Execute a detection experiment using the EleutherAI/gpt-j-6B model with top-k sampling. Configure the number of samples, perturbation list, and masking parameters.

```bash
python run.py \
    --base_model_name EleutherAI/gpt-j-6B \
    --mask_filling_model_name t5-3b \
    --dataset xsum \
    --n_samples 500 \
    --n_perturbation_list 1,10,100 \
    --pct_words_masked 0.3 \
    --span_length 2 \
    --do_top_k \
    --top_k 40
```

--------------------------------

### Run Detection with Top-p Sampling

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

Perform a detection experiment using the EleutherAI/gpt-neo-2.7B model with top-p (nucleus) sampling enabled. Adjust sample count and perturbation levels as needed.

```bash
python run.py \
    --base_model_name EleutherAI/gpt-neo-2.7B \
    --mask_filling_model_name t5-3b \
    --dataset xsum \
    --n_samples 500 \
    --n_perturbation_list 1,10,100 \
    --pct_words_masked 0.3 \
    --span_length 2 \
    --do_top_p \
    --top_p 0.96
```

--------------------------------

### Load Custom Datasets with custom_datasets.py

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

Demonstrates how to load different text corpora using the `custom_datasets` module. It shows how to access available datasets and load specific ones like 'pubmed', 'english', 'german', and 'writing'.

```python
import custom_datasets

# Available datasets
print(custom_datasets.DATASETS)  # ['writing', 'english', 'german', 'pubmed']

# Load PubMed Q&A data
cache_dir = "~/.cache"
pubmed_data = custom_datasets.load('pubmed', cache_dir)
# Returns: ['Question: {q} Answer:<<<SEP>>>{a}', ...]

# Load English text from WMT16 translation dataset
# Returns documents with 100-150 words
english_data = custom_datasets.load('english', cache_dir)

# Load German text from WMT16
german_data = custom_datasets.load('german', cache_dir)

# Load WritingPrompts (requires local data)
# Expects: data/writingPrompts/valid.wp_source and valid.wp_target
writing_data = custom_datasets.load('writing', cache_dir)
```

--------------------------------

### Run Baseline Experiments Only

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

This command executes only the baseline detection experiments, skipping the DetectGPT perturbation-based method. It specifies the dataset and sample size, and the output will include results for various baselines.

```bash
# Run only baseline experiments (skip perturbation-based DetectGPT)
python run.py \
    --base_model_name gpt2-xl \
    --mask_filling_model_name t5-3b \
    --dataset xsum \
    --n_samples 500 \
    --baselines_only

# Output includes:
# - likelihood_threshold_results.json (log-prob threshold baseline)
# - rank_threshold_results.json (token rank baseline)
# - logrank_threshold_results.json (log rank baseline)
# - entropy_threshold_results.json (entropy baseline)
# - roberta-base-openai-detector_results.json (supervised)
# - roberta-large-openai-detector_results.json (supervised)
```

--------------------------------

### Run Detection with OpenAI Models

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

Conduct a detection experiment using OpenAI's davinci model for text generation and log probability scoring. Requires an API key and specifies T5-11b for mask filling. Note API token usage.

```bash
python run.py \
    --output_name openai \
    --openai_model davinci \
    --openai_key "your-api-key-here" \
    --mask_filling_model_name t5-11b \
    --dataset pubmed \
    --n_samples 150 \
    --n_perturbation_list 1,10,75 \
    --pct_words_masked 0.3 \
    --span_length 2 \
    --do_top_p \
    --top_p 0.9 \
    --mask_top_p 0.95 \
    --batch_size 5
```

--------------------------------

### Run Detection on SQuAD Dataset

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

Perform a detection experiment using the SQuAD dataset (Wikipedia contexts). Configure the base model, mask-filling model, sample count, and perturbation list.

```bash
python run.py \
    --base_model_name gpt2-xl \
    --mask_filling_model_name t5-3b \
    --dataset squad \
    --dataset_key context \
    --n_samples 312 \
    --n_perturbation_list 1,10,100 \
    --pct_words_masked 0.3 \
    --span_length 2
```

--------------------------------

### Run DetectGPT with PubMed Dataset

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

This command initiates a DetectGPT run using the PubMed dataset, specifying models, sample count, perturbation levels, and masking parameters.

```bash
python run.py \
    --base_model_name gpt2-xl \
    --mask_filling_model_name t5-3b \
    --dataset pubmed \
    --n_samples 500 \
    --n_perturbation_list 1,10,100 \
    --pct_words_masked 0.3 \
    --span_length 2
```

--------------------------------

### Compute Batch Log Likelihoods

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

Provides a function to efficiently compute log likelihoods for a list of texts by processing them in batches. This is a common requirement for performance optimization in language model analysis.

```python
# Batch processing for efficiency
def get_lls(texts):
    return [get_ll(text) for text in texts]

# The detection criterion compares:
# original_ll - perturbed_ll  (d criterion)
# or normalized: (original_ll - perturbed_ll) / perturbed_ll_std  (z criterion)
```

--------------------------------

### Evaluate Supervised Detectors

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

Evaluate pre-trained transformer classifiers on original versus sampled text data.

```python
def eval_supervised(data, model='roberta-base-openai-detector'):
    """
    Evaluate a pre-trained detector on original vs sampled texts

    Args:
        data: dict with 'original' (human) and 'sampled' (machine) text lists
        model: HuggingFace model name for sequence classification

    Returns metrics including ROC AUC and PR AUC
    """
    detector = transformers.AutoModelForSequenceClassification.from_pretrained(model)
    tokenizer = transformers.AutoTokenizer.from_pretrained(model)

    # Get predictions - class 0 = human, class 1 = machine
    real_preds = []  # P(human) scores for human text
    fake_preds = []  # P(human) scores for machine text

    for batch in data['original']:
        inputs = tokenizer(batch, padding=True, truncation=True, max_length=512)
        real_preds.extend(detector(**inputs).logits.softmax(-1)[:,0].tolist())

    # Compute ROC and PR curves
    fpr, tpr, roc_auc = get_roc_metrics(real_preds, fake_preds)

    return {
        'name': model,
        'metrics': {'roc_auc': roc_auc, 'fpr': fpr, 'tpr': tpr},
        'pr_metrics': {'pr_auc': pr_auc, 'precision': p, 'recall': r}
    }
```

--------------------------------

### Full Text Perturbation Pipeline

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

Orchestrates the complete text perturbation process, including tokenization, masking, mask replacement using a T5 model, and applying the generated fills back to the original text. It processes texts in batches for efficiency.

```python
# Full perturbation pipeline
def perturb_texts(texts, span_length=2, pct=0.3):
    """Generate perturbed versions in batches"""
    outputs = []
    for i in range(0, len(texts), chunk_size):
        batch = texts[i:i + chunk_size]
        masked = [tokenize_and_mask(x, span_length, pct) for x in batch]
        raw_fills = replace_masks(masked)
        fills = extract_fills(raw_fills)
        perturbed = apply_extracted_fills(masked, fills)
        outputs.extend(perturbed)
    return outputs
```

--------------------------------

### Tokenize and Mask Text for Perturbation

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

This function prepares text for perturbation by masking approximately 30% of words in spans of a specified length. It inserts special tokens like '<extra_id_N>' which are used by T5 models for filling.

```python
# Tokenize text and insert mask tokens
def tokenize_and_mask(text, span_length=2, pct=0.3, ceil_pct=False):
    """
    Mask ~30% of words with span_length=2 word spans
    Returns text like: "The quick <extra_id_0> jumps over <extra_id_1> dog."
    """
    tokens = text.split(' ')
    n_spans = int(pct * len(tokens) / (span_length + buffer_size * 2))
    # Randomly select non-overlapping spans to mask
    # Replace with <extra_id_N> tokens for T5
    return masked_text
```

--------------------------------

### Compute Single Text Log Likelihood

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

Calculates the log likelihood of a given text using a pre-loaded base model and tokenizer. The function returns the negative cross-entropy loss, where a higher value indicates a more likely text.

```python
# Single text log likelihood (using loaded base_model and base_tokenizer)
def get_ll(text):
    with torch.no_grad():
        tokenized = base_tokenizer(text, return_tensors="pt").to(DEVICE)
        labels = tokenized.input_ids
        # Returns negative cross-entropy loss (higher = more likely)
        return -base_model(**tokenized, labels=labels).loss.item()

# Example usage within the framework:
# text = "The quick brown fox jumps over the lazy dog."
# log_likelihood = get_ll(text)  # e.g., -3.245
```

--------------------------------

### Compute ROC Metrics

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

Calculate ROC curve and AUC for detection tasks using scikit-learn.

```python
def get_roc_metrics(real_preds, sample_preds):
    """
    Compute ROC curve for detection task

    Args:
        real_preds: Detection scores for human text (label=0)
        sample_preds: Detection scores for machine text (label=1)

    Higher scores should indicate machine-generated text
    """
    from sklearn.metrics import roc_curve, auc

    labels = [0] * len(real_preds) + [1] * len(sample_preds)
    scores = real_preds + sample_preds

    fpr, tpr, thresholds = roc_curve(labels, scores)
    roc_auc = auc(fpr, tpr)

    return fpr.tolist(), tpr.tolist(), float(roc_auc)
```

--------------------------------

### Replace Masks using T5 Model

Source: https://context7.com/eric-mitchell/detect-gpt/llms.txt

This function utilizes a T5 model to generate replacements for masked spans in the input text. It takes a list of masked texts and returns the decoded output from the T5 model, which includes the filled spans.

```python
# Replace masks using T5 model
def replace_masks(texts):
    """
    Use T5 to generate fills for masked spans
    Input: ["The quick <extra_id_0> jumps over <extra_id_1> dog."]
    T5 output: "<extra_id_0> brown fox <extra_id_1> the lazy"
    Result: "The quick brown fox jumps over the lazy dog."
    """
    tokens = mask_tokenizer(texts, return_tensors="pt", padding=True).to(DEVICE)
    outputs = mask_model.generate(**tokens, max_length=150, do_sample=True, top_p=0.96)
    return mask_tokenizer.batch_decode(outputs, skip_special_tokens=False)
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.