### Instantiate Prompt and Get Responses

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/06-types.md

Example of creating a Prompt object and using it to get model responses.

```python
from heretic.utils import Prompt

prompt = Prompt(
    system="You are a helpful assistant.",
    user="Explain quantum computing in simple terms.",
)

responses = model.get_responses([prompt])
```

--------------------------------

### Minimal Heretic Model Example

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/08-programmatic-usage.md

A minimal example demonstrating how to load settings and a model, then generate a response using a prompt.

```python
from heretic.config import Settings
from heretic.model import Model
from heretic.utils import Prompt

# Load settings and model
settings = Settings(model="Qwen/Qwen3-4B-Instruct-2507")
model = Model(settings)

# Generate a response
prompt = Prompt(
    system="You are a helpful assistant.",
    user="What is 2+2?"
)
responses = model.get_responses([prompt])
print(responses[0])
```

--------------------------------

### Complete Heretic Workflow Example

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/08-programmatic-usage.md

This comprehensive example demonstrates the full Heretic workflow: loading settings and models, computing residuals, analyzing geometry, computing refusal directions, evaluating the baseline, optimizing parameters with Optuna, and exporting the best model.

```python
import torch
import optuna
from heretic.config import Settings
from heretic.model import Model, AbliterationParameters
from heretic.evaluator import Evaluator
from heretic.analyzer import Analyzer
from heretic.utils import load_prompts, Prompt, set_seed, get_trial_parameters

# Configure
set_seed(42)
settings = Settings(
    model="Qwen/Qwen3-4B-Instruct-2507",
    n_trials=200,
    quantization="bnb_4bit",
)

# Load
print("Loading model...")
model = Model(settings)

print("Loading datasets...")
good_prompts = load_prompts(settings, settings.good_prompts)
bad_prompts = load_prompts(settings, settings.bad_prompts)

print("Computing residuals...")
good_residuals = model.get_residuals_batched(good_prompts)
bad_residuals = model.get_residuals_batched(bad_prompts)

# Analyze
print("Analyzing geometry...")
analyzer = Analyzer(settings, model, good_residuals, bad_residuals)
if settings.print_residual_geometry:
    analyzer.print_residual_geometry()

# Compute refusal directions
refusal_directions = bad_residuals.mean(dim=0) - good_residuals.mean(dim=0)
refusal_directions = torch.nn.functional.normalize(refusal_directions, p=2, dim=1)

# Evaluate baseline
print("Evaluating baseline...")
evaluator = Evaluator(settings, model)
print(f"Baseline refusals: {evaluator.base_refusals}/{len(evaluator.bad_prompts)}")

# Optimize
print("Starting optimization...")

def objective(trial: optuna.Trial) -> tuple[float, float]:
    params = get_trial_parameters(trial, model)
    model.reset_model()
    
    model.abliterate(
        refusal_directions=refusal_directions,
        direction_index=trial.suggest_float("direction_index", -1, len(model.get_layers())),
        parameters=params,
    )
    
    (kld_score, ref_score), _, _ = evaluator.get_score()
    return kld_score, ref_score

study = optuna.create_study(
    directions=["minimize", "minimize"],
    sampler=optuna.samplers.TPESampler(seed=settings.seed),
)
study.optimize(objective, n_trials=settings.n_trials)

# Export best
print("Exporting best model...")
best_trial = study.best_trial
best_params = get_trial_parameters(best_trial, model)

model.reset_model()
model.abliterate(
    refusal_directions=refusal_directions,
    direction_index=best_trial.params.get("direction_index", 0),
    parameters=best_params,
)

merged = model.get_merged_model()
merged.save_pretrained("abliterated-model")
model.tokenizer.save_pretrained("abliterated-model")

print("Done!")
```

--------------------------------

### Initialize Model Instance

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Example of creating a Model instance with provided settings. Ensure necessary imports are present.

```python
from heretic.config import Settings
from heretic.model import Model

settings = Settings(model="Qwen/Qwen3-4B-Instruct-2507")
model = Model(settings)
```

--------------------------------

### Inspect Layer Components

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Example of getting and printing the number of modules for specific components within a layer.

```python
layer_modules = model.get_layer_modules(0)

for component, modules in layer_modules.items():
    print(f"{component}: {len(modules)} modules")
```

--------------------------------

### Install Heretic LLM

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/08-programmatic-usage.md

Install the Heretic LLM library using pip. For research features, install with the 'research' extra.

```bash
pip install heretic-llm
```

```bash
pip install heretic-llm[research]
```

--------------------------------

### Example TOML Configuration File

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/02-configuration.md

A sample TOML configuration file demonstrating how to set up model, quantization, dataset paths, and other parameters for Heretic.

```toml
# config.toml
model = "Qwen/Qwen3-4B-Instruct-2507"
quantization = "bnb_4bit"
batch_size = 0  # auto
n_trials = 200
n_startup_trials = 60
seed = 42

row_normalization = "full"
orthogonalize_direction = true
kl_divergence_scale = 1.0
kl_divergence_target = 0.01

system_prompt = "You are a helpful assistant."

[good_prompts]
dataset = "mlabonne/harmless_alpaca"
split = "train[:400]"
column = "text"

[bad_prompts]
dataset = "mlabonne/harmful_behaviors"
split = "train[:400]"
column = "text"

[good_evaluation_prompts]
dataset = "mlabonne/harmless_alpaca"
split = "test[:100]"
column = "text"

[bad_evaluation_prompts]
dataset = "mlabonne/harmful_behaviors"
split = "test[:100]"
column = "text"
```

--------------------------------

### List Abliterable Components

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Example of retrieving and displaying the list of abiterable components in the model.

```python
components = model.get_abliterable_components()
# ["attn.o_proj", "mlp.down_proj"]
```

--------------------------------

### Instantiate AbliterationParameters

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/06-types.md

Example of creating an instance of AbliterationParameters with specific values for model ablation.

```python
from heretic.model import AbliterationParameters

params = AbliterationParameters(
    max_weight=0.85,
    max_weight_position=15.2,
    min_weight=0.15,
    min_weight_distance=6.5,
)
```

--------------------------------

### Configuration File Example

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/07-main-cli.md

Specifies default model, quantization, trial counts, and prompt datasets for Heretic. Load order is CLI args > environment variables > config.toml.

```toml
model = "Qwen/Qwen3-4B-Instruct-2507"
quantization = "bnb_4bit"
n_trials = 200
n_startup_trials = 60
seed = 42

[good_prompts]
dataset = "mlabonne/harmless_alpaca"
split = "train[:400]"
column = "text"

[bad_prompts]
dataset = "mlabonne/harmful_behaviors"
split = "train[:400]"
column = "text"
```

--------------------------------

### Heretic integration example

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/04-analyzer-evaluator.md

Demonstrates the complete workflow of loading settings, models, prompts, analyzing residuals, and evaluating performance using Heretic components.

```python
from heretic.config import Settings
from heretic.model import Model
from heretic.analyzer import Analyzer
from heretic.evaluator import Evaluator
from heretic.utils import load_prompts

# Load settings and model
settings = Settings(model="Qwen/Qwen3-4B-Instruct-2507")
model = Model(settings)

# Load datasets and compute residuals
good_prompts = load_prompts(settings, settings.good_prompts)
bad_prompts = load_prompts(settings, settings.bad_prompts)

print("Computing residuals...")
good_residuals = model.get_residuals_batched(good_prompts)
bad_residuals = model.get_residuals_batched(bad_prompts)

# Analyze geometry
analyzer = Analyzer(settings, model, good_residuals, bad_residuals)
if settings.print_residual_geometry:
    analyzer.print_residual_geometry()

if settings.plot_residuals:
    analyzer.plot_residuals()

# Evaluate baseline
evaluator = Evaluator(settings, model)

print(f"Baseline refusals: {evaluator.base_refusals}/{len(evaluator.bad_prompts)}")

# After abliteration, score
(
    kld_score,
    ref_score
), kl_div, ref_count = evaluator.get_score()
print(f"Score: KLD={kld_score:.4f}, Refusals={ref_score:.4f}")
print(f"KL Divergence: {kl_div:.4f}")
print(f"Refusals: {ref_count}/{len(evaluator.bad_prompts)}")
```

--------------------------------

### Install Heretic LLM

Source: https://github.com/p-e-w/heretic/blob/master/README.md

Install the Heretic LLM package using pip. Ensure you have Python 3.10+ and PyTorch 2.2+ installed.

```bash
pip install -U heretic-llm
```

--------------------------------

### Install Heretic LLM with Research Extras

Source: https://github.com/p-e-w/heretic/blob/master/README.md

Install Heretic LLM with the 'research' extra to enable advanced interpretability features. This requires Python 3.10+ and PyTorch 2.2+.

```bash
pip install -U heretic-llm[research]
```

--------------------------------

### Adding Custom Components for Qwen3.5 MoE

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/09-architecture-algorithms.md

Example of how to extend the system to support new component types, specifically demonstrating the addition of Qwen3.5 MoE hybrid components by trying to add 'attn.o_proj'.

```python
# Qwen3.5 MoE hybrid
with suppress(Exception):
    try_add("attn.o_proj", layer.linear_attn.out_proj)
```

--------------------------------

### Pydantic Configuration Validation Example

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/06-types.md

Demonstrates how to use Pydantic's ValidationError to catch invalid configuration settings for Heretic. This example shows settings that will fail validation due to incorrect values.

```python
from heretic.config import Settings
from pydantic import ValidationError

try:
    settings = Settings(
        model="invalid model",  # Will fail
        n_trials=-1,  # Will fail
    )
except ValidationError as e:
    print(e)
```

--------------------------------

### Basic Programmatic Usage Example

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/INDEX.md

Demonstrates the basic programmatic usage of Heretic by initializing settings, loading a model, creating a prompt, and generating responses.

```python
from heretic.config import Settings
from heretic.model import Model
from heretic.utils import Prompt

settings = Settings(model="Qwen/Qwen3-4B-Instruct-2507")
model = Model(settings)

prompt = Prompt(system="You are helpful.", user="What is 2+2?")
responses = model.get_responses([prompt])
```

--------------------------------

### Complete Heretic Workflow Example

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/05-utilities.md

Demonstrates a complete workflow including hardware check, model loading, prompt processing, and memory management. Requires importing various components from the heretic library.

```python
from heretic.config import Settings
from heretic.model import Model
from heretic.utils import (
    load_prompts,
    set_seed,
    print_memory_usage,
    format_duration,
    is_hf_path,
)
from heretic.system import empty_cache, get_accelerator_info
import time

# Check hardware
device_type, device_count = get_accelerator_info()
print(f"Using {device_count} {device_type} devices")

# Set seed for reproducibility
set_seed(42)

# Load configuration
settings = Settings(
    model="Qwen/Qwen3-4B-Instruct-2507",
    n_trials=200,
)

# Load model
start = time.time()
model = Model(settings)
elapsed = time.time() - start
print(f"Model loaded in {format_duration(elapsed)}")

# Load datasets
good_prompts = load_prompts(settings, settings.good_prompts)
bad_prompts = load_prompts(settings, settings.bad_prompts)

print(f"Loaded {len(good_prompts)} good prompts, {len(bad_prompts)} bad prompts")

# Check memory
print_memory_usage()

# Process prompts
residuals = model.get_residuals_batched(good_prompts)

# Clean up
empty_cache()
print_memory_usage()
```

--------------------------------

### Install research dependencies for Analyzer

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/04-analyzer-evaluator.md

Installs optional research dependencies required for the analyzer module, including plotting and geometric computation libraries.

```bash
pip install heretic-llm[research]
```

--------------------------------

### Basic Abliteration CLI

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/07-main-cli.md

Starts the Heretic abliteration process with a specified model. Prompts for batch size, save options, and more.

```bash
heretic "Qwen/Qwen3-4B-Instruct-2507"
```

--------------------------------

### Heretic CLI with Full Options

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/07-main-cli.md

Example of running the Heretic CLI with various options specified, including model, quantization, batch size, optimization trials, and export strategy.

```bash
heretic \
    --model "Qwen/Qwen3-4B-Instruct-2507" \
    --quantization bnb_4bit \
    --batch-size 32 \
    --n-trials 200 \
    --n-startup-trials 60 \
    --seed 42 \
    --export-strategy merge
```

--------------------------------

### Get Readme Introduction Markdown

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/05-utilities.md

Generates a Markdown introduction string for a model card README file. Typically used to describe the model's origin or modification process.

```python
def get_readme_intro(settings: Settings) -> str:
    pass
```

```python
intro = get_readme_intro(settings)
# "This model was abliterated using Heretic..."
```

--------------------------------

### Initialize and Use Evaluator

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/INDEX.md

Instantiate the Evaluator with settings and a model. Use its methods to check for refusals, count refusals in a batch, or get a comprehensive score.

```python
from heretic.evaluator import Evaluator

evaluator = Evaluator(settings, model)

is_refusal = evaluator.is_refusal(response)
refusals = evaluator.count_refusals()
(kld_score, ref_score), kl_div, refusals = evaluator.get_score()
```

--------------------------------

### Access Model Layers

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Example of calling get_layers and determining the number of layers in the model.

```python
layers = model.get_layers()
num_layers = len(layers)
```

--------------------------------

### Error Handling Example

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/07-main-cli.md

Illustrates Heretic's detailed error messages for common issues like CUDA out of memory or NaN logits during model loading. Suggests solutions like reducing batch size or using quantization.

```text
[red]Error[/]: Failed to load model with all configured dtypes.
* Trying dtype auto... [red]Failed[/]: CUDA out of memory
* Trying dtype float16... [red]Failed[/]: NaN in logits
* Trying dtype bfloat16... [red]Failed[/]: ...
* Trying dtype float32... [red]Failed[/]: ...

Try reducing batch size, using quantization, or increasing max_memory.
```

--------------------------------

### Get Trial Parameters for Abliteration

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/05-utilities.md

Samples Optuna trial parameters for each abliterable component of a model and constructs corresponding parameter objects. Useful for hyperparameter optimization.

```python
def get_trial_parameters(trial: Trial, model: Model) -> dict[str, AbliterationParameters]:
    pass
```

```python
from heretic.utils import get_trial_parameters

params = get_trial_parameters(trial, model)
# {"attn.o_proj": AbliterationParameters(...), "mlp.down_proj": AbliterationParameters(...)}
```

--------------------------------

### Load Heretic Settings from Environment Variables

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/08-programmatic-usage.md

Configure Heretic settings by setting environment variables before initializing the Settings object. This example sets the model, quantization, and batch size.

```python
import os
from heretic.config import Settings

os.environ["HERETIC_MODEL"] = "Qwen/Qwen3-4B-Instruct-2507"
os.environ["HERETIC_QUANTIZATION"] = "bnb_4bit"
os.environ["HERETIC_BATCH_SIZE"] = "32"

settings = Settings()
```

--------------------------------

### Reproducibility Information JSON

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/07-main-cli.md

Example structure of `reproduce.json` saved after abliteration, containing version, settings, parameters, and metrics for exact regeneration.

```json
{
  "heretic_version": "1.3.0",
  "datetime": "2025-06-13T10:30:00Z",
  "model": "Qwen/Qwen3-4B-Instruct-2507",
  "settings": { /* full settings object */ },
  "best_trial_parameters": { /* per-component parameters */ },
  "best_trial_value": { /* score metrics */ },
  "study_checkpoint_dir": "checkpoints/qwen3_4b_instruct_2507/"
}
```

--------------------------------

### Tracking Library Versions

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/08-programmatic-usage.md

Retrieve and print the versions of installed libraries like Heretic and PyTorch. This is important for ensuring compatibility and documenting the environment used for experiments.

```python
from importlib.metadata import version

heretic_version = version("heretic-llm")
pytorch_version = version("torch")

print(f"Heretic: {heretic_version}")
print(f"PyTorch: {pytorch_version}")
```

--------------------------------

### Load Heretic Model

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/08-programmatic-usage.md

Load a Heretic model based on the provided settings. This example prints the number of layers in the loaded model.

```python
from heretic.model import Model
from heretic.config import Settings

settings = Settings(model="Qwen/Qwen3-4B-Instruct-2507")
model = Model(settings)

print(f"Loaded {len(model.get_layers())} layers")
```

--------------------------------

### Floating-Point Direction Index Example

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/09-architecture-algorithms.md

Illustrates the use of a floating-point direction index for continuous search over infinite directions in parameter space. This specific value interpolates between two layer directions.

```python
direction_index = 7.5
→ interpolate between layer_7_direction and layer_8_direction
```

--------------------------------

### Get Log Probabilities

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Computes log-probabilities for a list of prompts. Generates one token per prompt and computes the log-softmax over logits. Can move results to CPU if specified.

```python
def get_logprobs(self, prompts: list[Prompt]) -> Tensor:
    pass
```

```python
logprobs = model.get_logprobs(good_prompts)
print(logprobs.shape)  # torch.Size([100, 151936])
```

```python
# Compute KL divergence between two distributions
kl = F.kl_div(logprobs, baseline_logprobs, reduction="batchmean", log_target=True)
```

--------------------------------

### Generate Multiple Responses with Heretic

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/08-programmatic-usage.md

Generate responses for a list of prompts. This example shows how to iterate through prompts and their corresponding generated responses.

```python
from heretic.utils import Prompt

prompts = [
    Prompt(system="You are helpful.", user="What is AI?"),
    Prompt(system="You are helpful.", user="Explain ML."),
]

responses = model.get_responses(prompts)
for prompt, response in zip(prompts, responses):
    print(f"Q: {prompt.user}")
    print(f"A: {response}\n")
```

--------------------------------

### Get Responses in Batches

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Generates responses for a large list of prompts by splitting them into batches. Use this method to manage memory and improve performance for extensive prompt sets.

```python
def get_responses_batched(
    self,
    prompts: list[Prompt],
    skip_special_tokens: bool = False,
) -> list[str]:
    pass
```

```python
large_prompt_list = [...]  # 10,000 prompts

responses = model.get_responses_batched(large_prompt_list)
```

--------------------------------

### Get Accelerator Hardware Information

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/05-utilities.md

Retrieves information about the available hardware accelerators. Returns a tuple containing the device type (e.g., 'cuda', 'cpu') and the number of devices.

```python
def get_accelerator_info() -> tuple[str, int]:
    pass
```

```python
device_type, device_count = get_accelerator_info()
# ("cuda", 2)
```

--------------------------------

### Get Transformer Layers

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Retrieves the PyTorch ModuleList of all transformer layers from the model.

```python
def get_layers(self) -> ModuleList:
    pass
```

--------------------------------

### Initialize and Use Analyzer

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/INDEX.md

Instantiate the Analyzer with settings, a model, and residual data. Use it to print residual geometry metrics or generate PaCMAP visualizations.

```python
from heretic.analyzer import Analyzer

analyzer = Analyzer(settings, model, good_residuals, bad_residuals)

analyzer.print_residual_geometry()  # Metrics table
analyzer.plot_residuals()            # PaCMAP visualizations
```

--------------------------------

### Get Abliterable Components

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Returns a sorted list of component names that are eligible for ablation.

```python
def get_abliterable_components(self) -> list[str]:
    pass
```

--------------------------------

### Seeding for Reproducibility

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/09-architecture-algorithms.md

Demonstrates how to set seeds for various libraries (random, numpy, torch, optuna) to ensure deterministic behavior. It also shows how to manually seed PyTorch operations before specific steps like SVD.

```python
set_seed(settings.seed)  # Seeds random, numpy, torch, optuna
torch.manual_seed(...)  # Before SVD in full normalization
```

--------------------------------

### Get Batched Log Probabilities

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Computes log-probabilities for prompts by processing them in batches and concatenating the results.

```python
def get_logprobs_batched(self, prompts: list[Prompt]) -> Tensor:
    pass
```

--------------------------------

### Heretic CLI Entry Point

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/07-main-cli.md

The basic command to run the Heretic CLI. Configuration is typically managed via `pyproject.toml`.

```bash
heretic [OPTIONS]
```

```toml
heretic = "heretic.main:main"
```

--------------------------------

### get_readme_intro

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/05-utilities.md

Generates a markdown introduction for a model card README, typically used for abliterated models.

```APIDOC
## get_readme_intro

### Description
Generates a markdown introduction for a model card README, typically used for abliterated models.

### Signature
```python
def get_readme_intro(settings: Settings) -> str:
    pass
```

### Parameters
* `settings` (Settings) - Configuration settings object

### Returns
Markdown introduction for model card README.

### Example
```python
intro = get_readme_intro(settings)
# "This model was abliterated using Heretic..."
```
```

--------------------------------

### Get MPS Driver Version

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/05-utilities.md

Retrieves the Apple Silicon macOS version (MPS driver version).

```python
def get_mps_driver_version() -> str | None:
    pass
```

--------------------------------

### Get NPU Driver Version

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/05-utilities.md

Retrieves the Huawei NPU driver version. Returns None if not available.

```python
def get_npu_driver_version() -> str | None:
    pass
```

--------------------------------

### Get XPU Driver Version

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/05-utilities.md

Retrieves the Intel XPU driver version. Returns None if not available.

```python
def get_xpu_driver_version() -> str | None:
    pass
```

--------------------------------

### Reproduce Previous Run CLI

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/07-main-cli.md

Recreates a previous Heretic run by loading settings from a `reproduce.json` file hosted online.

```bash
heretic \
    --model "Qwen/Qwen3-4B-Instruct-2507" \
    --reproduce "https://huggingface.co/p-e-w/gemma-3-12b-it-heretic/blob/main/reproduce.json"
```

--------------------------------

### Get Model Class

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Determines and returns the appropriate model class (text-only or multimodal) based on the provided model identifier.

```python
def get_model_class(
    model: str,
) -> Type[AutoModelForImageTextToText] | Type[AutoModelForCausalLM]:
    pass
```

```python
model_class = get_model_class("Qwen/Qwen-VL-Chat")
# Returns: AutoModelForImageTextToText
```

--------------------------------

### Get Batched Residuals

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Computes residuals for a list of prompts by splitting them into batches. Returns a tensor containing residuals for each prompt.

```python
def get_residuals_batched(self, prompts: list[Prompt]) -> Tensor:
    pass
```

```python
all_residuals = model.get_residuals_batched(large_prompt_list)
```

--------------------------------

### Initialize Analyzer

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/04-analyzer-evaluator.md

Instantiate the Analyzer class with model settings and residual data. Requires pre-loaded settings, model, and residual tensors.

```python
from heretic.analyzer import Analyzer

analyzer = Analyzer(
    settings=settings,
    model=model,
    good_residuals=good_residual_tensor,
    bad_residuals=bad_residual_tensor,
)
```

--------------------------------

### Get Layer Modules by Index

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Retrieves specific component modules (e.g., attention out-projection, MLP down-projection) for a given layer index.

```python
def get_layer_modules(self, layer_index: int) -> dict[str, list[Module]]:
    pass
```

--------------------------------

### Utilities - Dataset Loading

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/INDEX.md

The `load_prompts` utility function facilitates the loading of prompt datasets from various sources, including Hugging Face Hub, local files, and plain text files, making it easy to prepare data for model analysis and generation.

```APIDOC
## Utilities

### Dataset Loading

```python
from heretic.utils import load_prompts

prompts = load_prompts(settings, dataset_spec)
```

Supports:
- HF Hub datasets
- Local datasets
- Text files (one prompt per line)
```

--------------------------------

### get_trial_parameters

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/05-utilities.md

Samples trial parameters for abliterable components of a model based on an Optuna trial object.

```APIDOC
## get_trial_parameters

### Description
Samples trial parameters for each abliterable component of a model and constructs parameter objects based on an Optuna trial.

### Signature
```python
def get_trial_parameters(trial: Trial, model: Model) -> dict[str, AbliterationParameters]:
    pass
```

### Parameters
* `trial` (Trial) - Optuna trial object
* `model` (Model) - Loaded model instance

### Returns
Dictionary mapping component name to `AbliterationParameters`.

### Behavior
Samples trial parameters for each abliterable component and constructs parameter objects.

### Example
```python
from heretic.utils import get_trial_parameters

params = get_trial_parameters(trial, model)
# {"attn.o_proj": AbliterationParameters(...), "mlp.down_proj": AbliterationParameters(...)}
```
```

--------------------------------

### Get Responses from Prompts

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Generates and decodes responses for a list of prompts, with an option to skip special tokens. Use for obtaining clean text responses.

```python
def get_responses(
    self,
    prompts: list[Prompt],
    skip_special_tokens: bool = False,
) -> list[str]:
    pass
```

```python
responses = model.get_responses(
    [Prompt(system="You are helpful.", user="Why is the sky blue?")],
    skip_special_tokens=True
)
print(responses[0])
```

--------------------------------

### Get NVIDIA Driver Version

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/05-utilities.md

Retrieves the NVIDIA driver version string if an NVIDIA GPU is present and the driver is detected. Returns None if not available.

```python
def get_nvidia_driver_version() -> str | None:
    pass
```

```python
from heretic.system import get_nvidia_driver_version

version = get_nvidia_driver_version()
# "555.42.02"
```

--------------------------------

### Model Constructor

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Initializes the Model class, loading necessary components and applying configurations.

```python
def __init__(self, settings: Settings):
    pass
```

--------------------------------

### Progress Bars for Prompt Loading in Notebooks

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/08-programmatic-usage.md

Utilize TQDM for progress bars when loading prompts, which functions correctly within Jupyter notebooks. This provides visual feedback during potentially long-running data loading operations.

```python
# Uses TQDM, works in notebooks
from heretic.utils import load_prompts

prompts = load_prompts(settings, settings.good_prompts)
# Shows progress bar in notebook
```

--------------------------------

### Compare Two Models Workflow

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/INDEX.md

Compares the refusal rates of a base model against an abliterated model. This workflow is useful for assessing the impact of abliteration on model behavior.

```python
from heretic.config import Settings
from heretic.model import Model
from heretic.evaluator import Evaluator

settings_base = Settings(model="Qwen/Qwen3-4B-Instruct-2507")
settings_abl = Settings(model="path/to/abliterated")

model_base = Model(settings_base)
evaluator_base = Evaluator(settings_base, model_base)

model_abl = Model(settings_abl)
evaluator_abl = Evaluator(settings_abl, model_abl)

print(f"Base refusals: {evaluator_base.base_refusals}")
print(f"Abl refusals: {evaluator_abl.base_refusals}")
```

--------------------------------

### LoRA Initialization for Abliteration

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/09-architecture-algorithms.md

Collects target modules for LoRA adaptation, sets a rank of 1 (or a configurable rank for full normalization), and applies LoRA to the model. Prepares the model with frozen base weights and trainable LoRA adapters.

```python
for layer_index in layers:
    for each component in ["attn.o_proj", "mlp.down_proj"]:
        add to target_modules

create LoRA config with:
    rank = 1 (or settings.full_normalization_lora_rank for FULL mode)
    target_modules = collected list
    lora_alpha = rank
    lora_dropout = 0
    task_type = "CAUSAL_LM"

apply LoRA to model
```

--------------------------------

### load_prompts

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/05-utilities.md

Loads prompts from various sources including Hugging Face datasets, local disk directories, or plain text files. It applies specified prefixes, suffixes, and system prompts, and handles dataset splitting.

```APIDOC
## load_prompts

### Description
Loads prompts from various sources including Hugging Face datasets, local disk directories, or plain text files. It applies specified prefixes, suffixes, and system prompts, and handles dataset splitting.

### Parameters
#### Path Parameters
- `settings` (Settings) - Required - Configuration object
- `dataset_spec` (DatasetSpecification) - Required - Dataset specification (see config.py)

### Returns
List of `Prompt` objects.

### Supported Sources:
- HF dataset ID (e.g., `"mlabonne/harmless_alpaca"`)
- Local dataset directory (e.g., `"/path/to/dataset"`)
- Plain text file (e.g., `"/path/to/prompts.txt"`)

### Example:
```python
from heretic.config import Settings, DatasetSpecification
from heretic.utils import load_prompts

settings = Settings()

# From HF dataset
dataset_spec = DatasetSpecification(
    dataset="mlabonne/harmless_alpaca",
    split="train[:400]",
    column="text",
    prefix="Q: ",
    suffix="\nA: ",
)
prompts = load_prompts(settings, dataset_spec)

# From text file
dataset_spec = DatasetSpecification(
    dataset="/path/to/prompts.txt",
    split="[:100]",  # Optional for text files
)
prompts = load_prompts(settings, dataset_spec)
```
```

--------------------------------

### Get performance score

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/04-analyzer-evaluator.md

Calculates and returns a performance score based on KL divergence and refusal rates. This score is used to evaluate the effectiveness of response generation.

```python
def get_score(self) -> tuple[tuple[float, float], float, int]:
    pass
```

```python
(
    kld_score,
    refusals_score
), kl_div, refusals = evaluator.get_score()

print(f"KL Divergence Score: {kld_score:.4f}")
print(f"Refusal Score: {refusals_score:.4f}")
print(f"Actual KL Divergence: {kl_div:.4f}")
print(f"Refusals: {refusals}/{len(evaluator.bad_prompts)}")
```

--------------------------------

### Prompt Processing Pipeline Steps

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/09-architecture-algorithms.md

Outlines the sequence of operations involved in processing a raw prompt, from loading data to generating tokens and extracting model outputs.

```text
Raw Prompt (dict)
    ↓
Load from dataset (HF, disk, or text file)
    ↓
Apply prefix/suffix
    ↓
Combine with system prompt → Prompt object
    ↓
Apply chat template
    ↓
Tokenize
    ↓
Batch and pad
    ↓
Generate tokens
    ↓
Extract residuals or logprobs
    ↓
Detach and optionally move to CPU
```

--------------------------------

### Import Configuration Classes

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/INDEX.md

Imports specific classes for defining dataset, benchmark, quantization, normalization, and export strategies within Heretic's configuration system.

```python
from heretic.config import (
    DatasetSpecification,
    BenchmarkSpecification,
    QuantizationMethod,
    RowNormalization,
    ExportStrategy,
)
```

--------------------------------

### Checkpoint Management for Optimization Studies

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/08-programmatic-usage.md

Save the state of an optimization study to a file for later resumption or analysis. This involves creating a checkpoint directory and using study.save() to store the study's database.

```python
import os
from pathlib import Path

checkpoint_dir = Path("checkpoints") / "my_experiment"
checkpoint_dir.mkdir(parents=True, exist_ok=True)

# Save optimization state
study.save(f"{checkpoint_dir}/study.db")
```

--------------------------------

### Get AMD GPU Driver Version

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/05-utilities.md

Retrieves the AMD GPU driver version. It checks multiple sources including 'amd-smi', 'rocm-smi', and the Linux kernel module.

```python
def get_amdgpu_driver_version() -> str | None:
    pass
```

```python
from heretic.system import get_amdgpu_driver_version

version = get_amdgpu_driver_version()
# "6.0.2"
```

--------------------------------

### Get Merged Model

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Retrieves the model with LoRA adapters applied. For quantized models, it reloads the base model in full precision before merging. This operation can consume significant VRAM.

```python
def get_merged_model(self) -> PreTrainedModel:
    pass
```

```python
merged = model.get_merged_model()
merged.save_pretrained("abliterated-model")
```

--------------------------------

### Get Mean Residuals

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Calculates the mean residuals for a list of prompts. Processes batches incrementally and uses float64 internally for precision, saving VRAM compared to computing all residuals at once.

```python
def get_residuals_mean(self, prompts: list[Prompt]) -> Tensor:
    pass
```

```python
mean_bad_residuals = model.get_residuals_mean(bad_prompts)
print(mean_bad_residuals.shape)  # torch.Size([19, 4096])
```

--------------------------------

### Optimization Loop - Trial Lifecycle

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/09-architecture-algorithms.md

Describes the lifecycle of a single trial within the Optuna optimization loop. It includes parameter sampling, model reset, abliteration, evaluation, reporting, and pruning based on performance.

```python
1. Sample parameters from TPE sampler
2. Reset model (clear LoRA adapters)
3. Apply abliteration with sampled parameters
4. Evaluate: obtain (kld_score, refusals_score), kl_div, refusal_count
5. Report to Optuna
6. Optuna prunes if:
   - refusals_score > baseline
   - kld_score > threshold
7. Store in study for resumability
8. Move to next trial
```

--------------------------------

### Get Residuals for Prompts

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Computes and extracts hidden states (residuals) for each layer after generating one token per prompt. Useful for analyzing model behavior and internal representations.

```python
def get_residuals(self, prompts: list[Prompt]) -> Tensor:
    pass
```

```python
good_prompts = [...] 
residuals = model.get_residuals(good_prompts)
print(residuals.shape)  # torch.Size([400, 19, 4096])

# Compute mean residual per layer
mean_residuals = residuals.mean(dim=0)  # (19, 4096)
```

--------------------------------

### Load Prompts from Dataset

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/05-utilities.md

Loads prompts from Hugging Face datasets, local disk, or plain text files. Supports applying prefixes, suffixes, and dataset-specific system prompts. Handles various source types and split specifications.

```python
from heretic.config import Settings, DatasetSpecification
from heretic.utils import load_prompts

settings = Settings()

# From HF dataset
dataset_spec = DatasetSpecification(
    dataset="mlabonne/harmless_alpaca",
    split="train[:400]",
    column="text",
    prefix="Q: ",
    suffix="\nA: ",
)
prompts = load_prompts(settings, dataset_spec)

# From text file
dataset_spec = DatasetSpecification(
    dataset="/path/to/prompts.txt",
    split="[:100]",  # Optional for text files
)
prompts = load_prompts(settings, dataset_spec)
```

--------------------------------

### Analyze Residual Geometry with Heretic Analyzer

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/08-programmatic-usage.md

Initialize the Analyzer with settings, model, and computed residuals. Optionally print residual geometry metrics or generate plots based on configuration settings.

```python
from heretic.analyzer import Analyzer

analyzer = Analyzer(settings, model, good_residuals, bad_residuals)

# Print metrics
if settings.print_residual_geometry:
    analyzer.print_residual_geometry()

# Generate plots
if settings.plot_residuals:
    analyzer.plot_residuals()
```

--------------------------------

### Optimization Workflow Loop

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/07-main-cli.md

Detailed steps within the Heretic optimization phase, including parameter sampling, abliteration, evaluation, scoring, and pruning using Optuna.

```text
For each trial:
  1. Sample parameters from Optuna TPE sampler
  2. Apply abliteration with sampled parameters
  3. Evaluate on test set (refusals + KL divergence)
  4. Score: (kld_score, refusals_score)
  5. Report score to Optuna
  6. Optuna prunes if score exceeds thresholds
  7. Reset model for next trial
```

--------------------------------

### Load Prompts from Dataset

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/INDEX.md

Load prompts using the load_prompts utility function, which supports datasets from the HF Hub, local files, or text files.

```python
from heretic.utils import load_prompts

prompts = load_prompts(settings, dataset_spec)
```

--------------------------------

### Generate Residual Vector Plots

Source: https://github.com/p-e-w/heretic/blob/master/README.md

Run Heretic with the '--plot-residuals' flag to compute, project, and visualize residual vectors for model interpretability. This feature requires the 'research' extra to be installed.

```bash
heretic --plot-residuals
```

--------------------------------

### Model Loading & Analysis Workflow

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/07-main-cli.md

A high-level overview of the initial stages in the Heretic workflow, involving model loading, residual computation, and geometric analysis.

```text
Load model → Compute residuals → Analyze geometry → Identify refusal directions
```

--------------------------------

### Manual Abliteration with Heretic

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/08-programmatic-usage.md

Apply abliteration to specific model components by defining parameters like max/min weights and distances. This example ablates attention output and MLP down projection layers.

```python
from heretic.model import AbliterationParameters

# Define per-component parameters
parameters = {
    "attn.o_proj": AbliterationParameters(
        max_weight=0.8,
        max_weight_position=15.0,
        min_weight=0.2,
        min_weight_distance=5.0,
    ),
    "mlp.down_proj": AbliterationParameters(
        max_weight=0.9,
        max_weight_position=16.0,
        min_weight=0.1,
        min_weight_distance=6.0,
    ),
}

# Apply abliteration
model.abliterate(
    refusal_directions=refusal_directions,
    direction_index=7.5,  # Use layer 8 direction, interpolated
    parameters=parameters,
)

# Test the abliterated model
test_response = model.get_responses([test_prompt])[0]
```

--------------------------------

### Import Utility and System Classes

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/INDEX.md

Imports various utility functions for data loading, prompting, formatting, path handling, and system-level operations like memory management and hardware info.

```python
from heretic.utils import (
    load_prompts,
    prompt_select,
    prompt_text,
    prompt_path,
    prompt_password,
    format_duration,
    format_exception,
    is_hf_path,
    set_seed,
    print_memory_usage,
    is_notebook,
    get_trial_parameters,
)

from heretic.system import (
    empty_cache,
    get_accelerator_info,
    get_nvidia_driver_version,
    get_amdgpu_driver_version,
)
```

--------------------------------

### prompt_path

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/05-utilities.md

Prompts the user for a directory path, supporting both existing and new paths. It uses a text input in notebooks and a path picker in terminals.

```APIDOC
## prompt_path

### Description
Prompts the user for a directory path, supporting both existing and new paths. It uses a text input in notebooks and a path picker in terminals.

### Parameters
#### Path Parameters
- `message` (str) - Required - Prompt text

### Returns
Directory path (existing or new).

### Example:
```python
repo_path = prompt_path("Where to upload?")
```
```

--------------------------------

### Load Heretic Settings from TOML

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/08-programmatic-usage.md

Load Heretic settings automatically from a 'config.toml' file if it exists in the current directory.

```python
from heretic.config import Settings

# Automatically loads config.toml if it exists
settings = Settings()
```

--------------------------------

### Model Constructor

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/03-model-api.md

Initializes the Model class, loading the tokenizer, processor, and applying LoRA adapters. It attempts to load the model with various dtypes and performs a test generation to ensure compatibility.

```APIDOC
## Model Constructor

### Description
Initializes the Model class, loading the tokenizer, processor, and applying LoRA adapters. It attempts to load the model with various dtypes and performs a test generation to ensure compatibility.

### Parameters
#### Request Body
- **settings** (Settings) - Required - Configuration object (see config.py)

### Behavior:
- Loads tokenizer from HF Hub
- Loads processor for multimodal models (if applicable)
- Tries each dtype in `settings.dtypes` until one succeeds
- Applies LoRA adapters to all target modules
- Performs test generation to validate dtype compatibility
- Prints layer count and abliterable components

### Raises:
`Exception` if model fails to load with all configured dtypes.

### Example:
```python
from heretic.config import Settings
from heretic.model import Model

settings = Settings(model="Qwen/Qwen3-4B-Instruct-2507")
model = Model(settings)
```
```

--------------------------------

### Batch Processing with Progress Bar

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/08-programmatic-usage.md

Process prompts in batches and display progress using tqdm. This is useful for large datasets to monitor the processing status.

```python
from tqdm import tqdm

prompts_batches = [prompts[i:i+100] for i in range(0, len(prompts), 100)]
all_responses = []

for batch in tqdm(prompts_batches):
    responses = model.get_responses(batch)
    all_responses.extend(responses)
```

--------------------------------

### Set Heretic Configuration via Environment Variables

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/02-configuration.md

Set Heretic configuration options using environment variables. The HERETIC_ prefix is used, and kebab-case settings are converted to SCREAMING_SNAKE_CASE.

```bash
export HERETIC_MODEL="Qwen/Qwen3-4B-Instruct-2507"
export HERETIC_QUANTIZATION="bnb_4bit"
export HERETIC_BATCH_SIZE=16
export HERETIC_N_TRIALS=200
```

--------------------------------

### Initialize Evaluator

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/04-analyzer-evaluator.md

Instantiate the Evaluator class with model settings. The constructor loads evaluation prompts, computes baselines, and counts initial refusals.

```python
from heretic.evaluator import Evaluator

evaluator = Evaluator(settings=settings, model=model)
# Evaluator now ready for scoring
```

--------------------------------

### Basic Tensor Operations

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/06-types.md

Demonstrates common tensor operations like calculating mean residuals per layer. Ensure torch and torch.nn.functional are imported.

```python
import torch
import torch.nn.functional as F

# Residuals
residuals = model.get_residuals(prompts)  # (N, L, D)

# Mean per layer
mean_residuals = residuals.mean(dim=0)  # (L, D)
```

--------------------------------

### Check XPU Availability

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/05-utilities.md

Checks if Intel XPU is available on the system.

```python
def is_xpu_available() -> bool:
    pass
```

--------------------------------

### Configure Heretic with Custom Datasets

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/08-programmatic-usage.md

Configure Heretic to use custom datasets for good and bad prompts, specifying the file path and a data split for each.

```python
from heretic.config import Settings, DatasetSpecification

settings = Settings(
    model="Qwen/Qwen3-4B-Instruct-2507",
    good_prompts=DatasetSpecification(
        dataset="/path/to/harmless.txt",
        split="[:400]",
    ),
    bad_prompts=DatasetSpecification(
        dataset="/path/to/harmful.txt",
        split="[:400]",
    ),
)
```

--------------------------------

### Formatted Console Output with Rich Markup

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/05-utilities.md

Shows how to use the Rich console for styled and colored output. Imports the 'print' function from heretic.utils.

```python
from heretic.utils import print

print("[bold]Important[/]")
print("[red]Error[/]")
print("[green]Success[/]")
print(f"[yellow]Model: {model_name}[/]")
```

--------------------------------

### Heretic CLI Entry Point

Source: https://github.com/p-e-w/heretic/blob/master/_autodocs/INDEX.md

The main command-line interface entry point for Heretic. It accepts options and a model argument, with configuration possible via CLI arguments, environment variables, config files, or interactive prompts.

```bash
heretic [OPTIONS] [MODEL]
```