### Install BioEmu-Benchmarks

Source: https://context7.com/microsoft/bioemu-benchmarks/llms.txt

Installs the bioemu-benchmarks package using pip. This is the primary method for setting up the library for use.

```bash
pip install bioemu-benchmarks
```

--------------------------------

### Get Sample Specifications with CLI

Source: https://context7.com/microsoft/bioemu-benchmarks/llms.txt

Generates CSV files containing sequences and recommended sample counts for benchmarks using the `bioemu-bench specs` command. This helps in preparing the correct number of samples for each benchmark.

```bash
# Get specifications for a single benchmark
bioemu-bench specs ./specs.csv --benchmarks multiconf_ood60

# Get specifications for multiple benchmarks
bioemu-bench specs ./all_specs.csv -b multiconf_ood60 folding_free_energies md_emulation

# Get specifications for all benchmarks
bioemu-bench specs ./complete_specs.csv --benchmarks all
```

--------------------------------

### Folding Free Energies Benchmark Setup

Source: https://context7.com/microsoft/bioemu-benchmarks/llms.txt

This snippet shows how to set up the Folding Free Energies benchmark, which evaluates the prediction of thermodynamic stability (dG) and mutation effects (ddG). It involves loading samples, indexing them for the benchmark, and filtering out unphysical data.

```python
from bioemu_benchmarks.benchmarks import Benchmark
from bioemu_benchmarks.samples import IndexedSamples, find_samples_in_dir, filter_unphysical_samples
from bioemu_benchmarks.eval.folding_free_energies.evaluate import evaluate_folding_free_energies

# Load and prepare samples
benchmark = Benchmark.FOLDING_FREE_ENERGIES
samples = find_samples_in_dir("/path/to/samples")
indexed = IndexedSamples.from_benchmark(benchmark, samples)
filtered, _ = filter_unphysical_samples(indexed)

```

--------------------------------

### Get Benchmark Metadata (Python)

Source: https://github.com/microsoft/bioemu-benchmarks/blob/main/README.md

Retrieves metadata for a specific benchmark using the BioEmu Python API. The metadata is returned as a pandas DataFrame containing information about the benchmark's sequences. Requires the `Benchmark` enum.

```python
from bioemu_benchmarks.benchmarks import Benchmark

# Returns a pandas df with info about the Multiconf OOD60 benchmark
metadata = Benchmark.MULTICONF_OOD60.metadata
```

--------------------------------

### Get Aggregate Metrics and Access Benchmark Info (Python)

Source: https://context7.com/microsoft/bioemu-benchmarks/llms.txt

This snippet demonstrates how to retrieve aggregate metrics from benchmark results and access the benchmark type. It iterates through the metrics dictionary and prints each metric name and its value, followed by the benchmark's value.

```python
# Get aggregate metrics as dictionary
metrics = results.get_aggregate_metrics()
for name, value in metrics.items():
    print(f"{name}: {value}")

# Access benchmark type
print(f"Benchmark: {results.benchmark.value}")
```

--------------------------------

### Generate Sample Specifications (CLI)

Source: https://github.com/microsoft/bioemu-benchmarks/blob/main/README.md

Generates a CSV file with sequence information and recommended sample counts for specified benchmarks using the `bioemu-bench` CLI. Requires an output CSV file path and a list of benchmarks.

```bash
bioemu-bench specs <output_csv> --benchmarks/-b [...] 
```

--------------------------------

### Finding and Loading Samples in Python

Source: https://context7.com/microsoft/bioemu-benchmarks/llms.txt

Discovers sample files (`.xtc` and `.pdb`) within specified directories and organizes them for use with benchmarks. It includes functions for finding samples and creating indexed sample objects.

```python
from bioemu_benchmarks.benchmarks import Benchmark
from bioemu_benchmarks.samples import (
    IndexedSamples,
    SequenceSample,
    find_samples_in_dir,
    filter_unphysical_samples,
)
```

--------------------------------

### Run Benchmark and Process Samples (Python)

Source: https://github.com/microsoft/bioemu-benchmarks/blob/main/README.md

Evaluates samples for a specified benchmark using the BioEmu Python API. It loads, validates, and optionally filters samples, then runs the evaluation, and provides methods for plotting and saving results. Dependencies include `bioemu_benchmarks` and `pandas`.

```python
from bioemu_benchmarks.benchmarks import Benchmark
from bioemu_benchmarks.samples import IndexedSamples, filter_unphysical_samples, find_samples_in_dir
from bioemu_benchmarks.evaluator_utils import evaluator_from_benchmark

# Specify the benchmark you want to run (e.g., OOD60)
benchmark = Benchmark.MULTICONF_OOD60

# This validates samples
sequence_samples = find_samples_in_dir("/path/to/your/sample_dir")
samples = IndexedSamples.from_benchmark(benchmark=benchmark, sequence_samples=sequence_samples)

# Filter unphysical-looking samples from getting evaluated
samples, _sample_stats = filter_unphysical_samples(samples)

# Instanstiate an evaluator for a given benchmark
evaluator = evaluator_from_benchmark(benchmark=benchmark)

# `results` has methods for plotting / computing summary metrics
results = evaluator(samples)
results.plot('/path/to/result/plots') 
results.save_results('/path/to/result/metrics/')
```

--------------------------------

### Multi-Conformational Benchmark Evaluation and Analysis

Source: https://context7.com/microsoft/bioemu-benchmarks/llms.txt

This snippet demonstrates how to run and analyze multi-conformational benchmarks, focusing on domain motion. It includes loading samples, filtering, obtaining an evaluator, running the evaluation, and accessing per-system results, coverage metrics, and k-recall metrics.

```python
from bioemu_benchmarks.benchmarks import Benchmark
from bioemu_benchmarks.samples import IndexedSamples, find_samples_in_dir, filter_unphysical_samples
from bioemu_benchmarks.evaluator_utils import evaluator_from_benchmark
from bioemu_benchmarks.multiconf_evaluator import (
    evaluate_multiconf_global,
    evaluate_multiconf_local,
    evaluate_crypticpocket,
    evaluate_local_unfolding,
)

# Example: Domain motion benchmark (global RMSD)
benchmark = Benchmark.MULTICONF_DOMAINMOTION
samples = find_samples_in_dir("/path/to/samples")
indexed = IndexedSamples.from_benchmark(benchmark, samples)
filtered, _ = filter_unphysical_samples(indexed)

evaluator = evaluator_from_benchmark(benchmark)
results = evaluator(filtered)

# Access per-system results
for test_case, system_result in list(results.per_system.items())[:3]:
    print(f"\nTest case: {test_case}")
    for metric_type, values in system_result.metrics_against_references.items():
        print(f"  {metric_type.value}: min={values.min():.2f}, mean={values.mean():.2f}")

# Coverage metrics (fraction of reference states covered)
for label, coverage_data in results.coverage.items():
    for metric_type, (thresholds, coverage_values) in coverage_data.items():
        coverage_at_2A = coverage_values[:, thresholds <= 2.0].mean()
        print(f"{label} - {metric_type.value} coverage at 2A: {coverage_at_2A:.2%}")

# K-recall metrics
for label, krecall_data in results.krecall.items():
    for metric_type, test_case_results in krecall_data.items():
        mean_krecall = sum(v[0] for v in test_case_results.values()) / len(test_case_results)
        print(f"{label} - {metric_type.value} mean k-recall: {mean_krecall:.2f}")
```

--------------------------------

### Discover and Process Molecular Samples

Source: https://context7.com/microsoft/bioemu-benchmarks/llms.txt

This snippet demonstrates how to find sequence samples within a directory, access their topology and trajectory files, create indexed samples for a benchmark, filter out unphysical samples, and retrieve trajectories for a specific test case.

```python
sequence_samples = find_samples_in_dir("/path/to/samples")
print(f"Found {len(sequence_samples)} sample files")

# Each SequenceSample contains paths to topology and trajectory
for sample in sequence_samples[:2]:
    print(f"Topology: {sample.topology_file}")
    print(f"Trajectory: {sample.trajectory_file}")

# Create indexed samples for a specific benchmark
benchmark = Benchmark.MULTICONF_OOD60
indexed_samples = IndexedSamples.from_benchmark(
    benchmark=benchmark,
    sequence_samples=sequence_samples
)
print(f"Test cases with samples: {list(indexed_samples.test_case_to_sequencesamples.keys())[:5]}")

# Filter out unphysical samples (clashes, broken bonds, etc.)
filtered_samples, filter_stats = filter_unphysical_samples(indexed_samples)
for test_case, retention_rates in list(filter_stats.items())[:3]:
    print(f"{test_case}: {retention_rates.mean():.1%} samples retained")

# Get trajectories for a specific test case
trajs = filtered_samples.get_trajs_for_test_case("P01112")
print(f"Number of trajectory objects: {len(trajs)}")
print(f"Total frames: {sum(t.n_frames for t in trajs)}")
```

--------------------------------

### Run Benchmarks with CLI

Source: https://context7.com/microsoft/bioemu-benchmarks/llms.txt

Evaluates protein samples against specified benchmarks using the `bioemu-bench eval` command. It can run single or multiple benchmarks, process samples from directories, and optionally skip filtering or overwrite existing results.

```bash
# Run a single benchmark
bioemu-bench eval ./results --benchmarks multiconf_ood60 --sample_dirs ./my_samples

# Run multiple benchmarks
bioemu-bench eval ./results -b multiconf_ood60 multiconf_domainmotion folding_free_energies -s ./my_samples

# Run all available benchmarks
bioemu-bench eval ./results --benchmarks all --sample_dirs ./samples_dir1 ./samples_dir2

# Skip sample filtering (not recommended for production)
bioemu-bench eval ./results -b md_emulation -s ./my_samples --skip_filtering

# Overwrite existing results
bioemu-bench eval ./results -b multiconf_ood60 -s ./my_samples --overwrite
```

--------------------------------

### Run MD Emulation Benchmark Evaluation

Source: https://context7.com/microsoft/bioemu-benchmarks/llms.txt

Performs an MD emulation benchmark by loading samples, indexing them, and filtering unphysical ones. It then runs the evaluation, accessing sample projections and metrics. The results are aggregated, saved, and plotted.

```python
from bioemu_benchmarks.benchmarks import Benchmark
from bioemu_benchmarks.samples import IndexedSamples, find_samples_in_dir, filter_unphysical_samples
from bioemu_benchmarks.eval.md_emulation.evaluate import evaluate_md_emulation

# Load samples
benchmark = Benchmark.MD_EMULATION
samples = find_samples_in_dir("/path/to/samples")
indexed = IndexedSamples.from_benchmark(benchmark, samples)
filtered, _ = filter_unphysical_samples(indexed)

# Run evaluation
results = evaluate_md_emulation(
    indexed_samples=filtered,
    temperature_K=300.0,  # Temperature for free energy computation
    random_seed=42        # For reproducible resampling
)

# Access sample projections (low-dimensional representations)
for test_case, projections in list(results.sample_projections.items())[:3]:
    print(f"{test_case}: {projections.shape[0]} samples, {projections.shape[1]}D projection")

# Access metrics DataFrame
print("\nMetrics per system:")
print(results.metrics[['mae', 'rmse', 'coverage']].head())

# Aggregate metrics
aggregate = results.get_aggregate_metrics()
print(f"\nMean MAE: {aggregate['mae']:.3f} kcal/mol")
print(f"Mean RMSE: {aggregate['rmse']:.3f} kcal/mol")
print(f"Mean Coverage: {aggregate['coverage']:.3f}")

# Save and plot
results.save_results("/path/to/output")  # Creates results_metrics.csv, results_projections.npz
results.plot("/path/to/output")          # Creates projections.png, metrics.png
```

--------------------------------

### Work with BenchmarkResults Objects

Source: https://context7.com/microsoft/bioemu-benchmarks/llms.txt

Illustrates common operations on `BenchmarkResults` objects, which are returned by various evaluators. This includes saving results to pickle format for later loading, saving in accessible formats like CSV and NPZ, and generating visualization plots.

```python
from bioemu_benchmarks.results import BenchmarkResults
from pathlib import Path

# Assuming 'evaluator' is a function that returns BenchmarkResults
# and 'filtered_samples' is pre-processed data
# results = evaluator(filtered_samples)

# Placeholder for results object
class MockResults(BenchmarkResults):
    def to_pickle(self, path):
        print(f"Mock saving to pickle: {path}")
    def save_results(self, path):
        print(f"Mock saving results to: {path}")
    def plot(self, path):
        print(f"Mock plotting to: {path}")

results = MockResults()

# Save to pickle for later loading
results.to_pickle("/path/to/results.pkl")

# Load from pickle
from bioemu_benchmarks.eval.multiconf.results import MulticonfResults
# loaded_results = MulticonfResults.from_pickle("/path/to/results.pkl")
print("Mock loading from pickle: /path/to/results.pkl")

# Save results in accessible formats (CSV, HDF5, NPZ)
results.save_results("/path/to/output_dir")

# Generate visualization plots
results.plot("/path/to/output_dir")
```

--------------------------------

### Complete Evaluation Workflow with Python API

Source: https://context7.com/microsoft/bioemu-benchmarks/llms.txt

This snippet outlines the complete workflow for running an evaluation using the BioEMU Benchmarks Python API. It covers loading and validating samples, filtering unphysical ones, obtaining the appropriate evaluator, running the evaluation, and accessing/saving the results.

```python
from bioemu_benchmarks.benchmarks import Benchmark
from bioemu_benchmarks.samples import IndexedSamples, find_samples_in_dir, filter_unphysical_samples
from bioemu_benchmarks.evaluator_utils import evaluator_from_benchmark

# Complete evaluation workflow
benchmark = Benchmark.MULTICONF_OOD60

# 1. Load and validate samples
sequence_samples = find_samples_in_dir("/path/to/your/sample_dir")
indexed_samples = IndexedSamples.from_benchmark(
    benchmark=benchmark,
    sequence_samples=sequence_samples
)

# 2. Filter unphysical samples
filtered_samples, stats = filter_unphysical_samples(indexed_samples)

# 3. Get the appropriate evaluator
evaluator = evaluator_from_benchmark(benchmark=benchmark)

# 4. Run evaluation
results = evaluator(filtered_samples)

# 5. Access results
aggregate_metrics = results.get_aggregate_metrics()
print("Aggregate metrics:")
for metric_name, value in aggregate_metrics.items():
    print(f"  {metric_name}: {value:.4f}")

# 6. Save results and generate plots
results.save_results("/path/to/output")
results.plot("/path/to/output")
results.to_pickle("/path/to/output/results.pkl")
```

--------------------------------

### Programmatic Benchmark Runner (Python)

Source: https://context7.com/microsoft/bioemu-benchmarks/llms.txt

This Python function, `run_all_benchmarks`, orchestrates the execution of multiple benchmarks. It collects samples, filters unphysical ones, runs evaluations using specified evaluators, saves results, and aggregates metrics. Dependencies include `pathlib`, `json`, and various modules from `bioemu_benchmarks`.

```python
from pathlib import Path
import json
from bioemu_benchmarks.benchmarks import Benchmark
from bioemu_benchmarks.samples import IndexedSamples, find_samples_in_dir, filter_unphysical_samples, NoSamples
from bioemu_benchmarks.evaluator_utils import evaluator_from_benchmark

def run_all_benchmarks(sample_dirs: list[str], output_dir: str):
    """Run all available benchmarks and collect results."""
    output_path = Path(output_dir)
    output_path.mkdir(parents=True, exist_ok=True)

    # Collect all samples
    all_samples = []
    for sample_dir in sample_dirs:
        all_samples.extend(find_samples_in_dir(sample_dir))
    print(f"Found {len(all_samples)} sample files")

    aggregate_results = {}

    for benchmark in Benchmark:
        print(f"\nRunning {benchmark.value}...")
        benchmark_dir = output_path / benchmark.value
        benchmark_dir.mkdir(exist_ok=True)

        try:
            # Load and filter samples
            indexed = IndexedSamples.from_benchmark(benchmark, all_samples)
            filtered, stats = filter_unphysical_samples(indexed)

            # Save filter statistics
            with open(benchmark_dir / "filter_stats.json", "w") as f:
                json.dump({k: v.mean() for k, v in stats.items()}, f, indent=2)

            # Run evaluation
            evaluator = evaluator_from_benchmark(benchmark)
            results = evaluator(filtered)

            # Save all outputs
            results.save_results(benchmark_dir)
            results.plot(benchmark_dir)
            results.to_pickle(benchmark_dir / "results.pkl")

            aggregate_results[benchmark.value] = results.get_aggregate_metrics()
            print(f"  Completed successfully")

        except NoSamples:
            print(f"  Skipped: no matching samples found")
            continue

    # Save aggregate results
    with open(output_path / "benchmark_metrics.json", "w") as f:
        json.dump(aggregate_results, f, indent=2, sort_keys=True)

    return aggregate_results

# Usage
results = run_all_benchmarks(
    sample_dirs=["./bioemu_samples", "./additional_samples"],
    output_dir="./benchmark_results"
)
```

--------------------------------

### Run Folding Free Energy Evaluation

Source: https://context7.com/microsoft/bioemu-benchmarks/llms.txt

Evaluates folding free energies at a specified temperature using provided indexed samples. It then accesses and prints metrics like fraction of native contacts, computed free energies, and aggregate metrics (MAE, correlation coefficients). Finally, it saves the results and generates plots.

```python
from bioemu_benchmarks.eval.folding.evaluate import evaluate_folding_free_energies

# Assuming 'filtered' is a pre-processed IndexedSamples object
results = evaluate_folding_free_energies(
    indexed_samples=filtered,
    temperature_K=295  # Temperature in Kelvin
)

# Access fraction of native contacts per system
for test_case, fnc_values in list(results.fnc_per_system.items())[:3]:
    print(f"{test_case}: mean FNC = {fnc_values.mean():.3f}, std = {fnc_values.std():.3f}")

# Access computed free energies
print("\nFree energies per system:")
print(results.free_energies_per_system[['test_case', 'dG_pred', 'dG_exp', 'ddG_pred', 'ddG_exp']].head())

# Aggregate metrics (MAE, correlation coefficients)
metrics = results.get_aggregate_metrics()
print(f"\ndG MAE: {metrics.get('dG_mae', 'N/A'):.2f} kcal/mol")
print(f"ddG MAE: {metrics.get('ddG_mae', 'N/A'):.2f} kcal/mol")

# Save results
results.save_results("/path/to/output")  # Creates results_systems.csv, results_metrics.csv
results.plot("/path/to/output")          # Creates scatter_dG.png, scatter_ddG.png
```

--------------------------------

### Filter Unphysical Samples

Source: https://context7.com/microsoft/bioemu-benchmarks/llms.txt

Demonstrates filtering unphysical samples using both high-level `filter_unphysical_samples` for `IndexedSamples` and low-level functions like `get_physical_traj_indices` and `filter_unphysical_traj` for individual trajectories loaded with `mdtraj`. It includes checking filtering statistics and applying trajectory-specific filters.

```python
from bioemu_benchmarks.samples import filter_unphysical_samples, IndexedSamples
from bioemu_benchmarks.utils import filter_unphysical_traj, get_physical_traj_indices
import mdtraj

# Assuming 'indexed_samples' is a pre-loaded IndexedSamples object

# High-level filtering for IndexedSamples
filtered_samples, filter_stats = filter_unphysical_samples(indexed_samples)

# Check filtering statistics
total_kept = []
for test_case, retention_rates in filter_stats.items():
    mean_retention = retention_rates.mean()
    total_kept.append(mean_retention)
    if mean_retention < 0.9:
        print(f"Warning: {test_case} only retained {mean_retention:.1%} of samples")
print(f"Overall mean retention: {sum(total_kept)/len(total_kept):.1%}")

# Low-level filtering for individual trajectories
traj = mdtraj.load("samples.xtc", top="topology.pdb")
print(f"Original frames: {traj.n_frames}")

# Get indices of physical frames
physical_indices = get_physical_traj_indices(
    traj,
    max_ca_seq_distance=4.5,  # Max CA-CA distance between neighbors (Angstrom)
    max_cn_seq_distance=2.0,  # Max C-N peptide bond distance (Angstrom)
    clash_distance=1.0,       # Min distance between non-bonded atoms (Angstrom)
)
print(f"Physical frames: {len(physical_indices)}")

# Filter trajectory directly
filtered_traj = filter_unphysical_traj(
    traj,
    max_ca_seq_distance=4.5,
    max_cn_seq_distance=2.0,
    clash_distance=1.0,
    strict=False  # Set True to raise error if all frames filtered
)
print(f"Filtered trajectory: {filtered_traj.n_frames} frames")
```

--------------------------------

### Benchmark Enum in Python

Source: https://context7.com/microsoft/bioemu-benchmarks/llms.txt

Defines and accesses available benchmark types within the BioEmu-Benchmarks Python API. It allows programmatic access to benchmark metadata, recommended sample sizes, and asset directory paths.

```python
from bioemu_benchmarks.benchmarks import Benchmark

# Available benchmarks
benchmark = Benchmark.MULTICONF_OOD60           # Out-of-distribution conformational changes
benchmark = Benchmark.MULTICONF_DOMAINMOTION    # Global domain motions (RMSD-based)
benchmark = Benchmark.MULTICONF_CRYPTICPOCKET   # Cryptic pocket backbone changes
benchmark = Benchmark.SINGLECONF_LOCALUNFOLDING # Local protein unfolding
benchmark = Benchmark.FOLDING_FREE_ENERGIES     # dG and ddG predictions
benchmark = Benchmark.MD_EMULATION              # MD distribution matching

# Access benchmark metadata (sequences and test cases)
metadata_df = Benchmark.MULTICONF_OOD60.metadata
print(metadata_df.columns)  # ['test_case', 'sequence', ...]
print(f"Number of test cases: {len(metadata_df)}")

# Get recommended sample sizes
sample_sizes = Benchmark.MULTICONF_OOD60.default_samplesize
print(f"Recommended samples per sequence: {sample_sizes[0]}")  # 4000 for multiconf

# Access asset directory path
asset_path = Benchmark.FOLDING_FREE_ENERGIES.asset_dir
print(f"Reference data location: {asset_path}")
```

--------------------------------

### BibTeX Citation for BioEmu Manuscript

Source: https://github.com/microsoft/bioemu-benchmarks/blob/main/DATASET_CARD.md

Provides the BibTeX formatted citation for the BioEmu manuscript, which should be used when referencing the dataset or its associated research. This format is commonly used in academic publications for bibliography management.

```bibtex
@misc{lewis_scalable_2024,
  title = {Scalable Emulation of Protein Equilibrium Ensembles with Generative Deep Learning},
  author = {Lewis, Sarah and Hempel, Tim and Jim{'e}nez Luna, Jos{'e} and Gastegger, Michael and Xie, Yu and Foong, Andrew Y. K. and Garc{'i}a Satorras, Victor and Abdin, Osama and Veeling, Bastiaan S. and Zaporozhets, Iryna and Chen, Yaoyi and Yang, Soojung and Schneuing, Arne and Nigam, Jigyasa and Barbero, Federico and Stimper, Vincent and Campbell, Andrew and Yim, Jason and Lienen, Marten and Shi, Yu and Zheng, Shuxin and Schulz, Hannes and Munir, Usman and Clementi, Cecilia and No{'e}, Frank},
  year = {2024},
  doi = {10.1101/2024.12.05.626885},
  archiveprefix = {BioRXiv},
  url = {https://www.biorxiv.org/content/10.1101/2024.12.05.626885}
}
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.