### Install PEFT library

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmc_finetune.ipynb

Installs the PEFT library, which is required for LoRA finetuning.

```python
# !pip install peft
```

--------------------------------

### Install ESMC

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmc_layer_sweep.ipynb

Install the ESMC library from its GitHub repository. This is necessary to use the ESMC model and its associated tools.

```python
# If you are working in colab, uncomment these lines to install dependencies
# !pip install esm@git+https://github.com/Biohub/esm.git@main
```

--------------------------------

### Install Dependencies and Import Libraries

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esm3_generate.ipynb

Installs necessary libraries and imports core modules for using the ESM SDK and visualization tools. Ensure you are in a Colab environment or have these libraries installed.

```python
%set_env TOKENIZERS_PARALLELISM=false
# If you are working in colab, uncomment these lines to install dependencies
# !pip install esm@git+https://github.com/Biohub/esm.git@main
# !pip install py3Dmol

import numpy as np
import py3Dmol
import torch

from esm.sdk import client
from esm.sdk.api import ESMProtein, GenerationConfig
from esm.utils.structure.protein_chain import ProteinChain
```

--------------------------------

### Install Dependencies

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/binder_design.ipynb

Installs the necessary Python packages for binder design, including ESM, Modal, and visualization libraries.

```bash
# Environment
! pip install esm@git+https://github.com/Biohub/esm.git@main
! pip install modal py3dmol pyarrow
```

--------------------------------

### Initialize ESM3 Guided Decoding with Constraints

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esm3_guided_generation.ipynb

Initializes ESM3GuidedDecodingWithConstraints with a radius of gyration scoring function and a pTM constraint. This setup guides generation towards globular proteins with a minimum pTM score.

```python
# Constrain generation to have pTM > 0.75
ptm_constraint = GenerationConstraint(
    scoring_function=PTMScoringFunction(),
    constraint_type=ConstraintType.GREATER_EQUAL,
    value=0.75,
)

radius_guided_decoding = ESM3GuidedDecodingWithConstraints(
    client=model,
    scoring_function=RadiousOfGyrationScoringFunction(),
    constraints=[ptm_constraint],  # Add list of constraints
    damping=1.0,  # Damping factor for the MMDM algorithm
    learning_rate=10.0,  # Learning rate for the MMDM algorithm
)
```

--------------------------------

### Initialize ESM3 Guided Decoding for No Cysteines

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esm3_guided_generation.ipynb

Initializes the ESM3GuidedDecoding class with a custom scoring function to prevent cysteine residues. This setup is used for generating proteins that adhere to the no-cysteine constraint.

```python
no_cysteine_guided_decoding = ESM3GuidedDecoding(
    client=model, scoring_function=NoCysteineScoringFunction()
)
```

--------------------------------

### Install ESMC and py3dmol

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmc_mutation_scoring.ipynb

Installs the ESMC library from GitHub and the py3dmol visualization library. Uncomment these lines if working in a Colab environment.

```python
# If you are working in colab, uncomment these lines to install dependencies
# !pip install esm@git+https://github.com/Biohub/esm.git@main
# !pip install py3dmol
```

--------------------------------

### Setup Batch Executor and Embedding Utilities

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/embed.ipynb

Import and configure utilities for batch execution and sequence embedding, including a configuration for requesting mean-pooled hidden states.

```python
from esm.sdk import batch_executor
from esm.sdk.api import ESMCInferenceClient, ESMProtein, LogitsConfig, LogitsOutput

# request mean-pooled hidden states to save memory
EMBEDDING_CONFIG = LogitsConfig(
    sequence=True, return_hidden_states=False, return_mean_hidden_states=True
)


def embed_sequence(model: ESMCInferenceClient, sequence: str) -> LogitsOutput:
    protein = ESMProtein(sequence=sequence)
    protein_tensor = model.encode(protein)
    output = model.logits(protein_tensor, EMBEDDING_CONFIG)
    return output
```

--------------------------------

### Install Dependencies

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/gfp_design.ipynb

Installs the esm package from GitHub and py3Dmol for visualization. Uncomment these lines if working in Google Colab.

```python
# If you are working in colab, uncomment these lines to install dependencies
# !pip install esm@git+https://github.com/Biohub/esm.git@main
# !pip install py3Dmol

from IPython.display import clear_output

clear_output()  # Suppress pip install log lines after installation is complete.
```

--------------------------------

### Set up ESM Development Environment

Source: https://github.com/biohub/esm/blob/main/CONTRIBUTIONS.md

Create and activate a micromamba environment for ESM development. Installs Python 3.10 and sets up the local package for testing.

```bash
micromamba create -n esm
micromamba activate esm
micromamba install -c conda-forge python=3.10

# in root level of repo
pip install -e .
pip install examples/requirements.txt
python -c 'from huggingface_hub import login; login()'

```

--------------------------------

### Initialize Guided Decoding for pTM

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esm3_guided_generation.ipynb

Creates an ESM3GuidedDecoding instance using the initialized model and the PTMScoringFunction. This object is used for guided protein generation.

```python
ptm_guided_decoding = ESM3GuidedDecoding(
    client=model, scoring_function=PTMScoringFunction()
)
```

--------------------------------

### Initialize Clustering Parameters

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/embed.ipynb

Sets the number of clusters for K-means and imports necessary scikit-learn modules for clustering and dimensionality reduction. Ensure scikit-learn is installed.

```python
from sklearn.cluster import KMeans
from sklearn.decomposition import PCA
from sklearn.metrics import adjusted_rand_score

N_KMEANS_CLUSTERS = 3
```

--------------------------------

### Perform Guided Generation

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esm3_guided_generation.ipynb

Starts a guided generation process using the ptm_guided_decoding instance. It begins with a fully masked protein and generates a new sequence over several decoding steps.

```python
# Start from a fully masked protein
PROTEIN_LENGTH = 256
starting_protein = ESMProtein(sequence="_" * PROTEIN_LENGTH)

# Call guided_generate
generated_protein = ptm_guided_decoding.guided_generate(
    protein=starting_protein,
    num_decoding_steps=len(starting_protein) // 8,
    num_samples_per_step=10,
)
```

--------------------------------

### Install ESM and py3Dmol

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esm3_guided_generation.ipynb

Installs the necessary libraries for using ESM and visualizing protein structures. Uncomment and run if working in a Colab environment.

```python
# # If you are working in colab, uncomment these lines to install dependencies
# !pip install esm@git+https://github.com/Biohub/esm.git@main
# !pip install py3Dmol
```

--------------------------------

### Install ESM and Dependencies

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmprotein.ipynb

Installs the ESM library from GitHub and other necessary dependencies like py3Dmol, matplotlib, and dna-features-viewer. Uncomment these lines if working in a Colab environment.

```python
# If you are working in colab, uncomment these lines to install dependencies
# ! pip install esm@git+https://github.com/Biohub/esm.git@main
# ! pip install py3Dmol
# ! pip install matplotlib
# ! pip install dna-features-viewer
```

--------------------------------

### Install ESM3 Python Library

Source: https://github.com/biohub/esm/blob/main/_assets/ESM3_README.md

Install the ESM3 Python library from its GitHub repository. This is a prerequisite for both Biohub and local execution.

```bash
pip install esm@git+https://github.com/Biohub/esm.git@main
```

--------------------------------

### Install ESM and Dependencies

Source: https://github.com/biohub/esm/blob/main/cookbook/local/open_generate.ipynb

Installs the ESM library from GitHub and other necessary packages like numpy, torch, and py3Dmol. Sets an environment variable to disable tokenizers parallelism.

```python
%set_env TOKENIZERS_PARALLELISM=false
!pip install esm@git+https://github.com/Biohub/esm.git@main
import numpy as np
import torch

!pip install py3Dmol
import py3Dmol

from esm.models.esm3 import ESM3
from esm.sdk.api import ESMProtein, GenerationConfig
from esm.utils.structure.protein_chain import ProteinChain
```

--------------------------------

### Install Dependencies

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/embed.ipynb

Install the necessary Python packages for ESMC and plotting. Uncomment these lines if running in Google Colab.

```python
# If you are working in colab, uncomment these lines to install dependencies
#! pip install esm@git+https://github.com/Biohub/esm.git@main
#! pip install matplotlib
#! pip install seaborn
```

--------------------------------

### Resume Binder Design Sweep on Colab

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/binder_design.ipynb

Example code to mount Google Drive for persistent storage when running on Colab. Ensure this is run before the Launch cell.

```python
from google.colab import drive
drive.mount('/content/drive')
save_dir = Path('/content/drive/MyDrive/binder_sweep')
```

--------------------------------

### Run a Single Design Job with Presets

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/binder_design.ipynb

Use this snippet to run a single binder design job using predefined target and binder names. It's a good starting point for a sanity check.

```python
# ---- Option 1: Use presets. ----
# Relies on the registry in modal_binder_design.py::{TARGET_SEQUENCES,BINDER_PROMPT_FACTORIES}, which can be modified.
future = app.design.spawn(target_name="ctla4", binder_name="minibinder")
future.get_dashboard_url()  # A clickable link to Modal dashboard
```

--------------------------------

### Load ESM3 Model Locally

Source: https://github.com/biohub/esm/blob/main/_assets/ESM3_README.md

Log in to Hugging Face Hub and load the ESM3 model weights locally. This example uses the 'esm3-sm-open-v1' model and can run on CUDA or CPU.

```python
from huggingface_hub import login
from esm.models.esm3 import ESM3
from esm.sdk.api import ESM3InferenceClient, ESMProtein, GenerationConfig

# Will instruct you how to get an API key from huggingface hub, make one with "Read" permission.
login()

# This will download the model weights and instantiate the model on your machine.
model: ESM3InferenceClient = ESM3.from_pretrained("esm3-sm-open-v1").to("cuda") # or "cpu"
```

--------------------------------

### Download MSA for ESMFold2

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmfold2.ipynb

Download a Multiple Sequence Alignment (MSA) file in A3M format for use with ESMFold2. This example downloads the MSA for ubiquitin.

```bash
!wget -q "https://drive.google.com/uc?export=download&id=1la9UUK_FnFQFR9VB35DIB797d87Zwm0b" -O ubiquitin.a3m
```

--------------------------------

### Training Loop for ESMC Model

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmc_finetune.ipynb

This snippet demonstrates the training loop for the ESMC model. It includes optimizer setup, batch creation, forward pass with loss calculation, backward pass, optimizer step, and periodic evaluation. It also handles data shuffling and progress bar updates.

```python
trainable = [p for p in model.parameters() if p.requires_grad]
optimizer = torch.optim.AdamW(trainable, lr=LEARNING_RATE)

rng = np.random.default_rng(SEED)
n_train = len(train_sequences)

losses: list[float] = []
accuracies: list[float] = []
val_history: list[dict] = []
postfix = {"train_loss": "n/a", "train_acc": "n/a", "val_loss": "n/a", "val_acc": "n/a"}
model.train()

perm = rng.permutation(n_train)
pbar = tqdm(range(NUM_TRAINING_STEPS), desc="training")
for step in pbar:
    start = (step * BATCH_SIZE) % n_train
    if start + BATCH_SIZE > n_train:
        perm = rng.permutation(n_train)
        start = 0
    idx = perm[start : start + BATCH_SIZE]
    batch = make_batch([train_sequences[int(i)] for i in idx], train_labels[idx])

    with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
        outputs = model(**batch)
        # our default model is intialized with a CrossEntropy loss.
        # to experiment with the loss, you can manually compute it here:
        # loss = my_loss-fn(outputs.logits, batch["labels"])
        loss = outputs.loss
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()

    preds = outputs.logits.argmax(dim=-1)
    acc = (preds == batch["labels"]).float().mean().item()
    losses.append(loss.item())
    accuracies.append(acc)

    if (step + 1) % 20 == 0:
        postfix["train_loss"] = f"{np.mean(losses[-20:]):.3f}"
        postfix["train_acc"] = f"{np.mean(accuracies[-20:]):.2f}"
        pbar.set_postfix(postfix)

    if (step + 1) % VAL_EVERY_STEPS == 0 and len(val_sequences) > 0:
        _, val_loss, val_acc = evaluate(model, "val", val_sequences, val_labels)
        val_history.append({"step": step + 1, "loss": val_loss, "acc": val_acc})
        postfix["val_loss"] = f"{val_loss:.3f}"
        postfix["val_acc"] = f"{val_acc:.2f}"
        pbar.set_postfix(postfix)
        model.train()
```

--------------------------------

### Generate Protein with No Cysteines

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esm3_guided_generation.ipynb

Generates a protein of a specified length starting from a fully masked sequence, using guided decoding to avoid cysteine residues. The number of decoding steps and samples per step can be adjusted.

```python
# Start from a fully masked protein
PROTEIN_LENGTH = 256
starting_protein = ESMProtein(sequence="_" * PROTEIN_LENGTH)

# Call guided_generate
no_cysteine_protein = no_cysteine_guided_decoding.guided_generate(
    protein=starting_protein,
    num_decoding_steps=len(starting_protein) // 8,
    num_samples_per_step=10,
)
```

--------------------------------

### Run ESMC Locally with Hugging Face

Source: https://github.com/biohub/esm/blob/main/README.md

Demonstrates how to set up and run the ESMC model locally using Hugging Face libraries. Requires logging in with Hugging Face credentials.

```python
import torch
from transformers import AutoModelForMaskedLM, AutoTokenizer
from huggingface_hub import login

# login with your Hugging Face credentials
login()
```

--------------------------------

### Deploy Binder Design App to Modal

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/binder_design.ipynb

Deploys the 'binder_design.py' application to Modal. This should be run once, or whenever the 'binder_design.py' file is updated.

```bash
# Deploy (or redeploy after changing binder_design.py).
# This only needs to be run a single time, unless code in binder_design.py changes.
! modal deploy binder_design.py
```

--------------------------------

### Initialize 3D Viewer Grid

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esm3_guided_generation.ipynb

Sets up a py3Dmol viewer grid to display multiple protein structures side-by-side for comparison.

```python
# Create a 1x2 grid of viewers (1 row, 2 columns)
view = py3Dmol.view(width=1000, height=500, viewergrid=(1, 2))
```

--------------------------------

### Launch Binder Design Jobs

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/binder_design.ipynb

Spawns binder design jobs using Modal and saves the manifest. This code should be run first to initiate the design process.

```python
df["call_id"] = [
    app.design.spawn(
        target_name=row.target_name,
        target_sequence=row.target_sequence,
        binder_name=row.binder_name,
        binder_sequence=row.binder_sequence,
        seed=row.seed,
        batch_size=row.batch_size,
    ).object_id
    for row in df.itertuples()
]
df.to_parquet(save_dir / "manifest.parquet", index=False)
print(
    f"Spawned {len(df)} jobs. It is safe to close the notebook."
    "The next cell will resume from call_id's, saved by Modal for up to 7 days."
)
```

--------------------------------

### Set up Biohub Client

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esm3_generate.ipynb

Initializes the Biohub client for accessing ESM models. Requires an API token obtained from Biohub. Protect your token as it manages account access and credits.

```python
from getpass import getpass

token = getpass("Token from Biohub: ")
model = client(model="esm3-open", url="https://biohub.ai", token=token)
```

--------------------------------

### Plot InterPro Annotations Example

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmprotein.ipynb

Example usage of the `visualize_function_annotations` function to plot InterPro annotations for a protein sequence. Assumes `interpro_function_annotations` and `protein` objects are defined.

```python
fig, ax = plt.subplots(figsize=(20.0, 4.0))
visualize_function_annotations(interpro_function_annotations, len(protein), ax)
```

--------------------------------

### Configure and Apply LoRA Adapters

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmc_finetune.ipynb

Sets up LoRA configuration with specified rank and alpha, targeting specific modules and parameters for adaptation. Applies the LoRA configuration to the model and prints the trainable parameters.

```python
LORA_RANK = 8
LORA_ALPHA = 16

# layernorm_qkv and ffn fused weights are bare nn.Parameter tensors (not Linear
# submodules), so they are reached via target_parameters rather than target_modules.
lora_config = LoraConfig(
    r=LORA_RANK,
    lora_alpha=LORA_ALPHA,
    lora_dropout=0.01,
    target_modules=["out_proj"],
    target_parameters=["layernorm_qkv.weight", "ffn.fc1_weight", "ffn.fc2_weight"],
    modules_to_save=["classifier"],
)

model = get_peft_model(model, lora_config)
print("\n=== Trainable Parameters ===")
model.print_trainable_parameters()

device = next(model.parameters()).device
```

--------------------------------

### Print pTM Scores

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esm3_guided_generation.ipynb

Prints the pTM scores for proteins generated with and without guidance. This helps in comparing the effectiveness of guided generation.

```python
print(f"pTM Without guidance: {generated_protein_no_guided.ptm:.3f}")
print(f"pTM With guidance: {generated_protein.ptm:.3f}")
```

--------------------------------

### ESM Utility Functions for Masked Inference

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmc_mutation_scoring.ipynb

Provides functions to get logits for a given sequence and to perform leave-one-out masked inference.

```python
def get_logits(client, sequence: str) -> LogitsOutput:
    protein = ESMProtein(sequence=sequence)
    protein_tensor = client.encode(protein)
    if isinstance(protein_tensor, ESMProteinError):
        raise protein_tensor
    return client.logits(protein_tensor, LogitsConfig(sequence=True))


def get_leave_one_out_logits(client, sequence):
    sequences = [sequence[:i] + "_" + sequence[i + 1 :] for i in range(len(sequence))]

    with batch_executor() as executor:
        outputs = executor.execute_batch(
            user_func=get_logits, client=client, sequence=sequences
        )

    return outputs
```

--------------------------------

### Deploy Modal App

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/binder_design.ipynb

Deploys a Modal application from a Python file. Running this on an existing app name redeploys a new version.

```bash
modal deploy path/to/app.py
```

--------------------------------

### Get Biohub API Token

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/embed.ipynb

Prompt the user for their Biohub API token using `getpass` to securely obtain the token without displaying it.

```python
from getpass import getpass

token = getpass("Token from Biohub: ")
```

--------------------------------

### Generate Protein Without Guidance (Baseline)

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esm3_guided_generation.ipynb

Generates a protein sequence without any guidance, serving as a baseline for comparison with guided generation. It then folds the generated sequence.

```python
# Generate a protein WITHOUT guidance
generated_protein_no_guided: ESMProtein = model.generate(
    input=starting_protein,
    config=GenerationConfig(track="sequence", num_steps=len(starting_protein) // 8), # type: ignore
)

# Fold
generated_protein_no_guided: ESMProtein = model.generate(
    input=generated_protein_no_guided,
    config=GenerationConfig(track="structure", num_steps=1), # type: ignore
)
```

--------------------------------

### Initialize ESMC Client with SAE Support

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmc_sae_feature_interpretation.ipynb

Sets up the ESMC client using a Biohub API token. Ensure your token is kept secure and not committed to repositories.

```python
token = getpass("Token from Biohub:")

# Initialize ESMC client with SAE support
model = ESMCForgeInferenceClient(
    model="esmc-6b-2024-12", url="https://biohub.ai", token=token
)

```

--------------------------------

### Initialize Biohub Client

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmc_layer_sweep.ipynb

Sets up the Biohub client using an API token and specifies the ESMC model name. It's recommended to use environment variables or getpass for API token security.

```python
token = getpass("Biohub API token: ")

model_name = "esmc-600m-2024-12"
forge_client = esmc_client(model=model_name, url="https://biohub.ai", token=token)
```

--------------------------------

### Get Color Strings for SASA Visualization

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmprotein.ipynb

Transforms SASA values into RGB color strings based on a colormap and clipping range. Requires numpy and a colormap.

```python
import numpy as np

def get_color_strings(sasa, clip_sasa_lower, clip_sasa_upper, cmap):
    transformed_sasa = np.clip(sasa, clip_sasa_lower, clip_sasa_upper)
    transformed_sasa = (transformed_sasa - clip_sasa_lower) / (
        clip_sasa_upper - clip_sasa_lower
    )
    rgbas = (cmap(transformed_sasa) * 255).astype(int)

    return [f"rgb({rgba[0]},{rgba[1]},{rgba[2]})" for rgba in rgbas]
```

--------------------------------

### Download and Load Enzyme Dataset

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmc_layer_sweep.ipynb

Downloads a pre-built CSV dataset of enzymes from Google Drive and loads it into a pandas DataFrame. It then prints class counts and the total number of proteins in the dataset.

```python
# Download the pre-built EC dataset from Google Drive
!wget --no-check-certificate "https://drive.google.com/uc?export=download&id=1zi-vnJOfvZs2uv3PmSxeiW7EFtK05tvV" -O ec_layer_sweep_dataset.csv

df = pd.read_csv("ec_layer_sweep_dataset.csv")

print("\nClass counts:")
print(df["ec_prefix"].value_counts().sort_index())
print(f"\nFinal dataset: {len(df)} proteins")
df.head()
```

--------------------------------

### Import necessary libraries

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmc_mutation_scoring.ipynb

Imports essential libraries for ESMC analysis, including PyTorch, Matplotlib, NumPy, and specific modules from the esm SDK. Clears the output after installation.

```python
from getpass import getpass

import matplotlib.pyplot as plt
import numpy as np
import torch
import torch.nn.functional as F

%matplotlib inline

from IPython.display import clear_output

from esm.sdk import batch_executor, esmc_client
from esm.sdk.api import ESMProtein, ESMProteinError, LogitsConfig, LogitsOutput
from esm.tokenization import get_esmc_model_tokenizers

clear_output()
```

--------------------------------

### Load and Visualize Protein Structure

Source: https://github.com/biohub/esm/blob/main/cookbook/local/open_generate.ipynb

Loads a protein structure from RCSB and visualizes it using py3Dmol, highlighting a specific region. This is a setup step for secondary structure editing.

```python
helix_shortening_chain = ProteinChain.from_rcsb("7XBQ", "A")
view = py3Dmol.view(width=500, height=500)
view.addModel(helix_shortening_chain.to_pdb_string(), "pdb")
view.setStyle({"cartoon": {"color": "lightgrey"}})
helix_region = np.arange(38, 111)  # zero-indexed
view.addStyle(
    {"resi": (helix_region + 1).tolist()}, {"cartoon": {"color": "lightblue"}}
)
view.zoomTo()
view.show()
```

--------------------------------

### Initialize ESMFold2 Client with Token

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmfold2.ipynb

Initializes the ESMFold2 client using a securely obtained API token. It's recommended to use `getpass` to avoid exposing your token.

```python
token = getpass("Biohub token: ")

client = esmfold2_client(
    model="esmfold2-fast-2026-05", url="https://biohub.ai", token=token
)
```

--------------------------------

### Initialize ESM3 Client (Biohub)

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esm3_guided_generation.ipynb

Initializes the ESM3 client for remote inference on Biohub using larger models. Requires a token from Biohub.

```python
## Locally with ESM3-open
# from esm.models.esm3 import ESM3
# model = ESM3.from_pretrained().to("cuda")

## On Biohub with larger ESM3 models
from getpass import getpass

from esm.sdk import client

token = getpass("Token from Biohub: ")
model = client(model="esm3-medium-2024-08", url="https://biohub.ai", token=token)
```

--------------------------------

### Initialize Layer Sweep Parameters

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmc_layer_sweep.ipynb

Sets up parameters for the layer sweep, including the number of cross-validation splits and a random state for reproducibility. It also prepares the target labels by mapping them to numerical indices.

```python
N_SPLITS = 5  # number of cross-validation folds
RANDOM_STATE = 44

classes = sorted(pd.unique(labels))
y = np.array([classes.index(l) for l in labels])

skf = StratifiedKFold(n_splits=N_SPLITS, shuffle=True, random_state=RANDOM_STATE)
```

--------------------------------

### Get Descriptions for Top Features

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmc_sae_feature_interpretation.ipynb

Retrieves and prints descriptions for the top 5 features ranked by maximum activation. It also shows the top associated UniRef ID for each feature.

```python
print("Top 5 feature descriptions (ranked by max activation):\n")
for idx, feature_id in enumerate(top_by_max[:5], 1):
    info = get_feature_info(feature_id)
    top_uniref = info["top_100_uniref_ids"][0]
    print(f"{idx}. Feature {feature_id}")
    print(f"   description: {info.get('description', 'No description available')}")
    print(f"   top uniref:  {top_uniref}")
```

--------------------------------

### Get SAE Features with Normalization

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmc_sae_feature_interpretation.ipynb

This snippet shows how to obtain SAE features from a protein tensor using a specified model. It configures the SAE to use TF-IDF normalization for features.

```python
sae_model_name = "esmc-6b-2024-12-sae-layer60-k64-codebook16384"
output = model.logits(
    protein_tensor,
    config=LogitsConfig(
        sae_config=SAEConfig(
            model=sae_model_name,
            normalize_features=True,  # TF-IDF normalization
        )
    ),
    return_bytes=False,
)
```

--------------------------------

### Set up ESMC client with API token

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmc_mutation_scoring.ipynb

Initializes the ESMC client using a Biohub API token obtained via getpass. It specifies the model name and the Biohub API URL. The tokenizer is also retrieved for use in subsequent steps.

```python
# Set up ESMC client

token = getpass("Token from Biohub console: ")

model_name = "esmc-600m-2024-12"
forge_client = esmc_client(model=model_name, url="https://biohub.ai", token=token)

tokenizer = get_esmc_model_tokenizers()
VOCAB = tokenizer.get_vocab()
```

--------------------------------

### Import necessary libraries

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esm3_guided_generation.ipynb

Imports core components for protein generation, structure manipulation, and visualization.

```python
import biotite.structure as bs
import py3Dmol

from esm.sdk.api import ESMProtein, GenerationConfig
from esm.sdk.experimental import ESM3GuidedDecoding, GuidedDecodingScoringFunction
```

--------------------------------

### Run ESMC Model for Protein Sequence Representation

Source: https://github.com/biohub/esm/blob/main/README.md

Load the ESMC model and tokenizer to get representations for a given protein sequence. Ensure the model and tokenizer are downloaded and accessible.

```python
sequences = ["MSKGEELFTGVVPILVELDGDVNGHKFSVSGEGEGDATYGKLTLKFICTTGKLPVPWPTLVTTFSYGVQCFSRYPDHMKQHDFFKSAMPEGYVQERTIFFKDDGNYKTRAEVKFEGDTLVNRIELKGIDFKEDGNILGHKLEYNYNSHNVYIMADKQKNGIKVNFKIRHNIEDGSVQLADHYQQNTPIGDGPVLLPDNHYLSTQSALSKDPNEKRDHMVLLEFVTAAGITHGMDELYK"]

model = AutoModelForMaskedLM.from_pretrained(
    "biohub/ESMC-6B",
    device_map="auto",
).eval()
tokenizer = AutoTokenizer.from_pretrained("biohub/ESMC-6B")

inputs = tokenizer(sequences, return_tensors="pt", padding=True)
inputs = {k: v.to(model.device) for k, v in inputs.items()}
with torch.inference_mode():
    output = model(**inputs)
```
```

--------------------------------

### Define No Cysteine Scoring Function

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esm3_guided_generation.ipynb

Defines a custom scoring function that penalizes the presence of cysteine residues in a protein sequence. Use this to guide generation away from sequences containing cysteine.

```python
class NoCysteineScoringFunction(GuidedDecodingScoringFunction):
    def __call__(self, protein: ESMProtein) -> float:
        # Penalize proteins that contain cysteine
        assert protein.sequence is not None, "Protein must have a sequence to be scored"
        # Note that we use a negative score here, to discourage the presence of cysteine
        return -protein.sequence.count("C")
```

--------------------------------

### Get Positions with Low Deleterious Fraction

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/esmc_mutation_scoring.ipynb

Identifies and returns protein sequence positions where the fraction of deleterious substitutions is below a specified threshold. Useful for selecting mutation-tolerant sites for experimental design.

```python
def get_positions_with_low_deleterious_fraction(
    logit_outputs, sequence, threshold=0.8, vocab=VOCAB
):
    """Select positions where the fraction of deleterious substitutions (LLR < 0) is below the given threshold."""
    log_likelihood_ratios = get_per_position_log_likelihood_ratios(
        logit_outputs, sequence
    )  # shape: (L, vocab)
    aa_list = VALID_AMINO_ACIDS  # all 20 AAs
    aa_inds = [vocab[aa] for aa in aa_list]
    llr_valid = log_likelihood_ratios[:, aa_inds]
    # count the number of negative log likelihood ratios
    negative_llr_count = (llr_valid < 0).sum(dim=1)

    # plot the percentage of negative log likelihood ratios as a scatterplot, one dot per position
    percent_negative = negative_llr_count / llr_valid.shape[1]

    # find positions with percent_negative < threshold
    positions = np.arange(1, len(percent_negative) + 1)  # 1-index the sequence position
    mask = percent_negative < threshold
    low_deleterious_positions = positions[mask]
    low_deleterious_fractions = percent_negative[mask]
    return low_deleterious_positions, low_deleterious_fractions
```

```python
good_positions, good_fractions = get_positions_with_low_deleterious_fraction(
    logit_outputs, capetase_sequence, threshold=0.8
)

print("Candidate mutation-tolerant positions (1-indexed):")
print(good_positions[:20])
```

--------------------------------

### Load and Preprocess ADK Data

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/embed.ipynb

Loads the ADK dataset using pandas, selects relevant columns, and filters out entries with 'other' lid types for simplification. Ensure pandas and matplotlib are installed.

```python
import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns

adk_path = "adk.csv"
df = pd.read_csv(adk_path)
df = df[["org_name", "sequence", "lid_type", "temperature"]]
df = df[df["lid_type"] != "other"]  # drop one structural class for simplicity
```

--------------------------------

### Initialize ESM3 Model Client

Source: https://github.com/biohub/esm/blob/main/cookbook/tutorials/gfp_design.ipynb

Creates a client stub for the ESM3 model ('esm3-medium-2024-03') hosted on Biohub. This stub facilitates remote model interaction for generation and other tasks.

```python
model = client(model="esm3-medium-2024-03", url="https://biohub.ai", token=token)
```