### Install the library

Source: https://github.com/lucidrains/bidirectional-cross-attention/blob/main/README.md

Use pip to install the package.

```bash
$ pip install bidirectional-cross-attention
```

--------------------------------

### Install the package

Source: https://context7.com/lucidrains/bidirectional-cross-attention/llms.txt

Install the library via pip.

```bash
pip install bidirectional-cross-attention
```

--------------------------------

### Initialize and use BidirectionalCrossAttention

Source: https://github.com/lucidrains/bidirectional-cross-attention/blob/main/README.md

Demonstrates how to instantiate the model and perform a forward pass with input tensors and masks.

```python
import torch
from bidirectional_cross_attention import BidirectionalCrossAttention

video = torch.randn(1, 4096, 512)
audio = torch.randn(1, 8192, 386)

video_mask = torch.ones((1, 4096)).bool()
audio_mask = torch.ones((1, 8192)).bool()

joint_cross_attn = BidirectionalCrossAttention(
    dim = 512,
    heads = 8,
    dim_head = 64,
    context_dim = 386
)

video_out, audio_out = joint_cross_attn(
    video,
    audio,
    mask = video_mask,
    context_mask = audio_mask
)

# attended output should have the same shape as input

assert video_out.shape == video.shape
assert audio_out.shape == audio.shape
```

--------------------------------

### Initialize and use BidirectionalCrossAttentionTransformer

Source: https://context7.com/lucidrains/bidirectional-cross-attention/llms.txt

Use this module to stack multiple bidirectional cross attention layers with feedforward networks and residual connections for complex cross-modal processing.

```python
import torch
from bidirectional_cross_attention import BidirectionalCrossAttentionTransformer

# Create sample sequences for cross-modal processing
# Example: text embeddings and image patch embeddings
text = torch.randn(4, 256, 768)    # (batch, seq_len_text, dim_text)
image = torch.randn(4, 196, 512)   # (batch, num_patches, dim_image)

# Create masks for variable-length sequences
text_mask = torch.ones((4, 256)).bool()
image_mask = torch.ones((4, 196)).bool()

# Initialize the transformer with multiple layers
transformer = BidirectionalCrossAttentionTransformer(
    dim=768,                    # dimension of source sequence
    depth=6,                    # number of transformer layers
    context_dim=512,            # dimension of context sequence
    heads=8,                    # number of attention heads
    dim_head=64,                # dimension per head
    ff_expansion_factor=4.,     # feedforward expansion factor
    dropout=0.1,                # attention dropout
    talking_heads=True,         # enable talking heads attention
    final_norms=True            # apply final layer normalization
)

# Forward pass through all layers
text_out, image_out = transformer(
    text,
    image,
    mask=text_mask,
    context_mask=image_mask
)

# Outputs have same shape as inputs
assert text_out.shape == text.shape    # (4, 256, 768)
assert image_out.shape == image.shape  # (4, 196, 512)

print(f"Text output shape: {text_out.shape}")
print(f"Image output shape: {image_out.shape}")
```

--------------------------------

### Implement DNA-Protein Binding Prediction Model

Source: https://context7.com/lucidrains/bidirectional-cross-attention/llms.txt

Demonstrates a custom nn.Module using BidirectionalCrossAttentionTransformer to model interactions between DNA and protein sequences. The model projects inputs to a common dimension before applying cross-attention and a final classification head.

```python
import torch
import torch.nn as nn
from bidirectional_cross_attention import BidirectionalCrossAttentionTransformer

class DNAProteinBindingModel(nn.Module):
    def __init__(self, dna_dim=128, protein_dim=256, hidden_dim=512):
        super().__init__()
        # Embedding layers for DNA and protein sequences
        self.dna_embed = nn.Linear(4, dna_dim)      # 4 nucleotides
        self.protein_embed = nn.Linear(20, protein_dim)  # 20 amino acids

        # Project to common dimension
        self.dna_proj = nn.Linear(dna_dim, hidden_dim)
        self.protein_proj = nn.Linear(protein_dim, hidden_dim)

        # Bidirectional cross attention transformer
        self.transformer = BidirectionalCrossAttentionTransformer(
            dim=hidden_dim,
            depth=4,
            context_dim=hidden_dim,
            heads=8,
            dim_head=64,
            dropout=0.1,
            final_norms=True
        )

        # Binding prediction head
        self.classifier = nn.Sequential(
            nn.Linear(hidden_dim * 2, hidden_dim),
            nn.ReLU(),
            nn.Linear(hidden_dim, 1),
            nn.Sigmoid()
        )

    def forward(self, dna_seq, protein_seq, dna_mask=None, protein_mask=None):
        # Embed and project sequences
        dna = self.dna_proj(self.dna_embed(dna_seq))
        protein = self.protein_proj(self.protein_embed(protein_seq))

        # Cross-attend between DNA and protein
        dna_out, protein_out = self.transformer(
            dna, protein,
            mask=dna_mask,
            context_mask=protein_mask
        )

        # Pool and predict binding
        dna_pooled = dna_out.mean(dim=1)
        protein_pooled = protein_out.mean(dim=1)
        combined = torch.cat([dna_pooled, protein_pooled], dim=-1)

        return self.classifier(combined)

# Example usage
batch_size = 8
dna_len = 1000
protein_len = 500

# One-hot encoded sequences
dna_seq = torch.randn(batch_size, dna_len, 4)
protein_seq = torch.randn(batch_size, protein_len, 20)

model = DNAProteinBindingModel()
binding_prob = model(dna_seq, protein_seq)

print(f"Binding probability shape: {binding_prob.shape}")  # (8, 1)
print(f"Binding probabilities: {binding_prob.squeeze()}")
```

--------------------------------

### Citation for Bidirectional Cross Attention

Source: https://github.com/lucidrains/bidirectional-cross-attention/blob/main/README.md

BibTeX citation for the underlying research paper.

```bibtex
@article{Hiller2024PerceivingLS,
    title   = {Perceiving Longer Sequences With Bi-Directional Cross-Attention Transformers},
    author  = {Markus Hiller and Krista A. Ehinger and Tom Drummond},
    journal = {ArXiv},
    year    = {2024},
    volume  = {abs/2402.12138},
    url     = {https://api.semanticscholar.org/CorpusID:267751060}
}
```

--------------------------------

### Apply Relative Position Bias with BidirectionalCrossAttention

Source: https://context7.com/lucidrains/bidirectional-cross-attention/llms.txt

Injects a relative position bias tensor into the attention mechanism to enable position-aware patterns. The bias tensor must match the batch, head, and sequence length dimensions of the input sequences.

```python
import torch
from bidirectional_cross_attention import BidirectionalCrossAttention

# Sequences with known positional relationships
seq_a = torch.randn(1, 100, 256)
seq_b = torch.randn(1, 50, 256)

# Create relative position bias tensor
# Shape: (batch, heads, seq_len_a, seq_len_b)
rel_pos_bias = torch.randn(1, 8, 100, 50)

cross_attn = BidirectionalCrossAttention(
    dim=256,
    heads=8,
    dim_head=32
)

# Apply attention with relative position bias
out_a, out_b = cross_attn(
    seq_a,
    seq_b,
    rel_pos_bias=rel_pos_bias
)

print(f"Output A shape: {out_a.shape}")  # (1, 100, 256)
print(f"Output B shape: {out_b.shape}")  # (1, 50, 256)
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.