tcrdist3

https://tcrdist3.readthedocs.io/en/latest/
Admin
tcrdist3 is an open-source Python package that enables comprehensive T cell receptor sequence...
Tokens:56,627
Snippets:386
Trust Score:9.6
Update:5 months ago
Show doc for...
Tokens:
Raw
### Install TCR Reference Files

Source: https://tcrdist3.readthedocs.io/en/latest/metaclonotypes.html

Installs necessary background reference files for meta-clonotype discovery. Ensure unzip is installed first.

```bash
apt-get install unzip
python3 -c "from tcrsampler.setup_db import install_all_next_gen; install_all_next_gen(dry_run = False)"
```

--------------------------------

### Set up tcrdist3 with Miniconda

Source: https://tcrdist3.readthedocs.io/en/latest/index.html

Create and activate a conda environment for tcrdist3, then install the package and related tools like notebook and ipython. This is an alternative installation method.

```bash
conda create --name tcrdist3 python=3.8
conda activate tcrdist3
pip install tcrdist3
pip install notebook ipython
ipython

```

--------------------------------

### Install tcrdist3 using pip

Source: https://tcrdist3.readthedocs.io/en/latest/index.html

Install the tcrdist3 package from GitHub using pip. Ensure you are using the correct version tag.

```bash
pip install git+https://github.com/kmayerb/tcrdist3.git@0.2.2

```

--------------------------------

### Generate TCRpublic HTML Report with Customizations

Source: https://tcrdist3.readthedocs.io/en/latest/public.html

This example demonstrates a full setup for generating a TCRpublic HTML report. It initializes TCRrep and TCRpublic objects, customizes the report by adding a 'cohort' summary column, setting a query string for 'nsubject > 6', and enabling a fixed radius of 18.

```python
import os
import pandas as pd
import numpy as np
from tcrdist.repertoire import TCRrep
from tcrdist.public import TCRpublic

# <aesr_fn> antigen-enriched subrepertoire
aesr_fn = os.path.join(
    'tcrdist',
    'data',
    'covid19',
    'mira_epitope_55_524_ALRKVPTDNYITTY_KVPTDNYITTY.tcrdist3.csv')

# <aesr_df> antigen-enriched subrepertoire
aesr_df = pd.read_csv(aesr_fn)
# <tr> TCR repertoire
tr = TCRrep(
    cell_df = aesr_df[[
        'cohort',
        'subject',
        'v_b_gene', 
        'j_b_gene',
        'cdr3_b_aa']].copy(), 
    organism = 'human', 
    chains = ['beta'], 
    db_file = 'alphabeta_gammadelta_db.tsv', 
    compute_distances = True)
# <tp> TCRpublic class for reporting publicities 
tp = TCRpublic(
    tcrrep = tr, 
    output_html_name = "quasi_public_clones2.html")
# set to True, if we want a universal radius
tp.fixed_radius = True
# must then specify maximum distance for finding similar TCRs
tp.radius = 18
# set criteria for being quasi-public
tp.query_str = 'nsubject > 6'
# Add additional columns to be summarized in the report
tp.kargs_member_summ['addl_cols'] = ['subject', 'cohort']
# Add cohort.summary to the labels column so it shows up in the report
tp.labels.append("cohort.summary")
# by calling, .report() an html report is made
public = tp.report()
```

--------------------------------

### Install Dependencies on OSX

Source: https://tcrdist3.readthedocs.io/en/latest/index.html

Install necessary dependencies for tcrdist3 on OSX using Homebrew. This step is crucial for the parasail library to build correctly.

```bash
brew install autoconf automake libtool

```

--------------------------------

### Set up tcrdist3 with venv

Source: https://tcrdist3.readthedocs.io/en/latest/index.html

Create a virtual environment for tcrdist3 using Python's venv module, activate it, and install the parasail and tcrdist3 packages. This method uses standard Python virtual environments.

```bash
python3 -m venv ./tcr3
source tcr3/bin/activate
pip3 install parasail==1.1.17
pip3 install tcrdist3

```

--------------------------------

### Complete Control Over TCRdist3 Parameters

Source: https://tcrdist3.readthedocs.io/en/latest/tcrdistances.html

This example demonstrates how to gain complete control over every parameter in TCRdist3. It involves setting up TCRrep with custom metrics, weights, and keyword arguments for distance calculations.

```python
import pwseqdist as pw
import pandas as pd
from tcrdist.repertoire import TCRrep

df = pd.read_csv("dash.csv")
tr = TCRrep(cell_df = df, 
            organism = 'mouse', 
            chains = ['alpha','beta'], 
            compute_distances = False,
            db_file = 'alphabeta_gammadelta_db.tsv')

metrics_a = {
    "cdr3_a_aa" : pw.metrics.nb_vector_tcrdist,
    "pmhc_a_aa" : pw.metrics.nb_vector_tcrdist,
    "cdr2_a_aa" : pw.metrics.nb_vector_tcrdist,
    "cdr1_a_aa" : pw.metrics.nb_vector_tcrdist}

metrics_b = {
    "cdr3_b_aa" : pw.metrics.nb_vector_tcrdist,
    "pmhc_b_aa" : pw.metrics.nb_vector_tcrdist,
    "cdr2_b_aa" : pw.metrics.nb_vector_tcrdist,
    "cdr1_b_aa" : pw.metrics.nb_vector_tcrdist }

weights_a= {
    "cdr3_a_aa" : 3,
    "pmhc_a_aa" : 1,
    "cdr2_a_aa" : 1,
    "cdr1_a_aa" : 1}

weights_b = {
    "cdr3_b_aa" : 3,
    "pmhc_b_aa" : 1,
    "cdr2_b_aa" : 1,
    "cdr1_b_aa" : 1}

kargs_a = {  
    'cdr3_a_aa' : 
        {'use_numba': True, 
        'distance_matrix': pw.matrices.tcr_nb_distance_matrix, 
        'dist_weight': 1, 
        'gap_penalty':4, 
        'ntrim':3, 
        'ctrim':2, 
        'fixed_gappos': False},
    'pmhc_a_aa' : {
        'use_numba': True,
        'distance_matrix': pw.matrices.tcr_nb_distance_matrix,
        'dist_weight':1,
        'gap_penalty':4,
        'ntrim':0,
        'ctrim':0,
        'fixed_gappos':True},
    'cdr2_a_aa' : {
        'use_numba': True,
        'distance_matrix': pw.matrices.tcr_nb_distance_matrix,
        'dist_weight': 1,
        'gap_penalty':4,
        'ntrim':0,
        'ctrim':0,
        'fixed_gappos':True},
    'cdr1_a_aa' : {
        'use_numba': True,
        'distance_matrix': pw.matrices.tcr_nb_distance_matrix,
        'dist_weight':1,
        'gap_penalty':4,
        'ntrim':0,
        'ctrim':0,
        'fixed_gappos':True}
    }
kargs_b= {  
    'cdr3_b_aa' : 
        {'use_numba': True, 
        'distance_matrix': pw.matrices.tcr_nb_distance_matrix, 
        'dist_weight': 1, 
        'gap_penalty':4, 
        'ntrim':3, 
        'ctrim':2, 
        'fixed_gappos': False},
    'pmhc_b_aa' : {
        'use_numba': True,
        'distance_matrix': pw.matrices.tcr_nb_distance_matrix,
        'dist_weight': 1,
        'gap_penalty':4,
        'ntrim':0,
        'ctrim':0,
        'fixed_gappos':True},
    'cdr2_b_aa' : {
        'use_numba': True,
        'distance_matrix': pw.matrices.tcr_nb_distance_matrix,
        'dist_weight':1,
        'gap_penalty':4,
        'ntrim':0,
        'ctrim':0,
        'fixed_gappos':True},
    'cdr1_b_aa' : {
        'use_numba': True,
        'distance_matrix': pw.matrices.tcr_nb_distance_matrix,
        'dist_weight':1,
        'gap_penalty':4,
        'ntrim':0,
        'ctrim':0,
        'fixed_gappos':True}
    }   

tr.metrics_a = metrics_a
tr.metrics_b = metrics_b

tr.weights_a = weights_a
tr.weights_b = weights_b

tr.kargs_a = kargs_a 
tr.kargs_b = kargs_b

```

--------------------------------

### Tabulate Meta-clonotype Frequency and Counts

Source: https://tcrdist3.readthedocs.io/en/latest/metaclonotypes.html

Use this example to tabulate the frequency and counts of each meta-clonotype across multiple bulk samples. Ensure you provide a valid path to your bulk files directory.

```bash
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
```

--------------------------------

### Example: Plotting Gene Pairings with Simulated Data

Source: https://tcrdist3.readthedocs.io/en/latest/visualizing.html

Demonstrates how to use the `tcrdist.plotting.plot_pairings` function with simulated pandas DataFrame containing TCR gene usage data and counts.

```APIDOC
## Example: Plotting Gene Pairings with Simulated Data

### Description
A popular feature of tcrdist is the ability to visualize the gene usage pattern of a T cell repertoire specific for a particular epitope. This feature has been re-implemented in tcrdist2 and is part of the plotting module. To illustrate how tcrdist.plotting.plot_pairings() is based on a data frame of gene usage we simulate some data. Notice only a few columns are necessary: (i) one for each gene (ii) and one specifying the count.

### Code Example
```python
import tcrdist as td
import pandas as pd
import numpy as np
import IPython
from tcrdist import plotting

np.random.seed(110820)
n = 50
df = pd.DataFrame({
    'v_a_gene': np.random.choice(['TRAV14', 'TRAV12', 'TRAV3', 'TRAV23', 'TRAV11', 'TRAV6', 'TRAV89'], n),
    'j_a_gene': np.random.choice(['TRAJ4', 'TRAJ2', 'TRAJ3', 'TRAJ5', 'TRAJ21', 'TRAJ13'], n),
    'v_b_gene': np.random.choice(['TRBV14', 'TRBV12', 'TRBV3'], n),
    'j_b_gene': np.random.choice(['TRBJ4', 'TRBJ2', 'TRBJ3', 'TRBJ5', 'TRBJ21', 'TRBJ13'], n)
})

df = df.assign(count=1)
df.loc[:10, 'count'] = 10 # Sim expansion of the genes used in the first 10 rows

svg = plotting.plot_pairings(
    cell_df=df,
    cols=['j_a_gene', 'v_a_gene', 'v_b_gene', 'j_b_gene'],
    count_col='count'
)
IPython.display.SVG(data=svg)
```

### Example: Bulk Unpaired Single-Chain Data

### Description
The same type of visualization can be made for bulk unpaired single-chain data, by specifying the cols list as follows. Note the plot depends on the gene order specified in the `cols` argument.

### Code Example
```python
svg = td.plotting.plot_pairings(
    cell_df=df,
    cols=['v_a_gene', 'j_a_gene'],
    count_col='count'
)
IPython.display.SVG(data=svg)
```
```

--------------------------------

### Add Radius, Motif, and Unique Clone Columns

Source: https://tcrdist3.readthedocs.io/en/latest/join.html

Adds boolean columns for radius and motif matching, and a column for unique clone counts. Requires prior setup of 'dist', 'radius_search', 'cdr3_b_aa_bulk', and 'regex_search' columns.

```python
df_join['RADIUS'] = df_join.apply(lambda x: x['dist'] <= x['radius_search'], axis = 1)
import re
df_join['MOTIF'] = df_join.apply(lambda x: re.search(string = x['cdr3_b_aa_bulk'], pattern = x['regex_search']) is not None, axis = 1)
df_join['RADIUSANDMOTIF'] =  df_join['RADIUS'] & df_join['MOTIF']
df_join['unique_clones'] = 1
```

--------------------------------

### Run Docker Container with Mounted Volume

Source: https://tcrdist3.readthedocs.io/en/latest/docker.html

Starts a tcrdist3 Docker container and mounts a local directory (${HOME}/mydata/) to a directory inside the container (/yourdata/). This allows for data persistence between container sessions. The `ls data` command is executed within the container to show the contents of the mounted directory.

```bash
docker run -v ${HOME}/mydata/:/yourdata/ -it quay.io/kmayerb/tcrdist3:0.1.9
ls data
```

--------------------------------

### Loading Adaptive Data into TCRrep

Source: https://tcrdist3.readthedocs.io/en/latest/adaptive.html

Example of loading processed Adaptive data into the TCRrep object for further analysis.

```APIDOC
## Loading Adaptive ImmunoSEQ Files

### Description
This section demonstrates how to load the cleaned Adaptive ImmunoSEQ data into a `TCRrep` object using the `import_adaptive_file` function and specifying parameters for analysis.

### Method
Python script using pandas, tcrdist.repertoire, and tcrdist.adpt_funcs.

### Code Example
```python
import pandas as pd
from tcrdist.repertoire import TCRrep
from tcrdist.adpt_funcs import import_adaptive_file, adaptive_to_imgt

df = import_adaptive_file(adaptive_filename = 'Adaptive2020.tsv')
# For larger datasets, make sure compute_distances is set to False
tr = TCRrep(cell_df = df, 
            organism = 'human', 
            chains = ['beta'], 
            db_file = 'alphabeta_gammadelta_db.tsv', compute_distances = False)
```
```

--------------------------------

### Calculate Mouse TCR Pgen Estimates

Source: https://tcrdist3.readthedocs.io/en/latest/pGen.html

Demonstrates how to compute Pgen estimates for mouse TCR sequences using OlgaModel and TCRrep. This example also highlights identifying low Pgen TCRs with convergent selection.

```python
import pandas as pd
from tcrdist.repertoire import TCRrep
from tcrdist.pgen import OlgaModel
import numpy as np

df = pd.read_csv("dash.csv")
tr = TCRrep(cell_df = df, 
            organism = 'mouse', 
            chains = ['alpha','beta'], 
            db_file = 'alphabeta_gammadelta_db.tsv',
            compute_distances = True)

# Load OLGA model as a python object
olga_beta  = OlgaModel(chain_folder = "mouse_T_beta", recomb_type="VDJ")
olga_alpha = OlgaModel(chain_folder = "mouse_T_alpha", recomb_type="VJ")

# An example computing a single Pgen
olga_beta.compute_aa_cdr3_pgen(tr.clone_df['cdr3_b_aa'][0])
olga_alpha.compute_aa_cdr3_pgen(tr.clone_df['cdr3_a_aa'][0])

# An example computing multiple Pgens 
olga_beta.compute_aa_cdr3_pgens(tr.clone_df['cdr3_b_aa'][0:5])
olga_alpha.compute_aa_cdr3_pgens(tr.clone_df['cdr3_a_aa'][0:5])
```

--------------------------------

### Compute Sparse Pairwise Distances

Source: https://tcrdist3.readthedocs.io/en/latest/welcome.html

This example shows how to compute pairwise distances using a sparse representation for large datasets. Set compute_distances to False during initialization and use `compute_sparse_rect_distances` with specified radius and chunk size. Adjust `tr.cpus` based on your system's cores. Note that true 0 distances are represented as -1 in the sparse matrix.

```python
import pandas as pd
from tcrdist.repertoire import TCRrep

df = pd.read_csv("dash.csv")
tr = TCRrep(cell_df = df,
            organism = 'mouse',
            chains = ['alpha','beta'],
            db_file = 'alphabeta_gammadelta_db.tsv',
            compute_distances = False)

tr.cpus = 2
tr.compute_sparse_rect_distances(radius = 50, chunk_size = 100)
tr.rw_beta
```

--------------------------------

### Prepare Output Directory and File Paths

Source: https://tcrdist3.readthedocs.io/en/latest/metaclonotypes.html

Sets up a tag for results, defines filenames for ranked centers and benchmark files, and creates a destination directory if it doesn't exist. It also specifies the path for bulk files.

```python
tag_level = '1E6'
		# <tag> the results with this symbol
		tag = f"{row['set']}_{tag_level}"
		ranked_centers_fn = f"{row['filename']}.ranked_centers_bkgd_ctlr_{tag_level}.tsv"
		benchmark_fn      = f"{row['filename']}.ranked_centers_bkgd_ctlr_{tag_level}.tsv.benchmark_tabulation.tsv"
		# <dest> place where .tsv files shall be saved
		dest = f'/fh/fast/gilbert_p/fg_data/tcrdist/t3/{tag}'
		if not os.path.isdir(dest):
			os.mkdir(dest)

		# <path> path where bulk files can be found 
		path = path_bulkfiles
```

--------------------------------

### Initialize TCR Sampler and Download Background

Source: https://tcrdist3.readthedocs.io/en/latest/motif_gallery.html

Initializes TCRsampler and downloads a background file for mouse TCR repertoire analysis. It then creates sampler instances for both beta and alpha chains using specific background files.

```python
from tcrsampler.sampler import TCRsampler

t = TCRsampler()
t.download_background_file("ruggiero_mouse_sampler.zip")
tcrsampler_beta = TCRsampler(default_background = 'ruggiero_mouse_beta_t.tsv.sampler.tsv')
tcrsampler_alpha = TCRsampler(default_background = 'ruggiero_mouse_alpha_t.tsv.sampler.tsv')
```

--------------------------------

### Initialize TCRsampler and Sample TCRs

Source: https://tcrdist3.readthedocs.io/en/latest/motif_gallery.html

Initializes the TCRsampler with a default background file and samples TCRs based on gene usage. Requires pandas for DataFrame creation.

```python
from tcrsampler.sampler import TCRsampler
import pandas as pd

t = TCRsampler()
#t.download_background_file("ruggiero_mouse_sampler.zip")
t = TCRsampler(default_background = 'ruggiero_mouse_beta_t.tsv.sampler.tsv')

df = pd.DataFrame( {
    "cdr3_b_aa": ['CASSPVRAGDTQYF', 'CASSPIRVGDTQYF', 'CASSPVRLGDTQYF'],
    "v_b_gene":['TRBV29*01', 'TRBV29*01','TRBV29*01'],
    "j_b_gene":['TRBJ2-7*01','TRBJ2-7*01','TRBJ2-5*01']
    })
gene_usage = df.groupby(['v_b_gene','j_b_gene']).size()
t.sample( gene_usage.reset_index().to_dict('split')['data'],
    flatten = True,
    depth = 2,
    seed = 1)

```

--------------------------------

### Modify TCRpublic Query String

Source: https://tcrdist3.readthedocs.io/en/latest/public.html

Changes the criteria for defining public clonotypes. This example sets the query to identify clonotypes present in more than 8 subjects.

```python
tp.query_str = 'nsubject > 8'
```

--------------------------------

### Initialize and Build TCR Sampler

Source: https://tcrdist3.readthedocs.io/en/latest/influenza_example.html

Initializes a TCRsampler object and builds a background distribution for sampling TCR sequences. This is useful for motif analysis. Requires tcrsampler.

```python
""" SEE TCRSAMPLER (https://github.com/kmayerb/tcrsampler/blob/master/docs/tcrsampler.md)
Here we used olga human alpha synthetic sequences for best coverage
"""
from tcrsampler.sampler import TCRsampler
t = TCRsampler()
#t.download_background_file('olga_sampler.zip') # ONLY IF NOT ALREADY DONE
tcrsampler_alpha = TCRsampler(default_background = 'olga_human_alpha_t.sampler.tsv')
tcrsampler_alpha.build_background(max_rows = 1000)

```

--------------------------------

### Create HTML Summary of Meta-Clonotypes

Source: https://tcrdist3.readthedocs.io/en/latest/metaclonotypes.html

Writes an HTML file summarizing the ranked meta-clonotypes, including their SVG logos and tabular data. The 'shrink' function is used to resize the SVGs. Requires pandas for DataFrame to HTML conversion.

```python
def shrink(s):
    return s.replace('height="100%"', 'height="20%"').replace('width="100%"', 'width="20%"')
labels =['cdr3_b_aa','v_b_gene', 'j_b_gene', 'pgen',
        'radius', 'regex','nsubject','K_neighbors', 
        'bkgd_hits_weighted','chi2dist','chi2re','chi2joint']

output_html_name = os.path.join(project_path, f'{antigen_enriched_file}.ranked_centers_bkgd_ctlr_{level_tag}.html')
with open(output_html_name, 'w') as output_handle:
    for i,r in ranked_centers_df.iterrows():
        svg, svg_raw = r['svg'],r['svg_raw']
        output_handle.write("<br></br>")
        output_handle.write(shrink(svg))
        output_handle.write(shrink(svg_raw))
        output_handle.write("<br></br>")
        output_handle.write(pd.DataFrame(r[labels]).transpose().to_html())
        output_handle.write("<br></br>")
```

--------------------------------

### Plot Pgen vs. Neighbors

Source: https://tcrdist3.readthedocs.io/en/latest/pGen.html

Generates a scatter plot visualizing the relationship between the negative log10 of generation probability and the number of K-neighbors. Facets are used to separate plots by epitope. Ensure plotnine is installed for this functionality.

```python
import plotnine
from plotnine import ggplot, geom_point, aes, ylab, xlab
(ggplot(tr.clone_df, 
    aes(x= 'pgen_cdr3_b_aa_nlog10', 
        y ='K_neighbors',
        color = 'nsubject')) + 
    geom_point(size = .1) + 
    plotnine.scales.xlim(0,20) + 
    plotnine.facets.facet_wrap('epitope', scales = "free_y")+ylab('Neighbors') + xlab('-Log10 Pgen'))
```

--------------------------------

### List Available Zip Files

Source: https://tcrdist3.readthedocs.io/en/latest/downloads.html

Use this function to see all available zip files that can be downloaded for testing. The returned list can be used to specify which zip file to download.

```python
from tcrdist.setup_tests import list_available_zip_files
list_available_zip_files()
```

--------------------------------

### Generate TCRpublic Report with Sequence-Specific Radius

Source: https://tcrdist3.readthedocs.io/en/latest/public.html

This example demonstrates how to configure TCRpublic to use a sequence-specific radius for similarity calculations. It initializes TCRrep with a 'radius' column in the DataFrame and sets `fixed_radius` to `False` in TCRpublic. The query string is set to 'nsubject > 5', and additional summary columns 'subject' and 'cohort' are included.

```python
import os
import pandas as pd
import numpy as np
from tcrdist.repertoire import TCRrep
from tcrdist.public import TCRpublic

fn = os.path.join(
    'tcrdist',
    'data',
    'covid19',
    'mira_epitope_55_524_ALRKVPTDNYITTY_KVPTDNYITTY.tcrdist3.radius.csv')

df = pd.read_csv(fn)

tr = TCRrep(cell_df = df[['cohort','subject','v_b_gene', 'j_b_gene','cdr3_b_aa', 'radius']], 
            organism = "human", 
            chains = ["beta"])

tp = TCRpublic(
    tcrrep = tr, 
    output_html_name = "quasi_public_clones3.html")

# set to True, if we want a universal radius
tp.fixed_radius = False
# must then specify maximum distance for finding similar TCRs
tp.radius = None
# set criteria for being quasi-public
tp.query_str = 'nsubject > 5'
# Add additional columns to be summarized in the report
tp.kargs_member_summ['addl_cols'] = ['subject', 'cohort']
# Add cohort.summary to the labels column so it shows up in the report
tp.labels.append("cohort.summary")
tp.labels.append("cdr3s")
```

--------------------------------

### Verify Downloaded Files

Source: https://tcrdist3.readthedocs.io/en/latest/downloads.html

After downloading and extracting a zip file, these assertions verify that the main zip file and its individual contents have been successfully created in the destination directory. This is useful for ensuring the integrity of the downloaded test data.

```python
import os
assert os.path.isfile("dash.zip")
```

```python
assert os.path.isfile("dash.csv")
assert os.path.isfile("dash2.csv")
assert os.path.isfile("dash_human.csv")
assert os.path.isfile("dash_beta_airr.csv")
```

--------------------------------

### Initialize TCRrep Instance

Source: https://tcrdist3.readthedocs.io/en/latest/radius.html

Load cell data into a TCRrep instance to infer CDR regions. Requires specifying organism, chains, and a database file. Distance computation can be enabled.

```python
tr = TCRrep(cell_df = df.copy(), 
            organism = 'mouse', 
            chains = ['beta'], 
            db_file = 'alphabeta_gammadelta_db.tsv',
            compute_distances = True)
```

--------------------------------

### Initialize TCRrep and Access Distance Matrices

Source: https://tcrdist3.readthedocs.io/en/latest/welcome.html

Use this snippet to initialize the TCRrep object with your data and access pre-computed pairwise distance matrices for alpha and beta chains, as well as CDR3 regions. Ensure 'dash.csv' and 'alphabeta_gammadelta_db.tsv' are in your working directory.

```python
import pandas as pd
from tcrdist.repertoire import TCRrep

df = pd.read_csv("dash.csv")
tr = TCRrep(cell_df = df, 
            organism = 'mouse', 
            chains = ['alpha','beta'], 
            db_file = 'alphabeta_gammadelta_db.tsv')

tr.pw_alpha
tr.pw_beta
tr.pw_cdr3_a_aa
tr.pw_cdr3_b_aa
```

--------------------------------

### Import VDJtools Data with Python

Source: https://tcrdist3.readthedocs.io/en/latest/vdjtools.html

Use this snippet to import VDJtools formatted data. Ensure your input file has the required columns: 'count', 'freq', 'cdr3aa', 'v', 'j', 'cdr3nt'. The function validates the input and can directly create a TCRrep instance.

```python
import pandas as pd
import numpy as np
import os
from tcrdist.paths import path_to_base
from tcrdist.vdjtools_funcs import import_vdjtools
from tcrdist.repertoire import TCRrep

# Reformat vdj_tools input format for tcrdist3
vdj_tools_file_beta = os.path.join(path_to_base, 'tcrdist','data','formats','vdj.M_15_CD8_beta.clonotypes.TRB.txt.gz')
df_beta = import_vdjtools(   vdj_tools_file = vdj_tools_file_beta ,
                    chain = 'beta',
                    organism = 'human',
                    db_file = 'alphabeta_gammadelta_db.tsv',
                    validate = True)
assert np.all(df_beta.columns == ['count', 'freq', 'cdr3_b_aa', 'v_b_gene', 'j_b_gene', 'cdr3_b_nucseq','valid_v', 'valid_j', 'valid_cdr3'])

# Can be directly imported into a TCRrep instance.
tr = TCRrep(
    cell_df = df_beta[['count', 'freq', 'cdr3_b_aa', 'v_b_gene', 'j_b_gene']], 
    chains = ['beta'], 
    organism = 'human', 
    compute_distances = False)

```

--------------------------------

### Download and extract data files

Source: https://tcrdist3.readthedocs.io/en/latest/join.html

Downloads necessary data files if they do not already exist. This includes bulk repertoires and meta-clonotype definitions.

```python
files = ['1588BW_20200417_PBMC_unsorted_cc1000000_ImmunRACE_050820_008_gDNA_TCRB.tsv.tcrdist3.tsv']
if not np.all([os.path.isfile(f) for f in files]):
    download_and_extract_zip_file("ImmunoSeq_MIRA_matched_tcrdist3_ready_2_files.zip", source = "dropbox", dest = ".")
if not os.path.isfile("bioRxiv_v2_metaclonotypes.tsv.zip"):
    download_and_extract_zip_file('bioRxiv_v2_metaclonotypes.tsv.zip', source = "dropbox", dest = ".")
```

--------------------------------

### Simulate Mouse T-beta CDR3 Sequences

Source: https://tcrdist3.readthedocs.io/en/latest/simulate.html

Generates one million simulated T-beta CDR3 sequences using specified model parameters and saves them to a CSV file. Requires tcrdist library and default model files.

```python
dfb= generate_simulated_beta_seqs(params_file_name = 'tcrdist/default_models/mouse_T_beta/model_params.txt',
                                marginals_file_name = 'tcrdist/default_models/mouse_T_beta/model_marginals.txt',
                                V_anchor_pos_file ='tcrdist/default_models/mouse_T_beta/V_gene_CDR3_anchors.csv',
                                J_anchor_pos_file = 'tcrdist/default_models/mouse_T_beta/J_gene_CDR3_anchors.csv',
                                output_cols = ['cdr3_b_aa', "v_b_gene",'j_b_gene'], n = 1000000)
```

--------------------------------

### Copy File from Docker Container

Source: https://tcrdist3.readthedocs.io/en/latest/docker.html

This snippet demonstrates how to copy a file from a running Docker container to your host machine. First, find the container ID using `docker ps`, then use `docker cp` to copy the specified file.

```bash
docker ps
```

```bash
docker cp 3a51162eb585:hierdiff_example_PA_v_PB1.html hierdiff_example_PA_v_PB1.html
```

--------------------------------

### Search Bulk Repertoire for Meta-Clonotypes using tcrdist3

Source: https://tcrdist3.readthedocs.io/en/latest/metaclonotypes.html

This script demonstrates searching a single bulk repertoire file for the presence of meta-clonotypes. It requires downloading a raw ImmunoSEQ file, formatting it for tcrdist3, and then using a meta-clonotype file for the search. Ensure the necessary database files and meta-clonotype files are accessible.

```python
import os
import pandas as pd 
from tcrdist.adpt_funcs import import_adaptive_file
from tcrdist.repertoire import TCRrep
from tcrdist.tabulate import tabulate
import math
import time
"""
You can download the file with wget or follow the link
!wget https://www.dropbox.com/s/1zbcf1ep8kgmlzy/KHBR20-00107_TCRB.tsv
"""
bulk_file = 'KHBR20-00107_TCRB.tsv'
df_bulk = import_adaptive_file(bulk_file)
df_bulk = df_bulk[['cdr3_b_aa',
                    'v_b_gene',
                    'j_b_gene',
                    'templates',
                    'productive_frequency',
                    'valid_cdr3']].\
        rename(columns = {'templates':'count'})

df_bulk = df_bulk[(df_bulk['v_b_gene'].notna()) & (df_bulk['j_b_gene'].notna())].reset_index()
tr_bulk = TCRrep(cell_df = df_bulk,
                 organism = 'human',
                 chains = ['beta'],
                 db_file = 'alphabeta_gammadelta_db.tsv',
                 compute_distances = False)

search_file = os.path.join(
    'tcrdist',
    'data',
    'covid19',
    'mira_epitope_48_610_YLQPRTFL_YLQPRTFLL_YYVGYLQPRTF.tcrdist3.csv.ranked_centers_bkgd_ctlr_1E6.tsv')

df_search = pd.read_csv(search_file, sep = '\t')
df_search = df_search[['cdr3_b_aa','v_b_gene','j_b_gene','pgen','radius','regex']]
tr_search = TCRrep(cell_df = df_search,
                   organism = 'human',
                   chains = ['beta'],
                   db_file = 'alphabeta_gammadelta_db.tsv',
                   compute_distances = False)
tr_search.cpus = 1
tic = time.perf_counter()
tr_search.compute_sparse_rect_distances(df = tr_search.clone_df, df2 = tr_bulk.clone_df, chunk_size = 50, radius = 50) 
results = tabulate(clone_df1 = tr_search.clone_df, clone_df2 = tr_bulk.clone_df, pwmat = tr_search.rw_beta)
toc = time.perf_counter()
print(f"TABULATED IN {toc - tic:0.4f} seconds")

```

--------------------------------

### Initialize TCRrep for Background Data

Source: https://tcrdist3.readthedocs.io/en/latest/radius.html

Load the synthesized background data into a new TCRrep instance. Distance computation is disabled for this instance.

```python
trb = TCRrep(cell_df = df_vj_background.copy(), 
            organism = 'mouse', 
            chains = ['beta'], 
            db_file = 'alphabeta_gammadelta_db.tsv',
            compute_distances = False)
```

--------------------------------

### Load and Initialize TCR Data

Source: https://tcrdist3.readthedocs.io/en/latest/bulkdata.html

Loads TCR data from a CSV file and initializes a TCRrep object. Ensure 'dash.csv' is available and specify organism, chains, and database file.

```python
from hierdiff.association_testing import cluster_association_test

df = pd.read_csv("dash.csv")
tr = TCRrep(cell_df = df.sample(100, random_state = 1),
            organism = 'mouse',
            chains = ['alpha','beta'],
            db_file = 'alphabeta_gammadelta_db.tsv',
            compute_distances = True,
            store_all_cdr = False)
```

--------------------------------

### Configure CPU Usage

Source: https://tcrdist3.readthedocs.io/en/latest/radius.html

Set the number of CPUs to use for parallel processing. This can significantly speed up computations.

```python
tr.cpus = 4
```

--------------------------------

### Download Simulated CDR3 Data

Source: https://tcrdist3.readthedocs.io/en/latest/simulate.html

Downloads and extracts a zip file containing one million simulated T-alpha and T-beta CDR3 sequences. This utility simplifies obtaining pre-generated simulation data.

```python
from tcrdist.setup_tests import download_and_extract_zip_file
download_and_extract_zip_file('olga_T_alpha_beta_1000K_simulated_cdr3.zip')
```

--------------------------------

### Initialize TCRrep objects

Source: https://tcrdist3.readthedocs.io/en/latest/join.html

Initializes two TCRrep objects: one for the meta-clonotype search set and one for the bulk repertoire. Distance computation is initially set to False.

```python
from tcrdist.repertoire import TCRrep

tr = TCRrep(
    cell_df = df_search,
    organism = "human",
    chains = ['beta'],
    compute_distances= False)
tr.cpus = ncpus

tr_bulk = TCRrep(
    cell_df = df_bulk,
    organism = "human",
    chains = ['beta'],
    compute_distances= False)
```

--------------------------------

### Main execution block for simulating CDR3 sequences

Source: https://tcrdist3.readthedocs.io/en/latest/simulate.html

This block demonstrates how to use the simulation functions to generate and save 1 million CDR3 sequences for human T-beta and T-alpha chains. It specifies the output file names.

```python
if __name__ == "__main__":
    """
    Using Olga See: 
    ---------------
    Zachary Sethna, Yuval Elhanati, Curtis G Callan, Aleksandra M Walczak, Thierry Mora
    `Bioinformatics (2019) <https://doi.org/10.1093/bioinformatics/btz035>`_ 
    OLGA: fast computation of generation probabilities of B- and T-cell receptor amino acid sequences and motifs


    Generate 1000K (1M) CDR3s using default Olga Models
    Human (Alpha/Beta) and Mouse (Beta)

    human_T_alpha_sim1000K.csv
    human_T_beta_sim1000K.csv
    mouse_T_beta_sim1000K.csv
    
    contained in: 
    olga_T_alpha_beta_1000K_simulated_cdr3.zip 
    """
    dfb= generate_simulated_beta_seqs(params_file_name = 'tcrdist/default_models/human_T_beta/model_params.txt',
                                marginals_file_name = 'tcrdist/default_models/human_T_beta/model_marginals.txt',
                                V_anchor_pos_file ='tcrdist/default_models/human_T_beta/V_gene_CDR3_anchors.csv',
                                J_anchor_pos_file = 'tcrdist/default_models/human_T_beta/J_gene_CDR3_anchors.csv',
                                output_cols = ['cdr3_b_aa', "v_b_gene",'j_b_gene'], n = 1000000)
    dfb.to_csv('human_T_beta_sim1000K.csv', index = False)

    dfa = generate_simulated_alpha_seqs(params_file_name = 'tcrdist/default_models/human_T_alpha/model_params.txt',
                                marginals_file_name = 'tcrdist/default_models/human_T_alpha/model_marginals.txt',
                                V_anchor_pos_file ='tcrdist/default_models/human_T_alpha/V_gene_CDR3_anchors.csv',
```

--------------------------------

### Run tcrdist3 Docker Container Interactively

Source: https://tcrdist3.readthedocs.io/en/latest/docker.html

Launches an interactive session within the tcrdist3 Docker container. This is useful for exploring the environment and running Python code.

```bash
docker run -it quay.io/kmayerb/tcrdist3:0.1.9
```

--------------------------------

### Load and preprocess dataframes

Source: https://tcrdist3.readthedocs.io/en/latest/join.html

Loads the bulk repertoire and meta-clonotype data into pandas DataFrames. It also sorts the bulk data by count and adds a rank column.

```python
df_search = pd.read_csv("bioRxiv_v2_metaclonotypes.tsv", sep = "\t")
df_bulk = pd.read_csv(files[0], sep = "\t")

df_bulk = df_bulk.sort_values('count').reset_index(drop = True)
df_bulk['rank'] = df_bulk.index.to_list()
```

--------------------------------

### Calculate Neighbor Radii and Thresholds

Source: https://tcrdist3.readthedocs.io/en/latest/radius.html

Compute radii and thresholds for neighbor discovery in the background repertoire. Sparse matrix storage is enabled for efficiency, limiting distances to a maximum radius.

```python
radii, thresholds, ecdfs = \
    calc_radii(tr = tr, 
        tr_bkgd = trb, 
        chain = 'beta', 
        ctrl_bkgd = 10**-5, 
        use_sparse = True, 
        max_radius=50)
```

--------------------------------

### import_adaptive_file Function

Source: https://tcrdist3.readthedocs.io/en/latest/adaptive.html

Details the `import_adaptive_file` function for preparing tcrdist3 input from Adaptive files.

```APIDOC
## import_adaptive_file

### Description
This function prepares tcrdist3 input from 2020 Adaptive files containing specific fields like 'bio_identity', 'productive_frequency', 'templates', and 'rearrangement'. It handles the conversion and cleaning necessary for tcrdist3.

### Parameters
* **adaptive_filename** (str) – Path to the Adaptive filename.
* **organism** (str) – 'human' or 'mouse'. Defaults to 'human'.
* **chain** (str) – 'beta' or 'alpha'. Defaults to 'beta'.
* **return_valid_cdr3_only** (bool) – If True, returns only valid CDR3 sequences. Defaults to True.
* **count** (str) – Column to use as count ('productive_frequency' or 'templates'). Defaults to 'productive_frequency'.
* **version_year** (int) – Year of the Adaptive file version. Defaults to 2020.
* **sep** (str) – Separator in the Adaptive file. Defaults to '\t'.
* **subject** (str or None) – Name of the epitope if known. Defaults to None.
* **epitope** (str or None) – If None, the filename will be used as the subject. Defaults to None.
* **log** (bool) – If True, write a log. Defaults to True.
* **swap_imgt_dictionary** (dict or None) – If None, the default dictionary `adaptive_to_imgt` is used.
* **additional_cols** (None or List) – List of any additional columns to keep.
* **use_cols** (list) – List of columns to retain from the original input file. Defaults to ['bio_identity', 'productive_frequency', 'templates', 'rearrangement'].

### Returns
* **bulk_df** (pd.DataFrame) – A Pandas DataFrame ready for tcrdist3 import.
```

--------------------------------

### List Available Zip Files

Source: https://tcrdist3.readthedocs.io/en/latest/downloads.html

Lists all available zip files that can be downloaded for use with tcrdist3. This is useful for knowing what data is accessible.

```APIDOC
## list_available_zip_files

### Description
Lists all available zip files downloadable from tcrdist3.

### Returns
- **Return type**: List of zipfile names that can be passed to zipfile argument in download_and_extract_zip_file()
```

--------------------------------

### Initialize TCRrep and Compute Pairwise Distances

Source: https://tcrdist3.readthedocs.io/en/latest/index.html

Initializes the TCRrep object with a DataFrame, organism, chains, and database file to compute pairwise distances between TCRs. By default, it uses the Dash et al. distance metric.

```python
import pandas as pd
from tcrdist.repertoire import TCRrep

df = pd.read_csv("dash.csv")
tr = TCRrep(cell_df = df, 
            organism = 'mouse', 
            chains = ['beta'], 
            db_file = 'alphabeta_gammadelta_db.tsv')

```

--------------------------------

### Multi-CDR Distances without OOP

Source: https://tcrdist3.readthedocs.io/en/latest/tcrdistances.html

Compute multi-CDR tcrdistances on a single chain using custom metrics and weights without relying on OOP. Requires pandas and tcrdist libraries.

```python
import multiprocessing
import pandas as pd
from tcrdist.rep_funcs import _pws, _pw

def my_own_metric(s1,s2):
    return Levenshtein.distance(s1,s2)

df = pd.read_csv("dash2.csv")

metrics_b = {
    "cdr3_b_aa" : my_own_metric ,
    "pmhc_b_aa" : my_own_metric ,
    "cdr2_b_aa" : my_own_metric ,
    "cdr1_b_aa" : my_own_metric }

weights_b = { 
    "cdr3_b_aa" : 1,
    "pmhc_b_aa" : 1,
    "cdr2_b_aa" : 1,
    "cdr1_b_aa" : 1}

kargs_b = {  
    'cdr3_b_aa' : 
        {'use_numba': False},
    'pmhc_b_aa' : {
        'use_numba': False},
    'cdr2_b_aa' : {
        'use_numba': False},
    'cdr1_b_aa' : {
        'use_numba': False}
    }

dmats =  _pws(df = df , 
            metrics = metrics_b, 
            weights = weights_b, 
            kargs   = kargs_b , 
            cpu     = 1, 
            uniquify= True, 
            store   = True)

print(dmats.keys())
```

--------------------------------

### Produce Summary Information for Tooltips

Source: https://tcrdist3.readthedocs.io/en/latest/motif_gallery.html

Generates summary information for tooltips in interactive graphics, such as the percentage of TCRs with a specific epitope at a given node. It concatenates this summary with existing cluster data.

```python
res_summary = member_summ(  res_df = tr.hcluster_df,
                            clone_df = tr.clone_df, 
                            addl_cols=['epitope'])

tr.hcluster_df_detailed = \
    pd.concat([tr.hcluster_df, res_summary], axis = 1)
```

--------------------------------

### Save Initial Meta-Clonotype Centers

Source: https://tcrdist3.readthedocs.io/en/latest/metaclonotypes.html

Saves the initial meta-clonotype center data frame to a TSV file. This is an intermediate step before further processing and filtering.

```python
centers_df.to_csv( os.path.join(project_path, f'{antigen_enriched_file}.centers_bkgd_ctlr_{level_tag}.tsv'), sep = "\t" )

```
Tokens:
Raw