### Install FPSim2 from PyPI Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/install.md Use this command to install the latest stable version of FPSim2 from the Python Package Index. ```bash pip install fpsim2 ``` -------------------------------- ### Install FPSim2 from Source with Native Optimizations Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/install.md Install FPSim2 from its GitHub repository. Set FPSIM2_MARCH_NATIVE=1 to enable compiler optimizations for potentially faster CPU similarity searches. ```bash export FPSIM2_MARCH_NATIVE=1 pip install git+https://github.com/chembl/FPSim2.git ``` -------------------------------- ### Install FPSim2 with SBGrid Source: https://github.com/chembl/fpsim2/blob/master/README.md Install FPSim2 using the sbgrid-cli tool. This method is suitable for users within the SBGrid ecosystem. ```bash sbgrid-cli install fpsim2 ``` -------------------------------- ### Install FPSim2 with conda Source: https://github.com/chembl/fpsim2/blob/master/README.md Install FPSim2 using conda from the conda-forge channel. This is recommended for managing complex dependencies. ```bash conda install conda-forge::fpsim2 ``` -------------------------------- ### Create Database Table with PostgreSQL Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/sql_backend.md Use this function to create a new database table for storing fingerprints. Specify the molecule format, fingerprint type, and parameters. This example uses PostgreSQL. ```python from FPSim2.io import create_db_table fp_type = "Morgan" fp_params = {"radius": 2, "fpSize": 256} db_url = "postgresql://user:password@hostname:5432/fpsim2" table_name = "fpsim2_fp_table" mol_format = "smiles" smiles_list = [ ["Cc1cc(-n2ncc(=O)[nH]c2=O)ccc1C(=O)c1ccccc1Cl", 1], ["Cc1cc(-n2ncc(=O)[nH]c2=O)ccc1C(=O)c1ccc(C#N)cc1", 2], ["Cc1cc(-n2ncc(=O)[nH]c2=O)cc(C)c1C(O)c1ccc(Cl)cc1", 3], ["Cc1ccc(C(=O)c2ccc(-n3ncc(=O)[nH]c3=O)cc2)cc1", 4], ["Cc1cc(-n2ncc(=O)[nH]c2=O)ccc1C(=O)c1ccc(Cl)cc1", 5], ] create_db_table(smiles_list, db_url, table_name, mol_format, fp_type, fp_params) ``` -------------------------------- ### Run GPU Tanimoto Similarity Search Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/gpu.md Instantiate FPSim2CudaEngine with a fingerprint file and use the similarity function to find similar molecules based on a query and threshold. Requires CuPy installation. ```python from FPSim2 import FPSim2CudaEngine fp_filename = 'chembl_35_v0.6.0.h5' fpce = FPSim2CudaEngine(fp_filename) query = 'CC(=O)Oc1ccccc1C(=O)O' results = fpce.similarity(query, threshold=0.7) ``` -------------------------------- ### GPU Tanimoto search with FPSim2CudaEngine Source: https://context7.com/chembl/fpsim2/llms.txt Performs Tanimoto similarity searches on an NVIDIA GPU using FPSim2CudaEngine. Requires CuPy to be installed. ```python from FPSim2 import FPSim2CudaEngine # Requires CuPy installed: pip install cupy-cuda12x fpce = FPSim2CudaEngine('fp_db.h5') query = 'CC(=O)Oc1ccccc1C(=O)O' results = fpce.similarity(query, threshold=0.7) # results dtype: [('mol_id', ' COALESCE((SELECT MAX(mol_id) FROM {table_name}), 0) ORDER BY mol_id """)) create_db_table(cursor, db_url, table_name, 'smiles', 'Morgan', {'radius': 2, 'fpSize': 2048}) # Load into FPSim2Engine and search from FPSim2 import FPSim2Engine fpe = FPSim2Engine(conn_url=db_url, table_name=table_name, storage_backend='sqla') results = fpe.similarity('CC(=O)Oc1ccccc1C(=O)O', threshold=0.7) # Export SQL table to h5 file for faster future loading fpe.save_h5('exported_fps.h5') ``` -------------------------------- ### Create DB from SDF file using Python Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/create_db_file.md Use the `create_db_file` function to generate a fingerprint database from an SDF file. Specify the SDF file path, output filename, fingerprint type, and parameters. The `mol_id_prop` argument is used to specify the property containing the molecule ID in the SDF file. ```python from FPSim2.io import create_db_file create_db_file( mols_source='sdf_file.sdf', filename='fp_db.h5', mol_format=None, # set to None fp_type='Morgan', fp_params={'radius': 2, 'fpSize': 256}, mol_id_prop='mol_id' ) ``` -------------------------------- ### Run On-Disk Substructure Screenout Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/subs_screenout.md Utilize the `FPSim2Engine.on_disk_substructure` function for substructure screenouts when the dataset exceeds available RAM. This method is slower but necessary for large datasets. ```python from FPSim2 import FPSim2Engine fp_filename = 'chembl_35_v0.6.0.h5' fpe = FPSim2Engine(fp_filename, in_memory_fps=False) query = 'CC(=O)Oc1ccccc1C(=O)O' results = fpe.on_disk_substructure(query, n_workers=1) ``` -------------------------------- ### Create Database with Partial Sanitization Source: https://context7.com/chembl/fpsim2/llms.txt Creates a fingerprint database file using partial sanitization during DB creation and search. This is useful when you want to bypass RDKit's full sanitization pipeline for performance or specific use cases. ```python from FPSim2.io import create_db_file from FPSim2 import FPSim2Engine from rdkit import Chem # Partial sanitization during DB creation and search create_db_file( mols_source='compounds.smi', filename='fp_db.h5', mol_format=None, fp_type='Morgan', fp_params={'radius': 2, 'fpSize': 2048}, full_sanitization=False, ) fpe = FPSim2Engine('fp_db.h5') results = fpe.similarity('CC(=O)Oc1ccccc1C(=O)O', threshold=0.7, full_sanitization=False) ``` -------------------------------- ### Create DB from SMI file using Python Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/create_db_file.md Use the `create_db_file` function to generate a fingerprint database from a SMILES file. Specify the SMILES file path, output filename, fingerprint type, and parameters. The `mol_format` should be set to `None` when reading from a SMILES file. ```python from FPSim2.io import create_db_file create_db_file( mols_source='smiles_file.smi', filename='fp_db.h5', mol_format=None, # set to None fp_type='Morgan', fp_params={'radius': 2, 'fpSize': 256} ) ``` -------------------------------- ### Create DB with Partial Sanitization (API) Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/sanitization.md Set `full_sanitization=False` in the `create_db_file` function to use FPSim2's partial sanitization. ```python from FPSim2.io import create_db_file mols = [['CC', 1], ['CCC', 2], ['CCCC', 3]] create_db_file( mols_source=mols, filename='fp_db.h5', mol_format='smiles', # required fp_type='Morgan', fp_params={'radius': 2, 'fpSize': 256}, full_sanitization=False ) ``` -------------------------------- ### Run Similarity Search with SQLAlchemy Backend Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/sql_backend.md Perform similarity searches using the FPSim2Engine initialized with the SQLAlchemy backend. The search interface is identical to the PyTables backend. ```python query = 'CC(=O)Oc1ccccc1C(=O)O' results = fpe.similarity(query, threshold=0.7, metric='tanimoto', n_workers=1) ``` -------------------------------- ### On-disk Similarity Search Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/similarity.md Use FPSim2Engine.on_disk_similarity for similarity searches directly on disk. This is slower but suitable for datasets exceeding available RAM. ```python from FPSim2 import FPSim2Engine fp_filename = 'chembl_35_v0.6.0.h5' fpe = FPSim2Engine(fp_filename, in_memory_fps=False) query = 'CC(=O)Oc1ccccc1C(=O)O' results = fpe.on_disk_similarity(query, threshold=0.7, metric='tanimoto') ``` -------------------------------- ### Run In-Memory Substructure Screenout Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/subs_screenout.md Use the `FPSim2Engine.substructure` function for in-memory substructure screenouts. This is the recommended approach when the dataset fits within available RAM. ```python from FPSim2 import FPSim2Engine fp_filename = 'chembl_35_v0.6.0.h5' fpe = FPSim2Engine(fp_filename) query = 'CC(=O)Oc1ccccc1C(=O)O' results = fpe.substructure(query, n_workers=1) ``` -------------------------------- ### On-disk Similarity Search Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/similarity.md Use the `FPSim2Engine.on_disk_similarity` function for similarity searches directly on disk. This method is slower but is suitable for datasets that exceed available RAM. It is recommended to use this only when the dataset does not fit in memory. ```APIDOC ## FPSim2Engine.on_disk_similarity ### Description Performs a symmetric similarity search on disk. This is suitable for very large datasets that do not fit into memory. ### Method ```python FPSim2Engine.on_disk_similarity(query: str, threshold: float = 0.7, metric: str = 'tanimoto') ``` ### Parameters #### Path Parameters None #### Query Parameters - **query** (str) - Required - The query molecule in SMILES format. - **threshold** (float) - Optional - The minimum similarity score to return. Defaults to 0.7. - **metric** (str) - Optional - The similarity metric to use. Options: 'tanimoto', 'dice', 'cosine'. Defaults to 'tanimoto'. ### Request Example ```python from FPSim2 import FPSim2Engine fp_filename = 'chembl_35_v0.6.0.h5' fpe = FPSim2Engine(fp_filename, in_memory_fps=False) query = 'CC(=O)Oc1ccccc1C(=O)O' results = fpe.on_disk_similarity(query, threshold=0.7, metric='tanimoto') ``` ### Response #### Success Response (200) - **results** (list) - A list of dictionaries, where each dictionary contains the 'id' and 'score' of similar compounds. ``` -------------------------------- ### Custom Sanitization and Database Creation Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/sanitization.md Provide pre-sanitized RDKit molecule objects to `create_db_file` by setting `mol_format='rdkit'`. This bypasses FPSim2's sanitization steps. ```python from FPSim2.io import create_db_file from rdkit import Chem mols = [['CC', 1], ['CCC', 2], ['CCCC', 3]] def parse_molecule(smiles): mol = Chem.MolFromSmiles(smiles, sanitize=False) # Apply custom sanitization steps Chem.SanitizeMol(mol, sanitizeOps=Chem.SANITIZE_ALL^Chem.SANITIZE_CLEANUP^Chem.SANITIZE_CLEANUPCHIRALITY) return mol # Create list of [mol, id] pairs mols = [[parse_molecule(smi), mol_id] for smi, mol_id in mols] create_db_file( mols_source=mols, filename='fp_db.h5', mol_format='rdkit', # Important: specify rdkit format fp_type='Morgan', fp_params={'radius': 2, 'fpSize': 256} ) ``` -------------------------------- ### On-disk Top K Similarity Search Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/similarity.md Perform top-K similarity searches on disk using FPSim2Engine.on_disk_top_k. This method is recommended only when the dataset does not fit into memory. ```python from FPSim2 import FPSim2Engine fp_filename = 'chembl_35_v0.6.0.h5' fpe = FPSim2Engine(fp_filename, in_memory_fps=False) query = 'CC(=O)Oc1ccccc1C(=O)O' results = fpe.on_disk_top_k(query, k=100, threshold=0.7, metric='tanimoto') ``` -------------------------------- ### On-disk Top K Search Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/similarity.md Use the `FPSim2Engine.on_disk_top_k` function for top-K similarity searches directly on disk. This is suitable for datasets that do not fit into memory and when you need a specific number of the most similar results. ```APIDOC ## FPSim2Engine.on_disk_top_k ### Description Retrieves the top K most similar compounds to a query molecule from a disk-based fingerprint database. This is suitable for datasets that do not fit into memory. ### Method ```python FPSim2Engine.on_disk_top_k(query: str, k: int, threshold: float = 0.7, metric: str = 'tanimoto') ``` ### Parameters #### Path Parameters None #### Query Parameters - **query** (str) - Required - The query molecule in SMILES format. - **k** (int) - Required - The number of top similar compounds to retrieve. - **threshold** (float) - Optional - The minimum similarity score to consider. Defaults to 0.7. - **metric** (str) - Optional - The similarity metric to use. Options: 'tanimoto', 'dice', 'cosine'. Defaults to 'tanimoto'. ### Request Example ```python from FPSim2 import FPSim2Engine fp_filename = 'chembl_35_v0.6.0.h5' fpe = FPSim2Engine(fp_filename, in_memory_fps=False) query = 'CC(=O)Oc1ccccc1C(=O)O' results = fpe.on_disk_top_k(query, k=100, threshold=0.7, metric='tanimoto') ``` ### Response #### Success Response (200) - **results** (list) - A list of dictionaries, where each dictionary contains the 'id' and 'score' of the top K similar compounds. ``` -------------------------------- ### FPSim2Engine.similarity / on_disk_similarity Source: https://context7.com/chembl/fpsim2/llms.txt Performs a threshold similarity search against the database. Supports Tanimoto, Dice, and Cosine metrics. Can be run in a multi-threaded fashion. ```APIDOC ## `FPSim2Engine.similarity` / `on_disk_similarity` — Threshold similarity search Searches the database for all molecules whose similarity to the query meets or exceeds a threshold. Supports `tanimoto` (default), `dice`, and `cosine` metrics. The `n_workers` parameter enables multi-threaded search by splitting the database range across threads. ### Method `similarity(query, threshold, metric='tanimoto', n_workers=1)` `on_disk_similarity(query, threshold, n_workers=4, chunk_size=100000)` ### Parameters - `query` (string or RDKit Mol or ExplicitBitVect): The query molecule or its fingerprint. - `threshold` (float): The minimum similarity score to return. - `metric` (string, optional): The similarity metric to use ('tanimoto', 'dice', 'cosine'). Defaults to 'tanimoto'. - `n_workers` (int, optional): Number of worker threads for multi-threaded search. - `chunk_size` (int, optional): Size of chunks for on-disk processing. ### Response - `results` (numpy.ndarray): An array of tuples, where each tuple contains `('mol_id', ' COALESCE((SELECT MAX(mol_id) FROM {table_name}), 0) ORDER BY mol_id """)) create_db_table(cursor, db_url, table_name, 'smiles', 'Morgan', {'radius': 2, 'fpSize': 2048}) # Load into FPSim2Engine and search from FPSim2 import FPSim2Engine fpe = FPSim2Engine(conn_url=db_url, table_name=table_name, storage_backend='sqla') results = fpe.similarity('CC(=O)Oc1ccccc1C(=O)O', threshold=0.7) # Export SQL table to h5 file for faster future loading fpe.save_h5('exported_fps.h5') ``` ``` -------------------------------- ### Substructure Screenout with FPSim2Engine Source: https://context7.com/chembl/fpsim2/llms.txt Performs a fast fingerprint-based substructure pre-filter using an optimized Tversky search. Use RDKit's `HasSubstructMatch` for confirmation. Recommended fingerprint type is `RDKitPattern`. Supports multi-threading and on-disk operations. ```python from FPSim2 import FPSim2Engine from rdkit import Chem # Build database with RDKitPattern fingerprints (best for substructure) from FPSim2.io import create_db_file create_db_file('compounds.smi', 'pattern_db.h5', None, 'RDKitPattern', {'fpSize': 2048}) fpe = FPSim2Engine('pattern_db.h5') query = 'c1ccccc1' # benzene ring — find all compounds containing a benzene # Screenout returns mol_ids (uint32 array), not similarity scores candidate_ids = fpe.substructure(query, n_workers=4) print(f"{len(candidate_ids)} candidates pass the screenout") # Confirm with full substructure search query_mol = Chem.MolFromSmarts(query) confirmed = [mol_id for mol_id in candidate_ids if your_mol_lookup(mol_id).HasSubstructMatch(query_mol)] # On-disk variant fpe_disk = FPSim2Engine('pattern_db.h5', in_memory_fps=False) candidate_ids = fpe_disk.on_disk_substructure(query, n_workers=4) ``` -------------------------------- ### PyTablesStorageBackend.append_fps / delete_fps Source: https://context7.com/chembl/fpsim2/llms.txt Allows incremental modification of an existing HDF5 fingerprint database by appending new molecules or deleting existing ones without requiring a full rebuild. After appending, `sort_db_file` should be called to restore optimal search performance. ```APIDOC ## PyTablesStorageBackend.append_fps / delete_fps ### Description Modify an existing database. Incrementally add molecules to or remove molecules from an existing HDF5 fingerprint database without rebuilding from scratch. After appending, re-sort the file with `sort_db_file` to restore sublinear search performance. ### Usage ```python from FPSim2.io.backends.pytables import PyTablesStorageBackend, sort_db_file backend = PyTablesStorageBackend('fp_db.h5', in_memory_fps=False) # Append new molecules from an iterable of (smiles, mol_id) pairs new_mols = [['CC(C)Cc1ccc(cc1)C(C)C(=O)O', 1001], ['c1ccc2ccccc2c1', 1002]] backend.append_fps(new_mols, mol_format='smiles') # Re-sort after appending to restore population-count order sort_db_file('fp_db.h5') # Delete molecules by their integer IDs backend.delete_fps([1001, 1002]) ``` ``` -------------------------------- ### FPSim2Engine Source: https://context7.com/chembl/fpsim2/llms.txt The main CPU-based similarity search engine. It loads a fingerprint database and provides methods for various similarity and substructure searches. Supports both in-memory and on-disk operation for datasets of different sizes, and can also load data from SQL backends. ```APIDOC ## `FPSim2Engine` — Main CPU search engine Loads a fingerprint database and exposes all search methods. By default fingerprints are loaded into memory (`in_memory_fps=True`) for maximum speed. Pass `in_memory_fps=False` to enable on-disk searches for datasets that exceed available RAM. ### Initialization Examples: #### In-memory engine (default) ```python from FPSim2 import FPSim2Engine fpe = FPSim2Engine('fp_db.h5') print(fpe) # FPSim2Engine(fp_type='Morgan', fp_params={...}, rdkit_ver='2024.x', fpsim2_ver='0.7.4') ``` #### On-disk engine (large datasets that don't fit in RAM) ```python from FPSim2 import FPSim2Engine fpe_disk = FPSim2Engine('fp_db.h5', in_memory_fps=False) ``` #### Load from SQL backend (PostgreSQL example) ```python from FPSim2 import FPSim2Engine fpe_sql = FPSim2Engine( conn_url='postgresql://user:password@localhost:5432/mydb', table_name='fpsim2_fp_table', storage_backend='sqla', ) ``` ### Inspecting Fingerprints: ```python # Inspect loaded fingerprints print(fpe.fps.shape) # (n_compounds, fp_fields) print(fpe.fp_type) # 'Morgan' print(fpe.fp_params) # {'radius': 2, 'fpSize': 2048, ...} ``` ``` -------------------------------- ### In-memory Top K Similarity Search Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/similarity.md Retrieve the top K most similar compounds in memory using FPSim2Engine.top_k. Parallel processing via 'n_workers' can improve performance. ```python from FPSim2 import FPSim2Engine fp_filename = 'chembl_35_v0.6.0.h5' fpe = FPSim2Engine(fp_filename) query = 'CC(=O)Oc1ccccc1C(=O)O' results = fpe.top_k(query, k=100, threshold=0.7, metric='tanimoto', n_workers=1) ``` -------------------------------- ### Incremental Database Load using SQLAlchemy CursorResult Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/sql_backend.md Append molecules to an existing database table by providing a SQLAlchemy CursorResult. This is useful for populating tables from other SQL instances. ```python from sqlalchemy import create_engine, text from FPSim2.io import create_db_table fp_type = "Morgan" fp_params = {"radius": 2, "fpSize": 256} db_url = "postgresql://user:password@hostname:5432/fpsim2" table_name = "fpsim2_fp_table" mol_format = "smiles" sql_query = text(f""" SELECT smiles, mol_id FROM structure WHERE mol_id > COALESCE(( SELECT MAX(mol_id) FROM {table_name} ), 0) ORDER BY mol_id """) engine = create_engine(db_url) with engine.connect() as conn: cursor = conn.execute(sql_query) create_db_table(cursor, db_url, table_name, mol_format, fp_type, fp_params) ``` -------------------------------- ### In memory substructure screenout Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/subs_screenout.md Use the `FPSim2Engine.substructure` function to run an in-memory substructure screenout. This method is efficient for datasets that fit within available RAM. ```APIDOC ## substructure ### Description Runs an optimized Tversky substructure screenout. This is not a full subgraph isomorphism search. ### Method `FPSim2Engine.substructure(query: str, n_workers: int = 1)` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python from FPSim2 import FPSim2Engine fp_filename = 'chembl_35_v0.6.0.h5' fpe = FPSim2Engine(fp_filename) query = 'CC(=O)Oc1ccccc1C(=O)O' results = fpe.substructure(query, n_workers=1) ``` ### Response #### Success Response (200) - **results** (list) - A list of identifiers for molecules that match the substructure query. #### Response Example ```json { "example": "[1, 5, 10, ... ]" } ``` ``` -------------------------------- ### Top-K Similarity Search with FPSim2Engine Source: https://context7.com/chembl/fpsim2/llms.txt Retrieves the K most similar molecules above a minimum threshold. More efficient than filtering full similarity results when a fixed number of hits is needed. Supports multi-threading and on-disk operations. ```python from FPSim2 import FPSim2Engine fpe = FPSim2Engine('fp_db.h5') query = 'CC(=O)Oc1ccccc1C(=O)O' # Return top 50 most similar compounds with Tanimoto >= 0.5 results = fpe.top_k(query, k=50, threshold=0.5, metric='tanimoto', n_workers=1) print(f"Returned {len(results)} hits") # up to 50 print(results['mol_id']) # array of molecule IDs print(results['coeff']) # array of similarity coefficients # On-disk variant fpe_disk = FPSim2Engine('fp_db.h5', in_memory_fps=False) results = fpe_disk.on_disk_top_k(query, k=100, threshold=0.4, n_workers=4) ``` -------------------------------- ### Integrated Similarity (iSIM) metrics with contrib.isim Source: https://context7.com/chembl/fpsim2/llms.txt Computes dataset-level similarity statistics using community module functions. Equivalent to average pairwise comparisons without enumerating all pairs. ```python from FPSim2.contrib.isim.isim_comp import ( get_sim_dict, calculate_medoid, calculate_outlier, calculate_comp_sim, ) from FPSim2 import FPSim2Engine fpe = FPSim2Engine('fp_db.h5') # Average pairwise similarity across the entire dataset sim_dict = get_sim_dict(fpe) print(sim_dict) # {'mean': 0.32, 'std': 0.14, ...} ``` -------------------------------- ### Save SQL Fingerprints to PyTables h5 File Source: https://github.com/chembl/fpsim2/blob/master/docs/user_guide/sql_backend.md Export fingerprints currently stored in the SQL database to a PyTables h5 file. This is useful for data sharing or distribution. ```python fpe.save_h5("my_fps.h5") ```