### Build and Install Oxbow Rust Library from Source Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/installation.md These commands outline the process of building and installing the Oxbow Rust library from its source code. It requires a working Rust toolchain. You clone the repository, navigate to the Rust library directory, and then build or install it. ```bash git clone https://github.com/abdenlab/oxbow.git cd oxbow cargo build --release ``` ```bash cd oxbow cargo install --path . ``` -------------------------------- ### Load BAM using File-like Objects Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md This example shows how to load BAM data using Python file-like objects, which are generated by callable functions. This approach offers flexibility for custom data transport, such as reading from various file systems or encodings. ```python ds = ox.from_bam( lambda : open("sample.bam", "rb"), index=lambda : open("sample.bam.bai", "rb"), ) ``` -------------------------------- ### Install Oxbow R Package from GitHub Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/installation.md This R code demonstrates how to install the Oxbow R bindings directly from GitHub. It requires the devtools or remotes package and a working Rust toolchain. You can install either by loading the package and calling install_github with the subdirectory, or by cloning locally and using install_local. ```R library(devtools) devtools::install_github("abdenlab/oxbow", subdir="r-oxbow") ``` ```bash git clone https://github.com/abdenlab/oxbow.git cd oxbow R ``` ```R library(devtools) devtools::install_local("r-oxbow") ``` -------------------------------- ### Load BAM with fsspec and S3FileSystem Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Demonstrates loading BAM data from remote sources (HTTP and S3) using `fsspec` and `s3fs` libraries. It utilizes caching file systems for HTTP and anonymous access for S3, showcasing advanced remote file handling capabilities. ```python from fsspec.implementations.cached import CachingFileSystem from s3fs import S3FileSystem url = "https://oxbow-ngs.s3.us-east-2.amazonaws.com/example.bam" httpfs = CachingFileSystem(target_protocol="https") ds = ox.from_bam( lambda : httpfs.open(url, "rb"), index=lambda : httpfs.open(url + ".bai", "rb"), ) s3fs = S3FileSystem(anon=True) s3_uri = "s3://oxbow-ngs/example.bam" ds = ox.from_bam( lambda : s3fs.open(s3_uri, "rb"), index=lambda : s3fs.open(s3_uri + ".bai", "rb"), tag_defs=[], ) ds.regions("chr1:82744-85000").pl() ``` -------------------------------- ### Install Oxbow Python Package from Source using pip Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/installation.md This method installs the Oxbow Python package by building it from its source code. It requires a Rust toolchain and maturin. The installation can be done by cloning the repository and using pip, or directly from GitHub specifying the subdirectory. ```bash git clone https://github.com/abdenlab/oxbow.git cd py-oxbow pip install . ``` ```bash cd py-oxbow uv sync ``` ```bash pip install 'git+https://github.com/abdenlab/oxbow.git@main#egg=oxbow&subdirectory=py-oxbow' ``` -------------------------------- ### Install Oxbow Python Package using pip Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/installation.md This command installs the Oxbow Python package directly from the Python Package Index (PyPI). It's the simplest way to get started with Oxbow in a Python environment. Ensure you have pip installed and updated. ```bash pip install oxbow ``` -------------------------------- ### Access Data Source Fragments Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md This code illustrates how Oxbow data sources are logically grouped into fragments. For data sources without random access, a single fragment is typically present. The `ds.fragments()` method provides access to these fragments. ```ipython3 ds = ox.from_bam("data/sample.bam") ds.fragments() ``` ```ipython3 ds = ox.from_bam("data/sample.bam").regions(["chr1", "chr3", "chrX"]) ds.fragments() ``` -------------------------------- ### Create BAM DataSource using Oxbow Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Initializes a DataSource object from a BAM file using the `ox.from_bam` convenience function. This function requires the path to the BAM file and optionally accepts parameters for compression, fields, tags, regions, index, and batch size. The returned DataSource object can then be used to access the data within the BAM file. ```ipython3 import oxbow as ox ds = ox.from_bam("data/sample.bam") ``` -------------------------------- ### Project VCF/BCF INFO Fields (None) Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Loads data from a VCF/BCF file using oxbow and explicitly excludes all INFO fields by setting `info_fields=[]`. It also excludes sample data. This is useful when only basic variant information is needed and INFO fields are not required. ```ipython3 ( ox.from_vcf( "data/sample.vcf.gz", info_fields=[] ) .pl() ).head() ``` -------------------------------- ### Project Specific VCF/BCF INFO Fields Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Loads data from a VCF/BCF file using oxbow and projects only a specified subset of INFO fields ('TYPE', 'snpeff.Effect', etc.). This allows for targeted loading of relevant annotation data, improving efficiency. Sample data is excluded. ```ipython3 df = ( ox.from_vcf( "data/sample.vcf.gz", info_fields=["TYPE", "snpeff.Effect", "snpeff.Gene_Name", "snpeff.Transcript_BioType"], samples=[] ) .pl() ) df.head() ``` ```ipython3 df.unnest("info").head() ``` -------------------------------- ### Load BAM from Remote URL with Index Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Loads data from a BAM file located at a remote URL, explicitly providing the URL for its corresponding index file (.bai). This is useful for accessing data stored in cloud storage or accessible via HTTP. ```python ds = ox.from_bam( "https://oxbow-ngs.s3.us-east-2.amazonaws.com/example.bam", index="https://oxbow-ngs.s3.us-east-2.amazonaws.com/example.bam.bai" ) ``` -------------------------------- ### Install Rust using rustup Source: https://github.com/abdenlab/oxbow/blob/main/docs/contributing-guide/development.md Installs the Rust programming language and its package manager, cargo, using the official rustup script. This is a prerequisite for building Rust components of the Oxbow project. ```bash curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ``` -------------------------------- ### Install py-oxbow from GitHub Source: https://github.com/abdenlab/oxbow/blob/main/py-oxbow/README.md Installs the latest development version of py-oxbow directly from GitHub. This requires Rust and maturin to be installed locally for building the package. Use this for the bleeding edge. ```shell pip install 'git+https://github.com/abdenlab/oxbow.git@main#egg=oxbow&subdirectory=py-oxbow' ``` -------------------------------- ### Download BAM and BAI Files Source: https://github.com/abdenlab/oxbow/blob/main/py-oxbow/notebooks/bench.ipynb This snippet downloads example BAM and BAI files using wget. It also creates an empty BAI file if it doesn't exist to prevent errors with pysam, though the direct download usually provides it. ```shell !wget https://aveit.s3.amazonaws.com/higlass/bam/example_higlass.bam -O example.bam !wget https://aveit.s3.amazonaws.com/higlass/bam/example_higlass.bam.bai -O example.bam.bai !touch example.bam.bai # ensures no errors from pysam ``` -------------------------------- ### Load BED File with bed3+ Schema Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Loads data from a BED file using oxbow, interpreting it according to the 'bed3+' schema. This schema implies at least three standard BED columns (chrom, start, end) plus additional columns. The result is a Polars DataFrame. ```ipython3 ox.from_bed("data/sample.bed", bed_schema="bed3+").pl().head() ``` -------------------------------- ### Serve Documentation with Sphinx Autobuild Source: https://github.com/abdenlab/oxbow/blob/main/py-oxbow/README.md Starts a live-reloading server for the project documentation using 'sphinx-autobuild'. This command also requires the 'docs' dependency group and is useful for development when viewing documentation changes in real-time. ```shell uv run --group=docs sphinx-autobuild docs docs/_build/html ``` -------------------------------- ### PyFastqScanner Initialization Source: https://github.com/abdenlab/oxbow/blob/main/docs/api/python/oxbow.core.PyFastqScanner.md Initializes the PyFastqScanner for reading FASTQ files. It can handle both plain and GZIP-compressed files. ```APIDOC ## PyFastqScanner Initialization ### Description Initializes the PyFastqScanner for reading FASTQ files. It can handle both plain and GZIP-compressed files. ### Method __init__ ### Parameters #### Path Parameters - **src** (str or file-like) - Required - The path to the FASTQ file or a file-like object. - **compressed** (bool) - Optional (default: False) - Whether the source is GZIP-compressed. ### Request Example ```python # Example of initializing PyFastqScanner scanner = PyFastqScanner('path/to/reads.fastq.gz', compressed=True) ``` ### Response #### Success Response (Initialization) - **None** - The constructor does not return a value, it initializes the object. #### Response Example (No explicit response, object is created in place) ``` -------------------------------- ### Add Oxbow Rust Crate to Cargo.toml Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/installation.md This command adds the Oxbow Rust library as a dependency to your project's Cargo.toml file. It allows you to easily include Oxbow's functionality in your Rust applications. Ensure you have Rust and Cargo installed. ```sh cargo add oxbow ``` -------------------------------- ### PyVcfScanner Initialization Source: https://github.com/abdenlab/oxbow/blob/main/docs/api/python/oxbow.core.PyVcfScanner.md Initializes the PyVcfScanner with a source VCF file and an option for compression. ```APIDOC ## PyVcfScanner Constructor ### Description Initializes a VCF file scanner. ### Method `__init__` ### Parameters #### Path Parameters - **src** (str or file-like) - Required - The path to the VCF file or a file-like object. - **compressed** (bool, optional) - Optional (default: False) - Whether the source is BGZF-compressed. ### Request Example ```python from oxbow.core import PyVcfScanner vcf_scanner = PyVcfScanner('path/to/your/file.vcf.gz', compressed=True) ``` ### Response #### Success Response (200) Initializes the PyVcfScanner object. #### Response Example ```json { "status": "success", "message": "PyVcfScanner initialized successfully" } ``` ``` -------------------------------- ### Collect and Display Head of Lazy Polars DataFrame Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Executes the computation plan for a lazy Polars DataFrame and displays the first few rows. The `.collect()` method triggers the evaluation of the lazy DataFrame. ```ipython3 df.head().collect() ``` -------------------------------- ### PyFastaScanner Initialization Source: https://github.com/abdenlab/oxbow/blob/main/docs/api/python/oxbow.core.PyFastaScanner.md Initializes the PyFastaScanner with a source FASTA file or file-like object. ```APIDOC ## PyFastaScanner ### Description A FASTA file scanner. ### Method __init__ ### Endpoint N/A (Class Constructor) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python from oxbow.core import PyFastaScanner # Example with a file path scanner = PyFastaScanner("path/to/your/file.fa") # Example with a file-like object # with open("path/to/your/file.fa", "r") as f: # scanner = PyFastaScanner(f) # Example with a compressed file # scanner_compressed = PyFastaScanner("path/to/your/file.fa.bgz", compressed=True) ``` ### Response #### Success Response (200) N/A (Constructor does not return a value, it initializes the object) #### Response Example N/A ``` -------------------------------- ### SamFile Initialization Source: https://github.com/abdenlab/oxbow/blob/main/docs/api/python/oxbow.core.SamFile.md Initializes a SamFile object to read data from a source. Supports various options for compression, fields, tags, regions, and indexing. ```APIDOC ## oxbow.core.SamFile ### Description Initializes a SamFile object to read data from a source. Supports various options for compression, fields, tags, regions, and indexing. ### Method `__init__` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **source** (str | Callable[[], IO[bytes] | str]) - Required - The data source path or a callable returning bytes or string. - **compressed** (bool) - Optional - Whether the source is compressed. Defaults to False. - **fields** (list[str] | None) - Optional - Specific fields to include. - **tag_defs** (list[tuple[str, str]] | None) - Optional - Definitions for tag records. - **tag_scan_rows** (int) - Optional - Number of rows to scan for tags. Defaults to 1024. - **regions** (str | list[str] | None) - Optional - Genomic regions to query. - **index** (str | Callable[[], IO[bytes] | str] | None) - Optional - Path to the index file or a callable returning it. - **batch_size** (int) - Optional - Size of record batches. Defaults to 131072. ### Request Example ```python # Example initialization (syntax will vary based on actual library usage) sam_file = oxbow.core.SamFile( source="path/to/your/file.sam", compressed=False, fields=["QNAME", "FLAG"], regions=["chr1:1-1000"] ) ``` ### Response This is a constructor, no direct response body is applicable. ``` -------------------------------- ### Convert Oxbow to Dask DataFrame Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Converts an Oxbow data source, potentially filtered by regions, into a Dask DataFrame. Oxbow maps its fragments to Dask partitions, enabling parallel processing capabilities of Dask. ```python df = ( ox.from_bam("data/sample.bam") .regions(["chr1", "chrX", "chrY"]) .dd() # or to_dask() ) df ``` ```python df.partitions[1].compute() ``` -------------------------------- ### Iterate Through BAM Record Batches Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Oxbow data sources stream data as Arrow RecordBatches. This snippet shows how to retrieve the next batch from an iterator returned by `ds.batches()`, with a specified `batch_size` for efficient processing. ```ipython3 ds = ox.from_bam("data/sample.bam", batch_size=100) batch = next(ds.batches()) batch ``` ```ipython3 pl.from_arrow(batch) ``` -------------------------------- ### Materialize BAM DataSource to Polars DataFrame Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Converts an Oxbow DataSource object, created from a BAM file, into a Polars DataFrame. This method is efficient for datasets that fit in memory. Use the `.pl()` method (or `.to_polars()`) for this conversion. ```ipython3 ds.pl() # or ds.to_polars() ``` -------------------------------- ### Create FASTQ Data Source with oxbow.from_fastq Source: https://github.com/abdenlab/oxbow/blob/main/docs/api/python/oxbow.from_fastq.md This snippet demonstrates how to create a FASTQ data source using the oxbow.from_fastq function. It takes the source file path, compression type, optional fields, and batch size as parameters. Note that indexed FASTQ files are not supported. ```python from pathlib import Path from oxbow import from_fastq # Example usage with a file path fastq_source_path = "/path/to/your/reads.fastq.gz" fastq_file = from_fastq(fastq_source_path) # Example usage with a callable def open_fastq_file(): return open("/path/to/your/reads.fastq", "rb") fastq_file_callable = from_fastq(open_fastq_file) # Example with specific fields and batch size fastq_file_filtered = from_fastq(fastq_source_path, fields=["sequence", "quality"], batch_size=65536) ``` -------------------------------- ### Materialize BAM DataSource to Pandas DataFrame Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Converts an Oxbow DataSource object, created from a BAM file, into a Pandas DataFrame. This is suitable for datasets that fit comfortably in memory. The `.pd()` method (or `.to_pandas()`) facilitates this conversion. ```ipython3 ds.pd() # or ds.to_pandas() ``` -------------------------------- ### FastaFile Class Initialization Source: https://github.com/abdenlab/oxbow/blob/main/docs/api/python/oxbow.core.FastaFile.md Initializes a FastaFile object, which represents a FASTA file and provides methods for accessing its data. ```APIDOC ## oxbow.core.FastaFile ### Description Initializes a FastaFile object for accessing FASTA data. ### Method `__init__` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body * **source** (str | Callable[[], IO[bytes] | str]) - Required - The source of the FASTA file (path or callable). * **compressed** (bool) - Optional - Whether the file is compressed (default: False). * **fields** (list[str] | None) - Optional - Specific fields to load. * **regions** (str | list[str] | None) - Optional - Genomic regions to query. * **index** (str | Callable[[], IO[bytes] | str] | None) - Optional - Path to or callable for the index file. * **gzi** (str | Callable[[], IO[bytes]] | None) - Optional - Path to or callable for the GZI index file. * **batch_size** (int) - Optional - The batch size for processing (default: 1). ### Request Example ```json { "source": "/path/to/your/file.fasta", "compressed": false, "fields": ["sequence"], "regions": "chr1:1-1000", "index": "/path/to/your/file.fasta.fai", "gzi": null, "batch_size": 10 } ``` ### Response #### Success Response (200) This method does not return a value, it initializes the object. #### Response Example None ``` -------------------------------- ### Parse BigBed with AutoSql Schema Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Oxbow can parse BigBed records using AutoSql definitions. This allows for structured interpretation of BigBed data based on a provided schema. The schema parameter defaults to 'autosql' if not explicitly provided. ```ipython3 ox.from_bigbed("data/autosql-sample.bb").pl().head() ``` ```ipython3 ox.from_bigbed("data/autosql-sample.bb", schema="autosql").pl().head() ``` -------------------------------- ### Create GTF Data Source with Default Settings Source: https://github.com/abdenlab/oxbow/blob/main/docs/api/python/oxbow.from_gtf.md This example demonstrates creating a GTF data source using the default settings. It assumes the GTF file is uncompressed and all default fields and attributes are to be used. No specific regions are queried. ```python from oxbow import from_gtf # Assuming 'your_file.gtf' is a valid GTF file gtffile = from_gtf('your_file.gtf') # You can then use the 'gtffile' object for further operations ``` -------------------------------- ### Load GTF/GFF Attributes Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Loads data from a GTF/GFF file using oxbow and exposes the 'attributes' column as a struct. It then unnests these attributes to make them directly accessible for further analysis in Polars. This handles complex, key-value pair data within these formats. ```ipython3 df = ( ox.from_gff("data/sample.gff") .pl() ) df.head() ``` ```ipython3 df['attributes'].struct.unnest().head() ``` -------------------------------- ### BigBedFile Methods Source: https://github.com/abdenlab/oxbow/blob/main/docs/api/python/oxbow.core.BigBedFile.md Provides documentation for various methods of the BigBedFile class, including data conversion, batch generation, and region querying. ```APIDOC ## BigBedFile Methods ### Description Provides documentation for various methods of the BigBedFile class, including data conversion, batch generation, and region querying. ### Method: batches() #### Description Generate record batches from the data source. #### Method batches #### Endpoint N/A (Instance method) #### Parameters None #### Request Example N/A #### Response ##### Success Response (Yields pa.RecordBatch) - **Yields:** pa.RecordBatch – A record batch from the data source. #### Response Example (Example of a yielded RecordBatch structure) ```json { "schema": { "fields": [ {"name": "chrom", "type": {"name": "string"}}, {"name": "start", "type": {"name": "int64"}}, {"name": "end", "type": {"name": "int64"}} ] }, "batches": [ { "columns": [ ["chr1", "chr1"], [1000, 1500], [1200, 1700] ] } ] } ``` ### Method: dataset() #### Description Convert the data source into a dataset. #### Method dataset #### Endpoint N/A (Instance method) #### Parameters None #### Request Example N/A #### Response ##### Success Response (Returns BatchReaderDataset) - **Returns:** A dataset representation of the data source. - **Return type:** [BatchReaderDataset](oxbow.arrow.BatchReaderDataset.md#oxbow.arrow.BatchReaderDataset) #### Response Example (Conceptual representation of a BatchReaderDataset object) ```json { "type": "BatchReaderDataset", "description": "Represents a dataset composed of record batches." } ``` ### Method: dd() #### Description Convert the data source to a Dask DataFrame. #### Method dd #### Endpoint N/A (Instance method) #### Parameters - **find_divisions** (bool) - Optional - If True, find divisions for the Dask DataFrame. Defaults to False. #### Request Example ```python df_dask = bigbed_file.dd(find_divisions=True) ``` #### Response ##### Success Response (Returns dask.dataframe.DataFrame) - **Returns:** A Dask DataFrame representation of the data source. - **Return type:** dask.dataframe.DataFrame #### Response Example (Conceptual representation of a Dask DataFrame object) ```json { "type": "dask.dataframe.DataFrame", "columns": ["chrom", "start", "end"], "divisions": [0, 1000, 2500] // Example divisions if find_divisions=True } ``` ### Method: pd() #### Description Convert the dataset to a Pandas DataFrame. #### Method pd #### Endpoint N/A (Instance method) #### Parameters None #### Request Example ```python df_pandas = bigbed_file.pd() ``` #### Response ##### Success Response (Returns pd.DataFrame) - **Returns:** A Pandas DataFrame representation of the data source. - **Return type:** pandas.DataFrame #### Response Example (Conceptual representation of a Pandas DataFrame object) ```json { "type": "pandas.DataFrame", "columns": ["chrom", "start", "end"], "data": [ ["chr1", 1000, 1200], ["chr1", 1500, 1700] ] } ``` ### Method: pl() #### Description Convert the data source to a Polars DataFrame or LazyFrame. #### Method pl #### Endpoint N/A (Instance method) #### Parameters - **lazy** (bool) - Optional - If True, return a LazyFrame. Defaults to False. #### Request Example ```python lf_polars = bigbed_file.pl(lazy=True) ``` #### Response ##### Success Response (Returns polars.DataFrame or polars.LazyFrame) - **Returns:** A Polars DataFrame or LazyFrame representation of the data source. - **Return type:** polars.DataFrame | polars.LazyFrame #### Response Example (Conceptual representation of a Polars DataFrame object) ```json { "type": "polars.DataFrame", "columns": ["chrom", "start", "end"], "shape": [2, 3] } ``` ### Method: regions() #### Description Query one or more genomic ranges within the data source. #### Method regions #### Endpoint N/A (Instance method) #### Parameters - **regions** (str | list[str]) - Required - The genomic region(s) to query. #### Request Example ```python region_data = bigbed_file.regions("chr1:1000-5000") ``` #### Response ##### Success Response (Returns data for specified regions) - **Returns:** Data corresponding to the queried genomic ranges, typically as a DataFrame or similar structure. #### Response Example (Conceptual representation of returned data) ```json { "query_regions": ["chr1:1000-5000"], "results": [ {"chrom": "chr1", "start": 1100, "end": 1300, "value": "some_data_1"}, {"chrom": "chr1", "start": 2500, "end": 2800, "value": "some_data_2"} ] } ``` ### Method: scanner() #### Description Create a low-level scanner for the data source. #### Method scanner #### Endpoint N/A (Instance method) #### Parameters None #### Request Example ```python scanner_obj = bigbed_file.scanner() ``` #### Response ##### Success Response (Returns a scanner object) - **Returns:** An object allowing low-level access to the data source. #### Response Example (Conceptual representation of a scanner object) ```json { "type": "ScannerObject", "description": "Provides low-level access to BigBed data." } ``` ### Method: to_dask() #### Description Convert the data source to a Dask DataFrame. #### Method to_dask #### Endpoint N/A (Instance method) #### Parameters - **find_divisions** (bool) - Optional - If True, find divisions for the Dask DataFrame. Defaults to False. #### Request Example ```python df_dask = bigbed_file.to_dask(find_divisions=True) ``` #### Response ##### Success Response (Returns dask.dataframe.DataFrame) - **Returns:** A Dask DataFrame representation of the data source. - **Return type:** dask.dataframe.DataFrame #### Response Example (Conceptual representation of a Dask DataFrame object) ```json { "type": "dask.dataframe.DataFrame", "columns": ["chrom", "start", "end"], "divisions": [0, 1000, 2500] // Example divisions if find_divisions=True } ``` ### Method: to_duckdb() #### Description Convert the data source into a DuckDB Relation. #### Method to_duckdb #### Endpoint N/A (Instance method) #### Parameters - **conn** (duckdb.DuckDBPyConnection) - Required - The DuckDB connection to use. #### Request Example ```python import duckdb conn = duckdb.connect(database=':memory:', read_only=False) duckdb_relation = bigbed_file.to_duckdb(conn) ``` #### Response ##### Success Response (Returns duckdb.DuckDBPyRelation) - **Returns:** A DuckDB Relation object representing the data source. - **Return type:** duckdb.DuckDBPyRelation #### Response Example (Conceptual representation of a DuckDB Relation object) ```json { "type": "duckdb.DuckDBPyRelation", "description": "Represents a relation within DuckDB." } ``` ### Method: to_ipc() #### Description Serialize the data source as an Arrow IPC stream. #### Method to_ipc #### Endpoint N/A (Instance method) #### Parameters None #### Request Example ```python ipc_stream = bigbed_file.to_ipc() ``` #### Response ##### Success Response (Returns bytes) - **Returns:** The data source serialized as an Arrow IPC stream (bytes). #### Response Example (Conceptual representation of Arrow IPC stream bytes) ```json { "type": "bytes", "content_type": "application/octet-stream", "description": "Arrow IPC stream data." } ``` ### Method: to_pandas() #### Description Convert the dataset to a Pandas DataFrame. #### Method to_pandas #### Endpoint N/A (Instance method) #### Parameters None #### Request Example ```python df_pandas = bigbed_file.to_pandas() ``` #### Response ##### Success Response (Returns pd.DataFrame) - **Returns:** A Pandas DataFrame representation of the data source. - **Return type:** pandas.DataFrame #### Response Example (Conceptual representation of a Pandas DataFrame object) ```json { "type": "pandas.DataFrame", "columns": ["chrom", "start", "end"], "data": [ ["chr1", 1000, 1200], ["chr1", 1500, 1700] ] } ``` ### Method: to_polars() #### Description Convert the data source to a Polars DataFrame or LazyFrame. #### Method to_polars #### Endpoint N/A (Instance method) #### Parameters - **lazy** (bool) - Optional - If True, return a LazyFrame. Defaults to False. #### Request Example ```python lf_polars = bigbed_file.to_polars(lazy=True) ``` #### Response ##### Success Response (Returns polars.DataFrame or polars.LazyFrame) - **Returns:** A Polars DataFrame or LazyFrame representation of the data source. - **Return type:** polars.DataFrame | polars.LazyFrame #### Response Example (Conceptual representation of a Polars DataFrame object) ```json { "type": "polars.DataFrame", "columns": ["chrom", "start", "end"], "shape": [2, 3] } ``` ### Method: zoom() #### Description Create a data source for a BBI file zoom level. #### Method zoom #### Endpoint N/A (Instance method) #### Parameters - **resolution** (int) - Required - The desired resolution for the zoom level. - **fields** (list[str] | None) - Optional - Fields to include. - **regions** (str | list[str] | None) - Optional - Regions to query within the zoom level. #### Request Example ```python zoom_data_source = bigbed_file.zoom(resolution=1000, regions="chr1:10000-20000") ``` #### Response ##### Success Response (Returns a data source for the zoom level) - **Returns:** A data source object representing the specified zoom level. #### Response Example (Conceptual representation of a zoom data source object) ```json { "type": "ZoomLevelDataSource", "resolution": 1000, "description": "Data source for a specific BBI zoom level." } ``` ``` -------------------------------- ### FastaFile Methods Source: https://github.com/abdenlab/oxbow/blob/main/docs/api/python/oxbow.core.FastaFile.md Provides methods to interact with the FastaFile data, including converting to different data formats and generating data batches. ```APIDOC ## FastaFile Methods ### Description Methods for processing and converting FastaFile data. ### Methods #### batches() ##### Description Generate record batches from the data source. ##### Method `batches` ##### Parameters None ##### Request Example ```json {} ``` ##### Response * **Yields:** *pa.RecordBatch* – A record batch from the data source. ##### Response Example ```json { "batch_data": "..." } ``` #### columns ##### Description The top-level column names of the projection. ##### Method `columns` (property) ##### Parameters None ##### Response * **Returns:** A list of strings representing the column names. * **Return type:** list[str] ##### Response Example ```json { "columns": ["sequence", "header"] } ``` #### dataset() ##### Description Convert the data source into a dataset. ##### Method `dataset` ##### Parameters None ##### Request Example ```json {} ``` ##### Response * **Returns:** A dataset representation of the data source. * **Return type:** [BatchReaderDataset](oxbow.arrow.BatchReaderDataset.md#oxbow.arrow.BatchReaderDataset) ##### Response Example ```json { "dataset_representation": "..." } ``` #### dd(find_divisions=False) ##### Description Convert the data source to a Dask DataFrame. ##### Method `dd` ##### Parameters * **find_divisions** (bool, optional) - If True, find divisions for the Dask DataFrame (default: False). ##### Request Example ```json { "find_divisions": true } ``` ##### Response * **Returns:** A Dask DataFrame representation of the data source. * **Return type:** dask.dataframe.DataFrame ##### Response Example ```json { "dask_dataframe": "..." } ``` #### fragments() ##### Description Get fragments of the data source. ##### Method `fragments` ##### Parameters None ##### Request Example ```json {} ``` ##### Response * **Returns:** A list of fragments representing parts of the data source. * **Return type:** list[[BatchReaderFragment](oxbow.arrow.BatchReaderFragment.md#oxbow.arrow.BatchReaderFragment)] ##### Response Example ```json { "fragments": [ { "fragment_id": 1, "start": 0, "end": 1000 } ] } ``` #### pd() ##### Description Convert the dataset to a Pandas DataFrame. ##### Method `pd` ##### Parameters None ##### Request Example ```json {} ``` ##### Response * **Returns:** A Pandas DataFrame representation of the dataset. * **Return type:** pandas.DataFrame ##### Response Example ```json { "pandas_dataframe": "..." } ``` ``` -------------------------------- ### Project Columns from BAM File Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Loads data from a BAM file, selecting only specified columns ('rname', 'pos', 'end', 'mapq') using oxbow. This projection is pushed down to the data source to avoid parsing unnecessary columns. The result is a Polars LazyFrame. ```ipython3 import polars as pl ox.from_bam( "data/sample.bam", fields=["rname", "pos", "end", "mapq"], tag_defs=[] ).regions( "chr1" ).pl() ``` -------------------------------- ### BigBedFile Initialization Source: https://github.com/abdenlab/oxbow/blob/main/docs/api/python/oxbow.core.BigBedFile.md Initializes a BigBedFile object to read data from a specified source. Supports various schemas and optional fields, regions, and batch sizes. ```APIDOC ## BigBedFile Initialization ### Description Initializes a BigBedFile object to read data from a specified source. Supports various schemas and optional fields, regions, and batch sizes. ### Method `__init__` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **source** (str | Callable[[], IO[bytes] | str]) - Required - The data source for the BigBed file. - **schema** (str) - Optional - The schema of the BigBed file. Defaults to 'bed3+'. - **fields** (list[str] | None) - Optional - A list of field names to use. - **regions** (str | list[str] | None) - Optional - Genomic regions to query. - **batch_size** (int) - Optional - The batch size for processing. Defaults to 131072. ### Request Example ```json { "source": "path/to/your/file.bb", "schema": "bed6+", "fields": ["chrom", "start", "end", "name", "score", "strand", "value"], "regions": ["chr1:1000-2000"], "batch_size": 65536 } ``` ### Response This is a constructor, so no direct response example is applicable. Initialization creates a BigBedFile object. #### Success Response (200) N/A #### Response Example N/A ``` -------------------------------- ### Create Lazy Polars DataFrame from BAM DataSource Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Loads data from an Oxbow DataSource into a lazy Polars DataFrame. This is recommended for very large datasets that do not fit entirely in memory. The `lazy=True` argument enables lazy evaluation, and `show_graph()` visualizes the computation plan. ```ipython3 df = ds.pl(lazy=True) df.show_graph() ``` -------------------------------- ### PyBedScanner Methods Source: https://github.com/abdenlab/oxbow/blob/main/docs/api/python/oxbow.core.PyBedScanner.md Documentation for the methods available in the PyBedScanner class, including scanning and schema-related operations. ```APIDOC ## Methods of oxbow.core.PyBedScanner ### `field_names()` #### Description Return the names of the BED fields. #### Method GET #### Endpoint N/A (Method of an instantiated object) #### Response #### Success Response (200) - **fields** (list[str]) - A list of field names. #### Response Example ```json { "fields": ["chrom", "start", "end", "name", "score", "strand", "thickStart", "thickEnd", "itemRgb", "blockCount", "blockSizes", "blockStarts"] } ``` ### `scan()` #### Description Scan batches of records from the file. #### Method GET #### Endpoint N/A (Method of an instantiated object) #### Parameters - **fields** (list[str], optional) - Names of the BED fields to project. - **batch_size** (int, optional) - The number of records to include in each batch. Defaults to 1024. - **limit** (int, optional) - The maximum number of records to scan. If None, records are scanned until EOF. #### Returns An iterator yielding Arrow record batches. #### Return Type arro3 RecordBatchReader (pycapsule) ### `scan_byte_ranges()` #### Description Scan batches of records from specified byte ranges in the file. The byte positions must align with record boundaries. #### Method GET #### Endpoint N/A (Method of an instantiated object) #### Parameters - **byte_ranges** (list[tuple[int, int]]) - List of (start, end) byte position tuples to read from. - **fields** (list[str], optional) - Names of the fixed fields to project. - **batch_size** (int, optional) - The number of records to include in each batch. Defaults to 1024. - **limit** (int, optional) - The maximum number of records to scan. If None, all records in the specified ranges are scanned. #### Returns An iterator yielding Arrow record batches. #### Return Type arro3 RecordBatchReader (pycapsule) ### `scan_query()` #### Description Scan batches of records from a genomic range query on a BGZF-encoded file. This operation requires an index file. #### Method GET #### Endpoint N/A (Method of an instantiated object) #### Parameters - **region** (str) - Genomic region in the format "chr:start-end". - **index** (path or file-like, optional) - The index file to use for querying the region. If None and the source was provided as a path, an index will be attempted to be loaded from the same path with an additional extension. - **fields** (list[str], optional) - Names of the BED fields to project. - **batch_size** (int, optional) - The number of records to include in each batch. Defaults to 1024. - **limit** (int, optional) - The maximum number of records to scan. If None, all records intersecting the query range are scanned. #### Returns An iterator yielding Arrow record batches. #### Return Type arro3 RecordBatchReader (pycapsule) ### `scan_virtual_ranges()` #### Description Scan batches of records from virtual position ranges in a BGZF file. #### Method GET #### Endpoint N/A (Method of an instantiated object) #### Parameters - **vpos_ranges** (list[tuple[int, int]]) - List of (start, end) virtual position tuples to read from. - **fields** (list[str], optional) - Names of the BED fields to project. - **batch_size** (int, optional) - The number of records to include in each batch. Defaults to 1024. - **limit** (int, optional) - The maximum number of records to scan. If None, all records in the specified ranges are scanned. #### Returns An iterator yielding Arrow record batches. #### Return Type arro3 RecordBatchReader (pycapsule) ### `schema()` #### Description Return the Arrow schema. #### Method GET #### Endpoint N/A (Method of an instantiated object) #### Parameters - **fields** (list[str], optional) - Names of the BED fields to project. #### Response #### Success Response (200) - **schema** (dict) - The Arrow schema definition. #### Response Example ```json { "schema": { "fields": [ {"name": "chrom", "type": {"name": "string"}}, {"name": "start", "type": {"name": "int64"}}, {"name": "end", "type": {"name": "int64"}} ], "metadata": {} } } ``` ``` -------------------------------- ### Load BED File with bed3+6 Schema Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Loads data from a BED file using oxbow, interpreting it according to the 'bed3+6' schema. This schema specifies the standard three BED columns plus exactly six additional columns. The output is a Polars DataFrame. ```ipython3 ox.from_bed("data/sample.bed", bed_schema="bed3+6").pl().head() ``` -------------------------------- ### PyBcfScanner Constructor Source: https://github.com/abdenlab/oxbow/blob/main/docs/api/python/oxbow.core.PyBcfScanner.md Initializes a BCF file scanner. It can take a file path or a file-like object as input and supports BGZF compression. ```APIDOC ## PyBcfScanner ### Description A BCF file scanner. ### Method __init__ ### Parameters #### Path Parameters - **src** (str or file-like) - Required - The path to the BCF file or a file-like object. - **compressed** (bool, optional) - Whether the source is BGZF-compressed. Defaults to True. ### Request Example ```json { "src": "/path/to/your/file.bcf", "compressed": true } ``` ### Response #### Success Response (200) - **PyBcfScanner** (object) - An instance of the PyBcfScanner class. #### Response Example ```json { "message": "PyBcfScanner initialized successfully" } ``` ``` -------------------------------- ### Create Scanner Source: https://github.com/abdenlab/oxbow/blob/main/docs/api/python/oxbow.core.GtfFile.md Creates a low-level scanner for the data source. ```APIDOC ## scanner() ### Description Creates a low-level scanner for the data source. ### Method GET ### Endpoint /scanner ### Parameters None ### Request Body None ### Request Example None ### Response #### Success Response (200) - **Scanner** (Any) - A low-level scanner object for the data source. #### Response Example ```json { "example": "" } ``` ``` -------------------------------- ### VcfFile Constructor Source: https://github.com/abdenlab/oxbow/blob/main/docs/api/python/oxbow.core.VcfFile.md Initializes a VcfFile object to read and process VCF data. It supports various configurations for data sources, compression, fields, and regions. ```APIDOC ## oxbow.core.VcfFile ### Description Initializes a VcfFile object to read and process VCF data. It supports various configurations for data sources, compression, fields, and regions. ### Method `__init__` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **source** (str | Callable[[], IO[bytes] | str]) - Required - The data source for the VCF file. - **compressed** (bool) - Optional - Whether the VCF file is compressed. Defaults to False. - **fields** (list[str] | None) - Optional - Specific fields to include. - **info_fields** (list[str] | None) - Optional - Specific INFO fields to include. - **samples** (list[str] | None) - Optional - Specific samples to include. - **genotype_fields** (list[str] | None) - Optional - Specific genotype fields to include. - **genotype_by** (Literal['sample', 'field']) - Optional - How to group genotype data. Defaults to 'sample'. - **regions** (str | list[str] | None) - Optional - Genomic regions to query. - **index** (str | Callable[[], IO[bytes] | str] | None) - Optional - Path to an index file or a function to load it. - **batch_size** (int) - Optional - The size of record batches. Defaults to 131072. ### Request Example ```json { "example": "VcfFile(source='/path/to/your.vcf.gz', compressed=True, samples=['sample1', 'sample2'])" } ``` ### Response #### Success Response (200) None (Constructor does not return a value) #### Response Example None ``` -------------------------------- ### zoom() Source: https://github.com/abdenlab/oxbow/blob/main/docs/api/python/oxbow.core.BigBedFile.md Creates a data source specifically for a zoom level of a BBI file, allowing access to data at a particular resolution. ```APIDOC ## GET /zoom ### Description Create a data source for a BBI file zoom level. ### Method GET ### Endpoint /zoom ### Parameters #### Query Parameters - **resolution** (int) - Required - The resolution / reduction level for zoomed data, in bp. - **fields** (list[str]) - Optional - Fields to include in the zoom data. - **regions** (str | list[str]) - Optional - Regions to query within the zoom level. - **batch_size** (int) - Optional - The batch size for processing zoom data. Defaults to 131072. ### Request Example ```json { "resolution": 10000, "regions": "chr1:1000-5000" } ``` ### Response #### Success Response (200) - **BbiZoom** (BbiZoom) - A data source representing a BBI file zoom level. #### Response Example ```json { "message": "BBI Zoom data source created successfully." } ``` ``` -------------------------------- ### Load BAM File Ignoring SAM Tags Source: https://github.com/abdenlab/oxbow/blob/main/docs/getting-started/quickstart.md Loads data from a BAM file using oxbow while explicitly ignoring all SAM tags by setting `tag_defs=[]`. This prevents oxbow from scanning rows to discover tags, which can be useful for performance or when tags are not needed. The output is a Polars DataFrame. ```ipython3 df = ( ox.from_bam( "data/sample.bam", tag_defs=[] ) .regions("chr1") .pl() ) df ```