### View partition information Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/tutorial.html Example output showing the actual number of partitions determined by the VCF index. ```text num_partitions 3 num_samples 3 num_variants 9 ``` -------------------------------- ### Install bio2zarr via pip Source: https://sgkit-dev.github.io/bio2zarr/_sources/installation.md Install the base package or include optional dependencies for VCF or tskit support. ```bash python3 -m pip install bio2zarr #Base package with only plink support python3 -m pip install bio2zarr[vcf] #Install with VCF support python3 -m pip install bio2zarr[tskit] #Install with tskit support python3 -m pip install bio2zarr[all] #Install with all optional dependencies ``` -------------------------------- ### Zarr inspection output table Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/tutorial.html Example output showing the structure and compression settings of the Zarr arrays. ```text name dtype stored size ratio nchunks chunk_size avg_chunk_stored shape chunk_shape compressor filters --------------------- ------------- --------- --------- ------- --------- ------------ ------------------ --------- ------------- -------------------------------------------------------------- ------------- /variant_allele StringDType() 558 bytes 576 bytes 1 1 576.0 bytes 558 bytes (9, 4) (9, 4) Blosc(cname='zstd', clevel=7, shuffle=SHUFFLE, blocksize=0) (VLenUTF8(),) /variant_id StringDType() 534 bytes 144 bytes 0.27 1 144.0 bytes 534 bytes (9,) (9,) Blosc(cname='zstd', clevel=7, shuffle=SHUFFLE, blocksize=0) (VLenUTF8(),) /variant_AC int8 503 bytes 18 bytes 0.036 1 18.0 bytes 503 bytes (9, 2) (9, 2) Blosc(cname='zstd', clevel=7, shuffle=SHUFFLE, blocksize=0) () /variant_contig int8 499 bytes 9 bytes 0.018 1 9.0 bytes 499 bytes (9,) (9,) Blosc(cname='zstd', clevel=7, shuffle=SHUFFLE, blocksize=0) () /variant_AF float32 498 bytes 72 bytes 0.14 1 72.0 bytes 498 bytes (9, 2) (9, 2) Blosc(cname='zstd', clevel=7, shuffle=SHUFFLE, blocksize=0) () /call_genotype_mask bool 488 bytes 54 bytes 0.11 1 54.0 bytes 488 bytes (9, 3, 2) (9, 3, 2) Blosc(cname='zstd', clevel=7, shuffle=BITSHUFFLE, blocksize=0) () /call_genotype int8 488 bytes 54 bytes 0.11 1 54.0 bytes 488 bytes (9, 3, 2) (9, 3, 2) Blosc(cname='zstd', clevel=7, shuffle=BITSHUFFLE, blocksize=0) () /region_index int32 480 bytes 72 bytes 0.15 1 72.0 bytes 480 bytes (3, 6) (3, 6) Blosc(cname='zstd', clevel=9, shuffle=NOSHUFFLE, blocksize=0) () /variant_AA StringDType() 478 bytes 144 bytes 0.3 1 144.0 bytes 478 bytes (9,) (9,) Blosc(cname='zstd', clevel=7, shuffle=SHUFFLE, blocksize=0) (VLenUTF8(),) /filter_description StringDType() 473 bytes 48 bytes 0.1 1 48.0 bytes 473 bytes (3,) (3,) Blosc(cname='zstd', clevel=7, shuffle=SHUFFLE, blocksize=0) (VLenUTF8(),) /variant_filter bool 461 bytes 27 bytes 0.059 1 27.0 bytes 461 bytes (9, 3) (9, 3) Blosc(cname='zstd', clevel=7, shuffle=BITSHUFFLE, blocksize=0) () /call_GQ int8 449 bytes 27 bytes 0.06 1 27.0 bytes 449 bytes (9, 3) (9, 3) Blosc(cname='zstd', clevel=7, shuffle=SHUFFLE, blocksize=0) () /call_DP int8 443 bytes 27 bytes 0.061 1 27.0 bytes 443 bytes (9, 3) (9, 3) Blosc(cname='zstd', clevel=7, shuffle=SHUFFLE, blocksize=0) () ``` -------------------------------- ### Install bio2zarr via bioconda Source: https://sgkit-dev.github.io/bio2zarr/_sources/installation.md Install the package with all optional dependencies included from the bioconda channel. ```bash conda install -c bioconda bio2zarr ``` -------------------------------- ### Initialize distributed explode Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Performs the initial setup for a distributed conversion of VCF files into an intermediate columnar format. ```bash vcf2zarr dexplode-init [OPTIONS] VCFS... ICF_PATH ``` -------------------------------- ### Clean up existing ICF directory Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcf2zarr/tutorial.md Removes any existing ICF directory before starting a new distributed process. ```bash rm -fR sample-dist.icf ``` -------------------------------- ### Equivalent PLINK and vcf2zarr conversion process Source: https://sgkit-dev.github.io/bio2zarr/_sources/plink2zarr/overview.md Demonstrates the manual conversion process using PLINK 1.9 and vcf2zarr that plink2zarr aims to replicate. ```bash $ plink1.9 --bfile fileset --keep-allele-order --recode vcf-iid --out tmp $ vcf2zarr convert tmp.vcf out.vcz ``` -------------------------------- ### List Zarr directory contents Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcf2zarr/tutorial.md Verify the creation of the Zarr dataset directory. ```bash ls sample.vcz ``` -------------------------------- ### Initialize Distributed Encoding Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Sets up the Zarr path for distributed conversion by defining the number of partitions. ```bash vcf2zarr dencode-init [OPTIONS] ICF_PATH ZARR_PATH ``` -------------------------------- ### Generate and preview storage schema Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcf2zarr/tutorial.md Create a JSON schema file from an ICF and display the first 20 lines. ```bash vcf2zarr mkschema sample.icf > sample.schema.json head -n 20 sample.schema.json ``` -------------------------------- ### Download sample VCF data Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/overview.html Retrieve indexed VCF files required for the conversion process. ```bash curl -O https://raw.githubusercontent.com/sgkit-dev/bio2zarr/main/tests/data/vcf/sample.vcf.gz curl -O https://raw.githubusercontent.com/sgkit-dev/bio2zarr/main/tests/data/vcf/sample.vcf.gz.tbi ``` -------------------------------- ### Pack a Zarr directory store with default output Source: https://sgkit-dev.github.io/bio2zarr/_sources/zipzarr/overview.md Compresses a Zarr directory store while managing the source directory based on the -k flag. ```bash zipzarr sample.vcz # writes sample.vcz.zip, removes sample.vcz zipzarr -k sample.vcz # writes sample.vcz.zip, keeps sample.vcz ``` -------------------------------- ### Partition a VCF file by size Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcfpartition/overview.md Use the -s flag to specify a target size for each partition, supporting suffixes like K, M, or G. ```bash ls -lh NA12878.prod.chr20snippet.g.vcf.gz ``` ```bash vcfpartition NA12878.prod.chr20snippet.g.vcf.gz -s 500K ``` -------------------------------- ### Process partitions in a shell loop Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcfpartition/overview.md Demonstrates iterating over the output of vcfpartition to perform operations on each region. ```bash vcfpartition CEUTrio.20.21.gatk3.4.g.bcf -n 3 | while read split; do bcftools view -Hr $split | wc -l done ``` -------------------------------- ### vcf2zarr dexplode-init Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Initializes the distributed conversion process for VCF files. ```APIDOC ## vcf2zarr dexplode-init ### Description Initial step for distributed conversion of VCF(s) to intermediate columnar format over some number of partitions. ### Usage `vcf2zarr dexplode-init [OPTIONS] VCFS... ICF_PATH` ### Options - **-n, --num-partitions** (int) - Target number of partitions - **-f, --force** - Force overwriting - **-c, --column-chunk-size** (int) - Chunk size in MiB - **-C, --compressor** (string) - Codec (lz4 | zstd) - **--json** - Output summary in JSON - **-v, --verbose** - Increase verbosity - **-P, --progress / -Q, --no-progress** - Show progress bars - **-p, --worker-processes** (int) - Number of worker processes ### Arguments - **VCFS** - Required - Input VCF file(s) - **ICF_PATH** - Required - Output path ``` -------------------------------- ### List Zarr directory structure Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcf2zarr/tutorial.md Examine the hierarchical directory structure of the Zarr dataset. ```bash find sample.vcz -maxdepth 2 -type d | sort ``` -------------------------------- ### zipzarr Usage Syntax Source: https://sgkit-dev.github.io/bio2zarr/zipzarr/cli_ref.html The basic command structure for invoking the zipzarr utility. ```bash zipzarr [OPTIONS] SRC [DEST] ``` -------------------------------- ### Initialize distributed explosion Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcf2zarr/tutorial.md Initializes the ICF directory with a specified number of partitions for parallel processing. ```bash vcf2zarr dexplode-init sample.vcf.gz sample-dist.icf -n 5 -Q ``` -------------------------------- ### Observe partition granularity limits Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcfpartition/overview.md Shows that the number of partitions is constrained by the underlying file index. ```bash vcfpartition CEUTrio.20.21.gatk3.4.g.bcf -n 30 ``` -------------------------------- ### Unzip a zipped Zarr store with default output Source: https://sgkit-dev.github.io/bio2zarr/_sources/zipzarr/overview.md Extracts a zipped Zarr store while managing the source archive based on the -k flag. ```bash zipzarr -u sample.vcz.zip # writes sample.vcz, removes the .zip zipzarr -u -k sample.vcz.zip # writes sample.vcz, keeps the .zip ``` -------------------------------- ### Partition a BCF file into a specific number of parts Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcfpartition/overview.md Use the -n flag to divide the file into the requested number of partitions. ```bash vcfpartition CEUTrio.20.21.gatk3.4.g.bcf -n 3 ``` -------------------------------- ### vcf2zarr create-index Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Creates an index for a Zarr store to enable efficient region queries. ```APIDOC ## vcf2zarr create-index ### Description Create an index to support efficient region queries. ### Usage `vcf2zarr create-index [OPTIONS] ZARR_PATH` ### Options - **-v, --verbose** - Increase verbosity ### Arguments - **ZARR_PATH** - Required - Path to the Zarr store ``` -------------------------------- ### Pack a Zarr directory store Source: https://sgkit-dev.github.io/bio2zarr/_sources/zipzarr/overview.md Compresses a Zarr directory store into a single .zip file. ```bash zipzarr sample.vcz sample.vcz.zip ``` -------------------------------- ### Generate plink2zarr CLI documentation Source: https://sgkit-dev.github.io/bio2zarr/_sources/plink2zarr/cli_ref.md Uses the click directive to automatically generate documentation for the plink2zarr convert command. ```rst .. _cmd-plink2zarr-convert: .. click:: bio2zarr.cli:convert_plink :prog: plink2zarr convert :nested: full ``` -------------------------------- ### vcf2zarr dencode-init Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Initializes the conversion of intermediate format to VCF Zarr for distributed processing. ```APIDOC ## vcf2zarr dencode-init ### Description Initialise conversion of intermediate format to VCF Zarr. This will set up the specified ZARR_PATH to perform this conversion over some number of partitions. ### Usage `vcf2zarr dencode-init [OPTIONS] ICF_PATH ZARR_PATH` ### Arguments - **ICF_PATH** (string) - Required - Path to the intermediate columnar format data. - **ZARR_PATH** (string) - Required - Path to the output Zarr directory. ### Options - **-n, --num-partitions** (int) - Optional - Target number of partitions to split into. - **-f, --force** - Optional - Force overwriting of existing directories. - **-s, --schema** (string) - Optional - Schema definition. - **-l, --variants-chunk-size** (int) - Optional - Chunk size in the variants dimension. - **-w, --samples-chunk-size** (int) - Optional - Chunk size in the samples dimension. - **-V, --max-variant-chunks** (int) - Optional - Truncate the output in the variants dimension. - **--json** - Optional - Output summary data in JSON format. - **-P, --progress / -Q, --no-progress** - Optional - Show progress bars. - **-v, --verbose** - Optional - Increase verbosity. - **--zarr-format** (2|3) - Optional - Zarr format version of output. ``` -------------------------------- ### vcfpartition command usage Source: https://sgkit-dev.github.io/bio2zarr/vcfpartition/cli_ref.html Basic syntax for invoking the vcfpartition command with VCF files. ```bash vcfpartition [OPTIONS] VCFS... ``` -------------------------------- ### Execute distributed partition tasks Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcf2zarr/tutorial.md Runs the explosion process for individual partitions. In production, these should be dispatched via a cluster scheduler. ```bash vcf2zarr dexplode-partition sample-dist.icf 0 vcf2zarr dexplode-partition sample-dist.icf 1 vcf2zarr dexplode-partition sample-dist.icf 2 ``` -------------------------------- ### Convert PLINK fileset to VCF Zarr Source: https://sgkit-dev.github.io/bio2zarr/_sources/plink2zarr/python_api.md Use the convert function to transform a PLINK fileset into a Zarr store. The plink_prefix must be the shared prefix for the .bed, .bim, and .fam files. ```python import bio2zarr.plink as p2z root = p2z.convert(plink_prefix, vcz_path) ``` -------------------------------- ### Create index for Zarr path Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Generates an index for a Zarr store to enable efficient region-based queries. ```bash vcf2zarr create-index [OPTIONS] ZARR_PATH ``` -------------------------------- ### Inspect VCF Zarr results Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/overview.html Verify the contents of the generated VCF Zarr dataset. ```bash vcf2zarr inspect sample.vcz ``` -------------------------------- ### Inspect final Zarr output Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcf2zarr/tutorial.md Verify the contents of the generated Zarr file. ```bash vcf2zarr inspect sample_noHQ.vcz ``` -------------------------------- ### Convert a Partition Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Converts a specific partition of the intermediate format to VCF Zarr, requiring prior initialization. ```bash vcf2zarr dencode-partition [OPTIONS] ZARR_PATH PARTITION ``` -------------------------------- ### Inspect Zarr or Columnar Format Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Inspects an intermediate columnar format or a Zarr path. ```bash vcf2zarr inspect [OPTIONS] PATH ``` -------------------------------- ### Convert VCF to Zarr Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcf2zarr/python_api.md Basic usage for converting a VCF file to a Zarr store on disk. ```python import bio2zarr.vcf as v2z v2z.convert([vcf_path], vcz_path) ``` -------------------------------- ### vcfpartition Source: https://sgkit-dev.github.io/bio2zarr/vcfpartition/cli_ref.html The vcfpartition command partitions indexed VCF/BCF files into regions. It requires either a target number of partitions (-n) or a target partition size (-s). ```APIDOC ## vcfpartition ### Description Output bcftools region strings that partition the indexed VCF/BCF files into either an approximate number of parts or parts of approximately a given size. ### Usage `vcfpartition [OPTIONS] VCFS...` ### Options - **--version** - Show the version and exit. - **-v, --verbose** - Increase verbosity. - **-n, --num-partitions** (integer) - Target number of partitions to split into. - **-s, --partition-size** (string) - Target (compressed) size of VCF partitions, e.g. 100KB, 10MiB, 1G. ### Arguments - **VCFS** - Required argument(s) representing the VCF/BCF files to partition. ``` -------------------------------- ### Enable Bash shell completion Source: https://sgkit-dev.github.io/bio2zarr/_sources/installation.md Enable command-line completion for bio2zarr tools in a Bash session. ```bash eval "$(_VCF2ZARR_COMPLETE=bash_source vcf2zarr)" eval "$(_TSKIT2ZARR_COMPLETE=bash_source tskit2zarr)" eval "$(_PLINK2ZARR_COMPLETE=bash_source plink2zarr)" ``` -------------------------------- ### bio2zarr.plink.convert Source: https://sgkit-dev.github.io/bio2zarr/_sources/plink2zarr/python_api.md Converts a PLINK fileset (consisting of .bed, .bim, and .fam files) into a VCF Zarr store. ```APIDOC ## bio2zarr.plink.convert(plink_prefix, vcz_path) ### Description Converts a PLINK fileset with the given path prefix (the shared prefix of the .bed, .bim, and .fam files) to VCF Zarr stored at the specified path. ### Parameters - **plink_prefix** (str) - The shared prefix of the input PLINK files (.bed, .bim, .fam). - **vcz_path** (str) - The destination path for the output VCF Zarr store. ### Usage Example ```python import bio2zarr.plink as p2z root = p2z.convert('path/to/plink_prefix', 'output_path.vcz') ``` ``` -------------------------------- ### zipzarr Source: https://sgkit-dev.github.io/bio2zarr/zipzarr/cli_ref.html The zipzarr command packs a Zarr directory store into a .zip archive or extracts a .zip archive into a directory. ```APIDOC ## zipzarr ### Description Pack a Zarr directory store into a single .zip archive, or extract a .zip archive into a directory using the --unzip flag. ### Usage `zipzarr [OPTIONS] SRC [DEST]` ### Arguments - **SRC** (string) - Required - The source directory to zip or the .zip archive to unzip. - **DEST** (string) - Optional - The destination path. Defaults to SRC with a .zip suffix (for zipping) or with the .zip suffix stripped (for unzipping). ### Options - **--version** - Show the version and exit. - **-u, --unzip** - Unzip SRC into DEST. - **-k, --keep** - Keep (don't delete) the source after a successful operation. - **-f, --force** - Force overwriting of existing directories. - **-v, --verbose** - Increase verbosity. - **-P, --progress** - Show progress bars (default). - **-Q, --no-progress** - Do not show progress bars. ``` -------------------------------- ### Customize VCF Zarr conversion with model mapping Source: https://sgkit-dev.github.io/bio2zarr/_sources/tskit2zarr/python_api.md Use map_to_vcf_model to specify subsets of individuals and output properties before conversion. ```python model_mapping = ts.map_to_vcf_model( individuals=[0, 1], contig_id="chr1", isolated_as_missing=True ) root = ts2z.convert(ts, vcz_path, model_mapping=model_mapping) ``` -------------------------------- ### Convert tskit to Zarr Source: https://sgkit-dev.github.io/bio2zarr/tskit2zarr/cli_ref.html Basic usage pattern for the tskit2zarr convert command. ```bash tskit2zarr convert [OPTIONS] TS_PATH ZARR_PATH ``` -------------------------------- ### Encode VCF using custom schema Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/tutorial.html Run the encoding process with the modified schema file. ```bash vcf2zarr encode sample.icf -Qs sample_noHQ.schema.json sample_noHQ.vcz ``` -------------------------------- ### Inspect ICF file Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcf2zarr/tutorial.md Examine the fields, data types, and statistics of an intermediate columnar format file. ```bash vcf2zarr inspect sample.icf ``` -------------------------------- ### Convert VCF to Zarr Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/overview.html Execute the two-step conversion process from VCF to ICF and then to the final VCF Zarr format. ```bash vcf2zarr explode sample.vcf.gz sample.icf vcf2zarr encode sample.icf sample.vcz ``` -------------------------------- ### vcf2zarr dexplode-finalise Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Finalizes the distributed conversion process. ```APIDOC ## vcf2zarr dexplode-finalise ### Description Final step for distributed conversion of VCF(s) to intermediate columnar format. ### Usage `vcf2zarr dexplode-finalise [OPTIONS] ICF_PATH` ### Options - **-v, --verbose** - Increase verbosity ### Arguments - **ICF_PATH** - Required - Path to the intermediate columnar format directory ``` -------------------------------- ### Query schema field with jq Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcf2zarr/tutorial.md Extract the schema definition for a specific field using jq. ```bash jq '.fields[] | select(.name == "call_genotype")' < sample.schema.json ``` -------------------------------- ### Invoke bio2zarr executables Source: https://sgkit-dev.github.io/bio2zarr/_sources/installation.md Run tools using the python module syntax as an alternative to calling executables directly. ```bash python3 -m bio2zarr vcf2zarr ``` ```bash vcf2zarr ``` -------------------------------- ### Convert PLINK fileset to VCF Zarr Source: https://sgkit-dev.github.io/bio2zarr/_sources/plink2zarr/overview.md Executes the conversion of a PLINK binary fileset to a VCF Zarr output directory. ```bash $ plink2zarr convert fileset out.vcz ``` -------------------------------- ### Convert VCF to Zarr Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/tutorial.html Converts an input VCF file to a Zarr dataset in a single pass. ```bash vcf2zarr convert ``` -------------------------------- ### Parallel processing with xargs Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcfpartition/overview.md Utilize xargs to process partitions in parallel across multiple CPU cores. ```bash vcfpartition CEUTrio.20.21.gatk3.4.g.bcf -n 3 \ | xargs -P 3 -I {} sh -c "bcftools view -Hr {} | wc -l" ``` -------------------------------- ### vcf2zarr inspect Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Inspects an intermediate columnar format or Zarr path. ```APIDOC ## vcf2zarr inspect ### Description Inspect an intermediate columnar format or Zarr path. ### Usage `vcf2zarr inspect [OPTIONS] PATH` ### Options - **-v, --verbose** - Increase verbosity ### Arguments - **PATH** - Required: Path to inspect ``` -------------------------------- ### Convert VCF to Zarr Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Converts input VCF files directly to VCF Zarr format. ```bash vcf2zarr convert [OPTIONS] VCFS... ZARR_PATH ``` -------------------------------- ### Use max-variants-chunks for schema tuning Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/tutorial.html Use this option to encode only the first few chunks of a dataset during schema tuning operations. ```bash max-variants-chunks ``` -------------------------------- ### Convert VCF partition Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Converts a specific partition of a VCF file to the intermediate columnar format, requiring prior initialization with dexplode-init. ```bash vcf2zarr dexplode-partition [OPTIONS] ICF_PATH PARTITION ``` -------------------------------- ### Unzip a zipped Zarr store Source: https://sgkit-dev.github.io/bio2zarr/_sources/zipzarr/overview.md Extracts a zipped Zarr store to a target directory. ```bash zipzarr -u sample.vcz.zip sample-roundtrip.vcz ``` -------------------------------- ### vcf2zarr convert Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Converts input VCF files directly to VCF Zarr format. ```APIDOC ## vcf2zarr convert ### Description Convert input VCF(s) directly to VCF Zarr. ### Usage `vcf2zarr convert [OPTIONS] VCFS... ZARR_PATH` ### Options - **-f, --force** - Force overwriting of existing directories - **-l, --variants-chunk-size** (int) - Chunk size in the variants dimension - **-w, --samples-chunk-size** (int) - Chunk size in the samples dimension - **-v, --verbose** - Increase verbosity - **-P, --progress, -Q, --no-progress** - Show progress bars (default: show) - **-p, --worker-processes** (int) - Number of worker processes (default: 0) - **--local-alleles, --no-local-alleles** - Use local allele fields to reduce storage requirements (default: False) - **--ploidy** (int) - Set the ploidy dimension explicitly - **--zarr-format** (2|3) - Zarr format version of output (default: 2) ### Arguments - **VCFS** - Required: Input VCF file(s) - **ZARR_PATH** - Required: Output Zarr path ``` -------------------------------- ### Finalize Distributed Encoding Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Completes the distributed conversion process for VCF Zarr. ```bash vcf2zarr dencode-finalise [OPTIONS] ZARR_PATH ``` -------------------------------- ### vcf2zarr dencode-finalise Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Finalizes the distributed conversion process. ```APIDOC ## vcf2zarr dencode-finalise ### Description Final step for distributed conversion of ICF to VCF Zarr. ### Usage `vcf2zarr dencode-finalise [OPTIONS] ZARR_PATH` ### Arguments - **ZARR_PATH** (string) - Required - Path to the Zarr directory. ### Options - **-v, --verbose** - Optional - Increase verbosity. - **-P, --progress / -Q, --no-progress** - Optional - Show progress bars. ``` -------------------------------- ### vcf2zarr dencode-partition Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Converts a specific partition from intermediate columnar format to VCF Zarr. ```APIDOC ## vcf2zarr dencode-partition ### Description Convert a partition from intermediate columnar format to VCF Zarr. Must be called after the Zarr path has been initialised with dencode_init. ### Usage `vcf2zarr dencode-partition [OPTIONS] ZARR_PATH PARTITION` ### Arguments - **ZARR_PATH** (string) - Required - Path to the Zarr directory. - **PARTITION** (int) - Required - The partition index to process. ### Options - **-v, --verbose** - Optional - Increase verbosity. - **--one-based** - Optional - Partition indexes are interpreted as one-based. ``` -------------------------------- ### Convert VCF to In-Memory Zarr Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcf2zarr/python_api.md Converts a VCF file directly into an in-memory Zarr store without writing to disk. ```python root = v2z.convert([vcf_path]) ``` -------------------------------- ### Encode Zarr with custom schema Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcf2zarr/tutorial.md Perform the final encoding step using a modified schema file. ```bash rm -fR sample_noHQ.vcz vcf2zarr encode sample.icf -Qs sample_noHQ.schema.json sample_noHQ.vcz ``` -------------------------------- ### vcf2zarr mkschema Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Generates a schema for Zarr encoding. ```APIDOC ## vcf2zarr mkschema ### Description Generate a schema for zarr encoding. ### Usage `vcf2zarr mkschema [OPTIONS] ICF_PATH` ### Options - **-l, --variants-chunk-size** (int) - Chunk size in the variants dimension - **-w, --samples-chunk-size** (int) - Chunk size in the samples dimension - **--local-alleles, --no-local-alleles** - Use local allele fields to reduce storage requirements (default: False) - **--ploidy** (int) - Set the ploidy dimension explicitly ### Arguments - **ICF_PATH** - Required: Path to the intermediate columnar format ``` -------------------------------- ### Generate Zarr Schema Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Generates a schema for Zarr encoding from an intermediate columnar format path. ```bash vcf2zarr mkschema [OPTIONS] ICF_PATH ``` -------------------------------- ### Explode VCF files Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Converts VCF files into an intermediate columnar format. ```bash vcf2zarr explode [OPTIONS] VCFS... ICF_PATH ``` -------------------------------- ### bio2zarr.vcf.convert Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcf2zarr/python_api.md The convert function is the primary entry point for transforming VCF data into Zarr format. It accepts a list of VCF paths and an optional output path for the resulting Zarr store. ```APIDOC ## bio2zarr.vcf.convert(vcf_paths, output_path=None) ### Description Converts VCF files to a Zarr store. If an output path is provided, the data is written to that location; otherwise, it returns an in-memory Zarr store. ### Parameters - **vcf_paths** (list) - Required - A list of paths to the input VCF files. - **output_path** (str) - Optional - The destination path for the Zarr store. If omitted, an in-memory store is created. ### Usage Examples #### Basic file conversion ```python import bio2zarr.vcf as v2z v2z.convert(['input.vcf'], 'output.zarr') ``` #### In-memory conversion ```python root = v2z.convert(['input.vcf']) ``` #### Zip archive conversion ```python root = v2z.convert(['input.vcf'], 'output.vcz.zip') ``` ``` -------------------------------- ### Convert tskit to VCF Zarr Source: https://sgkit-dev.github.io/bio2zarr/_sources/tskit2zarr/python_api.md Basic usage of the convert function to transform a tree sequence into VCF Zarr format. ```python import bio2zarr.tskit as ts2z root = ts2z.convert(ts_path, vcz_path, worker_processes=8) ``` -------------------------------- ### plink2zarr convert Source: https://sgkit-dev.github.io/bio2zarr/plink2zarr/cli_ref.html Converts a PLINK fileset to VCF Zarr format. ```APIDOC ## CLI Command: plink2zarr convert ### Description Converts a PLINK fileset to VCF Zarr. The results are equivalent to running plink1.9 to generate a VCF and then using vcf2zarr to convert that VCF to Zarr. ### Usage `plink2zarr convert [OPTIONS] IN_PATH ZARR_PATH` ### Arguments - **IN_PATH** (string) - Required - Path to the input PLINK fileset. - **ZARR_PATH** (string) - Required - Path to the output Zarr directory. ### Options - **-f, --force** (flag) - Optional - Force overwriting of existing directories. - **-p, --worker-processes** (integer) - Optional - Number of worker processes (Default: 0). - **-P, --progress / -Q, --no-progress** (flag) - Optional - Show or hide progress bars (Default: show). - **-v, --verbose** (flag) - Optional - Increase verbosity. - **-l, --variants-chunk-size** (integer) - Optional - Chunk size in the variants dimension. - **-w, --samples-chunk-size** (integer) - Optional - Chunk size in the samples dimension. - **--zarr-format** (integer) - Optional - Zarr format version of output (Options: 2, 3; Default: 2). ``` -------------------------------- ### tskit2zarr convert Source: https://sgkit-dev.github.io/bio2zarr/_sources/tskit2zarr/cli_ref.md The convert command is used to transform tskit data structures into the Zarr format. ```APIDOC ## tskit2zarr convert ### Description Converts tskit data into Zarr format. This command is the primary entry point for the tskit2zarr conversion utility. ### Usage `tskit2zarr convert [OPTIONS] [ARGS]` ``` -------------------------------- ### Convert VCF to Zip Archive Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcf2zarr/python_api.md Converts a VCF file into a Zarr-formatted zip archive. ```python root = v2z.convert([vcf_path], "output.vcz.zip") ``` -------------------------------- ### vcf2zarr dexplode-partition Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Converts a specific VCF partition to intermediate columnar format. ```APIDOC ## vcf2zarr dexplode-partition ### Description Convert a VCF partition to intermediate columnar format. Must be called after dexplode-init. ### Usage `vcf2zarr dexplode-partition [OPTIONS] ICF_PATH PARTITION` ### Options - **-v, --verbose** - Increase verbosity - **--one-based** - Interpret partition indexes as one-based ### Arguments - **ICF_PATH** - Required - Path initialized by dexplode-init - **PARTITION** - Required - Partition index ``` -------------------------------- ### vcf2zarr explode Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Converts VCF files into an intermediate columnar format. ```APIDOC ## vcf2zarr explode ### Description Convert VCF(s) to intermediate columnar format. ### Usage `vcf2zarr explode [OPTIONS] VCFS... ICF_PATH` ### Options - **-f, --force** - Force overwriting of existing directories - **-v, --verbose** - Increase verbosity - **-c, --column-chunk-size** (int) - Approximate uncompressed size of exploded column chunks in MiB - **-C, --compressor** (string) - Codec to use (lz4 | zstd, default=zstd) - **-P, --progress / -Q, --no-progress** - Show progress bars - **-p, --worker-processes** (int) - Number of worker processes (default: 0) ### Arguments - **VCFS** - Required - Input VCF file(s) - **ICF_PATH** - Required - Output path for intermediate columnar format ``` -------------------------------- ### plink2zarr convert Source: https://sgkit-dev.github.io/bio2zarr/_sources/plink2zarr/cli_ref.md The convert command is used to transform PLINK formatted data into the Zarr format. ```APIDOC ## plink2zarr convert ### Description Converts PLINK files to Zarr format. ### Usage `plink2zarr convert [OPTIONS] [ARGS]` ``` -------------------------------- ### Finalize distributed explosion Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcf2zarr/tutorial.md Completes the distributed explosion process after all partition tasks have finished. ```bash vcf2zarr dexplode-finalise sample-dist.icf ``` -------------------------------- ### vcf2zarr encode Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Converts intermediate columnar format to VCF Zarr. ```APIDOC ## vcf2zarr encode ### Description Convert intermediate columnar format to VCF Zarr. ### Usage `vcf2zarr encode [OPTIONS] ICF_PATH ZARR_PATH` ### Arguments - **ICF_PATH** (string) - Required - Path to the intermediate columnar format data. - **ZARR_PATH** (string) - Required - Path to the output Zarr directory. ### Options - **-f, --force** - Optional - Force overwriting of existing directories. - **-v, --verbose** - Optional - Increase verbosity. - **-s, --schema** (string) - Optional - Schema definition. - **-l, --variants-chunk-size** (int) - Optional - Chunk size in the variants dimension. - **-w, --samples-chunk-size** (int) - Optional - Chunk size in the samples dimension. - **-V, --max-variant-chunks** (int) - Optional - Truncate the output in the variants dimension. - **-M, --max-memory** (string) - Optional - Approximate bound on overall memory usage (e.g. 10G). - **-P, --progress / -Q, --no-progress** - Optional - Show progress bars. - **-p, --worker-processes** (int) - Optional - Number of worker processes (default: 0). - **--zarr-format** (2|3) - Optional - Zarr format version of output. ``` -------------------------------- ### bio2zarr.tskit.convert Source: https://sgkit-dev.github.io/bio2zarr/_sources/tskit2zarr/python_api.md Converts a tskit tree sequence to VCF Zarr format. This function handles the conversion process and allows for custom model mapping configurations. ```APIDOC ## bio2zarr.tskit.convert(ts_path, vcz_path, *, model_mapping=None, worker_processes=None, **kwargs) ### Description Converts a tskit tree sequence to VCF Zarr format. The function maps the tree sequence data to the VCF Zarr model and writes the output to the specified path. ### Parameters - **ts_path** (str or tskit.TreeSequence) - The path to the input tree sequence or the tree sequence object itself. - **vcz_path** (str) - The path where the output VCF Zarr will be stored. - **model_mapping** (dict, optional) - A mapping configuration generated by tskit.TreeSequence.map_to_vcf_model. If not provided, defaults are used. - **worker_processes** (int, optional) - The number of worker processes to use for parallel conversion. - **kwargs** (dict, optional) - Additional arguments passed to the underlying conversion process. ``` -------------------------------- ### Encode ICF to VCF Zarr Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Converts intermediate columnar format data to VCF Zarr format. ```bash vcf2zarr encode [OPTIONS] ICF_PATH ZARR_PATH ``` -------------------------------- ### Finalize distributed explode Source: https://sgkit-dev.github.io/bio2zarr/vcf2zarr/cli_ref.html Completes the distributed conversion process for the intermediate columnar format. ```bash vcf2zarr dexplode-finalise [OPTIONS] ICF_PATH ``` -------------------------------- ### Remove field from schema Source: https://sgkit-dev.github.io/bio2zarr/_sources/vcf2zarr/tutorial.md Use jq to delete a specific field from the schema definition. ```bash vcf2zarr mkschema sample.icf \ | jq 'del(.fields[] | select(.name == "call_HQ"))' > sample_noHQ.schema.json ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.