### Local Development Setup Source: https://github.com/quartzbio/quartzbio-python/blob/main/README.md Set up the quartzbio-python project for local development by cloning the repository and installing it in develop mode. ```bash git clone https://github.com/quartzbio/quartzbio-python.git cd quartzbio-python/ python setup.py develop ``` ```bash pip install -e . ``` -------------------------------- ### Install Libraries Source: https://github.com/quartzbio/quartzbio-python/blob/main/examples/Python_Template.ipynb Installs the quartzbio and plotly libraries. Run this cell before proceeding. ```python !pip install quartzbio !pip install plotly ``` -------------------------------- ### Install QuartzBio from Git Source: https://github.com/quartzbio/quartzbio-python/blob/main/README.md Install the quartzbio package directly from its GitHub repository using pip. ```bash pip install -e git+https://github.com/quartzbio/quartzbio-python.git#egg=quartzbio ``` -------------------------------- ### Install QuartzBio Python Client Source: https://github.com/quartzbio/quartzbio-python/blob/main/README.md Install the quartzbio package using pip. For interactive use, also install IPython and gnureadline. ```bash pip install quartzbio ``` ```bash pip install ipython ``` ```bash pip install gnureadline ``` -------------------------------- ### Install QuartzBio Python Client Source: https://github.com/quartzbio/quartzbio-python/blob/main/CLAUDE.md Clone the repository and install the client in development mode using pip. This command is useful for setting up the project for local development. ```bash git clone https://github.com/quartzbio/quartzbio-python.git cd quartzbio-python/ pip install -e . ``` ```bash python -m venv .venv source .venv/bin/activate # On Windows: .venv\Scripts\activate pip install -e . ``` -------------------------------- ### Install QuartzBio Python Library Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/authentication.md.txt Install or upgrade the QuartzBio Python package using pip. Ensure you are using a compatible Python version. ```bash pip install --upgrade quartzbio ``` -------------------------------- ### Example Output: Commit Object Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/metadata_and_global_beacons.md.txt This is an example of the output structure for a commit object, showing its properties and current status. ```python { 'id': 125, 'datastore_id': 6, 'dataset_id': 1658666726768179211, 'status': 'destroying', 'progress_percent': 0, 'is_deleted': False } ``` -------------------------------- ### Query String Syntax Examples Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/filters.html Examples of Lucene-based query strings for field-specific searches, wildcards, and existence checks. ```text status:active gene:"BRCA1" gene:(TTN) gene:(TTN OR BRCA1) gene_family:(Olfactory AND receptors) _missing_:sample_id _exists_:sample_id BRCA* ``` -------------------------------- ### Creating a New Dataset with Fields Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/creating_and_migrating_datasets.html Example demonstrating how to create a new dataset with predefined fields. ```APIDOC ## POST /api/datasets ### Description Creates a new dataset with specified fields and capacity. ### Method POST ### Endpoint `/api/datasets` ### Parameters #### Request Body - **fields** (list of objects) - Required - A list of dictionaries, where each dictionary defines a dataset field with properties like 'name', 'description', 'data_type', 'is_list', 'is_hidden', and 'ordering'. - **capacity** (string) - Optional - The capacity setting for the dataset (e.g., 'small'). ### Request Example ```json { "fields": [ { "name": "my_string_field", "description": "Just a string", "data_type": "string", "is_list": false, "is_hidden": false, "ordering": 0 }, { "name": "gene_symbol", "description": "HUGO gene symbol", "data_type": "string", "entity_type": "gene" } ], "capacity": "small" } ``` ### Response #### Success Response (200) - **dataset_id** (string) - The unique identifier of the created dataset. - **message** (string) - Confirmation message. #### Response Example ```json { "dataset_id": "some_dataset_id", "message": "Dataset created successfully." } ``` ``` -------------------------------- ### Run Tests with Tox Source: https://github.com/quartzbio/quartzbio-python/blob/main/README.md Install tox and run tests for the project. This is useful for ensuring code quality and compatibility. ```bash pip install tox tox ``` -------------------------------- ### Example Output Dictionary Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs-source/source/metadata_and_global_beacons.md Represents the structure of the returned object from the QuartzBio Python project. ```python {'id': 125, 'datastore_id': 6, 'dataset_id': 1658666726768179211, 'status': 'destroying', 'progress_percent': 0, 'is_deleted': False} ``` -------------------------------- ### Get or Create Dataset Source: https://context7.com/quartzbio/quartzbio-python/llms.txt Retrieves a dataset by its full path, creating it if it does not already exist. ```python # Get or create a dataset (creates if not exists) dataset = Dataset.get_or_create_by_full_path("~/my-project/analysis-results") ``` -------------------------------- ### String Expression Example Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/expressions.md.txt Illustrates a basic string concatenation expression. ```Python # String expression "hello" + " world" # output: "hello world" ``` -------------------------------- ### JSONL File Format Example Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/import_parameters.md.txt Illustrates the structure of a JSON Lines file, where each line is a complete JSON object. ```json {"field": 1} {"field": 2} {"field": 3} ``` -------------------------------- ### Define Dataset Paths Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs-source/source/creating_and_migrating_datasets.md Examples of path formats for identifying datasets within domains, vaults, and folders. ```default :: ``` ```default myDomain:MyVault:/folder/dataset ``` ```default ~/folder/dataset ``` -------------------------------- ### QuartzBio CLI Interactive Shell Source: https://context7.com/quartzbio/quartzbio-python/llms.txt Command to start an interactive shell session with QuartzBio. ```bash quartzbio shell ``` -------------------------------- ### Query a dataset Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/expression_functions.md.txt Syntax and examples for querying datasets with optional filters, fields, and entity constraints. ```text dataset_query(dataset, fields, limit, entities, filters, query) ``` ```text dataset_query("quartzbio:public:/ClinVar/5.1.0-20200720/Variants-GRCh38", fields=["clinical_significance"], query="*cancer*") ``` ```text dataset_query("quartzbio:public:/ClinVar/5.1.0-20200720/Variants-GRCh38", entities=[["variant", "GRCH38-13-32357842-32357842-TA"]]) ``` -------------------------------- ### Configure CLI Entry Point Source: https://github.com/quartzbio/quartzbio-python/blob/main/CLAUDE.md Register the console script in the setup.py file. ```python entry_points={ 'console_scripts': ['quartzbio = quartzbio.cli.main:main'] } ``` -------------------------------- ### Importing QuartzBio Library and Logging In Source: https://github.com/quartzbio/quartzbio-python/blob/main/examples/global_search.ipynb Demonstrates how to import the necessary QuartzBio modules and log in to the EDP environment. ```APIDOC ## Importing QuartzBio library and logging in ### Description Import the `login`, `Filter`, and `GlobalSearch` classes from the `quartzbio` library and authenticate with the EDP. ### Code Example ```python # Importing quartzbio library from quartzbio import login from quartzbio import Filter from quartzbio import GlobalSearch # Logging to EDP login() ``` ``` -------------------------------- ### Example XML Document Structure Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/import_parameters.md.txt A sample XML document used to demonstrate how item_depth affects record enumeration during import. ```xml EDP Docs, 3rd Edition ``` -------------------------------- ### UploadProgressTracker Get Average Speed Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/genindex.html Gets the average upload speed from the tracker. ```APIDOC ## UploadProgressTracker Get Average Speed ### Description Calculates and returns the average upload speed recorded by the tracker. ### Method GET ### Endpoint /quartzbio.resource.object.UploadProgressTracker.get_average_speed ### Parameters None ### Response #### Success Response (200) - **average_speed** (float) - The average upload speed in bytes per second. #### Response Example ```json { "average_speed": 102400.5 } ``` ``` -------------------------------- ### Adding Fields to an Existing Dataset Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/creating_and_migrating_datasets.html Example showing how to add a new field to a dataset that already exists. ```APIDOC ## POST /api/datasets/{dataset_id}/fields ### Description Adds a new field to an existing dataset. ### Method POST ### Endpoint `/api/datasets/{dataset_id}/fields` ### Parameters #### Path Parameters - **dataset_id** (string) - Required - The ID of the dataset to which the field will be added. #### Request Body - **name** (string) - Required - The name of the new field. - **data_type** (string) - Required - The data type of the new field (e.g., 'string'). ### Request Example ```json { "dataset_id": "existing_dataset_id", "name": "my_new_field", "data_type": "string" } ``` ### Response #### Success Response (200) - **field_id** (string) - The unique identifier of the newly created field. - **message** (string) - Confirmation message. #### Response Example ```json { "field_id": "new_field_id", "message": "Field added successfully." } ``` ``` -------------------------------- ### QuartzBio Client Initialization Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/modules.html Demonstrates how to initialize the QuartzBio client to interact with the API. ```APIDOC ## QuartzBio Python Client ### Description The QuartzBio Python client provides an interface to interact with the QuartzBio API for various bioinformatics tasks. ### Usage ```python from quartzbio.client import QuartzBioClient # Initialize the client client = QuartzBioClient(api_key='YOUR_API_KEY') ``` ``` -------------------------------- ### Flattening Algorithm Example Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/exporting_data.md.txt Illustrates how list fields in JSON records are expanded into multiple columns during CSV/XLSX export. ```Python {"a": "a", "b": ["x"]} {"a": "a", "b": ["x", "y"]} {"a": "a", "b": ["x", "y", "z"]} ``` -------------------------------- ### View import help Source: https://github.com/quartzbio/quartzbio-python/blob/main/examples/import/README.md Displays all available command line arguments for the import command. ```bash quartzbio import -h ``` -------------------------------- ### Get the Count of Subjects for an Entity Search Source: https://github.com/quartzbio/quartzbio-python/blob/main/examples/global_search.ipynb To get the number of subjects associated with an entity search, use the .subjects_count() method. This returns an integer count. ```python # Subjects count search.subjects_count() ``` -------------------------------- ### Query Facets Examples Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs-source/source/quartzbio.md Demonstrates how to request facets for specific fields using string arguments or keyword arguments with configuration options. ```python q.facets(‘field_1’, ‘field_2’) ``` ```python q.facets(field_1={‘limit’: 0}, field_2={‘limit’: 10}) ``` -------------------------------- ### Define Dataset Template Fields Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/dataset_templates.md.txt Example structure for defining fields within a dataset template. Each field includes properties like name, title, data type, and optional dependencies or expressions. Note the comma placement in the example. ```Python fields = [{ "name": "reason", "title": "Reason" "description": "The reasons for the significance value", "data_type": "string", "depends_on": [ "reason_list" ], "expression": ", ".join(record.reason_list) if record.reason_list else None", "ordering": 1 },{...}] ``` -------------------------------- ### Querying Datasets with Python Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/querying_datasets_and_files.html Examples demonstrating how to limit results, order by single or multiple columns, and handle ascending or descending sort orders. ```python # Users can set how many records they want to retrieve with the "limit" parameter Dataset.get_by_full_path('quartzbio:Public:/ClinVar/5.2.0-20210110/Variants-GRCH37').query(limit=1000) # Users can order query results using the ordering argument # Order the query results by clinical_significance ascending Dataset.get_by_full_path('quartzbio:Public:/ClinVar/5.2.0-20210110/Variants-GRCH37').query(ordering='clinical_significance') # Order the query results by clinical_significance descending Dataset.get_by_full_path('quartzbio:Public:/ClinVar/5.2.0-20210110/Variants-GRCH37').query(ordering='-clinical_significance') # Query results can be ordered by multiple columns # Order the query results by clinical_significance descending and gene_symbol ascending Dataset.get_by_full_path('quartzbio:Public:/ClinVar/5.2.0-20210110/Variants-GRCH37').query(ordering=['-clinical_significance', 'gene']) ``` -------------------------------- ### GET /beacons Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs-source/source/quartzbio.resource.beacon.md Lists available Beacon resources. ```APIDOC ## GET /beacons ### Description Retrieves a list of Beacon resources. ### Method GET ### Endpoint /beacons ### Response - **id** (string) - ID - **title** (string) - Title - **description** (string) - Description - **vault_object_id** (string) - Object ID ``` -------------------------------- ### Run QuartzBio Tests Source: https://github.com/quartzbio/quartzbio-python/blob/main/CLAUDE.md Execute all tests using setup.py or make commands. Specific test modules can be run using make targets or by directly invoking the unittest module. Ensure API credentials are set as environment variables. ```bash python setup.py test ``` ```bash make check ``` ```bash make test-sample # Runs quartzbio.test.test_sample ``` ```bash make test-dataset # Runs quartzbio.test.test_dataset ``` ```bash python -m unittest quartzbio.test.test_query ``` ```bash pip install tox tox ``` ```bash export QUARTZBIO_API_HOST=https://api.quartzbio.com export QUARTZBIO_ACCESS_TOKEN=your_token_here ``` -------------------------------- ### QuartzBioClient Get Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/genindex.html Retrieves a resource using the QuartzBio client. ```APIDOC ## QuartzBioClient Get ### Description Fetches a specific resource using the QuartzBio client. ### Method GET ### Endpoint /quartzbio.client.QuartzBioClient.get /quartzbio.QuartzBioClient.get ### Parameters #### Query Parameters - **resource_id** (string) - Required - The unique identifier of the resource to retrieve. ### Response #### Success Response (200) - **resource** (object) - The retrieved resource object. #### Response Example ```json { "resource": { "id": "resource_123", "name": "Example Resource" } } ``` ``` -------------------------------- ### Initialize QuartzBio and Plotly Source: https://github.com/quartzbio/quartzbio-python/blob/main/examples/generating_icgc_survival_curves.ipynb Imports necessary modules and authenticates the session with EDP credentials. ```python from quartzbio import login, Dataset, Filter import plotly.plotly as py import plotly.tools as tls from plotly.graph_objs import * # Load local EDP credentials login() ``` -------------------------------- ### GET /v2/datasets/{DATASET_ID} Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/creating_and_migrating_datasets.html Retrieve metadata about a specific dataset. ```APIDOC ## GET /v2/datasets/{DATASET_ID} ### Description Retrieve metadata about a dataset. This request requires an authorized user with permission to view the target dataset. ### Method GET ### Endpoint https:///v2/datasets/{DATASET_ID} ### Parameters #### Path Parameters - **DATASET_ID** (integer) - Required - The unique identifier of the dataset. ### Response #### Success Response (200) - **Dataset** (object) - The requested Dataset resource. ``` -------------------------------- ### Import data via command line Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/importing_data.md.txt Use the quartzbio CLI to import files with options for dataset creation and commit modes. ```bash # Import a file (create the dataset if necessary): quartzbio import --create-dataset --follow ~/test-dataset data.vcf.gz # Import files in upsert mode (create the dataset from a template if necessary): quartzbio import --create-dataset --template-file template.json --commit-mode=upsert --follow ~/test-dataset data.vcf.gz ``` -------------------------------- ### GET /v2/vaults/{ID} Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/vaults_and_objects.md.txt Retrieve the metadata for a specific vault. ```APIDOC ## GET https:///v2/vaults/{ID} ### Description Retrieve a vault's metadata. This request requires an authorized user with "read" permission or higher on the vault. ### Method GET ### Endpoint https:///v2/vaults/{ID} ### Parameters #### Path Parameters - **ID** (string) - Required - The unique identifier of the vault. ### Response #### Success Response (200) - **vault** (object) - The Vault resource. ``` -------------------------------- ### GET /v2/vaults Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/vaults_and_objects.md.txt List all available vaults accessible to the user. ```APIDOC ## GET https:///v2/vaults ### Description List all available vaults. All public vaults are included in this response. If the request is sent by an authenticated user, vaults which the user has "read" permission or higher on are also returned. ### Method GET ### Endpoint https:///v2/vaults ### Response #### Success Response (200) - **vaults** (array) - A list of vaults matching the provided filters. ``` -------------------------------- ### Launch QuartzBio IPython Shell (General) Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs-source/source/quartzbio.cli.ipython.md Launches the QuartzBio shell, a general wrapper for IPython. ```APIDOC ## quartzbio.cli.ipython.launch_ipython_shell(args) ### Description Open the QuartzBio shell (IPython wrapper) ### Method Not applicable (Python function call) ### Endpoint Not applicable (Python function call) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python quartzbio.cli.ipython.launch_ipython_shell(args) ``` ### Response None (Launches an interactive shell) ``` -------------------------------- ### GET /v2/dataset_restore_tasks Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/dataset_versioning.html Lists all available dataset restore tasks. ```APIDOC ## GET /v2/dataset_restore_tasks ### Description Retrieve a list of available dataset restore tasks. ### Method GET ### Endpoint https:///v2/dataset_restore_tasks ### Response #### Success Response (200) - **list** (array) - A list of DatasetRestoreTask resources. ``` -------------------------------- ### Function: main() Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs-source/source/quartzbio.cli.main.md The primary entry point for the QuartzBio CLI application. ```APIDOC ## Function: main() ### Description Main entry point for the QuartzBio CLI application. ``` -------------------------------- ### GET /v2/dataset_templates Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/dataset_templates.md.txt Retrieves a list of available dataset templates. ```APIDOC ## GET /v2/dataset_templates ### Description Retrieve a list of available dataset templates. ### Method GET ### Endpoint https:///v2/dataset_templates ### Parameters #### Query Parameters - **limit** (integer) - Optional - The number of objects to return per page. - **offset** (integer) - Optional - The offset within the list of available objects. ### Response #### Success Response (200) - **list** (array) - A list of resources visible to the requesting user. ``` -------------------------------- ### GET /v2/saved_queries Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs-source/source/querying_datasets_and_files.md List all Saved Queries available to the authenticated user. ```APIDOC ## GET /v2/saved_queries ### Description Retrieves all Saved Queries available to the current user. ### Method GET ### Endpoint https:///v2/saved_queries ### Response #### Success Response (200) - **SavedQueries** (array) - A list of SavedQuery resources. ``` -------------------------------- ### Build Documentation Locally Source: https://github.com/quartzbio/quartzbio-python/blob/main/README.md Build the project documentation locally using Sphinx and Sphinx Autobuild. This allows you to preview the documentation before deploying. ```bash make sphinx-autobuild ``` -------------------------------- ### Running the Streamlit Demo App Source: https://github.com/quartzbio/quartzbio-python/blob/main/quartzbio/contrib/streamlit/README.md Execute the secured Streamlit application using the command line interface. ```bash streamlit run app.py ``` -------------------------------- ### GET /v2/saved_queries/{ID} Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs-source/source/querying_datasets_and_files.md Retrieve a specific Saved Query by its ID. ```APIDOC ## GET /v2/saved_queries/{ID} ### Description Retrieve a Saved Query resource by its unique identifier. ### Method GET ### Endpoint https:///v2/saved_queries/{ID} ### Parameters #### Path Parameters - **ID** (string) - Required - The unique identifier of the saved query. ### Response #### Success Response (200) - **SavedQuery** (object) - The SavedQuery resource. ``` -------------------------------- ### GET /v2/dataset_fields/{ID} Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs-source/source/creating_and_migrating_datasets.md Retrieve a specific dataset field by its ID. ```APIDOC ## GET /v2/dataset_fields/{ID} ### Description Retrieve a dataset field. This request requires an authorized user with read permission on the dataset. ### Method GET ### Endpoint https:///v2/dataset_fields/{ID} ### Parameters #### Path Parameters - **ID** (string) - Required - The unique identifier of the dataset field. ### Response #### Success Response (200) - **DatasetField** (object) - The requested DatasetField resource. ``` -------------------------------- ### Manage QuartzBio Documentation Source: https://github.com/quartzbio/quartzbio-python/blob/main/CLAUDE.md Commands to build, serve, and generate documentation for the QuartzBio project. Use 'make sphinx-autobuild' for local development with live reloading. ```bash make sphinx-autobuild # Build and serve docs locally (http://localhost:8080) ``` ```bash make sphinx-apidoc ``` ```bash make sphinx-build ``` -------------------------------- ### Importing Quartzbio Library Source: https://github.com/quartzbio/quartzbio-python/blob/main/examples/global_search.ipynb Initializes the necessary modules for interacting with the Quartzbio API. ```python # Importing quartzbio library from quartzbio import login from quartzbio import Filter from quartzbio import GlobalSearch ``` -------------------------------- ### GET /v2/vaults Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/vaults_and_objects.html List all available vaults accessible to the authenticated user. ```APIDOC ## GET /v2/vaults ### Description List all available vaults. All public vaults are included in this response. If the request is sent by an authenticated user, vaults which the user has “read” permission or higher on are also returned. ### Method GET ### Endpoint https:///v2/vaults ### Response #### Success Response (200) - **vaults** (array) - A list of vaults matching the provided filters. ``` -------------------------------- ### Get Dataset Import Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs-source/source/import_parameters.md Retrieves details of a specific dataset import. ```APIDOC ## GET /v2/dataset_imports/{ID} ### Description Retrieves the details of a specific dataset import. ### Method GET ### Endpoint `//v2/dataset_imports/{ID}` ### Parameters #### Path Parameters - **ID** (string) - Required - The unique identifier of the dataset import to retrieve. ### Authorization This request requires an authorized user with write permissions on the dataset. ### Response #### Success Response (200) - **details** (object) - Contains the details of the dataset import. The response returns “HTTP 200 OK” when successful. #### Response Example ```json { "message": "HTTP 200 OK" } ``` ``` -------------------------------- ### Build Dash Auth JS Components Source: https://github.com/quartzbio/quartzbio-python/blob/main/quartzbio/contrib/dash/js/README.md Run these commands to build the Dash Auth JavaScript components. Ensure you have Node.js and Yarn installed. ```bash yarn ``` ```bash yarn run build ``` -------------------------------- ### QuartzBioClient Initialization Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs-source/source/quartzbio.client.md Initializes the QuartzBio API client with host and token information. ```APIDOC ## quartzbio.client.QuartzBioClient ### Description A requests-based HTTP client for QuartzBio API resources. ### Parameters - **host** (str) - Optional - The host URL for the API. - **token** (str) - Optional - The authentication token. - **token_type** (Literal['Bearer', 'Token']) - Optional - The type of token, defaults to 'Bearer'. - **include_resources** (bool) - Optional - Whether to include resources, defaults to True. - **retry_all** (bool) - Optional - Whether to retry all requests. ``` -------------------------------- ### Dataset Get By Full Path Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/genindex.html Retrieves a Dataset by its full path. ```APIDOC ## Dataset Get By Full Path ### Description Retrieves a Dataset object using its complete path. ### Method GET ### Endpoint /quartzbio.Dataset.get_by_full_path /quartzbio.Object.get_by_full_path ### Parameters #### Query Parameters - **full_path** (string) - Required - The full path to the dataset. ### Response #### Success Response (200) - **dataset** (Dataset) - The retrieved Dataset object. #### Response Example ```json { "dataset": { "id": "dataset_abc", "name": "MyDataset" } } ``` ``` -------------------------------- ### Create Dataset Template with Fields Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/dataset_templates.html Example of preparing a list of DatasetField objects to create a new template. Includes required and optional attributes for data transformation. ```python fields = [ { 'name': 'sample', "depends_on": ['subject'], "entity_type": "sample", 'description': 'Sample ID from SUBJECT', 'data_type': 'string', 'ordering': 1 , 'expression': "record.subject"},{ 'name': 'study', 'title': 'STUDY', 'description': 'Study Code', 'ordering': 2 , 'expression': "None if value == 'UNASSIGNED' else value", 'data_type': 'string'},{ "data_type": "string", "depends_on": [ "hgvs_c" ], "description": "EDP variant entity, computed from the short variant CDS change", "expression": "entity_ids('variant', record.hgvs_c) if record.hgvs_c else None", "is_transient": True, "name": "variant_cdna_grch38" } ] ``` -------------------------------- ### GET /user Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/expression_functions.html Retrieves the profile information for the currently authenticated user. ```APIDOC ## GET /user ### Description Returns the currently authenticated user's profile information. ### Method GET ### Endpoint /user ### Response #### Success Response (200) - **name** (string) - The user’s full name. - **email** (string) - The user’s email address. ``` -------------------------------- ### Launch QuartzBio IPython Shell (IPython 5+) Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs-source/source/quartzbio.cli.ipython.md Launches the QuartzBio shell using IPython version 5 or newer. ```APIDOC ## quartzbio.cli.ipython.launch_ipython_5_shell(args) ### Description Open the QuartzBio shell (IPython wrapper) with IPython 5+ ### Method Not applicable (Python function call) ### Endpoint Not applicable (Python function call) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python quartzbio.cli.ipython.launch_ipython_5_shell(args) ``` ### Response None (Launches an interactive shell) ``` -------------------------------- ### GET /v2/dataset_commits/{ID} Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/dataset_versioning.html Retrieves metadata for a specific dataset commit. ```APIDOC ## GET /v2/dataset_commits/{ID} ### Description Retrieve metadata about a dataset commit. ### Method GET ### Endpoint https:///v2/dataset_commits/{ID} ### Parameters #### Path Parameters - **ID** (string) - Required - The unique identifier of the dataset commit. ### Response #### Success Response (200) - **DatasetCommit** (object) - The resource containing metadata about the commit. ``` -------------------------------- ### GET /v2/dataset_templates/{ID} Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/dataset_templates.md.txt Retrieves a specific dataset template by its ID. ```APIDOC ## GET /v2/dataset_templates/{ID} ### Description Retrieve a dataset template. ### Method GET ### Endpoint https:///v2/dataset_templates/{ID} ### Parameters #### Path Parameters - **ID** (string) - Required - The unique identifier of the template. ### Response #### Success Response (200) - **DatasetTemplate** (object) - The requested DatasetTemplate resource. ``` -------------------------------- ### quartzbio.cli Package Overview Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/quartzbio.cli.rst.txt Overview of the available submodules within the quartzbio.cli package. ```APIDOC ## quartzbio.cli Package ### Description The quartzbio.cli package provides the command-line interface tools for the QuartzBio platform. It includes modules for handling user authentication, managing credentials, interacting with data, and initializing IPython environments. ### Submodules - **quartzbio.cli.auth**: Handles user authentication processes. - **quartzbio.cli.credentials**: Manages user credentials and configuration. - **quartzbio.cli.data**: Provides commands for data manipulation and retrieval. - **quartzbio.cli.ipython**: Integration for IPython environments. - **quartzbio.cli.ipython_init**: Initialization scripts for IPython. - **quartzbio.cli.main**: The main entry point for the CLI application. - **quartzbio.cli.tutorial**: Contains tutorial and onboarding command logic. ``` -------------------------------- ### Import Data using Command Line Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/importing_data.html Import data files using the quartzbio import command. Supports creating datasets and specifying commit modes. ```bash quartzbio import --create-dataset --follow ~/test-dataset data.vcf.gz ``` ```bash quartzbio import --create-dataset --template-file template.json --commit-mode=upsert --follow ~/test-dataset data.vcf.gz ``` -------------------------------- ### GET /v2/dataset_snapshot_tasks Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/dataset_versioning.md.txt Retrieves a list of all available dataset snapshot tasks. ```APIDOC ## GET /v2/dataset_snapshot_tasks ### Description Retrieve a list of available dataset snapshot tasks. ### Method GET ### Endpoint https:///v2/dataset_snapshot_tasks ### Response #### Success Response (200) - **List** (array) - The response contains a list of DatasetSnapshotTask resources. ``` -------------------------------- ### Upload Files using Command Line Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/importing_data.html Upload local folders to the personal vault using the quartzbio upload command. Only missing files and folders are uploaded. ```bash quartzbio upload ./local_folder ``` -------------------------------- ### GET /v2/dataset_exports/{ID} Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/exporting_data.md.txt Retrieve metadata about a specific dataset export. ```APIDOC ## GET /v2/dataset_exports/{ID} ### Description Retrieve metadata about an export. ### Method GET ### Endpoint https:///v2/dataset_exports/{ID} ### Parameters #### Path Parameters - **ID** (string) - Required - The unique identifier of the export. ### Response #### Success Response (200) - **DatasetExport** (object) - The DatasetExport resource. ``` -------------------------------- ### GET /v2/dataset_imports/{ID} Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/import_parameters.md.txt Retrieves details for a specific dataset import. ```APIDOC ## GET /v2/dataset_imports/{ID} ### Description Retrieves details for a specific dataset import. ### Method GET ### Endpoint https:///v2/dataset_imports/{ID} ### Parameters #### Path Parameters - **ID** (string) - Required - The unique identifier of the dataset import. ### Response #### Success Response (200) - **Status** - HTTP 200 OK ``` -------------------------------- ### Launch QuartzBio IPython Shell (Legacy) Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs-source/source/quartzbio.cli.ipython.md Launches the QuartzBio shell for older IPython versions. ```APIDOC ## quartzbio.cli.ipython.launch_ipython_legacy_shell(args) ### Description Open the QuartzBio shell (IPython wrapper) for older IPython versions ### Method Not applicable (Python function call) ### Endpoint Not applicable (Python function call) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python quartzbio.cli.ipython.launch_ipython_legacy_shell(args) ``` ### Response None (Launches an interactive shell) ``` -------------------------------- ### GET /v2/datasets/{DATASET_ID}/imports Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/import_parameters.html Lists all imports associated with a specific dataset. ```APIDOC ## GET /v2/datasets/{DATASET_ID}/imports ### Description Lists all imports associated with a specific dataset. ### Method GET ### Endpoint https:///v2/datasets/{DATASET_ID}/imports ### Parameters #### Path Parameters - **DATASET_ID** (string) - Required - The unique identifier of the dataset. #### Query Parameters - **limit** (integer) - Optional - The number of objects to return per page. - **offset** (integer) - Optional - The offset within the list of available objects. ### Response #### Success Response (200) - Returns a list of DatasetImport resources. ``` -------------------------------- ### Login to QuartzBio Source: https://github.com/quartzbio/quartzbio-python/blob/main/README.md Log in to your QuartzBio account using the CLI. Ensure you have your credentials ready. ```bash quartzbio login ``` -------------------------------- ### GET /v2/object_copy_tasks/{ID} Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/vaults_and_objects.md.txt Retrieves metadata for a specific object copy task. ```APIDOC ## GET /v2/object_copy_tasks/{ID} ### Description Retrieve metadata about an object copy task. ### Method GET ### Endpoint https:///v2/object_copy_tasks/{ID} ### Parameters #### Path Parameters - **ID** (integer) - Required - The unique identifier of the copy task. ``` -------------------------------- ### GET /v2/object_copy_tasks Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/vaults_and_objects.md.txt Lists all object copy tasks created by the current user. ```APIDOC ## GET /v2/object_copy_tasks ### Description List object copy tasks created by the current user. ### Method GET ### Endpoint https:///v2/object_copy_tasks ``` -------------------------------- ### Create Template and Dataset Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/dataset_templates.md.txt Initialize a template object and create a dataset using the template's field structure. ```Python from quartzbio import Dataset, DatasetTemplate template = DatasetTemplate.create(**template) dataset = Dataset.get_or_create_by_full_path('your dataset path', fields=template.fields) # Dataset will now have the non-transient fields from the template # with desired titles/descriptions and expressions print(dataset.fields()) # But no records print(dataset.documents_count) ``` -------------------------------- ### Test Credentials with `whoami` Command Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/authentication.md.txt Verify your authentication by running the `quartzbio whoami` command in the terminal. A successful output confirms your login status and domain. ```bash $ quartzbio whoami # You are logged-in to the "***" domain as ****@****.com with role member. ``` -------------------------------- ### GET /v2/objects Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/vaults_and_objects.md.txt Retrieves a list of objects based on provided filter criteria. ```APIDOC ## GET /v2/objects ### Description List all available objects that the user has read access to, with optional filtering. ### Method GET ### Endpoint https:///v2/objects ### Parameters #### Query Parameters - **id** (integer) - Optional - The ID of an object. - **vault_id** (integer) - Optional - The ID of the vault that will contain the object. - **vault_name** (string) - Optional - The name of the vault containing objects. - **vault_full_path** (text) - Optional - The full path of the vault containing objects. - **parent_object_id** (integer) - Optional - The ID of the existing folder object to place the new object into. - **filename** (string) - Optional - The filename of the object. - **path** (string) - Optional - The path of the object, including its parent folder. - **object_type** (string) - Optional - The type of the object. Must be one of "file", "folder", or "dataset". - **depth** (integer) - Optional - The depth of the object in the Vault. - **query** (string) - Optional - A string that matches any objects whose path contains that string. - **regex** (regex) - Optional - A regular expression which searches objects for matching paths. ### Response #### Success Response (200) - **List** (array) - The response returns a list of objects matching the provided filters. ``` -------------------------------- ### GET /v2/dataset_restore_tasks/{RESTORE_TASK_ID} Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/dataset_versioning.html Retrieves metadata for a specific dataset restore task. ```APIDOC ## GET /v2/dataset_restore_tasks/{RESTORE_TASK_ID} ### Description Retrieve metadata about a dataset restore task. ### Method GET ### Endpoint https:///v2/dataset_restore_tasks/{RESTORE_TASK_ID} ### Parameters #### Path Parameters - **RESTORE_TASK_ID** (string) - Required - The unique identifier of the restore task. ### Response #### Success Response (200) - **DatasetRestoreTask** (object) - The resource containing metadata about the restore task. ``` -------------------------------- ### Log out and Log in with Credentials Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/authentication.md.txt Clear existing credentials and log in using a personal access token and your account's domain. This command creates or updates the credentials file. ```bash # Clear your existing credentials quartzbio logout # Replace "TOKEN" with the Personal Access Token copied from the EDP web page # Replace "DOMAIN" with your account's subdomain (i.e. your company name) quartzbio login --access-token TOKEN --api-host https://DOMAIN.api.edp.aws.quartz.bio ``` -------------------------------- ### GET /v2/datasets/{DATASET_ID}/template Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/dataset_templates.md.txt Creates a dataset template from an existing dataset. ```APIDOC ## GET /v2/datasets/{DATASET_ID}/template ### Description Create a dataset template from an existing dataset. ### Method GET ### Endpoint https:///v2/datasets/{DATASET_ID}/template ### Parameters #### Path Parameters - **DATASET_ID** (string) - Required - The unique identifier of the source dataset. ### Response #### Success Response (200) - **DatasetTemplate** (object) - An unsaved DatasetTemplate object for the specified dataset. ``` -------------------------------- ### GET /v2/dataset_snapshot_tasks/{SNAPSHOT_TASK_ID} Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/dataset_versioning.md.txt Retrieves metadata for a specific dataset snapshot task. ```APIDOC ## GET /v2/dataset_snapshot_tasks/{SNAPSHOT_TASK_ID} ### Description Retrieve metadata about a dataset snapshot task. ### Method GET ### Endpoint https:///v2/dataset_snapshot_tasks/{SNAPSHOT_TASK_ID} ### Parameters #### Path Parameters - **SNAPSHOT_TASK_ID** (string) - Required - The unique identifier of the snapshot task. ### Response #### Success Response (200) - **DatasetSnapshotTask** (object) - The response contains a DatasetSnapshotTask resource. ``` -------------------------------- ### List Files and Folders Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs-source/source/quartzbio.cli.data.md Lists files and folders at a given QuartzBio remote path. ```APIDOC ## GET /api/data/ls ### Description Given a QuartzBio remote path, lists the files and folders. ### Method GET ### Endpoint /api/data/ls ### Parameters #### Query Parameters - **path** (string) - Required - The QuartzBio remote path. ``` -------------------------------- ### GET /v2/datasets/{DATASET_ID}/exports Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/exporting_data.md.txt List all exports associated with a specific dataset. ```APIDOC ## GET /v2/datasets/{DATASET_ID}/exports ### Description List the exports associated with a dataset. ### Method GET ### Endpoint https:///v2/datasets/{DATASET_ID}/exports ### Parameters #### Path Parameters - **DATASET_ID** (string) - Required - The unique identifier of the dataset. ### Response #### Success Response (200) - **List[DatasetExport]** (array) - A list of DatasetExport resources. ``` -------------------------------- ### GET /v2/dataset_exports/{ID}/download Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/exporting_data.md.txt Download a dataset export file by its ID. ```APIDOC ## GET /v2/dataset_exports/{ID}/download ### Description Download a dataset export by its ID. Requires an authorized user with read permissions on the dataset. Note that exports may expire after 24 hours. ### Method GET ### Endpoint https:///v2/dataset_exports/{ID}/download ### Parameters #### Path Parameters - **ID** (string) - Required - The unique identifier of the dataset export. #### Query Parameters - **redirect** (boolean) - Optional - Return a 302 redirect to the download location (default: true). ### Response #### Success Response (302/200) - **Response** (object) - Returns a 302 redirect to the download URL, or a URL to the file if redirect mode is disabled. ``` -------------------------------- ### GET /v2/object_copy_tasks/{ID} Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/vaults_and_objects.html Retrieve metadata for a specific object copy task by its ID. ```APIDOC ## GET /v2/object_copy_tasks/{ID} ### Description Retrieve metadata about an object copy task. This request requires that the authorized user is also the user who created the object copy task being retrieved. ### Method GET ### Endpoint https:///v2/object_copy_tasks/{ID} ### Parameters #### Path Parameters - **ID** (string) - Required - The unique identifier of the object copy task. ### Response #### Success Response (200) - **task** (object) - An object copy task resource. ``` -------------------------------- ### Enable Global Beacon Indexing for a Dataset Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/_sources/metadata_and_global_beacons.md.txt Demonstrates how to enable Global Beacon indexing for a dataset using the Python client. This process allows the dataset to be searched across the organization via specialized endpoints. ```Python # Getting the dataset dataset = Dataset.get_by_full_path('~/beacon-test-dataset') # Enabling Global Beacon on dataset dataset.enable_global_beacon() # Example Output: {'id': 125, 'datastore_id': 6, 'dataset_id': 1658666726768179211, 'status': 'indexing', 'progress_percent': 0, 'is_deleted': False} ``` -------------------------------- ### GET /v2/dataset_migrations/{ID} Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/creating_and_migrating_datasets.html Retrieves metadata about a specific dataset migration. Requires permission. ```APIDOC ## GET /v2/dataset_migrations/{ID} ### Description Retrieve metadata about a dataset migration. This request requires an authorized user with permission. The response contains a DatasetMigration resource. ### Method GET ### Endpoint https:///v2/dataset_migrations/{ID} ### Parameters #### Path Parameters - **ID** (integer) - Required - The ID of the dataset migration to retrieve. ``` -------------------------------- ### Launch IPython Shell Source: https://github.com/quartzbio/quartzbio-python/blob/main/docs/quartzbio.cli.ipython.html Functions to open the QuartzBio shell wrapper for different IPython versions. ```APIDOC ## launch_ipython_shell(args) ### Description Opens the standard QuartzBio shell (IPython wrapper). ### Parameters - **args** (object) - Required - Arguments to pass to the shell initialization. --- ## launch_ipython_5_shell(args) ### Description Opens the QuartzBio shell (IPython wrapper) specifically for IPython 5+. ### Parameters - **args** (object) - Required - Arguments to pass to the shell initialization. --- ## launch_ipython_legacy_shell(args) ### Description Opens the QuartzBio shell (IPython wrapper) for older IPython versions. ### Parameters - **args** (object) - Required - Arguments to pass to the shell initialization. ```