### Example Environment Configuration Source: https://docs.isabl.io/production-deployment This snippet provides an example of environment variable configuration, specifically for an admin user. This is typically used in production setups. ```javascript { "ADMIN_USER": "isablbot", ``` -------------------------------- ### Install Isabl CLI Source: https://docs.isabl.io/quick-start Install the CLI and apps, and configure environment variables. ```bash # install isabl-cli from github (Use a virtualenv as a good practice) pip install -e ./isabl_cli pip install -e ./isabl_apps # point to your API URL export ISABL_API_URL=http://localhost:8000/api/v1/ # set environment variable for demo client export ISABL_CLIENT_ID="demo-cli-client" ``` -------------------------------- ### Getting Instances from Database Source: https://docs.isabl.io/retrieve-data To get started, retrieve specific instances from the database. ```python To get started, we can retrieve specific instances from the database: ``` -------------------------------- ### Install Isabl CLI Source: https://docs.isabl.io/quick-start Instructions for installing the Isabl CLI from GitHub, ideally within a virtual environment. ```bash # install isabl-cli from github (Use a virtualenv as a good practice) ``` -------------------------------- ### Install Cookiecutter and Bootstrap Project Source: https://docs.isabl.io/writing-applications First, ensure cookiecutter is installed. Then, use it to bootstrap your project from the provided template. Finally, navigate to your project directory and install dependencies. ```bash # first make sure you have cookiecutter installed pip install cookiecutter # now lets bootstrap your project cookiecutter https://github.com/isabl-io/cookiecutter-apps # finally install you project in a new virtual environment cd && pip install -r requirements.txt ``` -------------------------------- ### Run Docker Hello World Source: https://docs.isabl.io/quick-start Verifies that Docker is installed and running correctly on your system. ```bash docker run hello-world ``` -------------------------------- ### Install and Configure Isabl CLI Source: https://docs.isabl.io/llms-full.txt Install the isabl-cli and isabl-apps from local paths. Set the ISABL_API_URL and ISABL_CLIENT_ID environment variables to configure the CLI to connect to your API. ```bash pip install -e ./isabl_cli pip install -e ./isabl_apps ``` ```bash export ISABL_API_URL=http://localhost:8000/api/v1/ ``` ```bash export ISABL_CLIENT_ID="demo-cli-client" ``` -------------------------------- ### Run Application Instance Source: https://docs.isabl.io/writing-applications Example of executing a HelloWorldApp instance with custom run arguments. ```python HelloWorldApp().run( tuples=[([target_experiment], [])], run_args=dict(message="custom message"), commit=True, ) ``` -------------------------------- ### Install Isabl CLI from GitHub Source: https://docs.isabl.io/production-deployment Use this command to install the isabl-cli package directly from a GitHub repository. Ensure you have pip installed. ```bash # install isabl-cli from Github pip install git+https://github.com/papaemmelab/isabl_cli#eggs=isabl-cli ``` -------------------------------- ### Run Production Services Source: https://docs.isabl.io/production-deployment Start the production services using the specified configuration file. ```bash docker-compose -f production.yml up ``` ```bash docker-compose -f production.yml up -d ``` -------------------------------- ### Start Demo Containers Source: https://docs.isabl.io/quick-start Launch the demo containers in detached mode. ```bash demo-compose up -d ``` -------------------------------- ### Install isabl_io_llms Package Source: https://docs.isabl.io/production-deployment Use this command to install the isabl_io_llms package. Ensure you have pip installed. ```bash pip install -e /isabl_cli ``` -------------------------------- ### Check Isabl CLI Help Source: https://docs.isabl.io/llms-full.txt Verify the installation and view available commands for the Isabl CLI. ```bash isabl --help ``` -------------------------------- ### Install Isabl CLI from GitHub Source: https://docs.isabl.io/production-deployment Use this command to install the Isabl CLI directly from its GitHub repository. This is useful for ensuring you have the latest version or for development purposes. ```bash # install isabl-cli from Github pip install git+https://github.com/papaemmelab/isabl_cli#eggs=isabl-cli ``` ```bash # or clone locally and install as editable git clone https://github.com/papaemmelab/isabl_cli /isabl_cli pip install -e /isabl_cli ``` -------------------------------- ### Setting Name Configuration Source: https://docs.isabl.io/import-data Example of how a setting name might be configured within the system. ```json { "object": "block", "type": "code-line", "isVoid": false, "data": {}, "nodes": [ { "object": "text", "leaves": [ { "object": "leaf", "text": "Setting Name", "marks": [] } ], "key": "0" } ] } ``` -------------------------------- ### Install ISABL CLI and Apps Source: https://docs.isabl.io/quick-start Install the ISABL CLI and associated applications using pip. Ensure you are in the correct project directory. ```bash pip install -e ./isabl_cli ``` ```bash pip install -e ./isabl_apps ``` -------------------------------- ### Start Isabl Web Development Server Source: https://docs.isabl.io/llms-full.txt Command to start the React development server for the isabl_web project using Yarn. ```bash yarn serve ``` -------------------------------- ### Install and Configure Isabl CLI Source: https://docs.isabl.io/llms-full.txt Commands to install the Isabl CLI from GitHub and configure the required environment variables for production use. ```bash # install isabl-cli from Github pip install git+https://github.com/papaemmelab/isabl_cli#eggs=isabl-cli # or clone locally and install as editable git clone https://github.com/papaemmelab/isabl_cli /isabl_cli pip install -e /isabl_cli ``` ```bash # let the client know what API should be used export ISABL_API_URL="https://isabl.mskcc.org/api/v1/" # set client id, you can create a new client in the admin site export ISABL_CLIENT_ID="" ``` -------------------------------- ### Start Isabl API Backend Source: https://docs.isabl.io/llms-full.txt Command to start the backend development server for the isabl_api project using docker-compose. ```bash docker-compose up ``` -------------------------------- ### Bash: Install Cookiecutter and Bootstrap Project Source: https://docs.isabl.io/llms-full.txt Steps to install Cookiecutter and use it to bootstrap a new Isabl project. This includes installing the package and setting up requirements. ```bash # first make sure you have cookiecutter installed pip install cookiecutter # now lets bootstrap your project cookiecutter https://github.com/isabl-io/cookiecutter-apps # finally install you project in a new virtual environment cd && pip install -r requirements.txt ``` -------------------------------- ### Get Help on a Method Source: https://docs.isabl.io/retrieve-data Use `?` to get help on a method, for example, `ii.get_instances?`. ```python ipython ``` ```python ? ``` ```python ii.get_instances? ``` -------------------------------- ### Isabl CLI Example Source: https://docs.isabl.io/retrieve-data This snippet demonstrates a basic interaction with the Isabl CLI. Further details on specific commands and their usage would typically be found in the CLI documentation. ```bash # Using Isabl CLI ``` -------------------------------- ### Install Project Dependencies Source: https://docs.isabl.io/writing-applications Install project dependencies using pip from a requirements file. Ensure you are in the project directory. ```bash cd && pip install -r requirements.txt ``` -------------------------------- ### Bootstrap Isabl API Project Source: https://docs.isabl.io/production-deployment Install cookiecutter and initialize the project from the official template. ```bash pip install cookiecutter # then bootstrap the project cookiecutter https://github.com/papaemmelab/cookiecutter-api ``` -------------------------------- ### Bootstrap New Application Project Source: https://docs.isabl.io/writing-applications Installs cookiecutter and bootstraps a new application project from the Isabl template. ```bash # first make sure you have cookiecutter installed pip install cookiecutter # now lets bootstrap your project cookiecutter https://github.com/isabl-io/cookiecutter-apps # finally install you project in a new virtual environment ``` -------------------------------- ### AbstractApplication Usage Examples Source: https://docs.isabl.io/writing-applications The `AbstractApplication` class provides a base for applications. This section shows common usage patterns and examples. ```text AbstractApplication comes with ``` -------------------------------- ### Install isabl_io_llms via pip Source: https://docs.isabl.io/production-deployment Install the isabl_io_llms repository using pip. This is useful for making the library available in your Python environment. ```bash pip install --editable /path/to/my/isabl/apps ``` -------------------------------- ### Install Isabl Web Dependencies Source: https://docs.isabl.io/llms-full.txt Command to install project dependencies for the isabl_web project using Yarn. ```bash yarn install ``` -------------------------------- ### Format Python Projects Source: https://docs.isabl.io/llms-full.txt Install and run the Black formatter for Python-based components. ```bash pip install black && black . ``` -------------------------------- ### Import BED Files Help Command Source: https://docs.isabl.io/llms-full.txt Execute this command to get assistance on importing BED files for sequencing techniques, which will be compressed, indexed, and registered. ```bash isabl import-bedfiles --help ``` -------------------------------- ### GET /isabl --help Source: https://docs.isabl.io/llms-full.txt Displays the help menu for the Isabl command-line tool, listing all available commands and options. ```APIDOC ## GET isabl --help ### Description Displays the help menu for the Isabl command-line tool, listing all available commands and options. ### Method CLI Command ### Endpoint isabl --help ### Parameters #### Options - **--version** (flag) - Optional - Show the version and exit. - **--help** (flag) - Optional - Show this message and exit. ``` -------------------------------- ### Environment Variable Setup and Client Configuration Source: https://docs.isabl.io/isabl-settings Before interacting with the API, ensure you set the necessary environment variables. Subsequently, configure your client object with the provided settings, referencing the default configurations for further details. ```APIDOC ## Environment Variable Setup and Client Configuration ### Description Before anything, you must set the following environment variables. Once this is set, you can configure your client object settings with the following configurations (see to check default settings). ### Method N/A (Configuration Steps) ### Endpoint N/A (Configuration Steps) ### Parameters N/A (Configuration Steps) ### Request Example N/A (Configuration Steps) ### Response N/A (Configuration Steps) ## API Endpoint Details (Example Table Structure) ### Description This section outlines the structure of API data, potentially representing different endpoints or data types. The table below provides a sample of available data fields. ### Method N/A (Table Structure) ### Endpoint N/A (Table Structure) ### Parameters N/A (Table Structure) ### Request Example N/A (Table Structure) ### Response #### Success Response (200) - **Field Name** (Type) - Description #### Response Example N/A (Table Structure) | Header 1 | Header 2 | Header 3 | |---|---|---| | Value 1a | Value 2a | Value 3a | | Value 1b | Value 2b | Value 3b | | Value 1c | Value 2c | Value 3c | ``` -------------------------------- ### Configure TLS with Certificate and Key Paths Source: https://docs.isabl.io/production-deployment Configure TLS by providing the paths to your certificate and key files. This is a common setup for production environments. ```bash tls /path/to/cert path/to/key ``` -------------------------------- ### Create Superuser for Isabl Source: https://docs.isabl.io/llms-full.txt Create a superuser account for the Isabl application using the provided example credentials. This is necessary for logging into the web interface. ```bash demo-django createsuperuser ``` -------------------------------- ### Define a new application class Source: https://docs.isabl.io/writing-applications This example demonstrates how to define a new application class by inheriting from AbstractApplication. It includes a placeholder for application-specific logic. ```python class MyApp(AbstractApplication): def __init__(self): super().__init__() ``` -------------------------------- ### Build and Run Production Stack Source: https://docs.isabl.io/llms-full.txt Commands to build, start, and manage the lifecycle of the production Docker containers. ```bash docker-compose -f production.yml build ``` ```bash docker-compose -f production.yml up ``` ```bash docker-compose -f production.yml up -d ``` ```bash docker-compose -f production.yml run --rm django python manage.py migrate ``` ```bash docker-compose -f production.yml run --rm django python manage.py createsuperuser ``` ```bash docker-compose -f production.yml run --rm django python manage.py shell ``` ```bash docker-compose -f production.yml logs ``` ```bash docker-compose -f production.yml scale django=4 docker-compose -f production.yml scale celeryworker=2 ``` ```bash docker-compose -f production.yml ps ``` -------------------------------- ### Get Count Function Example Source: https://docs.isabl.io/retrieve-data Demonstrates the usage of the 'get-count' function, likely for retrieving the number of items or results. ```javascript self.__next_f.push([1,"e5:[\"$\",\"div\",null,{\"aria-labelledby\":\"$undefined\",\"className\":\"blocks w-full space-y-2 lg:space-y-3 leading-normal self-center **:text-left text-left\",\"children\":[[ \"$\",\"p\",\"2199\",{\"className\":\"has-\\[.button,input\\]:flex has-\\[.button,input\\]:flex-wrap has-\\[.button,input\\]:gap-2 has-\\[.button,input\\]:items-center mx-auto page-width-wide:mx-0 decoration-primary/6 print:break-inside-avoid w-full max-w-\\[unset\"],\"children\":[[ \"$\",\"$1\",\"2198\",{\"children\":[[ \"$\",\"$1\",\"0\",{\"children\":[ \"$\",\"strong\",\"mark\",{\"className\":\"font-bold\",\"children\":\"get-count\"}]}]}]}]}]]) ``` -------------------------------- ### Rerun All Failed Signals Source: https://docs.isabl.io/operational-automations Use this command to rerun all signals that have previously failed. No additional setup is required beyond having isabl installed. ```bash # rerun all failed signals isabl rerun-signals ``` -------------------------------- ### Initialize and Configure HelloWorldApp Source: https://docs.isabl.io/writing-applications Instantiates the application and defines default settings for message and path configuration. ```python # create files if you may test with real data at some point app = HelloWorldApp() app.application.settings.default_client = { "default_message": "Hello from Elephant Island.", "echo_path": tmpdir.docker("ubuntu", "echo") } ``` -------------------------------- ### Install Isabl CLI Dependencies Source: https://docs.isabl.io/llms-full.txt Command to install project dependencies for isabl_cli using pip. It is recommended to perform this installation within a virtual environment. ```bash pip install -r requirements.txt ``` -------------------------------- ### Using Isabl SDK Source: https://docs.isabl.io/data-model Demonstrates how to import the Isabl CLI module and interact with instances. ```python import isabl_cli as ii center = ii.get_instance( 'centers', 'MEMORIAL SLOAN KETTERING (MSK)' ) individual = ii.create_instance( ``` -------------------------------- ### Isabl CLI Settings Configuration Source: https://docs.isabl.io/llms-full.txt Example configuration for the isabl CLI, specifying submission methods for analyses and merge operations, along with custom requirements and throttling settings. ```json // isabl cli's settings { ... "SUBMIT_ANALYSES": "isabl_cli.batch_systems.submit_slurm", "SUBMIT_CONFIGURATION": { "extra_args": " -p GPU_PARTITION ", "get_requirements": "my_isabl_extension.batch_systems.get_slurm_requirements", "throttle_by": 50, "unbuffer": True, }, "SUBMIT_MERGE_ANALYSIS": "my_isabl_extension.batch_systems.submit_merge_analysis_to_slurm", ... } ``` -------------------------------- ### Install Yarn Package Manager Source: https://docs.isabl.io/llms-full.txt Command to install Yarn, a JavaScript package manager, using Homebrew. This command installs Yarn without Node.js, assuming Node.js is managed separately. ```bash brew install yarn --without-node ``` -------------------------------- ### Isabl Configuration Settings Source: https://docs.isabl.io/production-deployment Example of Isabl's core configuration settings, including user, group, storage, and application integrations. Ensure these match your production environment. ```json { "ADMIN_USER": "isablbot", "DEFAULT_LINUX_GROUP": "isabl", "BASE_STORAGE_DIRECTORY": "/isabl/data", "SUBMIT_ANALYSES": "isabl_cli.batch_systems.submit_lsf", "ON_DATA_IMPORT": ["isabl_apps.signals.signal_data_import"], "CUSTOM_COMMANDS": ["isabl_apps.cli.one_click_genome"], "ON_STATUS_CHANGE": ["isabl_apps.signals.signal_apps_automation"], "INSTALLED_APPLICATIONS": ["isabl_apps.apps.BwaMemGRCh37"] } ``` -------------------------------- ### Get Target Experiment or Tumor Source: https://docs.isabl.io/retrieve-data This code snippet is used to get the target experiment or tumor. ```python # get the target experiment or tumor\n ``` -------------------------------- ### Filter by Starts With Query Source: https://docs.isabl.io/retrieve-data Use these parameters to filter results where a field starts with a specific substring. ```text [i]startswith ``` ```text name__startswith=isab ``` ```text name__istartswith=iSab ``` -------------------------------- ### Build Demo Containers Source: https://docs.isabl.io/quick-start Build the Docker containers for the demo environment. ```bash demo-compose build ``` -------------------------------- ### Define a Hello World Isabl Application Source: https://docs.isabl.io/writing-applications Inherit from AbstractApplication and override get_command to define the execution logic. The cli_options attribute specifies available command-line arguments. ```python from isabl_cli import AbstractApplication from isabl_cli import options class HelloWorldApp(AbstractApplication): NAME = "HELLO WORLD" VERSION = "1.0.0" cli_options = [options.TARGETS] def get_command(self, analysis): experiment = analysis.targets[0] return f"echo {experiment.sample.identifier} {experiment.raw_data} " ``` -------------------------------- ### Navigate to Demo Directory Source: https://docs.isabl.io/quick-start Change the current directory to the cloned ISABL demo repository. ```bash cd isabl_demo ``` -------------------------------- ### Install unbuffer with Miniconda Source: https://docs.isabl.io/llms-full.txt Commands to create a conda environment and install the expect package to provide the unbuffer utility. ```bash conda create -n myenv conda activate myenv conda install -c conda-forge expect ``` -------------------------------- ### Configure Experiment Metadata and Create Instance Source: https://docs.isabl.io/writing-applications Sets up experiment metadata fields and initializes an instance using the Isabl API. ```python meta_data["sample"]["individual"]["species"] = "HUMAN" meta_data["storage_url"] = hello_world_datadir experiment = api.create_instance("experiments", identifier="a", **meta_data) ``` -------------------------------- ### Import Reference Data Help Command Source: https://docs.isabl.io/llms-full.txt Run this command to view detailed help and options for importing arbitrary reference data for assemblies and techniques. ```bash isabl import-reference-data --help ``` -------------------------------- ### Create and Inspect CLI Client Source: https://docs.isabl.io/quick-start Create the client object and inspect the source script. ```bash # create and update the client object demo-cli python3.6 ./assets/metadata/create_cli_client.py # check the file used to create the client cat ./assets/metadata/create_cli_client.py ``` -------------------------------- ### Define CLI Options for Applications Source: https://docs.isabl.io/writing-applications Shows how to define CLI help text and options for an Isabl application. ```python from isabl_cli import options class HelloWorldApp(AbstractApplication): cli_help = "This is the Hello World App - a way to learn Isabl applications." cli_options = [options.TARGETS] ``` -------------------------------- ### Filtering by Name Starts With Source: https://docs.isabl.io/retrieve-data This endpoint allows you to filter results where the 'name' field starts with a specific query string, case-sensitively or case-insensitively. ```APIDOC ## GET /api/search ### Description Filters results where the 'name' field starts with the specified query string. ### Method GET ### Endpoint /api/search ### Query Parameters - **name__startswith** (string) - Required - The string to search for at the beginning of the 'name' field (case-sensitive). - **name__istartswith** (string) - Required - The string to search for at the beginning of the 'name' field (case-insensitive). ### Request Example ```json { "example": "/api/search?name__startswith=isa" } ``` ### Response #### Success Response (200) - **results** (array) - A list of matching records. #### Response Example ```json { "example": "{\"results\": [ { \"id\": 1, \"name\": \"isabelle\" } ]}" } ``` ``` -------------------------------- ### Create Isabl CLI Client Source: https://docs.isabl.io/quick-start Initializes and creates a client for the isabl-cli. ```bash isabl-cli client create ``` -------------------------------- ### Import and Initialize Isabl CLI Source: https://docs.isabl.io/retrieve-data Import the isabl_cli library and initialize the samples object. ```python import isabl_cli as ii samples = ii ``` -------------------------------- ### Source Demo Profile Source: https://docs.isabl.io/llms-full.txt Source the demo profile script to set up environment variables for the demo. ```bash source .demo-profile ``` -------------------------------- ### Retrieve analysis output directories Source: https://docs.isabl.io/llms-full.txt Get the storage paths for analysis instances. ```bash isabl get-outdirs -fi name STRELKA ``` -------------------------------- ### Filter by date parameter Source: https://docs.isabl.io/retrieve-data Example of applying a date filter to a query parameter. ```text created__date=2016-06-04 ``` -------------------------------- ### Initialize Isabl Data Lake Source: https://docs.isabl.io/production-deployment Run this snippet as the admin user to initialize the data lake. Ensure the BASE_STORAGE_URL is correctly configured in your Isabl CLI settings. ```bash python manage.py init_data_lake --base-storage-url $BASE_STORAGE_URL ``` -------------------------------- ### CLI Execution: isabl apps hello-world-1.0.0 Source: https://docs.isabl.io/writing-applications Describes the command line interface for the Hello World application, including required filters and execution flags. ```APIDOC ## CLI Command: isabl apps hello-world-1.0.0 ### Description Executes the Hello World application within the Isabl environment. This command requires specific target filters to identify the experiments to process. ### Usage `isabl apps hello-world-1.0.0 [OPTIONS]` ### Options - **--targets-filters** (TEXT) - Required - API filters for target experiments. - **--commit** (BOOLEAN) - Optional - Submit application analyses. Default: False. - **--force** (BOOLEAN) - Optional - Wipe unfinished analyses and start from scratch. ``` -------------------------------- ### Custom Application Validation Source: https://docs.isabl.io/writing-applications Example of implementing an application class with experiment validation logic. ```python class HelloWorldApp(AbstractApplication): def validate_experiments(self, targets, references): assert len(targets) == 1, "only one target experiment per analysis" assert targets[0].raw_data, "target experiment has no linked raw data" self.validate_dna_only(targets) # multiple validators are readily available ``` -------------------------------- ### Test HelloWorldApp with Sample Data Source: https://docs.isabl.io/llms-full.txt This Python function tests the HelloWorldApp by creating a sample experiment with FASTQ data and verifying the application's run with specific configurations. It uses temporary directories and data directories for testing. ```python def test_hello_world_app(tmpdir, datadir, commit): # path to hello_world test data hello_world_datadir = join(datadir, "hello_world") raw_data = [ dict( file_url=join(hello_world_datadir, "test.txt"), file_data=dict(extra="annotations"), file_type="FASTQ_R1", ), ] # overwrite default configuration for the default client meta_data = factories.ExperimentFactory(raw_data=raw_data) meta_data["sample"]["individual"]["species"] = "HUMAN" meta_data["storage_url"] = hello_world_datadir experiment = api.create_instance("experiments", identifier="a", **meta_data) # create files if you may test with real data at some point app = HelloWorldApp() app.application.settings.default_client = { "default_message": "Hello from Elephant Island.", "echo_path": tmpdir.docker("ubuntu", "echo") } # run application and make sure results are reported utils.assert_run( application=app, tuples=[([experiment], [])], commit=commit, results=["output", "count", "input"], ) ``` -------------------------------- ### Execute HelloWorldApp with custom arguments Source: https://docs.isabl.io/writing-applications Run the HelloWorldApp application class with specific experiment tuples and commit settings. ```python HelloWorldApp().run( tuples=([target_experiment], []), run_args=dict(message="custom message"), commit=True, ) ``` -------------------------------- ### Test unbuffer functionality Source: https://docs.isabl.io/llms-full.txt Verify that the unbuffer utility is correctly installed and functioning by echoing a string. ```bash $ unbuffer echo "Hello World!" Hello World ``` -------------------------------- ### Create Superuser Source: https://docs.isabl.io/quick-start Command to initialize an administrative user account. ```bash demo-django createsuperuser ``` ```bash username=admin password=admin email=admin@demo.io ``` -------------------------------- ### Initialize Isabl SDK Source: https://docs.isabl.io/llms-full.txt Import the Isabl CLI module to begin programmatic interaction. ```python import isabl_cli as ii # ii stands for `interactive isabl` 😎 ``` -------------------------------- ### Get count of database instances Source: https://docs.isabl.io/retrieve-data Use this command to count instances based on specific filters. ```bash isabl get-count analyses -fi status FAILED ``` -------------------------------- ### Import Reference Data Help and Structure Source: https://docs.isabl.io/import-data Command to view help for reference data import and the expected directory structure for assemblies. ```bash isabl import-reference-data --help # extra resources are included in the assembly directory assemblies/ ├── GRCh37 │ ├── chr_alias │ │ └── hg19_alias.tab │ ├── cytoband │ │ └── cytoBand.txt │ ├── genes │ │ └── refGene.txt │ └── genome_fasta ``` -------------------------------- ### Set up analysis and experiment directories Source: https://docs.isabl.io/production-deployment Define the directories for analyses and experiments. This script sets up environment variables for organizing your data lake. ```bash DIRS=\"00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99\" ``` ```bash BASE=\"analyses experiments\" ``` ```bash # go to your data lake base directory (see: BASE_STORAGE_DIRECTORY) ``` ```bash cd /path/to/my/data/lake ``` -------------------------------- ### Get Project Instance Source: https://docs.isabl.io/retrieve-data Retrieve a project instance using the endpoint and identifier. The signature is (endpoint, identifier). ```python project = ii.get_instance("projects", 100) # the signature is (endpoint, identifier) ``` -------------------------------- ### Initialize Data Lake Directories Source: https://docs.isabl.io/llms-full.txt Run this bash script as the admin user to create a hierarchical directory structure for the data lake. It sets permissions for read/write/execute for the owner and group. ```bash DIRS="00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99" BASE="analyses experiments" # go to your data lake base directory (see: BASE_STORAGE_DIRECTORY) cd /path/to/my/data/lake for i in $BASE; do for j in $DIRS; do for k in $DIRS; do DIR="$i/$j/$k" mkdir -p $DIR chmod u+wrX,g+wrX $DIR done; chmod g-w "$i/$j/" done; done ``` -------------------------------- ### Get Paths for Any Instance Source: https://docs.isabl.io/quick-start This command allows you to retrieve paths for any registered instance in the database using its ID. ```bash isabl get-paths --instance-id ``` -------------------------------- ### CLI Help and Restart Options Source: https://docs.isabl.io/writing-applications Standard command-line interface flags for managing analysis processes. ```bash --restart Attempt restarting failed analyses from previous checkpoint. --help Show this message and exit. ``` -------------------------------- ### GET /api/v1/ Source: https://docs.isabl.io/data-model Access the comprehensive RESTful API reference and interactive Swagger documentation for all database resources. ```APIDOC ## GET /api/v1/ ### Description Access the interactive Swagger API documentation to explore all available endpoints for each resource in the Isabl database. ### Method GET ### Endpoint http:///api/v1/ ### Response #### Success Response (200) - **HTML/Swagger UI** - Interactive documentation interface for the API. ``` -------------------------------- ### Create Experiments and Test Applications Source: https://docs.isabl.io/writing-applications Utilities for creating database instances via factories and running application tests with mock data. ```python from isabl_cli import api from isabl_cli import factories meta_data = factories.ExperimentFactory() meta_data["sample"]["individual"]["species"] = "HUMAN" experiment = api.create_instance("experiments", **meta_data) assert experiment.pk > 0, "failed to create database instance" ``` ```python def test_hello_world_app(tmpdir, datadir, commit): # path to hello_world test data hello_world_datadir = join(datadir, "hello_world") raw_data = [ dict( file_url=join(hello_world_datadir, "test.txt"), file_data=dict(extra="annotations"), file_type="FASTQ_R1", ), ] # overwrite default configuration for the default client meta_data = factories.ExperimentFactory(raw_data=raw_data) meta_data["sample"]["individual"]["species"] = "HUMAN" meta_data["storage_url"] = hello_world_datadir experiment = api.create_instance("experiments", identifier="a", **meta_data) # create files if you may test with real data at some point app = HelloWorldApp() app.application.settings.default_client = { "default_message": "Hello from Elephant Island.", "echo_path": tmpdir.docker("ubuntu", "echo") } # run application and make sure results are reported utils.assert_run( application=app, tuples=[([experiment], [])], commit=commit, results=["output", "count", "input"], ) ``` -------------------------------- ### Serve Isabl Frontend Externally Source: https://docs.isabl.io/llms-full.txt Example HTML structure for serving the Isabl frontend independently of the API. ```markup My Awesome App
``` -------------------------------- ### View Isabl Application Help Source: https://docs.isabl.io/writing-applications Command to display help information and available options for a specific Isabl application. ```bash $ isabl apps hello-world-1.0.0 --help Usage: isabl apps hello-world-1.0.0 [OPTIONS] This is the Hello World App - a way to learn Isabl applications. Options: --targets-filters API filters for target experiments [required] ... --commit Submit application analyses. [default: False] --force Wipe unfinished analyses and start from scratch. ``` -------------------------------- ### Run tests with py.test Source: https://docs.isabl.io/production-deployment Use this command to execute tests with the py.test framework. Ensure py.test is installed in your environment. ```bash py.test ``` -------------------------------- ### Get Project Analysis Results Source: https://docs.isabl.io/writing-applications Calculates the total number of lines in the 'merged.txt' file associated with a project analysis. ```python def get_project_analysis_results(self, analysis): merged = join(analysis.storage_url, "merged.txt") with open(merged) as f: count = sum(len(i) for i in f) ``` -------------------------------- ### Run Hello World Application via CLI Source: https://docs.isabl.io/writing-applications Executes the hello-world application filtered by project ID. ```bash isabl apps hello-world --filters projects 102 ``` -------------------------------- ### Initialize Global Isabl Configuration Source: https://docs.isabl.io/isabl-settings Sets up the global window.$isabl object with API host, application name, branding, and feature flags. ```javascript window.$isabl = { apiHost: "http://my.isabl.api.host", name: "My Cool App", logo: "/path/to/my/awesome/logo", customFields, restartButton: true, jira: true, oncoTree: true, darkTheme: { primary: '#1dc5a7', ``` -------------------------------- ### Get Projects by Owner Source: https://docs.isabl.io/retrieve-data Retrieves projects where the current user is the owner. Use this to filter projects based on ownership. ```python # get all the projects where I'm the owner projects = ii.get_projects(owner__startswith="besuhof") ``` -------------------------------- ### Get Multiple ISABL Trees Source: https://docs.isabl.io/retrieve-data Retrieve a collection of ISABL 'trees' (individuals and their associated data) by filtering them by project. ```python individuals = ii.get_trees(projects=267) ``` -------------------------------- ### Manage Production Docker Containers Source: https://docs.isabl.io/production-deployment Common Docker Compose commands for building, starting, and monitoring the production environment. ```bash docker-compose -f production.yml build ``` ```bash docker-compose -f production.yml up ``` ```bash docker-compose -f production.yml up -d ``` ```bash docker-compose -f production.yml logs ``` ```bash docker-compose -f production.yml ps ``` -------------------------------- ### Example of get_or_create_analyses override Source: https://docs.isabl.io/writing-applications Override the get_or_create_analyses method if custom logic is needed for determining analysis uniqueness or creation. ```python def get_or_create_analyses(self, experiments, application): return super().get_or_create_analyses(experiments, application) ``` -------------------------------- ### Filter Experiments by BAM File Source: https://docs.isabl.io/llms-full.txt Example of filtering experiments based on the presence of a specific BAM file reference. ```python experiments = ii.experiments(has_bam_for="GRCh37") ``` -------------------------------- ### Get Output Directories with FAILED Status Source: https://docs.isabl.io/retrieve-data Retrieve output directories where the status is FAILED and list their contents up to level 2. ```bash isabl get-outdirs -fi status FAILED | xargs tree -L 2 ``` -------------------------------- ### Initialize Theme Configuration Source: https://docs.isabl.io/master This JavaScript snippet initializes the application theme, handling system preferences and local storage persistence. ```javascript ((a,b,c,d,e,f,g,h)=>{let i=document.documentElement,j=["light","dark"];function k(b){var c;(Array.isArray(a)?a:[a]).forEach(a=>{let c="class"===a,d=c&&f?e.map(a=>f[a]||a):e;c?(i.classList.remove(...d),i.classList.add(f&&f[b]?f[b]:b)):i.setAttribute(a,b)}),c=b,h&&j.includes(c)&&(i.style.colorScheme=c)}if(d)k(d);else try{let a=localStorage.getItem(b)||c,d=g&&"system"===a?window.matchMedia("(prefers-color-scheme: dark)").matches?"dark":"light":a;k(d)}catch(a){}})("class","theme","system",null,["light","dark"],null,true,true) ``` -------------------------------- ### Code Snippet Examples Source: https://docs.isabl.io/retrieve-data These snippets represent serialized UI components containing code-styled elements for data filtering and formatting. ```json created__month=06 ``` ```json year ``` ```json YYYY ``` ```json created__year=2016 ``` ```json time ``` -------------------------------- ### Initialize Isabl Web Configuration Source: https://docs.isabl.io/isabl-settings Basic initialization script for the Isabl web frontend. ```html ``` -------------------------------- ### Update .eggs directory permissions Source: https://docs.isabl.io/production-deployment Ensure correct permissions for the .eggs directory. This is a necessary step after installation if you encounter permission issues. ```bash chmod -R g+rwX /path/to/my/isabl/apps/.eggs ``` -------------------------------- ### Check Platform Architecture Source: https://docs.isabl.io/quick-start Command to verify the current platform architecture. ```bash linux/amd64 ``` -------------------------------- ### Get Output Directories with FAILED Status and Specific Pattern Source: https://docs.isabl.io/retrieve-data Retrieve output directories with a FAILED status that match the pattern 'head_job.*'. ```bash isabl get-outdirs -fi status FAILED --pattern 'head_job.*' ``` -------------------------------- ### Build Production Containers Source: https://docs.isabl.io/production-deployment Build the project services defined in the production configuration file. ```bash docker-compose -f production.yml build ```