Illumina Connected Analytics

https://help.ica.illumina.com/
Admin
Illumina Connected Analytics is a cloud-based software platform designed to manage, analyze, and...
Tokens:218,044
Snippets:2,689
Trust Score:9.9
Update:5 months ago
Show doc for...
Tokens:
Raw
### Start Illumina Service Connector on Linux

Source: https://help.ica.illumina.com/project/p-connectivity/service-connector

Run this command from the connector's installation directory to start the service.

```bash
./illuminaserviceconnector start
```

--------------------------------

### Dynamic Documentation Query Example

Source: https://help.ica.illumina.com/project/p-bench/bench-command-line-interface

Demonstrates how to query documentation dynamically using an HTTP GET request with the `ask` query parameter. Useful for retrieving specific information not explicitly present on the page.

```http
GET https://help.ica.illumina.com/project/p-bench/bench-command-line-interface.md?ask=<question>
```

--------------------------------

### Query Documentation Dynamically

Source: https://help.ica.illumina.com/command-line-interface/cli-configsettings

Example of how to query the documentation dynamically using an HTTP GET request with the 'ask' query parameter.

```http
GET https://help.ica.illumina.com/command-line-interface/cli-configsettings.md?ask=<question>
```

--------------------------------

### Install Illumina Service Connector on Linux

Source: https://help.ica.illumina.com/project/p-connectivity/service-connector

Use this command to install the connector on Linux. Ensure Java 11 or later is installed. A UI will be displayed if an X server is running, otherwise, a command-line installation will proceed. A -c flag forces command-line installation.

```bash
bash illumina_unix_develop.sh
```

```bash
bash illumina_unix_develop.sh -c
```

--------------------------------

### Run Connector Installation Script (Linux)

Source: https://help.ica.illumina.com/project/p-connectivity/service-connector

Execute the connector installation script on Linux. Use 'sudo' if necessary for permissions. For headless systems, use the -c flag for command-line installation.

```bash
bash illumina_unix_develop.sh
```

```bash
sudo bash illumina_unix_develop.sh
```

```bash
bash illumina_unix_develop.sh -c
```

--------------------------------

### Example Configuration JSON

Source: https://help.ica.illumina.com/tutorials/launchpipecli

This is an example of the JSON response structure for configuration settings, showing parameter names, whether they are multi-valued, and their possible values.

```json
{
	"items": [{
		"name": "DRAGEN_Somatic__enable_variant_caller",
		"multiValue": false,
		"values": [
			"true"
		]
	}]
}
```

--------------------------------

### DRAGEN Software Mode Example

Source: https://help.ica.illumina.com/project/p-bench/run-dragen-in-bench-interactive

This example shows how to run DRAGEN in software mode, which is activated using the `--sw-mode` parameter. It includes similar steps to the FPGA mode example: downloading a reference, building a hash table, and running the DRAGEN mapper with software-specific parameters.

```sh
mkdir /data/demo 
cd /data/demo 

# download ref 
wget --progress=dot:giga https://s3.amazonaws.com/stratus-documentation-us-east-1-public/dragen/reference/Homo_sapiens/hg38.fa -O hg38.fa 
# => 0.5min 

# Build ht-ref 
mkdir ref 
dragen --build-hash-table true --ht-reference hg38.fa --output-directory ref 
# => 6.5min 

# run DRAGEN mapper 
FASTQ=/opt/edico/self_test/reads/midsize_chrM.fastq.gz

# Next line is needed to resolve "run the requested pipeline with a pangenome reference, but a linear reference was provided" in DRAGEN (4.4.1 and others). Comment out when encountering ERROR: unrecognised option '--validate-pangenome-reference=false'.
DRAGEN_VERSION_SPECIFIC_PARAMS="--validate-pangenome-reference=false"


```

--------------------------------

### Make Installation Script Executable (Linux)

Source: https://help.ica.illumina.com/project/p-connectivity/service-connector

Before running the connector installation script on Linux, make it executable using chmod +x.

```bash
chmod +x illumina_unix_develop.sh
```

--------------------------------

### DRAGEN FPGA Mode Example

Source: https://help.ica.illumina.com/project/p-bench/run-dragen-in-bench-interactive

This example demonstrates setting up a reference genome and running the DRAGEN mapper in FPGA mode. It includes downloading the reference, building a hash table, and executing the DRAGEN command with specific license and version parameters.

```sh
mkdir /data/demo 
cd /data/demo 

# download ref 
wget --progress=dot:giga https://s3.amazonaws.com/stratus-documentation-us-east-1-public/dragen/reference/Homo_sapiens/hg38.fa -O hg38.fa 
# => 0.5min 

# Build ht-ref 
mkdir ref 
dragen --build-hash-table true --ht-reference hg38.fa --output-directory ref 
# => 6.5min 

# run DRAGEN mapper 
FASTQ=/opt/edico/self_test/reads/midsize_chrM.fastq.gz

# Next line is needed to resolve "run the requested pipeline with a pangenome reference, but a linear reference was provided" in DRAGEN (4.4.1 and others). Comment out when encountering unrecognised option '--validate-pangenome-reference=false'.
DRAGEN_VERSION_SPECIFIC_PARAMS="--validate-pangenome-reference=false" 

# License Parameters
LICENSE_PARAMS="--lic-instance-id-location /opt/dragen-licence/instance-identity.protected --lic-credentials /opt/dragen-licence/instance-identity.protected/dragen-creds.lic"

mkdir out
dragen -r ref --output-directory out --output-file-prefix out -1 $FASTQ --enable-variant-caller false --RGID x --RGSM y ${LICENSE_PARAMS} ${DRAGEN_VERSION_SPECIFIC_PARAMS} 
# => 1.5min (10 sec if fpga already programmed)
```

--------------------------------

### Cursor-based Pagination Example

Source: https://help.ica.illumina.com/reference/r-api

This example demonstrates how to retrieve a list of projects using cursor-based pagination. It shows the initial request and how to use the nextPageToken for subsequent requests.

```APIDOC
## GET /api/projects

### Description
Retrieves a list of projects using cursor-based pagination.

### Method
GET

### Endpoint
`https://ica.illumina.com/ica/rest/api/projects`

### Query Parameters
- **includeHiddenProjects** (boolean) - Optional - Whether to include hidden projects.
- **pageSize** (integer) - Optional - The number of records to return per page.
- **pageToken** (string) - Optional - The cursor to get subsequent results.

### Request Example
```sh
curl -X 'GET' \
  'https://ica.illumina.com/ica/rest/api/projects?includeHiddenProjects=false&pageSize=2' \
  -H 'accept: application/vnd.illumina.v3+json' \
  -H 'X-API-Key: YOUR_API_KEY'
```

### Response
#### Success Response (200)
- **nextPageToken** (string) - A token to retrieve the next page of results.

#### Response Example
```json
{
  "nextPageToken": "A_STRING_OF_LETTERS_AND_NUMBERS"
}
```

## GET /api/projects (with pageToken)

### Description
Retrieves the next page of projects using the nextPageToken obtained from a previous request.

### Method
GET

### Endpoint
`https://ica.illumina.com/ica/rest/api/projects`

### Query Parameters
- **includeHiddenProjects** (boolean) - Optional - Whether to include hidden projects.
- **pageToken** (string) - Required - The cursor to get subsequent results.
- **pageSize** (integer) - Optional - The number of records to return per page.

### Request Example
```sh
curl -X 'GET' \
  'https://ica.illumina.com/ica/rest/api/projects?includeHiddenProjects=false&pageToken=A_STRING_OF_LETTERS_AND_NUMBERS_HERE&pageSize=2' \
  -H 'accept: application/vnd.illumina.v3+json' \
  -H 'X-API-Key: YOUR_API_KEY'
```
```

--------------------------------

### Query Documentation via HTTP GET

Source: https://help.ica.illumina.com/project/p-bench/pipeline-development-in-bench-experimental/creating-a-pipeline-from-scratch

Perform an HTTP GET request to query the documentation dynamically. Include your question as the value for the `ask` query parameter.

```http
GET https://help.ica.illumina.com/project/p-bench/pipeline-development-in-bench-experimental/creating-a-pipeline-from-scratch.md?ask=<question>
```

--------------------------------

### Offset-based Pagination Example

Source: https://help.ica.illumina.com/reference/r-api

This example demonstrates how to retrieve a list of analyses for a specific project using offset-based pagination, including sorting.

```APIDOC
## GET /api/projects/{projectId}/analyses

### Description
Retrieves a list of analyses for a specific project using offset-based pagination and allows for sorting.

### Method
GET

### Endpoint
`https://ica.illumina.com/ica/rest/api/projects/{projectId}/analyses`

### Path Parameters
- **projectId** (string) - Required - The ID of the project.

### Query Parameters
- **pageOffset** (integer) - Optional - The number of rows to skip.
- **pageSize** (integer) - Optional - The number of rows to return per page.
- **sort** (string) - Optional - The sorting criteria (e.g., 'reference desc').

### Response
#### Success Response (200)
- **totalItemCount** (integer) - The total number of records matching the search criteria.

### Request Example
```sh
curl -X 'GET' \
  '<SERVER>/ica/rest/api/projects/<PROJECT_ID>/analyses?pageOffset=2&pageSize=2&sort=reference%20desc' \
  -H 'accept: application/vnd.illumina.v3+json' \
  -H 'X-API-Key: <AUTH_KEY>'
```
```

--------------------------------

### Querying Documentation via HTTP GET

Source: https://help.ica.illumina.com/home/h-toolrepository

Perform an HTTP GET request on the current page URL with the 'ask' query parameter to dynamically query the documentation. The question should be specific and self-contained.

```http
GET https://help.ica.illumina.com/home/h-toolrepository.md?ask=<question>
```

--------------------------------

### Query Documentation via HTTP GET

Source: https://help.ica.illumina.com/project/p-bench/run-dragen-in-bench-interactive

Perform an HTTP GET request to query documentation dynamically using the 'ask' query parameter. This is useful for retrieving specific information or clarifications not explicitly present on the page.

```http
GET https://help.ica.illumina.com/project/p-bench/run-dragen-in-bench-interactive.md?ask=<question>
```

--------------------------------

### Query Documentation via HTTP GET

Source: https://help.ica.illumina.com/reference/r-resources

Perform an HTTP GET request to query the documentation dynamically. Use the 'ask' query parameter with a specific, self-contained question in natural language. The response includes the answer and relevant excerpts.

```HTTP
GET https://help.ica.illumina.com/reference/r-resources.md?ask=<question>
```

--------------------------------

### Query Documentation via HTTP GET

Source: https://help.ica.illumina.com/project/p-flow/f-pipelines/json-based-input-forms/json-scatter-gather-pipeline

Perform an HTTP GET request to the current page URL with the 'ask' query parameter to dynamically query documentation. Use this for clarifications, additional context, or retrieving related sections when information is not explicitly present.

```http
GET https://help.ica.illumina.com/project/p-flow/f-pipelines/json-based-input-forms/json-scatter-gather-pipeline.md?ask=<question>
```

--------------------------------

### Download All Files Matching a Pattern

Source: https://help.ica.illumina.com/command-line-interface/cli-indexcommands

Downloads all files starting with 'VariantCaller-' from a project. This example uses `jq` to extract file IDs and then iterates through them for download. Requires `jq` to be installed.

```bash
icav2 projectdata list --data-type FILE --file-name VariantCaller- --match-mode FUZZY -o json | jq -r '.items[].id' > filelist.txt; for item in $(cat filelist.txt); do echo "--- $item ---"; icav2 projectdata download $item . ; done;
```

--------------------------------

### Create Demo Directory and Input File

Source: https://help.ica.illumina.com/project/p-bench/pipeline-development-in-bench-experimental/creating-a-pipeline-from-scratch

Sets up a directory and a simple text file to be used as input for the pipeline.

```bash
mkdir demo_gzip
cd demo_gzip
echo test > test_input.txt
```

--------------------------------

### Launch Nextflow Pipeline with Parameters

Source: https://help.ica.illumina.com/tutorials/launchpipecli

Example of launching a Nextflow pipeline using the `icav2 projectpipelines start nextflow` command. Note that the input-type parameter is not used, but `--project-id` is required.

```bash
icav2 projectpipelines start nextflow b4dc6b91-5283-41f6-8095-62a5320ed092 \
--user-reference "somatic-3-10-test5" \
--project-id e501a0d5-f5e7-458c-a590-586c79bb87e0 \
--storage-size Small \
--input ref_tar:fil.35e27101fdec404fb37d08d9adf63307 \
--input tumor_fastqs:fil.e1ec77f2647f45804fe508d9aecb19c4,fil.d89018f0c7784fc4b76708d9adf63307 \
--parameters enable_map_align:true \
--parameters enable_map_align_output:true \
--parameters output_format:BAM \
--parameters enable_variant_caller:true \
--parameters vc_emit_ref_confidence:BP_RESOLUTION \
--parameters enable_cnv:false \
--parameters enable_sv:true \
--parameters repeat_genotype_enable:true \
--parameters enable_hla:false \
--parameters enable_variant_annotation:false \
--parameters output_file_prefix:Tumor
```

--------------------------------

### Override CWL Environment Variable with CLI

Source: https://help.ica.illumina.com/project/p-flow/f-pipelines/pi-cwl

Use the `icav2 projectpipelines start` command with a JSON input to override workflow requirements, such as an environment variable, at load time. This example changes the `MESSAGE` environment variable for `tool-fqTOfa.cwl`.

```bash
icav2 projectpipelines start cwl cli-tutorial --data-id fil.a725a68301ee4e6ad28908da12510c25 --input-json '{ 
  "ipFQ": { 
    "class": "File",
    "path": "test.fastq"
  },
  "cwltool:overrides": {
  "tool-fqTOfa.cwl": {
    "requirements": {
      "EnvVarRequirement": {
        "envDef": {
          "MESSAGE": "override_value"
          }
        }
       }
      }
    }
}' --type-input JSON --user-reference overrides-example
```

--------------------------------

### ICA API Response for Activation Code

Source: https://help.ica.illumina.com/tutorials/pipeline_chaining_aws

This JSON output is an example response from the ICA API when retrieving activation code details. It contains IDs for activation code details and analysis storage, which are necessary for starting a Nextflow pipeline.

```json
{
            "id": "6375eb43-e865-4d7c-a9e2-2c153c998a5c",
            "allowedSlots": -1,
            "usedSlots": 0,
            "movedSlots": 0,
            "originalSlots": -1,
            "pipelineBundle": {
                "id": "b4f2840c-4f79-44db-9e1c-5e7339a1b507",
                "name": "ICA_Ent-DE_Pipeline_Entitlement",
                "maxNumberOfAllowedSlots": -1,
                "activePipelines": [],
                "canceledPipelines": [],
                "retiredPipelines": [],
                "regions": [
                    {
                    }
                ],
                "analysisStorages": [
                    {},
                    {},
                    {
                        "id": "6e1b6c8f-f913-48b2-9bd0-7fc13eda0fd0",
                        "timeCreated": "2021-11-05T10:28:20Z",
                        "timeModified": "2021-11-05T10:28:20Z",
                        "ownerId": "8ec463f6-1acb-341b-b321-043c39d8716a",
                        "tenantId": "f91bb1a0-c55f-4bce-8014-b2e60c0ec7d3",
                        "tenantName": "ica-cp-admin",
                        "name": "Small",
                        "description": "1.2 TB"
                    }
                ]
            },
            "usages": []
        }
```

--------------------------------

### Get Analyses with Offset-based Pagination and Sorting

Source: https://help.ica.illumina.com/reference/r-api

Retrieve analyses for a specific project using offset-based pagination. This example demonstrates setting `pageOffset` and `pageSize`, and includes sorting results by `reference` in descending order. The `totalItemCount` is returned in the response.

```sh
curl -X 'GET' \
'<SERVER>/ica/rest/api/projects/<PROJECT_ID>/analyses?pageOffset=2&pageSize=2&sort=reference%20desc' \
-H 'accept: application/vnd.illumina.v3+json' \
-H 'X-API-Key: <AUTH_KEY>'
```

--------------------------------

### Define Pipeline Parameters (XML)

Source: https://help.ica.illumina.com/tutorials/cli-cwl

Create an XML parameter file to define inputs and settings for pipeline steps. This example defines a FASTQ input.

```xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<pd:pipeline xmlns:pd="xsd://www.illumina.com/ica/cp/pipelinedefinition" code="" version="1.0">
    <pd:dataInputs>
        <pd:dataInput code="ipFQ" format="FASTQ" type="FILE" required="true" multiValue="false">
            <pd:label>ipFQ</pd:label>
            <pd:description></pd:description>
        </pd:dataInput>
    </pd:dataInputs>
    <pd:steps/>
</pd:pipeline>
```

--------------------------------

### Retrieve Analysis Step Costs

Source: https://help.ica.illumina.com/project/p-base/base-tables/datacatalogue

Use this SQL query to get the costs of individual analysis steps for analyses run in the past week. It selects user, project, reference, status, price, duration, and start time, along with detailed step information like resource lifecycle, preset size, resource type, step duration, step price, status, and step ID.

```sql
SELECT
    USER_NAME as user_name,
    PROJECT_NAME as project,
    SUBSTRING(PIPELINE_ANALYSIS_DATA:reference, 1, 30) as reference,
    PIPELINE_ANALYSIS_DATA:status as status,
    ROUND(PIPELINE_ANALYSIS_DATA:computePrice,2) as price,
    PIPELINE_ANALYSIS_DATA:totalDurationInSeconds as duration,
    PIPELINE_ANALYSIS_DATA:startTime::TIMESTAMP as startAnalysis,
    f.value:bpeResourceLifeCycle::STRING as bpeResourceLifeCycle,
    f.value:bpeResourcePresetSize::STRING as bpeResourcePresetSize,
    f.value:bpeResourceType::STRING as bpeResourceType,
    f.value:durationInSeconds::INT as durationInSeconds,
    f.value:price::FLOAT as priceStep,
    f.value:status::STRING as status,
    f.value:stepId::STRING as stepId
FROM
    ICA_PIPELINE_ANALYSES_VIEW_project,
    LATERAL FLATTEN(input => PIPELINE_ANALYSIS_DATA:steps) f
WHERE
   PIPELINE_ANALYSIS_DATA:startTime > CURRENT_TIMESTAMP() - INTERVAL '1 WEEK'
ORDER BY
   priceStep DESC;
```

--------------------------------

### Build a Container Image

Source: https://help.ica.illumina.com/project/p-bench/containers-in-bench

Build a local container image using a Dockerfile. The example demonstrates creating a build context, copying a file, and then executing the docker build command.

```dockerfile
FROM alpine:latest 
RUN apk add rsync 
COPY myfile /root/myfile 
```

```bash
# Build a Container image locally 
/data $ mkdir /tmp/buildContext 
/data $ touch /tmp/buildContext/myFile 
/data $ docker build -f /tmp/Dockerfile -t myimage:1.0 /tmp/buildContext 
... 
/data $ docker images 
REPOSITORY                TAG         IMAGE ID      CREATED             SIZE 
docker.io/library/alpine  latest      aded1e1a5b37  3 weeks ago         8.13 MB 
localhost/myimage         1.0         06ef92e7544f  About a minute ago  12.1 MB 
```

--------------------------------

### Run Basic Git-Sourced Pipeline

Source: https://help.ica.illumina.com/project/p-flow/f-pipelines/git-sourced-pipelines-experimental/basic-git-sourced-pipeline-example

Use this command to start a Git-sourced pipeline. Replace placeholders with your specific project, pipeline, and file UUIDs. Ensure the storage size is available in your subscription.

```bash
icav2 projectpipelines start nextflowjson <your_pipeline_uuid> --storage-size 3XSmall --user-reference MyDemoGitPipeline --field-data "input":"<your_file_uuid>"
```

--------------------------------

### Query Documentation via HTTP GET

Source: https://help.ica.illumina.com/command-line-interface/cli-authentication

To get information not directly on the page, perform an HTTP GET request to the page URL with the `ask` query parameter. The response will include an answer and relevant documentation excerpts.

```http
GET https://help.ica.illumina.com/command-line-interface/cli-authentication.md?ask=<question>
```

--------------------------------

### icav2 projectpipelines start

Source: https://help.ica.illumina.com/command-line-interface/cli-indexcommands

Starts a Nextflow pipeline for a given pipeline ID or pipeline code from the current project.

```APIDOC
## icav2 projectpipelines start

### Description
This command starts a Nextflow pipeline for a given pipeline id, or for a pipeline code from the current project.

### Method
POST

### Endpoint
/api/v1/projectpipelines/start

### Parameters
#### Query Parameters
- **reference-tag** (stringArray) - Optional - Reference tag. Add flag multiple times for multiple values.
- **storage-size** (string) - Required - Name of the storage size. Can be fetched using the command 'icav2 list'.
- **technical-tag** (stringArray) - Optional - Technical tag. Add flag multiple times for multiple values.
- **user-reference** (string) - Required - User reference
- **user-tag** (stringArray) - Optional - User tag. Add flag multiple times for multiple values.

#### Global Flags
- **-t, --access-token** (string) - JWT used to call rest service
- **-o, --output-format** (string) - output format (default "table")
- **-s, --server-url** (string) - server url to direct commands
- **-k, --x-api-key** (string) - api key used to call rest service

### Request Example
```bash
icav2 projectpipelines start --storage-size my-storage --user-reference my-user-ref \
  --reference-tag tag1 --reference-tag tag2 \
  --technical-tag tech1 \
  --user-tag user1 \
  --access-token <your-access-token>
```

### Response
#### Success Response (200)
- **pipeline_id** (string) - The ID of the started pipeline.
- **status** (string) - The status of the pipeline operation.
```

--------------------------------

### Run Analysis with Parameters

Source: https://help.ica.illumina.com/tutorials/api-introduction

Construct and send a POST request to initiate an analysis. Ensure all placeholder variables like Pipeline_Identifier, Storage_Size, Parameters, and InputFile are correctly populated before execution. This method is used to start a Nextflow analysis.

```python
data = '{"userReference":"api_example","pipelineId":"'+(Pipeline_Identifier)+'","tags":{"technicalTags":[],"userTags":[],"referenceTags":[]},"analysisStorageId":"'+(Storage_Size)+'","analysisInput":{"inputs":[{"parameterCode":"'+(Parameters)+'","dataIds":["'+(InputFile)+'"]}]}}'
print (data)
response = requests.post('https://ica.illumina.com/ica/rest/api/projects/'+(Project_Identifier)+'/analysis:nextflow',headers=Postheaders,data=data,)
print("Post Response status code: ", response.status_code)
```

--------------------------------

### Query Documentation via HTTP GET

Source: https://help.ica.illumina.com/command-line-interface

To get information not directly on the current page, perform an HTTP GET request to the page URL with an 'ask' query parameter. The question should be specific and in natural language. Use this for clarifications or to retrieve related documentation.

```http
GET https://help.ica.illumina.com/command-line-interface.md?ask=<question>
```

--------------------------------

### workspace-ctl compute scale-pool Help

Source: https://help.ica.illumina.com/project/p-bench/bench-command-line-interface

Displays help for the 'scale-pool' subcommand, detailing required parameters for scaling a pool.

```bash
Usage:
  workspace-ctl compute scale-pool [flags]

Flags:
      --cluster-id string       Required. Cluster ID
  -h, --help                    help for scale-pool
      --help-tree               
      --help-verbose            
      --pool-id string          Required. Pool ID
      --pool-member-count int   Required. New pool size

Global Flags:
      --X-API-Key string   
      --base-path string   For example: / (default "/")
      --config string      config file path
      --debug              output debug logs
      --dry-run            do not send the request to server
      --hostname string    hostname of the service (default "api:8080")
      --print-curl         print curl equivalent do not send the request to server
      --scheme string      Choose from: [http] (default "http")
```

--------------------------------

### Set API GET Headers

Source: https://help.ica.illumina.com/project/p-data

Defines the headers required for making GET requests to the API. Ensure API_KEY is defined.

```python
headers = {
            'X-API-Key': API_KEY,
            'accept': 'application/vnd.illumina.v3+json'
            }
```

--------------------------------

### Example Deployment Interaction

Source: https://help.ica.illumina.com/project/p-bench/pipeline-development-in-bench-experimental/updating-an-existing-flow-pipeline

Illustrates the interactive prompts and output during the deployment of a pipeline as an ICA Flow Pipeline, including version selection and file uploads.

```bash
/data/demo $ pipeline-dev deploy-as-flow-pipeline

Generating ICA input specs...
Extracting nf-core test inputs...
Deploying project nf-core/demo
- Currently being developed as: dev-nf-core-demo
- Last version updated in ICA:  dev-nf-core-demo_v3
- Next suggested version:       dev-nf-core-demo_v4

How would you like to deploy?
1. Update dev-nf-core-demo (current version)
2. Create dev-nf-core-demo_v4
3. Enter new name
4. Update dev-nf-core-demo_v3 (latest version updated in ICA)

```

```bash
Sending docs/images/nf-core-demo-subway.svg
Sending docs/images/nf-core-demo_logo_dark.png
Sending docs/images/nf-core-demo_logo_light.png
Sending docs/images/nf-core-demo-subway.png
Sending docs/README. md
Sending docs/output.md

Pipeline successfully deployed
- Id : 26bc5aa5-0218-4e79-8a63-ee92954c6cd9
- URL: https://stage.v2.stratus.illumina.com/ica/projects/1873043/pipelines/26bc5aa5-0218-4e79-8a63-ee92954C6cd9

Suggested actions:
  pipeline-dev run-in-flow
```

--------------------------------

### List Files in a Folder

Source: https://help.ica.illumina.com/command-line-interface/cli-indexcommands

Example command to list all files within a specified folder in a project.

```bash
icav2 projectdata list --project-id <project_id> --parent-folder /SOURCE/
```

--------------------------------

### Import Libraries and Set Up Authentication

Source: https://help.ica.illumina.com/tutorials/api-introduction

Import necessary libraries (requests, json) and set up authentication headers for API requests. Replace placeholder values with your actual API key.

```python
# The requests library will allow you to make HTTP requests.
import requests

# JSON will allow us to format and interpret the output.
import json

# Replace <your_generated_API_key> with your actual generated API key here.
headers = {
    'X-API-Key': '<your_generated_API_key>',
}

# Replace <your_generated_API_key> with your actual generated API key here.
Postheaders = {
    'accept': 'application/vnd.illumina.v4+json',
    'X-API-Key': '<your_generated_API_key>',
    'Content-Type': 'application/vnd.illumina.v4+json',
}
```

--------------------------------

### Fish Shell Completion Setup

Source: https://help.ica.illumina.com/command-line-interface/cli-indexcommands

Set up autocompletion for the fish shell. Load completions for the current session or configure them for all new sessions by saving to the specified file.

```fish
icav2 completion fish | source
```

```fish
icav2 completion fish > ~/.config/fish/completions/icav2.fish
```

--------------------------------

### Fix for Erroneous 500 Error on Get Samples

Source: https://help.ica.illumina.com/reference/software-release-notes/2023

Fixed an issue where the API endpoint GET /api/samples erroneously returned a 500 Internal Server Error.

```APIDOC
## GET /api/samples

### Description
Retrieves a list of samples. This endpoint previously had an issue returning a 500 error, which has now been fixed.

### Method
GET

### Endpoint
/api/samples
```

--------------------------------

### Basic SQL Query Example

Source: https://help.ica.illumina.com/project/p-base/base-query

Use this syntax to select all data from a table. Ensure the table name is correct.

```sql
Select * From table_name
```

--------------------------------

### Get Projects API Error Handling

Source: https://help.ica.illumina.com/reference/software-release-notes/2024

Fixed an issue where the 'Get Projects' API endpoint would return an error when a tenant contained too many projects.

```APIDOC
## GET /projects

### Description
Retrieves a list of projects within a tenant. This issue has been resolved to prevent errors when a large number of projects exist.

### Method
GET

### Endpoint
/projects
```

--------------------------------

### workspace-ctl compute get-cluster-details Help

Source: https://help.ica.illumina.com/project/p-bench/bench-command-line-interface

Shows help information for the 'get-cluster-details' subcommand.

```bash
Usage:
  workspace-ctl compute get-cluster-details [flags]

Flags:
  -h, --help           help for get-cluster-details
      --help-tree      
      --help-verbose

Global Flags:
      --X-API-Key string   
      --base-path string   For example: / (default "/")
      --config string      config file path
      --debug              output debug logs
      --dry-run            do not send the request to server
      --hostname string    hostname of the service (default "api:8080")
      --print-curl         print curl equivalent do not send the request to server
      --scheme string      Choose from: [http] (default "http")
```

--------------------------------

### Query Bench Cluster Documentation

Source: https://help.ica.illumina.com/project/p-bench/bench-workspaces/bench-clusters

Perform an HTTP GET request with the `ask` query parameter to dynamically query documentation. This is useful for retrieving specific information not explicitly present on the page.

```http
GET https://help.ica.illumina.com/project/p-bench/bench-workspaces/bench-clusters.md?ask=<question>
```

--------------------------------

### icav2 projectpipelines start nextflowjson

Source: https://help.ica.illumina.com/command-line-interface/cli-indexcommands

Starts a Nextflow Json pipeline for a given pipeline id or code. Provides detailed flags for field and group configurations.

```APIDOC
## icav2 projectpipelines start nextflowjson

### Description
Starts a Nextflow Json pipeline for a given pipeline id, or for a pipeline code from the current project. See ICA CLI documentation for more information (https://help.ica.illumina.com/).

### Usage
`icav2 projectpipelines start nextflowjson [pipeline id] or [code] [flags]`

### Flags
- `--field stringArray` - Fields. Add flag multiple times for multiple fields. `--field fieldA:value --field multivalueFieldB:value1,value2`
- `--field-data stringArray` - Data fields. Add flag multiple times for multiple fields. `--field-data fieldA:fil.id --field-data multivalueFieldB:fil.id1,fil.id2`
- `--group stringArray` - Groups. Add flag multiple times for multiple fields in the group. `--group groupA.index1.multivalueFieldA:value1,value2 --group groupA.index1.fieldB:value --group groupB.index1.fieldA:value --group groupB.index2.fieldA:value`
- `--group-data stringArray` - Data groups. Add flag multiple times for multiple fields in the group. `--group-data groupA.index1.multivalueFieldA:fil.id1,fil.id2 --group-data groupA.index1.fieldB:fil.id --group-data groupB.index1.fieldA:fil.id --group-data groupB.index2.fieldA:fil.id`
- `-h, --help` - help for nextflowjson
- `--idempotency-key string` - Add a maximum 255 character idempotency key to prevent duplicate requests. The response is retained for 7 days so the key must be unique during that timeframe.
- `--output-parent-folder string` - The id of the folder in which the output folder should be created.
- `--project-id string` - project ID to set current project context
- `--reference-tag stringArray` - Reference tag. Add flag multiple times for multiple values.
- `--storage-size string` - (*) Name of the storage size. Can be fetched using the command 'icav2 list'.
- `--technical-tag stringArray` - Technical tag. Add flag multiple times for multiple values.
- `--user-reference string` - (*) User reference
- `--user-tag stringArray` - User tag. Add flag multiple times for multiple values.

### Global Flags
- `-t, --access-token string` - JWT used to call rest service
- `-o, --output-format string` - output format (default "table")
- `-s, --server-url string` - server url to direct commands
- `-k, --x-api-key string` - api key used to call rest service
```

--------------------------------

### icav2 projectpipelines start nextflow

Source: https://help.ica.illumina.com/command-line-interface/cli-indexcommands

Starts a Nextflow pipeline using its ID or code. Supports various flags for specifying data, parameters, storage, and tags.

```APIDOC
## icav2 projectpipelines start nextflow

### Description
Starts a Nextflow pipeline using its ID or code. Supports various flags for specifying data, parameters, storage, and tags.

### Usage
`icav2 projectpipelines start nextflow [pipeline id] or [code] [flags]`

### Flags
- `--data-parameters stringArray` - Enter data-parameters as follows : parameterCode:referenceDataId . Add flag multiple times for multiple values.
- `-h, --help` - help for nextflow
- `--idempotency-key string` - Add a maximum 255 character idempotency key to prevent duplicate requests. The response is retained for 7 days so the key must be unique during that timeframe.
- `--input stringArray` - Enter inputs as follows : parametercode:dataId,dataId{optional-mount-path},dataId,... . Add flag multiple times for multiple values. Mount path is optional and can be absolute and relative and can not contain curly braces and commas.
- `--output-parent-folder string` - The id of the folder in which the output folder should be created.
- `--parameters stringArray` - Enter single-value parameters as code:value. Enter multi-value parameters as code:"'value1','value2','value3'". To add multiple values, add the flag multiple times.
- `--project-id string` - project ID to set current project context
- `--reference-tag stringArray` - Reference tag. Add flag multiple times for multiple values.
- `--storage-size string` - (*) Name of the storage size. Can be fetched using the command 'icav2 list'.
- `--technical-tag stringArray` - Technical tag. Add flag multiple times for multiple values.
- `--user-reference string` - (*) User reference
- `--user-tag stringArray` - User tag. Add flag multiple times for multiple values.

### Global Flags
- `-t, --access-token string` - JWT used to call rest service
- `-o, --output-format string` - output format (default "table")
- `-s, --server-url string` - server url to direct commands
- `-k, --x-api-key string` - api key used to call rest service
```

--------------------------------

### workspace-ctl compute CLI Help

Source: https://help.ica.illumina.com/project/p-bench/bench-command-line-interface

Displays general help information for the 'workspace-ctl compute' command and lists available subcommands.

```bash
Usage:
  workspace-ctl compute [flags]
  workspace-ctl compute [command]

Available Commands:
  get-cluster-details 
  get-logs            
  get-pools           
  scale-pool          

Flags:
  -h, --help           help for compute
      --help-tree      
      --help-verbose

Global Flags:
      --X-API-Key string   
      --base-path string   For example: / (default "/")
      --config string      config file path
      --debug              output debug logs
      --dry-run            do not send the request to server
      --hostname string    hostname of the service (default "api:8080")
      --print-curl         print curl equivalent do not send the request to server
      --scheme string      Choose from: [http] (default "http")

Use "workspace-ctl compute [command] --help" for more information about a command.
```

--------------------------------

### Query Documentation via API

Source: https://help.ica.illumina.com/command-line-interface/cli-installation

Demonstrates how to perform an HTTP GET request to a documentation URL with an 'ask' query parameter to dynamically query the documentation.

```http
GET https://help.ica.illumina.com/command-line-interface/cli-installation.md?ask=<question>
```

--------------------------------

### icav2 projectpipelines start cwl

Source: https://help.ica.illumina.com/command-line-interface/cli-indexcommands

Starts a CWL pipeline by providing a pipeline ID or code from the current project. Supports various input and parameter configurations.

```APIDOC
## icav2 projectpipelines start cwl

### Description
Starts a CWL pipeline for a given pipeline id, or for a pipeline code from the current project.

### Method
POST

### Endpoint
/api/v1/projects/{project_id}/pipelines/start/cwl

### Parameters
#### Path Parameters
- **project_id** (string) - Required - The ID of the project.

#### Query Parameters
- **pipeline_id** (string) - Optional - The ID of the pipeline to start.
- **code** (string) - Optional - The code of the pipeline to start.

#### Request Body
- **data_id** (stringArray) - Optional - Enter data id's as follows : dataId{optional-mount-path}. Add flag multiple times for multiple values. Mount path is optional and can be absolute and relative and can not contain curly braces.
- **data_parameters** (stringArray) - Optional - Enter data-parameters as follows : parameterCode:referenceDataId. Add flag multiple times for multiple values.
- **idempotency_key** (string) - Optional - Add a maximum 255 character idempotency key to prevent duplicate requests. The response is retained for 7 days so the key must be unique during that timeframe.
- **input** (stringArray) - Optional - Enter inputs as follows : parametercode:dataId,dataId{optional-mount-path},dataId,.... Add flag multiple times for multiple values. Mount path is optional and can be absolute and relative and can not contain curly braces and commas.
- **input_json** (string) - Optional - Analysis input JSON string. JSON input works only with file-based CWL pipelines (built using code, not a graphical editor in ICA).
- **output_parent_folder** (string) - Optional - The id of the folder in which the output folder should be created.
- **parameters** (stringArray) - Optional - Enter single-value parameters as code:value. Enter multi-value parameters as code:"'value1','value2','value3'". To add multiple values, add the flag multiple times.
- **reference_tag** (stringArray) - Optional - Reference tag. Add flag multiple times for multiple values.
- **storage_size** (string) - Required - Name of the storage size. Can be fetched using the command 'icav2 analysisstorages list'.
- **technical_tag** (stringArray) - Optional - Technical tag. Add flag multiple times for multiple values.
- **type_input** (string) - Required - Input type STRUCTURED or JSON.
- **user_reference** (string) - Required - User reference.
- **user_tag** (stringArray) - Optional - User tag. Add flag multiple times for multiple values.

### Request Example
```json
{
  "data_id": ["data1:mount/path1", "data2"],
  "data_parameters": ["paramCode1:refData1"],
  "idempotency_key": "unique-key-123",
  "input": ["inputCode1:data3,data4:mount/path2"],
  "input_json": "{\"key\": \"value\"}",
  "output_parent_folder": "folderId123",
  "parameters": ["paramCode2:value1", "paramCode3:"'valueA','valueB'""],
  "reference_tag": ["refTag1"],
  "storage_size": "standard",
  "technical_tag": ["techTag1"],
  "type_input": "STRUCTURED",
  "user_reference": "userRef123",
  "user_tag": ["userTag1"]
}
```

### Response
#### Success Response (200)
- **pipeline_run_id** (string) - The ID of the pipeline run.
- **status** (string) - The status of the pipeline run.

#### Response Example
```json
{
  "pipeline_run_id": "prun-abcdef123456",
  "status": "RUNNING"
}
```
```

--------------------------------

### Rclone Configuration Example

Source: https://help.ica.illumina.com/tutorials/datatransfer

Example of a rclone.conf file configuration for connecting to AWS S3 using temporary credentials. This allows rclone to access ICA storage.

```ini
[s3-config]
type = s3
provider = AWS
env_auth = false
access_key_id = XXXXXXXXXX
secret_access_key = XXXXXXX
region = us-east-1
acl = private
session_token = XXXXXXXX
```

--------------------------------

### Select Deployment Option

Source: https://help.ica.illumina.com/project/p-bench/pipeline-development-in-bench-experimental/nf-core-pipelines

When prompted during deployment, choose option '3' to create a new version of the pipeline and provide a unique name to avoid conflicts.

```bash
Choice: 3
Creating ICA Flow pipeline dev-nf-core-demo_v4
Sending inputForm.json
Sending onRender.js
Sending main.nf
Sending nextflow.config
```

--------------------------------

### Query Documentation with GET Request

Source: https://help.ica.illumina.com/home/storage/s-awss3/s-sse-kms

Use this method to ask specific questions about the documentation. Append your question to the URL as the 'ask' parameter.

```http
GET https://help.ica.illumina.com/home/h-storage/s-awss3/s-sse-kms.md?ask=<question>
```

--------------------------------

### New Endpoint: Get Reports for Analysis

Source: https://help.ica.illumina.com/reference/software-release-notes/2025

The endpoint GET /api/projects/{projectId}/analyses/{analysisId}/reports now returns report configs with optional titles and matching data.

```APIDOC
## GET /api/projects/{projectId}/analyses/{analysisId}/reports

### Description
Retrieves report configurations and data for a specific analysis. The returned report configurations can include optional titles and matching data, which can be used in pipeline creation endpoints.

### Method
GET

### Endpoint
/api/projects/{projectId}/analyses/{analysisId}/reports

### Parameters
#### Path Parameters
- **projectId** (string) - Required - The ID of the project.
- **analysisId** (string) - Required - The ID of the analysis.

#### Response
#### Success Response (200)
- **reportConfigs** (array) - An array of report configuration objects. Each object may contain:
  - **title** (string) - Optional - The title of the report configuration.
  - **data** (object) - The matching data for the report configuration.
```

--------------------------------

### Perform HTTP GET Request with 'ask' Parameter

Source: https://help.ica.illumina.com/tutorials/launchpipecli

Use this command to query the documentation dynamically. The 'ask' parameter should contain a specific, natural language question.

```bash
GET https://help.ica.illumina.com/tutorials/launchpipecli.md?ask=<question>
```
Tokens:
Raw