### GDC API Overview Source: https://docs.gdc.cancer.gov/API/Users_Guide/Getting_Started The GDC API drives the GDC Data and Submission Portals, offering programmatic access to search, download, and submit data and metadata. ```APIDOC ## GDC API Overview ### Description The GDC API uses JSON for communication and supports standard HTTP methods (`GET`, `PUT`, `POST`, `DELETE`). It allows users to search for, download, and submit data and metadata programmatically. ### Tools Various third-party tools can be used for interacting with the GDC API, including command-line tools like `curl` and `HTTPie`, and graphical clients like `Postman` and `DHC`. Helper tools for JSON validation and formatting are also available. ### Authentication Authentication tokens are required for downloading controlled-access data and for all data submission operations. Tokens can be obtained from the GDC Data Portal and Submission Portal. ``` -------------------------------- ### API Endpoints Source: https://docs.gdc.cancer.gov/API/Users_Guide/Getting_Started Endpoints for accessing different GDC functionalities, including status, data retrieval, download, and submission. ```APIDOC ## API Endpoints ### Description Each GDC API endpoint represents specific functionality. The base URL for the latest version of an endpoint is `https://api.gdc.cancer.gov/`, and for a specific version, it is `https://api.gdc.cancer.gov//`. ### Endpoints Table | Endpoint | Type | Description | |-----------|------------------|------------------------------------------------------------------| | `status` | Status | Get the API status and version information | | `projects`| Search & Retrieval| Search all data generated by a project | | `cases` | Search & Retrieval| Find all files related to a specific case or sample donor | | `files` | Search & Retrieval| Find all files with specific characteristics | | `annotations` | Search & Retrieval| Search annotations added to data after curation | | `data` | Download | Used to download GDC data | | `manifest`| Download | Generates manifests for use with GDC Data Transfer Tool | | `slicing` | BAM Slicing | Allows remote slicing of BAM format objects | | `submission` | Submission | Returns available resources at the top level above programs | ``` -------------------------------- ### GET /files/{file_id} Source: https://docs.gdc.cancer.gov/API/Users_Guide/Getting_Started Retrieves information about a specific file using its UUID. ```APIDOC ## GET /files/{file_id} ### Description This endpoint retrieves detailed information about a specific file identified by its UUID. It is useful for obtaining metadata associated with a particular data file within the GDC. ### Method GET ### Endpoint `https://api.gdc.cancer.gov/files/` ### Parameters #### Path Parameters - **file_id** (string) - Required - The UUID of the file to retrieve. #### Query Parameters - **pretty** (boolean) - Optional - If set to `true`, the response will be formatted for human readability. ### Request Example (curl) ```bash curl https://api.gdc.cancer.gov/files/cb92f61d-041c-4424-a3e9-891b7545f351?pretty=true ``` ### Response #### Success Response (200) - **file_id** (string) - The UUID of the file. - **file_name** (string) - The name of the file. - **data_format** (string) - The format of the data (e.g., "MAF", "BAM"). - **md5sum** (string) - The MD5 checksum of the file. - ... (other file metadata fields) #### Response Example ```json { "cb92f61d-041c-4424-a3e9-891b7545f351": { "file_id": "cb92f61d-041c-4424-a3e9-891b7545f351", "file_name": "nationwidechildrens.org_OV-High.paired-end.Collapsibles.Level_3.mRNA_sequencing.TCGA-25-1704-01A-01W.18a940a7-6f29-4c1f-8f3a-84774e93367d.gdc_open.maf.txt", "data_format": "MAF", "md5sum": "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6", "state": "live", "size": 12345678, "creation_date": "2023-10-27T10:00:00.000Z" } } ``` ``` -------------------------------- ### GET /projects Source: https://docs.gdc.cancer.gov/API/Users_Guide/Additional_Examples Retrieves a list of projects contained within the GDC. This example demonstrates fetching the first five projects, sorted by project name in ascending order. ```APIDOC ## GET /projects ### Description Retrieves a list of projects contained within the GDC. This example demonstrates fetching the first five projects, sorted by project name in ascending order. ### Method GET ### Endpoint https://api.gdc.cancer.gov/projects ### Query Parameters - **from** (integer) - Optional - The starting index for the results. Defaults to 0. - **size** (integer) - Optional - The number of results to return per page. Defaults to 10. - **sort** (string) - Optional - The field and direction to sort the results by (e.g., 'name:asc'). - **pretty** (boolean) - Optional - If true, returns pretty-printed JSON. Defaults to false. ### Request Example ``` curl 'https://api.gdc.cancer.gov/projects?from=0&size=5&sort=name:asc&pretty=true' ``` ### Response #### Success Response (200) - **data** (object) - Contains the search results and pagination information. - **hits** (array) - An array of project objects. - **id** (string) - The unique identifier for the project. - **primary_site** (array) - A list of primary sites associated with the project. - **dbgap_accession_number** (string or null) - The dbGaP accession number for the project. - **project_id** (string) - The GDC project ID. - **disease_type** (array) - A list of disease types associated with the project. - **name** (string) - The name of the project. - **releasable** (boolean) - Indicates if the project is releasable. - **state** (string) - The current state of the project (e.g., 'open'). - **released** (boolean) - Indicates if the project has been released. - **pagination** (object) - Information about the pagination of the results. - **count** (integer) - The number of results on the current page. - **total** (integer) - The total number of results available. - **size** (integer) - The number of results per page. - **from** (integer) - The starting index for the current page. - **sort** (string) - The sort criteria applied. - **page** (integer) - The current page number. - **pages** (integer) - The total number of pages. - **warnings** (object) - Any warnings associated with the request. #### Response Example ```json { "data": { "hits": [ { "id": "APOLLO-LUAD", "primary_site": [ "Bronchus and lung" ], "dbgap_accession_number": "phs003011", "project_id": "APOLLO-LUAD", "disease_type": [ "Adenomas and Adenocarcinomas" ], "name": "APOLLO1: Proteogenomic characterization of lung adenocarcinoma", "releasable": false, "state": "open", "released": true }, { "id": "TARGET-ALL-P1", "primary_site": [ "Hematopoietic and reticuloendothelial systems" ], "dbgap_accession_number": "phs000463", "project_id": "TARGET-ALL-P1", "disease_type": [ "Lymphoid Leukemias" ], "name": "Acute Lymphoblastic Leukemia - Phase I", "releasable": true, "state": "open", "released": true } ], "pagination": { "count": 5, "total": 86, "size": 5, "from": 0, "sort": "None", "page": 1, "pages": 18 } }, "warnings": {} } ``` ``` -------------------------------- ### GraphQL Query Examples Source: https://docs.gdc.cancer.gov/API/Users_Guide/Submission Illustrative examples of GraphQL queries for common use cases. ```APIDOC ## GraphQL Query Examples ### Example: File UUID #### Description GraphQL query to find the file UUID based on file `submitter_id`. ### Method POST ### Endpoint `https://api.gdc.cancer.gov/v0/submission/graphql` ### Parameters #### Request Body - **query** (string) - Required - The GraphQL query string. ### Request Example ```json { "query": "{\n\n submitted_unaligned_reads (project_id: \"GDC-INTERNAL\", submitter_id: \"Blood-00001-aliquot_lane1_barcode23.fastq\") {\n id\n submitter_id\n file_name\n project_id\n} }", "variables": null } ``` ### Response Example ```json { "data": { "submitted_unaligned_reads": [ { "file_name": "dummy.fastq", "id": "616eab2f-791a-4641-8cd6-ee195a10a201", "project_id": "GDC-INTERNAL", "submitter_id": "Blood-00001-aliquot_lane1_barcode23.fastq" } ] } } ``` ### Example: Case Without Diagnosis #### Description GraphQL query for any one case in 'TCGA-LUAD' without Diagnosis information. ### Method POST ### Endpoint `https://api.gdc.cancer.gov/v0/submission/graphql` ### Parameters #### Request Body - **query** (string) - Required - The GraphQL query string. ### Request Example ```json { "query": "{\n case (project_id: \"TCGA-LUAD\", without_links: [\"diagnoses\"], first: 1) {\n submitter_id\n }\n}", "variables": null } ``` ### Response Example ```json { "data": { "case": [ { "submitter_id": "TCGA-17-Z050" } ] } } ``` ``` -------------------------------- ### Using Authentication Tokens Source: https://docs.gdc.cancer.gov/API/Users_Guide/Getting_Started All API requests that require authentication must include a token as an `X-Auth-Token` custom HTTP header. Below are examples of how to include this header in your requests. ```APIDOC ## Using Authentication Tokens ### Description All API requests that require authentication must include a token as an `X-Auth-Token` custom HTTP header. ### Method All HTTP methods (GET, POST, PUT, DELETE, etc.) ### Endpoint All GDC API endpoints ### Parameters #### Request Headers - **X-Auth-Token** (string) - Required - The authentication token obtained from the GDC. This token grants access to data based on the associated user account's permissions. ### Request Example (cURL) ```bash token=$(cat ) curl -O -J -H "X-Auth-Token: $token" 'https://api.gdc.cancer.gov/data/fd89bfa5-b3a7-4079-bf90-709580c006e5' ``` ### Request Example (Python) ```python import requests import json import re # Replace with the actual path to your token file TOKEN_FILE_PATH = "$TOKEN_FILE_PATH" with open(TOKEN_FILE_PATH, "r") as token_file: token_string = token_file.read().strip() headers = { 'X-Auth-Token': token_string } data_endpoint = 'https://api.gdc.cancer.gov/data/' data_uuid = 'fd89bfa5-b3a7-4079-bf90-709580c006e5' response = requests.get(data_endpoint + data_uuid, headers=headers) # Extract filename from Content-Disposition header response_head_cd = response.headers.get("Content-Disposition") file_name = None if response_head_cd: matches = re.findall(r"filename=(.+)", response_head_cd) if matches: file_name = matches[0] if file_name: with open(file_name, "wb") as output_file: output_file.write(response.content) print(f"File downloaded successfully as {file_name}") else: print("Could not determine filename from headers.") ``` ### Important Notes - Keep your authentication token secure, as it provides access to all data associated with your user account. - For more details on token expiration and rotation, refer to the Data Security section. ``` -------------------------------- ### Create Two Samples (TSV) via GDC API Source: https://docs.gdc.cancer.gov/API/Users_Guide/Submission This example demonstrates how to upload metadata for two samples in TSV format to the GDC using a dry run. It requires a TSV file with sample metadata and a curl command with an authentication token. ```bash type project_id submitter_id cases.submitter_id specimen_type tissue_type tumor_descriptor preservation_method sample GDC-INTERNAL GDC-INTERNAL-000093-sampleA GDC-INTERNAL-000093 Solid Tissue Tumor Primary Frozen sample GDC-INTERNAL GDC-INTERNAL-000093-sampleB GDC-INTERNAL-000093 Solid Tissue Normal Not Reported Frozen ``` ```bash curl --header "X-Auth-Token: $token" --header 'Content-Type: text/tsv' --request PUT --data-binary @Samples.tsv 'https://api.gdc.cancer.gov/submission/GDC/INTERNAL/_dry_run' ``` -------------------------------- ### Create Aliquot and Sample Entities using UUID (JSON) Source: https://docs.gdc.cancer.gov/API/Users_Guide/Submission This snippet demonstrates creating a sample and an aliquot entity, linking the sample to a case using the case's UUID. The aliquot is linked to the sample via its submitter_id. It requires a GDC API token and uses a curl command for submission. ```json [{"type": "sample", "submitter_id": "GDC-INTERNAL-000093-SAMPLE000093", "tissue_type": "Tumor", "preservation_method": "Fresh", "specimen_type": "Whole Bone Marrow", "tumor_descriptor": "Primary", "cases": {"id": "a00f076e-d694-47dd-8e50-24c28e90fd6a"}}, {"type": "aliquot", "submitter_id": "GDC-INTERNAL-000093-SAMPLE000093-ALIQUOT000093", "samples": {"submitter_id": "GDC-INTERNAL-000093-SAMPLE000093"}}] ``` -------------------------------- ### Accessing Developer Documentation Source: https://docs.gdc.cancer.gov/Data_Portal/Users_Guide/Developers_Guide Instructions on how to clone the repository and access the GDC API developer documentation locally. ```APIDOC ## Accessing Developer Documentation ### Description Instructions to clone the `gdc-frontend-framework` repository and access the local API documentation. ### Steps 1. Clone the repository: ```bash git clone git@github.com:NCI-GDC/gdc-frontend-framework.git ``` 2. Navigate into the directory: ```bash cd gdc-frontend-framework/ ``` 3. Checkout the specific branch: ```bash git checkout feat/with_api_docs ``` 4. Open the documentation file in your browser: `docs/api/index.html` ``` -------------------------------- ### Clone and Checkout GDC Frontend Framework for API Docs Source: https://docs.gdc.cancer.gov/Data_Portal/Users_Guide/Developers_Guide Provides terminal commands to clone the `gdc-frontend-framework` repository and checkout a specific branch (`feat/with_api_docs`) to access the GDC API developer documentation. ```bash git clone git@github.com:NCI-GDC/gdc-frontend-framework.git cd gdc-frontend-framework/ git checkout feat/with_api_docs ``` -------------------------------- ### GET /files Endpoint Example Source: https://docs.gdc.cancer.gov/API/Users_Guide/Search_and_Retrieval Example of an HTTP GET request to the 'files' endpoint. This demonstrates how to construct a query with URL-encoded parameters. While functional, GET requests are limited by URL length and POST is preferred for complex or large queries. ```APIDOC ## GET /files ### Description Searches and retrieves file information from the GDC using URL parameters. Suitable for simpler queries where URL length is not a concern. ### Method GET ### Endpoint `https://api.gdc.cancer.gov/files` ### Parameters #### Query Parameters - **filters** (string) - Required - URL-encoded JSON string specifying the search criteria. See POST example for structure. - **format** (string) - Optional - Desired response format (e.g., 'json', 'tsv', 'xml'). - **fields** (string) - Optional - Comma-separated list of data elements to include in the response. - **size** (integer) - Optional - Maximum number of results to return. ### Request Example ``` https://api.gdc.cancer.gov/files?filters=%7B%22op%22%3A%22and%22%2C%22content%22%3A%5B%7B%22op%22%3A%22in%22%2C%22content%22%3A%7B%22field%22%3A%22cases.submitter_id%22%2C%22value%22%3A%5B%22TCGA-CK-4948%22%2C%22TCGA-D1-A17N%22%2C%22TCGA-4V-A9QX%22%2C%22TCGA-4V-A9QM%22%5D%7D%7D%2C%7B%22op%22%3A%22%3D%22%2C%22content%22%3A%7B%22field%22%3A%22files.data_type%22%2C%22value%22%3A%22Gene%20Expression%20Quantification%22%7D%7D%5D%7D&format=tsv&fields=file_id,file_name,cases.submitter_id,cases.case_id,data_category,data_type,cases.samples.tumor_descriptor,cases.samples.tissue_type,cases.samples.sample_type,cases.samples.submitter_id,cases.samples.sample_id,analysis.workflow_type,cases.project.project_id,cases.samples.portions.analytes.aliquots.aliquot_id,cases.samples.portions.analytes.aliquots.submitter_id&size=1000 ``` ### Response #### Success Response (200) Returns data in the format specified by the `format` parameter (e.g., TSV, JSON). #### Response Example (TSV) (Same as POST example, response content is identical for equivalent queries) ``` -------------------------------- ### Retrieve File Information using Curl Source: https://docs.gdc.cancer.gov/API/Users_Guide/Getting_Started This snippet demonstrates how to retrieve information about a specific file using the GDC API's `files` endpoint with a file UUID. It utilizes the `curl` command-line tool and expects a JSON response. ```shell curl https://api.gdc.cancer.gov/files/cb92f61d-041c-4424-a3e9-891b7545f351?pretty=true ``` -------------------------------- ### Download Controlled-Access File with cURL using Auth Token Source: https://docs.gdc.cancer.gov/API/Users_Guide/Getting_Started This snippet demonstrates how to download a controlled-access file from the GDC API using cURL. It reads an authentication token from a file, sets it as an environment variable, and includes it in the request header. The token should be kept secure as it grants access to user-specific data. ```shell token=$(cat ) curl -O -J -H "X-Auth-Token: $token" 'https://api.gdc.cancer.gov/data/fd89bfa5-b3a7-4079-bf90-709580c006e5' ``` -------------------------------- ### Create Aliquot and Sample Entities using submitter_id (JSON) Source: https://docs.gdc.cancer.gov/API/Users_Guide/Submission This snippet demonstrates creating a sample and an aliquot entity in a single transaction, linking them using their respective submitter_ids. It requires a GDC API token and sends a POST request to the GDC submission endpoint. ```json [{"type": "sample", "submitter_id": "GDC-INTERNAL-000093-SAMPLE000093", "tissue_type": "Tumor", "preservation_method": "Fresh", "specimen_type": "Whole Bone Marrow", "tumor_descriptor": "Primary", "cases": {"submitter_id": "GDC-INTERNAL-000093"}}, {"type": "aliquot", "submitter_id": "GDC-INTERNAL-000093-SAMPLE000093-ALIQUOT000093", "samples": {"submitter_id": "GDC-INTERNAL-000093-SAMPLE000093"}}] ``` -------------------------------- ### Retrieve File Information using Python Requests Source: https://docs.gdc.cancer.gov/API/Users_Guide/Getting_Started This snippet shows how to programmatically retrieve file information from the GDC API using the `requests` library in Python. It constructs the request URL using the file endpoint and a specific UUID, then demonstrates two methods for handling the JSON response: saving to a file and printing to the console. ```python import requests import json file_endpt = 'https://api.gdc.cancer.gov/files/' file_uuid = 'cb92f61d-041c-4424-a3e9-891b7545f351' response = requests.get(file_endpt + file_uuid) # OUTPUT METHOD 1: Write to a file. file = open("sample_request.json", "w") file.write(response.text) file.close() # OUTPUT METHOD 2: View on screen. print(json.dumps(response.json(), indent=2)) ``` -------------------------------- ### POST /submission/GDC/INTERNAL - Creating Aliquot and Sample Entities using UUID Source: https://docs.gdc.cancer.gov/API/Users_Guide/Submission This endpoint allows for the creation of sample and aliquot entities, demonstrating how to link them using UUIDs. The example shows a single transaction for creating both entities. ```APIDOC ## POST /submission/GDC/INTERNAL ### Description Creates sample and aliquot entities, linking them using UUIDs for cases. ### Method POST ### Endpoint `https://api.gdc.cancer.gov/submission/GDC/INTERNAL` ### Parameters #### Request Body - **type** (string) - Required - The type of entity to create (e.g., "sample", "aliquot"). - **submitter_id** (string) - Required - A unique identifier for the entity. - **tissue_type** (string) - Optional - The type of tissue. - **preservation_method** (string) - Optional - The method used for preservation. - **specimen_type** (string) - Optional - The type of specimen. - **tumor_descriptor** (string) - Optional - Descriptor for the tumor. - **cases** (object) - Required (for sample) - Contains linking information for cases. - **id** (string) - Required - The UUID of the case to link to. - **samples** (object) - Required (for aliquot) - Contains linking information for samples. - **submitter_id** (string) - Required - The submitter ID of the sample to link to. ### Request Example ```json [ { "type": "sample", "submitter_id": "GDC-INTERNAL-000093-SAMPLE000093", "tissue_type": "Tumor", "preservation_method": "Fresh", "specimen_type": "Whole Bone Marrow", "tumor_descriptor": "Primary", "cases": { "id": "a00f076e-d694-47dd-8e50-24c28e90fd6a" } }, { "type": "aliquot", "submitter_id": "GDC-INTERNAL-000093-SAMPLE000093-ALIQUOT000093", "samples": { "submitter_id": "GDC-INTERNAL-000093-SAMPLE000093" } } ] ``` ### Request Command Example ```bash token=$(