### Install cmapPy in Development Mode

Source: https://cmappy.readthedocs.io/en/latest/build.html

Installs cmapPy in development mode after cloning the repository and activating the environment.

```bash
$ python setup.py develop
```

--------------------------------

### Install cmapPy via Pip (Windows)

Source: https://cmappy.readthedocs.io/en/latest/build.html

Installs cmapPy using pip after creating a conda environment on Windows.

```bash
pip install cmapPy
```

--------------------------------

### Create Development Conda Environment

Source: https://cmappy.readthedocs.io/en/latest/build.html

Creates a conda environment for development, excluding cmapPy from the initial installation.

```bash
conda create --name my_cmapPy_dev_env python=2.7.11 numpy=1.11.2 pandas=0.20.3 h5py=2.7.0 requests==2.13.0
```

--------------------------------

### Select Data from Multi-Index DataFrame (Example)

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/GCToo.html

Demonstrates how to select specific data subsets from a multi-index DataFrame using the `xs` method. It is crucial to use `drop_level=False` to retain metadata.

```python
# 1) Select the probe with pr_lua_id="LUA-3404":
            lua3404_df = multi_index_df.xs("LUA-3404", level="pr_lua_id", drop_level=False)
            # 2) Select all DMSO samples:
            DMSO_df = multi_index_df.xs("DMSO", level="pert_iname", axis=1, drop_level=False)
```

--------------------------------

### Test cmapPy Import

Source: https://cmappy.readthedocs.io/en/latest/build.html

Tests the installation of cmapPy by importing the parse_gct module within a Python interpreter.

```python
import cmapPy.pandasGEXpress.parse_gct as pg
```

--------------------------------

### Run Filter Query on CLUE API

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/clue_api_client/clue_api_client.html

Execute a GET request to retrieve a list of resources based on a filter clause. Ensure the resource name and filter are correctly formatted.

```python
results = client.run_filter_query('genes', {'where': {'gene_set_id': '123'}})
```

--------------------------------

### read

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/set_io/grp.html

Reads a GRP file from the specified path. It ignores lines starting with '#'.

```APIDOC
## read

### Description
Reads a grp file at the path specified by in_path.

### Parameters
#### Path Parameters
- **in_path** (string) - Required - path to GRP file

### Returns
- **grp** (list) - The content of the GRP file as a list.
```

--------------------------------

### Main Function for Concatenation Script

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/concat.html

Parses command-line arguments and initiates the concatenation process using the concat_main function. Includes setup for logging verbosity.

```python
def main():
    # get args
    args = build_parser().parse_args(sys.argv[1:])
    setup_logger.setup(verbose=args.verbose)
    logger.debug("args:  {}".format(args))

    concat_main(args)

```

--------------------------------

### run_count_query

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/clue_api_client/clue_api_client.html

Runs a count query (GET) against the CLUE API for a specified resource name and a 'where' clause.

```APIDOC
## run_count_query

### Description
Run a query (get) against CLUE api.

### Method
GET

### Endpoint
/{resource_name}/count

### Parameters
#### Path Parameters
- **resource_name** (str) - Required - Name of the resource/collection to query (e.g., genes, perts, cells).

#### Query Parameters
- **where** (dict) - Required - Contains the 'where' clause to pass to the API, using loopback specification.

### Response
#### Success Response (200)
- **dict** - A dictionary containing the results of the query (e.g., the count).
```

--------------------------------

### Run Count Query on CLUE API

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/clue_api_client/clue_api_client.html

Execute a GET request to count resources matching a specific 'where' clause. This is useful for determining the number of items without retrieving them all.

```python
count_results = client.run_count_query('perts', {'cell_id': 'abc'})
```

--------------------------------

### run_filter_query

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/clue_api_client/clue_api_client.html

Runs a GET query against the CLUE API using a specified resource name and filter clause.

```APIDOC
## run_filter_query

### Description
Run a query (get) against the CLUE api, using the API and user key fields of self and the fitler_clause provided.

### Method
GET

### Endpoint
/{resource_name}

### Parameters
#### Path Parameters
- **resource_name** (str) - Required - Name of the resource/collection to query (e.g., genes, perts, cells).

#### Query Parameters
- **filter** (dict) - Required - Contains the filter to pass to the API, using loopback specification.

### Response
#### Success Response (200)
- **list** (list of dict) - A list of dictionaries containing the results of the query.
```

--------------------------------

### Read GRP File

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/set_io/grp.html

Reads a GRP file from the specified path. Asserts that the file exists before opening. Ignores lines starting with '#'.

```python
import os
import re

def read(in_path):
    """ Read a grp file at the path specified by in_path.

    Args:
        in_path (string): path to GRP file

    Returns:
        grp (list)

    """
    assert os.path.exists(in_path), "The following GRP file can't be found. in_path: {}".format(in_path)

    with open(in_path, "r") as f:
        lines = f.readlines()
        # need the second conditional to ignore comment lines
        grp = [line.strip() for line in lines if line and not re.match("^#", line)]

    return grp
```

--------------------------------

### Get File List for Concatenation

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/concat.html

Searches for files matching a given wildcard pattern. This function is a basic implementation and can be expanded for more sophisticated file searching.

```python
def get_file_list(wildcard):
    """ Search for files to be concatenated. Currently very basic, but could
    expand to be more sophisticated.

    Args:
        wildcard (regular expression string)

    Returns:
        files (list of full file paths)

    """
    files = glob.glob(os.path.expanduser(wildcard))
    return files
```

--------------------------------

### Initialize ClueApiClient

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/clue_api_client/clue_api_client.html

Instantiate the client with an optional base URL and user key for authentication.

```python
client = ClueApiClient(base_url='https://dev-api.clue.io/api/', user_key='YOUR_USER_KEY')
```

--------------------------------

### Main Function for gctx2gct Script

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/gctx2gct.html

Parses command-line arguments, sets up logging, and calls the main conversion function. This is the entry point for the command-line tool.

```python
def main():
    args = build_parser().parse_args(sys.argv[1:])
    setup_logger.setup(verbose=args.verbose)
    gctx2gct_main(args)
```

--------------------------------

### Entry Point for Script Execution

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/gct2gctx.html

Standard Python construct to call the `main` function when the script is executed directly.

```python
if __name__ == "__main__":
    main()
```

--------------------------------

### ClueApiClient Initialization

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/clue_api_client/clue_api_client.html

Initializes the CLUE API client with an optional base URL and user key for authentication.

```APIDOC
## ClueApiClient

### Description
Basic class for running queries against CLUE api.

### Methods
#### __init__(self, base_url=None, user_key=None)
Initializes the client.

##### Parameters
- **base_url** (str) - Optional: specific URL to use for the CLUE api, e.g. https://dev-api.clue.io/api/
- **user_key** (str) - Optional: user key to use for authentication, available from CLUE account.
```

--------------------------------

### Entry point for the subset script

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/subset.html

Standard Python entry point that calls the main function when the script is executed directly.

```python
if __name__ == "__main__":
    main()

```

--------------------------------

### Main Function for gct2gctx Script

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/gct2gctx.html

Parses command-line arguments and initiates the GCT to GCTX conversion process. It calls `gct2gctx_main` to perform the actual conversion.

```python
def main():
    args = build_parser().parse_args(sys.argv[1:])
    setup_logger.setup(verbose=args.verbose)
    gct2gctx_main(args)
```

--------------------------------

### Build Argument Parser for gctx2gct

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/gctx2gct.html

Sets up an argument parser for the gctx2gct command-line script. It defines required and optional arguments for input file, output path, verbosity, and annotation file paths.

```python
import sys
import logging
import argparse
import os.path
import pandas as pd
import cmapPy.pandasGEXpress.setup_GCToo_logger as setup_logger
import cmapPy.pandasGEXpress.parse_gctx as parse_gctx
import cmapPy.pandasGEXpress.write_gct as write_gct

__author__ = "Oana Enache"
__email__ = "oana@broadinstitute.org"

logger = logging.getLogger(setup_logger.LOGGER_NAME)


def build_parser():
    parser = argparse.ArgumentParser(description=__doc__,
                                     formatter_class=argparse.ArgumentDefaultsHelpFormatter)
    # required
    parser.add_argument("-filename", "-f", required=True,
                        help=".gctx file that you would like to converted to .gct")
    # optional
    parser.add_argument("-output_filepath", "-o", default=None,
                        help=("out path/name for output gct file. " +
                              "Default is just to modify the extension"))
    parser.add_argument("-verbose", "-v",
                        help="Whether to print a bunch of output.", action="store_true", default=False)
    parser.add_argument("-row_annot_path", help="Path to annotations file for rows")
    parser.add_argument("-col_annot_path", help="Path to annotations file for columns")
    return parser
```

--------------------------------

### gctx2gct_main

Source: https://cmappy.readthedocs.io/en/latest/pandasGEXpress.html

Command-line script to convert a .gctx file to .gct. Supports v1.0 .gctx files.

```APIDOC
## gctx2gct_main

### Description
Command-line script to convert a .gctx file to .gct. It takes a .gctx file path and optionally an output path and/or name to save the equivalent .gct file.

### Parameters
* _args_ (any): Arguments for the command-line script.

### Note
Only supports v1.0 .gctx files.
```

--------------------------------

### gct2gctx_main

Source: https://cmappy.readthedocs.io/en/latest/pandasGEXpress.html

Command-line script to convert a .gct file to .gctx. Supports v1.3 .gct files.

```APIDOC
## gct2gctx_main

### Description
Command-line script to convert a .gct file to .gctx. It takes a .gct file path and optionally an output path and/or name to save the equivalent .gctx file.

### Parameters
* _args_ (any): Arguments for the command-line script.

### Note
Only supports v1.3 .gct files.
```

--------------------------------

### Main execution function for subset tool

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/subset.html

Parses command-line arguments and calls the core subsetting logic. It sets up logging based on verbosity and handles the main processing flow.

```python
def main():
    # Get args
    args = build_parser().parse_args(sys.argv[1:])
    setup_logger.setup(verbose=args.verbose)
    subset_main(args)

```

--------------------------------

### Create Conda Environment (Linux/Mac)

Source: https://cmappy.readthedocs.io/en/latest/build.html

Creates a new conda environment named 'my_cmapPy_env' with specified Python and package versions, including cmapPy from the bioconda channel.

```bash
conda create --name my_cmapPy_env -c bioconda python=2.7.11 numpy=1.11.2 pandas=0.20.3 h5py=2.7.0 requests==2.13.0 cmappy
```

--------------------------------

### Build Argument Parser for gct2gctx

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/gct2gctx.html

Sets up the command-line argument parser for the gct2gctx script. It defines required arguments like the input GCT filename and optional arguments for the output path, verbosity, and annotation file paths.

```python
import sys
import logging
import argparse
import os.path
import pandas as pd
import cmapPy.pandasGEXpress.setup_GCToo_logger as setup_logger
import cmapPy.pandasGEXpress.parse_gct as parse_gct
import cmapPy.pandasGEXpress.write_gctx as write_gctx

__author__ = "Oana Enache"
__email__ = "oana@broadinstitute.org"

logger = logging.getLogger(setup_logger.LOGGER_NAME)


def build_parser():
    parser = argparse.ArgumentParser(description=__doc__,
                                     formatter_class=argparse.ArgumentDefaultsHelpFormatter)
    # required
    parser.add_argument("-filename", "-f", required=True,
                        help=".gct file that you would like to convert to .gctx")
    # optional
    parser.add_argument("-output_filepath", "-o", default=None,
                        help=("out path/name for output gctx file. " +
                              "Default is just to modify the extension"))
    parser.add_argument("-verbose", "-v",
                        help="Whether to print a bunch of output.", action="store_true", default=False)
    parser.add_argument("-row_annot_path", help="Path to annotations file for rows")
    parser.add_argument("-col_annot_path", help="Path to annotations file for columns")
    return parser
```

--------------------------------

### write

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/set_io/grp.html

Writes a GRP object to a text file, with each element on a new line.

```APIDOC
## write

### Description
Write a GRP to a text file.

### Parameters
#### Request Body
- **grp** (list) - Required - GRP object to write to new-line delimited text file
- **out_path** (string) - Required - output path

### Returns
- None
```

--------------------------------

### Run POST Request to CLUE API

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/clue_api_client/clue_api_client.html

Submit data to a specified resource using a POST request. This is typically used for creating new entries.

```python
post_results = client.run_post('resources', {'key': 'value'})
```

--------------------------------

### Initialize GCToo Object

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/GCToo.html

Initializes a GCToo object with data, row metadata, and column metadata. Optionally creates a multi-index DataFrame.

```python
import numpy as np
import pandas as pd
import logging
import cmapPy.pandasGEXpress.setup_GCToo_logger as setup_logger


__authors__ = 'Oana Enache, Lev Litichevskiy, Dave Lahr'
__email__ = 'dlahr@broadinstitute.org'



class GCToo(object):
    """Class representing parsed gct(x) objects as pandas dataframes.
    Contains 3 component dataframes (row_metadata_df, column_metadata_df, 
    and data_df) as well as an assembly of these 3 into a multi index df
    that provides an alternate way of selecting data.
    """
    def __init__(self, data_df, row_metadata_df=None, col_metadata_df=None,
                 src=None, version=None, make_multiindex=False, logger_name=setup_logger.LOGGER_NAME):

        self.logger = logging.getLogger(logger_name)

        self.src = src
        self.version = version

        # Check data_df before setting
        self.check_df(data_df)
        self.data_df = data_df

        if row_metadata_df is None:
            self.row_metadata_df = pd.DataFrame(index=data_df.index)
        else:
            # Lots of checks will occur when this attribute is set (see __setattr__ below)
            self.row_metadata_df = row_metadata_df

        if col_metadata_df is None:
            self.col_metadata_df = pd.DataFrame(index=data_df.columns)
        else:
            # Lots of checks will occur when this attribute is set (see __setattr__ below)
            self.col_metadata_df = col_metadata_df

        # Create multi_index_df if explicitly requested
        if make_multiindex:
            self.assemble_multi_index_df()
        else:
            self.multi_index_df = None

        # This GCToo object is now initialized
        self._initialized = True

```

--------------------------------

### run_post

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/clue_api_client/clue_api_client.html

Executes a POST request to a specified resource name with provided data.

```APIDOC
## run_post

### Description
Runs a POST request against the CLUE API.

### Method
POST

### Endpoint
/{resource_name}

### Parameters
#### Path Parameters
- **resource_name** (str) - Required - Name of the resource/collection to post data to.

#### Request Body
- **data** (dict) - Required - The data to be sent in the request body.
```

--------------------------------

### Create Conda Environment (Windows)

Source: https://cmappy.readthedocs.io/en/latest/build.html

Creates a new conda environment named 'my_cmapPy_env' with specified Python and package versions, excluding cmapPy.

```bash
conda create --name my_cmapPy_env python=2.7.11 numpy=1.11.2 pandas=0.20.3 h5py=2.7.0 requests==2.13.0
```

--------------------------------

### Activate Conda Environment (Linux/Mac)

Source: https://cmappy.readthedocs.io/en/latest/build.html

Activates the previously created conda environment named 'my_cmapPy_env'.

```bash
source activate my_cmapPy_env
```

--------------------------------

### GCToo Class Initialization

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/GCToo.html

Initializes a GCToo object with data, row metadata, column metadata, and optional source information. It can also assemble a multi-index DataFrame.

```APIDOC
## GCToo Class

### Description
Class representing parsed gct(x) objects as pandas dataframes. Contains 3 component dataframes (row_metadata_df, column_metadata_df, and data_df) as well as an assembly of these 3 into a multi index df that provides an alternate way of selecting data.

### Method
__init__(self, data_df, row_metadata_df=None, col_metadata_df=None, src=None, version=None, make_multiindex=False, logger_name=setup_logger.LOGGER_NAME)

### Parameters
- **data_df** (pandas.DataFrame) - The primary data DataFrame.
- **row_metadata_df** (pandas.DataFrame, optional) - DataFrame containing row metadata. Defaults to None.
- **col_metadata_df** (pandas.DataFrame, optional) - DataFrame containing column metadata. Defaults to None.
- **src** (string, optional) - Source file path. Defaults to None.
- **version** (string, optional) - GCT file version. Defaults to None.
- **make_multiindex** (bool, optional) - If True, assembles a multi-index DataFrame. Defaults to False.
- **logger_name** (string, optional) - Name of the logger to use. Defaults to setup_logger.LOGGER_NAME.

### Attributes
- **logger**: Logger instance.
- **src**: Source file path.
- **version**: GCT file version.
- **data_df**: The main data DataFrame.
- **row_metadata_df**: DataFrame containing row metadata.
- **col_metadata_df**: DataFrame containing column metadata.
- **multi_index_df**: Multi-index DataFrame (assembled if make_multiindex is True).
- **_initialized**: Boolean flag indicating if the object is initialized.
```

--------------------------------

### build_parser

Source: https://cmappy.readthedocs.io/en/latest/pandasGEXpress.html

Builds the argument parser for the subset command-line script.

```APIDOC
## build_parser

### Description
Build argument parser for the subset command-line tool.

### Returns
* (ArgumentParser): The argument parser object.
```

--------------------------------

### Build Argument Parser for Concatenation

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/concat.html

Sets up the argument parser for the concatenation script, defining required and optional arguments for file input, concatenation direction, output type, and metadata handling.

```python
import argparse
import os
import sys
import glob
import logging
import numpy
import pandas as pd

import cmapPy.pandasGEXpress.GCToo as GCToo
import cmapPy.pandasGEXpress.setup_GCToo_logger as setup_logger
import cmapPy.pandasGEXpress.parse as parse
import cmapPy.pandasGEXpress.write_gct as write_gct
import cmapPy.pandasGEXpress.write_gctx as write_gctx


__author__ = "Lev Litichevskiy"
__email__ = "lev@broadinstitute.org"

logger = logging.getLogger(setup_logger.LOGGER_NAME)


def build_parser():
    parser = argparse.ArgumentParser(
        description=__doc__, formatter_class=argparse.ArgumentDefaultsHelpFormatter)

    # Required args
    parser.add_argument("--concat_direction", "-d", required=True,
                        choices=["horiz", "vert"],
                        help="which direction to concatenate")

    mutually_exclusive_group = parser.add_mutually_exclusive_group()
    mutually_exclusive_group.add_argument("--input_filepaths", "-if", nargs="+",
        help="full paths to gct(x) files to be concatenated")
    mutually_exclusive_group.add_argument("--file_wildcard", "-w", type=str,
        help=("wildcard specifying where files should be found " +
              "(make sure to surround in quotes if calling from command line!)"))

    parser.add_argument("--out_type", "-ot", default="gctx", choices=["gct", "gctx"],
                        help="whether to save output as a gct or gctx")
    parser.add_argument("--out_name", "-o", type=str, default="concated.gctx",
        help="what to name the output file")
    parser.add_argument("--fields_to_remove", "-ftr", nargs="+", default=[],
        help="fields to remove from the common metadata")
    parser.add_argument("--remove_all_metadata_fields", "-ramf", action="store_true", default=False,
                        help="remove all metadata fields during operation")
    parser.add_argument("--reset_ids", "-rsi", action="store_true", default=False,
        help="whether to reset ids (use this flag if ids are not unique)")

    parser.add_argument("-data_null", type=str, default="NaN",
        help="how to represent missing values in the data")
    parser.add_argument("-metadata_null", type=str, default="-666",
        help="how to represent missing values in the metadata")
    parser.add_argument("-filler_null", type=str, default="-666",
        help="what value to use for filling the top-left filler block if output is a .gct")
    parser.add_argument("-verbose", "-v", action="store_true", default=False,
        help="whether to print a bunch of output")

    parser.add_argument("--error_report_output_file", "-erof", type=str, default=None,
                        help=("destination file for writing out error report - currently information about inconsistent
                        metadata fields in the common dimension of the concat operation"))

    return parser

```

--------------------------------

### read

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/set_io/gmt.html

Reads a GMT file from the specified path. Each line in the GMT file is parsed into a dictionary with 'head', 'desc', and 'entry' keys.

```APIDOC
## read

### Description
Reads a gmt file at the path specified by file_path.

### Method
`read(file_path)`

### Parameters
#### Arguments
* **file_path** (string) - Required - path to gmt file

### Returns
* **gmt** (GMT object) - list of dicts, where each dict corresponds to one line of the GMT file

### Example
```python
# Assuming 'my_file.gmt' exists and is a valid GMT file
gmt_data = cmapPy.set_io.gmt.read('my_file.gmt')
```
```

--------------------------------

### write

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/set_io/gmt.html

Writes a GMT object (list of dictionaries) to a text file at the specified output path.

```APIDOC
## write

### Description
Write a GMT to a text file.

### Method
`write(gmt, out_path)`

### Parameters
#### Arguments
* **gmt** (GMT object) - Required - list of dicts
* **out_path** (string) - Required - output path

### Returns
* None

### Example
```python
# Assuming 'gmt_data' is a GMT object and 'output.gmt' is the desired output file path
cmapPy.set_io.gmt.write(gmt_data, 'output.gmt')
```
```

--------------------------------

### write (gctx)

Source: https://cmappy.readthedocs.io/en/latest/pandasGEXpress.html

Writes a GCToo instance to a specified .gctx file. Supports converting null values back to '-666', setting gzip compression level for metadata, defining maximum chunk size, and specifying the data matrix's data type.

```APIDOC
## write (gctx)

### Description
Writes a GCToo instance to specified file.

### Method
`cmapPy.pandasGEXpress.write_gctx.write`

### Parameters
#### Input
* **gctoo_object** (GCToo) - A GCToo instance.
* **out_file_name** (str) - File name to write gctoo_object to.
* **convert_back_to_neg_666** (bool) - Whether to convert np.NAN in metadata back to "-666". Default=True.
* **gzip_compression_level** (int) - Compression level to use for metadata. Default=6.
* **max_chunk_kb** (int) - The maximum number of KB a given chunk will occupy. Default=1024.
* **matrix_dtype** (numpy dtype) - Storage data type for data matrix. Default=numpy.float32.
```

--------------------------------

### subset_main

Source: https://cmappy.readthedocs.io/en/latest/pandasGEXpress.html

Command-line script to extract a subset of data from a GCT(x) file.

```APIDOC
## subset_main

### Description
Extract a subset of data from a GCT(x) file using the command line. IDs can be provided as a list or as a path to a .grp file. This is the command-line interface equivalent to the `subset_gctoo` method.

### Parameters
* _args_ (any): Arguments for the command-line script.
```

--------------------------------

### Build Argument Parser for Subset Tool

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/subset.html

Constructs the argument parser for the command-line subset tool. It defines required and optional arguments for input file, row/column selection, exclusion criteria, output name, output type, and verbosity.

```python
import logging
import sys
import os
import argparse

import cmapPy.pandasGEXpress.setup_GCToo_logger as setup_logger
import cmapPy.pandasGEXpress.parse_gct as parse_gct
import cmapPy.pandasGEXpress.parse_gctx as parse_gctx
import cmapPy.pandasGEXpress.subset_gctoo as sg
import cmapPy.pandasGEXpress.write_gct as wg
import cmapPy.pandasGEXpress.write_gct as wgx
import cmapPy.set_io.grp as grp

__author__ = "Lev Litichevskiy"
__email__ = "lev@broadinstitute.org"

logger = logging.getLogger(setup_logger.LOGGER_NAME)



[docs]
def build_parser():
    """Build argument parser."""

    parser = argparse.ArgumentParser(description=__doc__,
                                     formatter_class=argparse.ArgumentDefaultsHelpFormatter)

    # Required args
    parser.add_argument("--in_path", "-i", required=True,
                        help="file path to input GCT(x) file")

    parser.add_argument("--rid", nargs="+", help="filepath to grp file or string array for including rows")
    parser.add_argument("--cid", nargs="+", help="filepath to grp file or string array for including cols")
    parser.add_argument("--exclude_rid", "-er", nargs="+", help="filepath to grp file or string array for excluding rows")
    parser.add_argument("--exclude_cid", "-ec", nargs="+", help="filepath to grp file or string array for excluding cols")
    parser.add_argument("--out_name", "-o", default="ds_subsetted.gct",
                        help="what to name the output file")
    parser.add_argument("--out_type", default="gct", choices=["gct", "gctx"],
                        help="whether to write output as GCT or GCTx")
    parser.add_argument("--verbose", "-v", action="store_true", default=False,
                        help="whether to increase the # of messages reported")

    return parser

```

--------------------------------

### Build Common Metadata DataFrame

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/concat.html

Handles the construction of a common metadata dataframe, including deduplication and logging of shapes. Returns the unique metadata dataframe and the dataframe with duplicates.

```python
if all_meta_df_with_dups.empty:
        # Simply return unique ids
        all_meta_df = pd.DataFrame(index=all_meta_df_with_dups.index.unique())

    else:
        all_meta_df_with_dups["concat_column_for_index"] = all_meta_df_with_dups.index
        all_meta_df = all_meta_df_with_dups.copy(deep=True).drop_duplicates()
        all_meta_df.drop("concat_column_for_index", axis=1, inplace=True)
        all_meta_df_with_dups.drop("concat_column_for_index", axis=1, inplace=True)

    logger.debug("all_meta_df_with_dups.shape: {}".format(all_meta_df_with_dups.shape))
    logger.debug("all_meta_df.shape: {}".format(all_meta_df.shape))

    return (all_meta_df, all_meta_df_with_dups)
```

--------------------------------

### write

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/write_gctx.html

Writes a GCToo instance to a specified file in .gctx format. This function handles the entire process of converting a GCToo object into a .gctx file, including setting up the HDF5 file structure, writing version information, source attributes, data matrix, and metadata.

```APIDOC
## write

### Description
Writes a GCToo instance to specified file in .gctx format.

### Parameters
#### Arguments
- **gctoo_object** (GCToo) - A GCToo instance to be written.
- **out_file_name** (str) - The desired file name for the output .gctx file.
- **convert_back_to_neg_666** (bool, optional) - If True, converts np.NAN in metadata back to "-666". Defaults to True.
- **gzip_compression_level** (int, optional) - The gzip compression level to use for metadata. Defaults to 6.
- **max_chunk_kb** (int, optional) - The maximum size in KB for data matrix chunks. Defaults to 1024.
- **matrix_dtype** (numpy dtype, optional) - The data type for storing the matrix. Defaults to numpy.float32.
```

--------------------------------

### parse(file_path, convert_neg_666=True, rid=None, cid=None, ridx=None, cidx=None, row_meta_only=False, col_meta_only=False, make_multiindex=False)

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/parse.html

Identifies whether the file path corresponds to a .gct or .gctx file and calls the correct corresponding parse method. It supports GCT1.2, GCT1.3, and GCTX1.0 files.

```APIDOC
## parse

### Description
Generic parse method to parse either a .gct or .gctx file.

Takes in a file path corresponding to either a .gct or .gctx, and parses to a GCToo instance accordingly.

Note: Supports GCT1.2, GCT1.3, and GCTX1.0 files.

### Parameters
#### Mandatory
- **file_path** (str): full path to gct(x) file you want to parse.

#### Optional
- **convert_neg_666** (bool): whether to convert -666 values to numpy.nan or not. Default = True.
- **rid** (list of strings): list of row ids to specifically keep from gctx. Default=None.
- **cid** (list of strings): list of col ids to specifically keep from gctx. Default=None.
- **ridx** (list of integers): only read the rows corresponding to this list of integer ids. Default=None.
- **cidx** (list of integers): only read the columns corresponding to this list of integer ids. Default=None.
- **row_meta_only** (bool): Whether to load data + metadata (if False), or just row metadata (if True) as pandas DataFrame. Default = False.
- **col_meta_only** (bool): Whether to load data + metadata (if False), or just col metadata (if True) as pandas DataFrame. Default = False.
- **make_multiindex** (bool): whether to create a multi-index df combining the 3 component dfs. Default = False.

### Output
- **out** (GCToo object or pandas df): if row_meta_only or col_meta_only, then out is a metadata df; otherwise, it's a GCToo instance containing content of parsed gct(x) file.

### Note on `convert_neg_666`
In CMap, for somewhat obscure historical reasons, we use "-666" as our null value for metadata. However (so that users can take full advantage of pandas' methods, including those for filtering nan's etc) we provide the option of converting these into numpy.NaN values, the pandas default.
```

--------------------------------

### ClueApiClient Class

Source: https://cmappy.readthedocs.io/en/latest/clue_api_client.html

The main class for interacting with the CLUE API. It requires a base URL and a user key for authentication.

```APIDOC
## ClueApiClient Class

### Description
Basic class for running queries against CLUE api.

### Initialization
```python
ClueApiClient(base_url=None, user_key=None)
```

### Parameters
- **base_url** (str) - Optional - The base URL for the CLUE API.
- **user_key** (str) - Optional - The user key for authenticating with the CLUE API.
```

--------------------------------

### Write GCTx File

Source: https://cmappy.readthedocs.io/en/latest/pandasGEXpress.html

Writes a GCToo instance to a specified .gctx file. Options include controlling the conversion of np.NAN back to '-666' for metadata, setting gzip compression level for metadata, and defining the maximum chunk size in KB. You can also specify the data matrix's storage data type.

```python
cmapPy.pandasGEXpress.write_gctx.write(gctoo_object, out_file_name, convert_back_to_neg_666=True, gzip_compression_level=6, max_chunk_kb=1024, matrix_dtype=<type 'numpy.float32'>)
```

--------------------------------

### Writing GRP files

Source: https://cmappy.readthedocs.io/en/latest/set_io.html

Writes a GRP object to a text file.

```APIDOC
## cmapPy.set_io.grp.write

### Description
Write a GRP to a text file.

### Method
`write(grp, out_path)`

### Parameters
#### Path Parameters
- **grp** (list) - Required - GRP object to write to new-line delimited text file
- **out_path** (string) - Required - output path

### Returns
- **None**
```

--------------------------------

### Run PUT Request to CLUE API

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/clue_api_client/clue_api_client.html

Update an existing resource identified by its ID using a PUT request with new data. Returns the updated resource data.

```python
put_results = client.run_put('resources', 'resource_id', {'key': 'new_value'})
```

--------------------------------

### Write GRP File

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/set_io/grp.html

Writes a GRP object (list) to a new-line delimited text file at the specified output path.

```python
def write(grp, out_path):
    """ Write a GRP to a text file.

    Args:
        grp (list): GRP object to write to new-line delimited text file
        out_path (string): output path

    Returns:
        None

    """
    with open(out_path, "w") as f:
        for x in grp:
            f.write(str(x) + "\n")
```

--------------------------------

### assemble_common_meta

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/concat.html

Assembles common metadata DataFrames from a list, ensuring consistency and handling potential mismatches.

```APIDOC
## assemble_common_meta

### Description
Assembles common metadata DataFrames from a list of GCToo objects. It identifies and handles fields that are not present in all DataFrames, sorts the resulting index, and raises an exception if significant inconsistencies are found.

### Parameters
- **common_meta_dfs** (list of pandas dfs) - A list of pandas DataFrames, each representing the common metadata from a GCToo object.
- **fields_to_remove** (list of strings) - Fields to be removed from the common metadata because they do not agree across files.
- **sources** (list of strings) - List of source identifiers for the GCToo objects, used in error reporting.
- **remove_all_metadata_fields** (bool) - If True, all metadata fields are removed.
- **error_report_file** (str, optional) - Path to save a report of metadata mismatches.

### Returns
- **all_meta_df_sorted** (pandas df) - A sorted pandas DataFrame containing the assembled common metadata.
```

--------------------------------

### Reading GRP files

Source: https://cmappy.readthedocs.io/en/latest/set_io.html

Reads a GRP file from the specified path.

```APIDOC
## cmapPy.set_io.grp.read

### Description
Read a grp file at the path specified by in_path.

### Method
`read(in_path)`

### Parameters
#### Path Parameters
- **in_path** (string) - Required - path to GRP file

### Returns
- **grp** (list) - The content of the GRP file.
```

--------------------------------

### Write GMT File

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/set_io/gmt.html

Writes a GMT object (a list of dictionaries) to a text file at the specified output path. Each line in the output file will be tab-delimited, containing the set identifier, description, and members.

```python
def write(gmt, out_path):
    """ Write a GMT to a text file.

    Args:
        gmt (GMT object): list of dicts
        out_path (string): output path

    Returns:
        None

    """
    with open(out_path, 'w') as f:
        for _, each_dict in enumerate(gmt):
            f.write(each_dict[SET_IDENTIFIER_FIELD] + '\t')
            f.write(each_dict[SET_DESC_FIELD] + '\t')
            f.write('\t'.join([str(entry) for entry in each_dict[SET_MEMBERS_FIELD]]))
            f.write('\n')

```

--------------------------------

### Core subsetting logic for GCT(x) files

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/subset.html

Performs the actual subsetting of GCT or GCTx files based on provided arguments. It handles reading the input, applying subsetting operations using appropriate parsing functions, and preparing for output.

```python
def subset_main(args):
    """ Separate method from main() in order to make testing easier and to
    enable command-line access. """

    # Read in each of the command line arguments
    rid = _read_arg(args.rid)
    cid = _read_arg(args.cid)
    exclude_rid = _read_arg(args.exclude_rid)
    exclude_cid = _read_arg(args.exclude_cid)

    # If GCT, use subset_gctoo
    if args.in_path.endswith(".gct"):

        in_gct = parse_gct.parse(args.in_path)
        out_gct = sg.subset_gctoo(in_gct, rid=rid, cid=cid,
                                 exclude_rid=exclude_rid,
                                 exclude_cid=exclude_cid)

    # If GCTx, use parse_gctx
    else:

        if (exclude_rid is not None) or (exclude_cid is not None):
            msg = "exclude_{rid,cid} args not currently supported for parse_gctx."
            raise(Exception(msg))

        logger.info("Using hyperslab selection functionality of parse_gctx...")
        out_gct = parse_gctx.parse(args.in_path, rid=rid, cid=cid)

    # Write the output gct
    if args.out_type == "gctx":
        wgx.write(out_gct, args.out_name)
    else:
        wg.write(out_gct, args.out_name, data_null="NaN", metadata_null="NA", filler_null="NA")

```

--------------------------------

### Write GCT Version and Dimensions

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/write_gct.html

Writes the first two lines of a GCT file: the version string and the dimensions of the dataset. This is a helper function for the main `write` function.

```python
def write_version_and_dims(version, dims, f):
    """Write first two lines of gct file.

    Args:
        version (string): 1.3 by default
        dims (list of strings): length = 4
        f (file handle): handle of output file
    Returns:
        nothing
    """
    f.write(("#" + version + "\n"))
    f.write((dims[0] + "\t" + dims[1] + "\t" + dims[2] + "\t" + dims[3] + "\n"))
```

--------------------------------

### get_file_list

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/concat.html

Searches for files to be concatenated using a wildcard.

```APIDOC
## get_file_list

### Description
Search for files to be concatenated. Currently very basic, but could expand to be more sophisticated.

### Method
`get_file_list(wildcard)`

### Parameters
#### Arguments
- **wildcard** (regular expression string) - The wildcard pattern to search for files.

### Returns
- **files** (list of full file paths) - A list of full file paths matching the wildcard.
```

--------------------------------

### run_put

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/clue_api_client/clue_api_client.html

Executes a PUT request to update a resource by its ID with new data.

```APIDOC
## run_put

### Description
Runs a PUT request against the CLUE API to update a resource by its ID.

### Method
PUT

### Endpoint
/{resource_name}/{id}

### Parameters
#### Path Parameters
- **resource_name** (str) - Required - Name of the resource/collection to update.
- **id** (str) - Required - The ID of the resource to update.

#### Request Body
- **data** (dict) - Required - The data to update the resource with.
```

--------------------------------

### Core gctx to gct Conversion Logic

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/gctx2gct.html

Performs the conversion from a GCTX file to a GCT file. It parses the input GCTX, determines the output filename, optionally loads and applies row/column annotations, and writes the resulting GCT file.

```python
def gctx2gct_main(args):
    """ Separate from main() in order to make command-line tool. """

    in_gctoo = parse_gctx.parse(args.filename, convert_neg_666=False)

    if args.output_filepath is None:
        basename = os.path.basename(args.filename)
        out_name = os.path.splitext(basename)[0] + ".gct"
    else:
        out_name = args.output_filepath

    """ If annotations are supplied, parse table and set metadata_df """
    if args.row_annot_path is None:
        pass
    else:
        row_metadata = pd.read_csv(args.row_annot_path, sep='\t', index_col=0, header=0, low_memory=False)
        assert all(in_gctoo.data_df.index.isin(row_metadata.index)), \
            "Row ids in matrix missing from annotations file"
        in_gctoo.row_metadata_df = row_metadata.loc[row_metadata.index.isin(in_gctoo.data_df.index)]

    if args.col_annot_path is None:
        pass
    else:
        col_metadata = pd.read_csv(args.col_annot_path, sep='\t', index_col=0, header=0, low_memory=False)
        assert all(in_gctoo.data_df.columns.isin(col_metadata.index)), \
            "Column ids in matrix missing from annotations file"
        in_gctoo.col_metadata_df = col_metadata.loc[col_metadata.index.isin(in_gctoo.data_df.columns)]

    write_gct.write(in_gctoo, out_name)
```

--------------------------------

### String Representation of GCToo Object

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/GCToo.html

Provides a concise string summary of the GCToo object, including its version, source, and dimensions of its data and metadata DataFrames.

```python
version = "{}
".format(self.version)
        source = "src: {}
".format(self.src)


        data = "data_df: [{} rows x {} columns]
".format(
        self.data_df.shape[0], self.data_df.shape[1])

        row_meta = "row_metadata_df: [{} rows x {} columns]
".format(
        self.row_metadata_df.shape[0], self.row_metadata_df.shape[1])

        col_meta = "col_metadata_df: [{} rows x {} columns]".format(
        self.col_metadata_df.shape[0], self.col_metadata_df.shape[1])

        full_string = (
        version + source + data + row_meta + col_meta)
        return full_string
```

--------------------------------

### Generic parse method for GCT/GCTX files

Source: https://cmappy.readthedocs.io/en/latest/_modules/cmapPy/pandasGEXpress/parse.html

Use this function to parse either a .gct or .gctx file. It automatically detects the file type and calls the appropriate parsing function. Supports GCT1.2, GCT1.3, and GCTX1.0 files.

```python
import logging
import cmapPy.pandasGEXpress.setup_GCToo_logger as setup_logger
import cmapPy.pandasGEXpress.parse_gct as parse_gct
import cmapPy.pandasGEXpress.parse_gctx as parse_gctx

__author__ = "Oana Enache"
__email__ = "oana@broadinstitute.org"

# instantiate logger
logger = logging.getLogger(setup_logger.LOGGER_NAME)



def parse(file_path, convert_neg_666=True, rid=None, cid=None, ridx=None, cidx=None,
          row_meta_only=False, col_meta_only=False, make_multiindex=False):
    """
    Identifies whether file_path corresponds to a .gct or .gctx file and calls the
    correct corresponding parse method.

    Input:
        Mandatory:
        - gct(x)_file_path (str): full path to gct(x) file you want to parse.

        Optional:
        - convert_neg_666 (bool): whether to convert -666 values to numpy.nan or not
            (see Note below for more details on this). Default = False.
        - rid (list of strings): list of row ids to specifically keep from gctx. Default=None.
        - cid (list of strings): list of col ids to specifically keep from gctx. Default=None.
        - ridx (list of integers): only read the rows corresponding to this
            list of integer ids. Default=None.
        - cidx (list of integers): only read the columns corresponding to this
            list of integer ids. Default=None.
        - row_meta_only (bool): Whether to load data + metadata (if False), or just row metadata (if True)
            as pandas DataFrame
        - col_meta_only (bool): Whether to load data + metadata (if False), or just col metadata (if True)
            as pandas DataFrame
        - make_multiindex (bool): whether to create a multi-index df combining
            the 3 component dfs

    Output:
        - out (GCToo object or pandas df): if row_meta_only or col_meta_only, then
            out is a metadata df; otherwise, it's a GCToo instance containing
            content of parsed gct(x) file

    Note: why does convert_neg_666 exist?
        - In CMap--for somewhat obscure historical reasons--we use "-666" as our null value
        for metadata. However (so that users can take full advantage of pandas' methods,
        including those for filtering nan's etc) we provide the option of converting these
        into numpy.NaN values, the pandas default.
    """
    if file_path.endswith(".gct"):
        out = parse_gct.parse(file_path, convert_neg_666=convert_neg_666,
                              rid=rid, cid=cid, ridx=ridx, cidx=cidx,
                              row_meta_only=row_meta_only, col_meta_only=col_meta_only,
                              make_multiindex=make_multiindex)

    elif file_path.endswith(".gctx"):
        out = parse_gctx.parse(file_path, convert_neg_666=convert_neg_666,
                              rid=rid, cid=cid, ridx=ridx, cidx=cidx,
                              row_meta_only=row_meta_only, col_meta_only=col_meta_only,
                              make_multiindex=make_multiindex)

    else:
        err_msg = "File to parse must be .gct or .gctx!"
        logger.error(err_msg)
        raise Exception(err_msg)

    return out

```

--------------------------------

### Writing GMT files

Source: https://cmappy.readthedocs.io/en/latest/set_io.html

Writes a GMT object to a text file.

```APIDOC
## cmapPy.set_io.gmt.write

### Description
Write a GMT to a text file.

### Method
`write(gmt, out_path)`

### Parameters
#### Path Parameters
- **gmt** (GMT object) - Required - list of dicts
- **out_path** (string) - Required - output path

### Returns
- **None**
```