Domino Data Lab

https://docs.dominodatalab.com
Admin
Domino Data Lab is an enterprise MLOps platform designed for data scientists to develop, deploy, and...
Tokens:2,487,572
Snippets:16,450
Trust Score:10
Update:1 year ago
Show doc for...
Tokens:
Raw
### Get Started with Python in Domino

Source: https://docs.dominodatalab.com/en/6.0/user_guide/9a69d9/get-started-with-python

This tutorial guides users through a common model lifecycle in Domino using Python. It covers creating and configuring projects, starting workspaces, developing models, and deploying them, with examples of Jupyter, Dash, pandas, and Prophet.

```markdown
Get started with Python
This tutorial will guide you through a common model lifecycle in Domino. You will start by working with data from the Balancing Mechanism Reporting Service in the UK. We will be exploring the Electricity Generation by Fuel Type and predicting the electricity generation in the future. You’ll see examples of Jupyter, Dash, pandas, and Prophet used in Domino.
The following content is meant to be followed in sequence.
  * Create a Project
  * Configure your Project
  * Start a Workspace
  * Get your files and data
  * Develop your model
  * Clean up Workspaces
  * Deploy your model
```

--------------------------------

### Install Domino using Fleetcommand CLI

Source: https://docs.dominodatalab.com/en/5.9/admin_guide/4bb6a6/installation

This command initiates the installation of Domino using the Fleetcommand CLI. It requires an installation token provided by a Solutions Architect and assumes the fleetcommand-agent is set up.

```bash
fleetcommand install domino --token <your-token> --output domino.yaml

```

--------------------------------

### Download domino.yaml with Fleetcommand CLI

Source: https://docs.dominodatalab.com/en/5.9/admin_guide/4bb6a6/installation

This command downloads a pre-configured domino.yaml file, simplifying the Domino installation process. It requires the Fleetcommand CLI to be installed and accessible.

```bash
fleetcommand download domino.yaml --output domino.yaml

```

--------------------------------

### Install Domino with Fleetcommand CLI

Source: https://docs.dominodatalab.com/en/5.11/admin_guide/4bb6a6/installation

Installs Domino using the Fleetcommand CLI, requiring an installation token. This method is suitable for users with an existing token provided by a Solutions Architect.

```bash
# Example command structure (actual command will vary based on token and version)
fleetcommand-agent install --token <your_installation_token>
```

--------------------------------

### Domino CLI: Get Project Example

Source: https://docs.dominodatalab.com/en/5.11/user_guide/9355a5/domino-cli-reference

Demonstrates how to retrieve an existing Domino Project from the server onto a local machine. This command is used for initial project setup or accessing collaborative projects.

```bash
domino get otherUser/someProject
domino get myProject
```

--------------------------------

### Download domino.yaml Configuration

Source: https://docs.dominodatalab.com/en/5.11/admin_guide/4bb6a6/installation

Provides instructions or methods to download pre-configured `domino.yaml` files, which simplifies the Domino installation process by offering streamlined configuration options.

```bash
# Example command to download a pre-configured domino.yaml file
curl -O https://<domino_download_url>/domino.yaml
```

--------------------------------

### Download Streamlined domino.yaml

Source: https://docs.dominodatalab.com/en/5.10/admin_guide/4bb6a6/installation

This documentation mentions downloading pre-configured `domino.yaml` files to simplify the installation process. This approach is useful for users who want to quickly deploy Domino with common configurations.

```yaml
# Example of a streamlined domino.yaml file (content not provided in text)
# This file would contain pre-configured settings for Domino installation.
```

--------------------------------

### Fleetcommand CLI: Install Domino

Source: https://docs.dominodatalab.com/en/5.8/admin_guide/4bb6a6/installation

This snippet demonstrates how to install Domino using the Fleetcommand CLI. It assumes you have an installation token provided by your Solutions Architect. The command utilizes the 'install' subcommand with the necessary token.

```bash
fleetcommand-agent install --token <your_installation_token>
```

--------------------------------

### Register and Run Domino Flow

Source: https://docs.dominodatalab.com/en/5.11/user_guide/5b5259/get-started-with-flows

This command registers and executes a Domino Flow defined in `workflow.py`. It specifies the workflow name (`simple_math_workflow`) and provides the required input arguments (`a` and `b`). The execution is performed remotely.

```bash
pyflyte run --remote workflow.py simple_math_workflow --a 10 --b 6
```

--------------------------------

### Python Workspace in Domino

Source: https://docs.dominodatalab.com/en/cloud/user_guide/9a69d9/get-started-with-python

This snippet demonstrates how to start a Python workspace within Domino, a common environment for developing models. It assumes the user has already created and configured a project. The workspace provides an interactive environment for data exploration and model building.

```bash
# Example: Starting a Python Workspace (conceptual, actual steps within Domino UI)
# 1. Navigate to your project in Domino.
# 2. Click 'New' and select 'Workspace'.
# 3. Choose a Python environment (e.g., Python 3.9).
# 4. Click 'Start Workspace'.
# Once the workspace is running, you can open a Jupyter Notebook or a terminal.
```

--------------------------------

### Install Domino Data API R Package

Source: https://docs.dominodatalab.com/en/cloud/api_guide/5267b0/install-the-data-api

Installs the Domino Data API for R. It first installs the 'remotes' package and then installs the DominoDataR library from GitHub, allowing for version specification.

```bash
USER root

## Install Domino Data API as dependency of R library
RUN python -m pip install dominodatalab-data=={x.y.z}

## Install R library via github (using https://remotes.r-lib.org/)
RUN R --no-save -e "install.packages('remotes')"
RUN R --no-save -e "remotes::install_github('dominodatalab/DominoDataR@{a.b.c}'

USER ubuntu
```

--------------------------------

### Fleetcommand CLI: Download domino.yaml

Source: https://docs.dominodatalab.com/en/5.8/admin_guide/4bb6a6/installation

This command allows you to download a pre-configured `domino.yaml` file, which simplifies the Domino installation process. This file contains essential configuration parameters tailored for your environment.

```bash
fleetcommand-agent download domino.yaml
```

--------------------------------

### Developing a Model with Pandas and Prophet in Domino

Source: https://docs.dominodatalab.com/en/cloud/user_guide/9a69d9/get-started-with-python

This snippet illustrates a basic model development workflow using pandas for data manipulation and Prophet for time-series forecasting within a Domino workspace. It assumes data has been loaded into a pandas DataFrame. This is a simplified example of the 'Develop your model' step.

```python
import pandas as pd
from prophet import Prophet

# Assuming 'df' is a pandas DataFrame loaded from Domino datasets
# It should have columns 'ds' (datetime) and 'y' (value to predict)
# Example DataFrame creation (replace with actual data loading)
data = {
    'ds': pd.to_datetime(['2023-01-01', '2023-01-02', '2023-01-03', '2023-01-04', '2023-01-05']),
    'y': [10, 12, 15, 13, 16]
}
df = pd.DataFrame(data)

# Initialize and fit the Prophet model
model = Prophet()
model.fit(df)

# Create a future dataframe to make predictions
future = model.make_future_dataframe(periods=3)

# Make predictions
forecast = model.predict(future)

# Display the forecast
print(forecast[['ds', 'yhat', 'yhat_lower', 'yhat_upper']].tail())

# In a real scenario, you would save this model or its predictions
# For example, save the forecast to a CSV file in the project
# forecast.to_csv('electricity_generation_forecast.csv', index=False)
```

--------------------------------

### Install Domino with Fleetcommand CLI

Source: https://docs.dominodatalab.com/en/5.10/admin_guide/4bb6a6/installation

This snippet demonstrates how to install Domino using the Fleetcommand CLI. It assumes you have an installation token provided by your Solutions Architect. This method is suitable for environments where direct internet access is available and a token has been issued.

```bash
# Example command to install Domino using Fleetcommand CLI
# fleetcommand-agent install --token <your_installation_token>
```

--------------------------------

### Run Domino Job Task for Data Preparation

Source: https://docs.dominodatalab.com/en/5.11/user_guide/5b5259/get-started-with-flows

This Python code snippet demonstrates how to use the `run_domino_job_task` helper method to execute a data preparation task. It specifies the command, environment, hardware, inputs, and output specifications for the task. The task takes a data path as input and outputs a processed CSV file.

```python
data_prep_results = run_domino_job_task(
    flyte_task_name="Prepare data",
    command="python /mnt/code/scripts/prep-data.py",
    environment_name="Domino Standard Environment Py3.11 R4.4",
    hardware_tier_name="Small",
    inputs=[
        Input(name="data_path", type=str, value=data_path)
    ],
    output_specs=[
        Output(name="processed_data", type=FlyteFile[TypeVar("csv")])
    ],
    use_project_defaults_for_omitted=True
)
```

--------------------------------

### Install Domino Data API Python Package (PyPI)

Source: https://docs.dominodatalab.com/en/5.10/api_guide/5267b0/install-the-data-api

Installs the Domino Python package with the Data API as an extra dependency directly from the Python Package Index (PyPI). This is a recommended method for easier and consistent import.

```bash
python -m pip install dominodatalab[data]
```

--------------------------------

### Register and Run Flyte Workflow

Source: https://docs.dominodatalab.com/en/5.11/user_guide/5b5259/get-started-with-flows

This command registers and runs a Flyte workflow named 'training_workflow' defined in the 'workflow.py' file. It specifies the '--remote' option for running on a remote Flyte cluster and provides the 'data_path' argument with the path to the input data file.

```bash
pyflyte run --remote workflow.py training_workflow --data_path /mnt/code/data/data.csv
```

--------------------------------

### Fleetcommand CLI Reference - Core Commands

Source: https://docs.dominodatalab.com/en/5.10/admin_guide/4bb6a6/installation

This section provides a reference for the core commands and arguments available in the Fleetcommand CLI. These commands are essential for installing, configuring, and managing Domino deployments.

```bash
# fleetcommand-agent install [options]
# fleetcommand-agent configure [options]
# fleetcommand-agent upgrade [options]
# fleetcommand-agent uninstall [options]
```

--------------------------------

### Run Domino Job Task for Model Training

Source: https://docs.dominodatalab.com/en/5.11/user_guide/5b5259/get-started-with-flows

This Python code snippet illustrates the execution of a model training task using `run_domino_job_task`. It defines the command, environment, hardware, inputs (including the processed data from the previous step, epochs, and batch size), and output specifications for the trained model. The task outputs a trained model file.

```python
training_results = run_domino_job_task(
    flyte_task_name="Train model",
    command="python /mnt/code/scripts/train-model.py",
    environment_name="Domino Standard Environment Py3.11 R4.4",
    hardware_tier_name="Small",
    inputs=[
        Input(name="processed_data", type=FlyteFile[TypeVar("csv")], value=data_prep_results['processed_data']),
        Input(name="epochs", type=int, value=10),
        Input(name="batch_size", type=int, value=32)
    ],
    output_specs=[
        Output(name="model", type=FlyteFile)
    ],
    use_project_defaults_for_omitted=True
)
```

--------------------------------

### Deploying a Model in Domino

Source: https://docs.dominodatalab.com/en/cloud/user_guide/9a69d9/get-started-with-python

This snippet outlines the conceptual steps for deploying a model in Domino. Deployment makes your model available for predictions via an API or as a published application. The exact implementation depends on the deployment type (e.g., REST API, Shiny App).

```bash
# Example: Deploying a Model (conceptual steps within Domino UI)
# 1. Ensure your model artifact (e.g., a saved model file or a Docker image)
#    is available in your Domino project.
# 2. Navigate to the 'Deployments' section of your project.
# 3. Click 'New Deployment'.
# 4. Configure the deployment type (e.g., Model API, App).
# 5. Specify the model artifact and any required environment settings.
# 6. Set up scaling and monitoring parameters.
# 7. Click 'Create Deployment'.
# Once deployed, Domino provides an endpoint for accessing your model's predictions.
```

--------------------------------

### Domino Python Library Setup

Source: https://docs.dominodatalab.com/en/5.11/api_guide/c5ef26/the-python-domino-library

Instructions on how to set up and use the python-domino library for interacting with Domino APIs, including installation and connection setup.

```APIDOC
## python-domino Library Setup

This library provides bindings for the Domino APIs and is available in the Domino Standard Environment (DSE).

### Installation

To use the Python binding in a Domino workbook session, include `dominodatalab` in your project’s `requirements.txt` file.

For development, install dependencies from `setup.py`:
```bash
pipenv install -e ".[dev]"
```

For Airflow and data dependencies:
```bash
pipenv install -e ".[data]" ".[airflow]"
```

### Connection Setup

Instantiate the `Domino` class to set up the connection:

```python
_class_ Domino(project, api_key=None, host=None, domino_token_file=None, auth_token=None)
```

#### Parameters

*   **project** (string) - Required - A project identifier (e.g., `owner_user_name/projectname`).
*   **api_proxy** (string) - Optional - Location of the Domino API reverse proxy as `host:port`. This is the preferred method of authentication. Alternatively, set the `DOMINO_API_PROXY` environment variable. This mechanism does not work when connecting to an HTTPS endpoint; it is meant to be used inside Domino runs.
*   **api_key** (string) - Optional - An API key to authenticate with. If not provided, the library expects to find one in the `DOMINO_USER_API_KEY` environment variable.
*   **host** (string) - Optional - A host URL. If not provided, the library expects to find one in the `DOMINO_API_HOST` environment variable.
*   **domino_token_file** (string) - Optional - Path to the Domino token file containing auth token. If not provided, the library expects to find one in the `DOMINO_TOKEN_FILE` environment variable.
*   **auth_token** (string) - Optional - Authentication token.

### Authentication Order

Domino authenticates in the following order, using the first method found:

1.  `api_proxy`
2.  `auth_token`
3.  `domino_token_file`
4.  `api_key`
5.  `DOMINO_API_PROXY`
6.  `DOMINO_TOKEN_FILE`
7.  `DOMINO_USER_API_KEY`

### Additional Environment Variables

*   **DOMINO_LOG_LEVEL** (string) - Sets the log level (default: `INFO`). Example: `DEBUG`.
*   **DOMINO_VERIFY_CERTIFICATE** (boolean) - Set to `false` for testing or SSL certificate issues. Unset when not in use.
*   **DOMINO_MAX_RETRIES** (integer) - Determines the number of request attempts in case of `ConnectionError` (default: 4).
```

--------------------------------

### Download Domino Offline Installation Bundle using curl

Source: https://docs.dominodatalab.com/en/latest/admin_guide/87b601/offline-installation

Downloads the complete offline installation bundle for Domino using the curl command. It requires username and password for authentication and supports basic authentication and streaming the download.

```bash
curl -u username:password -#SfLOJ https://mirrors.domino.tech/s3/domino-artifacts/offline/fleetcommand-agent-${RELEASE_TAG}/docker-images-6.1.x.tar
```

--------------------------------

### Download Domino Offline Installation Bundle using wget

Source: https://docs.dominodatalab.com/en/latest/admin_guide/87b601/offline-installation

Downloads the complete offline installation bundle for Domino using the wget command. This method is useful for handling network interruptions with the --continue (-c) option and supports user authentication.

```bash
wget -c --user domino-registry --password xxxxxxx https://mirrors.domino.tech/s3/domino-artifacts/offline/fleetcommand-agent-${RELEASE_TAG}/docker-images-6.1.x.tar
```

--------------------------------

### R Package Installation Example

Source: https://docs.dominodatalab.com/en/5.8/user_guide/fda6d8/domino-reproducibility-tips

This snippet shows an analogous pattern for R practitioners to install packages. It involves having a single `install.R` file within the project directory, which is executed before other R scripts. This centralizes package management for reproducibility.

```r
# Example content of an install.R file
# install.packages("dplyr")
# devtools::install_github("someuser/somerepo")
```

--------------------------------

### Example Post Setup Script - Bash

Source: https://docs.dominodatalab.com/en/5.7/user_guide/5dd2c1/edit-environment-definition

An example of a bash command for a Post Setup Script. These scripts execute after Python packages from requirements.txt are installed.

```bash
# Example: Perform post-installation configuration
python /app/post_install_config.py
```

--------------------------------

### Example Pre Setup Script - Bash

Source: https://docs.dominodatalab.com/en/5.7/user_guide/5dd2c1/edit-environment-definition

An example of a bash command for a Pre Setup Script. These scripts execute before Python packages from requirements.txt are installed.

```bash
# Example: Install a system dependency before package installation
sudo apt-get update && sudo apt-get install -y some-package
```

--------------------------------

### Installation Configuration Reference

Source: https://docs.dominodatalab.com/en/6.0/admin_guide/7f4331/install-configuration-reference

This section details the configuration keys, their descriptions, requirement status, and accepted values for installing Domino.

```APIDOC
## Installation Configuration Reference

### Parameters

#### Core Configuration

- **`schema`** (string) - Required - YAML schema version. Accepted value: `1.10`
- **`name`** (string) - Required - Unique deployment name, must include the deployment owner's name. Accepts `[a-zA-Z0-9_-]+`.
- **`version`** (string) - Required - Domino version to install. Supported versions: `6.0.0`, `6.0.1`, `6.0.2`, `6.0.3`, `6.0.4`, `6.0.5`.
- **`hostname`** (string) - Required - Hostname Domino application will be accessed at. Must be a valid fully qualified domain name (FQDN).
- **`ssl_enabled`** (boolean) - Required - Determines if Domino should only be accessible with HTTPS. Accepts `true`, `false`.
- **`ssl_redirect`** (boolean) - Required - Determines if Domino should only be accessible with HTTPS. Accepts `true`, `false`.
- **`request_resources`** (boolean) - Required - Creates Kubernetes resource requests and limits for services. Accepts `true`, `false`.
- **`enable_network_policies`** (boolean) - Required - Uses network policies for fine-grained service access. Accepts `true`, `false`. Requires a compatible CNI plugin (e.g., Calico).
- **`external_docker_registry`** (string) - Optional - URL to an external Docker registry (ECR, ACR, GCR, GAR). The node must have permissions to the registry. Either `external_docker_registry` or `internal_docker_registry` must be configured.
- **`custom_certificates`** (object) - Required - Custom Certificates configuration.
- **`minimal_resources`** (boolean) - Optional - Uses the minimum amount of nodes and resources to run Domino. This may not be a stable configuration. Accepts `false`.
- **`cluster_domain`** (string) - Optional - The Kubernetes cluster domain name. Defaults to `cluster.local`.
- **`aws_account_id`** (string) - Optional - AWS account ID in which Domino is deployed.
- **`operator_role_arn`** (string) - Optional - IAM role ARN for the operator.
- **`operator_service_account_name`** (string) - Optional - Kubernetes service account name for the operator.

### Istio Configuration

Configures Istio Service Mesh deployment or integration. These configurations are for Domino use only and should be enabled only if intra-cluster encryption in transit is required.

- **`istio.enabled`** (boolean) - Required - Enables Istio in deployment (sidecar injection). Accepts `true`, `false`.
- **`istio.install`** (boolean) - Required - Installs Istio service with Domino. Accepts `true`, `false`.
- **`istio.cni`** (boolean) - Required - Configures Istio installation with a CNI. If `true`, installation uses CNI and requires fewer permissions (recommended). If `false`, installation adds `NET_ADMIN` and `NET_RAW` capabilities to every pod security policy. Accepts `true`, `false`.

### Ingress Controller Configuration

Configures the NGINX ingress controller deployed by the `fleetcommand-agent`.

- **`ingress_controller.install`** (boolean) - Required - Installs the NGINX ingress controller. Accepts `true`, `false`.
- **`ingress_controller.gcp_static_ip_name`** (string) - Optional - On GCP, uses a static IP address to expose Nginx. Specify the `name` of the IP address.
- **`ingress_controller.class_name`** (string) - Required - Name of the ingress class for Domino. Must be `nginx`.
- **`ingress_controller.istio_annotations`** (boolean) - Required - Includes annotations related to Istio. Accepts `true`, `false`.
- **`ingress_controller.allowed_cidrs`** (array) - Optional - List of allowed ingress CIDR blocks.

### Namespaces Configuration

Configures namespaces for segmenting Kubernetes executions. The installer requires that these namespaces do not exist at installation.

- **`namespaces.platform.name`** (string) - Required - Namespace to place Domino services. Must be a valid Kubernetes Name.
- **`namespaces.compute.name`** (string) - Required - Namespace for user executions. Must be a valid Kubernetes Name. Might be the same as the platform namespace.
- **`namespaces.system.name`** (string) - Required - Namespace for deployment metadata. Must be a valid Kubernetes Name.
- **`namespaces.istio.name`** (string) - Required - Namespace for Istio. Must be a valid Kubernetes Name.
- **`namespaces.*.annotations`** (object) - Optional - Optional annotations to apply to each namespace. Accepts Kubernetes Annotation format.
- **`namespaces.*.labels`** (object) - Optional - Optional labels to apply to each namespace. Accepts Kubernetes Labels and Selectors format.
```

--------------------------------

### Example: Post Setup Script in Domino Environments

Source: https://docs.dominodatalab.com/en/5.11/user_guide/5dd2c1/edit-environment-definition

This provides an example of bash code for the 'Post Setup Script' in Domino Environments. These commands execute after Python packages from requirements.txt are installed and run as root.

```bash
# Example: Configure a service or application after dependencies are installed
echo "Starting post-setup script..."
python /opt/scripts/configure_app.py --user ubuntu
chown -R ubuntu:ubuntu /opt/app_data
echo "Post-setup complete."
```

--------------------------------

### Fleetcommand Agent Tool Reference

Source: https://docs.dominodatalab.com/en/5.11/admin_guide/4bb6a6/installation

The fleetcommand-agent is the primary tool for installing, configuring, and upgrading Domino. This section provides reference information for its commands and arguments.

```bash
# Example usage of fleetcommand-agent for installation
fleetcommand-agent install --help

# Example usage for upgrading Domino
fleetcommand-agent upgrade --version <target_version>

# Example usage for configuration
fleetcommand-agent configure --file <config_file.yaml>
```

--------------------------------

### Accessing Data in Domino with Python

Source: https://docs.dominodatalab.com/en/cloud/user_guide/9a69d9/get-started-with-python

This snippet shows how to access data within a Domino project using Python. It utilizes the Domino Python SDK to interact with datasets, which are fundamental for model training and analysis. Ensure the Domino Python SDK is installed in your environment.

```python
from domino.datasets import Dataset

# Get a dataset from the current project
dataset = Dataset.load('your-dataset-name')

# Load the dataset into a pandas DataFrame
df = dataset.to_pandas_dataframe()

# Display the first few rows of the DataFrame
print(df.head())

# You can also access files directly within the dataset
# with open(dataset.path('your_file.csv'), 'r') as f:
#     content = f.read()
#     print(content)
```

--------------------------------

### Python Domino Library Usage Example

Source: https://docs.dominodatalab.com/en/5.11/api_guide/d9cc25/python-wrapper-for-domino-api

A comprehensive example demonstrating how to use the `dominodatalab` Python library to interact with Domino, including listing runs, files, getting blob content, and starting new runs.

```APIDOC
## Example Usage

The following is an example of usage:

```python
from domino import Domino

# By and large your commands will run against a single project,
# so you must specify the full project name
domino = Domino("chris/canon")

# List all runs in the project, most-recently queued first
all_runs = domino.runs_list()['data']

latest_100_runs = all_runs[0:100]

print(latest_100_runs)

# all runs have a commitId (the snapshot of the project when the
# run starts) and, if the run completed, an "outputCommitId"
# (the snapshot of the project after the run completed)
most_recent_run = all_runs[0]

commitId = most_recent_run['outputCommitId']

# list all the files in the output commit ID -- only showing the
# entries under the results directory.  If not provided, will
# list all files in the project.  Or you can say path=“/“ to
# list all files
files = domino.files_list(commitId, path='results/')

for file in files:
    print file['path'], '->', file['url']

print(files)

# Get the content (i.e. blob) for the file you're interested in.
# blobs_get returns a connection rather than the content, because
# the content can get quite large and it's up to you how you want
# to handle it
print(domino.blobs_get(files[0]['key']).read())

# Start a run of file main.py using the latest copy of that file
domino.runs_start(["main.py", "arg1", "arg2"])

# Start a "direct" command
domino.runs_start(["echo 'Hello, World!'"], isDirect=True)

# Start a run of a specific commit
domino.runs_start(["main.py"], commitId="aabbccddee")
```
```

--------------------------------

### python-domino Library Setup and Connection

Source: https://docs.dominodatalab.com/en/6.0/api_guide/c5ef26/the-python-domino-library

Instructions on how to install and set up the python-domino library, including connection parameters and authentication methods.

```APIDOC
## python-domino Library

This library provides bindings for the Domino APIs.

### Installation

To use the Python binding in a Domino workbook session, include `dominodatalab` in your project’s requirements.txt file.

For development, install dependencies from `setup.py`:
```bash
pipenv install -e "'.[dev]'"
```

For Airflow and data dependencies:
```bash
pipenv install -e "'.[data]'" "'.[airflow]'"
```

### Setup the Connection

Create a new instance of `Domino` to set up the connection:

```python
_class_ Domino(project, api_key=None, host=None, domino_token_file=None, auth_token=None)
```

#### Parameters

- **project** (string) - Required - A project identifier (in the form of owner_user_name/projectname).
- **api_proxy** (string) - Optional - Location of the Domino API reverse proxy as `host:port`. This is the preferred method of authentication.
- **api_key** (string) - Optional - An API key to authenticate with. Falls back to `DOMINO_USER_API_KEY` environment variable.
- **host** (string) - Optional - A host URL. Falls back to `DOMINO_API_HOST` environment variable.
- **domino_token_file** (string) - Optional - Path to the Domino token file containing auth token. Falls back to `DOMINO_TOKEN_FILE` environment variable.
- **auth_token** (string) - Optional - Authentication token.

### Authentication

Domino looks for authentication methods in the following order:
1. `api_proxy`
2. `auth_token`
3. `domino_token_file`
4. `api_key`
5. `DOMINO_API_PROXY` (environment variable)
6. `DOMINO_TOKEN_FILE` (environment variable)
7. `DOMINO_USER_API_KEY` (environment variable)

The API proxy is the preferred authentication method.

### Additional Environment Variables

- **DOMINO_LOG_LEVEL** (string) - Default: `INFO`. Controls the logging level (e.g., `DEBUG`).
- **DOMINO_VERIFY_CERTIFICATE** (boolean) - For testing SSL certificate issues, set to `false`. Ensure it's reset when not in use.
- **DOMINO_MAX_RETRIES** (integer) - Default: 4. Determines the number of retry attempts for requests in case of a `ConnectionError`.
```

--------------------------------

### Example: Pre Setup Script in Domino Environments

Source: https://docs.dominodatalab.com/en/5.11/user_guide/5dd2c1/edit-environment-definition

This shows bash code for the 'Pre Setup Script' in Domino Environments. These commands execute before Python packages from requirements.txt are installed and run as root.

```bash
# Example: Update package lists before installing dependencies
echo "Starting pre-setup script..."
apt-get update -y
apt-get install -y --no-install-recommends some-utility
echo "Pre-setup complete."
```

--------------------------------

### Start a Domino Job using a Service Account token

Source: https://docs.dominodatalab.com/en/6.0/user_guide/40b91f/domino-api-authentication

This example demonstrates how to start a Domino Job programmatically using a Service Account token. It is recommended to store the SA token as a custom environment variable for secure credential storage.

```APIDOC
## POST /api/jobs/v1/jobs

### Description
Starts a Domino Job using a Service Account token.

### Method
POST

### Endpoint
/api/jobs/v1/jobs

### Parameters
#### Header Parameters
- **Authorization** (string) - Required - Bearer token obtained from a Service Account.
- **Content-Type** (string) - Required - application/json

#### Request Body
- **projectId** (string) - Required - The ID of the project where the job will run.
- **runCommand** (string) - Required - The command to execute for the job.

### Request Example
```json
{
  "projectId": "12398gg098",
  "runCommand": "main.py"
}
```

### Response
#### Success Response (200)
- **jobId** (string) - The ID of the newly started job.
- **status** (string) - The status of the job.

#### Response Example
```json
{
  "jobId": "654321",
  "status": "Pending"
}
```
```

--------------------------------

### Create Initial Archive of Configs (Bash)

Source: https://docs.dominodatalab.com/en/6.0/user_guide/83e94a/connect-to-a-cloudera-cdh5-cluster-from-domino

This command compresses the gathered `hadoop-binaries-configs` directory into a tar.gz archive on the edge node. This archive will then be downloaded to the local machine for further preparation.

```bash
cd /tmp

tar -zcf hadoop-binaries-configs.tar.gz hadoop-binaries-configs

```

--------------------------------

### Domino CLI: Create Project Examples

Source: https://docs.dominodatalab.com/en/5.11/user_guide/9355a5/domino-cli-reference

Examples for creating new Domino Projects. The 'create' command makes a new folder and registers it with Domino, optionally taking a project name and owner. The 'init' command initializes a Domino Project within an existing directory.

```bash
domino create
domino create myProject
domino create --owner myOrg myProject
domino init
domino init myProject
domino init --owner myOrg myProject
```

--------------------------------

### Pre Setup Script Example - Domino Environment

Source: https://docs.dominodatalab.com/en/5.8/user_guide/5dd2c1/edit-environment-definition

Bash code executed before Python packages from `requirements.txt` are installed. This script runs as root and is suitable for system-level configurations or installations required before the project's Python dependencies are set up.

```bash
# Enter commands to execute before the Python packages in your Project’s requirements.txt are installed.
echo "Pre Setup script executed"
```

--------------------------------

### Domino Python Wrapper Usage Examples

Source: https://docs.dominodatalab.com/en/latest/api_guide/d9cc25/python-wrapper-for-domino-api

Provides practical code examples demonstrating common operations using the Domino Python wrapper, such as listing runs, files, and starting new runs.

```APIDOC
## Domino Python Wrapper Usage Examples

### Description
Illustrative examples of how to use the `dominodatalab` Python library to interact with the Domino API, covering project initialization, run management, and file operations.

### Initialization

```python
from domino import Domino

# Initialize Domino client for a specific project
domino = Domino("chris/canon")
```

### Listing Runs

```python
# List all runs in the project, most-recently queued first
all_runs = domino.runs_list()['data']

# Get the latest 100 runs
latest_100_runs = all_runs[0:100]

print(latest_100_runs)
```

### Listing Files in a Commit

```python
# Assuming 'all_runs' is already fetched and 'most_recent_run' is selected
most_recent_run = all_runs[0]
commitId = most_recent_run['outputCommitId']

# List files under the 'results/' directory for a specific commitId
files = domino.files_list(commitId, path='results/')

# To list all files in the project root:
# files = domino.files_list(commitId, path='/')

for file in files['data']:
    print(file['path'], '->', file['url'])

print(files)
```

### Getting Blob Content

```python
# Get the content (blob) for a file (e.g., the first file in the list)
# Note: blobs_get returns a connection, not the content directly
file_to_get = files['data'][0]
print(domino.blobs_get(file_to_get['key']).read())
```

### Starting Runs

```python
# Start a run of file 'main.py' with arguments 'arg1' and 'arg2'
domino.runs_start(["main.py", "arg1", "arg2"])

# Start a 'direct' command
domino.runs_start(["echo 'Hello, World!'"], isDirect=True)

# Start a run of a specific commit
domino.runs_start(["main.py"], commitId="aabbccddee")
```
```

--------------------------------

### Install Configuration Reference

Source: https://docs.dominodatalab.com/en/5.7/admin_guide/7f4331/install-configuration-reference

This section details the configuration parameters for installing Domino, including schema version, deployment naming, hostname, SSL settings, resource allocation, and external registry configurations.

```APIDOC
## Install Configuration Reference

### Description
Configuration parameters for installing Domino.

### Parameters
#### Request Body
- **schema** (string) - Required - YAML schema version. Supported value: `1.4`
- **name** (string) - Required - Unique deployment name. Must contain the name of the deployment owner. Format: `[a-zA-Z0-9_-]+`
- **version** (string) - Required - Domino version to install. Supported versions: `5.7.0`
- **hostname** (string) - Required - Hostname Domino application will be accessed at. Must be a valid fully qualified domain name (FQDN).
- **ssl_enabled** (boolean) - Required - Should Domino only be accessible with HTTPS. Values: `true`, `false`
- **ssl_redirect** (boolean) - Required - Should Domino only be accessible with HTTPS. Values: `true`, `false`
- **request_resources** (boolean) - Required - Create Kubernetes resource requests and limits for services. Values: `true`, `false`
- **enable_network_policies** (boolean) - Required - Use network policies for fine-grained service access. Values: `true`, `false`. Note: Requires a compatible CNI plugin (e.g., Calico).
- **external_docker_registry** (string) - Optional - A URL to ECR (Amazon Elastic Container Registry), ACR (Azure Container Registry), GCR (Google Container Registry) or GAR (Google Artifact Registry). The node must have permissions to the registry. Note: Either `external_docker_registry` or `internal_docker_registry` must be configured.
- **custom_certificates** (object) - Required - Custom Certificates configuration.
- **minimal_resources** (boolean) - Optional - Use the minimum amount of nodes and resources to run Domino. This may not be a stable configuration and should be used with caution. Default: `false`

### Request Example
```json
{
  "schema": "1.4",
  "name": "my-domino-deployment",
  "version": "5.7.0",
  "hostname": "domino.example.com",
  "ssl_enabled": true,
  "ssl_redirect": true,
  "request_resources": true,
  "enable_network_policies": true,
  "external_docker_registry": "https://my.registry.example.com",
  "custom_certificates": {},
  "minimal_resources": false
}
```

### Response
#### Success Response (200)
- **message** (string) - Indicates successful configuration update.

#### Response Example
```json
{
  "message": "Domino installation configuration updated successfully."
}
```
```

--------------------------------

### Helm 3 Installer Configuration Example

Source: https://docs.dominodatalab.com/en/5.11/admin_guide/559046/fleetcommand-agent-release-notes

This configuration snippet illustrates the setup for Helm 3 with the fleetcommand-agent installer. It omits the Tiller image and adjusts parameters for compatibility with Helm 3's architecture and supported chart repositories like GCR.io.

```yaml
helm:
  version: 3
  host: gcr.io
  namespace: domino-eng-service-artifacts
  insecure: false
  username: _json_key  # To support GCR authentication, this must be "_json_key"
  password: "<password>"
  tiller_image: null # Not required for Helm 3
  prefix: '' # Charts are stored without a prefix by default
```

--------------------------------

### Install Configuration Reference

Source: https://docs.dominodatalab.com/en/5.11/admin_guide/7f4331/install-configuration-reference

This section details the configuration parameters required for installing Domino. It outlines the key-value pairs, their descriptions, whether they are mandatory, and acceptable values.

```APIDOC
## Install Configuration Reference

This section details the configuration parameters required for installing Domino. It outlines the key-value pairs, their descriptions, whether they are mandatory, and acceptable values.

| Key | Description | Required | Values |
|---|---|---|---|
| `schema` | YAML schema version. | ✓ | `1.9` |
| `name` | Unique deployment name. This must contain the name of the deployment owner. | ✓ | `[a-zA-Z0-9_-]+` |
| `version` | Domino version to install. | ✓ | Supported versions: `5.11.0`, `5.11.1`, `5.11.2`, `5.11.3`, `5.11.4` |
| `hostname` | Hostname Domino application will be accessed at. | ✓ | Valid fully qualified domain name (FQDN) |
| `ssl_enabled` | Should Domino only be accessible with HTTPS. | ✓ | `true`, `false` |
| `ssl_redirect` | Should Domino only be accessible with HTTPS. | ✓ | `true`, `false` |
| `request_resources` | Create Kubernetes resource requests and limits for services. | ✓ | `true`, `false` |
| `enable_network_policies` | Use network policies for fine-grained service access. | ✓ | `true`, `false` | Note | Requires a compatible CNI plugin for example, Calico |
| `external_docker_registry` | A URL to ECR (Amazon Elastic Container Registry), ACR (Azure Container Registry), GCR (Google Container Registry) or GAR (Google Artifact Registry). The node must have permissions to the registry. | | | Note | Either `external_docker_registry` or `internal_docker_registry` must be configured. |
| `custom_certificates` | Custom Certificates | ✓ |  |
| `minimal_resources` | Use the minimum amount of nodes and resources to run Domino. This may not be a stable configuration and should be used with caution. | | `false` |
| `cluster_domain` | The Kubernetes cluster domain name. | | `cluster.local` |
| `extended_catalog` | Enables full image catalog for use. Default is `false`, which gates available images to only a core set of required services. | | `false` |
```

--------------------------------

### Initialize Domino Installer

Source: https://docs.dominodatalab.com/en/5.10/admin_guide/5765ba/installation-overview

Initializes the Domino installer by running the fleetcommand-agent Docker image. This generates a template configuration file named domino.yml. Requires a volume mount to persist the configuration file.

```bash
docker run --rm -it -v $(pwd):/install quay.io/domino/fleetcommand-agent:5.10.1.catalog-b1ce370
init --file /install/domino.yml
```

--------------------------------

### Kubernetes Autoscaling Error Message Example

Source: https://docs.dominodatalab.com/en/5.10/admin_guide/4482de/workspaces-and-jobs-troubleshooting

This example demonstrates a common error message encountered when Kubernetes autoscaling fails to scale up node pools. It highlights issues with node affinity, volume node affinity conflicts, and maximum node group sizes, which can prevent Workspaces, Jobs, Apps, and Model APIs from starting.

```json
{
  "description" : "pod didn't trigger scale-up: 8 node(s) didn't match Pod's node affinity/selector, 2 node(s) had volume node affinity conflict, 4 max node group size reached"
}
```

--------------------------------

### Python Domino Library Setup

Source: https://docs.dominodatalab.com/en/5.10/api_guide/c5ef26/the-python-domino-library

Instructions for installing and setting up the python-domino library for use within Domino environments or for local development.

```APIDOC
## Python Domino Library Setup

### Installation

To use the Python binding in a Domino workbook session, include `dominodatalab` in your project’s `requirements.txt` file.

For local development, use Pipenv to manage dependencies:

```bash
# Install development dependencies
pipenv install -e ".[dev]"

# Install data and airflow dependencies
pipenv install -e ".[data]" "-e ".[airflow]"
```

### Connection Setup

Instantiate the `Domino` class to set up the connection. Authentication methods are checked in a specific order.

#### `Domino` Class Constructor

```python
_class_ Domino(project, api_key=None, host=None, domino_token_file=None, auth_token=None)
```

#### Parameters

*   **`project`** (string) - Required - A project identifier (in the form of `owner_user_name/projectname`).
*   **`api_proxy`** (string) - Optional - Location of the Domino API reverse proxy as `host:port`. This is the preferred authentication method within Domino runs.
*   **`api_key`** (string) - Optional - An API key to authenticate with. If not provided, the library looks for the `DOMINO_USER_API_KEY` environment variable.
*   **`host`** (string) - Optional - A host URL. If not provided, the library looks for the `DOMINO_API_HOST` environment variable.
*   **`domino_token_file`** (string) - Optional - Path to the Domino token file containing auth token. If not provided, the library looks for the `DOMINO_TOKEN_FILE` environment variable.
*   **`auth_token`** (string) - Optional - Authentication token.

### Authentication Order

Domino authenticates using the first available method in this order:

1.  `api_proxy`
2.  `auth_token`
3.  `domino_token_file`
4.  `api_key`
5.  `DOMINO_API_PROXY` (Environment Variable)
6.  `DOMINO_TOKEN_FILE` (Environment Variable)
7.  `DOMINO_USER_API_KEY` (Environment Variable)

### Additional Environment Variables

*   **`DOMINO_LOG_LEVEL`**: Controls the logging level (e.g., `INFO`, `DEBUG`). Defaults to `INFO`.
*   **`DOMINO_VERIFY_CERTIFICATE`**: Set to `false` for testing SSL certificate issues. Use with caution.
*   **`DOMINO_MAX_RETRIES`**: Determines the number of retry attempts for requests in case of `ConnectionError`. Defaults to 4.
```

--------------------------------

### Get S3 File with R (aws.s3)

Source: https://docs.dominodatalab.com/en/latest/user_guide/dd93d4/connect-to-amazon-s3

R code example utilizing the aws.s3 package to fetch a file from an S3 bucket. Ensure the aws.s3 package is installed and AWS credentials are set up. Suitable for data analysis workflows in R.

```r
library(aws.s3)

bucket_name <- "your-bucket-name"
file_key <- "path/to/your/file.csv"
local_filename <- "downloaded_file.csv"

aws.s3::save_object(object = file_key, bucket = bucket_name, file = local_filename)
cat(paste0("File ", file_key, " downloaded to ", local_filename, "\n"))
```

--------------------------------

### Initialize Domino Installer and Generate Configuration

Source: https://docs.dominodatalab.com/en/5.9/admin_guide/5765ba/installation-overview

Runs the Domino fleetcommand-agent installer in a Docker container to generate a template configuration file named 'domino.yml'. It mounts the current directory to persist the generated file.

```bash
docker run --rm -it -v $(pwd):/install quay.io/domino/fleetcommand-agent:v66.3
init --file /install/domino.yml
```

--------------------------------

### Domino API Python Wrapper Usage Example

Source: https://docs.dominodatalab.com/en/5.9/api_guide/d9cc25/python-wrapper-for-domino-api

Demonstrates how to instantiate the Domino client, list runs, retrieve files from a specific commit, get file content, and start new runs with arguments or direct commands.

```python
from domino import Domino

# By and large your commands will run against a single project,
# so you must specify the full project name
domino = Domino("chris/canon")

# List all runs in the project, most-recently queued first
all_runs = domino.runs_list()['data']

latest_100_runs = all_runs[0:100]

print(latest_100_runs)

# all runs have a commitId (the snapshot of the project when the
# run starts) and, if the run completed, an "outputCommitId"
# (the snapshot of the project after the run completed)
most_recent_run = all_runs[0]

commitId = most_recent_run['outputCommitId']

# list all the files in the output commit ID -- only showing the
# entries under the results directory.  If not provided, will
# list all files in the project.  Or you can say path="/" to
# list all files
files = domino.files_list(commitId, path='results/')

for file in files:
    print file['path'], '->', file['url']

print(files)

# Get the content (i.e. blob) for the file you're interested in.
# blobs_get returns a connection rather than the content, because
# the content can get quite large and it's up to you how you want
# to handle it
print(domino.blobs_get(files[0]['key']).read())

# Start a run of file main.py using the latest copy of that file
domino.runs_start(["main.py", "arg1", "arg2"])

# Start a "direct" command
domino.runs_start(["echo 'Hello, World!'"], isDirect=True)

# Start a run of a specific commit
domino.runs_start(["main.py"], commitId="aabbccddee")
```
Tokens:
Raw