### Install TAO Launcher CLI Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/quick_start_guide/running_via_launcher.html Use the quick start script to install the TAO launcher and NGC CLI. ```bash bash setup/quickstart_launcher.sh --install ``` -------------------------------- ### Install TAO CLI Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao_deploy/nsight_dl_designer_integration.html Setup a Python virtual environment and install the TAO client library. ```bash # Create and activate virtual environment python3 -m venv tao_env source tao_env/bin/activate # Install TAO pip install nvidia-tao-client ``` -------------------------------- ### Install and Start Minikube Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao_toolkit_api/api_deployment.html Installs Minikube for local Kubernetes development and starts a cluster with Docker driver, Docker runtime, and GPU support. It also exposes port 32080. ```bash #install minikube curl -LO https://github.com/kubernetes/minikube/releases/latest/download/minikube-linux-amd64 sudo install minikube-linux-amd64 /usr/local/bin/minikube && rm minikube-linux-amd64 #start minikube minikube start --driver=docker --container-runtime=docker --gpus=all --ports=32080:32080 ``` -------------------------------- ### Environment Setup for Workflow Example Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao_toolkit_api/api_inference_microservice.html Set up environment variables for API base URL, organization name, and NGC API key. This script also handles authentication to obtain an access token. ```bash # Environment setup export BASE_URL="http://:/api/v2" export NGC_ORG="your-org-name" export NGC_API_KEY="your-ngc-api-key" # Authenticate TOKEN=$(curl -s -X POST $BASE_URL/login \ -H "Content-Type: application/json" \ -d '{ "ngc_key": "'"$NGC_API_KEY"'", "ngc_org_name": "'"$NGC_ORG"'" }' | jq -r '.token') echo "Authenticated with token" # Set experiment and training job IDs (from previous training) EXPERIMENT_ID="your-experiment-id" TRAIN_JOB_ID="your-completed-training-job-id" ``` -------------------------------- ### Object Detection Workflow Example Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao_toolkit_api/api_remote_client.html An example of initiating an object detection workflow using the v2 API. This includes fetching base experiments, getting training schemas, and setting up the initial training job configuration. Remember to edit the spec file as needed. ```bash # Complete object detection workflow with v2 API export WORKSPACE_ID="workspace_789" export DATASET_URI="://my-bucket/coco-dataset" # aws://, azure://, lustre://, file://, seaweedfs:// # 1. Get base experiment and training schema tao rtdetr list-base-experiments --filter-param network_arch=rtdetr --output json | jq . export SELECTED_PTM_ID="ptm_id_from_list" # Get training schema and save to file tao rtdetr get-job-schema --action train --base-experiment-id "$SELECTED_PTM_ID" --output @train_spec.yaml # Edit train_spec.yaml as needed. For example, update "epochs": 100 and "learning_rate": 0.001 ``` -------------------------------- ### Navigate to Setup Directory Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao_toolkit_api/api_docker_compose.html Change the working directory to the Docker Compose setup folder. ```bash cd tao_tutorials/setup/tao-docker-compose ``` -------------------------------- ### Launch NSight DL Designer Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao_deploy/nsight_dl_designer_integration.html Commands to navigate to the installation directory and start the NSight DL Designer application. ```bash cd /host/linux-desktop-dl-x64 ./nsight-dl ``` -------------------------------- ### Example Analyze Command (TAO Launcher) Source: https://docs.nvidia.com/tao/tao-toolkit/text/data_services/data_analytics.html An example of how to run the data analyze command with a default experiment specification file. ```bash tao dataset analytics analyze -e $DEFAULT_SPEC ``` -------------------------------- ### Execute Prune Command Examples Source: https://docs.nvidia.com/tao/tao-toolkit/text/cv_finetuning/tensorflow_2/image_classification_tf2.html Specific examples for triggering the pruning process via FTMS or the TAO Launcher. ```bash PRUNE_JOB_ID=$(tao classification_tf2 create-job \ --kind experiment \ --name "classification_tf2_prune" \ --action prune \ --workspace-id $WORKSPACE_ID \ --parent-job-id $TRAIN_JOB_ID \ --specs @prune_spec.yaml \ --base-experiment-id "$BASE_EXPERIMENT_ID" \ --encryption-key "nvidia_tlt" \ --output json | jq -r '.id') ``` ```bash tao model classification_tf2 prune -e /workspace/spec.yaml ``` -------------------------------- ### TAO Launcher Prune Command Example Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/cv_finetuning/tensorflow_2/image_classification_tf2.html Example of executing the prune command using the TAO Launcher. ```bash tao model classification_tf2 prune -e /workspace/spec.yaml ``` -------------------------------- ### Install TAO Client Versions Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/ds_tao/deepstream_tao_integration.html Commands to install specific versions of the TAO client in a virtual environment. ```bash pip3 install nvidia-pyindex pip3 install nvidia-tao-client==0.1.19 # for 3.0-21.08 pip3 install nvidia-tao-client==0.1.20 # for 3.0-21.11 ``` -------------------------------- ### Install Docker Engine Source: https://docs.nvidia.com/tao/tao-toolkit/text/tao_toolkit_api/api_setup.html Configures the Docker repository and installs the Docker engine, CLI, and plugins on Ubuntu. ```bash # Add Docker's official GPG key: sudo apt-get update sudo apt-get install ca-certificates curl sudo install -m 0755 -d /etc/apt/keyrings sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc sudo chmod a+r /etc/apt/keyrings/docker.asc # Add the repository to Apt sources: echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \ $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update #install docker sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin #add user to docker group sudo usermod -aG docker $USER newgrp docker ``` -------------------------------- ### Start Triton Server Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/cv_finetuning/pytorch/panoptic_3d_reconstruction/nvpanoptix3d.html Execute this command to start the Triton Inference Server. Ensure you are in the correct directory. ```bash bash scripts/nvpanoptix3d_e2e_inference/start_server.sh ``` -------------------------------- ### Set Up Environment Variables Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao_toolkit_api/api_airgapped_deployment.html Install the required package and configure environment variables for model downloading. ```bash # Install tao-core package pip install -U nvidia-tao-core # Set environment variables export AIRGAPPED_MODE=True export PTM_API_KEY="your_ngc_api_key" ``` -------------------------------- ### Clone TAO Tutorials Repository Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/quick_start_guide/index.html Use this command to download the TAO tutorials repository, which contains setup scripts and tutorial notebooks for TAO. ```bash git clone https://github.com/NVIDIA/tao_tutorials.git ``` -------------------------------- ### Start Inference Microservice via cURL Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao_toolkit_api/api_inference_microservice.html Example command to start an inference microservice using cURL and capture the job ID. ```bash INFERENCE_MS_JOB_ID=$(curl -s -X POST \ $BASE_URL/orgs/$NGC_ORG/inference_microservices:start \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "parent_job_id": "'"$TRAIN_JOB_ID"'" }' | jq -r '.job_id') echo "Inference Microservice Job ID: $INFERENCE_MS_JOB_ID" ``` -------------------------------- ### Start Jupyter Notebook Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/quick_start_guide/running_via_launcher.html Launch the Jupyter notebook server to begin working with TAO tutorials. ```bash jupyter notebook --ip 0.0.0.0 --port 8888 --allow-root ``` -------------------------------- ### Install Git LFS for DeepStream Apps Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/ds_tao/deformable_detr_ds.html Required setup for cloning the DeepStream TLT apps repository. ```bash curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash sudo apt-get install git-lfs git lfs install ``` -------------------------------- ### Initialize DeepStream Development Environment Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/ds_tao/nvocdr_ds.html Commands to start the appropriate DeepStream container and install OpenCV dependencies. ```bash docker run --gpus=all -v : --rm -it --privileged --net=host nvcr.io/nvidia/deepstream:6.2-triton bash # install opencv apt update && apt install -y libopencv-dev ``` ```bash docker run --gpus=all -v : --rm -it --privileged --net=host nvcr.io/nvidia/deepstream-l4t:6.2-triton bash # install opencv apt update && apt install -y libopencv-dev ``` ```bash # install opencv apt update && apt install -y libopencv-dev ``` -------------------------------- ### Example Sparse4D Training Command with TAO Launcher Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/cv_finetuning/pytorch/sparse4d/sparse4d.html An example command for initiating Sparse4D training using TAO Launcher, specifying the experiment specification file and results directory. ```bash tao model sparse4d train -e $DEFAULT_SPEC results_dir=$RESULTS_DIR ``` -------------------------------- ### Sample TAO EfficientDet Training Command Source: https://docs.nvidia.com/tao/tao-toolkit/text/cv_finetuning/tensorflow_2/efficientdet_tf2.html Example of how to initiate the training process for an EfficientDet model using the TAO launcher, specifying the experiment specification file and the number of GPUs to use. ```bash tao model efficientdet_tf2 train -e /path/to/spec.yaml num_gpus=2 ``` -------------------------------- ### Example TAO Help Output Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/quick_start_guide/running_via_launcher.html Sample output displayed when running the help command for the TAO launcher. ```text usage: tao [-h] {list,stop,info,dataset,deploy,model} ... Launcher for TAO optional arguments: -h, --help show this help message and exit task_groups: {list,stop,info,dataset,deploy,model} ``` -------------------------------- ### Example Training Status Output Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/cv_finetuning/pytorch/metric_learning_recognition/metric_learning_recognition.html Sample JSON output from the status.json file, indicating the start and successful completion of the training process. ```json {"date": "6/20/2023", "time": "23:11:2", "status": "STARTED", "verbosity": "INFO", "message": "Starting Training Loop."} ... {"date": "6/20/2023", "time": "23:11:22", "status": "SUCCESS", "verbosity": "INFO", "message": "Train finished successfully."} ``` -------------------------------- ### Sample Evaluation Status JSON Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao_deploy/metric_learning_recognition.html An example of the status.json file generated during evaluation, showing the start and success status of the ml_recog evaluation process. ```json {"date": "3/30/2023", "time": "6:7:14", "status": "STARTED", "verbosity": "INFO", "message": "Starting ml_recog evaluation."} {"date": "3/30/2023", "time": "6:7:24", "status": "SUCCESS", "verbosity": "INFO", "message": "Evaluation finished successfully."} ``` -------------------------------- ### Download and Run Jupyter Notebooks Source: https://docs.nvidia.com/tao-toolkit-api/hosted-ea/jupyter_notebook_examples.html Use these commands to download the TAO API notebooks, install necessary packages, and start the Jupyter Notebook server. ```bash wget https://api-ea4.tao.ngc.nvidia.com/tao_api_notebooks.zip sudo apt-get update sudo apt-get install unzip unzip tao_api_notebooks.zip pip3 install jupyter notebook jupyter notebook --ip 0.0.0.0 ``` -------------------------------- ### Object Detection Workflow Source: https://docs.nvidia.com/tao/tao-toolkit/text/tao_toolkit_api/api_remote_client.html An example of initiating an object detection workflow using the v2 API. This snippet shows how to get base experiments and training schemas, similar to the classification workflow. ```bash # Complete object detection workflow with v2 API export WORKSPACE_ID="workspace_789" export DATASET_URI="://my-bucket/coco-dataset" # aws://, azure://, lustre://, file://, seaweedfs:// # 1. Get base experiment and training schema tao rtdetr list-base-experiments --filter-param network_arch=rtdetr --output json | jq . export SELECTED_PTM_ID="ptm_id_from_list" # Get training schema and save to file tao rtdetr get-job-schema --action train --base-experiment-id "$SELECTED_PTM_ID" --output @train_spec.yaml # Edit train_spec.yaml as needed. For example, update "epochs": 100 and "learning_rate": 0.001 ``` -------------------------------- ### Sample ~/.tao_mounts.json configuration Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao_launcher.html A complete example of the configuration file defining mounts, environment variables, and Docker options. ```json { "Mounts": [ { "source": "/path/to/your/data", "destination": "/workspace/tao-experiments/data" }, { "source": "/path/to/your/local/results", "destination": "/workspace/tao-experiments/results" }, { "source": "/path/to/your/local/specs", "destination": "/workspace/tao-experiments/specs" } ], "Envs": [ { "variable": "CUDA_DEVICE_ORDER", "value": "PCI_BUS_ID" } ], "DockerOptions": { "shm_size": "16G", "ulimits": { "memlock": -1, "stack": 67108864 }, "user": "1000:1000", "ports": { "8888": "8888" } } } ``` -------------------------------- ### Sample Usage for TAO Launcher Evaluation Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao_deploy/deformable_detr.html An example command demonstrating how to run evaluation with the TAO launcher, specifying default variables for the spec file, results directory, and engine file. ```bash tao deploy deformable_detr evaluate -e $DEFAULT_SPEC results_dir=$RESULTS_DIR \ evaluate.trt_engine=$ENGINE_FILE ``` -------------------------------- ### Get Base Experiment ID and Training Specs Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/cv_finetuning/pytorch/depth_estimation/monocular_depth_estimation.html Retrieves the base experiment ID and default training specifications for a monocular depth estimation model using the FTMS client. Ensure jq is installed for JSON parsing. ```bash # Get the training spec BASE_EXPERIMENT_ID=$(tao monocular_depth_estimation list-base-experiments | jq -r '.[0].id') TRAIN_SPECS=$(tao monocular_depth_estimation get-job-schema --action train --base-experiment-id $BASE_EXPERIMENT_ID | jq -r '.default') ``` -------------------------------- ### TAO Launcher Training with Overrides Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/embedding/clip/training.html Example of running CLIP model training with TAO Launcher, demonstrating how to specify the experiment file and override parameters such as the number of GPUs and the results directory. ```bash tao model clip train -e /path/to/experiment_spec.yaml \ train.num_gpus=4 \ train.num_epochs=50 \ results_dir=/results/clip_run1 ``` -------------------------------- ### Sample SSD Model Training Command Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/cv_finetuning/tensorflow_1/object_detection/ssd.html An example of how to execute the SSD model training command, specifying the number of GPUs, the experiment specification file, the results directory, and the encryption key. ```bash tao model ssd train --gpus 2 -e /path/to/spec.txt -r /path/to/result -k $KEY ``` -------------------------------- ### Install NVIDIA GPU Driver Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao_toolkit_api/api_setup.html Installs NVIDIA GPU driver version 570 on Ubuntu. A reboot is required after installation. ```bash #install nv driver sudo apt-get update && sudo apt-get install nvidia-driver-570 sudo reboot ``` -------------------------------- ### Install Helm 3 Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao_toolkit_api/api_deployment.html Installs Helm version 3 on your system. This script downloads and executes the official Helm installation script. ```bash #install helm curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 chmod 700 get_helm.sh ./get_helm.sh ``` -------------------------------- ### Sample EfficientDet Training Command Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/cv_finetuning/tensorflow_2/efficientdet_tf2.html A basic example of executing the training command with an experiment specification and GPU configuration. ```bash tao model efficientdet_tf2 train -e /path/to/spec.yaml num_gpus=2 ``` -------------------------------- ### Start the Triton Server Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/cv_finetuning/pytorch/pose_classification/pose_classification.html Initializes the Triton inference server to enable engine generation. ```bash bash scripts/start_server.sh ``` -------------------------------- ### Install nvidia-tao-client Package Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/faqs.html Use these commands to install the latest TAO client package. Uninstall the previous `nvidia-tlt` if it was previously installed in your virtual environment. ```bash pip3 uninstall nvidia-tlt # This is only required if you had previously installed nvidia-tlt in your # Virtual environment. pip3 install nvidia-tao-client ``` -------------------------------- ### Sample Training Command Source: https://docs.nvidia.com/tao/tao-toolkit/text/cv_finetuning/pytorch/object_detection/grounding_dino.html Example usage of the training command specifying an experiment configuration file. ```bash tao grounding_dino model train -e /path/to/spec.yaml ``` -------------------------------- ### Install IPython Kernel for Conda Environment Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/quick_start_guide/running_via_launcher.html Install the ipykernel package and then install the kernel for the 'launcher' Conda environment. This ensures notebooks use the correct Python kernel. ```bash python -m pip install ipykernel python -m ipykernel install --user --name launcher --display-name "launcher" ``` -------------------------------- ### Evaluation Configuration Example Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/cv_finetuning/pytorch/image_classification_pyt.html Example configuration file for model evaluation. ```yaml evaluate: checkpoint: /path/to/model.pth ``` -------------------------------- ### Verify GPU Operator Installation Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao_toolkit_api/api_deployment.html Checks if the GPU operator pods are running and lists installed Helm charts. This command is useful for verifying the installation on various Kubernetes platforms. ```bash kubectl get pods -n gpu-operator helm list ``` -------------------------------- ### Example: Run Annotation Slice Command Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/data_services/annotations.html An example demonstrating how to use the `slice` command in Data Services with a specified experiment file. ```bash tao dataset annotations slice -e /path/to/spec.yaml ``` -------------------------------- ### Install NVIDIA GPU Operator Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao_toolkit_api/api_deployment.html Installs the NVIDIA GPU Operator using Helm. This command adds the NVIDIA Helm repository, updates it, and then installs the operator in the 'gpu-operator' namespace. ```bash #install gpu operator helm repo add nvidia https://helm.ngc.nvidia.com/nvidia \ && helm repo update helm install --wait --generate-name \ -n gpu-operator --create-namespace \ nvidia/gpu-operator \ --version=v25.3.0 \ --set driver.enabled=false \ --set toolkit.enabled=false ``` -------------------------------- ### Install TensorBoard Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tensorboard_visualization.html Use pip to install the TensorBoard package in your Python environment. ```bash python -m pip install tensorboard ``` -------------------------------- ### Example: Running SSD Inference Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/cv_finetuning/tensorflow_1/object_detection/ssd.html An example of using the 'inference' command with SSD models. Ensure all required paths and the model key are correctly provided. ```bash tao model ssd inference -i /path/to/input/images_dir -o /path/to/output/dir -m /path/to/trained_tlt_ssd_model -k -e /path/to/ssd_spec.txt ``` -------------------------------- ### Sample Evaluation Command (TAO Launcher) Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/cv_finetuning/tensorflow_2/efficientdet_tf2.html Example of how to use the `evaluate` command with the TAO launcher, specifying the experiment specification file. ```bash tao model efficientdet_tf2 evaluate -e /path/to/spec.yaml ``` -------------------------------- ### Example Pruning Command Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/cv_finetuning/tensorflow_1/image_classification.html Example usage of the pruning command with specific parameters. ```bash tao model classification_tf1 prune -m /workspace/output/weights/resnet_003.tlt -o /workspace/output/weights/resnet_003_pruned.tlt -eq union -pth 0.7 -k $KEY ``` -------------------------------- ### Install AWS CLI Source: https://docs.nvidia.com/tao/tao-toolkit/text/tao_toolkit_api/api_setup.html Downloads and installs the AWS Command Line Interface for Linux. ```bash #install aws cli curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" unzip awscliv2.zip sudo ./aws/install ``` -------------------------------- ### Create New Experiment Source: https://docs.nvidia.com/tao-toolkit-api/hosted-ea/tao_rest_api.html Initializes a new experiment using specified network architecture, datasets, and cloud configuration. ```bash EXPERIMENT_ID=$(curl -s -X POST \ $BASE_URL/orgs/ea-tlt/experiments \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $TOKEN" \ -d '{ "network_arch": "dino", "encryption_key": "tlt_encode", "checkpoint_choose_method": "best_model", "train_datasets": ["'"$TRAIN_DATASET_ID"'"], "eval_dataset": "'"$EVAL_DATASET_ID"'", "inference_dataset": "'"$EVAL_DATASET_ID"'", "calibration_dataset": "'"$TRAIN_DATASET_ID"'", "docker_env_vars": {}, "base_experiment": ["'"$BASE_EXPERIMENT_ID"'"], "cloud_details": { "cloud_type": "aws", "cloud_specific_details": { "cloud_region": "us-west-1", "cloud_bucket_name": "bucket_name", "access_key": "access_key", "secret_key": "secret_key" } } }' | jq -r '.id') echo $EXPERIMENT_ID ``` -------------------------------- ### Install TAO Deploy via Pip Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao_deploy/tao_deploy_overview.html Install the nvidia-tao-deploy package in a Python environment. ```bash pip install nvidia-tao-deploy ``` -------------------------------- ### Example of Using the Prune Command Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/cv_finetuning/tensorflow_1/semantic_segmentation/unet.html An example demonstrating how to use the `prune` task with specific parameters for equalization and pruning threshold. The key is provided as a shell variable. ```bash tao model unet prune -e -m -o -eq union -pth 0.7 -k $KEY ``` -------------------------------- ### Install OpenSSL Dependency Source: https://docs.nvidia.com/tao/tao-toolkit/6.26.03/text/tao-converter/tao_converter_deformable_detr.html Install the required OpenSSL development package on the target machine. ```bash sudo apt-get install libssl-dev ```