GPUhub

https://docs.gpuhub.com
Admin
GPUhub is a cloud platform offering GPU-accelerated container instances, AI gateway with unified...
Tokens:75,646
Snippets:885
Trust Score:6.3
Update:4 months ago
Show doc for...
Tokens:
Raw
### Starting Visdom with Custom Port

Source: https://docs.gpuhub.com/best-practices/visdom

This command starts the Visdom server on a specified custom port (e.g., 6006). Ensure Visdom is installed first using 'pip install visdom'.

```bash
# If Visdom is not installed, first execute pip install visdom to install it
# visdom -port 6006
Checking for scripts.
It's Alive!
INFO:root:Application Started
You can navigate to http://container-7e5111bf0c-8a82bc6b:6006
```

--------------------------------

### Example: Run a command in a screen session

Source: https://docs.gpuhub.com/container-instance/run-in-background

Demonstrates the workflow of starting a screen session, running a command, detaching, listing sessions, and reattaching. This illustrates how to keep processes running in the background.

```bash
# Start a new screen session
screen

# Run the command
ping www.huggingface.co

# Detach from the session (Ctrl + a + d)

# Find the detached session
screen -ls

# Reattach to the session
screen -r 12345

# You will see that `ping www.huggingface.co` is still running.
```

--------------------------------

### Install VNC and Dependencies

Source: https://docs.gpuhub.com/container-instance/remote-desktop

Installs essential packages, TurboVNC, and libjpeg-turbo on a Debian-based system. It also starts the VNC server with specified configurations.

```bash
# Install basic dependency packages
apt update && apt install -y libglu1-mesa-dev mesa-utils xterm xauth x11-xkb-utils xfonts-base xkb-data libxtst6 libxv1

# Install libjpeg-turbo and turbovnc
export TURBOVNC_VERSION=2.2.5
export LIBJPEG_VERSION=2.0.90
wget http://aivc.ks3-cn-beijing.ksyun.com/packages/libjpeg-turbo/libjpeg-turbo-official_${LIBJPEG_VERSION}_amd64.deb
wget http://aivc.ks3-cn-beijing.ksyun.com/packages/turbovnc/turbovnc_${TURBOVNC_VERSION}_amd64.deb
dpkg -i libjpeg-turbo-official_${LIBJPEG_VERSION}_amd64.deb
dpkg -i turbovnc_${TURBOVNC_VERSION}_amd64.deb
rm -rf *.deb

# Start the VNC server, which may involve configuring a VNC password (note that this is not the instance's account password). If there's an error saying xauth is not found, install it again with apt install xauth.
rm -rf /tmp/.X1*  # If starting again, remove the temporary files from the last time, otherwise it won't start properly
USER=root /opt/TurboVNC/bin/vncserver :1 -desktop X -auth /root/.Xauthority -geometry 1920x1080 -depth 24 -rfbwait 120000 -rfbauth /root/.vnc/passwd -fp /usr/share/fonts/X11/misc/,/usr/share/fonts -rfbport 6006

# Check if it has started; if there's a vncserver process, it means it has started
ps -ef | grep vnc
```

--------------------------------

### Install Dependencies

Source: https://docs.gpuhub.com/faqs/sd-image-generation-memory-leak

Install the necessary development package for Google Performance Tools before proceeding.

```bash
apt install libgoogle-perftools-dev
```

--------------------------------

### Python Example for Checking Balance

Source: https://docs.gpuhub.com/api-reference/api-elastic-deployment

This Python script demonstrates how to make a GET request to the /api/v1/dev/deployment/ddp/overview endpoint to check the duration package balance. Ensure you replace 'your_token_here' and 'your_deployment_uuid_here' with your actual credentials and deployment UUID.

```python
import requests

headers = {"Authorization": "your_token_here"}
url = "https://api.gpuhub.com/api/v1/dev/deployment/ddp/overview"
params = {"deployment_uuid": "your_deployment_uuid_here"}

response = requests.get(url, params=params, headers=headers)
print(response.json())
```

--------------------------------

### Install System Dependencies with Apt-get

Source: https://docs.gpuhub.com/environment/dependency-installation

Install system packages using apt-get. Remember to update the package list first.

```bash
# Example for installing zip:
apt-get update    # Only needs to be run once, not every time  
apt-get install -y zip
```

--------------------------------

### Test TensorFlow Installation

Source: https://docs.gpuhub.com/environment/miniconda

A simple Python script to verify the TensorFlow installation by printing a 'Hello, TensorFlow!' message and showing device placement.

```python
# Simple test using Python after installation
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
print(sess.run(hello))
```

--------------------------------

### Python SDK Example

Source: https://docs.gpuhub.com/ai-gateway/llms/claude-sonnet-4-6-cc

This Python code snippet demonstrates how to use the Anthropic SDK to interact with the `claude-sonnet-4-6-cc` model. It includes installation instructions, client initialization with API key and base URL, and an example of sending a message and printing the response.

```APIDOC
## Python SDK Integration

### Description
Integrate with the `claude-sonnet-4-6-cc` model using the Anthropic Python SDK. This example shows how to initialize the client, send a message, and process the response.

### Installation
```bash
pip install anthropic
```

### Usage

```python
from anthropic import Anthropic

client = Anthropic(
    api_key="YOUR_API_KEY",
    base_url="https://api.gpuhub.com/api/v1/anthropic",
)

message = client.messages.create(
    model="claude-sonnet-4-6-cc",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Hello! Review this implementation plan."}
    ],
)

print(message.content[0].text)
```
```

--------------------------------

### Start Lightweight Proxy Service

Source: https://docs.gpuhub.com/best-practices/exposing-multiple-services

Execute the `proxy_in_instance` program to start the lightweight proxy service, which will begin forwarding requests based on the `config.yaml` configuration.

```bash
# Start the proxy service:
./proxy_in_instance

```

--------------------------------

### Install screen

Source: https://docs.gpuhub.com/container-instance/run-in-background

Install the 'screen' utility on your system. This is a prerequisite for using screen to manage terminal sessions.

```bash
apt-get update && apt-get install -y screen
```

--------------------------------

### Install cuDNN Files

Source: https://docs.gpuhub.com/environment/cuda-cudnn

Move the dynamic libraries and header files from the unzipped cuDNN download to the appropriate CUDA installation directories.

```bash
mv cuda/include/* /usr/local/cuda/include/
chmod +x cuda/lib64/* && mv cuda/lib64/* /usr/local/cuda/lib64/
```

--------------------------------

### Install and Use Zip for Archives

Source: https://docs.gpuhub.com/best-practices/linux-basics

Install the zip utility and use it to compress directories into .zip files or decompress existing .zip archives.

```bash
#If you don't have zip installed, you can install it using:
apt-get update && apt-get install -y zip
#To compress a directory into a .zip file:
user@host:/tmp$ zip -r dir.zip test_directory/   # Compress the test_directory into a file named dir.zip
#To decompress a .zip file:
user@host:/tmp$ unzip dir.zip   # Decompress the dir.zip file
```

--------------------------------

### Install Pip if Missing

Source: https://docs.gpuhub.com/faqs/jupyterlab-fails-to-open

If the 'pip' command is not found, run this command to install it using conda.

```bash
conda install pip
```

--------------------------------

### Start Claude Code

Source: https://docs.gpuhub.com/ai-gateway/ides-agents/claude-code

Launch Claude Code from your project directory after setting the necessary environment variables.

```bash
claude
```

--------------------------------

### Install MPICH on Ubuntu 20.04

Source: https://docs.gpuhub.com/best-practices/mpi

Installs the MPICH implementation of MPI. Avoid installing the 'openmpi' package due to known issues.

```bash
# Note: Do not install or use the openmpi package, as it has issues.
apt update && apt-get install mpich
```

--------------------------------

### Install R and Dependencies

Source: https://docs.gpuhub.com/best-practices/rstudio

Update the package list and install R along with necessary dependencies using apt-get. This command ensures all required libraries for R to function correctly are present.

```bash
apt-get update
apt-get install -y r-base libgbm1 lib32gcc-s1 lib32stdc++6 libc6-i386 libclang-10-dev libclang-common-10-dev libclang-dev libclang1-10 libgc1c2 libllvm10 libobjc-9-dev libobjc4 libxcb-xkb1 libxkbcommon-x11-0
```

--------------------------------

### Python SDK Example

Source: https://docs.gpuhub.com/ai-gateway/llms/claude-opus-4-6

Example of how to use the Anthropic Python SDK to interact with the Claude Opus 4.6 model via AI Gateway.

```APIDOC
## Python SDK Usage

### Description
This example demonstrates how to initialize the Anthropic Python client and send a message to the `claude-opus-4-6` model.

### Method
`client.messages.create`

### Parameters
#### Request Body
- **model** (string) - Required - The model to use for the request, e.g., `"claude-opus-4-6"`
- **max_tokens** (integer) - Required - The maximum number of tokens to generate in the response.
- **messages** (array) - Required - A list of message objects, where each object has a `role` (user or assistant) and `content`.

### Request Example
```python
from anthropic import Anthropic

client = Anthropic(
    api_key="YOUR_API_KEY",
    base_url="https://api.gpuhub.com/api/v1/anthropic",
)

message = client.messages.create(
    model="claude-opus-4-6",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Hello! Please introduce yourself briefly."}
    ],
)

print(message.content[0].text)
```

### Response
#### Success Response (200)
- **content** (array) - The response content from the model.
```

--------------------------------

### Install Vulkan Dependencies

Source: https://docs.gpuhub.com/best-practices/vulkan

Install necessary Vulkan tools and libraries on your system. This includes `vulkan-tools`, `libvulkan1`, `libsm6`, and `libegl1`.

```bash
apt update && apt install -y vulkan-tools vulkan-tools libvulkan1 libsm6 libegl1
```

--------------------------------

### Install Python Dependencies with Pip

Source: https://docs.gpuhub.com/environment/dependency-installation

Install multiple Python packages at once using pip.

```bash
# Using pip: For example,
pip install opencv-python scipy numpy Pillow
```

--------------------------------

### SSH Tunneling Example Interaction

Source: https://docs.gpuhub.com/best-practices/visdom

This is an example of the interactive prompts you might encounter when establishing an SSH tunnel for Visdom. It includes host key verification and password entry.

```bash
@seeta:~$ ssh -CNg -L 8097:127.0.0.1:8097 root@region-1.gpuhub.com -p 37881
The authenticity of host '[region-1.gpuhub.com]:37881 ([120.92.100.9]:37881)' can't be established.
ECDSA key fingerprint is SHA256:eNwieuMEZiNCONaDJXFsAJRBfTnU7f3gQWZ+A2n1j1o.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[region-1.gpuhub.com]:37881,[120.92.100.9]:37881' (ECDSA) to the list of known hosts.
root@region-1.gpuhub.com's password:
```

--------------------------------

### Obtain Balances API Response Example

Source: https://docs.gpuhub.com/api-reference/api-obtain-balances

This is an example of a successful response from the Obtain Balances API, showing the structure of the returned data.

```javascript
{
    "code": "Success",
    "data": {
        "assets": 1000,
        "accumulate": 1000,
        "voucher_balance": 1000
    },
    "msg": ""
}
```

--------------------------------

### Install and Use Arc Compression Tool

Source: https://docs.gpuhub.com/data/compress-decompress

Download, install, and use the 'arc' tool for compressing and decompressing various archive formats. This tool supports .zip, .tar, .rar, and .7z.

```bash
# Download and Install
curl -L -o /usr/bin/arc http://gpuhub-public.ks3-sig-sig1.ksyun.com/tool/arc && chmod +x /usr/bin/arc

# Compress/Package
arc compress xxx.zip path/to/directory

# Decompress/Unpack
arc decompress xxx.zip

# Or decompress to a specific directory:
arc decompress xxx.zip path/to/directory
```

--------------------------------

### Verify R Installation

Source: https://docs.gpuhub.com/best-practices/rstudio

Enter the R interpreter by typing 'R' in the terminal. This snippet shows the R prompt and a sample command '1 + 1' to confirm that R is installed and functioning.

```r
# R

R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> 1 + 1
[1] 2
> 

```

--------------------------------

### SSH Tunnel Command Example

Source: https://docs.gpuhub.com/best-practices/ssh-tunnel

This is an example of an SSH tunnel command, demonstrating how to forward a specific port from the instance to your local machine. Replace placeholders with your actual instance details.

```bash
ssh -CNg -L 6006:127.0.0.1:6006 root@region-3.gpuhub.com -p 48332
```

--------------------------------

### Redeem Voucher Code Example

Source: https://docs.gpuhub.com/promotions/coupons-vouchers

This is an example of a voucher code that can be redeemed in the GPUhub console for an instant balance. Ensure you copy the code exactly as provided.

```text
A6I0Y54Q1L
```

--------------------------------

### List Containers Request Body Example

Source: https://docs.gpuhub.com/api-reference/api-elastic-deployment

This is an example of a request body for the list containers endpoint. It specifies the deployment UUID and pagination parameters. Ensure that 'deployment_uuid' is provided for all requests.

```JSON
{
    "deployment_uuid": "da497aea1eb8343", 
    "page_index": 1, 
    "page_size": 10
}
```

--------------------------------

### Stop Deployment (Python)

Source: https://docs.gpuhub.com/api-reference/api-elastic-deployment

Python example for stopping a deployment. Ensure the correct URL, headers, and request body are used.

```python

import requests
headers = {
    "Authorization": "your token",
    "Content-Type": "application/json"
}
url = "https://api.gpuhub.com/api/v1/dev/deployment/operate"
body = {
    "deployment_uuid": "5be3045703152b9",
    "operate": "stop"
}
response = requests.put(url, json=body, headers=headers)
print(response.content.decode())
```

--------------------------------

### Success Response Example

Source: https://docs.gpuhub.com/api-reference/api-elastic-deployment

This JSON object represents a successful API response when creating a deployment.

```json
{
   "code": "Success",
   "msg": "",
   "data": {
       "deployment_uuid": "833f1cd5a764fa3"
   }
}
```

--------------------------------

### Start Codex Command-Line Assistant

Source: https://docs.gpuhub.com/ai-gateway/ides-agents/codex

Launch the Codex command-line coding assistant. Ensure the OPENAI_BASE_URL is set before running this command.

```bash
codex
```

--------------------------------

### Create and Change Directories

Source: https://docs.gpuhub.com/best-practices/linux-basics

Use `mkdir` to create a new directory and `cd` to navigate into it. `..` refers to the parent directory and `./` refers to the current directory.

```bash
user@host:/tmp$ mkdir test_dir   # Create a new directory named test_dir
user@host:/tmp$ cd test_dir/     # Enter the test_dir directory
user@host:/tmp/test_dir$
user@host:/tmp/test_dir$ cd ../test_dir/   # Go to the test_dir directory which is in the parent directory
user@host:/tmp$
```

--------------------------------

### Python SDK Example for Claude Opus 4.6 Claude Code

Source: https://docs.gpuhub.com/ai-gateway/llms/claude-opus-4-6-cc

Example of how to use the Anthropic Python SDK to interact with the `claude-opus-4-6-cc` model. Ensure you have the SDK installed and replace 'YOUR_API_KEY' with your actual API key.

```python
# Install the SDK first if needed:
# pip install anthropic

from anthropic import Anthropic

client = Anthropic(
    api_key="YOUR_API_KEY",
    base_url="https://api.gpuhub.com/api/v1/anthropic",
)

message = client.messages.create(
    model="claude-opus-4-6-cc",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Hello! Please help me plan a coding task."}
    ],
)

print(message.content[0].text)
```

--------------------------------

### Check nvidia-opencl Library

Source: https://docs.gpuhub.com/best-practices/opencl

Use this command to verify if the nvidia-opencl dynamic link library is installed and accessible on your system. If this command returns no output, there might be an issue with your system's environment setup.

```bash
ldconfig -p | grep nvidia-opencl
```

--------------------------------

### Start TensorBoard with Custom Log Directory

Source: https://docs.gpuhub.com/best-practices/tensorboard

Execute this command to launch a new TensorBoard instance. Specify your desired log directory using the --logdir flag. The --port flag can be used to change the access port if needed.

```bash
tensorboard --port 6007 --logdir /path/to/your/tf-logs/directory
```

--------------------------------

### Check Conda Installed CUDA/cuDNN Versions

Source: https://docs.gpuhub.com/environment/cuda-cudnn

Check the versions of `cudatoolkit` and `cudnn` installed via Conda by listing installed packages and filtering the output.

```bash
conda list | grep cudatoolkit
cudatoolkit               10.1.243             h6bb024c_0    defaults

conda list | grep cudnn
cudnn                     7.6.5                cuda10.1_0    defaults
```

--------------------------------

### Scale Deployment Replicas (Python)

Source: https://docs.gpuhub.com/api-reference/api-elastic-deployment

Example of scaling deployment replicas using Python's requests library. Requires authentication token and correct content type.

```python
import requests

headers = {
    "Authorization": "your token",
    "Content-Type": "application/json"
}

url = "https://api.gpuhub.com/api/v1/dev/deployment/replica_num"

body = {
    "deployment_uuid": "5be3045703152b9",
    "replica_num": 16
}

response = requests.put(url, json=body, headers=headers)
print(response.json())
```

--------------------------------

### Install Python Dependencies with Conda

Source: https://docs.gpuhub.com/environment/dependency-installation

Install a Python package using Conda.

```bash
# Using conda: For example,
conda install numpy
```

--------------------------------

### Install TensorFlow 1.15.0 with GPU Support

Source: https://docs.gpuhub.com/environment/miniconda

Install TensorFlow 1.15.0 using Conda within an activated virtual environment. Conda automatically handles the installation of compatible CUDA and cuDNN versions.

```bash
# After switching to the Conda virtual environment
conda install tensorflow-gpu==1.15.0
# Conda will automatically resolve and install the required CUDA and cuDNN versions for TensorFlow 1.15.0
```

--------------------------------

### Make Proxy Executable

Source: https://docs.gpuhub.com/best-practices/exposing-multiple-services

Grant execute permissions to the downloaded `proxy_in_instance` program before running it.

```bash
# If the proxy program does not have executable permissions, add the permissions
chmod +x proxy_in_instance

```

--------------------------------

### Install FileZilla on Linux

Source: https://docs.gpuhub.com/best-practices/filezilla

Use this command to install FileZilla on Debian-based Linux distributions.

```bash
sudo apt install filezilla
```

--------------------------------

### Python SDK for Qwen3.5-397B-A17B Chat Completions

Source: https://docs.gpuhub.com/ai-gateway/llms/qwen3-5-397b-a17b

This Python script demonstrates how to use the OpenAI SDK to interact with the Qwen3.5-397B-A17B model. Replace 'YOUR_API_KEY' with your actual API key and ensure the SDK is installed.

```python
# Install the SDK first if needed:
# pip install openai

from openai import OpenAI

client = OpenAI(
    base_url="https://api.gpuhub.com/api/v1",
    api_key="YOUR_API_KEY",
)

stream = client.chat.completions.create(
    model="Qwen3.5-397B-A17B",
    messages=[
        {"role": "user", "content": "Hello! Please create a concise project risk checklist."}
    ],
    stream=True,
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
```

--------------------------------

### Configure and Prepare Gromacs Simulation

Source: https://docs.gpuhub.com/best-practices/gromacs

Creates a simulation parameter file ('options.mdp') and then uses 'gmx grompp' to generate a portable run input file ('esculentin.tpr'). The 'options.mdp' file specifies simulation settings like integration method, steps, and PME settings. 'gmx grompp' checks for errors and warnings, with a maximum of 100 warnings allowed.

```bash
cat > options.mdp << EOF
integrator        = md
nsteps            = 50000000
stxout           = 50000
coulombtype       = PME
courierspacing    = 0.15
tcoupl            = v-rescale
tau-t             = 0.2 0.2
ref-t             = 298.15 298.15
tc-grps           = Protein Non-protein
pcoupl            = berendsen
compressibility   = 4.5e-5
tau-p             = 0.2
ref-p             = 1.0
EOF
gmx grompp -f options.mdp -c solvated.gro -p esculentin.top -o esculentin.tpr -maxwarn 100
```

--------------------------------

### Fix Broken Dependencies

Source: https://docs.gpuhub.com/best-practices/rstudio

If dependency issues arise during the installation of R or other packages, this command attempts to resolve them by fixing broken installations.

```bash
apt -y --fix-broken install
```

--------------------------------

### Create Job Deployment

Source: https://docs.gpuhub.com/api-reference/api-elastic-deployment

This snippet demonstrates creating a Job deployment for batch processing tasks. Specify the number of replicas and parallelism.

```python
# Example 2: Create a Job deployment (batch processing)
{
    "name": "batch-training-job",
    "deployment_type": "Job",
    "replica_num": 4,
    "parallelism_num": 1,
    "reuse_container": True,
    "container_template": {
        "dc_list": ["Singpore-A"],
        "gpu_name_set": ["RTX 5090"],
        "gpu_num": 1,
        "cuda_v_from": 118,
        "cuda_v_to": 128,
        "cpu_num_from": 8,
        "cpu_num_to": 64,
        "memory_size_from": 32,
        "memory_size_to": 256,
        "cmd": "python train.py",
        "price_from": 100,
        "price_to": 9000,
        "image_uuid": "image-xxxxxxxx"
    }
}
```

--------------------------------

### Download Lightweight Proxy Program

Source: https://docs.gpuhub.com/best-practices/exposing-multiple-services

Download the `proxy_in_instance` executable, which will be used to manage route-based forwarding.

```bash
wget https://autodl-public.ks3-cn-beijing.ksyuncs.com/tool/api-proxy/proxy_in_instance

```

--------------------------------

### Install PyTorch and Torchvision

Source: https://docs.gpuhub.com/environment/dependency-installation

Install specific versions of PyTorch, torchvision, and torchaudio using Conda, specifying the CUDA toolkit version.

```bash
# PyTorch: Find the appropriate version from Previous PyTorch Versions. For example:
conda install pytorch==1.9.0 torchvision==0.10.0 torchaudio==0.9.0 cudatoolkit=11.3 -c pytorch -c conda-forge
```

--------------------------------

### Configure Proxy Routing

Source: https://docs.gpuhub.com/best-practices/exposing-multiple-services

Create a `config.yaml` file to define the routing rules for the lightweight proxy. This example forwards requests from `/v1/*` to port 2000 and `/v2/*` to port 3000.

```yaml
proxies:
  - host_and_port: http://127.0.0.1:2000  # Address of the service to proxy
    route_path: /v1/*                    # Route path to forward to this address

  - host_and_port: http://127.0.0.1:3000  # Multiple entries can be set to forward to different hosts
    route_path: /v2/*

```

--------------------------------

### List Files and Directories

Source: https://docs.gpuhub.com/best-practices/linux-basics

Use `ls` to list files and directories. The `-l` option provides detailed information including permissions, owner, group, size, and modification time.

```bash
user@host:/tmp/test_dir$ ls    # List files and directories in the current directory
a.txt  b
user@host:/tmp/test_dir$ ls -l  # List detailed information about files and directories: permissions, Owner, Group, and creation/update time
total 4
-rw-rw-r-- 1 root root 0 Nov 9 10:50 a.txt
drwxrwxr-x 2 root root 4096 Nov 9 10:50 b
```

--------------------------------

### Install TensorFlow GPU Version

Source: https://docs.gpuhub.com/environment/dependency-installation

Install a specific GPU-enabled version of TensorFlow using pip, by providing a direct wheel file URL.

```bash
# TensorFlow: Find the corresponding version link from Install TensorFlow with pip. For example:
pip install https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-2.6.0-cp38-cp38-manylinux2010_x86_64.whl
```

--------------------------------

### Example: Running a Python Application Directly

Source: https://docs.gpuhub.com/elastic-deployment/elastic-deployment

This method is recommended when the application's lifecycle can be directly tied to the container's lifecycle. The application runs in the foreground.

```shell
# Method 1 example:
python app.py
```

--------------------------------

### SSH Login Command for Instance

Source: https://docs.gpuhub.com/data/download-data

Use this command to establish an SSH connection to your instance. Ensure you replace the port number and remote address with your instance's specific details.

```bash
ssh -p 35394 root@region-1.gpuhub.com
```

--------------------------------

### Generate Content with Gemini 3.1 Pro Preview using Python SDK

Source: https://docs.gpuhub.com/ai-gateway/llms/gemini-3-1-pro-preview

This Python code snippet demonstrates how to initialize the Google GenAI SDK client with a specific API key and base URL for the AI Gateway, and then generate content using the gemini-3.1-pro-preview model.

```python
# Install the SDK first if needed:
# pip install google-genai

from google import genai

client = genai.Client(
    api_key="YOUR_API_KEY",
    http_options={"base_url": "https://api.gpuhub.com/api/v1/gemini"},
)

response = client.models.generate_content(
    model="gemini-3.1-pro-preview",
    contents="Hello! Please summarize the key tradeoffs of long-context models.",
)

print(response.text)
```

--------------------------------

### Compile and Run MPI C Program

Source: https://docs.gpuhub.com/best-practices/mpi

Demonstrates how to compile a simple MPI C program using `mpicc` and then run the resulting executable with `mpirun`. Ensure to use the absolute path for the executable.

```c
#include <stdio.h>
#include <mpi.h>
int main (int argc, char** argv) {
    int size, rank;
    MPI_Init(&argc, &argv);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &size);
    printf("size = %d rank = %d\n", size, rank);
    MPI_Finalize();
}
```

```bash
# Create a file named `test.c` with the following content:
#include <stdio.h>
#include <mpi.h>
int main (int argc, char** argv) {
    int size, rank;
    MPI_Init(&argc, &argv);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &size);
    printf("size = %d rank = %d\n", size, rank);
    MPI_Finalize();
}

# Compile the code to generate an executable file named `a.out`.
mpicc test.c

# Run the executable. Note that you need to use the absolute path since `a.out` is not in the PATH environment variable. For example, if the test directory is `/root`:
mpirun -n 1 /root/a.out
```

--------------------------------

### Check TensorFlow GPU Version

Source: https://docs.gpuhub.com/faqs/unable-to-call-gpu

Determine if your TensorFlow installation supports GPUs by checking for CUDA build information. This helps identify if a CPU-only version is installed.

```python
import tensorflow as tf
sys_details = tf.sysconfig.get_build_info()
print(sys_details["cuda_version"])
```

--------------------------------

### NCCL Connection Refused Log Example

Source: https://docs.gpuhub.com/container-instance/multi-node-multi-gpu

Example of NCCL debug logs showing 'Connection refused' errors, indicating potential network configuration problems.

```bash
container-xxx:152731:152731 [1] NCCL INFO Bootstrap : Using eth0:172.17.0.4<0>
container-xxx:152731:152731 [1] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation
container-xxx:152731:152731 [1] NCCL INFO NET/IB : No device found.
container-xxx:152731:152731 [1] NCCL INFO NET/Socket : Using [0]eth0:172.17.0.4<0> [1]eth1:172.18.0.16<0>
container-xxx:152731:152731 [1] NCCL INFO Using network Socket
container-xxx:152730:152730 [0] NCCL INFO Bootstrap : Using eth0:172.17.0.4<0>
container-xxx:152730:152730 [0] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation
container-xxx:152730:152730 [0] NCCL INFO NET/IB : No device found.
container-xxx:152730:152730 [0] NCCL INFO NET/Socket : Using [0]eth0:172.17.0.4<0> [1]eth1:172.18.0.16<0>
container-3e581195ae-be8c3f28:152730:152730 [0] NCCL INFO Using network Socket
container-3e581195ae-be8c3f28:152731:152775 [1] NCCL INFO Call to connect returned Connection refused, retrying
container-3e581195ae-be8c3f28:152730:152783 [0] NCCL INFO Call to connect returned Connection refused, retrying
container-3e581195ae-be8c3f28:152731:152775 [1] NCCL INFO Call to connect returned Connection refused, retrying
container-3e581195ae-be8c3f28:152730:152783 [0] NCCL INFO Call to connect returned Connection refused, retrying
```

--------------------------------

### Launch WebUI with tcmalloc

Source: https://docs.gpuhub.com/faqs/sd-image-generation-memory-leak

Start the AUTOMATIC1111 SD WebUI after setting the LD_PRELOAD environment variable. This ensures the memory leak fix is active.

```bash
python launch.py --port 6006
```

--------------------------------

### Create ReplicaSet Deployment

Source: https://docs.gpuhub.com/api-reference/api-elastic-deployment

Use this snippet to create a ReplicaSet deployment, which supports auto-scaling. Configure parameters like GPU type, CPU, memory, and pricing.

```python
import requests

headers = {
    "Authorization": "your token",
    "Content-Type": "application/json"
}

url = "https://api.gpuhub.com/api/v1/dev/deployment"

# Example 1: Create a ReplicaSet deployment (auto-scaling)
body = {
    "name": "my-inference-service",
    "deployment_type": "ReplicaSet",
    "replica_num": 2,
    "reuse_container": True,
    "container_template": {
        "dc_list": ["Singpore-A", "Singapore-B"],
        "gpu_name_set": ["RTX 5090", "RTX Pro 6000"],
        "gpu_num": 1,
        "cuda_v_from": 118,
        "cuda_v_to": 128,
        "cpu_num_from": 4,
        "cpu_num_to": 64,
        "memory_size_from": 16,
        "memory_size_to": 256,
        "cmd": "python main.py",
        "price_from": 100,   # $0.10/hr (price × 1000)
        "price_to": 9000,    # $9.00/hr (price × 1000)
        "image_uuid": "image-xxxxxxxx"
    }
}

response = requests.post(url, json=body, headers=headers)
print(response.content.decode())
```

--------------------------------

### Create and Navigate to Test Directory

Source: https://docs.gpuhub.com/best-practices/gromacs

Creates a new directory named 'test' and changes the current directory to it. This is the first step in setting up a Gromacs simulation.

```bash
mkdir test && cd test
```
Tokens:
Raw