### Setup vLLM Environment

Source: https://thundercompute.com/docs/llms-full.txt

Create and activate a Python virtual environment, then install vLLM with the auto torch backend. Ensure system libraries are up-to-date.

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -U uv
uv pip install vllm --torch-backend=auto
sudo /sbin/ldconfig
```

--------------------------------

### Start Services on Instance

Source: https://thundercompute.com/docs/docs/guides/using-instance-templates

Commands to start Ollama and ComfyUI services. These are example commands and may require adjustments based on your setup.

```shellscript
# Ollama
start-ollama

# ComfyUI
```

--------------------------------

### Tested Setup for Qwen3.6-27B with llama.cpp

Source: https://thundercompute.com/docs/docs/guides/models/qwen3-6-27b

This configuration was used for screenshots and command output in the guide. It specifies the GPU, model file, context length, and runtime.

```text
Thunder Compute GPU: RTX A6000 48 GB
Model file: Qwen3.6-27B-UD-Q4_K_XL.gguf (16.4 GiB)
Context: 32768
Runtime: llama.cpp Docker
Result: Runs successfully on the base template and serves /v1/chat/completions.
```

--------------------------------

### Install Unsloth Studio

Source: https://thundercompute.com/docs/docs/guides/unsloth-studio

Follow these commands to install Unsloth Studio on your created instance. This involves cloning the repository and running the installation script.

```shellscript
git clone https://github.com/unslothai/unsloth-studio.git
cd unsloth-studio
pip install -e .[all]
```

--------------------------------

### Unsloth Studio Installation

Source: https://thundercompute.com/docs/docs/guides/unsloth-studio

This snippet shows the command to install Unsloth Studio. Ensure you have the necessary environment setup.

```bash
pip install unsloth
```

--------------------------------

### Install Unsloth Studio dependencies and installer

Source: https://thundercompute.com/docs/llms-full.txt

Update package lists and install necessary system packages for Unsloth Studio. Then, download and run the official Unsloth Studio installer script.

```bash
sudo apt-get update
sudo apt-get install -y cmake libcurl4-openssl-dev
```

```bash
curl -fsSL https://unsloth.ai/install.sh | sh
```

--------------------------------

### Start Instance Services

Source: https://www.thundercompute.com/docs/guides/using-instance-templates

Initiate the specific service for your deployed instance using dedicated start commands. Required ports are automatically forwarded.

```bash
start-ollama
```

```bash
start-comfyui
```

```bash
start-forgeneo
```

```bash
start-unsloth
```

--------------------------------

### Minimal Wandb Training Example

Source: https://thundercompute.com/docs/llms-full.txt

This Python script initializes a wandb run, simulates a training loop logging metrics like loss and accuracy, and then finishes the run. Ensure wandb is installed.

```python
import wandb
import time

# Initialize wandb
wandb.init(
    project="thunder-resnet",
    name="quick-test",
    config={
        "learning_rate": 0.001,
        "batch_size": 32,
        "epochs": 5,
    },
)

# Simple training loop simulation
for epoch in range(5):
    # Simulate training metrics
    train_loss = 1.0 / (epoch + 1)
    train_acc = 0.5 + epoch * 0.1

    # Log metrics to wandb
    wandb.log({
        "epoch": epoch,
        "train/loss": train_loss,
        "train/accuracy": train_acc,
    })

    time.sleep(0.5)  # Simulate work

wandb.finish()
```

--------------------------------

### Install via Smithery CLI

Source: https://thundercompute.com/docs/docs/guides/mcp-server

Install the Thunder Compute MCP client using the Smithery CLI.

```APIDOC
## Install via Smithery CLI

### Description
Use the Smithery CLI to install the Thunder Compute MCP client.

### Command
```shell
npx @smithery/cli install @thunder-compute/thunder-compute
```
```

--------------------------------

### Run Official Unsloth Studio Installer

Source: https://thundercompute.com/docs/llms-full.txt

Download and execute the official Unsloth Studio installer script from unsloth.ai. This script sets up the Python environment and the 'unsloth' launcher.

```bash
curl -fsSL https://unsloth.ai/install.sh | sh
```

--------------------------------

### Add MCP Server via Interactive Setup (OpenCode)

Source: https://thundercompute.com/docs/docs/guides/mcp-server

Initiate the interactive setup for the OpenCode MCP server by running 'opencode mcp add'. Follow the prompts to enter the server name, type, URL, and OAuth requirements.

```bash
opencode mcp add
```

--------------------------------

### Start Ollama and Download GPT-OSS Model

Source: https://thundercompute.com/docs/docs/guides/gpt-oss-running-locally-on-thunder-compute

Inside the instance, start the Ollama service and download the GPT-OSS 120B model from the Ollama Model Library.

```bash
start-ollama
```

```bash
ollama pull gpt-oss:120b
```

--------------------------------

### Start Instance Services

Source: https://thundercompute.com/docs/docs/guides/using-instance-templates

After connecting, use the appropriate `start-*` command to launch the service for your specific instance template. Required ports are forwarded automatically.

```bash
# Ollama
start-ollama
```

```bash
# ComfyUI
start-comfyui
```

```bash
# Forge Neo
start-forgeneo
```

```bash
# Unsloth
start-unsloth
```

--------------------------------

### Initiate Interactive Port Forwarding

Source: https://thundercompute.com/docs/docs/cli/operations/port-forwarding

Run the 'tnr' command without arguments to start the interactive port forwarding interface. This is useful for a guided setup.

```shellscript
tnr
```

--------------------------------

### Start Development Instance with Ollama Template

Source: https://thundercompute.com/docs/docs/guides/gpt-oss-running-locally-on-thunder-compute

Use this command to launch a cost-effective development instance pre-configured with the Ollama template. Resources like GPU, vCPUs, and storage can be adjusted later.

```shell
thunder compute instances create --gpu a100xl --vcpus 4 --mode development --persistent-disk 200 --template "ollama"
```

--------------------------------

### Run Qwen3-6-27B with Direct Prompt

Source: https://thundercompute.com/docs/docs/guides/models/qwen3-6-27b

Execute a direct prompt to test model loading and GPU generation. Adjust '-c' for context size if memory issues arise.

```bash
llama-cli -m $MODEL -p $PROMPT --no-display-prompt --simple-io -no-cnv -p "$PROMPT"
```

--------------------------------

### Running Unsloth Studio with Docker

Source: https://thundercompute.com/docs/docs/guides/unsloth-studio

This example demonstrates how to run Unsloth Studio using Docker on Thunder Compute. Note that Docker support is experimental.

```bash
docker run --pull=always -v ".:/workspace" --device=/dev/nvidia0:/dev/nvidia0 -e NVIDIA_VISIBLE_DEVICES=0 --name unsloth_studio unsloth/unsloth
```

--------------------------------

### SSH Connection Command Example

Source: https://thundercompute.com/docs/docs/cli/operations/ssh

This command demonstrates how to initiate an SSH connection to an instance. Ensure you replace \'<port>\' with the actual port number and \'ubuntu\' with the correct username if it differs from your template.

```bash
ssh -p <port> ubuntu@<ip>
```

--------------------------------

### Start OpenAI-Compatible API with llama.cpp

Source: https://thundercompute.com/docs/docs/guides/models/qwen3-6-27b

This section describes how to start a persistent llama.cpp server on port 8000, making the Qwen3.6-27B model accessible via an OpenAI-compatible API. It includes the Docker command to run the server and example `curl` commands to check its health and list models.

```APIDOC
## Start An OpenAI-Compatible API

Start a persistent `llama.cpp` server on port `8000`:

```bash
sudo docker rm -f qwen36-llama >/dev/null 2>&1 || true

sudo docker run -d \
  --name qwen36-llama \
  --device nvidia.com/gpu=all \
  -p 8000:8000 \
  -v "$HOME/models/qwen3.6-27b:/models" \
  ghcr.io/ggml-org/llama.cpp:full-cuda --server \
  -m /models/Qwen3.6-27B-UD-Q4_K_XL.gguf \
  -ngl 99 \
  -c 32768 \
  --parallel 1 \
  --host 0.0.0.0 \
  --port 8000 \
  --jinja \
  --reasoning off \
  --temp 0.7 \
  --top-p 0.8 \
  --top-k 20
```

Check that the server is ready:

```bash
curl http://127.0.0.1:8000/health
curl http://127.0.0.1:8000/v1/models
```

The server is ready when `/health` returns `{"status":"ok"}`. You can use `/v1/models` as a quick check that the model is loaded.
```

--------------------------------

### Launch GPU Instance

Source: https://thundercompute.com/docs/docs/guides/gpt-oss-running-locally-on-thunder-compute

This command launches an 80 GB A100 instance on Thunder Compute, suitable for hosting large models like GPT-OSS 120B. Ensure you have a Thunder Compute account and have completed the Quickstart Guide.

```shellscript
tnr create --gpu a100:80gb --name gpt-oss-120b --region us-east-1 --runtime "docker run -it --shm-size=16g -p 8080:8080 -v $(pwd):/workspace -e HUGGING_FACE_HUB_TOKEN=$HUGGING_FACE_HUB_TOKEN ghcr.io/huggingface/text-generation-inference:latest"

```

--------------------------------

### Install Plugin Dependencies

Source: https://thundercompute.com/docs/llms-full.txt

Installs the necessary npm packages and then installs the OpenClaw plugin.

```bash
npm install
openclaw plugins install .
```

--------------------------------

### Create Model Directory and Download Qwen3.6-27B

Source: https://thundercompute.com/docs/docs/guides/models/qwen3-6-27b

Use these commands to create a directory for the model and download the Qwen3.6-27B GGUF file using wget. Ensure you are in the correct parent directory before executing.

```bash
mkdir -p ~/models/qwen3.6-27b
cd ~/models/qwen3.6-27b
wget -O Qwen3.6-27B-UD-Q4_K_XL.gguf \\
 https://huggingface.co/unsloth/Qwen3.6-27B-GGUF/resolve/main/Qwen3.6-27B-UD-Q4_K_XL.gguf
```

--------------------------------

### Run Unsloth Studio Installer

Source: https://thundercompute.com/docs/docs/guides/unsloth-studio

Executes the official Unsloth Studio installer script. This command should be run after installing the necessary system packages.

```shellscript
curl -L https://unsloth.ai/install.sh | bash
```

--------------------------------

### Install vLLM with uv

Source: https://thundercompute.com/docs/docs/guides/models/qwen3-6-27b

Installs the vLLM library using uv, a fast Python package installer. Ensure your virtual environment is activated.

```shell
source .venv/bin/activate
pip install -U uv
uv pip install vllm --torch-backend=auto
```

--------------------------------

### Start Ollama Server

Source: https://thundercompute.com/docs/docs/guides/deepseek-r1-running-locally-on-thunder-compute

Initiates the Ollama server within the connected instance. Allow approximately 30 seconds for the web UI to become accessible.

```bash
start-ollama
```

--------------------------------

### Get thunder templates

Source: https://thundercompute.com/docs/docs/console/operations/modifying-instances

Get available thunder templates for instance creation.

```APIDOC
## GET /thunder-templates

### Description
Get available thunder templates for instance creation.

### Method
GET

### Endpoint
/thunder-templates
```

--------------------------------

### Create A6000 Instance

Source: https://thundercompute.com/docs/llms-full.txt

Use this command to create a development instance with an A6000 GPU and 200GB disk space. Adjust disk size as needed for the GGUF file.

```bash
tnr create --mode development --gpu a6000 --vcpus 8 --template base --disk 200
```

--------------------------------

### VS Code Extension Installation Links

Source: https://thundercompute.com/docs/docs/vscode/quickstart

Provides direct links to install the Thunder Compute extension in VS Code, Cursor, and Windsurf. Ensure the respective editor is installed for these links to function.

```html
<a href="vscode:extension/ThunderCompute.thunder-compute">
VSCode extension
</a>

<a href="cursor:extension/ThunderCompute.thunder-compute">
Cursor extension
</a>

<a href="windsurf:extension/ThunderCompute.thunder-compute">
Windsurf extension
</a>
```

--------------------------------

### Install MCP Server

Source: https://thundercompute.com/docs/docs/guides/mcp-server

This command installs the Thunder Compute tools, making them available in your session.

```shellscript
opencode
```

--------------------------------

### Run Qwen3.6 27B

Source: https://thundercompute.com/docs/docs/guides/models/qwen3-6-27b

Instructions for launching Qwen3.6 27B on Thunder Compute, including quantization, llama.cpp commands, and OpenAI-compatible access.

```APIDOC
## Run Qwen3.6 27B

### Description
Launch Qwen3.6 27B dense on Thunder Compute with a GPU-fitting quantization, tested llama.cpp commands, and OpenAI-compatible access.

### Method
GET

### Endpoint
/guides/models/qwen3-6-27b

### Parameters
#### Query Parameters
- **model_format** (string) - Optional - Specify the model format (e.g., GGUF).
- **quantization** (string) - Optional - Specify the quantization level (e.g., Q4_K_M).
- **runtime** (string) - Optional - Specify the serving runtime (e.g., llama.cpp).

### Request Example
```json
{
  "example": "No request body needed for GET request"
}
```

### Response
#### Success Response (200)
- **instance_details** (object) - Details about the launched instance.
- **access_url** (string) - URL for OpenAI-compatible API access.

#### Response Example
```json
{
  "example": {
    "instance_details": {
      "instance_id": "i-12345abcde",
      "gpu_type": "A100",
      "status": "running"
    },
    "access_url": "https://your-instance-ip:8000/v1"
  }
}
```
```

--------------------------------

### Example User Request

Source: https://thundercompute.com/docs/llms-full.txt

An example of a natural language request that can be processed by the Thunder Compute plugin.

```text
Spin up an A100, wait until it is ready, run `nvidia-smi`, then tear it down and tell me the cost.
```

--------------------------------

### Connect to the instance and check GPU

Source: https://thundercompute.com/docs/llms-full.txt

Connect to your created instance using 'tnr connect' and verify GPU details with 'nvidia-smi'.

```bash
tnr status
tnr connect <instance-id>
```

```bash
nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv,noheader
```

--------------------------------

### Serve Qwen3.6-27B Model with vLLM

Source: https://thundercompute.com/docs/docs/guides/models/qwen3-6-27b

Starts a vLLM server to host the Qwen3.6-27B model. This command configures the host, port, and model name for the served model.

```shell
vllm serve Qwen/Qwen3.6-27B-FP8 \\\" --host 0.0.0.0 \\\" --port 8000 \\\" --served-model-name
```

--------------------------------

### Tested Setup Table for Qwen3.6 27B

Source: https://thundercompute.com/docs/docs/guides/models/qwen3-6-27b

Details the specific hardware and model configuration used for testing, including GPU, model file, context size, runtime, and result.

```markdown
## Tested Setup
The following configuration was used for the screenshots and command output in this guide.
| Thunder Compute GPU | Model file | Context | Runtime | Result |
|---|---|---|---|---|
| RTX A6000 48 GB | `Qwen3.6-27B-UD-Q4_K_XL.gguf` (16.4 GiB) | `32768` | llama.cpp Docker | Runs successfully on the `base` template and serves `/v1/chat/completions`. |
With these settings, Qwen3.6 27B runs cleanly on a single A6000 as a 32K-context endpoint. The server command uses `--parallel 1` to start with one request slot; this controls request concurrency, not the number of GPUs.
```

--------------------------------

### Add wandb to requirements.txt and install

Source: https://thundercompute.com/docs/llms-full.txt

Add wandb to your project's requirements file and then install it using pip.

```bash
echo "wandb" >> requirements.txt
pip install -r requirements.txt
```

--------------------------------

### Install wandb using pip

Source: https://thundercompute.com/docs/llms-full.txt

Install the Weights & Biases library in your Python environment on a Thunder Compute instance.

```bash
pip install wandb
```

--------------------------------

### Example Usage - Running nvidia-smi

Source: https://www.thundercompute.com/docs/guides/mcp-server

Example of how to ask the AI agent to run a specific command on an instance.

```natural language
"Run \"nvidia-smi\" on my instance"
```

--------------------------------

### Run Unsloth Studio

Source: https://thundercompute.com/docs/docs/guides/unsloth-studio

Execute this command to start Unsloth Studio. It will provide a local URL to access the Studio in your browser.

```shellscript
unsloth-studio
```

--------------------------------

### Start Unsloth Service

Source: https://thundercompute.com/docs/llms-full.txt

Execute 'start-unsloth' to start the Unsloth service. This command is used after connecting to an Unsloth instance.

```bash
start-unsloth
```

--------------------------------

### Start Unsloth Studio

Source: https://thundercompute.com/docs/llms-full.txt

Run Unsloth Studio in the background, listening on all interfaces on port 8888. Redirect output to a log file.

```bash
nohup unsloth studio -H 0.0.0.0 -p 8888 > ~/unsloth-studio.log 2>&1 &
```