Nosana

https://docs.nosana.com
Admin
Nosana is a decentralized cloud computing platform that enables users to deploy AI workloads and...
Tokens:39,235
Snippets:257
Trust Score:7.6
Update:11 months ago
Show doc for...
Tokens:
Raw
### Install Nosana CLI with Bun

Source: https://docs.nosana.com/inference/quick_start

Installs the Nosana CLI globally using Bun. This option is for developers using the Bun runtime for their JavaScript projects.

```bash
bun install -g @nosana/cli
```

--------------------------------

### Verify Nosana CLI Installation

Source: https://docs.nosana.com/inference/quick_start

Checks if the Nosana CLI has been installed correctly by displaying its version information. This command should be run after installation.

```bash
nosana --version
```

--------------------------------

### Install Nosana CLI with npm

Source: https://docs.nosana.com/inference/quick_start

Installs the Nosana CLI globally using npm. This is the first step to interacting with the Nosana network via the command line.

```bash
npm install -g @nosana/cli
```

--------------------------------

### Install Nosana CLI with Yarn

Source: https://docs.nosana.com/inference/quick_start

Installs the Nosana CLI globally using Yarn. This is an alternative method for Node.js environments that prefer Yarn for package management.

```bash
yarn install -g @nosana/cli
```

--------------------------------

### Install Nosana CLI with pnpm

Source: https://docs.nosana.com/inference/quick_start

Installs the Nosana CLI globally using pnpm. This method is suitable for users who utilize pnpm as their JavaScript package manager.

```bash
pnpm install -g @nosana/cli
```

--------------------------------

### Post a Nosana Job

Source: https://docs.nosana.com/inference/quick_start

Posts a simple 'echo hello world' job to the Nosana network using a specified market address. The --wait flag ensures the command waits for the job completion and result.

```bash
nosana job post echo hello world --wait --market 7AtiXMSH6R1jjBxrcYjehCkkSF7zvYWte63gwEDBcGHq
```

--------------------------------

### Run Podman API Service

Source: https://docs.nosana.com/hosts/grid-windows

Starts the Podman API service using a socket file in the user's home directory. This allows GPU Hosts to connect to the Podman service. This step is optional if the `start.sh` script has already been run.

```bash
mkdir -p $HOME/.nosana/podman
{ podman system service --time 0 unix://$HOME/.nosana/podman/podman.sock & } 2> podman.log
```

--------------------------------

### Verify Podman and GPU Support

Source: https://docs.nosana.com/hosts/grid-windows

Checks the installed Podman version and verifies if GPU support is correctly configured by running a test container that uses NVIDIA GPUs and executes `nvidia-smi -L`.

```bash
podman --version
```

```bash
podman run --rm --device nvidia.com/gpu=all --security-opt=label=disable ubuntu nvidia-smi -L
```

--------------------------------

### Display Node Information (Bash Output)

Source: https://docs.nosana.com/hosts/grid-ubuntu

This is an example of the output displayed after the Nosana script successfully runs, showing the node's status, wallet information, balances, and provider. It confirms the successful setup and connection to the network.

```text
  _   _ 
 | \ | | ___  ___  __ _ _ __   __ _
 |  \| |/ _ \/ __|/ _` | '_ \ / _` |
 | |\  | (_) \__ \ (_| | | | | (_| |
 |_| \_|\___/|___/\__,_|_| |_|\__,_|

Reading keypair from ~/.nosana/nosana_key.json

Network:	    mainnet
Wallet:		    <NODE_ADDRESS>
SOL balance:	0E-9 SOL
NOS balance:	0 NOS
Provider:	    podman

```

--------------------------------

### Verify NVIDIA Driver Installation

Source: https://docs.nosana.com/hosts/grid-windows

This command checks if the NVIDIA drivers are correctly installed and provides details about your GPU, driver version, and CUDA version. Successful execution indicates the drivers are ready for use.

```shell
nvidia-smi

```

--------------------------------

### Install NVIDIA Container Toolkit on Ubuntu

Source: https://docs.nosana.com/hosts/grid-ubuntu

These commands install the NVIDIA Container Toolkit by adding the official repository and then installing the package. This is essential for utilizing NVIDIA GPUs within containers.

```bash
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list \
  && \
    sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
```

--------------------------------

### Dockerfile for Multi-Service Container

Source: https://docs.nosana.com/inference/multi_service

This Dockerfile sets up a Python environment using uv, installs Open-WebUI and JupyterLab, exposes necessary ports, and includes a wrapper script to manage multiple services. It starts from a lightweight base image and configures a virtual environment.

```dockerfile
FROM ghcr.io/astral-sh/uv:debian

ENV PATH="$PATH:/root/.local/bin"

# Create & activate virtual‑env
RUN uv venv /opt/venv
ENV VIRTUAL_ENV=/opt/venv
ENV PATH="/opt/venv/bin:$PATH"

# Install Python deps
RUN uv pip install open-webui jupyterlab

EXPOSE 8000 9000

# Wrapper script starts both services
COPY start.sh /start.sh
RUN chmod +x /start.sh
ENTRYPOINT ["/start.sh"]

```

--------------------------------

### Verify Ubuntu WSL2 Version

Source: https://docs.nosana.com/hosts/grid-windows

This command verifies the installed Ubuntu version on WSL2. Ensure you are using Ubuntu 22.04 as older versions are not compatible with WSL2.

```shell
lsb_release -a

```

--------------------------------

### Install Podman on Ubuntu 22.04 (WSL2)

Source: https://docs.nosana.com/hosts/grid-windows

Installs Podman version 4.1 or later on Ubuntu 22.04 for WSL2. This involves adding the OpenSUSE Kubic repository and its GPG key, updating the package list, and installing the podman package.

```bash
echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/unstable/xUbuntu_22.04/ /" | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:unstable.list
curl -fsSL https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/unstable/xUbuntu_22.04/Release.key | sudo gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/devel_kubic_libcontainers_unstable.gpg > /dev/null
sudo apt update
sudo apt install podman
```

--------------------------------

### Retrieve a Nosana Job by ID

Source: https://docs.nosana.com/inference/quick_start

Retrieves the results of a previously posted Nosana job using its unique job ID. This allows users to fetch the output of their inference tasks.

```bash
nosana job get FQTP2F5hNP2rNGUtQm4Annrx462PgxPcSA6ND6ToPTxH
```

--------------------------------

### Join the Nosana Grid

Source: https://docs.nosana.com/hosts/grid-windows

A convenience script to set up a GPU Host on your machine by downloading and executing a bash script from nosana.com. This script does not require sudo privileges and performs various tests. Review the script before execution.

```bash
bash <(wget -qO- https://nosana.com/start.sh)
```

--------------------------------

### Wrapper Script to Start Multiple Services

Source: https://docs.nosana.com/inference/multi_service

This Bash script, intended to be run as the container's entrypoint, starts both the vLLM inference server and the Open-WebUI chat interface in the background. It uses `wait -n` to keep the container alive as long as either service is running and propagates the exit code of the first failing service.

```bash
#!/usr/bin/env bash
set -euo pipefail

# Start the LLM server (background)
vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B \
  --served-model-name R1-Qwen-1.5B \
  --port 9000 &


```

--------------------------------

### Confirm GPU Support in Podman (Docker Exec & Podman)

Source: https://docs.nosana.com/hosts/grid-ubuntu

These commands verify that GPU support is correctly configured within the Podman container. The first command enters the Podman container, and the second runs `nvidia-smi` to list available GPUs. Ensure NVIDIA drivers and nvidia-ctk are installed if this fails.

```bash
docker exec -it podman bash
podman run --rm --device nvidia.com/gpu=all --security-opt=label=disable ubuntu nvidia-smi -L

```

--------------------------------

### Launch Nosana GPU Host with Custom Parameters (Docker)

Source: https://docs.nosana.com/hosts/grid-windows

This command launches a Nosana GPU host using Docker. It allows specifying the Podman service socket location and mapping a custom Solana key into the container. Ensure the provided volume paths are correct for your system.

```docker
docker run \
      --pull=always \
      --network host  \
      --interactive -t \
      --volume ~/.config/solana/id.json:/root/.nosana/nosana_key.json \
      --volume ~/.nosana/podman:/root/.nosana/podman:ro \
      nosana/nosana-cli:latest \
         node join \
         --podman /root/.nosana/podman/podman.sock
```

--------------------------------

### Post Ollama Job to Nosana Network (CLI)

Source: https://docs.nosana.com/inference/examples/ollama

This command posts the previously defined Ollama job to the Nosana network. Replace `97G9NnvBDQ2WpKu6fasoMsAKmfj63C9rhysJnkeWodAf` with the actual market ID. The output will provide a service URL to interact with the deployed LLM.

```bash
nosana job post --file ollama.json --market 97G9NnvBDQ2WpKu6fasoMsAKmfj63C9rhysJnkeWodAf
```

--------------------------------

### Check Ubuntu Version

Source: https://docs.nosana.com/hosts/grid-ubuntu

This command allows you to verify your current Ubuntu version. Ensure it is 20.04 or higher for compatibility.

```bash
lsb_release -a
```

--------------------------------

### Multi-Operation Deployment Job Spec

Source: https://docs.nosana.com/inference/multi_service

Example of a job specification for deploying multiple operations in parallel with dependencies, such as vLLM and Open-WebUI.

```APIDOC
## Multi-Operation Deployment Job Spec

This section details how to create a job specification for deploying multiple operations that can run in parallel and have dependencies.

### Method
POST

### Endpoint
`@nosana/cli` job post

### Parameters
#### Request Body (JSON for vLLM + Open-WebUI)
```json
{
  "version": "0.1",
  "type": "container",
  "meta": {
    "trigger": "dashboard",
    "system_requirements": {
      "required_vram": 6
    }
  },
  "ops": [
    {
      "type": "container/run",
      "id": "vllm-server",
      "args": {
        "image": "vllm/vllm-openai:latest",
        "cmd": [
          "--model", "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B",
          "--served-model-name", "R1-Qwen-1.5B",
          "--port", "9000"
        ],
        "gpu": true,
        "expose": [
          {
            "port": 9000,
            "health_checks": [
              {
                "type": "http",
                "path": "/v1/models",
                "method": "GET",
                "expected_status": 200,
                "continuous": true
              }
            ]
          }
        ]
      },
      "execution": {
        "group": "inference"
      }
    },
    {
      "type": "container/run", 
      "id": "open-webui",
      "args": {
        "image": "ghcr.io/open-webui/open-webui:main",
        "env": {
          "OPENAI_API_BASE_URL": "http://vllm-server:9000/v1"
        },
        "expose": [8080]
      },
      "execution": {
        "group": "inference",
        "depends_on": ["vllm-server"]
      }
    }
  ]
}
```

- **version** (string) - Required - Version of the job spec.
- **type** (string) - Required - Type of job (e.g., `container`).
- **meta** (object) - Optional - Metadata for the job.
  - **trigger** (string) - Optional - How the job is triggered.
  - **system_requirements** (object) - Optional - System requirements for the job.
    - **required_vram** (number) - Optional - Required VRAM in GB.
- **ops** (array) - Required - List of operations.
  - **type** (string) - Required - Type of operation (e.g., `container/run`).
  - **id** (string) - Required - Unique identifier for the operation.
  - **args** (object) - Required - Arguments for the operation.
    - **image** (string) - Required - Docker image to use.
    - **cmd** (array) - Optional - Command to run inside the container.
    - **gpu** (boolean) - Optional - Whether to use GPU acceleration.
    - **expose** (array or object) - Optional - Ports to expose and their configurations.
      - **port** (number) - Required - The port number.
      - **health_checks** (array) - Optional - Health check configurations.
        - **type** (string) - Required - Type of health check (e.g., `http`).
        - **path** (string) - Required - Path for HTTP health check.
        - **method** (string) - Required - HTTP method for health check.
        - **expected_status** (number) - Required - Expected HTTP status code.
        - **continuous** (boolean) - Optional - Whether to perform continuous health checks.
    - **env** (object) - Optional - Environment variables for the container.
  - **execution** (object) - Optional - Execution details for the operation.
    - **group** (string) - Optional - Execution group.
    - **depends_on** (array) - Optional - List of operation IDs this operation depends on.

### Command Example
```bash
npx @nosana/cli job post \
  --file your-multi-op-job.json \
  --market nvidia-3090 \
  --timeout 60
```

### Response
#### Success Response (200)
- **job_id** (string) - The ID of the posted job.

#### Response Example
```json
{
  "job_id": "another-unique-job-id"
}
```
```

--------------------------------

### Post Job to Nosana using CLI

Source: https://docs.nosana.com/inference/multi_service

Posts a job definition file to the Nosana platform using the `@nosana/cli` command-line interface. This command specifies the job file ('webui-deepseek.json'), the market ('nvidia-3090'), and a timeout of 60 seconds. Ensure you have installed the CLI globally (`npm i -g @nosana/cli`).

```bash
npx @nosana/cli job post \
  --file webui-deepseek.json \
  --market nvidia-3090 \
  --timeout 60
```

--------------------------------

### Deploy Ollama Container with LLM on Nosana Network (JSON)

Source: https://docs.nosana.com/inference/examples/ollama

This JSON schema defines a containerized job to run an Ollama instance with a specific LLM image on the Nosana network. It exposes port 11434 for API access and includes a script to download and run LLM inference. Ensure the image `docker.io/nosana/ollama-7b:0.0.1` is accessible.

```json
{
  "version": "0.1",
  "type": "container",
  "meta": {
    "trigger": "cli"
  },
  "ops": [
    {
      "type": "container/run",
      "id": "ollama",
      "args": {
        "cmd": [
            "-c 'curl -s https://raw.githubusercontent.com/KeesGeerligs/nosana/main/benchmarking/images/command.sh -o /tmp/command.sh && chmod +x /tmp/command.sh && /tmp/command.sh'"
          ],
        "image": "docker.io/nosana/ollama-7b:0.0.1",
        "gpu": true,
        "expose": 11434
      }
    }
  ]
}
```

--------------------------------

### Start WebUI and LLM Endpoint in Background

Source: https://docs.nosana.com/inference/multi_service

This script starts the Open-WebUI service and a local LLM endpoint in the background. It uses `wait -n` to ensure the script waits for the first child process to exit and forwards its status code, preventing premature termination. This is crucial for maintaining service availability during deployment.

```bash
OPENAI_API_BASE_URL=http://127.0.0.1:9000/v1 open-webui serve --port 8000 &

wait -n
exit $?
```

--------------------------------

### Single Operation Deployment Job Spec

Source: https://docs.nosana.com/inference/multi_service

Example of a job specification JSON for deploying a single containerized application like Open-WebUI with a local LLM endpoint.

```APIDOC
## Single Operation Deployment Job Spec

This section details how to create a job specification for deploying a single containerized application.

### Method
POST

### Endpoint
`@nosana/cli` job post

### Parameters
#### Request Body (webui-deepseek.json)
```json
{
  "ops": [
    {
      "id": "webui-deepseek",
      "args": {
        "image": "docker.io/<username>/<image-name>:latest",
        "gpu": true,
        "expose": [8000, 9000]
      },
      "type": "container/run"
    }
  ],
  "meta": {
    "trigger": "dashboard",
    "system_requirements": {
      "required_vram": 6
    }
  },
  "type": "container",
  "version": "0.1"
}
```

- **ops** (array) - Required - List of operations to run.
  - **id** (string) - Required - Unique identifier for the operation.
  - **args** (object) - Required - Arguments for the operation.
    - **image** (string) - Required - Docker image to use.
    - **gpu** (boolean) - Optional - Whether to use GPU acceleration.
    - **expose** (array) - Optional - Ports to expose.
  - **type** (string) - Required - Type of operation (e.g., `container/run`).
- **meta** (object) - Optional - Metadata for the job.
  - **trigger** (string) - Optional - How the job is triggered.
  - **system_requirements** (object) - Optional - System requirements for the job.
    - **required_vram** (number) - Optional - Required VRAM in GB.
- **type** (string) - Required - Type of job (e.g., `container`).
- **version** (string) - Required - Version of the job spec.

### Command Example
```bash
npx @nosana/cli job post \
  --file webui-deepseek.json \
  --market nvidia-3090 \
  --timeout 60
```

### Response
#### Success Response (200)
- **job_id** (string) - The ID of the posted job.

#### Response Example
```json
{
  "job_id": "some-unique-job-id"
}
```
```

--------------------------------

### Generate Text with Ollama API

Source: https://docs.nosana.com/inference/examples/ollama

This endpoint allows you to generate text by sending a prompt to a specified Ollama model. You can configure the model, streaming behavior, and the prompt itself.

```APIDOC
## POST /api/generate

### Description
This endpoint generates text based on a provided prompt and model. It supports streaming responses and allows for detailed configuration.

### Method
POST

### Endpoint
`https://[SERVICE_URL]/api/generate`

### Parameters
#### Request Body
- **model** (string) - Required - The name of the Ollama model to use (e.g., "gemma:7b").
- **stream** (boolean) - Optional - If true, the response will be streamed.
- **prompt** (string) - Required - The text prompt to send to the model.

### Request Example
```json
{
  "model": "gemma:7b",
  "stream": false,
  "prompt": "What is water made of?"
}
```

### Response
#### Success Response (200)
- **model** (string) - The model used for generation.
- **created_at** (string) - Timestamp of when the response was created.
- **response** (string) - The generated text response.
- **done** (boolean) - Indicates if the generation is complete.
- **done_reason** (string) - The reason for completion (e.g., "stop").
- **context** (array) - Context information for the generation.
- **total_duration** (integer) - Total time taken for the generation in nanoseconds.
- **load_duration** (integer) - Time taken to load the model in nanoseconds.
- **prompt_eval_count** (integer) - Number of tokens evaluated in the prompt.
- **prompt_eval_duration** (integer) - Time taken to evaluate the prompt in nanoseconds.
- **eval_count** (integer) - Number of tokens generated.
- **eval_duration** (integer) - Time taken to generate tokens in nanoseconds.

#### Response Example
```json
{
  "model": "gemma:7b",
  "created_at": "2024-07-29T14:04:54.272390422Z",
  "response": "Water is composed of two hydrogen atoms and one oxygen atom together. Its chemical formula is H2O.",
  "done": true,
  "done_reason": "stop",
  "context": [
    968, 2997, 235298, 559, 235298, 15508, 235313, 1645, 108, 1841, 603, 2003, 1644, 576, 181537, 615, 235298, 559, 235298, 15508, 235313, 108, 235322, 2997, 235298, 559, 235298, 15508, 235313, 2516, 108, 11586, 603, 18588, 576, 1378, 20303, 25204, 578, 974, 16175, 24235, 74346, 3584, 235265, 9707, 9408, 10513, 603, 640, 235284, 235302, 35606, 615, 235298, 559, 235298, 15508, 235313, 108
  ],
  "total_duration": 11622510413,
  "load_duration": 11219650532,
  "prompt_eval_count": 32,
  "prompt_eval_duration": 74712000,
  "eval_count": 23,
  "eval_duration": 279790000
}
```
```

--------------------------------

### Check WSL Version and Default Distribution (Bash)

Source: https://docs.nosana.com/hosts/troubleshoot

Verifies the installed WSL version and sets the default distribution for Docker integration. Ensure Docker Desktop is running and configured with the WSL2 backend.

```bash
wsl -l -v

```

```bash
wsl --set-default <distro name>

```

--------------------------------

### Configure NVIDIA Container Toolkit for Podman on WSL2

Source: https://docs.nosana.com/hosts/grid-windows

Generates CDI configuration for the NVIDIA Container Toolkit to enable Podman v4 to natively use GPUs on WSL2. This requires a separate CDI configuration step found at the provided NVIDIA documentation link.

```bash
sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
```

```bash
nvidia-ctk cdi list
```

--------------------------------

### Check Available Storage with Free Command

Source: https://docs.nosana.com/hosts/grid-windows

This command displays the amount of free and used memory in the system, including RAM and swap. It's useful after configuring WSL2 to verify the updated memory allocation.

```shell
free -h

```

--------------------------------

### Nosana Job Posting Output Example

Source: https://docs.nosana.com/inference/examples/open_webui

This is an example of the output received after successfully posting a job to the Nosana network. It includes network details, wallet information, IPFS upload status, job posting confirmation, and the service endpoint.

```bash
  _   _ 
 | \ | | ___  ___  __ _ _ __   __ _ 
 |  \| |/ _ \/ __|/ _` | '_ \ / _` |
 | |\  | (_) |__ \ (_| | | | | (_| |
 |_| \_|\___/|___/\__,_|_| |_|\__,_|

Reading keypair from /home/user/.nosana/nosana_key.json

Network:	mainnet
Wallet:		4WtG17Vn3SSoTAVvXxpopQTG3Qo9NUK28Zotv4rL1ccv
SOL balance:	0.04212692 SOL
NOS balance:	65.125477 NOS
ipfs uploaded:	https://nosana.mypinata.cloud/ipfs/QmajmxiSFDfaZ4YmwsV92iqCz8LPZyLjKjjfqJ6m7kgWjw
posting job to market 97G9NnvBDQ2WpKu6fasoMsAKmfj63C9rhysJnkeWodAf for price 0.000115 NOS/s (total: 0.8280 NOS)
job posted with tx 62TjXTM2rme78RCpM7pZ5YAkCHLmybqHvWrYfTigrjMooBqww99Qn1BoGWtkcWVA5CCuWy3H2PgANWr7PaqsWBtg!
Service will be exposed at https://ChiscJkoWFYwiZrCPzWd1wKBeDonMqaekq7eCAqTt1YC.node.k8s.prd.nos.ci
Job:		https://dashboard.nosana.com/jobs/41s1bL8CYBctRgz6jCAYfUSxd8zueUkneHXHeMXVE3p5
JSON flow:	https://nosana.mypinata.cloud/ipfs/QmajmxiSFDfaZ4YmwsV92iqCz8LPZyLjKjjfqJ6m7kgWjw
Market:		https://dashboard.nosana.com/markets/97G9NnvBDQ2WpKu6fasoMsAKmfj63C9rhysJnkeWodAf
Price:		0.000115 NOS/s
Status:		RUNNING

run nosana job get 41s1bL8CYBctRgz6jCAYfUSxd8zueUkneHXHeMXVE3p5 --network mainnet to retrieve job and result

```

--------------------------------

### Configure WSL2 Memory and Processors

Source: https://docs.nosana.com/hosts/grid-windows

This configuration file allows customization of memory, processors, and swap space allocated to WSL2. Adjust values based on your system's resources to optimize performance for node operations.

```ini
[wsl2]
memory=58GB     # Adjust to ~90% of your total RAM (e.g., 58GB for 64GB system)
processors=16   # Match your CPU cores (e.g., 16 for an 16-core CPU)
swap=16GB       # Extra swap for memory overflow, once it runs out of RAM memory it borrows from SSD

```

--------------------------------

### Start Nosana GPU Host Script

Source: https://docs.nosana.com/hosts/grid-run

This command downloads and executes the `start.sh` script from the official Nosana website, simplifying the process of running your GPU Host. It ensures your GPU host is running within a Docker container.

```bash
bash <(wget -qO- https://nosana.com/start.sh)

```

--------------------------------

### Nosana Job Output Example

Source: https://docs.nosana.com/inference/examples/stable

This is an example of the output generated after successfully posting a job to the Nosana network. It includes network details, wallet information, balances, IPFS upload links, job posting confirmation, service exposure URL, and job details.

```text
  _   _ 
 | \ | | ___  ___  __ _ _ __   __ _
 |  \| |/ _ \/ __|/ _` | '_ \ / _` |
 | |\  | (_) \__ \ (_| | | | | (_| |
 |_| \_|\___/|___/\__,_|_| |_|\__,_|

Reading keypair from /home/user/.nosana/nosana_key.json

Network:	mainnet
Wallet:		4WtG17Vn3SSoTAVvXxpopQTG3Qo9NUK28Zotv4rL1ccv
SOL balance:	0.07779196 SOL
NOS balance:	72.163245 NOS
ipfs uploaded:	https://nosana.mypinata.cloud/ipfs/QmbYeFzM6gNCf32GqQG2GDYQsFXxhBWYPskHHzooQSURBW
posting job to market 97G9NnvBDQ2WpKu6fasoMsAKmfj63C9rhysJnkeWodAf for price 0.000097 NOS/s (total: 0.6984 NOS)
job posted with tx 3TveJgqzHiV1p97jxxzvS8Jeg29o7uKKtAJCU6NJ5SAtRsndtrkA6az3kLQwXp3aNatXD3ZUrBJ64YetWpTXsTFB!
Service will be exposed at https://GY1BDTVMQtwSJ7V3zFT5tYNimmyPRQfaJLKV1R5FDbz3.node.k8s.prd.nos.ci
Job:		https://dashboard.nosana.com/jobs/5hYyrw4jBkekaLDZviAvoBJPXjfUgJu5S8u1fjzdt5Wx
JSON flow:	https://nosana.mypinata.cloud/ipfs/QmbYeFzM6gNCf32GqQG2GDYQsFXxhBWYPskHHzooQSURBW
Market:		https://dashboard.nosana.com/markets/97G9NnvBDQ2WpKu6fasoMsAKmfj63C9rhysJnkeWodAf
Price:		0.000097 NOS/s
Status:		RUNNING

run nosana job get 5hYyrw4jBkekaLDZviAvoBJPXjfUgJu5S8u1fjzdt5Wx --network mainnet to retrieve job and result

```

--------------------------------

### Interact with Ollama API via cURL (POST Request)

Source: https://docs.nosana.com/inference/examples/ollama

This cURL command demonstrates how to send a POST request to the Ollama API endpoint to generate text. It specifies the model, stream settings, and the prompt. Replace the service URL with the one provided after posting the job.

```bash
curl -X POST https://B7hWYTyWHMLmYUH9sZFJL3TjgrHUAdDBi76xuEg9m9hX.node.k8s.prd.nos.ci/api/generate \
-H "Content-Type: application/json" \
-d "{\"model\": \"gemma:7b\", \"stream\": false, \"prompt\": \"What is water made of?\"}"
```

--------------------------------

### Configure Docker for NVIDIA Container Toolkit

Source: https://docs.nosana.com/hosts/grid-ubuntu

This command configures the Docker runtime to work with the NVIDIA Container Toolkit. After configuration, Docker must be restarted for the changes to take effect.

```bash
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
```

--------------------------------

### Interact with Ollama API via JavaScript (HTTPS Request)

Source: https://docs.nosana.com/inference/examples/ollama

This JavaScript code snippet shows how to make an HTTPS POST request to the Ollama API to generate text. It constructs the request payload and options, handling the response and potential errors. Ensure the `hostname` is set to the correct service URL.

```javascript
import https from "https";

const data = JSON.stringify({
  model: "gemma:7b",
  stream: false,
  prompt: "What is water made of?",
});

const options = {
  hostname: "https://B7hWYTyWHMLmYUH9sZFJL3TjgrHUAdDBi76xuEg9m9hX.node.k8s.prd.nos.ci",
  port: 443,
  path: "/api/generate",
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Content-Length": data.length,
  },
};

const req = https.request(options, (res) => {
  let responseData = "";
  res.on("data", (chunk) => {
    responseData += chunk;
  });
  res.on("end", () => {
    console.log("Response:", responseData);
  });
});

req.on("error", (e) => {
  console.error("Error:", e);
});

req.write(data);
req.end();

```

--------------------------------

### Nosana CLI: List and Get Market Resources

Source: https://docs.nosana.com/inference/examples/vllm

Utilize the Nosana CLI to list all available markets and retrieve specific market-required resources. This is essential for understanding and accessing cached model data.

```bash
// To list all markets
nosana market list

// Retrive market required resources
nosana market get 97G9NnvBDQ2WpKu6fasoMsAKmfj63C9rhysJnkeWodAf

```

--------------------------------

### Configure Docker Engine for Podman (JSON)

Source: https://docs.nosana.com/hosts/troubleshoot

Addresses Podman connection errors by adding a 'bip' configuration to the Docker engine settings. This requires modifying the Docker Desktop settings and restarting Docker.

```json
{
  "bip":"192.168.200.1/24",
}

```

--------------------------------

### List and Retrieve Nosana Market Resources (CLI)

Source: https://docs.nosana.com/inference/examples/lmdeploy

Commands to interact with the Nosana CLI for managing market resources. 'nosana market list' displays all available markets, and 'nosana market get' retrieves specific resources for a given market ID.

```bash
nosana market list
```

```bash
nosana market get 97G9NnvBDQ2WpKu6fasoMsAKmfj63C9rhysJnkeWodAf
```

--------------------------------

### Post Nosana Job via CLI

Source: https://docs.nosana.com/inference/examples/hello_world

This command-line instruction posts a predefined Nosana job to the network. It references a JSON job file and uses the '--wait' flag to monitor the job's execution and retrieve results.

```bash
nosana job post --file hello_world.json --market 97G9NnvBDQ2WpKu6fasoMsAKmfj63C9rhysJnkeWodAf --wait
```

--------------------------------

### Downgrade Conmon for Podman (Bash)

Source: https://docs.nosana.com/hosts/troubleshoot

Resolves Podman 'container create failed' errors by downgrading the 'conmon' package to a specific version. This involves downloading a .deb file and installing it using apt.

```bash
wget https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_22.04/amd64/conmon_2.1.2~0_amd64.deb -O /tmp/conmon_2.1.2.deb
sudo apt install /tmp/conmon_2.1.2.deb

```

```bash
podman run --rm --device nvidia.com/gpu=all --security-opt=label=disable ubuntu nvidia-smi -L

```

--------------------------------

### Run Podman in Docker (Docker CLI)

Source: https://docs.nosana.com/hosts/grid-ubuntu

This command sets up and runs a Podman instance within a Docker container, enabling GPU support. It creates a Docker volume for Podman's socket and cache, and mounts necessary devices and volumes for operation. This is an optional step for Ubuntu users running Docker.

```docker
docker volume create podman-socket
docker run -d \
    --pull=always \
    --gpus=all \
    --name podman \
    --device /dev/fuse \
    --mount source=podman-cache,target=/var/lib/containers \
    --volume podman-socket:/podman \
    --privileged \
    -e ENABLE_GPU=true \
    nosana/podman:v1.1.0 unix:/podman/podman.sock

```

--------------------------------

### Build Docker Image

Source: https://docs.nosana.com/inference/multi_service

Builds a Docker image from a Dockerfile in the current directory. The image is tagged with a specified username, image name, and 'latest' tag, preparing it for subsequent publishing to a container registry. Ensure you are in the same directory as your Dockerfile and start.sh script.

```docker
docker build -t <username>/<image-name>:latest .
```

--------------------------------

### Define Container Job for 'hello world'

Source: https://docs.nosana.com/inference/examples/hello_world

This JSON object defines a Nosana job to run a containerized shell command. It specifies the container image as 'ubuntu' and the command to execute as 'echo hello world'. The job is triggered via CLI.

```json
{
  "version": "0.1",
  "type": "container",
  "meta": {
    "trigger": "cli"
  },
  "ops": [
    {
      "type": "container/run",
      "id": "hello-world",
      "args": {
        "cmd": ["echo hello world"],
        "image": "ubuntu"
      }
    }
  ]
}
```

--------------------------------

### Execution Group and Dependencies Configuration

Source: https://docs.nosana.com/inference/multi_service

Defines how operations are grouped and their dependencies for execution within a larger job. 'group' specifies execution stages, where operations within the same group can run in parallel. 'depends_on' lists operation IDs that must complete before the current operation can start.

```json
{
  "execution": {
    "group": "string",
    "depends_on": ["op-id-1", "op-id-2"]
  }
}
```

--------------------------------

### Launch GPU Host with Custom Parameters (Docker CLI)

Source: https://docs.nosana.com/hosts/grid-ubuntu

This command manually launches the Nosana CLI Docker container, allowing for custom parameters such as specifying a different Podman socket location or mapping a custom Solana key. It uses host networking and mounts volumes for Podman and the Solana key.

```docker
docker run \
      --pull=always \
      --network host  \
      --interactive -t \
      --volume ~/.config/solana/id.json:/root/.nosana/nosana_key.json \
      --volume podman-socket:/root/.nosana/podman:ro \
      nosana/nosana-cli:latest node start \
         --podman /root/.nosana/podman/podman.sock

```

--------------------------------

### Check Operation Status via Node API

Source: https://docs.nosana.com/inference/multi_service

Demonstrates how to check the status of operations within a job on the Nosana platform using a GET request to the Node API. This endpoint allows you to monitor the progress and state of your deployed services, which can be useful for debugging and operational oversight.

```http
# Check operation status
GET https://{{node}}.node.k8s.prd.nos.ci/job/{{job}}/ops
```

--------------------------------

### View Nosana Node Solana Key

Source: https://docs.nosana.com/hosts/grid-run

This command displays the content of the `nosana_key.json` file, which contains your Node's Solana private key. It's crucial to back up this file securely.

```bash
sudo cat ~/.nosana/nosana_key.json

```

--------------------------------

### Login and Push Docker Image

Source: https://docs.nosana.com/inference/multi_service

Logs into a Docker registry and pushes the built Docker image. The `docker login` command is typically performed once, followed by `docker push` to upload the image tagged as '<username>/<image-name>:latest' to the registry. This makes the image available for deployment.

```docker
# (first time only)
docker login

docker push <username>/<image-name>:latest
```

--------------------------------

### Example Job Definition Structure (JSON)

Source: https://docs.nosana.com/inference/job_reference

This JSON object represents a complete example of a Nosana job definition. It includes fields for version, type, meta-information, global defaults, and a list of operations to be executed.

```json
{
  "version": "0.1",
  "type": "container",
  "meta": {
    "trigger": "cli",
    "system_resources": {
      "required_vram": 18
    }
  },
  "global": {
    "image": "ubuntu",
    "gpu": true,
    "entrypoint": "/bin/bash",
    "env": {
      "MY_VAR": "value"
    },
    "work_dir": "/workspace"
  },
  "ops": [
    {
      "id": "my-operation",
      "type": "container/run",
      "args": {
        "cmd": ["echo", "hello world"],
        "resources": [
          {
            "type": "S3",
            "url": "https://models.example.com",
            "target": "/data/"
          }
        ],
        "authentication": {
          "docker": {
            "username": "myDockerUser",
            "password": "myDockerPass"
          }
        }
      }
    }
  ]
}
```

--------------------------------

### Create a Job with Credits using Nosana API

Source: https://docs.nosana.com/inference/api_http

This example demonstrates how to create a new job using the Nosana API with credit allocation. It requires an API key and job details such as IPFS hash, market address, and a timeout period. The response will contain details about the created job.

```bash
export NOSANA_API_KEY="nos_xxx_your_api_key"

curl -s -X POST \
  -H "Authorization: Bearer $NOSANA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "ipfsHash": "QmYourJobHash",
    "market": "7AtiXMSH6R1jjBxrcYjehCkkSF7zvYWte63gwEDBcGHq",
    "timeout": 1800
  }' \
  https://dashboard.k8s.prd.nos.ci/api/jobs/create-with-credits | jq .
```

--------------------------------

### Post Open WebUI Job to Nosana Network

Source: https://docs.nosana.com/inference/examples/open_webui

This command posts the defined Open WebUI job to the Nosana network. It requires a file containing the job schema and the market ID for the network.

```bash
nosana job post --file open_webui.json --market 97G9NnvBDQ2WpKu6fasoMsAKmfj63C9rhysJnkeWodAf
```

--------------------------------

### Check Operation Status

Source: https://docs.nosana.com/inference/multi_service

API endpoint to check the status of operations within a job.

```APIDOC
## Check Operation Status

This API allows you to retrieve the current status of operations associated with a specific job.

### Method
GET

### Endpoint
`https://{{node}}.node.k8s.prd.nos.ci/job/{{job}}/ops`

### Parameters
#### Path Parameters
- **node** (string) - Required - The specific node where the job is running.
- **job** (string) - Required - The unique identifier of the job.

### Response
#### Success Response (200)
- **operations** (array) - A list of operations with their current states.
  - **id** (string) - The ID of the operation.
  - **state** (string) - The current state of the operation (e.g., `pending`, `running`, `completed`, `failed`).

#### Response Example
```json
{
  "operations": [
    {
      "id": "vllm-server",
      "state": "running"
    },
    {
      "id": "open-webui",
      "state": "pending"
    }
  ]
}
```
```

--------------------------------

### Stop/Restart Entire Groups

Source: https://docs.nosana.com/inference/multi_service

API endpoints to stop or restart all operations within a specified job group. These are POST requests targeting the group.

```http
POST https://{{node}}.node.k8s.prd.nos.ci/job/{{job}}/group/{{group}}/stop
POST https://{{node}}.node.k8s.prd.nos.ci/job/{{job}}/group/{{group}}/restart
```

--------------------------------

### Stop/Restart Individual Operations

Source: https://docs.nosana.com/inference/multi_service

API endpoints to stop or restart a specific operation within a job group. These are POST requests targeting the operation ID.

```http
POST https://{{node}}.node.k8s.prd.nos.ci/job/{{job}}/group/{{group}}/operation/{{opid}}/stop
POST https://{{node}}.node.k8s.prd.nos.ci/job/{{job}}/group/{{group}}/operation/{{opid}}/restart
```

--------------------------------

### Static Resource Mounting in JSON

Source: https://docs.nosana.com/inference/literals

This JSON snippet illustrates how an operation can mount resources, such as S3 buckets, by specifying their `url` and `target` location. This example shows a static configuration for an Ollama model.

```json
{
  "type": "container/run",
  "id": "ollama",
  "args": {
    "cmd": [],
    "image": "docker.io/ollama/ollama",
    "gpu": true,
    "expose": 11434,
    "resources": [
      {
        "type": "S3",
        "url": "s3://nos-ai-models-qllsn32u/ollama/llama3.1/70b",
        "target": "/root/.ollama/models"
      }
    ]
  }
}
```
Tokens:
Raw