### Building VLLM Example Chute Image (Bash) Source: https://github.com/rayonlabs/chutes-api/blob/main/data/example.md This snippet navigates to the data directory, installs vLLM, and initiates the build process for a public 'vllm_example:chute' image. It includes a wait period and then terminates the build process. ```bash cd data poetry run pip install vllm==0.6.2 poetry run chutes build vllm_example:chute --public --debug --wait & sleep 30 kill -9 %1 cd - ``` -------------------------------- ### Deploying VLLM Example Chute (Bash) Source: https://github.com/rayonlabs/chutes-api/blob/main/data/example.md This snippet uses the `chutes` CLI to deploy the previously built 'vllm_example:chute' image, making it publicly accessible. ```bash poetry run chutes deploy vllm_example:chute --public ``` -------------------------------- ### Starting Base Docker Compose Services Source: https://github.com/rayonlabs/chutes-api/blob/main/dev/dev.md Starts all services defined in the default `docker-compose.yml` file in detached mode. These are the core components of the Chutes API that do not require GPU. ```bash docker compose up -d ``` -------------------------------- ### Initializing Chutes API Docker Environment (Bash) Source: https://github.com/rayonlabs/chutes-api/blob/main/data/example.md This snippet initializes and recreates the Docker-based environment for the Chutes API, including GPU services, and registers a default user. It ensures a clean slate before starting services. ```bash #!/bin/bash set -eux # (re)create the entire environment. docker compose down docker compose -f docker-compose-gpu.yml down docker volume rm parachutes-api_minio_data parachutes-api_postgres_data || true docker compose up -d --build docker compose -f docker-compose-gpu.yml up --build vllm -d # Wait a second... echo "Waiting for services..." sleep 10 # Register a user. CHUTES_API_URL=http://127.0.0.1:8000 poetry run chutes register ``` -------------------------------- ### Running Bootstrap Script on Linux Source: https://github.com/rayonlabs/chutes-api/blob/main/dev/dev.md Executes the `bootstrap.sh` script with superuser privileges on Linux. This script is likely responsible for initial system-level setup or dependency installation required for the project. ```bash sudo -E bash dev/bootstrap.sh ``` -------------------------------- ### Starting Optional GPU-enabled vLLM Miner Source: https://github.com/rayonlabs/chutes-api/blob/main/dev/dev.md Starts the 'vllm' service from `docker-compose-gpu.yml` in detached mode. This is an optional component, acting as a dummy vLLM instance, and requires a local GPU to run. ```bash docker compose -f docker-compose-gpu.yml up -d vllm ``` -------------------------------- ### Setting up Python Environment on Mac Source: https://github.com/rayonlabs/chutes-api/blob/main/dev/dev.md This snippet configures the Python environment on macOS using `pyenv` and `poetry`. It sets the local Python version, creates a poetry environment, installs development dependencies, and activates the poetry shell. ```bash pyenv local 3.12.4 poetry env use $(pyenv which python) poetry install --with dev --no-root poetry shell ``` -------------------------------- ### Installing Python Dependencies on Linux Source: https://github.com/rayonlabs/chutes-api/blob/main/dev/dev.md Configures the Python environment on Linux using `poetry`. It sets the Python version to 3.12, installs project dependencies without creating a root package, and activates the poetry shell. ```bash poetry env use 3.12 poetry install --no-root poetry shell ``` -------------------------------- ### Starting Optional GPU-enabled Graval Node Validator Source: https://github.com/rayonlabs/chutes-api/blob/main/dev/dev.md Starts the 'graval' service from `docker-compose-gpu.yml` in detached mode. This is an optional component, serving as a node validator, and requires a local GPU to operate. ```bash docker compose -f docker-compose-gpu.yml up -d graval ``` -------------------------------- ### Creating Admin API Key (Bash) Source: https://github.com/rayonlabs/chutes-api/blob/main/data/example.md This snippet generates a new API key with administrative privileges using the `chutes keys create` command, which is essential for interacting with the API. ```bash poetrun run chutes keys create --name admin --admin ``` -------------------------------- ### Seeding Chute Instance Data (Bash/SQL/Python) Source: https://github.com/rayonlabs/chutes-api/blob/main/data/example.md This snippet retrieves the latest chute ID from PostgreSQL, sets the Python path, and then executes a Python script to seed the newly deployed chute instance with initial data. ```bash export CHUTE_ID=$(docker compose exec postgres psql -U user -d chutes -c "select chute_id from chutes order by created_at desc limit 1" -t) export PYTHONPATH=$(pwd) poetry run python ./api/bin/seed_instance --chute-id $CHUTE_ID ``` -------------------------------- ### Running Ansible Playbook for Extra Configurations Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This command executes the `extras.yml` Ansible playbook with the specified inventory. This playbook installs and configures additional components, such as the Kubernetes NVIDIA GPU operator, after the core cluster setup. ```bash ansible-playbook -i inventory.yml extras.yml ``` -------------------------------- ### Marking Chute Image as Ready in PostgreSQL (Bash/SQL) Source: https://github.com/rayonlabs/chutes-api/blob/main/data/example.md This snippet directly updates the PostgreSQL database to mark a newly built image's status as 'built and pushed', bypassing the need to wait for the build process to complete naturally. ```bash docker compose exec -it postgres psql -U user chutes -c "update images set status = 'built and pushed'" ``` -------------------------------- ### Installing EPEL Release on CentOS/RHEL/Fedora Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This command installs the Extra Packages for Enterprise Linux (EPEL) repository. EPEL provides additional high-quality packages for CentOS, RHEL, and Fedora systems, often including newer versions of software like Ansible. ```bash sudo dnf install epel-release -y ``` -------------------------------- ### Installing Ansible on CentOS/RHEL/Fedora Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This command installs Ansible on CentOS, RHEL, or Fedora systems using the DNF package manager. This step assumes the EPEL repository has been installed if necessary. ```bash sudo dnf install ansible -y ``` -------------------------------- ### Installing Homebrew on macOS Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This snippet installs Homebrew, a popular package manager for macOS, by executing a script downloaded from the official Homebrew website. Homebrew is a prerequisite for installing other tools like Ansible on macOS. ```bash /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" ``` -------------------------------- ### Installing Ansible Community General Collection Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This command installs the `community.general` collection for Ansible Galaxy. This collection provides a wide range of modules and plugins for common automation tasks across various platforms and services. ```bash ansible-galaxy collection install community.general ``` -------------------------------- ### Installing Ansible Kubernetes Core Collection Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This command installs the `kubernetes.core` collection for Ansible Galaxy. This collection is essential for managing and interacting with Kubernetes clusters using Ansible playbooks. ```bash ansible-galaxy collection install kubernetes.core ``` -------------------------------- ### Installing Ansible on macOS Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This command uses Homebrew, once installed, to install Ansible on a macOS system. Ansible is an open-source automation tool used for configuration management, application deployment, and orchestration. ```bash brew install ansible ``` -------------------------------- ### Testing Chat Completions API with Curl (Bash) Source: https://github.com/rayonlabs/chutes-api/blob/main/data/example.md This snippet demonstrates how to make a `curl` request to the Chutes API's chat completions endpoint. It sends a sample message to a specific model and expects a streamed response, requiring an API key for authorization. ```bash curl -s http://test-unsloth-llama-3-2-1b-instruct.lvh.me:8000/v1/chat/completions -d '{"model": "unsloth/Llama-3.2-1B-Instruct", "messages": [{"role": "user", "content": "What is the secret to life, the universe, everything?."}], "stream": true, "max_tokens": 25}' -H 'Authorization: Bearer ' ``` -------------------------------- ### Installing Ansible on Ubuntu/Debian Systems Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This command updates the package lists and then installs Ansible and Python 3 pip on Ubuntu or other aptitude-based Linux distributions. Python 3 pip is often required for Ansible's dependencies. ```bash sudo apt update && apt -y install -y ansible python3-pip ``` -------------------------------- ### Creating Docker Network for Chutes API Source: https://github.com/rayonlabs/chutes-api/blob/main/dev/dev.md Creates a custom Docker network named 'chutes'. This network allows containers to communicate with each other using their service names, isolating them from the default bridge network. ```bash docker network create chutes ``` -------------------------------- ### Generating Bittensor Wallet Encryption Keys (Python) Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This Python snippet provides an example of generating secure hexadecimal strings using the `secrets` module. It shows how to create a 64-character hex string (from 32 bytes) and a 128-character hex string (from 64 bytes), which are required for the two encryption layers of Bittensor wallet mnemonics stored in the PostgreSQL database. ```python >>> import secrets >>> secrets.token_bytes(64).hex() '8122a176b26d5e5b4bdd8a53a51137f2c3e988269a1f606e621f15eadeba049a8872ecabc4fa96502cc91a1350c247bc511ebd587db520aadfdfa85345f4867a' >>> secrets.token_bytes(32).hex() '81e161b658f53ee95dbb3457b1cc6205071ba4eadc1455d388a66b0a6b6d026d' ``` -------------------------------- ### Running Ansible Site Playbook for Node Provisioning Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This command executes the main Ansible playbook, `site.yml`, using the specified inventory file, `inventory.yml`. This playbook is responsible for provisioning and bootstrapping the primary components of the Chutes API infrastructure. ```bash ansible-playbook -i inventory.yml site.yml ``` -------------------------------- ### Deploying Kubernetes Services with Helm Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md These commands first render the Helm chart into a "prod.yaml" Kubernetes manifest file and then apply this manifest to the "chutes" namespace. This deploys all services defined in the Helm chart to the Kubernetes cluster. ```bash helm template . > prod.yaml microk8s kubectl apply -f prod.yaml -n chutes ``` -------------------------------- ### Running Ansible Playbook to Join Kubernetes Cluster Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This command runs the `join-cluster.yml` Ansible playbook, using the specified inventory. This playbook is designed to add newly provisioned Kubernetes nodes to a single primary Kubernetes instance. ```bash ansible-playbook -i inventory.yml join-cluster.yml ``` -------------------------------- ### Configuring Postgres Database Kubernetes Secret (Bash) Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This snippet shows how to create a Kubernetes generic secret for PostgreSQL database connection details. It includes the username, password, full connection URL, hostname/IP, port, and database name. This secret is critically important as the database stores encrypted Bittensor wallet mnemonics and other vital Chutes API data. ```bash microk8s kubectl create secret generic postgres-secret \ --from-literal="username=[username, preferred chutes]" \ --from-literal="password=[password]" \ --from-literal="url=postgresql+asyncpg://[username]:[URL safe password]@[hostname/IP]:[port]/chutes" \ --from-literal="hostname=[hostname/IP]" \ --from-literal="port=5432" \ --from-literal="database=chutes" \ -n chutes ``` -------------------------------- ### Configuring Bittensor Wallet Encryption Keys Kubernetes Secret (Bash) Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This snippet demonstrates how to create a Kubernetes generic secret for the Bittensor wallet encryption keys. It requires two hexadecimal strings: a 64-character 'wallet-key' and a 128-character 'pg-key', which are used to doubly encrypt wallet mnemonics stored in the PostgreSQL database. ```bash microk8s kubectl create secret generic wallet-secret \ --from-literal="wallet-key=[hex string with 64 chars]" \ --from-literal="pg-key=[hex string with 128 chars]" \ -n chutes ``` -------------------------------- ### Creating Kubernetes Secret for GraVal Database Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This command creates a Kubernetes generic secret named "gravaldb-secret" in the "chutes" namespace. It stores the PostgreSQL password for the GraVal database, enabling secure access for the daemonset. ```bash microk8s kubectl create secret generic gravaldb-secret \ --from-literal="password=[password]" \ -n chutes ``` -------------------------------- ### Configuring Validator Credentials Kubernetes Secret (Bash) Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This snippet demonstrates creating a Kubernetes generic secret for validator hotkey credentials. It stores the validator's ss58Address and secretSeed (without the '0x' prefix), which are used for signing requests to miners and setting weights within the Bittensor network. ```bash microk8s kubectl create secret generic validator-credentials \ --from-literal="ss58=[nhotkey ss58Address]" \ --from-literal="seed=[hotkey secretSeed, strip 0x prefix]" \ -n chutes ``` -------------------------------- ### Creating Kubernetes Secret for Redis Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This command creates a Kubernetes generic secret named "redis-secret" in the "chutes" namespace. It stores the Redis password and connection URL as literals, which are required for the Redis component to connect securely. ```bash microk8s kubectl create secret generic redis-secret \ --from-literal="password=[password]" \ --from-literal="url=redis://:[password]@redis.chutes.svc.cluster.local:6379/0" \ -n chutes ``` -------------------------------- ### Checking Kubernetes Pod Status Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This command retrieves the status of all pods in the "chutes" namespace, displaying detailed information in a wide format. It's used to verify that all deployed components are in a "Running" state. ```bash microk8s kubectl get po -n chutes -o wide ``` -------------------------------- ### Creating Kubernetes Secret for IP Check Salt Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This command creates a Kubernetes generic secret named "ip-check-salt" in the "chutes" namespace. It stores the generated salt, which is used for UUIDv5 token generation in the IP check mechanism. ```bash microk8s kubectl create secret generic ip-check-salt \ --from-literal="salt=[salt]" \ -n chutes ``` -------------------------------- ### Configuring Local Docker Registry Password Kubernetes Secret (Bash) Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This snippet shows how to create a Kubernetes generic secret for the local Docker registry's password. A randomly generated secure string should be used for the password. This secret is specifically required for the forge process within the Chutes API. ```bash microk8s kubectl create secret generic registry-secret \ --from-literal="password=[password] \ -n chutes ``` -------------------------------- ### Configuring Docker Pull Credentials for Forge Process Kubernetes Secret (Bash) Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This snippet demonstrates creating a Kubernetes generic secret specifically for the forge process to pull Docker images. It requires a username and password. This secret complements the local Docker registry password and is essential for the forge's image management operations. ```bash microk8s kubectl create secret generic docker-pull \ --from-literal="username=[username]" \ --from-literal="password=[password]" \ -n chutes ``` -------------------------------- ### Configuring Docker Hub Credentials Kubernetes Secret (Bash) Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This snippet illustrates how to create a Kubernetes Docker registry secret for Docker Hub credentials. It includes the Docker server URL, username, password, and email address. This secret is necessary to avoid rate-limiting when pulling or building Docker images. ```bash microk8s kubectl create secret docker-registry regcred \ --docker-server=docker.io \ --docker-username=[username] \ --docker-password=[password] \ --docker-email=[email address] \ -n chutes ``` -------------------------------- ### Configuring S3 Credentials Kubernetes Secret (Bash) Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This snippet demonstrates how to create a Kubernetes generic secret for S3-compatible object storage credentials. It requires an access key ID, secret access key, bucket name, endpoint URL, and AWS region (or 'auto' for GCS) to be provided as literal values. This secret is essential for storing image build contexts, logos, and Docker registry data. ```bash microk8s kubectl create secret generic s3-credentials \ --from-literal="access-key-id=[access key id]" \ --from-literal="secret-access-key=[secret access key]" \ --from-literal="bucket=[bucket name]" \ --from-literal="endpoint-url=[endpoint URL, e.g. https://storage.googleapis.com for GCS]" \ --from-literal="aws-region=[AWS region, or auto for GCS]" \ -n chutes ``` -------------------------------- ### Generating GraVal Database Password (Python) Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md Similar to Redis, this Python one-liner executed in bash generates a UUIDv4 suitable for use as the PostgreSQL password for the GraVal database. This ensures a unique and secure password. ```bash python3 -c 'import uuid; print(uuid.uuid4())' ``` -------------------------------- ### Generating Redis Password (Python) Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This snippet uses a Python one-liner executed in bash to generate a UUIDv4, which is recommended for use as the Redis password. UUIDs provide a strong, unique password. ```bash python3 -c 'import uuid; print(uuid.uuid4())' ``` -------------------------------- ### Generating IP Check Salt (Python) Source: https://github.com/rayonlabs/chutes-api/blob/main/README.md This snippet generates a UUIDv4 string using Python, which is used as a salt for creating UUIDv5 tokens for IP checks. The salt can be changed at any time without affecting existing tokens. ```bash python3 -c 'import uuid; print(uuid.uuid4())' ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.