### Update PyTorch Template README Example Source: https://github.com/runpod/containers/blob/main/scripts/README.md Example of how to update the README for the PyTorch template using its ID and path. ```bash # Update the PyTorch template README ./update-template-readme.sh abc123 official-templates/pytorch ``` -------------------------------- ### Update ComfyUI Template README Example Source: https://github.com/runpod/containers/blob/main/scripts/README.md Example of how to update the README for the ComfyUI template using its ID and path. ```bash # Update the ComfyUI template README ./update-template-readme.sh def456 official-templates/stable-diffusion-comfyui ``` -------------------------------- ### Run container with a pre-start hook Source: https://context7.com/runpod/containers/llms.txt Injects a custom pre-start script into the container to install dependencies before JupyterLab launches. Ensure the script is executable. ```bash # Inject a pre-start hook to install extra dependencies before JupyterLab starts docker run -d \ -v $(pwd)/my_pre_start.sh:/pre_start.sh \ -e JUPYTER_PASSWORD=mypassword \ runpod/base:1.0.3-cuda1290-ubuntu2404 ``` -------------------------------- ### Dockerfile for Custom RunPod Template Source: https://context7.com/runpod/containers/llms.txt This Dockerfile demonstrates extending a RunPod PyTorch image, installing custom Python packages, and setting up pre-start and post-start hooks for service initialization. ```dockerfile # Dockerfile — custom template example FROM runpod/pytorch:1.0.3-cu1281-torch280-ubuntu2404 # Install your application RUN pip install --no-cache-dir my-inference-server==1.2.3 # nginx proxy: expose internal port 5000 via external port 5001 # (add a server block to nginx.conf or use an additional config snippet) # Pre-start hook: runs before JupyterLab and SSH COPY pre_start.sh /pre_start.sh RUN chmod +x /pre_start.sh # Post-start hook: runs after everything is up COPY post_start.sh /post_start.sh RUN chmod +x /post_start.sh CMD ["/start.sh"] ``` -------------------------------- ### Post-start Script for Inference Server Source: https://context7.com/runpod/containers/llms.txt This bash script starts an inference server in the background after SSH and Jupyter are up. It redirects server logs to a file. ```bash #!/bin/bash nohup my-inference-server --port 5000 --model /workspace/models &> /workspace/server.log & ``` -------------------------------- ### Dockerfile for AutoResearch Template Source: https://context7.com/runpod/containers/llms.txt This Dockerfile sets up the AutoResearch environment by installing runpodctl, cloning the autoresearch repository, and preparing Python dependencies and data. It uses a base image and specific versions for runpodctl and autoresearch. ```dockerfile # official-templates/autoresearch/Dockerfile (key steps) ARG BASE_IMAGE=non-existing FROM ${BASE_IMAGE} ARG RUNPODCTL_VERSION=v2.1.6 RUN wget -qO- https://github.com/runpod/runpodctl/releases/download/${RUNPODCTL_VERSION}/runpodctl-linux-amd64.tar.gz | \ tar -xz -C /usr/local/bin runpodctl ARG AUTORESEARCH_REF=master RUN git clone --branch ${AUTORESEARCH_REF} --depth 1 \ https://github.com/runpod/autoresearch.git /opt/autoresearch WORKDIR /opt/autoresearch RUN uv sync # install Python deps RUN uv run prepare.py # download data + train tokenizer (~2 min) COPY pre_start.sh /pre_start.sh # syncs /opt/autoresearch → /workspace on first boot ENTRYPOINT ["/start.sh"] ``` -------------------------------- ### Base Dockerfile Build Arguments Source: https://context7.com/runpod/containers/llms.txt Defines essential build arguments for the base image Dockerfile, including the required `BASE_IMAGE` and optional arguments to skip Python or JupyterLab installations. ```dockerfile # official-templates/base/Dockerfile ARG BASE_IMAGE=non-existing # Required: the upstream CUDA or Ubuntu image ARG RP_SKIP_PYTHON # Set to any value to skip Python installation ARG RP_SKIP_JUPYTER # Set to any value to skip JupyterLab installation FROM ${BASE_IMAGE} # Workspace root — all user data, caches, and models live here ENV RP_WORKSPACE=/workspace ``` -------------------------------- ### Pull and verify PyTorch image Source: https://context7.com/runpod/containers/llms.txt Pulls a specific PyTorch image version and runs a quick check to confirm PyTorch installation and CUDA availability. ```bash # Pull PyTorch 2.8.0 + CUDA 12.8.1 on Ubuntu 24.04 docker pull runpod/pytorch:1.0.3-cu1281-torch280-ubuntu2404 # Quick GPU check docker run --rm --gpus all \ runpod/pytorch:1.0.3-cu1281-torch280-ubuntu2404 \ python -c "import torch; print(torch.__version__, torch.cuda.is_available())" # Output: 2.8.0+cu128 True ``` -------------------------------- ### Run container with SSH access Source: https://context7.com/runpod/containers/llms.txt Starts a container in detached mode (`-d`), maps port 22, and injects a public SSH key for remote access. ```bash # Run with SSH access enabled docker run -d --rm \ -p 22:22 \ -e PUBLIC_KEY="$(cat ~/.ssh/id_rsa.pub)" \ runpod/base:1.0.3-ubuntu2204 ``` -------------------------------- ### Update Template README Script Source: https://github.com/runpod/containers/blob/main/scripts/README.md Use this bash script to update a Runpod template's README.md file via the Runpod API. Ensure the RUNPOD_API_KEY environment variable is set and Node.js is installed for JSON escaping. ```bash ./update-template-readme.sh ``` -------------------------------- ### Container Startup Script Execution Order Source: https://context7.com/runpod/containers/llms.txt The `start.sh` script orchestrates the container startup process, including nginx, SSH, JupyterLab, environment variable export, and hook scripts. ```bash # container-template/start.sh — execution order: # 1. start nginx # 2. run /pre_start.sh (template or user hook, optional) # 3. setup SSH (if PUBLIC_KEY env var is set) # 4. start JupyterLab (if JUPYTER_PASSWORD env var is set) # 5. export env vars (writes /etc/rp_environment sourced in .bashrc) # 6. run /post_start.sh (template or user hook, optional) # 7. sleep infinity (keeps the container alive) ``` -------------------------------- ### Pull and Use Runpod Base Images Source: https://context7.com/runpod/containers/llms.txt Demonstrates how to pull pre-built base images from Docker Hub and how to use them as a base in your own Dockerfiles. ```bash # Pull a ready-made base image — GPU (CUDA 12.9.0, Ubuntu 24.04) docker pull runpod/base:1.0.3-cuda1290-ubuntu2404 ``` ```bash # Pull the CPU-only variant docker pull runpod/base:1.0.3-ubuntu2204 ``` ```dockerfile # Use as a FROM in your own Dockerfile FROM runpod/base:1.0.3-cuda1281-ubuntu2204 RUN pip install my-ml-library ``` -------------------------------- ### Build Images with bake.sh Source: https://context7.com/runpod/containers/llms.txt Use `bake.sh` as the primary entry point for local and CI builds. It ensures consistent versioning by prepending shared definitions to template-specific bake files. ```bash # Build all default targets for the base template ./bake.sh base ``` ```bash # Build a single named target ./bake.sh base cuda-ubuntu2204-1290 ``` ```bash # Build and load the resulting image into the local Docker daemon ./bake.sh base --load ``` ```bash # Build the pytorch template (all CUDA × Ubuntu × Torch combinations) ./bake.sh pytorch ``` ```bash # Build only the dev target (single fast build for local iteration) ./bake.sh pytorch --set "*.group=dev" --load ``` ```bash # Build the ROCm template ./bake.sh rocm ``` ```bash # Build the NVIDIA PyTorch NGC template ./bake.sh nvidia-pytorch ``` ```bash # Pass any extra docker buildx bake flags ./bake.sh base --progress=plain --no-cache ``` -------------------------------- ### Use PyTorch image as a base for custom Dockerfile Source: https://context7.com/runpod/containers/llms.txt Demonstrates how to use a pre-built PyTorch image as a base for your own Dockerfile, copying training scripts and setting the command to run them. ```Dockerfile FROM runpod/pytorch:1.0.3-cu1290-torch271-ubuntu2204 COPY train.py /workspace/train.py CMD ["python", "/workspace/train.py"] ``` -------------------------------- ### Pre-start Script for Model Download Source: https://context7.com/runpod/containers/llms.txt This bash script downloads model weights using huggingface-cli before the pod is marked ready. It checks if the model file already exists to avoid re-downloading. ```bash #!/bin/bash mkdir -p /workspace/models if [ ! -f /workspace/models/model.safetensors ]; then huggingface-cli download myorg/mymodel \ --local-dir /workspace/models \ --token "$HUGGINGFACE_TOKEN" fi ``` -------------------------------- ### Build and Run NVIDIA PyTorch Template Source: https://context7.com/runpod/containers/llms.txt Builds the NVIDIA PyTorch template using a bake script or pulls and runs a pre-built image with GPU access. Ensure your public SSH key is available. ```bash # Build the nvidia-pytorch template ./bake.sh nvidia-pytorch ``` ```bash # Pull and use the pre-built image docker pull runpod/nvidia-pytorch:1.0.3-25.11 ``` ```bash # Run with GPU access docker run --rm --gpus all \ -e PUBLIC_KEY="$(cat ~/.ssh/id_rsa.pub)" \ runpod/nvidia-pytorch:1.0.3-25.11 ``` -------------------------------- ### Initiate Autoresearch Experiment Source: https://github.com/runpod/containers/blob/main/official-templates/autoresearch/README.md Connect your coding agent via SSH and instruct it to read the program markdown to begin a new experiment. ```bash Read /workspace/autoresearch/program.md and let's kick off a new experiment! ``` -------------------------------- ### Launch and Connect AutoResearch Pod Source: https://context7.com/runpod/containers/llms.txt Launches the AutoResearch container with GPU access, port forwarding for SSH, and volume mounting for workspace persistence. After launching, connect via SSH to instruct the AI coding agent. ```bash # Pull image docker pull runpod/autoresearch:1.0.3-cuda1281-ubuntu2404 ``` ```bash # Launch and connect a coding agent via SSH docker run -d --gpus all \ -p 22:22 \ -v myvolume:/workspace \ -e PUBLIC_KEY="$(cat ~/.ssh/id_rsa.pub)" \ runpod/autoresearch:1.0.3-cuda1281-ubuntu2404 ``` ```bash # SSH in and instruct Claude Code / Cursor to start experimenting ssh root@ -p 22 # Then tell your agent: # "Read /workspace/autoresearch/program.md and kick off a new experiment!" ``` -------------------------------- ### Build NVIDIA PyTorch Container Source: https://github.com/runpod/containers/blob/main/official-templates/nvidia-pytorch/README.md Execute this script to build the NVIDIA PyTorch container image. ```bash ./bake.sh nvidia-pytorch ``` -------------------------------- ### Fetch and Convert README Markdown Source: https://github.com/runpod/containers/blob/main/container-template/proxy/readme.html Fetches the README.md file and converts its content from Markdown to HTML using the Showdown converter. The resulting HTML is then inserted into the page's content element. ```javascript fetch('/README.md') .then(response => response.text()) .then(text => { // Convert markdown to HTML var html = converter.makeHtml(text); // Insert HTML into the page document.getElementById('content').innerHTML = html; }); ``` -------------------------------- ### Instantiate Showdown Converter Source: https://github.com/runpod/containers/blob/main/container-template/proxy/readme.html Initializes the Showdown Markdown converter. This is required before converting any Markdown text. ```javascript var converter = new showdown.Converter(); ``` -------------------------------- ### Run container with JupyterLab Source: https://context7.com/runpod/containers/llms.txt Launches a container, maps port 8888, sets a Jupyter password, and mounts the current directory to the workspace. Use `-it` for interactive sessions. ```bash # Run the base image locally and activate JupyterLab on port 8888 docker run -it --rm \ -p 8888:8888 \ -e JUPYTER_PASSWORD=mysecretpassword \ -v $(pwd)/workspace:/workspace \ runpod/base:1.0.3-cuda1281-ubuntu2204 ``` -------------------------------- ### Build Runpod Containers with Bake Script Source: https://github.com/runpod/containers/blob/main/README.md Use the bake.sh script to build Runpod containers. It supports building default targets for a template, specific targets, or groups of targets, and can load them to the local Docker daemon. ```bash # Build the default targets for a template ./bake.sh base ``` ```bash # Build a specific target or group of targets ./bake.sh base cuda-ubuntu2204-1290 ``` ```bash # Build the default targets and load them to the local Docker daemon ./bake.sh base --load ``` -------------------------------- ### Run NCCL Verification Script Source: https://github.com/runpod/containers/blob/main/helper-templates/verify-nccl/README.md Execute this script after SSHing into the pod to test NCCL functionality. Ensure you are in the correct directory. ```bash ./check_nccl.sh ``` -------------------------------- ### GitHub Actions Workflow for Building Docker Images Source: https://context7.com/runpod/containers/llms.txt This workflow uses docker/bake-action to build and push Docker images based on HCL files. It depends on a previous 'build-base' job and checks for its success or skip status. ```yaml on: push: paths: - "official-templates/base/**" - "official-templates/pytorch/**" - "official-templates/autoresearch/**" - "official-templates/shared/**" jobs: build-base: runs-on: blacksmith-8vcpu-ubuntu-2204 steps: - uses: actions/checkout@v4 - uses: ./.github/actions/docker-setup id: setup with: dockerhub-username: ${{ secrets.DOCKERHUB_USERNAME }} dockerhub-token: ${{ secrets.DOCKERHUB_TOKEN }} - uses: docker/bake-action@v6 env: RELEASE_SUFFIX: ${{ steps.setup.outputs.release-suffix }} with: files: official-templates/shared/versions.hcl official-templates/base/docker-bake.hcl push: true build-pytorch: needs: build-base # waits for base images to be pushed first if: always() && (needs.build-base.result == 'success' || needs.build-base.result == 'skipped') runs-on: blacksmith-16vcpu-ubuntu-2404 steps: - uses: docker/bake-action@v6 with: files: official-templates/shared/versions.hcl official-templates/pytorch/docker-bake.hcl push: true ``` -------------------------------- ### Set HuggingFace and pip cache directories Source: https://context7.com/runpod/containers/llms.txt Configure environment variables to redirect HuggingFace, pip, and virtualenv caches to the persistent workspace volume for faster subsequent runs. ```Dockerfile ENV HF_HOME="${RP_WORKSPACE}/.cache/huggingface/" # pip / virtualenv / uv caches also live on the workspace volume ENV PIP_CACHE_DIR="${RP_WORKSPACE}/.cache/pip/" ENV UV_CACHE_DIR="${RP_WORKSPACE}/.cache/uv/" ENV VIRTUALENV_OVERRIDE_APP_DATA="${RP_WORKSPACE}/.cache/virtualenv/" # Accelerate HuggingFace Hub downloads ENV HF_HUB_ENABLE_HF_TRANSFER=1 ENV HF_XET_HIGH_PERFORMANCE=1 ``` -------------------------------- ### Update Template README via RunPod API Source: https://context7.com/runpod/containers/llms.txt Updates a template's README on the RunPod platform using a bash script and the RunPod GraphQL API. Requires setting the RUNPOD_API_KEY environment variable and providing the template ID and path. ```bash # Set your API key export RUNPOD_API_KEY=your_api_key_here # Update the pytorch template README ./scripts/update-template-readme.sh abc123templateid official-templates/pytorch # Update the base template README ./scripts/update-template-readme.sh def456templateid official-templates/base # Expected output on success: # Updating template abc123templateid with README from official-templates/pytorch/README.md... # ✅ GraphQL mutation successful for template abc123templateid: # {"data":{"adminUpdatePodTemplate":{"id":"abc123templateid","name":"Runpod PyTorch"}}} ``` -------------------------------- ### PyTorch Docker Bake configuration Source: https://context7.com/runpod/containers/llms.txt Defines CUDA-Torch combinations and tag patterns for building PyTorch images. Specifies base images and pip wheel sources. ```HCL # official-templates/pytorch/docker-bake.hcl (key variables) variable "CUDA_TORCH_COMBINATIONS" { default = [ { cuda_version = "12.8.1", torch = "2.7.1", whl_src = "128" }, { cuda_version = "12.8.1", torch = "2.8.0", whl_src = "128" }, { cuda_version = "12.9.0", torch = "2.8.0", whl_src = "129" }, { cuda_version = "12.9.0", torch = "2.9.1", whl_src = "129" }, { cuda_version = "13.0.0", torch = "2.9.1", whl_src = "130" }, # ... full matrix in docker-bake.hcl ] } target "pytorch-matrix" { # tag pattern: runpod/pytorch:-cu-torch- tags = ["runpod/pytorch:${RELEASE_VERSION}${RELEASE_SUFFIX}-cu${build.cuda_code}-torch${build.torch_code}-${build.ubuntu_name}"] args = { BASE_IMAGE = "runpod/base:${RELEASE_VERSION}${RELEASE_SUFFIX}-cuda${build.cuda_code}-${build.ubuntu_name}" WHEEL_SRC = build.wheel_src TORCH = "torch==${build.torch} torchvision==${build.torch_vision} torchaudio==${build.torch}" } } ``` -------------------------------- ### Activate ROCm conda environment and verify GPU Source: https://context7.com/runpod/containers/llms.txt Shows how to activate the conda environment within a ROCm container and verify that the AMD GPU is recognized by PyTorch. ```bash # Activate the conda environment after SSH into a ROCm pod conda init && source ~/.bashrc && conda activate py_3.12 # Verify AMD GPU is visible python -c "import torch; print(torch.cuda.is_available(), torch.version.hip)" # Output: True 6.4.42421-... ``` -------------------------------- ### Available ROCm images Source: https://context7.com/runpod/containers/llms.txt Lists the available RunPod base images with ROCm support for AMD GPU acceleration, specifying Python and PyTorch versions. ```bash # Available images docker pull runpod/base:1.0.3-rocm644-ubuntu2204-py310-pytorch260 docker pull runpod/base:1.0.3-rocm644-ubuntu2404-py312-pytorch271 ``` -------------------------------- ### Base Image Bake Definition Source: https://context7.com/runpod/containers/llms.txt Defines build targets for the base image, including CPU-only and CUDA-enabled variants using a matrix strategy. It specifies context, Dockerfile, platforms, and necessary arguments. ```hcl # official-templates/base/docker-bake.hcl (simplified) group "default" { targets = ["cpu", "cuda-matrix"] } target "common-base" { context = "official-templates/base" dockerfile = "Dockerfile" platforms = ["linux/amd64"] contexts = { scripts = "container-template" # provides start.sh proxy = "container-template/proxy" # provides nginx.conf + snippets logo = "container-template" # provides runpod.txt } } target "cpu-ubuntu2204" { inherits = ["common-base"] tags = ["runpod/base:${RELEASE_VERSION}${RELEASE_SUFFIX}-ubuntu2204"] args = { BASE_IMAGE = "ubuntu:22.04" } } target "cuda-matrix" { inherits = ["common-base"] name = "cuda-${combo.ubuntu_name}-${replace(combo.cuda_version, ".", "")}" matrix = { combo = flatten([ for cuda in CUDA_VERSIONS : [ for ubuntu in UBUNTU_VERSIONS : { ubuntu_version = ubuntu.version ubuntu_name = ubuntu.name cuda_code = replace(cuda.version, ".", "") cuda_version = cuda.version } if contains(cuda.ubuntu, ubuntu.version) ] ]) } tags = ["runpod/base:${RELEASE_VERSION}${RELEASE_SUFFIX}-cuda${combo.cuda_code}-${combo.ubuntu_name}"] args = { BASE_IMAGE = "nvidia/cuda:${combo.cuda_version}-cudnn-devel-ubuntu${combo.ubuntu_version}" } } ``` -------------------------------- ### Activate Python venv (Ubuntu 24.04) Source: https://github.com/runpod/containers/blob/main/official-templates/rocm/README.md Use this command to activate the Python 3.12 virtual environment for ROCm images on Ubuntu 24.04. ```bash conda init && source ~/.bashrc && conda activate py_3.12 ``` -------------------------------- ### Activate Python venv (Ubuntu 22.04) Source: https://github.com/runpod/containers/blob/main/official-templates/rocm/README.md Use this command to activate the Python 3.10 virtual environment for ROCm images on Ubuntu 22.04. ```bash conda init && source ~/.bashrc && conda activate py_3.10 ``` -------------------------------- ### Export environment variables for container sessions Source: https://context7.com/runpod/containers/llms.txt Environment variables set at pod launch are exported to `/etc/rp_environment` and sourced in `.bashrc`, making them available in interactive shells. ```bash # Variables are exported at container start and available in any shell: # /etc/rp_environment ← auto-sourced by /root/.bashrc # Example: pass a HuggingFace token at pod launch docker run -d \ -e HUGGINGFACE_TOKEN=hf_xxxx \ -e MY_MODEL=meta-llama/Llama-3-8B \ runpod/base:1.0.3-cuda1281-ubuntu2204 # Inside the container, both variables are immediately available: echo $HUGGINGFACE_TOKEN # hf_xxxx echo $MY_MODEL # meta-llama/Llama-3-8B ``` -------------------------------- ### GitHub Actions Workflow Step to Update Template README Source: https://github.com/runpod/containers/blob/main/scripts/README.md Integrate the template README update script into a GitHub Actions workflow. Set the RUNPOD_API_KEY and TEMPLATE_ID_SECRET environment variables appropriately. ```yaml - name: Update template README env: RUNPOD_API_KEY: ${{ secrets.RUNPOD_API_KEY }} run: ./scripts/update-template-readme.sh ${{ secrets.TEMPLATE_ID_SECRET }} official-templates/your-template ``` -------------------------------- ### Shared Version Matrix Configuration Source: https://context7.com/runpod/containers/llms.txt Centralizes CUDA and Ubuntu version declarations for consistent matrix builds across all templates. The `RELEASE_SUFFIX` variable is injected by CI for non-main builds. ```hcl # official-templates/shared/versions.hcl RELEASE_VERSION = "1.0.3" variable "RELEASE_SUFFIX" { default = "" # Injected by CI for non-main builds, e.g. "-dev-my-branch" } UBUNTU_VERSIONS = [ { version = "22.04", name = "ubuntu2204" }, { version = "24.04", name = "ubuntu2404" } ] CUDA_VERSIONS = [ { version = "12.8.1", ubuntu = ["22.04", "24.04"] }, { version = "12.9.0", ubuntu = ["22.04", "24.04"] }, { version = "13.0.0", ubuntu = ["24.04"] } ] ``` -------------------------------- ### GitHub Actions Workflow for README Update Source: https://context7.com/runpod/containers/llms.txt A GitHub Actions workflow that automatically updates a template's README on RunPod. It uses secrets for the API key and template ID, and executes the update script. ```yaml # .github/workflows/example-readme-update.yml - name: Update template README env: RUNPOD_API_KEY: ${{ secrets.RUNPOD_API_KEY }} run: | ./scripts/update-template-readme.sh \ ${{ secrets.PYTORCH_TEMPLATE_ID }} \ official-templates/pytorch ``` -------------------------------- ### NGINX Reverse Proxy Configuration Source: https://context7.com/runpod/containers/llms.txt Configures Nginx to proxy external ports to internal localhost ports for various services like JupyterLab, InvokeAI, Stable Diffusion WebUI, code-server, and RunPod CLI. Ensure the 'snippets/nginx-proxy.conf' is available. ```nginx # container-template/proxy/nginx.conf (key server blocks) # JupyterLab — port 8888 is handled natively by RunPod's proxy # InvokeAI: external 9091 → internal 9090 server { listen 9091; location / { include snippets/nginx-proxy.conf; proxy_pass http://localhost:9090; } } # Stable Diffusion WebUI / ComfyUI: external 3001 → internal 3000 server { listen 3001; location /ws { include snippets/nginx-proxy.conf; proxy_pass http://localhost:3000; } location / { include snippets/nginx-proxy.conf; proxy_pass http://localhost:3000; } } # code-server (VS Code): external 8081 → internal 8080 server { listen 8081; location / { include snippets/nginx-proxy.conf; proxy_pass http://localhost:8080; } } # RunPod CLI FastAPI (hex "rp" = 0x7270): external 7270 → internal 7271 server { listen 7270; location / { include snippets/nginx-proxy.conf; proxy_pass http://localhost:7271; } } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.