### Create and Activate a uv Virtual Environment Source: https://github.com/pytorch/helion/blob/main/docs/installation.md This section guides users on creating and activating a virtual environment using 'uv', a recommended tool for managing Python dependencies. ```bash # Create a new virtual environment in .venv (one-time) uv venv .venv # Activate the environment source .venv/bin/activate ``` -------------------------------- ### Install Triton from PyPI (Prebuilt wheel) Source: https://github.com/pytorch/helion/blob/main/docs/installation.md Recommended method for installing Triton using a prebuilt wheel from PyPI, which automatically selects the appropriate version for the platform. ```bash # Install Triton from PyPI (will pick the right wheel for your platform if available) pip install "triton>=3.5" # Verify python -c "import triton; print('Triton version:', triton.__version__)" ``` -------------------------------- ### Implementing RemoteCacheBackend Source: https://github.com/pytorch/helion/blob/main/docs/deployment_autotuning.md Example of how to implement the RemoteCacheBackend abstract base class. ```python from helion.autotuner.remote_cache import RemoteCacheBackend class MyBackend(RemoteCacheBackend): def get(self, key: str) -> str | None: ... def put(self, key: str, data: str) -> None: ... # Optional: implement to enable warm-start lookups # (from_best_available / helion.from_cache pull from the remote # when local matches are below autotune_best_available_max_configs). def list(self, max_results: int | None = None): ... ``` -------------------------------- ### Setup Dependencies Source: https://github.com/pytorch/helion/blob/main/docs/helion_tutorials.md Imports necessary libraries for Helion, PyTorch, and logging. ```python import logging import helion import helion.language as hl import torch from torch import Tensor # If you set this to info you will see the output Triton Code logging.getLogger().setLevel(logging.WARNING) ``` -------------------------------- ### Example (by name) Source: https://github.com/pytorch/helion/blob/main/docs/api/language.md Example of using inline_triton with a named Triton kernel. ```python @triton.jit def add_pairs(a, b): return a + b @helion.kernel() def k(x: torch.Tensor, y: torch.Tensor) -> torch.Tensor: out = torch.empty_like(x) for tile in hl.tile(x.shape): out[tile] = hl.triton_kernel("add_pairs", args=(x[tile], y[tile]), output_like=x[tile]) return out ``` -------------------------------- ### Build Documentation Locally Source: https://github.com/pytorch/helion/blob/main/docs/installation.md Instructions for building the project's documentation locally, requiring the 'docs' extra dependencies. ```bash pip install -e '.[docs]' cd docs make html ``` -------------------------------- ### Install Helion from Source Source: https://github.com/pytorch/helion/blob/main/README.md Commands to clone the repository, set up a virtual environment with uv, and install Helion in editable mode with development dependencies. ```bash git clone https://github.com/pytorch/helion.git cd helion # Create and activate a virtual environment with uv (one-time) uv venv .venv source .venv/bin/activate # To install in editable w/ required dev packages pip install -e .'[dev]' ``` -------------------------------- ### Example: Using Claude as the LLM provider Source: https://github.com/pytorch/helion/blob/main/docs/api/autotuner.md Configuration example for using Claude as the LLM provider with the autotuner. ```bash export HELION_AUTOTUNER=LLMSeededLFBOTreeSearch export HELION_LLM_PROVIDER=anthropic export HELION_LLM_MODEL=claude-opus-4-7 export HELION_LLM_API_KEY=your-key-here ``` -------------------------------- ### Install PyTorch and Helion Source: https://github.com/pytorch/helion/blob/main/notebooks/softmax.ipynb Installs PyTorch with CUDA support and the Helion library. ```python %pip install "torch==2.9.*" --index-url https://download.pytorch.org/whl/cu126 %pip install helion ``` -------------------------------- ### Install Helion (Post PyTorch/Triton) Source: https://github.com/pytorch/helion/blob/main/docs/installation.md Final step to install Helion after setting up PyTorch and Triton, offering both PyPI and development installation options. ```bash # From PyPI pip install helion # Development installation git clone https://github.com/pytorch/helion.git cd helion pip install -e '.[dev]' ``` -------------------------------- ### Deploying Multiple Configurations Source: https://github.com/pytorch/helion/blob/main/docs/deployment_autotuning.md Example of supplying a list of candidate configurations to the kernel decorator. ```python candidate_configs = [ helion.Config.load("configs/my_kernel_small.json"), helion.Config.load("configs/my_kernel_large.json"), ] @helion.kernel(configs=candidate_configs, static_shapes=True) def my_kernel(x, y): ... ``` -------------------------------- ### Deploying a Single Configuration Source: https://github.com/pytorch/helion/blob/main/docs/deployment_autotuning.md Example of baking a single winning configuration into the decorator. ```python best = helion.Config.load("configs/my_kernel.json") @helion.kernel(config=best) def my_kernel(x, y): ... ``` -------------------------------- ### Set up and use pre-commit hooks Source: https://github.com/pytorch/helion/blob/main/docs/installation.md Commands to set up and run pre-commit hooks for code formatting and linting in a development environment. ```bash # Install pre-commit hooks into your local git repo (one-time) pre-commit install # Run all checks across the repository pre-commit run --all-files ``` -------------------------------- ### Install PyTorch Source: https://github.com/pytorch/helion/blob/main/docs/installation.md Instructions for installing PyTorch version 2.9 or later, with specific commands for CUDA 12.8 and ROCm 7.0 environments. ```bash # CUDA 12.8 pip install "torch==2.9.*" --index-url https://download.pytorch.org/whl/cu128 # ROCm 7.0 pip install "torch==2.9.*" --index-url https://download.pytorch.org/whl/rocm7.0 ``` -------------------------------- ### Basic Usage Example: Add Kernel Source: https://github.com/pytorch/helion/blob/main/docs/tileir_backend.md A basic Helion kernel example demonstrating the use of the TileIR backend for an element-wise addition operation. ```python import torch import helion import helion.language as hl @helion.kernel( autotune_effort="none", config=helion.Config( block_sizes=[128, 128], num_ctas=2, occupancy=2, ), ) def add_kernel(x: torch.Tensor, y: torch.Tensor) -> torch.Tensor: result = torch.empty_like(x) for tile in hl.tile(x.shape): result[tile] = x[tile] + y[tile] return result # Run the kernel x = torch.randn(128, 128, device="cuda", dtype=torch.float32) y = torch.randn(128, 128, device="cuda", dtype=torch.float32) result = add_kernel(x, y) ``` -------------------------------- ### Run the AOT workflow Source: https://github.com/pytorch/helion/blob/main/docs/deployment_autotuning.md Examples of running the Helion AOT runner for both a tutorial example (layer_norm) and a user-authored kernel. ```bash # Tutorial example: tune layer_norm on the current GPU. python -m helion.experimental.aot_runner -- python pretuned_kernels/layer_norm/layer_norm.py # User-authored kernel: same pattern. python -m helion.experimental.aot_runner -- python my_benchmark.py ``` -------------------------------- ### Install Helion via pip Source: https://github.com/pytorch/helion/blob/main/docs/installation.md The simplest method to install Helion is by using pip from the Python Package Index (PyPI). ```bash pip install helion ``` -------------------------------- ### Build Triton from Source (Ubuntu/Debian) Source: https://github.com/pytorch/helion/blob/main/docs/installation.md Instructions for building Triton from source on Ubuntu/Debian systems, including installing build dependencies and setting compilers. ```bash # Install build dependencies (adjust versions as needed) sudo apt-get update sudo apt-get install -y git clang-14 clang++-14 zlib1g-dev python3-dev # (Optional) set compilers explicitly export CC=clang-14 export CXX=clang++-14 # Clone and install Triton git clone https://github.com/triton-lang/triton.git cd triton pip install -r python/requirements.txt MAX_JOBS=$(nproc) TRITON_PARALLEL_LINK_JOBS=2 pip install . # Verify and clean up python -c "import triton; print('Triton version:', triton.__version__)" cd .. ``` -------------------------------- ### Pre-commit Setup Source: https://github.com/pytorch/helion/blob/main/README.md Commands for one-time setup of pre-commit hooks and running all checks. ```bash pip install pre-commit pre-commit install ``` ```bash pre-commit run --all-files ``` -------------------------------- ### Vector Add Benchmark Example Source: https://github.com/pytorch/helion/blob/main/benchmarks/README.md Example of running the benchmark for the vector_add kernel. ```bash python benchmarks/run.py --metrics speedup,accuracy --kernel vector_add ``` -------------------------------- ### Install Documentation Dependencies Source: https://github.com/pytorch/helion/blob/main/docs/README.md Installs the necessary Python packages for building the documentation. ```bash pip install -e ".[docs]" ``` -------------------------------- ### Example: Running a kernel with autotuner Source: https://github.com/pytorch/helion/blob/main/docs/api/autotuner.md Example of running a PyTorch matmul operation after configuring the autotuner. ```python out = matmul(torch.randn([2048, 2048], device="cuda"), torch.randn([2048, 2048], device="cuda")) ``` -------------------------------- ### Setting autotune budget via environment variable Source: https://github.com/pytorch/helion/blob/main/docs/deployment_autotuning.md Example of setting the autotune budget in seconds using an environment variable. ```bash export HELION_AUTOTUNE_BUDGET_SECONDS=300 ``` -------------------------------- ### Using Environment Variables Source: https://github.com/pytorch/helion/blob/main/docs/api/settings.md Example of setting environment variables to configure Helion settings. ```bash env HELION_PRINT_OUTPUT_CODE=1 HELION_AUTOTUNE_EFFORT=none my_kernel.py ``` -------------------------------- ### 2D tiling Source: https://github.com/pytorch/helion/blob/main/docs/api/language.md Example of 2D tiling. ```python # 2D tiling for tile_i, tile_j in hl.tile([height, width]): # Each tile represents a portion of the 2D space pass ``` -------------------------------- ### Custom Key for Re-tuning Source: https://github.com/pytorch/helion/blob/main/docs/deployment_autotuning.md Example of using a custom key function for triggering re-tuning with power-of-two bucketing. ```python @helion.kernel( configs=candidate_configs, key=lambda x, y: helion.next_power_of_2(x.numel()), static_shapes=False, ) def my_kernel(x, y): ... ``` -------------------------------- ### Setting autotune budget via decorator Source: https://github.com/pytorch/helion/blob/main/docs/deployment_autotuning.md Example of setting the autotune budget in seconds using the helion.kernel decorator. ```python import helion import torch @helion.kernel(autotune_budget_seconds=300) def my_kernel(x: torch.Tensor) -> torch.Tensor: ... ``` -------------------------------- ### Manual Commands with sphinx-autobuild Source: https://github.com/pytorch/helion/blob/main/docs/README.md Examples of using sphinx-autobuild directly with custom options. ```bash # Basic live reload sphinx-autobuild docs/ site/ ``` ```bash # With custom port sphinx-autobuild docs/ site/ --port 8080 ``` ```bash # Watch additional directories (e.g., for theme development) sphinx-autobuild docs/ site/ --watch ../helion/ ``` ```bash # Force full rebuild on each change (useful for theme development) sphinx-autobuild -a docs/ site/ ``` -------------------------------- ### Manually running AOT phases Source: https://github.com/pytorch/helion/blob/main/docs/deployment_autotuning.md Examples of manually running specific phases of the AOT workflow using environment variables. ```bash HELION_AOT_MODE=collect HELION_AOT_DATA_DIR=./aot_data python my_benchmark.py ``` ```bash HELION_AOT_MODE=measure HELION_AOT_DATA_DIR=./aot_data python my_benchmark.py ``` -------------------------------- ### Simple 1D tiling Source: https://github.com/pytorch/helion/blob/main/docs/api/language.md Example of a simple 1D tiling loop. ```python # Simple 1D tiling for tile in hl.tile(1000): # tile.begin, tile.end, tile.block_size are available # Load entire tile (not just first element) data = tensor[tile] # or hl.load(tensor, tile) for explicit loading ``` -------------------------------- ### With explicit begin/end/block_size Source: https://github.com/pytorch/helion/blob/main/docs/api/language.md Example of tiling with explicit begin, end, and block_size. ```python # With explicit begin/end/block_size for tile in hl.tile(0, 1000, block_size=64): pass ``` -------------------------------- ### Build Documentation (Development Mode with Live Reload) Source: https://github.com/pytorch/helion/blob/main/docs/README.md Starts a local web server with live reload for active documentation development. ```bash cd docs/ make livehtml ``` -------------------------------- ### Manual Configuration for CUDA Tile IR Backend Source: https://github.com/pytorch/helion/blob/main/docs/tileir_backend.md Example of specifying multiple TileIR configurations for lightweight autotuning. ```python import torch import helion from helion import tileir as hl configs = [ helion.Config(block_sizes=[64, 64], num_ctas=1, occupancy=1), helion.Config(block_sizes=[64, 64], num_ctas=2, occupancy=2), helion.Config(block_sizes=[128, 128], num_ctas=2, occupancy=4), helion.Config(block_sizes=[128, 128], num_ctas=2, occupancy=8), ] @helion.kernel(configs=configs) def optimized_kernel(x: torch.Tensor) -> torch.Tensor: result = torch.empty_like(x) for tile in hl.tile(x.shape): result[tile] = x[tile] * 2 return result ``` -------------------------------- ### Setting autotune effort via environment variable Source: https://github.com/pytorch/helion/blob/main/docs/deployment_autotuning.md Example of setting the autotune effort to 'quick' using an environment variable. ```bash export HELION_AUTOTUNE_EFFORT=quick ``` -------------------------------- ### Example combining compilation control and configuration Source: https://github.com/pytorch/helion/blob/main/docs/index.md Demonstrates how to use the @helion.kernel decorator to control compilation behavior and GPU execution settings. ```python import torch import helion @helion.kernel( # Settings: Control compilation behavior autotune_effort="none", # Skip autotuning for development print_output_code=True, # Debug: show generated code # Config: Control GPU execution (when not using default) # config=helion.Config(block_sizes=[64, 32], num_warps=8) ) def debug_kernel(x: torch.Tensor, y: torch.Tensor) -> torch.Tensor: # Implementation pass ``` -------------------------------- ### Development Installation Source: https://github.com/pytorch/helion/blob/main/docs/installation.md For developers who need to modify the Helion source code, this method installs the package in editable mode along with development dependencies. ```bash # Clone the repository git clone https://github.com/pytorch/helion.git cd helion # Install in editable mode with development dependencies pip install -e '.[dev]' ``` -------------------------------- ### Manual Autotuner Construction Source: https://github.com/pytorch/helion/blob/main/docs/deployment_autotuning.md Demonstrates how to manually construct an autotuner, specifically `LFBOTreeSearch`, with custom parameters for more control over the tuning process. ```python from helion.autotuner import LFBOTreeSearch bound = my_kernel.bind(example_inputs) tuner = LFBOTreeSearch( bound, example_inputs, # Double the defaults to explore more candidates: initial_population=200, # Default is 100. copies=10, # Default is 5. max_generations=40, # Default is 20. ) best_config = tuner.autotune() best_config.save("configs/my_kernel.json") ``` -------------------------------- ### Build, Test, and Development Commands Source: https://github.com/pytorch/helion/blob/main/AGENTS.md Commands for linting, formatting, type-checking, running tests, and building documentation. ```bash ./lint.sh pre-commit run -a pytest pytest -k name_substring make -C docs html ``` -------------------------------- ### Autotuning Example Source: https://github.com/pytorch/helion/blob/main/docs/helion_tutorials.md Demonstrates how to use Helion's autotuning feature by omitting the 'config' parameter in the @helion.kernel decorator. This allows Helion to automatically find the optimal configuration parameters for the given hardware, problem size, and operation patterns. ```python @helion.kernel() # No config = automatic tuning def autotuned_add(x: torch.Tensor, y: torch.Tensor) -> torch.Tensor: m, n = x.size() out = torch.empty_like(x) for tile_m, tile_n in hl.tile([m, n]): out[tile_m, tile_n] = x[tile_m, tile_n] + y[tile_m, tile_n] ``` -------------------------------- ### Manual Routing Based on Input Size Source: https://github.com/pytorch/helion/blob/main/docs/deployment_autotuning.md Example demonstrating how to manually route kernel execution based on input tensor size using Helion's bind and compile_config methods. ```python bound = my_kernel.bind(example_inputs) small_cfg = helion.Config.load("configs/my_kernel_small.json") large_cfg = helion.Config.load("configs/my_kernel_large.json") small_run = bound.compile_config(small_cfg) # Returns a callable large_run = bound.compile_config(large_cfg) def routed_my_kernel(x, y): runner = small_run if x.numel() <= 2**16 else large_run return runner(x, y) ``` -------------------------------- ### Autotuning with Multiple Input Sizes Source: https://github.com/pytorch/helion/blob/main/docs/deployment_autotuning.md Shows how to explicitly call `autotune()` with a list of representative shapes to tune for different input sizes and save the configurations. ```python datasets = { "s": ( torch.randn(2**16, device="cuda"), torch.randn(2**16, device="cuda"), ), "m": ( torch.randn(2**20, device="cuda"), torch.randn(2**20, device="cuda"), ), "l": ( torch.randn(2**24, device="cuda"), torch.randn(2**24, device="cuda"), ), } for tag, args in datasets.items(): config = my_kernel.autotune(args) config.save(f"configs/my_kernel_{tag}.json") ``` -------------------------------- ### Verification Test Source: https://github.com/pytorch/helion/blob/main/docs/installation.md A Python code snippet to verify the Helion installation by running a simple kernel test using PyTorch and Helion's features. ```python import torch import helion import helion.language as hl @helion.kernel(autotune_effort="none") def test_kernel(x: torch.Tensor) -> torch.Tensor: out = torch.empty_like(x) for tile in hl.tile(x.shape[0]): out[tile] = x[tile] * 2 return out x = torch.randn(100, device='cuda') result = test_kernel(x) torch.testing.assert_close(result, x * 2) print("Verification successful!") ``` -------------------------------- ### Example Helion Kernel: Element-wise Addition Source: https://github.com/pytorch/helion/blob/main/docs/helion_tutorials.md A basic Helion kernel demonstrating element-wise addition using PyTorch operations within a device loop. ```python @helion.kernel(config=helion.Config(block_sizes = [128, 128])) # The @helion.kernel decorator marks this function for compilation def example_add(x: torch.Tensor, y: torch.Tensor) -> torch.Tensor: # Host code: Standard PyTorch operations m, n = x.size() out = torch.empty_like(x) # Allocate output tensor # The hl.tile loop defines the parallel execution structure for tile_m, tile_n in hl.tile([m, n]): # Device code: Everything inside the hl.tile loop runs on GPU out[tile_m, tile_n] = x[tile_m, tile_n] + y[tile_m, tile_n] # Simple element-wise addition expressed w/ pytorch ops return out # Return the result back to the host # Create some sample data x = torch.randn(10, 10, device="cuda") y = torch.randn(10, 10, device="cuda") # Run the kernel result = example_add(x, y) # Verify result expected = x + y torch.testing.assert_close(result, expected) print("✅ Results Match ✅") benchmark_kernel(example_add, x, y) compare_implementations(example_add, torch.add, x, y) ``` -------------------------------- ### Specifying multiple configurations for lightweight autotuning Source: https://github.com/pytorch/helion/blob/main/README.md This example shows how to provide multiple configurations to Helion for a more lightweight autotuning process, where the fastest configuration is selected. ```python @helion.kernel(configs=[ helion.Config(...), helion.Config(...), ]) def matmul(x: torch.Tensor, y: torch.Tensor) -> torch.Tensor: ... ``` -------------------------------- ### Internal Specialization with hl.specialize() Source: https://github.com/pytorch/helion/blob/main/docs/deployment_autotuning.md Example demonstrating how to use hl.specialize() inside a kernel to make a dimension a compile-time constant for all calls. This example shows a RMS Normalization forward pass where the hidden dimension 'n' is specialized. ```python import torch import helion import helion.language as hl @helion.kernel(static_shapes=False) def rms_norm_fwd( x: torch.Tensor, weight: torch.Tensor, eps: float = 1e-5 ) -> torch.Tensor: m, n = x.size() hl.specialize(n) # hidden dimension becomes a compile-time constant out = torch.empty_like(x) for tile_m in hl.tile(m): x_tile = x[tile_m, :].to(torch.float32) x_squared = x_tile * x_tile mean_x_squared = torch.mean(x_squared, dim=-1) inv_rms = torch.rsqrt(mean_x_squared + eps) normalized = x_tile * inv_rms[:, None] out[tile_m, :] = (normalized * weight[:].to(torch.float32)).to(out.dtype) return out # Every call specializes on n - different hidden sizes = different cache entries weight_4096 = torch.randn([4096], device="cuda") weight_2048 = torch.randn([2048], device="cuda") result1 = rms_norm_fwd(torch.randn([2048, 4096], device="cuda"), weight_4096) # compiles for n=4096 result2 = rms_norm_fwd(torch.randn([1024, 4096], device="cuda"), weight_4096) # reuses n=4096 result3 = rms_norm_fwd(torch.randn([2048, 2048], device="cuda"), weight_2048) # compiles for n=2048 ``` -------------------------------- ### External Specialization with torch._dynamo.mark_static() Source: https://github.com/pytorch/helion/blob/main/docs/deployment_autotuning.md Example demonstrating how to use torch._dynamo.mark_static() before calling a kernel to specialize dimensions on specific tensors. This allows the same kernel definition to serve both dynamic and specialized code paths. The example shows a matrix multiplication kernel. ```python @helion.kernel(static_shapes=False) def matmul(x: torch.Tensor, y: torch.Tensor) -> torch.Tensor: m, k = x.size() k2, n = y.size() out = torch.empty([m, n], device=x.device, dtype=x.dtype) for tile_m, tile_n in hl.tile([m, n]): acc = hl.zeros([tile_m, tile_n], dtype=torch.float32) for tile_k in hl.tile(k): acc = torch.addmm(acc, x[tile_m, tile_k], y[tile_k, tile_n]) out[tile_m, tile_n] = acc.to(x.dtype) return out # Dynamic call - all dimensions remain symbolic x_dyn = torch.randn([m, k], device="cuda", dtype=torch.float16) y_dyn = torch.randn([k, n], device="cuda", dtype=torch.float16) result = matmul(x_dyn, y_dyn) # Specialized call - mark specific dimensions as compile-time constants x_opt = torch.randn([64, 128], device="cuda", dtype=torch.float16) y_opt = torch.randn([128, 56], device="cuda", dtype=torch.float16) torch._dynamo.mark_static(x_opt, [0, -1]) # specialize dims 0 and -1 (M and K) torch._dynamo.mark_static(y_opt, 1) # specialize dim 1 (N) result = matmul(x_opt, y_opt) # generates code with 64, 128, 56 as constants ``` -------------------------------- ### Suppress ALL warnings Source: https://github.com/pytorch/helion/blob/main/docs/api/exceptions.md Example of suppressing all Helion warnings by using `ignore_warnings` with `helion.exc.BaseWarning`. ```python # Suppress ALL warnings by using BaseWarning @helion.kernel(ignore_warnings=[helion.exc.BaseWarning]) def quiet_kernel(x): # This kernel will suppress all Helion warnings pass ``` -------------------------------- ### RankMismatch Example Source: https://github.com/pytorch/helion/blob/main/docs/api/exceptions.md Raised when tensor rank doesn't match indexing dimensions. ```python x = torch.randn(10, 20) # 2D tensor for i in hl.grid(10): y = x[i, j, k] # Raises RankMismatch - too many indices ``` -------------------------------- ### Basic Kernel Autotuning Source: https://github.com/pytorch/helion/blob/main/docs/deployment_autotuning.md Demonstrates how to define a Helion kernel and trigger autotuning on the first call. The autotuning process is cached for subsequent calls. ```python import torch, helion @helion.kernel() def my_kernel(x: torch.Tensor, y: torch.Tensor) -> torch.Tensor: ... example_inputs = ( torch.randn(1048576, device="cuda"), torch.randn(1048576, device="cuda"), ) # First call triggers autotuning, which is cached for future calls, and prints the best config found. my_kernel(*example_inputs) ``` -------------------------------- ### NotInsideKernel Example Source: https://github.com/pytorch/helion/blob/main/docs/api/exceptions.md Raised when helion.language.* functions are called outside of a kernel context. ```python import helion.language as hl # This will raise NotInsideKernel result = hl.zeros([10]) # Called outside @helion.kernel ``` -------------------------------- ### Overriding initial population strategy with environment variables Source: https://github.com/pytorch/helion/blob/main/docs/deployment_autotuning.md Example of setting the autotune effort to 'full' and the initial population strategy to 'from_best_available' using environment variables. ```bash # Use from_best_available with the full search budget export HELION_AUTOTUNE_EFFORT=full export HELION_AUTOTUNER_INITIAL_POPULATION=from_best_available ```