### Setup live-reloading preview

Source: https://github.com/rocm/flydsl/blob/main/docs/README.md

Install sphinx-autobuild and start a live-reloading server for documentation development.

```bash
pip install sphinx-autobuild
make livehtml
```

--------------------------------

### Install documentation dependencies

Source: https://github.com/rocm/flydsl/blob/main/docs/README.md

Install the required Python packages for building the documentation.

```bash
pip install -r requirements.txt
```

--------------------------------

### Run Python Example

Source: https://github.com/rocm/flydsl/blob/main/docs/testing_benchmarking_guide.md

Execute a Python-based example script. This can be used to test MLIR generation and AOT compilation.

```python
python tests/python/examples/aot_example.py
```

--------------------------------

### Build and Install FlyDSL from Source

Source: https://github.com/rocm/flydsl/blob/main/CONTRIBUTING.md

Commands to clone the repository, build the required LLVM/MLIR dependencies, compile the project, and install it in development mode.

```bash
# Step 1: Clone the repository
git clone https://github.com/ROCm/FlyDSL.git
cd FlyDSL
git remote add upstream https://github.com/ROCm/FlyDSL.git

# Step 2: Build LLVM/MLIR (one-time, ~30min with -j64)
bash scripts/build_llvm.sh -j64

# Step 3: Build FlyDSL
bash scripts/build.sh -j64

# Step 4: Install in development mode
pip install -e .

# Step 5: Verify
bash scripts/run_tests.sh
```

--------------------------------

### Install FlyDSL

Source: https://github.com/rocm/flydsl/blob/main/docs/installation.md

Install FlyDSL in editable mode or build a distributable wheel.

```bash
pip install -e .
```

```bash
python setup.py develop
```

```bash
export PYTHONPATH=$(pwd)/build-fly/python_packages:$(pwd):$PYTHONPATH
export LD_LIBRARY_PATH=$(pwd)/build-fly/python_packages/flydsl/_mlir/_mlir_libs:$LD_LIBRARY_PATH
```

```bash
python setup.py bdist_wheel
ls dist/
```

--------------------------------

### Using Pre-built Kernels

Source: https://context7.com/rocm/flydsl/llms.txt

Examples of initializing and executing optimized LayerNorm, RMSNorm, and Softmax kernels.

```python
import torch

# LayerNorm kernel
from kernels.layernorm_kernel import build_layernorm_module

# Build LayerNorm executor for specific dimensions
M, N = 32768, 8192
dtype_str = "bf16"  # Options: "f32", "f16", "bf16"

layernorm_executor = build_layernorm_module(M=M, N=N, dtype_str=dtype_str)

# Prepare tensors
Input = torch.randn(M, N, dtype=torch.bfloat16, device="cuda")
Gamma = torch.ones(N, dtype=torch.bfloat16, device="cuda")
Beta = torch.zeros(N, dtype=torch.bfloat16, device="cuda")
Output = torch.empty(M, N, dtype=torch.bfloat16, device="cuda")

# Execute LayerNorm
layernorm_executor(Input, Gamma, Beta, Output, M)
torch.cuda.synchronize()


# RMSNorm kernel
from kernels.rmsnorm_kernel import build_rmsnorm_module

rmsnorm_executor = build_rmsnorm_module(M=M, N=N, dtype_str=dtype_str)

# Prepare tensors (no Beta for RMSNorm)
RmsOutput = torch.empty(M, N, dtype=torch.bfloat16, device="cuda")

# Execute RMSNorm
rmsnorm_executor(Input, Gamma, RmsOutput, M)
torch.cuda.synchronize()


# Softmax kernel
from kernels.softmax_kernel import build_softmax_module

softmax_executor = build_softmax_module(M=M, N=N, dtype_str=dtype_str)
```

--------------------------------

### Verify Installation

Source: https://github.com/rocm/flydsl/blob/main/docs/installation.md

Execute the test suite to ensure the build and environment are configured correctly.

```bash
bash scripts/run_tests.sh
```

--------------------------------

### Build and Install FlyDSL

Source: https://github.com/rocm/flydsl/blob/main/README.md

Commands for building LLVM/MLIR, compiling FlyDSL, and installing in development mode.

```bash
bash scripts/build_llvm.sh -j64   # one-time: build LLVM/MLIR
bash scripts/build.sh -j64        # build FlyDSL
pip install -e .                   # install in dev mode
bash scripts/run_tests.sh          # verify
```

```bash
bash scripts/build.sh -j64
```

--------------------------------

### Define and Launch Kernels with Python API

Source: https://github.com/rocm/flydsl/blob/main/README.md

Example of using @flyc.kernel to define a GPU kernel and @flyc.jit to launch it.

```python
import flydsl.compiler as flyc
import flydsl.expr as fx
from flydsl.expr import arith, gpu

@flyc.kernel
def my_kernel(arg_a: fx.Tensor, arg_b: fx.Tensor, n: fx.Constexpr[int]):
    tid = gpu.thread_idx.x
    bid = gpu.block_idx.x
    # ... kernel body using layout ops ...

@flyc.jit
def launch(arg_a: fx.Tensor, arg_b: fx.Tensor, n: fx.Constexpr[int],
           stream: fx.Stream = fx.Stream(None)):
    my_kernel(arg_a, arg_b, n).launch(
        grid=(grid_x, 1, 1),
        block=(256, 1, 1),
        stream=stream,
    )
```

--------------------------------

### Environment Setup Variables

Source: https://github.com/rocm/flydsl/blob/main/docs/testing_benchmarking_guide.md

Required environment variables for test execution.

```bash
PYTHONPATH="${BUILD_DIR}/python_packages:${REPO_ROOT}:${PYTHONPATH}"
LD_LIBRARY_PATH="${MLIR_LIBS_DIR}:${LD_LIBRARY_PATH}"
```

--------------------------------

### Define and Launch a Vector Addition Kernel with FlyDSL

Source: https://github.com/rocm/flydsl/blob/main/docs/kernel_authoring_guide.md

This example demonstrates the basic pattern of defining a GPU kernel using `@flyc.kernel` and creating a JIT-compiled host-side launcher with `@flyc.jit`. It shows how to configure the grid and block dimensions for the kernel launch.

```python
import flydsl.compiler as flyc
import flydsl.expr as fx
from flydsl.expr import arith, gpu

@flyc.kernel
def vec_add_kernel(
    A: fx.Tensor,
    B: fx.Tensor,
    C: fx.Tensor,
    N: fx.Constexpr[int],
):
    tid = gpu.thread_idx.x
    bid = gpu.block_idx.x
    idx = bid * 256 + tid
    # ... kernel body using arith/vector/buffer ops ...

@flyc.jit
def vec_add(
    A: fx.Tensor,
    B: fx.Tensor,
    C: fx.Tensor,
    N: fx.Constexpr[int],
    stream: fx.Stream = fx.Stream(None),
):
    vec_add_kernel(A, B, C, N).launch(
        grid=(N // 256,),
        block=(256,),
        stream=stream,
    )

# Usage:
import torch
A = torch.randn(1024, device="cuda", dtype=torch.float32)
B = torch.randn(1024, device="cuda", dtype=torch.float32)
C = torch.empty(1024, device="cuda", dtype=torch.float32)

vec_add(A, B, C, 1024)
```

--------------------------------

### Launch Kernels with JIT

Source: https://github.com/rocm/flydsl/blob/main/docs/tutorials/basic_usage.md

Launch kernels using the @flyc.jit decorator, specifying grid and block dimensions. This example demonstrates integration with PyTorch tensors and streams.

```python
@flyc.jit
def launch(
    data: fx.Tensor,
    n: fx.Constexpr[int],
    stream: fx.Stream = fx.Stream(None),
):
    my_kernel(data, n).launch(
        grid=(grid_x, 1, 1),
        block=(256, 1, 1),
        stream=stream,
    )

# Usage with PyTorch
import torch
data = torch.randn(1024, dtype=torch.float32).cuda()
launch(data, 1024, stream=torch.cuda.Stream())
torch.cuda.synchronize()
```

--------------------------------

### Benchmark Output Format

Source: https://github.com/rocm/flydsl/blob/main/docs/testing_benchmarking_guide.md

Example of the tabular output format for benchmark results.

```default
op             shape                              dtype       TB/s    TFLOPS
-------------- ---------------------------------- ---------- ---------- ----------
gemm           16x40960x5120                      fp8         1.234     56.789
```

--------------------------------

### Install FlyDSL in Development Mode

Source: https://github.com/rocm/flydsl/blob/main/README.md

Installs FlyDSL in an editable mode, allowing immediate reflection of changes in the python/flydsl/ directory. Alternatively, set environment variables to use the build directly.

```bash
pip install -e .
```

```bash
python setup.py develop
```

```bash
export PYTHONPATH=$(pwd)/build-fly/python_packages:$(pwd):$PYTHONPATH
export LD_LIBRARY_PATH=$(pwd)/build-fly/python_packages/flydsl/_mlir/_mlir_libs:$LD_LIBRARY_PATH
```

--------------------------------

### PyIR Test Pattern

Source: https://github.com/rocm/flydsl/blob/main/docs/testing_benchmarking_guide.md

Example of a test pattern for PyIR without requiring a GPU.

```python
# tests/python/test_my_feature.py
import flydsl.expr as fx
from flydsl.expr.typing import T

def test_my_layout_op(ctx, insert_point):
    shape = fx.make_shape(4, 8)
    stride = fx.make_stride(8, 1)
    layout = fx.make_layout(shape, stride)
    result = fx.size(layout)
    ir_str = str(ctx.module)
    assert "fly.make_layout" in ir_str
```

--------------------------------

### Writing New Tests

Source: https://github.com/rocm/flydsl/blob/main/docs/testing_benchmarking_guide.md

Examples for writing new tests, including a PyIR test pattern that does not require GPU and a GPU kernel test pattern using a new API.

```APIDOC
## Writing New Tests

### PyIR Test Pattern (No GPU)
```python
# tests/python/test_my_feature.py
import flydsl.expr as fx
from flydsl.expr.typing import T

def test_my_layout_op(ctx, insert_point):
    shape = fx.make_shape(4, 8)
    stride = fx.make_stride(8, 1)
    layout = fx.make_layout(shape, stride)
    result = fx.size(layout)
    ir_str = str(ctx.module)
    assert "fly.make_layout" in ir_str
```

### GPU Kernel Test Pattern (New API)
```python
# Placeholder for GPU kernel test pattern example
```
```

--------------------------------

### Launch a FlyDSL Kernel on a Specific CUDA Stream

Source: https://github.com/rocm/flydsl/blob/main/docs/kernel_authoring_guide.md

This example shows how to launch a FlyDSL kernel on a specific CUDA or HIP stream for asynchronous execution. It demonstrates passing a `torch.cuda.Stream` object wrapped in `fx.Stream` to the launch configuration.

```python
@flyc.jit
def launch(data: fx.Tensor, stream: fx.Stream = fx.Stream(None)):
    my_kernel(data).launch(grid=(1,), block=(256,), stream=stream)

# Launch on specific stream:
stream = torch.cuda.Stream()
launch(data, stream=fx.Stream(stream))
```

--------------------------------

### Build and Test FlyDSL

Source: https://github.com/rocm/flydsl/blob/main/CLAUDE.md

Commands to build LLVM/MLIR, build FlyDSL itself, install it in development mode, and run tests. Use `FLYDSL_DUMP_IR=1` to see MLIR IR output.

```bash
bash scripts/build_llvm.sh   # Build LLVM/MLIR (one-time)
bash scripts/build.sh        # Build FlyDSL (uses Ninja)
pip install -e .              # Install in dev mode
```

```bash
PYTHONPATH=./ pytest tests/   # All tests
PYTHONPATH=./ python tests/kernels/test_pa.py --num_iters 50  # Specific test
FLYDSL_DUMP_IR=1 PYTHONPATH=./ python tests/kernels/test_pa.py # Dump MLIR IR at each pipeline stage (also disables disk cache)
```

--------------------------------

### Example Usage of Tiled Copy

Source: https://context7.com/rocm/flydsl/llms.txt

Demonstrates the usage of the tiled copy functionality. It initializes source and destination tensors on CUDA, calls the tiledCopy function, synchronizes the CUDA stream, and verifies the result using torch.allclose.

```python
# Usage
M, N = 8 * 3, 24 * 5
A = torch.arange(M * N, dtype=torch.float32).reshape(M, N).cuda()
B = torch.zeros(M, N, dtype=torch.float32).cuda()

tiledCopy(A, B, stream=torch.cuda.Stream())
torch.cuda.synchronize()
print("Result correct:", torch.allclose(A, B))

```

--------------------------------

### MFMA Operation Setup and Execution

Source: https://github.com/rocm/flydsl/blob/main/docs/cute_layout_algebra_guide.md

Set up and execute Matrix Fused Multiply-Add (MFMA) operations using FlyDSL's direct access to AMD GPU intrinsics.

```python
mma_atom = fx.make_mma_atom(fx.rocdl.MFMA(16, 16, 4, fx.Float32))
tiled_mma = fx.make_tiled_mma(mma_atom, fx.make_layout((2, 2, 1), (1, 2, 0)))

# Block-level tensor views bA, bB, bC (e.g. after zipped_divide + slice by block)
thr_mma = tiled_mma.thr_slice(tid)
frag_A = thr_mma.make_fragment_A(bA)
frag_B = thr_mma.make_fragment_B(bB)
frag_C = thr_mma.make_fragment_C(bC)
fx.gemm(mma_atom, frag_C, frag_A, frag_B, frag_C)
```

--------------------------------

### Manual XOR-based Swizzling Implementation

Source: https://github.com/rocm/flydsl/blob/main/docs/cute_layout_algebra_guide.md

Implement XOR-based swizzling manually using arithmetic operations for bank conflict avoidance in shared/local memory. This example uses 16-byte granularity.

```python
# XOR-based swizzle at 16-byte granularity (manual implementation)
col_swizzled = col_bytes ^ ((row % k_blocks16) << 4)
```

--------------------------------

### Build HTML documentation

Source: https://github.com/rocm/flydsl/blob/main/docs/README.md

Generate the static HTML files from the Sphinx source.

```bash
make html
```

--------------------------------

### Configure and launch a GPU kernel

Source: https://github.com/rocm/flydsl/blob/main/docs/architecture_guide.md

Call a kernel function to obtain a KernelLauncher, then use .launch() to specify grid, block, shared memory, and stream parameters.

```python
launcher = my_kernel(a, b, 1024)
launcher.launch(
    grid=(num_blocks, 1, 1),
    block=(256, 1, 1),
    smem=shared_mem_bytes,
    stream=stream_value,
)
```

--------------------------------

### Performance results table example

Source: https://github.com/rocm/flydsl/blob/main/CONTRIBUTING.md

Format for reporting performance improvements in pull request descriptions.

```markdown
## Performance Results (MI300X)

| Config | Baseline | Optimized | Improvement |
|--------|----------|-----------|-------------|
| BF16, M=1024, N=4096 | 180 μs | 150 μs | 16.7% |
| BF16, M=2048, N=8192 | 720 μs | 600 μs | 16.7% |
```

--------------------------------

### Define and Launch a Vector Add Kernel

Source: https://github.com/rocm/flydsl/blob/main/docs/quickstart.md

Uses @flyc.kernel to define the GPU logic and @flyc.jit to handle host-side kernel launching with layout algebra for partitioning.

```python
import torch
import flydsl.compiler as flyc
import flydsl.expr as fx

@flyc.kernel
def vectorAddKernel(
    A: fx.Tensor, B: fx.Tensor, C: fx.Tensor,
    block_dim: fx.Constexpr[int],
):
    bid = fx.block_idx.x
    tid = fx.thread_idx.x

    # Partition tensors by block using layout algebra
    tA = fx.logical_divide(A, fx.make_layout(block_dim, 1))
    tB = fx.logical_divide(B, fx.make_layout(block_dim, 1))
    tC = fx.logical_divide(C, fx.make_layout(block_dim, 1))

    tA = fx.slice(tA, (None, bid))
    tB = fx.slice(tB, (None, bid))
    tC = fx.slice(tC, (None, bid))

    # Second divide gives a 2-level layout so per-thread slice (None, tid) matches
    # shape/stride rank (see examples/01-vectorAdd.py and tests/kernels/test_vec_add.py).
    tA = fx.logical_divide(tA, fx.make_layout(1, 1))
    tB = fx.logical_divide(tB, fx.make_layout(1, 1))
    tC = fx.logical_divide(tC, fx.make_layout(1, 1))

    # Allocate register fragments, load, compute, store
    RABTy = fx.MemRefType.get(fx.T.f32(), fx.LayoutType.get(1, 1),
                              fx.AddressSpace.Register)
    copyAtom = fx.make_copy_atom(fx.UniversalCopy32b(), fx.Float32)
    rA = fx.memref_alloca(RABTy, fx.make_layout(1, 1))
    rB = fx.memref_alloca(RABTy, fx.make_layout(1, 1))
    rC = fx.memref_alloca(RABTy, fx.make_layout(1, 1))

    fx.copy_atom_call(copyAtom, fx.slice(tA, (None, tid)), rA)
    fx.copy_atom_call(copyAtom, fx.slice(tB, (None, tid)), rB)

    vC = fx.arith.addf(fx.memref_load_vec(rA), fx.memref_load_vec(rB))
    fx.memref_store_vec(vC, rC)
    fx.copy_atom_call(copyAtom, rC, fx.slice(tC, (None, tid)))

@flyc.jit
def vectorAdd(
    A: fx.Tensor, B: fx.Tensor, C,
    n: fx.Int32,  # dynamic int32
    const_n: fx.Constexpr[int],  # static int32, affects JIT cache-key
    stream: fx.Stream = fx.Stream(None),
):
    block_dim = 64
    grid_x = (n + block_dim - 1) // block_dim
    vectorAddKernel(A, B, C, block_dim).launch(
        grid=(grid_x, 1, 1), block=[block_dim, 1, 1], stream=stream,
    )

# Usage
n = 128
A = torch.randint(0, 10, (n,), dtype=torch.float32).cuda()
B = torch.randint(0, 10, (n,), dtype=torch.float32).cuda()
C = torch.zeros(n, dtype=torch.float32).cuda()
vectorAdd(A, B, C, n, n + 1, stream=torch.cuda.Stream())
torch.cuda.synchronize()
print("Result correct:", torch.allclose(C, A + B))
```

--------------------------------

### Run Benchmarks

Source: https://github.com/rocm/flydsl/blob/main/docs/testing_benchmarking_guide.md

Execute the benchmark harness using the provided bash script. This is useful for performance measurement.

```bash
bash scripts/run_benchmark.sh
```

--------------------------------

### Get Thread Slice and Partition Tensors

Source: https://github.com/rocm/flydsl/blob/main/docs/cute_layout_algebra_guide.md

Obtain a thread slice and partition source and destination tensors for copy operations.

```python
thr_copy = tiled_copy.get_slice(tid)
src_partition = thr_copy.partition_S(src_tensor)
dst_partition = thr_copy.partition_D(dst_tensor)

# Execute copy
fx.copy(copy_atom, src_partition, dst_partition)
```

--------------------------------

### Run pytest with specific tier markers

Source: https://github.com/rocm/flydsl/blob/main/tests/README.md

Execute tests filtered by specific pytest markers, such as 'l0_backend_agnostic' or 'l1a_compile_no_target_dialect'. This is an example of forward-looking test annotation.

```bash
# python3 -m pytest tests/ -m "l0_backend_agnostic or l1a_compile_no_target_dialect" -v
# python3 -m pytest tests/ -m "l2_device" -v
```

--------------------------------

### Build and Run MLIR fly-opt

Source: https://github.com/rocm/flydsl/blob/main/docs/testing_benchmarking_guide.md

Build the fly-opt tool and run a single MLIR test case. This is useful for debugging specific MLIR transformations.

```bash
# Build fly-opt first if needed
cmake --build build-fly --target fly-opt -j$(nproc)

# Run a single test
build-fly/bin/fly-opt --fly-canonicalize tests/mlir/LayoutAlgebra/construction.mlir
```

--------------------------------

### Run all pytest tests

Source: https://github.com/rocm/flydsl/blob/main/tests/README.md

Execute all tests within the specified directories (kernels, unit, examples) with verbose output. This is the default behavior similar to scripts/run_tests.sh.

```bash
python3 -m pytest tests/kernels/ tests/unit/ tests/python/examples/ -v
```

--------------------------------

### Configure GEMM Test via CLI Arguments

Source: https://github.com/rocm/flydsl/blob/main/docs/testing_benchmarking_guide.md

Demonstrates how to run the GEMM test script with various command-line arguments to configure data types, dimensions, tiling, iteration counts, and testing modes like HIPGraph.

```bash
python tests/kernels/test_preshuffle_gemm.py \
    --in_dtype fp8 \
    -M 16 -N 5120 -K 8192 \
    --tile_m 16 --tile_n 128 --tile_k 256 \
    --lds_stage 2 \
    --num_iters 20 \
    --num_warmup 3 \
    --no_aiter_bench \
    --test_graph        # or -tg for HIPGraph mode
    --wfp4              # FP4 weight path (gfx950 only)

```

--------------------------------

### Benchmark Configuration

Source: https://github.com/rocm/flydsl/blob/main/docs/testing_benchmarking_guide.md

Default shape configurations for Softmax, LayerNorm, and GEMM benchmarks.

```bash
# Softmax/LayerNorm: "M,N,dtype"
SOFTMAX_SHAPES='32768,8192,bf16'
LAYERNORM_SHAPES='32768,8192,bf16'

# Preshuffle GEMM: "dtype,M,N,K,tile_m,tile_n,tile_k"
GEMM_SHAPES='
fp8,16,40960,5120,16,128,256
fp8,16,77824,5120,16,128,256
fp8,5120,5120,8320,64,256,128
fp8,9728,8192,8320,64,256,128
int8,9728,8192,8320,64,256,128
int4,9728,8192,8320,64,256,128
bf16,5120,5120,8320,64,256,128
'

# FP4 GEMM (gfx950 only): "M,N,K,tile_m,tile_n,tile_k"
GEMM_FP4_SHAPES='8192,8192,8192,64,128,256'
```

--------------------------------

### Define FlyDSL Kernel and Launch Wrapper

Source: https://github.com/rocm/flydsl/blob/main/docs/cute_layout_algebra_guide.md

Use `@flyc.kernel` to define GPU device functions and `@flyc.jit` to create host-side launch wrappers. Ensure correct tensor and constant types are used for arguments.

```python
import flydsl.compiler as flyc
import flydsl.expr as fx

@flyc.kernel
def my_kernel(
    A: fx.Tensor,
    B: fx.Tensor,
    C: fx.Tensor,
    block_dim: fx.Constexpr[int],
):
    tid = fx.thread_idx.x
    bid = fx.block_idx.x
    # Kernel body — use layout algebra here
    ...

@flyc.jit
def launch(
    A: fx.Tensor, B: fx.Tensor, C,
    n: fx.Int32,
    stream: fx.Stream = fx.Stream(None),
):
    my_kernel(A, B, C, block_dim).launch(
        grid=(grid_x, 1, 1), block=(block_dim, 1, 1), stream=stream,
    )
```

--------------------------------

### Troubleshooting Commands

Source: https://github.com/rocm/flydsl/blob/main/docs/installation.md

Commands to resolve common build and runtime issues.

```default
cmake --build build-fly --target fly-opt -j$(nproc)
```

```default
export PYTHONPATH=$(pwd)/build-fly/python_packages:$(pwd):$PYTHONPATH
```

```default
export LD_LIBRARY_PATH=$MLIR_PATH/lib:$LD_LIBRARY_PATH
```

--------------------------------

### Benchmark Execution Commands

Source: https://github.com/rocm/flydsl/blob/main/docs/testing_benchmarking_guide.md

Commands for running benchmarks with different filters and options.

```bash
bash scripts/run_benchmark.sh                    # default: GEMM only
bash scripts/run_benchmark.sh softmax             # only softmax
bash scripts/run_benchmark.sh gemm moe            # GEMM and MoE
bash scripts/run_benchmark.sh --only softmax,layernorm
bash scripts/run_benchmark.sh --list              # list available ops
```

--------------------------------

### Define and Launch Kernels with @flyc.kernel and @flyc.jit

Source: https://github.com/rocm/flydsl/blob/main/docs/api/compiler.md

Use @flyc.kernel to define GPU kernels and @flyc.jit to manage host-side kernel launching and JIT compilation.

```python
import flydsl.compiler as flyc
import flydsl.expr as fx

@flyc.kernel
def my_kernel(A: fx.Tensor, B: fx.Tensor, n: fx.Constexpr[int]):
    tid = fx.thread_idx.x
    bid = fx.block_idx.x
    # ... kernel body using layout ops ...

@flyc.jit
def launch(A: fx.Tensor, B: fx.Tensor, n: fx.Constexpr[int],
           stream: fx.Stream = fx.Stream(None)):
    my_kernel(A, B, n).launch(
        grid=(grid_x, 1, 1),
        block=(256, 1, 1),
        stream=stream,
    )
```

--------------------------------

### ROCDL Intrinsics Example

Source: https://context7.com/rocm/flydsl/llms.txt

Demonstrates the usage of ROCDL intrinsics for MFMA operations, buffer descriptors, scheduling barriers, hardware math functions, and warp shuffles within a FlyDSL kernel.

```python
import flydsl.compiler as flyc
import flydsl.expr as fx
from flydsl.expr import rocdl, arith
from flydsl.expr.typing import T

@flyc.kernel
def rocdl_example(A: fx.Tensor, B: fx.Tensor, C: fx.Tensor):
    tid = fx.thread_idx.x

    # Create buffer tensor with AMD buffer resource descriptor
    A_buf = rocdl.make_buffer_tensor(A)

    # MFMA atom constructor for 16x16 matrix multiply
    # MFMA(m, n, k, accumulator_type)
    mfma_type = rocdl.MFMA(16, 16, 4, fx.Float32)

    # Buffer copy atom types
    copy_128b = rocdl.BufferCopy128b()  # 128-bit buffer copy
    copy_64b = rocdl.BufferCopy64b()    # 64-bit buffer copy
    copy_32b = rocdl.BufferCopy32b()    # 32-bit buffer copy

    # Direct MFMA intrinsics (low-level)
    # result = rocdl.mfma_f32_16x16x16f16(result_type, [a, b, acc])
    # result = rocdl.mfma_f32_16x16x32_fp8_fp8(result_type, [a, b, acc])
    # result = rocdl.mfma_i32_16x16x32_i8(result_type, [a, b, acc])

    # Instruction scheduling barriers for performance tuning
    # Wait for N MFMA instructions to complete
    rocdl.sched_mfma(1)

    # Wait for N VMEM reads to complete
    rocdl.sched_vmem(2)

    # Wait for N DS (LDS) reads to complete
    rocdl.sched_dsrd(1)

    # Wait for N DS (LDS) writes to complete
    rocdl.sched_dswr(1)

    # Hardware math intrinsics (single VALU cycle, lower precision)
    x = fx.Float32(2.0)

    # Base-2 exponential (v_exp_f32)
    exp_result = rocdl.exp2(T.f32, x)

    # Reciprocal (v_rcp_f32)
    rcp_result = rocdl.rcp(T.f32, x)

    # Warp shuffle for communication between lanes
    src_val = fx.Float32(1.0)
    lane_idx = fx.Int32(0)
    shuffled = rocdl.ds_bpermute(lane_idx, src_val)
```

--------------------------------

### Allocate Shared Memory Array with SmemAllocator

Source: https://github.com/rocm/flydsl/blob/main/docs/cute_layout_algebra_guide.md

Use `SmemAllocator` to manage shared memory allocation. Specify the context, architecture, and array type. `finalize()` must be called before getting the base pointer.

```python
from flydsl.utils.smem_allocator import SmemAllocator

allocator = SmemAllocator(ctx, arch="gfx942")
lds_gen = allocator.allocate_array(T.f16(), num_elems=128*64)
allocator.finalize()

base = allocator.get_base()
lds_ptr = lds_gen(base)
```

--------------------------------

### Launch Kernels

Source: https://github.com/rocm/flydsl/blob/main/docs/kernel_authoring_guide.md

Configures kernel execution with grid, block, shared memory, and stream parameters.

```python
@flyc.jit
def launch(data: fx.Tensor, stream: fx.Stream = fx.Stream(None)):
    my_kernel(data).launch(
        grid=(num_blocks_x, num_blocks_y, num_blocks_z),
        block=(threads_x, threads_y, threads_z),
        smem=shared_mem_bytes,     # dynamic shared memory
        stream=stream,             # CUDA/HIP stream
    )
```

--------------------------------

### Create Generic Integer Tuple and Query Layout Properties

Source: https://context7.com/rocm/flydsl/llms.txt

Demonstrates creating a generic integer tuple and querying properties of a layout such as size, codomain size, shape, stride, specific dimension values, and rank.

```python
# Generic integer tuple
it = fx.make_int_tuple((4, 8, 2))
```

```python
# Query operations
s = fx.size(layout)           # Total elements (128 for 8x16)
cs = fx.cosize(layout)        # Codomain size (max index + 1)
shape_out = fx.get_shape(layout)
stride_out = fx.get_stride(layout)
v = fx.get(shape, 0)          # First dimension (8)
r = fx.rank(shape)            # Number of modes (2)
```

--------------------------------

### CUDA Graph Capture with FlyDSL

Source: https://context7.com/rocm/flydsl/llms.txt

Illustrates capturing FlyDSL kernel launches into a CUDA graph for efficient repeated execution. Includes warmup, stream management, and graph creation.

```python
# Setup tensors
n = 128
A = torch.randn(n, dtype=torch.float32, device="cuda")
B = torch.randn(n, dtype=torch.float32, device="cuda")
C = torch.zeros(n, dtype=torch.float32, device="cuda")
tA = flyc.from_dlpack(A).mark_layout_dynamic(leading_dim=0, divisibility=4)

# Warmup (triggers JIT compilation - required before graph capture)
my_launch(tA, B, C, n, n + 1, stream=torch.cuda.Stream())
torch.cuda.synchronize()

# Reset output
C.zero_()

# Create CUDA graph
graph = torch.cuda.CUDAGraph()
capture_stream = torch.cuda.Stream()
capture_stream.wait_stream(torch.cuda.current_stream())

# Capture kernel launches into graph
with torch.cuda.stream(capture_stream):
    with torch.cuda.graph(graph, stream=capture_stream):
        my_launch(tA, B, C, n, n + 1, stream=capture_stream)
```

--------------------------------

### Define a Kernel with Compile-Time Constant Parameters in FlyDSL

Source: https://github.com/rocm/flydsl/blob/main/docs/kernel_authoring_guide.md

This example illustrates defining a FlyDSL kernel that uses `fx.Constexpr` for parameters like size and data type. These values are embedded directly into the generated IR, and different constant values result in separate compiled kernels.

```python
@flyc.kernel
def my_kernel(data: fx.Tensor, N: fx.Constexpr[int], dtype: fx.Constexpr[str]):
    for i in range_constexpr(N // 64):  # unrolled at compile time
        ...
```

--------------------------------

### Run Ruff linting and formatting

Source: https://github.com/rocm/flydsl/blob/main/CONTRIBUTING.md

Execute these commands before submitting code to ensure compliance with project linting and formatting rules.

```bash
ruff check python/ kernels/ tests/
ruff format --check python/ kernels/ tests/
```

--------------------------------

### Create Tiled Copy and MMA

Source: https://github.com/rocm/flydsl/blob/main/docs/layout_system_guide.md

Initialize tiled copy and MMA structures based on atoms and layouts.

```python
# Make tiled copy matched to a TiledMma's A/B/C partitioning
tiled_copy_a = fx.make_tiled_copy_A(copy_atom, tiled_mma)
tiled_copy_b = fx.make_tiled_copy_B(copy_atom, tiled_mma)
tiled_copy_c = fx.make_tiled_copy_C(copy_atom, tiled_mma)

# Make tiled MMA from MMA atom + atom layout + optional permutation
tiled_mma = fx.make_tiled_mma(mma_atom, atom_layout)
tiled_mma = fx.make_tiled_mma(mma_atom, atom_layout, permutation)
```

--------------------------------

### Launch Tiled Copy Kernel with JIT Compilation

Source: https://context7.com/rocm/flydsl/llms.txt

JIT compiles and launches the previously defined copy kernel. It takes source and destination tensors and an optional CUDA stream. The launch configuration specifies grid and block dimensions.

```python
@flyc.jit
def tiledCopy(A: fx.Tensor, B: fx.Tensor, stream: fx.Stream = fx.Stream(None)):
    copy_kernel(A, B).launch(grid=(15, 1, 1), block=(4, 1, 1), stream=stream)

```

--------------------------------

### Run benchmark suite

Source: https://github.com/rocm/flydsl/blob/main/CONTRIBUTING.md

Execute the project's benchmark script to validate performance for kernel-related changes.

```bash
# Run the benchmark suite
bash scripts/run_benchmark.sh
```

--------------------------------

### Execute Buffer Operations for AMD CDNA

Source: https://github.com/rocm/flydsl/blob/main/docs/api/dsl.md

Shows how to create buffer resources and perform load/store operations with hardware bounds checking.

```python
from flydsl.expr import buffer_ops

rsrc = buffer_ops.create_buffer_resource(tensor, max_size=True)
data = buffer_ops.buffer_load(rsrc, offset, vec_width=4, dtype=T.i32)
buffer_ops.buffer_store(data, rsrc, offset, mask=is_valid)
```

--------------------------------

### Build LLVM/MLIR for FlyDSL

Source: https://github.com/rocm/flydsl/blob/main/README.md

Builds LLVM and MLIR from source, which is a prerequisite for FlyDSL. This process can take a significant amount of time. Ensure you have sufficient system resources.

```bash
bash scripts/build_llvm.sh -j64
```

--------------------------------

### Build Softmax Module

Source: https://github.com/rocm/flydsl/blob/main/docs/prebuilt_kernels_guide.md

Initializes the softmax module with specific dimensions and data type.

```python
from kernels.softmax_kernel import build_softmax_module

executor = build_softmax_module(M=32768, N=8192, dtype_str="bf16")
```

--------------------------------

### Construct Fly dialect Layouts in Python

Source: https://github.com/rocm/flydsl/blob/main/docs/layout_system_guide.md

Demonstrates creating shapes, strides, and layouts using the flydsl.expr module. Supports static constants and Python tuples for defining dimensions.

```python
import flydsl.expr as fx
from flydsl.expr import arith
from flydsl.expr.typing import T

# Shapes and strides (static constants auto-materialized)
shape = fx.make_shape(8, 16)              # !fly.int_tuple<(8, 16)>
stride = fx.make_stride(1, 8)             # !fly.int_tuple<(1, 8)>
layout = fx.make_layout(shape, stride)    # !fly.layout<(8, 16):(1, 8)>

# Shorthand — pass Python tuples directly
layout = fx.make_layout((8, 16), (1, 8))

# Coordinates
coord = fx.make_coord(i, j)

# Generic integer tuple
it = fx.make_int_tuple((4, 8, 2))

# Nested shapes
shape_nested = fx.make_shape(9, (4, 8))   # (9, (4, 8))

# Ordered layout — specify stride order (e.g., column-major vs row-major)
col_major = fx.make_ordered_layout((M, N), order=(0, 1))  # stride order: M-first
row_major = fx.make_ordered_layout((M, N), order=(1, 0))  # stride order: N-first

# Identity layout / tensor
identity = fx.make_identity_layout((M, N))
id_tensor = fx.make_identity_tensor((M, N))
```

--------------------------------

### Kernel Launch Configuration

Source: https://github.com/rocm/flydsl/blob/main/docs/kernel_authoring_guide.md

Configures the grid and block dimensions for GPU kernel execution.

```APIDOC
## .launch(grid, block, stream)

### Description
Configures and emits a GPU launch for a previously defined kernel.

### Parameters
- **grid** (tuple) - The grid dimensions (workgroup count).
- **block** (tuple) - The block dimensions (threads per workgroup).
- **stream** (fx.Stream) - Optional CUDA/HIP stream for asynchronous execution.
```

--------------------------------

### Run Preshuffle GEMM Kernel Test with Options

Source: https://github.com/rocm/flydsl/blob/main/docs/testing_benchmarking_guide.md

Execute the preshuffle GEMM kernel test with specified input data type and dimensions. This is for testing specific GEMM configurations.

```python
python tests/kernels/test_preshuffle_gemm.py --in_dtype fp8 -M 16 -N 5120 -K 8192
```

--------------------------------

### Create Ordered and Identity Layouts

Source: https://context7.com/rocm/flydsl/llms.txt

Illustrates creating ordered layouts (column-major, row-major) and identity layouts. Identity layouts represent a default memory layout where strides are prefix products of dimensions.

```python
# Ordered layouts - specify stride order (column-major vs row-major)
M, N = 64, 128
col_major = fx.make_ordered_layout((M, N), order=(0, 1))  # M-first strides
row_major = fx.make_ordered_layout((M, N), order=(1, 0))  # N-first strides
```

```python
# Identity layout (strides = prefix products)
identity = fx.make_identity_layout((M, N))
```

--------------------------------

### Compile and Launch Preshuffle GEMM

Source: https://context7.com/rocm/flydsl/llms.txt

Configures and launches high-performance matrix multiplication kernels supporting multiple data types and memory layouts.

```python
import torch
from kernels.preshuffle_gemm import compile_preshuffle_gemm_a8

# Compile preshuffle GEMM for specific configuration
# C[M,N] = A[M,K] @ B[N,K]^T
M, N, K = 16, 5120, 8192

launch_fn = compile_preshuffle_gemm_a8(
    M=M,                      # M dimension (can be 0 for dynamic)
    N=N,                      # N dimension (can be 0 for dynamic)
    K=K,                      # K dimension
    tile_m=16,                # Block tile M size
    tile_n=128,               # Block tile N size
    tile_k=256,               # Block tile K size
    in_dtype="fp8",           # Input dtype: "fp8", "int8", "int4", "fp16", "bf16", "fp4"
    lds_stage=2,              # LDS staging: 2=ping-pong (tuned), 1=single buffer
    use_cshuffle_epilog=False # CK-style LDS CShuffle epilogue
)

# Prepare tensors (FP8 example)
# Note: B must be in preshuffle layout: (N/16, K/64, 4, 16, kpack_bytes)
A = torch.randn(M, K, dtype=torch.float8_e4m3fn, device="cuda")
B_preshuffled = torch.randn(N // 16, K // 64, 4, 16, 4, dtype=torch.float8_e4m3fn, device="cuda")
C = torch.zeros(M, N, dtype=torch.float32, device="cuda")

# Scale tensors for FP8
scale_a = torch.ones(1, dtype=torch.float32, device="cuda")
scale_b = torch.ones(1, dtype=torch.float32, device="cuda")

# Launch GEMM
stream = torch.cuda.Stream()
launch_fn(C, A, B_preshuffled, scale_a, scale_b, M, N, stream=stream)
torch.cuda.synchronize()


# Example with BF16
launch_fn_bf16 = compile_preshuffle_gemm_a8(
    M=0, N=0, K=K,            # Dynamic M and N
    tile_m=64, tile_n=128, tile_k=64,
    in_dtype="bf16",
    lds_stage=2,
)

A_bf16 = torch.randn(M, K, dtype=torch.bfloat16, device="cuda")
B_bf16 = torch.randn(N, K, dtype=torch.bfloat16, device="cuda")  # Will be preshuffled internally
C_bf16 = torch.zeros(M, N, dtype=torch.float32, device="cuda")

# Scale tensors (usually 1.0 for non-quantized)
scale_a_bf16 = torch.ones(1, dtype=torch.float32, device="cuda")
scale_b_bf16 = torch.ones(1, dtype=torch.float32, device="cuda")
```

--------------------------------

### Compile Preshuffle GEMM

Source: https://github.com/rocm/flydsl/blob/main/docs/prebuilt_kernels_guide.md

Compiles a GEMM kernel with specified tile sizes and data types, returning a JIT-decorated launch function.

```python
from kernels.preshuffle_gemm import compile_preshuffle_gemm_a8

launch_fn = compile_preshuffle_gemm_a8(
    M=16, N=5120, K=8192,
    tile_m=16, tile_n=128, tile_k=256,
    in_dtype="fp8",
    lds_stage=2,
    use_cshuffle_epilog=False,
)
```

--------------------------------

### Define and Launch a GEMM Kernel

Source: https://github.com/rocm/flydsl/blob/main/docs/prebuilt_kernels_guide.md

The standard pattern for defining a kernel using the @flyc.kernel decorator and launching it via the @flyc.jit wrapper.

```python
import flydsl.compiler as flyc
import flydsl.expr as fx
from flydsl.expr import arith, gpu, buffer_ops, rocdl

@flyc.kernel
def gemm_kernel(arg_c: fx.Tensor, arg_a: fx.Tensor, ...):
    tid = gpu.thread_idx.x
    # ... uses fx.*, arith.*, buffer_ops.*, rocdl.* ...

@flyc.jit
def launch_fn(arg_c: fx.Tensor, ..., stream: fx.Stream = fx.Stream(None)):
    gemm_kernel(arg_c, ...).launch(grid=..., block=..., stream=stream)
```

--------------------------------

### Execute Softmax Kernel

Source: https://context7.com/rocm/flydsl/llms.txt

Demonstrates executing a softmax operation and verifying the output against a PyTorch reference implementation.

```python
# Prepare tensors
SoftmaxInput = torch.randn(M, N, dtype=torch.bfloat16, device="cuda")
SoftmaxOutput = torch.empty(M, N, dtype=torch.bfloat16, device="cuda")

# Execute Softmax
softmax_executor(SoftmaxInput, SoftmaxOutput, M)
torch.cuda.synchronize()

# Verify against PyTorch reference
expected_softmax = torch.nn.functional.softmax(SoftmaxInput.float(), dim=-1).bfloat16()
print("Softmax correct:", torch.allclose(SoftmaxOutput, expected_softmax, atol=1e-2))
```

--------------------------------

### Run Benchmark Script

Source: https://github.com/rocm/flydsl/blob/main/docs/testing_benchmarking_guide.md

The `run_benchmark.sh` script is a specialized harness for performance characterization. It supports default configurations for various operations like Softmax, LayerNorm, and GEMM, and allows selective execution of benchmarks.

```APIDOC
## scripts/run_benchmark.sh

### Description
Specialized benchmarking harness for performance characterization.

### Default Configurations
```bash
# Softmax/LayerNorm: "M,N,dtype"
SOFTMAX_SHAPES='32768,8192,bf16'
LAYERNORM_SHAPES='32768,8192,bf16'

# Preshuffle GEMM: "dtype,M,N,K,tile_m,tile_n,tile_k"
GEMM_SHAPES=' 
fp8,16,40960,5120,16,128,256
fp8,16,77824,5120,16,128,256
fp8,5120,5120,8320,64,256,128
fp8,9728,8192,8320,64,256,128
int8,9728,8192,8320,64,256,128
int4,9728,8192,8320,64,256,128
bf16,5120,5120,8320,64,256,128
'

# FP4 GEMM (gfx950 only): "M,N,K,tile_m,tile_n,tile_k"
GEMM_FP4_SHAPES='8192,8192,8192,64,128,256'
```

### Selective Execution
```bash
bash scripts/run_benchmark.sh                    # default: GEMM only
bash scripts/run_benchmark.sh softmax             # only softmax
bash scripts/run_benchmark.sh gemm moe            # GEMM and MoE
bash scripts/run_benchmark.sh --only softmax,layernorm
bash scripts/run_benchmark.sh --list              # list available ops
```

### Output Format
Tabular with TB/s and TFLOPS columns:
```default
op             shape                              dtype       TB/s    TFLOPS
-------------- ---------------------------------- ---------- ---------- ----------
gemm           16x40960x5120                      fp8         1.234     56.789
```

### Logs
Written to `${BENCH_LOG_DIR:-/tmp/flydsl_bench}/`
```

--------------------------------

### Troubleshoot MLIR Library Path

Source: https://github.com/rocm/flydsl/blob/main/README.md

Command to add the MLIR build library directory to the loader path when encountering .so load errors.

```bash
export LD_LIBRARY_PATH=$(pwd)/build-fly/python_packages/flydsl/_mlir/_mlir_libs:$LD_LIBRARY_PATH
```

--------------------------------

### Define and Launch a GPU Kernel with FlyDSL

Source: https://github.com/rocm/flydsl/blob/main/docs/testing_benchmarking_guide.md

Defines a GPU kernel using the `@flyc.kernel` decorator and a launch function with `@flyc.jit`. The launch function configures grid and block dimensions for kernel execution. This is typically used for custom GPU computations.

```python
import torch
import flydsl.compiler as flyc
import flydsl.expr as fx
from flydsl.expr import arith, gpu
from tests.test_common import checkAllclose

@flyc.kernel
def my_kernel(A: fx.Tensor, B: fx.Tensor, N: fx.Constexpr[int]):
    tid = gpu.thread_idx.x
    bid = gpu.block_idx.x
    # ... kernel body ...

@flyc.jit
def launch(A: fx.Tensor, B: fx.Tensor, N: fx.Constexpr[int],
           stream: fx.Stream = fx.Stream(None)):
    my_kernel(A, B, N).launch(grid=(N // 256,), block=(256,), stream=stream)

def test_my_kernel():
    N = 1024
    A = torch.randn(N, device="cuda", dtype=torch.float32)
    B = torch.empty(N, device="cuda", dtype=torch.float32)

    launch(A, B, N)

    # Reference
    ref = A  # or some computation

    # Validate
    err = checkAllclose(B, ref, rtol=1e-2, atol=1e-2)
    assert err == 0, f"Mismatch: {err * 100:.2f}%"

```

--------------------------------

### Dump IR to Files

Source: https://github.com/rocm/flydsl/blob/main/docs/kernel_authoring_guide.md

Environment variables to enable IR dumping to a specified directory.

```bash
FLYDSL_DUMP_IR=1 FLYDSL_DUMP_DIR=./my_dumps python my_script.py
```

--------------------------------

### Configure Dynamic Grid and Block Dimensions

Source: https://github.com/rocm/flydsl/blob/main/docs/kernel_authoring_guide.md

Uses dynamic values for grid and block dimensions during kernel launch.

```python
@flyc.jit
def launch(data: fx.Tensor, M: fx.Int32, stream: fx.Stream = fx.Stream(None)):
    grid_x = M // 256
    my_kernel(data, M).launch(
        grid=(grid_x, 1, 1),
        block=(256, 1, 1),
        stream=stream,
    )
```

--------------------------------

### GEMM Kernels

Source: https://github.com/rocm/flydsl/blob/main/docs/api/kernels.md

Overview of available pre-built GEMM kernels in FlyDSL.

```APIDOC
## GEMM Kernel List

- `kernels.preshuffle_gemm`: MFMA-based GEMM with LDS pipeline and pre-shuffled weights (FP8, INT8, BF16).
- `kernels.preshuffle_gemm_flyc`: Preshuffle GEMM using the `@flyc.kernel` API.
- `kernels.mixed_preshuffle_gemm`: Mixed-precision GEMM with pre-shuffled layouts.
- `kernels.blockscale_preshuffle_gemm`: Block-scale (MXFP4) preshuffle GEMM.
```

--------------------------------

### Add License Header to C++/TableGen/MLIR Files

Source: https://github.com/rocm/flydsl/blob/main/CONTRIBUTING.md

Include this SPDX license and copyright header at the top of all new C++, TableGen, and MLIR source files. Replace '20xx' with the current year.

```cpp
// SPDX-License-Identifier: Apache-2.0
// Copyright (c) 20xx FlyDSL Project Contributors
```