### Install DeepGEMM

Source: https://github.com/deepseek-ai/deepgemm/blob/main/README.md

To install the DeepGEMM library, execute the install script. This makes the `deep_gemm` Python package available for use in your projects.

```bash
cat install.sh
./install.sh
```

--------------------------------

### Project Setup and Flags

Source: https://github.com/deepseek-ai/deepgemm/blob/main/CMakeLists.txt

Initializes the CMake project, sets C++ and CUDA standards, and defines optimization and PIC flags. Use this for basic project configuration.

```cmake
cmake_minimum_required(VERSION 3.10)
project(deep_gemm LANGUAGES CXX CUDA)
set(CMAKE_VERBOSE_MAKEFILE ON)

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -O3 -fPIC -Wno-psabi -Wno-deprecated-declarations")
set(CUDA_SEPARABLE_COMPILATION ON)
list(APPEND CUDA_NVCC_FLAGS "-DENABLE_FAST_DEBUG")
list(APPEND CUDA_NVCC_FLAGS "-O3")
list(APPEND CUDA_NVCC_FLAGS "--ptxas-options=--verbose,--register-usage-level=10,--warn-on-local-memory-usage")

set(USE_SYSTEM_NVTX on)
set(CUDA_ARCH_LIST "9.0" CACHE STRING "List of CUDA architectures to compile")
set(TORCH_CUDA_ARCH_LIST "${CUDA_ARCH_LIST}")

set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CUDA_STANDARD 20)
```

--------------------------------

### Mega MoE Kernel Setup

Source: https://github.com/deepseek-ai/deepgemm/blob/main/README.md

Allocates symmetric memory buffer required for Mega MoE operations. Requires PyTorch version 2.9 or higher.

```python
# Allocate symmetric memory buffer
# NOTES: requires PyTorch >= 2.9
buffer = deep_gemm.get_symm_buffer_for_mega_moe(
    group, num_experts, num_max_tokens_per_rank, num_topk, hidden, intermediate_hidden
)
```

--------------------------------

### Finding and Linking Dependencies

Source: https://github.com/deepseek-ai/deepgemm/blob/main/CMakeLists.txt

Locates required packages like CUDAToolkit, pybind11, and PyTorch, and sets include and link directories. Ensure these packages are installed and discoverable by CMake.

```cmake
find_package(CUDAToolkit REQUIRED)
find_package(pybind11 REQUIRED)
find_package(Torch REQUIRED)

include_directories(deep_gemm/include third-party/cutlass/include third-party/cutlass/tools/util/include third-party/fmt/include)
include_directories(${CUDA_TOOLKIT_ROOT_DIR}/include/cccl ${TORCH_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
link_directories(${TORCH_INSTALL_PREFIX}/lib ${CUDA_TOOLKIT_ROOT_DIR}/lib64 ${CUDA_TOOLKIT_ROOT_DIR}/lib64/stubs)
```

--------------------------------

### Develop DeepGEMM CPP JIT Module

Source: https://github.com/deepseek-ai/deepgemm/blob/main/README.md

After cloning the repository, link essential includes and build the C++ Just-In-Time (JIT) compilation module using the provided develop script.

```bash
cat develop.sh
./develop.sh
```

--------------------------------

### Clone DeepGEMM Repository and Submodules

Source: https://github.com/deepseek-ai/deepgemm/blob/main/README.md

To set up the development environment, clone the DeepGEMM repository and its recursive submodules. This ensures all necessary dependencies, like CUTLASS and {fmt}, are included.

```bash
git clone --recursive git@github.com:deepseek-ai/DeepGEMM.git
cd DeepGEMM
```

--------------------------------

### Building Python API Module

Source: https://github.com/deepseek-ai/deepgemm/blob/main/CMakeLists.txt

Creates a Python extension module named '_C' from the specified C++ source file. Links against PyTorch libraries for Python integration.

```cmake
# The main Python API entrance
pybind11_add_module(_C csrc/python_api.cpp)
target_link_libraries(_C PRIVATE ${TORCH_LIBRARIES} torch_python)
```

--------------------------------

### Mega MoE Weight Transformation

Source: https://github.com/deepseek-ai/deepgemm/blob/main/README.md

Transforms weights from FP4 with UE8M0 SF into the layout required by the Mega MoE kernel.

```python
# Transform weights (FP4 with UE8M0 SF) into the required layout
transformed_l1, transformed_l2 = deep_gemm.transform_weights_for_mega_moe(l1_weights, l2_weights)
```

--------------------------------

### Building CUDA Indexing Library

Source: https://github.com/deepseek-ai/deepgemm/blob/main/CMakeLists.txt

Compiles a static CUDA library for kernel code indexing. This is primarily for IDE indexing and not the main JIT compilation.

```cmake
# Enable kernel code indexing with CMake-based IDEs
cuda_add_library(deep_gemm_indexing_cuda STATIC csrc/indexing/main.cu)
```

--------------------------------

### Mega MoE Kernel Execution

Source: https://github.com/deepseek-ai/deepgemm/blob/main/README.md

Executes the fused Mega MoE kernel, which combines EP dispatch, linear transformations, SwiGLU, and EP combine.

```python
# Run the fused mega MoE kernel
y = torch.empty((num_tokens, hidden), dtype=torch.bfloat16, device='cuda')
deep_gemm.fp8_fp4_mega_moe(y, transformed_l1, transformed_l2, buffer)
```

--------------------------------

### Mega MoE Input Copy

Source: https://github.com/deepseek-ai/deepgemm/blob/main/README.md

Copies input data into the symmetric memory buffer before executing the Mega MoE kernel. These copies can be fused into previous kernels.

```python
# Copy inputs into the buffer before each call
# You may fuse these into previous kernels
buffer.x[:num_tokens].copy_(x_fp8)
buffer.x_sf[:num_tokens].copy_(x_sf)
buffer.topk_idx[:num_tokens].copy_(topk_idx)
buffer.topk_weights[:num_tokens].copy_(topk_weights)
```

--------------------------------

### MQA Logits Calculation (Non-Paged)

Source: https://github.com/deepseek-ai/deepgemm/blob/main/README.md

Calculates token-to-token logits for MQA. This non-paged version is for prefilling.

```python
fp8_mqa_logits
```

--------------------------------

### Perform FP8 GEMM Operation

Source: https://github.com/deepseek-ai/deepgemm/blob/main/README.md

This function performs a basic non-grouped FP8 GEMM operation. The specific function to call depends on the desired memory layout (e.g., `fp8_gemm_nt` for non-transposed A and transposed B).

```python
fp8_gemm_{nt, nn, tn, tt}
```

--------------------------------

### Contiguous Grouped GEMM (M-axis)

Source: https://github.com/deepseek-ai/deepgemm/blob/main/README.md

Used for training forward passes or inference prefilling where experts share the same shape. Requires experts to be aligned to the GEMM M block size.

```python
kv_j = kv[0][j, :] * kv[1][j].unsqueeze(1)  # [head_dim]
out_ij = q[i, :, :] @ kv_j  # [num_heads]
out_ij = out_ij.relu() * weights[i, :]  # [num_heads]
out_ij = out_ij.sum()  # Scalar
```

--------------------------------

### Masked Grouped GEMM (M-axis)

Source: https://github.com/deepseek-ai/deepgemm/blob/main/README.md

Supports masked grouped GEMMs during inference decoding with CUDA graphs enabled. Computes only valid portions using a mask tensor.

```python
m_grouped_fp8_gemm_nt_masked
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.