### Install spconv and cumm from Source on Windows

Source: https://github.com/traveller59/spconv/blob/master/README.md

Detailed steps to set up the development environment and install spconv and cumm in editable mode on Windows. This involves Visual Studio setup, PowerShell script execution, cloning repositories, and pip installation.

```PowerShell
tools/msvc_setup.ps1
```

```Shell
git clone https://github.com/FindDefinition/cumm
cd ./cumm
pip install -e .
```

```Shell
git clone https://github.com/traveller59/spconv
cd ./spconv
pip install -e .
```

```Python
import spconv
```

--------------------------------

### Check Installed SpConv and CUMM Packages

Source: https://github.com/traveller59/spconv/blob/master/README.md

Lists all installed SpConv and CUMM packages to verify existing installations, a recommended step before updating the library.

```Shell
pip list | grep spconv
pip list | grep cumm
```

--------------------------------

### Unfusable Spconv Residual Block Example for Quantization

Source: https://github.com/traveller59/spconv/blob/master/docs/INT8_GUIDE.md

This Python code demonstrates a `SparseBasicBlock` implementation in spconv that cannot be fused for int8 quantization due to the use of `out.replace_feature` and direct access to `out.features`. These operations create standalone nodes in the `torch.fx` graph, preventing proper recognition and fusion for optimized int8 operations.

```Python
class SparseBasicBlock(spconv.SparseModule):
    expansion = 1
    def __init__(self,
                 in_planes, out_planes,
                 stride=1,
                 downsample=None):
        spconv.SparseModule.__init__(self)
        conv1 = spconv.SubMConv2d(in_planes, out_planes, 3, stride, 1, bias=False)
        conv2 = spconv.SubMConv2d(out_planes, out_planes, 3, stride, 1, bias=False)

        norm1 = nn.BatchNorm1d(out_planes, momentum=0.1)
        norm2 = nn.BatchNorm1d(out_planes, momentum=0.1)

        self.conv1_bn_relu = spconv.SparseSequential(conv=conv1, bn=norm1, relu=nn.ReLU(inplace=True))
        self.conv2_bn = spconv.SparseSequential(conv=conv2, bn=norm2)
        self.relu = nn.ReLU(inplace=True)
        self.downsample = downsample
        self.iden_for_fx_match = nn.Identity()

    def forward(self, x: spconv.SparseConvTensor):
        identity = x.features
        out = self.conv1_bn_relu(x)
        out = self.conv2_bn(out)

        if self.downsample is not None:
            identity = self.downsample(x)

        out = out.replace_feature(self.relu(out.features + identity))
        return out
```

--------------------------------

### Fusible Spconv Residual Block Example for Quantization

Source: https://github.com/traveller59/spconv/blob/master/docs/INT8_GUIDE.md

This Python code presents a `SparseBasicBlock` designed for spconv int8 quantization, enabling residual fusion. It achieves this by replacing `nn.ReLU` with `spconv.SparseReLU` and `nn.Identity` with `spconv.SparseIdentity`, and modifying the residual addition to avoid `replace_feature` and direct feature access, making it compatible with `torch.fx` graph fusion for improved performance.

```Python
class SparseBasicBlock(spconv.SparseModule):
    """residual block that supported by spconv quantization.
    """
    expansion = 1
    def __init__(self,
                 in_planes, out_planes,
                 stride=1,
                 downsample=None):
        spconv.SparseModule.__init__(self)
        conv1 = spconv.SubMConv2d(in_planes, out_planes, 3, stride, 1, bias=False)
        conv2 = spconv.SubMConv2d(out_planes, out_planes, 3, stride, 1, bias=False)

        norm1 = nn.BatchNorm1d(out_planes, momentum=0.1)
        norm2 = nn.BatchNorm1d(out_planes, momentum=0.1)

        self.conv1_bn_relu = spconv.SparseSequential(conv=conv1, bn=norm1, relu=nn.ReLU(inplace=True))
        self.conv2_bn = spconv.SparseSequential(conv=conv2, bn=norm2)

        self.relu = spconv.SparseReLU(inplace=True)
        self.downsample = downsample
        self.iden_for_fx_match = spconv.SparseIdentity()

    def forward(self, x: spconv.SparseConvTensor):
        identity = x
        # if self.training:
        #     assert x.features.dim() == 2, f'x.features.dim()={x.features.dim()}'
        out = self.conv1_bn_relu(x)
        out = self.conv2_bn(out)

        if self.downsample is not None:
            identity = self.downsample(x)
        out = self.relu(out + identity)
        return out
```

--------------------------------

### Run Training for Quantization Aware Training (QAT) in PyTorch

Source: https://github.com/traveller59/spconv/blob/master/docs/INT8_GUIDE.md

This Python snippet demonstrates how to perform training on the `prepared_model` after it has been prepared for Quantization Aware Training (QAT). During this phase, the model learns with the inserted fake quantization nodes, allowing it to adapt to the quantized environment and improve int8 inference accuracy.

```Python
train(prepared_model)
```

--------------------------------

### Run Inference for PTQ Observer Calibration in PyTorch

Source: https://github.com/traveller59/spconv/blob/master/docs/INT8_GUIDE.md

This Python snippet shows how to run inference on the `prepared_model` after preparation for PTQ. This step is crucial for the observers to collect statistics and calculate the necessary int8 scales, which are then used for the final quantization.

```Python
for x in loader:
    prepared_model(x)
```

--------------------------------

### Prepare Model for Quantization Aware Training (QAT) in PyTorch

Source: https://github.com/traveller59/spconv/blob/master/docs/INT8_GUIDE.md

This Python snippet illustrates how to prepare a PyTorch model for Quantization Aware Training (QAT) using `spconv.pytorch.quantization` and `torch.fx`. It configures `qconfig_mapping` specifically for QAT (`is_qat=True`) and then applies `qfx.prepare_qat_fx` to insert observers and fake quantize nodes into the model, making it ready for training.

```Python
import spconv.pytorch.quantization as spconvq
import torch.ao.quantization.quantize_fx as qfx

model = ...
is_qat = True
qconfig_mapping = spconvq.get_default_spconv_qconfig_mapping(is_qat)
prepare_cfg = spconvq.get_spconv_prepare_custom_config()
backend_cfg = spconvq.get_spconv_backend_config()
# keep in mind that all model user attrs are lost in prepared_model.
prepared_model = qfx.prepare_qat_fx(model, qconfig_mapping, (), backend_config=backend_cfg, prepare_custom_config=prepare_cfg)
```

--------------------------------

### Install Spconv for CUDA 10.2

Source: https://github.com/traveller59/spconv/blob/master/README.md

This command installs the prebuilt Spconv binary compatible with CUDA 10.2. Ensure you have the necessary CUDA toolkit and driver installed.

```Python
pip install spconv-cu102
```

--------------------------------

### Build spconv Wheel from Source on Windows

Source: https://github.com/traveller59/spconv/blob/master/README.md

Instructions for building a spconv wheel package from source on Windows, including Visual Studio setup, PowerShell script execution, and disabling JIT compilation. This method is generally not recommended for end-users.

```PowerShell
tools/msvc_setup.ps1
```

```PowerShell
$Env:SPCONV_DISABLE_JIT = "1"
```

```PowerShell
pip install pccm cumm wheel
```

```PowerShell
python setup.py bdist_wheel
```

```PowerShell
pip install dists/xxx.whl
```

--------------------------------

### Install Spconv for CUDA 12.0

Source: https://github.com/traveller59/spconv/blob/master/README.md

This command installs the prebuilt Spconv binary compatible with CUDA 12.0. Ensure you have the necessary CUDA toolkit and driver installed.

```Python
pip install spconv-cu120
```

--------------------------------

### Prepare Model for Post Training Quantization (PTQ) in PyTorch

Source: https://github.com/traveller59/spconv/blob/master/docs/INT8_GUIDE.md

This Python snippet demonstrates how to prepare a PyTorch model for Post Training Quantization (PTQ) using `spconv.pytorch.quantization` and `torch.fx`. It involves configuring `qconfig_mapping` to control quantization behavior, including options to disable quantization for specific layers or types, and setting `prepare_cfg` and `backend_cfg` before applying `qfx.prepare_fx`.

```Python
import spconv.pytorch.quantization as spconvq
import torch.ao.quantization.quantize_fx as qfx

model = ...
is_qat = False
qconfig_mapping = spconvq.get_default_spconv_qconfig_mapping(is_qat)
# disable quantization for some layers here:
qconfig_mapping.set_module_name_regex("foo.*bar.*", None)
# disable quantization by type here:
qconfig_mapping.set_object_type(ModuleClass, None)

prepare_cfg = spconvq.get_spconv_prepare_custom_config()
# preserve static attrs for your module here:
prepare_cfg.preserved_attributes = [...]
# add nontraceable modules here:
prepare_cfg.non_traceable_module_classes.extend([...])

backend_cfg = spconvq.get_spconv_backend_config()
# add custom qconfig for your non-traceable operators:
backend_cfg.set_backend_pattern_config(BackendPatternConfig(some_op_or_module_class).set_observation_type(
    ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT).set_dtype_configs(
        [non_weighted_op_qint8_dtype_config]))

# keep in mind that all model user attrs are lost in prepared_model.
prepared_model = qfx.prepare_fx(model, qconfig_mapping, (), backend_config=backend_cfg, prepare_custom_config=prepare_cfg)
```

--------------------------------

### Install Spconv for CUDA 11.4

Source: https://github.com/traveller59/spconv/blob/master/README.md

This command installs the prebuilt Spconv binary compatible with CUDA 11.4. Ensure you have the necessary CUDA toolkit and driver installed.

```Python
pip install spconv-cu114
```

--------------------------------

### Install Spconv for CUDA 11.3 (Linux Only)

Source: https://github.com/traveller59/spconv/blob/master/README.md

This command installs the prebuilt Spconv binary compatible with CUDA 11.3. This version is available for Linux only. Ensure you have the necessary CUDA toolkit and driver installed.

```Python
pip install spconv-cu113
```

--------------------------------

### Build spconv Wheel from Source on Linux

Source: https://github.com/traveller59/spconv/blob/master/README.md

Instructions for building a spconv wheel package from source on Linux, including dependency installation and disabling JIT compilation. This method is generally not recommended for end-users.

```Bash
export SPCONV_DISABLE_JIT="1"
```

```Bash
pip install pccm cumm wheel
```

```Bash
python setup.py bdist_wheel
```

```Bash
pip install dists/xxx.whl
```

--------------------------------

### Install SpConv for CPU (Linux Only)

Source: https://github.com/traveller59/spconv/blob/master/README.md

Installs the CPU-only version of the SpConv library, designed for Linux environments without CUDA dependencies.

```Shell
pip install spconv
```

--------------------------------

### Implementing Sparse Convolutional Networks with SparseSequential

Source: https://github.com/traveller59/spconv/blob/master/docs/USAGE.md

This example defines an `ExampleNet` class that utilizes `spconv.SparseSequential` to build a sparse convolutional neural network. It showcases the use of `SparseConv3d`, `SubMConv3d` (with shared `indice_key`), `SparseConvTranspose3d`, and `ToDense` layers, along with standard PyTorch layers like `BatchNorm1d` and `ReLU`. This demonstrates how to construct complex sparse architectures.

```Python
import spconv.pytorch as spconv
from torch import nn
class ExampleNet(nn.Module):
    def __init__(self, shape):
        super().__init__()
        self.net = spconv.SparseSequential(
            spconv.SparseConv3d(32, 64, 3), # just like nn.Conv3d but don't support group
            nn.BatchNorm1d(64), # non-spatial layers can be used directly in SparseSequential.
            nn.ReLU(),
            spconv.SubMConv3d(64, 64, 3, indice_key="subm0"),
            nn.BatchNorm1d(64),
            nn.ReLU(),
            # when use submanifold convolutions, their indices can be shared to save indices generation time.
            spconv.SubMConv3d(64, 64, 3, indice_key="subm0"),
            nn.BatchNorm1d(64),
            nn.ReLU(),
            spconv.SparseConvTranspose3d(64, 64, 3, 2),
            nn.BatchNorm1d(64),
            nn.ReLU(),
            spconv.ToDense(), # convert spconv tensor to dense and convert it to NCHW format.
            nn.Conv3d(64, 64, 3),
            nn.BatchNorm1d(64),
            nn.ReLU(),
        )
        self.shape = shape

    def forward(self, features, coors, batch_size):
        coors = coors.int() # unlike torch, this library only accept int coordinates.
        x = spconv.SparseConvTensor(features, coors, self.shape, batch_size)
        return self.net(x)# .dense()
```

--------------------------------

### Install Spconv for CUDA 11.7

Source: https://github.com/traveller59/spconv/blob/master/README.md

This command installs the prebuilt Spconv binary compatible with CUDA 11.7. Note that `spconv-cu117` may require CUDA Driver >= 515. Ensure you have the necessary CUDA toolkit and driver installed.

```Python
pip install spconv-cu117
```

--------------------------------

### Build Spconv and Cumm from Source on Linux

Source: https://github.com/traveller59/spconv/blob/master/README.md

This sequence of commands outlines the steps to build Spconv from source on Linux for development purposes, including installing `cumm` and `spconv` in editable mode. This allows for automatic recompilation of C++ code changes.

```Shell
git clone https://github.com/FindDefinition/cumm
cd ./cumm
pip install -e .
git clone https://github.com/traveller59/spconv
cd ./spconv
pip install -e .
```

--------------------------------

### Prepare Spconv PyTorch Inference with Optional Linear Layer Handling

Source: https://github.com/traveller59/spconv/blob/master/docs/TENSORRT_INT8_GUIDE.md

This Python code shows a simpler way to prepare a Spconv PyTorch model for inference using `spconvq.prepare_spconv_torch_inference`. It provides an option (`with_linear=False`) to control whether linear layers are included in the preparation process, which is useful when specific layers should not undergo QDQ fusion.

```Python
spconvq.prepare_spconv_torch_inference(with_linear=False)
```

--------------------------------

### Install SpConv for CUDA 11.6

Source: https://github.com/traveller59/spconv/blob/master/README.md

Installs the SpConv library version compiled for CUDA 11.6, enabling GPU-accelerated sparse convolutions.

```Shell
pip install spconv-cu116
```

--------------------------------

### Install SpConv for CUDA 11.8

Source: https://github.com/traveller59/spconv/blob/master/README.md

Installs the SpConv library version compiled for CUDA 11.8, enabling GPU-accelerated sparse convolutions.

```Shell
pip install spconv-cu118
```

--------------------------------

### Install SpConv for CUDA 11.3

Source: https://github.com/traveller59/spconv/blob/master/README.md

Installs the SpConv library version compiled for CUDA 11.3, enabling GPU-accelerated sparse convolutions.

```Shell
pip install spconv-cu113
```

--------------------------------

### Example Convolution Parameters and Coordinates Causing CUDA Kernel Error

Source: https://github.com/traveller59/spconv/blob/master/docs/COMMON_PROBLEMS.md

Illustrates example convolution parameters and input coordinates that can lead to a 'CUDA kernel launch blocks must be positive' error. This specific example shows how a convolution with `spatial shape=[8, 200, 200], ksize=3, stride=2, padding=[0, 1, 1], dilation=1` applied to coordinates like `[[0, 7, 153, 142]]` can drop all points in the z-axis (z == 7), resulting in an empty output and the error. Modifying padding is suggested as a solution.

```text
Conv Params:
spatial shape=[8, 200, 200],ksize=3,stride=2,padding=[0, 1, 1],dilation=1
Coordinates:
[[0, 7, 153, 142]]
```

--------------------------------

### Install SpConv for CUDA 12.0

Source: https://github.com/traveller59/spconv/blob/master/README.md

Installs the SpConv library version compiled for CUDA 12.0, enabling GPU-accelerated sparse convolutions.

```Shell
pip install spconv-cu120
```

--------------------------------

### Install SpConv for CUDA 11.7

Source: https://github.com/traveller59/spconv/blob/master/README.md

Installs the SpConv library version compiled for CUDA 11.7, enabling GPU-accelerated sparse convolutions.

```Shell
pip install spconv-cu117
```

--------------------------------

### Install SpConv for CUDA 10.2

Source: https://github.com/traveller59/spconv/blob/master/README.md

Installs the SpConv library version compiled for CUDA 10.2, enabling GPU-accelerated sparse convolutions.

```Shell
pip install spconv-cu102
```

--------------------------------

### Convert and Test PTQ Model for Int8 Inference in PyTorch

Source: https://github.com/traveller59/spconv/blob/master/docs/INT8_GUIDE.md

This Python snippet outlines the steps to convert a PTQ-prepared model for int8 inference in PyTorch. It includes calling `spconvq.prepare_spconv_torch_inference` to enable per-channel weight support for Linear layers, converting the model with `qfx.convert_fx`, transforming QDQ nodes with `spconvq.transform_qdq` to support `SparseConvTensor`, and removing DQ nodes with `spconvq.remove_conv_add_dq` for debug testing.

```Python
with_linear = True
# pytorch don't support per-channel weight for Linear in native backend, we implement a simple fake-quantization to do this.
spconvq.prepare_spconv_torch_inference(with_linear) # must call before convert_fx

converted_model = qfx.convert_fx(prepared_model, qconfig_mapping=qconfig_mapping, backend_config=backend_cfg)
# transform all torch quantize_per_tensor to custom quantize_per_tensor to support SparseConvTensor
converted_model = spconvq.transform_qdq(converted_model)
# remove all dq node in fused residual node
converted_model = spconvq.remove_conv_add_dq(converted_model)

test(converted_model)
```

--------------------------------

### Configure spconv Project with CMake

Source: https://github.com/traveller59/spconv/blob/master/example/libspconv/spconv/CMakeLists.txt

This snippet defines the build configuration for the 'spconv' project using CMake. It sets the minimum CMake version, specifies C++ and CUDA as project languages, includes the 'include' directory, adds the 'src' subdirectory, sets an include path for a parent project ('SPCONV2_INCLUDE_PATH'), and defines installation destinations for the compiled 'spconv' target.

```CMake
cmake_minimum_required(VERSION 3.20)

project(spconv LANGUAGES CXX CUDA)
include_directories(include)

add_subdirectory(src)
# tell parent spconv2 include path.
set(SPCONV2_INCLUDE_PATH ${${PROJECT_NAME}_SOURCE_DIR}/include PARENT_SCOPE)

install (TARGETS spconv
         ARCHIVE DESTINATION lib
         LIBRARY DESTINATION lib
         RUNTIME DESTINATION bin)
```

--------------------------------

### Install Spconv for CPU Only (Linux)

Source: https://github.com/traveller59/spconv/blob/master/README.md

This command installs a CPU-only version of Spconv via pip. It is primarily for debugging purposes on Linux as performance is not optimized due to manylinux limitations (no OpenMP support).

```Python
pip install spconv
```

--------------------------------

### Install SpConv for CUDA 11.4

Source: https://github.com/traveller59/spconv/blob/master/README.md

Installs the SpConv library version compiled for CUDA 11.4, enabling GPU-accelerated sparse convolutions. This version is recommended for potentially faster kernel compilation.

```Shell
pip install spconv-cu114
```

--------------------------------

### Verify Spconv Source Build in Python

Source: https://github.com/traveller59/spconv/blob/master/README.md

After installing Spconv in editable mode from source, import it in Python. This action will trigger the C++ code compilation if it hasn't been built yet.

```Python
import spconv
```

--------------------------------

### Spconv Int8 Scale/Bias Conversion Basic Rule

Source: https://github.com/traveller59/spconv/blob/master/docs/TENSORRT_INT8_GUIDE.md

This C++-like pseudocode defines the fundamental conversion rules between FP32 and Int8 data types used in Spconv's quantization scheme. It illustrates how floating-point data is converted to Int8 by scaling, rounding, and saturating, and how Int8 data is converted back to FP32 using a scale factor.

```C++
fp32_data = float(int8_data) * scale
int8_data = int8_t(saturate(round(fp32_data / scale)))
```

--------------------------------

### Set Environment Variables for Spconv C++ Build

Source: https://github.com/traveller59/spconv/blob/master/docs/PURE_CPP_BUILD.md

Sets crucial environment variables required for the spconv C++ build process. This includes specifying the CUDA version, disabling JIT compilation for both CUMM and SPCONV, defining the CUMM include path for subdirectory setups, and listing target CUDA architectures.

```Bash
export CUMM_CUDA_VERSION=11.4 # cuda version, required
export CUMM_DISABLE_JIT=1
export SPCONV_DISABLE_JIT=1
export CUMM_INCLUDE_PATH="\${CUMM_INCLUDE_PATH}" # if you use cumm as a subdirectory, you need this to find cumm includes.
export CUMM_CUDA_ARCH_LIST="6.1;7.5;8.6" # cuda arch flags
```

--------------------------------

### Call Spconv Implicit GEMM with Calculated Scales and Bias

Source: https://github.com/traveller59/spconv/blob/master/docs/TENSORRT_INT8_GUIDE.md

This C++ code snippet shows how to invoke the `ConvGemmOps::implicit_gemm` function in Spconv. It demonstrates feeding the pre-calculated `bias_for_spconv_implicit_gemm` and `scale_for_spconv_implicit_gemm` along with other necessary parameters for sparse convolution operations, highlighting their use in the GEMM kernel.

```C++
// output_add and output_add_scale: for fused conv-add-relu layer
ConvGemmOps::implicit_gemm(
    allocator, tuner, features_int8, weight_int8, pair_fwd,
    pair_mask_splits, mask_argsort_splits, actual_out_feature_num,
    mask_tensor, arch, false, is_subm,
    reinterpret_cast<std::uintptr_t>(stream), tv::CUDAKernelTimer(false),
    false, false, bias_for_spconv_implicit_gemm, 1.0,
    0.0, tv::gemm::Activation::kReLU, false /*use_tf32*/, output_scale, 
    scale_for_spconv_implicit_gemm, output_add, output_add_scale);
```

--------------------------------

### Optimize Int8 Kernels by Disabling Sort in Implicit Gemm (C++)

Source: https://github.com/traveller59/spconv/blob/master/docs/INT8_GUIDE.md

This C++ snippet shows how to disable sorting in `SpconvOps::get_indice_pairs_implicit_gemm` for int8 kernel performance optimization. Setting `do_sort` to `false` can significantly improve performance for implicit GEMM operations within spconv by avoiding unnecessary sorting.

```C++
bool do_sort = false;
pair_res = SpconvOps::get_indice_pairs_implicit_gemm(
    alloc, input_indices_real, batch_size, input_dims,
    static_cast<int>(conv_algo), ksize, stride, padding, dilation,
    {0, 0, 0}, is_subm, transpose, false /*is_train*/,
    reinterpret_cast<std::uintptr_t>(stream), out_inds_num_limit,
    tv::CUDAKernelTimer(false), use_direct_table, do_sort);
```

--------------------------------

### TensorRT Int8 Plugin Requirements and Limitations

Source: https://github.com/traveller59/spconv/blob/master/docs/TENSORRT_INT8_GUIDE.md

This section outlines the critical requirements and limitations for developing and using custom Int8 plugins within TensorRT. It specifies conditions such as static input shapes, voxel count handling, strict `supportsFormatCombination` behavior, minimum tensor dimensions for Int8, and TensorRT version compatibility. These are essential behavioral specifications for TensorRT plugins.

```APIDOC
TensorRT Int8 Plugin Requirements:
1. Pad all inputs to a static shape.
2. Use a tensor to save current number of voxels, copy it to CPU and slice all inputs to real shape during inference (enqueue).
3. `supportsFormatCombination` must allow exactly one combination, i.e., set dtype of this layer during network build. For example, if fp16 is desired, this function must accept fp16 and reject other dtypes to avoid TensorRT performing dtype/format selection during engine build.
4. Number of dimensions of int8 tensor for plugin must be >= 3 (tested in TensorRT 8.4).
5. TensorRT version >= 8.4 (TensorRT 8.0 does not support int8 plugin).
```

--------------------------------

### Calculate Spconv Int8 Scale and Bias from PyTorch Quantized Layer

Source: https://github.com/traveller59/spconv/blob/master/docs/TENSORRT_INT8_GUIDE.md

This Python code snippet demonstrates how to extract and calculate the necessary `scale_for_spconv_implicit_gemm` and `bias_for_spconv_implicit_gemm` from a PyTorch quantized layer. It supports different types of Spconv quantized layers (e.g., `snnqr.SpConv`, `snniq.SparseConvReLU`) and prepares the scales and biases for Spconv's implicit GEMM operations.

```Python
import spconv.pytorch.quantization.quantized.reference as snnqr
import spconv.pytorch.quantization.intrinsic.quantized as snniq
import spconv.pytorch.quantization.quantized as snnq

input_scale = ...
output_scale = ...
if isinstance(layer, snnqr.SpConv):
    q_weight = layer.get_quantized_weight() # for snnqr.SpConv
    bias_np = layer.bias.detach().cpu().numpy()
elif isinstance(layer, (snniq.SparseConvReLU, snniq.SparseConvAddReLU, snnq.SparseConv)):
    q_weight = layer.weight() # for quantized layers
    bias_np = layer.bias().detach().cpu().numpy()
else:
    raise NotImplementedError
w_perchannel_scales = q_weight.q_per_channel_scales().detach().cpu().numpy().astype(np.float32)

scale_for_spconv_implicit_gemm = (input_scale * w_perchannel_scales) / output_scale
bias_for_spconv_implicit_gemm = bias_np / output_scale
```

--------------------------------

### Gathering Point-wise Features from Voxelized Data using spconv.pytorch.utils.gather_features_by_pc_voxel_id

Source: https://github.com/traveller59/spconv/blob/master/docs/USAGE.md

This example shows how to retrieve point-wise features from a semantic segmentation network's output, which is typically in a voxelized format. It utilizes gen.generate_voxel_with_id to obtain pc_voxel_id for each point, then uses gather_features_by_pc_voxel_id to map voxel features back to the original points. Points outside the defined range or without space in a voxel will have zero features.

```Python
voxels, coords, num_points_per_voxel, pc_voxel_id = gen.generate_voxel_with_id(pc_th, empty_mean=True)
seg_features = YourSegNet(...)
# if voxel id is invalid (point out of range, or no space left in a voxel)
# features will be zero.
point_features = gather_features_by_pc_voxel_id(seg_features, pc_voxel_id)
```

--------------------------------

### Using spconv.pytorch.functional.sparse_add with Inverse Operations

Source: https://github.com/traveller59/spconv/blob/master/docs/USAGE.md

This example demonstrates the Fsp.sparse_add function for combining sparse tensors with different indices. It highlights a critical limitation: if the added tensors' combined indices change the spatial structure in a way that prevents inverse operations (e.g., with SparseInverseConv), the usage is incorrect. The correct approach ensures that the output spatial structure is maintained, typically by including a tensor that already encompasses all necessary indices, allowing for subsequent inverse transformations.

```Python
from spconv.pytorch import functional as Fsp
res_1x3 = conv1x3(x)
res_3x1 = conv3x1(x)
# WRONG
# because we can't "inverse" this operation
wrong_usage_cant_inverse = Fsp.sparse_add(res_1x3, res_3x1)

# CORRECT
# res_3x3 already contains all indices of res_1x3 and res_3x1, 
# so output spatial structure isn't changed, we can "inverse" back.
res_3x3 = conv3x3(x)
correct = Fsp.sparse_add(res_1x3, res_3x1, res_3x3)
```

--------------------------------

### Customize PyTorch convert_fx for Spconv QDQ Fusion

Source: https://github.com/traveller59/spconv/blob/master/docs/TENSORRT_INT8_GUIDE.md

This Python code demonstrates a method to modify PyTorch's quantization FX graph transformation (`convert_fx`) to correctly handle Spconv layers and prevent unwanted QDQ fusion for specific regular layers (e.g., Linear). It involves updating internal PyTorch maps with Spconv-specific configurations and removing entries for layers that should retain their QDQs for TensorRT explicit quantization.

```Python
import torch.ao.nn.intrinsic as nni
import torch.nn.quantized._reference as nnqr

from torch.ao.quantization.fx._lower_to_native_backend import \
    STATIC_LOWER_FUSED_MODULE_MAP, STATIC_LOWER_MODULE_MAP, QBIN_OP_MAPPING
from spconv.pytorch.quantization.backend_cfg import \
    SPCONV_STATIC_LOWER_FUSED_MODULE_MAP, SPCONV_STATIC_LOWER_MODULE_MAP
# add spconv layers to support QDQ fusion for sparse conv layers
STATIC_LOWER_FUSED_MODULE_MAP.update(SPCONV_STATIC_LOWER_FUSED_MODULE_MAP)
STATIC_LOWER_MODULE_MAP.update(SPCONV_STATIC_LOWER_MODULE_MAP)
# remove linear layers to avoid QDQ fusion for linear.
STATIC_LOWER_FUSED_MODULE_MAP.pop(nni.LinearReLU)
STATIC_LOWER_MODULE_MAP.pop(nnqr.Linear)
# run above BEFORE convert_fx
converted_model = qfx.convert_fx(prepared_model, qconfig_mapping=qconfig_mapping, backend_config=backend_cfg)
```

--------------------------------

### Importing spconv.pytorch Modules

Source: https://github.com/traveller59/spconv/blob/master/docs/USAGE.md

Demonstrates the standard way to import core spconv.pytorch modules, including functional APIs, utilities for point cloud processing, and hash tables.

```Python
import spconv.pytorch as spconv
from spconv.pytorch import functional as Fsp
from torch import nn
from spconv.pytorch.utils import PointToVoxel
from spconv.pytorch.hash import HashTable
```

--------------------------------

### Debug Manylinux Build with Docker for spconv

Source: https://github.com/traveller59/spconv/blob/master/tools/README.md

This snippet provides commands to set up a Docker environment for debugging manylinux builds of spconv. It mounts the current working directory and the user's home directory, then executes the build script within the container.

```Bash
docker run --rm -it -e PLAT=manylinux2014_x86_64 -v `pwd`:/io -v $HOME:/myhome scrin/manylinux2014-cuda:cu114-devel bash
/io/tools/build-wheels.sh
```

--------------------------------

### spconv.pytorch.utils API Reference

Source: https://github.com/traveller59/spconv/blob/master/docs/USAGE.md

Detailed API documentation for key utility functions and classes within spconv.pytorch.utils for point cloud processing.

```APIDOC
Class: spconv.pytorch.utils.PointToVoxel
  Description: A utility class to convert point cloud data into a voxelized representation.
  __init__(self, vsize_xyz: list, coors_range_xyz: list, num_point_features: int, max_num_voxels: int, max_num_points_per_voxel: int)
    vsize_xyz: List of 3 floats, voxel size in XYZ order.
    coors_range_xyz: List of 6 floats, min/max coordinates [x_min, y_min, z_min, x_max, y_max, z_max].
    num_point_features: Integer, number of features per point.
    max_num_voxels: Integer, maximum number of voxels to generate.
    max_num_points_per_voxel: Integer, maximum number of points allowed in a single voxel.
  __call__(self, pc_th: torch.Tensor, empty_mean: bool = True) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor]
    Description: Generates voxels, coordinates, and number of points per voxel.
    pc_th: Input point cloud tensor (N, 3+features).
    empty_mean: Boolean, if True, empty voxels will have mean features.
    Returns:
      voxels: Tensor of voxel features.
      coords: Tensor of voxel coordinates (ZYX order, no batch axis).
      num_points_per_voxel: Tensor of number of points in each voxel.
  generate_voxel_with_id(self, pc_th: torch.Tensor, empty_mean: bool = True) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]
    Description: Generates voxels, coordinates, number of points per voxel, and point-to-voxel IDs.
    pc_th: Input point cloud tensor (N, 3+features).
    empty_mean: Boolean, if True, empty voxels will have mean features.
    Returns:
      voxels: Tensor of voxel features.
      coords: Tensor of voxel coordinates (ZYX order, no batch axis).
      num_points_per_voxel: Tensor of number of points in each voxel.
      pc_voxel_id: Tensor mapping each point to its voxel ID. Invalid IDs result in zero features when gathered.

Function: spconv.pytorch.utils.gather_features_by_pc_voxel_id(seg_features: torch.Tensor, pc_voxel_id: torch.Tensor) -> torch.Tensor
  Description: Gathers features from voxelized segmentation results back to original points using point-to-voxel IDs.
  seg_features: Tensor of segmentation features per voxel.
  pc_voxel_id: Tensor mapping each point to its voxel ID, obtained from generate_voxel_with_id.
  Returns:
    point_features: Tensor of features for each original point. Features are zero if voxel ID is invalid (point out of range or no space).
```

--------------------------------

### spconv.pytorch Layer APIs Comparison with torchsparse and MinkowskiEngine

Source: https://github.com/traveller59/spconv/blob/master/docs/USAGE.md

A comparative reference of spconv.pytorch layer APIs against their equivalents in torchsparse and MinkowskiEngine, highlighting similar functionalities across different sparse convolution frameworks.

```APIDOC
spconv.SparseConv3d:
  torchsparse: Conv3d(stride!=1, transpose=False)
  MinkowskiEngine: MinkowskiConvolution(stride!=1)

spconv.SubMConv3d:
  torchsparse: Conv3d(stride=1, transpose=False)
  MinkowskiEngine: MinkowskiConvolution(stride=1)

spconv.SparseInverseConv3d:
  torchsparse: Conv3d(stride!=1, transpose=True)
  MinkowskiEngine: MinkowskiConvolutionTranspose

spconv.SparseConvTranspose3d:
  torchsparse: N/A
  MinkowskiEngine: MinkowskiConvolutionTranspose

spconv.SparseMaxPool3d:
  torchsparse: N/A
  MinkowskiEngine: MinkowskiMaxPooling
```

--------------------------------

### spconv.pytorch Input APIs Reference

Source: https://github.com/traveller59/spconv/blob/master/docs/USAGE.md

Documentation for input processing APIs in spconv.pytorch, including `PointToVoxel` for converting point clouds to voxels.

```APIDOC
PointToVoxel:
  Usage: point cloud to voxels
```

--------------------------------

### spconv.pytorch Miscellaneous APIs Reference

Source: https://github.com/traveller59/spconv/blob/master/docs/USAGE.md

Documentation for miscellaneous utility APIs in spconv.pytorch, such as `HashTable` for one-slot hash table operations.

```APIDOC
HashTable:
  Usage: hash table, one-slot
```

--------------------------------

### spconv.pytorch Layer APIs Reference

Source: https://github.com/traveller59/spconv/blob/master/docs/USAGE.md

Comprehensive documentation for spconv.pytorch's sparse convolution layers, detailing their common usage, equivalent dense versions, and important notes regarding `indice_key` for data saving and reuse.

```APIDOC
spconv.SparseConv3d:
  Usage: Downsample
  Dense Version: nn.Conv3d
  Note: Use indice_key to save data for inverse

spconv.SubMConv3d:
  Usage: Convolution
  Dense Version: N/A
  Note: Use indice_key to save data for reuse

spconv.SparseInverseConv3d:
  Usage: Upsample
  Dense Version: N/A
  Note: Use pre-saved indice_key to upsample

spconv.SparseConvTranspose3d:
  Usage: Upsample (for generative model)
  Dense Version: nn.ConvTranspose3d
  Note: VERY SLOW and CAN'T RECOVER ORIGIN POINT CLOUD

spconv.SparseMaxPool3d:
  Usage: Downsample
  Dense Version: nn.MaxPool3d
  Note: Use indice_key to save data for inverse

spconv.SparseSequential:
  Usage: Container
  Dense Version: nn.Sequential
  Note: support layers above and nn.ReLU, nn.BatchNorm, ...

spconv.SparseGlobalMaxPool:
  Usage: global pool
  Dense Version: N/A
  Note: return dense tensor instead of SparseConvTensor

spconv.SparseGlobalAvgPool:
  Usage: global pool
  Dense Version: N/A
  Note: return dense tensor instead of SparseConvTensor
```

--------------------------------

### Run Basic spconv Network Benchmark (120k voxels)

Source: https://github.com/traveller59/spconv/blob/master/docs/BENCHMARK.md

This snippet provides commands to execute the spconv benchmark for a basic network configuration, processing 120,000 voxels. Users can specify either F16 or TF32 precision to measure forward and backward pass performance on their GPU. Ensure other applications are closed for accurate results.

```python
python -m spconv.benchmark bench_basic f16
```

```python
python -m spconv.benchmark bench_basic tf32
```

--------------------------------

### Configure Spconv Project with CMake and CUDA

Source: https://github.com/traveller59/spconv/blob/master/example/libspconv/CMakeLists.txt

This CMake configuration sets up the Spconv project, defines its version, and specifies C++ and CUDA as primary languages. It includes external dependencies like CUMM and Spconv, and links them to a CUDA executable named 'main'. It also demonstrates how to manage include paths and library linking for a CUDA application.

```CMake
cmake_minimum_required(VERSION 3.18 FATAL_ERROR)
project(SpconvExample LANGUAGES CXX CUDA VERSION 0.1)

set(CUMM_DISABLE_CMAKE_INSTALL ON CACHE BOOL "enable X functionality" FORCE)

add_subdirectory(cumm)
add_subdirectory(spconv)

add_executable(main main.cu)
# SPCONV2_INCLUDE_PATH come from spconv/CMakeLists.txt
target_include_directories(main PRIVATE ${SPCONV2_INCLUDE_PATH})
target_link_libraries(main spconv cumm::cumm)
```

--------------------------------

### Build libspconv shared library

Source: https://github.com/traveller59/spconv/blob/master/example/libspconv/README.md

Executes the `run_build.sh` script to compile and link the `libspconv.so` shared library. This library is the C++ core of the spconv python package and is essential for its functionality.

```Shell
run_build.sh
```

--------------------------------

### Creating and Converting SparseConvTensor in Python

Source: https://github.com/traveller59/spconv/blob/master/docs/USAGE.md

This snippet demonstrates how to initialize a `SparseConvTensor` using features, indices, spatial shape, and batch size. It also shows how to convert the sparse tensor to a dense NCHW tensor using the `dense()` method, which is useful for integrating sparse outputs with dense network components.

```Python
import spconv.pytorch as spconv
features = # your features with shape [N, num_channels]
indices = # your indices/coordinates with shape [N, ndim + 1], batch index must be put in indices[:, 0]
spatial_shape = # spatial shape of your sparse tensor, spatial_shape[i] is shape of indices[:, 1 + i].
batch_size = # batch size of your sparse tensor.
x = spconv.SparseConvTensor(features, indices, spatial_shape, batch_size)
x_dense_NCHW = x.dense() # convert sparse tensor to dense NCHW tensor.
```

--------------------------------

### Generate Spconv C++ Code Including Training Operations

Source: https://github.com/traveller59/spconv/blob/master/docs/PURE_CPP_BUILD.md

Runs the spconv code generation script to produce C++ sources that include both inference and training operations. This command requires specifying the include and source directories of spconv without the 'inference_only' flag.

```Bash
python -m spconv.gencode --include=/path/to/spconv/include --src=/path/to/spconv/src
```

--------------------------------

### Converting Point Clouds to Voxels using spconv.pytorch.utils.PointToVoxel

Source: https://github.com/traveller59/spconv/blob/master/docs/USAGE.md

This snippet demonstrates how to use the PointToVoxel utility from spconv.pytorch.utils to convert raw point cloud data into a voxelized representation. It specifies the voxel grid parameters, including voxel size, coordinate range, number of point features, and maximum number of voxels/points per voxel. It's important to note that the generated indices are in ZYX order, while input parameters are XYZ, and the batch axis needs to be added manually.

```Python
from spconv.pytorch.utils import PointToVoxel, gather_features_by_pc_voxel_id
# this generator generate ZYX indices.
gen = PointToVoxel(
    vsize_xyz=[0.1, 0.1, 0.1], 
    coors_range_xyz=[-80, -80, -2, 80, 80, 6], 
    num_point_features=3, 
    max_num_voxels=5000, 
    max_num_points_per_voxel=5)
pc = np.random.uniform(-10, 10, size=[1000, 3])
pc_th = torch.from_numpy(pc)
voxels, coords, num_points_per_voxel = gen(pc_th, empty_mean=True)
```

--------------------------------

### Import Spconv in Version 2.x

Source: https://github.com/traveller59/spconv/blob/master/README.md

In Spconv 2.x, the library should be imported specifically from `spconv.pytorch` to ensure proper functionality.

```Python
import spconv.pytorch as spconv
```

--------------------------------

### spconv.pytorch Functional APIs Reference

Source: https://github.com/traveller59/spconv/blob/master/docs/USAGE.md

Documentation for functional APIs within spconv.pytorch, specifically `Fsp.sparse_add` for adding sparse tensors with different indices but same shape.

```APIDOC
Fsp.sparse_add:
  Usage: Add sparse tensors with same shape and different indices
```

--------------------------------

### Run Large spconv Network Benchmark (900k voxels)

Source: https://github.com/traveller59/spconv/blob/master/docs/BENCHMARK.md

This snippet provides commands to execute the spconv benchmark for a large network configuration, processing 900,000 voxels. Users can specify either F16 or TF32 precision to measure forward and backward pass performance on their GPU. It's recommended to close other applications for optimal benchmark accuracy.

```python
python -m spconv.benchmark bench_large f16
```

```python
python -m spconv.benchmark bench_large tf32
```

--------------------------------

### Set CUDA Architecture for NVIDIA Embedded Platforms

Source: https://github.com/traveller59/spconv/blob/master/README.md

Before building Spconv from source on NVIDIA embedded platforms like Xavier, TX2, or Orin, you need to export the specific CUDA compute capability (architecture) for your device. This ensures the correct kernels are compiled.

```Shell
export CUMM_CUDA_ARCH_LIST="7.2"
```

```Shell
export CUMM_CUDA_ARCH_LIST="6.2"
```

```Shell
export CUMM_CUDA_ARCH_LIST="8.7"
```

--------------------------------

### Cite spconv Project in Research

Source: https://github.com/traveller59/spconv/blob/master/README.md

BibTeX entry for academic citation of the spconv library project in research papers or publications.

```LaTeX
@misc{spconv2022,
    title={Spconv: Spatially Sparse Convolution Library},
    author={Spconv Contributors},
    howpublished = {\url{https://github.com/traveller59/spconv}},
    year={2022}
}
```

--------------------------------

### spconv.pytorch Core Concepts Definitions

Source: https://github.com/traveller59/spconv/blob/master/docs/USAGE.md

Definitions of fundamental concepts in spconv.pytorch, including Sparse Conv Tensor, Sparse Convolution, and Submanifold Convolution, explaining their characteristics and operational principles.

```APIDOC
Sparse Conv Tensor:
  Description: Like hybrid torch.sparse_coo_tensor with two differences:
    1. Only one dense dimension.
    2. Indice is transposed.
  See: torch doc for more details.

Sparse Convolution:
  Description: Equivalent to performing dense convolution when converting SparseConvTensor to dense.
  Operation: Only runs calculation on valid data.

Submanifold Convolution (SubMConv):
  Description: Similar to Sparse Convolution, but indices remain the same.
  Operation: Copies the same spatial structure to output, iterates, gets input coordinates by convolution rule, and applies convolution ONLY in these output coordinates.
```

--------------------------------

### Generate Spconv C++ Code for Inference Only

Source: https://github.com/traveller59/spconv/blob/master/docs/PURE_CPP_BUILD.md

Executes the spconv code generation script to create C++ sources specifically for inference operations. This command requires specifying the include and source directories of spconv and setting the 'inference_only' flag to true.

```Bash
python -m spconv.gencode --include=/path/to/spconv/include --src=/path/to/spconv/src --inference_only=True
```

--------------------------------

### Enable TF32 Kernels for Spconv

Source: https://github.com/traveller59/spconv/blob/master/README.md

This snippet shows how to enable TF32 kernels in Spconv 2.x for faster FP32 training. TF32 kernels are disabled by default and can be activated by setting a constant.

```Python
import spconv as spconv_core; spconv_core.constants.SPCONV_ALLOW_TF32 = True
```

--------------------------------

### Using SparseInverseConv for Inverse Sparse Operations

Source: https://github.com/traveller59/spconv/blob/master/docs/USAGE.md

This snippet illustrates the application of `spconv.SparseInverseConv3d`, which is used for inverse sparse convolution operations. It highlights that `SparseInverseConv` is distinct from `SparseConvTranspose` and is typically employed in tasks like semantic segmentation where the output indices should match the input indices of a previous sparse convolution.

```Python
class ExampleNet(nn.Module):
    def __init__(self, shape):
        super().__init__()
        self.net = spconv.SparseSequential(
            spconv.SparseConv3d(32, 64, 3, 2, indice_key="cp0"),
            spconv.SparseInverseConv3d(64, 32, 3, indice_key="cp0"), # need provide kernel size to create weight
        )
        self.shape = shape

    def forward(self, features, coors, batch_size):
        coors = coors.int()
        x = spconv.SparseConvTensor(features, coors, self.shape, batch_size)
        return self.net(x)
```

--------------------------------

### Correcting Sparse Inverse Convolution Usage in spconv

Source: https://github.com/traveller59/spconv/blob/master/docs/USAGE.md

This snippet illustrates a common error when chaining SparseConv3d and SparseInverseConv3d layers in spconv. It demonstrates that if an intermediate SparseConv3d changes the spatial structure (e.g., by using a different indice_key), a corresponding SparseInverseConv3d is required to correctly inverse that specific operation before further inverse convolutions can be applied. The CorrectNet class shows the proper way to handle such scenarios to ensure successful reconstruction of features.

```Python
class WrongNet(nn.Module):
    def __init__(self, shape):
        super().__init__()
        self.Encoder = spconv.SparseConv3d(channels, channels, kernel_size=3, stride=2, indice_key="cp1",algo=algo)
        self.Sparse_Conv = spconv.SparseConv3d(channels, channels, kernel_size=3, stride=1,algo=algo)
        self.Decoder = spconv.SparseInverseConv3d(channels, channels, kernel_size=3, indice_key="cp1",algo=algo)   

    def forward(self, sparse_tensor):
        encoded = self.Encoder(sparse_tensor)
        s_conv = self.Sparse_Conv(encoded)
        return self.Decoder(s_conv).features

class CorrectNet(nn.Module):
    def __init__(self, shape):
        super().__init__()
        self.Encoder = spconv.SparseConv3d(channels, channels, kernel_size=3, stride=2, indice_key="cp1",algo=algo)
        self.Sparse_Conv = spconv.SparseConv3d(channels, channels, kernel_size=3, stride=1, indice_key="cp2",algo=algo)
        self.Sparse_Conv_Decoder = spconv.SparseInverseConv3d(channels, channels, kernel_size=3, indice_key="cp2",algo=algo)   
        self.Decoder = spconv.SparseInverseConv3d(channels, channels, kernel_size=3, indice_key="cp1",algo=algo)   

    def forward(self, sparse_tensor):
        encoded = self.Encoder(sparse_tensor)
        s_conv = self.Sparse_Conv(encoded)
        return self.Decoder(self.Sparse_Conv_Decoder(s_conv)).features
```

--------------------------------

### Updating Spconv Module Import Paths

Source: https://github.com/traveller59/spconv/blob/master/docs/SPCONV_2_BREAKING_CHANGEs.md

In Spconv 2.x, the core modules have moved from `spconv.xxx` to `spconv.pytorch.xxx`. This snippet shows how to update import statements to reflect the new module structure, ensuring compatibility with the 2.x API.

```Python
import spconv.pytorch as spconv
from spconv.pytorch.xxx import
```

--------------------------------

### Correcting Feature Manipulation Syntax in Spconv 2.x

Source: https://github.com/traveller59/spconv/blob/master/docs/SPCONV_2_BREAKING_CHANGEs.md

Spconv 2.x no longer allows direct assignment to `x.features` after operations like `F.relu`. Instead, users must use the `replace_feature` method to update the sparse tensor's features, which ensures proper internal state management.

```Python
# Old (raises error in Spconv 2.x)
x.features = F.relu(x.features)

# New (correct way in Spconv 2.x)
x = x.replace_feature(F.relu(x.features))
```