### Run DBCSR Example via SLURM

Source: https://github.com/cp2k/dbcsr/blob/develop/docs/guide/2-user-guide/3-examples/index.md

Example command to execute a compiled DBCSR example binary using the SLURM workload manager with MPI.

```bash
srun -N 1 --ntasks-per-core 2 --ntasks-per-node 12 --cpus-per-task 2 ./examples/dbcsr_example_1
```

--------------------------------

### Install and Configure Pre-commit Hooks

Source: https://github.com/cp2k/dbcsr/wiki/Development

Instructions for installing the pre-commit framework and initializing the hooks in the local repository. These hooks ensure code quality by running checks before every commit.

```bash
pip install pre-commit
pre-commit install
pre-commit run --all-files
```

--------------------------------

### Reserve and Populate Sparse Matrix Blocks in C

Source: https://context7.com/cp2k/dbcsr/llms.txt

This example shows the lifecycle of a DBCSR matrix, including distribution setup, block reservation based on MPI rank, and filling blocks with data using the iterator interface. It requires an MPI-enabled environment and the DBCSR C API.

```cpp
#include <mpi.h>
#include <dbcsr.h>
#include <vector>
#include <algorithm>
#include <cstdlib>

int main(int argc, char* argv[]) {
    MPI_Init(&argc, &argv);

    int mpi_rank, mpi_size;
    MPI_Comm_size(MPI_COMM_WORLD, &mpi_size);
    MPI_Comm_rank(MPI_COMM_WORLD, &mpi_rank);

    int dims[2] = {0, 0};
    MPI_Dims_create(mpi_size, 2, dims);
    int periods[2] = {1, 1};
    MPI_Comm group;
    MPI_Cart_create(MPI_COMM_WORLD, 2, dims, periods, 0, &group);

    c_dbcsr_init_lib(MPI_COMM_WORLD, nullptr);

    std::vector<int> row_blk_sizes = {2, 3, 5, 2};
    std::vector<int> col_blk_sizes = {3, 3, 4, 6, 2};

    std::vector<int> row_dist(4), col_dist(5);
    for (int i = 0; i < 4; i++) row_dist[i] = i % dims[0];
    for (int i = 0; i < 5; i++) col_dist[i] = i % dims[1];

    dbcsr_distribution dist = nullptr;
    c_dbcsr_distribution_new(&dist, group,
                             row_dist.data(), row_dist.size(),
                             col_dist.data(), col_dist.size());

    dbcsr_matrix matrix = nullptr;
    c_dbcsr_create_new(&matrix, "sparse matrix", dist, dbcsr_type_no_symmetry,
                       row_blk_sizes.data(), row_blk_sizes.size(),
                       col_blk_sizes.data(), col_blk_sizes.size(),
                       nullptr, nullptr, nullptr, nullptr, nullptr, nullptr);

    std::vector<int> all_rows = {0, 0, 0, 1, 1, 2, 2, 3, 3};
    std::vector<int> all_cols = {0, 1, 2, 0, 2, 1, 3, 0, 4};

    std::vector<int> local_rows, local_cols;
    for (size_t i = 0; i < all_rows.size(); ++i) {
        int proc = -1;
        c_dbcsr_get_stored_coordinates(matrix, all_rows[i], all_cols[i], &proc);
        if (proc == mpi_rank) {
            local_rows.push_back(all_rows[i]);
            local_cols.push_back(all_cols[i]);
        }
    }

    c_dbcsr_reserve_blocks(matrix, local_rows.data(), local_cols.data(), local_rows.size());

    dbcsr_iterator iter = nullptr;
    c_dbcsr_iterator_start(&iter, matrix, nullptr, nullptr, nullptr, nullptr, nullptr);

    while (c_dbcsr_iterator_blocks_left(iter)) {
        int row, col, nblk, rsize, csize, roff, coff;
        bool tr;
        double* blk = nullptr;

        c_dbcsr_iterator_next_2d_block_d(iter, &row, &col, &blk, &tr,
                                          &nblk, &rsize, &csize, &roff, &coff);

        std::generate(blk, blk + rsize * csize,
                      []() { return static_cast<double>(rand()) / RAND_MAX; });
    }

    c_dbcsr_iterator_stop(&iter);
    c_dbcsr_finalize(matrix);

    c_dbcsr_print(matrix);

    c_dbcsr_release(&matrix);
    c_dbcsr_distribution_release(&dist);
    MPI_Comm_free(&group);
    c_dbcsr_finalize_lib();
    MPI_Finalize();
    return 0;
}
```

--------------------------------

### Create DBCSR Matrix

Source: https://github.com/cp2k/dbcsr/blob/develop/docs/guide/2-user-guide/3-examples/index.md

Demonstrates the initialization and creation of a DBCSR matrix structure. This example is implemented in Fortran.

```fortran
call dbcsr_example_1()
```

--------------------------------

### Define Installation Rules for DBCSR

Source: https://github.com/cp2k/dbcsr/blob/develop/src/CMakeLists.txt

This snippet defines the installation paths and components for the DBCSR library, including headers, Fortran modules, and CMake package configuration files.

```cmake
install(TARGETS dbcsr EXPORT DBCSRTargets LIBRARY DESTINATION "${CMAKE_INSTALL_LIBDIR}" ARCHIVE DESTINATION "${CMAKE_INSTALL_LIBDIR}")
install(FILES "${CMAKE_CURRENT_BINARY_DIR}/dbcsr_api.mod" DESTINATION "${CMAKE_INSTALL_Fortran_MODULES}")
configure_package_config_file(cmake/DBCSRConfig.cmake.in "${CMAKE_CURRENT_BINARY_DIR}/DBCSRConfig.cmake" INSTALL_DESTINATION "${config_install_dir}")
```

--------------------------------

### Navigate to Tuning Directory

Source: https://github.com/cp2k/dbcsr/blob/develop/src/acc/libsmm_acc/tune/README.md

Changes the current directory to the `libsmm_acc/tune` directory, which is the starting point for the auto-tuning procedure.

```bash
cd dbcsr/src/acc/libsmm_acc/tune
```

--------------------------------

### DBCSR API Usage Example (Fortran)

Source: https://github.com/cp2k/dbcsr/blob/develop/CONTRIBUTING.md

Illustrates how external packages should use the DBCSR API. It highlights the importance of using the public API for stability and major release considerations. This example assumes the API is defined in `dbcsr_api.F`.

```Fortran
PROGRAM main
  USE dbcsr_api, ONLY: initialize, finalize, create_matrix, set_value, get_value
  IMPLICIT NONE

  CALL initialize()

  ! Example: Create a matrix and set a value
  TYPE(matrix_type) :: A
  CALL create_matrix(A, rows=10, cols=10, block_size=1)
  CALL set_value(A, row=1, col=2, value=3.14)

  ! Example: Get a value from the matrix
  REAL :: val
  CALL get_value(A, row=1, col=2, value=val)
  PRINT *, "Value at (1,2):", val

  ! Clean up
  CALL finalize()
END PROGRAM main
```

--------------------------------

### Run Ubuntu Build Environment Container

Source: https://github.com/cp2k/dbcsr/blob/develop/tools/docker/README.md

Starts an interactive Ubuntu 22.04 container with the necessary build tools mounted to the current directory. It maps the local directory to /app and executes a build sequence using CMake and Ninja.

```console
cd dbcsr
docker run --rm -it -v $PWD:/app --workdir /app --user $(id -u):$(id -g) ghcr.io/cp2k/dbcsr-build-env-ubuntu-22.04 /bin/bash
mkdir build && cd build/
cmake -G Ninja ..
cmake --build .
```

--------------------------------

### Install DBCSR using CMake

Source: https://github.com/cp2k/dbcsr/blob/develop/docs/guide/2-user-guide/1-installation/3-using-dbcsr-in-a-cmake-project.md

Commands to install DBCSR using CMake. It shows how to install to the default location (/usr/local) using sudo or to a custom prefix using CMAKE_INSTALL_PREFIX.

```bash
sudo cmake --build . --install  # will install to /usr/local
```

```bash
cmake -DCMAKE_INSTALL_PREFIX=/my/custom/prefix ..
cmake --build . -- install
```

--------------------------------

### Install Python Dependencies

Source: https://github.com/cp2k/dbcsr/blob/develop/src/acc/libsmm_acc/tune/README.md

Installs all necessary Python packages for the auto-tuning procedure using pip. It is recommended to use a virtual environment to avoid conflicts with other projects.

```bash
pip install -r requirements.txt
```

--------------------------------

### Set DBCSR Matrix Values

Source: https://github.com/cp2k/dbcsr/blob/develop/docs/guide/2-user-guide/3-examples/index.md

Shows how to populate a previously created DBCSR matrix with data. This example is implemented in Fortran.

```fortran
call dbcsr_example_2()
```

--------------------------------

### DBCSR Preprocessor Flag Convention (Fortran)

Source: https://github.com/cp2k/dbcsr/blob/develop/CONTRIBUTING.md

Shows the convention for defining and documenting preprocessor flags in DBCSR, which must start with two underscores and be documented in the project's guide.

```Fortran
#define __DBCSR_ENABLE_DEBUG__ 1

#if __DBCSR_ENABLE_DEBUG__ == 1
  ! Debug specific code
  PRINT *, "Debug mode is enabled."
#endif

#include "base/dbcsr_base_uses.f90"
```

--------------------------------

### Create and Contract Multi-dimensional Tensors

Source: https://context7.com/cp2k/dbcsr/llms.txt

Illustrates the setup of a 3D process grid, distribution vectors, and the creation of tensors. It concludes with a tensor contraction operation using the DBCSR Tensor API.

```fortran
PROGRAM tensor_example
   USE mpi
   USE dbcsr_api, ONLY: dbcsr_init_lib, dbcsr_finalize_lib, dbcsr_type_real_8, dbcsr_scalar
   USE dbcsr_tensor_api, ONLY: &
      dbcsr_t_create, dbcsr_t_destroy, dbcsr_t_contract, &
      dbcsr_t_pgrid_create, dbcsr_t_pgrid_destroy, dbcsr_t_pgrid_type, &
      dbcsr_t_distribution_new, dbcsr_t_distribution_destroy, &
      dbcsr_t_distribution_type, dbcsr_t_type, dbcsr_t_default_distvec, &
      dbcsr_t_reserve_blocks, dbcsr_t_put_block, dbcsr_t_filter, dbcsr_t_copy
   IMPLICIT NONE

   TYPE(dbcsr_t_pgrid_type) :: pgrid_3d
   TYPE(dbcsr_t_distribution_type) :: dist_tensor
   TYPE(dbcsr_t_type) :: tensor_a, tensor_b, tensor_c
   INTEGER :: ierr, numnodes
   INTEGER, DIMENSION(3) :: pdims_3d, shape_3d
   INTEGER, DIMENSION(:), ALLOCATABLE :: blk_size_i, blk_size_j, blk_size_k
   INTEGER, DIMENSION(:), ALLOCATABLE :: dist_1, dist_2, dist_3
   INTEGER(KIND=8) :: nflop
   REAL(KIND=8), PARAMETER :: filter_eps = 1.0D-10

   CALL mpi_init(ierr)
   CALL mpi_comm_size(MPI_COMM_WORLD, numnodes, ierr)
   CALL dbcsr_init_lib(MPI_COMM_WORLD)

   shape_3d = [10, 10, 10]
   ALLOCATE(blk_size_i(shape_3d(1)), blk_size_j(shape_3d(2)), blk_size_k(shape_3d(3)))
   blk_size_i(:) = 5
   blk_size_j(:) = 5
   blk_size_k(:) = 5

   pdims_3d(:) = 0
   CALL dbcsr_t_pgrid_create(MPI_COMM_WORLD, pdims_3d, pgrid_3d)

   ALLOCATE(dist_1(shape_3d(1)), dist_2(shape_3d(2)), dist_3(shape_3d(3)))
   CALL dbcsr_t_default_distvec(shape_3d(1), pdims_3d(1), blk_size_i, dist_1)
   CALL dbcsr_t_default_distvec(shape_3d(2), pdims_3d(2), blk_size_j, dist_2)
   CALL dbcsr_t_default_distvec(shape_3d(3), pdims_3d(3), blk_size_k, dist_3)

   CALL dbcsr_t_distribution_new(dist_tensor, pgrid_3d, dist_1, dist_2, dist_3)

   CALL dbcsr_t_create(tensor_a, "A[ij|k]", dist_tensor, &
                       map1_2d=[1, 2], map2_2d=[3], &
                       data_type=dbcsr_type_real_8, &
                       blk_size_1=blk_size_i, blk_size_2=blk_size_j, blk_size_3=blk_size_k)

   CALL dbcsr_t_create(tensor_b, "B[ij|k]", dist_tensor, &
                       map1_2d=[1, 2], map2_2d=[3], &
                       data_type=dbcsr_type_real_8, &
                       blk_size_1=blk_size_i, blk_size_2=blk_size_j, blk_size_k=blk_size_k)

   CALL dbcsr_t_create(tensor_c, "C[ij|lm]", dist_tensor, &
                       map1_2d=[1, 2], map2_2d=[3], &
                       data_type=dbcsr_type_real_8, &
                       blk_size_1=blk_size_i, blk_size_2=blk_size_j, blk_size_3=blk_size_k)

   CALL dbcsr_t_contract(alpha=dbcsr_scalar(1.0D0), tensor_1=tensor_a, tensor_2=tensor_b, &
                         beta=dbcsr_scalar(0.0D0), tensor_3=tensor_c, &
                         contract_1=[3], notcontract_1=[1, 2], &
                         contract_2=[3], notcontract_2=[1, 2], &
                         map_1=[1, 2], map_2=[3], &
                         filter_eps=filter_eps, flop=nflop)

   CALL dbcsr_t_destroy(tensor_a)
   CALL dbcsr_t_destroy(tensor_b)
   CALL dbcsr_t_destroy(tensor_c)
   CALL dbcsr_t_distribution_destroy(dist_tensor)
   CALL dbcsr_t_pgrid_destroy(pgrid_3d)

   DEALLOCATE(blk_size_i, blk_size_j, blk_size_k, dist_1, dist_2, dist_3)

   CALL dbcsr_finalize_lib()
   CALL mpi_finalize(ierr)
END PROGRAM tensor_example
```

--------------------------------

### DBCSR Error Handling Example (Fortran)

Source: https://github.com/cp2k/dbcsr/blob/develop/CONTRIBUTING.md

Demonstrates the preferred method for handling errors and warnings in DBCSR using predefined macros like DBCSR_WARN and DBCSR_ABORT, instead of using STOP or direct MPI calls.

```Fortran
SUBROUTINE process_data(data)
  USE dbcsr_api, ONLY: DBCSR_WARN, DBCSR_ABORT
  IMPLICIT NONE

  REAL, DIMENSION(:), INTENT(IN) :: data

  IF (SIZE(data) == 0) THEN
    CALL DBCSR_WARN("Input data array is empty.")
    RETURN
  END IF

  IF (any(data < 0.0))
    CALL DBCSR_ABORT("Negative values found in data array.")
  END IF

  ! ... process data ...

END SUBROUTINE process_data
```

--------------------------------

### Create DBCSR Matrix (C/C++ API)

Source: https://context7.com/cp2k/dbcsr/llms.txt

Creates a new DBCSR sparse matrix using the C/C++ API, providing fine-grained control over the block structure. This example initializes the MPI environment and DBCSR library, defines block sizes and distribution using C++ vectors, creates the matrix, and includes placeholders for matrix usage and cleanup.

```cpp
#include <mpi.h>
#include <dbcsr.h>
#include <vector>

int main(int argc, char* argv[]) {
    MPI_Init(&argc, &argv);

    int mpi_size;
    MPI_Comm_size(MPI_COMM_WORLD, &mpi_size);

    int dims[2] = {0, 0};
    MPI_Dims_create(mpi_size, 2, dims);
    int periods[2] = {1, 1};
    MPI_Comm group;
    MPI_Cart_create(MPI_COMM_WORLD, 2, dims, periods, 0, &group);

    c_dbcsr_init_lib(MPI_COMM_WORLD, nullptr);

    // Block sizes: 4 row blocks, 5 column blocks
    std::vector<int> row_blk_sizes = {2, 3, 5, 2};  // Total: 12 rows
    std::vector<int> col_blk_sizes = {3, 3, 4, 6, 2};  // Total: 18 columns

    // Create distribution
    std::vector<int> row_dist(4), col_dist(5);
    for (int i = 0; i < 4; i++) row_dist[i] = i % dims[0];
    for (int i = 0; i < 5; i++) col_dist[i] = i % dims[1];

    dbcsr_distribution dist = nullptr;
    c_dbcsr_distribution_new(&dist, group,
                             row_dist.data(), row_dist.size(),
                             col_dist.data(), col_dist.size());

    // Create matrix
    dbcsr_matrix matrix_a = nullptr;
    c_dbcsr_create_new(&matrix_a,
                       "matrix a",           // name
                       dist,                 // distribution
                       dbcsr_type_no_symmetry,  // symmetry type
                       row_blk_sizes.data(), row_blk_sizes.size(),
                       col_blk_sizes.data(), col_blk_sizes.size(),
                       nullptr,              // nze (optional)
                       nullptr,              // data_type (default: real_8)
                       nullptr, nullptr, nullptr, nullptr);  // optional params

    c_dbcsr_finalize(matrix_a);

    // ... use matrix ...

    c_dbcsr_release(&matrix_a);
    c_dbcsr_distribution_release(&dist);

    MPI_Comm_free(&group);
    c_dbcsr_finalize_lib();
    MPI_Finalize();
    return 0;
}

```

--------------------------------

### Run tune_setup.py for Auto-tuning

Source: https://github.com/cp2k/dbcsr/blob/develop/src/acc/libsmm_acc/tune/README.md

Executes the `tune_setup.py` script to initiate the auto-tuning process. Users specify the target GPU parameters file using `-p` and the compiler (`nvcc` or `hipcc`) using `-b`. The script also accepts block sizes as arguments or can read them from another parameter file.

```bash
$ ./tune_setup.py 5 8 -p ../parameters/parameters_P100.json
```

```bash
$ ./tune_setup.py -p ../parameters/parameters_P100.json ../parameters/parameters_K40.json
```

--------------------------------

### Execute Benchmark with Auto-tuned Kernels

Source: https://github.com/cp2k/dbcsr/blob/develop/src/acc/opencl/smm/README-bulktune.md

Run the accelerator benchmark utility to exercise the auto-tuned kernels.

```bash
cd src/acc
./acc_bench 5 30000 13 5 7
```

--------------------------------

### Configure DBCSR with Intel MPI, Intel Compiler, and Intel MKL

Source: https://github.com/cp2k/dbcsr/blob/develop/docs/guide/2-user-guide/1-installation/1-cmake-build-recipes.md

Full Intel stack configuration using Intel MPI, Intel compilers, and Intel MKL.

```bash
source /sw/intel/bin/compilervars.sh intel64
CC=mpiicc FC=mpiifort CXX=mpiicxx cmake -DBLA_VENDOR=Intel10_64lp_seq ..
```

--------------------------------

### Set DBCSR CMake Configuration Path

Source: https://github.com/cp2k/dbcsr/blob/develop/docs/guide/2-user-guide/1-installation/3-using-dbcsr-in-a-cmake-project.md

Command to set the DBCSR_DIR environment variable when calling CMake, which is necessary if DBCSR was installed to a custom prefix.

```bash
DBCSR_DIR=/my/custom/prefix cmake ..
```

--------------------------------

### Build DBCSR Documentation with CMake

Source: https://github.com/cp2k/dbcsr/blob/develop/docs/guide/3-developer-guide/2-documentation/index.md

This snippet shows the CMake commands to configure and build the DBCSR documentation. It requires the FORD tool and highlights the importance of enabling the WITH_EXAMPLES option for comprehensive documentation. The generated HTML is found in the 'doc/' directory.

```bash
mkdir build
cd build
cmake ..
make doc
```

```bash
cmake -DUSE_MPI=ON -DWITH_EXAMPLES=ON ..
# these options are default and recommended.
# If set off, the examples' documentation is not generated.
```

--------------------------------

### Update and Re-tune Existing Parameters

Source: https://github.com/cp2k/dbcsr/blob/develop/src/acc/opencl/smm/README-bulktune.md

Workflow for rebuilding the binary with specific GPU parameters and re-tuning existing JSON files.

```bash
cd src/acc/opencl
make realclean
echo "Rebuild and embed smm/params/tune_multiply_P100.csv"
make WITH_GPU=P100
echo "Retune original parameters"
smm/tune_multiply.sh -p smm/params/p100 -u
echo "Override original parameters"
cp tune_multiply.csv smm/params/tune_multiply_P100.csv
```

--------------------------------

### Find DBCSR in CMake Project

Source: https://github.com/cp2k/dbcsr/blob/develop/docs/guide/2-user-guide/1-installation/3-using-dbcsr-in-a-cmake-project.md

CMake code to find the installed DBCSR library and link it to executables. It demonstrates how to find DBCSR, MPI, and link them to Fortran, C, and C++ targets.

```cmake
cmake_minimum_required(VERSION 3.22)

enable_language(Fortran C CXX)  # only request the required language

find_package(DBCSR 2.0.0 CONFIG REQUIRED)
find_package(MPI)

# for Fortran:
set(CMAKE_Fortran_FLAGS "-std=f2018")  # your Fortran code likely needs to be F2018+ compatible as well
add_executable(dbcsr_example_fortran dbcsr_example.f90)
target_link_libraries(dbcsr_example_fortran DBCSR::dbcsr)

# for C:
add_executable(dbcsr_example_c dbcsr_example.c)
target_link_libraries(dbcsr_example_c DBCSR::dbcsr_c MPI::MPI_C)

# for C++:
add_executable(dbcsr_example_cpp dbcsr_example.cpp)
target_link_libraries(dbcsr_example_cpp DBCSR::dbcsr_c MPI::MPI_CXX)
```

--------------------------------

### Build Ubuntu Environment Image

Source: https://github.com/cp2k/dbcsr/blob/develop/tools/docker/README.md

Builds the local Docker image for the Ubuntu build environment using the provided Dockerfile. This command must be executed from the tools/docker directory within the project.

```console
cd dbcsr/tools/docker
docker build -t dbcsr-build-env-ubuntu-22.04 -f Dockerfile.build-env-ubuntu .
```

--------------------------------

### Configure DBCSR with GNU, System MPI, and Intel MKL

Source: https://github.com/cp2k/dbcsr/blob/develop/docs/guide/2-user-guide/1-installation/1-cmake-build-recipes.md

Configures the build to use Intel MKL instead of system-provided BLAS libraries. Requires the MKL environment to be sourced prior to execution.

```bash
source /sw/intel/mkl/bin/mklvars.sh intel64
cmake -DBLA_VENDOR=Intel10_64lp_seq ..
```

--------------------------------

### Prepare Test Files for Documentation

Source: https://github.com/cp2k/dbcsr/blob/develop/tests/CMakeLists.txt

This snippet copies test source files into the build directory to ensure they are accessible for documentation generation tools like FORD. It includes dependency management to ensure generated files are ready before copying.

```cmake
set(DBCSR_TESTS dbcsr_performance_driver.F ${DBCSR_TESTS_SRCS_FTN} ${DBCSR_TESTS_SRCS_CPP} libsmm_acc_unittest_transpose.cpp)

set(test_copy_commands)
foreach (test ${DBCSR_TESTS})
  list(APPEND test_copy_commands COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_SOURCE_DIR}/tests/${test} ${CMAKE_BINARY_DIR}/tests)
endforeach ()

add_custom_target(
  doc_copy_tests
  COMMENT "Copy tests for documentation generation"
  COMMAND mkdir -p ${CMAKE_BINARY_DIR}/tests ${test_copy_commands}
  VERBATIM)

add_dependencies(doc_copy_tests generate_libsmm_acc_unittest_multiply_test_cpp)
add_dependencies(doc_copy_tests generate_libsmm_acc_timer_multiply_test_cpp)
```

--------------------------------

### Generate Job Files with tune_setup.py

Source: https://github.com/cp2k/dbcsr/blob/develop/src/acc/libsmm_acc/tune/README.md

This Python script generates job files for submitting auto-tuning tasks to a supercomputer. It requires customization for the target environment, including module loading and compiler selection (nvcc for CUDA or hipcc for HIP).

```python
import os

class TuneSetup:
    def __init__(self, compiler='nvcc', time='00:30:00', num_nodes=1, cpus_per_node=40):
        self.compiler = compiler
        self.time = time
        self.num_nodes = num_nodes
        self.cpus_per_node = cpus_per_node

    def gen_jobfile(self, outdir, m, n, k):
        output = "#!/bin/bash -l\n"
        output += "#SBATCH --nodes=%d\n" % self.num_nodes
        output += "#SBATCH --ntasks-per-core=1\n"
        output += "#SBATCH --ntasks-per-node=1\n"
        output += "#SBATCH --cpus-per-task=%d\n" % self.cpus_per_node
        output += "#SBATCH --time=%s\n" % self.time
        output += "#SBATCH --partition=normal\n"
        output += "#SBATCH --constraint=gpu\n"
        output += "\n"
        output += "source ${MODULESHOME}/init/sh;\n"
        output += "module load daint-gpu\n"
        output += "module unload PrgEnv-cray\n"
        output += "module load PrgEnv-gnu\n"
        if self.compiler == "nvcc":
            output += "module load cudatoolkit/8.0.61_2.4.9-6.0.7.0_17.1__g899857c\n"
        else: # i.e. compiler = hipcc
            output += "module load hip\n"
        output += "module list\n"
        output += "export CRAY_CUDA_MPS=1\n"
        output += "cd $SLURM_SUBMIT_DIR \n"
        output += "\n"
        output += "date\n"
        # ... (rest of the job file generation logic)
        return output

# Example usage:
tune_setup = TuneSetup(compiler='nvcc', time='01:00:00')
job_content = tune_setup.gen_jobfile('/path/to/output', 5, 8, 5)
print(job_content)
```

--------------------------------

### MPI-Accelerated Kernel Tuning with tune_multiply.py

Source: https://github.com/cp2k/dbcsr/blob/develop/src/acc/opencl/smm/README-bulktune.md

This script demonstrates how to use MPI for SPMD execution to tune multiple kernels concurrently across different devices. It assumes one process per device and allows for parallel tuning, which is more efficient than tuning the same kernels redundantly. Environment variables are preferred over command-line options for configuration.

```bash
MAXTIME=200 NPARTS=8 UPDATE=1 JSONDIR=params/pvc mpirun \
  ./tune_multiply.sh -i 1 : \
  ./tune_multiply.sh -i 2 : \
  ./tune_multiply.sh -i 3 : \
  ./tune_multiply.sh -i 4 : \
  ./tune_multiply.sh -i 5 : \
  ./tune_multiply.sh -i 6 : \
  ./tune_multiply.sh -i 7 : \
  ./tune_multiply.sh -i 8 \
>out.log 2>&1
```

--------------------------------

### Configure DBCSR with Intel MPI, GNU Compiler, and Intel MKL

Source: https://github.com/cp2k/dbcsr/blob/develop/docs/guide/2-user-guide/1-installation/1-cmake-build-recipes.md

Combines Intel MPI wrappers, GNU compilers, and Intel MKL for optimized performance.

```bash
source /sw/intel/bin/compilervars.sh intel64
CC=mpicc FC=mpifc CXX=mpicxx cmake -DBLA_VENDOR=Intel10_64lp_seq ..
```

--------------------------------

### Fortran Initialization: dbcsr_init_lib

Source: https://context7.com/cp2k/dbcsr/llms.txt

Initializes the DBCSR library within a Fortran application, requiring an MPI communicator.

```APIDOC
## dbcsr_init_lib (Fortran)

### Description
Initializes the DBCSR library. Must be called after MPI initialization and before any other DBCSR operations.

### Method
Subroutine Call

### Parameters
- **comm** (MPI_Comm) - Required - The MPI communicator to be used by the library.

### Request Example
CALL dbcsr_init_lib(MPI_COMM_WORLD)

### Response
- **status** (Integer) - Returns 0 on success, non-zero on failure.
```

--------------------------------

### Build DBCSR with CMake (Basic)

Source: https://context7.com/cp2k/dbcsr/llms.txt

This snippet shows the basic steps to clone the DBCSR repository and build it using CMake with system-provided compilers and libraries. It assumes MPI and OpenBLAS are available on the system.

```bash
git clone https://github.com/cp2k/dbcsr.git
cd dbcsr

# Create build directory
mkdir build && cd build

# Configure with CMake (uses system MPI and OpenBLAS)
cmake ..

# Build
make -j$(nproc)

# Run tests (optional)
make test
```

--------------------------------

### Copy Parameters File

Source: https://github.com/cp2k/dbcsr/blob/develop/src/acc/libsmm_acc/tune/README.md

Copies the `parameters.h` file from a build directory to the current directory. This file contains kernel multiplication data and their optimal parameters, which is required for the auto-tuning process.

```bash
cp ~/dbcsr/build_dir/src/acc/libsmm_acc/parameters.h ../
```

--------------------------------

### Compile Projects with DBCSR

Source: https://context7.com/cp2k/dbcsr/llms.txt

Shows the command-line steps to configure and build a project that depends on the DBCSR library using CMake.

```bash
mkdir build && cd build
cmake -DDBCSR_DIR=/path/to/dbcsr/install/lib/cmake/dbcsr ..
make
```

--------------------------------

### Perform Binary I/O for DBCSR Matrices

Source: https://context7.com/cp2k/dbcsr/llms.txt

Demonstrates how to save a DBCSR matrix to a binary file and subsequently load it back into memory. This is primarily used for checkpointing and restart functionality in scientific simulations.

```fortran
PROGRAM io_example
   USE dbcsr_api, ONLY: dbcsr_type, dbcsr_binary_write, dbcsr_binary_read, &
                        dbcsr_distribution_type
   IMPLICIT NONE

   TYPE(dbcsr_type) :: matrix_a, matrix_loaded
   TYPE(dbcsr_distribution_type) :: dist

   ! Save matrix to binary file
   CALL dbcsr_binary_write(matrix_a, "matrix_checkpoint.dbcsr")

   ! Later: load matrix from file
   CALL dbcsr_binary_read("matrix_checkpoint.dbcsr", dist, matrix_loaded)
END PROGRAM io_example
```

--------------------------------

### C/C++ Initialization: c_dbcsr_init_lib

Source: https://context7.com/cp2k/dbcsr/llms.txt

Initializes the DBCSR library for C/C++ applications, handling MPI communicator integration.

```APIDOC
## c_dbcsr_init_lib (C/C++)

### Description
Initializes the DBCSR library from C or C++ code. This function handles the conversion of the MPI communicator for internal library use.

### Method
Function Call

### Parameters
- **comm** (MPI_Comm) - Required - The MPI communicator.
- **io_unit** (void*) - Optional - Pointer to an I/O unit, pass nullptr for default.

### Request Example
c_dbcsr_init_lib(MPI_COMM_WORLD, nullptr);

### Response
- **status** (int) - Returns 0 on success, non-zero on failure.
```

--------------------------------

### Rebuild DBCSR with Optimized Kernels

Source: https://github.com/cp2k/dbcsr/blob/develop/src/acc/opencl/smm/README-bulktune.md

Clean and rebuild the project to incorporate auto-tuned kernel parameters into the binary.

```bash
cd src/acc/opencl
make realclean
make
```

--------------------------------

### Initialize DBCSR Library (Fortran API)

Source: https://context7.com/cp2k/dbcsr/llms.txt

This Fortran code snippet shows how to initialize the DBCSR library using the `dbcsr_init_lib` function. It must be called after MPI initialization and before any other DBCSR operations. The library is finalized with `dbcsr_finalize_lib`.

```fortran
PROGRAM init_example
   USE mpi
   USE dbcsr_api, ONLY: dbcsr_init_lib, dbcsr_finalize_lib
   IMPLICIT NONE

   INTEGER :: ierr

   ! Initialize MPI
   CALL mpi_init(ierr)

   ! Initialize DBCSR library with MPI communicator
   CALL dbcsr_init_lib(MPI_COMM_WORLD)

   ! ... perform DBCSR operations ...

   ! Finalize DBCSR library
   CALL dbcsr_finalize_lib()

   ! Finalize MPI
   CALL mpi_finalize(ierr)
END PROGRAM init_example
```

--------------------------------

### Initialize DBCSR Library (C API)

Source: https://context7.com/cp2k/dbcsr/llms.txt

This C code snippet demonstrates how to initialize the DBCSR library from C/C++ code using `c_dbcsr_init_lib`. It automatically handles the MPI communicator conversion. The library is finalized using `c_dbcsr_finalize_lib`.

```c
#include <mpi.h>
#include <dbcsr.h>

int main(int argc, char* argv[]) {
    MPI_Init(&argc, &argv);

    // Initialize DBCSR library
    // Second parameter is optional I/O unit (nullptr for default)
    c_dbcsr_init_lib(MPI_COMM_WORLD, nullptr);

    // ... perform DBCSR operations ...

    // Finalize DBCSR library
    c_dbcsr_finalize_lib();

    MPI_Finalize();
    return 0;
}
```

--------------------------------

### Fortran: DBCSR Sparse Matrix-Matrix Multiplication

Source: https://context7.com/cp2k/dbcsr/llms.txt

Demonstrates sparse matrix-matrix multiplication using the DBCSR Fortran API. It initializes the library, creates matrices, fills them with sample data, performs the multiplication, and prints the result. Dependencies include MPI and the DBCSR library.

```Fortran
PROGRAM multiply_example
   USE mpi
   USE dbcsr_api, ONLY: dbcsr_init_lib, dbcsr_finalize_lib, &
                        dbcsr_distribution_new, dbcsr_distribution_release, &
                        dbcsr_distribution_type, dbcsr_distribution_get, &
                        dbcsr_create, dbcsr_release, dbcsr_finalize, dbcsr_print, &
                        dbcsr_type, dbcsr_type_no_symmetry, dbcsr_type_real_8, &
                        dbcsr_put_block, dbcsr_get_stored_coordinates, &
                        dbcsr_nblkrows_total, dbcsr_nblkcols_total, dbcsr_multiply
   IMPLICIT NONE

   TYPE(dbcsr_type) :: matrix_a, matrix_b, matrix_c
   TYPE(dbcsr_distribution_type) :: dist
   INTEGER, DIMENSION(:), POINTER :: row_blk_sizes, col_blk_sizes
   INTEGER, DIMENSION(:), POINTER :: row_dist, col_dist
   INTEGER :: group, numnodes, mynode, ierr, row, col, nze
   INTEGER :: node_holds_blk, max_nze
   INTEGER, DIMENSION(2) :: npdims
   LOGICAL, DIMENSION(2) :: period = .TRUE.
   REAL(KIND=8), DIMENSION(:), ALLOCATABLE :: values

   CALL mpi_init(ierr)
   CALL mpi_comm_size(MPI_COMM_WORLD, numnodes, ierr)

   npdims(:) = 0
   CALL mpi_dims_create(numnodes, 2, npdims, ierr)
   CALL mpi_cart_create(MPI_COMM_WORLD, 2, npdims, period, .FALSE., group, ierr)

   CALL dbcsr_init_lib(MPI_COMM_WORLD)

   ! Create square 4x4 block matrices
   ALLOCATE(row_blk_sizes(4), col_blk_sizes(4))
   row_blk_sizes(:) = 2
   col_blk_sizes(:) = 2

   ALLOCATE(row_dist(4), col_dist(4))
   row_dist = [MODULO(0, npdims(1)), MODULO(1, npdims(1)), &
               MODULO(2, npdims(1)), MODULO(3, npdims(1))]
   col_dist = [MODULO(0, npdims(2)), MODULO(1, npdims(2)), &
               MODULO(2, npdims(2)), MODULO(3, npdims(2))]

   CALL dbcsr_distribution_new(dist, group=group, &
                               row_dist=row_dist, col_dist=col_dist, &
                               reuse_arrays=.TRUE.)

   ! Create matrices A, B, and C
   CALL dbcsr_create(matrix=matrix_a, name="Matrix A", dist=dist, &
                     matrix_type=dbcsr_type_no_symmetry, &
                     row_blk_size=row_blk_sizes, col_blk_size=col_blk_sizes, &
                     data_type=dbcsr_type_real_8)

   CALL dbcsr_create(matrix=matrix_b, name="Matrix B", dist=dist, &
                     matrix_type=dbcsr_type_no_symmetry, &
                     row_blk_size=row_blk_sizes, col_blk_size=col_blk_sizes, &
                     data_type=dbcsr_type_real_8)

   CALL dbcsr_create(matrix=matrix_c, name="Matrix C", dist=dist, &
                     matrix_type=dbcsr_type_no_symmetry, &
                     row_blk_size=row_blk_sizes, col_blk_size=col_blk_sizes, &
                     data_type=dbcsr_type_real_8)

   CALL dbcsr_distribution_get(dist, mynode=mynode)
   max_nze = MAXVAL(row_blk_sizes) * MAXVAL(col_blk_sizes)
   ALLOCATE(values(max_nze))

   ! Fill matrices A and B with tridiagonal pattern
   DO row = 1, dbcsr_nblkrows_total(matrix_a)
      DO col = MAX(row-1, 1), MIN(row+1, dbcsr_nblkcols_total(matrix_a))
         CALL dbcsr_get_stored_coordinates(matrix_a, row, col, node_holds_blk)
         IF (node_holds_blk == mynode) THEN
            nze = row_blk_sizes(row) * col_blk_sizes(col)
            CALL RANDOM_NUMBER(values(1:nze))
            CALL dbcsr_put_block(matrix_a, row, col, values(1:nze))
            CALL RANDOM_NUMBER(values(1:nze))
            CALL dbcsr_put_block(matrix_b, row, col, values(1:nze))
         END IF
      END DO
   END DO
   DEALLOCATE(values)

   CALL dbcsr_finalize(matrix_a)
   CALL dbcsr_finalize(matrix_b)
   CALL dbcsr_finalize(matrix_c)

   ! Perform multiplication: C = 1.0 * A * B + 0.0 * C
   ! 'N' = no transpose, 'T' = transpose, 'C' = conjugate transpose
   CALL dbcsr_multiply('N', 'N', 1.0D0, matrix_a, matrix_b, 0.0D0, matrix_c)

   ! Print result
   CALL dbcsr_print(matrix_c)

   ! Cleanup
   CALL dbcsr_release(matrix_a)
   CALL dbcsr_release(matrix_b)
   CALL dbcsr_release(matrix_c)
   CALL dbcsr_distribution_release(dist)
   CALL mpi_comm_free(group, ierr)
   CALL dbcsr_finalize_lib()
   CALL mpi_finalize(ierr)
END PROGRAM multiply_example
```

--------------------------------

### Perform Manual Release Procedure

Source: https://github.com/cp2k/dbcsr/wiki/Development

This sequence of commands demonstrates the manual process for creating a new release in the DBCSR repository. It involves creating a release branch, updating the version, merging into master, tagging, and syncing back to develop.

```bash
DBCSR_VERSION=X.Y.Z
git checkout -b release-${DBCSR_VERSION} develop
$EDITOR VERSION
git commit -m "Bump version to ${DBCSR_VERSION}" VERSION
git checkout master
git merge --no-ff release-${DBCSR_VERSION}
git tag -a -m "Version ${DBCSR_VERSION}" v${DBCSR_VERSION}
git checkout develop
git merge --no-ff master
git remote add upstream git@github.com:cp2k/dbcsr.git
git push --atomic --follow-tags upstream master develop
git branch -d release-${DBCSR_VERSION}
```

--------------------------------

### Build DBCSR C API Library

Source: https://github.com/cp2k/dbcsr/blob/develop/src/CMakeLists.txt

This snippet configures the build process for the C API, including generating sources with fypp, setting library properties, and linking necessary dependencies like MPI.

```cmake
if (WITH_C_API)
  add_fypp_sources(DBCSR_C_SRCS dbcsr.h dbcsr_api_c.F tensors/dbcsr_tensor_api_c.F tensors/dbcsr_tensor.h)
  add_library(dbcsr_c ${DBCSR_C_SRCS})
  set_target_properties(dbcsr_c PROPERTIES LINKER_LANGUAGE Fortran)
  target_link_libraries(dbcsr_c PRIVATE dbcsr)
  target_link_libraries(dbcsr_c PUBLIC MPI::MPI_C)
endif ()
```

--------------------------------

### Create DBCSR Matrix (Fortran API)

Source: https://context7.com/cp2k/dbcsr/llms.txt

Creates a new DBCSR sparse matrix using the Fortran API. It initializes the DBCSR library, defines matrix block sizes and distribution, creates the matrix, finalizes its structure for operations, prints its information, and then cleans up allocated resources.

```Fortran
PROGRAM create_matrix_example
   USE mpi
   USE dbcsr_api, ONLY: dbcsr_init_lib, dbcsr_finalize_lib, & 
                        dbcsr_distribution_new, dbcsr_distribution_release, & 
                        dbcsr_distribution_type, dbcsr_create, dbcsr_release, & 
                        dbcsr_finalize, dbcsr_print, dbcsr_type, & 
                        dbcsr_type_no_symmetry, dbcsr_type_real_8
   IMPLICIT NONE

   TYPE(dbcsr_type) :: matrix_a
   TYPE(dbcsr_distribution_type) :: dist
   INTEGER, DIMENSION(:), POINTER :: row_blk_sizes, col_blk_sizes
   INTEGER, DIMENSION(:), POINTER :: row_dist, col_dist
   INTEGER :: group, numnodes, ierr, i
   INTEGER, DIMENSION(2) :: npdims
   LOGICAL, DIMENSION(2) :: period = .TRUE.

   CALL mpi_init(ierr)
   CALL mpi_comm_size(MPI_COMM_WORLD, numnodes, ierr)

   npdims(:) = 0
   CALL mpi_dims_create(numnodes, 2, npdims, ierr)
   CALL mpi_cart_create(MPI_COMM_WORLD, 2, npdims, period, .FALSE., group, ierr)

   CALL dbcsr_init_lib(MPI_COMM_WORLD)

   ! Define 4x3 block matrix with varying block sizes
   ! Total matrix size: 8x9 (2+2+2+2 rows, 3+3+3 cols)
   ALLOCATE(row_blk_sizes(4), col_blk_sizes(3))
   row_blk_sizes(:) = 2  ! Each row block has 2 rows
   col_blk_sizes(:) = 3  ! Each column block has 3 columns

   ! Create distribution
   ALLOCATE(row_dist(4), col_dist(3))
   DO i = 1, 4
      row_dist(i) = MODULO(i-1, npdims(1))
   END DO
   DO i = 1, 3
      col_dist(i) = MODULO(i-1, npdims(2))
   END DO

   CALL dbcsr_distribution_new(dist, group=group, & 
                               row_dist=row_dist, col_dist=col_dist, & 
                               reuse_arrays=.TRUE.)

   ! Create DBCSR matrix
   CALL dbcsr_create(matrix=matrix_a, & 
                     name="Example Matrix A", & 
                     dist=dist, & 
                     matrix_type=dbcsr_type_no_symmetry, & 
                     row_blk_size=row_blk_sizes, & 
                     col_blk_size=col_blk_sizes, & 
                     data_type=dbcsr_type_real_8, & 
                     reuse_arrays=.TRUE.)

   ! Finalize matrix structure (required before operations)
   CALL dbcsr_finalize(matrix_a)

   ! Print matrix information
   CALL dbcsr_print(matrix_a)

   ! Cleanup
   CALL dbcsr_release(matrix_a)
   CALL dbcsr_distribution_release(dist)
   CALL mpi_comm_free(group, ierr)
   CALL dbcsr_finalize_lib()
   CALL mpi_finalize(ierr)
END PROGRAM create_matrix_example
```

--------------------------------

### Load PrgEnv-intel Module

Source: https://github.com/cp2k/dbcsr/blob/develop/docs/guide/2-user-guide/1-installation/1-cmake-build-recipes.md

Loads the PrgEnv-intel module, which sets up the environment for using the Intel compiler suite.

```bash
module load PrgEnv-intel
```

--------------------------------

### Configure DBCSR with GNU, System MPI, and OpenBLAS

Source: https://github.com/cp2k/dbcsr/blob/develop/docs/guide/2-user-guide/1-installation/1-cmake-build-recipes.md

Standard configuration for Linux systems using the default GNU compiler, system-provided MPI, and OpenBLAS libraries.

```bash
cmake ..
```

--------------------------------

### Configure DBCSR with Intel MPI and GNU Compiler

Source: https://github.com/cp2k/dbcsr/blob/develop/docs/guide/2-user-guide/1-installation/1-cmake-build-recipes.md

Uses Intel MPI wrappers with the GNU toolchain. Requires sourcing the Intel compiler environment variables first.

```bash
source /sw/intel/bin/compilervars.sh intel64
CC=mpicc FC=mpifc CXX=mpicxx cmake ..
```

--------------------------------

### POST /kernels/smm_acc_dnt_ALGORITHM

Source: https://github.com/cp2k/dbcsr/blob/develop/src/acc/libsmm_acc/kernels/README.md

Defines the signature and execution requirements for the batched matrix multiplication kernels.

```APIDOC
## POST /kernels/smm_acc_dnt_ALGORITHM

### Description
Executes a batched matrix multiplication on the GPU using one of the five available kernels (tiny, small, medium, largeDB1, largeDB2). The kernel processes a stack of matrix-matrix product descriptors stored in global memory.

### Method
POST

### Endpoint
/kernels/smm_acc_dnt_ALGORITHM

### Parameters
#### Request Body
- **param_stack** (int*) - Required - Pointer to the stack of matrix-matrix product descriptors in global memory.
- **stack_size** (int) - Required - Number of entries in the product stack.
- **a_data** (double*) - Required - Pointer to the A matrix blocks in global memory.
- **b_data** (double*) - Required - Pointer to the B matrix blocks in global memory.
- **c_data** (double*) - Required - Pointer to the C matrix blocks in global memory.

### Request Example
{
  "param_stack": "0x00001",
  "stack_size": 1024,
  "a_data": "0x10000",
  "b_data": "0x20000",
  "c_data": "0x30000"
}

### Response
#### Success Response (200)
- **status** (string) - Kernel execution completed successfully.

#### Response Example
{
  "status": "success"
}
```

--------------------------------

### Configure LIBXSMM via PkgConfig in CMake

Source: https://github.com/cp2k/dbcsr/blob/develop/CMakeLists.txt

This snippet uses PkgConfig to locate and configure LIBXSMM libraries. It handles both static and shared library variants based on project build settings.

```cmake
if (USE_SMM MATCHES "libxsmm|auto")
  find_package(PkgConfig ${LIBXSMM_REQUIRED})
  if (BUILD_SHARED_LIBS OR USE_SMM MATCHES "libxsmm-shared")
    pkg_check_modules(LIBXSMM IMPORTED_TARGET GLOBAL libxsmmf-shared)
  else ()
    pkg_check_modules(LIBXSMM IMPORTED_TARGET GLOBAL libxsmmf-static)
  endif ()
endif ()
```