### Python Installation - Quick Start

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/README_BUILD.md

Steps to install LoadGen as a Python module, including fetching source, building, and running a demo.

```bash
pip install absl-py numpy
git clone --recurse-submodules https://github.com/mlcommons/inference.git mlperf_inference
cd mlperf_inference/loadgen
CFLAGS="-std=c++14 -O3" python -m pip install .
```

--------------------------------

### Install dependencies

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/README.md

Installs dependencies using requirements.txt.

```bash
pip3 install -r requirements.txt
```

--------------------------------

### C/C++ API Setup for Find Peak Performance

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/README.md

Example of configuring and starting a test in Find Peak Performance mode using the C/C++ API.

```cpp
mlperf::TestSettings settings;
setting.server_target_qps = 100;
settings.scenario = mlperf::TestScenario::Server;
settings.mode = mlperf::TestMode::FindPeakPerformance;
mlperf::LogSettings log_settings;
/*
Construct QSL and SUT
*/
mlperf::StartTest(&sut, &qsl, settings, log_settings);
```

--------------------------------

### Start Test (Single Machine)

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/demos/lon/README.md

Starts the LoadGen client for the demo.

```shell
python demos/lon/py_demo_server_lon.py --sut_server http://localhost:8000
```

--------------------------------

### Basic SUT + jemalloc Experiment

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/benchmark/README.md

Example command for running the Basic SUT with jemalloc, achieving 800-900k i/s.

```bash
bash run.sh 800000 0
```

--------------------------------

### Bootstrap gn and ninja, and generate build project

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/tests/README.md

Initial setup commands to build gn and ninja and create a release project.

```bash
make bootstrap_gn_ninja
third_party/gn/gn gen out/Release --args="is_debug=false"
```

--------------------------------

### C++ Installation

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/README_BUILD.md

Steps to build the LoadGen as a C++ library.

```bash
git clone https://github.com/mlcommons/inference.git mlperf_inference
cd mlperf_inference
mkdir loadgen/build/ && cd loadgen/build/
cmake .. && cmake --build .
cp libmlperf_loadgen.a ..
```

--------------------------------

### Install Python Packages

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/demos/lon/README.md

Installs necessary Python packages for the demo.

```shell
pip install absl-py numpy wheel flask requests
```

--------------------------------

### Python API Setup for Find Peak Performance

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/README.md

Example of configuring and starting a test in Find Peak Performance mode using the Python API.

```python
settings = mlperf_loadgen.TestSettings()
settings.server_target_qps = 100
settings.scenario = mlperf_loadgen.TestScenario.Server
settings.mode = mlperf_loadgen.TestMode.FindPeakPerformance
...

mlperf_loadgen.StartTest(sut, qsl, settings)
```

--------------------------------

### Manual dependency installation

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/README.md

Manually installs PyTorch and other required libraries.

```bash
pip3 install gin-config pandas fbgemm_gpu torchrec tensorboard
```

--------------------------------

### Basic SUT with Server Coalesce and IssueQueryThreads Experiment

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/benchmark/README.md

Example command for running the Basic SUT with jemalloc, server_coalesce_queries, and 4 IssueQueryThreads, achieving 2.4-2.5M i/s.

```bash
bash run.sh 2400000 0 2 512 1 4
```

--------------------------------

### Testing Python Installation

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/README_BUILD.md

Command to run a simple end-to-end demo after installing LoadGen as a Python module.

```bash
python mlperf_inference/loadgen/demos/py_demo_single_stream.py
```

--------------------------------

### Python Installation - Using Wheels

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/README_BUILD.md

Alternative installation method using pre-built wheels for various Python versions and operating systems.

```bash
pip install mlperf-loadgen
```

--------------------------------

### Queued SUT Experiment

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/benchmark/README.md

Example command for running the Queued SUT with 2 complete threads and jemalloc, achieving 1.2-1.3M i/s.

```bash
bash run.sh 1200000 1 2 2048
```

--------------------------------

### Prerequisites for LoadGen Installation

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/README_BUILD.md

Commands to install necessary development libraries and Python packages.

```bash
sudo apt-get install libglib2.0-dev python-pip python3-pip
pip2 install absl-py numpy
pip3 install absl-py numpy
```

--------------------------------

### Build Loadgen Wheel

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/demos/lon/README.md

Builds the Loadgen wheel and installs it.

```shell
cd mlperf_inference/loadgen
CFLAGS="-std=c++14 -O3" python setup.py bdist_wheel
cd ..; pip install --force-reinstall loadgen/dist/`ls -r loadgen/dist/ | head -n1` ; cd -
```

--------------------------------

### Installation Rules

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/CMakeLists.txt

Defines installation rules for the 'mlperf_loadgen' library and its header files.

```cmake
install(TARGETS mlperf_loadgen
	DESTINATION ${CMAKE_INSTALL_PREFIX}/lib)
install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/
	DESTINATION ${CMAKE_INSTALL_PREFIX}/include FILES_MATCHING PATTERN "*.h")
```

--------------------------------

### Queued SUT with Server Coalesce Experiment

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/benchmark/README.md

Example command for running the Queued SUT with server_coalesce_queries enabled, achieving 1.4-1.5M i/s.

```bash
bash run.sh 1400000 1 2 512 1
```

--------------------------------

### Start Test (Over Network)

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/demos/lon/README.md

Starts the LoadGen client, replacing localhost with the correct IP for network operation.

```shell
python demos/lon/py_demo_server_lon.py --sut_server IP1:8000,IP2:8000,IP3:8000
```

--------------------------------

### Install dependencies

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/README.md

Installs the project dependencies using pip.

```bash
cd generative_recommenders/
pip install -e .

```

--------------------------------

### Run Demo SUT Server (Single Machine)

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/demos/lon/README.md

Starts the demo SUT server on a separate terminal.

```shell
python demos/lon/sut_over_network_demo.py --port 8000
```

--------------------------------

### Run Demo SUT Server (Over Network)

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/demos/lon/README.md

Starts the demo SUT server on a different machine, specifying the node number.

```shell
python demos/lon/sut_over_network_demo.py --port 8000 --node N1
```

--------------------------------

### Launch TensorBoard

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/README.md

Commands to launch TensorBoard for visualizing experimental logs.

```bash
tensorboard --logdir ~/generative-recommenders/exps/ml-1m-l200/ --port 24001 --bind_all
tensorboard --logdir ~/generative-recommenders/exps/ml-20m-l200/ --port 24001 --bind_all
tensorboard --logdir ~/generative-recommenders/exps/amzn-books-l50/ --port 24001 --bind_all
```

--------------------------------

### Run all unit tests

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/tests/README.md

Command to execute all unit tests.

```bash
out/Release/mlperf_loadgen_tests_basic .
```

--------------------------------

### Build unit tests

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/tests/README.md

Command to build the mlperf_loadgen_tests_basic executable.

```bash
third_party/ninja/ninja -C out/Release mlperf_loadgen_tests_basic
```

--------------------------------

### Run all performance tests

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/tests/README.md

Command to execute all performance tests.

```bash
out/Release/mlperf_loadgen_perftests .
```

--------------------------------

### Build performance tests

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/tests/README.md

Command to build the mlperf_loadgen_perftests executable.

```bash
third_party/ninja/ninja -C out/Release mlperf_loadgen_perftests
```

--------------------------------

### Run specific unit tests

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/tests/README.md

Command to run unit tests matching a regular expression.

```bash
out/Release/mlperf_loadgen_tests_basic <regex>
```

```bash
out/Release/mlperf_loadgen_tests_basic SingleStream
```

--------------------------------

### Download and preprocess data

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/README.md

Downloads and preprocesses public datasets for experiments.

```bash
mkdir -p tmp/ && python3 preprocess_public_data.py
```

--------------------------------

### Clone Repository

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/demos/lon/README.md

Clones the mlcommons inference repository with submodules.

```shell
git clone --recurse-submodules https://github.com/mlcommons/inference.git mlperf_inference
```

--------------------------------

### Download trained checkpoint

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/README.md

Downloads the 1TB trained checkpoint for DLRMv3.

```bash
bash <(curl -s https://raw.githubusercontent.com/mlcommons/r2-downloader/refs/heads/main/mlc-r2-downloader.sh) https://inference.mlcommons-storage.org/metadata/dlrm-v3-checkpoint.uri

```

--------------------------------

### Import necessary libraries and set up plotting environment

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/tools/mlperf-trace.ipynb

This code block imports libraries like matplotlib, seaborn, pandas, and numpy, and sets up plotting configurations such as figure size and font size. It also ignores warnings.

```python
%matplotlib inline
# Ignore warnings
import warnings
warnings.filterwarnings('ignore')

import json
import os
import seaborn as sns
from operator import itemgetter
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

figsize=(10, 5)
font=10

plt.figure(dpi=600)
plt.rc('xtick', labelsize=font) 
plt.rc('font', size=font)
sns.set(font_scale=1.4, style="whitegrid");
```

--------------------------------

### QDL FlushQueries Method

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/README.md

The FlushQueries method that QDL needs to implement.

```CPP
void FlushQueries();
```

--------------------------------

### Run model training with 4 GPUs

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/README.md

This command trains the DLRM-v3 model using 4 GPUs.

```bash
LOCAL_WORLD_SIZE=4 WORLD_SIZE=4 python3 generative_recommenders/dlrm_v3/train/train_ranker.py --dataset debug --mode train
```

--------------------------------

### Download sampled dataset for inference benchmark

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/README.md

Downloads the sampled DLRMv3 dataset used in the inference benchmark.

```bash
bash <(curl -s https://raw.githubusercontent.com/mlcommons/r2-downloader/refs/heads/main/mlc-r2-downloader.sh) https://inference.mlcommons-storage.org/metadata/dlrm-v3-dataset.uri

```

--------------------------------

### Generate Doxygen HTML with gn/ninja

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/docs/src/README.md

Command to generate Doxygen HTML documentation using the gn build system and ninja.

```bash
ninja -C out/Release generate_doxygen_html
```

--------------------------------

### Run public experiments

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/README.md

Runs the public experiments on MovieLens and Amazon Reviews datasets.

```bash
CUDA_VISIBLE_DEVICES=0 python3 main.py --gin_config_file=configs/ml-1m/hstu-sampled-softmax-n128-large-final.gin --master_port=12345
```

--------------------------------

### LoadGen Benchmark Command

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/benchmark/README.md

The command to run the LoadGen benchmark with specified parameters.

```bash
bash run.sh <target_qps> <0=Basic,1=Queue> <numCompleteThreads> <maxSizeInComplete> <server_coalesce_queries=0or1>
```

--------------------------------

### Run specific performance tests

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/tests/README.md

Command to run performance tests matching a regular expression.

```bash
out/Release/mlperf_loadgen_perftests <regex>
```

```bash
out/Release/mlperf_loadgen_tests_basic ServerPool
```

--------------------------------

### Download Synthetic Dataset

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/README.md

Commands to download the pre-generated synthetic dataset using gdown.

```bash
pip3 install gdown
mkdir -p tmp/ && cd tmp/
gdown https://drive.google.com/uc?id=1-jZ6k0el7e7PyFnwqMLfqUTRh_Qdumt-
unzip ml-3b.zip && rm ml-3b.zip
```

--------------------------------

### Package Dependencies

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/requirements.txt

This file lists the essential Python packages and their version requirements for the project.

```text
torch>=2.6.0
fbgemm_gpu>=1.1.0
torchrec>=1.1.0
gin_config>=0.5.0
pandas>=2.2.0
tensorboard>=2.19.0
pybind11
```

--------------------------------

### QDL Name Method

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/README.md

The Name function implemented by QDL to identify over-the-network SUTs.

```CPP
const std::string& Name();
```

--------------------------------

### Generate Synthetic Dataset

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/README.md

Command to generate the synthetic dataset using fractal expansion.

```bash
python3 run_fractal_expansion.py --input-csv-file tmp/ml-20m/ratings.csv --write-dataset True --output-prefix tmp/ml-3b/
```

--------------------------------

### Run model inference with 4 GPUs

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/README.md

This command runs inference for the DLRM-v3 model using 4 GPUs.

```bash
git clone --recurse-submodules https://github.com/mlcommons/inference.git mlperf_inference
cd mlperf_inference/loadgen
CFLAGS="-std=c++14 -O3" python -m pip install .

LOCAL_WORLD_SIZE=4 WORLD_SIZE=4 python3 generative_recommenders/dlrm_v3/inference/main.py --dataset debug
```

--------------------------------

### Run unit tests

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/README.md

Executes the unit tests for the inference module.

```bash
python tests/inference_test.py

```

--------------------------------

### Inference benchmark

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/README.md

Runs the inference benchmark using the specified number of GPUs and dataset.

```bash
cd generative_recommenders/generative_recommenders/dlrm_v3/inference/
WORLD_SIZE=8 python main.py --dataset sampled-streaming-100b

```

--------------------------------

### CMake Minimum Required and Project Definition

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/CMakeLists.txt

Sets the minimum required CMake version and defines the project name.

```cmake
cmake_minimum_required(VERSION 3.12)

project(mlperf_loadgen)
```

--------------------------------

### Library Output Path

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/CMakeLists.txt

Configures the output directory for libraries.

```cmake
set(LIBRARY_OUTPUT_PATH ${CMAKE_BINARY_DIR})
message(STATUS "Using output path: ${LIBRARY_OUTPUT_PATH}")
```

--------------------------------

### QDL Query Completion Response

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/README.md

The structure and method used by QDL to return query responses to LoadGen.

```CPP
struct QuerySampleResponse {
  ResponseId id;
  uintptr_t data;
  size_t size;
};
void QuerySamplesComplete(QuerySampleResponse* responses, 
                          size_t response_count);
```

--------------------------------

### Build loadgen

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/README.md

Builds the loadgen library with specified CFLAGS.

```bash
cd generative_recommenders/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/
CFLAGS="-std=c++14 -O3" python -m pip install .

```

--------------------------------

### Compiler Flags Configuration

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/CMakeLists.txt

Sets default C++ compiler flags for optimization and warnings, and allows for additional flags via an environment variable.

```cmake
if (NOT MSVC)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -O3 -W -Wall")
endif()
# Extra build options can be specified by setting the MLPERF_LOADGEN_CXX_FLAGS variable
if (MLPERF_LOADGEN_CXX_FLAGS)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${MLPERF_LOADGEN_CXX_FLAGS}")
endif()
```

--------------------------------

### QDL Query Issue

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/README.md

The method used by QDL to receive queries from LoadGen.

```CPP
void IssueQuery(const std::vector<QuerySample>& samples)
```

--------------------------------

### Manually Generate Doxygen HTML

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/docs/src/README.md

Command to manually generate Doxygen HTML documentation using a Python script.

```bash
python docs/src/doxygen_html_generator.py <target_dir> <loadgen_root>
```

--------------------------------

### Generate dataset

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/README.md

Generates the synthetic dataset for DLRMv3 training and inference.

```bash
cd generative_recommenders/dlrm_v3/
python streaming_synthetic_data.py

```

--------------------------------

### Version Extraction from VERSION.txt

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/CMakeLists.txt

Reads the VERSION.txt file and extracts major, minor, and patch versions using regular expressions.

```cmake
file(READ "${CMAKE_SOURCE_DIR}/VERSION.txt" VERSION_CONTENTS)

string(REGEX MATCH "^([0-9]+)\.([0-9]+)\.([0-9]+)" VERSION_MATCH ${VERSION_CONTENTS})

set(mlperf_loadgen_VERSION_MAJOR "${CMAKE_MATCH_1}")
set(mlperf_loadgen_VERSION_MINOR "${CMAKE_MATCH_2}")
set(mlperf_loadgen_VERSION_PATCH "${CMAKE_MATCH_3}")
```

--------------------------------

### Version Format Validation

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/CMakeLists.txt

Checks if the version components were successfully parsed and throws a fatal error if the format is incorrect.

```cmake
if(NOT DEFINED mlperf_loadgen_VERSION_MAJOR OR NOT DEFINED mlperf_loadgen_VERSION_MINOR OR NOT DEFINED mlperf_loadgen_VERSION_PATCH)
    message(FATAL_ERROR "Version format in VERSION.txt is incorrect. Expected format: MAJOR.MINOR.PATCH")
endif()
```

--------------------------------

### Analyze and plot inference time for SingleStream (commented out)

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/tools/mlperf-trace.ipynb

This is a commented-out section intended to analyze and plot the duration of 'QuerySamplesComplete' events, representing inference time, for SingleStream scenarios. It includes histograms and line plots.

```python
# for SingleStream
if False:
    df1 = df[df["name"].isin(["QuerySamplesComplete"])]
    ax = df1['dur'].plot.hist(bins=BINS, alpha=0.5, figsize=figsize)
    ax.set_title('Inference time (usec)');
    plt.show()
    ax = df1['dur'].plot(figsize=figsize)
    ax.set(ylim=(0, 100))
    ax.set_title('Individual inference time (usec)');
```

--------------------------------

### C++ Standard and Linker Flags

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/CMakeLists.txt

Sets the C++ standard to 14 and displays the configured C++ compiler flags, static linker flags, and shared linker flags.

```cmake
set(CMAKE_CXX_STANDARD "14")
message(STATUS "Using C++ compiler flags: ${CMAKE_CXX_FLAGS}")
message(STATUS "Using C++ standard: ${CMAKE_CXX_STANDARD}")
message(STATUS "Using static linker flags: ${CMAKE_STATIC_LINKER_FLAGS}")
message(STATUS "Using shared linker flags: ${CMAKE_SHARED_LINKER_FLAGS}")
```

--------------------------------

### Accuracy test

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/README.md

Runs the accuracy test and dumps prediction outputs.

```bash
python accuracy.py --path path/to/mlperf_log_accuracy.json

```

--------------------------------

### Executable Target Definition

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/CMakeLists.txt

Adds an executable target named 'benchmark' using 'benchmark/repro.cpp' and links it with the 'mlperf_loadgen' library and platform-specific libraries.

```cmake
if(WIN32)
set (LIBS "")
else()
set (LIBS pthread)
endif()

add_executable(benchmark benchmark/repro.cpp)
target_link_libraries(benchmark PUBLIC mlperf_loadgen ${LIBS})
```

--------------------------------

### Version Generator Execution

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/CMakeLists.txt

Executes a Python script to generate a C++ source file containing version information.

```cmake
execute_process(COMMAND ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/version_generator.py ${CMAKE_BINARY_DIR}/version_generated.cc ${CMAKE_CURRENT_SOURCE_DIR})
```

--------------------------------

### Version Generator Script

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/README_FAQ.md

The version_generator.py script is used to generate the version_generated.cc file, which records git stats and source file SHAs at build time for verification purposes. This script should ideally be integrated with custom build systems.

```python
version_generator.py
```

--------------------------------

### Source File List

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/CMakeLists.txt

Defines the list of source files to be compiled for the mlperf_loadgen library.

```cmake
set(SOURCE
  ${CMAKE_CURRENT_SOURCE_DIR}/bindings/c_api.h
  ${CMAKE_CURRENT_SOURCE_DIR}/bindings/c_api.cc
  ${CMAKE_CURRENT_SOURCE_DIR}/early_stopping.cc  
  ${CMAKE_CURRENT_SOURCE_DIR}/issue_query_controller.cc
  ${CMAKE_CURRENT_SOURCE_DIR}/loadgen.cc
  ${CMAKE_CURRENT_SOURCE_DIR}/logging.cc
  ${CMAKE_CURRENT_SOURCE_DIR}/logging.h
  ${CMAKE_CURRENT_SOURCE_DIR}/test_settings_internal.cc
  ${CMAKE_CURRENT_SOURCE_DIR}/test_settings_internal.h
  ${CMAKE_CURRENT_SOURCE_DIR}/utils.cc
  ${CMAKE_CURRENT_SOURCE_DIR}/utils.h
  ${CMAKE_CURRENT_SOURCE_DIR}/results.h
  ${CMAKE_CURRENT_SOURCE_DIR}/results.cc
  ${CMAKE_CURRENT_SOURCE_DIR}/version.cc
  ${CMAKE_CURRENT_SOURCE_DIR}/version.h
  ${CMAKE_CURRENT_SOURCE_DIR}/mlperf_conf.h
  ${CMAKE_CURRENT_SOURCE_DIR}/VERSION.txt
  ${CMAKE_BINARY_DIR}/version_generated.cc
)
```

--------------------------------

### Library Target Definition

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/CMakeLists.txt

Adds a static library target named 'mlperf_loadgen' using the defined source files and links it.

```cmake
include_directories(${CMAKE_CURRENT_SOURCE_DIR})

add_library(mlperf_loadgen STATIC ${SOURCE})
target_link_libraries(mlperf_loadgen)
```

--------------------------------

### Python Detection

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/CMakeLists.txt

Finds the Python interpreter to be used for generating source files.

```cmake
find_package(PythonInterp)
message(STATUS "Using Python interpreter: ${PYTHON_EXECUTABLE}")
```

--------------------------------

### Analyze and plot time between QuerySamplesComplete events

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/tools/mlperf-trace.ipynb

Filters the DataFrame for 'QuerySamplesComplete' events, calculates the time difference between consecutive events, and plots histograms for this time delta and the duration of the 'QuerySamplesComplete' events themselves.

```python
df1 = df[df["name"].isin(["QuerySamplesComplete"])]
df1['delta'] = df1['ts'].diff()
ax = df1['delta'].plot.hist(bins=BINS, alpha=0.5, figsize=figsize)
ax.set_title('Time between QuerySamplesComplete (usec)');
plt.show()

ax = df1['dur'].plot.hist(bins=BINS, alpha=0.5, figsize=figsize)
ax.set_title('Time QuerySamplesComplete (usec)');
```

--------------------------------

### Analyze and plot latency (issue_to_done) for Latency events

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/tools/mlperf-trace.ipynb

Filters the DataFrame for 'Latency' events and plots histograms and line plots for the 'issue_to_done' metric, representing inference time in microseconds.

```python
df1 = df[df["name"].isin(["Latency"])]
ax = df1['issue_to_done'].plot.hist(bins=BINS, alpha=0.5, figsize=figsize)
ax.set_title('Inference time (usec)');
#ax.set(xlim=(0, 25000))
plt.xticks(rotation=60)
plt.show()

ax = df1['issue_to_done'].plot(figsize=figsize)
ax.set_title('Individual inference time (usec)');
#ax.set(ylim=(0, 200))
plt.show()


# df1['issue_to_done'].describe()
```

--------------------------------

### Analyze and plot IssueQuery duration and time between queries

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/tools/mlperf-trace.ipynb

Filters the DataFrame for 'IssueQuery' events, calculates the time difference between consecutive queries, and plots histograms for the duration of 'IssueQuery' and the time delta between them.

```python
df1 = df[df["name"].isin(["IssueQuery"])]
df1['delta'] = df1['ts'].diff()
ax = df1['dur'].plot.hist(bins=BINS, alpha=0.5, figsize=figsize)
ax.set_title('IssueQuery duration (usec)');
plt.show()
ax = df1['delta'].plot.hist(bins=BINS, alpha=0.5, figsize=figsize)
ax.set_title('Time between IssueQuery (usec)');

# df1['delta'].describe()
```

--------------------------------

### Load trace data and display descriptive statistics

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/tools/mlperf-trace.ipynb

Loads the mlperf trace data from a specified JSON file into a pandas DataFrame and then displays descriptive statistics for the DataFrame.

```python
df = trace_to_df('/tmp/mlperf_log_trace.json')
df.describe()
```

--------------------------------

### Configuration File Processing

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/CMakeLists.txt

Reads a configuration file (mlperf.conf), escapes special characters (quotes, backslashes, newlines), and formats it into a C++ string literal for inclusion in a header file (mlperf_conf.h).

```cmake
set(CONF_FILE "mlperf.conf")
set(HEADER_FILE "mlperf_conf.h")

file(READ ${CONF_FILE} CONF_CONTENTS)

string(REPLACE "\\" "\\\\" CONF_CONTENTS "${CONF_CONTENTS}")
string(REPLACE "\"" "\\\"" CONF_CONTENTS "${CONF_CONTENTS}")

string(REPLACE "\n" "\\n\"
\"" CONF_CONTENTS "${CONF_CONTENTS}")

set(FORMATTED_CONTENT "const char* mlperf_conf =
\"${CONF_CONTENTS}\";\n")

file(WRITE ${HEADER_FILE} "${FORMATTED_CONTENT}")
```

--------------------------------

### Function to convert trace data to a pandas DataFrame

Source: https://github.com/meta-recsys/generative-recommenders/blob/main/generative_recommenders/dlrm_v3/inference/thirdparty/loadgen/tools/mlperf-trace.ipynb

Defines a function `trace_to_df` that reads a JSON trace file, extracts relevant event data (Latency, Sample, QuerySamplesComplete, IssueQuery), and converts it into a pandas DataFrame, sorting by timestamp.

```python
def trace_to_df(fname):
    with open(fname, "r") as f:
        j = json.load(f)
    if type(j) == dict:
        j = j['traceEvents']
        
    result = []
    for item in j:
        name = item['name']
        if name not in ["Latency", "Sample", "QuerySamplesComplete", "IssueQuery"]:
            continue

        args = item.get('args')
        d = {"ts": item['ts'], "name": name, "dur": item.get("dur")}

        if name == "Latency":
            d["issue_delay"] = args["issue_delay"]
            d["issue_to_done"] = args["issue_to_done"] / 1e3
            result.append(d)
        elif name == "Sample":
            if args:
                d["issue_start_ns"] = args["issue_start_ns"]
                d["complete_ns"] = args["complete_ns"]
                d["issue_to_done"] = (args["complete_ns"] - args["issue_start_ns"]) / 1e3
                result.append(d)
        elif name == "QuerySamplesComplete":
            result.append(d)
        elif name == "IssueQuery":
            result.append(d)

    df = pd.DataFrame(result)
    df = df.sort_values(by=["ts"])
    return df

BINS = 10
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.