### Environment Setup with Conda and Pip

Source: https://github.com/princetonuniversity/llmcompass/blob/main/README.md

Set up a Python 3.9 environment using Conda, activate it, and install necessary packages including PyTorch, matplotlib, seaborn, and scipy.

```bash
conda create -n llmcompass_ae python=3.9
conda activate llmcompass_ae
pip3 install scalesim
conda install pytorch==2.0.0 -c pytorch
pip3 install matplotlib
pip3 install seaborn
pip3 install scipy

```

--------------------------------

### Instantiate System from hardware spec dict

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

Constructs a `System` object from an architecture specification dictionary. This is the primary entry point for creating a simulation-ready object.

```python
from design_space_exploration.dse import read_architecture_template, template_to_system

specs = read_architecture_template("configs/GA100.json")
system = template_to_system(specs)

# system.device holds the compute + I/O + memory hierarchy
# system.interconnect holds topology and link parameters
print(system.device.compute_module.core_count)
print(system.device.io_module.bandwidth)
print(system.interconnect.device_count)
print(system.interconnect.topology)
```

--------------------------------

### template_to_system

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

Instantiates a System object from a hardware specification dictionary. This function takes a dictionary representing the hardware architecture and constructs a fully parameterized `System` object, which includes a `Device` and an `InterConnectModule`. It is the primary entry point for converting a JSON hardware description into a simulation-ready object.

```APIDOC
## template_to_system

### Description
Instantiates a System object from a hardware spec dict. Constructs a fully parameterized `System` object (containing a `Device` and an `InterConnectModule`) from an architecture spec dictionary. This is the primary entry point for turning a JSON hardware description into a simulation-ready object.

### Method
```python
template_to_system(specs: dict)
```

### Parameters
#### Request Body
- **specs** (dict) - Required - A dictionary containing the hardware architecture specification, typically obtained from `read_architecture_template`.

### Request Example
```python
from design_space_exploration.dse import read_architecture_template, template_to_system

specs = read_architecture_template("configs/GA100.json")
system = template_to_system(specs)
```

### Response
#### Success Response (System)
- **system** (System) - A `System` object ready for simulation, containing device and interconnect modules.
```

--------------------------------

### Run AE Experiment for Figure 5

Source: https://github.com/princetonuniversity/llmcompass/blob/main/README.md

Navigate to the 'ae/figure5' directory and execute the 'run_figure5.sh' script to generate Figure 5. This experiment is estimated to take around 100 minutes.

```bash
cd ae/figure5
bash run_figure5.sh

```

--------------------------------

### Simulate Multi-device AllReduce with AllReduceMultiPCB

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

Simulates an AllReduce communication collective over a multi-PCB system, supporting Fully-Connected (FC) and Ring topologies. Accounts for link bandwidth, latency, packet header overhead, and crossbar bandwidth. Requires importing AllReduceMultiPCB and Tensor utilities.

```python
from software_model.communication_primitives import AllReduceMultiPCB
from software_model.utils import Tensor, data_type_dict
from design_space_exploration.dse import read_architecture_template, template_to_system

specs = read_architecture_template("configs/GA100.json")
system = template_to_system(specs)
interconnect = system.interconnect

# AllReduce after MHA output projection: tensor shape [batch, seq, d_model]
allreduce = AllReduceMultiPCB(data_type_dict["fp16"])
tensor = Tensor([8, 2048, 12288], data_type_dict["fp16"])
_ = allreduce(tensor)

latency = allreduce.simulate(interconnect)
data_size_mb = tensor.size * data_type_dict["fp16"].word_size / 1e6
print(f"AllReduce data size: {data_size_mb:.1f} MB")
print(f"AllReduce latency (FC topology): {latency * 1e3:.3f} ms")
# Two AllReduces per transformer block (after MHA and FFN)
print(f"Communication overhead per block: {latency * 2 * 1e3:.3f} ms")
```

--------------------------------

### Run AE Experiment for Figure 6

Source: https://github.com/princetonuniversity/llmcompass/blob/main/README.md

Navigate to the 'ae/figure6' directory and execute the 'run_figure6.sh' script to generate Figure 6. This experiment is estimated to take around 1 minute.

```bash
cd ae/figure6
bash run_figure6.sh

```

--------------------------------

### Run AE Experiment for Figure 7

Source: https://github.com/princetonuniversity/llmcompass/blob/main/README.md

Navigate to the 'ae/figure7' directory and execute the 'run_figure7.sh' script to generate Figure 7. This experiment is estimated to take around 20 minutes.

```bash
cd ae/figure7
bash run_figure7.sh

```

--------------------------------

### Run AE Experiment for Figure 10

Source: https://github.com/princetonuniversity/llmcompass/blob/main/README.md

Navigate to the 'ae/figure10' directory and execute the 'run_figure10.sh' script to generate Figure 10. This experiment is estimated to take around 45 minutes.

```bash
cd ae/figure10
bash run_figure10.sh

```

--------------------------------

### Run AE Experiment for Figure 8

Source: https://github.com/princetonuniversity/llmcompass/blob/main/README.md

Navigate to the 'ae/figure8' directory and execute the 'run_figure8.sh' script to generate Figure 8. This experiment is estimated to take around 40 minutes.

```bash
cd ae/figure8
bash run_figure8.sh

```

--------------------------------

### Run AE Experiment for Figure 11

Source: https://github.com/princetonuniversity/llmcompass/blob/main/README.md

Navigate to the 'ae/figure11' directory and execute the 'run_figure11.sh' script to generate Figure 11. This experiment is estimated to take around 5 minutes.

```bash
cd ae/figure11
bash run_figure11.sh

```

--------------------------------

### Compile and Simulate Model

Source: https://github.com/princetonuniversity/llmcompass/blob/main/docs/run.md

Compiles and simulates the instantiated model using a heuristic-GPU strategy with the provided system configuration. Returns the simulated latency.

```python
auto_regression_latency_simulated = model_auto_regression.compile_and_simulate(
	system, "heuristic-GPU"
)
```

--------------------------------

### Run AE Experiment for Figure 9

Source: https://github.com/princetonuniversity/llmcompass/blob/main/README.md

Navigate to the 'ae/figure9' directory and execute the 'run_figure9.sh' script to generate Figure 9. This experiment is estimated to take around 30 minutes.

```bash
cd ae/figure9
bash run_figure9.sh

```

--------------------------------

### Find Cheapest Hardware Design for LLM

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

Use this function to exhaustively search a discrete design space for the minimum-area hardware that meets specified prefill and decoding latency SLOs. Results are written to `configs/best_arch_specs.json`.

```python
from design_space_exploration.dse import find_cheapest_design

# Find the cheapest hardware design for GPT-3 (96 layers, 12288 hidden dim, 96 heads)
# that achieves:
#   - Prefill (2048-token context, batch=8): <= 5 seconds
#   - Decoding (1024-token output): <= 0.1 seconds per token
find_cheapest_design(
    d_model=12288,
    n_heads=96,
    n_layers=96,
    batch_size=8,
    input_seq_length=2048,
    init_latency=5.0,          # seconds: max allowed prefill latency
    output_seq_length=1024,
    auto_regression_latency=0.1,  # seconds: max allowed per-token decoding latency
)
# Output: prints candidate count and writes configs/best_arch_specs.json
# Example output:
#   number of potential designs=1247
#   (best_arch_specs.json contains the minimum total die area configuration)
```

--------------------------------

### Simulate Softmax Operator

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

Simulates the online softmax kernel, modeling compute and I/O phases with a two-level tiling strategy. Accounts for multi-pass reduction cost.

```python
from software_model.softmax import Softmax
from software_model.utils import Tensor, data_type_dict
from design_space_exploration.dse import read_architecture_template, template_to_system

specs = read_architecture_template("configs/GA100.json")
system = template_to_system(specs)
device = system.device

# Attention scores for GPT-3 prefill: [batch * n_heads_per_device, seq, seq]
# With 4-way TP: 96/4=24 heads per device, batch=8
softmax_op = Softmax(data_type_dict["fp16"])
attn_scores = Tensor([8 * 24, 2048, 2048], data_type_dict["fp16"])
output = softmax_op(attn_scores)  # shape unchanged

roofline_lat = softmax_op.roofline_model(device)
sim_lat = softmax_op.compile_and_simulate(device)
print(f"Softmax roofline: {roofline_lat * 1e6:.2f} µs")
print(f"Softmax simulated: {sim_lat * 1e6:.2f} µs")
```

--------------------------------

### Load hardware config from JSON

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

Reads a hardware architecture specification from a JSON file. Inspect or mutate the returned dictionary before passing it to `template_to_system`.

```python
from design_space_exploration.dse import read_architecture_template, template_to_system

# Load the NVIDIA A100 x4 configuration
specs = read_architecture_template("configs/GA100.json")

# specs is a plain dict — inspect or mutate before passing to template_to_system
print(specs["name"])
print(specs["device_count"])
print(specs["device"]["compute_chiplet"]["core_count"])
print(specs["interconnect"]["topology"])
```

--------------------------------

### Run AE Experiment for Figure 12

Source: https://github.com/princetonuniversity/llmcompass/blob/main/README.md

Navigate to the 'ae/figure12' directory and execute the 'run_figure12.sh' script to generate Figure 12. This experiment is estimated to take around 4 hours.

```bash
cd ae/figure12
bash run_figure12.sh

```

--------------------------------

### Instantiate Transformer Model

Source: https://github.com/princetonuniversity/llmcompass/blob/main/docs/run.md

Initiates and instantiates a TransformerBlockAutoRegressionTP model with specified parameters like d_model, n_heads, device_count, and data_type. The model is then called with input tensors.

```python
model_auto_regression = TransformerBlockAutoRegressionTP(
        d_model=12288,
        n_heads=96,
        device_count=1,
        data_type=data_type_dict["fp16"],
    )
_ = model_auto_regression(
	Tensor([bs, 1, 12288], data_type_dict["fp16"]),
	seq_len,
)
```

--------------------------------

### Read Hardware Configuration

Source: https://github.com/princetonuniversity/llmcompass/blob/main/docs/run.md

Reads the hardware architecture template from a specified JSON file and parses it into a system object for LLMCompass.

```python
from design_space_exploration.dse import template_to_system, read_architecture_template

specs = read_architecture_template("PATH/TO/YOUR/JSON")
system = template_to_system(specs)
```

--------------------------------

### Simulate Transformer Model Prefill Latency

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

Builds a computational graph for a transformer model and estimates its prefill latency using roofline and cycle-accurate simulations.

```python
from software_model.transformer import TransformerBlockAutoRegressionTP
from software_model.utils import data_type_dict, Tensor
from design_space_exploration.dse import read_architecture_template, template_to_system

specs = read_architecture_template("configs/GA100.json")
system = template_to_system(specs)

model = TransformerBlockAutoRegressionTP(
    d_model=12288,
    n_heads=96,
    device_count=4,
    data_type=data_type_dict["fp16"],
)

batch_size = 8
seq_len = 1024   # current KV-cache length

# Input is always shape [bs, 1, d_model] for decoding (one new token)
output = model(Tensor([batch_size, 1, 12288], data_type_dict["fp16"])), seq_len)
assert output.shape == [batch_size, 1, 12288]

# Check memory requirement for KV-cache + weights (bytes, per device)
print(f"Memory per layer: {model.memory_requirement / 1e9:.2f} GB")

# Simulate decoding latency for one token generation step
latency = model.compile_and_simulate(system, "heuristic-GPU")
print(f"Decoding latency per block: {latency * 1e3:.3f} ms")

# Full model throughput: tokens/second for 96-layer GPT-3
n_layers = 96
tokens_per_second = batch_size / (latency * n_layers)
print(f"Throughput: {tokens_per_second:.1f} tokens/s")
```

--------------------------------

### Simulate Transformer Block Prefill

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

Builds a computational graph for a transformer model and estimates its prefill latency using roofline and cycle-accurate simulations.

```python
from software_model.transformer import TransformerBlockAutoRegressionTP
from software_model.utils import data_type_dict, Tensor
from design_space_exploration.dse import read_architecture_template, template_to_system

specs = read_architecture_template("configs/GA100.json")
system = template_to_system(specs)

output = model(Tensor([batch_size, seq_len, 12288], data_type_dict["fp16"]))
assert output.shape == [batch_size, seq_len, 12288]

# Fast roofline estimate (seconds per transformer block)
roofline_latency = model.roofline_model(system)
print(f"Roofline prefill latency: {roofline_latency * 1e3:.3f} ms")

# Cycle-accurate simulation with GPU-style heuristic tiling search
simulated_latency = model.compile_and_simulate(system, "heuristic-GPU")
print(f"Simulated prefill latency: {simulated_latency * 1e3:.3f} ms")

# Multiply by number of layers for full-model latency
n_layers = 96
total_prefill_latency = simulated_latency * n_layers
print(f"Full GPT-3 prefill (96 layers): {total_prefill_latency:.3f} s")
# Expected: ~5 s for a 4xA100 system
```

--------------------------------

### Simulate GeLU Activation with GeLU

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

Simulates the GELU activation function (tanh approximation variant) as a memory-bandwidth-bound vector operation. Models IO and compute separately with roofline analysis. Requires importing GeLU and Tensor utilities.

```python
from software_model.gelu import GeLU
from software_model.utils import Tensor, data_type_dict
from design_space_exploration.dse import read_architecture_template, template_to_system

specs = read_architecture_template("configs/GA100.json")
system = template_to_system(specs)
device = system.device

# FFN intermediate activations: [batch * seq, 4 * d_model / device_count]
# With 4-way TP: 4*12288/4 = 12288
gelu_op = GeLU(data_type_dict["fp16"])
ffn_hidden = Tensor([8 * 2048, 12288], data_type_dict["fp16"])
output = gelu_op(ffn_hidden)

roofline_lat = gelu_op.roofline_model(device)
sim_lat = gelu_op.compile_and_simulate(device, compile_mode="heuristic-GPU")
print(f"GeLU roofline: {roofline_lat * 1e6:.2f} µs")
print(f"GeLU simulated: {sim_lat * 1e6:.2f} µs")
```

--------------------------------

### Simulate Transformer Block Prefill Stage (TP)

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

Simulates a single transformer block during the prefill stage with Tensor Parallelism. Call `__call__` to build the graph, then `compile_and_simulate` or `roofline_model`.

```python
from software_model.transformer import TransformerBlockInitComputationTP
from software_model.utils import data_type_dict, Tensor
from design_space_exploration.dse import read_architecture_template, template_to_system

# Build system
specs = read_architecture_template("configs/GA100.json")
system = template_to_system(specs)

# GPT-3 scale model: d_model=12288, n_heads=96, 4-way tensor parallelism
model = TransformerBlockInitComputationTP(
    d_model=12288,
    n_heads=96,
    device_count=4,
    data_type=data_type_dict["fp16"],
)

batch_size = 8
seq_len = 2048
```

--------------------------------

### read_architecture_template

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

Loads a hardware architecture specification from a JSON file. This function reads a JSON file that describes all hardware parameters, including device count, interconnect topology, compute core configuration, memory protocol, and I/O subsystem, returning the raw dictionary representation.

```APIDOC
## read_architecture_template

### Description
Loads a hardware architecture specification from a JSON file and returns the raw dictionary. The JSON schema describes all hardware parameters including device count, interconnect topology, compute core configuration, memory protocol, and I/O subsystem.

### Method
```python
read_architecture_template(file_path: str)
```

### Parameters
#### Path Parameters
- **file_path** (str) - Required - The path to the JSON configuration file.

### Request Example
```python
from design_space_exploration.dse import read_architecture_template

specs = read_architecture_template("configs/GA100.json")
```

### Response
#### Success Response (dict)
- **specs** (dict) - A dictionary containing the hardware architecture specification.
```

--------------------------------

### TransformerBlockInitComputationTP

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

Simulates a single transformer block during the initial computation/prefill stage with Tensor Parallelism across multiple devices. This class implements the full multi-head attention and FFN graph. Users can call `__call__` to build the computational graph and then `compile_and_simulate` or `roofline_model` for evaluation.

```APIDOC
## TransformerBlockInitComputationTP

### Description
Simulates a single transformer block during the initial computation / prefill stage with Tensor Parallelism across multiple devices. Implements the full multi-head attention + FFN graph. Call `__call__` to build the computational graph (tensor shapes propagated symbolically), then call `compile_and_simulate` or `roofline_model`.

### Method
```python
TransformerBlockInitComputationTP(d_model: int, n_heads: int, device_count: int, data_type: DataType)
```

### Parameters
#### Path Parameters
- **d_model** (int) - Required - The dimensionality of the model's hidden states.
- **n_heads** (int) - Required - The number of attention heads.
- **device_count** (int) - Required - The number of devices to use for tensor parallelism.
- **data_type** (DataType) - Required - The data type for computations. Use `data_type_dict` for available types.

### Request Example
```python
from software_model.transformer import TransformerBlockInitComputationTP
from software_model.utils import data_type_dict
from design_space_exploration.dse import read_architecture_template, template_to_system

specs = read_architecture_template("configs/GA100.json")
system = template_to_system(specs)

model = TransformerBlockInitComputationTP(
    d_model=12288,
    n_heads=96,
    device_count=4,
    data_type=data_type_dict["fp16"],
)
```

### Response
#### Success Response (TransformerBlockInitComputationTP)
- **model** (TransformerBlockInitComputationTP) - An instance of the transformer block simulation class.
```

--------------------------------

### Clone LLMCompass Repository from GitHub

Source: https://github.com/princetonuniversity/llmcompass/blob/main/README.md

Clone the LLMCompass repository from GitHub, specifically the 'ISCA_AE' branch, and initialize/update its submodules.

```bash
git clone -b ISCA_AE https://github.com/PrincetonUniversity/LLMCompass
cd LLMCompass
git submodule init
git submodule update --recursive

```

--------------------------------

### NVIDIA GA100 Hardware Configuration

Source: https://github.com/princetonuniversity/llmcompass/blob/main/docs/run.md

This JSON defines a hardware configuration for a system with 4 NVIDIA GA100 devices connected via NVLinks. It specifies device count, interconnect details (bandwidth, latency, topology), and device-specific attributes like frequency, core counts, memory protocol, and global buffer characteristics.

```json
{
    "name": "NVIDIA A100(80GB)x4", 
    "device_count": 4, 
    "interconnect": {
        "link": {
            "name": "NVLink3",
            "bandwidth_per_direction_byte": 25e9,
            "bandwidth_both_directions_byte": 50e9,
            "latency_second": 8.92e-6,
            "flit_size_byte": 16,
            "header_size_byte": 16,
            "max_payload_size_byte": 256
        },
        "link_count_per_device": 12,
        "topology": "FC"
    },
    "device": {
        "frequency_Hz": 1410e6,
        "compute_chiplet_count": 1,
        "compute_chiplet": {
            "physical_core_count": 128, 
            "core_count": 128, 
            "process_node": "7nm", 
            "core": {
                "sublane_count": 4,
                "systolic_array": {
                    "array_width": 16,
                    "array_height": 16,
                    "data_type": "fp16",
                    "mac_per_cycle": 1
                },
                "vector_unit": {
                    "vector_width": 32,
                    "flop_per_cycle": 4, 
                    "data_type": "fp16",
                    "int32_count": 16, 
                    "fp16_count": 0,
                    "fp32_count": 16,
                    "fp64_count": 8
                },
                "register_file": {
                    "num_reg_files": 1,
                    "num_registers": 16384,
                    "register_bitwidth":32,
                    "num_rdwr_ports":4
                },
                "SRAM_KB": 192
            }
        },
        "memory_protocol": "HBM2e",
        "_memory_protocol_list": [
            "HBM2e",
            "DDR4",
            "DDR5",
            "PCIe4",
            "PCIe5"
        ],
        "io": {
            "process_node": "7nm",
            "global_buffer_MB": 48,
            "physical_global_buffer_MB": 48,
            "global_buffer_bandwidth_per_cycle_byte": 5120,
            "memory_channel_physical_count": 6, 
            "memory_channel_active_count": 5, 
            "pin_count_per_channel": 1024,
            "bandwidth_per_pin_bit": 3.2e9
        },
        "memory": {
            "total_capacity_GB": 80
        }
    }
}

```

--------------------------------

### Simulate Layer Normalization with LayerNorm

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

Simulates layer normalization over the last dimension of an input tensor using a three-pass compute model. Captures reduction costs at L1 and L2 tile boundaries. Requires importing LayerNorm and Tensor utilities.

```python
from software_model.layernorm import LayerNorm
from software_model.utils import Tensor, data_type_dict
from design_space_exploration.dse import read_architecture_template, template_to_system

specs = read_architecture_template("configs/GA100.json")
system = template_to_system(specs)
device = system.device

# Post-attention layer norm: [batch * seq, d_model]
layernorm_op = LayerNorm(data_type_dict["fp16"])
hidden_states = Tensor([8 * 2048, 12288], data_type_dict["fp16"])
output = layernorm_op(hidden_states)

roofline_lat = layernorm_op.roofline_model(device)
sim_lat = layernorm_op.compile_and_simulate(device, compile_mode="heuristic-GPU")
print(f"LayerNorm roofline: {roofline_lat * 1e6:.2f} µs")
print(f"LayerNorm simulated: {sim_lat * 1e6:.2f} µs")
```

--------------------------------

### Simulate Matmul Operator

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

Simulates a 2D matrix multiplication using a hierarchical tiling model. Supports multiple compile modes for optimization.

```python
from software_model.matmul import Matmul
from software_model.utils import Tensor, data_type_dict
from design_space_exploration.dse import read_architecture_template, template_to_system

specs = read_architecture_template("configs/GA100.json")
system = template_to_system(specs)
device = system.device

# Simulate the QKV projection in GPT-3 prefill: [8*2048, 12288] × [12288, 3072]
matmul = Matmul(data_type_dict["fp16"])
output = matmul(
    Tensor([8 * 2048, 12288], data_type_dict["fp16"]),  # [M, K]
    Tensor([12288, 3072], data_type_dict["fp16"]),        # [K, N]
)
print(f"Output shape: {output.shape}")         # [16384, 3072]
print(f"FLOPs: {matmul.flop_count:,}")         # 2*M*K*N

# Roofline bound (seconds)
roofline = matmul.roofline_model(device)
print(f"Roofline latency: {roofline * 1e6:.2f} µs")

# Cycle-accurate simulation
latency = matmul.compile_and_simulate(device, compile_mode="heuristic-GPU")
print(f"Simulated latency: {latency * 1e6:.2f} µs")
print(f"Best tiling: L2={matmul.best_mapping.l2_tile_M}x{matmul.best_mapping.l2_tile_N}x{matmul.best_mapping.l2_tile_K}")
```

--------------------------------

### Hardware Configuration JSON Schema

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

This JSON schema defines the structure for specifying hardware configurations, including details on interconnect, device, compute chiplets, cores, vector units, memory protocols, and SRAM.

```json
{
    "name": "MyCustomAccelerator",
    "device_count": 2,
    "interconnect": {
        "link": {
            "name": "NVLink3",
            "bandwidth_per_direction_byte": 25e9,
            "bandwidth_both_directions_byte": 50e9,
            "latency_second": 8.92e-6,
            "flit_size_byte": 16,
            "header_size_byte": 16,
            "max_payload_size_byte": 256
        },
        "link_count_per_device": 12,
        "topology": "FC"
    },
    "device": {
        "frequency_Hz": 1.5e9,
        "compute_chiplet_count": 1,
        "compute_chiplet": {
            "physical_core_count": 64,
            "core_count": 64,
            "process_node": "5nm",
            "core": {
                "sublane_count": 4,
                "systolic_array": {
                    "array_width": 32,
                    "array_height": 32,
                    "data_type": "fp16",
                    "mac_per_cycle": 1
                },
                "vector_unit": {
                    "vector_width": 32,
                    "flop_per_cycle": 4,
                    "data_type": "fp16",
                    "int32_count": 16,
                    "fp16_count": 0,
                    "fp32_count": 16,
                    "fp64_count": 8
                },
                "register_file": {
                    "num_reg_files": 1,
                    "num_registers": 16384,
                    "register_bitwidth": 32,
                    "num_rdwr_ports": 4
                },
                "SRAM_KB": 256
            }
        },
        "memory_protocol": "HBM2e",
        "io": {
            "process_node": "5nm",
            "global_buffer_MB": 64,
            "physical_global_buffer_MB": 64,
            "global_buffer_bandwidth_per_cycle_byte": 8192,
            "memory_channel_physical_count": 6,
            "memory_channel_active_count": 5,
            "pin_count_per_channel": 1024,
            "bandwidth_per_pin_bit": 3.2e9
        },
        "memory": {
            "total_capacity_GB": 80
        }
    }
}
```

--------------------------------

### BibTeX Entry for LLMCompass Paper

Source: https://github.com/princetonuniversity/llmcompass/blob/main/README.md

This is the BibTeX entry for the LLMCompass paper, useful for academic citations.

```bibtex
@inproceedings{LLMCompass,
author = {Zhang, Hengrui and Ning, August and Prabhakar, Rohan Baskar and Wentzlaff, David},
title = {LLMCompass: Enabling Efficient Hardware Design for Large Language Model Inference},
year = {2024},
booktitle = {Proceedings of the 51st Annual International Symposium on Computer Architecture},
}

```

--------------------------------

### Estimate Compute Chiplet Area with calc_compute_chiplet_area_mm2

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

Estimates the silicon area (mm²) of the compute chiplet based on a hardware configuration dictionary. Models systolic arrays, vector units, register files, L1 SRAM, and interconnects, normalized to target process nodes. Supports verbose mode for component breakdown.

```python
from cost_model.cost_model import calc_compute_chiplet_area_mm2, calc_io_die_area_mm2
from design_space_exploration.dse import read_architecture_template

specs = read_architecture_template("configs/GA100.json")

# Basic area query
compute_area = calc_compute_chiplet_area_mm2(specs)
print(f"Compute chiplet area: {compute_area:.1f} mm²")

# Verbose mode returns a per-component breakdown
compute_area_v, core_breakdown, total_die_map = calc_compute_chiplet_area_mm2(
    specs, verbose=True
)
print(f"Total: {total_die_map['total_area']:.1f} mm²")
print(f"  Cores: {total_die_map['cores_area']:.1f} mm²")
print(f"  Crossbar: {total_die_map['crossbar_area']:.1f} mm²")
print(f"  SA per core: {core_breakdown['sa_area']:.2f} mm²")
print(f"  L1 SRAM per core: {core_breakdown['local_buffer_area']:.2f} mm²")

# IO die area (global buffer + memory PHY + controller + NVLink PHY + controller)
io_area = calc_io_die_area_mm2(specs)
print(f"IO die area: {io_area:.1f} mm²")
print(f"Total device area: {compute_area + io_area:.1f} mm²")
```

--------------------------------

### Define Tensor shape descriptor

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

Creates a lightweight descriptor for tensor shapes and data types used in symbolic shape propagation. Does not hold actual data.

```python
from software_model.utils import Tensor, data_type_dict

# data_type_dict keys: "int8", "fp16", "fp32"
# Create a batch of 8 sequences, each of length 2048, with hidden dim 12288, in fp16
x = Tensor([8, 2048, 12288], data_type_dict["fp16"])

print(x.shape)
print(x.size)
print(x.data_type.word_size)
```

--------------------------------

### Tensor

Source: https://context7.com/princetonuniversity/llmcompass/llms.txt

A lightweight descriptor for tensor shapes and data types. The `Tensor` class is used throughout the software model for symbolic shape propagation during graph construction and simulation. It does not hold actual data.

```APIDOC
## Tensor

### Description
A lightweight descriptor used throughout the software model to represent tensor shapes and data types. Does not hold actual data — it is used purely for symbolic shape propagation during graph construction and simulation.

### Method
```python
Tensor(shape: list[int], data_type: DataType)
```

### Parameters
#### Path Parameters
- **shape** (list[int]) - Required - A list of integers representing the dimensions of the tensor.
- **data_type** (DataType) - Required - The data type of the tensor elements. Use `data_type_dict` for available types.

### Request Example
```python
from software_model.utils import Tensor, data_type_dict

x = Tensor([8, 2048, 12288], data_type_dict["fp16"])
```

### Response
#### Success Response (Tensor)
- **x** (Tensor) - A `Tensor` object representing the shape and data type.
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.