### Run tt-metal Programming Example

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/installing

Executes a sample programming example to verify the tt-metal installation. This command runs an operation on the device using the ttnn module.

```python
python3 -m ttnn.examples.usage.run_op_on_device
```

--------------------------------

### Install TT-Installer Script

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/installing

Downloads and executes the TT-Installer script for a quick setup of Tenstorrent's software stack. It offers options to skip specific component installations like Podman and Metalium container. This script is recommended for most users.

```shell
curl -fsSL https://github.com/tenstorrent/tt-installer/releases/latest/download/install.sh -O
chmod +x install.sh
./install.sh --no-install-podman --no-install-metalium-container
```

--------------------------------

### Clone tt-metal Repository

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/installing

Clones the tt-metal repository and its submodules. This is the first step before building the library.

```sh
git clone https://github.com/tenstorrent/tt-metal.git --recurse-submodules
```

--------------------------------

### Set Up Development Environment for TT-NN Models

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/installing

Configures the environment for running TT-NN models, including setting the PYTHONPATH, installing development requirements, and ensuring optimal CPU performance by setting the governor to 'performance'. This is necessary for users working with models in the 'models/' directory.

```shell
export PYTHONPATH=$(pwd)
pip install -r tt_metal/python_env/requirements-dev.txt
sudo apt-get install cpufrequtils
sudo cpupower frequency-set -g performance
```

--------------------------------

### Install ttnn-visualizer via pip

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/tutorials/2025_dx_rework/ttnn_visualizer

Installs the ttnn-visualizer package using pip. This is a simple way to get the visualizer tool ready for use after building the necessary dependencies.

```bash
pip install ttnn-visualizer
```

--------------------------------

### Example of Test Infrastructure Setup in test_ttnn_functional_resnet50.py

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/demos

This snippet references an example Python file, test_ttnn_functional_resnet50.py, and specifically highlights the ResNet50TestInfra class as a model for organizing tests for device and end-to-end performance collection. This serves as a practical guide for developers setting up their own test infrastructure.

```python
# For the purpose of organizing your tests to be managed for collecting both device and end-to-end performance, please take a look at test_ttnn_functional_resnet50.py and how the ResNet50TestInfra class was setup.
```

--------------------------------

### Install TT-NN from PyPI

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/installing

Installs the TT-NN Python package using pip. This method provides quick access to TT-NN APIs and is suitable for users who prefer installing from pre-built binaries. It requires Linux with glibc 2.34 or newer.

```shell
pip install ttnn
```

--------------------------------

### Build tt-metal Library with Build Script

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/installing

Builds the tt-metal library using the provided build script. This is the recommended and simplest method for building the library.

```sh
./build_metal.sh
```

--------------------------------

### Set Environment Variables for Installation Verification

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/installing

Sets the TT_METAL_HOME and PYTHONPATH environment variables required to verify the tt-metal installation. TT_METAL_HOME should point to the tt-metal directory.

```sh
export TT_METAL_HOME=</path/to/your/tt-metal>
export PYTHONPATH="${TT_METAL_HOME}" # Same path
```

--------------------------------

### Build tt-metal Library Manually with CMake

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/installing

Manually builds the tt-metal library using CMake, offering more control over build options. Requires creating a build directory, configuring with CMake, and then using Ninja to build and install.

```bash
mkdir build
cd build
cmake .. -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebugInfo -DCMAKE_CXX_COMPILER=<your compiler>
ninja
ninja install # Installs to build directory by default, required for Python environment
```

--------------------------------

### Running a Python Tutorial Script

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/tutorials

Demonstrates how to execute a TT-NN tutorial script from the command line using Python 3. Ensure Python 3.10.12 or a compatible version is installed.

```bash
python3 --version
python3 example.py
```

--------------------------------

### Build TT-Metal Library using CMake and Ninja

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/installing

Manually builds the TT-Metal library using CMake and Ninja. This method is for users who require more control over the build process or have custom build configurations. It involves creating a build directory, configuring with CMake, and then using Ninja to build and install the library.

```bash
mkdir build
cd build
cmake .. -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebugInfo -DCMAKE_CXX_COMPILER=<your compiler>
ninja
ninja install
```

--------------------------------

### Install and Launch TTNN Visualizer (Python/Shell)

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/tutorials/2025_dx_rework/ttnn_visualizer

These commands install the TTNN Visualizer package using pip and then launch the visualizer application. This enables the exploration of profiling data. It requires internet access for installation and a compatible Python environment. The visualizer typically opens in a web browser.

```shell
pip install ttnn-visualizer
ttnn-visualizer
```

--------------------------------

### Configure TT-Installer with Specific Versions

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/installing

Installs Tenstorrent software using the TT-Installer script, allowing specification of exact versions for components like TT-SMI, firmware, and KMD. This provides control over dependency versions, crucial for specific hardware like Galaxy Wormhole 4U and Blackhole systems.

```shell
./install.sh \
  --smi-version=v3.0.17 \
  --fw-version=18.3.0 \
  --kmd-version=1.34 \
  --no-install-podman \
  --no-install-metalium-container
```

--------------------------------

### Launch TT-NN Visualizer Application

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/tutorials/2025_dx_rework/ttnn_visualizer

Launches the TT-NN Visualizer application, which starts a local server at http://localhost:8000. This server hosts the web interface for visualizing model profiling data.

```bash
ttnn-visualizer

```

--------------------------------

### Pull and Run TT-Metalium Docker Image

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/installing

Downloads the latest TT-Metalium release image from the Docker registry and runs it in an interactive container. This method provides a quick way to access TT-NN APIs and run AI models within a containerized environment, with direct access to Tenstorrent hardware devices.

```shell
docker pull ghcr.io/tenstorrent/tt-metal/tt-metalium-ubuntu-22.04-release-amd64:latest-rc
docker run -it --rm --device /dev/tenstorrent ghcr.io/tenstorrent/tt-metal/tt-metalium-ubuntu-22.04-release-amd64:latest-rc bash
```

--------------------------------

### Create and Activate tt-metal Python Virtual Environment

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/installing

Sets up the Python virtual environment for tt-metal and activates it. This script creates a new environment in './python_env' if PYTHON_ENV_DIR is not specified.

```sh
./create_venv.sh
source python_env/bin/activate
```

--------------------------------

### Python: ResNet50TestInfra Class Setup

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/demos

Demonstrates the setup of the ResNet50TestInfra class in Python, used for organizing tests for collecting device and end-to-end performance metrics. This serves as a reference for structuring similar tests.

```python
class ResNet50TestInfra:
    def __init__(self, ...):
        pass

    @pytest.mark.models_device_performance_bare_metal
    def test_device_performance(self):
        # Test logic for device performance
        pass

    @pytest.mark.models_performance_bare_metal
    def test_end_to_end_performance(self):
        # Test logic for end-to-end performance
        pass

    @pytest.mark.models_demo_test
    def test_demo_functionality(self):
        # Test logic for demo functionality
        pass
```

--------------------------------

### Install ttnn-visualizer Package

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/tutorials/2025_dx_rework/ttnn_visualizer

Installs the ttnn-visualizer package using pip. This is a required step to use the TT-NN Visualizer for analyzing model performance and memory usage.

```bash
pip install ttnn-visualizer

```

--------------------------------

### Launch TTNN Visualizer

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/tutorials/2025_dx_rework/ttnn_visualizer

Launches the TT-NN Visualizer tool. This command starts a local server, typically at http://localhost:8000, which you can access through a web browser to analyze your model's performance reports.

```bash
ttnn-visualizer
```

--------------------------------

### Set Python Virtual Environment Directory

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/installing

Optionally sets the directory for an existing Python virtual environment. If not set, a new environment will be created in './python_env'.

```sh
export PYTHON_ENV_DIR=<path_to_your_env_directory>
```

--------------------------------

### Register Example Operation with ttnn in C++

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/adding_new_ttnn_operation

This C++ code snippet demonstrates how to register the `ExampleDeviceOperation` with the `ttnn` framework. It uses `ttnn::register_operation` to make the operation accessible through the `ttnn::prim` namespace under the name `example`. This allows users to call the operation directly after it has been defined and compiled.

```cpp
// Register the operation with the ttnn::register_operation API to make it available to the user as ttnn::prim::example
namespace ttnn::prim {
constexpr auto example =
    ttnn::register_operation<"ttnn::prim::example", ttnn::operations::examples::ExampleDeviceOperation>();
}  // namespace ttnn::prim

```

--------------------------------

### TTNN Usage Examples (Python)

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/usage

This section includes various Python script examples for TTNN usage, such as `visualizer_example.py`, `falling_back_to_torch.py`, and `graph_capture.py`. These examples demonstrate specific functionalities like visualization, fallback mechanisms, and graph capturing.

```python
.. literalinclude:: ../../../../ttnn/ttnn/examples/usage/visualizer_example.py
   :language: python
```

```python
.. literalinclude:: ../../../../ttnn/ttnn/examples/usage/falling_back_to_torch.py
   :language: python
```

```python
.. literalinclude:: ../../../../ttnn/ttnn/examples/usage/graph_capture.py
   :language: python
```

--------------------------------

### Import TT-NN and Logging Libraries

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/tutorials/2025_dx_rework/ttnn_add_tensors

Imports the necessary TT-NN library for tensor operations and the loguru library for logging. These are essential first steps for any TT-NN script.

```python
import ttnn
from loguru import logger
```

--------------------------------

### Import Libraries for ttnn Convolution

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/tutorials/2025_dx_rework/ttnn_basic_conv

Imports the necessary libraries for PyTorch tensor operations, ttnn functionalities, and logging. These are fundamental for setting up and executing the convolution example.

```python
import torch
import ttnn
from loguru import logger
```

--------------------------------

### Device Management APIs

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/api

APIs for opening, closing, managing, and synchronizing TT Metal devices, as well as setting and getting the default device.

```APIDOC
## Device Management APIs

### `ttnn.open_device`

#### Description
Opens a device with the given device_id.

### Method
`ttnn.open_device`

### Endpoint
N/A (Function Call)

### Parameters
- **device_id** (int) - Required - The ID of the device to open.

### Response
(Returns a device object)

### `ttnn.close_device`

#### Description
Closes the device and removes it from the device cache.

### Method
`ttnn.close_device`

### Endpoint
N/A (Function Call)

### Parameters
- **device** (ttnn.Device) - Required - The device object to close.

### Response
(No return value)

### `ttnn.manage_device`

#### Description
Context manager for opening and closing a device.

### Method
`ttnn.manage_device`

### Endpoint
N/A (Context Manager)

### Parameters
- **device_id** (int) - Required - The ID of the device to manage.

### Response
(Yields a device object within the context)

### `ttnn.synchronize_device`

#### Description
Synchronizes the device with the host by waiting for all operations to complete.

### Method
`ttnn.synchronize_device`

### Endpoint
N/A (Function Call)

### Parameters
- **device** (ttnn.Device) - Required - The device to synchronize.

### Response
(No return value)

### `ttnn.SetDefaultDevice`

#### Description
Sets the default device to use for operations when inputs are not on the device.

### Method
`ttnn.SetDefaultDevice`

### Endpoint
N/A (Function Call)

### Parameters
- **device** (ttnn.Device) - Required - The device to set as default.

### Response
(No return value)

### `ttnn.GetDefaultDevice`

#### Description
Gets the default device to use for operations when inputs aren't on the device.

### Method
`ttnn.GetDefaultDevice`

### Endpoint
N/A (Function Call)

### Response
(Returns the default ttnn.Device object)

```

--------------------------------

### Define Example Device Operation Header

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/adding_new_ttnn_operation

This C++ header file defines the structure for an example device operation within the ttnn framework. It includes definitions for operation attributes, tensor arguments, return types, and the structure for single and multi-core implementations, including their create and override_runtime_arguments methods.

```cpp
// SPDX-FileCopyrightText: © 2023 Tenstorrent Inc.
//
// SPDX-License-Identifier: Apache-2.0

#pragma once

#include <optional>
#include <variant>

#include "ttnn/tensor/tensor.hpp"
#include "ttnn/core.hpp"
#include "ttnn/device_operation.hpp"
#include "ttnn/types.hpp"
#include "ttnn/decorators.hpp"

namespace ttnn::operations::examples {

struct ExampleDeviceOperation {
    // Define the operation attributes. This is it to store all variables needed by operations that aren't tensors
    struct operation_attributes_t {
        bool attribute;
        int some_other_attribute;
    };

    // Define the tensor arguments. This is it to store all tensors passed in and/or out of the operation
    // Tensor arguments don't need to be just input tensors, they can be output tensors, input/output tensors, optional
    // tensors, etc.
    struct tensor_args_t {
        // This example will use a tensor that can only be used as an input
        const Tensor& input_tensor;

        // However, the following examples show what else can be done with tensor_args_t

        // An example of the tensor that can be used for input/output or just for pre-allocated output
        // Tensor& io_tensor;

        // An example of an optional tensor
        // std::optional<Tensor> optional_output_tensor;

        // An example of a vector of tensors
        // std::vector<Tensor> vector_of_tensors;

        // An example of a tuple of tensors
        // std::tuple<Tensor, ...> tuple_of_tensors;

        // An example of a vector of optional tensors
        // std::vector<std::optional<Tensor>> vector_of_optional_tensors;

        // An example of a tuple of tensors
        // std::tuple<std::vector<std::optional<Tensor>>, std::optional<Tensor>> some_crazy_tuple_of_tensors;
    };

    // Define the return types for the spec(s) of the operation
    // Can be a single ttnn::TensorSpec, std::optional<ttnn::TensorSpec>, std::vector<ttnn::TensorSpec>,
    // std::tuple<ttnn::TensorSpec> etc.
    using spec_return_value_t = ttnn::TensorSpec;

    // Define the return types for the tensor(s) of the operation
    // Can be a single Tensor, std::optional<Tensor, ...>, std::vector<Tensor>, std::tuple<Tensor, ...> etc.
    using tensor_return_value_t = Tensor;

    // Note spec_return_value_t and tensor_return_value_t should follow the same pattern
    // i.e. if spec_return_value_t is a std::vector<std::optional<ttnn::TensorSpec>> then tensor_return_value_t should
    // be std::vector<std::optional<Tensor>>

    struct SingleCore {
        // Shared variables are the variables that are shared between the create and override_runtime_arguments methods
        struct shared_variables_t {
            tt::tt_metal::KernelHandle unary_reader_kernel_id;
            tt::tt_metal::KernelHandle unary_writer_kernel_id;
        };
        using cached_program_t = ttnn::device_operation::CachedProgram<shared_variables_t>;

        static cached_program_t create(
            const operation_attributes_t& operation_attributes,
            const tensor_args_t& tensor_args,
            tensor_return_value_t& tensor_return_value);

        static void override_runtime_arguments(
            cached_program_t& cached_program,
            const operation_attributes_t& operation_attributes,
            const tensor_args_t& tensor_args,
            tensor_return_value_t& tensor_return_value);
    };

    struct MultiCore {
        // Shared variables are the variables that are shared between the create and override_runtime_arguments methods
        struct shared_variables_t {
            tt::tt_metal::KernelHandle unary_reader_kernel_id;
            tt::tt_metal::KernelHandle unary_writer_kernel_id;
            std::size_t num_cores;
            std::size_t num_cores_y;
        };
        using cached_program_t = ttnn::device_operation::CachedProgram<shared_variables_t>;

        static cached_program_t create(
            const operation_attributes_t& operation_attributes,

```

--------------------------------

### Adding Demo Tests to Demonstration Scripts

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/demos

This snippet details how to add end-to-end demo tests to the appropriate shell scripts based on the target hardware. It guides users on selecting the correct script (e.g., run_demos_single_card_n150_tests.sh, run_t3000_demo_tests.sh) for their specific hardware to ensure the demo tests are properly integrated.

```shell
# Please add your test within run_demos_single_card_n150_tests.sh, run_demos_single_card_n300_tests.sh, and/or run_t3000_demo_tests.sh depending on the hardware you are targeting.
```

--------------------------------

### Define Example Device Operation in C++

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/adding_new_ttnn_operation

This C++ code defines the `ExampleDeviceOperation` class, which inherits from `ttnn::DeviceOperation`. It includes methods for selecting a program factory, validating operations, computing output tensor specifications, creating output tensors, and invoking the operation. This forms the core logic for a custom device operation in tt-metal.

```cpp
#include "example_device_operation.hpp"

namespace ttnn::operations::examples {

ExampleDeviceOperation::program_factory_t ExampleDeviceOperation::select_program_factory(
    const operation_attributes_t& operation_attributes, const tensor_args_t& tensor_args) {
    bool some_condition_based_on_operation_attributes_and_or_tensor_args = true;
    if (some_condition_based_on_operation_attributes_and_or_tensor_args) {
        return SingleCore{};
    }
    return MultiCore{};
}

void ExampleDeviceOperation::validate_on_program_cache_miss(
    const operation_attributes_t& attributes, const tensor_args_t& tensor_args) {}

void ExampleDeviceOperation::validate_on_program_cache_hit(
    const operation_attributes_t& attributes, const tensor_args_t& tensor_args) {}

ExampleDeviceOperation::spec_return_value_t ExampleDeviceOperation::compute_output_specs(
    const operation_attributes_t&, const tensor_args_t& tensor_args) {
    const auto& input_tensor = tensor_args.input_tensor;
    return TensorSpec(
        input_tensor.logical_shape(),
        tt::tt_metal::TensorLayout(
            input_tensor.dtype(), tt::tt_metal::PageConfig(input_tensor.layout()), MemoryConfig{}));
}

ExampleDeviceOperation::tensor_return_value_t ExampleDeviceOperation::create_output_tensors(
    const operation_attributes_t& operation_attributes, const tensor_args_t& tensor_args) {
    auto output_spec = compute_output_specs(operation_attributes, tensor_args);
    return create_device_tensor(output_spec, tensor_args.input_tensor.device());
}

std::tuple<ExampleDeviceOperation::operation_attributes_t, ExampleDeviceOperation::tensor_args_t>
ExampleDeviceOperation::invoke(const Tensor& input_tensor) {
    return {operation_attributes_t{true, 42}, tensor_args_t{input_tensor}};
}

}  // namespace ttnn::operations::examples

```

--------------------------------

### Example: C++ Device Operation for TT-NN

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/adding_new_ttnn_operation

An example of implementing a device operation in C++ for TT-NN. This specific example demonstrates a simple operation that copies an input tensor to an output tensor on the device.

```cpp
struct ExampleOperation {
    // Method to create output tensors
    // Method to define the program to run on the device
};
// Registering the operation would follow this implementation.
```

--------------------------------

### Shell: Device Performance Test Integration

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/demos

Example of how to add a test to the `run_device_perf_models()` function in `run_performance.sh` for collecting device performance metrics. This script is part of the CI pipeline.

```bash
# Example addition to run_performance.sh

run_device_perf_models() {
  # ... existing logic ...

  # Add your test command here, ensuring it's marked with @pytest.mark.models_device_performance_bare_metal
  pytest tests/path/to/your_resnet50_tests.py::ResNet50TestInfra::test_device_performance

  # ... rest of the logic ...
}
```

--------------------------------

### Open Tenstorrent Device

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/tutorials/2025_dx_rework/ttnn_add_tensors

Opens a connection to a Tenstorrent device, identified by its device ID. This is a prerequisite for performing any operations on the Tenstorrent hardware.

```python
# Open the Device
device = ttnn.open_device(device_id=0)
```

--------------------------------

### Run Operation on Device using TT-NN

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/usage

Shows how to execute a TT-NN operation on the specified device. This example covers basic tensor creation and applying an operation like exponentiation. Ensure the device is opened before use and closed afterward.

```python
import torch
import ttnn

# Assuming device is already opened and configured
# device = ttnn.open_device(device_id=0)

torch_input_tensor = torch.rand(32, 32, dtype=torch.float32)
input_tensor = ttnn.from_torch(torch_input_tensor, dtype=ttnn.bfloat16, layout=ttnn.TILE_LAYOUT, device=device)

# Perform an operation on the device
output_tensor = ttnn.exp(input_tensor)

# Convert back to torch tensor to view results if needed
# torch_output_tensor = ttnn.to_torch(output_tensor)

# print(f"Output tensor shape: {output_tensor.shape}")

# ttnn.close_device(device)
```

--------------------------------

### Set TTNN Configuration Path (Bash)

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/tutorials/2025_dx_rework/ttnn_visualizer

This bash command sets the environment variable TTNN_CONFIG_PATH to point to the generated setup file. This is required for TT-NN to load the specified configuration for memory profiling. Remember to export other required global variables from tt-metal.

```bash
export TTNN_CONFIG_PATH=path/to/vis.setup
```

--------------------------------

### Trace and Visualize Operation Graph

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/usage

This example demonstrates how to trace the graph of operations performed by TT-NN and visualize it as an SVG file. This is useful for understanding the computation flow and identifying potential optimizations. Requires PyTorch and a Tenstorrent device. Note: This snippet is under construction.

```python
import torch
import ttnn

device_id = 0
device = ttnn.open_device(device_id=device_id)

with ttnn.tracer.trace():
    torch_input_tensor = torch.rand(32, 32, dtype=torch.float32)
    input_tensor = ttnn.from_torch(torch_input_tensor, dtype=ttnn.bfloat16, layout=ttnn.TILE_LAYOUT, device=device)
    output_tensor = ttnn.exp(input_tensor)
    torch_output_tensor = ttnn.to_torch(output_tensor)
ttnn.tracer.visualize(torch_output_tensor, file_name="exp_trace.svg")

ttnn.close_device(device)

```

--------------------------------

### Create and Initialize TT-NN Tensors

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/tutorials/2025_dx_rework/ttnn_add_tensors

Creates two TT-NN tensors with a shape of (32, 32), initialized with float32 values 1.0 and 2.0 respectively, using TILE_LAYOUT. The tensors are configured to reside on the specified Tenstorrent device. This section also logs the created tensors.

```python
# Create two TT-NN tensors with TILE_LAYOUT
tt_tensor1 = ttnn.full(
    shape=(32, 32),
    fill_value=1.0,
    dtype=ttnn.float32,
    layout=ttnn.TILE_LAYOUT,
    device=device,
)
tt_tensor2 = ttnn.full(
    shape=(32, 32),
    fill_value=2.0,
    dtype=ttnn.float32,
    layout=ttnn.TILE_LAYOUT,
    device=device,
)
logger.info("Input tensors:")
logger.info(tt_tensor1)
logger.info(tt_tensor2)
```

--------------------------------

### Install tt-metalium with Conda

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/installing

Installs the tt-metalium package using conda, creating a new environment named 'metalium' with Python 3.10. This command is for Linux distributions with glibc 2.34 or newer.

```sh
conda create -n metalium python=3.10 tt-metalium -c conda-forge
```

--------------------------------

### ttnn::prim::example Registration

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/adding_new_ttnn_operation

Demonstrates how the ExampleDeviceOperation is registered with TTNN, making it accessible to users via the `ttnn::prim::example` interface.

```APIDOC
## `ttnn::prim::example` Registration

### Description
This section describes the registration process of the `ExampleDeviceOperation` within the `ttnn::prim` namespace. This registration makes the operation callable by users through the `ttnn::prim::example` alias.

### Method
`ttnn::register_operation<"ttnn::prim::example", ttnn::operations::examples::ExampleDeviceOperation>()`

### Endpoint
Not applicable (this is a C++ API registration, not a REST endpoint).

### Parameters
- `"ttnn::prim::example"` (string): The unique name or alias under which the operation will be available to users.
- `ttnn::operations::examples::ExampleDeviceOperation`: The C++ class implementing the device operation.

### Request Example
```cpp
// The registration happens internally in the TTNN library.
// Users access the operation via:
tensor_return_value_t result = ttnn::prim::example(input_tensor, operation_parameters);
```

### Response
Not applicable (this is a C++ API registration).

### Error Handling
Errors during registration would typically manifest as compilation errors or runtime issues if the operation class itself has problems during instantiation or if the name is already taken. Specific error handling details depend on the `ttnn::register_operation` implementation.
```

--------------------------------

### ExampleDeviceOperation API

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/adding_new_ttnn_operation

Core methods for the ExampleDeviceOperation, handling program factory selection, validation, output specification computation, tensor creation, and user invocation.

```APIDOC
## ExampleDeviceOperation

### Description
This class defines a sample device operation within the TTNN framework. It includes methods for managing program execution, validating operations, computing output tensor specifications, creating output tensors, and handling user invocations.

### Methods

- `select_program_factory`: Determines whether to use `SingleCore` or `MultiCore` execution based on operation attributes and tensor arguments.
- `validate_on_program_cache_miss`: Performs validation when a program is not found in the cache.
- `validate_on_program_cache_hit`: Performs validation when a program is found in the cache.
- `compute_output_specs`: Calculates the specifications for the output tensors based on input tensor properties.
- `create_output_tensors`: Creates the actual output tensors on the device.
- `invoke`: The primary method for users to call the operation, mapping user arguments to internal operation attributes and tensor arguments.

### Path Parameters
None

### Query Parameters
None

### Request Body
(Not applicable for the `invoke` method as it takes a `Tensor` object directly)

### Request Example
```cpp
// Example of invoking the operation:
tensor_return_value_t output_tensor = ttnn::prim::example(input_tensor);
```

### Response
#### Success Response (200)
- `std::tuple<operation_attributes_t, tensor_args_t>`: Returned by the `invoke` method, containing the operation's attributes and arguments.

#### Response Example
```json
{
  "operation_attributes": {
    "some_attribute": true,
    "another_attribute": 42
  },
  "tensor_args": {
    "input_tensor": "<Tensor object details>"
  }
}
```

### Error Handling
Error handling is managed internally by the TTNN framework. Specific errors related to validation or tensor creation would typically be exceptions or specific return codes, depending on the implementation details not shown here.
```

--------------------------------

### Close Tenstorrent Device

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/tutorials/2025_dx_rework/ttnn_add_tensors

Closes the connection to the Tenstorrent device, releasing resources. This is a standard practice for cleaning up after computations on the hardware.

```python
ttnn.close_device(device)
logger.info("Device closed successfully.")
```

--------------------------------

### Convert Tensors Between PyTorch and TT-NN

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/usage

Demonstrates how to convert PyTorch tensors to TT-NN tensors and back. This is a fundamental operation for using TT-NN with PyTorch models. Ensure PyTorch is installed.

```python
# SPDX-FileCopyrightText: © 2024 Tenstorrent Inc.

# SPDX-License-Identifier: Apache-2.0


import torch
import ttnn

torch_input_tensor = torch.zeros(2, 4, dtype=torch.float32)
tensor = ttnn.from_torch(torch_input_tensor, dtype=ttnn.bfloat16)
torch_output_tensor = ttnn.to_torch(tensor)

```

--------------------------------

### Shell: Demo Test Script Integration

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/demos

Illustrates where to add demo tests within shell scripts like `run_demos_single_card_n150_tests.sh` for integrating end-to-end demo functionality tests into the CI pipeline.

```bash
# Example addition to run_demos_single_card_n150_tests.sh

# ... existing demo test commands ...

# Add your demo test execution command here
pytest tests/path/to/your_resnet50_tests.py::ResNet50TestInfra::test_demo_functionality
```

--------------------------------

### Tensor Creation API

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/api

APIs for creating tensors with various initializations.

```APIDOC
## Tensor Creation

### `ttnn.arange(start: int, end: int, step: int = 1, device: tt_device, dtype: tt_dtype, layout: tt_memory_layout)`

Creates a tensor with values ranging from `start` to `end` with a given `step`.

### `ttnn.empty(shape: Shape, device: tt_device, dtype: tt_dtype, layout: tt_memory_layout)`

Creates an uninitialized tensor of the specified shape, device, dtype, and layout.

### `ttnn.empty_like(input_tensor: Tensor)`

Creates an uninitialized tensor with the same shape, device, dtype, and layout as the input tensor.

### `ttnn.zeros(shape: Shape, device: tt_device, dtype: tt_dtype, layout: tt_memory_layout)`

Creates a tensor of zeros with the specified shape, device, dtype, and layout.

### `ttnn.zeros_like(input_tensor: Tensor)`

Creates a tensor of zeros with the same shape, device, dtype, and layout as the input tensor.

### `ttnn.ones(shape: Shape, device: tt_device, dtype: tt_dtype, layout: tt_memory_layout)`

Creates a tensor of ones with the specified shape, device, dtype, and layout.

### `ttnn.ones_like(input_tensor: Tensor)`

Creates a tensor of ones with the same shape, device, dtype, and layout as the input tensor.

### `ttnn.full(shape: Shape, value: float, device: tt_device, dtype: tt_dtype, layout: tt_memory_layout)`

Creates a tensor filled with a specified value.

### `ttnn.full_like(input_tensor: Tensor, value: float)`

Creates a tensor filled with a specified value, with the same shape, device, dtype, and layout as the input tensor.

### `ttnn.rand(shape: Shape, device: tt_device, dtype: tt_dtype, layout: tt_memory_layout)`

Creates a tensor with random values.
```

--------------------------------

### Softmax Program Configurations

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/api

Program configuration classes for Softmax operations.

```APIDOC
## ttnn.SoftmaxProgramConfig

### Description
Base program configuration class for Softmax operations. Used to configure execution details.

### Method
`ttnn.SoftmaxProgramConfig` (Class)

## ttnn.SoftmaxDefaultProgramConfig

### Description
Default program configuration for Softmax operations. Provides standard settings.

### Method
`ttnn.SoftmaxDefaultProgramConfig` (Class)

## ttnn.SoftmaxShardedMultiCoreProgramConfig

### Description
Multi-core sharded program configuration for Softmax operations. Optimized for sharded execution on multiple cores.

### Method
`ttnn.SoftmaxShardedMultiCoreProgramConfig` (Class)

### Parameters (Applicable to sharded configs)
- **sharded_input_halo_config** - Configuration for input halo in sharded execution.
- **sharded_output_halo_config** - Configuration for output halo in sharded execution.
- **compute_with_storage_types** - Data types for compute with storage.
- **output_sharding_factor** - Factor for output sharding.
- **num_cores_to_sum_reducer** - Number of cores for the sum reducer in sharded execution.
```

--------------------------------

### Close Tenstorrent Device

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/tutorials/2025_dx_rework/ttnn_mlp_inference_mnist

Closes the connection to the Tenstorrent device, releasing associated resources. This is a necessary cleanup step after computation is complete. It requires a device object as input.

```python
ttnn.close_device(device)

```

--------------------------------

### Normalization Program Configs

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/api

Configuration classes for softmax operations.

```APIDOC
## Normalization Program Configs

### Description
Configuration classes for customizing the behavior of softmax operations, including sharded multi-core execution.

### Classes
- `ttnn.SoftmaxProgramConfig`
- `ttnn.SoftmaxDefaultProgramConfig`
- `ttnn.SoftmaxShardedMultiCoreProgramConfig
```

--------------------------------

### Enable Program Cache in TT-NN

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/usage

Demonstrates how to enable the program cache feature in TT-NN. This can improve performance by caching compiled programs for reuse. The snippet shows the basic setup and execution flow.

```python
import torch
import ttnn

# Assuming device is already opened and configured
# device = ttnn.open_device(device_id=0)

# Enable program cache (often implicitly handled or configured via flags/env vars, 
# but this snippet assumes a direct usage pattern if available or related setup)
# Note: Actual cache enabling might be more involved or set globally.
# This example is a placeholder for demonstrating the concept.

torch_input_tensor = torch.rand(32, 32, dtype=torch.float32)
input_tensor = ttnn.from_torch(torch_input_tensor, dtype=ttnn.bfloat16, layout=ttnn.TILE_LAYOUT, device=device)

# Running an operation that would benefit from caching
output_tensor = ttnn.exp(input_tensor)

# The program cache would store the compiled kernel for ttnn.exp
# on this device with these input properties for potential reuse.

# ttnn.close_device(device)
```

--------------------------------

### Device Management API

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/index

APIs for managing Tenstorrent devices, including opening, closing, synchronizing, and setting default devices.

```APIDOC
## Device Management API

### Description
This section details the TT-NN APIs for managing Tenstorrent devices.

### APIs

- **ttnn.open_device(device_id: int, num_loops: int = 1, io_timeout_ns: int = 1000000000)**
  - Opens a Tenstorrent device.

- **ttnn.close_device(device: ttnn.Device)**
  - Closes a Tenstorrent device.

- **ttnn.manage_device(device: ttnn.Device)**
  - Manages a Tenstorrent device.

- **ttnn.synchronize_device(device: ttnn.Device)**
  - Synchronizes operations on a Tenstorrent device.

- **ttnn.SetDefaultDevice(device: ttnn.Device)**
  - Sets the default Tenstorrent device.

- **ttnn.GetDefaultDevice() -> ttnn.Device**
  - Retrieves the default Tenstorrent device.
```

--------------------------------

### Debug Intermediate Tensors in TT-NN

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/usage

Provides an example of how to debug intermediate tensors within TT-NN operations. This is crucial for understanding data flow and identifying issues during complex computations. Requires an opened device.

```python
import torch
import ttnn

# Assuming device is already opened and configured
# device = ttnn.open_device(device_id=0)

torch_input_tensor = torch.rand(32, 32, dtype=torch.float32)
input_tensor = ttnn.from_torch(torch_input_tensor, dtype=ttnn.bfloat16, layout=ttnn.TILE_LAYOUT, device=device)

# Example: Perform an operation and potentially inspect intermediate results
# The exact mechanism for debugging intermediate tensors might involve specific flags
# or wrapper functions provided by TT-NN for debugging purposes.

# This snippet is conceptual, as direct intermediate tensor access often requires 
# specific debugging APIs or environment configurations.

output_tensor = ttnn.exp(input_tensor) # Replace with a more complex operation if needed for debugging

# Example of how one might conceptually debug (actual API may differ):
# debug_info = ttnn.get_debug_info(output_tensor)
# print(debug_info)

# ttnn.close_device(device)
```

--------------------------------

### Device Management API

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/api

APIs for managing the device, including opening, closing, synchronizing, and retrieving default device information.

```APIDOC
## Device Management

### `ttnn.open_device(device_id: int, num_queues: int)`

Opens a device with the specified ID and number of queues.

### `ttnn.close_device(device_id: int)`

Closes the device with the specified ID.

### `ttnn.manage_device(device_id: int)`

Manages the specified device.

### `ttnn.synchronize_device(device_id: int)`

Synchronizes the specified device.

### `ttnn.SetDefaultDevice(device_id: int)`

Sets the default device to the one with the specified ID.

### `ttnn.GetDefaultDevice()`

Returns the default device.

### `ttnn.format_input_tensor(input_tensor: Tensor, device: tt_device, layout: tt_memory_layout, padding: Tuple[int, int, int, int], pad_value: int = 0)`

Formats an input tensor according to the specified device, layout, and padding.

### `ttnn.format_output_tensor(output_tensor: Tensor, device: tt_device, layout: tt_memory_layout, padding: Tuple[int, int, int, int], pad_value: int = 0)`

Formats an output tensor according to the specified device, layout, and padding.

### `ttnn.pad_to_tile_shape(input_tensor: Tensor, padding_val: int = 0)`

Pads the input tensor to a tile shape.
```

--------------------------------

### Build tt-metal with Profiler Enabled

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/tutorials/2025_dx_rework/ttnn_visualizer

This command builds the tt-metal project with the profiler enabled, which is a prerequisite for generating profiling data required by the TT-NN Visualizer.

```bash
./build_metal.sh -p

```

--------------------------------

### Perform Tensor Addition and Log Result

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/tutorials/2025_dx_rework/ttnn_add_tensors

Performs an element-wise addition operation between the two previously created TT-NN tensors on the device and logs the resulting tensor. This demonstrates a fundamental arithmetic operation within TT-NN.

```python
# Perform eltwise addition on the device
tt_result = ttnn.add(tt_tensor1, tt_tensor2)

# Log output tensor
logger.info("Output tensor:")
logger.info(tt_result)
```

--------------------------------

### Convert to and from PyTorch Tensor using TT-NN

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/usage

Demonstrates how to convert PyTorch tensors to TT-NN tensors for device operations and back. This is essential for integrating TT-NN with existing PyTorch workflows. Ensure PyTorch is installed if using wheel or Docker releases.

```python
import ttnn
import torch

# Assuming device is already opened and configured
# device = ttnn.open_device(device_id=0)

# Example: Convert torch tensor to TT-NN tensor
torch_tensor = torch.rand(3, 3, dtype=torch.float32)
input_tensor = ttnn.from_torch(torch_tensor, dtype=ttnn.bfloat16, layout=ttnn.TILE_LAYOUT, device=device)

# Example: Convert TT-NN tensor back to torch tensor
output_tensor = ttnn.to_torch(input_tensor)

# print(f"Input TT-NN Tensor: {input_tensor}")
# print(f"Output Torch Tensor: {output_tensor}")

# ttnn.close_device(device)
```

--------------------------------

### Create and Display TTNN Tensor (Python)

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/tutorials/2025_dx_rework/ttnn_add_tensors

This snippet demonstrates the creation of a TTNN tensor with a specific shape and data type. It then shows how the tensor's contents and properties are represented in the output. Dependencies include the ttnn library.

```python
ttnn.Tensor([[ 2.00000,  2.00000,  ...,  2.00000,  2.00000]], shape=Shape([32, 32]), dtype=DataType::FLOAT32, layout=Layout::TILE)
Output tensor:
ttnn.Tensor([[ 3.00000,  3.00000,  ...,  3.00000,  3.00000],
            [ 3.00000,  3.00000,  ...,  3.00000,  3.00000],
            ...,
            [ 3.00000,  3.00000,  ...,  3.00000,  3.00000],
            [ 3.00000,  3.00000,  ...,  3.00000,  3.00000]], shape=Shape([32, 32]), dtype=DataType::FLOAT32, layout=Layout::TILE)
```

--------------------------------

### Import Libraries for CNN Inference with TT-NN

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/tutorials/2025_dx_rework/ttnn_simplecnn_inference

Imports essential Python libraries for image classification tasks. This includes `os` for file operations, `torch` and `torchvision` for dataset handling and model loading, `ttnn` for Tenstorrent's neural network operations, and `loguru` for logging. These libraries are crucial for setting up the environment, preprocessing data, and interacting with Tenstorrent hardware.

```python
import os
import torch
import torchvision
import torchvision.transforms as transforms
import ttnn
from loguru import logger
```

--------------------------------

### Build tt-metal with Profiler Enabled

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/tutorials/2025_dx_rework/ttnn_visualizer

This command builds the tt-metal project with the profiler enabled, which is a prerequisite for generating profiling data required by the TT-NN Visualizer. Ensure you have cloned the tt-metal repository before running this command.

```bash
./build_metal.sh -p
```

--------------------------------

### Memory Configuration API

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/api

API for creating sharded memory configurations.

```APIDOC
## Memory Configuration

### `ttnn.create_sharded_memory_config(batch_size: int, num_shards: int, shard_batch_size: int, shard_memory_layout: tt_memory_layout)`

Creates a sharded memory configuration.
```

--------------------------------

### Enable and Utilize Program Cache

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/ttnn/usage

Demonstrates how TT-NN's program cache improves performance by compiling and caching operations on the first run, leading to significantly faster subsequent executions. This example measures and prints the duration of both runs. Requires PyTorch and a Tenstorrent device.

```python
# SPDX-FileCopyrightText: © 2024 Tenstorrent Inc.

# SPDX-License-Identifier: Apache-2.0


import torch
import ttnn
import time

device_id = 0
device = ttnn.open_device(device_id=device_id)

torch_input_tensor = torch.rand(2, 4, dtype=torch.float32)
input_tensor = ttnn.from_torch(torch_input_tensor, dtype=ttnn.bfloat16, layout=ttnn.TILE_LAYOUT, device=device)

# Running the first time will compile the program and cache it
start_time = time.time()
output_tensor = ttnn.exp(input_tensor)
torch_output_tensor = ttnn.to_torch(output_tensor)
end_time = time.time()
duration = end_time - start_time
print(f"duration of the first run: {duration}")
# stdout: duration of the first run: 0.6391518115997314

# Running the subsequent time will use the cached program
start_time = time.time()
output_tensor = ttnn.exp(input_tensor)
torch_output_tensor = ttnn.to_torch(output_tensor)
end_time = time.time()
duration = end_time - start_time
print(f"duration of the second run: {duration}")
# stdout: duration of the subsequent run: 0.0007393360137939453

ttnn.close_device(device)

```

--------------------------------

### Test BERT Intermediate Layer with ttnn

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/converting_torch_model_to_ttnn

This Python code defines a pytest test function for the bert_intermediate implementation. It uses torch and transformers for model setup and comparison, and ttnn for the optimized computation. The test verifies the output against a PyTorch reference using PCC.

```python
import pytest
import torch
import transformers

import ttnn
import ttnn_bert

from models.common.utility_functions import torch_random
from tests.ttnn.utils_for_testing import assert_with_pcc

@pytest.mark.parametrize("model_name", ["phiyodr/bert-large-finetuned-squad2"])
@pytest.mark.parametrize("batch_size", [1])
@pytest.mark.parametrize("sequence_size", [384])
def test_bert_intermediate(device, model_name, batch_size, sequence_size):
    torch.manual_seed(0)

    config = transformers.BertConfig.from_pretrained(model_name)
    model = transformers.models.bert.modeling_bert.BertIntermediate(config).eval()

    torch_hidden_states = torch_random((batch_size, sequence_size, config.hidden_size), -0.1, 0.1)
    torch_output = model(torch_hidden_states)

    parameters = preprocess_model_parameters(
        initialize_model=lambda: model,
        device=device, # Device to put the parameters on
        custom_preprocessor=ttnn_bert.custom_preprocessor, # Use custom_preprocessor to set ttnn.bfloat8_b data type for the weights and biases
    )

    hidden_states = ttnn.from_torch(torch_hidden_states, dtype=ttnn.bfloat16, layout=ttnn.TILE_LAYOUT, device=device)
    output = ttnn_bert.bert_intermediate(
        hidden_states,
        parameters=parameters,
    )
    output = ttnn.to_torch(output)

    assert_with_pcc(torch_output, output.to(torch_output.dtype), 0.999)

```

--------------------------------

### Initialize TT-NN Device

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/tutorials/2025_dx_rework/ttnn_multihead_attention

This snippet initializes the TT-NN device for computation. It imports necessary libraries like `os`, `time`, `torch`, `ttnn`, and `loguru`. A specific device ID is opened to prepare for tensor operations.

```python
import os
import time
import torch
import ttnn
from loguru import logger

torch.manual_seed(0)

device_id = 0
device = ttnn.open_device(device_id=device_id)
```

--------------------------------

### Operation Hooks

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/api

Utilities for registering pre and post operation hooks.

```APIDOC
## Operation Hooks

### Description
This section provides utilities for registering custom hooks to be executed before or after specific operations.

### Functions
- `ttnn.register_pre_operation_hook`
- `ttnn.register_post_operation_hook
```

--------------------------------

### Run Profiling with Tracy

Source: https://docs.tenstorrent.com/tt-metal/latest/ttnn/_sources/ttnn/tutorials/2025_dx_rework/ttnn_visualizer

This command executes a specific Pytest test case for the YOLOv4 model using the tracy profiler. It generates detailed performance data for operations and device usage. The output includes CSV files and a Tracy file, which are essential for the TT-NN Visualizer.

```bash
python -m tracy -p -r -v -m pytest models/demos/yolov4/tests/pcc/test_ttnn_yolov4.py::test_yolov4[0-pretrained_weight_true-0]
```