PyNVMe3

https://pynv.me/
Admin
PyNVMe3 is a fast, open, and extensible NVMe SSD test platform that uses Python with a user-space...
Tokens:101,114
Snippets:1,470
Trust Score:5.7
Update:4 months ago
Show doc for...
Tokens:
Raw
### Install and Enable SSH Server

Source: https://pynv.me/ssd/pynvme3-developers-guide

Installs the OpenSSH server on Ubuntu, starts the service, enables it to start on boot, and configures the firewall to allow SSH connections.

```bash
sudo apt update
sudo apt install -y openssh-server
```

```bash
sudo systemctl start ssh
sudo systemctl enable ssh
```

```bash
sudo systemctl status ssh
```

```bash
sudo ufw allow ssh
sudo ufw enable
sudo ufw status
```

--------------------------------

### Start QuestDB service

Source: https://pynv.me/ssd/pynvme3-platforms

Starts the pre-installed QuestDB service on the F6 Test Server. This command is used after a system reboot to enable monitoring data storage.

```bash
./questdb/bin/questdb.sh start
```

--------------------------------

### Navigate to PyNVMe3 Directory

Source: https://pynv.me/ssd/pynvme3-developers-guide

Change the current directory to the PyNVMe3 installation path. This is a prerequisite for subsequent setup steps.

```bash
cd /usr/local/PyNVMe3/
```

--------------------------------

### Controller List Setup

Source: https://pynv.me/ssd/pynvme3-api

Initializes a buffer to hold a list of controllers, specifying parameters for the list.

```python
ctrl_buf = Buffer(4096)
ctrl_buf.set_controller_list(1, 2, 3)
```

--------------------------------

### PyNVMe3 Hello World Example

Source: https://pynv.me/ssd/pynvme3-design

Demonstrates asynchronous NVMe write and read operations using ns.cmd, including callback chaining and explicit synchronization with waitdone().

```python
import time
import pytest
import logging

import nvme as d

def test_hello_world(nvme0, nvme0n1, qpair, verify):
    # allocate DMA-capable buffers
    read_buf = d.Buffer(512)
    write_buf = d.Buffer(512)
    write_buf[10:21] = b'hello world'  # insert test pattern

    # callback triggered when the write completes
    def write_cb(cdw0, status):
        nvme0n1.read(qpair, read_buf, 0, 1)

    # issue the write; the callback submits the read
    nvme0n1.write(qpair, write_buf, 0, 1, cb=write_cb)

    # verify data pattern before and after
    assert read_buf[10:21] != b'hello world'
    qpair.waitdone(2)  # wait for both write and read to complete
    assert read_buf[10:21] == b'hello world'

```

--------------------------------

### Execute All Tests

Source: https://pynv.me/ssd/pynvme3-developers-guide

Run all default conformance tests. This is the basic command to start the test suite.

```bash
make test
```

--------------------------------

### Buffer Memory Layout Example

Source: https://pynv.me/ssd/pynvme3-api

Illustrates the relationship between length, offset, and size in a PyNVMe3 Buffer.

```text
 0             offset                             length
 |===============|===================================|
                 |<========= size =========>|
```

--------------------------------

### Install PyNVMe3 using pip3

Source: https://pynv.me/ssd/pynvme3-developers-guide

Installs the PyNVMe3 package using pip3. Ensure the specified .tar.gz file is available in the current directory or provide the correct path.

```bash
sudo pip3 install PyNVMe3-xx.yy.z.tar.gz
```

--------------------------------

### 4K Full Disk Random Write Example

Source: https://pynv.me/ssd/pynvme3-io-models

Shows how to execute a 4K full disk random write operation for a specified duration using pynvme's ioworker. This example sets a read percentage of 0 to ensure only writes are performed.

```python
def test_ioworker_qpair(nvme0n1):
    nvme0n1.ioworker(lba_random=True,
                     io_size=8,
                     read_percentage=0,
                     time=3600).start().close()
```

--------------------------------

### Setup and Run VF Namespace Tests

Source: https://pynv.me/ssd/pynvme3-platforms

These commands set up the environment with virtual functions and then execute specific test utilities for creating namespaces and initializing virtual functions. Ensure virtual functions are enabled on the DUT.

```bash
make setup numvfs=5
make test TESTS=./scripts/test_utilities.py::test_create_ns_attach_vf
make test TESTS=./scripts/test_utilities.py::test_vf_init
```

--------------------------------

### IOWorker IO Size Examples

Source: https://pynv.me/ssd/pynvme3-io-models

Demonstrates different configurations for io_size: fixed size, list of sizes, and mixed sizes with percentages. Each worker runs for 5 seconds.

```python
def test_ioworker_io_size(nvme0n1):
    nvme0n1.ioworker(io_size=8,
                     time=5).start().close()
    nvme0n1.ioworker(io_size=[8, 16, 256],
                     time=5).start().close()
    nvme0n1.ioworker(io_size={8:50, 16:25, 256:25},
                     time=5).start().close()
```

--------------------------------

### SPDM Get Capabilities with Invalid Sequence

Source: https://pynv.me/ssd/pynvme3-platforms

Example test function demonstrating an SPDM GET_CAPABILITIES request with an invalid sequence, expecting a UserWarning for an SPDM ERROR response.

```python
def test_spdm_get_capabilities_multiple(spdm):
    """ Test SPDM GET_CAPABILITIES request with an invalid sequence. """
    spdm.get_version().receive()
    spdm.get_capabilities().receive()
    with pytest.warns(UserWarning, match="SPDM ERROR response, code: 0x04"):
        spdm.get_capabilities().receive()

```

--------------------------------

### Basic Qpair Creation and Deletion

Source: https://pynv.me/ssd/pynvme3-api

Illustrates how to create a queue pair (Qpair) with a specified depth and then explicitly delete it to free resources. This is a fundamental step before submitting I/O commands.

```python
def test_qpair_basic(nvme0):
    # Create a Qpair with depth 16
    qpair = Qpair(nvme0, depth=16)
    # Use it with I/O commands...
    qpair.delete()  # Cleanup
```

--------------------------------

### IOWorker LBA Alignment Example

Source: https://pynv.me/ssd/pynvme3-io-models

Configures an IOWorker to issue 4KiB random reads, ensuring each IO starts on a 4KiB boundary using lba_align=8. The worker runs for 5 seconds.

```python
def test_ioworker_lba_align(nvme0n1):
    # Issue 4KiB random reads; each IO starts on a 4KiB boundary.
    nvme0n1.ioworker(io_size=8,        # 8 LBAs = 4KiB
                     lba_align=8,      # align starts to 4KiB boundaries
                     lba_random=True,
                     time=5).start().close()
```

--------------------------------

### Parameterize Test Cases with Pytest

Source: https://pynv.me/ssd/pynvme3-developers-guide

Use pytest.mark.parametrize to run tests with multiple combinations of input values, improving test coverage. This example parameterizes LBA start and count for write operations.

```python
import pytest

@pytest.mark.parametrize("lba_start", [0, 1, 8, 32])
@pytest.mark.parametrize("lba_count", [1, 8, 9, 32])
def test_write_lba(nvme0, nvme0n1, qpair, lba_start, lba_count):
    buf = Buffer(512 * lba_count)
    nvme0n1.write(qpair, buf, lba_start, lba_count).waitdone()

```

--------------------------------

### Update Ubuntu and Install pip3

Source: https://pynv.me/ssd/pynvme3-developers-guide

Updates the Ubuntu package list and installed packages, then installs pip3, the package installer for Python.

```bash
sudo apt update
sudo apt upgrade
```

```bash
sudo apt install -y python3-pip
```

--------------------------------

### Uninstall and Remove Previous PyNVMe3 Installation

Source: https://pynv.me/ssd/pynvme3-developers-guide

Removes any existing PyNVMe3 installation using pip3 and then deletes the installation directory.

```bash
sudo pip3 uninstall PyNVMe3
sudo rm -rf /usr/local/PyNVMe3
```

--------------------------------

### Simplified Metamode Read Command Example

Source: https://pynv.me/ssd/pynvme3-io-models

This script re-implements the read IO process using a simplified metamode interface, demonstrating a more concise way to send common IOs.

```python
def test_metamode_example(nvme0):
    # Create an IOCQ and IOSQ
    cq = IOCQ(nvme0, 1, 10, PRP(10*16))
    sq = IOSQ(nvme0, 1, 10, PRP(10*64), cq=cq)

    # Step1: Host writes command to SQ Entry
    sq.read(cid=0, nsid=1, lba=0, lba_count=1, prp1=PRP(4096))

    # Step2: The host updates the SQ tail doorbell register to notify the SSD that a new command is pending.
    sq.tail = 1

    # Step7: Host processes Completion entry
    cq.waitdone(1)

    # Step8: The host writes the CQ head doorbell to release the completion entry
    cq.head = 1

    # print first CQE's status field
    logging.info(cq[0].status)
```

--------------------------------

### Install pip3

Source: https://pynv.me/ssd/user-guide

Installs the pip3 package manager for Python 3 on Ubuntu systems.

```bash
sudo apt install -y python3-pip
```

--------------------------------

### Setup and Run SRIOV Test

Source: https://pynv.me/ssd/pynvme3-platforms

Use this command to set up the environment with a specified number of virtual functions and then run an SRIOV test script. Ensure virtual functions are enabled on the DUT.

```bash
make setup numvfs=5
make test TESTS=./scripts/features/sriov_test.py pciaddr=0000:01:00.0
```

--------------------------------

### IOWorker Time Example

Source: https://pynv.me/ssd/pynvme3-io-models

An example of an IOWorker running for a specified duration of 5 seconds with a fixed IO size.

```python
def test_ioworker_time(nvme0n1):
    nvme0n1.ioworker(io_size=8,
                     time=5).start().close()
```

--------------------------------

### NVMe Firmware Download and Commit Example

Source: https://pynv.me/ssd/pynvme3-api

Shows the process of downloading a firmware image to the NVMe controller and committing it to a specific slot. PyNVMe3 also offers a convenience helper `downfw()` for file-based firmware updates.

```python
def test_fw_update_flow(nvme0):
    image = Buffer(4096, ptype=0)
    nvme0.fw_download(image, offset=0).waitdone()
    nvme0.fw_commit(slot=1, action=1).waitdone()

```

--------------------------------

### Specify ioworker LBA region start

Source: https://pynv.me/ssd/pynvme3-io-models

Defines the starting LBA for IO operations within the ioworker. IOs are sent from region_start up to region_end.

```python
def test_ioworker_region_start(nvme0n1):
    nvme0n1.ioworker(region_start=0x10,
                     lba_random=True,
                     time=5).start().close()
```

--------------------------------

### 4K Full Disk Sequential Reading Example

Source: https://pynv.me/ssd/pynvme3-io-models

Demonstrates how to perform a 4K full disk sequential read operation using pynvme's ioworker. Ensure the NVMe device and namespace size are correctly identified.

```python
def test_ioworker_full_disk(nvme0n1):
    ns_size = nvme0n1.id_data(7, 0)
    nvme0n1.ioworker(lba_random=False,
                     io_size=8,
                     read_percentage=100,
                     region_end=ns_size).start().close()
```

--------------------------------

### Disable PEP 668 for System Package Installation

Source: https://pynv.me/ssd/pynvme3-developers-guide

Removes the externally managed file to allow pip to install Python packages into the system directory on Ubuntu 24.04.

```bash
sudo rm -f /usr/lib/python3.12/EXTERNALLY-MANAGED
```

--------------------------------

### NVMe Namespace Management Example

Source: https://pynv.me/ssd/pynvme3-api

Illustrates basic namespace management operations including creation, attachment, and deletion. These operations are crucial for SR-IOV, multi-controller, FDP, and enterprise provisioning tests.

```python
def test_namespace_management(nvme0):
    ns_buf = Buffer(4096)

    nsid = nvme0.ns_create(ns_buf)
    nvme0.ns_attach(nsid, 1)
    nvme0.ns_delete(nsid)

```

--------------------------------

### Start ioworker workload

Source: https://pynv.me/ssd/pynvme3-platforms

Initiates an ioworker workload on the specified NVMe device for a set duration with a specific read percentage. Ensure the device is properly configured before starting.

```python
nvme0n1.ioworker(time=15, read_percentage=100).start().close()
```

--------------------------------

### Configure Sequential IO with lba_step

Source: https://pynv.me/ssd/pynvme3-io-models

Sets up sequential IO with a specific IO size and a step between LBAs.

```python
def test_ioworker_lba_step(nvme0n1):
    nvme0n1.ioworker(io_size=8,
                     lba_random=False,
                     lba_step=16,
                     time=5).start().close()
```

--------------------------------

### Complicated SGL Chain Example

Source: https://pynv.me/ssd/pynvme3-design

Implements a complex SGL chain using SegmentSGL, LastSegmentSGL, DataBlockSGL, and BitBucketSGL as per the NVMe Specification. This example demonstrates constructing SGL descriptors with memory blocks and holes.

```python
   def test_sgl_segment_example(nvme0, nvme0n1):
       sector_size = nvme0n1.sector_size
       cq = IOCQ(nvme0, 1, 10, PRP(10*16))
       sq = IOSQ(nvme0, 1, 10, PRP(10*64), cq=cq)

       sseg0 = SegmentSGL(16*2)
       sseg1 = SegmentSGL(16*3)
       sseg2 = LastSegmentSGL(16)
       sseg0[0] = DataBlockSGL(sector_size*6)
       sseg0[1] = sseg1
       sseg1[0] = DataBlockSGL(sector_size*8)
       sseg1[1] = BitBucketSGL(sector_size*4)
       sseg1[2] = sseg2
       sseg2[0] = DataBlockSGL(sector_size*8)

       sq.read(cid=0, nsid=1, lba=100, lba_count=26, sgl=sseg0)
       sq.tail = sq.wpointer
       cq.waitdone(1)
       cq.head = cq.rpointer

       sq.delete()
       cq.delete()

```

--------------------------------

### Asymmetric SQ and CQ Management

Source: https://pynv.me/ssd/pynvme3-io-models

Demonstrates creating multiple submission queues (IOSQ) mapped to a single completion queue (IOCQ). It shows how to construct and submit write and read commands, manage queue doorbells, and wait for command completions.

```python
def test_multi_sq_and_single_cq(nvme0):
    # Create 3 IOSQ, and mapping to a single IOCQ
    cq = IOCQ(nvme0, 1, 10, PRP(10*16))
    sq1 = IOSQ(nvme0, 1, 10, PRP(10*64), cq=cq)
    sq2 = IOSQ(nvme0, 2, 10, PRP(10*64), cq=cq)
    sq3 = IOSQ(nvme0, 3, 10, PRP(10*64), cq=cq)

    # Construct a write command
    cmd_write = SQE(1, 1)
    cmd_write.cid = 1
    cmd_write[12] = 1<<30

    # Set buffer of the write command
    buf2 = PRP(4096)
    buf2[10:21] = b'hello world'
    cmd_write.prp1 = buf2

    # Construct a read command
    cmd_read1 = SQE(2, 1)
    buf1 = PRP(4096)
    cmd_read1.prp1 = buf1
    cmd_read1.cid = 2

    # Construct another read command
    cmd_read2 = SQE(2, 1)
    buf3 = PRP(4096)
    cmd_read2.prp1 = buf3
    cmd_read2.cid = 3

    # Place the commands into the SQs
    sq1[0] = cmd_write
    sq2[0] = cmd_read1
    sq3[0] = cmd_read2

    # Update SQ1 Tail doorbell to 1
    sq1.tail = 1

    # Wait for the Phase Tag of the head of CQE to be 1
    cq.wait_pbit(cq.head, 1)

    # Update CQ Head doorbell to 1
    cq.head = 1

    # Update SQ2 Tail doorbell to 1, and wait for the command completion
    sq2.tail = 1
    cq.wait_pbit(cq.head, 1)

    # Update CQ Head doorbell to 2
    cq.head = 2

    # Update SQ3 Tail doorbell to 1
    sq3.tail = 1
    cq.wait_pbit(cq.head, 1)

    # update CQ Head doorbell to 3
    cq.head = 3

    # Get the completion status and CID from the first CQ entry
    logging.info(cq[0].status)
    logging.info(cq[0].cid)

```

--------------------------------

### End-to-End Data Integrity Example

Source: https://pynv.me/ssd/pynvme3-api

Demonstrates sequential writes, overwriting a specific LBA, and reading back data for verification using PyNVMe3's ioworker and write/waitdone methods. This example showcases transparent data validation with CRCs, injected metadata, and LBA locks.

```python
def test_verify_integrity(nvme0n1, qpair, buf, verify):
    # Step 1: Perform sequential writes
    nvme0n1.ioworker(io_size=8, lba_random=False,
                     read_percentage=0, qdepth=2,
                     io_count=1000).start().close()

    # Step 2: Overwrite specific LBA
    nvme0n1.write(qpair, buf, 0).waitdone()

    # Step 3: Read back data for verification
    nvme0n1.ioworker(io_size=8, lba_random=False,
                     read_percentage=100,
                     io_count=1000).start().close()
```

--------------------------------

### Configure PyNVMe3 Runtime Environment

Source: https://pynv.me/ssd/pynvme3-developers-guide

Set up the PyNVMe3 runtime by replacing the kernel driver with the user-mode driver and reserving hugepages for testing. This command ensures the system is ready for NVMe device testing.

```bash
make setup
```

--------------------------------

### Initialize NVMe Controller using Fixture

Source: https://pynv.me/ssd/pynvme3-api

Initializes an NVMe controller using a pre-configured fixture. Assumes the controller is already set up. The getfeatures(7) call retrieves controller capabilities.

```python
def test_fixture_nvme_init(nvme0):
    nvme0.getfeatures(7).waitdone()
```
Tokens:
Raw