### Install DPDK Prerequisites

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Installs git, kernel development headers, and tools. Also installs Python build tools required for DPDK.

```sh
sudo yum install -y git kernel-devel kernel-headers
pip3 install meson ninja pyelftools
```

--------------------------------

### Install Git

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/README.rst

Installs the git package, which is required for cloning the FreeBSD source tree.

```sh
pkg install git
```

--------------------------------

### Install Build Environment Packages

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/rpm/README-rpm.txt

Installs essential build tools including gcc, make, and rpm-build using yum.

```bash
bash$ sudo yum -y install gcc make rpm-build
```

--------------------------------

### Install Subversion

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/README.rst

Installs the subversion package, which may be required for older FreeBSD installations or specific AMIs.

```sh
pkg install subversion
```

--------------------------------

### Clone and Install DKMS from Source

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Clones the dkms repository and installs it from source code.

```shell
git clone https://github.com/dell/dkms.git
cd dkms
sudo make install
```

--------------------------------

### Install Binary RPM using Yum

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/rpm/README-rpm.txt

Installs the compiled binary RPM packages using the 'yum localinstall' command.

```bash
bash$ sudo yum -y localinstall
/home/ec2-user/rpmbuild/RPMS/x86_64/kmod-ena-2.17.0-1.el7.3.x86_64.rpm
```

--------------------------------

### Start DPDK TestPMD Application

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Launches the DPDK testpmd application to initialize the ENA device and expose its attributes for verification.

```shell
sudo ./dpdk-build-dir/apps/dpdk-testpmd
```

--------------------------------

### Install Additional RHEL 8 Packages

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/rpm/README-rpm.txt

Installs kernel-abi-whitelists and kernel-rpm-macros for RHEL 8.x using dnf.

```bash
bash$ sudo dnf install kernel-abi-whitelists kernel-rpm-macros
```

--------------------------------

### Install Additional RHEL 9 Packages

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/rpm/README-rpm.txt

Installs kernel-abi-stablelists and kernel-rpm-macros for RHEL 9.x using dnf.

```bash
bash$ sudo dnf install kernel-abi-stablelists kernel-rpm-macros
```

--------------------------------

### Install Kernel Headers and Development Packages

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/rpm/README-rpm.txt

Installs kernel-devel and kernel-headers packages matching the running kernel version using yum.

```bash
bash$ sudo yum -y install kernel-{devel,headers}-$(uname -r)
```

--------------------------------

### Install DKMS and EPEL Repository

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Installs the necessary DKMS package and enables the EPEL repository for additional packages.

```shell
sudo yum install --enablerepo=extras epel-release
sudo yum install dkms
```

--------------------------------

### Configure Automatic ENA Driver Start

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/README.rst

Configure the system to automatically load the ENA driver upon boot by adding 'if_ena_load="YES"' to '/boot/loader.conf' and copying the module to '/boot/modules/'.

```shell
vi /boot/loader.conf
# insert 'if_ena_load="YES"' in the above file

cp if_ena.ko /boot/modules/
sync; sleep 30;
```

--------------------------------

### Example Flow Steering Rule Configuration Output

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Illustrates the output format for viewing configured flow steering rules, detailing filter type, IP addresses, ports, and action.

```shell
# ethtool -n eth1
8 RX rings available
Total 1 rules

Filter: 6
Rule Type: TCP over IPv4
Src IP addr: 0.0.0.0 mask: 255.255.255.255
Dest IP addr: 0.0.0.0 mask: 255.255.255.255
TOS: 0x0 mask: 0xff
Src port: 0 mask: 0xffff
Dest port: 5001 mask: 0x0
Action: Direct to queue 1
```

--------------------------------

### Add, Build, and Install ENA Driver with DKMS

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Registers the ENA driver with DKMS, builds the module, and installs it.

```shell
sudo dkms add -m amzn-drivers -v X.Y.Z
sudo dkms build -m amzn-drivers -v X.Y.Z
sudo dkms install -m amzn-drivers -v X.Y.Z
```

--------------------------------

### Install Kernel Headers and Build Tools on Ubuntu

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Installs essential packages for building kernel modules on Ubuntu, including linux-headers, make, and gcc.

```shell
sudo apt-get update
sudo apt install linux-headers-$(uname -r)
sudo apt-get install make gcc
```

--------------------------------

### Configure ENA Debugging via Meson Build Arguments

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Enable ENA driver debug logging during Meson configuration by passing C arguments. This example enables RX and TX debug logging.

```sh
meson build -Dc_args='-DRTE_LIBRTE_ENA_DEBUG_RX=1 -DRTE_LIBRTE_ENA_DEBUG_TX=1'
```

--------------------------------

### Update System and Install Kernel

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Updates the system package list and installs the latest kernel. A reboot is required after kernel installation.

```sh
sudo yum update
sudo yum install -y kernel
sudo reboot
```

--------------------------------

### Set ENA Devargs for Runtime Options

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Use device arguments to modify ENA driver behavior at runtime. This example sets 'large_llq_hdr' for a specific PCI BDF.

```sh
./dpdk_app -w 00:06.0,large_llq_hdr=1
```

--------------------------------

### Reboot Instance

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Reboots the instance to load the newly installed ENA driver.

```shell
sudo reboot
```

--------------------------------

### Install Kernel Development Packages on RHEL

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Installs necessary packages for kernel development on RHEL, including gcc, kernel-devel, and git.

```shell
sudo yum update
sudo reboot
sudo yum install gcc kernel-devel-$(uname -r) git
```

--------------------------------

### Enable ENA Debugging in rte_config.h

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Define ENA driver debug options directly in the rte_config.h file for Meson builds. This example enables RX, TX, and COM debug logging.

```c
#define RTE_LIBRTE_ENA_DEBUG_RX 1
#define RTE_LIBRTE_ENA_DEBUG_TX 1
#define RTE_LIBRTE_ENA_COM_DEBUG 1
```

--------------------------------

### Install DKMS on RHEL

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Installs the dkms package on RHEL, including enabling the EPEL repository for RHEL 9.

```shell
# Replace the 9 with the RHEL version you have (For example - for RHEL 8 use 8)
# For EOL RHEL versions please look for the rpm in https://dl.fedoraproject.org/pub/archive/epel/
sudo yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm
sudo yum install dkms
```

--------------------------------

### Install Specific Ubuntu Kernel Version

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Install the linux-image, linux-headers, and linux-tools for kernel version 5.4.0-1060-aws, which provides vfio-pci as a module.

```bash
sudo apt install -y linux-image-5.4.0-1060-aws linux-headers-5.4.0-1060-aws linux-tools-5.4.0-1060-aws
```

--------------------------------

### Compile ENA Driver with Custom Flags

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/README.rst

Example of compiling the ENA driver with specific flags, forcing DEV_NETMAP to be enabled and RSS to be disabled.

```sh
make DEV_NETMAP=1 RSS=0
```

--------------------------------

### Calculate Toeplitz Hash Value for RSS

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/ENA_Linux_Best_Practices.rst

Use this script to simulate the Toeplitz hash calculation for a given network flow, which is used in RSS configuration. This example demonstrates calculating the hash for a specific flow and shows expected results for different operating systems and driver states.

```python
python3 toeplitz_calc.py -t 1.2.3.4 -T 7000 -r 1.2.3.5 -R 7000 -k 77:d1:c9:34:a4:c9:bd:87:6e:35:dd:17:b2:e3:23:9e:39:6d:8a:93:2a:95:b4:72:3a:b3:7f:56:8e:de:b6:01:97:af:3b:2f:3a:70:e7:04
Sending traffic from 1.2.3.4:7000 to 1.2.3.5:7000
to an instance which supports changing the key

Should result in the following hash for each driver:
DPDK                                                    0xa9828bd4 (RSS table entry: 84)
FreeBSD                                                 0xa9828bd4 (RSS table entry: 84)
Linux (before setting the key with ethtool)             0xa4a1471a (RSS table entry: 26)
Linux (after setting the key with ethtool)              0x5b5eb8e5 (RSS table entry: 101)
Windows                                                 0x5b5eb8e5 (RSS table entry: 101)
```

--------------------------------

### Configure GRUB Default Boot Entry

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Set the GRUB_DEFAULT value to "1>2" in /etc/default/grub to specify the kernel boot entry. This is typically used after installing a specific kernel version.

```bash
grep GRUB_DEFAULT /etc/default/grub
GRUB_DEFAULT="1>2"
```

--------------------------------

### Retrieve PHC timestamp using testptp

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Example of retrieving PHC timestamp using the testptp utility. Limits PHC get time requests to 125 per second.

```shell
testptp -d /dev/ptp$(ethtool -T <interface> | awk '/PTP Hardware Clock:/ {print $NF}') -k 1
```

--------------------------------

### Install DKMS on Ubuntu

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Installs the dkms package on Ubuntu, which is used for managing kernel module installations.

```shell
sudo apt-get install dkms
```

--------------------------------

### Configure GRUB for IOMMU Support

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/enav2-vfio-patch/README.md

For instances that support IOMMU, enable it by adding kernel boot parameters to the GRUB configuration. This typically involves 'iommu=1' and 'intel_iommu=on'.

```bash
GRUB_CMDLINE_LINUX="console=ttyS0,115200n8 console=tty0 net.ifnames=0 crashkernel=auto iommu=1 intel_iommu=on"
```

--------------------------------

### Install DKMS on Amazon Linux

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Installs the dkms package on Amazon Linux, which helps manage kernel module installations.

```shell
sudo yum install dkms
```

--------------------------------

### DPDK v16.11 - v17.11 Initial Configuration

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Performs initial configuration for DPDK v16.11 to v17.11 using 'make config'. Navigate to the build directory after execution.

```shell
make config T=x86_64-native-linuxapp-gcc
cd build
```

--------------------------------

### DPDK v18.02 - v21.05 Initial Configuration (Makefile)

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Performs initial configuration for DPDK v18.02 to v21.05 using 'make defconfig'. Navigate to the build directory after execution. This method is deprecated since v20.11.

```shell
make defconfig
cd build
```

--------------------------------

### Load and Insert igb_uio with WC Support

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Load the dependent 'uio' kernel module and then insert the 'igb_uio' module with write combining support enabled.

```shell
modprobe uio # Load dependent kernel module uio
insmod igb_uio.ko wc_activate=1 # insert igb_uio with WC support
```

--------------------------------

### Verify vm.min_free_kbytes Setting

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/ENA_Linux_Best_Practices.rst

Verify the current setting of vm.min_free_kbytes. This command retrieves the value without any additional output.

```bash
$ sudo sysctl -n vm.min_free_kbytes
```

--------------------------------

### Install DKMS on SLES 15

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Installs the dkms package on SLES 15, ensuring the PackageHub is enabled and repositories are refreshed.

```shell
# Replace the 15.4 with the correct SLES 15 version
# For arm instances replace x86_64 with aarch64
sudo SUSEConnect --product PackageHub/15.4/x86_64
sudo zypper refresh
sudo zypper install dkms
```

--------------------------------

### Build DPDK with Meson and Ninja

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Builds the DPDK using the meson build system and ninja. These are the primary commands for compiling DPDK.

```sh
meson build
ninja -C build
```

--------------------------------

### Install Kernel Development Packages on SLES

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Installs necessary packages for kernel development on SLES, including make, gcc, kernel-default-devel, and git.

```shell
sudo zypper update
sudo reboot
sudo zypper install make gcc kernel-default-devel git
```

--------------------------------

### Apply GRUB Changes and Reboot

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Update the GRUB configuration to apply changes and then reboot the system to boot with the selected kernel.

```bash
sudo update-grub
sudo reboot
```

--------------------------------

### Install Kernel Development Packages on Amazon Linux

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Installs necessary packages for kernel development on Amazon Linux, including kernel-devel and git.

```shell
sudo yum update
sudo reboot
sudo yum install kernel-devel-$(uname -r) git
```

--------------------------------

### List available PTP clock sources

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

List all available PTP hardware clock devices in the system.

```shell
ls /sys/class/ptp
```

--------------------------------

### Apply sysctl Configuration

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/ENA_Linux_Best_Practices.rst

Apply the system control settings, including vm.min_free_kbytes, to the running system. This command reads and applies settings from /etc/sysctl.conf.

```bash
$ sudo sysctl -p
```

--------------------------------

### Enable IOMMU on x86_64 Metal Instances

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Updates GRUB configuration to enable IOMMU on x86_64 metal instances. This is necessary for vfio-pci driver compatibility when IOMMU is enabled.

```text
iommu=1 intel_iommu=on
```

--------------------------------

### Load igb_uio Module with Write Combining

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Loads the generic uio module and then the igb_uio module with the wc_activate=1 option, which is required for ENAv2 efficiency.

```sh
# igb_uio depends on the generic uio module
sudo modprobe uio
# Load igb_uio with Write Combining activated
sudo insmod ./igb_uio.ko wc_activate=1
```

--------------------------------

### Get Running Kernel Version

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/README.rst

Displays information about the running kernel, including the branch and commit ID, to help determine the correct source tree to clone.

```sh
uname -a
```

--------------------------------

### Modify ENA Debugging via Meson Configure

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Update ENA driver debug logging after initial Meson configuration. This example enables TX debug logging.

```sh
cd build
meson configure -Dc_args='-DRTE_LIBRTE_ENA_DEBUG_TX=1'
```

--------------------------------

### Enable PHC using devlink

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Activate PHC support using the devlink utility to set the 'enable_phc' parameter and reload the device. Supported from Linux Kernel v6.17+.

```shell
sudo devlink dev param set pci/<domain:bus:slot.function> name enable_phc value true cmode driverinit
sudo devlink dev reload pci/<domain:bus:slot.function>
# for example:
sudo devlink dev param set pci/0000:00:06.0 name enable_phc value true cmode driverinit
sudo devlink dev reload pci/0000:00:06.0
```

--------------------------------

### Configuring Kbuild File

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/efa/src/CMakeLists.txt

Replaces list separators with spaces and configures the Kbuild file using a template.

```cmake
string(REPLACE ";" " " efa_sources_string "${efa_sources}")
configure_file(Kbuild.in Kbuild @ONLY)
```

--------------------------------

### Run vfio-pci with WC Patch Script

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Execute the script to download kernel sources, apply the WC patch, rebuild vfio with no-iommu-mode support, and replace the existing vfio module.

```bash
cd amzn-drivers/userspace/dpdk/enav2-vfio-patch
sudo get-vfio-with-wc.sh
```

--------------------------------

### Configuring Kbuild

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/efa_nv_peermem/src/CMakeLists.txt

Replaces semicolons with spaces in the source list and configures the Kbuild file.

```cmake
string(REPLACE ";" " " efa_nv_peermem_sources_string "${efa_nv_peermem_sources}")
configure_file(Kbuild.in Kbuild @ONLY)

```

--------------------------------

### Activate Receive Packet Steering (RPS)

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/ENA_Linux_Best_Practices.rst

Activate RPS to offload Rx traffic processing to other vCPUs. This example assigns cores 8-15 to RPS for the eth0 interface. Ensure RPS vCPU cores are on the same NUMA node as ENA IRQ vCPUs.

```bash
$ for i in `seq 0 7`; do echo $(printf "00000000,0000ff00") | sudo tee /sys/class/net/eth0/queues/rx-$i/rps_cpus; done
```

--------------------------------

### DPDK v18.02 - v21.05 Initial Configuration (meson)

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Configures the DPDK project using meson, enabling specific debug flags for ENA drivers. This is for DPDK versions 18.02 to 21.05.

```shell
meson build -Dc_args='-DRTE_LIBRTE_ENA_DEBUG_TX=1 -DRTE_LIBRTE_ENA_DEBUG_TX_FREE=1 -DRTE_LIBRTE_ENA_DEBUG_RX=1 -DRTE_LIBRTE_ENA_COM_DEBUG=1'
cd build
```

--------------------------------

### List IOMMU Groups

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/enav2-vfio-patch/README.md

Once IOMMU is active, verify its functionality by checking the contents of the /sys/kernel/iommu_groups/ directory. A non-empty directory indicates IOMMU is working.

```bash
ls /sys/kernel/iommu_groups/
```

--------------------------------

### Query Large LLQ configuration using ethtool

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Check the current hardware settings for TX push buffer length to verify Large LLQ configuration.

```shell
ethtool -g [interface]
```

--------------------------------

### Configure dkms.conf for ENA Driver

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Sets up the dkms.conf file with package details and build/install commands for the ENA driver.

```shell
PACKAGE_NAME="ena"
PACKAGE_VERSION="1.0.0"
CLEAN="make -C kernel/linux/ena clean"
MAKE="make -C kernel/linux/ena/ BUILD_KERNEL=${kernelver}"
BUILT_MODULE_NAME[0]="ena"
BUILT_MODULE_LOCATION="kernel/linux/ena"
DEST_MODULE_LOCATION[0]="/updates"
DEST_MODULE_NAME[0]="ena"
REMAKE_INITRD="yes"
AUTOINSTALL="yes"
```

--------------------------------

### Copying Source Files

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/efa_nv_peermem/src/CMakeLists.txt

Iterates through the source list and copies each file to the build directory.

```cmake
foreach(src ${efa_nv_peermem_sources})
  configure_file(${src} ${src} COPYONLY)
endforeach()

```

--------------------------------

### Configure Kernel Boot Options for C-states (Amazon Linux AMI)

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/ENA_Linux_Best_Practices.rst

Edit the grub.conf file to limit CPU C-states for low-latency performance on Amazon Linux AMI. This involves adding specific parameters to the kernel line.

```bash
kernel /boot/vmlinuz-4.14.62-65.117.amzn1.x86_64 root=LABEL=/ console=tty1 console=ttyS0 selinux=0 nvme_core.io_timeout=4294967295 xen_nopvspin=1 clocksource=tsc intel_idle.max_cstate=1 processor.max_cstate=1
```

--------------------------------

### Configure Large LLQ TX push buffer length using ethtool

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Use ethtool to set the TX push buffer length for Large LLQ. Requires Linux Kernel v6.4+ and ethtool v6.4+.

```shell
ethtool -G [interface] tx-push-buf-len [number of bytes: 96 or 224]
```

--------------------------------

### Enable PHC using insmod parameter

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Load the ENA kernel module with the 'phc_enable' parameter set to 1 to activate PHC support.

```shell
sudo insmod ena.ko phc_enable=1
```

--------------------------------

### Rebuild Binary RPM from Source RPM

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/rpm/README-rpm.txt

Compiles the binary RPM packages from the created source RPM (src.rpm) using rpmbuild --rebuild.

```bash
bash$ rpmbuild --rebuild ena-2.17.0-1.el7.3.src.rpm
```

--------------------------------

### DPDK v18.02 - v21.05 Build Project (ninja)

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Builds the DPDK project using ninja after configuration with meson. This is for DPDK versions 18.02 to 21.05.

```shell
ninja
```

--------------------------------

### Configure 9k mbufs for ENA Driver

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/README.rst

Enable the use of 9k mbufs for ENA Rx descriptors by setting the 'hw.ena.enable_9k_mbufs' tunable to 1 in '/boot/loader.conf'. This can improve Rx throughput but requires sufficient memory.

```shell
hw.ena.enable_9k_mbufs=1
```

--------------------------------

### Clone FreeBSD Source Tree (Main/Current)

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/README.rst

Clones the FreeBSD source tree for the main branch (CURRENT), used for building the kernel.

```sh
git clone https://git.FreeBSD.org/src.git /usr/src --single-branch --branch main
```

--------------------------------

### Setting Custom RSS Hash Key and Length

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Demonstrates how to fill the 'struct rte_eth_rss_conf' to set a custom RSS hash key and its length for predictable flow-to-queue mapping. Requires DPDK v21.08 and ENA v2.4.0+.

```c
/*
 * Example of filling the structure for RSS hash update.
 * .rss_key - pointer to the 40 bytes table containing RSS hash key
 * .rss_key_len - size of the RSS key, which must be 40
 */
struct rte_eth_rss_conf rss_conf;

/* Assume rss_key_data is a 40-byte buffer containing the desired key */
rss_conf.rss_key = rss_key_data;
rss_conf.rss_key_len = 40;

/* If hardware doesn't support RSS hash fields modification, set .rss_hf to 0 */
rss_conf.rss_hf = 0;

/* Pass the filled structure to the update function */
ret = rte_eth_dev_rss_hash_update(port_id, &rss_conf);

```

--------------------------------

### Build Source RPM Package

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/rpm/README-rpm.txt

Builds the kmod src.rpm package for the Amazon ENA driver by running 'make' in the driver's rpm directory.

```bash
bash$ make
cd .. && git archive --format=tar --prefix=ena-2.17.0/ -o rpm/ena-2.17.0.tar ena_linux_2.17.0
rpmbuild -bs \
            --define '_topdir %(pwd)' --define '_ntopdir %(pwd)' \
            --define '_builddir  %{_ntopdir}' \
            --define '_sourcedir %{_ntopdir}' \
            --define '_specdir   %{_ntopdir}' \
            --define '_rpmdir    %{_ntopdir}' \
            --define '_srcrpmdir %{_ntopdir}' \
        ena.spec
```

--------------------------------

### Load Patched VFIO-PCI Module

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/enav2-vfio-patch/README.md

After applying the necessary patches and building the VFIO driver, load the custom module using modprobe. Ensure the script is executed with root privileges.

```bash
modprobe vfio-pci
```

--------------------------------

### Clone and Build igb_uio Module

Source: https://github.com/amzn/amzn-drivers/blob/master/userspace/dpdk/README.md

Clones the DPDK kernel modules repository and builds the igb_uio module. This module is used for ENA support.

```sh
git clone git://dpdk.org/dpdk-kmods
cd dpdk-kmods/linux/igb_uio
make
```

--------------------------------

### Enable devlink support in ENA driver build

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Sets the ENA_DEVLINK_INCLUDE environment variable to 1 to enable devlink support during the ENA driver build process. This is required for using devlink functionalities.

```shell
ENA_DEVLINK_INCLUDE=1 make
```

--------------------------------

### Load ENA Kernel Module

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/README.rst

Load the ENA kernel module ('if_ena.ko') into the running system using the 'kldload' command.

```shell
kldload ./if_ena.ko
```

--------------------------------

### View Configured Flow Steering Rules

Source: https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/README.rst

Displays the list of all configured flow steering rules on the specified network interface, including available rings and filter details.

```shell
ethtool -n eth1
```