### Install QAT Resource Driver Helm Chart Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/charts/intel-qat-resource-driver/README.md Installs the Intel QAT Resource Driver Helm chart to the 'intel-qat-resource-driver' namespace. Creates the namespace if it does not exist. ```bash helm install \ --namespace intel-qat-resource-driver \ --create-namespace \ intel-qat-resource-driver oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-qat-resource-driver ``` -------------------------------- ### Install Kubernetes Code Generation Tools Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/DEV.md Install `controller-gen` and `client-gen` using `go install`. Ensure `GO111MODULE` is set to `on`. These tools are essential for generating Kubernetes API code. ```bash GO111MODULE=on go install sigs.k8s.io/controller-tools/cmd/controller-gen@latest GO111MODULE=on go install k8s.io/code-generator/cmd/client-gen@latest ``` -------------------------------- ### Install Intel QAT Resource Driver Helm Chart Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/qat/USAGE.md Installs the Intel QAT Resource Driver using its Helm chart published to the GitHub OCI registry. Ensure the namespace is created. ```console helm install \ --namespace "intel-qat-resource-driver" \ --create-namespace \ intel-qat-resource-driver oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-qat-resource-driver-chart ``` -------------------------------- ### Install Intel Gaudi Resource Driver Chart Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/charts/intel-gaudi-resource-driver/README.md Installs the Intel Gaudi Resource Driver Helm chart into a specified namespace. Ensure the namespace is created beforehand if it does not exist. ```console helm install \ --namespace intel-gaudi-resource-driver \ --create-namespace \ intel-gaudi-resource-driver oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-gaudi-resource-driver-chart ``` -------------------------------- ### KubeVirt Pod YAML Example with Inline VFIO Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gpu/USAGE.md This example demonstrates how to request a GPU device for a KubeVirt VM using the `gpu-vfio.intel.com` DeviceClass, enabling PCI passthrough mode. ```yaml apiVersion: v1 kind: Pod metadata: name: my-gpu-vm spec: containers: - name: vm image: kubevirt/virt-launcher:v1.8.3 resources: limits: nvidia.com/gpu: 1 volumeMounts: - name: gpu-volume mountPath: /dev/dxg volumes: - name: gpu-volume persistentVolumeClaim: claimName: gpu-pvc ``` -------------------------------- ### Create Namespace and Install QAT Resource Driver Helm Chart Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/charts/intel-qat-resource-driver/README.md Pre-creates the 'intel-qat-resource-driver' namespace with the 'privileged' pod security label, then installs the Helm chart. This is necessary for Kubernetes clusters using Pod Security Standards with HostPath Volumes. ```bash kubectl create namespace intel-qat-resource-driver kubectl label --overwrite namespace intel-qat-resource-driver pod-security.kubernetes.io/enforce=privileged helm install \ --namespace intel-qat-resource-driver \ intel-qat-resource-driver oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-qat-resource-driver ``` -------------------------------- ### Example: Generate CDI Specs for GPUs Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/cdi-spec-generator/README.md Run the tool with 'gpu' as an argument to generate CDI specifications for supported GPUs on the system. ```bash intel-cdi-specs-generator gpu ``` -------------------------------- ### Install Intel GPU Resource Driver Helm Chart Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/charts/intel-gpu-resource-driver/README.md Installs the Intel GPU Resource Driver chart with a dedicated namespace. Ensure the namespace is created if not already present. ```console helm install \ --namespace "intel-gpu-resource-driver" \ --create-namespace \ intel-gpu-resource-driver oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-gpu-resource-driver-chart ``` -------------------------------- ### Start Minikube with Feature Gates Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/CLUSTER_SETUP.md Starts a minikube cluster with specific feature gates enabled for DRA device taints. ```shell minikube start \ --extra-config=apiserver.feature-gates=DRADeviceTaints=true \ --extra-config=scheduler.feature-gates=DRADeviceTaints=true \ --extra-config=controller-manager.feature-gates=DRADeviceTaints=true \ --container-runtime=containerd \ --kubernetes-version=1.34.0 ``` -------------------------------- ### Install Intel Gaudi Resource Driver Helm Chart Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gaudi/USAGE.md Install the Intel Gaudi Resource Driver using its Helm chart published to the GitHub OCI registry. Ensure the namespace is created if it doesn't exist. ```bash helm install \ --namespace "intel-gaudi-resource-driver" \ --create-namespace \ intel-gaudi-resource-driver oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-gaudi-resource-driver-chart ``` -------------------------------- ### Install on OpenShift 4.20 Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/charts/intel-gaudi-resource-driver/README.md Installs the Intel Gaudi Resource Driver Helm chart on OpenShift 4.20, enabling specific OpenShift configurations. ```console helm install \ --set openshift.enabled=true \ --set openshift.version=4.20 \ --namespace intel-gaudi-resource-driver \ --create-namespace \ intel-gaudi-resource-driver oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-gaudi-resource-driver-chart ``` -------------------------------- ### Install QAT Resource Driver on OpenShift 4.20 Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/charts/intel-qat-resource-driver/README.md Installs the Intel QAT Resource Driver Helm chart on OpenShift 4.20, enabling OpenShift-specific configurations and setting the version to 4.20. Requires cluster admin privileges. ```bash helm install \ --set openshift.enabled=true \ --set openshift.version=4.20 \ --namespace "intel-qat-resource-driver" \ --create-namespace \ intel-qat-resource-driver oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-qat-resource-driver-chart ``` -------------------------------- ### Build Kubernetes Code Generation Tools from Source Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/DEV.md Clone the `controller-tools` and `code-generator` repositories and build `controller-gen` and `client-gen` binaries locally. This method is an alternative to `go install`. ```bash git clone https://github.com/kubernetes-sigs/controller-tools.git cd controller-tools go build ./cmd/controller-gen cd - git clone https://github.com/kubernetes/code-generator.git cd code-generator go build ./cmd/client-gen cd - ``` -------------------------------- ### Install Intel Gaudi Resource Driver Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/deployments/gaudi/overlays/openshift/README.md Apply the OpenShift overlay to install the Intel Gaudi resource driver. Ensure you are using OpenShift version 4.21 or later. ```shell kubectl apply -k deployments/gaudi/overlays/openshift ``` -------------------------------- ### Show Helm Chart Values Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/charts/intel-qat-resource-driver/README.md Displays all configurable options for the Intel QAT Resource Driver Helm chart with detailed comments. Useful for understanding customization possibilities before installation. ```bash helm show values oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-qat-resource-driver ``` -------------------------------- ### Install QAT Resource Driver on OpenShift 4.21+ Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/charts/intel-qat-resource-driver/README.md Installs the Intel QAT Resource Driver Helm chart on OpenShift 4.21 or later, enabling OpenShift configurations. The version defaults to 4.21, so it doesn't need to be explicitly set. Requires cluster admin privileges. ```bash helm install \ --set openshift.enabled=true \ --namespace "intel-qat-resource-driver" \ --create-namespace \ intel-qat-resource-driver oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-qat-resource-driver-chart ``` -------------------------------- ### Install with Pod Security Standards Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/charts/intel-gaudi-resource-driver/README.md Installs the Intel Gaudi Resource Driver Helm chart on clusters using Pod Security Standards. This requires pre-creating the namespace with the 'privileged' label for HostPath Volumes. ```console kubectl create namespace intel-gaudi-resource-driver kubectl label --overwrite namespace intel-gaudi-resource-driver pod-security.kubernetes.io/enforce=privileged helm install \ --namespace intel-gaudi-resource-driver \ intel-gaudi-resource-driver oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-gaudi-resource-driver-chart ``` -------------------------------- ### Install Node Feature Discovery (NFD) Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/qat/USAGE.md Installs Node Feature Discovery (NFD) using its default overlay. NFD is required to label nodes with QAT capabilities for targeted deployments. ```bash kubectl apply -k "https://github.com/kubernetes-sigs/node-feature-discovery/deployment/overlays/default?ref=v0.18.3" ``` -------------------------------- ### Example Pod with QAT Accelerator ResourceClaim Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/qat/USAGE.md Demonstrates how to define a ResourceClaimTemplate and a Deployment to request and utilize a QAT accelerator. The `IPC_LOCK` capability is required for VFIO-based device access. ```yaml apiVersion: resource.k8s.io/v1 kind: ResourceClaimTemplate metadata: name: qat-template-sym spec: spec: devices: requests: - name: qat-request-sym exactly: deviceClassName: qat.intel.com selectors: - cel: expression: |- device.attributes["qat.intel.com"].services == "sym" || device.attributes["qat.intel.com"].services == "sym;asym" || device.attributes["qat.intel.com"].services == "sym;dc" || device.attributes["qat.intel.com"].services == "asym;sym" || device.attributes["qat.intel.com"].services == "dc;sym" --- apiVersion: apps/v1 kind: Deployment metadata: name: qat-sample-sym labels: app: inline-qat-deployment spec: replicas: 1 selector: matchLabels: app: inline-qat-deployment template: metadata: labels: app: inline-qat-deployment spec: containers: - name: with-resource image: registry.k8s.io/e2e-test-images/busybox:1.29-2 command: ["sh", "-c", "ls -la /dev/vfio/ && sleep 300"] securityContext: capabilities: add: ["IPC_LOCK"] resources: claims: - name: resource-sym resourceClaims: - name: resource-sym resourceClaimTemplateName: qat-template-sym ``` -------------------------------- ### OpenShift Pod Security Warning Example Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/deployments/qat/overlays/openshift/README.md This warning may appear during installation if the SecurityContextConstraints are created after the DaemonSet. Kubernetes will retry Pod creation, and the issue should resolve itself. ```shell Warning: would violate PodSecurity "restricted:latest": privileged (container "kubelet-plugin" must not set securityContext.privileged=true), allowPrivilegeEscalation != false (container "kubelet-plugin" must set securityContext.allowPrivilegeEscalation=false), restricted volume types (volumes "plugins-registry", "plugins", "cdi", "varruncdi", "sysfs" use restricted volume type "hostPath"), runAsNonRoot != true (pod or container "kubelet-plugin" must set securityContext.runAsNonRoot=true), runAsUser=0 (container "kubelet-plugin" must not set runAsUser=0) ``` -------------------------------- ### Install on OpenShift 4.21+ Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/charts/intel-gpu-resource-driver/README.md Deploy the Intel GPU Resource Driver Helm chart to an OpenShift 4.21 or later cluster. The default `openshift.version` is used, simplifying the command. ```bash helm install \ --set openshift.enabled=true \ --namespace "intel-gpu-resource-driver" \ --create-namespace \ intel-gpu-resource-driver oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-gpu-resource-driver-chart ``` -------------------------------- ### Apply OpenShift Kustomization Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/deployments/qat/overlays/openshift/README.md Use this command to apply the OpenShift overlay for the Intel QAT resource driver installation. ```shell kubectl apply -k deployments/qat/overlays/openshift ``` -------------------------------- ### Basic Gaudi Accelerator Request with ResourceClaim Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gaudi/USAGE.md This example demonstrates the basic use case for requesting a Gaudi accelerator by creating a ResourceClaim and referencing it in a Pod specification. The resource driver handles device allocation. ```yaml apiVersion: resource.k8s.io/v1 kind: ResourceClaim metadata: name: claim1 spec: devices: requests: - name: gaudi exactly: deviceClassName: gaudi.intel.com --- apiVersion: v1 kind: Pod metadata: name: test-inline-claim spec: restartPolicy: Never containers: - name: with-resource image: registry.k8s.io/e2e-test-images/busybox:1.29-2 command: ["sh", "-c", "ls -la /dev/accel/ && sleep 60"] resources: claims: - name: resource resourceClaims: - name: resource resourceClaimName: claim1 ``` -------------------------------- ### Install on OpenShift 4.21+ Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/charts/intel-gaudi-resource-driver/README.md Installs the Intel Gaudi Resource Driver Helm chart on OpenShift 4.21 and later. The `openshift.version` defaults to `4.21`, so it does not need to be explicitly set. Versions older than 4.20 are not supported. ```console helm install \ --set openshift.enabled=true \ --namespace intel-gaudi-resource-driver \ --create-namespace \ intel-gaudi-resource-driver oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-gaudi-resource-driver-chart ``` -------------------------------- ### Install Helm Chart with Pod Security Standards Enabled Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/charts/intel-gpu-resource-driver/README.md Installs the Intel GPU Resource Driver chart while configuring the namespace for Pod Security Standards, allowing HostPath Volumes. This involves pre-creating the namespace and applying the 'privileged' label. ```console kubectl create namespace intel-gpu-resource-driver kubectl label --overwrite namespace intel-gpu-resource-driver pod-security.kubernetes.io/enforce=privileged helm install \ --namespace intel-gpu-resource-driver \ intel-gpu-resource-driver oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-gpu-resource-driver-chart ``` -------------------------------- ### Show Helm Chart Values Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/charts/intel-gaudi-resource-driver/README.md Displays all configurable options for the Intel Gaudi Resource Driver Helm chart with detailed comments. This is useful for understanding customization possibilities before installation. ```console helm show values oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-gaudi-resource-driver-chart ``` -------------------------------- ### Pod with Generated Resource Claim Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gaudi/USAGE.md This example shows how to define a Pod that references a ResourceClaimTemplate. A new ResourceClaim will be generated for this Pod, allocating Gaudi accelerators. Generated claims are deleted when the Pod is deleted. ```YAML apiVersion: resource.k8s.io/v1 kind: ResourceClaimTemplate metadata: name: claim1 spec: spec: devices: requests: - name: gaudi exactly: deviceClassName: gaudi.intel.com --- apiVersion: v1 kind: Pod metadata: name: test-inline-claim spec: restartPolicy: Never containers: - name: with-resource image: registry.k8s.io/e2e-test-images/busybox:1.29-2 command: ["sh", "-c", "ls -la /dev/accel/ && sleep 60"] resources: claims: - name: resource resourceClaims: - name: resource resourceClaimTemplateName: claim1 ``` -------------------------------- ### Install on OpenShift 4.20 Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/charts/intel-gpu-resource-driver/README.md Deploy the Intel GPU Resource Driver Helm chart to an OpenShift 4.20 cluster. This command sets specific OpenShift version and namespace configurations. ```bash helm install \ --set openshift.enabled=true \ --set openshift.version=4.20 \ --namespace "intel-gpu-resource-driver" \ --create-namespace \ intel-gpu-resource-driver oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-gpu-resource-driver-chart ``` -------------------------------- ### Pod with Generated Resource Claim from Template Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gpu/USAGE.md This example demonstrates how a Pod can reference a ResourceClaimTemplate, which K8s then uses to generate a ResourceClaim for GPU resources. The generated claim is automatically deleted when the Pod is deleted. ```yaml apiVersion: resource.k8s.io/v1 kind: ResourceClaimTemplate metadata: name: claim1 spec: spec: devices: requests: - name: gpu exactly: deviceClassName: gpu.intel.com --- apiVersion: v1 kind: Pod metadata: name: test-inline-claim spec: restartPolicy: Never containers: - name: with-resource image: registry.k8s.io/e2e-test-images/busybox:1.29-2 command: ["sh", "-c", "ls -la /dev/dri/ && sleep 60"] resources: claims: - name: resource resourceClaims: - name: resource resourceClaimTemplateName: claim1 ``` -------------------------------- ### Add Go Binaries Directory to PATH Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/DEV.md Append the directory containing Go-generated binaries to your system's PATH environment variable by adding an export command to your `.bashrc` file. This ensures newly installed tools are accessible. ```bash export PATH=":$PATH" ``` -------------------------------- ### Resource Claim Requesting 2 Gaudi2 Accelerators Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gaudi/USAGE.md This example demonstrates how to create a ResourceClaim that explicitly requests two Gaudi2 accelerators. It uses a CEL expression to filter devices based on their model attribute. ```YAML apiVersion: resource.k8s.io/v1 kind: ResourceClaim metadata: name: claim1 spec: devices: requests: - name: gaudi deviceClassName: gaudi.intel.com exactly: count: 2 selectors: - cel: expression: device.attributes["gaudi.intel.com"].model == 'Gaudi2' ``` -------------------------------- ### Pod Logs Showing Assigned GPU Device Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gpu/USAGE.md Example output from 'kubectl logs' showing the devices available within the test pod, including the assigned GPU device (e.g., card0, renderD128). This confirms the GPU is visible and accessible inside the pod. ```bash Defaulted container "with-resource" out of: with-resource, without-resource total 0 drwxr-xr-x 2 root root 80 Sep 27 09:17 . drwxr-xr-x 6 root root 380 Sep 27 09:17 .. crw-rw-rw- 1 root root 226, 0 Sep 27 09:17 card0 crw-rw-rw- 1 root root 226, 128 Sep 27 09:17 renderD128 ``` -------------------------------- ### Resource Claim Template Requesting Multiple GPUs with Memory Selector Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gpu/USAGE.md This example shows how to customize a ResourceClaimTemplate to request a specific count of GPUs and use a CEL expression to filter for GPUs with at least 16 Gi of local memory. ```yaml apiVersion: resource.k8s.io/v1 kind: ResourceClaimTemplate metadata: name: claim1 spec: spec: devices: requests: - name: gpu exactly: deviceClassName: gpu.intel.com count: 2 selectors: - cel: expression: device.capacity["gpu.intel.com"].memory.compareTo(quantity("16Gi")) >= 0 ``` -------------------------------- ### Initialize Kubernetes Cluster with kubeadm Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/CLUSTER_SETUP.md Initializes a Kubernetes cluster using a specific configuration file. ```bash sudo -E kubeadm init --config hack/clusterconfig.yaml ``` -------------------------------- ### Deploy Intel QAT Resource Driver from Sources Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/qat/USAGE.md Applies the Kubernetes resource definitions for the QAT resource driver from the local deployment directory. This is a quick way to deploy if you have the sources. ```bash kubectl apply -k deployments/qat/ ``` -------------------------------- ### Copy Built Tools and Update PATH Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/DEV.md Copy the locally built `controller-gen` and `client-gen` binaries to a directory in your PATH, such as `$HOME/go/bin`. Verify the directory is included in the PATH. ```bash cp controller-tools/controller-gen code-generator/client-gen $HOME/go/bin # ensure it's in the path. You may want to add export to $HOME/.bashrc echo $PATH | grep -q $HOME/go/bin || export PATH=$HOME/go/bin:$PATH ``` -------------------------------- ### List ResourceSlices Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gaudi/USAGE.md Use this command to list all ResourceSlice objects after the kubelet-plugin pods are ready. This helps verify that hardware discovery and announcement are functioning correctly. ```bash $ kubectl get resourceSlices NAME NODE DRIVER POOL AGE rpl-s-gaudi.intel.com-x8m4h rpl-s gaudi.intel.com rpl-s 4d1h ``` -------------------------------- ### Build container image locally Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/qat/BUILD.md Executes the default build process from the repository root. ```bash make qat-container-build ``` -------------------------------- ### Build the Executable Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/cdi-spec-generator/BUILD.md Build the intel-cdi-specs-generator executable using the provided Makefile. ```bash make bin/intel-cdi-specs-generator ``` -------------------------------- ### Build Intel Gaudi Resource Driver Image Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gaudi/BUILD.md Builds the container image locally using the default configuration. ```bash make gaudi-container-build ``` -------------------------------- ### Deploy Intel GPU Resource Driver from Sources Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gpu/USAGE.md Deploys the Intel GPU Resource Driver using Kustomize from its GitHub repository. Replace with the desired release tag. ```bash kubectl apply -k 'https://github.com/intel/intel-resource-drivers-for-kubernetes/deployments/gpu?ref=' ``` -------------------------------- ### Build Intel GPU Resource Driver container image Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gpu/BUILD.md Builds the container image locally from the repository root using the default configuration. ```bash make gpu-container-build ``` -------------------------------- ### Get ResourceSlice YAML Details Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gaudi/USAGE.md Retrieve the full YAML definition of a specific ResourceSlice object. This is useful for inspecting the discovered hardware attributes, driver information, and node details. ```bash $ kubectl get resourceslices.resource.k8s.io rpl-s-gaudi.intel.com-x8m4h -o yaml apiVersion: resource.k8s.io/v1 kind: ResourceSlice metadata: creationTimestamp: "2026-04-15T07:17:43Z" generateName: rpl-s-gaudi.intel.com- generation: 1 name: rpl-s-gaudi.intel.com-x8m4h ownerReferences: - apiVersion: v1 controller: true kind: Node name: rpl-s uid: 3a243a6b-e6db-4613-94f2-169f938c87ae resourceVersion: "6266269" uid: d093f601-9ad5-4234-91d8-410733e32784 spec: devices: - attributes: healthy: bool: true model: string: Gaudi2 pciRoot: string: "01" resource.kubernetes.io/pcieRoot: string: pci0000:01 serial: string: "" name: 0000-a0-00-0-0x1020 - attributes: healthy: bool: true model: string: Gaudi2 pciRoot: string: "02" resource.kubernetes.io/pcieRoot: string: pci0000:02 serial: string: "" name: 0000-b0-00-0-0x1020 driver: gaudi.intel.com nodeName: rpl-s pool: generation: 1 name: rpl-s resourceSliceCount: 1 ``` -------------------------------- ### Apply Test Case Deployments Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/qat/TESTING.md Apply the resource claim template and test case deployments using kubectl. ```bash kubectl apply -f deployments/qat/tests/resource-claim-template.yaml kubectl apply -k deployments/qat/tests/qatlib-sample-code kubectl apply -k deployments/qat/tests/qat-dpdk-test ``` -------------------------------- ### Get Detailed ResourceSlice YAML Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gpu/USAGE.md Retrieve the full YAML definition of a specific ResourceSlice object. This is crucial for detailed validation of device attributes, capacity, and driver configuration as reported by the node. ```bash $ kubectl get resourceslice/rpl-s-gpu.intel.com-mbr6p -o yaml apiVersion: resource.k8s.io/v1 kind: ResourceSlice metadata: creationTimestamp: "2024-09-27T09:11:24Z" generateName: rpl-s-gpu.intel.com- generation: 1 name: rpl-s-gpu.intel.com-mbr6p ownerReferences: - apiVersion: v1 controller: true kind: Node name: rpl-s uid: 0894e000-e7a3-49ad-8749-04b27be61c03 resourceVersion: "2479360" uid: 305a8e03-fe9b-44ea-831e-01ce70edb1a7 spec: devices: - attributes: driver: string: i915 family: string: Unknown health: string: Healthy model: string: Unknown pciAddress: string: "0000:00:02.0" pciId: string: "0x7d67" pciRoot: string: "00" resource.kubernetes.io/pciBusID: string: "0000:00:02.0" resource.kubernetes.io/pcieRoot: string: pci0000:00 sriov: bool: true capacity: memory: value: "0" millicores: value: 1k name: 0000-00-02-0-0x7d67 - attributes: driver: string: xe family: string: Unknown health: string: Healthy model: string: Unknown pciAddress: string: "0000:04:00.0" pciId: string: "0xe211" pciRoot: string: "00" resource.kubernetes.io/pciBusID: string: "0000:04:00.0" resource.kubernetes.io/pcieRoot: string: pci0000:00 sriov: bool: true capacity: memory: value: 24480Mi millicores: value: 1k name: 0000-04-00-0-0xe211 driver: gpu.intel.com nodeName: rpl-s pool: generation: 0 name: rpl-s resourceSliceCount: 1 ``` -------------------------------- ### Basic GPU Resource Request with ResourceClaim Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gpu/USAGE.md This snippet demonstrates the fundamental use case of requesting a GPU resource using a ResourceClaim and then associating that claim with a Pod. Ensure the `gpu.intel.com` device class is available. ```yaml apiVersion: resource.k8s.io/v1 kind: ResourceClaim metadata: name: claim1 spec: devices: requests: - name: gpu exactly: deviceClassName: gpu.intel.com --- apiVersion: v1 kind: Pod metadata: name: test-claim spec: restartPolicy: Never containers: - name: with-resource image: registry.k8s.io/e2e-test-images/busybox:1.29-2 command: ["sh", "-c", "ls -la /dev/dri/ && sleep 60"] resources: claims: - name: resource resourceClaims: - name: resource resourceClaimName: claim1 ``` -------------------------------- ### Build and Push Container Image Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gaudi/BUILD.md Builds and pushes the container image to a specified registry. ```bash REGISTRY=registry.local make gaudi-container-push ``` ```bash GAUDI_IMAGE_TAG=registry.local/intel-gaudi-resource-driver:latest make gaudi-container-push ``` -------------------------------- ### Show Helm Chart Values Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/charts/intel-gpu-resource-driver/README.md View all configurable options for the Intel GPU Resource Driver Helm chart with detailed comments. ```bash helm show values oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-gpu-resource-driver-chart ``` -------------------------------- ### Verify the Build Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/cdi-spec-generator/BUILD.md Check the version of the built intel-cdi-specs-generator tool to confirm a successful build. ```bash bin/intel-cdi-specs-generator --version ``` -------------------------------- ### Create Kind Cluster Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/CLUSTER_SETUP.md Creates a Kubernetes cluster using kind with a provided configuration file. ```shell kind create cluster --config ./hack/kind-config.yaml ``` -------------------------------- ### Device-Faker Command-Line Help Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/device-faker/README.md Display help information for the device-faker command-line tool. This shows available commands and flags for generating fake sysfs and devfs. ```shell $ device-faker -h ``` -------------------------------- ### Build and Run Tests in Container Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/DEV.md Build a Docker image for testing using `make test-image` and then run tests within that containerized environment using `make test-containerized`. This ensures consistent test execution. ```shell $ make test-image $ make test-containerized ``` -------------------------------- ### Configure ldconfig Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/hack/fake_libhlml/README.md Configure the dynamic linker to use the deployed fake libhlml by adding its path to ld.so.conf.d and then running ldconfig. ```bash cat << EOF | sudo tee /etc/ld.so.conf.d/habanalabs.conf /usr/lib/habanalabs/ EOF sudo ldconfig ``` -------------------------------- ### Clone the Repository Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/cdi-spec-generator/BUILD.md Clone the source code repository for the Intel resource drivers for Kubernetes. ```bash git clone https://github.com/intel/intel-resource-drivers-for-kubernetes.git cd intel-resource-drivers-for-kubernetes ``` -------------------------------- ### Build fake_libhlml Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/hack/fake_libhlml/README.md Build the fake_libhlml shared library by navigating to its directory and running make. ```bash cd hack/fake_libhlml make ``` -------------------------------- ### Deploy fake_libhlml Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/hack/fake_libhlml/README.md Deploy the built fake_libhlml.so to the expected location and rename it to libhlml.so. ```bash sudo mkdir /usr/lib/habanalabs sudo cp hack/fake_libhlml/fake_libhlml.so /usr/lib/habanalabs/libhlml.so ``` -------------------------------- ### List ResourceSlice Objects Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gpu/USAGE.md Use this command to list all ResourceSlice objects currently present in the cluster. This helps in verifying that the kubelet-plugin pods have successfully created and registered the resource slices. ```bash $ kubectl get resourceslices NAME NODE DRIVER POOL AGE rpl-s-gpu.intel.com-mbr6p rpl-s gpu.intel.com rpl-s 30s ``` -------------------------------- ### Build and Deploy Fake libhlml Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/DEV.md Compile a fake `libhlml.so` shared library using `make` in the `hack/fake_libhlml` directory. This mock library is used for testing in environments without actual Gaudi hardware. ```shell $ cd hack/fake_libhlml $ make clean rm -f fake_libhlml.o fake_libhlml.so $ make gcc -O -Wall -Wextra -Wno-unused-parameter -fPIC -c fake_libhlml.c -o fake_libhlml.o gcc -shared -o fake_libhlml.so fake_libhlml.o $ sudo cp ./fake_libhlml.so /usr/lib/habanalabs/libhlml.so $ cat << EOF | sudo tee /etc/ld.so.conf.d/habanalabs.conf /usr/lib/habanalabs/ EOF $ sudo ldconfig ``` -------------------------------- ### Generate Fake File System from Template Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/device-faker/README.md Generates a fake file system structure using a provided device template. This is useful for simulating device environments for testing. ```shell device-faker -t /tmp/gpu-template-3524438793.json gpu ``` ```shell $ device-faker -t /tmp/gpu-template-3524438793.json gpu fake file system: /tmp/test-2503111759/ fake sysfs: /tmp/test-2503111759/sysfs fake devfs: /tmp/test-2503111759/dev fake CDI: /tmp/test-2503111759/cdi $ sudo tree /tmp/test-2503111759/ /tmp/test-2503111759/ ├── cdi ├── dev │   ├── dri │   │   ├── by-path │   │   │   ├── pci-0000:03:00.0-card -> ../card0 │   │   │   ├── pci-0000:03:00.0-render -> ../renderD128 │   │   │   ├── pci-0000:04:00.1-card -> ../card1 │   │   │   └── pci-0000:04:00.1-render -> ../renderD129 │   │   ├── card0 │   │   ├── card1 │   │   ├── renderD128 │   │   └── renderD129 │   ├── mei0 │   └── mei1 ├── kubelet-plugin │   ├── plugins │   │   └── gpu.intel.com │   └── plugins_registry └── sysfs ├── bus │   └── pci │   ├── devices │   │   ├── 0000:03:00.0 -> ../../../devices/pci0000:01/0000:03:00.0 │   │   └── 0000:04:00.1 -> ../../../devices/pci0000:02/0000:04:00.1 │   └── drivers │   ├── i915 │   │   ├── 0000:03:00.0 -> ../../../../devices/pci0000:01/0000:03:00.0 │   │   └── bind │   └── xe │   ├── 0000:04:00.1 -> ../../../../devices/pci0000:02/0000:04:00.1 │   └── bind ├── class │   ├── drm │   │   ├── card0 -> /tmp/test-2503111759/sysfs/bus/pci/drivers/i915/0000:03:00.0/drm/card0 │   │   └── card1 -> /tmp/test-2503111759/sysfs/bus/pci/drivers/xe/0000:04:00.1/drm/card1 │   └── mei │   ├── mei0 -> ../../devices/pci0000:01/0000:03:00.0/i915.mei-gscfi.2304/mei/mei0 │   └── mei1 -> ../../devices/pci0000:02/0000:04:00.1/xe.mei-gscfi.768/mei/mei1 └── devices ├── pci0000:01 │   └── 0000:03:00.0 │   ├── device │   ├── drm │   │   ├── card0 │   │   │   ├── lmem_total_bytes │   │   │   └── prelim_iov │   │   │   ├── pf │   │   │   │   └── auto_provisioning │   │   ├── vf1 │   │   │   └── gt │   │   │   ├── contexts_quota │   │   │   ├── doorbells_quota │   │   │   ├── exec_quantum_ms │   │   │   ├── ggtt_quota │   │   │   ├── lmem_quota │   │   │   └── preempt_timeout_us │   │   ├── vf2 │   │   │   └── gt │   │   │   ├── contexts_quota │   │   │   ├── doorbells_quota │   │   │   ├── exec_quantum_ms │   │   │   ├── ggtt_quota │   │   │   ├── lmem_quota │   │   │   └── preempt_timeout_us │   │   ├── vf3 │   │   │   └── gt │   │   │   ├── contexts_quota │   │   │   ├── doorbells_quota │   │   │   ├── exec_quantum_ms │   │   │   ├── ggtt_quota │   │   │   ├── lmem_quota │   │   │   └── preempt_timeout_us │   │   ├── vf4 │   │   │   └── gt │   │   │   ├── contexts_quota │   │   │   ├── doorbells_quota │   │   │   ├── exec_quantum_ms │   │   │   ├── ggtt_quota ``` -------------------------------- ### Deploy Gaudi Resource Driver from Sources Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gaudi/USAGE.md Apply the Kubernetes resource definitions for the Gaudi resource driver directly from the deployment sources. This method is suitable for development or custom deployments. ```bash kubectl apply -k deployments/gaudi/ ``` -------------------------------- ### Enable Health Monitoring Namespace Labeling Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/charts/intel-gpu-resource-driver/README.md Prepare a Kubernetes namespace for health monitoring by creating it and applying the necessary label for admin access. This is required when health monitoring is enabled. ```bash kubectl create namespace intel-gpu-resource-driver kubectl label namespace intel-gpu-resource-driver resource.kubernetes.io/admin-access=true ``` -------------------------------- ### Generate CDI Specs for Devices Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/cdi-spec-generator/README.md Execute the built executable with the type of device (gpu or gaudi) to generate CDI specs. ```bash intel-cdi-specs-generator ``` -------------------------------- ### Deploy DRA Driver with NFD Rules for Intel GPUs Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gpu/USAGE.md Applies Kustomize configuration to deploy the DRA driver on nodes labeled by NFD for Intel GPUs. Replace with the appropriate release tag. ```bash kubectl apply -k 'https://github.com/intel/intel-resource-drivers-for-kubernetes/deployments/gpu/overlays/nfd_labeled_nodes?ref=' ``` -------------------------------- ### Deploy Test Pod for Gaudi Resource Driver Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gaudi/USAGE.md Apply a Kubernetes manifest to deploy a test pod that utilizes the Gaudi resource driver. This is used to verify the driver's functionality. ```bash $ kubectl apply -f deployments/examples/pod-inline.yaml resourceclaim.resource.k8s.io/claim1 created pod/test-inline-claim created ``` -------------------------------- ### Generate New Device Template Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/device-faker/README.md Generates a new template file for a specified device type (e.g., 'gpu'). The output includes the path to the newly created template file and its contents. ```shell device-faker -n gpu ``` ```shell $ device-faker -n gpu new template: /tmp/gpu-template-3524438793.json $ cat /tmp/gpu-template-3524438793.json { "card0": { "uid": "0000-03-00-0-0x56c0", "pciaddress": "0000:03:00.0", "model": "0x56c0", "modelname": "", "familyname": "", "meiname": "mei0", "cardname": "card0", "renderdname": "renderD128", "memorymib": 1024, "millicores": 1000, "devicetype": "gpu", "maxvfs": 8, "parentuid": "", "vfprofile": "", "vfindex": 0, "provisioned": false, "driver": "i915", "currentdriver": "", "pciroot": "pci0000:01", "health": "", "healthstatus": null }, "card1": { "uid": "0000-04-00-1-0xe20b", "pciaddress": "0000:04:00.1", "model": "0xe20b", "modelname": "", "familyname": "", "meiname": "mei1", "cardname": "card1", "renderdname": "renderD129", "memorymib": 2048, "millicores": 1000, "devicetype": "gpu", "maxvfs": 0, "parentuid": "0000-04-00-0-0xe20b", "vfprofile": "", "vfindex": 0, "provisioned": false, "driver": "xe", "currentdriver": "", "pciroot": "pci0000:02", "health": "", "healthstatus": null } } ``` -------------------------------- ### Build with Custom Image Variables Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gaudi/BUILD.md Overrides registry, image name, or version variables during the build process. ```bash REGISTRY=myregistry GAUDI_IMAGE_NAME=myimage GAUDI_IMAGE_VERSION=myversion make gaudi-container-build ``` ```bash GAUDI_IMAGE_TAG=myregistry/myimagename:myversion make gaudi-container-build ``` -------------------------------- ### Deploy Test Pod with Inline GPU Resource Claim Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gpu/USAGE.md Apply a Kubernetes resource definition to deploy a test pod that utilizes an inline GPU resource claim. This is the initial step to verify GPU resource driver functionality. ```bash $ kubectl apply -f 'https://raw.githubusercontent.com/intel/intel-resource-drivers-for-kubernetes/refs/heads/main/deployments/gpu/examples/pod-inline-gpu.yaml' ``` -------------------------------- ### Apply NFD Rules for QAT Labeled Nodes Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/qat/USAGE.md Applies NFD rules to label nodes that have QAT capabilities. This ensures the resource driver is deployed only on relevant nodes. ```bash kubectl apply -k deployments/qat/overlays/nfd_labeled_nodes/ ``` -------------------------------- ### Display CDI Spec Generator Version Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/cdi-spec-generator/README.md Use the --version flag to display the current version of the binary. ```bash intel-cdi-specs-generator --version ``` -------------------------------- ### Deploy DRA Driver with Faked Devices Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/device-faker/README.md Apply the kustomization overlay for device-faker to deploy the DRA driver with simulated devices. Ensure the device-faker container image is built locally before deployment. ```shell kubectl apply -k deployments/gpu/overlay/device-faker ``` -------------------------------- ### Check Assigned GPU in Running Pod Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gpu/USAGE.md After the test pod enters the 'Running' state, retrieve its logs to confirm that a GPU resource has been successfully assigned to it. This verifies the driver's allocation mechanism. ```bash $ kubectl logs pod/test-inline-claim ``` -------------------------------- ### Deploy Gaudi DRA Driver and NFD Rules Together Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gaudi/USAGE.md Apply the Gaudi DRA driver configurations along with NFD rules to ensure the driver is deployed only on nodes labeled by NFD as having Gaudi accelerators. ```bash kubectl apply -k deployments/gaudi/overlays/nfd_labeled_nodes/ ``` -------------------------------- ### Check Assigned Gaudi Device in Pod Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gaudi/USAGE.md After the test pod is running, check its logs to verify that a Gaudi device has been assigned to it. This confirms successful resource allocation. ```bash $ kubectl logs po/test-inline-claim Defaulted container "with-resource" out of: with-resource, without-resource total 0 drwxr-xr-x 2 root root 80 Sep 27 14:30 . drwxr-xr-x 6 root root 380 Sep 27 14:30 .. crw-rw-rw- 1 root root 1, 3 Sep 27 14:30 accel0 crw-rw-rw- 1 root root 1, 3 Sep 27 14:30 accel_controlD0 ``` -------------------------------- ### Device-Faker Usage and Flags Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/device-faker/README.md Shows the usage and available flags for the device-faker tool. Use flags to specify the accelerator type, target directory, template files, and whether to create real device files. ```shell device-faker [flags] Flags: -c, --cleanup Wait for SIGTERM, cleanup before exiting -h, --help help for device-faker -n, --new-template Create new template file for given accelerator -p, --print Print resulting file-system tree -r, --real-devices Create real device files (requires root) -d, --target-dir string Target directory, default is random /tmp/test-* -t, --template string Template file to populate devices from -v, --version Show the version of the binary ``` -------------------------------- ### KubeVirt Device Class Configuration for VM Workloads Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gpu/USAGE.md This configuration snippet shows how to uncomment the selector for the `driver` attribute in a device class definition to prevent GPUs bound to DRM drivers from being allocated for VM workloads when binding management is disabled. ```yaml spec: selector: # driver: vfio-pci ``` -------------------------------- ### Describe Gaudi Device Class Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/gaudi/USAGE.md Use `kubectl describe deviceclass` to check if the DRA feature is enabled and successfully activated, specifically looking for the 'Extended Resource Name' for the Gaudi device class. ```shell kubectl describe deviceclass/gaudi.intel.com ``` -------------------------------- ### Generate Coverage Report Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/DEV.md Generate a coverage report for the tests by running the `make gaudi-coverage` target. This provides insights into test coverage for the Gaudi driver. ```shell make gaudi-coverage ``` -------------------------------- ### Enable CDI in Containerd Configuration Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/DEV.md Configure Containerd to enable CDI injection by setting `enable_cdi` to true and specifying `cdi_spec_dirs` in the `config.toml` file. Ensure Containerd version is v1.7+. ```toml version = 2 [plugins] [plugins."io.containerd.grpc.v1.cri"] enable_cdi = true cdi_spec_dirs = ["/etc/cdi", "/var/run/cdi"] ``` -------------------------------- ### Restart Containerd and Exit Minikube Source: https://github.com/intel/intel-resource-drivers-for-kubernetes/blob/main/doc/CLUSTER_SETUP.md Commands to restart the containerd service and exit the minikube shell. ```bash systemctl restart containerd ``` ```bash exit ```