### Setup Kind Cluster and Install Volcano Source: https://github.com/volcano-sh/volcano/blob/master/benchmark/README.md Sets up a Kind cluster, builds local images, and installs Volcano with monitoring. Use this to test Volcano from local source. ```bash cd benchmark # Setup: create Kind cluster, build images, install Volcano + monitoring make setup VOLCANO_VERSION=v1.14.2 # Run gang scheduling benchmark make test-gang-env JOBS=10 REPLICAS=100 MIN_AVAILABLE=100 # Cleanup make cleanup-all ``` -------------------------------- ### Install Argo Workflows and Submit MPI Job Source: https://github.com/volcano-sh/volcano/blob/master/example/integrations/argo/mpi/README.md Installs Argo Workflows using the quick-start guide, applies necessary configuration files, submits an MPI hello-world job using a workflow template, and watches its execution. ```bash kubectl create namespace argo kubectl apply -n argo -f "https://github.com/argoproj/argo-workflows/releases/download/v3.7.10/quick-start-minimal.yaml" kubectl apply -f rbac.yaml kubectl apply -f hello-world.yaml WF_NAME=volcano-mpi-hello-$(date +%s) argo submit --from workflowtemplate/volcano-mpi-hello -n argo --name "$WF_NAME" argo watch "$WF_NAME" -n argo ``` -------------------------------- ### Node Allocatable Resources Example Source: https://github.com/volcano-sh/volcano/blob/master/docs/user-guide/how_to_use_volcano_vgpu.md Example of node allocatable resources that should include vGPU memory and number. Ensure these are present after setup. ```yaml volcano.sh/vgpu-memory: "89424" volcano.sh/vgpu-number: "8" ``` -------------------------------- ### Apply MindSpore CPU Example Configuration Source: https://github.com/volcano-sh/volcano/blob/master/example/MindSpore-example/README.md Apply the Kubernetes configuration file to run the MindSpore CPU example. ```bash kubectl apply -f mindspore-cpu.yaml ``` -------------------------------- ### Kubelet Configuration Example Source: https://github.com/volcano-sh/volcano/blob/master/docs/user-guide/how_to_use_numa_aware.md Example snippet of a kubelet configuration file showing static CPU management and best-effort topology management policies. ```yaml {...} cpuManagerPolicy: static topologyManagerPolicy: best-effort kubeReserved: cpu: 1000m ``` -------------------------------- ### Install Volcano from Code Source: https://github.com/volcano-sh/volcano/blob/master/README.md Use this script for a one-click installation from the code base if you do not have a Kubernetes cluster. This method is temporarily only available for x86_64. ```bash ./hack/local-up-volcano.sh ``` -------------------------------- ### Check MindSpore CPU Example Logs Source: https://github.com/volcano-sh/volcano/blob/master/example/MindSpore-example/README.md Retrieve the logs from the MindSpore CPU example pod to check the results. ```bash kubectl logs mindspore-cpu-pod-0 ``` -------------------------------- ### Install Monitoring with Audit-Exporter Source: https://github.com/volcano-sh/volcano/blob/master/benchmark/README.md Install the monitoring stack, optionally specifying a custom audit-exporter image from your private registry. ```bash # Using the default audit-exporter image name make install-monitoring # Using a custom image from your private registry make install-monitoring AUDIT_EXPORTER_IMAGE=myregistry.example.com/audit-exporter:dev ``` -------------------------------- ### Detailed Dynamic Plugin Configuration Example Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/plugin-conf.md An example demonstrating the dynamic configuration of actions and plugin tiers, including priority settings and specific plugin options like disabling task order for 'drf'. ```yaml actions: "reclaim, allocate, backfill, preempt" tiers: - plugins: - name: "priority" - name: "gang" - plugins: - name: "drf" disableTaskOrder: true - name: "predicates" - name: "proportion" ``` -------------------------------- ### Install Volcano with Helm Source: https://github.com/volcano-sh/volcano/blob/master/README.md Add the Volcano Helm repository and install the official release. For development, install from the chart in the source code. ```bash helm repo add volcano-sh https://volcano-sh.github.io/helm-charts helm install volcano volcano-sh/volcano -n volcano-system --create-namespace ``` ```bash helm install volcano installer/helm/chart/volcano --namespace volcano-system --create-namespace ``` ```bash helm list -n volcano-system ``` -------------------------------- ### Scheduler Policy Configuration Example Source: https://github.com/volcano-sh/volcano/blob/master/pkg/controllers/sharding/policy/README.md Example YAML configuration for defining scheduler policies, including their names, weights, and arguments. ```yaml schedulerConfigs: - name: volcano type: volcano policies: - name: allocation-rate weight: 1 arguments: minCPUUtil: 0.0 maxCPUUtil: 0.6 - name: warmup weight: 2 # higher weight = prefer warmup nodes first arguments: warmupLabel: "node.volcano.sh/warmup" warmupLabelValue: "true" - name: node-limit arguments: minNodes: 2 maxNodes: 100 - name: agent-scheduler type: agent policies: - name: allocation-rate weight: 1 arguments: minCPUUtil: 0.7 maxCPUUtil: 1.0 - name: warmup weight: 5 # very strong warmup preference for agent shard - name: node-limit arguments: minNodes: 2 maxNodes: 50 ``` -------------------------------- ### Run JobFlow Example Source: https://github.com/volcano-sh/volcano/blob/master/example/jobflow/README.md Deploy a JobTemplate first, then a JobFlow. Check their status using kubectl commands. ```bash # deploy jobTemplate first kubectl apply -f JobTemplate.yaml # deploy jobFlow second kubectl apply -f JobFlow.yaml # check them kubectl get jt kubectl get jf kubectl get po ``` -------------------------------- ### CPU Throttle Configuration Example Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/cpu-throttle-design.md Provides an example YAML configuration for the CPU throttling feature, specifying thresholds for throttling, jitter, and recovery. ```yaml cpuThrottlingConfig: enable: true cpuThrottlingThreshold: 80 # Allow BE quota up to 80% of allocatable CPU cpuJitterLimitPercent: 1 # Emit updates when quota changes by >=1% cpuRecoverLimitPercent: 10 # Cap quota increases to 10% per update ``` -------------------------------- ### Whole-Device Quota Example Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/capacity-dra-support.md Use 'deviceclass/' when the queue should be limited by device count. This example defines a queue with a whole-device GPU quota. ```yaml apiVersion: scheduling.volcano.sh/v1beta1 kind: Queue metadata: name: gpu-team spec: reclaimable: true capability: cpu: "64" memory: "256Gi" "deviceclass/gpu.nvidia.com": "8" deserved: "deviceclass/gpu.nvidia.com": "4" guarantee: "deviceclass/gpu.nvidia.com": "1" ``` -------------------------------- ### Install Volcano Source: https://github.com/volcano-sh/volcano/blob/master/benchmark/README.md Command to install Volcano on the cluster if it's not already present. Use SKIP_INSTALL_VOLCANO=true if Volcano is pre-installed. ```bash make install-volcano VOLCANO_VERSION=v1.14.2 ``` -------------------------------- ### Example TensorFlow Plugin Configuration Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/distributed-framework-plugins.md Users can enable the TensorFlow plugin by adding a few lines to their job spec. This configuration is a minimal example. ```yaml spec: plugins: tensorflow: [] ``` -------------------------------- ### Pod Example for GPU Exclusivity Source: https://github.com/volcano-sh/volcano/blob/master/docs/user-guide/how_to_use_volcano_vgpu.md This pod example demonstrates how to apply GPU exclusivity by including labels that match a defined rule. Ensure the pod's labels align with the `GPUExclusiveRules` configured in the deviceshare plugin to receive dedicated GPUs. ```yaml metadata: name: training-job labels: workloadType: training spec: schedulerName: volcano containers: - name: trainer image: nvidia/cuda:12.0-base resources: limits: volcano.sh/vgpu-number: "4" volcano.sh/vgpu-memory: "16384" ``` -------------------------------- ### Install Volcano Monitoring System Source: https://github.com/volcano-sh/volcano/blob/master/README.md Apply the monitoring YAML file to set up Prometheus and Grafana dashboards for Volcano after installation. ```bash kubectl create -f installer/volcano-monitoring.yaml ``` -------------------------------- ### Pod Status and Gates Example Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/scheduling-gates-queue-admission.md Illustrates the state of pods and their associated gates during the scheduling process. ```text NAME PHASE CONDITION GATES pod-1 Pending SchedulingGated scheduling.volcano.sh/queue-allocation-gate pod-2 Pending SchedulingGated scheduling.volcano.sh/queue-allocation-gate pod-3 Pending SchedulingGated scheduling.volcano.sh/queue-allocation-gate ``` ```text NAME PHASE CONDITION GATES pod-1 Running pod-2 Pending SchedulingGated scheduling.volcano.sh/queue-allocation-gate pod-3 Pending SchedulingGated scheduling.volcano.sh/queue-allocation-gate ``` ```text NAME PHASE CONDITION GATES pod-2 Pending Unschedulable pod-3 Running ``` -------------------------------- ### Network Topology Discovery Configuration Example Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/hyperNode-auto-discovery.md Example configuration for network topology discovery sources in `volcano-controller.conf`. Supports UFM, RoCE, and label-based discovery with customizable intervals and credentials. ```yaml networkTopologyDiscovery: - source: ufm enabled: true interval: 10m credentials: secretRef: name: ufm-credentials namespace: volcano-system config: [...] - source: roce enabled: false interval: 15m config: [...] - source: label enabled: true config: networkTopologyTypes: rack-level-topology: # 2-level topology - nodeLabel: "volcano.sh/tor" - nodeLabel: "kubernetes.io/hostname" cluster-level-topology: # 3-level topology - nodeLabel: "volcano.sh/hypercluster" - nodeLabel: "volcano.sh/hypernode" - nodeLabel: "kubernetes.io/hostname" ``` -------------------------------- ### HyperNode Manifest Example Source: https://github.com/volcano-sh/volcano/blob/master/benchmark/README.md Example YAML manifest for a HyperNode, demonstrating how to define topology tiers and member selectors for real nodes. ```yaml apiVersion: topology.volcano.sh/v1alpha1 kind: HyperNode metadata: name: rack-0 spec: tier: 1 tierName: rack members: - type: Node selector: labelMatch: matchLabels: topology-rack: "rack-0" ``` -------------------------------- ### Example Job YAML with Multiple Pod Templates Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/job-api.md An example YAML configuration for a Volcano Job demonstrating the use of multiple task specifications (ps and worker) with different replicas and container images. ```yaml apiVersion: batch.volcano.sh/v1alpha1 kind: Job metadata: name: tf-job spec: tasks: - name: "ps" replicas: 2 template: spec: containers: - name: ps image: ps-img - name: "worker" replicas: 5 template: spec: containers: - name: worker image: worker-img ``` -------------------------------- ### Apply MindSpore GPU Example Configuration Source: https://github.com/volcano-sh/volcano/blob/master/example/MindSpore-example/README.md Apply the Kubernetes configuration file to run the MindSpore GPU example using Volcano's mpimaster and mpiworker. ```bash kubectl apply -f mindspore-gpu.yaml ``` -------------------------------- ### Install Monitoring Stack Source: https://github.com/volcano-sh/volcano/blob/master/benchmark/README.md Deploys a monitoring stack including Prometheus, kube-state-metrics, Grafana, and audit-exporter. ```bash make install-monitoring ``` -------------------------------- ### Queue CLI Commands Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/queue/queue.md Provides examples of using the `vcctl` command-line tool to manage queues. ```APIDOC ## Queue CLI Commands ### Description Provides examples of using the `vcctl` command-line tool to manage queues. ### Create Queue Creates a new queue with a specified name and weight. #### Command ```shell $ vcctl queue create --name --weight ``` #### Example ```shell $ vcctl queue create --name myqueue --weight 10 ``` ### View Queue Displays the details of a specific queue. #### Command ```shell $ vcctl queue view ``` #### Example ```shell $ vcctl queue view myqueue ``` ### List Queues Lists all available queues to the current user. #### Command ```shell $ vcctl queue list ``` #### Example Output ``` Name Weight Total Pending Running myqueue 10 10 5 5 ``` ``` -------------------------------- ### Online Pod Log Example Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/colocation/Overview.md Example output from an online pod's iperf log, showing high bandwidth utilization (e.g., ~900 Mbits/sec). This indicates the workload is receiving the expected network throughput. ```shell Connecting to host 192.168.2.30, port 5201 [ 5] local 192.168.2.115 port 58492 connected to 192.168.2.30 port 5201 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 118 MBytes 990 Mbits/sec [ 5] 1.00-2.00 sec 106 MBytes 889 Mbits/sec [ 5] 2.00-3.00 sec 107 MBytes 897 Mbits/sec [ 5] 3.00-4.00 sec 107 MBytes 903 Mbits/sec [ 5] 4.00-5.00 sec 107 MBytes 899 Mbits/sec [ 5] 5.00-6.00 sec 107 MBytes 902 Mbits/sec [ 5] 6.00-7.00 sec 705 MBytes 884 Mbits/sec ... ``` -------------------------------- ### Mixed Resource Queues Example Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/capacity-dra-support.md A queue can include traditional resources, device-plugin resources, whole-device DRA quota, and DRA capacity dimensions simultaneously. This example demonstrates a queue with a mix of these resource types. ```yaml apiVersion: scheduling.volcano.sh/v1beta1 kind: Queue metadata: name: ml-team spec: reclaimable: true capability: cpu: "100" memory: "200Gi" "nvidia.com/gpu": "4" "deviceclass/gpu.nvidia.com": "8" "cores.deviceclass/hami-core-gpu.project-hami.io": "800" "memory.deviceclass/hami-core-gpu.project-hami.io": "320Gi" ``` -------------------------------- ### Offline Pod Log Example Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/colocation/Overview.md Example output from an offline pod's iperf log, showing significantly lower bandwidth utilization (e.g., ~70 Mbits/sec). This demonstrates bandwidth suppression when online workloads are active. ```shell Connecting to host 192.168.2.30, port 5201 [ 5] local 192.168.2.115 port 44362 connected to 192.168.2.30 port 5201 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 8 MBytes 70 Mbits/sec [ 5] 1.00-2.00 sec 12 MBytes 102 Mbits/sec [ 5] 2.00-3.00 sec 11 MBytes 98 Mbits/sec [ 5] 3.00-4.00 sec 11 MBytes 99 Mbits/sec [ 5] 4.00-5.00 sec 11 MBytes 99 Mbits/sec [ 5] 5.00-6.00 sec 11 MBytes 97 Mbits/sec [ 5] 6.00-7.00 sec 11 MBytes 98 Mbits/sec ... ``` -------------------------------- ### Example Hierarchy Structure Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/hdrf.md Illustrates a hierarchical structure with queues and their weights, showing how resources might be distributed. ```text n / \ n1 w=50% n2 w=50% (0 CPU,1 GPU) / \ n2,1 w=50% n2,2 w=50% (1 CPU,0 GPU) (0 CPU,1 GPU) ``` -------------------------------- ### Demo Job with Dedicated Volumes in YAML Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/dedicated-volume.md An example Volcano Job demonstrating how to configure dedicated volumes using `generateName` within the `volumes` section of the task template. This setup ensures each pod gets its own PVC. ```yaml apiVersion: batch.volcano.sh/v1alpha1 kind: Job metadata: name: demo spec: tasks: - name: task replicas: 2 template: spec: containers: - name: ps image: ps-img volumes: - mountPath: /data generateName: train-pvc- volumeClaim: accessModes: - ReadWriteOnce resources: requests: storage: 8Gi ``` -------------------------------- ### Setup for Existing Clusters Source: https://github.com/volcano-sh/volcano/blob/master/benchmark/README.md Commands to set up the benchmarking environment on an existing Kubernetes cluster, skipping Kind cluster creation and image builds. ```bash cd benchmark ``` ```bash # Volcano not installed yet, let the framework install it make setup USE_EXISTING_CLUSTER=true VOLCANO_VERSION=v1.14.2 ``` ```bash # Volcano already installed (e.g., a custom or modified build) make setup USE_EXISTING_CLUSTER=true SKIP_INSTALL_VOLCANO=true ``` ```bash # Both Volcano and monitoring already deployed, only create KWOK nodes make setup USE_EXISTING_CLUSTER=true SKIP_INSTALL_VOLCANO=true SKIP_INSTALL_MONITORING=true ``` ```bash # Cluster already has real nodes, skip KWOK entirely make setup USE_EXISTING_CLUSTER=true SKIP_INSTALL_VOLCANO=true SKIP_KWOK=true ``` ```bash # Custom kubeconfig KUBECONFIG=/path/to/kubeconfig make setup USE_EXISTING_CLUSTER=true SKIP_INSTALL_VOLCANO=true ``` -------------------------------- ### Install Volcano Chart with --set Source: https://github.com/volcano-sh/volcano/blob/master/installer/README.md Use the `--set` argument with `helm install` to override default configuration values. This example sets the `basic.image_pull_policy` to `Always`, ensuring Docker images are pulled on each installation. ```bash helm install --name volcano-release --set basic.image_pull_policy=Always volcano/volcano ``` -------------------------------- ### Example Curl Command for Dynamic Log Level Update Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/dynamic-conf.md This command demonstrates how to dynamically update the log level of the Volcano scheduler at runtime using a Unix domain socket. Ensure the socket path and parameters match your setup. ```bash sudo curl --unix-socket /tmp/socks/klog.sock "http://localhost/setlevel?level=5&duration=60s" ``` -------------------------------- ### Verify Prerequisites Source: https://github.com/volcano-sh/volcano/blob/master/benchmark/README.md Checks if Go, Docker, Kind, kubectl, Helm, jq, and make are installed and meet the minimum version requirements. ```bash go version && docker info > /dev/null && kind version && kubectl version --client && helm version && jq --version ``` -------------------------------- ### Install Volcano Source: https://github.com/volcano-sh/volcano/blob/master/benchmark/README.md Installs Volcano into the cluster. Can install from a specific Helm repository version or from local source. ```bash make install-volcano VOLCANO_VERSION=v1.14.2 # from Helm repo ``` ```bash make install-volcano # from local source ``` -------------------------------- ### Build Specific Component (Example) Source: https://github.com/volcano-sh/volcano/blob/master/docs/development/development.md Builds a specific component like `vc-controller-manager` after making local code changes. Ensure `HUB` and `TAG` environment variables are set. ```bash pwd ``` ```bash export HUB=docker.io/yourrepo export TAG=citadel ``` ```bash make vc-controller-manager ``` -------------------------------- ### Task Labels Example Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/Network Topology Aware Scheduling.md Provides an example of labels applied to a Task, which are used for matching against subGroupPolicy. ```yaml labels: volcano.sh/partition-id: "12" example.com/role: worker ``` -------------------------------- ### Setup Kind Cluster with Local Source Source: https://github.com/volcano-sh/volcano/blob/master/benchmark/README.md Builds and sets up a Kind cluster using Volcano from local source code instead of a release version. ```bash make setup ``` -------------------------------- ### Start Custom Informers in SchedulerCache Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/event-handler-framework.md Ensure custom informers are started when the SchedulerCache begins its run cycle. ```go func (sc *SchedulerCache) Run(stopCh <-chan struct{}) { go sc.podInformer.Informer().Run(stopCh) ...... for _, i := range sc.CustomInformers { go i.Run(stopCh) } ...... } ``` -------------------------------- ### Download and Deploy hami-scheduler-device ConfigMap (HAMi Core) Source: https://github.com/volcano-sh/volcano/blob/master/docs/user-guide/how_to_use_vnpu.md Download the ConfigMap, set 'hamiVnpuCore' to true, and then deploy it. This is for the HAMi Core mode. ```bash curl -O https://github.com/Project-HAMi/ascend-device-plugin/blob/main/ascend-device-configmap.yaml ``` ```bash kubectl apply -f ascend-device-configmap.yaml ``` -------------------------------- ### vcctl queue create vs. Slurm scontrol create PartitionName Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/command-line-enhancement.md Compares creating a queue with `vcctl` to creating a partition with `scontrol create` in Slurm. ```bash vcctl queue create -n -w ``` ```bash scontrol create PartitionName= ``` -------------------------------- ### Install Volcano Agent Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/colocation/Overview.md Installs the Volcano agent using kubectl. This command is suitable for both x86_64 and arm64 architectures. ```shell kubectl apply -f https://raw.githubusercontent.com/volcano-sh/volcano/master/installer/volcano-agent-development.yaml ``` -------------------------------- ### Expected Plugin Configuration Output Source: https://github.com/volcano-sh/volcano/blob/master/docs/user-guide/how_to_use_resource_strategy_fit_plugin.md Example output from checking scheduler logs, indicating the resource-strategy-fit plugin has been initialized with specific configuration parameters. ```text Initialize resource-strategy-fit plugin with configuration: {resourceStrategyFitWeight: 10, resources: {...}} ``` -------------------------------- ### Pull MindSpore GPU Example Image Source: https://github.com/volcano-sh/volcano/blob/master/example/MindSpore-example/README.md Use this command to pull the Docker image for the MindSpore GPU example. ```bash docker pull lyd911/mindspore-gpu-example:0.2.0 ``` -------------------------------- ### Scheduler Initialization and Gate Manager Integration Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/scheduling-gates-queue-admission.md Illustrates how the Scheduler creates, starts, and stops the SchGateManager. It also shows how the manager is passed to each scheduling session. ```go // In Scheduler.Run(): pc.schGateManager = gate.NewSchGateManager(pc.cache.Client(), options.ServerOpts.GateRemovalWorkerNum) pc.schGateManager.Start() go func() { <-stopCh pc.schGateManager.Stop() }() // Passed to each session: ssn := framework.OpenSession(pc.cache, plugins, configurations, pc.schGateManager) ``` -------------------------------- ### Pull MindSpore CPU Example Image Source: https://github.com/volcano-sh/volcano/blob/master/example/MindSpore-example/README.md Use this command to pull the Docker image for the MindSpore CPU example. ```bash docker pull lyd911/mindspore-cpu-example:0.2.0 ``` -------------------------------- ### View Kubelet Configuration Source: https://github.com/volcano-sh/volcano/blob/master/docs/user-guide/how_to_use_numa_aware.md Display the current kubelet configuration file to check CPU and topology manager settings. ```bash cat /var/lib/kubelet/config.yaml ``` -------------------------------- ### Install Volcano with YAML Source: https://github.com/volcano-sh/volcano/blob/master/README.md Apply the provided YAML file to install Volcano on your Kubernetes cluster. This method is suitable for both x86_64 and arm64 architectures. ```bash kubectl apply -f https://raw.githubusercontent.com/volcano-sh/volcano/master/installer/volcano-development.yaml ``` -------------------------------- ### Mixed Strategy Resource Strategy Fit Configuration Source: https://github.com/volcano-sh/volcano/blob/master/docs/user-guide/how_to_use_resource_strategy_fit_plugin.md An advanced global configuration example demonstrating a mix of LeastAllocated and MostAllocated strategies for different resources like CPU, memory, and a custom resource. ```yaml arguments: resourceStrategyFitWeight: 15 resources: cpu: type: "LeastAllocated" weight: 3 memory: type: "MostAllocated" weight: 1 example.com/custom-resource: type: "LeastAllocated" weight: 2 ``` -------------------------------- ### Configuration Format Example Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/hyperNode-auto-discovery.md Defines the structure for network topology discovery configuration, including enabled sources and intervals. This format is loaded by the Config Loader. ```yaml apiVersion: v1 kind: ConfigMap metadata: name: network-topology-discovery namespace: volcano-system data: config.yaml: | enabled: true interval: "30s" sources: - type: "ufm" address: "ufm.example.com:8080" auth: secret: "ufm-auth-secret" - type: "roce" address: "/dev/infiniband/ucm0" - type: "label" label_selector: "topology.kubernetes.io/zone" ``` -------------------------------- ### Volcano Resource Configuration Example Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/resource-strategy-fit-scheduling.md This YAML configuration demonstrates how to define resources with exact and wildcard matches, specifying allocation types and weights. It includes CPU and various GPU resource types. ```yaml resources: nvidia.com/gpu-v100: # Exact match - highest priority type: MostAllocated weight: 3 nvidia.com/gpu/*: # Wildcard match - covers other NVIDIA GPUs type: MostAllocated weight: 2 amd.com/gpu/*: # Wildcard match for AMD GPUs type: LeastAllocated weight: 2 cpu: type: LeastAllocated weight: 1 ``` -------------------------------- ### Build Scheduler with Plugin Support (Ubuntu) Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/custom-plugin.md Use this command to build the scheduler with plugin support when using an Ubuntu Linux base image. ```bash make images SUPPORT_PLUGINS=yes ``` -------------------------------- ### Install Volcano with DRA Feature Gates (kubectl) Source: https://github.com/volcano-sh/volcano/blob/master/docs/user-guide/how_to_enable_dra.md Enable the 'DynamicResourceAllocation' feature gate when installing Volcano directly using 'kubectl apply -f'. ```yaml --feature-gates=DynamicResourceAllocation=true ``` -------------------------------- ### Example DRA Usage in Queue Status Source: https://github.com/volcano-sh/volcano/blob/master/docs/user-guide/how_to_use_dra_quota.md An example of how DRA usage is reported in the status.allocated field of a queue, showing allocated resources like GPUs and cores. ```yaml status: allocated: "deviceclass/gpu.nvidia.com": "4" "cores.deviceclass/hami-core-gpu.project-hami.io": "400" ``` -------------------------------- ### Install Volcano with DRA Feature Gates (Helm) Source: https://github.com/volcano-sh/volcano/blob/master/docs/user-guide/how_to_enable_dra.md Install Volcano using Helm, enabling the 'DynamicResourceAllocation' feature gate. Other necessary Helm values can be included as needed. ```bash helm install volcano volcano/volcano --namespace volcano-system --create-namespace \ --set custom.scheduler_feature_gates="DynamicResourceAllocation=true" \ # Add other necessary Helm values for your installation ``` -------------------------------- ### Get Scheduler Metrics Source: https://github.com/volcano-sh/volcano/blob/master/docs/user-guide/how_to_use_volcano_vgpu.md Retrieve scheduler metrics by sending a GET request to the /metrics endpoint. Ensure you replace with the actual IP address of your Volcano scheduler. ```bash curl http://:8080/metrics ``` -------------------------------- ### Example HyperNode CR for Spine-Leaf Topology Source: https://github.com/volcano-sh/volcano/blob/master/docs/design/Network Topology Aware Scheduling.md Illustrates the definition of a spine-leaf network topology using HyperNode CRs. This example shows multiple tiers representing switches and nodes. ```yaml version: topology.volcano.sh/v1alpha1 kind: HyperNode name: s0 spec: tier: 1 members: - name: node0 - name: node1 --- version: topology.volcano.sh/v1alpha1 kind: HyperNode name: s1 spec: tier: 1 members: - name: node2 - name: node3 --- version: topology.volcano.sh/v1alpha1 kind: HyperNode name: s2 spec: tier: 1 members: - name: node4 - name: node5 --- version: topology.volcano.sh/v1alpha1 kind: HyperNode name: s3 spec: tier: 1 members: - name: node6 - name: node7 --- version: topology.volcano.sh/v1alpha1 kind: HyperNode name: s4 spec: tier: 2 members: - name: s0 - name: s1 --- version: topology.volcano.sh/v1alpha1 kind: HyperNode name: s5 spec: tier: 2 members: - name: s2 - name: s3 --- version: topology.volcano.sh/v1alpha1 kind: HyperNode name: s6 spec: tier: 3 members: - name: s4 - name: s5 ```