### ControllerPublishVolume Example Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/csi-driver.md Shows how to attach a LINSTOR volume to a specific node, making it available for mounting. The example specifies the volume ID, node ID, and the required access mode. ```go resp, err := driver.ControllerPublishVolume(ctx, &csi.ControllerPublishVolumeRequest{ VolumeId: "vol-1/0", NodeId: "node-1", VolumeCapability: &csi.VolumeCapability{ AccessMode: &csi.VolumeCapability_AccessMode{ Mode: csi.VolumeCapability_AccessMode_SINGLE_NODE_WRITER, }, }, }) if err != nil { log.Fatal(err) } // resp.PublishContext["devicePath"] contains /dev/drbd1000 ``` -------------------------------- ### Multiple Zones (OR Logic) Example Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/topology-scheduler.md Example of a TopologyRequirement with multiple requisite zones, implementing OR logic. ```go // Pod can run in zone 1a OR zone 1b topoReq := &csi.TopologyRequirement{ Requisite: []*csi.Topology{ {Segments: map[string]string{ "topology.kubernetes.io/zone": "us-west-1a", }}, {Segments: map[string]string{ "topology.kubernetes.io/zone": "us-west-1b", }}, }, } ``` -------------------------------- ### Complete Linstor CSI Configuration Example Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/configuration.md A comprehensive example demonstrating how to configure the Linstor CSI driver with various options, including LINSTOR endpoint, CSI endpoint, node name, log level, API request limits, property namespaces, caching timeouts, resync intervals, and NFS-specific settings. ```bash ./linstor-csi \ --linstor-endpoint http://linstor-controller:3370 \ --csi-endpoint unix:///var/lib/kubelet/plugins/linstor.csi.linbit.com/csi.sock \ --node k8s-node-1 \ --log-level info \ --linstor-api-requests-per-second 50 \ --linstor-api-burst 5 \ --property-namespace Aux \ --label-by-storage-pool=true \ --node-cache-timeout 1m \ --resource-cache-timeout 30s \ --resync-after 5m \ --enable-rwx \ --nfs-service-namespace linstor-csi \ --nfs-reactor-config-map-name linstor-csi-nfs-reactor-config \ --default-remote-access-policy=true ``` -------------------------------- ### Volume Creation with Topology Constraints Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/topology-scheduler.md Example of creating a volume with specific topology constraints. The AccessibleTopology field guides the scheduler to place the volume in a desired zone. ```go req := &csi.CreateVolumeRequest{ Name: "my-volume", CapacityRange: &csi.CapacityRange{RequiredBytes: 1e9}, VolumeCapabilities: []*csi.VolumeCapability{{ AccessMode: &csi.VolumeCapability_AccessMode{ Mode: csi.VolumeCapability_AccessMode_SINGLE_NODE_WRITER, }, }}, Parameters: map[string]string{ "linstor.csi.linbit.com/placementPolicy": "autoplaceTopology", "linstor.csi.linbit.com/placementCount": "3", }, AccessibleTopology: []*csi.Topology{{ Segments: map[string]string{ "topology.kubernetes.io/zone": "us-west-1a", }}, }, } resp, err := driver.CreateVolume(ctx, req) ``` -------------------------------- ### Complete LINSTOR CSI StorageClass Example Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/volume-parameters.md A comprehensive example of a Kubernetes StorageClass for LINSTOR CSI, demonstrating configurations for replica placement, filesystem options, security, capacity, DRBD, and snapshots. ```yaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: linstor-production provisioner: linstor.csi.linbit.com allowVolumeExpansion: true parameters: # Replica placement linstor.csi.linbit.com/placementCount: "3" linstor.csi.linbit.com/storagePool: "lvm-thin nvme" linstor.csi.linbit.com/placementPolicy: "autoplaceTopology" linstor.csi.linbit.com/replicasOnDifferent: "topology.kubernetes.io/zone" # Filesystem csi.storage.k8s.io/fstype: xfs linstor.csi.linbit.com/fsopts: "-m 0" linstor.csi.linbit.com/mountopts: "noatime,nodiratime,discard" linstor.csi.linbit.com/postMountXfsOpts: "extsize=16m" # Security linstor.csi.linbit.com/encryption: "false" linstor.csi.linbit.com/allowRemoteVolumeAccess: "true" # Capacity linstor.csi.linbit.com/overProvision: "0.8" # DRBD options property.linstor.csi.linbit.com/DrbdOptions/auto-quorum: "suspend-io" # Snapshots snap.linstor.csi.linbit.com/backupRemote: "s3" snap.linstor.csi.linbit.com/backupRetentionCount: "7" ``` -------------------------------- ### Preferred Constraint Example Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/topology-scheduler.md Example of a TopologyRequirement with a preferred zone, allowing fallback to other zones. ```go // Pod prefers zone 1a but can use other zones topoReq := &csi.TopologyRequirement{ Preferred: []*csi.Topology{ {Segments: map[string]string{ "topology.kubernetes.io/zone": "us-west-1a", }}, }, } ``` -------------------------------- ### Start CSI gRPC Server Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/csi-driver.md Starts the CSI gRPC server on a given listener. The server runs until the provided context is cancelled. Ensure a network listener is created before calling this function. ```go listener, err := net.Listen("unix", "/var/run/csi.sock") if err != nil { log.Fatal(err) } c, cancel := context.WithCancel(context.Background()) deffer cancel() if err := driver.Serve(c, listener); err != nil { log.Fatal(err) } ``` -------------------------------- ### CreateVolume Example Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/csi-driver.md Demonstrates how to create a new LINSTOR volume with specified capacity, access mode, and storage class parameters. Ensure the necessary LINSTOR parameters like placementCount and storagePool are provided. ```go resp, err := driver.CreateVolume(ctx, &csi.CreateVolumeRequest{ Name: "my-volume", CapacityRange: &csi.CapacityRange{ RequiredBytes: 1073741824, // 1 GB }, VolumeCapabilities: []*csi.VolumeCapability{{ AccessMode: &csi.VolumeCapability_AccessMode{ Mode: csi.VolumeCapability_AccessMode_SINGLE_NODE_WRITER, }, }}, Parameters: map[string]string{ "linstor.csi.linbit.com/placementCount": "2", "linstor.csi.linbit.com/storagePool": "my-pool", }, }) if err != nil { log.Fatal(err) } fmt.Printf("Created volume: %s\n", resp.Volume.VolumeId) ``` -------------------------------- ### Follow Topology Scheduler Example Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/topology-scheduler.md Demonstrates the followtopology scheduler, which strictly adheres to specified topology segments and fails if constraints cannot be met. This is useful for strict compliance scenarios. ```go sched := &followtopology.Scheduler{} topoReq := &csi.TopologyRequirement{ Requisite: []*csi.Topology{ {Segments: map[string]string{ "topology.kubernetes.io/zone": "us-west-1a", }}, {Segments: map[string]string{ "topology.kubernetes.io/zone": "us-west-1b", }}, }, } // Places 1 replica in zone 1a, 1 in zone 1b nodes, err := sched.Schedule(ctx, allNodes, 2, topoReq) ``` -------------------------------- ### Zone Constraint Example Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/topology-scheduler.md Example of a TopologyRequirement specifying a single requisite zone. ```go // Pod must run in zone us-west-1a topoReq := &csi.TopologyRequirement{ Requisite: []*csi.Topology{ {Segments: map[string]string{ "topology.kubernetes.io/zone": "us-west-1a", }}, }, } ``` -------------------------------- ### Manual Scheduler Example Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/topology-scheduler.md Illustrates the manual scheduler, which uses an explicitly provided node list and ignores topology requirements. The Schedule method here is called with the provided node list. ```go sched := &manual.Scheduler{} params := &volume.Parameters{ NodeList: []string{"node-1", "node-2", "node-3"}, } // Schedule doesn't make decisions for manual placement nodes, err := sched.Schedule(ctx, params.NodeList, int32(len(params.NodeList)), nil) ``` -------------------------------- ### Autoplace Scheduler Example Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/topology-scheduler.md Uses the autoplace scheduler for fastest placement in homogeneous clusters. It selects nodes with the most available capacity and ignores topology requirements. ```go sched := &autoplace.Scheduler{} nodes, err := sched.Schedule(ctx, allNodes, 3, nil) // Returns: 3 nodes with highest available capacity ``` -------------------------------- ### Get Plugin Information Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/csi-driver.md Retrieves the driver's name and vendor version. Use this to identify the CSI driver. ```go func (d *Driver) GetPluginInfo(ctx context.Context, req *csi.GetPluginInfoRequest) (*csi.GetPluginInfoResponse, error) ``` ```go info, err := driver.GetPluginInfo(ctx, &csi.GetPluginInfoRequest{}) if err != nil { log.Fatal(err) } fmt.Printf("Driver: %s v%s\n", info.Name, info.VendorVersion) ``` -------------------------------- ### Example Topology Segments in Go Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/README.md Illustrates the structure of topology segments used for volume placement. This map defines properties that control where a volume can be scheduled. ```go map[string]string{ "topology.kubernetes.io/zone": "us-west-1a", "kubernetes.io/hostname": "node-1", "linstor.csi.linbit.com/node": "node-1", } ``` -------------------------------- ### Create and Configure Linstor Client Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/linstor-client.md Instantiates a new Linstor client with custom configuration options. This example demonstrates setting the API client, log level, property namespace, and node labeling behavior. ```go client, err := client.NewLinstor( client.APIClient(myAPIClient), client.LogLevel("debug"), client.PropertyNamespace("Aux"), client.LabelBySP(true), ) if err != nil { log.Fatal(err) } ``` -------------------------------- ### Go Example for Topology Caching Configuration Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/topology-scheduler.md Shows how to configure caching for topology queries in Go using `lapicache`. This improves performance by caching node properties and resource deployments for a specified duration. ```go lapicache.WithCaches( &lapicache.NodeCache{Timeout: 1 * time.Minute}, // Cache node properties &lapicache.ResourceCache{Timeout: 30 * time.Second}, // Cache deployments ) ``` -------------------------------- ### Specify Explicit Node with LinstorNodeKey Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/topology-scheduler.md Example of using the LinstorNodeKey to explicitly request a specific node for volume placement. ```go segments := map[string]string{ topology.LinstorNodeKey: "node-1", // Explicitly request node-1 } ``` -------------------------------- ### Serve CSI Driver Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/csi-driver.md Starts the CSI gRPC server on the given listener and blocks until the context is cancelled. This is the primary method for running the CSI driver. ```APIDOC ## Serve ### Description Starts the CSI gRPC server on the given listener and blocks until context is cancelled. ### Method Signature ```go func (d *Driver) Serve(ctx context.Context, listener net.Listener) error ``` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters Table | Parameter | Type | Description | |-----------|------|-------------| | ctx | context.Context | Server context; cancellation stops the server | | listener | `net.Listener` | Network listener (typically Unix socket) | ### Returns Error if server fails to start or serve ### Example ```go listener, err := net.Listen("unix", "/var/run/csi.sock") if err != nil { log.Fatal(err) } cancel := context.WithCancel(context.Background()) def cancel() // Ensure cancel is called if err := driver.Serve(ctx, listener); err != nil { log.Fatal(err) } ``` ``` -------------------------------- ### DRBD Reactor Promoter Configuration for NFS Volume Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/docs/design/read-write-many.md This TOML configuration defines the DRBD Reactor Promoter setup for an NFS volume. It specifies services to start and stop, metadata for the NFS export, and squash options. Ensure systemd escaping is applied to template parameters when using this in a production environment. ```toml [[promoters]] [promoter.resources.vol1] runner = "systemd" target-as = "BindsTo" stop-services-on-exit = true start = [ "prepare-device-links@pvc-12345.service", "clean-nfs-endpoint@pvc-12345.service", "mount-recovery@pvc-12345.service", "mount-export@pvc-12345.service", "growfs@/srv/exports/pvc-12345.timer", "chmod@/srv/exports/pvc-12345.service", "nfs-ganesha@pvc-12345.service", "advertise-nfs-endpoint@pvc-12345.service", ] [promoter.resources.vol1.metadata] service-name = "linstor-csi-nfs" port = 2345 export-id = 2345 config-template-path = "/etc/nfs-helper/default-config.tmpl" squash = "no_root_squash" ``` -------------------------------- ### Go Example for Scheduling Error Handling Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/topology-scheduler.md Demonstrates how to handle potential errors during the scheduling process in Go. It specifically checks for topology-related errors and other general scheduling failures, returning appropriate status codes. ```go nodes, err := scheduler.Schedule(ctx, availNodes, placementCount, topoReq) if err != nil { if strings.Contains(err.Error(), "topology") { // Topology constraint violation return nil, status.Errorf(codes.FailedPrecondition, "cannot satisfy topology: %v", err) } // Other errors return nil, status.Errorf(codes.Unavailable, "scheduling failed: %v", err) } ``` -------------------------------- ### Example Error Message Format Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/errors.md Illustrates the expected format for error messages when a delete operation fails due to the resource being in a deletion state. ```text '' failed, is being deleted ``` -------------------------------- ### Example Error Message Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/errors.md A concrete example of an error message indicating a volume deletion is in progress. ```text 'Delete' failed, Volume my-volume is being deleted ``` -------------------------------- ### Example StorageClass Configuration Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/configuration.md Defines a basic Linstor CSI StorageClass with parameters for placement count, storage pool, encryption, and DRBD options. Ensure the specified storage pool exists in your LINSTOR configuration. ```yaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: linstor-basic provisioner: linstor.csi.linbit.com allowVolumeExpansion: true parameters: linstor.csi.linbit.com/placementCount: "2" linstor.csi.linbit.com/storagePool: "lvm-thin" linstor.csi.linbit.com/encryption: "true" property.linstor.csi.linbit.com/DrbdOptions/auto-quorum: "suspend-io" ``` -------------------------------- ### List All LINSTOR Volumes with Status (Go) Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/linstor-client.md Lists all LINSTOR-managed volumes and retrieves their current status and conditions. Use this to get an overview of all volumes. ```go volumes, err := client.ListAllWithStatus(ctx) if err != nil { log.Fatal(err) } for _, vol := range volumes { fmt.Printf("Volume: %s, Size: %d bytes\n", vol.ResourceName, vol.Size()) } ``` -------------------------------- ### Kubernetes StorageClass for Linstor CSI Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/README.md Define a Kubernetes StorageClass to provision LINSTOR volumes. This example specifies the provisioner, allows volume expansion, and configures parameters like placement count, storage pool, resource group, and filesystem type. It also shows how to override LINSTOR properties. ```yaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: linstor-basic-storage provisioner: linstor.csi.linbit.com allowVolumeExpansion: true parameters: linstor.csi.linbit.com/placementCount: "2" linstor.csi.linbit.com/storagePool: "my-storage-pool" linstor.csi.linbit.com/resourceGroup: "linstor-basic-storage" csi.storage.k8s.io/fstype: xfs # You can override LINSTOR properties by adding the property.linstor.csi.linbit.com prefix: property.linstor.csi.linbit.com/DrbdOptions/auto-quorum: suspend-io ``` -------------------------------- ### Get Plugin Capabilities Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/csi-driver.md Declares the CSI capabilities supported by the driver, such as controller service, volume expansion, and topology awareness. This method is called by the CSI controller to understand what features the driver offers. ```go func (d *Driver) GetPluginCapabilities(ctx context.Context, req *csi.GetPluginCapabilitiesRequest) (*csi.GetPluginCapabilitiesResponse, error) ``` -------------------------------- ### Topology Segment Creation Example Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/topology-scheduler.md Illustrates the transformation of a LINSTOR property into a topology segment. This is used when calculating accessible topologies by reading LINSTOR node properties and filtering them within a configured namespace. ```text LINSTOR Property: "Aux/zone=us-west-1a" → Topology Segment: {"zone": "us-west-1a", "linstor.csi.linbit.com/node": "node-1"} ``` -------------------------------- ### Prepare VM Image with qemu-img Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/examples/kubevirt/README.md Create a qcow2 image, boot a VM from it to configure, convert to raw, compress, and transfer to a storage node. ```bash $ qemu-img create -f qcow2 alpine.qcow2 950M $ qemu-system-x86_64 -m 512 -nic user -boot d -cdrom alpine-virt-3.11.5-x86_64.iso -enable-kvm -hda alpine.qcow2 # setup-alpine; no ssh; reboot; disable the NIC/eth0 in /etc/network/interfaces $ qemu-img convert -O raw alpine.qcow2 -f qcow2 alpine.raw $ xz alpine.raw # scp alpine.raw.xz to one of the storage nodes/hypervisors ``` -------------------------------- ### Set Filesystem Creation Options Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/volume-parameters.md Provide custom options for the `mkfs` command using `linstor.csi.linbit.com/fsopts`. These options are applied during filesystem creation. ```yaml parameters: linstor.csi.linbit.com/fsopts: "-m 0 -O sparse_super" ``` -------------------------------- ### Create Snapshot with Backup Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/linstor-client.md Use this to create one or more snapshots of volumes. Optionally configure backup to remote storage. ```go snapParams := &volume.SnapshotParameters{ BackupRemote: "s3", } snaps, err := client.SnapCreate(ctx, "snap-1", snapParams, volId1, volId2) if err != nil { log.Fatal(err) } fmt.Printf("Created %d snapshots\n", len(snaps)) ``` -------------------------------- ### ListVolumes Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/csi-driver.md Lists LINSTOR-managed volumes with pagination support. Accepts maximum entries and a starting token. ```APIDOC ## ListVolumes ### Description Lists LINSTOR-managed volumes with pagination. ### Method `ListVolumes` ### Request Fields - `MaxEntries` (int32): Maximum volumes to return - `StartingToken` (string): Pagination token ### Response Fields - `Entries` (`[]*csi.ListVolumesResponse_Entry`): Volume list - `NextToken` (string): Token for next page (empty if no more) ``` -------------------------------- ### Filesystem Configuration Parameters Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/volume-parameters.md Configure filesystem type, options, and mount options for your volumes. ```APIDOC ## Filesystem Configuration ### fsType (csi.storage.k8s.io/fstype) **Namespace:** `csi.storage.k8s.io/` (CSI standard) **Type:** String **Required:** No **Default:** `ext4` **Description:** Filesystem type to create on volume. Only set at creation time. **Valid Values:** | Value | Description | |-------|-------------| | `ext4` | ext4 filesystem (default) | | `xfs` | XFS filesystem | | `ext3` | ext3 filesystem | **Example:** ```yaml parameters: csi.storage.k8s.io/fstype: xfs ``` --- ### fsopts **Namespace:** `linstor.csi.linbit.com/` **Type:** String (mkfs command options) **Required:** No **Default:** None **Description:** Options passed to mkfs command during filesystem creation. **Example:** ```yaml parameters: linstor.csi.linbit.com/fsopts: "-m 0 -O sparse_super" ``` --- ### mountopts **Namespace:** `linstor.csi.linbit.com/` **Type:** Comma-separated list of mount options **Required:** No **Default:** None **Description:** Mount options passed to mount command. Cannot be changed after creation. **Example:** ```yaml parameters: linstor.csi.linbit.com/mountopts: "noatime,nodiratime,discard" ``` --- ### postMountXfsOpts **Namespace:** `linstor.csi.linbit.com/` **Type:** String (xfs_io command options) **Required:** No **Default:** None **Description:** Options passed to xfs_io after mounting XFS filesystem. Only used with fsType=xfs. **Example:** ```yaml parameters: linstor.csi.linbit.com/postMountXfsOpts: "extsize=16m" ``` ``` -------------------------------- ### ListAllWithStatus Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/linstor-client.md Lists all LINSTOR-managed volumes with their current status and conditions. This is useful for getting an overview of all managed volumes. ```APIDOC ## ListAllWithStatus ### Description Lists all LINSTOR-managed volumes with current status and conditions. ### Method ```go func (s *Linstor) ListAllWithStatus(ctx context.Context) ([]volume.VolumeStatus, error) ``` ### Returns - `[]VolumeStatus`: List of volumes with their current conditions - `error`: List operation error ### Example ```go volumes, err := client.ListAllWithStatus(ctx) if err != nil { log.Fatal(err) } for _, vol := range volumes { fmt.Printf("Volume: %s, Size: %d bytes\n", vol.ResourceName, vol.Size()) } ``` ``` -------------------------------- ### Optimize Topology Queries with High-Level Client Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/highlevel-client.md Demonstrates fast-path (direct node lookup, uses cache) versus slow-path (property-based lookup, queries LINSTOR) topology resolution. Prefer explicit node names when available. ```go // Fast: direct node lookup (uses cache) nodes, _ := hlc.NodesForTopology(ctx, map[string]string{ "linstor.csi.linbit.com/node": "node-1", }) // Slow: property-based lookup (queries LINSTOR) nodes, _ := hlc.NodesForTopology(ctx, map[string]string{ "topology.kubernetes.io/zone": "us-west-1a", }) ``` -------------------------------- ### Get Volume Status Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/linstor-client.md Retrieves the current status and condition of a volume. Use this to check the health and state of a volume. ```go func (s *Linstor) Status(ctx context.Context, id volume.ID) (*csi.VolumeCondition, error) ``` -------------------------------- ### Get Linstor CSI Driver Version Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/csi-driver.md Retrieves the driver version, which is set at compile time. This value is typically used in the GetPluginInfo response. ```go var Version = "UNKNOWN" ``` -------------------------------- ### Balancer Scheduler with Client Configuration Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/topology-scheduler.md Initializes the balancer scheduler, which places replicas on nodes with the least utilized storage pools to balance load. Requires a HighLevelClient for capacity queries. ```go sched := &balancer.Scheduler{ Client: hlc, // HighLevelClient for capacity queries } nodes, err := sched.Schedule(ctx, allNodes, 3, topoReq) // Returns: 3 nodes with lowest current utilization ``` -------------------------------- ### Get Volume Group Snapshot Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/csi-driver.md Retrieves group snapshot metadata and status using its identifier. Optional credentials can be provided via secrets. ```go func (d *Driver) GetVolumeGroupSnapshot(ctx context.Context, req *csi.GetVolumeGroupSnapshotRequest) (*csi.GetVolumeGroupSnapshotResponse, error) ``` -------------------------------- ### Create New CSI Driver Instance Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/csi-driver.md Instantiates a new CSI driver with a LINSTOR client and optional configuration functions. Ensure the LINSTOR client is properly initialized before creating the driver. ```go drv, err := driver.NewDriver( linstorClient, driver.NodeID("node-1"), driver.LogLevel("info"), ) if err != nil { log.Fatal(err) } ``` -------------------------------- ### Handle Errors in High-Level Client Operations Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/highlevel-client.md Example of handling potential errors when calling GenericAccessibleTopologies. Differentiates between 'not found' errors and other API/network issues. ```go topos, err := hlc.GenericAccessibleTopologies(ctx, volId, policy) if err != nil { if strings.Contains(err.Error(), "not found") { // Volume doesn't exist in LINSTOR return nil, status.Errorf(codes.NotFound, "volume not found") } // Network or API error return nil, status.Errorf(codes.Unavailable, "LINSTOR API error: %v", err) } ``` -------------------------------- ### Direct CSI Driver Logs to an Output Writer Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/csi-driver.md Specifies an `io.Writer` to direct all log output from the CSI driver. This is useful for custom logging setups. ```go func LogOut(out io.Writer) func(*Driver) error ``` -------------------------------- ### Create High-Level LINSTOR Client Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/highlevel-client.md Instantiate a HighLevelClient with optional configuration. Common options include setting the LINSTOR controller URL, providing an authentication token, configuring logging, and enabling response caching. ```go import ( lapi "github.com/LINBIT/golinstor/client" lapicache "github.com/LINBIT/golinstor/cache" ) hlc, err := highlevelclient.NewHighLevelClient( lapi.BaseURL(url), lapi.Log(logger), lapicache.WithCaches( &lapicache.NodeCache{Timeout: 1 * time.Minute}, &lapicache.ResourceCache{Timeout: 30 * time.Second}, ), ) if err != nil { log.Fatal(err) } ``` -------------------------------- ### Access Linstor Storage Pools Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/highlevel-client.md Retrieves all defined storage pools in Linstor. This is an example of using the embedded client to access underlying Linstor API functionalities. ```go // Storage pools pools, err := hlc.StoragePoolDefinitions.GetAll(ctx) ``` -------------------------------- ### Configure Mount Options Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/volume-parameters.md Specify mount options for the volume using `linstor.csi.linbit.com/mountopts`. These are passed to the `mount` command and cannot be changed after creation. ```yaml parameters: linstor.csi.linbit.com/mountopts: "noatime,nodiratime,discard" ``` -------------------------------- ### Get Accessible Topologies for Volume Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/linstor-client.md Retrieve a list of topologies where a specific volume is accessible. This is crucial for understanding data locality and availability in distributed storage environments. ```go func (s *Linstor) AccessibleTopologies(ctx context.Context, id volume.ID, params *volume.Parameters) ([]*csi.Topology, error) ``` -------------------------------- ### Specify Filesystem Type Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/volume-parameters.md Use `csi.storage.k8s.io/fstype` to define the filesystem type for a new volume. Common options include `ext4`, `xfs`, and `ext3`. This parameter is only effective at volume creation. ```yaml parameters: csi.storage.k8s.io/fstype: xfs ``` -------------------------------- ### Get Accessible Topologies for Volume Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/highlevel-client.md Returns CSI topology segments where a volume is accessible based on its deployment and remote access policy. Use this to determine where a volume can be mounted. ```go policy := volume.RemoteAccessPolicyAnywhere topos, err := hlc.GenericAccessibleTopologies(ctx, "vol-1", policy) if err != nil { log.Fatal(err) } for _, topo := range topos { fmt.Printf("Accessible segment: %v\n", topo.Segments) } ``` -------------------------------- ### Set Post-Mount XFS Options Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/volume-parameters.md Apply specific options to the `xfs_io` command after mounting an XFS filesystem using `linstor.csi.linbit.com/postMountXfsOpts`. This parameter is only relevant when `fsType` is set to `xfs`. ```yaml parameters: linstor.csi.linbit.com/postMountXfsOpts: "extsize=16m" ``` -------------------------------- ### Enable RWX Volume Support Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/configuration.md Enable RWX volume support via NFS Ganesha. Requires running in Kubernetes and proper NFS infrastructure. ```bash ./linstor-csi --enable-rwx ``` -------------------------------- ### Topology and Scheduler Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/README.md Documentation on topology types, PlacementPolicy enum, scheduler interfaces, and various scheduling algorithms like Autoplace and Balancer. ```APIDOC ## Topology and Scheduler ### Description Topology types and PlacementPolicy enum, scheduler interface and implementations. ### Details - Autoplace, AutoplaceTopology, Balancer, FollowTopology, Manual schedulers - Constraint matching and policy selection - Advanced placement configuration ``` -------------------------------- ### Configuration Reference Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/README.md Details all command-line flags for LINSTOR connection, CSI service, logging, performance tuning, topology, and RWX support. ```APIDOC ## Configuration Reference ### LINSTOR Connection - `--linstor-endpoint` — Controller API endpoint - `--linstor-skip-tls-verification` — TLS validation - `--bearer-token` — Authentication token ### CSI Service - `--csi-endpoint` — Socket address for CSI server - `--node` — Node identifier ### Logging - `--log-level` — Logging intensity ### Performance - `--linstor-api-requests-per-second` — Rate limiting - `--linstor-api-burst` — Request burst allowance - `--node-cache-timeout` — Node property cache duration - `--resource-cache-timeout` — Resource cache duration ### Topology - `--property-namespace` — LINSTOR property namespace for topology - `--label-by-storage-pool` — Node labeling by pool ### RWX Support - `--enable-rwx` — Enable NFS-based RWX - `--nfs-service-namespace` — NFS service location - `--nfs-reactor-config-map-name` — NFS configuration ### Other - `--resync-after` — Reconciliation interval - `--disable-rwx-block-validation` — KubeVirt integration ``` -------------------------------- ### SnapCreate Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/linstor-client.md Creates snapshots of one or more volumes with optional backup to remote storage. ```APIDOC ## SnapCreate ### Description Creates snapshots of one or more volumes with optional backup to remote storage. ### Method Signature ```go func (s *Linstor) SnapCreate(ctx context.Context, id string, params *volume.SnapshotParameters, sourceVolIds ...volume.ID) ([]*volume.Snapshot, error) ``` ### Parameters #### Path Parameters - **ctx** (context.Context) - Required - Request context - **id** (string) - Required - Snapshot identifier/name - **params** (*volume.SnapshotParameters) - Required - Snapshot configuration (backup remote, retention, etc.) - **sourceVolIds** (...volume.ID) - Required - Source volumes to snapshot ### Returns - `[]*Snapshot`: Created snapshots - `error`: Creation error ### Example ```go snapParams := &volume.SnapshotParameters{ BackupRemote: "s3", } snaps, err := client.SnapCreate(ctx, "snap-1", snapParams, volId1, volId2) if err != nil { log.Fatal(err) } fmt.Printf("Created %d snapshots\n", len(snaps)) ``` ``` -------------------------------- ### Define Volume Parameters Structure Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/types.md Defines the structure for volume creation parameters, including node lists, replica placement, storage pools, encryption, and NFS configurations. Use this to configure storage classes for dynamic volume provisioning. ```go type Parameters struct { ClientList []string NodeList []string ReplicasOnDifferent []string ReplicasOnSame []string XReplicasOnDifferent map[string]int DisklessStoragePool string DoNotPlaceWithRegex string FSOpts string MountOpts string StoragePools []string PlacementCount int32 Disklessonremaining bool Encryption bool AllowRemoteVolumeAccess RemoteAccessPolicy LayerList []devicelayerkind.DeviceLayerKind PlacementPolicy topology.PlacementPolicy PostMountXfsOpts string ResourceGroup string Properties map[string]string UsePvcName bool OverProvision *float64 NfsConfigTemplatePath string NfsServiceName string NfsSquash string NfsRecoveryVolumeBytes int64 IOQoSLimits meta.Limits } ``` -------------------------------- ### Handle Delete In Progress Error Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/linstor-client.md This error type is returned when an operation is attempted on a resource that is currently in the process of being deleted. The example shows how to type-assert the error to access details like resource kind, name, and the operation that failed. ```go type DeleteInProgressError struct { Kind string Name string Operation string } ``` ```go err := client.Delete(ctx, volId) if deleteErr, ok := err.(*client.DeleteInProgressError); ok { fmt.Printf("%s %s is being deleted\n", deleteErr.Kind, deleteErr.Name) } ``` -------------------------------- ### Create a New LINSTOR Volume (Go) Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/linstor-client.md Creates a new LINSTOR volume with specified parameters and topology constraints. Ensure that the specified storage pools are available and placement constraints can be met. ```go info := &volume.Info{ ID: volume.ID{ResourceName: "vol-1", VolumeNumber: 0}, DeviceBytes: map[int]int64{0: 1073741824}, // 1 GB FsType: "ext4", } params := &volume.Parameters{ PlacementCount: 2, StoragePools: []string{"lvm-thin"}, } err := client.Create(ctx, info, params, nil) ``` -------------------------------- ### Create Volume from Snapshot Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/linstor-client.md Provisions a new volume using an existing snapshot as the source. Allows specifying volume parameters and topology constraints. ```go volInfo := &volume.Info{ ... } volParams := &volume.Parameters{ ... } snapParams := &volume.SnapshotParameters{ ... } err := client.VolFromSnap(ctx, snapshot, volInfo, volParams, snapParams, nil) if err != nil { log.Fatal(err) } ``` -------------------------------- ### Enable Debug Logging for Linstor CSI Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/errors.md Enable debug logging for the Linstor CSI driver to get detailed information including error stack traces, LINSTOR API requests/responses, and Kubernetes API calls. This is useful for diagnosing complex issues. ```bash ./linstor-csi --log-level debug ``` -------------------------------- ### Create Volume Group Snapshot Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/csi-driver.md Creates snapshots of multiple volumes atomically. Requires a name, source volume IDs, and optional parameters or secrets. ```go func (d *Driver) CreateVolumeGroupSnapshot(ctx context.Context, req *csi.CreateVolumeGroupSnapshotRequest) (*csi.CreateVolumeGroupSnapshotResponse, error) ``` -------------------------------- ### FollowTopology Scheduler Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/topology-scheduler.md Strictly follows pod topology hints. Places replicas only in specified topology segments. ```APIDOC ## FollowTopology Scheduler ### Description Strictly follows pod topology hints. Places replicas only in specified topology segments and fails if the topology cannot be satisfied. There is no fallback to auto-placement, making it useful for strict compliance scenarios. ### Example ```go sched := &followtopology.Scheduler{} topoReq := &csi.TopologyRequirement{ Requisite: []*csi.Topology{ {Segments: map[string]string{ "topology.kubernetes.io/zone": "us-west-1a", }}, {Segments: map[string]string{ "topology.kubernetes.io/zone": "us-west-1b", }}, }, } // Places 1 replica in zone 1a, 1 in zone 1b nodes, err := sched.Schedule(ctx, allNodes, 2, topoReq) ``` ``` -------------------------------- ### GetPluginCapabilities Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/csi-driver.md Declares supported CSI capabilities (controller service, volume expansion, etc.). ```APIDOC ## GetPluginCapabilities ### Description Declares supported CSI capabilities such as controller service, volume expansion, and topology-aware volume placement. ### Method `GetPluginCapabilities` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Response #### Success Response (200) - **Capabilities** (array of strings) - A list of supported capabilities. Example values include CONTROLLER_SERVICE, VOLUME_ACCESSIBILITY_CONSTRAINTS, GROUP_CONTROLLER_SERVICE, VOLUME_EXPANSION. #### Response Example ```json { "Capabilities": [ "CONTROLLER_SERVICE", "VOLUME_ACCESSIBILITY_CONSTRAINTS", "GROUP_CONTROLLER_SERVICE", "VOLUME_EXPANSION" ] } ``` ``` -------------------------------- ### List All Snapshots Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/linstor-client.md Retrieves a paginated list of all available snapshots. ```go snaps, err := client.ListSnaps(ctx, 0, 10) if err != nil { log.Fatal(err) } fmt.Printf("Listed %d snapshots\n", len(snaps)) ``` -------------------------------- ### volume.Parameters Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/types.md Defines the parameters that can be configured for volume creation via storage class. ```APIDOC ## volume.Parameters ### Description These are the storage class parameters used for volume creation. ### Fields - **ClientList** (`[]string`) - Nodes for diskless assignment at creation - **NodeList** (`[]string`) - Nodes for diskful assignment (overrides auto-placement) - **ReplicasOnDifferent** (`[]string`) - Topology keys for replica separation - **ReplicasOnSame** (`[]string`) - Topology keys for replica co-location - **XReplicasOnDifferent** (`map[string]int`) - Extended replica placement constraints - **DisklessStoragePool** (`string`) - Pool for diskless assignments. Default: `"DfltDisklessStorPool"` - **DoNotPlaceWithRegex** (`string`) - Regex to exclude matching nodes - **FSOpts** (`string`) - Filesystem creation options - **MountOpts** (`string`) - Mount options (comma-separated) - **StoragePools** (`[]string`) - Diskful storage pools - **PlacementCount** (`int32`) - Number of replicas. Default: 1 - **Disklessonremaining** (`bool`) - Create diskless on remaining nodes. Default: false - **Encryption** (`bool`) - Enable LUKS encryption. Default: false - **AllowRemoteVolumeAccess** (`RemoteAccessPolicy`) - Network access policy. Default: `RemoteAccessPolicyAnywhere` - **LayerList** (`[]devicelayerkind.DeviceLayerKind`) - Device layers. Default: `[DRBD, Storage]` - **PlacementPolicy** (`topology.PlacementPolicy`) - Automatic placement strategy. Default: `AutoPlaceTopology` - **PostMountXfsOpts** (`string`) - Post-mount xfs_io commands - **ResourceGroup** (`string`) - LINSTOR resource group - **Properties** (`map[string]string`) - Custom LINSTOR properties - **UsePvcName** (`bool`) - Use PVC name for volume name. Default: false - **OverProvision** (`*float64`) - Overprovision factor for free capacity - **NfsConfigTemplatePath** (`string`) - NFS Ganesha config template. Default: `/etc/nfs-helper/default-config.tmpl` - **NfsServiceName** (`string`) - Kubernetes service name for NFS. Default: `"linstor-csi-nfs"` - **NfsSquash** (`string`) - NFS squash policy. Default: `"no_root_squash"` - **NfsRecoveryVolumeBytes** (`int64`) - Recovery volume size. Default: 314572800 (300 MiB) - **IOQoSLimits** (`meta.Limits`) - I/O QoS limits from qos.linbit.com/* parameters ### Functions - **NewParameters**(params map[string]string, topologyPrefix string) (Parameters, error) — Parses storage class parameters ### Used By - Linstor.Create() — Volume creation - Linstor.Clone() — Cloning - Linstor.VolFromSnap() — Restore from snapshot - Driver.CreateVolume() — CSI create request ``` -------------------------------- ### Configure Caching for High-Level Client Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/highlevel-client.md Enable response caching for the High-Level Client to improve performance by reducing repeated queries. Adjust timeouts based on cluster change frequency. ```go lapicache.WithCaches( &lapicache.NodeCache{Timeout: 1 * time.Minute}, &lapicache.ResourceCache{Timeout: 30 * time.Second}, ) ``` -------------------------------- ### Define Scheduler Interface Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/topology-scheduler.md Defines the contract for volume replica placement algorithms. Implementations determine node selection based on capacity, topology, or manual input. ```go type Interface interface { // Schedule returns a list of nodes where replicas should be placed Schedule(ctx context.Context, node []string, placementCount int32, topologies *csi.TopologyRequirement) ([]string, error) } ``` -------------------------------- ### Volume Parameters Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/README.md Reference for all StorageClass parameters, covering placement, replication, encryption, filesystem, NFS/RWX settings, and custom properties. ```APIDOC ## Volume Parameters ### Description Complete StorageClass parameter reference. ### Details - Placement and replica configuration - Encryption and security options - Filesystem configuration - NFS/RWX settings - Resource group and custom properties - Snapshot parameters ``` -------------------------------- ### Configuration Options Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/linstor-client.md Provides various functional options to configure the Linstor client during initialization. ```APIDOC ## Configuration Options ### APIClient ```go func APIClient(c *lc.HighLevelClient) func(*Linstor) error ``` #### Description Sets the LINSTOR API client used for cluster communication. #### Parameters - **c** (`*HighLevelClient`): Required - Configured high-level client. ### LogOut ```go func LogOut(out io.Writer) func(*Linstor) error ``` #### Description Directs logs to the provided writer instead of discarding them. #### Parameters - **out** (`io.Writer`): Required - Output destination for logs. ### LogFmt ```go func LogFmt(fmt logrus.Formatter) func(*Linstor) error ``` #### Description Sets the log output format using a logrus formatter. #### Parameters - **fmt** (`logrus.Formatter`): Required - Logrus formatter instance. ### PropertyNamespace ```go func PropertyNamespace(ns string) func(*Linstor) error ``` #### Description Sets the LINSTOR property namespace for topology keys and custom properties. #### Parameters - **ns** (`string`): Required - Property namespace in LINSTOR. Default: `"Aux"`. ### LabelBySP ```go func LabelBySP(b bool) func(*Linstor) error ``` #### Description Enable or disable labeling of Kubernetes nodes based on configured storage pools. #### Parameters - **b** (`bool`): Required - If true, nodes are labeled with storage pool names. Default: `true`. ### LogLevel ```go func LogLevel(s string) func(*Linstor) error ``` #### Description Sets logging intensity. Debug level additionally reports the calling function. #### Parameters - **s** (`string`): Required - One of: panic, fatal, error, warn, info, debug. #### Returns Error if invalid log level provided. ``` -------------------------------- ### Assign Diskless Clients Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/api-reference/volume-parameters.md Specify nodes to assign disklessly at volume creation time, useful for pre-populating diskless access for known clients. ```yaml parameters: linstor.csi.linbit.com/clientList: "node-4 node-5" ``` -------------------------------- ### volume.SnapshotParameters Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/types.md Configuration for snapshot creation and backup operations. ```APIDOC ## volume.SnapshotParameters ### Description Configuration for snapshot creation and backup. ### Fields - **BackupRemote** (`string`) - Remote name for backup (e.g., "s3") - **BackupRemoteName** (`string`) - Backup resource name (computed if empty) - **BackupRetentionCount** (`*int`) - Maximum number of backups to retain - **BackupRetentionDays** (`*int`) - Maximum age of backups in days - **BackupIncremental** (`bool`) - Whether to create incremental backup - **BackupIncrementalBase** (`string`) - Base backup name for incremental ### Used By - Linstor.SnapCreate() — Snapshot creation - Linstor.VolFromSnap() — Restore configuration ``` -------------------------------- ### Topology Scheduler Reference Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/_autodocs/MANIFEST.txt Documentation for the topology scheduler, including types, policies, and constraint matching. ```APIDOC ## Topology Scheduler ### Description Describes the topology scheduler component, its types, placement policies, and how it handles topology constraints for volume placement. ### Types - **Topology types**: Defines the structure for representing topological information. - **PlacementPolicy enum**: Enumerates different strategies for placing storage volumes. ### Scheduler Interface - **Scheduler interface**: Defines the contract for all scheduler implementations. ### Scheduler Types - **Autoplace**: Automatic placement strategy. - **AutoplaceTopology**: Automatic placement considering topology. - **Balancer**: Load balancing placement strategy. - **FollowTopology**: Placement that follows existing topology. - **Manual**: Manual placement configuration. ### Constraint Matching - Explains how topology constraints are matched against available nodes and topologies. ### Placement Policy Selection - Details the logic for selecting and automating placement policies. ### Advanced Constraints - **ReplicasOnDifferent**: Ensures replicas are on different nodes. - **ReplicasOnSame**: Allows replicas on the same node. - **XReplicasOnDifferent**: Ensures X replicas are on different nodes. ### Topology Segment Resolution - Describes how the scheduler resolves topology segments. ### Error Handling - Covers error scenarios and handling within the scheduling process. ### Performance Tuning - Provides guidance on optimizing scheduler performance. ```