DRBD

https://linbit.com/drbd-user-guide/drbd-guide-9_0-en/
Admin
DRBD (Distributed Replicated Block Device) is a software-defined storage solution for Linux that...
Tokens:109,396
Snippets:832
Trust Score:10
Update:1 week ago
Show doc for...
Tokens:
Raw
### Build and install DRBD userspace utilities

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Builds the DRBD userspace utilities and installs them with sudo.

```bash
$ make
$ sudo make install
```

--------------------------------

### Install DRBD and DRBD utils

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Installs the DRBD kernel module and userspace tools after a successful build.

```bash
$ cd drbd && sudo make install && cd ..
$ cd drbd-utils && sudo make install && cd ..
```

--------------------------------

### Install DRBD Proxy with apt

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Use this command to install DRBD Proxy on systems using the apt package manager.

```bash
# apt install drbd-proxy
```

--------------------------------

### Specify OCF resource agent in promoter start list

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Example of using an OCF resource agent (ocf:heartbeat:IPaddr2) in the start list of the promoter plugin. The syntax is ocf:$vendor:$agent instance-id [key=value ...], with key=value pairs passed as environment variables to the created systemd unit file.

```plaintext
[[promoter]]
[...]
start = ["ocf:heartbeat:IPaddr2 ip_mysql ip=10.43.7.223 cidr_netmask=16"]
[...]
```

--------------------------------

### Start the gfs service

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Command to start the gfs service on both nodes to mount the filesystem.

```bash
# service gfs start
```

--------------------------------

### Install DRBD on SLES using zypper

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Alternative command-line installation method for DRBD on SLES using zypper.

```bash
# zypper install drbd
```

--------------------------------

### Multi-line start list service strings

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Example of splitting long service strings across multiple lines in a promoter plugin start list using TOML multi-line basic strings. A backslash at the end of a line prevents newline insertion.

```TOML
[...]
start = [
"""
ocf:heartbeat:Filesystem fs_mysql device=/dev/drbd1001 \
directory=/var/lib/mysql fstype=ext4 run_fsck=no""",
"mariadb.service",
"""ocf:heartbeat:IPaddr2 db_virtip ip=192.168.222.65 \
cidr_netmask=24 iflabel=virtualip"""
]
[...]

```

--------------------------------

### Install DRBD on Ubuntu using apt

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Install DRBD on Ubuntu LTS using apt. Requires the LINBIT PPA repository to be added.

```bash
# apt install drbd-utils drbd-dkms
```

--------------------------------

### OCFS2 cluster.conf example

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Example /etc/ocfs2/cluster.conf file with two nodes and a cluster. Default port 7777 is used; choose a non-conflicting port if changed. Distribute to both nodes using scp.

```text
node:
    ip_port = 7777
    ip_address = 10.1.1.31
    number = 0
    name = alice
    cluster = ocfs2

node:
    ip_port = 7777
    ip_address = 10.1.1.32
    number = 1
    name = bob
    cluster = ocfs2

cluster:
    node_count = 2
    name = ocfs2
```

--------------------------------

### Install DRBD Proxy with dnf

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Use this command to install DRBD Proxy on systems using the dnf package manager.

```bash
# dnf install drbd-proxy
```

--------------------------------

### Start DRBD manually with systemctl

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Starts the DRBD target for a specific resource on a node when not using a cluster manager. Replace <resource> with the actual resource name.

```bash
node-2# systemctl start drbd@<resource>.target
```

--------------------------------

### Enable and start snmpd service

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Enables and starts the snmpd service using systemctl. Use this after configuring the SNMP settings.

```bash
systemctl enable --now snmpd.service
```

--------------------------------

### Start DRBD resource target

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Command to start a DRBD resource target, typically used during upgrade to bring the resource online.

```bash
systemctl start drbd@<resource>.target
```

--------------------------------

### Install SNMP packages on RPM-based systems

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Install net-snmp and net-snmp-utils on RPM-based systems as a prerequisite for the AgentX plugin.

```bash
dnf -y install net-snmp net-snmp-utils
```

--------------------------------

### Variable-rate synchronization configuration example

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Example DRBD resource configuration for variable-rate synchronization, assuming deployment with DRBD Proxy. It sets c-plan-ahead, c-max-rate, and c-fill-target in the disk section.

```drbd
resource <resource> {
  disk {
    c-plan-ahead 5;
    c-max-rate 10M;
    c-fill-target 2M;
  }
}
```

--------------------------------

### Download and run LINBIT manage nodes helper script

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Run these commands on all nodes, one node at a time, as superuser. If the error 'no python interpreter found :-( ' appears, install Python 3 with 'dnf -y install python3' (RPM) or 'apt -y install python3' (DEB).

```bash
# curl -O https://my.linbit.com/linbit-manage-node.py
# chmod +x ./linbit-manage-node.py
# ./linbit-manage-node.py
```

--------------------------------

### Enable and set primary on stacked resource

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

After creating metadata, bring up the stacked resource and set it to primary to start replication.

```bash
# drbdadm up --stacked r0-U
# drbdadm primary --stacked r0-U
```

--------------------------------

### Get DRBD error code information with Perl

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Command to get information about a specific error code number using Perl. Example shows error code 11 returning 'Resource temporarily unavailable'.

```bash
# perl -e 'print $! = 11, "\n"'
Resource temporarily unavailable
```

--------------------------------

### Display node management hints with ./linbit-manage-node.py --hints

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Run this command to display hints for managing a LINBIT DRBD node. The output is not shown in the source.

```bash
./linbit-manage-node.py --hints
```

--------------------------------

### Example promoter plugin configuration snippet

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

TOML snippet file for configuring the promoter plugin for a DRBD resource. Place in /etc/drbd-reactor.d or another directory specified in the main configuration.

```toml
[[promoter]]

```

--------------------------------

### Configuration for permanently diskless node

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Example DRBD resource configuration with three diskful nodes and one permanently diskless client node. The diskless node uses 'disk none;' and gets no bitmap slot.

```plaintext
resource kvm-mail {
  device      /dev/drbd6;
  disk        /dev/vg/kvm-mail;
  meta-disk   internal;

  on store1 {
    address   10.1.10.1:7006;
    node-id   0;
  }
  on store2 {
    address   10.1.10.2:7006;
    node-id   1;
  }
  on store3 {
    address   10.1.10.3:7006;
    node-id   2;
  }

  on for-later-rebalancing {
    address   10.1.10.4:7006;
    node-id   3;
  }

  # DRBD "client"
  floating 10.1.11.6:8006 {
    disk      none;
    node-id   4;
  }

  # rest omitted for brevity
  ...
}
```

--------------------------------

### Configure O2CB driver with /etc/init.d/o2cb configure

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Interactive configuration of the O2CB driver on boot. Prompts for loading on boot, cluster name, heartbeat threshold, and network timeouts. Defaults shown in brackets; pressing ENTER keeps the current value. Output confirms successful configuration and module loading.

```bash
# /etc/init.d/o2cb configure
Configuring the O2CB driver.

This will configure the on-boot properties of the O2CB driver.
The following questions will determine whether the driver is loaded on
boot.  The current values will be shown in brackets ('[]').  Hitting
<ENTER> without typing an answer will keep that current value.  Ctrl-C
will abort.

Load O2CB driver on boot (y/n) [y]:
Cluster to start on boot (Enter "none" to clear) [ocfs2]:
Specify heartbeat dead threshold (>=7) [31]:
Specify network idle timeout in ms (>=5000) [30000]:
Specify network keepalive delay in ms (>=1000) [2000]:
Specify network reconnect delay in ms (>=2000) [2000]:
Use user-space driven heartbeat? (y/n) [n]:
Writing O2CB configuration: OK
Loading module "configfs": OK
Mounting configfs filesystem at /sys/kernel/config: OK
Loading module "ocfs2_nodemanager": OK
Loading module "ocfs2_dlm": OK
Loading module "ocfs2_dlmfs": OK
Mounting ocfs2_dlmfs filesystem at /dlm: OK
Starting O2CB cluster ocfs2: OK
```

--------------------------------

### Get DRBD resource name for LINSTOR resource

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Uses snmpget to retrieve the DRBD resource name for the first LINSTOR-created resource, which starts at minor number 1000. The OID .1.3.6.1.4.1.23302.1.2.1.2.1000 corresponds to the ResourceName entry for minor 1000.

```bash
snmpget -m ALL -v 2c -c public localhost .1.3.6.1.4.1.23302.1.2.1.2.1000
```

--------------------------------

### Enable a DRBD resource with drbdadm up

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Use this command to bring a DRBD resource online. You can use the keyword 'all' instead of a specific resource name to enable all resources configured in /etc/drbd.conf at once.

```bash
# drbdadm up <resource>
```

--------------------------------

### Initialize stacked resource metadata

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Run this command on the stacked resource to initialize its metadata before enabling it.

```bash
# drbdadm create-md --stacked r0-U
```

--------------------------------

### Bring up resource on backup node for three-node replication

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

On the backup node, create metadata and bring up the resource to enable three-node replication.

```bash
# drbdadm create-md r0-U
# drbdadm up r0-U
```

--------------------------------

### Enable DRBD resource with drbdadm up

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Associates the resource with its backing device, sets replication parameters, and connects to the peer. This is required before the resource can be used.

```bash
# drbdadm up <resource>

```

--------------------------------

### Install SNMP packages on DEB-based systems

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Install snmp and snmpd on DEB-based systems as a prerequisite for the AgentX plugin.

```bash
apt -y install snmp snmpd
```

--------------------------------

### Install DRBD on CentOS using yum

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Install DRBD on CentOS with yum. Requires a correct repository to be enabled.

```bash
# yum install drbd kmod-drbd
```

--------------------------------

### Simple DRBD configuration (/etc/drbd.d/global_common.conf)

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Defines the global and common sections. Sets usage-count to yes to opt into usage statistics, and configures protocol C for fully synchronous replication by default.

```plaintext
global {
  usage-count yes;
}
common {
  net {
    protocol C;
  }
}

```

--------------------------------

### Configure DRBD userspace build

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Recommended configuration command for DRBD userspace utilities, setting prefix, localstatedir, and sysconfdir to standard system paths.

```bash
$ ./configure --prefix=/usr --localstatedir=/var --sysconfdir=/etc
```

--------------------------------

### Create Prometheus plugin configuration file

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Creates /etc/drbd-reactor.d/prometheus.toml with the [[prometheus]] section, enabling enums and setting the listen address to 0.0.0.0:9942. Run on all DRBD Reactor nodes being monitored.

```bash
cat << EOF > /etc/drbd-reactor.d/prometheus.toml
[[prometheus]]
enums = true
address = "0.0.0.0:9942"
EOF
```

--------------------------------

### Enable and start DRBD Reactor service

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Enables and starts the DRBD Reactor service on all nodes. If already running, reload it instead.

```bash
systemctl enable drbd-reactor.service --now
```

--------------------------------

### Manually promote DRBD resource on the new Primary node

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Run these commands on the node that will become Primary. Promote the resource and mount the DRBD device.

```bash
# drbdadm primary <resource>
# mount /dev/drbd/by-res/<resource>/<vol-nr> <mountpoint>
```

--------------------------------

### Install DRBD on SLES using YaST

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Install DRBD on SUSE Linux Enterprise Server via the YaST command-line interface.

```bash
# yast -i drbd
```

--------------------------------

### Reconfigure ocfs2-tools on Debian with dpkg-reconfigure

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Debian does not support the configure option for /etc/init.d/o2cb. Instead, reconfigure the ocfs2-tools package to enable the driver. The command prompts for boot-time cluster start, cluster name, and heartbeat threshold. Output shows successful loading and mounting of OCFS2 components.

```bash
# dpkg-reconfigure -p medium -f readline ocfs2-tools
Configuring ocfs2-tools
Would you like to start an OCFS2 cluster (O2CB) at boot time? yes
Name of the cluster to start at boot time: ocfs2
The O2CB heartbeat threshold sets up the maximum time in seconds that a node
awaits for an I/O operation. After it, the node "fences" itself, and you will
probably see a crash.

It is calculated as the result of: (threshold - 1) x 2.

Its default value is 31 (60 seconds).

Raise it if you have slow disks and/or crashes with kernel messages like:

o2hb_write_timeout: 164 ERROR: heartbeat write timeout to device XXXX after NNNN
milliseconds
O2CB Heartbeat threshold: `31`
		Loading filesystem "configfs": OK
Mounting configfs filesystem at /sys/kernel/config: OK
Loading stack plugin "o2cb": OK
Loading filesystem "ocfs2_dlmfs": OK
Mounting ocfs2_dlmfs filesystem at /dlm: OK
Setting cluster stack "o2cb": OK
Starting O2CB cluster ocfs2: OK
```

--------------------------------

### Start DRBD with Pacemaker crm node online

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

Brings a node online in a Pacemaker cluster, which starts the DRBD resources managed by the cluster. Replace node-2 with the actual node name.

```bash
# crm node online node-2
```

--------------------------------

### Helper script suggested package manager command

Source: https://linbit.com/drbd-user-guide/drbd-guide-9_0-en

The LINBIT helper script shows a suggested package manager command after this line. Use this command to install DRBD. If you need to refer to the helper script's suggested actions later, run the script again with the --hints flag.

```bash
If you don't intend to run an SDS satellite or controller, a useful set is:
[...]
```
Tokens:
Raw