TransWikia.com

DRBD resources not coming back online - Pacemaker + Corosync

Server Fault Asked by bakasan on December 30, 2021

I am using DRBD for replication. For testing purpose I am using 2 VMs. I have noticed that if I disconnect network interface on a node then it moves to standalone, and after I reconnect it does not go back to connected or WFconnection mode. Is there a way to run $drbdadm connect r0 or equivalent using pacemaker?

My config:

[root@CentOS1 ~]# cat /etc/drbd.d/nfs.res 
resource r0 {

    syncer {
        c-plan-ahead 20;
        c-fill-target 50k;
        c-min-rate 25M;
        al-extents 3833;
        rate 90M;

    }
    disk {

        no-md-flushes;
        fencing resource-only;
    }
    handlers {
        fence-peer "/usr/lib/drbd/crm-fence-peer.sh --timeout 120 --dc-timeout 120";
        after-resync-target "/usr/lib/drbd/crm-unfence-peer.sh";

    }
    net { 

        sndbuf-size 0;
        max-buffers 8000;
        max-epoch-size 8000;    
        after-sb-0pri discard-least-changes;
        after-sb-1pri consensus;
        after-sb-2pri call-pri-lost-after-sb;
    }


    device /dev/drbd0;
    disk /dev/centos/drbd;
    meta-disk internal;

    on CentOS1 {
        address 172.25.1.11:7790;
    }
    on CentOS2 {
        address 172.25.1.12:7790;
    }
}

resource status after disconnecting network and reconnecting:

[root@CentOS1 ~]#cat /proc/drbd 
version: 8.4.10-1 (api:1/proto:86-101)
GIT-hash: a4d5de01fffd7e4cde48a080e2c686f9e8cebf4c build by mockbuild@, 2017-09-15 14:23:22
 0: cs:StandAlone ro:Secondary/Unknown ds:UpToDate/Outdated   r-----
    ns:0 nr:0 dw:0 dr:2128 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

log:

[root@CentOS2 ~]# cat /dev/kmsg 

6,1171,16427206172,-;e1000: enp0s9 NIC Link is Down
3,1172,16427206435,-;e1000 0000:00:09.0 enp0s9: Reset adapter
 SUBSYSTEM=pci
 DEVICE=+pci:0000:00:09.0
3,1173,16431488757,-;drbd r0: PingAck did not arrive in time.
6,1174,16431488808,-;drbd r0: peer( Secondary -> Unknown ) conn( Connected -> NetworkFailure ) pdsk( UpToDate -> DUnknown ) 
6,1175,16431489116,-;drbd r0: ack_receiver terminated
6,1176,16431489121,-;drbd r0: Terminating drbd_a_r0
6,1177,16431489277,-;block drbd0: new current UUID A5A2A33E423F0679:0F9B7A2A6938EE77:4471A80A92A109A4:4470A80A92A109A4
 SUBSYSTEM=block
 DEVICE=b147:0
6,1178,16431489574,-;drbd r0: Connection closed
6,1179,16431489660,-;drbd r0: conn( NetworkFailure -> Unconnected ) 
6,1180,16431489667,-;drbd r0: receiver terminated
6,1181,16431489669,-;drbd r0: Restarting receiver thread
6,1182,16431489671,-;drbd r0: receiver (re)started
6,1183,16431489712,-;drbd r0: conn( Unconnected -> WFConnection ) 
6,1184,16431489801,-;drbd r0: helper command: /sbin/drbdadm fence-peer r0
4,1185,16431673341,-;drbd r0: helper command: /sbin/drbdadm fence-peer r0 exit code 5 (0x500)
6,1186,16431673351,-;drbd r0: fence-peer helper returned 5 (peer is unreachable, assumed to be dead)
6,1187,16431673374,-;drbd r0: pdsk( DUnknown -> Outdated ) 
3,1188,16450060876,-;drbd r0: bind before connect failed, err = -99
6,1189,16450060962,-;drbd r0: conn( WFConnection -> Disconnecting ) 
4,1190,16461488768,-;drbd r0: Discarding network configuration.
6,1191,16461488824,-;drbd r0: Connection closed
6,1192,16461488855,-;drbd r0: conn( Disconnecting -> StandAlone ) 
6,1193,16461488860,-;drbd r0: receiver terminated
6,1194,16461488862,-;drbd r0: Terminating drbd_r_r0

One Answer

There is not. DRBD and Pacemaker don't really know about each other aside from what the resource agents return codes were on given actions.

When you pull the interface that DRBD is bound to out from under it, DRBD will go StandAlone. You simply need to reconnect the resource after doing that: drbdadm connect <res> --OR-- drbdadm connect all

Which seems fine considering network interfaces don't usually disappear completely and then come back without an admin getting involved. If you want to test what happens when the network communication breaks (packet drops), you should use IP tables rules; you'll then see that DRBD goes into WFConnection (waiting for connection), and back to Connected when the rules are removed.

Answered by Matt Kereczman on December 30, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP