How to create an ocfs2 volume with the global heartbeat feature

Since ocfs2-1.8, global heartbeat feature is available. Earlier versions of OCFS2 had its own heartbeat for each mounted volume, which caused a lot of overhead. This has now been changed to what is called “global heartbeat”, where there is only one heartbeat/disk/network for all mounted volumes.

1. Check the ocfs2 version: Both ocfs2 kernel modules and ocfs2-tools utility must be 1.8 or later.

# modinfo ocfs2
filename:       /lib/modules/2.6.39-400.17.1.el5uek/kernel/fs/ocfs2/ocfs2.ko
license:        GPL
author:         Oracle
version:        1.8.0
description:    OCFS2 1.8.0
srcversion:     A90323B2B73A562F6A6353E
depends:        jbd2,ocfs2_stackglue,ocfs2_nodemanager
vermagic:       2.6.39-400.17.1.el5uek SMP mod_unload modversions

Earlier versions of ocfs2-tools-1.8.0-10 do not recognize xvd (virtual) devices. If you need to build global heartbeat on xvd device, make sure to upgrade to ocfs2-tools-1.8.0-11 or later.

# rpm -q ocfs2-tools
ocfs2-tools-1.8.0-10.el5

2. Create an ocfs2 volume by specifying the cluster-name and cluster-stack forcefully:

# mkfs.ocfs2 -b 4K -C 4K -J size=4M -N 4 -L myocfs2 --cluster-name=ocfs2 --cluster-stack=o2cb --force --global-heartbeat /dev/sdb
mkfs.ocfs2 1.8.0
User requested cluster (o2cb,ocfs2,1) does not match the active cluster (o2cb,mycluster,1).
mkfs.ocfs2 will not be able to determine if this operation can be done safely.
Format is forced.
Cluster stack: o2cb
Cluster name: ocfs2
Stack Flags: 0x1
NOTE: Feature extended slot map may be enabled
Overwriting existing ocfs2 partition.
WARNING: Cluster check disabled.
Proceed (y/N): y
Label: myocfs2
Features: sparse extended-slotmap backup-super unwritten inline-data strict-journal-super xattr indexed-dirs refcount discontig-bg
Block size: 4096 (12 bits)
Cluster size: 4096 (12 bits)
Volume size: 5368709120 (1310720 clusters) (1310720 blocks)
Cluster groups: 41 (tail covers 20480 clusters, rest cover 32256 clusters)
Extent allocator size: 4194304 (1 groups)
Journal size: 4194304
Node slots: 4
Creating bitmaps: done
Initializing superblock: done
Writing system files: done
Writing superblock: done
Writing backup superblock: 2 block(s)
Formatting Journals: done
Growing extent allocator: done
Formatting slot map: done
Formatting quota files: done
Writing lost+found: done
mkfs.ocfs2 successful

3. Setup the ocfs2 cluster.conf and add the cluster nodes:

# o2cb add-cluster ocfs2
# o2cb add-node ocfs2 [node1] --ip 10.182.32.148 --number 0
# o2cb add-node ocfs2 [node2] --ip 10.182.32.149 --number 1
# o2cb add-heartbeat ocfs2 /dev/sdb
# o2cb heartbeat-mode ocfs2 global

4. Check the ocfs2 cluster configuration:

# o2cb list-cluster ocfs2
heartbeat:
        region = 9796292A20374A7EBA0A4054AEC1859B
        cluster = ocfs2

node:
        number = 0
        name = node1
        ip_address = 10.182.32.148
        ip_port = 7777
        cluster = ocfs2

node:
        number = 1
        name = node2
        ip_address = 10.182.32.149
        ip_port = 7777
        cluster = ocfs2

cluster:
        node_count = 2
        heartbeat_mode = global
        name = ocfs2

5. Enable o2cb:

# service o2cb enable
Writing O2CB configuration: OK
Loading filesystem "configfs": OK
Mounting configfs filesystem at /sys/kernel/config: OK
Loading stack plugin "o2cb": OK
Loading filesystem "ocfs2_dlmfs": OK
Creating directory '/dlm': OK
Mounting ocfs2_dlmfs filesystem at /dlm: OK
Setting cluster stack "o2cb": OK
Registering O2CB cluster "ocfs2": OK
Setting O2CB cluster timeouts : OK
Starting global heartbeat for cluster "ocfs2": OK

6. Check the global heartbeat

# ll /sys/kernel/config/cluster/ocfs2/heartbeat/
total 0
drwxr-xr-x 2 root root    0 Mar  5 08:55 9796292A20374A7EBA0A4054AEC1859B
-rw-r--r-- 1 root root 4096 Mar  5 09:05 dead_threshold
-rw-r--r-- 1 root root 4096 Mar  5 09:05 mode

# cat /sys/kernel/config/cluster/ocfs2/heartbeat/mode
global

# o2cb cluster-status
Cluster 'ocfs2' is online

# service o2cb status
Driver for "configfs": Loaded
Filesystem "configfs": Mounted
Stack glue driver: Loaded
Stack plugin "o2cb": Loaded
Driver for "ocfs2_dlmfs": Loaded
Filesystem "ocfs2_dlmfs": Mounted
Checking O2CB cluster "ocfs2": Online
  Heartbeat dead threshold: 31
  Network idle timeout: 30000
  Network keepalive delay: 2000
  Network reconnect delay: 2000
  Heartbeat mode: Global
Checking O2CB heartbeat: Active
  9796292A20374A7EBA0A4054AEC1859B /dev/sdb