Troubleshooting CGROUP I/O Throttling in CentOS/RHEL 7
The Problem
The system is configured to use user slice cgroups to throttle the bandwidth on its Oracle ASM DATA and RECO disks. The bandwidth is limited to 50000000 bytes (50Mb) per second but CGROUP I/O throttling does not work. Here is some of the output from cgsnapshot:
group user.slice {
blkio {
blkio.leaf_weight="500";
blkio.leaf_weight_device="";
blkio.weight="1000";
blkio.weight_device="";
blkio.throttle.write_iops_device="";
blkio.throttle.read_iops_device="";
blkio.throttle.write_bps_device="8:144 50000000
8:96 50000000";
blkio.throttle.read_bps_device="8:144 50000000
8:96 50000000";
}
}
From configuration, it is defined that throttling should work for devices 8:144 and 8:96.
The Solution
The BlockIOAccounting parameter needs to be enabled for IO throttling to work, however it was not enabled on the system. This is shown by the following:
# systemctl status user.slice
● user.slice - User and Session Slice
Loaded: loaded (/usr/lib/systemd/system/user.slice; static; vendor preset: disabled)
Drop-In: /etc/systemd/system/user.slice.d
└─50-BlockIOWriteBandwidth.conf
There was no parameter showing for BlockIOAccounting in user.slice. To manage the Block IO, use the following directives in the [Service] section of the unit configuration file.
1. Enable the BlockIOAccounting parameter for user.slice with below commands:
# systemctl set-property user.slice BlockIOAccounting=1
# systemctl daemon-reload
# systemctl restart user.slice
2. Now check the status with the below command:
# systemctl status user.slice
● user.slice - User and Session Slice
Loaded: loaded (/usr/lib/systemd/system/user.slice; static; vendor preset: disabled)
Drop-In: /etc/systemd/system/user.slice.d
└─50-BlockIOAccounting.conf, 50-BlockIOReadBandwidth.conf, 50-BlockIOWriteBandwidth.conf <<<<< we can see that BlockIOAccounting is enabled now and cgroup should do the IO throttle now.
Active: active since Wed 2018-12-05 14:18:03 GMT; 9s ago
Docs: man:systemd.special(7)