How to disable I/O merge for Linux block device

Posted on Edited on In , Views:

iomerges parameter

In Linux, the file nomerges under /sys/block/xxx/queue/ enables the user to disable the lookup logic involved with I/O merging requests in the block layer.

By default, all merges are enabled.

1
2
$ cat /sys/block/sdb/queue/nomerges
0

When set to 1, only simple one-hit merges will be tried. When set to 2, no merge algorithms will be tried (including one-hit or more complex tree/hash lookups).

Tuning exercise

Firstly, we run 4k sequential write benchmarking with fio on a SSD disk. The write IOPS achieved is 104k as below. However, it doesn’t really tell us how many 4k sequential writes can be handled by this SSD disk because the 4k writes are merged to larger blocksize, ~20k in this case.

1
2
3
4
5
6
7
8
9
10
$ cat /sys/block/sdb/queue/nomerges
0

$ fio --ioengine=libaio --blocksize=4k --readwrite=write --filesize=1G --end_fsync=1 --iodepth=128 --direct=1 --group_reporting --numjobs=1 --name=fiojob1 --filename=/dev/sdb
  write: IOPS=104k, BW=406MiB/s (426MB/s)(1024MiB/2522msec)

$ iostat -ktdx 1 sdb
01/28/2024 12:57:51 AM
Device:   rrqm/s   wrqm/s     r/s    w/s        rkB/s   wkB/s       avgrq-sz   avgqu-sz   await   r_await   w_await  svctm  %util
sdb       0.00     62222.00   0.00   41947.00   0.00    416708.00   19.87      27.62      0.66    0.00      0.66     0.02   100.00

Now, we disable I/O merges and rerun the same fio benchmarking. The write IOPS achieved is 79.2k which reflects the actual capabilty to handle 4k sequential writes for this SSD disk.

1
2
3
4
5
6
7
8
9
10
11
$ echo 2 > /sys/block/sdb/queue/nomerges
$ cat /sys/block/sdb/queue/nomerges
2

$ fio --ioengine=libaio --blocksize=4k --readwrite=write --filesize=1G --end_fsync=1 --iodepth=128 --direct=1 --group_reporting --numjobs=1 --name=fiojob1 --filename=/dev/sdb
  write: IOPS=79.2k, BW=309MiB/s (324MB/s)(1024MiB/3311msec)

$ iostat -ktdx 1 sdb
01/28/2024 01:00:24 AM
Device:   rrqm/s   wrqm/s   r/s    w/s        rkB/s   wkB/s       avgrq-sz   avgqu-sz   await   r_await   w_await  svctm   %util
sdb       0.00     0.00     0.00   78923.00   0.00    315692.00   8.00       41.15      0.52    0.00      0.52     0.01    100.20

Note: The fio and iostat command above should be run from different terminal in order to monitor the disk I/O statistics in real time.

Reference