Fio initial write, overwrite and append write

Posted on Edited on In , Views:

In this post, we try to study how the overwrite and file_append options work in fio benchmark.

overwrite and file_append options

  • overwrite=bool

If true, writes to a file will always overwrite existing data. If the file doesn’t already exist, it will be created before the write phase begins. If the file exists and is large enough for the specified write phase, nothing will be done. Default: false.

  • file_append=bool

Perform I/O after the end of the file. Normally fio will operate within the size of a file. If this option is set, then fio will append to the file instead. This has identical behavior to setting offset to the size of a file. This option is ignored on non-regular files.

  • offset=int

Start I/O at the provided offset in the file, given as either a fixed size in bytes, zones or a percentage. If a percentage is given, the generated offset will be aligned to the minimum blocksize or to the value of offset_align if provided. Data before the given offset will not be touched. This effectively caps the file size at real_size - offset. Can be combined with size to constrain the start and end range of the I/O workload. A percentage can be specified by a number between 1 and 100 followed by ‘%’, for example, offset=20% to specify 20%. In ZBD mode, value can be set as number of zones using ‘z’.

Fio commands

The following fio commands are used to study initial write, overwrite and append write. We also have one more randread run at the last. Before each command run, we drop kernel cache with command “echo 3>/proc/sys/vm/drop_caches”.

$ fio --name=test --ioengine=libaio --blocksize=4k --readwrite=randwrite --directory=/mnt/bench1 --nrfiles=1 --size=1g --end_fsync=1 --numjobs=1 --direct=1
$ fio --name=test --ioengine=libaio --blocksize=4k --readwrite=randwrite --directory=/mnt/bench1 --nrfiles=1 --size=1g --end_fsync=1 --numjobs=1 --direct=1
$ fio --name=test --ioengine=libaio --blocksize=4k --readwrite=randwrite --directory=/mnt/bench1 --nrfiles=1 --size=1g --end_fsync=1 --numjobs=1 --direct=1 --overwrite=1
$ fio --name=test --ioengine=libaio --blocksize=4k --readwrite=randwrite --directory=/mnt/bench1 --nrfiles=1 --size=1g --end_fsync=1 --numjobs=1 --direct=1 --file_append=1
$ fio --name=test --ioengine=libaio --blocksize=4k --readwrite=randread --directory=/mnt/bench1 --nrfiles=1 --size=1g --end_fsync=1 --numjobs=1 --direct=1

Fio results

With the option “–file_append=1”, fio will append to the end of the file and expand it to the proper size. The write performance is similar to the initial write.

With the option “–overwrite=1”, fio will overwrite the existing data. If the file doesn’t already exist, it will be created before the write phase begins. If the file exists and is large enough for the specified write phase, nothing will be done.

The following table shows the fio results.
Test NameFio OptionsResultFile SizeInitial Writewrite: IOPS=7009, BW=27.4MiB/s0GB -> 1GBOverwrite–overwrite=0(default)write: IOPS=10.4k, BW=40.4MiB/s1GBOverwrite–overwrite=1write: IOPS=10.0k, BW=39.2MiB/s1GBAppend Write–file_append=1write: IOPS=6667, BW=26.0MiB/s1GB -> 2GBRandom Readread: IOPS=7769, BW=30.3MiB/s2GB

Fio syscalls for create, write and read

The file space will be allocated when to lay out the file.

29247 write(1, "Starting 1 process\n", 19) = 19
29247 stat("/mnt/bench1/test.0.0", 0x7ffd59df9db0) = -1 ENOENT (No such file or directory)
29247 write(1, "test: Laying out IO file (1 file"..., 44) = 44
29247 unlink("/mnt/bench1/test.0.0")    = -1 ENOENT (No such file or directory)
29247 open("/mnt/bench1/test.0.0", O_WRONLY|O_CREAT, 0644) = 3
29247 fallocate(3, 0, 0, 1073741824)    = 0
29247 fadvise64(3, 0, 1073741824, POSIX_FADV_DONTNEED) = 0
29247 close(3)

The following are the syscalls when fio writes data with ioengine libaio.

29429 open("/mnt/bench1/test.0.0", O_RDWR|O_CREAT|O_DIRECT, 0600) = 3
29429 fadvise64(3, 1073741824, 1073741824, POSIX_FADV_DONTNEED) = 0
29429 fadvise64(3, 1073741824, 1073741824, POSIX_FADV_RANDOM) = 0
29429 io_submit(0x7fbe9301c000, 1, [{aio_lio_opcode=IOCB_CMD_PWRITE, aio_fildes=3, aio_buf="5\340(\3148\240\231\26\6\234j\251\362\315\351\n\200S*\7\t\345\r\25pJ%\367\v9\235\30"..., aio_nbytes=4096, aio_offset=1138499584}]) = 1
29429 io_getevents(0x7fbe9301c000, 1, 1, [{data=0, obj=0xa77de0, res=4096, res2=0}], NULL) = 1
29429 io_submit(0x7fbe9301c000, 1, [{aio_lio_opcode=IOCB_CMD_PWRITE, aio_fildes=3, aio_buf="\0\340`o\0\0\0\0\6\234j\251\362\315\351\n\200S*\7\t\345\r\25pJ%\367\v9\235\30"..., aio_nbytes=4096, aio_offset=1868619776}]) = 1
29429 io_getevents(0x7fbe9301c000, 1, 1, [{data=0, obj=0xa77de0, res=4096, res2=0}], NULL) = 1
[..]
29429 io_submit(0x7fbe9301c000, 1, [{aio_lio_opcode=IOCB_CMD_PWRITE, aio_fildes=3, aio_buf="\0\300\357y\0\0\0\0\6\234j\251\362\315\351\n\200S*\7\t\345\r\25pJ%\367\v9\235\30"..., aio_nbytes=4096, aio_offset=2046550016}]) = 1
29429 io_getevents(0x7fbe9301c000, 1, 1, [{data=0, obj=0xa77de0, res=4096, res2=0}], NULL) = 1
29429 fsync(3)                          = 0
29429 close(3)

The following are the syscalls when fio reads data with ioengine libaio.

29484 open("/mnt/bench1/test.0.0", O_RDONLY|O_DIRECT) = 3
29484 fadvise64(3, 0, 1073741824, POSIX_FADV_DONTNEED) = 0
29484 fadvise64(3, 0, 1073741824, POSIX_FADV_RANDOM) = 0
29484 io_submit(0x7f069d23a000, 1, [{aio_lio_opcode=IOCB_CMD_PREAD, aio_fildes=3, aio_buf=0x9d1000, aio_nbytes=4096, aio_offset=64757760}]) = 1
29484 io_getevents(0x7f069d23a000, 1, 1, [{data=0, obj=0x9d3da0, res=4096, res2=0}], NULL) = 1
29484 io_submit(0x7f069d23a000, 1, [{aio_lio_opcode=IOCB_CMD_PREAD, aio_fildes=3, aio_buf=0x9d1000, aio_nbytes=4096, aio_offset=794877952}]) = 1
29484 io_getevents(0x7f069d23a000, 1, 1, [{data=0, obj=0x9d3da0, res=4096, res2=0}], NULL) = 1
[..]
29484 io_submit(0x7f069d23a000, 1, [{aio_lio_opcode=IOCB_CMD_PREAD, aio_fildes=3, aio_buf=0x9d1000, aio_nbytes=4096, aio_offset=972808192}]) = 1
29484 io_getevents(0x7f069d23a000, 1, 1, [{data=0, obj=0x9d3da0, res=4096, res2=0}], NULL) = 1
29484 getrusage(RUSAGE_THREAD, {ru_utime={tv_sec=1, tv_usec=706077}, ru_stime={tv_sec=7, tv_usec=828474}, ...}) = 0
29484 close(3)