How to use IOzone for File System Performance Measurements
Abstract
This article explains how to use the IOzone open-source tool to perform file system performance assessments.
Contents
Getting and Compiling the Software
1. Obtain the latest version of the IOzone source code.
2. Compile IOzone from source.
HP-UX Bundle or ANSI C Compiler
Use the standard hpux-11.0w make definition to create a 64 bit binary.
# make hpux-11.0w
Building iozone for HP-UX (11.0w)
cc -c +DD64 +O3 -Dunix -D_LARGEFILE64_SOURCE -D_HPUX_SOURCE \
-DNAME='"hpux-11.0w"' -DHAVE_ANSIC_C -DASYNC_IO -DVXFS iozone.c \
-o iozone_hpux-11.0w.o
cc: "iozone.c", line 19848: warning 604: Pointers are not assignment-compatible.
cc: "iozone.c", line 19848: warning 563: Argument #3 is not the correct type.
cc: "iozone.c", line 21966: warning 604: Pointers are not assignment-compatible.
cc: "iozone.c", line 21966: warning 563: Argument #3 is not the correct type.
cc: "iozone.c", line 22188: warning 604: Pointers are not assignment-compatible.
cc: "iozone.c", line 22188: warning 563: Argument #3 is not the correct type.
cc -c +DD64 +O3 -Dunix -D_LARGEFILE64_SOURCE -D_HPUX_SOURCE \
-DHAVE_ANSIC_C -DASYNC_IO -DVXFS libasync.c -o libasyncw.o
cc -c +DD64 +O3 -Dunix -D_LARGEFILE64_SOURCE -D_HPUX_SOURCE \
-DHAVE_ANSIC_C -DASYNC_IO -DVXFS libbif.c -o libbif.o
cc +DD64 +O3 iozone_hpux-11.0w.o libasyncw.o \
libbif.o -lpthread -lrt -o iozone
# file iozone
iozone: ELF-64 executable object file - PA-RISC 2.0 (LP64)
DSPP GCC 4.4.3
Edit makefile text file and change the ghpux definition to optimize for the itanium 2 platform. Remove the -static flag to create a dynamic library file to support the unwind library. It will create a 32 binary.
# make ghpux
Building iozone for GCC HP-UX (9.05)
gcc -c -O -Dunix -D_HPUX_SOURCE -DHAVE_ANSIC_C -DNO_THREADS iozone.c \
-DNAME='"h=ghpux"' -o iozone_ghpux.o
iozone.c: In function 'main':
iozone.c:1658: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c:1806: warning: incompatible implicit declaration of built-in function 'strlen'
iozone.c:1883: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c:2913: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'record_command_line':
iozone.c:2984: warning: incompatible implicit declaration of built-in function 'strlen'
iozone.c:2985: warning: incompatible implicit declaration of built-in function 'strcat'
iozone.c: In function 'show_help':
iozone.c:3151: warning: incompatible implicit declaration of built-in function 'strlen'
iozone.c: In function 'throughput_test':
iozone.c:3382: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'purge_buffer_cache':
iozone.c:11661: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c:11662: warning: incompatible implicit declaration of built-in function 'strcat'
iozone.c: In function 'fill_area':
iozone.c:18093: warning: incompatible implicit declaration of built-in function 'bcopy'
iozone.c: In function 'get_traj':
iozone.c:18641: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'r_traj_size':
iozone.c:18778: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'w_traj_size':
iozone.c:18873: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'start_master_listen':
iozone.c:19318: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'child_send':
iozone.c:19410: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c:19419: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'master_send':
iozone.c:19471: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c:19480: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'start_child_send':
iozone.c:19635: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'start_child_listen':
iozone.c:19776: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'child_listen':
iozone.c:19878: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'start_child_listen_async':
iozone.c:19942: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'start_master_send':
iozone.c:20081: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'start_master_send_async':
iozone.c:20166: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'pick_client':
iozone.c:20269: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c:20281: warning: incompatible implicit declaration of built-in function 'strcat'
iozone.c:20350: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'become_client':
iozone.c:20487: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c:20526: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'tell_master_stats':
iozone.c:20914: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'tell_master_ready':
iozone.c:20967: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'wait_for_master_go':
iozone.c:20996: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'start_child_listen_loop':
iozone.c:21143: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'tell_children_begin':
iozone.c:21206: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'terminate_child_async':
iozone.c:21377: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'distribute_stop':
iozone.c:21420: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'send_stop':
iozone.c:21446: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'cleanup_children':
iozone.c:21485: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'find_remote_shell':
iozone.c:21547: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c:21551: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'find_external_mon':
iozone.c:21572: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c:21577: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'speed_main':
iozone.c:21666: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c:21679: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'sp_start_master_send':
iozone.c:21845: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'sp_start_child_listen':
iozone.c:21930: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'sp_start_master_listen':
iozone.c:22152: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'sp_start_child_send':
iozone.c:22247: warning: incompatible implicit declaration of built-in function 'bzero'
iozone.c: In function 'get_date':
iozone.c:22335: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c: In function 'get_pattern':
iozone.c:22366: warning: incompatible implicit declaration of built-in function 'strcpy'
iozone.c:22367: warning: incompatible implicit declaration of built-in function 'strlen'
iozone.c: In function 'check_filename':
iozone.c:22428: warning: incompatible implicit declaration of built-in function 'strlen'
iozone.c: In function 'start_monitor':
iozone.c:22454: warning: incompatible implicit declaration of built-in function 'strlen'
iozone.c: In function 'stop_monitor':
iozone.c:22472: warning: incompatible implicit declaration of built-in function 'strlen'
gcc -c -O -Dunix -D_HPUX_SOURCE -DHAVE_ANSIC_C -DNO_THREADS \
libbif.c -o libbif.o
libbif.c: In function 'create_xls':
libbif.c:194: warning: incompatible implicit declaration of built-in function 'exit'
libbif.c: In function 'do_label':
libbif.c:361: warning: incompatible implicit declaration of built-in function 'strlen'
libbif.c:377: warning: incompatible implicit declaration of built-in function 'strcpy'
gcc -O -mtune=itanium2 iozone_ghpux.o libbif.o -o iozone
# file iozone
iozone: ELF-32 executable object file - IA64
3. Save a copy of the binary executable iozone on /usr/contrib/bin directory for day-to-day usage.
# cp iozone /usr/contrib/bin
# chown root:sys /usr/contrib/bin/iozone
# chmod 0755 /usr/contrib/bin/iozone
# ls -l /usr/contrib/bin/iozone
-rwxr-xr-x 1 root sys 458000 Sep 9 18:54 /usr/contrib/bin/iozone
Using iozone
Full documentation on the usage of iozone command is available at Iozone Filesystem Benchmark manual. Here is an overview of the options used to collect data for this article.
# iozone -e -r 1 -r 2 -r 4 -r 8 -r 16 -r 32 -r 64 -r 128 -r 256 -r 512 -r 1m -r 2m -r 4m -r 8m \
-s 64m -f /perf/tfile \
-i 0 -i 1
| -e | include flush (fsync,fflsuh) in the timing calcultations |
| -c | include close() in the timing calculations |
| -r | used to specify record size in Kbytes, to test. |
| -s | Used to specify the size, in Kbytes of the file to test. |
| -f | Used to specify the filename of the temporary file under test. |
| -t | Run IOzone in throughput mode. This option allows the user to specify how many threads or process to have active during the measurement. |
Testing Unbuffered File Systems
The following chart present the results of performing I/O Read/Write transaction with incremental block size, 1k up to 4096k (4Mb) on a VxFS file system. Buffer Cache has been disable using the OnlineJFS options mincache=direct,convosync=direct. These options are commonly used on SAN devices that provided their own buffering mechanisms.
The backing storage is a direct attached 36 GB 10k Ultra320 parallel SCSI drive.
|
|
|
Note that increasing the block size improves the overall transfer rate of the file system, up to 1024k (1 MB), when additional increase of the block size does not provides additional I/O throughput.
Testing Buffered File Systems
|
|
|
Note that the read results for the Buffered File System are almost an order of magnitude higher than the Unbuffered File System when the block size reaches the 256k sweet spot. At the same time, write speed is on the same range of the previous test.
Once the block size is higher than 256k, the read speed heavily falls. The drop is explained by the discovered_direct_iosz VxFS tunable. These parameters controls the discovery of Direct I/O transactions.
Always keep in mind that no matter how large and fast the buffer cache may, in the long run, all I/O transaction will need to access the backing storage device. Always perform unbuffered tests to understand the physical capacities of your storage subsystem.
VxFS DirectIO
On VxFS file systems, when transactions larger that 256k are made the kernels stops using the buffer cache and performs it's I/O directly from the backing storage. This is done to keep a balance over the system performance because heavy Direct I/O transactions will thrash the buffer with data that, due to it's nature, will not likely be reused in upcoming transactions, reducing the opportunity for cache data re-usability.
A common scenario is a file system [backup | restore], where data is read once [to | from] the backup media. Knowing in advance the block size used by the backup application allow to set the boundaries for these transactions to optimize the task.
# vxtunefs -p /perf | grep direct
discovered_direct_iosz = 262144
max_direct_iosz = 1048576
For example, to set discovery of Direct I/O toward applications using 64k blocks.
# vxtunefs -s -o discovered_direct_iosz=65536 /perf
UX:vxfs vxtunefs: INFO: V-3-22525: Parameters successfully set for /perf
|
|
|
