How to Measure Disk Performance using Fio in Linux

Fio is a free and open-source tool that can be used for benchmark and hardware verification. It supports lots of I/O engines including, libaio, sync, nmap, syslet, network, slice and many more. It also supports many distributions such as Linux, OpenBSD, OpenSolaris, HP-UX, Android and Windows.It can work on both files and block devices.

In this tutorial, we will learn how to install Fio in Linux, we will also learn how to use Fio with some examples.

Requirements

• Ubuntu 16.04 desktop or later install on your system.
• A normal user with sudo privileges.

Install Fio

Here, we will install Fio in all major Linux distributions such as CentOS 7, Ubuntu 16.04 and Debian 9.

Install Fio on Ubuntu 16.04

By default, Fio is available in Ubuntu default repository, so you can easily install it by running the following command:

sudo apt-get install fio -y

Install Fio on CentOS 7

By default, Fio is not available in CentOS default repository. So you will need to install EPEL repository to your system. You can install it by running the following command:

sudo yum install epel-release -y

Once EPEL repository is installed, you can install Fio by running the following command:

sudo yum install fio -y

Install Fio on Debian 9

By default, Fio is available in Debian repository, you can install it by just running the following command:

sudo apt-get install fio -y

Working with Fio

Now, Fio is installed on your system. It's time to see how to use Fio with example.

Example 1: Random Write Test

Lets start by running the following command. This command will writes a total 2GB files [4 jobs x 512 MB = 2GB] running 4 processes at a time:

sudo fio --name=randwrite --ioengine=libaio --iodepth=1 --rw=randwrite --bs=4k --direct=0 --size=512M --numjobs=4 --runtime=240 --group_reporting

You should see the following output:

randwrite: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
fio-2.1.3
Starting 4 processes
randwrite: Laying out IO file(s) (1 file(s) / 512MB)
randwrite: Laying out IO file(s) (1 file(s) / 512MB)
randwrite: Laying out IO file(s) (1 file(s) / 512MB)
randwrite: Laying out IO file(s) (1 file(s) / 512MB)
Jobs: 1 (f=1): [__w_] [-.-% done] [0KB/27032KB/0KB /s] [0/6758/0 iops] [eta 00m:00s] 
randwrite: (groupid=0, jobs=4): err= 0: pid=28810: Tue Jul 25 20:28:20 2017
  write: io=2048.0MB, bw=297722KB/s, iops=74430, runt=  7044msec
    slat (usec): min=9, max=47469, avg=38.13, stdev=488.37
    clat (usec): min=1, max=10595, avg= 2.26, stdev=30.99
     lat (usec): min=11, max=47478, avg=40.74, stdev=489.66
    clat percentiles (usec):
     |  1.00th=[    1],  5.00th=[    2], 10.00th=[    2], 20.00th=[    2],
     | 30.00th=[    2], 40.00th=[    2], 50.00th=[    2], 60.00th=[    2],
     | 70.00th=[    2], 80.00th=[    2], 90.00th=[    2], 95.00th=[    2],
     | 99.00th=[    3], 99.50th=[    4], 99.90th=[   16], 99.95th=[   61],
     | 99.99th=[  454]
    bw (KB  /s): min=10626, max=244432, per=32.42%, avg=96530.07, stdev=85594.86
    lat (usec) : 2=4.28%, 4=95.15%, 10=0.44%, 20=0.03%, 50=0.03%
    lat (usec) : 100=0.04%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%
  cpu          : usr=11.91%, sys=27.58%, ctx=4646, majf=0, minf=104
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=524288/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=2048.0MB, aggrb=297721KB/s, minb=297721KB/s, maxb=297721KB/s, mint=7044msec, maxt=7044msec

Disk stats (read/write):
  sda: ios=129/2495, merge=0/1599, ticks=11520/91476, in_queue=143340, util=78.35%

Example 2: Random Read Test

The following command will reads a total 2GB of files, running 4 processes at once.

sudo fio --name=randread --ioengine=libaio --iodepth=16 --rw=randread --bs=4k --direct=0 --size=512M --numjobs=4 --runtime=240 --group_reporting

You should see the following result:

randread: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
...
randread: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio-2.1.3
Starting 4 processes
randread: Laying out IO file(s) (1 file(s) / 512MB)
randread: Laying out IO file(s) (1 file(s) / 512MB)
randread: Laying out IO file(s) (1 file(s) / 512MB)
randread: Laying out IO file(s) (1 file(s) / 512MB)
Jobs: 4 (f=4): [rrrr] [100.0% done] [467KB/0KB/0KB /s] [116/0/0 iops] [eta 00m:00s]
randread: (groupid=0, jobs=4): err= 0: pid=14521: Tue Jul 25 22:01:24 2017
  read : io=117912KB, bw=503032B/s, iops=122, runt=240028msec
    slat (usec): min=93, max=1178.5K, avg=32554.59, stdev=45398.62
    clat (usec): min=12, max=2611.2K, avg=487669.64, stdev=288615.07
     lat (msec): min=49, max=2667, avg=520.23, stdev=303.69
    clat percentiles (msec):
     |  1.00th=[  167],  5.00th=[  217], 10.00th=[  247], 20.00th=[  289],
     | 30.00th=[  330], 40.00th=[  367], 50.00th=[  404], 60.00th=[  445],
     | 70.00th=[  502], 80.00th=[  603], 90.00th=[  898], 95.00th=[ 1057],
     | 99.00th=[ 1614], 99.50th=[ 1811], 99.90th=[ 2245], 99.95th=[ 2376],
     | 99.99th=[ 2573]
    bw (KB  /s): min=    4, max=  318, per=25.59%, avg=125.65, stdev=59.18
    lat (usec) : 20=0.01%
    lat (msec) : 50=0.01%, 100=0.02%, 250=10.45%, 500=59.10%, 750=15.73%
    lat (msec) : 1000=8.21%, 2000=6.23%, >=2000=0.25%
  cpu          : usr=0.05%, sys=0.17%, ctx=29816, majf=0, minf=165
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=99.8%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=29478/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=117912KB, aggrb=491KB/s, minb=491KB/s, maxb=491KB/s, mint=240028msec, maxt=240028msec

Disk stats (read/write):
  sda: ios=29552/2717, merge=0/2735, ticks=969368/97672, in_queue=1067872, util=100.00%

Example 3: Read Write Performance Test

The following command will measure random read/write performance of USB Pen drive (/dev/sdc1):

sudo fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=random_read_write.fio --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75

You should see the following test result:

test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.1.3
Starting 1 process
Jobs: 1 (f=1): [m] [98.0% done] [191KB/95KB/0KB /s] [47/23/0 iops] [eta 00m:03s]
test: (groupid=0, jobs=1): err= 0: pid=24800: Tue Jul 25 22:10:38 2017
  read : io=15512KB, bw=107237B/s, iops=26, runt=148123msec
  write: io=4968.0KB, bw=34344B/s, iops=8, runt=148123msec
  cpu          : usr=0.08%, sys=0.18%, ctx=5080, majf=0, minf=24
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.3%, 32=0.6%, >=64=98.8%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=3878/w=1242/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=15512KB, aggrb=104KB/s, minb=104KB/s, maxb=104KB/s, mint=148123msec, maxt=148123msec
  WRITE: io=4968KB, aggrb=33KB/s, minb=33KB/s, maxb=33KB/s, mint=148123msec, maxt=148123msec

Disk stats (read/write):
  sdc: ios=3506/1203, merge=372/28, ticks=7386380/1400600, in_queue=8796700, util=100.00%

The above command will write a 20MB data in USB Pen drive and perform 4KB reads and writes using three reads for every write ratio (rwmixread=75).

Fio is a very useful tool that can give information about the status of disk speed and latency. You can now easily test the performance of the Disk using Fio.