TEST-4K
RandomRW: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 5 processes
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: (groupid=0, jobs=5): err= 0: pid=19513: Wed Jan 22 12:49:57 2020
read: IOPS=449, BW=1797KiB/s (1840kB/s)(106MiB/60110msec)
slat (usec): min=2, max=908870, avg=6594.47, stdev=38638.85
clat (usec): min=775, max=2006.2k, avg=477166.55, stdev=212949.23
lat (msec): min=82, max=2054, avg=483.76, stdev=213.28
clat percentiles (msec):
| 1.00th=[ 146], 5.00th=[ 199], 10.00th=[ 241], 20.00th=[ 309],
| 30.00th=[ 355], 40.00th=[ 397], 50.00th=[ 439], 60.00th=[ 498],
| 70.00th=[ 558], 80.00th=[ 625], 90.00th=[ 751], 95.00th=[ 860],
| 99.00th=[ 1133], 99.50th=[ 1318], 99.90th=[ 1737], 99.95th=[ 2005],
| 99.99th=[ 2005]
bw ( KiB/s): min= 7, max= 912, per=20.59%, avg=369.98, stdev=156.88, samples=579
iops : min= 1, max= 228, avg=92.44, stdev=39.23, samples=579
write: IOPS=196, BW=785KiB/s (804kB/s)(46.1MiB/60110msec)
slat (usec): min=3, max=751211, avg=6352.81, stdev=37598.05
clat (msec): min=10, max=2006, avg=498.95, stdev=211.36
lat (msec): min=10, max=2006, avg=505.30, stdev=211.26
clat percentiles (msec):
| 1.00th=[ 163], 5.00th=[ 211], 10.00th=[ 255], 20.00th=[ 330],
| 30.00th=[ 380], 40.00th=[ 414], 50.00th=[ 468], 60.00th=[ 527],
| 70.00th=[ 575], 80.00th=[ 642], 90.00th=[ 768], 95.00th=[ 877],
| 99.00th=[ 1133], 99.50th=[ 1301], 99.90th=[ 1737], 99.95th=[ 2005],
| 99.99th=[ 2005]
bw ( KiB/s): min= 7, max= 360, per=20.93%, avg=164.09, stdev=69.28, samples=570
iops : min= 1, max= 90, avg=40.96, stdev=17.32, samples=570
lat (usec) : 1000=0.03%
lat (msec) : 2=0.02%, 20=0.01%, 100=0.03%, 250=10.80%, 500=49.15%
lat (msec) : 750=29.23%, 1000=8.58%
cpu : usr=0.09%, sys=0.16%, ctx=2448, majf=0, minf=151
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=0.4%, >=64=99.2%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=27008,11794,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=1797KiB/s (1840kB/s), 1797KiB/s-1797KiB/s (1840kB/s-1840kB/s), io=106MiB (111MB), run=60110-60110msec
WRITE: bw=785KiB/s (804kB/s), 785KiB/s-785KiB/s (804kB/s-804kB/s), io=46.1MiB (48.3MB), run=60110-60110msec
Disk stats (read/write):
sda: ios=27011/11824, merge=4/80, ticks=6686698/3129618, in_queue=9834334, util=99.87%
TEST-8K
RandomRW: (g=0): rw=randrw, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 5 processes
RandomRW: (groupid=0, jobs=5): err= 0: pid=19524: Wed Jan 22 12:50:58 2020
read: IOPS=451, BW=3614KiB/s (3700kB/s)(212MiB/60136msec)
slat (usec): min=2, max=638175, avg=6614.52, stdev=37336.76
clat (usec): min=678, max=1555.8k, avg=477141.20, stdev=201151.57
lat (usec): min=1124, max=1555.8k, avg=483756.98, stdev=201136.43
clat percentiles (msec):
| 1.00th=[ 146], 5.00th=[ 194], 10.00th=[ 234], 20.00th=[ 317],
| 30.00th=[ 368], 40.00th=[ 405], 50.00th=[ 443], 60.00th=[ 493],
| 70.00th=[ 550], 80.00th=[ 625], 90.00th=[ 735], 95.00th=[ 860],
| 99.00th=[ 1083], 99.50th=[ 1183], 99.90th=[ 1368], 99.95th=[ 1418],
| 99.99th=[ 1552]
bw ( KiB/s): min= 15, max= 1744, per=20.25%, avg=731.69, stdev=305.05, samples=589
iops : min= 1, max= 218, avg=91.39, stdev=38.15, samples=589
write: IOPS=197, BW=1578KiB/s (1616kB/s)(92.7MiB/60136msec)
slat (usec): min=3, max=612189, avg=6613.92, stdev=37831.04
clat (msec): min=41, max=1555, avg=502.06, stdev=209.33
lat (msec): min=41, max=1591, avg=508.68, stdev=209.63
clat percentiles (msec):
| 1.00th=[ 159], 5.00th=[ 205], 10.00th=[ 266], 20.00th=[ 334],
| 30.00th=[ 384], 40.00th=[ 422], 50.00th=[ 472], 60.00th=[ 518],
| 70.00th=[ 575], 80.00th=[ 659], 90.00th=[ 785], 95.00th=[ 894],
| 99.00th=[ 1133], 99.50th=[ 1301], 99.90th=[ 1368], 99.95th=[ 1552],
| 99.99th=[ 1552]
bw ( KiB/s): min= 15, max= 832, per=20.39%, avg=321.52, stdev=140.16, samples=585
iops : min= 1, max= 104, avg=40.12, stdev=17.53, samples=585
lat (usec) : 750=0.01%, 1000=0.03%
lat (msec) : 2=0.02%, 20=0.01%, 50=0.01%, 100=0.04%, 250=10.59%
lat (msec) : 500=49.29%, 750=30.08%, 1000=8.29%
cpu : usr=0.09%, sys=0.17%, ctx=2495, majf=0, minf=161
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=0.4%, >=64=99.2%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=27164,11860,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=3614KiB/s (3700kB/s), 3614KiB/s-3614KiB/s (3700kB/s-3700kB/s), io=212MiB (223MB), run=60136-60136msec
WRITE: bw=1578KiB/s (1616kB/s), 1578KiB/s-1578KiB/s (1616kB/s-1616kB/s), io=92.7MiB (97.2MB), run=60136-60136msec
Disk stats (read/write):
sda: ios=26994/11818, merge=0/33, ticks=6612528/3113492, in_queue=9741592, util=99.95%
TEST-16K
RandomRW: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 5 processes
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: (groupid=0, jobs=5): err= 0: pid=19546: Wed Jan 22 12:56:47 2020
read: IOPS=417, BW=6685KiB/s (6845kB/s)(393MiB/60199msec)
slat (usec): min=3, max=1393.8k, avg=6941.57, stdev=40534.65
clat (usec): min=768, max=1744.3k, avg=517365.23, stdev=232362.86
lat (msec): min=34, max=1744, avg=524.31, stdev=232.31
clat percentiles (msec):
| 1.00th=[ 171], 5.00th=[ 218], 10.00th=[ 257], 20.00th=[ 338],
| 30.00th=[ 384], 40.00th=[ 426], 50.00th=[ 468], 60.00th=[ 531],
| 70.00th=[ 600], 80.00th=[ 676], 90.00th=[ 827], 95.00th=[ 936],
| 99.00th=[ 1284], 99.50th=[ 1401], 99.90th=[ 1720], 99.95th=[ 1737],
| 99.99th=[ 1737]
bw ( KiB/s): min= 31, max= 3658, per=20.49%, avg=1369.34, stdev=607.02, samples=582
iops : min= 1, max= 228, avg=85.51, stdev=37.96, samples=582
write: IOPS=181, BW=2911KiB/s (2981kB/s)(171MiB/60199msec)
slat (usec): min=3, max=1075.3k, avg=7036.56, stdev=42601.59
clat (msec): min=78, max=1744, avg=542.34, stdev=237.48
lat (msec): min=108, max=1943, avg=549.38, stdev=238.24
clat percentiles (msec):
| 1.00th=[ 180], 5.00th=[ 224], 10.00th=[ 275], 20.00th=[ 363],
| 30.00th=[ 401], 40.00th=[ 447], 50.00th=[ 506], 60.00th=[ 558],
| 70.00th=[ 625], 80.00th=[ 701], 90.00th=[ 844], 95.00th=[ 978],
| 99.00th=[ 1334], 99.50th=[ 1485], 99.90th=[ 1720], 99.95th=[ 1737],
| 99.99th=[ 1737]
bw ( KiB/s): min= 31, max= 1568, per=21.12%, avg=614.50, stdev=272.80, samples=566
iops : min= 1, max= 98, avg=38.32, stdev=17.06, samples=566
lat (usec) : 1000=0.01%
lat (msec) : 2=0.03%, 4=0.01%, 20=0.01%, 50=0.01%, 100=0.01%
lat (msec) : 250=8.12%, 500=44.95%, 750=32.26%, 1000=10.59%
cpu : usr=0.09%, sys=0.17%, ctx=2219, majf=0, minf=150
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=0.4%, >=64=99.1%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=25151,10952,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=6685KiB/s (6845kB/s), 6685KiB/s-6685KiB/s (6845kB/s-6845kB/s), io=393MiB (412MB), run=60199-60199msec
WRITE: bw=2911KiB/s (2981kB/s), 2911KiB/s-2911KiB/s (2981kB/s-2981kB/s), io=171MiB (179MB), run=60199-60199msec
Disk stats (read/write):
sda: ios=25114/10950, merge=0/70, ticks=6773795/3124289, in_queue=9920477, util=99.96%
TEST-32K
RandomRW: (g=0): rw=randrw, bs=(R) 32.0KiB-32.0KiB, (W) 32.0KiB-32.0KiB, (T) 32.0KiB-32.0KiB, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 5 processes
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: (groupid=0, jobs=5): err= 0: pid=19587: Wed Jan 22 13:02:46 2020
read: IOPS=280, BW=8983KiB/s (9198kB/s)(530MiB/60440msec)
slat (usec): min=3, max=1212.4k, avg=10411.97, stdev=60066.42
clat (usec): min=879, max=2358.0k, avg=769624.82, stdev=325524.63
lat (usec): min=1041, max=2358.0k, avg=780038.10, stdev=325497.66
clat percentiles (msec):
| 1.00th=[ 245], 5.00th=[ 300], 10.00th=[ 368], 20.00th=[ 498],
| 30.00th=[ 575], 40.00th=[ 651], 50.00th=[ 726], 60.00th=[ 818],
| 70.00th=[ 936], 80.00th=[ 1028], 90.00th=[ 1200], 95.00th=[ 1368],
| 99.00th=[ 1720], 99.50th=[ 1888], 99.90th=[ 2366], 99.95th=[ 2366],
| 99.99th=[ 2366]
bw ( KiB/s): min= 63, max= 5504, per=21.40%, avg=1922.09, stdev=1088.46, samples=557
iops : min= 1, max= 172, avg=59.99, stdev=34.01, samples=557
write: IOPS=122, BW=3931KiB/s (4026kB/s)(232MiB/60440msec)
slat (usec): min=5, max=1029.8k, avg=10747.75, stdev=61183.66
clat (msec): min=78, max=2358, avg=797.52, stdev=333.00
lat (msec): min=78, max=2358, avg=808.27, stdev=333.84
clat percentiles (msec):
| 1.00th=[ 255], 5.00th=[ 309], 10.00th=[ 380], 20.00th=[ 523],
| 30.00th=[ 600], 40.00th=[ 667], 50.00th=[ 760], 60.00th=[ 852],
| 70.00th=[ 953], 80.00th=[ 1036], 90.00th=[ 1250], 95.00th=[ 1401],
| 99.00th=[ 1787], 99.50th=[ 1955], 99.90th=[ 2366], 99.95th=[ 2366],
| 99.99th=[ 2366]
bw ( KiB/s): min= 63, max= 2816, per=22.80%, avg=896.13, stdev=497.04, samples=524
iops : min= 1, max= 88, avg=27.91, stdev=15.55, samples=524
lat (usec) : 1000=0.03%
lat (msec) : 2=0.03%, 4=0.01%, 50=0.01%, 100=0.01%, 250=0.98%
lat (msec) : 500=19.14%, 750=31.87%, 1000=24.58%
cpu : usr=0.06%, sys=0.13%, ctx=1534, majf=0, minf=151
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.3%, 32=0.7%, >=64=98.7%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=16966,7425,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=8983KiB/s (9198kB/s), 8983KiB/s-8983KiB/s (9198kB/s-9198kB/s), io=530MiB (556MB), run=60440-60440msec
WRITE: bw=3931KiB/s (4026kB/s), 3931KiB/s-3931KiB/s (4026kB/s-4026kB/s), io=232MiB (243MB), run=60440-60440msec
Disk stats (read/write):
sda: ios=16875/7429, merge=1/68, ticks=6721867/3138635, in_queue=9912692, util=99.94%
TEST-64K
RandomRW: (g=0): rw=randrw, bs=(R) 64.0KiB-64.0KiB, (W) 64.0KiB-64.0KiB, (T) 64.0KiB-64.0KiB, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 5 processes
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: (groupid=0, jobs=5): err= 0: pid=19611: Wed Jan 22 13:08:39 2020
read: IOPS=228, BW=14.3MiB/s (14.0MB/s)(859MiB/60145msec)
slat (usec): min=4, max=2729.5k, avg=12057.03, stdev=78466.45
clat (usec): min=13, max=5217.2k, avg=948311.42, stdev=565267.73
lat (msec): min=96, max=5217, avg=960.37, stdev=567.59
clat percentiles (msec):
| 1.00th=[ 262], 5.00th=[ 326], 10.00th=[ 393], 20.00th=[ 558],
| 30.00th=[ 642], 40.00th=[ 718], 50.00th=[ 860], 60.00th=[ 944],
| 70.00th=[ 1062], 80.00th=[ 1234], 90.00th=[ 1586], 95.00th=[ 1938],
| 99.00th=[ 3306], 99.50th=[ 3675], 99.90th=[ 5201], 99.95th=[ 5201],
| 99.99th=[ 5201]
bw ( KiB/s): min= 127, max=11648, per=23.67%, avg=3460.71, stdev=2148.36, samples=500
iops : min= 1, max= 182, avg=53.97, stdev=33.58, samples=500
write: IOPS=99, BW=6384KiB/s (6537kB/s)(375MiB/60145msec)
slat (usec): min=7, max=1887.2k, avg=12454.79, stdev=76947.64
clat (msec): min=6, max=5217, avg=981.57, stdev=585.66
lat (msec): min=173, max=5217, avg=994.02, stdev=587.56
clat percentiles (msec):
| 1.00th=[ 284], 5.00th=[ 347], 10.00th=[ 447], 20.00th=[ 584],
| 30.00th=[ 651], 40.00th=[ 743], 50.00th=[ 885], 60.00th=[ 978],
| 70.00th=[ 1099], 80.00th=[ 1267], 90.00th=[ 1603], 95.00th=[ 1938],
| 99.00th=[ 3306], 99.50th=[ 4463], 99.90th=[ 5201], 99.95th=[ 5201],
| 99.99th=[ 5201]
bw ( KiB/s): min= 127, max= 4992, per=26.21%, avg=1673.15, stdev=930.46, samples=451
iops : min= 1, max= 78, avg=26.05, stdev=14.52, samples=451
lat (usec) : 20=0.01%, 250=0.01%
lat (msec) : 2=0.05%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01%
lat (msec) : 250=0.54%, 500=13.81%, 750=28.19%, 1000=22.18%
cpu : usr=0.06%, sys=0.14%, ctx=1140, majf=0, minf=157
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.4%, 32=0.8%, >=64=98.4%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=13740,5999,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=14.3MiB/s (14.0MB/s), 14.3MiB/s-14.3MiB/s (14.0MB/s-14.0MB/s), io=859MiB (900MB), run=60145-60145msec
WRITE: bw=6384KiB/s (6537kB/s), 6384KiB/s-6384KiB/s (6537kB/s-6537kB/s), io=375MiB (393MB), run=60145-60145msec
Disk stats (read/write):
sda: ios=13691/6015, merge=1/66, ticks=6917215/3142085, in_queue=10111053, util=99.93%
TEST-128K
RandomRW: (g=0): rw=randrw, bs=(R) 128KiB-128KiB, (W) 128KiB-128KiB, (T) 128KiB-128KiB, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 5 processes
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: (groupid=0, jobs=5): err= 0: pid=19635: Wed Jan 22 13:14:31 2020
read: IOPS=176, BW=22.1MiB/s (23.1MB/s)(1333MiB/60384msec)
slat (usec): min=8, max=1719.4k, avg=17069.51, stdev=96755.57
clat (usec): min=1381, max=4744.2k, avg=1213496.72, stdev=624788.47
lat (msec): min=217, max=4744, avg=1230.57, stdev=627.55
clat percentiles (msec):
| 1.00th=[ 338], 5.00th=[ 481], 10.00th=[ 584], 20.00th=[ 726],
| 30.00th=[ 844], 40.00th=[ 978], 50.00th=[ 1099], 60.00th=[ 1217],
| 70.00th=[ 1368], 80.00th=[ 1586], 90.00th=[ 2005], 95.00th=[ 2433],
| 99.00th=[ 3406], 99.50th=[ 3943], 99.90th=[ 4732], 99.95th=[ 4732],
| 99.99th=[ 4732]
bw ( KiB/s): min= 255, max=16896, per=24.83%, avg=5613.63, stdev=3621.35, samples=476
iops : min= 1, max= 132, avg=43.74, stdev=28.31, samples=476
write: IOPS=76, BW=9823KiB/s (10.1MB/s)(579MiB/60384msec)
slat (usec): min=13, max=1551.5k, avg=18518.81, stdev=102342.05
clat (msec): min=235, max=4744, avg=1283.20, stdev=654.13
lat (msec): min=288, max=4782, avg=1301.72, stdev=656.34
clat percentiles (msec):
| 1.00th=[ 342], 5.00th=[ 518], 10.00th=[ 634], 20.00th=[ 768],
| 30.00th=[ 885], 40.00th=[ 1011], 50.00th=[ 1167], 60.00th=[ 1267],
| 70.00th=[ 1469], 80.00th=[ 1670], 90.00th=[ 2123], 95.00th=[ 2500],
| 99.00th=[ 3675], 99.50th=[ 4178], 99.90th=[ 4732], 99.95th=[ 4732],
| 99.99th=[ 4732]
bw ( KiB/s): min= 255, max= 7680, per=28.28%, avg=2777.79, stdev=1665.69, samples=418
iops : min= 1, max= 60, avg=21.58, stdev=13.04, samples=418
lat (msec) : 2=0.06%, 4=0.06%, 10=0.01%, 250=0.03%, 500=5.27%
lat (msec) : 750=15.82%, 1000=20.37%
cpu : usr=0.06%, sys=0.11%, ctx=1087, majf=0, minf=152
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.3%, 16=0.5%, 32=1.0%, >=64=97.9%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=10664,4634,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=22.1MiB/s (23.1MB/s), 22.1MiB/s-22.1MiB/s (23.1MB/s-23.1MB/s), io=1333MiB (1398MB), run=60384-60384msec
WRITE: bw=9823KiB/s (10.1MB/s), 9823KiB/s-9823KiB/s (10.1MB/s-10.1MB/s), io=579MiB (607MB), run=60384-60384msec
Disk stats (read/write):
sda: ios=10647/4651, merge=3/59, ticks=6213746/2900566, in_queue=9144148, util=99.91%
TEST-256K
RandomRW: (g=0): rw=randrw, bs=(R) 256KiB-256KiB, (W) 256KiB-256KiB, (T) 256KiB-256KiB, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 5 processes
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: (groupid=0, jobs=5): err= 0: pid=19659: Wed Jan 22 13:20:27 2020
read: IOPS=182, BW=45.5MiB/s (47.7MB/s)(2739MiB/60140msec)
slat (usec): min=9, max=1521.1k, avg=17565.61, stdev=95848.61
clat (usec): min=12, max=3376.2k, avg=1183428.74, stdev=490665.20
lat (msec): min=134, max=3657, avg=1201.00, stdev=492.08
clat percentiles (msec):
| 1.00th=[ 397], 5.00th=[ 523], 10.00th=[ 625], 20.00th=[ 785],
| 30.00th=[ 894], 40.00th=[ 986], 50.00th=[ 1099], 60.00th=[ 1200],
| 70.00th=[ 1385], 80.00th=[ 1552], 90.00th=[ 1821], 95.00th=[ 2140],
| 99.00th=[ 2668], 99.50th=[ 2869], 99.90th=[ 3104], 99.95th=[ 3373],
| 99.99th=[ 3373]
bw ( KiB/s): min= 510, max=33724, per=23.66%, avg=11033.83, stdev=6723.85, samples=498
iops : min= 1, max= 131, avg=42.97, stdev=26.28, samples=498
write: IOPS=79, BW=19.8MiB/s (20.7MB/s)(1189MiB/60140msec)
slat (usec): min=17, max=1382.7k, avg=18346.45, stdev=95889.78
clat (msec): min=23, max=3376, avg=1230.18, stdev=502.88
lat (msec): min=248, max=3376, avg=1248.53, stdev=504.90
clat percentiles (msec):
| 1.00th=[ 405], 5.00th=[ 558], 10.00th=[ 693], 20.00th=[ 827],
| 30.00th=[ 919], 40.00th=[ 1011], 50.00th=[ 1133], 60.00th=[ 1267],
| 70.00th=[ 1435], 80.00th=[ 1603], 90.00th=[ 1921], 95.00th=[ 2198],
| 99.00th=[ 2869], 99.50th=[ 2937], 99.90th=[ 3239], 99.95th=[ 3373],
| 99.99th=[ 3373]
bw ( KiB/s): min= 506, max=14818, per=26.80%, avg=5425.89, stdev=2843.68, samples=439
iops : min= 1, max= 57, avg=21.05, stdev=11.13, samples=439
lat (usec) : 20=0.01%
lat (msec) : 2=0.02%, 4=0.03%, 10=0.01%, 50=0.03%, 250=0.02%
lat (msec) : 500=3.28%, 750=12.27%, 1000=24.81%
cpu : usr=0.07%, sys=0.17%, ctx=971, majf=0, minf=152
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.3%, 16=0.5%, 32=1.0%, >=64=98.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=10954,4756,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=45.5MiB/s (47.7MB/s), 45.5MiB/s-45.5MiB/s (47.7MB/s-47.7MB/s), io=2739MiB (2872MB), run=60140-60140msec
WRITE: bw=19.8MiB/s (20.7MB/s), 19.8MiB/s-19.8MiB/s (20.7MB/s-20.7MB/s), io=1189MiB (1247MB), run=60140-60140msec
Disk stats (read/write):
sda: ios=10910/4766, merge=11/70, ticks=6201806/2901136, in_queue=9149819, util=99.88%
TEST-512K
RandomRW: (g=0): rw=randrw, bs=(R) 512KiB-512KiB, (W) 512KiB-512KiB, (T) 512KiB-512KiB, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 5 processes
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: (groupid=0, jobs=5): err= 0: pid=19687: Wed Jan 22 13:26:28 2020
read: IOPS=131, BW=65.6MiB/s (68.8MB/s)(3959MiB/60320msec)
slat (usec): min=20, max=1712.4k, avg=24932.94, stdev=133250.21
clat (msec): min=2, max=4393, avg=1624.41, stdev=729.41
lat (msec): min=60, max=4788, avg=1649.34, stdev=735.75
clat percentiles (msec):
| 1.00th=[ 550], 5.00th=[ 684], 10.00th=[ 776], 20.00th=[ 1003],
| 30.00th=[ 1167], 40.00th=[ 1334], 50.00th=[ 1485], 60.00th=[ 1670],
| 70.00th=[ 1888], 80.00th=[ 2198], 90.00th=[ 2735], 95.00th=[ 3037],
| 99.00th=[ 3675], 99.50th=[ 3708], 99.90th=[ 4144], 99.95th=[ 4144],
| 99.99th=[ 4396]
bw ( KiB/s): min= 1021, max=53248, per=28.45%, avg=19121.59, stdev=12651.28, samples=411
iops : min= 1, max= 104, avg=37.26, stdev=24.70, samples=411
write: IOPS=56, BW=28.3MiB/s (29.7MB/s)(1706MiB/60320msec)
slat (usec): min=37, max=1325.1k, avg=27028.99, stdev=136095.20
clat (msec): min=269, max=4522, avg=1733.42, stdev=760.48
lat (msec): min=309, max=4522, avg=1760.45, stdev=767.35
clat percentiles (msec):
| 1.00th=[ 567], 5.00th=[ 718], 10.00th=[ 827], 20.00th=[ 1083],
| 30.00th=[ 1267], 40.00th=[ 1401], 50.00th=[ 1569], 60.00th=[ 1821],
| 70.00th=[ 2005], 80.00th=[ 2333], 90.00th=[ 2903], 95.00th=[ 3205],
| 99.00th=[ 3742], 99.50th=[ 4144], 99.90th=[ 4396], 99.95th=[ 4530],
| 99.99th=[ 4530]
bw ( KiB/s): min= 1021, max=27648, per=32.99%, avg=9554.46, stdev=5727.36, samples=357
iops : min= 1, max= 54, avg=18.57, stdev=11.14, samples=357
lat (msec) : 4=0.07%, 10=0.02%, 50=0.01%, 100=0.01%, 500=0.34%
lat (msec) : 750=6.36%, 1000=11.63%
cpu : usr=0.07%, sys=0.18%, ctx=709, majf=0, minf=152
IO depths : 1=0.1%, 2=0.1%, 4=0.2%, 8=0.4%, 16=0.7%, 32=1.4%, >=64=97.2%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=7917,3412,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=65.6MiB/s (68.8MB/s), 65.6MiB/s-65.6MiB/s (68.8MB/s-68.8MB/s), io=3959MiB (4151MB), run=60320-60320msec
WRITE: bw=28.3MiB/s (29.7MB/s), 28.3MiB/s-28.3MiB/s (29.7MB/s-29.7MB/s), io=1706MiB (1789MB), run=60320-60320msec
Disk stats (read/write):
sda: ios=8055/3504, merge=7732/3413, ticks=6166858/2894175, in_queue=9088467, util=99.80%
TEST-1024K
RandomRW: (g=0): rw=randrw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 5 processes
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: Laying out IO file (1 file / 5120MiB)
RandomRW: (groupid=0, jobs=5): err= 0: pid=19719: Wed Jan 22 13:32:26 2020
read: IOPS=57, BW=57.9MiB/s (60.7MB/s)(3510MiB/60661msec)
slat (usec): min=48, max=1997.2k, avg=59810.00, stdev=202101.84
clat (msec): min=172, max=8239, avg=3641.43, stdev=1251.37
lat (msec): min=287, max=8563, avg=3701.24, stdev=1259.03
clat percentiles (msec):
| 1.00th=[ 995], 5.00th=[ 1737], 10.00th=[ 2022], 20.00th=[ 2567],
| 30.00th=[ 2970], 40.00th=[ 3272], 50.00th=[ 3608], 60.00th=[ 3910],
| 70.00th=[ 4245], 80.00th=[ 4665], 90.00th=[ 5269], 95.00th=[ 5805],
| 99.00th=[ 7080], 99.50th=[ 7416], 99.90th=[ 7886], 99.95th=[ 8221],
| 99.99th=[ 8221]
bw ( KiB/s): min= 2035, max=90112, per=27.88%, avg=16517.00, stdev=12298.65, samples=408
iops : min= 1, max= 88, avg=16.07, stdev=12.01, samples=408
write: IOPS=25, BW=25.6MiB/s (26.9MB/s)(1555MiB/60661msec)
slat (usec): min=86, max=1993.8k, avg=58651.53, stdev=195074.96
clat (msec): min=378, max=8239, avg=3739.21, stdev=1282.77
lat (msec): min=700, max=8467, avg=3797.86, stdev=1296.81
clat percentiles (msec):
| 1.00th=[ 995], 5.00th=[ 1770], 10.00th=[ 2072], 20.00th=[ 2635],
| 30.00th=[ 3071], 40.00th=[ 3306], 50.00th=[ 3675], 60.00th=[ 4010],
| 70.00th=[ 4396], 80.00th=[ 4799], 90.00th=[ 5336], 95.00th=[ 5805],
| 99.00th=[ 7148], 99.50th=[ 7617], 99.90th=[ 8221], 99.95th=[ 8221],
| 99.99th=[ 8221]
bw ( KiB/s): min= 2027, max=47104, per=31.88%, avg=8368.67, stdev=6653.35, samples=357
iops : min= 1, max= 46, avg= 8.10, stdev= 6.51, samples=357
lat (msec) : 250=0.02%, 500=0.20%, 750=0.39%, 1000=0.77%
cpu : usr=0.06%, sys=0.14%, ctx=781, majf=0, minf=147
IO depths : 1=0.1%, 2=0.2%, 4=0.4%, 8=0.8%, 16=1.6%, 32=3.2%, >=64=93.8%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=3510,1555,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=57.9MiB/s (60.7MB/s), 57.9MiB/s-57.9MiB/s (60.7MB/s-60.7MB/s), io=3510MiB (3681MB), run=60661-60661msec
WRITE: bw=25.6MiB/s (26.9MB/s), 25.6MiB/s-25.6MiB/s (26.9MB/s-26.9MB/s), io=1555MiB (1631MB), run=60661-60661msec
Disk stats (read/write):
sda: ios=10514/4689, merge=0/80, ticks=6230183/3064867, in_queue=9329265, util=99.65%
*****************************************************************************