SD Card vs. eMMC Benchmarks

Here is an update with a 320GB HDD (TOSHIBA MQ01ABD0) attached to A20-OLinuXino-Lime2.

HDD - Random Write: 3237KiB/s

admin@freedombox:~$ fio --name=randwrite --ioengine=libaio --iodepth=1 --rw=randwrite --bs=4k --direct=0 --size=512M --runtime=240
randwrite: (g=0): rw=randwrite, bs=® 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.21
Starting 1 process
Jobs: 1 (f=1): [w(1)][100.0%][w=10.7MiB/s][w=2740 IOPS][eta 00m:00s]
randwrite: (groupid=0, jobs=1): err= 0: pid=1213: Sat Jul 25 02:04:13 2020
write: IOPS=809, BW=3237KiB/s (3314kB/s)(512MiB/161985msec); 0 zone resets
slat (usec): min=53, max=568483, avg=1153.27, stdev=8460.15
clat (usec): min=5, max=40214, avg=25.70, stdev=329.48
lat (usec): min=62, max=568538, avg=1188.78, stdev=8476.09
clat percentiles (usec):
| 1.00th=[ 8], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 10],
| 30.00th=[ 11], 40.00th=[ 12], 50.00th=[ 13], 60.00th=[ 15],
| 70.00th=[ 17], 80.00th=[ 21], 90.00th=[ 29], 95.00th=[ 37],
| 99.00th=[ 83], 99.50th=[ 176], 99.90th=[ 2040], 99.95th=[ 4490],
| 99.99th=[14353]
bw ( KiB/s): min= 8, max=19984, per=100.00%, avg=3239.61, stdev=3046.63, samples=323
iops : min= 2, max= 4996, avg=809.68, stdev=761.67, samples=323
lat (usec) : 10=25.33%, 20=52.99%, 50=19.18%, 100=1.72%, 250=0.37%
lat (usec) : 500=0.13%, 750=0.06%, 1000=0.04%
lat (msec) : 2=0.08%, 4=0.05%, 10=0.04%, 20=0.01%, 50=0.01%
cpu : usr=4.12%, sys=16.67%, ctx=33375, majf=1, minf=34
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,131072,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
WRITE: bw=3237KiB/s (3314kB/s), 3237KiB/s-3237KiB/s (3314kB/s-3314kB/s), io=512MiB (537MB), run=161985-161985msec

HDD - Sequential Write: 33.4MiB/s

admin@freedombox:~$ fio --name=write --ioengine=libaio --iodepth=1 --rw=write --bs=4k --direct=0 --size=512M --runtime=240
write: (g=0): rw=write, bs=® 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.21
Starting 1 process
Jobs: 1 (f=1): [W(1)][100.0%][w=36.4MiB/s][w=9326 IOPS][eta 00m:00s]
write: (groupid=0, jobs=1): err= 0: pid=1221: Sat Jul 25 02:05:53 2020
write: IOPS=8550, BW=33.4MiB/s (35.0MB/s)(512MiB/15330msec); 0 zone resets
slat (usec): min=46, max=118899, avg=88.26, stdev=771.85
clat (usec): min=5, max=65971, avg= 9.77, stdev=259.47
lat (usec): min=54, max=118944, avg=101.12, stdev=824.24
clat percentiles (usec):
| 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 7], 20.00th=[ 7],
| 30.00th=[ 7], 40.00th=[ 8], 50.00th=[ 8], 60.00th=[ 8],
| 70.00th=[ 9], 80.00th=[ 9], 90.00th=[ 11], 95.00th=[ 12],
| 99.00th=[ 20], 99.50th=[ 30], 99.90th=[ 149], 99.95th=[ 289],
| 99.99th=[ 857]
bw ( KiB/s): min=25639, max=41269, per=99.87%, avg=34156.13, stdev=4447.53, samples=30
iops : min= 6409, max=10317, avg=8538.77, stdev=1111.87, samples=30
lat (usec) : 10=86.64%, 20=12.39%, 50=0.66%, 100=0.05%, 250=0.19%
lat (usec) : 500=0.05%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 10=0.01%, 100=0.01%
cpu : usr=17.56%, sys=60.18%, ctx=1615, majf=1, minf=32
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,131072,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
WRITE: bw=33.4MiB/s (35.0MB/s), 33.4MiB/s-33.4MiB/s (35.0MB/s-35.0MB/s), io=512MiB (537MB), run=15330-15330msec

HDD - Random Read: 381KiB/s

admin@freedombox:~$ fio --name=read --ioengine=libaio --iodepth=1 --rw=randread --bs=4k --direct=0 --size=512M --runtime=240

read: (g=0): rw=randread, bs=® 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.21
Starting 1 process
Jobs: 1 (f=1): [r(1)][100.0%][r=228KiB/s][r=57 IOPS][eta 00m:00s]
read: (groupid=0, jobs=1): err= 0: pid=1302: Sat Jul 25 02:17:03 2020
read: IOPS=95, BW=381KiB/s (390kB/s)(89.3MiB/240011msec)
slat (usec): min=959, max=108969, avg=10269.11, stdev=4198.59
clat (usec): min=16, max=6721, avg=64.84, stdev=120.96
lat (usec): min=1006, max=109098, avg=10369.46, stdev=4201.37
clat percentiles (usec):
| 1.00th=[ 26], 5.00th=[ 35], 10.00th=[ 37], 20.00th=[ 39],
| 30.00th=[ 41], 40.00th=[ 44], 50.00th=[ 63], 60.00th=[ 66],
| 70.00th=[ 68], 80.00th=[ 71], 90.00th=[ 74], 95.00th=[ 77],
| 99.00th=[ 293], 99.50th=[ 594], 99.90th=[ 1516], 99.95th=[ 2638],
| 99.99th=[ 5669]
bw ( KiB/s): min= 135, max= 471, per=100.00%, avg=381.14, stdev=36.95, samples=478
iops : min= 33, max= 117, avg=94.71, stdev= 9.23, samples=478
lat (usec) : 20=0.07%, 50=42.88%, 100=54.87%, 250=0.70%, 500=0.84%
lat (usec) : 750=0.32%, 1000=0.06%
lat (msec) : 2=0.17%, 4=0.06%, 10=0.02%
cpu : usr=1.97%, sys=9.19%, ctx=25694, majf=0, minf=33
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=22851,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
READ: bw=381KiB/s (390kB/s), 381KiB/s-381KiB/s (390kB/s-390kB/s), io=89.3MiB (93.6MB), run=240011-240011msec

HDD - Sequential Read: 20.8MiB/s

admin@freedombox:~$ fio --name=read --ioengine=libaio --iodepth=1 --rw=read --bs=4k --direct=0 --size=512M --runtime=240
read: (g=0): rw=read, bs=® 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.21
Starting 1 process
Jobs: 1 (f=1): [R(1)][100.0%][r=23.0MiB/s][r=6140 IOPS][eta 00m:00s]
read: (groupid=0, jobs=1): err= 0: pid=1297: Sat Jul 25 02:12:21 2020
read: IOPS=5329, BW=20.8MiB/s (21.8MB/s)(512MiB/24594msec)
slat (usec): min=12, max=477916, avg=166.13, stdev=3299.99
clat (usec): min=3, max=8573, avg= 6.39, stdev=45.86
lat (usec): min=17, max=477963, avg=175.04, stdev=3304.32
clat percentiles (usec):
| 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 5],
| 30.00th=[ 5], 40.00th=[ 5], 50.00th=[ 5], 60.00th=[ 6],
| 70.00th=[ 6], 80.00th=[ 6], 90.00th=[ 7], 95.00th=[ 8],
| 99.00th=[ 25], 99.50th=[ 36], 99.90th=[ 178], 99.95th=[ 310],
| 99.99th=[ 1975]
bw ( KiB/s): min= 8510, max=37976, per=98.70%, avg=21039.19, stdev=6647.59, samples=48
iops : min= 2127, max= 9494, avg=5259.56, stdev=1661.99, samples=48
lat (usec) : 4=3.08%, 10=94.02%, 20=1.70%, 50=0.80%, 100=0.20%
lat (usec) : 250=0.12%, 500=0.04%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%
cpu : usr=8.68%, sys=26.94%, ctx=2171, majf=0, minf=323
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=131072,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
READ: bw=20.8MiB/s (21.8MB/s), 20.8MiB/s-20.8MiB/s (21.8MB/s-21.8MB/s), io=512MiB (537MB), run=24594-24594msec