Hi Gao, How you got the read amplification? I ran FIO on enwik9 (both Erofs and SquashFs) and got the below output. Is there anyway to calculate the read amplification from the below logs. Here filename (/mnt/enwik9_erofs/enwik9, /mnt/enwiki_sqfs/enwik9) points to the mounted readonly file system(squasfs, erofs). But if I give directory as a parameter instead of filename I am getting error(see the logs at the end). *FIO on Erofs:* localhost:~> fio --name=randread --ioengine=libaio --iodepth=16 --rw=randread --bs=4k --direct=0 --size=512M --numjobs=4 --runtime=240 --group_reporting --filename=/mnt/enwik9_erofs/enwik9 randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16 ... fio-3.17-90-gd9b7 Starting 4 processes Jobs: 4 (f=4): [r(4)][100.0%][r=381MiB/s][r=97.6k IOPS][eta 00m:00s] randread: (groupid=0, jobs=4): err= 0: pid=34282: Mon Jan 27 01:04:55 2020 read: IOPS=36.7k, BW=144MiB/s (150MB/s)(2048MiB/14271msec) slat (nsec): min=1305, max=135688k, avg=106650.48, stdev=493480.73 clat (nsec): min=1970, max=136593k, avg=1629459.90, stdev=2639786.83 lat (usec): min=3, max=136625, avg=1736.29, stdev=2772.32 clat percentiles (usec): | 1.00th=[ 48], 5.00th=[ 69], 10.00th=[ 251], 20.00th=[ 437], | 30.00th=[ 570], 40.00th=[ 701], 50.00th=[ 848], 60.00th=[ 1029], | 70.00th=[ 1336], 80.00th=[ 2147], 90.00th=[ 4015], 95.00th=[ 5932], | 99.00th=[ 11600], 99.50th=[ 13304], 99.90th=[ 17171], 99.95th=[ 20579], | 99.99th=[135267] bw ( KiB/s): min=16510, max=295435, per=76.91%, avg=113025.79, stdev=23830.42, samples=112 iops : min= 4126, max=73857, avg=28254.82, stdev=5957.62, samples=112 lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=1.37% lat (usec) : 100=5.45%, 250=3.15%, 500=14.74%, 750=18.99%, 1000=14.99% lat (msec) : 2=20.14%, 4=11.09%, 10=8.42%, 20=1.62%, 50=0.04% lat (msec) : 250=0.01% cpu : usr=1.87%, sys=8.28%, ctx=144023, majf=1, minf=114 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=524288,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=144MiB/s (150MB/s), 144MiB/s-144MiB/s (150MB/s-150MB/s), io=2048MiB (2147MB), run=14271-14271msec Disk stats (read/write): loop0: ios=137357/0, merge=0/0, ticks=23020/0, in_queue=460, util=97.70% *FIO on SquashFs:* localhost:~/Downloads/erofs-utils> fio --name=randread --ioengine=libaio --iodepth=16 --rw=randread --bs=4k --direct=0 --size=512M --numjobs=4 --runtime=240 --group_reporting --filename=/mnt/enwik9_sqsh/enwik9 randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16 ... fio-3.17-90-gd9b7 Starting 4 processes Jobs: 4 (f=4): [r(4)][66.7%][r=1175MiB/s][r=301k IOPS][eta 00m:05s] randread: (groupid=0, jobs=4): err= 0: pid=34389: Mon Jan 27 01:07:56 2020 read: IOPS=55.4k, BW=216MiB/s (227MB/s)(2048MiB/9467msec) slat (nsec): min=1194, max=61065k, avg=67581.76, stdev=754174.73 clat (usec): min=2, max=222014, avg=1075.25, stdev=5969.94 lat (usec): min=3, max=235437, avg=1143.13, stdev=6341.32 clat percentiles (usec): | 1.00th=[ 39], 5.00th=[ 40], 10.00th=[ 40], 20.00th=[ 41], | 30.00th=[ 42], 40.00th=[ 43], 50.00th=[ 43], 60.00th=[ 44], | 70.00th=[ 45], 80.00th=[ 48], 90.00th=[ 63], 95.00th=[ 3163], | 99.00th=[ 28443], 99.50th=[ 41157], 99.90th=[ 78119], 99.95th=[ 89654], | 99.99th=[125305] bw ( KiB/s): min= 1985, max=991826, per=63.49%, avg=140649.83, stdev=78204.76, samples=72 iops : min= 495, max=247955, avg=35161.00, stdev=19551.19, samples=72 lat (usec) : 4=0.01%, 10=0.01%, 20=0.01%, 50=84.82%, 100=8.18% lat (usec) : 250=0.37%, 500=0.09%, 750=0.24%, 1000=0.54% lat (msec) : 2=0.43%, 4=0.46%, 10=1.29%, 20=1.93%, 50=1.30% lat (msec) : 100=0.33%, 250=0.02% cpu : usr=1.76%, sys=16.29%, ctx=14519, majf=0, minf=104 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=524288,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=216MiB/s (227MB/s), 216MiB/s-216MiB/s (227MB/s-227MB/s), io=2048MiB (2147MB), run=9467-9467msec Disk stats (read/write): loop1: ios=177240/0, merge=0/0, ticks=199386/0, in_queue=75984, util=73.95% Fio Test on SquashFs dir: localhost:~/Downloads/erofs-utils> fio --name=randread --ioengine=libaio --iodepth=16 --rw=randread --bs=4k --direct=0 --size=512M --numjobs=4 --runtime=240 --group_reporting --directory=/mnt/enwik9_sqsh/ randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16 ... fio-3.17-90-gd9b7 Starting 4 processes randread: Laying out IO file (1 file / 512MiB) fio: pid=0, err=30/file:filesetup.c:150, func=unlink, error=Read-only file system randread: Laying out IO file (1 file / 512MiB) fio: pid=0, err=30/file:filesetup.c:150, func=unlink, error=Read-only file system randread: Laying out IO file (1 file / 512MiB) fio: pid=0, err=30/file:filesetup.c:150, func=unlink, error=Read-only file system randread: Laying out IO file (1 file / 512MiB) fio: pid=0, err=30/file:filesetup.c:150, func=unlink, error=Read-only file system Run status group 0 (all jobs): On Wed, Jan 22, 2020 at 10:07 AM Gao Xiang wrote: > On Wed, Jan 22, 2020 at 09:27:45AM +0530, Saumya Panda wrote: > > Hi Gao, > > Thanks for the info. After I enabled the said configuration, I am now > > able to read the files after mount. But I am seeing Squashfs has better > > compression ratio compared to Erofs (more than 60% than that of Erofs). > Am > > I missing something? I used lz4hc while making the Erofs image. > > > > ls -l enwik* > > -rw-r--r-- 1 saumya users 61280256 Jan 21 03:22 enwik8.erofs.img > > -rw-r--r-- 1 saumya users 37355520 Jan 21 03:34 enwik8.sqsh > > -rw-r--r-- 1 saumya users 558133248 Jan 21 03:25 enwik9.erofs.img > > -rw-r--r-- 1 saumya users 331481088 Jan 21 03:35 enwik9.sqsh > > Yes, it's working as expect. Currently EROFS is compressed in 4k > fixed-sized output compression granularity as mentioned in many > available materials. That is the use case for our smartphones. > You should compare with similar block configuration of squashfs. > and there are some 3rd data by other folks as well [1]. > > In the future, we will support other compression algorithms and > larger compressed size (> 4k). > > [1] In chinese, > https://blog.csdn.net/scnutiger/article/details/102507596 > > Thanks, > Gao Xiang > > -- Thanks, Saumya Prakash Panda