Re: [RFC PATCH v2 0/8] implement containerized syncfs for overlayfs

From: Chengguang Xu <cgxu519@mykernel.net>
To: "Miklos Szeredi" <miklos@szeredi.hu>
Cc: "Amir Goldstein" <amir73il@gmail.com>, "Jan Kara" <jack@suse.cz>,
	"overlayfs" <linux-unionfs@vger.kernel.org>,
	"linux-fsdevel" <linux-fsdevel@vger.kernel.org>
Subject: Re: [RFC PATCH v2 0/8] implement containerized syncfs for overlayfs
Date: Sat, 31 Oct 2020 20:22:05 +0800	[thread overview]
Message-ID: <1757e9b90db.10c024288229.900734846219816716@mykernel.net> (raw)
In-Reply-To: <CAJfpegu-bn2BjkLaykk-gZLRv71n=PgrsrwBnuAav1GHzWO5iQ@mail.gmail.com>

 ---- 在 星期五, 2020-10-30 23:46:00 Miklos Szeredi <miklos@szeredi.hu> 撰写 ----
 > On Sun, Oct 25, 2020 at 4:42 AM Chengguang Xu <cgxu519@mykernel.net> wrote:
 > >
 > > Current syncfs(2) syscall on overlayfs just calls sync_filesystem()
 > > on upper_sb to synchronize whole dirty inodes in upper filesystem
 > > regardless of the overlay ownership of the inode. In the use case of
 > > container, when multiple containers using the same underlying upper
 > > filesystem, it has some shortcomings as below.
 > >
 > > (1) Performance
 > > Synchronization is probably heavy because it actually syncs unnecessary
 > > inodes for target overlayfs.
 > >
 > > (2) Interference
 > > Unplanned synchronization will probably impact IO performance of
 > > unrelated container processes on the other overlayfs.
 > >
 > > This series try to implement containerized syncfs for overlayfs so that
 > > only sync target dirty upper inodes which are belong to specific overlayfs
 > > instance. By doing this, it is able to reduce cost of synchronization and
 > > will not seriously impact IO performance of unrelated processes.
 > 
 > Series looks good at first glance.  Still need to do an in-depth review.
 > 
 > In the meantime can you post some numbers showing the performance improvements?
 > 

Actually, the performance improvement depends on workload and the number of overlay instances.

I did a very simple test on my dev box (intel NUC ) and the result seems quite obvious.

I made two overlay instances and untar a kernel source to an overlay then compare

umount time(sys part) of two instances. 

------------------------------------------
test env:
Intel(R) Core(TM) i7-8809G CPU @ 3.10GHz  8core
32GB memory
Samsung NVMe SSD 970 pro

test script:

[root@localhost ovl]# cat do.sh
#!/bin/sh

rm -rf upper/* &> /dev/null
echo 3 > /proc/sys/vm/drop_caches

mount -t overlay overlay -o lowerdir=lower,workdir=work,upperdir=upper merged
mount -t overlay overlay -o lowerdir=lower2,workdir=work2,upperdir=upper2 merged2

tar Jxvf linux-5.9.tar.xz -C merged &> /dev/null

time umount merged2
time umount merged

---------------------------------
test result:

[root@localhost ovl]# seq 3 | while read line ; do sh -x do.sh ; done
+ rm -rf upper/linux-5.9
+ echo 3
+ mount -t overlay overlay -o lowerdir=lower,workdir=work,upperdir=upper merged
+ mount -t overlay overlay -o lowerdir=lower2,workdir=work2,upperdir=upper2 merged2
+ tar Jxvf linux-5.9.tar.xz -C merged
+ umount merged2

real    0m0.437s
user    0m0.001s
sys     0m0.001s
+ umount merged

real    0m2.107s
user    0m0.002s
sys     0m0.090s
+ rm -rf upper/linux-5.9
+ echo 3
+ mount -t overlay overlay -o lowerdir=lower,workdir=work,upperdir=upper merged
+ mount -t overlay overlay -o lowerdir=lower2,workdir=work2,upperdir=upper2 merged2
+ tar Jxvf linux-5.9.tar.xz -C merged
+ umount merged2

real    0m0.443s
user    0m0.002s
sys     0m0.001s
+ umount merged

real    0m2.032s
user    0m0.001s
sys     0m0.105s
+ rm -rf upper/linux-5.9
+ echo 3
+ mount -t overlay overlay -o lowerdir=lower,workdir=work,upperdir=upper merged
+ mount -t overlay overlay -o lowerdir=lower2,workdir=work2,upperdir=upper2 merged2
+ tar Jxvf linux-5.9.tar.xz -C merged
+ umount merged2

real    0m0.263s
user    0m0.001s
sys     0m0.001s
+ umount merged

real    0m2.094s
user    0m0.000s
sys     0m0.083s
[root@localhost o

Thanks,
Chengguang