Re: shiftfs status and future development

From: Amir Goldstein <amir73il@gmail.com>
To: James Bottomley <James.Bottomley@hansenpartnership.com>
Cc: Seth Forshee <seth.forshee@canonical.com>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	Tyler Hicks <tyler.hicks@canonical.com>,
	Linux Containers <containers@lists.linux-foundation.org>,
	Christian Brauner <christian.brauner@canonical.com>
Subject: Re: shiftfs status and future development
Date: Mon, 18 Jun 2018 20:11:52 +0300	[thread overview]
Message-ID: <CAOQ4uxiLrhgwc1uNDmL4+DisWFB6EkfeYfybZ-s7db5RzvXZ2g@mail.gmail.com> (raw)
In-Reply-To: <1529333819.4021.4.camel@HansenPartnership.com>

On Mon, Jun 18, 2018 at 5:56 PM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
[...]
>> > >  - Does not break inotify
>> >
>> > I don't expect it does, but I haven't checked.
>>
>> I haven't checked either; I'm planning to do so soon. This is a
>> concern that was expressed to me by others, I think because inotify
>> doesn't work with overlayfs.
>
> I think shiftfs does work simply because it doesn't really do overlays,
> so lots of stuff that doesn't work with overlays does work with it.
>

I'm afraid shiftfs suffers from the same problems that the old naiive
overlayfs inode implementation suffered from.

This problem is demonstrated with LTP tests inotify08 inotify09.
shiftfs_new_inode() is called on every lookup, so inotify watch
may be set on an inode object, then dentry is evicted from cache
and then all events on new dentry are not reported on the watched
inode. You will need to implement hashed inodes to solve it.
Can be done as overlay does - hashing by real inode pointer.

This is just one of those subtle things about stacked fs and there may
be other in present and more in future - if we don't have a shared code
base for the two stacked fs, I wager you are going to end up "cherry
picking" fixes often.

IMO, an important question to ask is, since both shiftfs and overlayfs
are strongly coupled with container use cases, are there users that
are interested in both layering AND shifting? on the same "mark"?
If the answer is yes, then this may be an argument in favor of
integrating at least some of shittfs functionality into overlayfs.

Another argument is that shiftfs itself takes the maximum allowed
2 levels of s_stack_depth for it's 2 mounts, so it is actually not
possible with current VFS limitation to combine shiftfs with overlayfs.

This could be solved relatively easily by adding "-o mark" support
to overlayfs and allowing to mount shiftfs also over "marked" overlayfs
inside container.

Anyway, I'm just playing devil's advocate to the idea of two stacked fs
implementation, so presenting this point of view. I am fully aware that
there are also plenty of disadvantages to couple two unrelated
functionalities together.

Cheers,
Amir.