Re: shiftfs status and future development

From: Phil Estes <estesp@gmail.com>
To: Amir Goldstein <amir73il@gmail.com>
Cc: James Bottomley <James.Bottomley@hansenpartnership.com>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	Seth Forshee <seth.forshee@canonical.com>,
	Linux Containers <containers@lists.linux-foundation.org>,
	Tyler Hicks <tyler.hicks@canonical.com>,
	Christian Brauner <christian.brauner@canonical.com>
Subject: Re: shiftfs status and future development
Date: Mon, 18 Jun 2018 15:53:43 -0400	[thread overview]
Message-ID: <424a9472-91cb-e7d4-b933-2f8b772782fa@gmail.com> (raw)
In-Reply-To: <CAOQ4uxiLrhgwc1uNDmL4+DisWFB6EkfeYfybZ-s7db5RzvXZ2g@mail.gmail.com>

Amir Goldstein wrote on 6/18/18 1:11 PM:
> On Mon, Jun 18, 2018 at 5:56 PM, James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
> [...]
>>>>>   - Does not break inotify
>>>> I don't expect it does, but I haven't checked.
>>> I haven't checked either; I'm planning to do so soon. This is a
>>> concern that was expressed to me by others, I think because inotify
>>> doesn't work with overlayfs.
>> I think shiftfs does work simply because it doesn't really do overlays,
>> so lots of stuff that doesn't work with overlays does work with it.
>>
> I'm afraid shiftfs suffers from the same problems that the old naiive
> overlayfs inode implementation suffered from.
>
> This problem is demonstrated with LTP tests inotify08 inotify09.
> shiftfs_new_inode() is called on every lookup, so inotify watch
> may be set on an inode object, then dentry is evicted from cache
> and then all events on new dentry are not reported on the watched
> inode. You will need to implement hashed inodes to solve it.
> Can be done as overlay does - hashing by real inode pointer.
>
> This is just one of those subtle things about stacked fs and there may
> be other in present and more in future - if we don't have a shared code
> base for the two stacked fs, I wager you are going to end up "cherry
> picking" fixes often.
>
> IMO, an important question to ask is, since both shiftfs and overlayfs
> are strongly coupled with container use cases, are there users that
> are interested in both layering AND shifting? on the same "mark"?
> If the answer is yes, then this may be an argument in favor of
> integrating at least some of shittfs functionality into overlayfs.
>
> Another argument is that shiftfs itself takes the maximum allowed
> 2 levels of s_stack_depth for it's 2 mounts, so it is actually not
> possible with current VFS limitation to combine shiftfs with overlayfs.

I'm not sure if this was my problem or not, but I did try the v2 
patchset with overlay (by marking and mounting a filesystem tree with 
shiftfs, and then using it as the component of an overlay mount) and 
could not get it to work. I was trying to prepare a demo, but with 
limited time gave up and wasn't able to find time to debug with possible 
help from James.

I think for shiftfs to be useful in certain container contexts, 
combination with overlay is definitely a desirable and/or required 
feature. Of course if it is limited to overlay (e.g. implemented within 
overlayfs) that would be limiting for container use cases for other 
contexts (zfs, btrfs, etc.).
>
> This could be solved relatively easily by adding "-o mark" support
> to overlayfs and allowing to mount shiftfs also over "marked" overlayfs
> inside container.
>
> Anyway, I'm just playing devil's advocate to the idea of two stacked fs
> implementation, so presenting this point of view. I am fully aware that
> there are also plenty of disadvantages to couple two unrelated
> functionalities together.
>
> Cheers,
> Amir.
> _______________________________________________
> Containers mailing list
> Containers@lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/containers