Re: shiftfs status and future development

[Christian Brauner <christian@brauner.io>]

On Mon, Jun 18, 2018 at 08:11:52PM +0300, Amir Goldstein wrote:
> On Mon, Jun 18, 2018 at 5:56 PM, James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
> [...]
> >> > >  - Does not break inotify
> >> >
> >> > I don't expect it does, but I haven't checked.
> >>
> >> I haven't checked either; I'm planning to do so soon. This is a
> >> concern that was expressed to me by others, I think because inotify
> >> doesn't work with overlayfs.
> >
> > I think shiftfs does work simply because it doesn't really do overlays,
> > so lots of stuff that doesn't work with overlays does work with it.
> >
> 
> I'm afraid shiftfs suffers from the same problems that the old naiive
> overlayfs inode implementation suffered from.
> 
> This problem is demonstrated with LTP tests inotify08 inotify09.
> shiftfs_new_inode() is called on every lookup, so inotify watch
> may be set on an inode object, then dentry is evicted from cache
> and then all events on new dentry are not reported on the watched
> inode. You will need to implement hashed inodes to solve it.
> Can be done as overlay does - hashing by real inode pointer.
> 
> This is just one of those subtle things about stacked fs and there may
> be other in present and more in future - if we don't have a shared code
> base for the two stacked fs, I wager you are going to end up "cherry
> picking" fixes often.
> 
> IMO, an important question to ask is, since both shiftfs and overlayfs
> are strongly coupled with container use cases, are there users that
> are interested in both layering AND shifting? on the same "mark"?
> If the answer is yes, then this may be an argument in favor of
> integrating at least some of shittfs functionality into overlayfs.

I think that there are use-cases for shiftfs and overlayfs. Especially
since shifts and overlayfs have decent conceptual overlap. They both
provide a mechanism to isolate or at least protect a filesystem tree.
The mechanisms differ but I see no obvious reasons why they couldn't be
combined. That is one of the reasons why we considered it an option to
make shiftfs an extension of overlayfs. But this has been just a thought
experiment so far.
One of the benefits of combining shiftfs and overlayfs I see could be
that the underlay could be made read-only and when combined with shifts
writing a file (e.g. as root) would only hit the upperdir. Although, for
shifts we really also want to allow an option to write through to the
underlay. If done right we also get the nice features of merging
multiple underlays together which seems a good feature to have for
containers. Additionally, it would also allow us to not have to push
another filesystem upstream. (That of course also would not be necessary
if this would be a pure vfs feature which Seth already mentioned in a
prior mail.)

> 
> Another argument is that shiftfs itself takes the maximum allowed
> 2 levels of s_stack_depth for it's 2 mounts, so it is actually not
> possible with current VFS limitation to combine shiftfs with overlayfs.
> 
> This could be solved relatively easily by adding "-o mark" support
> to overlayfs and allowing to mount shiftfs also over "marked" overlayfs
> inside container.
> 
> Anyway, I'm just playing devil's advocate to the idea of two stacked fs
> implementation, so presenting this point of view. I am fully aware that
> there are also plenty of disadvantages to couple two unrelated
> functionalities together.
> 
> Cheers,
> Amir.
> _______________________________________________
> Containers mailing list
> Containers@lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/containers