Re: Streams support in Linux

From: Andreas Dilger <adilger@dilger.ca>
To: Jeremy Allison <jra@samba.org>
Cc: Tom Talpey <ttalpey@microsoft.com>,
	Steve French <smfrench@gmail.com>, Theodore Tso <tytso@mit.edu>,
	Matthew Wilcox <willy@infradead.org>,
	"ebiggers@kernel.org" <ebiggers@kernel.org>,
	Al Viro <viro@zeniv.linux.org.uk>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>
Subject: Re: Streams support in Linux
Date: Wed, 29 Aug 2018 12:52:47 -0600	[thread overview]
Message-ID: <B4347D04-AA86-416C-A0E1-0D3D7C494C90@dilger.ca> (raw)
In-Reply-To: <20180829155922.GA114671@jra3>

[-- Attachment #1: Type: text/plain, Size: 2236 bytes --]

On Aug 29, 2018, at 9:59 AM, Jeremy Allison <jra@samba.org> wrote:
> 
> On Wed, Aug 29, 2018 at 01:46:30PM +0000, Tom Talpey wrote:
>>> 
>>> This sounds like a case for a couple of ioctls. One to enumerate
>>> the streams on an open fd, one to open a given stream name on an
>>> open fd.
>> 
>> Wait, you're saying that Macs and Windows clients need to start issuing these new magical ioctl's to Samba/Linux servers? Not a solution, IMO.
> 
> Nope, that's not an option obviously. The ioctls only exist
> on the Linux userspace client side. Inside cifsfs they get
> converted to standard SMB1/2/3 calls.
> 
> I'm trying to create a way for Linux userspace to get
> access to these Windows/Mac/Samba features without having
> to completely change the VFS model to introduce the
> horrible Solaris semantics of opening a file as a
> directory to get access to internal streams.

Note that ext4 recently added support for xattrs larger than the
previous one-block (usually 4KB) total size.  The large xattrs were
implemented as a separate inode attached to the file/directory to
reference the multiple data blocks allocated to the xattr, which
simplifies e2fsck and ext4 block allocation/freeing/traversal.

There is no real limit on the size of such xattr inodes, except as
currently imposed by the xattr userspace APIs.  If there was an API
(e.g. openat() or an ioctl() to start) that allowed opening the
named xattrs, normal read/write should be useable to modify them.

I don't think I'd go as far as Matthew to allow link, rename, etc.
of these attributes, as that introduces unnecessary complexity for
no clear use case or benefit.  That doesn't preclude future usage
in this manner, but I don't think it makes sense to add that until
there is a clear use for this.

For the original FS Verity case, using xattrs attached to the inode
is IMHO far preferable to using a separate hidden directory and/or
special file to store the verity information, as it will properly
follow the lifetime of the original file.  Rename will keep the
verity data, and unlink will delete the verity data, without the
need for fsck to keep them consistent or do garbage collection.

Cheers, Andreas

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 873 bytes --]