Streams support in Linux

Streams support in Linux

[Matthew Wilcox]

[starting a separate thread to not hijack the fs-verity submission]

Eric Biggers wrote:
> In theory it would be a much cleaner design to store verity metadata
> separately from the data.  But the Merkle tree can be very large.
> For example, a 1 GB file using SHA-512 would have a 16.6 MB Merkle tree.
> So the Merkle tree can't be an extended attribute, since the xattrs API
> requires xattrs to be small (<= 64 KB), and most filesystems further limit
> xattr sizes in their on-disk format to as little as 4 KB.  Furthermore,
> even if both of these limits were to be increased, the xattrs functions
> (both the syscalls, and the internal functions that filesystems have)
> are all based around getting/setting the entire xattr value.
> 
> Also when used with fscrypt, we want the Merkle tree and
> fsverity_descriptor to be encrypted, so they doesn't leak plaintext
> hashes.  And we want the Merkle tree to be paged into memory, just like
> the file contents, to take advantage of the usual Linux memory management.
> 
> What we really need is *streams*, like NTFS has.  But the filesystems
> we're targetting don't support streams, nor does the Linux syscall
> interface have any API for accessing streams, nor does the VFS support
> them.
> 
> Adding streams support to all those things would be a huge multi-year
> effort, controversial, and almost certainly not worth it just for
> fs-verity.

There are, of course, other clients for file streams.  Samba is one,
GNOME could use streams for various desktoppy things, and I'm certain
other users would come out of the woodwork if we had them.

Let's go over the properties of a file stream:

 - It has no life independent of the file it's attached to; you can't move
   it from one file to another
 - If the file is deleted, it is also deleted
 - If the file is renamed, it travels with the file
 - If the file is copied, the copying program decides whether any named
   streams are copied along with it.
 - Can be created, deleted.  Can be renamed?
 - Openable, seekable, cachable
 - Does not have sub-streams of its own
 - Directories may also have streams which are distinct from the files
   in the directory
 - Can pipes / sockets / device nodes / symlinks / ... have streams?  Unclear.
   Probably not useful.

NTFS, UDF and SMB all support streams already.  Microsoft opted to
include the functionality in ReFS (which dropped some of the less-used
functionality of NTFS), so it's clearly useful.

Here's my proposed syscall API for this:

openat()
To access a named stream, we need to be able to get a file descriptor for
it.  The new openat() syscall seems like the best way to accompish this;
specify a file descriptor, a new AT_NAMED_STREAM flag and a filename,
and the last component of the filename will be treated as the name of
the stream within the object.  This permits us to distinguish between
a named stream on a directory and a file within a directory.

fstat()
st_ino may be different for different names.  st_dev may be different.
st_mode will match the object for files, even if it is changed after
creation.  For directories, it will match except that execute permission
will be removed and S_IFMT will be S_ISREG (do we want to define a
new S_ISSTRM?).  st_nlink will be 1.  st_uid and st_gid will match.  It
will have its own st_atime/st_mtime/st_ctime.  Accessing a stream will not
update its parent's atime/mtime/ctime.

mmap(), read(), write(), close(), splice(), sendfile(), fallocate(),
ftruncate(), dup(), dup2(), dup3(), utimensat(), futimens(), select(), poll(),
lseek(), 
fcntl(): F_DUPFD, F_GETFD, F_GETFL, F_SETFL, F_SETLK, F_SETLKW, F_GETLK,
F_GETOWN, F_SETOWN, F_GETSIG, F_SETSIG, F_SETLEASE, F_GETLEASE)

These system calls work as expected

linkat(), symlinkat(), mknodat(), mkdirat(), 
These system calls will return -EPERM.

renameat()
If olddirfd + oldpath refers to a stream then newdirfd + newpath must
refer to a stream within the same parent object.  If that stream exists,
it is removed.  If olddirfd + oldpath does not refer to a stream, then
newdirfd + newpath must not refer to a stream.

The two file specifications must resolve to the same parent object.  It
is possible to use renameat() to rename a stream within an object, but
not to move a stream from one object to another.  If newpath refers to
an existing named stream, it is removed.  

unlinkat()
This is how you remove an individual named stream

unlink()
Unlinking a file with named streams removes all named streams from that
file and then unlinks the file.  Open streams will continue to exist in
the filesystem until they are closed, just as unlinked files do.

link(), rename()
Renaming or linking to a file with named streams does not affect the streams.

We may need a new system call for enumerating the streams associated
with a file or directory.  We can't use getdents() because there's no
way to distinguish between wanting to read the contents of a directory
and the named streams on a directory.


For shell programming, I would suggest a new program:

	strcat [FILE] [STREAM]...

which opens [FILE], then each named stream within that file, concatenating
said STREAMs to stdout.  We probably need a strls too.

Re: Streams support in Linux

[Al Viro]

On Sat, Aug 25, 2018 at 06:51:07AM -0700, Matthew Wilcox wrote:

> openat()
> To access a named stream, we need to be able to get a file descriptor for
> it.  The new openat() syscall seems like the best way to accompish this;
> specify a file descriptor, a new AT_NAMED_STREAM flag and a filename,
> and the last component of the filename will be treated as the name of
> the stream within the object.  This permits us to distinguish between
> a named stream on a directory and a file within a directory.

What do you do for access via /proc/*/fd/*, when descriptor is a sodding
stream?

> renameat()
> If olddirfd + oldpath refers to a stream then newdirfd + newpath must
> refer to a stream within the same parent object.  If that stream exists,
> it is removed.  If olddirfd + oldpath does not refer to a stream, then
> newdirfd + newpath must not refer to a stream.

And here the shit begins - what to do when parents have different dentries?
Directories can't have hardlinks; regular files very much can.

Where do dentries of these things belong, anyway?

> unlinkat()
> This is how you remove an individual named stream

Yeah... and what happens to access via other hardlinks to the same parent
file?

> unlink()
> Unlinking a file with named streams removes all named streams from that
> file and then unlinks the file.  Open streams will continue to exist in
> the filesystem until they are closed, just as unlinked files do.

Can they be looked up by openat() from such a starting point?

The only new part in that proposal is requesting explicit AT_... flag to
even get to those.  In effect, you are multiplexing a lot of syscalls
that way, which does, in theory, allow different locking scheme to be
used from the very beginning.  However, that doesn't take care of quite
a few other problems; a lot of code fundamentally assumes that ability
to have children mean having only one chain of ancestors.

And frankly, we already have one pile of garbage of that sort; what's the
benefit of adding another?  About the only difference between those and
xattrs is the ability to open them; is the resulting headache worth the
trouble, seeing that naive programs can't use that shit and somebody
arranging redirects for them can bloody well use pipe and stick around
feeding xattr contents into it...

Re: Streams support in Linux

[Matthew Wilcox]

On Sat, Aug 25, 2018 at 03:47:45PM +0100, Al Viro wrote:
> On Sat, Aug 25, 2018 at 06:51:07AM -0700, Matthew Wilcox wrote:
> 
> > openat()
> > To access a named stream, we need to be able to get a file descriptor for
> > it.  The new openat() syscall seems like the best way to accompish this;
> > specify a file descriptor, a new AT_NAMED_STREAM flag and a filename,
> > and the last component of the filename will be treated as the name of
> > the stream within the object.  This permits us to distinguish between
> > a named stream on a directory and a file within a directory.
> 
> What do you do for access via /proc/*/fd/*, when descriptor is a sodding
> stream?

I don't know.  Is it useful to be able to access them that way?

> > renameat()
> > If olddirfd + oldpath refers to a stream then newdirfd + newpath must
> > refer to a stream within the same parent object.  If that stream exists,
> > it is removed.  If olddirfd + oldpath does not refer to a stream, then
> > newdirfd + newpath must not refer to a stream.
> 
> And here the shit begins - what to do when parents have different dentries?
> Directories can't have hardlinks; regular files very much can.

I presume you mean from a locking point of view?  In my mind, we check that
old and new resolve to the same inode and take a lock on that inode.

> Where do dentries of these things belong, anyway?

Do streams have to have dentries?  They don't have names in the hierarchy.
Can they share their parent's dentry?  Can they have a NULL dentry?  Or
a magic dentry of some kind?

> > unlinkat()
> > This is how you remove an individual named stream
> 
> Yeah... and what happens to access via other hardlinks to the same parent
> file?

I'm not sure I understand the problem here.  You have, say /a/b/c
and /d which refer to the same inode.  This inode has streams x and y.
If you unlinkat() /a/b/c::x then it is no longer openable.  Any currently
existing open fds for x keep x alive until the last one is closed, no
matter which path was used to open them.

> > unlink()
> > Unlinking a file with named streams removes all named streams from that
> > file and then unlinks the file.  Open streams will continue to exist in
> > the filesystem until they are closed, just as unlinked files do.
> 
> Can they be looked up by openat() from such a starting point?

No, you can't use openat() on a stream fd.  I should have been clearer
on that above; this proposal should have split the syscall list in two;
one for how the syscall behaves on a fd-for-a-file-with-streams and one
for how the syscall behaves on a stream-fd.

> The only new part in that proposal is requesting explicit AT_... flag to
> even get to those.  In effect, you are multiplexing a lot of syscalls
> that way, which does, in theory, allow different locking scheme to be
> used from the very beginning.  However, that doesn't take care of quite
> a few other problems; a lot of code fundamentally assumes that ability
> to have children mean having only one chain of ancestors.

Right, but that code also assumes it's dealing with files.  Streams can't
have their own children (either files or streams), which makes the locking
easier as they only have one parent and can't be cross-parent-renamed.

> And frankly, we already have one pile of garbage of that sort; what's the
> benefit of adding another?  About the only difference between those and
> xattrs is the ability to open them; is the resulting headache worth the
> trouble, seeing that naive programs can't use that shit and somebody
> arranging redirects for them can bloody well use pipe and stick around
> feeding xattr contents into it...

Eric says xattrs are limited to 64kB which is too small for fs-verity.

Re: Streams support in Linux

[Al Viro]

On Sat, Aug 25, 2018 at 08:51:50AM -0700, Matthew Wilcox wrote:

> > What do you do for access via /proc/*/fd/*, when descriptor is a sodding
> > stream?
> 
> I don't know.  Is it useful to be able to access them that way?

I don't know - I've no idea who needs that... little gem of elegance in the
first place.

> > > renameat()
> > > If olddirfd + oldpath refers to a stream then newdirfd + newpath must
> > > refer to a stream within the same parent object.  If that stream exists,
> > > it is removed.  If olddirfd + oldpath does not refer to a stream, then
> > > newdirfd + newpath must not refer to a stream.
> > 
> > And here the shit begins - what to do when parents have different dentries?
> > Directories can't have hardlinks; regular files very much can.
> 
> I presume you mean from a locking point of view?  In my mind, we check that
> old and new resolve to the same inode and take a lock on that inode.

Now, if that was all we ever needed...

> > Where do dentries of these things belong, anyway?
> 
> Do streams have to have dentries?  They don't have names in the hierarchy.
> Can they share their parent's dentry?  Can they have a NULL dentry?  Or
> a magic dentry of some kind?

If you want them as opened files - yes, they have to, no, they can't and no,
they can't.

> > > unlinkat()
> > > This is how you remove an individual named stream
> > 
> > Yeah... and what happens to access via other hardlinks to the same parent
> > file?
> 
> I'm not sure I understand the problem here.  You have, say /a/b/c
> and /d which refer to the same inode.  This inode has streams x and y.
> If you unlinkat() /a/b/c::x then it is no longer openable.  Any currently
> existing open fds for x keep x alive until the last one is closed, no
> matter which path was used to open them.

Again, where it the tree would the dentries live?  And I really hope you are
not suggesting to reserve '::' - not when we have files like
/usr/share/man/man3/Params::Check.3perl.gz
just to pull a random line out of locate :: output here...

> > > Unlinking a file with named streams removes all named streams from that
> > > file and then unlinks the file.  Open streams will continue to exist in
> > > the filesystem until they are closed, just as unlinked files do.
> > 
> > Can they be looked up by openat() from such a starting point?
> 
> No, you can't use openat() on a stream fd.  I should have been clearer
> on that above; this proposal should have split the syscall list in two;
> one for how the syscall behaves on a fd-for-a-file-with-streams and one
> for how the syscall behaves on a stream-fd.

I'm not suggesting passing a stream fd as the starting point; just a normal
opened-and-then-unlinked regular file.

> Right, but that code also assumes it's dealing with files.

I'm looking forward to your analysis of all code related to dentry tree,
with an eye towards the places where such asserts are quietly made...

Re: Streams support in Linux

[Matthew Wilcox]

On Sat, Aug 25, 2018 at 07:00:26PM +0100, Al Viro wrote:
> > > Where do dentries of these things belong, anyway?
> > 
> > Do streams have to have dentries?  They don't have names in the hierarchy.
> > Can they share their parent's dentry?  Can they have a NULL dentry?  Or
> > a magic dentry of some kind?
> 
> If you want them as opened files - yes, they have to, no, they can't and no,
> they can't.

I was hoping for something like alloc_file_pseudo() rather than having
real dentries in the dentry tree.

> > > > unlinkat()
> > > > This is how you remove an individual named stream
> > > 
> > > Yeah... and what happens to access via other hardlinks to the same parent
> > > file?
> > 
> > I'm not sure I understand the problem here.  You have, say /a/b/c
> > and /d which refer to the same inode.  This inode has streams x and y.
> > If you unlinkat() /a/b/c::x then it is no longer openable.  Any currently
> > existing open fds for x keep x alive until the last one is closed, no
> > matter which path was used to open them.
> 
> Again, where it the tree would the dentries live?  And I really hope you are
> not suggesting to reserve '::' - not when we have files like
> /usr/share/man/man3/Params::Check.3perl.gz
> just to pull a random line out of locate :: output here...

I was just using '::' as shorthand for this email.  Just to be clear,
there's no intent to have these streams in the filesystem namespace.

> > > > Unlinking a file with named streams removes all named streams from that
> > > > file and then unlinks the file.  Open streams will continue to exist in
> > > > the filesystem until they are closed, just as unlinked files do.
> > > 
> > > Can they be looked up by openat() from such a starting point?
> > 
> > No, you can't use openat() on a stream fd.  I should have been clearer
> > on that above; this proposal should have split the syscall list in two;
> > one for how the syscall behaves on a fd-for-a-file-with-streams and one
> > for how the syscall behaves on a stream-fd.
> 
> I'm not suggesting passing a stream fd as the starting point; just a normal
> opened-and-then-unlinked regular file.

Ah, I see your point.  I think that it makes most sense to define the
semantics of an open-but-unlinked file that you can create new streams,
delete existing streams and rename streams until the last reference to
that file are gone, and all references to streams within a file count
as a reference on that file.  Does that make sense?

> > Right, but that code also assumes it's dealing with files.
> 
> I'm looking forward to your analysis of all code related to dentry tree,
> with an eye towards the places where such asserts are quietly made...

I really was hoping not to change the dentry tree at all.

Re: Streams support in Linux

[Al Viro]

On Sat, Aug 25, 2018 at 01:57:16PM -0700, Matthew Wilcox wrote:

> > > Right, but that code also assumes it's dealing with files.
> > 
> > I'm looking forward to your analysis of all code related to dentry tree,
> > with an eye towards the places where such asserts are quietly made...
> 
> I really was hoping not to change the dentry tree at all.

Umhm...  Just what, pray tell, would be used to hold the stream name when
you are creating/removing/renaming it?  And why are we talking about the
use of existing syscalls for that thing?  In effect, you are multiplexing
entirely new syscalls, with different pathname resolution, etc. upon the
existing ones, using that new AT_... flag to select them.

Better yet, you need some new objects to represent those things, since
you don't want any informative dentries.  And not fs-private ones, at
that, since those new syscalls of yours would have to operate on them
(after all, renaming something opened would probably be expected to
have the opened descriptor to keep accessing the same object, wouldn't
it?)

Re: Streams support in Linux

[Steve French]

On Sat, Aug 25, 2018 at 5:37 PM Al Viro via samba-technical
<samba-technical@lists.samba.org> wrote:
>
> On Sat, Aug 25, 2018 at 01:57:16PM -0700, Matthew Wilcox wrote:
>
> > > > Right, but that code also assumes it's dealing with files.
> > >
> > > I'm looking forward to your analysis of all code related to dentry tree,
> > > with an eye towards the places where such asserts are quietly made...
> >
> > I really was hoping not to change the dentry tree at all.
>
> Umhm...  Just what, pray tell, would be used to hold the stream name when
> you are creating/removing/renaming it?  And why are we talking about the
> use of existing syscalls for that thing?  In effect, you are multiplexing
> entirely new syscalls, with different pathname resolution, etc. upon the
> existing ones, using that new AT_... flag to select them.
>
> Better yet, you need some new objects to represent those things, since
> you don't want any informative dentries.  And not fs-private ones, at
> that, since those new syscalls of yours would have to operate on them
> (after all, renaming something opened would probably be expected to
> have the opened descriptor to keep accessing the same object, wouldn't
> it?)

These are interesting questions, and there are cases where streams
have been shown to have value in Windows, and for Apple (in Macs).
Don't know whether the Solaris equivalent was useful - but presumably
was.

Samba has had mixed feelings about streams (see various emails
from JRA e.g.) due to concerns that we don't want to accidentally
create security holes (and some Windows admins don't like streams
because virus checkers have to check them for viruses, not just
the normal file data - $DATA).   Unfortunately, whether Samba,
in a perfect world, would want streams - the "Content Indexing
Service" (and to some extent Internet Explorer) rely on streams
for some useful information (and perhaps to support Mac clients
really well it is even more important).

In the current implementation of cifs.ko we can enumerate
alternate data streams (via an ioctl) - this would be nice to
convert to a standard interface so NTFS and CIFS/SMB3
etc. could use similar tools to list streams.   It is not really
possible to open alternate data streams with cifs.ko if
posix path mapping is enabled (since a ":" in a filename such
as "filename:streamname" would get remapped (to SFM_COLON ie 0xF022
just as Macs do)  as ":" is a reserved character in CIFS/SMB2/SMB3
but not in POSIX.

If there were a way to open streams that would be a help (if
nothing else for backup applications which have to go through hoops
of turning off posix path name mapping in order to get at stream data
today with cifs.ko)

-- 
Thanks,

Steve

Re: Streams support in Linux

[Jeremy Allison]

On Sat, Aug 25, 2018 at 08:03:03PM -0500, Steve French via samba-technical wrote:
> On Sat, Aug 25, 2018 at 5:37 PM Al Viro via samba-technical
> >
> > Better yet, you need some new objects to represent those things, since
> > you don't want any informative dentries.  And not fs-private ones, at
> > that, since those new syscalls of yours would have to operate on them
> > (after all, renaming something opened would probably be expected to
> > have the opened descriptor to keep accessing the same object, wouldn't
> > it?)
> 
> These are interesting questions, and there are cases where streams
> have been shown to have value in Windows, and for Apple (in Macs).
> Don't know whether the Solaris equivalent was useful - but presumably
> was.

Sorry Steve, can't let this pass :-). Please name *one* case
where streams have value in Windows or Mac. And I'm not talking
about the case for EA's, these clearly have value (plus we already
have them :-).

I'm talking about a case where there is clear value in having
an openable/seekable stream on a file/directory.

I can't think of a *single* case where a stream adds more
utility than an EA used in the same case.

I don't want theoretical "well it would be nice if..",
I want clear "we couldn't have done it any other way"
kinds of things.

Re: Streams support in Linux

[Jeremy Allison]

On Mon, Aug 27, 2018 at 10:05:31AM -0700, Jeremy Allison via samba-technical wrote:
> On Sat, Aug 25, 2018 at 08:03:03PM -0500, Steve French via samba-technical wrote:
> > On Sat, Aug 25, 2018 at 5:37 PM Al Viro via samba-technical
> > >
> > > Better yet, you need some new objects to represent those things, since
> > > you don't want any informative dentries.  And not fs-private ones, at
> > > that, since those new syscalls of yours would have to operate on them
> > > (after all, renaming something opened would probably be expected to
> > > have the opened descriptor to keep accessing the same object, wouldn't
> > > it?)
> > 
> > These are interesting questions, and there are cases where streams
> > have been shown to have value in Windows, and for Apple (in Macs).
> > Don't know whether the Solaris equivalent was useful - but presumably
> > was.
> 
> Sorry Steve, can't let this pass :-). Please name *one* case
> where streams have value in Windows or Mac. And I'm not talking
> about the case for EA's, these clearly have value (plus we already
> have them :-).
> 
> I'm talking about a case where there is clear value in having
> an openable/seekable stream on a file/directory.
> 
> I can't think of a *single* case where a stream adds more
> utility than an EA used in the same case.
> 
> I don't want theoretical "well it would be nice if..",
> I want clear "we couldn't have done it any other way"
> kinds of things.

Actually, to answer my own question, I do know of one
valid application that uses named streams.

The CIA exfiltration tools exposed by WikiLeaks used a
named stream on a top-level share directory to hide data being
stolen from the target (which is why I guess the CIA
doesn't employ NSA-level people, the NSA almost certainly
use the hidden data area behind Windows ACL store instead
as no known scanning tools look at that :-).

So if we really want to enable such things, by all means
add named streams to Linux :-) :-).

Re: Streams support in Linux

[Matthew Wilcox]

On Mon, Aug 27, 2018 at 10:05:31AM -0700, Jeremy Allison wrote:
> I can't think of a *single* case where a stream adds more
> utility than an EA used in the same case.
> 
> I don't want theoretical "well it would be nice if..",
> I want clear "we couldn't have done it any other way"
> kinds of things.

I started this thread with such an example.  The fs-verity patch proposed
wants to store hundreds of megabytes of data associated with a particular
file.  The current solution is to append it to the end of the data then
magic to set i_size lower but not remove the data from the file like a
truncate would.  Then more magic to read that data.

It can't be stored in an xattr; xattrs are limited to 64k in size.
And you have to read them / write them all-in-one-go; you can't read
a little bit of them.  How would you solve the fs-verity problem with
less magic?

Re: Streams support in Linux

[Al Viro]

On Mon, Aug 27, 2018 at 11:21:43AM -0700, Matthew Wilcox wrote:
> On Mon, Aug 27, 2018 at 10:05:31AM -0700, Jeremy Allison wrote:
> > I can't think of a *single* case where a stream adds more
> > utility than an EA used in the same case.
> > 
> > I don't want theoretical "well it would be nice if..",
> > I want clear "we couldn't have done it any other way"
> > kinds of things.
> 
> I started this thread with such an example.  The fs-verity patch proposed
> wants to store hundreds of megabytes of data associated with a particular
> file.  The current solution is to append it to the end of the data then
> magic to set i_size lower but not remove the data from the file like a
> truncate would.  Then more magic to read that data.
> 
> It can't be stored in an xattr; xattrs are limited to 64k in size.
> And you have to read them / write them all-in-one-go; you can't read
> a little bit of them.  How would you solve the fs-verity problem with
> less magic?

* create a subdirectory in root called e.g. .turdis
* prohibit lookups of that after mount has done the initial one and got a dentry
for it
* kernel-side, do O_TEMPFILE open, crap into it and link it in /.turdis/pile#42
* put that 42 into xattrs of the file that has served as, er, nutrient source.

No fs structure changes needed...

Re: Streams support in Linux

[Jeremy Allison]

On Mon, Aug 27, 2018 at 11:21:43AM -0700, Matthew Wilcox wrote:
> On Mon, Aug 27, 2018 at 10:05:31AM -0700, Jeremy Allison wrote:
> > I can't think of a *single* case where a stream adds more
> > utility than an EA used in the same case.
> > 
> > I don't want theoretical "well it would be nice if..",
> > I want clear "we couldn't have done it any other way"
> > kinds of things.
> 
> I started this thread with such an example.  The fs-verity patch proposed
> wants to store hundreds of megabytes of data associated with a particular
> file.  The current solution is to append it to the end of the data then
> magic to set i_size lower but not remove the data from the file like a
> truncate would.  Then more magic to read that data.
> It can't be stored in an xattr; xattrs are limited to 64k in size.
> And you have to read them / write them all-in-one-go; you can't read
> a little bit of them.  How would you solve the fs-verity problem with
> less magic?

OK, fair enough. I concede that is a use-case that would
be easier if Linux had streams.

But as Al just pointed out, this is a case of a poor design.
There are other ways of solving this problem without introducing
a feature that adds *known* security issues and causes headaches for
people attempting to secure systems (scanners for "hidden"
data areas etc.). The greatest users of streams in Windows
are the virus-writing fraternity. Let's not add features
for them please.

But you also said:

> There are, of course, other clients for file streams.  Samba is one,
> GNOME could use streams for various desktoppy things, and I'm certain
> other users would come out of the woodwork if we had them.

This is not a good justification (IMHO). You have one
case where streams would make things easier. Samba might
use them on Linux if there were there, but we'd always
have to keep our non-Linux stream emulation code around
and up to date for our other important system FreeBSD
(I'm considering Solaris dead now, plus we never implemented
Solaris native streams in all the time they were available,
due to their limited availability on all other platforms).

This is a case of adding a nasty feature that encourages bad
security design "just because" it might be useful for user
space that currently works fine without it.

EA's work OK for Gnome. I doubt they want to store
hundreds of megabytes of extra meta-data hidden inside
a file, and on the off chance they do they also have
to emulate this on non-Linux systems anyway (as does
Samba).

Apple gets away with requiring streams in MacOSX as
they don't care about interop with anything other than
themselves and Windows. We're not in that position.

All pain, no gain IMHO :-).

Re: Streams support in Linux

[Theodore Y. Ts'o]

On Mon, Aug 27, 2018 at 11:21:43AM -0700, Matthew Wilcox wrote:
> On Mon, Aug 27, 2018 at 10:05:31AM -0700, Jeremy Allison wrote:
> > I can't think of a *single* case where a stream adds more
> > utility than an EA used in the same case.
> > 
> > I don't want theoretical "well it would be nice if..",
> > I want clear "we couldn't have done it any other way"
> > kinds of things.
> 
> I started this thread with such an example.  The fs-verity patch proposed
> wants to store hundreds of megabytes of data associated with a particular
> file.  The current solution is to append it to the end of the data then
> magic to set i_size lower but not remove the data from the file like a
> truncate would.  Then more magic to read that data.

Sure, but so what?  The Merkle Tree only needs to be read by the
kernel, and we only need to install the fs-verity data once.  And
putting it at the end of the file works just fine.  Theoretically we
could do it other ways, but this is almost certainly easier and
cleaner than if we had to open a streams file from kernel code.

Doing it using streams would actually be *harder* and is only of use
for people who like more structured designs.  But to be clear, this is
*not* something I'm asking for as one of the original designers of
fs-verity.

	     	       	       	 	 - Ted

Re: Streams support in Linux

[Steve French]

On Mon, Aug 27, 2018 at 7:45 PM Theodore Y. Ts'o <tytso@mit.edu> wrote:
>
> On Mon, Aug 27, 2018 at 11:21:43AM -0700, Matthew Wilcox wrote:
> > On Mon, Aug 27, 2018 at 10:05:31AM -0700, Jeremy Allison wrote:
> > > I can't think of a *single* case where a stream adds more
> > > utility than an EA used in the same case.
> > >
> > > I don't want theoretical "well it would be nice if..",
> > > I want clear "we couldn't have done it any other way"
> > > kinds of things.
> >
> > I started this thread with such an example.  The fs-verity patch proposed
> > wants to store hundreds of megabytes of data associated with a particular
> > file.  The current solution is to append it to the end of the data then
> > magic to set i_size lower but not remove the data from the file like a
> > truncate would.  Then more magic to read that data.
>
> Sure, but so what?  The Merkle Tree only needs to be read by the
> kernel, and we only need to install the fs-verity data once.  And
> putting it at the end of the file works just fine.  Theoretically we
> could do it other ways, but this is almost certainly easier and
> cleaner than if we had to open a streams file from kernel code.
>
> Doing it using streams would actually be *harder* and is only of use
> for people who like more structured designs.  But to be clear, this is
> *not* something I'm asking for as one of the original designers of
> fs-verity.

Given that streams need to be read to backup Macs and Windows
(and for a few features of these servers mentioned earlier)
and would be exposed in ntfs (and SMB3 remotely) locally on Linux,
seems useful to me to have some consistent way to open and read
them on Linux even if we don't want to generalize it to other local fs.
The protocol supports it fine (just a file with a reserved character
':' in it) but a little tricky to avoid name conflict with posix ':'
in filenames


-- 
Thanks,

Steve

Re: Streams support in Linux

[Jeremy Allison]

On Mon, Aug 27, 2018 at 08:07:35PM -0500, Steve French wrote:
> On Mon, Aug 27, 2018 at 7:45 PM Theodore Y. Ts'o <tytso@mit.edu> wrote:
> >
> > On Mon, Aug 27, 2018 at 11:21:43AM -0700, Matthew Wilcox wrote:
> > > On Mon, Aug 27, 2018 at 10:05:31AM -0700, Jeremy Allison wrote:
> > > > I can't think of a *single* case where a stream adds more
> > > > utility than an EA used in the same case.
> > > >
> > > > I don't want theoretical "well it would be nice if..",
> > > > I want clear "we couldn't have done it any other way"
> > > > kinds of things.
> > >
> > > I started this thread with such an example.  The fs-verity patch proposed
> > > wants to store hundreds of megabytes of data associated with a particular
> > > file.  The current solution is to append it to the end of the data then
> > > magic to set i_size lower but not remove the data from the file like a
> > > truncate would.  Then more magic to read that data.
> >
> > Sure, but so what?  The Merkle Tree only needs to be read by the
> > kernel, and we only need to install the fs-verity data once.  And
> > putting it at the end of the file works just fine.  Theoretically we
> > could do it other ways, but this is almost certainly easier and
> > cleaner than if we had to open a streams file from kernel code.
> >
> > Doing it using streams would actually be *harder* and is only of use
> > for people who like more structured designs.  But to be clear, this is
> > *not* something I'm asking for as one of the original designers of
> > fs-verity.
> 
> Given that streams need to be read to backup Macs and Windows
> (and for a few features of these servers mentioned earlier)
> and would be exposed in ntfs (and SMB3 remotely) locally on Linux,
> seems useful to me to have some consistent way to open and read
> them on Linux even if we don't want to generalize it to other local fs.
> The protocol supports it fine (just a file with a reserved character
> ':' in it) but a little tricky to avoid name conflict with posix ':'
> in filenames

This sounds like a case for a couple of ioctls. One to enumerate
the streams on an open fd, one to open a given stream name on an
open fd.

Re: Streams support in Linux

[Steve French]

On Tue, Aug 28, 2018 at 1:12 PM Jeremy Allison <jra@samba.org> wrote:
>
> On Mon, Aug 27, 2018 at 08:07:35PM -0500, Steve French wrote:
> >
> > Given that streams need to be read to backup Macs and Windows
> > (and for a few features of these servers mentioned earlier)
> > and would be exposed in ntfs (and SMB3 remotely) locally on Linux,
> > seems useful to me to have some consistent way to open and read
> > them on Linux even if we don't want to generalize it to other local fs.
> > The protocol supports it fine (just a file with a reserved character
> > ':' in it) but a little tricky to avoid name conflict with posix ':'
> > in filenames
>
> This sounds like a case for a couple of ioctls. One to enumerate
> the streams on an open fd, one to open a given stream name on an
> open fd.

We already have a (cifs.ko) ioctl to enumerate streams, but I was
less comfortable with how to structure an ioctl to read/write a
stream (other than $DATA of course).  Ideas on what a "read stream"
ioctl might look like?

-- 
Thanks,

Steve

Re: Streams support in Linux

[Jeremy Allison]

On Tue, Aug 28, 2018 at 01:32:39PM -0500, Steve French wrote:
> On Tue, Aug 28, 2018 at 1:12 PM Jeremy Allison <jra@samba.org> wrote:
> >
> > On Mon, Aug 27, 2018 at 08:07:35PM -0500, Steve French wrote:
> > >
> > > Given that streams need to be read to backup Macs and Windows
> > > (and for a few features of these servers mentioned earlier)
> > > and would be exposed in ntfs (and SMB3 remotely) locally on Linux,
> > > seems useful to me to have some consistent way to open and read
> > > them on Linux even if we don't want to generalize it to other local fs.
> > > The protocol supports it fine (just a file with a reserved character
> > > ':' in it) but a little tricky to avoid name conflict with posix ':'
> > > in filenames
> >
> > This sounds like a case for a couple of ioctls. One to enumerate
> > the streams on an open fd, one to open a given stream name on an
> > open fd.
> 
> We already have a (cifs.ko) ioctl to enumerate streams, but I was
> less comfortable with how to structure an ioctl to read/write a
> stream (other than $DATA of course).  Ideas on what a "read stream"
> ioctl might look like?

You shouldn't need a read stream ioctl. You only need 3 I think.

struct open_stream {
	const char *stream_name;
	int open_flags,
	int stream_fd;
};

struct open_stream os = {
		"MyStreamName",
		O_CREAT,
		-1
};

1). ioctl(file_or_dir_fd, FIO_OPEN_STREAM, &os);

Now read/write the os->stream_fd for the created
stream as desired. Use O_RDONLY|O_WRONLY|O_RDWR
in the flags field as needed.

2). ioctl(file_or_dir_fd, FIO_ENUM_STREAMS_DIR, &new_fd);

Now use readdir() to get the list.

3). ioctl(stream_fd, FIO_DELETE_STREAM, 0);

Delete the stream opened on stream_fd.

Doesn't that cover everything ?

Re: Streams support in Linux

[Steve French]

On Tue, Aug 28, 2018 at 1:41 PM Jeremy Allison <jra@samba.org> wrote:
>
> On Tue, Aug 28, 2018 at 01:32:39PM -0500, Steve French wrote:
> > On Tue, Aug 28, 2018 at 1:12 PM Jeremy Allison <jra@samba.org> wrote:
> > >
> > > On Mon, Aug 27, 2018 at 08:07:35PM -0500, Steve French wrote:
> > > >
> > > > Given that streams need to be read to backup Macs and Windows
> > > > (and for a few features of these servers mentioned earlier)
> > > > and would be exposed in ntfs (and SMB3 remotely) locally on Linux,
> > > > seems useful to me to have some consistent way to open and read
> > > > them on Linux even if we don't want to generalize it to other local fs.
> > > > The protocol supports it fine (just a file with a reserved character
> > > > ':' in it) but a little tricky to avoid name conflict with posix ':'
> > > > in filenames
> > >
> > > This sounds like a case for a couple of ioctls. One to enumerate
> > > the streams on an open fd, one to open a given stream name on an
> > > open fd.
> >
> > We already have a (cifs.ko) ioctl to enumerate streams, but I was
> > less comfortable with how to structure an ioctl to read/write a
> > stream (other than $DATA of course).  Ideas on what a "read stream"
> > ioctl might look like?
>
> You shouldn't need a read stream ioctl. You only need 3 I think.
>
> struct open_stream {
>         const char *stream_name;
>         int open_flags,
>         int stream_fd;
> };
>
> struct open_stream os = {
>                 "MyStreamName",
>                 O_CREAT,
>                 -1
> };
>
> 1). ioctl(file_or_dir_fd, FIO_OPEN_STREAM, &os);
>
> Now read/write the os->stream_fd for the created
> stream as desired. Use O_RDONLY|O_WRONLY|O_RDWR
> in the flags field as needed.
>
> 2). ioctl(file_or_dir_fd, FIO_ENUM_STREAMS_DIR, &new_fd);
>
> Now use readdir() to get the list.
>
> 3). ioctl(stream_fd, FIO_DELETE_STREAM, 0);
>
> Delete the stream opened on stream_fd.
>
> Doesn't that cover everything ?

This would be trickier to code than you might think.

In particular I think the reaadir to list streams is going to be
harder than the ioctl
I had coded for it.   I also had code at one point to convert streams
to xattrs etc.

-- 
Thanks,

Steve

Re: Streams support in Linux

[Jeremy Allison]

On Tue, Aug 28, 2018 at 02:43:12PM -0500, Steve French wrote:
> > > On Tue, Aug 28, 2018 at 1:12 PM Jeremy Allison <jra@samba.org> wrote:
> >
> > You shouldn't need a read stream ioctl. You only need 3 I think.
> >
> > struct open_stream {
> >         const char *stream_name;
> >         int open_flags,
> >         int stream_fd;
> > };
> >
> > struct open_stream os = {
> >                 "MyStreamName",
> >                 O_CREAT,
> >                 -1
> > };
> >
> > 1). ioctl(file_or_dir_fd, FIO_OPEN_STREAM, &os);
> >
> > Now read/write the os->stream_fd for the created
> > stream as desired. Use O_RDONLY|O_WRONLY|O_RDWR
> > in the flags field as needed.
> >
> > 2). ioctl(file_or_dir_fd, FIO_ENUM_STREAMS_DIR, &new_fd);
> >
> > Now use readdir() to get the list.
> >
> > 3). ioctl(stream_fd, FIO_DELETE_STREAM, 0);
> >
> > Delete the stream opened on stream_fd.
> >
> > Doesn't that cover everything ?
> 
> This would be trickier to code than you might think.
> 
> In particular I think the reaadir to list streams is going to be
> harder than the ioctl
> I had coded for it.   I also had code at one point to convert streams
> to xattrs etc.

Why is that ? You control any fd's you return,
so you can vector any readdir call into an
enum_streams request ?

I wouldn't return streams as xattrs, as they
are separately exposed and already have meaning
in SMB1/2/3.

Re: Streams support in Linux

[Steve French]

On Tue, Aug 28, 2018 at 2:47 PM Jeremy Allison <jra@samba.org> wrote:
>
> On Tue, Aug 28, 2018 at 02:43:12PM -0500, Steve French wrote:
> > > > On Tue, Aug 28, 2018 at 1:12 PM Jeremy Allison <jra@samba.org> wrote:
> > >
> > > You shouldn't need a read stream ioctl. You only need 3 I think.
<snip>
> > > 3). ioctl(stream_fd, FIO_DELETE_STREAM, 0);
> > >
> > > Delete the stream opened on stream_fd.
> > >
> > > Doesn't that cover everything ?
> >
> > This would be trickier to code than you might think.
> >
> > In particular I think the reaadir to list streams is going to be
> > harder than the ioctl
> > I had coded for it.   I also had code at one point to convert streams
> > to xattrs etc.
>
> Why is that ? You control any fd's you return,
> so you can vector any readdir call into an
> enum_streams request ?
>
> I wouldn't return streams as xattrs, as they
> are separately exposed and already have meaning
> in SMB1/2/3.

In theory I could construct a struct fd by setting up a struct file on the fly
(see "alloc_empty_file" for example) and the few dozen fields in it,
but it would be a lot easier to return them as xattrs with a reserved
prefix (e.g. "streams")
since most of these are small (some examples) and could have been
stored in xattrs
if the Apple or Windows developers had wanted to

"streams.ZoneIdentifier" (for Internet Explorer download security info)
"streams.favicon"  (for Internet explorer icons)
"streams.AFP_AfpInfo" and "streams.AFP_Resource" (for two common Mac streams)
"streams.XPRESS" (for compression)

-- 
Thanks,

Steve

Re: Streams support in Linux

[Jeremy Allison]

On Tue, Aug 28, 2018 at 03:43:45PM -0500, Steve French wrote:
> 
> In theory I could construct a struct fd by setting up a struct file on the fly
> (see "alloc_empty_file" for example) and the few dozen fields in it,
> but it would be a lot easier to return them as xattrs with a reserved
> prefix (e.g. "streams")
> since most of these are small (some examples) and could have been
> stored in xattrs
> if the Apple or Windows developers had wanted to
> 
> "streams.ZoneIdentifier" (for Internet Explorer download security info)
> "streams.favicon"  (for Internet explorer icons)
> "streams.AFP_AfpInfo" and "streams.AFP_Resource" (for two common Mac streams)
> "streams.XPRESS" (for compression)

Hmmm. Yeah, I guess this works if you can ensure
the prefix is reserved.

Re: Streams support in Linux

[Steve French]

On Tue, Aug 28, 2018 at 3:47 PM Jeremy Allison <jra@samba.org> wrote:
>
> On Tue, Aug 28, 2018 at 03:43:45PM -0500, Steve French wrote:
> >
> > In theory I could construct a struct fd by setting up a struct file on the fly
> > (see "alloc_empty_file" for example) and the few dozen fields in it,
> > but it would be a lot easier to return them as xattrs with a reserved
> > prefix (e.g. "streams")
> > since most of these are small (some examples) and could have been
> > stored in xattrs
> > if the Apple or Windows developers had wanted to
> >
> > "streams.ZoneIdentifier" (for Internet Explorer download security info)
> > "streams.favicon"  (for Internet explorer icons)
> > "streams.AFP_AfpInfo" and "streams.AFP_Resource" (for two common Mac streams)
> > "streams.XPRESS" (for compression)
>
> Hmmm. Yeah, I guess this works if you can ensure
> the prefix is reserved.

And it gives Ronnie another chance to show off the performance benefits of
his good work on compounding (open/read/close in this case)


-- 
Thanks,

Steve

Re: Streams support in Linux

[Stefan Metzmacher]

[-- Attachment #1.1: Type: text/plain, Size: 1123 bytes --]

Am 28.08.2018 um 22:47 schrieb Jeremy Allison via samba-technical:
> On Tue, Aug 28, 2018 at 03:43:45PM -0500, Steve French wrote:
>>
>> In theory I could construct a struct fd by setting up a struct file on the fly
>> (see "alloc_empty_file" for example) and the few dozen fields in it,
>> but it would be a lot easier to return them as xattrs with a reserved
>> prefix (e.g. "streams")
>> since most of these are small (some examples) and could have been
>> stored in xattrs
>> if the Apple or Windows developers had wanted to
>>
>> "streams.ZoneIdentifier" (for Internet Explorer download security info)
>> "streams.favicon"  (for Internet explorer icons)
>> "streams.AFP_AfpInfo" and "streams.AFP_Resource" (for two common Mac streams)
>> "streams.XPRESS" (for compression)
> 
> Hmmm. Yeah, I guess this works if you can ensure
> the prefix is reserved.

Remember some of the AFP stream can get very large, Ralph should have
more details about it.

Also from reading [MS-FSA] I have the impression that opens on stream
would also support oplocks/leases as well as byte range locks.

metze



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Streams support in Linux

[Jeremy Allison]

On Tue, Aug 28, 2018 at 11:19:09PM +0200, Stefan Metzmacher wrote:
> Am 28.08.2018 um 22:47 schrieb Jeremy Allison via samba-technical:
> > On Tue, Aug 28, 2018 at 03:43:45PM -0500, Steve French wrote:
> >>
> >> In theory I could construct a struct fd by setting up a struct file on the fly
> >> (see "alloc_empty_file" for example) and the few dozen fields in it,
> >> but it would be a lot easier to return them as xattrs with a reserved
> >> prefix (e.g. "streams")
> >> since most of these are small (some examples) and could have been
> >> stored in xattrs
> >> if the Apple or Windows developers had wanted to
> >>
> >> "streams.ZoneIdentifier" (for Internet Explorer download security info)
> >> "streams.favicon"  (for Internet explorer icons)
> >> "streams.AFP_AfpInfo" and "streams.AFP_Resource" (for two common Mac streams)
> >> "streams.XPRESS" (for compression)
> > 
> > Hmmm. Yeah, I guess this works if you can ensure
> > the prefix is reserved.
> 
> Remember some of the AFP stream can get very large, Ralph should have
> more details about it.

Steve is talking about the enumeration step, not the open/read
method. I still think my ioctl approach works best there.

> Also from reading [MS-FSA] I have the impression that opens on stream
> would also support oplocks/leases as well as byte range locks.

Yep, but we have no way to express that in POSIX anyway
(so I guess the strategy is always ask for the best you
can get).

Re: Streams support in Linux

[Steve French]

On Tue, Aug 28, 2018 at 4:19 PM Stefan Metzmacher <metze@samba.org> wrote:
>
> Am 28.08.2018 um 22:47 schrieb Jeremy Allison via samba-technical:
> > On Tue, Aug 28, 2018 at 03:43:45PM -0500, Steve French wrote:
> >>
> >> In theory I could construct a struct fd by setting up a struct file on the fly
> >> (see "alloc_empty_file" for example) and the few dozen fields in it,
> >> but it would be a lot easier to return them as xattrs with a reserved
> >> prefix (e.g. "streams")
> >> since most of these are small (some examples) and could have been
> >> stored in xattrs
> >> if the Apple or Windows developers had wanted to
> >>
> >> "streams.ZoneIdentifier" (for Internet Explorer download security info)
> >> "streams.favicon"  (for Internet explorer icons)
> >> "streams.AFP_AfpInfo" and "streams.AFP_Resource" (for two common Mac streams)
> >> "streams.XPRESS" (for compression)
> >
> > Hmmm. Yeah, I guess this works if you can ensure
> > the prefix is reserved.
>
> Remember some of the AFP stream can get very large, Ralph should have
> more details about it.
>
> Also from reading [MS-FSA] I have the impression that opens on stream
> would also support oplocks/leases as well as byte range locks.

Larger than 8MB (maximum i/o size for read)?


-- 
Thanks,

Steve

Re: Streams support in Linux

[Ralph Böhme]

On Tue, Aug 28, 2018 at 04:23:24PM -0500, Steve French wrote:
>On Tue, Aug 28, 2018 at 4:19 PM Stefan Metzmacher <metze@samba.org> wrote:
>> Remember some of the AFP stream can get very large, Ralph should have
>> more details about it.
>>
>> Also from reading [MS-FSA] I have the impression that opens on stream
>> would also support oplocks/leases as well as byte range locks.
>
>Larger than 8MB (maximum i/o size for read)?

yes.

-slow

-- 
Ralph Boehme, Samba Team       https://samba.org/
Samba Developer, SerNet GmbH   https://sernet.de/en/samba/
GPG Key Fingerprint:           FAE2 C608 8A24 2520 51C5
                               59E4 AA1E 9B71 2639 9E46

RE: Streams support in Linux

[Tom Talpey]

> -----Original Message-----
> From: samba-technical <samba-technical-bounces@lists.samba.org> On Behalf
> Of Jeremy Allison via samba-technical
> Sent: Tuesday, August 28, 2018 2:13 PM
> To: Steve French <smfrench@gmail.com>
> Cc: Theodore Tso <tytso@mit.edu>; samba-technical <samba-
> technical@lists.samba.org>; Matthew Wilcox <willy@infradead.org>;
> ebiggers@kernel.org; Al Viro <viro@zeniv.linux.org.uk>; linux-fsdevel <linux-
> fsdevel@vger.kernel.org>
> Subject: Re: Streams support in Linux
> 
> On Mon, Aug 27, 2018 at 08:07:35PM -0500, Steve French wrote:
> > On Mon, Aug 27, 2018 at 7:45 PM Theodore Y. Ts'o <tytso@mit.edu> wrote:
> > >
> > > On Mon, Aug 27, 2018 at 11:21:43AM -0700, Matthew Wilcox wrote:
> > > > On Mon, Aug 27, 2018 at 10:05:31AM -0700, Jeremy Allison wrote:
> > > > > I can't think of a *single* case where a stream adds more
> > > > > utility than an EA used in the same case.
> > > > >
> > > > > I don't want theoretical "well it would be nice if..",
> > > > > I want clear "we couldn't have done it any other way"
> > > > > kinds of things.
> > > >
> > > > I started this thread with such an example.  The fs-verity patch proposed
> > > > wants to store hundreds of megabytes of data associated with a particular
> > > > file.  The current solution is to append it to the end of the data then
> > > > magic to set i_size lower but not remove the data from the file like a
> > > > truncate would.  Then more magic to read that data.
> > >
> > > Sure, but so what?  The Merkle Tree only needs to be read by the
> > > kernel, and we only need to install the fs-verity data once.  And
> > > putting it at the end of the file works just fine.  Theoretically we
> > > could do it other ways, but this is almost certainly easier and
> > > cleaner than if we had to open a streams file from kernel code.
> > >
> > > Doing it using streams would actually be *harder* and is only of use
> > > for people who like more structured designs.  But to be clear, this is
> > > *not* something I'm asking for as one of the original designers of
> > > fs-verity.
> >
> > Given that streams need to be read to backup Macs and Windows
> > (and for a few features of these servers mentioned earlier)
> > and would be exposed in ntfs (and SMB3 remotely) locally on Linux,
> > seems useful to me to have some consistent way to open and read
> > them on Linux even if we don't want to generalize it to other local fs.
> > The protocol supports it fine (just a file with a reserved character
> > ':' in it) but a little tricky to avoid name conflict with posix ':'
> > in filenames
> 
> This sounds like a case for a couple of ioctls. One to enumerate
> the streams on an open fd, one to open a given stream name on an
> open fd.

Wait, you're saying that Macs and Windows clients need to start issuing these new magical ioctl's to Samba/Linux servers? Not a solution, IMO.

Tom.

RE: Streams support in Linux

[Aurélien Aptel]

Tom Talpey <ttalpey@microsoft.com> writes:
> Wait, you're saying that Macs and Windows clients need to start issuing these new magical ioctl's to Samba/Linux servers? Not a solution, IMO.

I think what Jeremy is saying is the other way around: linux programs
issuing ioctls that cifs.ko will translate to what SMB uses to handle
stream on the wire.

-- 
Aurélien Aptel / SUSE Labs Samba Team
GPG: 1839 CB5F 9F5B FB9B AA97  8C99 03C8 A49B 521B D5D3
SUSE Linux GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)

RE: Streams support in Linux

[Tom Talpey]

> -----Original Message-----
> From: Aurélien Aptel <aaptel@suse.com>
> Sent: Wednesday, August 29, 2018 9:55 AM
> To: Tom Talpey <ttalpey@microsoft.com>; Jeremy Allison <jra@samba.org>;
> Steve French <smfrench@gmail.com>
> Cc: Theodore Tso <tytso@mit.edu>; Matthew Wilcox <willy@infradead.org>;
> ebiggers@kernel.org; Al Viro <viro@zeniv.linux.org.uk>; linux-fsdevel <linux-
> fsdevel@vger.kernel.org>
> Subject: RE: Streams support in Linux
> 
> Tom Talpey <ttalpey@microsoft.com> writes:
> > Wait, you're saying that Macs and Windows clients need to start issuing these
> new magical ioctl's to Samba/Linux servers? Not a solution, IMO.
> 
> I think what Jeremy is saying is the other way around: linux programs
> issuing ioctls that cifs.ko will translate to what SMB uses to handle
> stream on the wire.

Well ok, but the same question might apply to local applications,
or local libraries. Will there be a liblinuxsmb3streams.so, just to
implement these?

Tom.

Re: Streams support in Linux

[Jeremy Allison]

On Wed, Aug 29, 2018 at 03:02:01PM +0000, Tom Talpey wrote:
> > -----Original Message-----
> > From: Aur�lien Aptel <aaptel@suse.com>
> > Sent: Wednesday, August 29, 2018 9:55 AM
> > To: Tom Talpey <ttalpey@microsoft.com>; Jeremy Allison <jra@samba.org>;
> > Steve French <smfrench@gmail.com>
> > Cc: Theodore Tso <tytso@mit.edu>; Matthew Wilcox <willy@infradead.org>;
> > ebiggers@kernel.org; Al Viro <viro@zeniv.linux.org.uk>; linux-fsdevel <linux-
> > fsdevel@vger.kernel.org>
> > Subject: RE: Streams support in Linux
> > 
> > Tom Talpey <ttalpey@microsoft.com> writes:
> > > Wait, you're saying that Macs and Windows clients need to start issuing these
> > new magical ioctl's to Samba/Linux servers? Not a solution, IMO.
> > 
> > I think what Jeremy is saying is the other way around: linux programs
> > issuing ioctls that cifs.ko will translate to what SMB uses to handle
> > stream on the wire.
> 
> Well ok, but the same question might apply to local applications,
> or local libraries. Will there be a liblinuxsmb3streams.so, just to
> implement these?

Something like that. Remember, streams are not a part of POSIX
(and IMHO nor should they be :-).

Re: Streams support in Linux

[Jeremy Allison]

On Wed, Aug 29, 2018 at 01:46:30PM +0000, Tom Talpey wrote:
> > 
> > This sounds like a case for a couple of ioctls. One to enumerate
> > the streams on an open fd, one to open a given stream name on an
> > open fd.
> 
> Wait, you're saying that Macs and Windows clients need to start issuing these new magical ioctl's to Samba/Linux servers? Not a solution, IMO.

Nope, that's not an option obviously. The ioctls only exist
on the Linux userspace client side. Inside cifsfs they get
converted to standard SMB1/2/3 calls.

I'm trying to create a way for Linux userspace to get
access to these Windows/Mac/Samba features without having
to completely change the VFS model to introduce the
horrible Solaris semantics of opening a file as a
directory to get access to internal streams.

Re: Streams support in Linux

[Andreas Dilger]

[-- Attachment #1: Type: text/plain, Size: 2236 bytes --]

On Aug 29, 2018, at 9:59 AM, Jeremy Allison <jra@samba.org> wrote:
> 
> On Wed, Aug 29, 2018 at 01:46:30PM +0000, Tom Talpey wrote:
>>> 
>>> This sounds like a case for a couple of ioctls. One to enumerate
>>> the streams on an open fd, one to open a given stream name on an
>>> open fd.
>> 
>> Wait, you're saying that Macs and Windows clients need to start issuing these new magical ioctl's to Samba/Linux servers? Not a solution, IMO.
> 
> Nope, that's not an option obviously. The ioctls only exist
> on the Linux userspace client side. Inside cifsfs they get
> converted to standard SMB1/2/3 calls.
> 
> I'm trying to create a way for Linux userspace to get
> access to these Windows/Mac/Samba features without having
> to completely change the VFS model to introduce the
> horrible Solaris semantics of opening a file as a
> directory to get access to internal streams.

Note that ext4 recently added support for xattrs larger than the
previous one-block (usually 4KB) total size.  The large xattrs were
implemented as a separate inode attached to the file/directory to
reference the multiple data blocks allocated to the xattr, which
simplifies e2fsck and ext4 block allocation/freeing/traversal.

There is no real limit on the size of such xattr inodes, except as
currently imposed by the xattr userspace APIs.  If there was an API
(e.g. openat() or an ioctl() to start) that allowed opening the
named xattrs, normal read/write should be useable to modify them.

I don't think I'd go as far as Matthew to allow link, rename, etc.
of these attributes, as that introduces unnecessary complexity for
no clear use case or benefit.  That doesn't preclude future usage
in this manner, but I don't think it makes sense to add that until
there is a clear use for this.

For the original FS Verity case, using xattrs attached to the inode
is IMHO far preferable to using a separate hidden directory and/or
special file to store the verity information, as it will properly
follow the lifetime of the original file.  Rename will keep the
verity data, and unlink will delete the verity data, without the
need for fsck to keep them consistent or do garbage collection.

Cheers, Andreas






[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 873 bytes --]

Re: Streams support in Linux

[Matthew Wilcox]

On Sat, Aug 25, 2018 at 11:36:15PM +0100, Al Viro wrote:
> On Sat, Aug 25, 2018 at 01:57:16PM -0700, Matthew Wilcox wrote:
> 
> > > > Right, but that code also assumes it's dealing with files.
> > > 
> > > I'm looking forward to your analysis of all code related to dentry tree,
> > > with an eye towards the places where such asserts are quietly made...
> > 
> > I really was hoping not to change the dentry tree at all.
> 
> Umhm...  Just what, pray tell, would be used to hold the stream name when
> you are creating/removing/renaming it?  And why are we talking about the
> use of existing syscalls for that thing?  In effect, you are multiplexing
> entirely new syscalls, with different pathname resolution, etc. upon the
> existing ones, using that new AT_... flag to select them.
> 
> Better yet, you need some new objects to represent those things, since
> you don't want any informative dentries.  And not fs-private ones, at
> that, since those new syscalls of yours would have to operate on them
> (after all, renaming something opened would probably be expected to
> have the opened descriptor to keep accessing the same object, wouldn't
> it?)

Let's look at it this way ... suppose we have an fcntl F_SETSTREAM.
And our API looks like:

	fd = open("/a/b/c", O_RDWR);
	fcntl(fd, F_SETSTREAM, "verity");
	read(fd, ...);

>From a VFS point of view, we just change file->f_inode to point to a
different inode.  We probably also have to change file->f_op.  Do we
also need to change file->f_path.dentry?

Re: Streams support in Linux

[Theodore Y. Ts'o]

On Sat, Aug 25, 2018 at 06:51:07AM -0700, Matthew Wilcox wrote:
> 
> Let's go over the properties of a file stream:
> 
>  - It has no life independent of the file it's attached to; you can't move
>    it from one file to another
>  - If the file is deleted, it is also deleted
>  - If the file is renamed, it travels with the file
>  - If the file is copied, the copying program decides whether any named
>    streams are copied along with it.
>  - Can be created, deleted.  Can be renamed?
>  - Openable, seekable, cachable
>  - Does not have sub-streams of its own
>  - Directories may also have streams which are distinct from the files
>    in the directory
>  - Can pipes / sockets / device nodes / symlinks / ... have streams?  Unclear.
>    Probably not useful.

Let's *not* make the mistakes Solaris did, and don't allow an fchdirat()
into a streams directory.  Let's also not allow executing
rootkits^H^H^H^H^H^H^H^H binaries as a stream.   :-)

			 	       - Ted

Re: Streams support in Linux

[Jeremy Allison]

On Sat, Aug 25, 2018 at 12:25:47PM -0400, Theodore Y. Ts'o via samba-technical wrote:
> On Sat, Aug 25, 2018 at 06:51:07AM -0700, Matthew Wilcox wrote:
> > 
> > Let's go over the properties of a file stream:
> > 
> >  - It has no life independent of the file it's attached to; you can't move
> >    it from one file to another
> >  - If the file is deleted, it is also deleted
> >  - If the file is renamed, it travels with the file
> >  - If the file is copied, the copying program decides whether any named
> >    streams are copied along with it.
> >  - Can be created, deleted.  Can be renamed?
> >  - Openable, seekable, cachable
> >  - Does not have sub-streams of its own
> >  - Directories may also have streams which are distinct from the files
> >    in the directory
> >  - Can pipes / sockets / device nodes / symlinks / ... have streams?  Unclear.
> >    Probably not useful.
> 
> Let's *not* make the mistakes Solaris did, and don't allow an fchdirat()
> into a streams directory.  Let's also not allow executing
> rootkits^H^H^H^H^H^H^H^H binaries as a stream.   :-)

After long and bitter experience with them, I have
come to the conclusion (aided greatly by Ted :-) that
streams are a *FUNDAMENTAL* mistake in filesystem
APIs and we should not have them implemented in
Linux.

I don't write the code, so don't get to chose
of course, but if I had to chose one essential
feature between file streams and RichACLs (which
are already supported for NFSv4) I would chose
RichACLs every time.

For anyone who is claiming that streams are essential
for Windows or Mac application compatibility I'd advise
them to take a look at the initial implementation of
Microsoft ReFS (which didn't support streams, might
do now) and Microsoft Azure SMB file server, which
*still* doesn't support named streams (although I'm
sure they're working on it):

https://docs.microsoft.com/en-us/rest/api/storageservices/features-not-supported-by-the-azure-file-service

Now tell me, if streams are so essential, how are
Microsoft getting *anyone* to migrate to Azure SMB
file service ?

Please let's not make the same mistake Windows NTFS
did here.

Jeremy.

Re: Streams support in Linux

[Shahbaz Youssefi]

> Let's go over the properties of a file stream:
>
>  - It has no life independent of the file it's attached to; you can't move
>    it from one file to another
>  - If the file is deleted, it is also deleted
>  - If the file is renamed, it travels with the file
>  - If the file is copied, the copying program decides whether any named
>    streams are copied along with it.
>  - Can be created, deleted.  Can be renamed?
>  - Openable, seekable, cachable
>  - Does not have sub-streams of its own
>  - Directories may also have streams which are distinct from the files
>    in the directory
>  - Can pipes / sockets / device nodes / symlinks / ... have streams?  Unclear.
>    Probably not useful.

This certainly sounds useful! And it's called tar.

With fs-verity as well, I don't see why they have to put the tree and
the data in the same file, when they can just bundle them in a
tarball.