NFSv4 alternate data streams?

NFSv4 alternate data streams?

[Martin Wege]

Hello,

does the Linux NFSv4 server has support for alternate data streams?
Solaris surely has, but we want to replace it. As our Windows
applications (DB) rely on alternate data streams the question is
whether the Linux NFSv4 server can fully replace the Solaris NFSv4
server in that respect.

Thanks,
Martin

Re: NFSv4 alternate data streams?

[Jeff Layton]

On Thu, 2023-11-30 at 04:59 +0100, Martin Wege wrote:
> Hello,
> 
> does the Linux NFSv4 server has support for alternate data streams?
> Solaris surely has, but we want to replace it. As our Windows
> applications (DB) rely on alternate data streams the question is
> whether the Linux NFSv4 server can fully replace the Solaris NFSv4
> server in that respect.

No. These are not supported on Linux and (likely) never will be.
Alternate data streams are just the sort of misfeature that virus
writers love.
-- 
Jeff Layton <jlayton@kernel.org>

Re: NFSv4 alternate data streams?

[Chuck Lever III]

> On Nov 29, 2023, at 10:59 PM, Martin Wege <martin.l.wege@gmail.com> wrote:
> 
> Hello,
> 
> does the Linux NFSv4 server has support for alternate data streams?
> Solaris surely has, but we want to replace it. As our Windows
> applications (DB) rely on alternate data streams the question is
> whether the Linux NFSv4 server can fully replace the Solaris NFSv4
> server in that respect.

Hi Martin -

Linux NFSD does not support alternate data streams because none of
the underlying file systems on Linux implement them. Very much like
the HIDDEN and ARCHIVE attributes.

I believe Solaris and their storage appliance are the only
implementations of NFS that do support them, since they have
implemented streams in ZFS.

Instead, Linux NFSD implements extended attributes (that's what
our native file systems and user space support). I realize that
the semantics of those are not the same as stream support.

--
Chuck Lever

Re: NFSv4 alternate data streams?

[Martin Wege]

On Thu, Nov 30, 2023 at 3:03 PM Chuck Lever III <chuck.lever@oracle.com> wrote:
>
>
>
> > On Nov 29, 2023, at 10:59 PM, Martin Wege <martin.l.wege@gmail.com> wrote:
> >
> > Hello,
> >
> > does the Linux NFSv4 server has support for alternate data streams?
> > Solaris surely has, but we want to replace it. As our Windows
> > applications (DB) rely on alternate data streams the question is
> > whether the Linux NFSv4 server can fully replace the Solaris NFSv4
> > server in that respect.
>
> Hi Martin -
>
> Linux NFSD does not support alternate data streams because none of
> the underlying file systems on Linux implement them. Very much like
> the HIDDEN and ARCHIVE attributes.
>
> I believe Solaris and their storage appliance are the only
> implementations of NFS that do support them, since they have
> implemented streams in ZFS.
>
> Instead, Linux NFSD implements extended attributes (that's what
> our native file systems and user space support). I realize that
> the semantics of those are not the same as stream support.

SMB server on Linux supports Alternate Data Streams - why can't the
same be done for NFSv4? Otherwise we're stuck with using SMB

Thanks,
Martin

Re: NFSv4 alternate data streams?

[Chuck Lever III]

> On Dec 18, 2023, at 3:33 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
> 
> On Thu, Nov 30, 2023 at 3:03 PM Chuck Lever III <chuck.lever@oracle.com> wrote:
>> 
>>> On Nov 29, 2023, at 10:59 PM, Martin Wege <martin.l.wege@gmail.com> wrote:
>>> 
>>> Hello,
>>> 
>>> does the Linux NFSv4 server has support for alternate data streams?
>>> Solaris surely has, but we want to replace it. As our Windows
>>> applications (DB) rely on alternate data streams the question is
>>> whether the Linux NFSv4 server can fully replace the Solaris NFSv4
>>> server in that respect.
>> 
>> Hi Martin -
>> 
>> Linux NFSD does not support alternate data streams because none of
>> the underlying file systems on Linux implement them. Very much like
>> the HIDDEN and ARCHIVE attributes.
>> 
>> I believe Solaris and their storage appliance are the only
>> implementations of NFS that do support them, since they have
>> implemented streams in ZFS.
>> 
>> Instead, Linux NFSD implements extended attributes (that's what
>> our native file systems and user space support). I realize that
>> the semantics of those are not the same as stream support.
> 
> SMB server on Linux supports Alternate Data Streams -

I was not aware of that support. I need more information about
how that is done.


> why can't the same be done for NFSv4?

I mean, yes the standard NFSv4 protocol provides a way to access
these, and NFSD can be made to support that. But where would it
store that content?

NFSD can support what is readily available from the VFS API. If
alternate streams were to become a premier part of the Linux
file system stack, it would be straightforward for NFSD to
support them.

IOW first NFSD needs the communities responsible for the VFS and
file systems to implement them. Everyone has to agree on how
these are stored, we can't just make something up. Otherwise
there is no hope for interoperation between local applications
and applications that access these via SMB or NFS.


--
Chuck Lever

Re: NFSv4 alternate data streams?

[Martin Wege]

On Mon, Dec 18, 2023 at 3:49 PM Chuck Lever III <chuck.lever@oracle.com> wrote:
>
>
> > On Dec 18, 2023, at 3:33 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
> >
> > On Thu, Nov 30, 2023 at 3:03 PM Chuck Lever III <chuck.lever@oracle.com> wrote:
> >>
> >>> On Nov 29, 2023, at 10:59 PM, Martin Wege <martin.l.wege@gmail.com> wrote:
> >>>
> >>> Hello,
> >>>
> >>> does the Linux NFSv4 server has support for alternate data streams?
> >>> Solaris surely has, but we want to replace it. As our Windows
> >>> applications (DB) rely on alternate data streams the question is
> >>> whether the Linux NFSv4 server can fully replace the Solaris NFSv4
> >>> server in that respect.
> >>
> >> Hi Martin -
> >>
> >> Linux NFSD does not support alternate data streams because none of
> >> the underlying file systems on Linux implement them. Very much like
> >> the HIDDEN and ARCHIVE attributes.
> >>
> >> I believe Solaris and their storage appliance are the only
> >> implementations of NFS that do support them, since they have
> >> implemented streams in ZFS.
> >>
> >> Instead, Linux NFSD implements extended attributes (that's what
> >> our native file systems and user space support). I realize that
> >> the semantics of those are not the same as stream support.
> >
> > SMB server on Linux supports Alternate Data Streams -
>
> I was not aware of that support. I need more information about
> how that is done.
>
>
> > why can't the same be done for NFSv4?
>
> I mean, yes the standard NFSv4 protocol provides a way to access
> these, and NFSD can be made to support that. But where would it
> store that content?
>
> NFSD can support what is readily available from the VFS API. If
> alternate streams were to become a premier part of the Linux
> file system stack, it would be straightforward for NFSD to
> support them.
>
> IOW first NFSD needs the communities responsible for the VFS and
> file systems to implement them. Everyone has to agree on how
> these are stored, we can't just make something up. Otherwise
> there is no hope for interoperation between local applications
> and applications that access these via SMB or NFS.

Yeah, that's a good one(TM). Seriously? You try to pull on a 'John
Reiser' on me?
for those who do not remember, or are too young. Once upon a time
there was ReiserFS-NG, which had support for that, and the VFS people
dragged him and his project through the mud for religious reasons.

The 'religion' here being that Alternate Data Streams are from
WIndows, and therefore this is BAD(TM), EVIL(TM), and SATAN(TM).Even
if someone would come up with a serious, technical sound patch they
would just bicker at the patches so long and so often that they just
rot. Death by a thousand cuts (or nasty review comments). Remember,
for faith people will do anything, just look up the "dark ages" in
Europe.
And just the tip of the iceberg, I bet someone will deliver the
argument that John Reiser murdered his wife, children and family dog,
and that's why Alternate Data Streams will never be part of VFS.

So back to the topic: SAMBA people just support ADS by sticking a :
between filename and stream name on the SAMBA server side
("filename:adsname"), and are happy with that.
Why can't NFSv4 do the same?

Thanks,
Martin

Re: NFSv4 alternate data streams?

[Chuck Lever]

On Thu, Dec 21, 2023 at 01:26:31PM +0100, Martin Wege wrote:
> On Mon, Dec 18, 2023 at 3:49 PM Chuck Lever III <chuck.lever@oracle.com> wrote:
> >
> >
> > > On Dec 18, 2023, at 3:33 AM, Martin Wege <martin.l.wege@gmail.com> wrote:
> > >
> > > On Thu, Nov 30, 2023 at 3:03 PM Chuck Lever III <chuck.lever@oracle.com> wrote:
> > >>
> > >>> On Nov 29, 2023, at 10:59 PM, Martin Wege <martin.l.wege@gmail.com> wrote:
> > >>>
> > >>> Hello,
> > >>>
> > >>> does the Linux NFSv4 server has support for alternate data streams?
> > >>> Solaris surely has, but we want to replace it. As our Windows
> > >>> applications (DB) rely on alternate data streams the question is
> > >>> whether the Linux NFSv4 server can fully replace the Solaris NFSv4
> > >>> server in that respect.
> > >>
> > >> Hi Martin -
> > >>
> > >> Linux NFSD does not support alternate data streams because none of
> > >> the underlying file systems on Linux implement them. Very much like
> > >> the HIDDEN and ARCHIVE attributes.
> > >>
> > >> I believe Solaris and their storage appliance are the only
> > >> implementations of NFS that do support them, since they have
> > >> implemented streams in ZFS.
> > >>
> > >> Instead, Linux NFSD implements extended attributes (that's what
> > >> our native file systems and user space support). I realize that
> > >> the semantics of those are not the same as stream support.
> > >
> > > SMB server on Linux supports Alternate Data Streams -
> >
> > I was not aware of that support. I need more information about
> > how that is done.
> >
> >
> > > why can't the same be done for NFSv4?
> >
> > I mean, yes the standard NFSv4 protocol provides a way to access
> > these, and NFSD can be made to support that. But where would it
> > store that content?
> >
> > NFSD can support what is readily available from the VFS API. If
> > alternate streams were to become a premier part of the Linux
> > file system stack, it would be straightforward for NFSD to
> > support them.
> >
> > IOW first NFSD needs the communities responsible for the VFS and
> > file systems to implement them. Everyone has to agree on how
> > these are stored, we can't just make something up. Otherwise
> > there is no hope for interoperation between local applications
> > and applications that access these via SMB or NFS.
> 
> Yeah, that's a good one(TM). Seriously? You try to pull on a 'John
> Reiser' on me?
> for those who do not remember, or are too young. Once upon a time
> there was ReiserFS-NG, which had support for that, and the VFS people
> dragged him and his project through the mud for religious reasons.
> 
> The 'religion' here being that Alternate Data Streams are from
> WIndows, and therefore this is BAD(TM), EVIL(TM), and SATAN(TM).Even
> if someone would come up with a serious, technical sound patch they
> would just bicker at the patches so long and so often that they just
> rot. Death by a thousand cuts (or nasty review comments). Remember,
> for faith people will do anything, just look up the "dark ages" in
> Europe.
> And just the tip of the iceberg, I bet someone will deliver the
> argument that John Reiser murdered his wife, children and family dog,
> and that's why Alternate Data Streams will never be part of VFS.
> 
> So back to the topic: SAMBA people just support ADS by sticking a :
> between filename and stream name on the SAMBA server side
> ("filename:adsname"), and are happy with that.
> Why can't NFSv4 do the same?

IIRC ":" is the SMB filename separator.

NFS can't use ":" to denote an alternate stream because ":" is a
character that is legal and valid to use in POSIX (and NFS)
filenames. Therefore...

... the NFSv4 protocol /does/ support alternate data streams as
a separate protocol element. They are referred to in the standard
as "named attributes". See:

https://www.rfc-editor.org/rfc/rfc8881#name-named-attributes


Named attributes are an optional feature of the protocol. That means
that a standards-compliant NFS server implementation is not required
to implement them, and there are mechanisms in the protocol for NFS
clients to detect when that support is present or absent on a
server.

Linux file systems do not implement named attributes. Rather, they
implement extended attributes, which are different enough that they
cannot substitute. For instance, you can use read(2), write(2), and
lseek(2) on a named attribute. Extended attributes have to be read
or written in a single operation, and are thus limited in total
size, and xattr I/O always starts at byte position zero.

There's also no VFS plumbing for named attributes in Linux, and
no user space API (for applications to use with the NFS client).

There are two file system implementations on Linux that can store
named attributes: NTFS and ZFS (which is out of tree). But the
practical matter is that the Linux NFS implementation cannot support
NFSv4 named attributes until the Linux VFS does.

I'm personally agnostic about named versus extended attributes.
There's no good-vs-evil here. If the VFS were to grow support, I
don't have a problem with implementing NFSv4 named attributes in
NFSD.


-- 
Chuck Lever