bindfs over NFS shows the underlying file system

bindfs over NFS shows the underlying file system

[Alkis Georgopoulos]

I'm not sure if this is an NFS issue, or a bindfs issue, or if I'm not 
using the appropriate NFS options.

I export my /home via NFS with:

     /home *(rw,async,crossmnt,no_subtree_check,no_root_squash,insecure)

Inside my /home I'm providing a shared folder with a bindfs mount:

     bindfs -u 1000 --create-for-user=1000 -g 100 --create-for-group=100 
-p 770,af-x /home/share /home/share

I.e. this just sets fixed permissions for anything under /home/share.

And finally I mount /home on some NFS client (or on localhost):

     mount -t nfs server:/home /home

The problem is that /home/share on the client doesn't show the bindfs 
permissions, but it shows the underlying file system of the server's 
/home/share.
The crossmnt NFS option follows submounts with other file systems, but 
not with bindfs.

On the other hand, if the bindfs source is on a different file system 
than the bindfs target directory, everything works fine (i.e. bindfs 
/other/filesystem/share /home/share).

Is there any way to configure either NFS or bindfs, so that this works 
when I only have one partition, i.e. when the share is on the same file 
system as /home?

If anyone answers, please Cc me as I'm not in the list.

Thank you very much,
Alkis Georgopoulos
LTSP developer

Re: bindfs over NFS shows the underlying file system

[J. Bruce Fields]

On Fri, Oct 11, 2019 at 08:24:14AM +0300, Alkis Georgopoulos wrote:
> I'm not sure if this is an NFS issue, or a bindfs issue, or if I'm
> not using the appropriate NFS options.
> 
> I export my /home via NFS with:
> 
>     /home *(rw,async,crossmnt,no_subtree_check,no_root_squash,insecure)
> 
> Inside my /home I'm providing a shared folder with a bindfs mount:
> 
>     bindfs -u 1000 --create-for-user=1000 -g 100
> --create-for-group=100 -p 770,af-x /home/share /home/share
> 
> I.e. this just sets fixed permissions for anything under /home/share.
> 
> And finally I mount /home on some NFS client (or on localhost):
> 
>     mount -t nfs server:/home /home
> 
> The problem is that /home/share on the client doesn't show the
> bindfs permissions, but it shows the underlying file system of the
> server's /home/share.
> The crossmnt NFS option follows submounts with other file systems,
> but not with bindfs.
> 
> On the other hand, if the bindfs source is on a different file
> system than the bindfs target directory, everything works fine (i.e.
> bindfs /other/filesystem/share /home/share).

Huh.  I wonder if nfsd is for some reason determining the existence of a
mountpoint by comparing some kind of filesystem id and not seeing a
change.  Looking at the code to remind myself how this works....

nfsd_mountpoint() is using d_mountpoint() and follow_down(), which
should be right.  Then it's making an upcall to mountd.  That's handled
by nfs-utils/mountd/cache.c:nfsd_export().

The is_mountpoint() check there is indeed going to return false in your
case because it's just comparing inode and device numbers....  But I
think that case is only for the "mountpoint" export option.

So I think all that matters is that export_matches() does the right
thing, and it certainly looks like it does--it should succeed as long as
there's a parent directory that's exported with crossmnt.

There's some debugging you could try by looking at
net/sunrpc/nfsd.*/content or using strace to watch rpc.mountd's reads
and writes of net/sunrpc/nfsd.*/channel.

What version of nfs-utils are you on?

--b.

> 
> Is there any way to configure either NFS or bindfs, so that this
> works when I only have one partition, i.e. when the share is on the
> same file system as /home?
> 
> If anyone answers, please Cc me as I'm not in the list.
> 
> Thank you very much,
> Alkis Georgopoulos
> LTSP developer

Re: bindfs over NFS shows the underlying file system

[Alkis Georgopoulos]

Thank you very much for the feedback,

I'm testing with Ubuntu 18.04, nfs-common 1:1.3.4-2.1ubuntu5.2.
I think this means "nfs utils 1.3.4".

I tried explicitly listing all submounts in exports, and specifying an 
fsid everywhere, and that worked, for example:

exports:
/home 
*(fsid=4858dab5b4ac16ad2b7d274698c2532a,rw,async,crossmnt,no_subtree_check,no_root_squash,insecure)
/home/share 
*(fsid=8c1748909cac2548372caead5bab9aa5,rw,async,no_subtree_check,no_root_squash,insecure)

...and the clients only mount /home, and then properly see the bindfs 
share permissions.

But I'd really like to avoid that as in the real scenario there are many 
submounts which are frequently added/removed, not just /home/share.

I'll try to follow your advice for debugging information.



On 10/11/19 7:47 PM, J. Bruce Fields wrote:
> On Fri, Oct 11, 2019 at 08:24:14AM +0300, Alkis Georgopoulos wrote:
>> I'm not sure if this is an NFS issue, or a bindfs issue, or if I'm
>> not using the appropriate NFS options.
>>
>> I export my /home via NFS with:
>>
>>      /home *(rw,async,crossmnt,no_subtree_check,no_root_squash,insecure)
>>
>> Inside my /home I'm providing a shared folder with a bindfs mount:
>>
>>      bindfs -u 1000 --create-for-user=1000 -g 100
>> --create-for-group=100 -p 770,af-x /home/share /home/share
>>
>> I.e. this just sets fixed permissions for anything under /home/share.
>>
>> And finally I mount /home on some NFS client (or on localhost):
>>
>>      mount -t nfs server:/home /home
>>
>> The problem is that /home/share on the client doesn't show the
>> bindfs permissions, but it shows the underlying file system of the
>> server's /home/share.
>> The crossmnt NFS option follows submounts with other file systems,
>> but not with bindfs.
>>
>> On the other hand, if the bindfs source is on a different file
>> system than the bindfs target directory, everything works fine (i.e.
>> bindfs /other/filesystem/share /home/share).
> 
> Huh.  I wonder if nfsd is for some reason determining the existence of a
> mountpoint by comparing some kind of filesystem id and not seeing a
> change.  Looking at the code to remind myself how this works....
> 
> nfsd_mountpoint() is using d_mountpoint() and follow_down(), which
> should be right.  Then it's making an upcall to mountd.  That's handled
> by nfs-utils/mountd/cache.c:nfsd_export().
> 
> The is_mountpoint() check there is indeed going to return false in your
> case because it's just comparing inode and device numbers....  But I
> think that case is only for the "mountpoint" export option.
> 
> So I think all that matters is that export_matches() does the right
> thing, and it certainly looks like it does--it should succeed as long as
> there's a parent directory that's exported with crossmnt.
> 
> There's some debugging you could try by looking at
> net/sunrpc/nfsd.*/content or using strace to watch rpc.mountd's reads
> and writes of net/sunrpc/nfsd.*/channel.
> 
> What version of nfs-utils are you on?
> 
> --b.
> 
>>
>> Is there any way to configure either NFS or bindfs, so that this
>> works when I only have one partition, i.e. when the share is on the
>> same file system as /home?
>>
>> If anyone answers, please Cc me as I'm not in the list.
>>
>> Thank you very much,
>> Alkis Georgopoulos
>> LTSP developer