Re: Linux 5.11 Kernel: NFS re-export errors with older nfs-utils package versions

[Benjamin Maynard <benmaynard@google.com>]

Thanks for the response. Comments inline below.

On Tue, 19 Jan 2021 at 18:02, J. Bruce Fields <bfields@fieldses.org> wrote:
>
> On Mon, Jan 18, 2021 at 05:57:54PM +0000, Benjamin Maynard wrote:
> > Hi,
> >
> > I was recently experimenting with NFS re-exporting using the new patch
> > set that is in the Linux 5.11 kernel
> > (https://patchwork.kernel.org/project/linux-nfs/list/?series=393561).
> >
> > After applying these patches, I consistently faced an error when
> > trying to perform a previously working NFS re-export: "exportfs:
> > /files does not support NFS export".
> >
> > I (with help from some other interested parties) began troubleshooting
> > and after stepping through each patch individually we identified that
> > the error only occurred when the following patch was applied:
> > https://patchwork.kernel.org/project/linux-nfs/patch/20201130220319.501064-3-trond.myklebust@hammerspace.com/.
>
> That link isn't working for me for some reason.
>
> Looks like we're talking about ba5e8187c555 "nfsd: allow filesystems to
> opt out of subtree checking".
>

Correct, this is the patch I am referencing.

> > This patch prevents re-exporting if subtree checking is enabled on the
> > originating NFS server.
>
> That's not correct.
>
> I'm assuming there are two servers: a reexporting server, which mounts
> the originating NFS server, which is mounting ext4 or xfs or some other
> local filesystem.
>
> It's hard for the reexporting server to even tell if the originating
> server is using subtree checking, so I'm surprised that would make a
> difference in behavior.
>

That is correct, there is an originating NFS Server (Ubuntu 20.04 -
5.4.0-1034-gcp) that is exporting a directory from the local ext4
filesystem. This is exported with the following options:

/files 10.0.0.0/8(rw,no_subtree_check,fsid=10)

This is then mounted from the re-exporting server (export from /proc/mounts):

10.70.1.2:/files /files nfs
rw,sync,noatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,acregmin=600,acregmax=600,acdirmin=600,acdirmax=600,hard,nocto,proto=tcp,nconnect=16,timeo=600,retrans=2,sec=sys,mountaddr=10.70.1.2,mountvers=3,mountport=20048,mountproto=udp,fsc,local_lock=none,addr=10.70.1.2
0 0

We then attempt to re-export the mounted directory from the
re-exporting server with the following entry in /etc/exports:

/files   10.67.0.0/16(rw,wdelay,no_root_squash,no_subtree_check,fsid=10,sec=sys,rw,secure,no_root_squash,no_all_squash)

If you perform this set of steps with the 5.10 kernel with nfs-utils
1.3.4 (Ubuntu & Debian default version), the re-export will work. If
you perform the same set of steps with the ba5e8187c555 patch applied
(still on nfs-utils 1.3.4) then the re-export will fail with the error
message "exportfs: /files does not support NFS export". dmesg further
reveals the cause "check_export: nfs does not support subtree
checking!".

This message appears even though we have no_subtree_check set on both
the exports of the originating NFS server, and the re-export server.

If you then upgrade nfs-utils to 2.5.2 on the re-export server, the
re-export works as expected.


> > The strange thing was that no_subtree_check
> > export option was already set on the export from the originating NFS
> > Filer, but the error message persisted.
>
> So, this patch is only checks whether you've got no_subtree_check set on
> the reexporting server.
>

Thanks. I misunderstood and thought it checked the exports on the
originating server. Regardless as detailed above no_subtree_check is
also set on the re-export server and the error persists when using an
older nfs-utils version.

> > After lots of troubleshooting, eventually we tried updating NFS Utils
> > from 1.3.4 to 2.5.2 and we were able to successfully perform
> > re-export. It appears that the old version of the nfs-utils package
> > was the cause of the issue.
>
> I'm a little confused about what happened here.  Which server were you
> applying the above patches on?  Which server did you upgrade NFS utils
> on?
>

The re-exporting server in both cases. The originating server is using
an older, unmodified kernel and version of nfs-utils. See above for a
more detailed overview.


> Could be that you're actually running into some filehandle size limit or
> something.  (Subtree checking can make that problem worse.)  Hard to
> tell.
>
> --b.
>
> > I appreciate that 1.3.4 is a very old version of nfs-utils, but it is
> > the default version that ships with Ubuntu and Debian and the error
> > message does not immediately point to the outdated version being the
> > cause of the problem.
> >
> > I was wondering if it was possible to detail the requirement for a
> > more recent version of nfs-utils in the NFS Re-exporting section of
> > the Wiki (http://wiki.linux-nfs.org/wiki/index.php/NFS_re-export) to
> > help others who may encounter this problem in the future?