Re: [PATCH/RFC] NFSD: handle BTRFS subvolumes better.

From: "NeilBrown" <neilb@suse.de>
To: "Josef Bacik" <josef@toxicpanda.com>
Cc: "J. Bruce Fields" <bfields@fieldses.org>,
	"Chuck Lever" <chuck.lever@oracle.com>,
	"Chris Mason" <clm@fb.com>, "David Sterba" <dsterba@suse.com>,
	linux-nfs@vger.kernel.org, "Wang Yugui" <wangyugui@e16-tech.com>,
	"Ulli Horlacher" <framstag@rus.uni-stuttgart.de>,
	linux-btrfs@vger.kernel.org
Subject: Re: [PATCH/RFC] NFSD: handle BTRFS subvolumes better.
Date: Fri, 16 Jul 2021 09:02:16 +1000	[thread overview]
Message-ID: <162639013675.13764.11555673325105489888@noble.neil.brown.name> (raw)
In-Reply-To: <edd94b15-90df-c540-b9aa-8eac89b6713b@toxicpanda.com>

On Fri, 16 Jul 2021, Josef Bacik wrote:
> 
> I'm going to restate what I think the problem is you're having just so I'm sure 
> we're on the same page.
> 
> 1. We export a btrfs volume via nfsd that has multiple subvolumes.
> 2. We run find, and when we stat a file, nfsd doesn't send along our bogus 
> st_dev, it sends it's own thing (I assume?).  This confuses du/find because you 
> get the same inode number with different parents.
> 
> Is this correct?  If that's the case then it' be relatively straightforward to 
> add another callback into export_operations to grab this fsid right?  Hell we 
> could simply return the objectid of the root since that's unique across the 
> entire file system.  We already do our magic FH encoding to make sure we keep 
> all this straight for NFS, another callback to give that info isn't going to 
> kill us.  Thanks,

Fairly close.
As well as the fsid I need a "mounted-on" inode number, so one callback
to provide both would do.
If zero was reported, that would be equivalent to not providing the
callback.
- Is "u64" always enough for the subvol-id?
- Should we make these details available to user-space with a new STATX
  flag?
- Should it be a new export_operations callback, or new fields in
  "struct kstat" ??

... though having asked those question, I begin to wonder if I took a
wrong turn.
I can already get some fsid information form statfs, though it is only
64bits and for BTRFS is combines the filesystem uuid and the subvol
id.  For that reason I avoided it.

But I'm already caching the fsid for the export-point.  If, when I find
a different fsid lower down, I xor the result with the export-point
fsid, the result would be fairly clean (the xor difference between the
two subvol ids) and could be safely mixed into the fsid we currently
report.

So all I REALLY need from btrfs is a "mounted-on" inode number, matching
what readdir() reports.
I wouldn't argue AGAINST getting cleaner fsid information.  A 128-bit
uuid and a 64bit subvol id would be ideal.
I'd rather see them as new STATX flags than a new export_operations
callback.

NeilBrown