Re: [LSF/MM/BPF TOPIC] Cloud storage optimizations

[Matthew Wilcox <willy@infradead.org>]

On Sat, Mar 04, 2023 at 08:15:50PM -0800, Luis Chamberlain wrote:
> On Sat, Mar 04, 2023 at 04:39:02PM +0000, Matthew Wilcox wrote:
> > I'm getting more and more
> > comfortable with the idea that "Linux doesn't support block sizes >
> > PAGE_SIZE on 32-bit machines" is an acceptable answer.
> 
> First of all filesystems would need to add support for a larger block
> sizes > PAGE_SIZE, and that takes effort. It is also a support question
> too.
> 
> I think garnering consensus from filesystem developers we don't want
> to support block sizes > PAGE_SIZE on 32-bit systems would be a good
> thing to review at LSFMM or even on this list. I hightly doubt anyone
> is interested in that support.

Agreed.

> > XFS already works with arbitrary-order folios. 
> 
> But block sizes > PAGE_SIZE is work which is still not merged. It
> *can* be with time. That would allow one to muck with larger block
> sizes than 4k on x86-64 for instance. Without this, you can't play
> ball.

Do you mean that XFS is checking that fs block size <= PAGE_SIZE and
that check needs to be dropped?  If so, I don't see where that happens.

Or do you mean that the blockdev "filesystem" needs to be enhanced to
support large folios?  That's going to be kind of a pain because it
uses buffer_heads.  And ext4 depends on it using buffer_heads.  So,
yup, more work needed than I remembered (but as I said, it's FS side,
not block layer or driver work).

Or were you referring to the NVMe PAGE_SIZE sanity check that Keith
mentioned upthread?

> > The only needed piece is
> > specifying to the VFS that there's a minimum order for this particular
> > inode, and having the VFS honour that everywhere.
> 
> Other than the above too, don't we still also need to figure out what
> fs APIs would incur larger order folios? And then what about corner cases
> with the page cache?
> 
> I was hoping some of these nooks and crannies could be explored with tmpfs.

I think we're exploring all those with XFS.  Or at least, many of
them.  A lot of the folio conversion patches you see flowing past
are pure efficiency gains -- no need to convert between pages and
folios implicitly; do the explicit conversions and save instructions.
Most of the correctness issues were found & fixed a long time ago when
PMD support was added to tmpfs.  One notable exception would be the
writeback path since tmpfs doesn't writeback, it has that special thing
it does with swap.

tmpfs is a rather special case as far as its use of the filesystem APIs
go, but I suspect I've done most of the needed work to have it work with
arbitrary order folios instead of just PTE and PMD sizes.  There's
probably some left-over assumptions that I didn't find yet.  Maybe in
the swap path, for example ;-)