Re: [PATCH v3 6/8] filemap: Allow __filemap_get_folio to allocate large folios

[Dave Chinner <david@fromorbit.com>]

On Tue, Jun 13, 2023 at 01:42:51AM +0100, Matthew Wilcox wrote:
> On Tue, Jun 13, 2023 at 08:49:05AM +1000, Dave Chinner wrote:
> > On Mon, Jun 12, 2023 at 09:39:08PM +0100, Matthew Wilcox (Oracle) wrote:
> > > Allow callers of __filemap_get_folio() to specify a preferred folio
> > > order in the FGP flags.  This is only honoured in the FGP_CREATE path;
> > > if there is already a folio in the page cache that covers the index,
> > > we will return it, no matter what its order is.  No create-around is
> > > attempted; we will only create folios which start at the specified index.
> > > Unmodified callers will continue to allocate order 0 folios.
> > .....
> > > -		/* Init accessed so avoid atomic mark_page_accessed later */
> > > -		if (fgp_flags & FGP_ACCESSED)
> > > -			__folio_set_referenced(folio);
> > > +		if (!mapping_large_folio_support(mapping))
> > > +			order = 0;
> > > +		if (order > MAX_PAGECACHE_ORDER)
> > > +			order = MAX_PAGECACHE_ORDER;
> > > +		/* If we're not aligned, allocate a smaller folio */
> > > +		if (index & ((1UL << order) - 1))
> > > +			order = __ffs(index);
> > 
> > If I read this right, if we pass in an unaligned index, we won't get
> > the size of the folio we ask for?
> 
> Right.  That's implied by (but perhaps not obvious from) the changelog.
> Folios are always naturally aligned in the file, so an order-4 folio
> has to start at a multiple of 16.  If the index you pass in is not
> a multiple of 16, we can't create an order-4 folio without starting
> at an earlier index.
> 
> For a 4kB block size filesystem, that's what we want.  Applications
> _generally_ don't write backwards, so creating an order-4 folio is just
> wasting memory.
> 
> > e.g. if we want an order-4 folio (64kB) because we have a 64kB block
> > size in the filesystem, then we have to pass in an index that
> > order-4 aligned, yes?
> > 
> > I ask this, because the later iomap code that asks for large folios
> > only passes in "pos >> PAGE_SHIFT" so it looks to me like it won't
> > allocate large folios for anything other than large folio aligned
> > writes, even if we need them.
> > 
> > What am I missing?
> 
> Perhaps what you're missing is that this isn't trying to solve the
> problem of supporting a bs > ps filesystem?

No, that's not what I'm asking about. I know there's other changes
needed to enforce minimum folio size/alignment for bs > ps.

What I'm asking about is when someone does a 16kB write at offset
12kB, they won't get a large folio allocated at all, right? Even
though the write is large enough to enable it?

Indeed, if we do a 1MB write at offset 4KB, we'll get 4kB at 4KB, 8KB
and 12kB (because we can't do order-1 folios), then order-2 at 16KB,
order-3 at 32kB, and so on until we hit offset 1MB where we will do
an order-0 folio allocation again (because the remaining length is
4KB). The next 1MB write will then follow the same pattern, right?

I think this ends up being sub-optimal and fairly non-obvious
non-obvious behaviour from the iomap side of the fence which is
clearly asking for high-order folios to be allocated. i.e. a small
amount of allocate-around to naturally align large folios when the
page cache is otherwise empty would make a big difference to the
efficiency of non-large-folio-aligned sequential writes...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com