Re: [PATCH RFC v3 3/3] iomap: bound ioend size to 4096 pages

[Brian Foster <bfoster@redhat.com>]

On Wed, May 19, 2021 at 02:28:39PM +0100, Christoph Hellwig wrote:
> On Mon, May 17, 2021 at 01:17:22PM -0400, Brian Foster wrote:
> > The iomap writeback infrastructure is currently able to construct
> > extremely large bio chains (tens of GBs) associated with a single
> > ioend. This consolidation provides no significant value as bio
> > chains increase beyond a reasonable minimum size. On the other hand,
> > this does hold significant numbers of pages in the writeback
> > state across an unnecessarily large number of bios because the ioend
> > is not processed for completion until the final bio in the chain
> > completes. Cap an individual ioend to a reasonable size of 4096
> > pages (16MB with 4k pages) to avoid this condition.
> 
> Note that once we get huge page/folio support in the page cache this
> sucks as we can trivially handle much larger sizes with very little
> iteration.
> 
> I wonder if both this limit and the previous one should be based on the
> number of pages added instead.  And in fact maybe if we only want the
> limit at add to ioend time and skip the defer to workqueue part entirely.
> 

Both limits are already based on pages. I imagine they could change to
folios when appropriate.

The defer to workqueue part was based on your suggestion[1]. The primary
purpose of this series is to address the completion processing soft
lockup warning, so I don't have a strong preference on whether we do
that by capping ioend size, processing (and yielding) from non-atomic
context, or both.

Brian

[1] https://lore.kernel.org/linux-fsdevel/20200917080455.GY26262@infradead.org/