linux-xfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <djwong@kernel.org>
To: Matthew Wilcox <willy@infradead.org>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>,
	Brian Foster <bfoster@redhat.com>,
	linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org
Subject: Re: [PATCH v2 2/2] xfs: kick extra large ioends to completion workqueue
Date: Thu, 6 May 2021 12:34:06 -0700	[thread overview]
Message-ID: <20210506193406.GE8582@magnolia> (raw)
In-Reply-To: <20201006140720.GQ20115@casper.infradead.org>

On Tue, Oct 06, 2020 at 03:07:20PM +0100, Matthew Wilcox wrote:
> On Mon, Oct 05, 2020 at 08:55:37PM -0700, Darrick J. Wong wrote:
> > On Mon, Oct 05, 2020 at 11:21:02AM -0400, Brian Foster wrote:
> > > We've had reports of soft lockup warnings in the iomap ioend
> > > completion path due to very large bios and/or bio chains. Divert any
> > > ioends with 256k or more pages to process to the workqueue so
> > > completion occurs in non-atomic context and can reschedule to avoid
> > > soft lockup warnings.
> > 
> > Hmmmm... is there any way we can just make end_page_writeback faster?
> 
> There are ways to make it faster.  I don't know if they're a "just"
> solution ...
> 
> 1. We can use THPs.  That will reduce the number of pages being operated
> on.  I hear somebody might have a patch set for that.  Incidentally,
> this patch set will clash with the THP patchset, so one of us is going to
> have to rebase on the other's work.  Not a complaint, just acknowledging
> that some coordination will be needed for the 5.11 merge window.

How far off is this, anyway?  I assume it's in line behind the folio
series?

> 2. We could create end_writeback_pages(struct pagevec *pvec) which
> calls a new test_clear_writeback_pages(pvec).  That could amortise
> taking the memcg lock and finding the lruvec and taking the mapping
> lock -- assuming these pages are sufficiently virtually contiguous.
> It can definitely amortise all the statistics updates.

/me kinda wonders if THPs arent the better solution for people who want
to run large ios.

> 3. We can make wake_up_page(page, PG_writeback); more efficient.  If
> you can produce this situation on demand, I had a patch for that which
> languished due to lack of interest.

I can (well, someone can) so I'll talk to you internally about their
seeekret reproducer.

> https://lore.kernel.org/linux-fsdevel/20200416220130.13343-1-willy@infradead.org/

--D

  reply	other threads:[~2021-05-06 19:34 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-02 15:33 [PATCH 0/2] iomap: avoid soft lockup warnings on large ioends Brian Foster
2020-10-02 15:33 ` [PATCH 1/2] iomap: resched ioend completion when in non-atomic context Brian Foster
2020-10-02 15:33 ` [PATCH 2/2] xfs: kick extra large ioends to completion workqueue Brian Foster
2020-10-02 16:19   ` Darrick J. Wong
2020-10-02 16:38     ` Brian Foster
2020-10-03  0:26   ` kernel test robot
2020-10-05 15:21   ` [PATCH v2 " Brian Foster
2020-10-06  3:55     ` Darrick J. Wong
2020-10-06 12:44       ` Brian Foster
2021-05-06 19:31         ` Darrick J. Wong
2021-05-07 14:06           ` Brian Foster
2021-05-07 14:40             ` Matthew Wilcox
2021-05-10  2:45               ` Dave Chinner
2020-10-06 14:07       ` Matthew Wilcox
2021-05-06 19:34         ` Darrick J. Wong [this message]
2021-05-06 19:45           ` Matthew Wilcox

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210506193406.GE8582@magnolia \
    --to=djwong@kernel.org \
    --cc=bfoster@redhat.com \
    --cc=darrick.wong@oracle.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-xfs@vger.kernel.org \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).