linux-block.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jens Axboe <axboe@kernel.dk>
To: Sagi Grimberg <sagi@grimberg.me>,
	"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>
Subject: Re: soft lockup with io_uring
Date: Wed, 21 Aug 2019 18:55:46 -0600	[thread overview]
Message-ID: <0ac352c8-851c-9976-118b-afb5839d6746@kernel.dk> (raw)
In-Reply-To: <2f87ee3f-d61f-e572-08f5-96a8ef8843b0@grimberg.me>

On 8/21/19 4:48 PM, Sagi Grimberg wrote:
> Hey,
> 
> Just ran io-uring-bench on my VM to /dev/nullb0 and got the following
> soft lockup [1], the reproducer is as simple as:
> 
> modprobe null_blk
> tools/io_uring/io_uring-bench /dev/nullb0
> 
> It looks like io_iopoll_getevents() can hog the cpu, however I don't
> yet really know what is preventing it from quickly exceeding min and
> punting back...
> 
> Adding this makes the problem go away:
> --
> diff --git a/fs/io_uring.c b/fs/io_uring.c
> index 8b9dbf3b2298..aba03eee5c81 100644
> --- a/fs/io_uring.c
> +++ b/fs/io_uring.c
> @@ -779,6 +779,7 @@ static int io_iopoll_getevents(struct io_ring_ctx
> *ctx, unsigned int *nr_events,
>                           return ret;
>                   if (!min || *nr_events >= min)
>                           return 0;
> +               cond_resched();
>           }
> 
>           return 1;
> --
> 
> But I do not know if this is the correct way to fix this, or what
> exactly is the issue, but thought I send it out given its so
> easy to reproduce.

I wonder what your .config is, can you attach it?

Also, please try my for-linus branch, it's got a few tweaks for how
we handle polling (and when we back off). Doesn't affect the inner
loop, so might not change anything for you.

If not, might be better to have a need_resched() terminator in there,
like we have in the outer loop.

-- 
Jens Axboe


  reply	other threads:[~2019-08-22  0:55 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-21 22:48 soft lockup with io_uring Sagi Grimberg
2019-08-22  0:55 ` Jens Axboe [this message]
     [not found]   ` <e14439c4-b59a-2aa0-6cf1-1ef54e70b14b@grimberg.me>
2019-08-22  1:33     ` Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0ac352c8-851c-9976-118b-afb5839d6746@kernel.dk \
    --to=axboe@kernel.dk \
    --cc=linux-block@vger.kernel.org \
    --cc=sagi@grimberg.me \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).