stable.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jens Axboe <axboe@kernel.dk>
To: Song Liu <song@kernel.org>, Thomas Deutschmann <whissi@whissi.de>
Cc: Christoph Hellwig <hch@lst.de>,
	Vishal Verma <vverma@digitalocean.com>,
	Thorsten Leemhuis <regressions@leemhuis.info>,
	"stable@vger.kernel.org" <stable@vger.kernel.org>,
	"regressions@lists.linux.dev" <regressions@lists.linux.dev>
Subject: Re: [REGRESSION] v5.17-rc1+: FIFREEZE ioctl system call hangs
Date: Thu, 25 Aug 2022 13:12:05 -0600	[thread overview]
Message-ID: <3114ed77-3802-8ca4-52e4-0e4b917d88b2@kernel.dk> (raw)
In-Reply-To: <CAPhsuW6OcFeyVO-=12opRDijXHpQh1drQZZ6nLAq9P5TzvpnMA@mail.gmail.com>

On 8/25/22 10:47 AM, Song Liu wrote:
> On Tue, Aug 23, 2022 at 10:13 AM Song Liu <song@kernel.org> wrote:
>>
>> On Mon, Aug 22, 2022 at 8:15 PM Thomas Deutschmann <whissi@whissi.de> wrote:
>>>
>>> On 2022-08-23 03:37, Song Liu wrote:
>>>> Thomas, have you tried to bisect with the fio repro?
>>>
>>> Yes, just finished:
>>>
>>>> d32d3d0b47f7e34560ae3c55ddfcf68694813501 is the first bad commit
>>>> commit d32d3d0b47f7e34560ae3c55ddfcf68694813501
>>>> Author: Christoph Hellwig
>>>> Date:   Mon Jun 14 13:17:34 2021 +0200
>>>>
>>>>     nvme-multipath: set QUEUE_FLAG_NOWAIT
>>>>
>>>>     The nvme multipathing code just dispatches bios to one of the blk-mq
>>>>     based paths and never blocks on its own, so set QUEUE_FLAG_NOWAIT
>>>>     to support REQ_NOWAIT bios.
>>>
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d32d3d0b47f7e34560ae3c55ddfcf68694813501
>>>
>>>
>>> So another NOWAIT issue -- similar to the bad commit which is causing
>>> the mdraid issue I already found
>>> (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0f9650bd838efe5c52f7e5f40c3204ad59f1964d).
>>>
>>> Reverting the commit, i.e. deleting
>>>
>>>    blk_queue_flag_set(QUEUE_FLAG_NOWAIT, head->disk->queue);
>>>
>>> fixes the problem for me. Well, sort of. Looks like this will disable
>>> io_uring. fio reproducer fails with
>>
>> My system doesn't have multipath enabled. I guess bisect will point to something
>> else here.
>>
>> I am afraid we won't get more information from bisect.
> 
> OK, I am able to pinpoint the issue, and Jens found the proper fix for
> it (see below,
> also available in [1]). It survived 100 runs of the repro fio job.
> 
> Thomas, please give it a try.
> 
> Thanks,
> Song
> 
> diff --git c/fs/io_uring.c w/fs/io_uring.c
> index 3f8a79a4affa..72a39f5ec5a5 100644
> --- c/fs/io_uring.c
> +++ w/fs/io_uring.c
> @@ -4551,7 +4551,12 @@ static int io_write(struct io_kiocb *req,
> unsigned int issue_flags)
>  copy_iov:
>                 iov_iter_restore(&s->iter, &s->iter_state);
>                 ret = io_setup_async_rw(req, iovec, s, false);
> -               return ret ?: -EAGAIN;
> +               if (!ret) {
> +                       if (kiocb->ki_flags & IOCB_WRITE)
> +                               kiocb_end_write(req);
> +                       return -EAGAIN;
> +               }
> +               return 0;

This should be 'return ret;' for that last line. I had to double check
the ones I did, but they did get it right. But I did a double take when
I saw this one :-)

It'll work fine for testing as we won't hit errors here unless we run
out of memory, so...

-- 
Jens Axboe

  reply	other threads:[~2022-08-25 19:12 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-08-03 14:35 [REGRESSION] v5.17-rc1+: FIFREEZE ioctl system call hangs Thomas Deutschmann
2022-08-11 12:34 ` Thomas Deutschmann
2022-08-15 10:58   ` Thorsten Leemhuis
2022-08-15 15:46     ` Vishal Verma
2022-08-17  6:19       ` Song Liu
2022-08-17  6:53         ` Thomas Deutschmann
2022-08-17 18:29           ` Thomas Deutschmann
2022-08-19  2:46             ` Thomas Deutschmann
2022-08-20  1:04               ` Song Liu
2022-08-22 15:29                 ` Thomas Deutschmann
2022-08-22 16:30                   ` Thomas Deutschmann
2022-08-22 21:52                     ` Song Liu
2022-08-22 22:44                       ` Thomas Deutschmann
2022-08-22 22:59                         ` Song Liu
2022-08-23  1:37                           ` Song Liu
2022-08-23  3:15                             ` Thomas Deutschmann
2022-08-23 17:13                               ` Song Liu
2022-08-25 16:47                                 ` Song Liu
2022-08-25 19:12                                   ` Jens Axboe [this message]
2022-08-25 22:24                                     ` Song Liu
2022-08-26 20:10                                       ` Thomas Deutschmann
2022-09-08 13:25     ` [REGRESSION] v5.17-rc1+: FIFREEZE ioctl system call hangs #forregzbot Thorsten Leemhuis

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3114ed77-3802-8ca4-52e4-0e4b917d88b2@kernel.dk \
    --to=axboe@kernel.dk \
    --cc=hch@lst.de \
    --cc=regressions@leemhuis.info \
    --cc=regressions@lists.linux.dev \
    --cc=song@kernel.org \
    --cc=stable@vger.kernel.org \
    --cc=vverma@digitalocean.com \
    --cc=whissi@whissi.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).