From: Jens Axboe <axboe@kernel.dk>
To: Song Liu <song@kernel.org>, Thomas Deutschmann <whissi@whissi.de>
Cc: Christoph Hellwig <hch@lst.de>,
Vishal Verma <vverma@digitalocean.com>,
Thorsten Leemhuis <regressions@leemhuis.info>,
"stable@vger.kernel.org" <stable@vger.kernel.org>,
"regressions@lists.linux.dev" <regressions@lists.linux.dev>
Subject: Re: [REGRESSION] v5.17-rc1+: FIFREEZE ioctl system call hangs
Date: Thu, 25 Aug 2022 13:12:05 -0600 [thread overview]
Message-ID: <3114ed77-3802-8ca4-52e4-0e4b917d88b2@kernel.dk> (raw)
In-Reply-To: <CAPhsuW6OcFeyVO-=12opRDijXHpQh1drQZZ6nLAq9P5TzvpnMA@mail.gmail.com>
On 8/25/22 10:47 AM, Song Liu wrote:
> On Tue, Aug 23, 2022 at 10:13 AM Song Liu <song@kernel.org> wrote:
>>
>> On Mon, Aug 22, 2022 at 8:15 PM Thomas Deutschmann <whissi@whissi.de> wrote:
>>>
>>> On 2022-08-23 03:37, Song Liu wrote:
>>>> Thomas, have you tried to bisect with the fio repro?
>>>
>>> Yes, just finished:
>>>
>>>> d32d3d0b47f7e34560ae3c55ddfcf68694813501 is the first bad commit
>>>> commit d32d3d0b47f7e34560ae3c55ddfcf68694813501
>>>> Author: Christoph Hellwig
>>>> Date: Mon Jun 14 13:17:34 2021 +0200
>>>>
>>>> nvme-multipath: set QUEUE_FLAG_NOWAIT
>>>>
>>>> The nvme multipathing code just dispatches bios to one of the blk-mq
>>>> based paths and never blocks on its own, so set QUEUE_FLAG_NOWAIT
>>>> to support REQ_NOWAIT bios.
>>>
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d32d3d0b47f7e34560ae3c55ddfcf68694813501
>>>
>>>
>>> So another NOWAIT issue -- similar to the bad commit which is causing
>>> the mdraid issue I already found
>>> (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0f9650bd838efe5c52f7e5f40c3204ad59f1964d).
>>>
>>> Reverting the commit, i.e. deleting
>>>
>>> blk_queue_flag_set(QUEUE_FLAG_NOWAIT, head->disk->queue);
>>>
>>> fixes the problem for me. Well, sort of. Looks like this will disable
>>> io_uring. fio reproducer fails with
>>
>> My system doesn't have multipath enabled. I guess bisect will point to something
>> else here.
>>
>> I am afraid we won't get more information from bisect.
>
> OK, I am able to pinpoint the issue, and Jens found the proper fix for
> it (see below,
> also available in [1]). It survived 100 runs of the repro fio job.
>
> Thomas, please give it a try.
>
> Thanks,
> Song
>
> diff --git c/fs/io_uring.c w/fs/io_uring.c
> index 3f8a79a4affa..72a39f5ec5a5 100644
> --- c/fs/io_uring.c
> +++ w/fs/io_uring.c
> @@ -4551,7 +4551,12 @@ static int io_write(struct io_kiocb *req,
> unsigned int issue_flags)
> copy_iov:
> iov_iter_restore(&s->iter, &s->iter_state);
> ret = io_setup_async_rw(req, iovec, s, false);
> - return ret ?: -EAGAIN;
> + if (!ret) {
> + if (kiocb->ki_flags & IOCB_WRITE)
> + kiocb_end_write(req);
> + return -EAGAIN;
> + }
> + return 0;
This should be 'return ret;' for that last line. I had to double check
the ones I did, but they did get it right. But I did a double take when
I saw this one :-)
It'll work fine for testing as we won't hit errors here unless we run
out of memory, so...
--
Jens Axboe
next prev parent reply other threads:[~2022-08-25 19:12 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-08-03 14:35 [REGRESSION] v5.17-rc1+: FIFREEZE ioctl system call hangs Thomas Deutschmann
2022-08-11 12:34 ` Thomas Deutschmann
2022-08-15 10:58 ` Thorsten Leemhuis
2022-08-15 15:46 ` Vishal Verma
2022-08-17 6:19 ` Song Liu
2022-08-17 6:53 ` Thomas Deutschmann
2022-08-17 18:29 ` Thomas Deutschmann
2022-08-19 2:46 ` Thomas Deutschmann
2022-08-20 1:04 ` Song Liu
2022-08-22 15:29 ` Thomas Deutschmann
2022-08-22 16:30 ` Thomas Deutschmann
2022-08-22 21:52 ` Song Liu
2022-08-22 22:44 ` Thomas Deutschmann
2022-08-22 22:59 ` Song Liu
2022-08-23 1:37 ` Song Liu
2022-08-23 3:15 ` Thomas Deutschmann
2022-08-23 17:13 ` Song Liu
2022-08-25 16:47 ` Song Liu
2022-08-25 19:12 ` Jens Axboe [this message]
2022-08-25 22:24 ` Song Liu
2022-08-26 20:10 ` Thomas Deutschmann
2022-09-08 13:25 ` [REGRESSION] v5.17-rc1+: FIFREEZE ioctl system call hangs #forregzbot Thorsten Leemhuis
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3114ed77-3802-8ca4-52e4-0e4b917d88b2@kernel.dk \
--to=axboe@kernel.dk \
--cc=hch@lst.de \
--cc=regressions@leemhuis.info \
--cc=regressions@lists.linux.dev \
--cc=song@kernel.org \
--cc=stable@vger.kernel.org \
--cc=vverma@digitalocean.com \
--cc=whissi@whissi.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).