* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
[not found] ` <87ee0091-9c2f-50e8-c8f2-dcebebb9de48@kernel.dk>
@ 2021-11-11 12:58 ` Geert Uytterhoeven
2021-11-11 13:19 ` Martin K. Petersen
2021-11-11 13:44 ` Ming Lei
0 siblings, 2 replies; 17+ messages in thread
From: Geert Uytterhoeven @ 2021-11-11 12:58 UTC (permalink / raw)
To: Jens Axboe; +Cc: Christoph Hellwig, linux-block, linux-m68k, linux-kernel
Hi Jens,
On Thu, 4 Nov 2021, Jens Axboe wrote:
> On 11/4/21 1:04 PM, Christoph Hellwig wrote:
>> On Thu, Nov 04, 2021 at 01:02:54PM -0600, Jens Axboe wrote:
>>> On 11/4/21 12:52 PM, Christoph Hellwig wrote:
>>>> Looks good:
>>>>
>>>> Reviewed-by: Christoph Hellwig <hch@lst.de>
>>>
>>> So these two are now:
>>>
>>> https://git.kernel.dk/cgit/linux-block/commit/?h=for-5.16/block&id=c98cb5bbdab10d187aff9b4e386210eb2332af96
>>>
>>> which is the one I sent here, and then the next one gets cleaned up to
>>> remove that queue enter helper:
>>>
>>> https://git.kernel.dk/cgit/linux-block/commit/?h=for-5.16/block&id=7f930eb31eeb07f1b606b3316d8ad3ab6a92905b
>>>
>>> Can I add your reviewed-by to this last one as well? Only change is the
>>> removal of blk_mq_enter_queue() and the weird construct there, it's just
>>> bio_queue_enter() now.
>>
>> Sure.
>
> Thanks, prematurely already done, as you could tell :-)
The updated version is now commit 900e080752025f00 ("block: move queue
enter logic into blk_mq_submit_bio()") in Linus' tree.
I have bisected failures on m68k/atari (on ARAnyM, using nfhd as the
root device) to this commit, e.g.:
sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
Buffer I/O error on dev sda1, logical block 0, lost sync page write
EXT4-fs (sda1): I/O error while writing superblock
sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
Buffer I/O error on dev sda1, logical block 0, lost sync page write
EXT4-fs (sda1): I/O error while writing superblock
This may happen either when mounting the root file system (leading to an
unable to mount root fs panic), or later (leading to a read-only
rootfs).
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
2021-11-11 12:58 ` [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio() Geert Uytterhoeven
@ 2021-11-11 13:19 ` Martin K. Petersen
2021-11-11 14:48 ` Geert Uytterhoeven
2021-11-11 13:44 ` Ming Lei
1 sibling, 1 reply; 17+ messages in thread
From: Martin K. Petersen @ 2021-11-11 13:19 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: Jens Axboe, Christoph Hellwig, linux-block, linux-m68k, linux-kernel
Geert,
> sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
> sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
> sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
> sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
> critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
> Buffer I/O error on dev sda1, logical block 0, lost sync page write
Peculiar. That write command looks OK to me. I wonder if it's the FUA
bit that trips it?
What does:
# dmesg | grep FUA
say?
--
Martin K. Petersen Oracle Linux Engineering
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
2021-11-11 12:58 ` [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio() Geert Uytterhoeven
2021-11-11 13:19 ` Martin K. Petersen
@ 2021-11-11 13:44 ` Ming Lei
2021-11-11 14:51 ` Geert Uytterhoeven
1 sibling, 1 reply; 17+ messages in thread
From: Ming Lei @ 2021-11-11 13:44 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: Jens Axboe, Christoph Hellwig, linux-block, linux-m68k, linux-kernel
On Thu, Nov 11, 2021 at 01:58:38PM +0100, Geert Uytterhoeven wrote:
> Hi Jens,
>
> On Thu, 4 Nov 2021, Jens Axboe wrote:
> > On 11/4/21 1:04 PM, Christoph Hellwig wrote:
> > > On Thu, Nov 04, 2021 at 01:02:54PM -0600, Jens Axboe wrote:
> > > > On 11/4/21 12:52 PM, Christoph Hellwig wrote:
> > > > > Looks good:
> > > > >
> > > > > Reviewed-by: Christoph Hellwig <hch@lst.de>
> > > >
> > > > So these two are now:
> > > >
> > > > https://git.kernel.dk/cgit/linux-block/commit/?h=for-5.16/block&id=c98cb5bbdab10d187aff9b4e386210eb2332af96
> > > >
> > > > which is the one I sent here, and then the next one gets cleaned up to
> > > > remove that queue enter helper:
> > > >
> > > > https://git.kernel.dk/cgit/linux-block/commit/?h=for-5.16/block&id=7f930eb31eeb07f1b606b3316d8ad3ab6a92905b
> > > >
> > > > Can I add your reviewed-by to this last one as well? Only change is the
> > > > removal of blk_mq_enter_queue() and the weird construct there, it's just
> > > > bio_queue_enter() now.
> > >
> > > Sure.
> >
> > Thanks, prematurely already done, as you could tell :-)
>
> The updated version is now commit 900e080752025f00 ("block: move queue
> enter logic into blk_mq_submit_bio()") in Linus' tree.
>
> I have bisected failures on m68k/atari (on ARAnyM, using nfhd as the
> root device) to this commit, e.g.:
>
> sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
> sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
> sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
> sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
> critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
> Buffer I/O error on dev sda1, logical block 0, lost sync page write
>
> EXT4-fs (sda1): I/O error while writing superblock
> sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
> sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
> sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
> sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
> critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
> Buffer I/O error on dev sda1, logical block 0, lost sync page write
> EXT4-fs (sda1): I/O error while writing superblock
>
> This may happen either when mounting the root file system (leading to an
> unable to mount root fs panic), or later (leading to a read-only
> rootfs).
BTW, today I just found that hang in blk_mq_freeze_queue_wait() is
caused by commit 900e080752025f00, and the following patch can fix it:
- blk-mq: don't grab ->q_usage_counter in blk_mq_sched_bio_merge
https://lore.kernel.org/linux-block/20211111085650.GA476@lst.de/T/#m759b88fda094a65ebf29bc81b780967cdaf9cf28
Maybe you can try the above patch.
Thanks,
Ming
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
2021-11-11 13:19 ` Martin K. Petersen
@ 2021-11-11 14:48 ` Geert Uytterhoeven
2021-11-11 15:36 ` Martin K. Petersen
2021-11-11 21:35 ` Michael Schmitz
0 siblings, 2 replies; 17+ messages in thread
From: Geert Uytterhoeven @ 2021-11-11 14:48 UTC (permalink / raw)
To: Martin K. Petersen
Cc: Jens Axboe, Christoph Hellwig, linux-block, linux-m68k,
Linux Kernel Mailing List
Hi Martin,
On Thu, Nov 11, 2021 at 2:19 PM Martin K. Petersen
<martin.petersen@oracle.com> wrote:
> > sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
> > sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
> > sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
> > sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
> > critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
> > Buffer I/O error on dev sda1, logical block 0, lost sync page write
>
> Peculiar. That write command looks OK to me. I wonder if it's the FUA
> bit that trips it?
>
> What does:
>
> # dmesg | grep FUA
>
> say?
sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't
support DPO or FUA
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
2021-11-11 13:44 ` Ming Lei
@ 2021-11-11 14:51 ` Geert Uytterhoeven
2021-11-11 15:23 ` Ming Lei
0 siblings, 1 reply; 17+ messages in thread
From: Geert Uytterhoeven @ 2021-11-11 14:51 UTC (permalink / raw)
To: Ming Lei
Cc: Jens Axboe, Christoph Hellwig, linux-block, linux-m68k,
Linux Kernel Mailing List
Hi Ming,
On Thu, Nov 11, 2021 at 2:45 PM Ming Lei <ming.lei@redhat.com> wrote:
> On Thu, Nov 11, 2021 at 01:58:38PM +0100, Geert Uytterhoeven wrote:
> > On Thu, 4 Nov 2021, Jens Axboe wrote:
> > > On 11/4/21 1:04 PM, Christoph Hellwig wrote:
> > > > On Thu, Nov 04, 2021 at 01:02:54PM -0600, Jens Axboe wrote:
> > > > > On 11/4/21 12:52 PM, Christoph Hellwig wrote:
> > > > > > Looks good:
> > > > > >
> > > > > > Reviewed-by: Christoph Hellwig <hch@lst.de>
> > > > >
> > > > > So these two are now:
> > > > >
> > > > > https://git.kernel.dk/cgit/linux-block/commit/?h=for-5.16/block&id=c98cb5bbdab10d187aff9b4e386210eb2332af96
> > > > >
> > > > > which is the one I sent here, and then the next one gets cleaned up to
> > > > > remove that queue enter helper:
> > > > >
> > > > > https://git.kernel.dk/cgit/linux-block/commit/?h=for-5.16/block&id=7f930eb31eeb07f1b606b3316d8ad3ab6a92905b
> > > > >
> > > > > Can I add your reviewed-by to this last one as well? Only change is the
> > > > > removal of blk_mq_enter_queue() and the weird construct there, it's just
> > > > > bio_queue_enter() now.
> > > >
> > > > Sure.
> > >
> > > Thanks, prematurely already done, as you could tell :-)
> >
> > The updated version is now commit 900e080752025f00 ("block: move queue
> > enter logic into blk_mq_submit_bio()") in Linus' tree.
> >
> > I have bisected failures on m68k/atari (on ARAnyM, using nfhd as the
> > root device) to this commit, e.g.:
> >
> > sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
> > sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
> > sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
> > sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
> > critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
> > Buffer I/O error on dev sda1, logical block 0, lost sync page write
> >
> > EXT4-fs (sda1): I/O error while writing superblock
> > sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
> > sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
> > sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
> > sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
> > critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
> > Buffer I/O error on dev sda1, logical block 0, lost sync page write
> > EXT4-fs (sda1): I/O error while writing superblock
> >
> > This may happen either when mounting the root file system (leading to an
> > unable to mount root fs panic), or later (leading to a read-only
> > rootfs).
>
> BTW, today I just found that hang in blk_mq_freeze_queue_wait() is
> caused by commit 900e080752025f00, and the following patch can fix it:
>
> - blk-mq: don't grab ->q_usage_counter in blk_mq_sched_bio_merge
>
> https://lore.kernel.org/linux-block/20211111085650.GA476@lst.de/T/#m759b88fda094a65ebf29bc81b780967cdaf9cf28
>
> Maybe you can try the above patch.
Thanks! I have applied both patches, but it doesn't make a difference.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
2021-11-11 14:51 ` Geert Uytterhoeven
@ 2021-11-11 15:23 ` Ming Lei
2021-11-11 22:17 ` Jens Axboe
0 siblings, 1 reply; 17+ messages in thread
From: Ming Lei @ 2021-11-11 15:23 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: Jens Axboe, Christoph Hellwig, linux-block, linux-m68k,
Linux Kernel Mailing List
Hi Geert,
On Thu, Nov 11, 2021 at 03:51:28PM +0100, Geert Uytterhoeven wrote:
> Hi Ming,
>
> On Thu, Nov 11, 2021 at 2:45 PM Ming Lei <ming.lei@redhat.com> wrote:
> > On Thu, Nov 11, 2021 at 01:58:38PM +0100, Geert Uytterhoeven wrote:
> > > On Thu, 4 Nov 2021, Jens Axboe wrote:
> > > > On 11/4/21 1:04 PM, Christoph Hellwig wrote:
> > > > > On Thu, Nov 04, 2021 at 01:02:54PM -0600, Jens Axboe wrote:
> > > > > > On 11/4/21 12:52 PM, Christoph Hellwig wrote:
> > > > > > > Looks good:
> > > > > > >
> > > > > > > Reviewed-by: Christoph Hellwig <hch@lst.de>
> > > > > >
> > > > > > So these two are now:
> > > > > >
> > > > > > https://git.kernel.dk/cgit/linux-block/commit/?h=for-5.16/block&id=c98cb5bbdab10d187aff9b4e386210eb2332af96
> > > > > >
> > > > > > which is the one I sent here, and then the next one gets cleaned up to
> > > > > > remove that queue enter helper:
> > > > > >
> > > > > > https://git.kernel.dk/cgit/linux-block/commit/?h=for-5.16/block&id=7f930eb31eeb07f1b606b3316d8ad3ab6a92905b
> > > > > >
> > > > > > Can I add your reviewed-by to this last one as well? Only change is the
> > > > > > removal of blk_mq_enter_queue() and the weird construct there, it's just
> > > > > > bio_queue_enter() now.
> > > > >
> > > > > Sure.
> > > >
> > > > Thanks, prematurely already done, as you could tell :-)
> > >
> > > The updated version is now commit 900e080752025f00 ("block: move queue
> > > enter logic into blk_mq_submit_bio()") in Linus' tree.
> > >
> > > I have bisected failures on m68k/atari (on ARAnyM, using nfhd as the
> > > root device) to this commit, e.g.:
> > >
> > > sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
> > > sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
> > > sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
> > > sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
> > > critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
> > > Buffer I/O error on dev sda1, logical block 0, lost sync page write
> > >
> > > EXT4-fs (sda1): I/O error while writing superblock
> > > sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
> > > sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
> > > sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
> > > sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
> > > critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
> > > Buffer I/O error on dev sda1, logical block 0, lost sync page write
> > > EXT4-fs (sda1): I/O error while writing superblock
> > >
> > > This may happen either when mounting the root file system (leading to an
> > > unable to mount root fs panic), or later (leading to a read-only
> > > rootfs).
> >
> > BTW, today I just found that hang in blk_mq_freeze_queue_wait() is
> > caused by commit 900e080752025f00, and the following patch can fix it:
> >
> > - blk-mq: don't grab ->q_usage_counter in blk_mq_sched_bio_merge
> >
> > https://lore.kernel.org/linux-block/20211111085650.GA476@lst.de/T/#m759b88fda094a65ebf29bc81b780967cdaf9cf28
> >
> > Maybe you can try the above patch.
>
> Thanks! I have applied both patches, but it doesn't make a difference.
Thanks for your test!
Can you try the following patch?
diff --git a/block/blk-mq.c b/block/blk-mq.c
index f511db395c7f..a5ab2f2e9f67 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -2517,7 +2517,6 @@ static struct request *blk_mq_get_new_requests(struct request_queue *q,
struct blk_mq_alloc_data data = {
.q = q,
.nr_tags = 1,
- .cmd_flags = bio->bi_opf,
};
struct request *rq;
@@ -2525,6 +2524,7 @@ static struct request *blk_mq_get_new_requests(struct request_queue *q,
return NULL;
if (unlikely(!submit_bio_checks(bio)))
goto put_exit;
+ data.cmd_flags = bio->bi_opf;
if (blk_mq_attempt_bio_merge(q, bio, nsegs, same_queue_rq))
goto put_exit;
--
Ming
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
2021-11-11 14:48 ` Geert Uytterhoeven
@ 2021-11-11 15:36 ` Martin K. Petersen
2021-11-11 21:35 ` Michael Schmitz
1 sibling, 0 replies; 17+ messages in thread
From: Martin K. Petersen @ 2021-11-11 15:36 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: Martin K. Petersen, Jens Axboe, Christoph Hellwig, linux-block,
linux-m68k, Linux Kernel Mailing List
Geert,
>> > sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
^^
FUA
> sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't
> support DPO or FUA
The device is correct in rejecting the command, then. But I'm not sure
how the commit in question would cause us to inadvertently set that
flag...
--
Martin K. Petersen Oracle Linux Engineering
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
2021-11-11 14:48 ` Geert Uytterhoeven
2021-11-11 15:36 ` Martin K. Petersen
@ 2021-11-11 21:35 ` Michael Schmitz
2021-11-12 7:37 ` Geert Uytterhoeven
1 sibling, 1 reply; 17+ messages in thread
From: Michael Schmitz @ 2021-11-11 21:35 UTC (permalink / raw)
To: Geert Uytterhoeven, Martin K. Petersen
Cc: Jens Axboe, Christoph Hellwig, linux-block, linux-m68k,
Linux Kernel Mailing List
Hi Geert,
how easy is that to reproduce?
sd_setup_read_write_cmnd() does not validate the request's FUA flag
against sdkp->DPOFUA (not suggesting that it should ...). I'd like to
try and trace when such a mismatch happens.
Cheers,
Michael
On 12/11/21 03:48, Geert Uytterhoeven wrote:
> Hi Martin,
>
> On Thu, Nov 11, 2021 at 2:19 PM Martin K. Petersen
> <martin.petersen@oracle.com> wrote:
>>> sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
>>> sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
>>> sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
>>> sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
>>> critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
>>> Buffer I/O error on dev sda1, logical block 0, lost sync page write
>>
>> Peculiar. That write command looks OK to me. I wonder if it's the FUA
>> bit that trips it?
>>
>> What does:
>>
>> # dmesg | grep FUA
>>
>> say?
>
> sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't
> support DPO or FUA
>
> Gr{oetje,eeting}s,
>
> Geert
>
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
>
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like that.
> -- Linus Torvalds
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
2021-11-11 15:23 ` Ming Lei
@ 2021-11-11 22:17 ` Jens Axboe
2021-11-12 0:44 ` Ming Lei
0 siblings, 1 reply; 17+ messages in thread
From: Jens Axboe @ 2021-11-11 22:17 UTC (permalink / raw)
To: Ming Lei, Geert Uytterhoeven
Cc: Christoph Hellwig, linux-block, linux-m68k, Linux Kernel Mailing List
On 11/11/21 8:23 AM, Ming Lei wrote:
> Hi Geert,
>
> On Thu, Nov 11, 2021 at 03:51:28PM +0100, Geert Uytterhoeven wrote:
>> Hi Ming,
>>
>> On Thu, Nov 11, 2021 at 2:45 PM Ming Lei <ming.lei@redhat.com> wrote:
>>> On Thu, Nov 11, 2021 at 01:58:38PM +0100, Geert Uytterhoeven wrote:
>>>> On Thu, 4 Nov 2021, Jens Axboe wrote:
>>>>> On 11/4/21 1:04 PM, Christoph Hellwig wrote:
>>>>>> On Thu, Nov 04, 2021 at 01:02:54PM -0600, Jens Axboe wrote:
>>>>>>> On 11/4/21 12:52 PM, Christoph Hellwig wrote:
>>>>>>>> Looks good:
>>>>>>>>
>>>>>>>> Reviewed-by: Christoph Hellwig <hch@lst.de>
>>>>>>>
>>>>>>> So these two are now:
>>>>>>>
>>>>>>> https://git.kernel.dk/cgit/linux-block/commit/?h=for-5.16/block&id=c98cb5bbdab10d187aff9b4e386210eb2332af96
>>>>>>>
>>>>>>> which is the one I sent here, and then the next one gets cleaned up to
>>>>>>> remove that queue enter helper:
>>>>>>>
>>>>>>> https://git.kernel.dk/cgit/linux-block/commit/?h=for-5.16/block&id=7f930eb31eeb07f1b606b3316d8ad3ab6a92905b
>>>>>>>
>>>>>>> Can I add your reviewed-by to this last one as well? Only change is the
>>>>>>> removal of blk_mq_enter_queue() and the weird construct there, it's just
>>>>>>> bio_queue_enter() now.
>>>>>>
>>>>>> Sure.
>>>>>
>>>>> Thanks, prematurely already done, as you could tell :-)
>>>>
>>>> The updated version is now commit 900e080752025f00 ("block: move queue
>>>> enter logic into blk_mq_submit_bio()") in Linus' tree.
>>>>
>>>> I have bisected failures on m68k/atari (on ARAnyM, using nfhd as the
>>>> root device) to this commit, e.g.:
>>>>
>>>> sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
>>>> sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
>>>> sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
>>>> sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
>>>> critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
>>>> Buffer I/O error on dev sda1, logical block 0, lost sync page write
>>>>
>>>> EXT4-fs (sda1): I/O error while writing superblock
>>>> sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
>>>> sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
>>>> sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
>>>> sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
>>>> critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
>>>> Buffer I/O error on dev sda1, logical block 0, lost sync page write
>>>> EXT4-fs (sda1): I/O error while writing superblock
>>>>
>>>> This may happen either when mounting the root file system (leading to an
>>>> unable to mount root fs panic), or later (leading to a read-only
>>>> rootfs).
>>>
>>> BTW, today I just found that hang in blk_mq_freeze_queue_wait() is
>>> caused by commit 900e080752025f00, and the following patch can fix it:
>>>
>>> - blk-mq: don't grab ->q_usage_counter in blk_mq_sched_bio_merge
>>>
>>> https://lore.kernel.org/linux-block/20211111085650.GA476@lst.de/T/#m759b88fda094a65ebf29bc81b780967cdaf9cf28
>>>
>>> Maybe you can try the above patch.
>>
>> Thanks! I have applied both patches, but it doesn't make a difference.
>
> Thanks for your test!
>
> Can you try the following patch?
>
>
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index f511db395c7f..a5ab2f2e9f67 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -2517,7 +2517,6 @@ static struct request *blk_mq_get_new_requests(struct request_queue *q,
> struct blk_mq_alloc_data data = {
> .q = q,
> .nr_tags = 1,
> - .cmd_flags = bio->bi_opf,
> };
> struct request *rq;
>
> @@ -2525,6 +2524,7 @@ static struct request *blk_mq_get_new_requests(struct request_queue *q,
> return NULL;
> if (unlikely(!submit_bio_checks(bio)))
> goto put_exit;
> + data.cmd_flags = bio->bi_opf;
> if (blk_mq_attempt_bio_merge(q, bio, nsegs, same_queue_rq))
> goto put_exit;
That's definitely a real fix, akin to the other pre-enter variants, this
one just post checks. Geert, can you give this a whirl?
Ming, would you mind sending this as a real patch?
--
Jens Axboe
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
2021-11-11 22:17 ` Jens Axboe
@ 2021-11-12 0:44 ` Ming Lei
2021-11-12 7:51 ` Geert Uytterhoeven
0 siblings, 1 reply; 17+ messages in thread
From: Ming Lei @ 2021-11-12 0:44 UTC (permalink / raw)
To: Jens Axboe
Cc: Geert Uytterhoeven, Christoph Hellwig, linux-block, linux-m68k,
Linux Kernel Mailing List
On Thu, Nov 11, 2021 at 03:17:27PM -0700, Jens Axboe wrote:
> On 11/11/21 8:23 AM, Ming Lei wrote:
> > Hi Geert,
> >
> > On Thu, Nov 11, 2021 at 03:51:28PM +0100, Geert Uytterhoeven wrote:
> >> Hi Ming,
> >>
> >> On Thu, Nov 11, 2021 at 2:45 PM Ming Lei <ming.lei@redhat.com> wrote:
> >>> On Thu, Nov 11, 2021 at 01:58:38PM +0100, Geert Uytterhoeven wrote:
> >>>> On Thu, 4 Nov 2021, Jens Axboe wrote:
> >>>>> On 11/4/21 1:04 PM, Christoph Hellwig wrote:
> >>>>>> On Thu, Nov 04, 2021 at 01:02:54PM -0600, Jens Axboe wrote:
> >>>>>>> On 11/4/21 12:52 PM, Christoph Hellwig wrote:
> >>>>>>>> Looks good:
> >>>>>>>>
> >>>>>>>> Reviewed-by: Christoph Hellwig <hch@lst.de>
> >>>>>>>
> >>>>>>> So these two are now:
> >>>>>>>
> >>>>>>> https://git.kernel.dk/cgit/linux-block/commit/?h=for-5.16/block&id=c98cb5bbdab10d187aff9b4e386210eb2332af96
> >>>>>>>
> >>>>>>> which is the one I sent here, and then the next one gets cleaned up to
> >>>>>>> remove that queue enter helper:
> >>>>>>>
> >>>>>>> https://git.kernel.dk/cgit/linux-block/commit/?h=for-5.16/block&id=7f930eb31eeb07f1b606b3316d8ad3ab6a92905b
> >>>>>>>
> >>>>>>> Can I add your reviewed-by to this last one as well? Only change is the
> >>>>>>> removal of blk_mq_enter_queue() and the weird construct there, it's just
> >>>>>>> bio_queue_enter() now.
> >>>>>>
> >>>>>> Sure.
> >>>>>
> >>>>> Thanks, prematurely already done, as you could tell :-)
> >>>>
> >>>> The updated version is now commit 900e080752025f00 ("block: move queue
> >>>> enter logic into blk_mq_submit_bio()") in Linus' tree.
> >>>>
> >>>> I have bisected failures on m68k/atari (on ARAnyM, using nfhd as the
> >>>> root device) to this commit, e.g.:
> >>>>
> >>>> sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
> >>>> sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
> >>>> sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
> >>>> sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
> >>>> critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
> >>>> Buffer I/O error on dev sda1, logical block 0, lost sync page write
> >>>>
> >>>> EXT4-fs (sda1): I/O error while writing superblock
> >>>> sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
> >>>> sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
> >>>> sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
> >>>> sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
> >>>> critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
> >>>> Buffer I/O error on dev sda1, logical block 0, lost sync page write
> >>>> EXT4-fs (sda1): I/O error while writing superblock
> >>>>
> >>>> This may happen either when mounting the root file system (leading to an
> >>>> unable to mount root fs panic), or later (leading to a read-only
> >>>> rootfs).
> >>>
> >>> BTW, today I just found that hang in blk_mq_freeze_queue_wait() is
> >>> caused by commit 900e080752025f00, and the following patch can fix it:
> >>>
> >>> - blk-mq: don't grab ->q_usage_counter in blk_mq_sched_bio_merge
> >>>
> >>> https://lore.kernel.org/linux-block/20211111085650.GA476@lst.de/T/#m759b88fda094a65ebf29bc81b780967cdaf9cf28
> >>>
> >>> Maybe you can try the above patch.
> >>
> >> Thanks! I have applied both patches, but it doesn't make a difference.
> >
> > Thanks for your test!
> >
> > Can you try the following patch?
> >
> >
> > diff --git a/block/blk-mq.c b/block/blk-mq.c
> > index f511db395c7f..a5ab2f2e9f67 100644
> > --- a/block/blk-mq.c
> > +++ b/block/blk-mq.c
> > @@ -2517,7 +2517,6 @@ static struct request *blk_mq_get_new_requests(struct request_queue *q,
> > struct blk_mq_alloc_data data = {
> > .q = q,
> > .nr_tags = 1,
> > - .cmd_flags = bio->bi_opf,
> > };
> > struct request *rq;
> >
> > @@ -2525,6 +2524,7 @@ static struct request *blk_mq_get_new_requests(struct request_queue *q,
> > return NULL;
> > if (unlikely(!submit_bio_checks(bio)))
> > goto put_exit;
> > + data.cmd_flags = bio->bi_opf;
> > if (blk_mq_attempt_bio_merge(q, bio, nsegs, same_queue_rq))
> > goto put_exit;
>
> That's definitely a real fix, akin to the other pre-enter variants, this
> one just post checks. Geert, can you give this a whirl?
>
> Ming, would you mind sending this as a real patch?
Hi Jens,
The above patch may not be enough, since submit_bio_checks() is done in
case of using cached request, so how about the following patch(un-tested)?
diff --git a/block/blk-mq.c b/block/blk-mq.c
index f511db395c7f..f84044c8de3f 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -2517,7 +2517,6 @@ static struct request *blk_mq_get_new_requests(struct request_queue *q,
struct blk_mq_alloc_data data = {
.q = q,
.nr_tags = 1,
- .cmd_flags = bio->bi_opf,
};
struct request *rq;
@@ -2525,6 +2524,7 @@ static struct request *blk_mq_get_new_requests(struct request_queue *q,
return NULL;
if (unlikely(!submit_bio_checks(bio)))
goto put_exit;
+ data.cmd_flags = bio->bi_opf;
if (blk_mq_attempt_bio_merge(q, bio, nsegs, same_queue_rq))
goto put_exit;
@@ -2564,13 +2564,15 @@ static inline struct request *blk_mq_get_request(struct request_queue *q,
if (blk_mq_attempt_bio_merge(q, bio, nsegs,
same_queue_rq))
return NULL;
+ if (bio->bi_opf != rq->cmd_flags)
+ goto fallback;
plug->cached_rq = rq_list_next(rq);
INIT_LIST_HEAD(&rq->queuelist);
rq_qos_throttle(q, bio);
return rq;
}
}
-
+fallback:
return blk_mq_get_new_requests(q, plug, bio, nsegs, same_queue_rq);
}
--
Ming
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
2021-11-11 21:35 ` Michael Schmitz
@ 2021-11-12 7:37 ` Geert Uytterhoeven
2021-11-12 22:34 ` Michael Schmitz
2021-11-13 22:11 ` Michael Schmitz
0 siblings, 2 replies; 17+ messages in thread
From: Geert Uytterhoeven @ 2021-11-12 7:37 UTC (permalink / raw)
To: Michael Schmitz
Cc: Martin K. Petersen, Jens Axboe, Christoph Hellwig, linux-block,
linux-m68k, Linux Kernel Mailing List
Hi Michael,
On Thu, Nov 11, 2021 at 10:35 PM Michael Schmitz <schmitzmic@gmail.com> wrote:
> how easy is that to reproduce?
Fairly easy: it happens either on mounting, or after a few seconds booting
into my old Debian userspace.
> sd_setup_read_write_cmnd() does not validate the request's FUA flag
> against sdkp->DPOFUA (not suggesting that it should ...). I'd like to
> try and trace when such a mismatch happens.
Thanks!
> On 12/11/21 03:48, Geert Uytterhoeven wrote:
> > On Thu, Nov 11, 2021 at 2:19 PM Martin K. Petersen
> > <martin.petersen@oracle.com> wrote:
> >>> sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
> >>> sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
> >>> sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
> >>> sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
> >>> critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
> >>> Buffer I/O error on dev sda1, logical block 0, lost sync page write
> >>
> >> Peculiar. That write command looks OK to me. I wonder if it's the FUA
> >> bit that trips it?
> >>
> >> What does:
> >>
> >> # dmesg | grep FUA
> >>
> >> say?
> >
> > sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't
> > support DPO or FUA
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
2021-11-12 0:44 ` Ming Lei
@ 2021-11-12 7:51 ` Geert Uytterhoeven
2021-11-15 19:23 ` Michael Schmitz
0 siblings, 1 reply; 17+ messages in thread
From: Geert Uytterhoeven @ 2021-11-12 7:51 UTC (permalink / raw)
To: Ming Lei
Cc: Jens Axboe, Christoph Hellwig, linux-block, linux-m68k,
Linux Kernel Mailing List
Hi Ming,
On Fri, Nov 12, 2021 at 1:44 AM Ming Lei <ming.lei@redhat.com> wrote:
> On Thu, Nov 11, 2021 at 03:17:27PM -0700, Jens Axboe wrote:
> > On 11/11/21 8:23 AM, Ming Lei wrote:
> > > On Thu, Nov 11, 2021 at 03:51:28PM +0100, Geert Uytterhoeven wrote:
> > >> On Thu, Nov 11, 2021 at 2:45 PM Ming Lei <ming.lei@redhat.com> wrote:
> > >>> On Thu, Nov 11, 2021 at 01:58:38PM +0100, Geert Uytterhoeven wrote:
> > >>>> On Thu, 4 Nov 2021, Jens Axboe wrote:
> > >>>>> On 11/4/21 1:04 PM, Christoph Hellwig wrote:
> > >>>>>> On Thu, Nov 04, 2021 at 01:02:54PM -0600, Jens Axboe wrote:
> > >>>>>>> On 11/4/21 12:52 PM, Christoph Hellwig wrote:
> > >>>>>>>> Looks good:
> > >>>>>>>>
> > >>>>>>>> Reviewed-by: Christoph Hellwig <hch@lst.de>
> > >>>>>>>
> > >>>>>>> So these two are now:
> > >>>>>>>
> > >>>>>>> https://git.kernel.dk/cgit/linux-block/commit/?h=for-5.16/block&id=c98cb5bbdab10d187aff9b4e386210eb2332af96
> > >>>>>>>
> > >>>>>>> which is the one I sent here, and then the next one gets cleaned up to
> > >>>>>>> remove that queue enter helper:
> > >>>>>>>
> > >>>>>>> https://git.kernel.dk/cgit/linux-block/commit/?h=for-5.16/block&id=7f930eb31eeb07f1b606b3316d8ad3ab6a92905b
> > >>>>>>>
> > >>>>>>> Can I add your reviewed-by to this last one as well? Only change is the
> > >>>>>>> removal of blk_mq_enter_queue() and the weird construct there, it's just
> > >>>>>>> bio_queue_enter() now.
> > >>>>>>
> > >>>>>> Sure.
> > >>>>>
> > >>>>> Thanks, prematurely already done, as you could tell :-)
> > >>>>
> > >>>> The updated version is now commit 900e080752025f00 ("block: move queue
> > >>>> enter logic into blk_mq_submit_bio()") in Linus' tree.
> > >>>>
> > >>>> I have bisected failures on m68k/atari (on ARAnyM, using nfhd as the
> > >>>> root device) to this commit, e.g.:
> > >>>>
> > >>>> sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
> > >>>> sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
> > >>>> sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
> > >>>> sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
> > >>>> critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
> > >>>> Buffer I/O error on dev sda1, logical block 0, lost sync page write
> > >>>>
> > >>>> EXT4-fs (sda1): I/O error while writing superblock
> > >>>> sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
> > >>>> sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
> > >>>> sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
> > >>>> sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
> > >>>> critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
> > >>>> Buffer I/O error on dev sda1, logical block 0, lost sync page write
> > >>>> EXT4-fs (sda1): I/O error while writing superblock
> > >>>>
> > >>>> This may happen either when mounting the root file system (leading to an
> > >>>> unable to mount root fs panic), or later (leading to a read-only
> > >>>> rootfs).
> > >>>
> > >>> BTW, today I just found that hang in blk_mq_freeze_queue_wait() is
> > >>> caused by commit 900e080752025f00, and the following patch can fix it:
> > >>>
> > >>> - blk-mq: don't grab ->q_usage_counter in blk_mq_sched_bio_merge
> > >>>
> > >>> https://lore.kernel.org/linux-block/20211111085650.GA476@lst.de/T/#m759b88fda094a65ebf29bc81b780967cdaf9cf28
> > >>>
> > >>> Maybe you can try the above patch.
> > >>
> > >> Thanks! I have applied both patches, but it doesn't make a difference.
> > >
> > > Thanks for your test!
> > >
> > > Can you try the following patch?
[...]
> > That's definitely a real fix, akin to the other pre-enter variants, this
> > one just post checks. Geert, can you give this a whirl?
With both of
blk-mq: don't grab ->q_usage_counter in blk_mq_sched_bio_merge
blk-mq: rename blk_attempt_bio_merge
applied, and the version above, I no longer saw the error, but the
boot sometimes hangs after:
ext3 filesystem being remounted at / supports timestamps until
2038 (0x7fffffff)
I don't know how easy that is to trigger: it hung on my first try, but
the second and third tries it booted fully into old Debian userspace.
> > Ming, would you mind sending this as a real patch?
>
> The above patch may not be enough, since submit_bio_checks() is done in
> case of using cached request, so how about the following patch(un-tested)?
Worked fine in five subsequent boots. Thanks!
Tested-by: Geert Uytterhoeven <geert@linux-m68k.org>
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
2021-11-12 7:37 ` Geert Uytterhoeven
@ 2021-11-12 22:34 ` Michael Schmitz
2021-11-13 7:02 ` Michael Schmitz
2021-11-13 10:06 ` Geert Uytterhoeven
2021-11-13 22:11 ` Michael Schmitz
1 sibling, 2 replies; 17+ messages in thread
From: Michael Schmitz @ 2021-11-12 22:34 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: Martin K. Petersen, Jens Axboe, Christoph Hellwig, linux-block,
linux-m68k, Linux Kernel Mailing List
Hi Geert,
On 12/11/21 20:37, Geert Uytterhoeven wrote:
> Hi Michael,
>
> On Thu, Nov 11, 2021 at 10:35 PM Michael Schmitz <schmitzmic@gmail.com> wrote:
>> how easy is that to reproduce?
>
> Fairly easy: it happens either on mounting, or after a few seconds booting
> into my old Debian userspace.
I must be too thick for this:
EXT4-fs (sda1): Cannot load crc32c driver.
VFS: Cannot open root device "sda1" or unknown-block(8,1): error -80
(linux-block 5833291ab6de, from kernel.org)
Same config on m68k 5.15 works just fine.
But this looks resolved now, so I'll punt for now ...
Cheers,
Michael
>
>> sd_setup_read_write_cmnd() does not validate the request's FUA flag
>> against sdkp->DPOFUA (not suggesting that it should ...). I'd like to
>> try and trace when such a mismatch happens.
>
> Thanks!
>
>> On 12/11/21 03:48, Geert Uytterhoeven wrote:
>>> On Thu, Nov 11, 2021 at 2:19 PM Martin K. Petersen
>>> <martin.petersen@oracle.com> wrote:
>>>>> sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
>>>>> sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
>>>>> sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
>>>>> sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
>>>>> critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
>>>>> Buffer I/O error on dev sda1, logical block 0, lost sync page write
>>>>
>>>> Peculiar. That write command looks OK to me. I wonder if it's the FUA
>>>> bit that trips it?
>>>>
>>>> What does:
>>>>
>>>> # dmesg | grep FUA
>>>>
>>>> say?
>>>
>>> sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't
>>> support DPO or FUA
>
> Gr{oetje,eeting}s,
>
> Geert
>
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
>
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like that.
> -- Linus Torvalds
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
2021-11-12 22:34 ` Michael Schmitz
@ 2021-11-13 7:02 ` Michael Schmitz
2021-11-13 10:06 ` Geert Uytterhoeven
1 sibling, 0 replies; 17+ messages in thread
From: Michael Schmitz @ 2021-11-13 7:02 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: Martin K. Petersen, Jens Axboe, Christoph Hellwig, linux-block,
linux-m68k, Linux Kernel Mailing List
Hi Geert, Jens,
On 13/11/21 11:34, Michael Schmitz wrote:
> Hi Geert,
>
> On 12/11/21 20:37, Geert Uytterhoeven wrote:
>> Hi Michael,
>>
>> On Thu, Nov 11, 2021 at 10:35 PM Michael Schmitz
>> <schmitzmic@gmail.com> wrote:
>>> how easy is that to reproduce?
>>
>> Fairly easy: it happens either on mounting, or after a few seconds
>> booting
>> into my old Debian userspace.
>
> I must be too thick for this:
>
> EXT4-fs (sda1): Cannot load crc32c driver.
> VFS: Cannot open root device "sda1" or unknown-block(8,1): error -80
>
> (linux-block 5833291ab6de, from kernel.org)
>
> Same config on m68k 5.15 works just fine.
FWIW - I had to revert cad439fc040efe5f4381e3a7d583c5c200dbc186 to
overcome this, and reproduce Geert's finding.
Cheers,
Michael
>
> But this looks resolved now, so I'll punt for now ...
>
> Cheers,
>
> Michael
>
>>
>>> sd_setup_read_write_cmnd() does not validate the request's FUA flag
>>> against sdkp->DPOFUA (not suggesting that it should ...). I'd like to
>>> try and trace when such a mismatch happens.
>>
>> Thanks!
>>
>>> On 12/11/21 03:48, Geert Uytterhoeven wrote:
>>>> On Thu, Nov 11, 2021 at 2:19 PM Martin K. Petersen
>>>> <martin.petersen@oracle.com> wrote:
>>>>>> sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK
>>>>>> driverbyte=DRIVER_OK cmd_age=0s
>>>>>> sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
>>>>>> sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
>>>>>> sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
>>>>>> critical target error, dev sda, sector 1 op 0x1:(WRITE) flags
>>>>>> 0x20800 phys_seg 1 prio class 0
>>>>>> Buffer I/O error on dev sda1, logical block 0, lost sync page write
>>>>>
>>>>> Peculiar. That write command looks OK to me. I wonder if it's the FUA
>>>>> bit that trips it?
>>>>>
>>>>> What does:
>>>>>
>>>>> # dmesg | grep FUA
>>>>>
>>>>> say?
>>>>
>>>> sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't
>>>> support DPO or FUA
>>
>> Gr{oetje,eeting}s,
>>
>> Geert
>>
>> --
>> Geert Uytterhoeven -- There's lots of Linux beyond ia32 --
>> geert@linux-m68k.org
>>
>> In personal conversations with technical people, I call myself a
>> hacker. But
>> when I'm talking to journalists I just say "programmer" or something
>> like that.
>> -- Linus Torvalds
>>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
2021-11-12 22:34 ` Michael Schmitz
2021-11-13 7:02 ` Michael Schmitz
@ 2021-11-13 10:06 ` Geert Uytterhoeven
1 sibling, 0 replies; 17+ messages in thread
From: Geert Uytterhoeven @ 2021-11-13 10:06 UTC (permalink / raw)
To: Michael Schmitz
Cc: Martin K. Petersen, Jens Axboe, Christoph Hellwig, linux-block,
linux-m68k, Linux Kernel Mailing List
On Fri, Nov 12, 2021 at 11:34 PM Michael Schmitz <schmitzmic@gmail.com> wrote:
> On 12/11/21 20:37, Geert Uytterhoeven wrote:
> > On Thu, Nov 11, 2021 at 10:35 PM Michael Schmitz <schmitzmic@gmail.com> wrote:
> >> how easy is that to reproduce?
> >
> > Fairly easy: it happens either on mounting, or after a few seconds booting
> > into my old Debian userspace.
>
> I must be too thick for this:
>
> EXT4-fs (sda1): Cannot load crc32c driver.
> VFS: Cannot open root device "sda1" or unknown-block(8,1): error -80
I know ;-) (or :-(
Please cherry-pick commit beaaaa37c664e9af ("crypto: api - Fix boot-up
crash when crypto manager is disabled"), which is now upstream.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
2021-11-12 7:37 ` Geert Uytterhoeven
2021-11-12 22:34 ` Michael Schmitz
@ 2021-11-13 22:11 ` Michael Schmitz
1 sibling, 0 replies; 17+ messages in thread
From: Michael Schmitz @ 2021-11-13 22:11 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: Martin K. Petersen, Jens Axboe, Christoph Hellwig, linux-block,
linux-m68k, Linux Kernel Mailing List
Hi Geert, Martin,
On 12/11/21 20:37, Geert Uytterhoeven wrote:
> Hi Michael,
>
> On Thu, Nov 11, 2021 at 10:35 PM Michael Schmitz <schmitzmic@gmail.com> wrote:
>> how easy is that to reproduce?
>
> Fairly easy: it happens either on mounting, or after a few seconds booting
> into my old Debian userspace.
>
>> sd_setup_read_write_cmnd() does not validate the request's FUA flag
>> against sdkp->DPOFUA (not suggesting that it should ...). I'd like to
>> try and trace when such a mismatch happens.
>
> Thanks!
Also happens with the pata-falcon driver (which stats it doesnt support
DPO or FUA).
Seen twice, once on boot, once on reboot (FUA bit not set in the SCSI
command descriptor when the disk doesn't support that, to avoid a panic,
otherwise still 5833291ab6de). Each time when a remount operation happens:
EXT4-fs (sda1): re-mounted. Opts: . Quota mode: disabled.
------------[ cut here ]------------
WARNING: CPU: 0 PID: 695 at drivers/scsi/sd.c:1271
sd_init_command+0x6aa/0x886
Modules linked in:
CPU: 0 PID: 695 Comm: mount Not tainted
5.15.0-atari-fpuemu-clean-signalfixes-remove-set_fs-ataflopfix4+ #88
Stack from 00be1c10:
00be1c10 0035cc02 0035cc02 002cac7a 00379b57 000004f7 00000009
00ad80a4
002cacd4 00379b57 000004f7 0020ac5e 00000009 00000000 00000000
00209d00
00000008 00000000 00000001 00000000 00000001 00ab6400 0020ac5e
00379b57
000004f7 00000009 00000000 00adc800 00add9d0 00ab6400 00000001
001269c4
00003a98 00ad8000 00adc800 00ad80a4 00ab6400 00adc800 00000008
00020801
00000000 00000008 00000000 00000001 00000001 00201876 00ad80a4
00ad80a4
Call Trace: [<002cac7a>] __warn+0x9e/0xb6
[<002cacd4>] warn_slowpath_fmt+0x42/0x62
[<0020ac5e>] sd_init_command+0x6aa/0x886
[<00209d00>] max_write_same_blocks_store+0x6c/0x88
[<0020ac5e>] sd_init_command+0x6aa/0x886
[<001269c4>] ext4_get_group_desc+0x0/0xb0
[<00003a98>] setup_frame+0xba/0x20a
[<00020801>] _I_CALL_TOP+0xf21/0x1900
[<00201876>] scsi_queue_rq+0x4f8/0x67e
[<00194aaa>] __blk_mq_try_issue_directly+0x8a/0xf8
[<00194adc>] __blk_mq_try_issue_directly+0xbc/0xf8
[<00194fd0>] blk_mq_try_issue_directly+0x28/0x7a
[<0019584c>] blk_mq_submit_bio+0x3fc/0x422
[<0018c6f6>] __submit_bio+0x0/0x78
[<0018b5cc>] bio_list_pop+0x0/0x1a
[<00092800>] btf_check_type_match+0x274/0x2e4
[<00020801>] _I_CALL_TOP+0xf21/0x1900
[<00002000>] _start+0x0/0x8
[<0018c7dc>] submit_bio_noacct+0x6e/0x152
[<00020801>] _I_CALL_TOP+0xf21/0x1900
[<000f21fc>] test_bit+0x0/0x12
[<00051f68>] ktime_get_real_seconds+0x0/0x30
[<000f21fc>] test_bit+0x0/0x12
[<00051f68>] ktime_get_real_seconds+0x0/0x30
[<000f329c>] submit_bh_wbc.isra.54+0x178/0x182
[<000f21fc>] test_bit+0x0/0x12
[<000f32b8>] submit_bh+0x12/0x18
[<00020800>] _I_CALL_TOP+0xf20/0x1900
[<000f362e>] __sync_dirty_buffer+0x78/0xaa
[<00020800>] _I_CALL_TOP+0xf20/0x1900
[<00158684>] test_bit+0x0/0x12
[<0015a508>] ext4_commit_super+0xa8/0xf8
[<00020800>] _I_CALL_TOP+0xf20/0x1900
[<0015be82>] ext4_setup_super+0x132/0x1ac
[<0015c970>] ext4_remount+0x49c/0x57e
[<00004003>] do_rt_sigreturn+0xe1/0x1b8
[<000dbe3e>] dentry_lru_isolate_shrink+0x0/0x1c
[<000efaf0>] legacy_reconfigure+0x30/0x42
[<000ce664>] reconfigure_super+0xae/0x178
[<000e4a38>] path_mount+0x274/0x61c
[<000e4e20>] do_mount+0x40/0x5c
[<000e50c4>] sys_mount+0xf8/0x10e
[<00002922>] syscall+0x8/0xc
[<0018c00c>] blk_cleanup_queue+0x5a/0xc6
---[ end trace b0cd975622ec08c4 ]---
EXT4-fs (sda1): re-mounted. Opts: errors=remount-ro. Quota mode: disabled.
ext4 filesystem being remounted at / supports timestamps until 2038
(0x7fffffff)
------------[ cut here ]------------
WARNING: CPU: 0 PID: 1174 at drivers/scsi/sd.c:1271
sd_init_command+0x6aa/0x886
Modules linked in:
CPU: 0 PID: 1174 Comm: umount Tainted: G W
5.15.0-atari-fpuemu-clean-signalfixes-remove-set_fs-ataflopfix4+ #88
Stack from 00b6dc70:
00b6dc70 0035cc02 0035cc02 002cac7a 00379b57 000004f7 00000009
00ad80a4
002cacd4 00379b57 000004f7 0020ac5e 00000009 00000000 00000000
00209d00
00000008 00000000 00000001 00000000 00000001 00ab6400 0020ac5e
00379b57
000004f7 00000009 00000000 00adc800 00add9d0 00ab6400 00000001
00000000
00003a98 00ad8000 00adc800 00ad80a4 00ab6400 00adc800 00000008
00023801
00000000 00000008 00000000 00000001 00000001 00201876 00ad80a4
00ad80a4
Call Trace: [<002cac7a>] __warn+0x9e/0xb6
[<002cacd4>] warn_slowpath_fmt+0x42/0x62
[<0020ac5e>] sd_init_command+0x6aa/0x886
[<00209d00>] max_write_same_blocks_store+0x6c/0x88
[<0020ac5e>] sd_init_command+0x6aa/0x886
[<00003a98>] setup_frame+0xba/0x20a
[<00023801>] fp_fadd+0x1d5/0x1e4
[<00201876>] scsi_queue_rq+0x4f8/0x67e
[<00194aaa>] __blk_mq_try_issue_directly+0x8a/0xf8
[<00194adc>] __blk_mq_try_issue_directly+0xbc/0xf8
[<00194fd0>] blk_mq_try_issue_directly+0x28/0x7a
[<0019584c>] blk_mq_submit_bio+0x3fc/0x422
[<0018c6f6>] __submit_bio+0x0/0x78
[<0018b5cc>] bio_list_pop+0x0/0x1a
[<00092800>] btf_check_type_match+0x274/0x2e4
[<00023801>] fp_fadd+0x1d5/0x1e4
[<00002000>] _start+0x0/0x8
[<0018c7dc>] submit_bio_noacct+0x6e/0x152
[<00023801>] fp_fadd+0x1d5/0x1e4
[<000f21fc>] test_bit+0x0/0x12
[<000f21fc>] test_bit+0x0/0x12
[<000f329c>] submit_bh_wbc.isra.54+0x178/0x182
[<00158684>] test_bit+0x0/0x12
[<000f21fc>] test_bit+0x0/0x12
[<000f32b8>] submit_bh+0x12/0x18
[<00020800>] _I_CALL_TOP+0xf20/0x1900
[<000f362e>] __sync_dirty_buffer+0x78/0xaa
[<00020800>] _I_CALL_TOP+0xf20/0x1900
[<00158684>] test_bit+0x0/0x12
[<00158684>] test_bit+0x0/0x12
[<0015a508>] ext4_commit_super+0xa8/0xf8
[<00020800>] _I_CALL_TOP+0xf20/0x1900
[<00158684>] test_bit+0x0/0x12
[<0015c80e>] ext4_remount+0x33a/0x57e
[<00004003>] do_rt_sigreturn+0xe1/0x1b8
[<000dbe3e>] dentry_lru_isolate_shrink+0x0/0x1c
[<000efaf0>] legacy_reconfigure+0x30/0x42
[<000ce664>] reconfigure_super+0xae/0x178
[<000e3ab0>] path_umount+0x17c/0x2e8
[<000e3c62>] ksys_umount+0x46/0x54
[<000e3c8a>] sys_oldumount+0xa/0xe
[<00002922>] syscall+0x8/0xc
[<0018c00c>] blk_cleanup_queue+0x5a/0xc6
---[ end trace b0cd975622ec08c5 ]---
EXT4-fs (sda1): re-mounted. Opts: (null). Quota mode: disabled.
EXT4-fs (sda1): re-mounted. Opts: errors=remount-ro. Quota mode: disabled.
Not sure this helps much.
Cheers,
Michael
>
>> On 12/11/21 03:48, Geert Uytterhoeven wrote:
>>> On Thu, Nov 11, 2021 at 2:19 PM Martin K. Petersen
>>> <martin.petersen@oracle.com> wrote:
>>>>> sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
>>>>> sd 0:0:0:0: [sda] tag#0 Sense Key : Illegal Request [current]
>>>>> sd 0:0:0:0: [sda] tag#0 Add. Sense: Invalid field in cdb
>>>>> sd 0:0:0:0: [sda] tag#0 CDB: Write(10) 2a 08 00 00 00 01 00 00 08 00
>>>>> critical target error, dev sda, sector 1 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
>>>>> Buffer I/O error on dev sda1, logical block 0, lost sync page write
>>>>
>>>> Peculiar. That write command looks OK to me. I wonder if it's the FUA
>>>> bit that trips it?
>>>>
>>>> What does:
>>>>
>>>> # dmesg | grep FUA
>>>>
>>>> say?
>>>
>>> sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't
>>> support DPO or FUA
>
> Gr{oetje,eeting}s,
>
> Geert
>
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
>
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like that.
> -- Linus Torvalds
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio()
2021-11-12 7:51 ` Geert Uytterhoeven
@ 2021-11-15 19:23 ` Michael Schmitz
0 siblings, 0 replies; 17+ messages in thread
From: Michael Schmitz @ 2021-11-15 19:23 UTC (permalink / raw)
To: Geert Uytterhoeven, Ming Lei
Cc: Jens Axboe, Christoph Hellwig, linux-block, linux-m68k,
Linux Kernel Mailing List
On 12/11/21 20:51, Geert Uytterhoeven wrote:
[...]
>>>>>> BTW, today I just found that hang in blk_mq_freeze_queue_wait() is
>>>>>> caused by commit 900e080752025f00, and the following patch can fix it:
>>>>>>
>>>>>> - blk-mq: don't grab ->q_usage_counter in blk_mq_sched_bio_merge
>>>>>>
>>>>>> https://lore.kernel.org/linux-block/20211111085650.GA476@lst.de/T/#m759b88fda094a65ebf29bc81b780967cdaf9cf28
>>>>>>
>>>>>> Maybe you can try the above patch.
>>>>>
>>>>> Thanks! I have applied both patches, but it doesn't make a difference.
>>>>
>>>> Thanks for your test!
>>>>
>>>> Can you try the following patch?
>
> [...]
>
>>> That's definitely a real fix, akin to the other pre-enter variants, this
>>> one just post checks. Geert, can you give this a whirl?
>
> With both of
>
> blk-mq: don't grab ->q_usage_counter in blk_mq_sched_bio_merge
> blk-mq: rename blk_attempt_bio_merge
>
> applied, and the version above, I no longer saw the error, but the
> boot sometimes hangs after:
>
> ext3 filesystem being remounted at / supports timestamps until
> 2038 (0x7fffffff)
>
> I don't know how easy that is to trigger: it hung on my first try, but
> the second and third tries it booted fully into old Debian userspace.
>
>>> Ming, would you mind sending this as a real patch?
>>
>> The above patch may not be enough, since submit_bio_checks() is done in
>> case of using cached request, so how about the following patch(un-tested)?
>
> Worked fine in five subsequent boots. Thanks!
> Tested-by: Geert Uytterhoeven <geert@linux-m68k.org>
For good measure: block-5.16-2021-11-13 tested fine running IO stress
tests on a mix of IDE and SCSI disks, only one of those supporting DPO/FUA.
Using
WARN_ON(rq->cmd_flags & REQ_FUA && !sdkp->DPOFUA);
in sd.c:sd_setup_read_write_cmnd(), nothing seen in the logs.
Cheers,
Michael
>
> Gr{oetje,eeting}s,
>
> Geert
>
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
>
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like that.
> -- Linus Torvalds
>
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2021-11-16 0:04 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20211104182201.83906-1-axboe@kernel.dk>
[not found] ` <20211104182201.83906-5-axboe@kernel.dk>
[not found] ` <YYQoLzMn7+s9hxpX@infradead.org>
[not found] ` <2865c289-7014-2250-0f5b-a9ed8770d0ec@kernel.dk>
[not found] ` <YYQo4ougXZvgv11X@infradead.org>
[not found] ` <8c6163f4-0c0f-5254-5f79-9074f5a73cfe@kernel.dk>
[not found] ` <461c4758-2675-1d11-ac8a-6f25ef01d781@kernel.dk>
[not found] ` <YYQr3jl3avsuOUAJ@infradead.org>
[not found] ` <3d29a5ce-aace-6198-3ea9-e6f603e74aa1@kernel.dk>
[not found] ` <YYQuyt2/y1MgzRi0@infradead.org>
[not found] ` <87ee0091-9c2f-50e8-c8f2-dcebebb9de48@kernel.dk>
2021-11-11 12:58 ` [PATCH 4/5] block: move queue enter logic into blk_mq_submit_bio() Geert Uytterhoeven
2021-11-11 13:19 ` Martin K. Petersen
2021-11-11 14:48 ` Geert Uytterhoeven
2021-11-11 15:36 ` Martin K. Petersen
2021-11-11 21:35 ` Michael Schmitz
2021-11-12 7:37 ` Geert Uytterhoeven
2021-11-12 22:34 ` Michael Schmitz
2021-11-13 7:02 ` Michael Schmitz
2021-11-13 10:06 ` Geert Uytterhoeven
2021-11-13 22:11 ` Michael Schmitz
2021-11-11 13:44 ` Ming Lei
2021-11-11 14:51 ` Geert Uytterhoeven
2021-11-11 15:23 ` Ming Lei
2021-11-11 22:17 ` Jens Axboe
2021-11-12 0:44 ` Ming Lei
2021-11-12 7:51 ` Geert Uytterhoeven
2021-11-15 19:23 ` Michael Schmitz
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).