From: Damien Le Moal <damien.lemoal@opensource.wdc.com> To: "Krzysztof Wilczyński" <kw@linux.com>, "Bjorn Helgaas" <helgaas@kernel.org> Cc: Christian Zigotzky <chzigotzky@xenosoft.de>, "bhelgaas@google.com >> Bjorn Helgaas" <bhelgaas@google.com>, linuxppc-dev <linuxppc-dev@lists.ozlabs.org>, Olof Johansson <olof@lixom.net>, "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>, Darren Stevens <darren@stevens-zone.net>, "R.T.Dickinson" <rtd2@xtra.co.nz>, mad skateman <madskateman@gmail.com>, Matthew Leaman <matthew@a-eon.biz>, Christian Zigotzky <info@xenosoft.de>, Jens Axboe <axboe@kernel.dk> Subject: Re: [PASEMI] Nemo board doesn't recognize any ATA disks with the pci-v5.16 updates Date: Wed, 10 Nov 2021 12:52:09 +0900 [thread overview] Message-ID: <1400f0f2-9247-6540-2685-be257d9ef243@opensource.wdc.com> (raw) In-Reply-To: <YYr4x1xWfptXRmqt@rocinante> On 2021/11/10 7:40, Krzysztof Wilczyński wrote: > [+CC Adding Jens and Damien to get their opinion about the problem at hand] > > Hello Jens and Damien, > > Sorry to bother both of you, but we are having a problem that most > definitely requires someone with an extensive expertise in storage, > as per the quoted message from Christian below: > >>>> The Nemo board [1] doesn't recognize any ATA disks with the pci-v5.16 >>>> updates [2]. >>>> >>>> Error messages: >>>> >>>> ata4.00: gc timeout cmd 0xec >>>> ata4.00: failed to IDENTIFY (I/O error, error_mask=0x4) >>>> ata1.00: gc timeout cmd 0xec >>>> ata1.00: failed to IDENTIFY (I/O error, error_mask=0x4) >>>> ata3.00: gc timeout cmd 0xec >>>> ata3.00: failed to IDENTIFY (I/O error, error_mask=0x4) IDENTIFY is the first command sent to a device when it is being probed. This means that at least the AHCI (is it AHCI ?) adapter found the ports and drives connected. But the qc timeout indicates that there is no response from the drive. This could be due to interrupts not being received for the command completion. One thing to try would be to increase the identify command timeout to see things simply got slow (for whatever reason) or if indeed there is no response at all. Note that after the first timeout, normally the port is reset and the command retried. That does not seem to be the case here. Weird... Maybe try something like this: diff --git a/drivers/ata/libata-eh.c b/drivers/ata/libata-eh.c index 1d4a6f1e88cd..16e105bcb899 100644 --- a/drivers/ata/libata-eh.c +++ b/drivers/ata/libata-eh.c @@ -79,7 +79,7 @@ enum { * take an exceptionally long time to recover from reset. */ static const unsigned long ata_eh_reset_timeouts[] = { - 10000, /* most drives spin up by 10sec */ + 30000, /* most drives spin up by 10sec */ 10000, /* > 99% working drives spin up before 20sec */ 35000, /* give > 30 secs of idleness for outlier devices */ 5000, /* and sweet one last chance */ Also note that I posted a patch a couple of days ago fixing a qc timeout for read log commands during device probe. This is not what you are hitting here though. I have not yet sent this to Linus. https://lore.kernel.org/linux-ide/20211105073106.422623-1-damien.lemoal@opensource.wdc.com/ > > The error message is also not very detailed and we aren't really sure what > the issue coming from the PCI sub-system might be causing or leading to > this. > >>>> >>>> I was able to revert the new pci-v5.16 updates [2]. After a new compiling, >>>> the kernel recognize all ATA disks correctly. >>>> >>>> Could you please check the pci-v5.16 updates [2]? >>>> >>>> Please find attached the kernel config. >>>> >>>> Thanks, >>>> Christian >>>> >>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000 >>>> [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0c5c62ddf88c34bc83b66e4ac9beb2bb0e1887d4 >> >> Sorry for the breakage, and thank you very much for the report. Can >> you please collect the complete dmesg logs before and after the >> pci-v5.16 changes and the "sudo lspci -vv" output from before the >> changes? >> >> You can attach them at https://bugzilla.kernel.org if you don't have >> a better place to put them. >> >> You could attach the kernel config there, too, since it didn't make it >> to the mailing list (vger may discard them -- see >> http://vger.kernel.org/majordomo-info.html). > > Bjorn and I looked at which commits that went with a recent Pull Request > from us might be causing this, but we are a little bit at loss, and were > hoping that you could give us a hand in troubleshooting this. > > Thank you in advance! > > Krzysztof > > -- Damien Le Moal Western Digital Research
WARNING: multiple messages have this Message-ID (diff)
From: Damien Le Moal <damien.lemoal@opensource.wdc.com> To: "Krzysztof Wilczyński" <kw@linux.com>, "Bjorn Helgaas" <helgaas@kernel.org> Cc: Jens Axboe <axboe@kernel.dk>, Darren Stevens <darren@stevens-zone.net>, mad skateman <madskateman@gmail.com>, "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>, Olof Johansson <olof@lixom.net>, "R.T.Dickinson" <rtd2@xtra.co.nz>, Christian Zigotzky <chzigotzky@xenosoft.de>, "bhelgaas@google.com >> Bjorn Helgaas" <bhelgaas@google.com>, Matthew Leaman <matthew@a-eon.biz>, linuxppc-dev <linuxppc-dev@lists.ozlabs.org>, Christian Zigotzky <info@xenosoft.de> Subject: Re: [PASEMI] Nemo board doesn't recognize any ATA disks with the pci-v5.16 updates Date: Wed, 10 Nov 2021 12:52:09 +0900 [thread overview] Message-ID: <1400f0f2-9247-6540-2685-be257d9ef243@opensource.wdc.com> (raw) In-Reply-To: <YYr4x1xWfptXRmqt@rocinante> On 2021/11/10 7:40, Krzysztof Wilczyński wrote: > [+CC Adding Jens and Damien to get their opinion about the problem at hand] > > Hello Jens and Damien, > > Sorry to bother both of you, but we are having a problem that most > definitely requires someone with an extensive expertise in storage, > as per the quoted message from Christian below: > >>>> The Nemo board [1] doesn't recognize any ATA disks with the pci-v5.16 >>>> updates [2]. >>>> >>>> Error messages: >>>> >>>> ata4.00: gc timeout cmd 0xec >>>> ata4.00: failed to IDENTIFY (I/O error, error_mask=0x4) >>>> ata1.00: gc timeout cmd 0xec >>>> ata1.00: failed to IDENTIFY (I/O error, error_mask=0x4) >>>> ata3.00: gc timeout cmd 0xec >>>> ata3.00: failed to IDENTIFY (I/O error, error_mask=0x4) IDENTIFY is the first command sent to a device when it is being probed. This means that at least the AHCI (is it AHCI ?) adapter found the ports and drives connected. But the qc timeout indicates that there is no response from the drive. This could be due to interrupts not being received for the command completion. One thing to try would be to increase the identify command timeout to see things simply got slow (for whatever reason) or if indeed there is no response at all. Note that after the first timeout, normally the port is reset and the command retried. That does not seem to be the case here. Weird... Maybe try something like this: diff --git a/drivers/ata/libata-eh.c b/drivers/ata/libata-eh.c index 1d4a6f1e88cd..16e105bcb899 100644 --- a/drivers/ata/libata-eh.c +++ b/drivers/ata/libata-eh.c @@ -79,7 +79,7 @@ enum { * take an exceptionally long time to recover from reset. */ static const unsigned long ata_eh_reset_timeouts[] = { - 10000, /* most drives spin up by 10sec */ + 30000, /* most drives spin up by 10sec */ 10000, /* > 99% working drives spin up before 20sec */ 35000, /* give > 30 secs of idleness for outlier devices */ 5000, /* and sweet one last chance */ Also note that I posted a patch a couple of days ago fixing a qc timeout for read log commands during device probe. This is not what you are hitting here though. I have not yet sent this to Linus. https://lore.kernel.org/linux-ide/20211105073106.422623-1-damien.lemoal@opensource.wdc.com/ > > The error message is also not very detailed and we aren't really sure what > the issue coming from the PCI sub-system might be causing or leading to > this. > >>>> >>>> I was able to revert the new pci-v5.16 updates [2]. After a new compiling, >>>> the kernel recognize all ATA disks correctly. >>>> >>>> Could you please check the pci-v5.16 updates [2]? >>>> >>>> Please find attached the kernel config. >>>> >>>> Thanks, >>>> Christian >>>> >>>> [1] https://en.wikipedia.org/wiki/AmigaOne_X1000 >>>> [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0c5c62ddf88c34bc83b66e4ac9beb2bb0e1887d4 >> >> Sorry for the breakage, and thank you very much for the report. Can >> you please collect the complete dmesg logs before and after the >> pci-v5.16 changes and the "sudo lspci -vv" output from before the >> changes? >> >> You can attach them at https://bugzilla.kernel.org if you don't have >> a better place to put them. >> >> You could attach the kernel config there, too, since it didn't make it >> to the mailing list (vger may discard them -- see >> http://vger.kernel.org/majordomo-info.html). > > Bjorn and I looked at which commits that went with a recent Pull Request > from us might be causing this, but we are a little bit at loss, and were > hoping that you could give us a hand in troubleshooting this. > > Thank you in advance! > > Krzysztof > > -- Damien Le Moal Western Digital Research
next prev parent reply other threads:[~2021-11-10 3:52 UTC|newest] Thread overview: 137+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-04-30 11:59 Radeon NI: GIT kernel with the nislands_smc commit doesn't boot on a Freescale P5040 board and P.A.Semi Nemo board Christian Zigotzky 2021-04-30 11:59 ` Christian Zigotzky 2021-04-30 15:26 ` Deucher, Alexander 2021-04-30 15:26 ` Deucher, Alexander 2021-04-30 15:26 ` Deucher, Alexander 2021-04-30 15:56 ` Gustavo A. R. Silva 2021-04-30 15:56 ` Gustavo A. R. Silva 2021-04-30 15:56 ` Gustavo A. R. Silva 2021-05-03 22:25 ` [FSL P50x0] Xorg always restarts again and again after the the PowerPC updates 5.13-1 Christian Zigotzky 2021-05-04 4:56 ` Christophe Leroy 2021-05-04 7:21 ` Christian Zigotzky 2021-05-04 7:47 ` Christophe Leroy 2021-05-04 8:29 ` Christian Zigotzky 2021-05-04 8:58 ` Christophe Leroy 2021-05-04 9:09 ` Christian Zigotzky 2021-05-04 9:11 ` Christophe Leroy 2021-05-04 9:46 ` Christian Zigotzky 2021-05-04 9:49 ` Christophe Leroy 2021-05-04 10:07 ` Christian Zigotzky 2021-05-04 11:02 ` Christian Zigotzky 2021-05-04 13:48 ` Christian Zigotzky 2021-05-04 14:48 ` Christophe Leroy 2021-05-04 14:52 ` Christian Zigotzky 2021-05-04 14:41 ` Christophe Leroy 2021-05-04 14:59 ` Christian Zigotzky 2021-05-04 15:17 ` Christophe Leroy 2021-05-05 12:43 ` Christian Zigotzky 2021-05-06 6:12 ` Christophe Leroy 2021-05-06 7:56 ` Christian Zigotzky 2021-05-06 8:09 ` Christophe Leroy 2021-05-06 13:58 ` Christian Zigotzky 2021-05-08 16:39 ` Christian Zigotzky 2021-05-09 12:27 ` Christian Zigotzky 2021-05-09 17:43 ` Christophe Leroy 2021-05-09 21:46 ` Christian Zigotzky 2021-05-09 17:36 ` Christophe Leroy 2021-05-13 6:47 ` Christian Zigotzky 2021-05-13 10:01 ` Christophe Leroy 2021-05-13 15:19 ` Christian Zigotzky 2021-05-13 15:51 ` Christophe Leroy 2021-05-13 16:35 ` Christian Zigotzky 2021-05-13 17:00 ` Christophe Leroy 2021-05-13 20:20 ` Christian Zigotzky 2021-05-13 22:58 ` Nicholas Piggin 2021-05-14 0:19 ` Christian Zigotzky 2021-05-04 14:44 ` Christophe Leroy 2021-05-06 23:55 ` Radeon NI: GIT kernel with the nislands_smc commit doesn't boot on a Freescale P5040 board and P.A.Semi Nemo board Gustavo A. R. Silva 2021-05-06 23:55 ` Gustavo A. R. Silva 2021-05-07 6:43 ` Christian Zigotzky 2021-05-07 6:43 ` Christian Zigotzky 2021-05-08 11:33 ` Christian Zigotzky 2021-05-08 11:33 ` Christian Zigotzky 2021-05-09 23:04 ` Gustavo A. R. Silva 2021-05-09 23:04 ` Gustavo A. R. Silva 2023-11-02 14:45 ` Fbdev issue after the drm updates 'drm-next-2023-10-31-1' Christian Zigotzky 2023-11-07 8:36 ` Christian Zigotzky 2023-11-12 14:23 ` Christian Zigotzky 2023-11-13 12:48 ` Geert Uytterhoeven 2023-11-13 12:48 ` Geert Uytterhoeven 2023-11-13 13:12 ` Christian Zigotzky 2023-11-13 13:12 ` Christian Zigotzky 2023-11-14 9:45 ` Christian Zigotzky 2023-11-14 9:45 ` Christian Zigotzky 2023-11-15 8:33 ` Geert Uytterhoeven 2023-11-15 8:33 ` Geert Uytterhoeven 2023-11-15 8:33 ` Geert Uytterhoeven 2023-11-15 9:22 ` Gerd Hoffmann 2023-11-15 9:22 ` Gerd Hoffmann 2023-11-15 9:22 ` Gerd Hoffmann 2023-11-16 13:13 ` Geert Uytterhoeven 2023-11-16 13:13 ` Geert Uytterhoeven 2023-11-16 13:13 ` Geert Uytterhoeven 2021-11-04 10:45 ` [VirtIO GPU] Xorg doesn't start with the DRM updates 'drm-next-2021-11-03' in a virtual e5500 QEMU KVM-HV machine on a Freescale P5040 board Christian Zigotzky 2021-11-04 21:42 ` [PATCH] drm/virtio: Fix NULL dereference error in virtio_gpu_poll Vivek Kasireddy 2021-11-05 18:05 ` Christian Zigotzky 2021-11-05 18:05 ` Christian Zigotzky 2021-11-08 12:56 ` Gerd Hoffmann 2021-11-08 14:48 ` [FSL P50x0] Cyrus+ board doesn't boot with the PowerPC updates 5.16-1 Christian Zigotzky 2021-11-09 14:45 ` [PASEMI] Nemo board doesn't recognize any ATA disks with the pci-v5.16 updates Christian Zigotzky 2021-11-09 15:10 ` Christian Zigotzky 2021-11-09 15:10 ` Christian Zigotzky 2021-11-09 16:58 ` Bjorn Helgaas 2021-11-09 16:58 ` Bjorn Helgaas 2021-11-09 22:40 ` Krzysztof Wilczyński 2021-11-09 22:40 ` Krzysztof Wilczyński 2021-11-09 23:05 ` Arnd Bergmann 2021-11-09 23:05 ` Arnd Bergmann 2021-11-09 23:18 ` Krzysztof Wilczyński 2021-11-09 23:18 ` Krzysztof Wilczyński 2021-11-10 16:42 ` Robert Święcki 2021-11-10 16:42 ` Robert Święcki 2021-11-10 3:52 ` Damien Le Moal [this message] 2021-11-10 3:52 ` Damien Le Moal 2021-11-10 18:07 ` Christian Zigotzky 2021-11-10 18:07 ` Christian Zigotzky 2021-11-10 18:41 ` Bjorn Helgaas 2021-11-10 18:41 ` Bjorn Helgaas 2021-11-10 19:09 ` Marc Zyngier 2021-11-10 19:09 ` Marc Zyngier 2021-11-11 5:24 ` Christian Zigotzky 2021-11-11 5:24 ` Christian Zigotzky 2021-11-11 7:13 ` Marc Zyngier 2021-11-11 7:13 ` Marc Zyngier 2021-11-11 7:47 ` Christian Zigotzky 2021-11-11 7:47 ` Christian Zigotzky 2021-11-11 10:20 ` Marc Zyngier 2021-11-11 10:20 ` Marc Zyngier 2021-11-11 10:44 ` Christian Zigotzky 2021-11-11 10:44 ` Christian Zigotzky 2021-11-11 11:24 ` Marc Zyngier 2021-11-11 11:24 ` Marc Zyngier 2021-11-11 11:54 ` Christian Zigotzky 2021-11-11 11:54 ` Christian Zigotzky 2021-11-11 22:21 ` Olof Johansson 2021-11-11 22:21 ` Olof Johansson 2021-11-12 11:49 ` Segher Boessenkool 2021-11-12 11:49 ` Segher Boessenkool 2021-11-11 17:39 ` Marc Zyngier 2021-11-11 17:39 ` Marc Zyngier 2021-11-12 9:40 ` Christian Zigotzky 2021-11-12 9:40 ` Christian Zigotzky 2021-11-12 10:11 ` Christian Zigotzky 2021-11-12 10:11 ` Christian Zigotzky 2021-11-12 11:00 ` Christian Zigotzky 2021-11-12 11:00 ` Christian Zigotzky 2021-11-12 13:41 ` Marc Zyngier 2021-11-12 13:41 ` Marc Zyngier 2021-11-12 14:15 ` Christian Zigotzky 2021-11-12 14:15 ` Christian Zigotzky 2021-11-12 14:46 ` Marc Zyngier 2021-11-12 14:46 ` Marc Zyngier 2021-11-12 15:01 ` Christian Zigotzky 2021-11-12 15:01 ` Christian Zigotzky 2021-11-12 15:05 ` Christian Zigotzky 2021-11-12 15:05 ` Christian Zigotzky 2021-11-12 16:14 ` [irqchip: irq/irqchip-fixes] of/irq: Don't ignore interrupt-controller when interrupt-map failed irqchip-bot for Marc Zyngier 2021-11-09 16:13 ` [FSL P50x0] Cyrus+ board doesn't boot with the PowerPC updates 5.16-1 Christophe Leroy
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1400f0f2-9247-6540-2685-be257d9ef243@opensource.wdc.com \ --to=damien.lemoal@opensource.wdc.com \ --cc=axboe@kernel.dk \ --cc=bhelgaas@google.com \ --cc=chzigotzky@xenosoft.de \ --cc=darren@stevens-zone.net \ --cc=helgaas@kernel.org \ --cc=info@xenosoft.de \ --cc=kw@linux.com \ --cc=linux-pci@vger.kernel.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=madskateman@gmail.com \ --cc=matthew@a-eon.biz \ --cc=olof@lixom.net \ --cc=rtd2@xtra.co.nz \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.