* Failure to shutdown/reboot with intel_iommu=on @ 2020-05-06 14:46 Uwe Kleine-König 2020-05-08 15:07 ` Joerg Roedel 0 siblings, 1 reply; 10+ messages in thread From: Uwe Kleine-König @ 2020-05-06 14:46 UTC (permalink / raw) To: linux-kernel; +Cc: rafael.j.wysocki, jroedel, tglx, x86 [-- Attachment #1: Type: text/plain, Size: 1014 bytes --] Hello, On my Lenovo T460p I cannot shutdown and reboot when the iommu is enabled. This is using linux 5.2.7 as provided by Debian, 5.6.4 has the same problem. Suspend/resume also fails; I suspect this is the same issue. When requesting power off the kernel messages just end with: sd 0:0:0:0: [sda] Synchronizing SCSI cache sd 0:0:0:0: [sda] Stopping disk e1000e: EEE TX LPI TIMER: 00000011 ACPI: Preparing to enter system sleep state S5 reboot: Power down acpi_power_off called (photo at https://www.kleine-koenig.org/tmp/uklsiommu.jpg in case I mistyped something. Full dmesg and lspci -vvv at https://www.kleine-koenig.org/tmp/uklsiommu.tar.gz with and without iommu enabled.) With the iommu disabled (CONFIG_INTEL_IOMMU_DEFAULT_ON unset or intel_iommu=off on cmdline) the machine just works as expected (including working suspend/resume). I already talked to tglx on irc but unfortunately no new insights resulted from that. Any ideas how to fix or continue debugging the issue? Best regards Uwe [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Failure to shutdown/reboot with intel_iommu=on 2020-05-06 14:46 Failure to shutdown/reboot with intel_iommu=on Uwe Kleine-König @ 2020-05-08 15:07 ` Joerg Roedel 2020-05-09 1:58 ` Lu Baolu 2020-05-11 13:43 ` Lenny Szubowicz 0 siblings, 2 replies; 10+ messages in thread From: Joerg Roedel @ 2020-05-08 15:07 UTC (permalink / raw) To: Uwe Kleine-König; +Cc: linux-kernel, rafael.j.wysocki, tglx, x86, Lu Baolu + Baolu, Maintainer of Intel IOMMU Baolu, does that ring any bells? On Wed, May 06, 2020 at 04:46:02PM +0200, Uwe Kleine-König wrote: > Hello, > > On my Lenovo T460p I cannot shutdown and reboot when the iommu is > enabled. This is using linux 5.2.7 as provided by Debian, 5.6.4 has the > same problem. Suspend/resume also fails; I suspect this is the same > issue. > > When requesting power off the kernel messages just end with: > > sd 0:0:0:0: [sda] Synchronizing SCSI cache > sd 0:0:0:0: [sda] Stopping disk > e1000e: EEE TX LPI TIMER: 00000011 > ACPI: Preparing to enter system sleep state S5 > reboot: Power down > acpi_power_off called > > (photo at https://www.kleine-koenig.org/tmp/uklsiommu.jpg in case I > mistyped something. Full dmesg and lspci -vvv at > https://www.kleine-koenig.org/tmp/uklsiommu.tar.gz with and without > iommu enabled.) > > With the iommu disabled (CONFIG_INTEL_IOMMU_DEFAULT_ON unset or > intel_iommu=off on cmdline) the machine just works as expected > (including working suspend/resume). > > I already talked to tglx on irc but unfortunately no new insights > resulted from that. > > Any ideas how to fix or continue debugging the issue? > > Best regards > Uwe ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Failure to shutdown/reboot with intel_iommu=on 2020-05-08 15:07 ` Joerg Roedel @ 2020-05-09 1:58 ` Lu Baolu 2020-05-11 19:59 ` Uwe Kleine-König 2020-05-11 13:43 ` Lenny Szubowicz 1 sibling, 1 reply; 10+ messages in thread From: Lu Baolu @ 2020-05-09 1:58 UTC (permalink / raw) To: Joerg Roedel, Uwe Kleine-König Cc: baolu.lu, linux-kernel, rafael.j.wysocki, tglx, x86 Hi Uwe, Have you tried commenting out intel_disable_iommus() in intel_iommu_shutdowan()? diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index 0182cff2c7ac..532e62600f95 100644 --- a/drivers/iommu/intel-iommu.c +++ b/drivers/iommu/intel-iommu.c @@ -4928,8 +4928,10 @@ void intel_iommu_shutdown(void) for_each_iommu(iommu, drhd) iommu_disable_protect_mem_regions(iommu); +#if 0 /* Make sure the IOMMUs are switched off */ intel_disable_iommus(); +#endif up_write(&dmar_global_lock); } Best regards, baolu On 5/8/20 11:07 PM, Joerg Roedel wrote: > + Baolu, Maintainer of Intel IOMMU > > Baolu, does that ring any bells? > > On Wed, May 06, 2020 at 04:46:02PM +0200, Uwe Kleine-König wrote: >> Hello, >> >> On my Lenovo T460p I cannot shutdown and reboot when the iommu is >> enabled. This is using linux 5.2.7 as provided by Debian, 5.6.4 has the >> same problem. Suspend/resume also fails; I suspect this is the same >> issue. >> >> When requesting power off the kernel messages just end with: >> >> sd 0:0:0:0: [sda] Synchronizing SCSI cache >> sd 0:0:0:0: [sda] Stopping disk >> e1000e: EEE TX LPI TIMER: 00000011 >> ACPI: Preparing to enter system sleep state S5 >> reboot: Power down >> acpi_power_off called >> >> (photo at https://www.kleine-koenig.org/tmp/uklsiommu.jpg in case I >> mistyped something. Full dmesg and lspci -vvv at >> https://www.kleine-koenig.org/tmp/uklsiommu.tar.gz with and without >> iommu enabled.) >> >> With the iommu disabled (CONFIG_INTEL_IOMMU_DEFAULT_ON unset or >> intel_iommu=off on cmdline) the machine just works as expected >> (including working suspend/resume). >> >> I already talked to tglx on irc but unfortunately no new insights >> resulted from that. >> >> Any ideas how to fix or continue debugging the issue? >> >> Best regards >> Uwe > > ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: Failure to shutdown/reboot with intel_iommu=on 2020-05-09 1:58 ` Lu Baolu @ 2020-05-11 19:59 ` Uwe Kleine-König 2020-05-11 20:16 ` Uwe Kleine-König 0 siblings, 1 reply; 10+ messages in thread From: Uwe Kleine-König @ 2020-05-11 19:59 UTC (permalink / raw) To: Lu Baolu, Joerg Roedel; +Cc: linux-kernel, rafael.j.wysocki, tglx, x86 [-- Attachment #1.1: Type: text/plain, Size: 833 bytes --] On 5/9/20 3:58 AM, Lu Baolu wrote: > Hi Uwe, > > Have you tried commenting out intel_disable_iommus() in > intel_iommu_shutdowan()? > > diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c > index 0182cff2c7ac..532e62600f95 100644 > --- a/drivers/iommu/intel-iommu.c > +++ b/drivers/iommu/intel-iommu.c > @@ -4928,8 +4928,10 @@ void intel_iommu_shutdown(void) > for_each_iommu(iommu, drhd) > iommu_disable_protect_mem_regions(iommu); > > +#if 0 > /* Make sure the IOMMUs are switched off */ > intel_disable_iommus(); > +#endif > > up_write(&dmar_global_lock); > } I just tested that and it didn't help. The machine still hangs with the same symptoms as reported before. Best regards Uwe [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Failure to shutdown/reboot with intel_iommu=on 2020-05-11 19:59 ` Uwe Kleine-König @ 2020-05-11 20:16 ` Uwe Kleine-König 0 siblings, 0 replies; 10+ messages in thread From: Uwe Kleine-König @ 2020-05-11 20:16 UTC (permalink / raw) To: Lu Baolu, Joerg Roedel; +Cc: linux-kernel, rafael.j.wysocki, tglx, x86 [-- Attachment #1: Type: text/plain, Size: 2210 bytes --] Hello again, On Mon, May 11, 2020 at 09:59:31PM +0200, Uwe Kleine-König wrote: > On 5/9/20 3:58 AM, Lu Baolu wrote: > > Hi Uwe, > > > > Have you tried commenting out intel_disable_iommus() in > > intel_iommu_shutdowan()? > > > > diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c > > index 0182cff2c7ac..532e62600f95 100644 > > --- a/drivers/iommu/intel-iommu.c > > +++ b/drivers/iommu/intel-iommu.c > > @@ -4928,8 +4928,10 @@ void intel_iommu_shutdown(void) > > for_each_iommu(iommu, drhd) > > iommu_disable_protect_mem_regions(iommu); > > > > +#if 0 > > /* Make sure the IOMMUs are switched off */ > > intel_disable_iommus(); > > +#endif > > > > up_write(&dmar_global_lock); > > } > > I just tested that and it didn't help. The machine still hangs with the > same symptoms as reported before. I patched the file a bit differently: diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index ef0a5246700e..b76acae6a6ac 100644 --- a/drivers/iommu/intel-iommu.c +++ b/drivers/iommu/intel-iommu.c @@ -4922,16 +4922,24 @@ void intel_iommu_shutdown(void) if (no_iommu || dmar_disabled) return; + pr_warn("%s:%d\n", __func__, __LINE__); down_write(&dmar_global_lock); + pr_warn("%s:%d\n", __func__, __LINE__); /* Disable PMRs explicitly here. */ - for_each_iommu(iommu, drhd) + for_each_iommu(iommu, drhd) { + pr_warn("%s:%d\n", __func__, __LINE__); iommu_disable_protect_mem_regions(iommu); + pr_warn("%s:%d\n", __func__, __LINE__); + } + pr_warn("%s:%d\n", __func__, __LINE__); /* Make sure the IOMMUs are switched off */ intel_disable_iommus(); + pr_warn("%s:%d\n", __func__, __LINE__); up_write(&dmar_global_lock); + pr_warn("%s:%d\n", __func__, __LINE__); } static inline struct intel_iommu *dev_to_intel_iommu(struct device *dev) and the output shows that the for_each_iommu loop runs twice and the last pr_warn is reached, too. So the hang doesn't occur in intel_iommu_shutdown() but later. I don't know enough about x86 and iommus to judge what that means or even if this was a useful test. Best regards Uwe [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: Failure to shutdown/reboot with intel_iommu=on 2020-05-08 15:07 ` Joerg Roedel 2020-05-09 1:58 ` Lu Baolu @ 2020-05-11 13:43 ` Lenny Szubowicz 2020-05-11 19:58 ` Uwe Kleine-König 2020-05-12 13:34 ` Joerg Roedel 1 sibling, 2 replies; 10+ messages in thread From: Lenny Szubowicz @ 2020-05-11 13:43 UTC (permalink / raw) To: Joerg Roedel, Uwe Kleine-König Cc: linux-kernel, rafael.j.wysocki, tglx, x86, Lu Baolu On 5/8/20 11:07 AM, Joerg Roedel wrote: > + Baolu, Maintainer of Intel IOMMU > > Baolu, does that ring any bells? > > On Wed, May 06, 2020 at 04:46:02PM +0200, Uwe Kleine-König wrote: >> Hello, >> >> On my Lenovo T460p I cannot shutdown and reboot when the iommu is >> enabled. This is using linux 5.2.7 as provided by Debian, 5.6.4 has the >> same problem. Suspend/resume also fails; I suspect this is the same >> issue. >> >> When requesting power off the kernel messages just end with: >> >> sd 0:0:0:0: [sda] Synchronizing SCSI cache >> sd 0:0:0:0: [sda] Stopping disk >> e1000e: EEE TX LPI TIMER: 00000011 >> ACPI: Preparing to enter system sleep state S5 >> reboot: Power down >> acpi_power_off called >> >> (photo at https://www.kleine-koenig.org/tmp/uklsiommu.jpg in case I >> mistyped something. Full dmesg and lspci -vvv at >> https://www.kleine-koenig.org/tmp/uklsiommu.tar.gz with and without >> iommu enabled.) >> >> With the iommu disabled (CONFIG_INTEL_IOMMU_DEFAULT_ON unset or >> intel_iommu=off on cmdline) the machine just works as expected >> (including working suspend/resume). >> >> I already talked to tglx on irc but unfortunately no new insights >> resulted from that. >>> Any ideas how to fix or continue debugging the issue? >> >> Best regards >> Uwe > > I suspect that you have TPM 2.x functionality enabled in the BIOS/firmware. Unless you are actually using the TPM, try setting it to TPM 1.2 mode. I've seen an incompatiblity on other Lenovo laptops between using the IOMMU, TPM 2.x implementation in firmware, and shutdown/suspend. -Lenny. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Failure to shutdown/reboot with intel_iommu=on 2020-05-11 13:43 ` Lenny Szubowicz @ 2020-05-11 19:58 ` Uwe Kleine-König 2020-05-12 13:34 ` Joerg Roedel 1 sibling, 0 replies; 10+ messages in thread From: Uwe Kleine-König @ 2020-05-11 19:58 UTC (permalink / raw) To: Lenny Szubowicz, Joerg Roedel Cc: linux-kernel, rafael.j.wysocki, tglx, x86, Lu Baolu [-- Attachment #1.1: Type: text/plain, Size: 486 bytes --] Hello Lenny, On 5/11/20 3:43 PM, Lenny Szubowicz wrote: > I suspect that you have TPM 2.x functionality enabled in the BIOS/firmware. Indeed. > Unless you are actually using the TPM, try setting it to TPM 1.2 mode. > I've seen an incompatiblity on other Lenovo laptops between using the > IOMMU, TPM 2.x implementation in firmware, and shutdown/suspend. When setting it to TPM 1.2 reboot works again. Didn't test poweroff and suspend/resume yet. Best regards Uwe [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Failure to shutdown/reboot with intel_iommu=on 2020-05-11 13:43 ` Lenny Szubowicz 2020-05-11 19:58 ` Uwe Kleine-König @ 2020-05-12 13:34 ` Joerg Roedel 2020-05-12 20:00 ` Lenny Szubowicz 1 sibling, 1 reply; 10+ messages in thread From: Joerg Roedel @ 2020-05-12 13:34 UTC (permalink / raw) To: Lenny Szubowicz Cc: Uwe Kleine-König, linux-kernel, rafael.j.wysocki, tglx, x86, Lu Baolu On Mon, May 11, 2020 at 09:43:11AM -0400, Lenny Szubowicz wrote: > I suspect that you have TPM 2.x functionality enabled in the BIOS/firmware. > > Unless you are actually using the TPM, try setting it to TPM 1.2 mode. > I've seen an incompatiblity on other Lenovo laptops between using the > IOMMU, TPM 2.x implementation in firmware, and shutdown/suspend. Interesting, has this been debugged further into the TPM code? Joerg ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Failure to shutdown/reboot with intel_iommu=on 2020-05-12 13:34 ` Joerg Roedel @ 2020-05-12 20:00 ` Lenny Szubowicz 2020-05-12 21:13 ` Joerg Roedel 0 siblings, 1 reply; 10+ messages in thread From: Lenny Szubowicz @ 2020-05-12 20:00 UTC (permalink / raw) To: Joerg Roedel Cc: Uwe Kleine-König, linux-kernel, rafael.j.wysocki, tglx, x86, Lu Baolu On 5/12/20 9:34 AM, Joerg Roedel wrote: > On Mon, May 11, 2020 at 09:43:11AM -0400, Lenny Szubowicz wrote: >> I suspect that you have TPM 2.x functionality enabled in the BIOS/firmware. >> >> Unless you are actually using the TPM, try setting it to TPM 1.2 mode. >> I've seen an incompatiblity on other Lenovo laptops between using the >> IOMMU, TPM 2.x implementation in firmware, and shutdown/suspend. > > Interesting, has this been debugged further into the TPM code? > > > Joerg > I believe the problem is in the Lenovo firmware and not in the kernel. There are essentially two problems: 1. TPM 2.0 doesn't work when the IOMMU is enabled 2. Suspend/shutdown hangs when problem 1 is encountered on boot Lenovo's firmware implementation of TPM 2.0 functionality on some of their laptops uses DMA. When you ask the kernel to enable the IOMMU, this DMA access is correctly blocked by the IOMMU hardware. If you look at your dmesg log from when you have TPM 2.0 and the IOMMU enabled, there are TPM timeout messages that indicate the inability to initialize and use the TPM capability. The hang on shutdown or S3 suspend appears to be in firmware, i.e. after the kernel has transferred control back to the firmware. It makes no difference if the kernel actively shuts down the IOMMU before transferring control to the firmware on a suspend or shutdown. The hang still occurs. My guess is that the firmware wants to do some TPM related processing on shutdown and suspend and can't handle the TPM state that exists due to the startup failure. But that's just a guess. I don't know what the firmware is actually doing. Some Lenovo laptops provide an ACPI DMAR RMRR that identifies the memory range that the kernel should open up for permissable DMA access for this purpose. Unfortunately, the PCI device that performs these DMA operations is hidden from the kernel by the BIOS. Given that the associated PCI device is hidden, the Linux kernel does not act upon the associated DMAR RMRR. -Lenny. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Failure to shutdown/reboot with intel_iommu=on 2020-05-12 20:00 ` Lenny Szubowicz @ 2020-05-12 21:13 ` Joerg Roedel 0 siblings, 0 replies; 10+ messages in thread From: Joerg Roedel @ 2020-05-12 21:13 UTC (permalink / raw) To: Lenny Szubowicz Cc: Uwe Kleine-König, linux-kernel, rafael.j.wysocki, tglx, x86, Lu Baolu Hi Lenny, On Tue, May 12, 2020 at 04:00:26PM -0400, Lenny Szubowicz wrote: > Some Lenovo laptops provide an ACPI DMAR RMRR that identifies the memory > range that the kernel should open up for permissable DMA access > for this purpose. Unfortunately, the PCI device that performs these > DMA operations is hidden from the kernel by the BIOS. Given that the > associated PCI device is hidden, the Linux kernel does not act upon > the associated DMAR RMRR. That sounds aweful. We should add to the VT-d driver that it sets up RMRR mappings for request-ids which are not present as a PCI device, to fix the Laptops which have it. For the others, is the region the TPM talks to via DMA known so that we can add a quirk? Joerg ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2020-05-12 21:13 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-05-06 14:46 Failure to shutdown/reboot with intel_iommu=on Uwe Kleine-König 2020-05-08 15:07 ` Joerg Roedel 2020-05-09 1:58 ` Lu Baolu 2020-05-11 19:59 ` Uwe Kleine-König 2020-05-11 20:16 ` Uwe Kleine-König 2020-05-11 13:43 ` Lenny Szubowicz 2020-05-11 19:58 ` Uwe Kleine-König 2020-05-12 13:34 ` Joerg Roedel 2020-05-12 20:00 ` Lenny Szubowicz 2020-05-12 21:13 ` Joerg Roedel
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.
