[PATCH v2] PCI: Add DMA alias for Intel Corporation 8 Series HECI

[PATCH v2] PCI: Add DMA alias for Intel Corporation 8 Series HECI

[Francisco Blas Izquierdo Riera]

PCI: Add function 7 DMA alias quirk for Intel Corporation 8 Series HECI.

Intel Corporation 8 Series HECIs include support for a CRB TPM 2.0
device. When the device is enabled on the BIOS, the TPM 2.0 device is
detected but the IOMMU prevents it from being accessed.

Even on a computer with a fixed DMAR table, device initialization
fails with DMA errors:
  DMAR: DRHD: handling fault status reg 3
  DMAR: [DMA Read NO_PASID] Request device [00:16.7] fault addr 0xdceff000 [fault reason 0x06] PTE Read access is not set
  DMAR: DRHD: handling fault status reg 2
  DMAR: [DMA Write NO_PASID] Request device [00:16.7] fault addr 0xdceff000 [fault reason 0x05] PTE Write access is not set
  DMAR: DRHD: handling fault status reg 2
  DMAR: [DMA Write NO_PASID] Request device [00:16.7] fault addr 0xdceff000 [fault reason 0x05] PTE Write access is not set
  tpm tpm0: Operation Timed out
  DMAR: DRHD: handling fault status reg 3
  tpm tpm0: Operation Timed out
  tpm_crb: probe of MSFT0101:00 failed with error -62

After patching the DMAR table and adding this patch, the TPM 2.0
device is initialized correctly and no DMA errors appear. Accessing
the TPM 2.0 PCR banks also works as expected.

Since most Haswell computers supporting this do not seem to have a
valid DMAR table patching the table with an appropriate RMRR is
usually also needed. I have published a blogpost describing the
process.

This patch currently adds the alias only for function 0. Since this
is the only function I have seen provided by the HECI on actual
hardware.


V2: Resent using a sendmail to fix tab mangling made by Thunderbird.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=108251
Link: https://klondike.es/klog/2022/11/21/patching-the-acpi-dmar-table-to-allow-tpm2-0/
Reported-by: Pierre Chifflier <chifflier@gmail.com>
Tested-by: Francisco Blas Izquierdo Riera (klondike) <klondike@klondike.es>
Signed-off-by: Francisco Blas Izquierdo Riera <klondike@klondike.es>
Suggested-by: Baolu Lu <baolu.lu@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: stable@vger.kernel.org

--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -4162,6 +4162,22 @@
 			 0x0122, /* Plextor M6E (Marvell 88SS9183)*/
 			 quirk_dma_func1_alias);
 
+static void quirk_dma_func7_alias(struct pci_dev *dev)
+{
+	if (PCI_FUNC(dev->devfn) == 0)
+		pci_add_dma_alias(dev, PCI_DEVFN(PCI_SLOT(dev->devfn), 7), 1);
+}
+
+/*
+ * Certain HECIs in Haswell systems support TPM 2.0. Unfortunately they
+ * perform DMA using the hidden function 7. Fixing this requires this
+ * alias and a patch of the DMAR ACPI table to include the appropriate
+ *  MTRR.
+ * https://bugzilla.kernel.org/show_bug.cgi?id=108251
+ */
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x9c3a,
+			 quirk_dma_func7_alias);
+
 /*
  * Some devices DMA with the wrong devfn, not just the wrong function.
  * quirk_fixed_dma_alias() uses this table to create fixed aliases, where

Re: [PATCH v2] PCI: Add DMA alias for Intel Corporation 8 Series HECI

[Bjorn Helgaas]

[+to Jarkko, Tomas, Alexander; can any of you confirm this behavior of
the HECI device?  Are there any other HECI devices that should be
included in this quirk?]

[+cc David, Lu, iommu list]

On Mon, Nov 21, 2022 at 05:40:37PM +0100, Francisco Blas Izquierdo Riera wrote:
> PCI: Add function 7 DMA alias quirk for Intel Corporation 8 Series HECI.
> 
> Intel Corporation 8 Series HECIs include support for a CRB TPM 2.0
> device. When the device is enabled on the BIOS, the TPM 2.0 device is
> detected but the IOMMU prevents it from being accessed.
> 
> Even on a computer with a fixed DMAR table, device initialization
> fails with DMA errors:
>   DMAR: DRHD: handling fault status reg 3
>   DMAR: [DMA Read NO_PASID] Request device [00:16.7] fault addr 0xdceff000 [fault reason 0x06] PTE Read access is not set
>   DMAR: DRHD: handling fault status reg 2
>   DMAR: [DMA Write NO_PASID] Request device [00:16.7] fault addr 0xdceff000 [fault reason 0x05] PTE Write access is not set
>   DMAR: DRHD: handling fault status reg 2
>   DMAR: [DMA Write NO_PASID] Request device [00:16.7] fault addr 0xdceff000 [fault reason 0x05] PTE Write access is not set
>   tpm tpm0: Operation Timed out
>   DMAR: DRHD: handling fault status reg 3
>   tpm tpm0: Operation Timed out
>   tpm_crb: probe of MSFT0101:00 failed with error -62
> 
> After patching the DMAR table and adding this patch, the TPM 2.0
> device is initialized correctly and no DMA errors appear. Accessing
> the TPM 2.0 PCR banks also works as expected.

Francisco, is the DMAR patch *also* required?  We have several similar
quirks for devices that use unexpected function numbers, but I don't
remember any that require DMAR changes.

A kernel quirk requires no action on the part of users, so that's
easy.  But I don't think it's practical for ordinary users to extract
the DMAR, disassemble it, patch it, recompile it, and update the
initramfs as described in your blog post.

Is there a way to add a kernel quirk to accomplish the same DMAR
override?

> Since most Haswell computers supporting this do not seem to have a
> valid DMAR table patching the table with an appropriate RMRR is
> usually also needed. I have published a blogpost describing the
> process.
> 
> This patch currently adds the alias only for function 0. Since this
> is the only function I have seen provided by the HECI on actual
> hardware.
> 
> 
> V2: Resent using a sendmail to fix tab mangling made by Thunderbird.
> 
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=108251
> Link: https://klondike.es/klog/2022/11/21/patching-the-acpi-dmar-table-to-allow-tpm2-0/
> Reported-by: Pierre Chifflier <chifflier@gmail.com>
> Tested-by: Francisco Blas Izquierdo Riera (klondike) <klondike@klondike.es>
> Signed-off-by: Francisco Blas Izquierdo Riera <klondike@klondike.es>
> Suggested-by: Baolu Lu <baolu.lu@linux.intel.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: stable@vger.kernel.org
> 
> --- a/drivers/pci/quirks.c
> +++ b/drivers/pci/quirks.c
> @@ -4162,6 +4162,22 @@
>  			 0x0122, /* Plextor M6E (Marvell 88SS9183)*/
>  			 quirk_dma_func1_alias);
>  
> +static void quirk_dma_func7_alias(struct pci_dev *dev)
> +{
> +	if (PCI_FUNC(dev->devfn) == 0)
> +		pci_add_dma_alias(dev, PCI_DEVFN(PCI_SLOT(dev->devfn), 7), 1);
> +}
> +
> +/*
> + * Certain HECIs in Haswell systems support TPM 2.0. Unfortunately they
> + * perform DMA using the hidden function 7. Fixing this requires this
> + * alias and a patch of the DMAR ACPI table to include the appropriate
> + *  MTRR.
> + * https://bugzilla.kernel.org/show_bug.cgi?id=108251
> + */
> +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x9c3a,
> +			 quirk_dma_func7_alias);
> +
>  /*
>   * Some devices DMA with the wrong devfn, not just the wrong function.
>   * quirk_fixed_dma_alias() uses this table to create fixed aliases, where

Re: [PATCH v2] PCI: Add DMA alias for Intel Corporation 8 Series HECI

[Robin Murphy]

On 2022-11-21 17:54, Bjorn Helgaas wrote:
> [+to Jarkko, Tomas, Alexander; can any of you confirm this behavior of
> the HECI device?  Are there any other HECI devices that should be
> included in this quirk?]
> 
> [+cc David, Lu, iommu list]
> 
> On Mon, Nov 21, 2022 at 05:40:37PM +0100, Francisco Blas Izquierdo Riera wrote:
>> PCI: Add function 7 DMA alias quirk for Intel Corporation 8 Series HECI.
>>
>> Intel Corporation 8 Series HECIs include support for a CRB TPM 2.0
>> device. When the device is enabled on the BIOS, the TPM 2.0 device is
>> detected but the IOMMU prevents it from being accessed.
>>
>> Even on a computer with a fixed DMAR table, device initialization
>> fails with DMA errors:
>>    DMAR: DRHD: handling fault status reg 3
>>    DMAR: [DMA Read NO_PASID] Request device [00:16.7] fault addr 0xdceff000 [fault reason 0x06] PTE Read access is not set
>>    DMAR: DRHD: handling fault status reg 2
>>    DMAR: [DMA Write NO_PASID] Request device [00:16.7] fault addr 0xdceff000 [fault reason 0x05] PTE Write access is not set
>>    DMAR: DRHD: handling fault status reg 2
>>    DMAR: [DMA Write NO_PASID] Request device [00:16.7] fault addr 0xdceff000 [fault reason 0x05] PTE Write access is not set
>>    tpm tpm0: Operation Timed out
>>    DMAR: DRHD: handling fault status reg 3
>>    tpm tpm0: Operation Timed out
>>    tpm_crb: probe of MSFT0101:00 failed with error -62
>>
>> After patching the DMAR table and adding this patch, the TPM 2.0
>> device is initialized correctly and no DMA errors appear. Accessing
>> the TPM 2.0 PCR banks also works as expected.
> 
> Francisco, is the DMAR patch *also* required?  We have several similar
> quirks for devices that use unexpected function numbers, but I don't
> remember any that require DMAR changes.
> 
> A kernel quirk requires no action on the part of users, so that's
> easy.  But I don't think it's practical for ordinary users to extract
> the DMAR, disassemble it, patch it, recompile it, and update the
> initramfs as described in your blog post.
> 
> Is there a way to add a kernel quirk to accomplish the same DMAR
> override?

Yes, in principle it's possible to quirk a reserved region for a 
particular device - like intel-iommu does for ISA bridges for instance - 
however figuring out exactly *where* to reserve might be a lot more 
tricky if it's not in a fixed place. The other possibility is to use 
.def_domain_type to quirk the entire device to use an identity domain - 
like for certain integrated GPUs - but that might depend on the scope of 
how much you want to trust it and whether there are also other devices 
in the same IOMMU group.

If anything I'm not sure that the PCI DMA alias quirk really sounds like 
the most correct thing to do, if the hidden function is doing its own 
DMA unrelated to whatever function 0 and its driver might do. If I'm 
understanding the situation correctly, a quirk for the ACPI TPM device 
in probe_acpi_namespace_devices() (assuming it doesn't have an ANDD 
entry already) might be more elegant (but admittedly more work).

Robin.

>> Since most Haswell computers supporting this do not seem to have a
>> valid DMAR table patching the table with an appropriate RMRR is
>> usually also needed. I have published a blogpost describing the
>> process.
>>
>> This patch currently adds the alias only for function 0. Since this
>> is the only function I have seen provided by the HECI on actual
>> hardware.
>>
>>
>> V2: Resent using a sendmail to fix tab mangling made by Thunderbird.
>>
>> Link: https://bugzilla.kernel.org/show_bug.cgi?id=108251
>> Link: https://klondike.es/klog/2022/11/21/patching-the-acpi-dmar-table-to-allow-tpm2-0/
>> Reported-by: Pierre Chifflier <chifflier@gmail.com>
>> Tested-by: Francisco Blas Izquierdo Riera (klondike) <klondike@klondike.es>
>> Signed-off-by: Francisco Blas Izquierdo Riera <klondike@klondike.es>
>> Suggested-by: Baolu Lu <baolu.lu@linux.intel.com>
>> Cc: Bjorn Helgaas <bhelgaas@google.com>
>> Cc: stable@vger.kernel.org
>>
>> --- a/drivers/pci/quirks.c
>> +++ b/drivers/pci/quirks.c
>> @@ -4162,6 +4162,22 @@
>>   			 0x0122, /* Plextor M6E (Marvell 88SS9183)*/
>>   			 quirk_dma_func1_alias);
>>   
>> +static void quirk_dma_func7_alias(struct pci_dev *dev)
>> +{
>> +	if (PCI_FUNC(dev->devfn) == 0)
>> +		pci_add_dma_alias(dev, PCI_DEVFN(PCI_SLOT(dev->devfn), 7), 1);
>> +}
>> +
>> +/*
>> + * Certain HECIs in Haswell systems support TPM 2.0. Unfortunately they
>> + * perform DMA using the hidden function 7. Fixing this requires this
>> + * alias and a patch of the DMAR ACPI table to include the appropriate
>> + *  MTRR.
>> + * https://bugzilla.kernel.org/show_bug.cgi?id=108251
>> + */
>> +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x9c3a,
>> +			 quirk_dma_func7_alias);
>> +
>>   /*
>>    * Some devices DMA with the wrong devfn, not just the wrong function.
>>    * quirk_fixed_dma_alias() uses this table to create fixed aliases, where
> 

Re: [PATCH v2] PCI: Add DMA alias for Intel Corporation 8 Series HECI

[klondike]

[-- Attachment #1.1.1: Type: text/plain, Size: 1609 bytes --]

El 21/11/22 a las 20:39, Robin Murphy escribió:
> Yes, in principle it's possible to quirk a reserved region for a particular device - like intel-iommu does for ISA bridges for instance - however figuring out exactly *where* to reserve might be a lot more tricky if it's not in a fixed place. The other possibility is to use .def_domain_type to quirk the entire device to use an identity domain - like for certain integrated GPUs - but that might depend on the scope of how much you want to trust it and whether there are also other devices in the same IOMMU group.

Well, in theory you can use the TPM2 ACPI table to find the (physical) address of the device. And identity mapping for the domain is then needed. The real issue is that function 7 of the HECI is hidden which is why just patching the ACPI DMAR table doesn't solve it.

On the computers I was testing the mapping is fixed though but that might not always be the case.

> If anything I'm not sure that the PCI DMA alias quirk really sounds like the most correct thing to do, if the hidden function is doing its own DMA unrelated to whatever function 0 and its driver might do. If I'm understanding the situation correctly, a quirk for the ACPI TPM device in probe_acpi_namespace_devices() (assuming it doesn't have an ANDD entry already) might be more elegant (but admittedly more work).

The ACPI tables I saw did not have any ANDD devices and although my first patch used the ANDD+MTRR approach I was asked to drop that as no further support to MTRRs was going to be added and linux does not support ANDD based MTRRs.

Klondike


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 82721 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: [PATCH v2] PCI: Add DMA alias for Intel Corporation 8 Series HECI

[klondike]

[-- Attachment #1.1.1: Type: text/plain, Size: 715 bytes --]

El 21/11/22 a las 18:54, Bjorn Helgaas escribió:
> Francisco, is the DMAR patch*also*  required?  We have several similar
> quirks for devices that use unexpected function numbers, but I don't
> remember any that require DMAR changes.

At least on the Lenovos X240 I was addressing this on it is. The bios (I installed the latest) does not add the RMRR for the fwTPM on the HECI.

> Is there a way to add a kernel quirk to accomplish the same DMAR
> override?

I think you could add a quirk depending on both, ACPI device and PCI device. If you see a MSFT0101:00 device on a 8 Series HECI, you will need to add a mapping for function 7 of the HECI for the memory range used by the MSFT0101:00 device.


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 82721 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: [PATCH v2] PCI: Add DMA alias for Intel Corporation 8 Series HECI

[Bjorn Helgaas]

[+cc Jarkko, tpm_crb author]

On Mon, Nov 21, 2022 at 05:40:37PM +0100, Francisco Blas Izquierdo Riera wrote:
> PCI: Add function 7 DMA alias quirk for Intel Corporation 8 Series HECI.
> 
> Intel Corporation 8 Series HECIs include support for a CRB TPM 2.0
> device. When the device is enabled on the BIOS, the TPM 2.0 device is
> detected but the IOMMU prevents it from being accessed.
> 
> Even on a computer with a fixed DMAR table, device initialization
> fails with DMA errors:
>   DMAR: DRHD: handling fault status reg 3
>   DMAR: [DMA Read NO_PASID] Request device [00:16.7] fault addr 0xdceff000 [fault reason 0x06] PTE Read access is not set
>   DMAR: DRHD: handling fault status reg 2
>   DMAR: [DMA Write NO_PASID] Request device [00:16.7] fault addr 0xdceff000 [fault reason 0x05] PTE Write access is not set
>   DMAR: DRHD: handling fault status reg 2
>   DMAR: [DMA Write NO_PASID] Request device [00:16.7] fault addr 0xdceff000 [fault reason 0x05] PTE Write access is not set
>   tpm tpm0: Operation Timed out
>   DMAR: DRHD: handling fault status reg 3
>   tpm tpm0: Operation Timed out
>   tpm_crb: probe of MSFT0101:00 failed with error -62
> 
> After patching the DMAR table and adding this patch, the TPM 2.0
> device is initialized correctly and no DMA errors appear. Accessing
> the TPM 2.0 PCR banks also works as expected.
> 
> Since most Haswell computers supporting this do not seem to have a
> valid DMAR table patching the table with an appropriate RMRR is
> usually also needed. I have published a blogpost describing the
> process.
> 
> This patch currently adds the alias only for function 0. Since this
> is the only function I have seen provided by the HECI on actual
> hardware.
> 
> 
> V2: Resent using a sendmail to fix tab mangling made by Thunderbird.
> 
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=108251
> Link: https://klondike.es/klog/2022/11/21/patching-the-acpi-dmar-table-to-allow-tpm2-0/
> Reported-by: Pierre Chifflier <chifflier@gmail.com>
> Tested-by: Francisco Blas Izquierdo Riera (klondike) <klondike@klondike.es>
> Signed-off-by: Francisco Blas Izquierdo Riera <klondike@klondike.es>
> Suggested-by: Baolu Lu <baolu.lu@linux.intel.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: stable@vger.kernel.org

Given that significant additional effort to override the DMAR table is
required before this quirk is useful, I'm deferring this quirk for
now.  People willing to recompile the DMAR will probably also be able
to apply the quirk and rebuild the kernel.

Does the TPM work under Windows?  If so, it would suggest that there's
a different way to use it that doesn't require the quirk or the DMAR
override.  Or maybe it only works on Windows without the IOMMU being
enabled?

Naive question: apparently the TPM is doing DMA reads/writes.  I see
tpm_crb.c doing MMIO mappings (ioremap()), but I don't see any DMA
mappings.  Is that implicit or done elsewhere?

Bjorn

> --- a/drivers/pci/quirks.c
> +++ b/drivers/pci/quirks.c
> @@ -4162,6 +4162,22 @@
>  			 0x0122, /* Plextor M6E (Marvell 88SS9183)*/
>  			 quirk_dma_func1_alias);
>  
> +static void quirk_dma_func7_alias(struct pci_dev *dev)
> +{
> +	if (PCI_FUNC(dev->devfn) == 0)
> +		pci_add_dma_alias(dev, PCI_DEVFN(PCI_SLOT(dev->devfn), 7), 1);
> +}
> +
> +/*
> + * Certain HECIs in Haswell systems support TPM 2.0. Unfortunately they
> + * perform DMA using the hidden function 7. Fixing this requires this
> + * alias and a patch of the DMAR ACPI table to include the appropriate
> + *  MTRR.
> + * https://bugzilla.kernel.org/show_bug.cgi?id=108251
> + */
> +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x9c3a,
> +			 quirk_dma_func7_alias);
> +
>  /*
>   * Some devices DMA with the wrong devfn, not just the wrong function.
>   * quirk_fixed_dma_alias() uses this table to create fixed aliases, where

Re: [PATCH v2] PCI: Add DMA alias for Intel Corporation 8 Series HECI

[Jarkko Sakkinen]

On Tue, Nov 29, 2022 at 12:27:56PM -0600, Bjorn Helgaas wrote:
> Does the TPM work under Windows?  If so, it would suggest that there's
> a different way to use it that doesn't require the quirk or the DMAR
> override.  Or maybe it only works on Windows without the IOMMU being
> enabled?

Haswell was the first generation having firmware TPM, and had fTPM
implementation that was flakky in many ways (e.g. Intel TXT still required
a discrete TPM so servers never used it). We have some workarounds for it
already in place.

I'm almost 100% confident nobody seriously tried to use it on server
side and/or together with MMIO.

> Naive question: apparently the TPM is doing DMA reads/writes.  I see
> tpm_crb.c doing MMIO mappings (ioremap()), but I don't see any DMA
> mappings.  Is that implicit or done elsewhere?

Firmware does this. Kernel does not and should not care how it does
it because it is not part of the specification [1]. Kernel cares only
of getting a buffer pointed out by the firmware.

[1] Section 6.5.3 in https://trustedcomputinggroup.org/resource/pc-client-platform-tpm-profile-ptp-specification/

BR, Jarkko