[RFC] thunderbolt: Automatically authorize PCIe tunnels when IOMMU is active

[RFC] thunderbolt: Automatically authorize PCIe tunnels when IOMMU is active

[Mario Limonciello]

Historically TBT3 in Linux used "Thunderbolt security levels" as a primary
means of "security" against DMA attacks. This mean that users would need to
ack any device plugged in via userspace.  In ~2018 machines started to use
the IOMMU for protection, but instead of dropping security levels a
convoluted flow was introduced:
* User hotplugs device
* Driver discovers supported tunnels
* Driver emits a uevent to userspace that a PCIe tunnel is present
* Userspace reads 'iommu_dma_protection' attribute (which currently
  indicates an Intel IOMMU is present and was enabled pre-boot not that
  it's active "now")
* Based on that value userspace then authorizes automatically or prompts
  the user like how security level based support worked.

This is further confused because a pre-OS CM may authorize PCIe tunnels
and the Linux USB4 CM will re-use tunnels if the paths discovered are the
same. This means if you plug in a dock pre-boot it works in Linux
immediately, but if you plug it in after boot you need to authorize it with
userspace.

All the hand waving malarkey should be unnecessary when the IOMMU
translation layer is active.
* First detect that the firmware has allowed PCIe tunnels to be created.
* Then check if an IOMMU domain is assigned to the NHI.
  - If it is, set TBT security level to "NONE".
  - If it is not, leave things as before.
* Lastly when a port is scanned via `tb_scan_port`, check the security
  level.
* If it was set as "NONE", create the PCIe tunnel and update the
  'authorized' attribute to reflect this.

Link: https://lore.kernel.org/linux-iommu/20220315162455.5190-1-mario.limonciello@amd.com/T/#mbe71d35c27bb91f6f9781eba7676d38ca467f6d5
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
---
Remaining items to be done/investigated/discussed:
 * Documentation updates for this
 * Is this a good enough method to check IOMMU is active?
 * Should the IOMMU check be on PCIe root port for tunneling instead
   of the router? If so, need to discover it via the device link
   (possibly at time of device link creation).
 * Maybe create a TBT new security level definition "iommu" that
   functionally behaves same to "none" to make things clearer to userspace
   how the device was authorized.
 * Drop iommu_dma_protection attribute?
 drivers/thunderbolt/tb.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/drivers/thunderbolt/tb.c b/drivers/thunderbolt/tb.c
index cbd0ad85ffb1..eaa8429e0156 100644
--- a/drivers/thunderbolt/tb.c
+++ b/drivers/thunderbolt/tb.c
@@ -6,6 +6,7 @@
  * Copyright (C) 2019, Intel Corporation
  */
 
+#include <linux/iommu.h>
 #include <linux/slab.h>
 #include <linux/errno.h>
 #include <linux/delay.h>
@@ -18,6 +19,8 @@
 
 #define TB_TIMEOUT	100 /* ms */
 
+static int tb_tunnel_pci(struct tb *tb, struct tb_switch *sw);
+
 /**
  * struct tb_cm - Simple Thunderbolt connection manager
  * @tunnel_list: List of active tunnels
@@ -690,6 +693,16 @@ static void tb_scan_port(struct tb_port *port)
 	if (tcm->hotplug_active && tb_tunnel_usb3(sw->tb, sw))
 		tb_sw_warn(sw, "USB3 tunnel creation failed\n");
 
+	/*
+	 * Create PCIe tunnels if security has been turned off
+	 * (such as by IOMMU translation enabled)
+	 */
+	if (port->sw->tb->security_level == TB_SECURITY_NONE) {
+		if (tb_tunnel_pci(port->sw->tb, sw))
+			tb_sw_warn(sw, "PCIe tunnel creation failed\n");
+		else
+			sw->authorized = 1;
+	}
 	tb_add_dp_resources(sw);
 	tb_scan_switch(sw);
 }
@@ -1702,7 +1715,8 @@ struct tb *tb_probe(struct tb_nhi *nhi)
 		return NULL;
 
 	if (tb_acpi_may_tunnel_pcie())
-		tb->security_level = TB_SECURITY_USER;
+		tb->security_level = iommu_get_domain_for_dev(&nhi->pdev->dev) ?
+			TB_SECURITY_NONE : TB_SECURITY_USER;
 	else
 		tb->security_level = TB_SECURITY_NOPCIE;
 
-- 
2.34.1

Re: [RFC] thunderbolt: Automatically authorize PCIe tunnels when IOMMU is active

[Mika Westerberg]

Hi Mario,

On Tue, Mar 15, 2022 at 04:30:08PM -0500, Mario Limonciello wrote:
> Historically TBT3 in Linux used "Thunderbolt security levels" as a primary
> means of "security" against DMA attacks. This mean that users would need to
> ack any device plugged in via userspace.  In ~2018 machines started to use
> the IOMMU for protection, but instead of dropping security levels a
> convoluted flow was introduced:
> * User hotplugs device
> * Driver discovers supported tunnels
> * Driver emits a uevent to userspace that a PCIe tunnel is present
> * Userspace reads 'iommu_dma_protection' attribute (which currently
>   indicates an Intel IOMMU is present and was enabled pre-boot not that
>   it's active "now")
> * Based on that value userspace then authorizes automatically or prompts
>   the user like how security level based support worked.

There are legitimate reasons to disable PCIe tunneling even if the IOMMU
bits are in place. The ACPI _OSC allows the boot firmware to do so and
our "security levels" allows the userspace policy to do the same. I
would not like to change that unless absolutely necessary.

RE: [RFC] thunderbolt: Automatically authorize PCIe tunnels when IOMMU is active

[Limonciello, Mario]

[AMD Official Use Only]



> -----Original Message-----
> From: Mika Westerberg <mika.westerberg@linux.intel.com>
> Sent: Wednesday, March 16, 2022 01:30
> To: Limonciello, Mario <Mario.Limonciello@amd.com>
> Cc: Andreas Noever <andreas.noever@gmail.com>; Michael Jamet
> <michael.jamet@intel.com>; Yehezkel Bernat <YehezkelShB@gmail.com>;
> open list:THUNDERBOLT DRIVER <linux-usb@vger.kernel.org>; open list
> <linux-kernel@vger.kernel.org>
> Subject: Re: [RFC] thunderbolt: Automatically authorize PCIe tunnels when
> IOMMU is active
> 
> Hi Mario,
> 
> On Tue, Mar 15, 2022 at 04:30:08PM -0500, Mario Limonciello wrote:
> > Historically TBT3 in Linux used "Thunderbolt security levels" as a primary
> > means of "security" against DMA attacks. This mean that users would need
> to
> > ack any device plugged in via userspace.  In ~2018 machines started to use
> > the IOMMU for protection, but instead of dropping security levels a
> > convoluted flow was introduced:
> > * User hotplugs device
> > * Driver discovers supported tunnels
> > * Driver emits a uevent to userspace that a PCIe tunnel is present
> > * Userspace reads 'iommu_dma_protection' attribute (which currently
> >   indicates an Intel IOMMU is present and was enabled pre-boot not that
> >   it's active "now")
> > * Based on that value userspace then authorizes automatically or prompts
> >   the user like how security level based support worked.
> 
> There are legitimate reasons to disable PCIe tunneling even if the IOMMU
> bits are in place. The ACPI _OSC allows the boot firmware to do so and
> our "security levels" allows the userspace policy to do the same. I
> would not like to change that unless absolutely necessary.

Actually I intentionally left that in the RFC patch, to only do this based off
of tb_acpi_may_tunnel_pcie, so I think that should still work as you described
if boot firmware turned off PCIe tunneling.

Re: [RFC] thunderbolt: Automatically authorize PCIe tunnels when IOMMU is active

[Mika Westerberg]

On Wed, Mar 16, 2022 at 01:06:24PM +0000, Limonciello, Mario wrote:
> [AMD Official Use Only]
> 
> 
> 
> > -----Original Message-----
> > From: Mika Westerberg <mika.westerberg@linux.intel.com>
> > Sent: Wednesday, March 16, 2022 01:30
> > To: Limonciello, Mario <Mario.Limonciello@amd.com>
> > Cc: Andreas Noever <andreas.noever@gmail.com>; Michael Jamet
> > <michael.jamet@intel.com>; Yehezkel Bernat <YehezkelShB@gmail.com>;
> > open list:THUNDERBOLT DRIVER <linux-usb@vger.kernel.org>; open list
> > <linux-kernel@vger.kernel.org>
> > Subject: Re: [RFC] thunderbolt: Automatically authorize PCIe tunnels when
> > IOMMU is active
> > 
> > Hi Mario,
> > 
> > On Tue, Mar 15, 2022 at 04:30:08PM -0500, Mario Limonciello wrote:
> > > Historically TBT3 in Linux used "Thunderbolt security levels" as a primary
> > > means of "security" against DMA attacks. This mean that users would need
> > to
> > > ack any device plugged in via userspace.  In ~2018 machines started to use
> > > the IOMMU for protection, but instead of dropping security levels a
> > > convoluted flow was introduced:
> > > * User hotplugs device
> > > * Driver discovers supported tunnels
> > > * Driver emits a uevent to userspace that a PCIe tunnel is present
> > > * Userspace reads 'iommu_dma_protection' attribute (which currently
> > >   indicates an Intel IOMMU is present and was enabled pre-boot not that
> > >   it's active "now")
> > > * Based on that value userspace then authorizes automatically or prompts
> > >   the user like how security level based support worked.
> > 
> > There are legitimate reasons to disable PCIe tunneling even if the IOMMU
> > bits are in place. The ACPI _OSC allows the boot firmware to do so and
> > our "security levels" allows the userspace policy to do the same. I
> > would not like to change that unless absolutely necessary.
> 
> Actually I intentionally left that in the RFC patch, to only do this based off
> of tb_acpi_may_tunnel_pcie, so I think that should still work as you described
> if boot firmware turned off PCIe tunneling.

Right but if the user still wants to disable it, like say you are
travelling and you want to be sure that no PCIe devices get attached
while your laptop is charging from a public "charging station" (whatever
is the right term).

RE: [RFC] thunderbolt: Automatically authorize PCIe tunnels when IOMMU is active

[Limonciello, Mario]

[Public]

> > > IOMMU is active
> > >
> > > Hi Mario,
> > >
> > > On Tue, Mar 15, 2022 at 04:30:08PM -0500, Mario Limonciello wrote:
> > > > Historically TBT3 in Linux used "Thunderbolt security levels" as a primary
> > > > means of "security" against DMA attacks. This mean that users would
> need
> > > to
> > > > ack any device plugged in via userspace.  In ~2018 machines started to
> use
> > > > the IOMMU for protection, but instead of dropping security levels a
> > > > convoluted flow was introduced:
> > > > * User hotplugs device
> > > > * Driver discovers supported tunnels
> > > > * Driver emits a uevent to userspace that a PCIe tunnel is present
> > > > * Userspace reads 'iommu_dma_protection' attribute (which currently
> > > >   indicates an Intel IOMMU is present and was enabled pre-boot not
> that
> > > >   it's active "now")
> > > > * Based on that value userspace then authorizes automatically or
> prompts
> > > >   the user like how security level based support worked.
> > >
> > > There are legitimate reasons to disable PCIe tunneling even if the
> IOMMU
> > > bits are in place. The ACPI _OSC allows the boot firmware to do so and
> > > our "security levels" allows the userspace policy to do the same. I
> > > would not like to change that unless absolutely necessary.
> >
> > Actually I intentionally left that in the RFC patch, to only do this based off
> > of tb_acpi_may_tunnel_pcie, so I think that should still work as you
> described
> > if boot firmware turned off PCIe tunneling.
> 
> Right but if the user still wants to disable it, like say you are
> travelling and you want to be sure that no PCIe devices get attached
> while your laptop is charging from a public "charging station" (whatever
> is the right term).

So wouldn't you flip the default in BIOS setup to disable PCIe tunnels then for
this use case?

Otherwise with how it is today you end up with the PCIe tunnel created in the
boot FW and then coming into the OS if it's the same path the tunnel stays
in place with no opportunity for userspace to authorize it, no?

Re: [RFC] thunderbolt: Automatically authorize PCIe tunnels when IOMMU is active

[Mika Westerberg]

On Wed, Mar 16, 2022 at 01:48:49PM +0000, Limonciello, Mario wrote:
> [Public]
> 
> > > > IOMMU is active
> > > >
> > > > Hi Mario,
> > > >
> > > > On Tue, Mar 15, 2022 at 04:30:08PM -0500, Mario Limonciello wrote:
> > > > > Historically TBT3 in Linux used "Thunderbolt security levels" as a primary
> > > > > means of "security" against DMA attacks. This mean that users would
> > need
> > > > to
> > > > > ack any device plugged in via userspace.  In ~2018 machines started to
> > use
> > > > > the IOMMU for protection, but instead of dropping security levels a
> > > > > convoluted flow was introduced:
> > > > > * User hotplugs device
> > > > > * Driver discovers supported tunnels
> > > > > * Driver emits a uevent to userspace that a PCIe tunnel is present
> > > > > * Userspace reads 'iommu_dma_protection' attribute (which currently
> > > > >   indicates an Intel IOMMU is present and was enabled pre-boot not
> > that
> > > > >   it's active "now")
> > > > > * Based on that value userspace then authorizes automatically or
> > prompts
> > > > >   the user like how security level based support worked.
> > > >
> > > > There are legitimate reasons to disable PCIe tunneling even if the
> > IOMMU
> > > > bits are in place. The ACPI _OSC allows the boot firmware to do so and
> > > > our "security levels" allows the userspace policy to do the same. I
> > > > would not like to change that unless absolutely necessary.
> > >
> > > Actually I intentionally left that in the RFC patch, to only do this based off
> > > of tb_acpi_may_tunnel_pcie, so I think that should still work as you
> > described
> > > if boot firmware turned off PCIe tunneling.
> > 
> > Right but if the user still wants to disable it, like say you are
> > travelling and you want to be sure that no PCIe devices get attached
> > while your laptop is charging from a public "charging station" (whatever
> > is the right term).
> 
> So wouldn't you flip the default in BIOS setup to disable PCIe tunnels then for
> this use case?

What if you are on Chromebook? Or something where this is not user
configurable?

> Otherwise with how it is today you end up with the PCIe tunnel created in the
> boot FW and then coming into the OS if it's the same path the tunnel stays
> in place with no opportunity for userspace to authorize it, no?

The boot FW does not need to support CM capabilites nor does it need to
provide the ACPI _OSC.

RE: [RFC] thunderbolt: Automatically authorize PCIe tunnels when IOMMU is active

[Limonciello, Mario]

[Public]

> > > > Actually I intentionally left that in the RFC patch, to only do this based
> off
> > > > of tb_acpi_may_tunnel_pcie, so I think that should still work as you
> > > described
> > > > if boot firmware turned off PCIe tunneling.
> > >
> > > Right but if the user still wants to disable it, like say you are
> > > travelling and you want to be sure that no PCIe devices get attached
> > > while your laptop is charging from a public "charging station" (whatever
> > > is the right term).
> >
> > So wouldn't you flip the default in BIOS setup to disable PCIe tunnels then
> for
> > this use case?
> 
> What if you are on Chromebook? Or something where this is not user
> configurable?
> 
> > Otherwise with how it is today you end up with the PCIe tunnel created in
> the
> > boot FW and then coming into the OS if it's the same path the tunnel stays
> > in place with no opportunity for userspace to authorize it, no?
> 
> The boot FW does not need to support CM capabilites nor does it need to
> provide the ACPI _OSC.

Ah right - my thoughts were entirely UEFI firmware centric.  Chromebooks don't
have BIOS setup, nor do they all have the USB4 _OSC.

Then yes I agree we do need to "keep" this authorization decision in userspace.