[RFC PATCH v2 0/6] Towards a shared TSM sysfs-ABI for Confidential Computing

[RFC PATCH v2 0/6] Towards a shared TSM sysfs-ABI for Confidential Computing

[Dan Williams]

Here is a revised attempt at creating a shared sysfs-ABI for the concept
of a TSM (TEE Security Manager) as described by PCIe TDISP (PCIe 6.2
Section 11 TEE Device Interface Security Protocol). It remains an RFC
until at least one vendor (Intel, AMD, Rivos...) completes integration
with their low level TSM driver. I am actively working on that with Hao
Wu and Yilun Xu, but if another vendor adopts this before us, great.

Changes since v1: [1]
* Major simplifications:
  * Drop the 'struct pci_tsm_req' concept (Yilun), but keep a common
    @exec entry point from the PCI core to the low level driver.
  * Drop Link IDE and related sysfs attributes (Alexey). This
    sophistication may come back later, but no need to tackle that
    complexity now.
  * Move policy choice of requiring native CMA before TSM connection
    to userspace policy. This removes the need to build on top of the
    moving CMA baseline, and these series can now be considered on
    indpendent timelines.
* Create a guest/ vs host/ split in drivers/virt/coco/ (Sathya)
* Require a parent device for the common TSM class device (Jonathan)
* Create a 'tdx' virtual bus and 'tdx_tsm' device to parent the TSM
  class device
* Create a 'tdx_tsm' for the low-level TDX calls
* Rebase on v6.9-rc1 that includes a DEFINE_SYSFS_GROUP_VISIBLE()
* Cleanup usage of __free() to match the proposed style guide [2]
  (Jonathan)
* Cleanup, clarifications, and fixes (Kevin)
* Improve the cover letter prose below (Bjorn, Kevin)

[1]: http://lore.kernel.org/r/170660662589.224441.11503798303914595072.stgit@dwillia2-xfh.jf.intel.com
[2]: http://lore.kernel.org/r/171140738438.1574931.15717256954707430472.stgit@dwillia2-xfh.jf.intel.com

Confidential Computing (CC) introduces the concept of hardware protected
(integrity and confidentiality) guest private memory. The next phase of
that journey is private memory access for guest assigned devices. To
date, assigned devices for CC guests are constrained to accessing shared
memory, unprotected clear-text memory. That mode incurs a bounce buffer
performance penalty as every DMA (direct-memory-access) performed by the
device must be later copied from shared-to-private memory for
device-write and private-to-shared copies for device-reads.

The PCIe TEE Device Interface Security Protocol (TDISP) arranges for
devices to be permitted to DMA to private memory directly, but it
requires significant infrastructure to authenticate, validate, and
provision a virtual-device interface to be used in CC guest.

TDISP specifies a TEE Security Manager (TSM) as a platform agent that
can manage the IOMMU, PCI host, and endpoint Device Security Manager
capabilities to convert an guest assigned device (physical function or
sriov-virtual function) into private mode operation.

What follows is common shared infrastructure for the PCI core to
interface with the platform TSM and a TDX as an example low level
consumer of these core capabilities.

Enable the PCI core to export a "connect" verb via sysfs for a given
device which, when the low level platform implementation is added,
arranges for the device to be authenticated and its link protected by
encryption and integrity checks.

---

Dan Williams (6):
      configfs-tsm: Namespace TSM report symbols
      coco/guest: Move shared guest CC infrastructure to drivers/virt/coco/guest/
      x86/tdx: Introduce a "tdx" subsystem and "tsm" device
      coco/tsm: Introduce a class device for TEE Security Managers
      PCI/TSM: Authenticate devices via platform TSM
      tdx_tsm: TEE Security Manager driver for TDX


 Documentation/ABI/testing/sysfs-bus-pci |   46 +++++
 MAINTAINERS                             |    7 +
 arch/x86/include/asm/shared/tdx.h       |    3 
 arch/x86/virt/vmx/tdx/tdx.c             |   70 ++++++++
 drivers/pci/Kconfig                     |   13 +
 drivers/pci/Makefile                    |    2 
 drivers/pci/pci-sysfs.c                 |    4 
 drivers/pci/pci.h                       |   10 +
 drivers/pci/probe.c                     |    1 
 drivers/pci/remove.c                    |    1 
 drivers/pci/tsm.c                       |  270 +++++++++++++++++++++++++++++++
 drivers/virt/coco/Kconfig               |    8 -
 drivers/virt/coco/Makefile              |    3 
 drivers/virt/coco/guest/Kconfig         |    7 +
 drivers/virt/coco/guest/Makefile        |    2 
 drivers/virt/coco/guest/tsm_report.c    |   32 ++--
 drivers/virt/coco/host/Kconfig          |   12 +
 drivers/virt/coco/host/Makefile         |    8 +
 drivers/virt/coco/host/tdx_tsm.c        |   68 ++++++++
 drivers/virt/coco/host/tsm-core.c       |  131 +++++++++++++++
 drivers/virt/coco/sev-guest/sev-guest.c |    8 -
 drivers/virt/coco/tdx-guest/tdx-guest.c |    8 -
 include/linux/pci-tsm.h                 |   80 +++++++++
 include/linux/pci.h                     |   11 +
 include/linux/tsm.h                     |   31 ++--
 include/uapi/linux/pci_regs.h           |    4 
 26 files changed, 795 insertions(+), 45 deletions(-)
 create mode 100644 drivers/pci/tsm.c
 create mode 100644 drivers/virt/coco/guest/Kconfig
 create mode 100644 drivers/virt/coco/guest/Makefile
 rename drivers/virt/coco/{tsm.c => guest/tsm_report.c} (92%)
 create mode 100644 drivers/virt/coco/host/Kconfig
 create mode 100644 drivers/virt/coco/host/Makefile
 create mode 100644 drivers/virt/coco/host/tdx_tsm.c
 create mode 100644 drivers/virt/coco/host/tsm-core.c
 create mode 100644 include/linux/pci-tsm.h

base-commit: 4cece764965020c22cff7665b18a012006359095

[RFC PATCH v2 1/6] configfs-tsm: Namespace TSM report symbols

[Dan Williams]

In preparation for new + common TSM (TEE Security Manager)
infrastructure, namespace the TSM report symbols in tsm.h with an
_REPORT suffix to differentiate them from other incoming tsm symbols.

Cc: Wu Hao <hao.wu@intel.com>
Cc: Yilun Xu <yilun.xu@intel.com>
Cc: Samuel Ortiz <sameo@rivosinc.com>
Cc: Alexey Kardashevskiy <aik@amd.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
 drivers/virt/coco/sev-guest/sev-guest.c |    8 ++++----
 drivers/virt/coco/tdx-guest/tdx-guest.c |    8 ++++----
 drivers/virt/coco/tsm.c                 |   24 ++++++++++++------------
 include/linux/tsm.h                     |   24 ++++++++++++------------
 4 files changed, 32 insertions(+), 32 deletions(-)

diff --git a/drivers/virt/coco/sev-guest/sev-guest.c b/drivers/virt/coco/sev-guest/sev-guest.c
index 87f241825bc3..ce60e3ce8aa3 100644
--- a/drivers/virt/coco/sev-guest/sev-guest.c
+++ b/drivers/virt/coco/sev-guest/sev-guest.c
@@ -786,7 +786,7 @@ struct snp_msg_cert_entry {
 static int sev_report_new(struct tsm_report *report, void *data)
 {
 	struct snp_msg_cert_entry *cert_table;
-	struct tsm_desc *desc = &report->desc;
+	struct tsm_report_desc *desc = &report->desc;
 	struct snp_guest_dev *snp_dev = data;
 	struct snp_msg_report_resp_hdr hdr;
 	const u32 report_size = SZ_4K;
@@ -885,14 +885,14 @@ static int sev_report_new(struct tsm_report *report, void *data)
 	return 0;
 }
 
-static const struct tsm_ops sev_tsm_ops = {
+static const struct tsm_report_ops sev_tsm_report_ops = {
 	.name = KBUILD_MODNAME,
 	.report_new = sev_report_new,
 };
 
 static void unregister_sev_tsm(void *data)
 {
-	tsm_unregister(&sev_tsm_ops);
+	tsm_report_unregister(&sev_tsm_ops);
 }
 
 static int __init sev_guest_probe(struct platform_device *pdev)
@@ -968,7 +968,7 @@ static int __init sev_guest_probe(struct platform_device *pdev)
 	snp_dev->input.resp_gpa = __pa(snp_dev->response);
 	snp_dev->input.data_gpa = __pa(snp_dev->certs_data);
 
-	ret = tsm_register(&sev_tsm_ops, snp_dev, &tsm_report_extra_type);
+	ret = tsm_report_register(&sev_tsm_ops, snp_dev, &tsm_report_extra_type);
 	if (ret)
 		goto e_free_cert_data;
 
diff --git a/drivers/virt/coco/tdx-guest/tdx-guest.c b/drivers/virt/coco/tdx-guest/tdx-guest.c
index 1253bf76b570..904f16461492 100644
--- a/drivers/virt/coco/tdx-guest/tdx-guest.c
+++ b/drivers/virt/coco/tdx-guest/tdx-guest.c
@@ -163,7 +163,7 @@ static int tdx_report_new(struct tsm_report *report, void *data)
 {
 	u8 *buf, *reportdata = NULL, *tdreport = NULL;
 	struct tdx_quote_buf *quote_buf = quote_data;
-	struct tsm_desc *desc = &report->desc;
+	struct tsm_report_desc *desc = &report->desc;
 	int ret;
 	u64 err;
 
@@ -278,7 +278,7 @@ static const struct x86_cpu_id tdx_guest_ids[] = {
 };
 MODULE_DEVICE_TABLE(x86cpu, tdx_guest_ids);
 
-static const struct tsm_ops tdx_tsm_ops = {
+static const struct tsm_report_ops tdx_tsm_ops = {
 	.name = KBUILD_MODNAME,
 	.report_new = tdx_report_new,
 };
@@ -301,7 +301,7 @@ static int __init tdx_guest_init(void)
 		goto free_misc;
 	}
 
-	ret = tsm_register(&tdx_tsm_ops, NULL, NULL);
+	ret = tsm_report_register(&tdx_tsm_ops, NULL, NULL);
 	if (ret)
 		goto free_quote;
 
@@ -318,7 +318,7 @@ module_init(tdx_guest_init);
 
 static void __exit tdx_guest_exit(void)
 {
-	tsm_unregister(&tdx_tsm_ops);
+	tsm_report_unregister(&tdx_tsm_ops);
 	free_quote_buf(quote_data);
 	misc_deregister(&tdx_misc_dev);
 }
diff --git a/drivers/virt/coco/tsm.c b/drivers/virt/coco/tsm.c
index d1c2db83a8ca..6cb0a0e6783d 100644
--- a/drivers/virt/coco/tsm.c
+++ b/drivers/virt/coco/tsm.c
@@ -13,7 +13,7 @@
 #include <linux/configfs.h>
 
 static struct tsm_provider {
-	const struct tsm_ops *ops;
+	const struct tsm_report_ops *ops;
 	const struct config_item_type *type;
 	void *data;
 } provider;
@@ -98,7 +98,7 @@ static ssize_t tsm_report_privlevel_store(struct config_item *cfg,
 	 * SEV-SNP GHCB) and a minimum of a TSM selected floor value no less
 	 * than 0.
 	 */
-	if (provider.ops->privlevel_floor > val || val > TSM_PRIVLEVEL_MAX)
+	if (provider.ops->privlevel_floor > val || val > TSM_REPORT_PRIVLEVEL_MAX)
 		return -EINVAL;
 
 	guard(rwsem_write)(&tsm_rwsem);
@@ -134,7 +134,7 @@ static ssize_t tsm_report_inblob_write(struct config_item *cfg,
 	memcpy(report->desc.inblob, buf, count);
 	return count;
 }
-CONFIGFS_BIN_ATTR_WO(tsm_report_, inblob, NULL, TSM_INBLOB_MAX);
+CONFIGFS_BIN_ATTR_WO(tsm_report_, inblob, NULL, TSM_REPORT_INBLOB_MAX);
 
 static ssize_t tsm_report_generation_show(struct config_item *cfg, char *buf)
 {
@@ -201,7 +201,7 @@ static ssize_t tsm_report_read(struct tsm_report *report, void *buf,
 			       size_t count, enum tsm_data_select select)
 {
 	struct tsm_report_state *state = to_state(report);
-	const struct tsm_ops *ops;
+	const struct tsm_report_ops *ops;
 	ssize_t rc;
 
 	/* try to read from the existing report if present and valid... */
@@ -241,7 +241,7 @@ static ssize_t tsm_report_outblob_read(struct config_item *cfg, void *buf,
 
 	return tsm_report_read(report, buf, count, TSM_REPORT);
 }
-CONFIGFS_BIN_ATTR_RO(tsm_report_, outblob, NULL, TSM_OUTBLOB_MAX);
+CONFIGFS_BIN_ATTR_RO(tsm_report_, outblob, NULL, TSM_REPORT_OUTBLOB_MAX);
 
 static ssize_t tsm_report_auxblob_read(struct config_item *cfg, void *buf,
 				       size_t count)
@@ -250,7 +250,7 @@ static ssize_t tsm_report_auxblob_read(struct config_item *cfg, void *buf,
 
 	return tsm_report_read(report, buf, count, TSM_CERTS);
 }
-CONFIGFS_BIN_ATTR_RO(tsm_report_, auxblob, NULL, TSM_OUTBLOB_MAX);
+CONFIGFS_BIN_ATTR_RO(tsm_report_, auxblob, NULL, TSM_REPORT_OUTBLOB_MAX);
 
 #define TSM_DEFAULT_ATTRS() \
 	&tsm_report_attr_generation, \
@@ -353,10 +353,10 @@ static struct configfs_subsystem tsm_configfs = {
 	.su_mutex = __MUTEX_INITIALIZER(tsm_configfs.su_mutex),
 };
 
-int tsm_register(const struct tsm_ops *ops, void *priv,
-		 const struct config_item_type *type)
+int tsm_report_register(const struct tsm_report_ops *ops, void *priv,
+			const struct config_item_type *type)
 {
-	const struct tsm_ops *conflict;
+	const struct tsm_report_ops *conflict;
 
 	if (!type)
 		type = &tsm_report_default_type;
@@ -375,9 +375,9 @@ int tsm_register(const struct tsm_ops *ops, void *priv,
 	provider.type = type;
 	return 0;
 }
-EXPORT_SYMBOL_GPL(tsm_register);
+EXPORT_SYMBOL_GPL(tsm_report_register);
 
-int tsm_unregister(const struct tsm_ops *ops)
+int tsm_report_unregister(const struct tsm_report_ops *ops)
 {
 	guard(rwsem_write)(&tsm_rwsem);
 	if (ops != provider.ops)
@@ -387,7 +387,7 @@ int tsm_unregister(const struct tsm_ops *ops)
 	provider.type = NULL;
 	return 0;
 }
-EXPORT_SYMBOL_GPL(tsm_unregister);
+EXPORT_SYMBOL_GPL(tsm_report_unregister);
 
 static struct config_group *tsm_report_group;
 
diff --git a/include/linux/tsm.h b/include/linux/tsm.h
index de8324a2223c..9bbb1d130d01 100644
--- a/include/linux/tsm.h
+++ b/include/linux/tsm.h
@@ -5,25 +5,25 @@
 #include <linux/sizes.h>
 #include <linux/types.h>
 
-#define TSM_INBLOB_MAX 64
-#define TSM_OUTBLOB_MAX SZ_32K
+#define TSM_REPORT_INBLOB_MAX 64
+#define TSM_REPORT_OUTBLOB_MAX SZ_32K
 
 /*
  * Privilege level is a nested permission concept to allow confidential
  * guests to partition address space, 4-levels are supported.
  */
-#define TSM_PRIVLEVEL_MAX 3
+#define TSM_REPORT_PRIVLEVEL_MAX 3
 
 /**
- * struct tsm_desc - option descriptor for generating tsm report blobs
+ * struct tsm_report_desc - option descriptor for generating tsm report blobs
  * @privlevel: optional privilege level to associate with @outblob
  * @inblob_len: sizeof @inblob
  * @inblob: arbitrary input data
  */
-struct tsm_desc {
+struct tsm_report_desc {
 	unsigned int privlevel;
 	size_t inblob_len;
-	u8 inblob[TSM_INBLOB_MAX];
+	u8 inblob[TSM_REPORT_INBLOB_MAX];
 };
 
 /**
@@ -35,7 +35,7 @@ struct tsm_desc {
  * @auxblob: (optional) auxiliary data to the report (e.g. certificate data)
  */
 struct tsm_report {
-	struct tsm_desc desc;
+	struct tsm_report_desc desc;
 	size_t outblob_len;
 	u8 *outblob;
 	size_t auxblob_len;
@@ -43,7 +43,7 @@ struct tsm_report {
 };
 
 /**
- * struct tsm_ops - attributes and operations for tsm instances
+ * struct tsm_report_ops - attributes and operations for tsm_report instances
  * @name: tsm id reflected in /sys/kernel/config/tsm/report/$report/provider
  * @privlevel_floor: convey base privlevel for nested scenarios
  * @report_new: Populate @report with the report blob and auxblob
@@ -52,7 +52,7 @@ struct tsm_report {
  * Implementation specific ops, only one is expected to be registered at
  * a time i.e. only one of "sev-guest", "tdx-guest", etc.
  */
-struct tsm_ops {
+struct tsm_report_ops {
 	const char *name;
 	const unsigned int privlevel_floor;
 	int (*report_new)(struct tsm_report *report, void *data);
@@ -63,7 +63,7 @@ extern const struct config_item_type tsm_report_default_type;
 /* publish @privlevel, @privlevel_floor, and @auxblob attributes */
 extern const struct config_item_type tsm_report_extra_type;
 
-int tsm_register(const struct tsm_ops *ops, void *priv,
-		 const struct config_item_type *type);
-int tsm_unregister(const struct tsm_ops *ops);
+int tsm_report_register(const struct tsm_report_ops *ops, void *priv,
+			const struct config_item_type *type);
+int tsm_report_unregister(const struct tsm_report_ops *ops);
 #endif /* __TSM_H */

[RFC PATCH v2 2/6] coco/guest: Move shared guest CC infrastructure to drivers/virt/coco/guest/

[Dan Williams]

In preparation for creating a new drivers/virt/coco/host/ directory to
house shared host driver infrastructure for confidential computing, move
configfs-tsm to a guest/ sub-directory. The tsm.ko module is renamed to
tsm_reports.ko. The old tsm.ko module was only ever demand loaded by
kernel internal dependencies, so it should not affect existing userspace
module install scripts.

Cc: Wu Hao <hao.wu@intel.com>
Cc: Yilun Xu <yilun.xu@intel.com>
Cc: Samuel Ortiz <sameo@rivosinc.com>
Cc: Alexey Kardashevskiy <aik@amd.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
 MAINTAINERS                             |    2 +-
 drivers/virt/coco/Kconfig               |    6 ++----
 drivers/virt/coco/Makefile              |    2 +-
 drivers/virt/coco/guest/Kconfig         |    7 +++++++
 drivers/virt/coco/guest/Makefile        |    2 ++
 drivers/virt/coco/guest/tsm_report.c    |    8 ++++----
 drivers/virt/coco/sev-guest/sev-guest.c |    4 ++--
 7 files changed, 19 insertions(+), 12 deletions(-)
 create mode 100644 drivers/virt/coco/guest/Kconfig
 create mode 100644 drivers/virt/coco/guest/Makefile
 rename drivers/virt/coco/{tsm.c => guest/tsm_report.c} (98%)

diff --git a/MAINTAINERS b/MAINTAINERS
index aa3b947fb080..65beba4e704a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -22466,7 +22466,7 @@ M:	Dan Williams <dan.j.williams@intel.com>
 L:	linux-coco@lists.linux.dev
 S:	Maintained
 F:	Documentation/ABI/testing/configfs-tsm
-F:	drivers/virt/coco/tsm.c
+F:	drivers/virt/coco/guest/tsm_report.c
 F:	include/linux/tsm.h
 
 TTY LAYER AND SERIAL DRIVERS
diff --git a/drivers/virt/coco/Kconfig b/drivers/virt/coco/Kconfig
index 87d142c1f932..7c41e0abd423 100644
--- a/drivers/virt/coco/Kconfig
+++ b/drivers/virt/coco/Kconfig
@@ -3,12 +3,10 @@
 # Confidential computing related collateral
 #
 
-config TSM_REPORTS
-	select CONFIGFS_FS
-	tristate
-
 source "drivers/virt/coco/efi_secret/Kconfig"
 
 source "drivers/virt/coco/sev-guest/Kconfig"
 
 source "drivers/virt/coco/tdx-guest/Kconfig"
+
+source "drivers/virt/coco/guest/Kconfig"
diff --git a/drivers/virt/coco/Makefile b/drivers/virt/coco/Makefile
index 18c1aba5edb7..621111811a76 100644
--- a/drivers/virt/coco/Makefile
+++ b/drivers/virt/coco/Makefile
@@ -2,7 +2,7 @@
 #
 # Confidential computing related collateral
 #
-obj-$(CONFIG_TSM_REPORTS)	+= tsm.o
 obj-$(CONFIG_EFI_SECRET)	+= efi_secret/
 obj-$(CONFIG_SEV_GUEST)		+= sev-guest/
 obj-$(CONFIG_INTEL_TDX_GUEST)	+= tdx-guest/
+obj-$(CONFIG_TSM_REPORTS)	+= guest/
diff --git a/drivers/virt/coco/guest/Kconfig b/drivers/virt/coco/guest/Kconfig
new file mode 100644
index 000000000000..ed9bafbdd854
--- /dev/null
+++ b/drivers/virt/coco/guest/Kconfig
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Confidential computing shared guest collateral
+#
+config TSM_REPORTS
+	select CONFIGFS_FS
+	tristate
diff --git a/drivers/virt/coco/guest/Makefile b/drivers/virt/coco/guest/Makefile
new file mode 100644
index 000000000000..1f5fad59fc96
--- /dev/null
+++ b/drivers/virt/coco/guest/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_TSM_REPORTS)	+= tsm_report.o
diff --git a/drivers/virt/coco/tsm.c b/drivers/virt/coco/guest/tsm_report.c
similarity index 98%
rename from drivers/virt/coco/tsm.c
rename to drivers/virt/coco/guest/tsm_report.c
index 6cb0a0e6783d..272077a02da5 100644
--- a/drivers/virt/coco/tsm.c
+++ b/drivers/virt/coco/guest/tsm_report.c
@@ -391,7 +391,7 @@ EXPORT_SYMBOL_GPL(tsm_report_unregister);
 
 static struct config_group *tsm_report_group;
 
-static int __init tsm_init(void)
+static int __init tsm_report_init(void)
 {
 	struct config_group *root = &tsm_configfs.su_group;
 	struct config_group *tsm;
@@ -412,14 +412,14 @@ static int __init tsm_init(void)
 
 	return 0;
 }
-module_init(tsm_init);
+module_init(tsm_report_init);
 
-static void __exit tsm_exit(void)
+static void __exit tsm_report_exit(void)
 {
 	configfs_unregister_default_group(tsm_report_group);
 	configfs_unregister_subsystem(&tsm_configfs);
 }
-module_exit(tsm_exit);
+module_exit(tsm_report_exit);
 
 MODULE_LICENSE("GPL");
 MODULE_DESCRIPTION("Provide Trusted Security Module attestation reports via configfs");
diff --git a/drivers/virt/coco/sev-guest/sev-guest.c b/drivers/virt/coco/sev-guest/sev-guest.c
index ce60e3ce8aa3..dc0e3ad21cbf 100644
--- a/drivers/virt/coco/sev-guest/sev-guest.c
+++ b/drivers/virt/coco/sev-guest/sev-guest.c
@@ -892,7 +892,7 @@ static const struct tsm_report_ops sev_tsm_report_ops = {
 
 static void unregister_sev_tsm(void *data)
 {
-	tsm_report_unregister(&sev_tsm_ops);
+	tsm_report_unregister(&sev_tsm_report_ops);
 }
 
 static int __init sev_guest_probe(struct platform_device *pdev)
@@ -968,7 +968,7 @@ static int __init sev_guest_probe(struct platform_device *pdev)
 	snp_dev->input.resp_gpa = __pa(snp_dev->response);
 	snp_dev->input.data_gpa = __pa(snp_dev->certs_data);
 
-	ret = tsm_report_register(&sev_tsm_ops, snp_dev, &tsm_report_extra_type);
+	ret = tsm_report_register(&sev_tsm_report_ops, snp_dev, &tsm_report_extra_type);
 	if (ret)
 		goto e_free_cert_data;
 

[RFC PATCH v2 3/6] x86/tdx: Introduce a "tdx" subsystem and "tsm" device

[Dan Williams]

TDX depends on a platform firmware module that is invoked via
instructions similar to vmenter (i.e. enter into a new privileged
"root-mode" context to manage private memory and private device
mechanisms). It is a software construct that depends on the CPU vmxon
state to enable invocation of TDX-module ABIs. Unlike other
Trusted Execution Environment (TEE) platform implementations that employ
a firmware module running on a PCI device with an MMIO mailbox for
communication, TDX has no hardware device to point to as the "TSM".

The "/sys/devices/virtual" hierarchy is intended for "software
constructs which need sysfs interface", which aligns with what TDX
needs.

The new tdx_subsys will export global attributes populated by the
TDX-module "sysinfo". A tdx_tsm device is published on this bus to
enable a typical driver model for the low level "TEE Security Manager"
(TSM) flows that talk TDISP to capable PCIe devices.

For now, this is only the base tdx_subsys and tdx_tsm device
registration with attribute definition and TSM driver to follow later.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
 arch/x86/virt/vmx/tdx/tdx.c |   63 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 62 insertions(+), 1 deletion(-)

diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 4d6826a76f78..e23bddf31daa 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -28,6 +28,8 @@
 #include <linux/acpi.h>
 #include <linux/suspend.h>
 #include <linux/acpi.h>
+#include <linux/device.h>
+#include <linux/cleanup.h>
 #include <asm/page.h>
 #include <asm/special_insns.h>
 #include <asm/msr-index.h>
@@ -1097,9 +1099,56 @@ static int init_tdmrs(struct tdmr_info_list *tdmr_list)
 	return 0;
 }
 
+static const struct bus_type tdx_subsys = {
+	.name = "tdx",
+};
+
+struct tdx_tsm {
+	struct device dev;
+};
+
+static struct tdx_tsm *alloc_tdx_tsm(void)
+{
+	struct tdx_tsm *tsm = kzalloc(sizeof(*tsm), GFP_KERNEL);
+	struct device *dev;
+
+	if (!tsm)
+		return ERR_PTR(-ENOMEM);
+
+	dev = &tsm->dev;
+	dev->bus = &tdx_subsys;
+	device_initialize(dev);
+
+	return tsm;
+}
+
+DEFINE_FREE(tdx_tsm_put, struct tdx_tsm *,
+	    if (!IS_ERR_OR_NULL(_T)) put_device(&_T->dev))
+static struct tdx_tsm *init_tdx_tsm(void)
+{
+	struct device *dev;
+	int ret;
+
+	struct tdx_tsm *tsm __free(tdx_tsm_put) = alloc_tdx_tsm();
+	if (IS_ERR(tsm))
+		return tsm;
+
+	dev = &tsm->dev;
+	ret = dev_set_name(dev, "tdx_tsm");
+	if (ret)
+		return ERR_PTR(ret);
+
+	ret = device_add(dev);
+	if (ret)
+		return ERR_PTR(ret);
+
+	return no_free_ptr(tsm);
+}
+
 static int init_tdx_module(void)
 {
 	struct tdx_tdmr_sysinfo tdmr_sysinfo;
+	struct tdx_tsm *tsm;
 	int ret;
 
 	/*
@@ -1122,10 +1171,15 @@ static int init_tdx_module(void)
 	if (ret)
 		goto err_free_tdxmem;
 
+	/* Establish subsystem for global TDX module attributes */
+	ret = subsys_virtual_register(&tdx_subsys, NULL);
+	if (ret)
+		goto err_free_tdxmem;
+
 	/* Allocate enough space for constructing TDMRs */
 	ret = alloc_tdmr_list(&tdx_tdmr_list, &tdmr_sysinfo);
 	if (ret)
-		goto err_free_tdxmem;
+		goto err_unregister_subsys;
 
 	/* Cover all TDX-usable memory regions in TDMRs */
 	ret = construct_tdmrs(&tdx_memlist, &tdx_tdmr_list, &tdmr_sysinfo);
@@ -1149,6 +1203,11 @@ static int init_tdx_module(void)
 
 	pr_info("%lu KB allocated for PAMT\n", tdmrs_count_pamt_kb(&tdx_tdmr_list));
 
+	/* Register 'tdx_tsm' for driving optional TDX Connect functionality */
+	tsm = init_tdx_tsm();
+	if (IS_ERR(tsm))
+		pr_err("failed to initialize TSM device (%pe)\n", tsm);
+
 out_put_tdxmem:
 	/*
 	 * @tdx_memlist is written here and read at memory hotplug time.
@@ -1177,6 +1236,8 @@ static int init_tdx_module(void)
 	tdmrs_free_pamt_all(&tdx_tdmr_list);
 err_free_tdmrs:
 	free_tdmr_list(&tdx_tdmr_list);
+err_unregister_subsys:
+	bus_unregister(&tdx_subsys);
 err_free_tdxmem:
 	free_tdx_memlist(&tdx_memlist);
 	goto out_put_tdxmem;

[RFC PATCH v2 4/6] coco/tsm: Introduce a class device for TEE Security Managers

[Dan Williams]

A "TSM" is a platform component that provides an API for securely
provisioning resources for a confidential guest (TVM) to consume. The
name originates from the PCI specification for platform agent that
carries out operations for PCIe TDISP (TEE Device Interface Security
Protocol).

Instances of this class device are parented by a device representing the
platform security capability like CONFIG_CRYPTO_DEV_CCP or
CONFIG_INTEL_TDX_HOST.

This class device interface is a frontend to the aspects of a TSM and
TEE I/O that are cross-architecture common. This includes mechanisms
like enumerating available platform TEE I/O capabilities and
provisioning connections between the platform TSM and device DSMs
(Device Security Manager (TDISP)).

For now this is just the scaffolding for registering a TSM device sysfs
interface.

Cc: Xiaoyao Li <xiaoyao.li@intel.com>
Cc: Isaku Yamahata <isaku.yamahata@intel.com>
Cc: Alexey Kardashevskiy <aik@amd.com>
Cc: Wu Hao <hao.wu@intel.com>
Cc: Yilun Xu <yilun.xu@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: John Allen <john.allen@amd.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
 MAINTAINERS                       |    3 +
 drivers/virt/coco/Kconfig         |    2 +
 drivers/virt/coco/Makefile        |    1 
 drivers/virt/coco/host/Kconfig    |    6 ++
 drivers/virt/coco/host/Makefile   |    6 ++
 drivers/virt/coco/host/tsm-core.c |  113 +++++++++++++++++++++++++++++++++++++
 include/linux/tsm.h               |    5 ++
 7 files changed, 135 insertions(+), 1 deletion(-)
 create mode 100644 drivers/virt/coco/host/Kconfig
 create mode 100644 drivers/virt/coco/host/Makefile
 create mode 100644 drivers/virt/coco/host/tsm-core.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 65beba4e704a..8d5bcd9d43ac 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -22461,12 +22461,13 @@ W:	https://github.com/srcres258/linux-doc
 T:	git git://github.com/srcres258/linux-doc.git doc-zh-tw
 F:	Documentation/translations/zh_TW/
 
-TRUSTED SECURITY MODULE (TSM) ATTESTATION REPORTS
+TRUSTED (TEE) SECURITY MANAGER (TSM)
 M:	Dan Williams <dan.j.williams@intel.com>
 L:	linux-coco@lists.linux.dev
 S:	Maintained
 F:	Documentation/ABI/testing/configfs-tsm
 F:	drivers/virt/coco/guest/tsm_report.c
+F:	drivers/virt/coco/host/
 F:	include/linux/tsm.h
 
 TTY LAYER AND SERIAL DRIVERS
diff --git a/drivers/virt/coco/Kconfig b/drivers/virt/coco/Kconfig
index 7c41e0abd423..ae92da620168 100644
--- a/drivers/virt/coco/Kconfig
+++ b/drivers/virt/coco/Kconfig
@@ -10,3 +10,5 @@ source "drivers/virt/coco/sev-guest/Kconfig"
 source "drivers/virt/coco/tdx-guest/Kconfig"
 
 source "drivers/virt/coco/guest/Kconfig"
+
+source "drivers/virt/coco/host/Kconfig"
diff --git a/drivers/virt/coco/Makefile b/drivers/virt/coco/Makefile
index 621111811a76..3557f556e782 100644
--- a/drivers/virt/coco/Makefile
+++ b/drivers/virt/coco/Makefile
@@ -6,3 +6,4 @@ obj-$(CONFIG_EFI_SECRET)	+= efi_secret/
 obj-$(CONFIG_SEV_GUEST)		+= sev-guest/
 obj-$(CONFIG_INTEL_TDX_GUEST)	+= tdx-guest/
 obj-$(CONFIG_TSM_REPORTS)	+= guest/
+obj-y				+= host/
diff --git a/drivers/virt/coco/host/Kconfig b/drivers/virt/coco/host/Kconfig
new file mode 100644
index 000000000000..4fbc6ef34f12
--- /dev/null
+++ b/drivers/virt/coco/host/Kconfig
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# TSM (TEE Security Manager) Common infrastructure and host drivers
+#
+config TSM
+	tristate
diff --git a/drivers/virt/coco/host/Makefile b/drivers/virt/coco/host/Makefile
new file mode 100644
index 000000000000..be0aba6007cd
--- /dev/null
+++ b/drivers/virt/coco/host/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# TSM (TEE Security Manager) Common infrastructure and host drivers
+
+obj-$(CONFIG_TSM) += tsm.o
+tsm-y := tsm-core.o
diff --git a/drivers/virt/coco/host/tsm-core.c b/drivers/virt/coco/host/tsm-core.c
new file mode 100644
index 000000000000..0ee738fc40ed
--- /dev/null
+++ b/drivers/virt/coco/host/tsm-core.c
@@ -0,0 +1,113 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright(c) 2024 Intel Corporation. All rights reserved. */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/tsm.h>
+#include <linux/rwsem.h>
+#include <linux/device.h>
+#include <linux/module.h>
+#include <linux/cleanup.h>
+
+static DECLARE_RWSEM(tsm_core_rwsem);
+static struct class *tsm_class;
+static struct tsm_subsys {
+	struct device dev;
+} *tsm_subsys;
+
+static struct tsm_subsys *
+alloc_tsm_subsys(struct device *parent, const struct attribute_group **groups)
+{
+	struct tsm_subsys *subsys = kzalloc(sizeof(*subsys), GFP_KERNEL);
+	struct device *dev;
+
+	if (!subsys)
+		return ERR_PTR(-ENOMEM);
+	dev = &subsys->dev;
+	dev->parent = parent;
+	dev->groups = groups;
+	dev->class = tsm_class;
+	device_initialize(dev);
+	return subsys;
+}
+
+static void put_tsm_subsys(struct tsm_subsys *subsys)
+{
+	if (!IS_ERR_OR_NULL(subsys))
+		put_device(&subsys->dev);
+}
+
+DEFINE_FREE(put_tsm_subsys, struct tsm_subsys *,
+	    if (!IS_ERR_OR_NULL(_T)) put_tsm_subsys(_T))
+struct tsm_subsys *tsm_register(struct device *parent,
+				const struct attribute_group **groups)
+{
+	struct device *dev;
+	int rc;
+
+	guard(rwsem_write)(&tsm_core_rwsem);
+	if (tsm_subsys) {
+		dev_warn(parent, "failed to register: %s already registered\n",
+			 dev_name(tsm_subsys->dev.parent));
+		return ERR_PTR(-EBUSY);
+	}
+
+	struct tsm_subsys *subsys __free(put_tsm_subsys) =
+		alloc_tsm_subsys(parent, groups);
+	if (IS_ERR(subsys))
+		return subsys;
+
+	dev = &subsys->dev;
+	rc = dev_set_name(dev, "tsm0");
+	if (rc)
+		return ERR_PTR(rc);
+
+	rc = device_add(dev);
+	if (rc)
+		return ERR_PTR(rc);
+
+	tsm_subsys = no_free_ptr(subsys);
+
+	return tsm_subsys;
+}
+EXPORT_SYMBOL_GPL(tsm_register);
+
+void tsm_unregister(struct tsm_subsys *subsys)
+{
+	guard(rwsem_write)(&tsm_core_rwsem);
+	if (!tsm_subsys || subsys != tsm_subsys) {
+		pr_warn("failed to unregister, not currently registered\n");
+		return;
+	}
+
+	device_unregister(&subsys->dev);
+	tsm_subsys = NULL;
+}
+EXPORT_SYMBOL_GPL(tsm_unregister);
+
+static void tsm_release(struct device *dev)
+{
+	struct tsm_subsys *subsys = container_of(dev, typeof(*subsys), dev);
+
+	kfree(subsys);
+}
+
+static int __init tsm_init(void)
+{
+	tsm_class = class_create("tsm");
+	if (IS_ERR(tsm_class))
+		return PTR_ERR(tsm_class);
+
+	tsm_class->dev_release = tsm_release;
+	return 0;
+}
+module_init(tsm_init)
+
+static void __exit tsm_exit(void)
+{
+	class_destroy(tsm_class);
+}
+module_exit(tsm_exit)
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("TEE Security Manager core");
diff --git a/include/linux/tsm.h b/include/linux/tsm.h
index 9bbb1d130d01..2867c2ecbd11 100644
--- a/include/linux/tsm.h
+++ b/include/linux/tsm.h
@@ -4,6 +4,7 @@
 
 #include <linux/sizes.h>
 #include <linux/types.h>
+#include <linux/device.h>
 
 #define TSM_REPORT_INBLOB_MAX 64
 #define TSM_REPORT_OUTBLOB_MAX SZ_32K
@@ -66,4 +67,8 @@ extern const struct config_item_type tsm_report_extra_type;
 int tsm_report_register(const struct tsm_report_ops *ops, void *priv,
 			const struct config_item_type *type);
 int tsm_report_unregister(const struct tsm_report_ops *ops);
+struct tsm_subsys;
+struct tsm_subsys *tsm_register(struct device *parent,
+				const struct attribute_group **groups);
+void tsm_unregister(struct tsm_subsys *subsys);
 #endif /* __TSM_H */

[RFC PATCH v2 5/6] PCI/TSM: Authenticate devices via platform TSM

[Dan Williams]

The PCIe 6.1 specification, section 11, introduces the Trusted Execution
Environment (TEE) Device Interface Security Protocol (TDISP).  This
interface definition builds upon Component Measurement and
Authentication (CMA), and link Integrity and Data Encryption (IDE). It
adds support for assigning devices (PCI physical or virtual function) to
a confidential VM such that the assigned device is enabled to access
guest private memory protected by technologies like Intel TDX, AMD
SEV-SNP, RISCV COVE, or ARM CCA.

The "TSM" (TEE Security Manager) is a concept in the TDISP specification
of an agent that mediates between a "DSM" (Device Security Manager) and
system software in both a VMM and a confidential VM. A VMM uses TSM ABIs
to setup link security and assign devices. A confidential VM uses TSM
ABIs to transition an assigned device into the TDISP "RUN" state and
validate its configuration. From a Linux perspective the TSM abstracts
many of the details of TDISP, IDE, and CMA. Some of those details leak
through at times, but for the most part TDISP is an internal
implementation detail of the TSM.

Similar to the PCI core extensions to support CONFIG_PCI_CMA,
CONFIG_PCI_TSM builds upon that to reuse the "authenticated" sysfs
attribute, and add more properties + controls in a tsm/ subdirectory of
the PCI device sysfs interface. Unlike CMA that can depend on a local to
the PCI core implementation, PCI_TSM needs to be prepared for late
loading of the platform TSM driver. Consider that the TSM driver may
itself be a PCI driver. Userspace can depend on the common TSM device
uevent to know when the PCI core has TSM services enabled. The PCI
device tsm/ subdirectory is supplemented by the TSM device pci/
directory for platform global TSM properties + controls.

The common verbs that the low-level TSM drivers implement are defined by
'enum pci_tsm_cmd'. For now only connect and disconnect are defined for
establishing a trust relationship between the host and the device,
securing the interconnect (optionally establishing IDE), and tearing
that down.

The locking allows for multiple devices to be executing commands
simultaneously, one outstanding command per-device and an rwsem flushes
all inflight commands when a TSM low-level driver/device is removed.

In addition to commands submitted through an 'exec' operation the
low-level TSM driver is notified of device arrival and departure events
via 'add' and 'del' operations. With those it can setup per-device
context, or filter devices that the TSM is not prepared to support.

Cc: Wu Hao <hao.wu@intel.com>
Cc: Yilun Xu <yilun.xu@intel.com>
Cc: Lukas Wunner <lukas@wunner.de>
Cc: Samuel Ortiz <sameo@rivosinc.com>
Cc: Alexey Kardashevskiy <aik@amd.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Co-developed-by: Xu Yilun <yilun.xu@linux.intel.com>
Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
 Documentation/ABI/testing/sysfs-bus-pci |   46 +++++
 MAINTAINERS                             |    2 
 drivers/pci/Kconfig                     |   13 +
 drivers/pci/Makefile                    |    2 
 drivers/pci/pci-sysfs.c                 |    4 
 drivers/pci/pci.h                       |   10 +
 drivers/pci/probe.c                     |    1 
 drivers/pci/remove.c                    |    1 
 drivers/pci/tsm.c                       |  270 +++++++++++++++++++++++++++++++
 drivers/virt/coco/host/tsm-core.c       |   22 ++-
 include/linux/pci-tsm.h                 |   80 +++++++++
 include/linux/pci.h                     |   11 +
 include/linux/tsm.h                     |    4 
 include/uapi/linux/pci_regs.h           |    4 
 14 files changed, 466 insertions(+), 4 deletions(-)
 create mode 100644 drivers/pci/tsm.c
 create mode 100644 include/linux/pci-tsm.h

diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci
index ecf47559f495..4ae50621e65b 100644
--- a/Documentation/ABI/testing/sysfs-bus-pci
+++ b/Documentation/ABI/testing/sysfs-bus-pci
@@ -500,3 +500,49 @@ Description:
 		console drivers from the device.  Raw users of pci-sysfs
 		resourceN attributes must be terminated prior to resizing.
 		Success of the resizing operation is not guaranteed.
+
+What:		/sys/bus/pci/devices/.../authenticated
+Date:		March 2024
+Contact:	linux-pci@vger.kernel.org
+Description:
+		This file contains 1 if the device authenticated successfully.
+		It contains 0 if the device failed authentication (and may thus
+		be malicious). There are 2 potential authentication methods:
+		native PCI CMA (Component Measurement and Authentication) and
+		PCI TSM (TEE Security Manager). In the PCI TSM case the device's
+		PCI CMA interface is subsumed by the TSM driver. A TSM
+		implementation uses its own private certificate store + keys to
+		authenticate the device. Without a TSM the kernel can
+		authenticate using its own certificate chain.
+
+		In the TSM case, "authenticated" is read-only (0444) and the
+		"tsm/connect" attribute reflects whether the device is TSM
+		"connected" which includes not only CMA authentication, but
+		optionally IDE (link Integrity and Data encryption) if the TSM
+		deems that is necessary. When the device is disconnected from
+		the TSM the kernel may attempt authentication with its own
+		certificate chain. See
+		Documentation/ABI/testing/sysfs-devices-spdm.
+
+		The file is not visible if authentication is unsupported by the
+		device, or if PCI CMA support is disabled and the TSM
+		driver has no available authentication methods for the device.
+
+What:		/sys/bus/pci/devices/.../tsm/
+Date:		March 2024
+Contact:	linux-coco@lists.linux.dev
+Description:
+		This directory only appears if a device supports CMA and IDE,
+		and only after a TSM driver has loaded and evaluated this
+		PCI device. All present devices shall be dispositioned
+		after the 'add' event for /sys/class/tsm/tsm0 triggers.
+
+What:		/sys/bus/pci/devices/.../tsm/connect
+Date:		March 2024
+Contact:	linux-coco@lists.linux.dev
+Description:
+		(RW) Writing "1" to this file triggers the TSM to establish a
+		connection with the device. This typically includes an SPDM
+		(DMTF Security Protocols and Data Models) session over PCIe DOE
+		(Data Object Exchange) and may also include PCIe IDE (Integrity
+		and Data Encryption) establishment.
diff --git a/MAINTAINERS b/MAINTAINERS
index 8d5bcd9d43ac..0e1d995e7a16 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -22466,8 +22466,10 @@ M:	Dan Williams <dan.j.williams@intel.com>
 L:	linux-coco@lists.linux.dev
 S:	Maintained
 F:	Documentation/ABI/testing/configfs-tsm
+F:	drivers/pci/tsm.c
 F:	drivers/virt/coco/guest/tsm_report.c
 F:	drivers/virt/coco/host/
+F:	include/linux/pci-tsm.h
 F:	include/linux/tsm.h
 
 TTY LAYER AND SERIAL DRIVERS
diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
index d35001589d88..cd863c5e49ca 100644
--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
@@ -121,6 +121,19 @@ config XEN_PCIDEV_FRONTEND
 config PCI_ATS
 	bool
 
+config PCI_TSM
+	bool "TEE Security Manager for PCI Device Security"
+	help
+	  The TEE (Trusted Execution Environment) Device Interface
+	  Security Protocol (TDISP) defines a "TSM" as a platform agent
+	  that manages device authentication, link encryption, link
+	  integrity protection, and assignment of PCI device functions
+	  (virtual or physical) to confidential computing VMs that can
+	  access (DMA) guest private memory.
+
+	  Enable a platform TSM driver to use this capability.
+
+
 config PCI_DOE
 	bool
 
diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile
index 175302036890..b9884a669c5f 100644
--- a/drivers/pci/Makefile
+++ b/drivers/pci/Makefile
@@ -35,6 +35,8 @@ obj-$(CONFIG_VGA_ARB)		+= vgaarb.o
 obj-$(CONFIG_PCI_DOE)		+= doe.o
 obj-$(CONFIG_PCI_DYNAMIC_OF_NODES) += of_property.o
 
+obj-$(CONFIG_PCI_TSM)		+= tsm.o
+
 # Endpoint library must be initialized before its users
 obj-$(CONFIG_PCI_ENDPOINT)	+= endpoint/
 
diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c
index 40cfa716392f..c6ea624dd76c 100644
--- a/drivers/pci/pci-sysfs.c
+++ b/drivers/pci/pci-sysfs.c
@@ -1661,6 +1661,10 @@ const struct attribute_group *pci_dev_attr_groups[] = {
 #endif
 #ifdef CONFIG_PCIEASPM
 	&aspm_ctrl_attr_group,
+#endif
+#ifdef CONFIG_PCI_TSM
+	&pci_tsm_auth_attr_group,
+	&pci_tsm_attr_group,
 #endif
 	NULL,
 };
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 17fed1846847..9b864cbf8682 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -335,6 +335,16 @@ static inline void pci_doe_destroy(struct pci_dev *pdev) { }
 static inline void pci_doe_disconnected(struct pci_dev *pdev) { }
 #endif
 
+#ifdef CONFIG_PCI_TSM
+void pci_tsm_init(struct pci_dev *pdev);
+void pci_tsm_destroy(struct pci_dev *pdev);
+extern const struct attribute_group pci_tsm_attr_group;
+extern const struct attribute_group pci_tsm_auth_attr_group;
+#else
+static inline void pci_tsm_init(struct pci_dev *pdev) { }
+static inline void pci_tsm_destroy(struct pci_dev *pdev) { }
+#endif
+
 /**
  * pci_dev_set_io_state - Set the new error state if possible.
  *
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 1325fbae2f28..d89b67541d02 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -2481,6 +2481,7 @@ static void pci_init_capabilities(struct pci_dev *dev)
 	pci_dpc_init(dev);		/* Downstream Port Containment */
 	pci_rcec_init(dev);		/* Root Complex Event Collector */
 	pci_doe_init(dev);		/* Data Object Exchange */
+	pci_tsm_init(dev);		/* TEE Security Manager connection */
 
 	pcie_report_downtraining(dev);
 	pci_init_reset_methods(dev);
diff --git a/drivers/pci/remove.c b/drivers/pci/remove.c
index d749ea8250d6..d94b2458934a 100644
--- a/drivers/pci/remove.c
+++ b/drivers/pci/remove.c
@@ -39,6 +39,7 @@ static void pci_destroy_dev(struct pci_dev *dev)
 	list_del(&dev->bus_list);
 	up_write(&pci_bus_sem);
 
+	pci_tsm_destroy(dev);
 	pci_doe_destroy(dev);
 	pcie_aspm_exit_link_state(dev);
 	pci_bridge_d3_update(dev);
diff --git a/drivers/pci/tsm.c b/drivers/pci/tsm.c
new file mode 100644
index 000000000000..9c5fb2c46662
--- /dev/null
+++ b/drivers/pci/tsm.c
@@ -0,0 +1,270 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * TEE Security Manager for the TEE Device Interface Security Protocol
+ * (TDISP, PCIe r6.1 sec 11)
+ *
+ * Copyright(c) 2024 Intel Corporation. All rights reserved.
+ */
+
+#define dev_fmt(fmt) "TSM: " fmt
+
+#include <linux/pci.h>
+#include <linux/pci-doe.h>
+#include <linux/sysfs.h>
+#include <linux/xarray.h>
+#include <linux/pci-tsm.h>
+#include <linux/bitfield.h>
+#include "pci.h"
+
+/* collect TSM capable devices to rendezvous with the tsm driver */
+static DEFINE_XARRAY(pci_tsm_devs);
+
+/*
+ * Provide a read/write lock against the init / exit of pdev tsm
+ * capabilities and arrival/departure of a tsm instance
+ */
+static DECLARE_RWSEM(pci_tsm_rwsem);
+static const struct pci_tsm_ops *tsm_ops;
+
+static int pci_tsm_disconnect(struct pci_dev *pdev)
+{
+	struct pci_tsm *pci_tsm = pdev->tsm;
+
+	lockdep_assert_held_read(&pci_tsm_rwsem);
+	scoped_cond_guard(mutex_intr, return -EINTR, &pci_tsm->exec_lock) {
+		int rc;
+
+		if (pci_tsm->state < PCI_TSM_CONNECT)
+			return 0;
+		if (pci_tsm->state < PCI_TSM_INIT)
+			return -ENXIO;
+
+		rc = tsm_ops->exec(pdev, TSM_EXEC_DISCONNECT);
+		if (rc)
+			return rc;
+		pci_tsm->state = PCI_TSM_INIT;
+	}
+	return 0;
+}
+
+static int pci_tsm_connect(struct pci_dev *pdev)
+{
+	struct pci_tsm *pci_tsm = pdev->tsm;
+
+	lockdep_assert_held_read(&pci_tsm_rwsem);
+	scoped_cond_guard(mutex_intr, return -EINTR, &pci_tsm->exec_lock) {
+		int rc;
+
+		if (pci_tsm->state >= PCI_TSM_CONNECT)
+			return 0;
+		if (pci_tsm->state < PCI_TSM_INIT)
+			return -ENXIO;
+
+		rc = tsm_ops->exec(pdev, TSM_EXEC_CONNECT);
+		if (rc)
+			return rc;
+		pci_tsm->state = PCI_TSM_CONNECT;
+	}
+	return 0;
+}
+
+static ssize_t connect_store(struct device *dev, struct device_attribute *attr,
+			     const char *buf, size_t len)
+{
+	int rc;
+	bool connect;
+	struct pci_dev *pdev = to_pci_dev(dev);
+
+	rc = kstrtobool(buf, &connect);
+	if (rc)
+		return rc;
+
+	if (connect)
+		rc = pci_tsm_connect(pdev);
+	else
+		rc = pci_tsm_disconnect(pdev);
+	if (rc)
+		return rc;
+	return len;
+}
+
+static ssize_t connect_show(struct device *dev, struct device_attribute *attr,
+			    char *buf)
+{
+	struct pci_dev *pdev = to_pci_dev(dev);
+
+	return sysfs_emit(buf, "%d\n", pdev->tsm->state >= PCI_TSM_CONNECT);
+}
+static DEVICE_ATTR_RW(connect);
+
+static bool pci_tsm_group_visible(struct kobject *kobj)
+{
+	struct device *dev = kobj_to_dev(kobj);
+	struct pci_dev *pdev = to_pci_dev(dev);
+
+	if (pdev->tsm && pdev->tsm->state > PCI_TSM_IDLE)
+		return true;
+	return false;
+}
+DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE(pci_tsm);
+
+static struct attribute *pci_tsm_attrs[] = {
+	&dev_attr_connect.attr,
+	NULL,
+};
+
+const struct attribute_group pci_tsm_attr_group = {
+	.name = "tsm",
+	.attrs = pci_tsm_attrs,
+	.is_visible = SYSFS_GROUP_VISIBLE(pci_tsm),
+};
+
+static ssize_t authenticated_show(struct device *dev,
+				  struct device_attribute *attr, char *buf)
+{
+	/*
+	 * When device authentication is TSM owned, 'authenticated' is
+	 * identical to the connect state.
+	 */
+	return connect_show(dev, attr, buf);
+}
+static DEVICE_ATTR_RO(authenticated);
+
+static struct attribute *pci_tsm_auth_attrs[] = {
+	&dev_attr_authenticated.attr,
+	NULL,
+};
+
+const struct attribute_group pci_tsm_auth_attr_group = {
+	.attrs = pci_tsm_auth_attrs,
+	.is_visible = SYSFS_GROUP_VISIBLE(pci_tsm),
+};
+
+static int pci_tsm_add(struct pci_dev *pdev)
+{
+	struct pci_tsm *pci_tsm = pdev->tsm;
+
+	lockdep_assert_held(&pci_tsm_rwsem);
+	if (!tsm_ops)
+		return 0;
+	if (pci_tsm->state < PCI_TSM_INIT) {
+		int rc = tsm_ops->add(pdev);
+
+		if (rc)
+			return rc;
+	}
+	pci_tsm->state = PCI_TSM_INIT;
+	return sysfs_update_group(&pdev->dev.kobj, &pci_tsm_attr_group);
+}
+
+static void pci_tsm_del(struct pci_dev *pdev)
+{
+	struct pci_tsm *pci_tsm = pdev->tsm;
+
+	lockdep_assert_held(&pci_tsm_rwsem);
+	/* shutdown sysfs operations before tsm delete */
+	scoped_guard(mutex, &pdev->tsm->exec_lock)
+		pci_tsm->state = PCI_TSM_IDLE;
+	sysfs_update_group(&pdev->dev.kobj, &pci_tsm_attr_group);
+	tsm_ops->del(pdev);
+}
+
+int pci_tsm_register(const struct pci_tsm_ops *ops)
+{
+	struct pci_dev *pdev;
+	unsigned long index;
+
+	if (!ops)
+		return 0;
+	guard(rwsem_write)(&pci_tsm_rwsem);
+	if (tsm_ops)
+		return -EBUSY;
+	tsm_ops = ops;
+	xa_for_each(&pci_tsm_devs, index, pdev)
+		pci_tsm_add(pdev);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(pci_tsm_register);
+
+void pci_tsm_unregister(const struct pci_tsm_ops *ops)
+{
+	struct pci_dev *pdev;
+	unsigned long index;
+
+	if (!ops)
+		return;
+	guard(rwsem_write)(&pci_tsm_rwsem);
+	if (ops != tsm_ops)
+		return;
+	xa_for_each(&pci_tsm_devs, index, pdev)
+		pci_tsm_del(pdev);
+	tsm_ops = NULL;
+}
+EXPORT_SYMBOL_GPL(pci_tsm_unregister);
+
+int pci_tsm_doe_transfer(struct pci_dev *pdev, enum pci_doe_proto type,
+			 const void *req, size_t req_sz, void *resp,
+			 size_t resp_sz)
+{
+	if (!pdev->tsm || !pdev->tsm->doe_mb)
+		return -ENXIO;
+
+	return pci_doe(pdev->tsm->doe_mb, PCI_VENDOR_ID_PCI_SIG, type, req,
+		       req_sz, resp, resp_sz);
+}
+EXPORT_SYMBOL_GPL(pci_tsm_doe_transfer);
+
+static unsigned long pci_tsm_devid(struct pci_dev *pdev)
+{
+	return FIELD_PREP(GENMASK(15, 0),
+			  PCI_DEVID(pdev->bus->number, pdev->devfn)) |
+	       FIELD_PREP(GENMASK(31, 16), pci_domain_nr(pdev->bus));
+}
+
+void pci_tsm_init(struct pci_dev *pdev)
+{
+	bool tee_cap;
+	u16 ide_cap;
+	int rc;
+
+	ide_cap = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_IDE);
+	tee_cap = pdev->devcap & PCI_EXP_DEVCAP_TEE;
+
+	if (ide_cap || tee_cap)
+		pci_dbg(pdev,
+			"Device security capabailities detected (%s%s ), init TSM\n",
+			ide_cap ? " ide" : "", tee_cap ? " tee" : "");
+	else
+		return;
+
+	struct pci_tsm *pci_tsm __free(kfree) = kzalloc(sizeof(*pci_tsm), GFP_KERNEL);
+	if (!pci_tsm)
+		return;
+
+	pci_tsm->ide_cap = ide_cap;
+	mutex_init(&pci_tsm->exec_lock);
+
+	pci_tsm->doe_mb = pci_find_doe_mailbox(pdev, PCI_VENDOR_ID_PCI_SIG,
+					       PCI_DOE_PROTO_CMA);
+	if (!pci_tsm->doe_mb)
+		return;
+
+	rc = xa_insert(&pci_tsm_devs, pci_tsm_devid(pdev), pdev, GFP_KERNEL);
+	if (rc) {
+		pci_dbg(pdev, "failed to register TSM capable device\n");
+		return;
+	}
+
+	guard(rwsem_write)(&pci_tsm_rwsem);
+	pdev->tsm = no_free_ptr(pci_tsm);
+	pci_tsm_add(pdev);
+}
+
+void pci_tsm_destroy(struct pci_dev *pdev)
+{
+	guard(rwsem_write)(&pci_tsm_rwsem);
+	pci_tsm_del(pdev);
+	xa_erase(&pci_tsm_devs, pci_tsm_devid(pdev));
+	kfree(pdev->tsm);
+	pdev->tsm = NULL;
+}
diff --git a/drivers/virt/coco/host/tsm-core.c b/drivers/virt/coco/host/tsm-core.c
index 0ee738fc40ed..994a7f77c5c9 100644
--- a/drivers/virt/coco/host/tsm-core.c
+++ b/drivers/virt/coco/host/tsm-core.c
@@ -8,11 +8,13 @@
 #include <linux/device.h>
 #include <linux/module.h>
 #include <linux/cleanup.h>
+#include <linux/pci-tsm.h>
 
 static DECLARE_RWSEM(tsm_core_rwsem);
 static struct class *tsm_class;
 static struct tsm_subsys {
 	struct device dev;
+	const struct pci_tsm_ops *pci_ops;
 } *tsm_subsys;
 
 static struct tsm_subsys *
@@ -40,7 +42,8 @@ static void put_tsm_subsys(struct tsm_subsys *subsys)
 DEFINE_FREE(put_tsm_subsys, struct tsm_subsys *,
 	    if (!IS_ERR_OR_NULL(_T)) put_tsm_subsys(_T))
 struct tsm_subsys *tsm_register(struct device *parent,
-				const struct attribute_group **groups)
+				const struct attribute_group **groups,
+				const struct pci_tsm_ops *pci_ops)
 {
 	struct device *dev;
 	int rc;
@@ -62,10 +65,20 @@ struct tsm_subsys *tsm_register(struct device *parent,
 	if (rc)
 		return ERR_PTR(rc);
 
+	rc = pci_tsm_register(pci_ops);
+	if (rc) {
+		dev_err(parent, "PCI initialization failure: %pe\n",
+			ERR_PTR(rc));
+		return ERR_PTR(rc);
+	}
+
 	rc = device_add(dev);
-	if (rc)
+	if (rc) {
+		pci_tsm_unregister(pci_ops);
 		return ERR_PTR(rc);
+	}
 
+	subsys->pci_ops = pci_ops;
 	tsm_subsys = no_free_ptr(subsys);
 
 	return tsm_subsys;
@@ -74,13 +87,18 @@ EXPORT_SYMBOL_GPL(tsm_register);
 
 void tsm_unregister(struct tsm_subsys *subsys)
 {
+	const struct pci_tsm_ops *pci_ops;
+
 	guard(rwsem_write)(&tsm_core_rwsem);
 	if (!tsm_subsys || subsys != tsm_subsys) {
 		pr_warn("failed to unregister, not currently registered\n");
 		return;
 	}
 
+	pci_ops = subsys->pci_ops;
 	device_unregister(&subsys->dev);
+
+	pci_tsm_unregister(pci_ops);
 	tsm_subsys = NULL;
 }
 EXPORT_SYMBOL_GPL(tsm_unregister);
diff --git a/include/linux/pci-tsm.h b/include/linux/pci-tsm.h
new file mode 100644
index 000000000000..d17f5e0574c4
--- /dev/null
+++ b/include/linux/pci-tsm.h
@@ -0,0 +1,80 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __PCI_TSM_H
+#define __PCI_TSM_H
+#include <linux/mutex.h>
+
+enum pci_tsm_cmd {
+	TSM_EXEC_CONNECT,
+	TSM_EXEC_DISCONNECT,
+};
+
+struct pci_dev;
+/**
+ * struct pci_tsm_ops - Low-level TSM-exported interface to the PCI core
+ * @add: accept device for tsm operation
+ * @del: teardown tsm context for @pdev
+ * @exec: synchronously execute @cmd
+ *
+ * Note that @add, and @del run in down_write(&pci_tsm_rswem) context to
+ * synchronize with TSM driver bind/unbind events and
+ * pci_device_add()/pci_destroy_dev(). @exec runs in
+ * @pdev->tsm->exec_lock context to synchronize @exec results with
+ * @pdev->tsm->state
+ */
+struct pci_tsm_ops {
+	int (*add)(struct pci_dev *pdev);
+	void (*del)(struct pci_dev *pdev);
+	int (*exec)(struct pci_dev *pdev, enum pci_tsm_cmd cmd);
+};
+
+enum pci_tsm_state {
+	PCI_TSM_IDLE,
+	PCI_TSM_INIT,
+	PCI_TSM_CONNECT,
+};
+
+/**
+ * struct pci_tsm - per device TSM context
+ * @state: reflect device initialized, connected, or bound
+ * @ide_cap: PCIe IDE Extended Capability offset
+ * @exec_lock: protect @state vs pci_tsm_ops.exec() results
+ * @doe_mb: PCIe Data Object Exchange mailbox
+ * @tsm_data: TSM driver private context
+ */
+struct pci_tsm {
+	enum pci_tsm_state state;
+	u16 ide_cap;
+	struct mutex exec_lock;
+	struct pci_doe_mb *doe_mb;
+	void *tsm_data;
+};
+
+enum pci_doe_proto {
+	PCI_DOE_PROTO_CMA = 1,
+	PCI_DOE_PROTO_SSESSION = 2,
+};
+
+#ifdef CONFIG_PCI_TSM
+int pci_tsm_register(const struct pci_tsm_ops *ops);
+void pci_tsm_unregister(const struct pci_tsm_ops *ops);
+int pci_tsm_doe_transfer(struct pci_dev *pdev, enum pci_doe_proto type,
+			 const void *req, size_t req_sz, void *resp,
+			 size_t resp_sz);
+#else
+static inline int pci_tsm_register(const struct pci_tsm_ops *ops)
+{
+	return 0;
+}
+static inline void pci_tsm_unregister(const struct pci_tsm_ops *ops)
+{
+}
+static inline int pci_tsm_doe_transfer(struct pci_dev *pdev,
+				       enum pci_doe_proto type, const void *req,
+				       size_t req_sz, void *resp,
+				       size_t resp_sz)
+{
+	return -ENOENT;
+}
+
+#endif
+#endif /*__PCI_TSM_H */
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 16493426a04f..dd4dc8719c5c 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -515,6 +515,9 @@ struct pci_dev {
 #endif
 #ifdef CONFIG_PCI_DOE
 	struct xarray	doe_mbs;	/* Data Object Exchange mailboxes */
+#endif
+#ifdef CONFIG_PCI_TSM
+	struct pci_tsm *tsm;		/* TSM operation state */
 #endif
 	u16		acs_cap;	/* ACS Capability offset */
 	phys_addr_t	rom;		/* Physical address if not from BAR */
@@ -550,6 +553,12 @@ static inline int pci_channel_offline(struct pci_dev *pdev)
 	return (pdev->error_state != pci_channel_io_normal);
 }
 
+/* id resources that may be shared across host-bridges */
+struct pci_hb_id_pool {
+	int nr_stream_ids;
+	int nr_cxl_cache_ids;
+};
+
 /*
  * Currently in ACPI spec, for each PCI host bridge, PCI Segment
  * Group number is limited to a 16-bit value, therefore (int)-1 is
@@ -568,6 +577,8 @@ struct pci_host_bridge {
 	void		*sysdata;
 	int		busnr;
 	int		domain_nr;
+	struct pci_hb_id_pool __pool;
+	struct pci_hb_id_pool *pool;	/* &self->__pool, unless shared */
 	struct list_head windows;	/* resource_entry */
 	struct list_head dma_ranges;	/* dma ranges resource list */
 	u8 (*swizzle_irq)(struct pci_dev *, u8 *); /* Platform IRQ swizzler */
diff --git a/include/linux/tsm.h b/include/linux/tsm.h
index 2867c2ecbd11..6481cc99ea6d 100644
--- a/include/linux/tsm.h
+++ b/include/linux/tsm.h
@@ -68,7 +68,9 @@ int tsm_report_register(const struct tsm_report_ops *ops, void *priv,
 			const struct config_item_type *type);
 int tsm_report_unregister(const struct tsm_report_ops *ops);
 struct tsm_subsys;
+struct pci_tsm_ops;
 struct tsm_subsys *tsm_register(struct device *parent,
-				const struct attribute_group **groups);
+				const struct attribute_group **groups,
+				const struct pci_tsm_ops *ops);
 void tsm_unregister(struct tsm_subsys *subsys);
 #endif /* __TSM_H */
diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
index a39193213ff2..9aaffa379cae 100644
--- a/include/uapi/linux/pci_regs.h
+++ b/include/uapi/linux/pci_regs.h
@@ -498,6 +498,7 @@
 #define  PCI_EXP_DEVCAP_PWR_VAL	0x03fc0000 /* Slot Power Limit Value */
 #define  PCI_EXP_DEVCAP_PWR_SCL	0x0c000000 /* Slot Power Limit Scale */
 #define  PCI_EXP_DEVCAP_FLR     0x10000000 /* Function Level Reset */
+#define  PCI_EXP_DEVCAP_TEE     0x40000000 /* TEE I/O (TDISP) Support */
 #define PCI_EXP_DEVCTL		0x08	/* Device Control */
 #define  PCI_EXP_DEVCTL_CERE	0x0001	/* Correctable Error Reporting En. */
 #define  PCI_EXP_DEVCTL_NFERE	0x0002	/* Non-Fatal Error Reporting Enable */
@@ -742,7 +743,8 @@
 #define PCI_EXT_CAP_ID_PL_16GT	0x26	/* Physical Layer 16.0 GT/s */
 #define PCI_EXT_CAP_ID_PL_32GT  0x2A    /* Physical Layer 32.0 GT/s */
 #define PCI_EXT_CAP_ID_DOE	0x2E	/* Data Object Exchange */
-#define PCI_EXT_CAP_ID_MAX	PCI_EXT_CAP_ID_DOE
+#define PCI_EXT_CAP_ID_IDE	0x30	/* Integrity and Data Encryption */
+#define PCI_EXT_CAP_ID_MAX	PCI_EXT_CAP_ID_IDE
 
 #define PCI_EXT_CAP_DSN_SIZEOF	12
 #define PCI_EXT_CAP_MCAST_ENDPOINT_SIZEOF 40

[RFC PATCH v2 6/6] tdx_tsm: TEE Security Manager driver for TDX

[Dan Williams]

Recall that a TEE Security Manager (TSM) is a platform agent that speaks
the TEE Device Interface Security Protocol (TDISP) to PCIe devices and
manages private memory resources for the platform. The tdx_tsm driver
loads against a device of the same name registered at TDX Module
initialization time. The device lives on the "tdx" bus which is a
virtual subsystem that hosts the TDX module sysfs ABI.

It allows for device-security enumeration and initialization flows to be
deferred from TDX Module init time. Crucially, when / if TDX Module
init moves earlier in x86 initialization flow this driver is still
guaranteed to run after IOMMU and PCI init (i.e. subsys_initcall() vs
device_initcall()).

The ability to unload the module, or unbind the driver is also useful
for debug and coarse grained transitioning between PCI TSM operation and
PCI CMA operation (native kernel PCI device authentication).

For now this is the basic boilerplate with sysfs attributes and
operation flows to be added later.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
 arch/x86/include/asm/shared/tdx.h |    3 ++
 arch/x86/virt/vmx/tdx/tdx.c       |    9 ++++-
 drivers/virt/coco/host/Kconfig    |    6 +++
 drivers/virt/coco/host/Makefile   |    2 +
 drivers/virt/coco/host/tdx_tsm.c  |   68 +++++++++++++++++++++++++++++++++++++
 5 files changed, 87 insertions(+), 1 deletion(-)
 create mode 100644 drivers/virt/coco/host/tdx_tsm.c

diff --git a/arch/x86/include/asm/shared/tdx.h b/arch/x86/include/asm/shared/tdx.h
index fdfd41511b02..7cc5cfb65e8d 100644
--- a/arch/x86/include/asm/shared/tdx.h
+++ b/arch/x86/include/asm/shared/tdx.h
@@ -132,5 +132,8 @@ static __always_inline u64 hcall_func(u64 exit_reason)
         return exit_reason;
 }
 
+/* tdx_tsm driver interfaces */
+extern const struct bus_type tdx_subsys;
+
 #endif /* !__ASSEMBLY__ */
 #endif /* _ASM_X86_SHARED_TDX_H */
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index e23bddf31daa..13b285e4e91e 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -1099,9 +1099,16 @@ static int init_tdmrs(struct tdmr_info_list *tdmr_list)
 	return 0;
 }
 
-static const struct bus_type tdx_subsys = {
+static int tdx_uevent(const struct device *dev, struct kobj_uevent_env *env)
+{
+	return add_uevent_var(env, "MODALIAS=%s", dev_name(dev));
+}
+
+const struct bus_type tdx_subsys = {
 	.name = "tdx",
+	.uevent = tdx_uevent,
 };
+EXPORT_SYMBOL_NS_GPL(tdx_subsys, TDX);
 
 struct tdx_tsm {
 	struct device dev;
diff --git a/drivers/virt/coco/host/Kconfig b/drivers/virt/coco/host/Kconfig
index 4fbc6ef34f12..2155507b8516 100644
--- a/drivers/virt/coco/host/Kconfig
+++ b/drivers/virt/coco/host/Kconfig
@@ -4,3 +4,9 @@
 #
 config TSM
 	tristate
+
+config TDX_TSM
+	depends on INTEL_TDX_HOST
+	select PCI_TSM
+	select TSM
+	tristate
diff --git a/drivers/virt/coco/host/Makefile b/drivers/virt/coco/host/Makefile
index be0aba6007cd..2612f17ec966 100644
--- a/drivers/virt/coco/host/Makefile
+++ b/drivers/virt/coco/host/Makefile
@@ -4,3 +4,5 @@
 
 obj-$(CONFIG_TSM) += tsm.o
 tsm-y := tsm-core.o
+
+obj-$(CONFIG_TDX_TSM) += tdx_tsm.o
diff --git a/drivers/virt/coco/host/tdx_tsm.c b/drivers/virt/coco/host/tdx_tsm.c
new file mode 100644
index 000000000000..95d1352589c9
--- /dev/null
+++ b/drivers/virt/coco/host/tdx_tsm.c
@@ -0,0 +1,68 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright(c) 2024 Intel Corporation. All rights reserved. */
+#include <linux/tsm.h>
+#include <linux/pci-tsm.h>
+#include <asm/tdx.h>
+
+static int tdx_tsm_add(struct pci_dev *pdev)
+{
+	return 0;
+}
+
+static void tdx_tsm_del(struct pci_dev *pdev)
+{
+}
+
+static int tdx_tsm_exec(struct pci_dev *pdev, enum pci_tsm_cmd cmd)
+{
+	return -EOPNOTSUPP;
+}
+
+static const struct pci_tsm_ops tdx_pci_tsm_ops = {
+	.add = tdx_tsm_add,
+	.del = tdx_tsm_del,
+	.exec = tdx_tsm_exec,
+};
+
+static void unregister_tsm(void *subsys)
+{
+	tsm_unregister(subsys);
+}
+
+static int tdx_tsm_probe(struct device *dev)
+{
+	struct tsm_subsys *subsys;
+
+	subsys = tsm_register(dev, NULL, &tdx_pci_tsm_ops);
+	if (IS_ERR(subsys)) {
+		dev_err(dev, "failed to register TSM: (%pe)\n", subsys);
+		return PTR_ERR(subsys);
+	}
+
+	return devm_add_action_or_reset(dev, unregister_tsm, subsys);
+}
+
+static struct device_driver tdx_tsm_driver = {
+	.probe = tdx_tsm_probe,
+	.bus = &tdx_subsys,
+	.owner = THIS_MODULE,
+	.name = KBUILD_MODNAME,
+	.mod_name = KBUILD_MODNAME,
+};
+
+static int __init tdx_tsm_init(void)
+{
+	return driver_register(&tdx_tsm_driver);
+}
+module_init(tdx_tsm_init);
+
+static void __exit tdx_tsm_exit(void)
+{
+	driver_unregister(&tdx_tsm_driver);
+}
+module_exit(tdx_tsm_exit);
+
+MODULE_IMPORT_NS(TDX);
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("tdx_tsm");
+MODULE_DESCRIPTION("TDX TEE Security Manager");

Re: [RFC PATCH v2 5/6] PCI/TSM: Authenticate devices via platform TSM

[Bjorn Helgaas]

On Fri, Apr 12, 2024 at 01:52:13AM -0700, Dan Williams wrote:
> The PCIe 6.1 specification, section 11, introduces the Trusted Execution
> Environment (TEE) Device Interface Security Protocol (TDISP).  This
> interface definition builds upon Component Measurement and
> Authentication (CMA), and link Integrity and Data Encryption (IDE). It
> adds support for assigning devices (PCI physical or virtual function) to
> a confidential VM such that the assigned device is enabled to access
> guest private memory protected by technologies like Intel TDX, AMD
> SEV-SNP, RISCV COVE, or ARM CCA.
> 
> The "TSM" (TEE Security Manager) is a concept in the TDISP specification
> of an agent that mediates between a "DSM" (Device Security Manager) and
> system software in both a VMM and a confidential VM. A VMM uses TSM ABIs
> to setup link security and assign devices. A confidential VM uses TSM
> ABIs to transition an assigned device into the TDISP "RUN" state and
> validate its configuration. From a Linux perspective the TSM abstracts
> many of the details of TDISP, IDE, and CMA. Some of those details leak
> through at times, but for the most part TDISP is an internal
> implementation detail of the TSM.
> 
> Similar to the PCI core extensions to support CONFIG_PCI_CMA,
> CONFIG_PCI_TSM builds upon that to reuse the "authenticated" sysfs
> attribute, and add more properties + controls in a tsm/ subdirectory of
> the PCI device sysfs interface. Unlike CMA that can depend on a local to
> the PCI core implementation, PCI_TSM needs to be prepared for late
> loading of the platform TSM driver. Consider that the TSM driver may
> itself be a PCI driver. Userspace can depend on the common TSM device
> uevent to know when the PCI core has TSM services enabled. The PCI
> device tsm/ subdirectory is supplemented by the TSM device pci/
> directory for platform global TSM properties + controls.
> 
> The common verbs that the low-level TSM drivers implement are defined by
> 'enum pci_tsm_cmd'. For now only connect and disconnect are defined for
> establishing a trust relationship between the host and the device,
> securing the interconnect (optionally establishing IDE), and tearing
> that down.
> 
> The locking allows for multiple devices to be executing commands
> simultaneously, one outstanding command per-device and an rwsem flushes
> all inflight commands when a TSM low-level driver/device is removed.
> 
> In addition to commands submitted through an 'exec' operation the
> low-level TSM driver is notified of device arrival and departure events
> via 'add' and 'del' operations. With those it can setup per-device
> context, or filter devices that the TSM is not prepared to support.
> 
> Cc: Wu Hao <hao.wu@intel.com>
> Cc: Yilun Xu <yilun.xu@intel.com>
> Cc: Lukas Wunner <lukas@wunner.de>
> Cc: Samuel Ortiz <sameo@rivosinc.com>
> Cc: Alexey Kardashevskiy <aik@amd.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Co-developed-by: Xu Yilun <yilun.xu@linux.intel.com>
> Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Acked-by: Bjorn Helgaas <bhelgaas@google.com>	# PCI parts

> +What:		/sys/bus/pci/devices/.../tsm/
> +Date:		March 2024
> +Contact:	linux-coco@lists.linux.dev
> +Description:
> +		This directory only appears if a device supports CMA and IDE,
> +		and only after a TSM driver has loaded and evaluated this
> +		PCI device. All present devices shall be dispositioned
> +		after the 'add' event for /sys/class/tsm/tsm0 triggers.

What does "dispositioned" mean?

What devices does "all present devices" cover?

Is "tsm0" a special global thing?  Is there doc for
/sys/class/tsm/...?

> +++ b/drivers/pci/Makefile
> @@ -35,6 +35,8 @@ obj-$(CONFIG_VGA_ARB)		+= vgaarb.o
>  obj-$(CONFIG_PCI_DOE)		+= doe.o
>  obj-$(CONFIG_PCI_DYNAMIC_OF_NODES) += of_property.o
>  
> +obj-$(CONFIG_PCI_TSM)		+= tsm.o

Maybe put it next to CONFIG_PCI_DOE or at least not off in a special
separate list?

Bjorn

Re: [RFC PATCH v2 5/6] PCI/TSM: Authenticate devices via platform TSM

[Alexey Kardashevskiy]

On 12/4/24 18:52, Dan Williams wrote:
> The PCIe 6.1 specification, section 11, introduces the Trusted Execution
> Environment (TEE) Device Interface Security Protocol (TDISP).  This
> interface definition builds upon Component Measurement and
> Authentication (CMA), and link Integrity and Data Encryption (IDE). It
> adds support for assigning devices (PCI physical or virtual function) to
> a confidential VM such that the assigned device is enabled to access
> guest private memory protected by technologies like Intel TDX, AMD
> SEV-SNP, RISCV COVE, or ARM CCA.
> 
> The "TSM" (TEE Security Manager) is a concept in the TDISP specification
> of an agent that mediates between a "DSM" (Device Security Manager) and
> system software in both a VMM and a confidential VM. A VMM uses TSM ABIs
> to setup link security and assign devices. A confidential VM uses TSM
> ABIs to transition an assigned device into the TDISP "RUN" state and
> validate its configuration. From a Linux perspective the TSM abstracts
> many of the details of TDISP, IDE, and CMA. Some of those details leak
> through at times, but for the most part TDISP is an internal
> implementation detail of the TSM.
> 
> Similar to the PCI core extensions to support CONFIG_PCI_CMA,
> CONFIG_PCI_TSM builds upon that to reuse the "authenticated" sysfs
> attribute, and add more properties + controls in a tsm/ subdirectory of
> the PCI device sysfs interface. Unlike CMA that can depend on a local to
> the PCI core implementation, PCI_TSM needs to be prepared for late
> loading of the platform TSM driver. Consider that the TSM driver may
> itself be a PCI driver. Userspace can depend on the common TSM device
> uevent to know when the PCI core has TSM services enabled. The PCI
> device tsm/ subdirectory is supplemented by the TSM device pci/
> directory for platform global TSM properties + controls.
> 
> The common verbs that the low-level TSM drivers implement are defined by
> 'enum pci_tsm_cmd'. For now only connect and disconnect are defined for
> establishing a trust relationship between the host and the device,
> securing the interconnect (optionally establishing IDE), and tearing
> that down.
> 
> The locking allows for multiple devices to be executing commands
> simultaneously, one outstanding command per-device and an rwsem flushes
> all inflight commands when a TSM low-level driver/device is removed.
> 
> In addition to commands submitted through an 'exec' operation the
> low-level TSM driver is notified of device arrival and departure events
> via 'add' and 'del' operations. With those it can setup per-device
> context, or filter devices that the TSM is not prepared to support.
> 
> Cc: Wu Hao <hao.wu@intel.com>
> Cc: Yilun Xu <yilun.xu@intel.com>
> Cc: Lukas Wunner <lukas@wunner.de>
> Cc: Samuel Ortiz <sameo@rivosinc.com>
> Cc: Alexey Kardashevskiy <aik@amd.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Co-developed-by: Xu Yilun <yilun.xu@linux.intel.com>
> Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> ---
>   Documentation/ABI/testing/sysfs-bus-pci |   46 +++++
>   MAINTAINERS                             |    2
>   drivers/pci/Kconfig                     |   13 +
>   drivers/pci/Makefile                    |    2
>   drivers/pci/pci-sysfs.c                 |    4
>   drivers/pci/pci.h                       |   10 +
>   drivers/pci/probe.c                     |    1
>   drivers/pci/remove.c                    |    1
>   drivers/pci/tsm.c                       |  270 +++++++++++++++++++++++++++++++
>   drivers/virt/coco/host/tsm-core.c       |   22 ++-
>   include/linux/pci-tsm.h                 |   80 +++++++++
>   include/linux/pci.h                     |   11 +
>   include/linux/tsm.h                     |    4
>   include/uapi/linux/pci_regs.h           |    4
>   14 files changed, 466 insertions(+), 4 deletions(-)
>   create mode 100644 drivers/pci/tsm.c
>   create mode 100644 include/linux/pci-tsm.h
> 
> diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci
> index ecf47559f495..4ae50621e65b 100644
> --- a/Documentation/ABI/testing/sysfs-bus-pci
> +++ b/Documentation/ABI/testing/sysfs-bus-pci
> @@ -500,3 +500,49 @@ Description:
>   		console drivers from the device.  Raw users of pci-sysfs
>   		resourceN attributes must be terminated prior to resizing.
>   		Success of the resizing operation is not guaranteed.
> +
> +What:		/sys/bus/pci/devices/.../authenticated
> +Date:		March 2024
> +Contact:	linux-pci@vger.kernel.org
> +Description:
> +		This file contains 1 if the device authenticated successfully.
> +		It contains 0 if the device failed authentication (and may thus
> +		be malicious). There are 2 potential authentication methods:
> +		native PCI CMA (Component Measurement and Authentication) and
> +		PCI TSM (TEE Security Manager). In the PCI TSM case the device's
> +		PCI CMA interface is subsumed by the TSM driver. A TSM
> +		implementation uses its own private certificate store + keys to
> +		authenticate the device. Without a TSM the kernel can
> +		authenticate using its own certificate chain.
> +
> +		In the TSM case, "authenticated" is read-only (0444) and the
> +		"tsm/connect" attribute reflects whether the device is TSM
> +		"connected" which includes not only CMA authentication, but
> +		optionally IDE (link Integrity and Data encryption) if the TSM
> +		deems that is necessary. When the device is disconnected from
> +		the TSM the kernel may attempt authentication with its own
> +		certificate chain. See
> +		Documentation/ABI/testing/sysfs-devices-spdm.
> +
> +		The file is not visible if authentication is unsupported by the
> +		device, or if PCI CMA support is disabled and the TSM
> +		driver has no available authentication methods for the device.
> +
> +What:		/sys/bus/pci/devices/.../tsm/
> +Date:		March 2024
> +Contact:	linux-coco@lists.linux.dev
> +Description:
> +		This directory only appears if a device supports CMA and IDE,
> +		and only after a TSM driver has loaded and evaluated this
> +		PCI device. All present devices shall be dispositioned
> +		after the 'add' event for /sys/class/tsm/tsm0 triggers.
> +
> +What:		/sys/bus/pci/devices/.../tsm/connect
> +Date:		March 2024
> +Contact:	linux-coco@lists.linux.dev
> +Description:
> +		(RW) Writing "1" to this file triggers the TSM to establish a
> +		connection with the device. This typically includes an SPDM
> +		(DMTF Security Protocols and Data Models) session over PCIe DOE
> +		(Data Object Exchange) and may also include PCIe IDE (Integrity
> +		and Data Encryption) establishment.
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 8d5bcd9d43ac..0e1d995e7a16 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -22466,8 +22466,10 @@ M:	Dan Williams <dan.j.williams@intel.com>
>   L:	linux-coco@lists.linux.dev
>   S:	Maintained
>   F:	Documentation/ABI/testing/configfs-tsm
> +F:	drivers/pci/tsm.c
>   F:	drivers/virt/coco/guest/tsm_report.c
>   F:	drivers/virt/coco/host/
> +F:	include/linux/pci-tsm.h
>   F:	include/linux/tsm.h
>   
>   TTY LAYER AND SERIAL DRIVERS
> diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
> index d35001589d88..cd863c5e49ca 100644
> --- a/drivers/pci/Kconfig
> +++ b/drivers/pci/Kconfig
> @@ -121,6 +121,19 @@ config XEN_PCIDEV_FRONTEND
>   config PCI_ATS
>   	bool
>   
> +config PCI_TSM
> +	bool "TEE Security Manager for PCI Device Security"
> +	help
> +	  The TEE (Trusted Execution Environment) Device Interface
> +	  Security Protocol (TDISP) defines a "TSM" as a platform agent
> +	  that manages device authentication, link encryption, link
> +	  integrity protection, and assignment of PCI device functions
> +	  (virtual or physical) to confidential computing VMs that can
> +	  access (DMA) guest private memory.
> +
> +	  Enable a platform TSM driver to use this capability.
> +
> +
>   config PCI_DOE
>   	bool
>   
> diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile
> index 175302036890..b9884a669c5f 100644
> --- a/drivers/pci/Makefile
> +++ b/drivers/pci/Makefile
> @@ -35,6 +35,8 @@ obj-$(CONFIG_VGA_ARB)		+= vgaarb.o
>   obj-$(CONFIG_PCI_DOE)		+= doe.o
>   obj-$(CONFIG_PCI_DYNAMIC_OF_NODES) += of_property.o
>   
> +obj-$(CONFIG_PCI_TSM)		+= tsm.o
> +
>   # Endpoint library must be initialized before its users
>   obj-$(CONFIG_PCI_ENDPOINT)	+= endpoint/
>   
> diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c
> index 40cfa716392f..c6ea624dd76c 100644
> --- a/drivers/pci/pci-sysfs.c
> +++ b/drivers/pci/pci-sysfs.c
> @@ -1661,6 +1661,10 @@ const struct attribute_group *pci_dev_attr_groups[] = {
>   #endif
>   #ifdef CONFIG_PCIEASPM
>   	&aspm_ctrl_attr_group,
> +#endif
> +#ifdef CONFIG_PCI_TSM
> +	&pci_tsm_auth_attr_group,
> +	&pci_tsm_attr_group,
>   #endif
>   	NULL,
>   };
> diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
> index 17fed1846847..9b864cbf8682 100644
> --- a/drivers/pci/pci.h
> +++ b/drivers/pci/pci.h
> @@ -335,6 +335,16 @@ static inline void pci_doe_destroy(struct pci_dev *pdev) { }
>   static inline void pci_doe_disconnected(struct pci_dev *pdev) { }
>   #endif
>   
> +#ifdef CONFIG_PCI_TSM
> +void pci_tsm_init(struct pci_dev *pdev);
> +void pci_tsm_destroy(struct pci_dev *pdev);
> +extern const struct attribute_group pci_tsm_attr_group;
> +extern const struct attribute_group pci_tsm_auth_attr_group;
> +#else
> +static inline void pci_tsm_init(struct pci_dev *pdev) { }
> +static inline void pci_tsm_destroy(struct pci_dev *pdev) { }
> +#endif
> +
>   /**
>    * pci_dev_set_io_state - Set the new error state if possible.
>    *
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index 1325fbae2f28..d89b67541d02 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -2481,6 +2481,7 @@ static void pci_init_capabilities(struct pci_dev *dev)
>   	pci_dpc_init(dev);		/* Downstream Port Containment */
>   	pci_rcec_init(dev);		/* Root Complex Event Collector */
>   	pci_doe_init(dev);		/* Data Object Exchange */
> +	pci_tsm_init(dev);		/* TEE Security Manager connection */
>   
>   	pcie_report_downtraining(dev);
>   	pci_init_reset_methods(dev);
> diff --git a/drivers/pci/remove.c b/drivers/pci/remove.c
> index d749ea8250d6..d94b2458934a 100644
> --- a/drivers/pci/remove.c
> +++ b/drivers/pci/remove.c
> @@ -39,6 +39,7 @@ static void pci_destroy_dev(struct pci_dev *dev)
>   	list_del(&dev->bus_list);
>   	up_write(&pci_bus_sem);
>   
> +	pci_tsm_destroy(dev);
>   	pci_doe_destroy(dev);
>   	pcie_aspm_exit_link_state(dev);
>   	pci_bridge_d3_update(dev);
> diff --git a/drivers/pci/tsm.c b/drivers/pci/tsm.c
> new file mode 100644
> index 000000000000..9c5fb2c46662
> --- /dev/null
> +++ b/drivers/pci/tsm.c
> @@ -0,0 +1,270 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * TEE Security Manager for the TEE Device Interface Security Protocol
> + * (TDISP, PCIe r6.1 sec 11)
> + *
> + * Copyright(c) 2024 Intel Corporation. All rights reserved.
> + */
> +
> +#define dev_fmt(fmt) "TSM: " fmt
> +
> +#include <linux/pci.h>
> +#include <linux/pci-doe.h>
> +#include <linux/sysfs.h>
> +#include <linux/xarray.h>
> +#include <linux/pci-tsm.h>
> +#include <linux/bitfield.h>
> +#include "pci.h"
> +
> +/* collect TSM capable devices to rendezvous with the tsm driver */
> +static DEFINE_XARRAY(pci_tsm_devs);

imho either this or pci_dev::tsm is enough but not necessarily both.

> +
> +/*
> + * Provide a read/write lock against the init / exit of pdev tsm
> + * capabilities and arrival/departure of a tsm instance
> + */
> +static DECLARE_RWSEM(pci_tsm_rwsem);
> +static const struct pci_tsm_ops *tsm_ops;
> +
> +static int pci_tsm_disconnect(struct pci_dev *pdev)
> +{
> +	struct pci_tsm *pci_tsm = pdev->tsm;
> +
> +	lockdep_assert_held_read(&pci_tsm_rwsem);
> +	scoped_cond_guard(mutex_intr, return -EINTR, &pci_tsm->exec_lock) {
> +		int rc;
> +
> +		if (pci_tsm->state < PCI_TSM_CONNECT)
> +			return 0;
> +		if (pci_tsm->state < PCI_TSM_INIT)
> +			return -ENXIO;
> +
> +		rc = tsm_ops->exec(pdev, TSM_EXEC_DISCONNECT);
> +		if (rc)
> +			return rc;
> +		pci_tsm->state = PCI_TSM_INIT;
> +	}
> +	return 0;
> +}
> +
> +static int pci_tsm_connect(struct pci_dev *pdev)
> +{
> +	struct pci_tsm *pci_tsm = pdev->tsm;
> +
> +	lockdep_assert_held_read(&pci_tsm_rwsem);
> +	scoped_cond_guard(mutex_intr, return -EINTR, &pci_tsm->exec_lock) {
> +		int rc;
> +
> +		if (pci_tsm->state >= PCI_TSM_CONNECT)
> +			return 0;
> +		if (pci_tsm->state < PCI_TSM_INIT)
> +			return -ENXIO;
> +
> +		rc = tsm_ops->exec(pdev, TSM_EXEC_CONNECT);
> +		if (rc)
> +			return rc;
> +		pci_tsm->state = PCI_TSM_CONNECT;
> +	}
> +	return 0;
> +}
> +
> +static ssize_t connect_store(struct device *dev, struct device_attribute *attr,
> +			     const char *buf, size_t len)
> +{
> +	int rc;
> +	bool connect;
> +	struct pci_dev *pdev = to_pci_dev(dev);
> +
> +	rc = kstrtobool(buf, &connect);
> +	if (rc)
> +		return rc;
> +
> +	if (connect)
> +		rc = pci_tsm_connect(pdev);
> +	else
> +		rc = pci_tsm_disconnect(pdev);
> +	if (rc)
> +		return rc;
> +	return len;
> +}
> +
> +static ssize_t connect_show(struct device *dev, struct device_attribute *attr,
> +			    char *buf)
> +{
> +	struct pci_dev *pdev = to_pci_dev(dev);
> +
> +	return sysfs_emit(buf, "%d\n", pdev->tsm->state >= PCI_TSM_CONNECT);
> +}
> +static DEVICE_ATTR_RW(connect);
> +
> +static bool pci_tsm_group_visible(struct kobject *kobj)
> +{
> +	struct device *dev = kobj_to_dev(kobj);
> +	struct pci_dev *pdev = to_pci_dev(dev);
> +
> +	if (pdev->tsm && pdev->tsm->state > PCI_TSM_IDLE)
> +		return true;
> +	return false;
> +}
> +DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE(pci_tsm);
> +
> +static struct attribute *pci_tsm_attrs[] = {
> +	&dev_attr_connect.attr,
> +	NULL,
> +};
> +
> +const struct attribute_group pci_tsm_attr_group = {
> +	.name = "tsm",
> +	.attrs = pci_tsm_attrs,
> +	.is_visible = SYSFS_GROUP_VISIBLE(pci_tsm),
> +};
> +
> +static ssize_t authenticated_show(struct device *dev,
> +				  struct device_attribute *attr, char *buf)
> +{
> +	/*
> +	 * When device authentication is TSM owned, 'authenticated' is
> +	 * identical to the connect state.
> +	 */
> +	return connect_show(dev, attr, buf);
> +}
> +static DEVICE_ATTR_RO(authenticated);
> +
> +static struct attribute *pci_tsm_auth_attrs[] = {
> +	&dev_attr_authenticated.attr,
> +	NULL,
> +};
> +
> +const struct attribute_group pci_tsm_auth_attr_group = {
> +	.attrs = pci_tsm_auth_attrs,
> +	.is_visible = SYSFS_GROUP_VISIBLE(pci_tsm),
> +};
> +
> +static int pci_tsm_add(struct pci_dev *pdev)
> +{
> +	struct pci_tsm *pci_tsm = pdev->tsm;
> +
> +	lockdep_assert_held(&pci_tsm_rwsem);
> +	if (!tsm_ops)
> +		return 0;
> +	if (pci_tsm->state < PCI_TSM_INIT) {
> +		int rc = tsm_ops->add(pdev);
> +
> +		if (rc)
> +			return rc;
> +	}
> +	pci_tsm->state = PCI_TSM_INIT;
> +	return sysfs_update_group(&pdev->dev.kobj, &pci_tsm_attr_group);
> +}
> +
> +static void pci_tsm_del(struct pci_dev *pdev)
> +{
> +	struct pci_tsm *pci_tsm = pdev->tsm;
> +
> +	lockdep_assert_held(&pci_tsm_rwsem);
> +	/* shutdown sysfs operations before tsm delete */
> +	scoped_guard(mutex, &pdev->tsm->exec_lock)
> +		pci_tsm->state = PCI_TSM_IDLE;
> +	sysfs_update_group(&pdev->dev.kobj, &pci_tsm_attr_group);
> +	tsm_ops->del(pdev);
> +}
> +
> +int pci_tsm_register(const struct pci_tsm_ops *ops)
> +{
> +	struct pci_dev *pdev;
> +	unsigned long index;
> +
> +	if (!ops)
> +		return 0;
> +	guard(rwsem_write)(&pci_tsm_rwsem);
> +	if (tsm_ops)
> +		return -EBUSY;
> +	tsm_ops = ops;
> +	xa_for_each(&pci_tsm_devs, index, pdev)
> +		pci_tsm_add(pdev);
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(pci_tsm_register);
> +
> +void pci_tsm_unregister(const struct pci_tsm_ops *ops)
> +{
> +	struct pci_dev *pdev;
> +	unsigned long index;
> +
> +	if (!ops)
> +		return;
> +	guard(rwsem_write)(&pci_tsm_rwsem);
> +	if (ops != tsm_ops)
> +		return;
> +	xa_for_each(&pci_tsm_devs, index, pdev)
> +		pci_tsm_del(pdev);
> +	tsm_ops = NULL;
> +}
> +EXPORT_SYMBOL_GPL(pci_tsm_unregister);
> +
> +int pci_tsm_doe_transfer(struct pci_dev *pdev, enum pci_doe_proto type,
> +			 const void *req, size_t req_sz, void *resp,
> +			 size_t resp_sz)
> +{
> +	if (!pdev->tsm || !pdev->tsm->doe_mb)
> +		return -ENXIO;
> +
> +	return pci_doe(pdev->tsm->doe_mb, PCI_VENDOR_ID_PCI_SIG, type, req,
> +		       req_sz, resp, resp_sz);
> +}
> +EXPORT_SYMBOL_GPL(pci_tsm_doe_transfer);
> +
> +static unsigned long pci_tsm_devid(struct pci_dev *pdev)
> +{
> +	return FIELD_PREP(GENMASK(15, 0),
> +			  PCI_DEVID(pdev->bus->number, pdev->devfn)) |
> +	       FIELD_PREP(GENMASK(31, 16), pci_domain_nr(pdev->bus));
> +}
> +
> +void pci_tsm_init(struct pci_dev *pdev)
> +{
> +	bool tee_cap;
> +	u16 ide_cap;
> +	int rc;
> +
> +	ide_cap = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_IDE);
> +	tee_cap = pdev->devcap & PCI_EXP_DEVCAP_TEE;
> +	if (ide_cap || tee_cap)

I'd swap if with else.

> +		pci_dbg(pdev,
> +			"Device security capabailities detected (%s%s ), init TSM\n",

capabailities

> +			ide_cap ? " ide" : "", tee_cap ? " tee" : "");
> +	else
> +		return;


If (!ide_cap && tee_cap), we get here but doing the below does not make 
sense for TEE (which are likely to be VFs).


> +	struct pci_tsm *pci_tsm __free(kfree) = kzalloc(sizeof(*pci_tsm), GFP_KERNEL);
> +	if (!pci_tsm)
> +		return;
> +
> +	pci_tsm->ide_cap = ide_cap;
> +	mutex_init(&pci_tsm->exec_lock);
> +
> +	pci_tsm->doe_mb = pci_find_doe_mailbox(pdev, PCI_VENDOR_ID_PCI_SIG,
> +					       PCI_DOE_PROTO_CMA);
> +	if (!pci_tsm->doe_mb)
> +		return;
> +
> +	rc = xa_insert(&pci_tsm_devs, pci_tsm_devid(pdev), pdev, GFP_KERNEL);
> +	if (rc) {
> +		pci_dbg(pdev, "failed to register TSM capable device\n");
> +		return;
> +	}
> +
> +	guard(rwsem_write)(&pci_tsm_rwsem);
> +	pdev->tsm = no_free_ptr(pci_tsm);
> +	pci_tsm_add(pdev);
> +}
> +
> +void pci_tsm_destroy(struct pci_dev *pdev)
> +{
> +	guard(rwsem_write)(&pci_tsm_rwsem);
> +	pci_tsm_del(pdev);
> +	xa_erase(&pci_tsm_devs, pci_tsm_devid(pdev));
> +	kfree(pdev->tsm);
> +	pdev->tsm = NULL;
> +}
> diff --git a/drivers/virt/coco/host/tsm-core.c b/drivers/virt/coco/host/tsm-core.c
> index 0ee738fc40ed..994a7f77c5c9 100644
> --- a/drivers/virt/coco/host/tsm-core.c
> +++ b/drivers/virt/coco/host/tsm-core.c
> @@ -8,11 +8,13 @@
>   #include <linux/device.h>
>   #include <linux/module.h>
>   #include <linux/cleanup.h>
> +#include <linux/pci-tsm.h>
>   
>   static DECLARE_RWSEM(tsm_core_rwsem);
>   static struct class *tsm_class;
>   static struct tsm_subsys {
>   	struct device dev;
> +	const struct pci_tsm_ops *pci_ops;
>   } *tsm_subsys;
>   
>   static struct tsm_subsys *
> @@ -40,7 +42,8 @@ static void put_tsm_subsys(struct tsm_subsys *subsys)
>   DEFINE_FREE(put_tsm_subsys, struct tsm_subsys *,
>   	    if (!IS_ERR_OR_NULL(_T)) put_tsm_subsys(_T))
>   struct tsm_subsys *tsm_register(struct device *parent,
> -				const struct attribute_group **groups)
> +				const struct attribute_group **groups,
> +				const struct pci_tsm_ops *pci_ops)
>   {
>   	struct device *dev;
>   	int rc;
> @@ -62,10 +65,20 @@ struct tsm_subsys *tsm_register(struct device *parent,
>   	if (rc)
>   		return ERR_PTR(rc);
>   
> +	rc = pci_tsm_register(pci_ops);
> +	if (rc) {
> +		dev_err(parent, "PCI initialization failure: %pe\n",
> +			ERR_PTR(rc));
> +		return ERR_PTR(rc);
> +	}
> +
>   	rc = device_add(dev);
> -	if (rc)
> +	if (rc) {
> +		pci_tsm_unregister(pci_ops);
>   		return ERR_PTR(rc);
> +	}
>   
> +	subsys->pci_ops = pci_ops;
>   	tsm_subsys = no_free_ptr(subsys);
>   
>   	return tsm_subsys;
> @@ -74,13 +87,18 @@ EXPORT_SYMBOL_GPL(tsm_register);
>   
>   void tsm_unregister(struct tsm_subsys *subsys)
>   {
> +	const struct pci_tsm_ops *pci_ops;
> +
>   	guard(rwsem_write)(&tsm_core_rwsem);
>   	if (!tsm_subsys || subsys != tsm_subsys) {
>   		pr_warn("failed to unregister, not currently registered\n");
>   		return;
>   	}
>   
> +	pci_ops = subsys->pci_ops;
>   	device_unregister(&subsys->dev);
> +
> +	pci_tsm_unregister(pci_ops);
>   	tsm_subsys = NULL;
>   }
>   EXPORT_SYMBOL_GPL(tsm_unregister);
> diff --git a/include/linux/pci-tsm.h b/include/linux/pci-tsm.h
> new file mode 100644
> index 000000000000..d17f5e0574c4
> --- /dev/null
> +++ b/include/linux/pci-tsm.h
> @@ -0,0 +1,80 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __PCI_TSM_H
> +#define __PCI_TSM_H
> +#include <linux/mutex.h>
> +
> +enum pci_tsm_cmd {
> +	TSM_EXEC_CONNECT,
> +	TSM_EXEC_DISCONNECT,
> +};
> +
> +struct pci_dev;
> +/**
> + * struct pci_tsm_ops - Low-level TSM-exported interface to the PCI core
> + * @add: accept device for tsm operation


What does "accept" means here? "Accept" sounds like some action needed 
from something but this is what exec() for. So far I have not noticed 
that allocating platform-specific structures prior calling a verb is any 
useful, i.e. having a separate state - PCI_TSM_INIT - is just an extra 
noise in the "painfully simple first TSM implementation".


> + * @del: teardown tsm context for @pdev
> + * @exec: synchronously execute @cmd
> + *
> + * Note that @add, and @del run in down_write(&pci_tsm_rswem) context to
> + * synchronize with TSM driver bind/unbind events and
> + * pci_device_add()/pci_destroy_dev(). @exec runs in
> + * @pdev->tsm->exec_lock context to synchronize @exec results with
> + * @pdev->tsm->state
> + */
> +struct pci_tsm_ops {
> +	int (*add)(struct pci_dev *pdev);
> +	void (*del)(struct pci_dev *pdev);
> +	int (*exec)(struct pci_dev *pdev, enum pci_tsm_cmd cmd);


A nit: the verbs seem working (especially reducing the amount of 
cut-n-paste of all this spdm forwarding) until I get to things like 
"get_status" which returns a structure to dump in the sysfs. Doing it 
like this means adding a structure in pci_tsm and manage its state 
(valid/partial/empty/...). Or we might want to generalize some input 
parameters for the verbs, adding u64 params is meh.


> +};
> +
> +enum pci_tsm_state {
> +	PCI_TSM_IDLE,
> +	PCI_TSM_INIT,
> +	PCI_TSM_CONNECT,
> +};
> +
> +/**
> + * struct pci_tsm - per device TSM context
> + * @state: reflect device initialized, connected, or bound
> + * @ide_cap: PCIe IDE Extended Capability offset
> + * @exec_lock: protect @state vs pci_tsm_ops.exec() results
> + * @doe_mb: PCIe Data Object Exchange mailbox
> + * @tsm_data: TSM driver private context
> + */
> +struct pci_tsm {
> +	enum pci_tsm_state state;
> +	u16 ide_cap;
> +	struct mutex exec_lock;
> +	struct pci_doe_mb *doe_mb;
> +	void *tsm_data;
> +};
> +
> +enum pci_doe_proto {
> +	PCI_DOE_PROTO_CMA = 1,
> +	PCI_DOE_PROTO_SSESSION = 2,
> +};
> +
> +#ifdef CONFIG_PCI_TSM
> +int pci_tsm_register(const struct pci_tsm_ops *ops);
> +void pci_tsm_unregister(const struct pci_tsm_ops *ops);
> +int pci_tsm_doe_transfer(struct pci_dev *pdev, enum pci_doe_proto type,
> +			 const void *req, size_t req_sz, void *resp,
> +			 size_t resp_sz);
> +#else
> +static inline int pci_tsm_register(const struct pci_tsm_ops *ops)
> +{
> +	return 0;
> +}
> +static inline void pci_tsm_unregister(const struct pci_tsm_ops *ops)
> +{
> +}
> +static inline int pci_tsm_doe_transfer(struct pci_dev *pdev,
> +				       enum pci_doe_proto type, const void *req,
> +				       size_t req_sz, void *resp,
> +				       size_t resp_sz)
> +{
> +	return -ENOENT;
> +}
> +
> +#endif
> +#endif /*__PCI_TSM_H */
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 16493426a04f..dd4dc8719c5c 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -515,6 +515,9 @@ struct pci_dev {
>   #endif
>   #ifdef CONFIG_PCI_DOE
>   	struct xarray	doe_mbs;	/* Data Object Exchange mailboxes */
> +#endif
> +#ifdef CONFIG_PCI_TSM
> +	struct pci_tsm *tsm;		/* TSM operation state */


I am wondering if pdev->dev.platform_data can be used for this.


>   #endif
>   	u16		acs_cap;	/* ACS Capability offset */
>   	phys_addr_t	rom;		/* Physical address if not from BAR */
> @@ -550,6 +553,12 @@ static inline int pci_channel_offline(struct pci_dev *pdev)
>   	return (pdev->error_state != pci_channel_io_normal);
>   }
>   
> +/* id resources that may be shared across host-bridges */
> +struct pci_hb_id_pool {
> +	int nr_stream_ids;
> +	int nr_cxl_cache_ids;
> +};
> +
>   /*
>    * Currently in ACPI spec, for each PCI host bridge, PCI Segment
>    * Group number is limited to a 16-bit value, therefore (int)-1 is
> @@ -568,6 +577,8 @@ struct pci_host_bridge {
>   	void		*sysdata;
>   	int		busnr;
>   	int		domain_nr;
> +	struct pci_hb_id_pool __pool;
> +	struct pci_hb_id_pool *pool;	/* &self->__pool, unless shared */


What are these for? Something for IDE (which I also have in the works, 
very basic set of wrappers)? Thanks,

>   	struct list_head windows;	/* resource_entry */
>   	struct list_head dma_ranges;	/* dma ranges resource list */
>   	u8 (*swizzle_irq)(struct pci_dev *, u8 *); /* Platform IRQ swizzler */
> diff --git a/include/linux/tsm.h b/include/linux/tsm.h
> index 2867c2ecbd11..6481cc99ea6d 100644
> --- a/include/linux/tsm.h
> +++ b/include/linux/tsm.h
> @@ -68,7 +68,9 @@ int tsm_report_register(const struct tsm_report_ops *ops, void *priv,
>   			const struct config_item_type *type);
>   int tsm_report_unregister(const struct tsm_report_ops *ops);
>   struct tsm_subsys;
> +struct pci_tsm_ops;
>   struct tsm_subsys *tsm_register(struct device *parent,
> -				const struct attribute_group **groups);
> +				const struct attribute_group **groups,
> +				const struct pci_tsm_ops *ops);
>   void tsm_unregister(struct tsm_subsys *subsys);
>   #endif /* __TSM_H */
> diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
> index a39193213ff2..9aaffa379cae 100644
> --- a/include/uapi/linux/pci_regs.h
> +++ b/include/uapi/linux/pci_regs.h
> @@ -498,6 +498,7 @@
>   #define  PCI_EXP_DEVCAP_PWR_VAL	0x03fc0000 /* Slot Power Limit Value */
>   #define  PCI_EXP_DEVCAP_PWR_SCL	0x0c000000 /* Slot Power Limit Scale */
>   #define  PCI_EXP_DEVCAP_FLR     0x10000000 /* Function Level Reset */
> +#define  PCI_EXP_DEVCAP_TEE     0x40000000 /* TEE I/O (TDISP) Support */
>   #define PCI_EXP_DEVCTL		0x08	/* Device Control */
>   #define  PCI_EXP_DEVCTL_CERE	0x0001	/* Correctable Error Reporting En. */
>   #define  PCI_EXP_DEVCTL_NFERE	0x0002	/* Non-Fatal Error Reporting Enable */
> @@ -742,7 +743,8 @@
>   #define PCI_EXT_CAP_ID_PL_16GT	0x26	/* Physical Layer 16.0 GT/s */
>   #define PCI_EXT_CAP_ID_PL_32GT  0x2A    /* Physical Layer 32.0 GT/s */
>   #define PCI_EXT_CAP_ID_DOE	0x2E	/* Data Object Exchange */
> -#define PCI_EXT_CAP_ID_MAX	PCI_EXT_CAP_ID_DOE
> +#define PCI_EXT_CAP_ID_IDE	0x30	/* Integrity and Data Encryption */
> +#define PCI_EXT_CAP_ID_MAX	PCI_EXT_CAP_ID_IDE
>   
>   #define PCI_EXT_CAP_DSN_SIZEOF	12
>   #define PCI_EXT_CAP_MCAST_ENDPOINT_SIZEOF 40
> 

-- 
Alexey

Re: [RFC PATCH v2 5/6] PCI/TSM: Authenticate devices via platform TSM

[Dan Williams]

Bjorn Helgaas wrote:
> On Fri, Apr 12, 2024 at 01:52:13AM -0700, Dan Williams wrote:
> > The PCIe 6.1 specification, section 11, introduces the Trusted Execution
> > Environment (TEE) Device Interface Security Protocol (TDISP).  This
> > interface definition builds upon Component Measurement and
> > Authentication (CMA), and link Integrity and Data Encryption (IDE). It
> > adds support for assigning devices (PCI physical or virtual function) to
> > a confidential VM such that the assigned device is enabled to access
> > guest private memory protected by technologies like Intel TDX, AMD
> > SEV-SNP, RISCV COVE, or ARM CCA.
> > 
> > The "TSM" (TEE Security Manager) is a concept in the TDISP specification
> > of an agent that mediates between a "DSM" (Device Security Manager) and
> > system software in both a VMM and a confidential VM. A VMM uses TSM ABIs
> > to setup link security and assign devices. A confidential VM uses TSM
> > ABIs to transition an assigned device into the TDISP "RUN" state and
> > validate its configuration. From a Linux perspective the TSM abstracts
> > many of the details of TDISP, IDE, and CMA. Some of those details leak
> > through at times, but for the most part TDISP is an internal
> > implementation detail of the TSM.
> > 
> > Similar to the PCI core extensions to support CONFIG_PCI_CMA,
> > CONFIG_PCI_TSM builds upon that to reuse the "authenticated" sysfs
> > attribute, and add more properties + controls in a tsm/ subdirectory of
> > the PCI device sysfs interface. Unlike CMA that can depend on a local to
> > the PCI core implementation, PCI_TSM needs to be prepared for late
> > loading of the platform TSM driver. Consider that the TSM driver may
> > itself be a PCI driver. Userspace can depend on the common TSM device
> > uevent to know when the PCI core has TSM services enabled. The PCI
> > device tsm/ subdirectory is supplemented by the TSM device pci/
> > directory for platform global TSM properties + controls.
> > 
> > The common verbs that the low-level TSM drivers implement are defined by
> > 'enum pci_tsm_cmd'. For now only connect and disconnect are defined for
> > establishing a trust relationship between the host and the device,
> > securing the interconnect (optionally establishing IDE), and tearing
> > that down.
> > 
> > The locking allows for multiple devices to be executing commands
> > simultaneously, one outstanding command per-device and an rwsem flushes
> > all inflight commands when a TSM low-level driver/device is removed.
> > 
> > In addition to commands submitted through an 'exec' operation the
> > low-level TSM driver is notified of device arrival and departure events
> > via 'add' and 'del' operations. With those it can setup per-device
> > context, or filter devices that the TSM is not prepared to support.
> > 
> > Cc: Wu Hao <hao.wu@intel.com>
> > Cc: Yilun Xu <yilun.xu@intel.com>
> > Cc: Lukas Wunner <lukas@wunner.de>
> > Cc: Samuel Ortiz <sameo@rivosinc.com>
> > Cc: Alexey Kardashevskiy <aik@amd.com>
> > Cc: Bjorn Helgaas <bhelgaas@google.com>
> > Co-developed-by: Xu Yilun <yilun.xu@linux.intel.com>
> > Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com>
> > Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> 
> Acked-by: Bjorn Helgaas <bhelgaas@google.com>	# PCI parts

Great, thanks Bjorn! This lets us move forward on the TSM details
knowing that it looks like this meets your expectations on the PCI
integration aspects. Will continue to include you on revisions as this
evolves, but this is good news for the cross vendor collaboration effort
in process here.

> > +What:		/sys/bus/pci/devices/.../tsm/
> > +Date:		March 2024
> > +Contact:	linux-coco@lists.linux.dev
> > +Description:
> > +		This directory only appears if a device supports CMA and IDE,
> > +		and only after a TSM driver has loaded and evaluated this
> > +		PCI device. All present devices shall be dispositioned
> > +		after the 'add' event for /sys/class/tsm/tsm0 triggers.
> 
> What does "dispositioned" mean?

When /sys/class/tsm/tsm0/uevent signals KOBJ_ADD, arrival of the TSM
device, a udev script can assume that the kernel has walked all PCI
devices and toggled the visibility of the all
/sys/bus/pci/devices/$pdev/tsm/ attribute directories.

> What devices does "all present devices" cover?

A snapshot of the state of /sys/bus/pci/devices at TSM arrival. So a
udev script that, for example, wants to perform the TSM "connect"
operation as soon as possible should watch PCI KOBJ_ADD uevents and try
to connect if the TSM is present at that time, but if not, try again
after TSM KOBJ_ADD.

Similar for the PCI-hot-add case. TSM may arrive first, so that policy
script should recheck when PCI KOBJ_ADD, for the device it wants to
connect, fires.

> 
> Is "tsm0" a special global thing?  Is there doc for
> /sys/class/tsm/...?

Yes, it is a special common object that any platform with a TSM will
export. I had yet to create a document since there are no common
attributes defined in this version of the patch set, but I can go ahead
and create it and mention what userspace can expect about the state of
/sys/bus/pci/devices when the device shows up.

> 
> > +++ b/drivers/pci/Makefile
> > @@ -35,6 +35,8 @@ obj-$(CONFIG_VGA_ARB)		+= vgaarb.o
> >  obj-$(CONFIG_PCI_DOE)		+= doe.o
> >  obj-$(CONFIG_PCI_DYNAMIC_OF_NODES) += of_property.o
> >  
> > +obj-$(CONFIG_PCI_TSM)		+= tsm.o
> 
> Maybe put it next to CONFIG_PCI_DOE or at least not off in a special
> separate list?

Sure, will do.

Re: [RFC PATCH v2 5/6] PCI/TSM: Authenticate devices via platform TSM

[Dan Williams]

Alexey Kardashevskiy wrote:
> On 12/4/24 18:52, Dan Williams wrote:
> > The PCIe 6.1 specification, section 11, introduces the Trusted Execution
> > Environment (TEE) Device Interface Security Protocol (TDISP).  This
> > interface definition builds upon Component Measurement and
> > Authentication (CMA), and link Integrity and Data Encryption (IDE). It
> > adds support for assigning devices (PCI physical or virtual function) to
> > a confidential VM such that the assigned device is enabled to access
> > guest private memory protected by technologies like Intel TDX, AMD
> > SEV-SNP, RISCV COVE, or ARM CCA.
> > 
> > The "TSM" (TEE Security Manager) is a concept in the TDISP specification
> > of an agent that mediates between a "DSM" (Device Security Manager) and
> > system software in both a VMM and a confidential VM. A VMM uses TSM ABIs
> > to setup link security and assign devices. A confidential VM uses TSM
> > ABIs to transition an assigned device into the TDISP "RUN" state and
> > validate its configuration. From a Linux perspective the TSM abstracts
> > many of the details of TDISP, IDE, and CMA. Some of those details leak
> > through at times, but for the most part TDISP is an internal
> > implementation detail of the TSM.
> > 
> > Similar to the PCI core extensions to support CONFIG_PCI_CMA,
> > CONFIG_PCI_TSM builds upon that to reuse the "authenticated" sysfs
> > attribute, and add more properties + controls in a tsm/ subdirectory of
> > the PCI device sysfs interface. Unlike CMA that can depend on a local to
> > the PCI core implementation, PCI_TSM needs to be prepared for late
> > loading of the platform TSM driver. Consider that the TSM driver may
> > itself be a PCI driver. Userspace can depend on the common TSM device
> > uevent to know when the PCI core has TSM services enabled. The PCI
> > device tsm/ subdirectory is supplemented by the TSM device pci/
> > directory for platform global TSM properties + controls.
> > 
> > The common verbs that the low-level TSM drivers implement are defined by
> > 'enum pci_tsm_cmd'. For now only connect and disconnect are defined for
> > establishing a trust relationship between the host and the device,
> > securing the interconnect (optionally establishing IDE), and tearing
> > that down.
> > 
> > The locking allows for multiple devices to be executing commands
> > simultaneously, one outstanding command per-device and an rwsem flushes
> > all inflight commands when a TSM low-level driver/device is removed.
> > 
> > In addition to commands submitted through an 'exec' operation the
> > low-level TSM driver is notified of device arrival and departure events
> > via 'add' and 'del' operations. With those it can setup per-device
> > context, or filter devices that the TSM is not prepared to support.
> > 
> > Cc: Wu Hao <hao.wu@intel.com>
> > Cc: Yilun Xu <yilun.xu@intel.com>
> > Cc: Lukas Wunner <lukas@wunner.de>
> > Cc: Samuel Ortiz <sameo@rivosinc.com>
> > Cc: Alexey Kardashevskiy <aik@amd.com>
> > Cc: Bjorn Helgaas <bhelgaas@google.com>
> > Co-developed-by: Xu Yilun <yilun.xu@linux.intel.com>
> > Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com>
> > Signed-off-by: Dan Williams <dan.j.williams@intel.com>
[..]
> > diff --git a/drivers/pci/tsm.c b/drivers/pci/tsm.c
> > new file mode 100644
> > index 000000000000..9c5fb2c46662
> > --- /dev/null
> > +++ b/drivers/pci/tsm.c
> > @@ -0,0 +1,270 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * TEE Security Manager for the TEE Device Interface Security Protocol
> > + * (TDISP, PCIe r6.1 sec 11)
> > + *
> > + * Copyright(c) 2024 Intel Corporation. All rights reserved.
> > + */
> > +
> > +#define dev_fmt(fmt) "TSM: " fmt
> > +
> > +#include <linux/pci.h>
> > +#include <linux/pci-doe.h>
> > +#include <linux/sysfs.h>
> > +#include <linux/xarray.h>
> > +#include <linux/pci-tsm.h>
> > +#include <linux/bitfield.h>
> > +#include "pci.h"
> > +
> > +/* collect TSM capable devices to rendezvous with the tsm driver */
> > +static DEFINE_XARRAY(pci_tsm_devs);
> 
> imho either this or pci_dev::tsm is enough but not necessarily both.

You mean:

s/pci_tsm_devs/tsm_devs/

?

[..]
> > +void pci_tsm_init(struct pci_dev *pdev)
> > +{
> > +	bool tee_cap;
> > +	u16 ide_cap;
> > +	int rc;
> > +
> > +	ide_cap = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_IDE);
> > +	tee_cap = pdev->devcap & PCI_EXP_DEVCAP_TEE;
> > +	if (ide_cap || tee_cap)
> 
> I'd swap if with else.

Oh, you mean:

	if (!(ide_cap || tee_cap))
		return;

?

> 
> > +		pci_dbg(pdev,
> > +			"Device security capabailities detected (%s%s ), init TSM\n",
> 
> capabailities

Incapabail of spelling correctly apparently. checkpatch spell check let
me down.

Fixed.

> 
> > +			ide_cap ? " ide" : "", tee_cap ? " tee" : "");
> > +	else
> > +		return;
> 
> 
> If (!ide_cap && tee_cap), we get here but doing the below does not make 
> sense for TEE (which are likely to be VFs).

The "!ide_cap && tee_cap" case may also be the "TSM wants to setup IDE
without TDISP flow".

[..]
> > +struct pci_dev;
> > +/**
> > + * struct pci_tsm_ops - Low-level TSM-exported interface to the PCI core
> > + * @add: accept device for tsm operation
> 
> 
> What does "accept" means here? "Accept" sounds like some action needed 
> from something but this is what exec() for.

At the lowest level an "accepted" device, one that returns successfully
from "->add()" and has its tsm/ attribute group in sysfs enabled.

> So far I have not noticed that allocating platform-specific structures
> prior calling a verb is any useful, i.e. having a separate state -
> PCI_TSM_INIT - is just an extra noise in the "painfully simple first
> TSM implementation".

As far as I know, not all TSM implementations care about the "!ide_cap
&& tee_cap" case. That said, regarding painfully simple, if the only
difference is some TSMs support IDE without TDISP and some do not, that
standalone-IDE support can be added later.

> > + * @del: teardown tsm context for @pdev
> > + * @exec: synchronously execute @cmd
> > + *
> > + * Note that @add, and @del run in down_write(&pci_tsm_rswem) context to
> > + * synchronize with TSM driver bind/unbind events and
> > + * pci_device_add()/pci_destroy_dev(). @exec runs in
> > + * @pdev->tsm->exec_lock context to synchronize @exec results with
> > + * @pdev->tsm->state
> > + */
> > +struct pci_tsm_ops {
> > +	int (*add)(struct pci_dev *pdev);
> > +	void (*del)(struct pci_dev *pdev);
> > +	int (*exec)(struct pci_dev *pdev, enum pci_tsm_cmd cmd);
> 
> 
> A nit: the verbs seem working (especially reducing the amount of 
> cut-n-paste of all this spdm forwarding) until I get to things like 
> "get_status" which returns a structure to dump in the sysfs. Doing it 
> like this means adding a structure in pci_tsm and manage its state 
> (valid/partial/empty/...). Or we might want to generalize some input 
> parameters for the verbs, adding u64 params is meh.

If it is just for sysfs cases then I would just have a facility for low
level TSM drivers to publish some attributes directly at
pci_tsm_register time. Something like the following, and just require
low level TSM implementations to agree on filenames and formats, but
otherwise avoid complicating ->exec().

diff --git a/drivers/pci/tsm.c b/drivers/pci/tsm.c
index 44b998a1c824..d34c3477ddc3 100644
--- a/drivers/pci/tsm.c
+++ b/drivers/pci/tsm.c
@@ -113,9 +113,9 @@ static struct attribute *pci_tsm_attrs[] = {
 	NULL,
 };
 
-const struct attribute_group pci_tsm_attr_group = {
+struct attribute_group pci_tsm_attr_group = {
 	.name = "tsm",
-	.attrs = pci_tsm_attrs,
+	.attrs = pci_tsm_default_attrs,
 	.is_visible = SYSFS_GROUP_VISIBLE(pci_tsm),
 };
 
@@ -169,7 +169,7 @@ static void pci_tsm_del(struct pci_dev *pdev)
 	tsm_ops->del(pdev);
 }
 
-int pci_tsm_register(const struct pci_tsm_ops *ops)
+int pci_tsm_register(const struct pci_tsm_ops *ops, struct attribute **attrs)
 {
 	struct pci_dev *pdev;
 	unsigned long index;
@@ -180,6 +180,7 @@ int pci_tsm_register(const struct pci_tsm_ops *ops)
 	if (tsm_ops)
 		return -EBUSY;
 	tsm_ops = ops;
+	pci_tsm_attr_group.attrs = attrs;
 	xa_for_each(&pci_tsm_devs, index, pdev)
 		pci_tsm_add(pdev);
 	return 0;
@@ -198,6 +199,7 @@ void pci_tsm_unregister(const struct pci_tsm_ops *ops)
 		return;
 	xa_for_each(&pci_tsm_devs, index, pdev)
 		pci_tsm_del(pdev);
+	pci_tsm_attr_group.attrs = pci_tsm_default_attrs;
 	tsm_ops = NULL;
 }
 EXPORT_SYMBOL_GPL(pci_tsm_unregister);

> > diff --git a/include/linux/pci.h b/include/linux/pci.h
> > index 16493426a04f..dd4dc8719c5c 100644
> > --- a/include/linux/pci.h
> > +++ b/include/linux/pci.h
> > @@ -515,6 +515,9 @@ struct pci_dev {
> >   #endif
> >   #ifdef CONFIG_PCI_DOE
> >   	struct xarray	doe_mbs;	/* Data Object Exchange mailboxes */
> > +#endif
> > +#ifdef CONFIG_PCI_TSM
> > +	struct pci_tsm *tsm;		/* TSM operation state */
> 
> 
> I am wondering if pdev->dev.platform_data can be used for this.

No, platform_data is for ACPI or OF to populate.

For example, see sst_acpi_probe().

> 
> 
> >   #endif
> >   	u16		acs_cap;	/* ACS Capability offset */
> >   	phys_addr_t	rom;		/* Physical address if not from BAR */
> > @@ -550,6 +553,12 @@ static inline int pci_channel_offline(struct pci_dev *pdev)
> >   	return (pdev->error_state != pci_channel_io_normal);
> >   }
> >   
> > +/* id resources that may be shared across host-bridges */
> > +struct pci_hb_id_pool {
> > +	int nr_stream_ids;
> > +	int nr_cxl_cache_ids;
> > +};
> > +
> >   /*
> >    * Currently in ACPI spec, for each PCI host bridge, PCI Segment
> >    * Group number is limited to a 16-bit value, therefore (int)-1 is
> > @@ -568,6 +577,8 @@ struct pci_host_bridge {
> >   	void		*sysdata;
> >   	int		busnr;
> >   	int		domain_nr;
> > +	struct pci_hb_id_pool __pool;
> > +	struct pci_hb_id_pool *pool;	/* &self->__pool, unless shared */
> 
> 
> What are these for? Something for IDE (which I also have in the works, 
> very basic set of wrappers)? Thanks,

Yes, this is something I had sitting in my tree as a rough draft, but
did not intend to send out, but I guess a useful accident. Hao Wu has
taken this concept further. @Hao, lets post what we are thinking here.

The concept is that stream-ids are a limited resource. Host bridges, at
least Intel ones, might share their stream-id pool with another
host-bridge. Do AMD platforms have similar constraints?

The end goal is to be able to convey to a system owner which devices are
consuming which stream-ids and which host-bridges have available
stream-ids to allocate.

Re: [RFC PATCH v2 5/6] PCI/TSM: Authenticate devices via platform TSM

[Xu Yilun]

> > If (!ide_cap && tee_cap), we get here but doing the below does not make 
> > sense for TEE (which are likely to be VFs).
> 
> The "!ide_cap && tee_cap" case may also be the "TSM wants to setup IDE
> without TDISP flow".

IIUC, should be "TSM wants to setup TDISP without IDE flow"?

But I think aik is talking about VFs (which fit "!ide_cap && tee_cap"),
VFs should not be rejected by the following:

      pci_tsm->doe_mb = pci_find_doe_mailbox(pdev, PCI_VENDOR_ID_PCI_SIG,
                                             PCI_DOE_PROTO_CMA);
      if (!pci_tsm->doe_mb)
              return;

VF should check its PF's doe/ide/tee cap and then be added to
pci_tsm_devs, is it?

Thanks,
Yilun

Re: [RFC PATCH v2 5/6] PCI/TSM: Authenticate devices via platform TSM

[Xu Yilun]

> > > +
> > > +/* collect TSM capable devices to rendezvous with the tsm driver */
> > > +static DEFINE_XARRAY(pci_tsm_devs);
> > 
> > imho either this or pci_dev::tsm is enough but not necessarily both.
> 
> You mean:
> 
> s/pci_tsm_devs/tsm_devs/
> 
> ?

I don't think the concern is just a renaming. My understanding is, we
already have a struct pci_tsm embedded in struct pci_dev, we could loop
and find all TSM capable devices by:

	for_each_pci_dev(pdev) {
		if (pdev->tsm)
			pci_tsm_add/del(pdev);
	}

A dedicated list for TSM capable devices seems not necessary.

But my concern is about VFs.  VFs are as well TSM capable but not
applicable for tsm_ops->exec(TSM_EXEC_CONNECT), maybe not applicable
for tsm_ops->add() either.  One way to distinguish PF/VFs is we only
collect PFs in pci_tsm_devs, but all TSM capable devices have
valid pci_dev::tsm pointer.

TSM capable devices in Guest should not been collected in pci_tsm_devs
either.

Thanks,
Yilun

Re: [RFC PATCH v2 5/6] PCI/TSM: Authenticate devices via platform TSM

[Dan Williams]

Xu Yilun wrote:
> > > If (!ide_cap && tee_cap), we get here but doing the below does not make 
> > > sense for TEE (which are likely to be VFs).
> > 
> > The "!ide_cap && tee_cap" case may also be the "TSM wants to setup IDE
> > without TDISP flow".
> 
> IIUC, should be "TSM wants to setup TDISP without IDE flow"?

Both are possible. The TSM may need to be involved in IDE key
establishment even if the PF or its VFs are ever assigned as TDIs. Also,
as you say, it is possible for a TSM to trust that a device does not
need IDE established because it is has knowledge that the device is
integrated into the SOC without physical exposure of its links.

> But I think aik is talking about VFs (which fit "!ide_cap && tee_cap"),
> VFs should not be rejected by the following:
> 
>       pci_tsm->doe_mb = pci_find_doe_mailbox(pdev, PCI_VENDOR_ID_PCI_SIG,
>                                              PCI_DOE_PROTO_CMA);
>       if (!pci_tsm->doe_mb)
>               return;
> 
> VF should check its PF's doe/ide/tee cap and then be added to
> pci_tsm_devs, is it?

This path should probably skip VFs because the 'connect' operation is a
PF-only semantic. I will fix that up.

I still think the PF needs to go through an ->add() callback because I
do not think we have a cross-vendor unified concept of when IDE without
TDISP, or TDISP without IDE is supported.

Re: [RFC PATCH v2 5/6] PCI/TSM: Authenticate devices via platform TSM

[Dan Williams]

Xu Yilun wrote:
> > > > +
> > > > +/* collect TSM capable devices to rendezvous with the tsm driver */
> > > > +static DEFINE_XARRAY(pci_tsm_devs);
> > > 
> > > imho either this or pci_dev::tsm is enough but not necessarily both.
> > 
> > You mean:
> > 
> > s/pci_tsm_devs/tsm_devs/
> > 
> > ?
> 
> I don't think the concern is just a renaming. My understanding is, we
> already have a struct pci_tsm embedded in struct pci_dev, we could loop
> and find all TSM capable devices by:
> 
> 	for_each_pci_dev(pdev) {
> 		if (pdev->tsm)
> 			pci_tsm_add/del(pdev);
> 	}
> A dedicated list for TSM capable devices seems not necessary.

Not the first time this criticism has been raised against pci_tsm_devs.
I think for_each_pci_dev() is potentially wasteful, but it is trivial to
add back if for_each_pci_dev() scanning becomes too expensive.

> But my concern is about VFs.  VFs are as well TSM capable but not
> applicable for tsm_ops->exec(TSM_EXEC_CONNECT), maybe not applicable
> for tsm_ops->add() either.  One way to distinguish PF/VFs is we only
> collect PFs in pci_tsm_devs, but all TSM capable devices have
> valid pci_dev::tsm pointer.
> 
> TSM capable devices in Guest should not been collected in pci_tsm_devs
> either.

Yes, for this initial phase of the enabling only PF operations are
relevant.

Re: [RFC PATCH v2 5/6] PCI/TSM: Authenticate devices via platform TSM

[Xu Yilun]

On Tue, May 07, 2024 at 11:21:37AM -0700, Dan Williams wrote:
> Xu Yilun wrote:
> > > > If (!ide_cap && tee_cap), we get here but doing the below does not make 
> > > > sense for TEE (which are likely to be VFs).
> > > 
> > > The "!ide_cap && tee_cap" case may also be the "TSM wants to setup IDE
> > > without TDISP flow".
> > 
> > IIUC, should be "TSM wants to setup TDISP without IDE flow"?
> 
> Both are possible. The TSM may need to be involved in IDE key
> establishment even if the PF or its VFs are ever assigned as TDIs. Also,
> as you say, it is possible for a TSM to trust that a device does not
> need IDE established because it is has knowledge that the device is
> integrated into the SOC without physical exposure of its links.
> 
> > But I think aik is talking about VFs (which fit "!ide_cap && tee_cap"),
> > VFs should not be rejected by the following:
> > 
> >       pci_tsm->doe_mb = pci_find_doe_mailbox(pdev, PCI_VENDOR_ID_PCI_SIG,
> >                                              PCI_DOE_PROTO_CMA);
> >       if (!pci_tsm->doe_mb)
> >               return;
> > 
> > VF should check its PF's doe/ide/tee cap and then be added to
> > pci_tsm_devs, is it?
> 
> This path should probably skip VFs because the 'connect' operation is a
> PF-only semantic. I will fix that up.

Agreed. I drafted some simple changes for the idea, that we keep
pci_dev::tsm for every TEE capable device (PF & VF) to execute tsm_ops,
but only adds PFs to pci_tsm_devs for "connect".


diff --git a/drivers/pci/tsm.c b/drivers/pci/tsm.c
index 9c5fb2c46662..31707f0351c6 100644
--- a/drivers/pci/tsm.c
+++ b/drivers/pci/tsm.c
@@ -241,9 +241,14 @@ void pci_tsm_init(struct pci_dev *pdev)
        if (!pci_tsm)
                return;

-       pci_tsm->ide_cap = ide_cap;
        mutex_init(&pci_tsm->exec_lock);

+       if (pdev->is_virtfn) {
+               pdev->tsm = no_free_ptr(pci_tsm);
+               return;
+       }
+
+       pci_tsm->ide_cap = ide_cap;
        pci_tsm->doe_mb = pci_find_doe_mailbox(pdev, PCI_VENDOR_ID_PCI_SIG,
                                               PCI_DOE_PROTO_CMA);
        if (!pci_tsm->doe_mb)
@@ -262,9 +267,14 @@ void pci_tsm_init(struct pci_dev *pdev)

 void pci_tsm_destroy(struct pci_dev *pdev)
 {
+       if (!pdev->tsm)
+               return;
+
        guard(rwsem_write)(&pci_tsm_rwsem);
-       pci_tsm_del(pdev);
-       xa_erase(&pci_tsm_devs, pci_tsm_devid(pdev));
+       if (!pdev->is_virtfn) {
+               pci_tsm_del(pdev);
+               xa_erase(&pci_tsm_devs, pci_tsm_devid(pdev));
+       }
        kfree(pdev->tsm);
        pdev->tsm = NULL;
 }

Thanks,
Yilun

> 
> I still think the PF needs to go through an ->add() callback because I
> do not think we have a cross-vendor unified concept of when IDE without
> TDISP, or TDISP without IDE is supported.