[PATCH 0/2] i2c-designware: Add support for AMD PSP semaphore

[PATCH 0/2] i2c-designware: Add support for AMD PSP semaphore

[Jan Dabros]

This patchset comprises support for new i2c-designware controller setup on some
AMD Cezanne SoCs, where x86 is sharing i2c bus with PSP. PSP uses the same
controller and acts as an i2c arbitrator there (x86 is leasing bus from it).

First commit aims to improve generic i2c-designware code by adding extra locking
on probe() and disable() paths. I would like to ask someone with access to
boards which use Intel BayTrail(CONFIG_I2C_DESIGNWARE_BAYTRAIL) to verify
behavior of my changes on such setup.

Second commit adds support for new PSP semaphore arbitration mechanism.
Implementation is similar to the one from i2c-designware-baytrail.c however
there are two main differences:
1) Add new ACPI ID in order to protect against silent binding of the old driver
to the setup with PSP semaphore. Extra flag ARBITRATION_SEMAPHORE added to this
new _HID allows to recognize setup with PSP.
2) Beside acquire_lock() and release_lock() methods we are also applying quirks
to the lock_bus() and unlock_bus() global adapter methods. With this in place
all i2c clients drivers may lock i2c bus for a desired number of i2c
transactions (e.g. write-wait-read) without being aware of that such bus is
shared with another entity.

This patchset is a follow-up to the RFC sent earlier on LKML [1], with review
comments applied.

Looking forward to some feedback.

[1] https://lkml.org/lkml/2021/12/22/219

Jan Dabros (2):
  i2c: designware: Add missing locks
  i2c: designware: Add AMD PSP I2C bus support

 MAINTAINERS                                  |   1 +
 drivers/acpi/acpi_apd.c                      |   1 +
 drivers/i2c/busses/Kconfig                   |  10 +
 drivers/i2c/busses/Makefile                  |   1 +
 drivers/i2c/busses/i2c-designware-amdpsp.c   | 357 +++++++++++++++++++
 drivers/i2c/busses/i2c-designware-baytrail.c |  10 +-
 drivers/i2c/busses/i2c-designware-common.c   |  12 +
 drivers/i2c/busses/i2c-designware-core.h     |  18 +-
 drivers/i2c/busses/i2c-designware-master.c   |   6 +
 drivers/i2c/busses/i2c-designware-platdrv.c  |  61 ++++
 10 files changed, 469 insertions(+), 8 deletions(-)
 create mode 100644 drivers/i2c/busses/i2c-designware-amdpsp.c

-- 
2.34.1.703.g22d0c6ccf7-goog

[PATCH 1/2] i2c: designware: Add missing locks

[Jan Dabros]

All accesses to controller's registers should be protected on
probe, disable and xfer paths. This is needed for i2c bus controllers
that are shared with but not controller by kernel.

Signed-off-by: Jan Dabros <jsd@semihalf.com>
---
 drivers/i2c/busses/i2c-designware-common.c | 12 ++++++++++++
 drivers/i2c/busses/i2c-designware-master.c |  6 ++++++
 2 files changed, 18 insertions(+)

diff --git a/drivers/i2c/busses/i2c-designware-common.c b/drivers/i2c/busses/i2c-designware-common.c
index bf2a4920638a..9f8574320eb2 100644
--- a/drivers/i2c/busses/i2c-designware-common.c
+++ b/drivers/i2c/busses/i2c-designware-common.c
@@ -578,7 +578,12 @@ int i2c_dw_set_fifo_size(struct dw_i2c_dev *dev)
 	 * Try to detect the FIFO depth if not set by interface driver,
 	 * the depth could be from 2 to 256 from HW spec.
 	 */
+	ret = i2c_dw_acquire_lock(dev);
+	if (ret)
+		return ret;
+
 	ret = regmap_read(dev->map, DW_IC_COMP_PARAM_1, &param);
+	i2c_dw_release_lock(dev);
 	if (ret)
 		return ret;
 
@@ -607,6 +612,11 @@ u32 i2c_dw_func(struct i2c_adapter *adap)
 void i2c_dw_disable(struct dw_i2c_dev *dev)
 {
 	u32 dummy;
+	int ret;
+
+	ret = i2c_dw_acquire_lock(dev);
+	if (ret)
+		return;
 
 	/* Disable controller */
 	__i2c_dw_disable(dev);
@@ -614,6 +624,8 @@ void i2c_dw_disable(struct dw_i2c_dev *dev)
 	/* Disable all interrupts */
 	regmap_write(dev->map, DW_IC_INTR_MASK, 0);
 	regmap_read(dev->map, DW_IC_CLR_INTR, &dummy);
+
+	i2c_dw_release_lock(dev);
 }
 
 void i2c_dw_disable_int(struct dw_i2c_dev *dev)
diff --git a/drivers/i2c/busses/i2c-designware-master.c b/drivers/i2c/busses/i2c-designware-master.c
index 9177463c2cbb..1a4b23556db3 100644
--- a/drivers/i2c/busses/i2c-designware-master.c
+++ b/drivers/i2c/busses/i2c-designware-master.c
@@ -905,7 +905,13 @@ int i2c_dw_probe_master(struct dw_i2c_dev *dev)
 		irq_flags = IRQF_SHARED | IRQF_COND_SUSPEND;
 	}
 
+	ret = i2c_dw_acquire_lock(dev);
+	if (ret)
+		return ret;
+
 	i2c_dw_disable_int(dev);
+	i2c_dw_release_lock(dev);
+
 	ret = devm_request_irq(dev->dev, dev->irq, i2c_dw_isr, irq_flags,
 			       dev_name(dev->dev), dev);
 	if (ret) {
-- 
2.34.1.703.g22d0c6ccf7-goog

[PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dabros]

Implement an I2C controller sharing mechanism between the host (kernel)
and PSP co-processor on some platforms equipped with AMD Cezanne SoC.

On these platforms we need to implement "software" i2c arbitration.
Default arbitration owner is PSP and kernel asks for acquire as well
as inform about release of the i2c bus via mailbox mechanism.

            +---------+
 <- ACQUIRE |         |
  +---------|   CPU   |\
  |         |         | \      +----------+  SDA
  |         +---------+  \     |          |-------
MAILBOX                   +--> |  I2C-DW  |  SCL
  |         +---------+        |          |-------
  |         |         |        +----------+
  +---------|   PSP   |
   <- ACK   |         |
            +---------+

            +---------+
 <- RELEASE |         |
  +---------|   CPU   |
  |         |         |        +----------+  SDA
  |         +---------+        |          |-------
MAILBOX                   +--> |  I2C-DW  |  SCL
  |         +---------+  /     |          |-------
  |         |         | /      +----------+
  +---------|   PSP   |/
   <- ACK   |         |
            +---------+

The solution is similar to i2c-designware-baytrail.c implementation, where
we are using a generic i2c-designware-* driver with a small "wrapper".

In contrary to baytrail semaphore implementation, beside internal
acquire_lock() and release_lock() methods we are also applying quirks to
lock_bus() and unlock_bus() global adapter methods. With this in place
all i2c clients drivers may lock i2c bus for a desired number of i2c
transactions (e.g. write-wait-read) without being aware of that such bus
is shared with another entity.

Modify i2c_dw_probe_lock_support() to select correct semaphore
implementation at runtime, since now we have more than one available.

Configure new matching ACPI ID "AMDI0019" and register
ARBITRATION_SEMAPHORE flag in order to distinguish setup with PSP
arbitration.

Add new entry in MAINTAINERS file to cover new module.

Signed-off-by: Jan Dabros <jsd@semihalf.com>
---
 MAINTAINERS                                  |   1 +
 drivers/acpi/acpi_apd.c                      |   1 +
 drivers/i2c/busses/Kconfig                   |  10 +
 drivers/i2c/busses/Makefile                  |   1 +
 drivers/i2c/busses/i2c-designware-amdpsp.c   | 357 +++++++++++++++++++
 drivers/i2c/busses/i2c-designware-baytrail.c |  10 +-
 drivers/i2c/busses/i2c-designware-core.h     |  18 +-
 drivers/i2c/busses/i2c-designware-platdrv.c  |  61 ++++
 8 files changed, 451 insertions(+), 8 deletions(-)
 create mode 100644 drivers/i2c/busses/i2c-designware-amdpsp.c

diff --git a/MAINTAINERS b/MAINTAINERS
index b84e2d5642bc..3c81084bc6e6 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -18659,6 +18659,7 @@ SYNOPSYS DESIGNWARE I2C DRIVER
 M:	Jarkko Nikula <jarkko.nikula@linux.intel.com>
 R:	Andy Shevchenko <andriy.shevchenko@linux.intel.com>
 R:	Mika Westerberg <mika.westerberg@linux.intel.com>
+R:	Jan Dabros <jsd@semihalf.com>
 L:	linux-i2c@vger.kernel.org
 S:	Maintained
 F:	drivers/i2c/busses/i2c-designware-*
diff --git a/drivers/acpi/acpi_apd.c b/drivers/acpi/acpi_apd.c
index e7934ba79b02..4a812476a418 100644
--- a/drivers/acpi/acpi_apd.c
+++ b/drivers/acpi/acpi_apd.c
@@ -236,6 +236,7 @@ static const struct acpi_device_id acpi_apd_device_ids[] = {
 	{ "AMD0020", APD_ADDR(cz_uart_desc) },
 	{ "AMDI0020", APD_ADDR(cz_uart_desc) },
 	{ "AMDI0022", APD_ADDR(cz_uart_desc) },
+	{ "AMDI0019", APD_ADDR(wt_i2c_desc) },
 	{ "AMD0030", },
 	{ "AMD0040", APD_ADDR(fch_misc_desc)},
 	{ "HYGO0010", APD_ADDR(wt_i2c_desc) },
diff --git a/drivers/i2c/busses/Kconfig b/drivers/i2c/busses/Kconfig
index 42da31c1ab70..9177a56d2e94 100644
--- a/drivers/i2c/busses/Kconfig
+++ b/drivers/i2c/busses/Kconfig
@@ -553,6 +553,16 @@ config I2C_DESIGNWARE_PLATFORM
 	  This driver can also be built as a module.  If so, the module
 	  will be called i2c-designware-platform.
 
+config I2C_DESIGNWARE_AMDPSP
+	bool "AMD PSP I2C semaphore support"
+	depends on ACPI
+	depends on I2C_DESIGNWARE_PLATFORM
+	help
+	  This driver enables managed host access to the selected I2C bus shared
+	  between AMD CPU and AMD PSP.
+
+	  You should say Y if running on an AMD system equipped with the PSP.
+
 config I2C_DESIGNWARE_BAYTRAIL
 	bool "Intel Baytrail I2C semaphore support"
 	depends on ACPI
diff --git a/drivers/i2c/busses/Makefile b/drivers/i2c/busses/Makefile
index 1d00dce77098..752f47be3fc1 100644
--- a/drivers/i2c/busses/Makefile
+++ b/drivers/i2c/busses/Makefile
@@ -54,6 +54,7 @@ i2c-designware-core-y					+= i2c-designware-master.o
 i2c-designware-core-$(CONFIG_I2C_DESIGNWARE_SLAVE) 	+= i2c-designware-slave.o
 obj-$(CONFIG_I2C_DESIGNWARE_PLATFORM)			+= i2c-designware-platform.o
 i2c-designware-platform-y 				:= i2c-designware-platdrv.o
+i2c-designware-platform-$(CONFIG_I2C_DESIGNWARE_AMDPSP)	+= i2c-designware-amdpsp.o
 i2c-designware-platform-$(CONFIG_I2C_DESIGNWARE_BAYTRAIL) += i2c-designware-baytrail.o
 obj-$(CONFIG_I2C_DESIGNWARE_PCI)			+= i2c-designware-pci.o
 i2c-designware-pci-y					:= i2c-designware-pcidrv.o
diff --git a/drivers/i2c/busses/i2c-designware-amdpsp.c b/drivers/i2c/busses/i2c-designware-amdpsp.c
new file mode 100644
index 000000000000..e86cc56df921
--- /dev/null
+++ b/drivers/i2c/busses/i2c-designware-amdpsp.c
@@ -0,0 +1,357 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <asm/msr.h>
+#include <linux/i2c.h>
+#include <linux/psp-sev.h>
+
+#include "i2c-designware-core.h"
+
+#define MSR_AMD_PSP_ADDR	0xc00110a2
+#define PSP_MBOX_OFFSET		0x10570
+#define PSP_CMD_TIMEOUT_MS	500
+
+#define PSP_I2C_REQ_BUS_CMD		0x64
+#define PSP_I2C_REQ_RETRY_CNT		10
+#define PSP_I2C_REQ_RETRY_DELAY_MSEC	50
+#define PSP_I2C_REQ_STS_OK		0x0
+#define PSP_I2C_REQ_STS_BUS_BUSY	0x1
+#define PSP_I2C_REQ_STS_INV_PARAM	0x3
+
+union psp_req_buffer_hdr {
+	struct {
+		u32 total_size;
+		u32 status;
+	} __packed;
+	u64 hdr_val;
+};
+
+enum psp_i2c_req_type {
+	PSP_I2C_REQ_ACQUIRE,
+	PSP_I2C_REQ_RELEASE,
+	PSP_I2C_REQ_MAX,
+};
+
+struct psp_i2c_req {
+	union psp_req_buffer_hdr hdr;
+	enum psp_i2c_req_type type;
+} __packed __aligned(32);
+
+union psp_mbox_cmd_reg {
+	struct psp_mbox_cmd_fields {
+		u16 mbox_status;
+		u8 mbox_cmd;
+		u8 reserved:6;
+		u8 recovery:1;
+		u8 ready:1;
+	} __packed fields;
+	u32 val;
+};
+
+struct psp_mbox {
+	union psp_mbox_cmd_reg fields;
+	uintptr_t i2c_req_addr;
+} __packed;
+
+static DEFINE_MUTEX(psp_i2c_access_mutex);
+static unsigned long psp_i2c_sem_acquired;
+static void __iomem *mbox_iomem;
+static u32 psp_i2c_access_count;
+static bool psp_i2c_mbox_fail;
+static struct device *psp_i2c_dev;
+
+static int psp_get_mbox_addr(unsigned long *mbox_addr)
+{
+	unsigned long long psp_mmio;
+
+	if (rdmsrl_safe(MSR_AMD_PSP_ADDR, &psp_mmio))
+		return -EIO;
+
+	*mbox_addr = (unsigned long)(psp_mmio + PSP_MBOX_OFFSET);
+
+	return 0;
+}
+
+static int psp_mbox_probe(void)
+{
+	unsigned long mbox_addr;
+
+	if (psp_get_mbox_addr(&mbox_addr))
+		return -1;
+
+	mbox_iomem = ioremap(mbox_addr, sizeof(struct psp_mbox));
+	if (!mbox_iomem)
+		return -ENOMEM;
+
+	return 0;
+}
+
+/* Recovery field should be equal 0 to start sending commands */
+static int psp_check_mbox_recovery(struct psp_mbox *mbox)
+{
+	union psp_mbox_cmd_reg tmp = {0};
+
+	tmp.val = readl(&mbox->fields.val);
+	return !!tmp.fields.recovery;
+}
+
+static int psp_wait_cmd(struct psp_mbox *mbox)
+{
+	union psp_mbox_cmd_reg expected = { .val = 0 };
+	u32 tmp;
+
+	/* Expect mbox_cmd to be cleared and ready bit to be set by PSP */
+	expected.fields.ready = 1;
+
+	return readl_poll_timeout(&mbox->fields.val, tmp, (tmp == expected.val),
+				  0, 1000 * PSP_CMD_TIMEOUT_MS);
+}
+
+/* Status equal to 0 means that PSP succeed processing command */
+static int psp_check_mbox_sts(struct psp_mbox *mbox)
+{
+	union psp_mbox_cmd_reg cmd_reg = {0};
+
+	cmd_reg.val = readl(&mbox->fields.val);
+	return cmd_reg.fields.mbox_status;
+}
+
+static int psp_send_cmd(struct psp_i2c_req *req)
+{
+	struct psp_mbox *mbox = (struct psp_mbox *)mbox_iomem;
+	union psp_mbox_cmd_reg cmd_reg = {0};
+
+	if (psp_check_mbox_recovery(mbox))
+		return -EIO;
+
+	if (psp_wait_cmd(mbox))
+		return -EBUSY;
+
+	/* Fill address of command-response buffer */
+	writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr);
+
+	/* Write command register to trigger processing */
+	cmd_reg.fields.mbox_cmd = PSP_I2C_REQ_BUS_CMD;
+	writel(cmd_reg.val, &mbox->fields.val);
+
+	if (psp_wait_cmd(mbox))
+		return -ETIMEDOUT;
+
+	if (psp_check_mbox_sts(mbox))
+		return -EIO;
+
+	return 0;
+}
+
+/* Helper to verify status returned by PSP */
+static int check_i2c_req_sts(struct psp_i2c_req *req)
+{
+	int status;
+
+	status = readl(&req->hdr.status);
+
+	switch (status) {
+	case PSP_I2C_REQ_STS_OK:
+		return 0;
+	case PSP_I2C_REQ_STS_BUS_BUSY:
+		return -EBUSY;
+	case PSP_I2C_REQ_STS_INV_PARAM:
+	default:
+		return -EIO;
+	};
+}
+
+static int psp_send_i2c_req(enum psp_i2c_req_type i2c_req_type)
+{
+	int status, ret, retry_cnt = PSP_I2C_REQ_RETRY_CNT;
+	struct psp_i2c_req *req;
+	unsigned long start;
+
+	/* Allocate command-response buffer */
+	req = kzalloc(sizeof(*req), GFP_KERNEL);
+	if (!req)
+		return -ENOMEM;
+
+	req->hdr.total_size = sizeof(*req);
+	req->type = i2c_req_type;
+
+	start = jiffies;
+	do {
+		if (psp_send_cmd(req)) {
+			ret = -EIO;
+			goto cleanup;
+		}
+
+		status = check_i2c_req_sts(req);
+		if (!status) {
+			dev_dbg(psp_i2c_dev, "Request accepted by PSP after %ums\n",
+				jiffies_to_msecs(jiffies - start));
+			ret = 0;
+			goto cleanup;
+		} else if (status == -EBUSY) {
+			retry_cnt--;
+		} else {
+			ret = -EIO;
+			goto cleanup;
+		};
+
+		/* IF EBUSY, give PSP time to finish its i2c activities */
+		mdelay(PSP_I2C_REQ_RETRY_DELAY_MSEC);
+	} while (retry_cnt);
+
+
+	ret = -ETIMEDOUT;
+
+cleanup:
+	kfree(req);
+	return ret;
+}
+
+static int psp_acquire_i2c_bus(void)
+{
+	int status;
+
+	mutex_lock(&psp_i2c_access_mutex);
+
+	/* Return early if mailbox malfunctioned */
+	if (psp_i2c_mbox_fail)
+		goto cleanup;
+
+	/*
+	 * Simply increment usage counter and return if PSP semaphore was
+	 * already taken by kernel
+	 */
+	if (psp_i2c_access_count > 0) {
+		psp_i2c_access_count++;
+		goto cleanup;
+	};
+
+	status = psp_send_i2c_req(PSP_I2C_REQ_ACQUIRE);
+	if (!status) {
+		psp_i2c_sem_acquired = jiffies;
+		psp_i2c_access_count++;
+		goto cleanup;
+	} else if (status == -ETIMEDOUT) {
+		dev_err(psp_i2c_dev, "Timed out waiting for PSP to release I2C bus\n");
+	} else {
+		dev_err(psp_i2c_dev, "PSP communication error\n");
+	};
+
+	dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n");
+	psp_i2c_mbox_fail = true;
+
+cleanup:
+	mutex_unlock(&psp_i2c_access_mutex);
+	return 0;
+}
+
+static void psp_release_i2c_bus(void)
+{
+	int status;
+
+	mutex_lock(&psp_i2c_access_mutex);
+
+	/* Return early if mailbox was malfunctional */
+	if (psp_i2c_mbox_fail)
+		goto cleanup;
+
+	/*
+	 * If we are last owner of PSP semaphore, need to release aribtration
+	 * via mailbox
+	 */
+	psp_i2c_access_count--;
+	if (psp_i2c_access_count > 0)
+		goto cleanup;
+
+	/* Send a release command to PSP */
+	status = psp_send_i2c_req(PSP_I2C_REQ_RELEASE);
+	if (!status) {
+		dev_dbg(psp_i2c_dev, "PSP semaphore held for %ums\n",
+			jiffies_to_msecs(jiffies - psp_i2c_sem_acquired));
+		goto cleanup;
+	} else if (status == -ETIMEDOUT) {
+		dev_err(psp_i2c_dev, "Timed out waiting for PSP to acquire I2C bus\n");
+	} else {
+		dev_err(psp_i2c_dev, "PSP communication error\n");
+	}
+
+	dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n");
+	psp_i2c_mbox_fail = true;
+
+cleanup:
+	mutex_unlock(&psp_i2c_access_mutex);
+}
+
+/*
+ * Locking methods are based on the default implementation from
+ * drivers/i2c/i2c-core.base.c, but with psp acquire and release operations
+ * added. With this in place we can ensure that i2c clients on the bus shared
+ * with psp are able to lock HW access to the bus for arbitrary number of
+ * operations - that is e.g. write-wait-read.
+ */
+static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter,
+					unsigned int flags)
+{
+	psp_acquire_i2c_bus();
+	rt_mutex_lock_nested(&adapter->bus_lock, i2c_adapter_depth(adapter));
+}
+
+static int i2c_adapter_dw_psp_trylock_bus(struct i2c_adapter *adapter,
+					  unsigned int flags)
+{
+	int ret;
+
+	ret = rt_mutex_trylock(&adapter->bus_lock);
+	if (!ret)
+		psp_acquire_i2c_bus();
+
+	return ret;
+}
+
+static void i2c_adapter_dw_psp_unlock_bus(struct i2c_adapter *adapter,
+					  unsigned int flags)
+{
+	psp_release_i2c_bus();
+	rt_mutex_unlock(&adapter->bus_lock);
+}
+
+static const struct i2c_lock_operations i2c_dw_psp_lock_ops = {
+	.lock_bus = i2c_adapter_dw_psp_lock_bus,
+	.trylock_bus = i2c_adapter_dw_psp_trylock_bus,
+	.unlock_bus = i2c_adapter_dw_psp_unlock_bus,
+};
+
+int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev)
+{
+	if (!dev || !dev->dev)
+		return -ENODEV;
+
+	if (!(dev->flags & ARBITRATION_SEMAPHORE))
+		return -ENODEV;
+
+	/* Allow to bind only one instance of a driver */
+	if (!psp_i2c_dev)
+		psp_i2c_dev = dev->dev;
+	else
+		return -EEXIST;
+
+	if (psp_mbox_probe())
+		return -EIO;
+
+	dev_info(psp_i2c_dev, "I2C bus managed by AMD PSP\n");
+
+	/*
+	 * Install global locking callbacks for adapter as well as internal i2c
+	 * controller locks
+	 */
+	dev->adapter.lock_ops = &i2c_dw_psp_lock_ops;
+	dev->acquire_lock = psp_acquire_i2c_bus;
+	dev->release_lock = psp_release_i2c_bus;
+
+	return 0;
+}
+
+/* Unmap area used as a mailbox with PSP */
+void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev)
+{
+	iounmap(mbox_iomem);
+}
diff --git a/drivers/i2c/busses/i2c-designware-baytrail.c b/drivers/i2c/busses/i2c-designware-baytrail.c
index c6a7a00e1d52..0c674542dd99 100644
--- a/drivers/i2c/busses/i2c-designware-baytrail.c
+++ b/drivers/i2c/busses/i2c-designware-baytrail.c
@@ -12,25 +12,25 @@
 
 #include "i2c-designware-core.h"
 
-int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
+int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev)
 {
 	acpi_status status;
 	unsigned long long shared_host = 0;
 	acpi_handle handle;
 
 	if (!dev || !dev->dev)
-		return 0;
+		return -ENODEV;
 
 	handle = ACPI_HANDLE(dev->dev);
 	if (!handle)
-		return 0;
+		return -ENODEV;
 
 	status = acpi_evaluate_integer(handle, "_SEM", NULL, &shared_host);
 	if (ACPI_FAILURE(status))
-		return 0;
+		return -ENODEV;
 
 	if (!shared_host)
-		return 0;
+		return -ENODEV;
 
 	if (!iosf_mbi_available())
 		return -EPROBE_DEFER;
diff --git a/drivers/i2c/busses/i2c-designware-core.h b/drivers/i2c/busses/i2c-designware-core.h
index 4b26cba40139..1d65212fddbd 100644
--- a/drivers/i2c/busses/i2c-designware-core.h
+++ b/drivers/i2c/busses/i2c-designware-core.h
@@ -227,6 +227,8 @@ struct reset_control;
  * @hs_lcnt: high speed LCNT value
  * @acquire_lock: function to acquire a hardware lock on the bus
  * @release_lock: function to release a hardware lock on the bus
+ * @semaphore_idx: Index of table with semaphore type attached to the bus. It's
+ *	-1 if there is no semaphore.
  * @shared_with_punit: true if this bus is shared with the SoCs PUNIT
  * @disable: function to disable the controller
  * @disable_int: function to disable all interrupts
@@ -285,6 +287,7 @@ struct dw_i2c_dev {
 	u16			hs_lcnt;
 	int			(*acquire_lock)(void);
 	void			(*release_lock)(void);
+	int			semaphore_idx;
 	bool			shared_with_punit;
 	void			(*disable)(struct dw_i2c_dev *dev);
 	void			(*disable_int)(struct dw_i2c_dev *dev);
@@ -297,6 +300,7 @@ struct dw_i2c_dev {
 
 #define ACCESS_INTR_MASK	BIT(0)
 #define ACCESS_NO_IRQ_SUSPEND	BIT(1)
+#define ARBITRATION_SEMAPHORE	BIT(2)
 
 #define MODEL_MSCC_OCELOT	BIT(8)
 #define MODEL_BAIKAL_BT1	BIT(9)
@@ -310,6 +314,11 @@ struct dw_i2c_dev {
 #define AMD_UCSI_INTR_REG	0x474
 #define AMD_UCSI_INTR_EN	0xd
 
+struct i2c_dw_semaphore_callbacks {
+	int	(*probe)(struct dw_i2c_dev *dev);
+	void	(*remove)(struct dw_i2c_dev *dev);
+};
+
 int i2c_dw_init_regmap(struct dw_i2c_dev *dev);
 u32 i2c_dw_scl_hcnt(u32 ic_clk, u32 tSYMBOL, u32 tf, int cond, int offset);
 u32 i2c_dw_scl_lcnt(u32 ic_clk, u32 tLOW, u32 tf, int offset);
@@ -370,9 +379,12 @@ static inline void i2c_dw_configure(struct dw_i2c_dev *dev)
 }
 
 #if IS_ENABLED(CONFIG_I2C_DESIGNWARE_BAYTRAIL)
-extern int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev);
-#else
-static inline int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev) { return 0; }
+int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev);
+#endif
+
+#if IS_ENABLED(CONFIG_I2C_DESIGNWARE_AMDPSP)
+int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev);
+void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev);
 #endif
 
 int i2c_dw_validate_speed(struct dw_i2c_dev *dev);
diff --git a/drivers/i2c/busses/i2c-designware-platdrv.c b/drivers/i2c/busses/i2c-designware-platdrv.c
index 2bd81abc86f6..5844a4df4141 100644
--- a/drivers/i2c/busses/i2c-designware-platdrv.c
+++ b/drivers/i2c/busses/i2c-designware-platdrv.c
@@ -51,6 +51,7 @@ static const struct acpi_device_id dw_i2c_acpi_match[] = {
 	{ "AMD0010", ACCESS_INTR_MASK },
 	{ "AMDI0010", ACCESS_INTR_MASK },
 	{ "AMDI0510", 0 },
+	{ "AMDI0019", ACCESS_INTR_MASK | ARBITRATION_SEMAPHORE },
 	{ "APMC0D0F", 0 },
 	{ "HISI02A1", 0 },
 	{ "HISI02A2", 0 },
@@ -204,6 +205,64 @@ static const struct dmi_system_id dw_i2c_hwmon_class_dmi[] = {
 	{ } /* terminate list */
 };
 
+static const struct i2c_dw_semaphore_callbacks i2c_dw_semaphore_cb_table[] = {
+#ifdef CONFIG_I2C_DESIGNWARE_BAYTRAIL
+	{
+		.probe = i2c_dw_baytrail_probe_lock_support,
+		.remove = NULL,
+	},
+#endif
+#ifdef CONFIG_I2C_DESIGNWARE_AMDPSP
+	{
+		.probe = i2c_dw_amdpsp_probe_lock_support,
+		.remove = i2c_dw_amdpsp_remove_lock_support,
+	},
+#endif
+	{
+		.probe = NULL,
+		.remove = NULL,
+	},
+};
+
+static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
+{
+	int ret;
+	int i;
+
+	dev->semaphore_idx = -1;
+
+	for (i = 0; i < ARRAY_SIZE(i2c_dw_semaphore_cb_table); i++) {
+		if (!i2c_dw_semaphore_cb_table[i].probe)
+			continue;
+
+		ret = i2c_dw_semaphore_cb_table[i].probe(dev);
+		if (!ret) {
+			dev->semaphore_idx = i;
+			break;
+		} else if (ret == -ENODEV) {
+			/*
+			 * If there is no semaphore device attached to this
+			 * controller, we shouldn't abort general i2c_controller
+			 * probe.
+			 */
+			continue;
+		} else {
+			return ret;
+		}
+	}
+
+	return 0;
+}
+
+static void i2c_dw_remove_lock_support(struct dw_i2c_dev *dev)
+{
+	if (dev->semaphore_idx < 0)
+		return;
+
+	if (i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove)
+		i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove(dev);
+}
+
 static int dw_i2c_plat_probe(struct platform_device *pdev)
 {
 	struct i2c_adapter *adap;
@@ -334,6 +393,8 @@ static int dw_i2c_plat_remove(struct platform_device *pdev)
 	pm_runtime_put_sync(&pdev->dev);
 	dw_i2c_plat_pm_cleanup(dev);
 
+	i2c_dw_remove_lock_support(dev);
+
 	reset_control_assert(dev->rst);
 
 	return 0;
-- 
2.34.1.703.g22d0c6ccf7-goog

Re: [PATCH 0/2] i2c-designware: Add support for AMD PSP semaphore

[Hans de Goede]

Hi Jan,

On 1/20/22 01:16, Jan Dabros wrote:
> This patchset comprises support for new i2c-designware controller setup on some
> AMD Cezanne SoCs, where x86 is sharing i2c bus with PSP. PSP uses the same
> controller and acts as an i2c arbitrator there (x86 is leasing bus from it).
> 
> First commit aims to improve generic i2c-designware code by adding extra locking
> on probe() and disable() paths. I would like to ask someone with access to
> boards which use Intel BayTrail(CONFIG_I2C_DESIGNWARE_BAYTRAIL) to verify
> behavior of my changes on such setup.
> 
> Second commit adds support for new PSP semaphore arbitration mechanism.
> Implementation is similar to the one from i2c-designware-baytrail.c however
> there are two main differences:
> 1) Add new ACPI ID in order to protect against silent binding of the old driver
> to the setup with PSP semaphore. Extra flag ARBITRATION_SEMAPHORE added to this
> new _HID allows to recognize setup with PSP.
> 2) Beside acquire_lock() and release_lock() methods we are also applying quirks
> to the lock_bus() and unlock_bus() global adapter methods. With this in place
> all i2c clients drivers may lock i2c bus for a desired number of i2c
> transactions (e.g. write-wait-read) without being aware of that such bus is
> shared with another entity.
> 
> This patchset is a follow-up to the RFC sent earlier on LKML [1], with review
> comments applied.
> 
> Looking forward to some feedback.
> 
> [1] https://lkml.org/lkml/2021/12/22/219


Thank you for your patch series.

As you may have seen I've done a lot of work on the Bay Trail semaphore
thing. I also own several Bay Trail and Cherry Trail based devices which
use this setup.

I'll add your patches to my personal WIP tree which I regularly run
on these devices and I'll report back if I notice any issues.

One remark, I notice that there are no AMD people in the Cc, it
would be good if you can find someone from AMD to look at this,
also see my remarks to the 2nd patch in my reply to that patch.

Regards,

Hans




> 
> Jan Dabros (2):
>   i2c: designware: Add missing locks
>   i2c: designware: Add AMD PSP I2C bus support
> 
>  MAINTAINERS                                  |   1 +
>  drivers/acpi/acpi_apd.c                      |   1 +
>  drivers/i2c/busses/Kconfig                   |  10 +
>  drivers/i2c/busses/Makefile                  |   1 +
>  drivers/i2c/busses/i2c-designware-amdpsp.c   | 357 +++++++++++++++++++
>  drivers/i2c/busses/i2c-designware-baytrail.c |  10 +-
>  drivers/i2c/busses/i2c-designware-common.c   |  12 +
>  drivers/i2c/busses/i2c-designware-core.h     |  18 +-
>  drivers/i2c/busses/i2c-designware-master.c   |   6 +
>  drivers/i2c/busses/i2c-designware-platdrv.c  |  61 ++++
>  10 files changed, 469 insertions(+), 8 deletions(-)
>  create mode 100644 drivers/i2c/busses/i2c-designware-amdpsp.c
> 

Re: [PATCH 1/2] i2c: designware: Add missing locks

[Hans de Goede]

Hi,

On 1/20/22 01:16, Jan Dabros wrote:
> All accesses to controller's registers should be protected on
> probe, disable and xfer paths. This is needed for i2c bus controllers
> that are shared with but not controller by kernel.
> 
> Signed-off-by: Jan Dabros <jsd@semihalf.com>

Thanks, patch looks good to me:

Reviewed-by: Hans de Goede <hdegoede@redhat.com>

Regards,

Hans


> ---
>  drivers/i2c/busses/i2c-designware-common.c | 12 ++++++++++++
>  drivers/i2c/busses/i2c-designware-master.c |  6 ++++++
>  2 files changed, 18 insertions(+)
> 
> diff --git a/drivers/i2c/busses/i2c-designware-common.c b/drivers/i2c/busses/i2c-designware-common.c
> index bf2a4920638a..9f8574320eb2 100644
> --- a/drivers/i2c/busses/i2c-designware-common.c
> +++ b/drivers/i2c/busses/i2c-designware-common.c
> @@ -578,7 +578,12 @@ int i2c_dw_set_fifo_size(struct dw_i2c_dev *dev)
>  	 * Try to detect the FIFO depth if not set by interface driver,
>  	 * the depth could be from 2 to 256 from HW spec.
>  	 */
> +	ret = i2c_dw_acquire_lock(dev);
> +	if (ret)
> +		return ret;
> +
>  	ret = regmap_read(dev->map, DW_IC_COMP_PARAM_1, &param);
> +	i2c_dw_release_lock(dev);
>  	if (ret)
>  		return ret;
>  
> @@ -607,6 +612,11 @@ u32 i2c_dw_func(struct i2c_adapter *adap)
>  void i2c_dw_disable(struct dw_i2c_dev *dev)
>  {
>  	u32 dummy;
> +	int ret;
> +
> +	ret = i2c_dw_acquire_lock(dev);
> +	if (ret)
> +		return;
>  
>  	/* Disable controller */
>  	__i2c_dw_disable(dev);
> @@ -614,6 +624,8 @@ void i2c_dw_disable(struct dw_i2c_dev *dev)
>  	/* Disable all interrupts */
>  	regmap_write(dev->map, DW_IC_INTR_MASK, 0);
>  	regmap_read(dev->map, DW_IC_CLR_INTR, &dummy);
> +
> +	i2c_dw_release_lock(dev);
>  }
>  
>  void i2c_dw_disable_int(struct dw_i2c_dev *dev)
> diff --git a/drivers/i2c/busses/i2c-designware-master.c b/drivers/i2c/busses/i2c-designware-master.c
> index 9177463c2cbb..1a4b23556db3 100644
> --- a/drivers/i2c/busses/i2c-designware-master.c
> +++ b/drivers/i2c/busses/i2c-designware-master.c
> @@ -905,7 +905,13 @@ int i2c_dw_probe_master(struct dw_i2c_dev *dev)
>  		irq_flags = IRQF_SHARED | IRQF_COND_SUSPEND;
>  	}
>  
> +	ret = i2c_dw_acquire_lock(dev);
> +	if (ret)
> +		return ret;
> +
>  	i2c_dw_disable_int(dev);
> +	i2c_dw_release_lock(dev);
> +
>  	ret = devm_request_irq(dev->dev, dev->irq, i2c_dw_isr, irq_flags,
>  			       dev_name(dev->dev), dev);
>  	if (ret) {
> 

Re: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[kernel test robot]

Hi Jan,

I love your patch! Yet something to improve:

[auto build test ERROR on wsa/i2c/for-next]
[also build test ERROR on linux/master rafael-pm/linux-next linus/master v5.16 next-20220120]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Jan-Dabros/i2c-designware-Add-support-for-AMD-PSP-semaphore/20220120-081854
base:   https://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux.git i2c/for-next
config: ia64-allyesconfig (https://download.01.org/0day-ci/archive/20220120/202201201946.aWAj339S-lkp@intel.com/config)
compiler: ia64-linux-gcc (GCC) 11.2.0
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # https://github.com/0day-ci/linux/commit/f21a75f68baddffd9c50ffdc95d419ad7dbe3f68
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Jan-Dabros/i2c-designware-Add-support-for-AMD-PSP-semaphore/20220120-081854
        git checkout f21a75f68baddffd9c50ffdc95d419ad7dbe3f68
        # save the config file to linux build tree
        mkdir build_dir
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=ia64 SHELL=/bin/bash

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

>> drivers/i2c/busses/i2c-designware-amdpsp.c:3:10: fatal error: asm/msr.h: No such file or directory
       3 | #include <asm/msr.h>
         |          ^~~~~~~~~~~
   compilation terminated.


vim +3 drivers/i2c/busses/i2c-designware-amdpsp.c

     2	
   > 3	#include <asm/msr.h>
     4	#include <linux/i2c.h>
     5	#include <linux/psp-sev.h>
     6	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

Re: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Hans de Goede]

Hi,

On 1/20/22 01:16, Jan Dabros wrote:
> Implement an I2C controller sharing mechanism between the host (kernel)
> and PSP co-processor on some platforms equipped with AMD Cezanne SoC.
> 
> On these platforms we need to implement "software" i2c arbitration.
> Default arbitration owner is PSP and kernel asks for acquire as well
> as inform about release of the i2c bus via mailbox mechanism.
> 
>             +---------+
>  <- ACQUIRE |         |
>   +---------|   CPU   |\
>   |         |         | \      +----------+  SDA
>   |         +---------+  \     |          |-------
> MAILBOX                   +--> |  I2C-DW  |  SCL
>   |         +---------+        |          |-------
>   |         |         |        +----------+
>   +---------|   PSP   |
>    <- ACK   |         |
>             +---------+
> 
>             +---------+
>  <- RELEASE |         |
>   +---------|   CPU   |
>   |         |         |        +----------+  SDA
>   |         +---------+        |          |-------
> MAILBOX                   +--> |  I2C-DW  |  SCL
>   |         +---------+  /     |          |-------
>   |         |         | /      +----------+
>   +---------|   PSP   |/
>    <- ACK   |         |
>             +---------+
> 
> The solution is similar to i2c-designware-baytrail.c implementation, where
> we are using a generic i2c-designware-* driver with a small "wrapper".
> 
> In contrary to baytrail semaphore implementation, beside internal
> acquire_lock() and release_lock() methods we are also applying quirks to
> lock_bus() and unlock_bus() global adapter methods. With this in place
> all i2c clients drivers may lock i2c bus for a desired number of i2c
> transactions (e.g. write-wait-read) without being aware of that such bus
> is shared with another entity.
> 
> Modify i2c_dw_probe_lock_support() to select correct semaphore
> implementation at runtime, since now we have more than one available.
> 
> Configure new matching ACPI ID "AMDI0019" and register
> ARBITRATION_SEMAPHORE flag in order to distinguish setup with PSP
> arbitration.
> 
> Add new entry in MAINTAINERS file to cover new module.
> 
> Signed-off-by: Jan Dabros <jsd@semihalf.com>
> ---
>  MAINTAINERS                                  |   1 +
>  drivers/acpi/acpi_apd.c                      |   1 +
>  drivers/i2c/busses/Kconfig                   |  10 +
>  drivers/i2c/busses/Makefile                  |   1 +
>  drivers/i2c/busses/i2c-designware-amdpsp.c   | 357 +++++++++++++++++++
>  drivers/i2c/busses/i2c-designware-baytrail.c |  10 +-
>  drivers/i2c/busses/i2c-designware-core.h     |  18 +-
>  drivers/i2c/busses/i2c-designware-platdrv.c  |  61 ++++
>  8 files changed, 451 insertions(+), 8 deletions(-)
>  create mode 100644 drivers/i2c/busses/i2c-designware-amdpsp.c
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index b84e2d5642bc..3c81084bc6e6 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -18659,6 +18659,7 @@ SYNOPSYS DESIGNWARE I2C DRIVER
>  M:	Jarkko Nikula <jarkko.nikula@linux.intel.com>
>  R:	Andy Shevchenko <andriy.shevchenko@linux.intel.com>
>  R:	Mika Westerberg <mika.westerberg@linux.intel.com>
> +R:	Jan Dabros <jsd@semihalf.com>
>  L:	linux-i2c@vger.kernel.org
>  S:	Maintained
>  F:	drivers/i2c/busses/i2c-designware-*
> diff --git a/drivers/acpi/acpi_apd.c b/drivers/acpi/acpi_apd.c
> index e7934ba79b02..4a812476a418 100644
> --- a/drivers/acpi/acpi_apd.c
> +++ b/drivers/acpi/acpi_apd.c
> @@ -236,6 +236,7 @@ static const struct acpi_device_id acpi_apd_device_ids[] = {
>  	{ "AMD0020", APD_ADDR(cz_uart_desc) },
>  	{ "AMDI0020", APD_ADDR(cz_uart_desc) },
>  	{ "AMDI0022", APD_ADDR(cz_uart_desc) },
> +	{ "AMDI0019", APD_ADDR(wt_i2c_desc) },
>  	{ "AMD0030", },
>  	{ "AMD0040", APD_ADDR(fch_misc_desc)},
>  	{ "HYGO0010", APD_ADDR(wt_i2c_desc) },
> diff --git a/drivers/i2c/busses/Kconfig b/drivers/i2c/busses/Kconfig
> index 42da31c1ab70..9177a56d2e94 100644
> --- a/drivers/i2c/busses/Kconfig
> +++ b/drivers/i2c/busses/Kconfig
> @@ -553,6 +553,16 @@ config I2C_DESIGNWARE_PLATFORM
>  	  This driver can also be built as a module.  If so, the module
>  	  will be called i2c-designware-platform.
>  
> +config I2C_DESIGNWARE_AMDPSP
> +	bool "AMD PSP I2C semaphore support"
> +	depends on ACPI
> +	depends on I2C_DESIGNWARE_PLATFORM
> +	help
> +	  This driver enables managed host access to the selected I2C bus shared
> +	  between AMD CPU and AMD PSP.
> +
> +	  You should say Y if running on an AMD system equipped with the PSP.
> +
>  config I2C_DESIGNWARE_BAYTRAIL
>  	bool "Intel Baytrail I2C semaphore support"
>  	depends on ACPI
> diff --git a/drivers/i2c/busses/Makefile b/drivers/i2c/busses/Makefile
> index 1d00dce77098..752f47be3fc1 100644
> --- a/drivers/i2c/busses/Makefile
> +++ b/drivers/i2c/busses/Makefile
> @@ -54,6 +54,7 @@ i2c-designware-core-y					+= i2c-designware-master.o
>  i2c-designware-core-$(CONFIG_I2C_DESIGNWARE_SLAVE) 	+= i2c-designware-slave.o
>  obj-$(CONFIG_I2C_DESIGNWARE_PLATFORM)			+= i2c-designware-platform.o
>  i2c-designware-platform-y 				:= i2c-designware-platdrv.o
> +i2c-designware-platform-$(CONFIG_I2C_DESIGNWARE_AMDPSP)	+= i2c-designware-amdpsp.o
>  i2c-designware-platform-$(CONFIG_I2C_DESIGNWARE_BAYTRAIL) += i2c-designware-baytrail.o
>  obj-$(CONFIG_I2C_DESIGNWARE_PCI)			+= i2c-designware-pci.o
>  i2c-designware-pci-y					:= i2c-designware-pcidrv.o
> diff --git a/drivers/i2c/busses/i2c-designware-amdpsp.c b/drivers/i2c/busses/i2c-designware-amdpsp.c
> new file mode 100644
> index 000000000000..e86cc56df921
> --- /dev/null
> +++ b/drivers/i2c/busses/i2c-designware-amdpsp.c
> @@ -0,0 +1,357 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +#include <asm/msr.h>
> +#include <linux/i2c.h>
> +#include <linux/psp-sev.h>
> +
> +#include "i2c-designware-core.h"

So all the stuff starting here:

> +
> +#define MSR_AMD_PSP_ADDR	0xc00110a2
> +#define PSP_MBOX_OFFSET		0x10570
> +#define PSP_CMD_TIMEOUT_MS	500
> +
> +#define PSP_I2C_REQ_BUS_CMD		0x64
> +#define PSP_I2C_REQ_RETRY_CNT		10
> +#define PSP_I2C_REQ_RETRY_DELAY_MSEC	50
> +#define PSP_I2C_REQ_STS_OK		0x0
> +#define PSP_I2C_REQ_STS_BUS_BUSY	0x1
> +#define PSP_I2C_REQ_STS_INV_PARAM	0x3
> +
> +union psp_req_buffer_hdr {
> +	struct {
> +		u32 total_size;
> +		u32 status;
> +	} __packed;
> +	u64 hdr_val;
> +};
> +
> +enum psp_i2c_req_type {
> +	PSP_I2C_REQ_ACQUIRE,
> +	PSP_I2C_REQ_RELEASE,
> +	PSP_I2C_REQ_MAX,
> +};
> +
> +struct psp_i2c_req {
> +	union psp_req_buffer_hdr hdr;
> +	enum psp_i2c_req_type type;
> +} __packed __aligned(32);
> +
> +union psp_mbox_cmd_reg {
> +	struct psp_mbox_cmd_fields {
> +		u16 mbox_status;
> +		u8 mbox_cmd;
> +		u8 reserved:6;
> +		u8 recovery:1;
> +		u8 ready:1;
> +	} __packed fields;
> +	u32 val;
> +};
> +
> +struct psp_mbox {
> +	union psp_mbox_cmd_reg fields;
> +	uintptr_t i2c_req_addr;
> +} __packed;
> +
> +static DEFINE_MUTEX(psp_i2c_access_mutex);
> +static unsigned long psp_i2c_sem_acquired;
> +static void __iomem *mbox_iomem;
> +static u32 psp_i2c_access_count;
> +static bool psp_i2c_mbox_fail;
> +static struct device *psp_i2c_dev;
> +
> +static int psp_get_mbox_addr(unsigned long *mbox_addr)
> +{
> +	unsigned long long psp_mmio;
> +
> +	if (rdmsrl_safe(MSR_AMD_PSP_ADDR, &psp_mmio))
> +		return -EIO;
> +
> +	*mbox_addr = (unsigned long)(psp_mmio + PSP_MBOX_OFFSET);
> +
> +	return 0;
> +}
> +
> +static int psp_mbox_probe(void)
> +{
> +	unsigned long mbox_addr;
> +
> +	if (psp_get_mbox_addr(&mbox_addr))
> +		return -1;
> +
> +	mbox_iomem = ioremap(mbox_addr, sizeof(struct psp_mbox));
> +	if (!mbox_iomem)
> +		return -ENOMEM;
> +
> +	return 0;
> +}
> +
> +/* Recovery field should be equal 0 to start sending commands */
> +static int psp_check_mbox_recovery(struct psp_mbox *mbox)
> +{
> +	union psp_mbox_cmd_reg tmp = {0};
> +
> +	tmp.val = readl(&mbox->fields.val);
> +	return !!tmp.fields.recovery;
> +}
> +
> +static int psp_wait_cmd(struct psp_mbox *mbox)
> +{
> +	union psp_mbox_cmd_reg expected = { .val = 0 };
> +	u32 tmp;
> +
> +	/* Expect mbox_cmd to be cleared and ready bit to be set by PSP */
> +	expected.fields.ready = 1;
> +
> +	return readl_poll_timeout(&mbox->fields.val, tmp, (tmp == expected.val),
> +				  0, 1000 * PSP_CMD_TIMEOUT_MS);
> +}
> +
> +/* Status equal to 0 means that PSP succeed processing command */
> +static int psp_check_mbox_sts(struct psp_mbox *mbox)
> +{
> +	union psp_mbox_cmd_reg cmd_reg = {0};
> +
> +	cmd_reg.val = readl(&mbox->fields.val);
> +	return cmd_reg.fields.mbox_status;
> +}
> +
> +static int psp_send_cmd(struct psp_i2c_req *req)
> +{
> +	struct psp_mbox *mbox = (struct psp_mbox *)mbox_iomem;
> +	union psp_mbox_cmd_reg cmd_reg = {0};
> +
> +	if (psp_check_mbox_recovery(mbox))
> +		return -EIO;
> +
> +	if (psp_wait_cmd(mbox))
> +		return -EBUSY;
> +
> +	/* Fill address of command-response buffer */
> +	writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr);
> +
> +	/* Write command register to trigger processing */
> +	cmd_reg.fields.mbox_cmd = PSP_I2C_REQ_BUS_CMD;
> +	writel(cmd_reg.val, &mbox->fields.val);
> +
> +	if (psp_wait_cmd(mbox))
> +		return -ETIMEDOUT;
> +
> +	if (psp_check_mbox_sts(mbox))
> +		return -EIO;
> +
> +	return 0;
> +}

Through here seems to all be generic code for accessing
the AMD PSP. To me this seems like something which belongs
in a separate AMD-PSP-mbox driver/lib, which can then be
shared between other kernel drivers which may also want
to access PSP.

Sorta like the generic iosf_mbi_read() and
iosf_mbi_write() functions from:

arch/x86/platform/intel/iosf_mbi.c

used on the Intel chips, which are also used outside of
the I2C bus-locking code.

This is also one of the reasons why I think it would be
good to get some AMD folks involved in this, since they
may be aware of other drivers which also need to access
the PSP mbox.




> +
> +/* Helper to verify status returned by PSP */
> +static int check_i2c_req_sts(struct psp_i2c_req *req)
> +{
> +	int status;
> +
> +	status = readl(&req->hdr.status);
> +
> +	switch (status) {
> +	case PSP_I2C_REQ_STS_OK:
> +		return 0;
> +	case PSP_I2C_REQ_STS_BUS_BUSY:
> +		return -EBUSY;
> +	case PSP_I2C_REQ_STS_INV_PARAM:
> +	default:
> +		return -EIO;
> +	};
> +}
> +
> +static int psp_send_i2c_req(enum psp_i2c_req_type i2c_req_type)
> +{
> +	int status, ret, retry_cnt = PSP_I2C_REQ_RETRY_CNT;
> +	struct psp_i2c_req *req;
> +	unsigned long start;
> +
> +	/* Allocate command-response buffer */
> +	req = kzalloc(sizeof(*req), GFP_KERNEL);
> +	if (!req)
> +		return -ENOMEM;
> +
> +	req->hdr.total_size = sizeof(*req);
> +	req->type = i2c_req_type;
> +
> +	start = jiffies;
> +	do {
> +		if (psp_send_cmd(req)) {
> +			ret = -EIO;
> +			goto cleanup;
> +		}
> +
> +		status = check_i2c_req_sts(req);
> +		if (!status) {
> +			dev_dbg(psp_i2c_dev, "Request accepted by PSP after %ums\n",
> +				jiffies_to_msecs(jiffies - start));
> +			ret = 0;
> +			goto cleanup;
> +		} else if (status == -EBUSY) {
> +			retry_cnt--;
> +		} else {
> +			ret = -EIO;
> +			goto cleanup;
> +		};
> +
> +		/* IF EBUSY, give PSP time to finish its i2c activities */
> +		mdelay(PSP_I2C_REQ_RETRY_DELAY_MSEC);
> +	} while (retry_cnt);
> +
> +
> +	ret = -ETIMEDOUT;
> +
> +cleanup:
> +	kfree(req);
> +	return ret;
> +}
> +
> +static int psp_acquire_i2c_bus(void)
> +{
> +	int status;
> +
> +	mutex_lock(&psp_i2c_access_mutex);
> +
> +	/* Return early if mailbox malfunctioned */
> +	if (psp_i2c_mbox_fail)
> +		goto cleanup;
> +
> +	/*
> +	 * Simply increment usage counter and return if PSP semaphore was
> +	 * already taken by kernel
> +	 */
> +	if (psp_i2c_access_count > 0) {
> +		psp_i2c_access_count++;
> +		goto cleanup;
> +	};
> +
> +	status = psp_send_i2c_req(PSP_I2C_REQ_ACQUIRE);
> +	if (!status) {
> +		psp_i2c_sem_acquired = jiffies;
> +		psp_i2c_access_count++;
> +		goto cleanup;
> +	} else if (status == -ETIMEDOUT) {
> +		dev_err(psp_i2c_dev, "Timed out waiting for PSP to release I2C bus\n");
> +	} else {
> +		dev_err(psp_i2c_dev, "PSP communication error\n");
> +	};
> +
> +	dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n");
> +	psp_i2c_mbox_fail = true;
> +
> +cleanup:
> +	mutex_unlock(&psp_i2c_access_mutex);
> +	return 0;
> +}
> +
> +static void psp_release_i2c_bus(void)
> +{
> +	int status;
> +
> +	mutex_lock(&psp_i2c_access_mutex);
> +
> +	/* Return early if mailbox was malfunctional */
> +	if (psp_i2c_mbox_fail)
> +		goto cleanup;
> +
> +	/*
> +	 * If we are last owner of PSP semaphore, need to release aribtration
> +	 * via mailbox
> +	 */
> +	psp_i2c_access_count--;
> +	if (psp_i2c_access_count > 0)
> +		goto cleanup;
> +
> +	/* Send a release command to PSP */
> +	status = psp_send_i2c_req(PSP_I2C_REQ_RELEASE);
> +	if (!status) {
> +		dev_dbg(psp_i2c_dev, "PSP semaphore held for %ums\n",
> +			jiffies_to_msecs(jiffies - psp_i2c_sem_acquired));
> +		goto cleanup;
> +	} else if (status == -ETIMEDOUT) {
> +		dev_err(psp_i2c_dev, "Timed out waiting for PSP to acquire I2C bus\n");
> +	} else {
> +		dev_err(psp_i2c_dev, "PSP communication error\n");
> +	}
> +
> +	dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n");
> +	psp_i2c_mbox_fail = true;
> +
> +cleanup:
> +	mutex_unlock(&psp_i2c_access_mutex);
> +}
> +
> +/*
> + * Locking methods are based on the default implementation from
> + * drivers/i2c/i2c-core.base.c, but with psp acquire and release operations
> + * added. With this in place we can ensure that i2c clients on the bus shared
> + * with psp are able to lock HW access to the bus for arbitrary number of
> + * operations - that is e.g. write-wait-read.
> + */
> +static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter,
> +					unsigned int flags)
> +{
> +	psp_acquire_i2c_bus();
> +	rt_mutex_lock_nested(&adapter->bus_lock, i2c_adapter_depth(adapter));

This does not do what you think it does and you will still deadlock
when things nest because of someone taking the bus-lock and then
the main i2c-designware transfer function calling the acquire_lock
callback.

The _nested postfix is only for the lockdep lock-debugger, this
actually turns into a regular mutex_lock when lockdep is not enabled:

#ifdef CONFIG_DEBUG_LOCK_ALLOC
extern void rt_mutex_lock_nested(struct rt_mutex *lock, unsigned int subclass);
#define rt_mutex_lock(lock) rt_mutex_lock_nested(lock, 0)
#else
extern void rt_mutex_lock(struct rt_mutex *lock);
#define rt_mutex_lock_nested(lock, subclass) rt_mutex_lock(lock)
#endif

The _nested postfix as such is only to tell the lockdep code that
even though it seems we are trying to take the same mutex twice
since in both cases it is of i2c_adapter.rt_mutex "lock class"
that we are sure it is never the same i2c_adapter (but rather
one which always gets called in a nested fashion from another
i2c_adapter).

IOW this only disables a false-positive lockdep warning, it does
not allow taking the same mutex twice, you will still hang on
the second mutex_lock call on the same lock.

Also I don't think you are allowed to use the bus_locking code
like this. The i2c bus-locking code is intended to deal with
busses which have muxes in them, where the mux must be set
to the right branch of the bus to reach the client and then
not be changed during the transfer to that client.

So i2c-client drivers are never supposed to directly call
the bus-locking functions.

This is why in the Bay Trail case we have i2c-drivers
directly calling iosf_mbi_block_punit_i2c_access() and
iosf_mbi_unblock_punit_i2c_access() to lock the bus
for multiple i2c-transfers. We can get away with this there
because the bus in question is only used to access the
PMIC and that PMIC is only used on Bay Trail (and CHT)
boards, so the PMIC drivers can just hard-code these
calls.

If you need to take the PSP I2C semaphore for multiple
transfers in some generic drivers, then I guess that the
i2c-subsys will need to get some new i2c_adapter callbacks
to acquire / release the bus for i2c-controllers where
the bus/controller is shared with some co-processor like
in the PSP case.

Also note that iosf_mbi_block_punit_i2c_access() and
iosf_mbi_unblock_punit_i2c_access() do their own
ref/lock-counting to allow calling them multiple times and
the first block call takes the bus and the last unblock
call releases it.

Regards,

Hans





> +}
> +
> +static int i2c_adapter_dw_psp_trylock_bus(struct i2c_adapter *adapter,
> +					  unsigned int flags)
> +{
> +	int ret;
> +
> +	ret = rt_mutex_trylock(&adapter->bus_lock);
> +	if (!ret)
> +		psp_acquire_i2c_bus();
> +
> +	return ret;
> +}
> +
> +static void i2c_adapter_dw_psp_unlock_bus(struct i2c_adapter *adapter,
> +					  unsigned int flags)
> +{
> +	psp_release_i2c_bus();
> +	rt_mutex_unlock(&adapter->bus_lock);
> +}
> +
> +static const struct i2c_lock_operations i2c_dw_psp_lock_ops = {
> +	.lock_bus = i2c_adapter_dw_psp_lock_bus,
> +	.trylock_bus = i2c_adapter_dw_psp_trylock_bus,
> +	.unlock_bus = i2c_adapter_dw_psp_unlock_bus,
> +};
> +
> +int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev)
> +{
> +	if (!dev || !dev->dev)
> +		return -ENODEV;
> +
> +	if (!(dev->flags & ARBITRATION_SEMAPHORE))
> +		return -ENODEV;
> +
> +	/* Allow to bind only one instance of a driver */
> +	if (!psp_i2c_dev)
> +		psp_i2c_dev = dev->dev;
> +	else
> +		return -EEXIST;
> +
> +	if (psp_mbox_probe())
> +		return -EIO;
> +
> +	dev_info(psp_i2c_dev, "I2C bus managed by AMD PSP\n");
> +
> +	/*
> +	 * Install global locking callbacks for adapter as well as internal i2c
> +	 * controller locks
> +	 */
> +	dev->adapter.lock_ops = &i2c_dw_psp_lock_ops;
> +	dev->acquire_lock = psp_acquire_i2c_bus;
> +	dev->release_lock = psp_release_i2c_bus;
> +
> +	return 0;
> +}
> +
> +/* Unmap area used as a mailbox with PSP */
> +void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev)
> +{
> +	iounmap(mbox_iomem);
> +}
> diff --git a/drivers/i2c/busses/i2c-designware-baytrail.c b/drivers/i2c/busses/i2c-designware-baytrail.c
> index c6a7a00e1d52..0c674542dd99 100644
> --- a/drivers/i2c/busses/i2c-designware-baytrail.c
> +++ b/drivers/i2c/busses/i2c-designware-baytrail.c
> @@ -12,25 +12,25 @@
>  
>  #include "i2c-designware-core.h"
>  
> -int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
> +int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev)
>  {
>  	acpi_status status;
>  	unsigned long long shared_host = 0;
>  	acpi_handle handle;
>  
>  	if (!dev || !dev->dev)
> -		return 0;
> +		return -ENODEV;
>  
>  	handle = ACPI_HANDLE(dev->dev);
>  	if (!handle)
> -		return 0;
> +		return -ENODEV;
>  
>  	status = acpi_evaluate_integer(handle, "_SEM", NULL, &shared_host);
>  	if (ACPI_FAILURE(status))
> -		return 0;
> +		return -ENODEV;
>  
>  	if (!shared_host)
> -		return 0;
> +		return -ENODEV;
>  
>  	if (!iosf_mbi_available())
>  		return -EPROBE_DEFER;
> diff --git a/drivers/i2c/busses/i2c-designware-core.h b/drivers/i2c/busses/i2c-designware-core.h
> index 4b26cba40139..1d65212fddbd 100644
> --- a/drivers/i2c/busses/i2c-designware-core.h
> +++ b/drivers/i2c/busses/i2c-designware-core.h
> @@ -227,6 +227,8 @@ struct reset_control;
>   * @hs_lcnt: high speed LCNT value
>   * @acquire_lock: function to acquire a hardware lock on the bus
>   * @release_lock: function to release a hardware lock on the bus
> + * @semaphore_idx: Index of table with semaphore type attached to the bus. It's
> + *	-1 if there is no semaphore.
>   * @shared_with_punit: true if this bus is shared with the SoCs PUNIT
>   * @disable: function to disable the controller
>   * @disable_int: function to disable all interrupts
> @@ -285,6 +287,7 @@ struct dw_i2c_dev {
>  	u16			hs_lcnt;
>  	int			(*acquire_lock)(void);
>  	void			(*release_lock)(void);
> +	int			semaphore_idx;
>  	bool			shared_with_punit;
>  	void			(*disable)(struct dw_i2c_dev *dev);
>  	void			(*disable_int)(struct dw_i2c_dev *dev);
> @@ -297,6 +300,7 @@ struct dw_i2c_dev {
>  
>  #define ACCESS_INTR_MASK	BIT(0)
>  #define ACCESS_NO_IRQ_SUSPEND	BIT(1)
> +#define ARBITRATION_SEMAPHORE	BIT(2)
>  
>  #define MODEL_MSCC_OCELOT	BIT(8)
>  #define MODEL_BAIKAL_BT1	BIT(9)
> @@ -310,6 +314,11 @@ struct dw_i2c_dev {
>  #define AMD_UCSI_INTR_REG	0x474
>  #define AMD_UCSI_INTR_EN	0xd
>  
> +struct i2c_dw_semaphore_callbacks {
> +	int	(*probe)(struct dw_i2c_dev *dev);
> +	void	(*remove)(struct dw_i2c_dev *dev);
> +};
> +
>  int i2c_dw_init_regmap(struct dw_i2c_dev *dev);
>  u32 i2c_dw_scl_hcnt(u32 ic_clk, u32 tSYMBOL, u32 tf, int cond, int offset);
>  u32 i2c_dw_scl_lcnt(u32 ic_clk, u32 tLOW, u32 tf, int offset);
> @@ -370,9 +379,12 @@ static inline void i2c_dw_configure(struct dw_i2c_dev *dev)
>  }
>  
>  #if IS_ENABLED(CONFIG_I2C_DESIGNWARE_BAYTRAIL)
> -extern int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev);
> -#else
> -static inline int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev) { return 0; }
> +int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev);
> +#endif
> +
> +#if IS_ENABLED(CONFIG_I2C_DESIGNWARE_AMDPSP)
> +int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev);
> +void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev);
>  #endif
>  
>  int i2c_dw_validate_speed(struct dw_i2c_dev *dev);
> diff --git a/drivers/i2c/busses/i2c-designware-platdrv.c b/drivers/i2c/busses/i2c-designware-platdrv.c
> index 2bd81abc86f6..5844a4df4141 100644
> --- a/drivers/i2c/busses/i2c-designware-platdrv.c
> +++ b/drivers/i2c/busses/i2c-designware-platdrv.c
> @@ -51,6 +51,7 @@ static const struct acpi_device_id dw_i2c_acpi_match[] = {
>  	{ "AMD0010", ACCESS_INTR_MASK },
>  	{ "AMDI0010", ACCESS_INTR_MASK },
>  	{ "AMDI0510", 0 },
> +	{ "AMDI0019", ACCESS_INTR_MASK | ARBITRATION_SEMAPHORE },
>  	{ "APMC0D0F", 0 },
>  	{ "HISI02A1", 0 },
>  	{ "HISI02A2", 0 },
> @@ -204,6 +205,64 @@ static const struct dmi_system_id dw_i2c_hwmon_class_dmi[] = {
>  	{ } /* terminate list */
>  };
>  
> +static const struct i2c_dw_semaphore_callbacks i2c_dw_semaphore_cb_table[] = {
> +#ifdef CONFIG_I2C_DESIGNWARE_BAYTRAIL
> +	{
> +		.probe = i2c_dw_baytrail_probe_lock_support,
> +		.remove = NULL,
> +	},
> +#endif
> +#ifdef CONFIG_I2C_DESIGNWARE_AMDPSP
> +	{
> +		.probe = i2c_dw_amdpsp_probe_lock_support,
> +		.remove = i2c_dw_amdpsp_remove_lock_support,
> +	},
> +#endif
> +	{
> +		.probe = NULL,
> +		.remove = NULL,
> +	},
> +};
> +
> +static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
> +{
> +	int ret;
> +	int i;
> +
> +	dev->semaphore_idx = -1;
> +
> +	for (i = 0; i < ARRAY_SIZE(i2c_dw_semaphore_cb_table); i++) {
> +		if (!i2c_dw_semaphore_cb_table[i].probe)
> +			continue;
> +
> +		ret = i2c_dw_semaphore_cb_table[i].probe(dev);
> +		if (!ret) {
> +			dev->semaphore_idx = i;
> +			break;
> +		} else if (ret == -ENODEV) {
> +			/*
> +			 * If there is no semaphore device attached to this
> +			 * controller, we shouldn't abort general i2c_controller
> +			 * probe.
> +			 */
> +			continue;
> +		} else {
> +			return ret;
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static void i2c_dw_remove_lock_support(struct dw_i2c_dev *dev)
> +{
> +	if (dev->semaphore_idx < 0)
> +		return;
> +
> +	if (i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove)
> +		i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove(dev);
> +}
> +
>  static int dw_i2c_plat_probe(struct platform_device *pdev)
>  {
>  	struct i2c_adapter *adap;
> @@ -334,6 +393,8 @@ static int dw_i2c_plat_remove(struct platform_device *pdev)
>  	pm_runtime_put_sync(&pdev->dev);
>  	dw_i2c_plat_pm_cleanup(dev);
>  
> +	i2c_dw_remove_lock_support(dev);
> +
>  	reset_control_assert(dev->rst);
>  
>  	return 0;
> 

Re: [PATCH 0/2] i2c-designware: Add support for AMD PSP semaphore

[Jan Dąbroś]

Hi Hans,


czw., 20 sty 2022 o 12:15 Hans de Goede <hdegoede@redhat.com> napisał(a):
>
> Hi Jan,
>
> On 1/20/22 01:16, Jan Dabros wrote:
> > This patchset comprises support for new i2c-designware controller setup on some
> > AMD Cezanne SoCs, where x86 is sharing i2c bus with PSP. PSP uses the same
> > controller and acts as an i2c arbitrator there (x86 is leasing bus from it).
> >
> > First commit aims to improve generic i2c-designware code by adding extra locking
> > on probe() and disable() paths. I would like to ask someone with access to
> > boards which use Intel BayTrail(CONFIG_I2C_DESIGNWARE_BAYTRAIL) to verify
> > behavior of my changes on such setup.
> >
> > Second commit adds support for new PSP semaphore arbitration mechanism.
> > Implementation is similar to the one from i2c-designware-baytrail.c however
> > there are two main differences:
> > 1) Add new ACPI ID in order to protect against silent binding of the old driver
> > to the setup with PSP semaphore. Extra flag ARBITRATION_SEMAPHORE added to this
> > new _HID allows to recognize setup with PSP.
> > 2) Beside acquire_lock() and release_lock() methods we are also applying quirks
> > to the lock_bus() and unlock_bus() global adapter methods. With this in place
> > all i2c clients drivers may lock i2c bus for a desired number of i2c
> > transactions (e.g. write-wait-read) without being aware of that such bus is
> > shared with another entity.
> >
> > This patchset is a follow-up to the RFC sent earlier on LKML [1], with review
> > comments applied.
> >
> > Looking forward to some feedback.
> >
> > [1] https://lkml.org/lkml/2021/12/22/219
>
>
> Thank you for your patch series.
>
> As you may have seen I've done a lot of work on the Bay Trail semaphore
> thing. I also own several Bay Trail and Cherry Trail based devices which
> use this setup.
>
> I'll add your patches to my personal WIP tree which I regularly run
> on these devices and I'll report back if I notice any issues.

Thanks in advance, this will be really helpful! I don't have Bay
Trail/Cherry Trail, so I've only tested that build of Bay Trail
semaphore isn't broken.

I would like to point to new locks in i2c_dw_disable() method as
something to be the most fragile and error-prone, will be great if you
can verify this thoroughly. This function is invoked on both
dw_i2c_driver.remove() and dw_i2c_plat_suspend() paths. Considering
that Bay Trail semaphore means that i2c bus is shared with PMIC, I'm
not sure whether all corner cases are secured especially on platform
suspend.

>
> One remark, I notice that there are no AMD people in the Cc, it
> would be good if you can find someone from AMD to look at this,
> also see my remarks to the 2nd patch in my reply to that patch.

This was partially discussed with AMD folks and you are right that I
should include someone from AMD to take a look at this. Thanks for all
your comments!

> Regards,
>
> Hans
>
>
>
>
> >
> > Jan Dabros (2):
> >   i2c: designware: Add missing locks
> >   i2c: designware: Add AMD PSP I2C bus support
> >
> >  MAINTAINERS                                  |   1 +
> >  drivers/acpi/acpi_apd.c                      |   1 +
> >  drivers/i2c/busses/Kconfig                   |  10 +
> >  drivers/i2c/busses/Makefile                  |   1 +
> >  drivers/i2c/busses/i2c-designware-amdpsp.c   | 357 +++++++++++++++++++
> >  drivers/i2c/busses/i2c-designware-baytrail.c |  10 +-
> >  drivers/i2c/busses/i2c-designware-common.c   |  12 +
> >  drivers/i2c/busses/i2c-designware-core.h     |  18 +-
> >  drivers/i2c/busses/i2c-designware-master.c   |   6 +
> >  drivers/i2c/busses/i2c-designware-platdrv.c  |  61 ++++
> >  10 files changed, 469 insertions(+), 8 deletions(-)
> >  create mode 100644 drivers/i2c/busses/i2c-designware-amdpsp.c
> >
>

Re: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Andy Shevchenko]

On Thu, Jan 20, 2022 at 01:16:21AM +0100, Jan Dabros wrote:
> Implement an I2C controller sharing mechanism between the host (kernel)
> and PSP co-processor on some platforms equipped with AMD Cezanne SoC.
> 
> On these platforms we need to implement "software" i2c arbitration.
> Default arbitration owner is PSP and kernel asks for acquire as well
> as inform about release of the i2c bus via mailbox mechanism.
> 
>             +---------+
>  <- ACQUIRE |         |
>   +---------|   CPU   |\
>   |         |         | \      +----------+  SDA
>   |         +---------+  \     |          |-------
> MAILBOX                   +--> |  I2C-DW  |  SCL
>   |         +---------+        |          |-------
>   |         |         |        +----------+
>   +---------|   PSP   |
>    <- ACK   |         |
>             +---------+
> 
>             +---------+
>  <- RELEASE |         |
>   +---------|   CPU   |
>   |         |         |        +----------+  SDA
>   |         +---------+        |          |-------
> MAILBOX                   +--> |  I2C-DW  |  SCL
>   |         +---------+  /     |          |-------
>   |         |         | /      +----------+
>   +---------|   PSP   |/
>    <- ACK   |         |
>             +---------+
> 
> The solution is similar to i2c-designware-baytrail.c implementation, where
> we are using a generic i2c-designware-* driver with a small "wrapper".
> 
> In contrary to baytrail semaphore implementation, beside internal
> acquire_lock() and release_lock() methods we are also applying quirks to
> lock_bus() and unlock_bus() global adapter methods. With this in place
> all i2c clients drivers may lock i2c bus for a desired number of i2c
> transactions (e.g. write-wait-read) without being aware of that such bus
> is shared with another entity.
> 
> Modify i2c_dw_probe_lock_support() to select correct semaphore
> implementation at runtime, since now we have more than one available.
> 
> Configure new matching ACPI ID "AMDI0019" and register
> ARBITRATION_SEMAPHORE flag in order to distinguish setup with PSP
> arbitration.

> Add new entry in MAINTAINERS file to cover new module.

It's confusing. You added yourself as a reviewer for I2C DesignWare driver,
which is great, but not described in the commit message.

...

>  	{ "AMD0020", APD_ADDR(cz_uart_desc) },

>  	{ "AMDI0020", APD_ADDR(cz_uart_desc) },
>  	{ "AMDI0022", APD_ADDR(cz_uart_desc) },
> +	{ "AMDI0019", APD_ADDR(wt_i2c_desc) },

This addition adds more chaos in the ordering (the group of AMDI should be
after AMD as far as I can see here). Can you order the entries by IDs?

>  	{ "AMD0030", },
>  	{ "AMD0040", APD_ADDR(fch_misc_desc)},

...

> +#include <asm/msr.h>

Usually linux/* followed by asm/*.

> +#include <linux/i2c.h>
> +#include <linux/psp-sev.h>

types.h?

...

> +union psp_req_buffer_hdr {
> +	struct {
> +		u32 total_size;
> +		u32 status;
> +	} __packed;

What does packet bring you here?

> +	u64 hdr_val;

And why does this not have the same alignment since it's also part of
the union?

> +};
> +
> +enum psp_i2c_req_type {
> +	PSP_I2C_REQ_ACQUIRE,
> +	PSP_I2C_REQ_RELEASE,

> +	PSP_I2C_REQ_MAX,

Is MAX a terminator or not?
If former, no comma.

> +};
> +
> +struct psp_i2c_req {
> +	union psp_req_buffer_hdr hdr;
> +	enum psp_i2c_req_type type;

> +} __packed __aligned(32);

Can you explain, what this means and how it's supposed to work?

> +union psp_mbox_cmd_reg {
> +	struct psp_mbox_cmd_fields {
> +		u16 mbox_status;
> +		u8 mbox_cmd;
> +		u8 reserved:6;
> +		u8 recovery:1;
> +		u8 ready:1;

> +	} __packed fields;

So, what is the __packed purpose here?

> +	u32 val;
> +};
> +
> +struct psp_mbox {
> +	union psp_mbox_cmd_reg fields;
> +	uintptr_t i2c_req_addr;
> +} __packed;

...

> +static int psp_mbox_probe(void)
> +{
> +	unsigned long mbox_addr;
> +
> +	if (psp_get_mbox_addr(&mbox_addr))

> +		return -1;

Use error code.

> +	mbox_iomem = ioremap(mbox_addr, sizeof(struct psp_mbox));
> +	if (!mbox_iomem)
> +		return -ENOMEM;
> +
> +	return 0;
> +}

...

> +	union psp_mbox_cmd_reg tmp = {0};

> +	tmp.val = readl(&mbox->fields.val);
> +	return !!tmp.fields.recovery;

OK, I understood the purpose of unions, no, please use bitfield.h APIs.

...

> +	struct psp_mbox *mbox = (struct psp_mbox *)mbox_iomem;

Heck, no!

...

> +	/* Fill address of command-response buffer */
> +	writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr);

What does this voodoo mean?!

...

> +	start = jiffies;
> +	do {
> +		if (psp_send_cmd(req)) {
> +			ret = -EIO;
> +			goto cleanup;
> +		}
> +
> +		status = check_i2c_req_sts(req);
> +		if (!status) {
> +			dev_dbg(psp_i2c_dev, "Request accepted by PSP after %ums\n",
> +				jiffies_to_msecs(jiffies - start));
> +			ret = 0;
> +			goto cleanup;
> +		} else if (status == -EBUSY) {
> +			retry_cnt--;
> +		} else {
> +			ret = -EIO;
> +			goto cleanup;
> +		};
> +
> +		/* IF EBUSY, give PSP time to finish its i2c activities */
> +		mdelay(PSP_I2C_REQ_RETRY_DELAY_MSEC);
> +	} while (retry_cnt);

NIH iopoll.h API(s).

> +	ret = -ETIMEDOUT;

...

> +	status = psp_send_i2c_req(PSP_I2C_REQ_ACQUIRE);
> +	if (!status) {

Handle errors first.

...

> +		goto cleanup;
> +	} else if (status == -ETIMEDOUT) {

In this case it's redundant 'else'.

...

> +	/* Send a release command to PSP */
> +	status = psp_send_i2c_req(PSP_I2C_REQ_RELEASE);
> +	if (!status) {
> +		dev_dbg(psp_i2c_dev, "PSP semaphore held for %ums\n",
> +			jiffies_to_msecs(jiffies - psp_i2c_sem_acquired));
> +		goto cleanup;
> +	} else if (status == -ETIMEDOUT) {
> +		dev_err(psp_i2c_dev, "Timed out waiting for PSP to acquire I2C bus\n");
> +	} else {
> +		dev_err(psp_i2c_dev, "PSP communication error\n");
> +	}

As per above comments.

...

> +	int ret;
> +
> +	ret = rt_mutex_trylock(&adapter->bus_lock);
> +	if (!ret)

	if (ret)
		...

> +		psp_acquire_i2c_bus();
> +
> +	return ret;

...

> +	/* Allow to bind only one instance of a driver */
> +	if (!psp_i2c_dev)
> +		psp_i2c_dev = dev->dev;
> +	else
> +		return -EEXIST;

As per above.

...

> +	if (psp_mbox_probe())
> +		return -EIO;

Why error code is hidden?

...

> +	/*
> +	 * Install global locking callbacks for adapter as well as internal i2c
> +	 * controller locks

Missed period.

> +	 */

...

>  	{ "AMD0010", ACCESS_INTR_MASK },
>  	{ "AMDI0010", ACCESS_INTR_MASK },
>  	{ "AMDI0510", 0 },
> +	{ "AMDI0019", ACCESS_INTR_MASK | ARBITRATION_SEMAPHORE },

It's not in order.

...

> +static const struct i2c_dw_semaphore_callbacks i2c_dw_semaphore_cb_table[] = {
> +#ifdef CONFIG_I2C_DESIGNWARE_BAYTRAIL
> +	{
> +		.probe = i2c_dw_baytrail_probe_lock_support,

> +		.remove = NULL,

See below.

> +	},
> +#endif
> +#ifdef CONFIG_I2C_DESIGNWARE_AMDPSP
> +	{
> +		.probe = i2c_dw_amdpsp_probe_lock_support,
> +		.remove = i2c_dw_amdpsp_remove_lock_support,
> +	},
> +#endif

> +	{
> +		.probe = NULL,
> +		.remove = NULL,
> +	},

First of all, it should be terminating entry, so no comma.
On top of that, no need to assign 0/NULL to static variables.
So here, it will become as simple as

	{}

> +};

...

> +static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
> +{
> +	int ret;
> +	int i;
> +
> +	dev->semaphore_idx = -1;
> +
> +	for (i = 0; i < ARRAY_SIZE(i2c_dw_semaphore_cb_table); i++) {

> +		if (!i2c_dw_semaphore_cb_table[i].probe)
> +			continue;

Huh?

> +	}
> +
> +	return 0;
> +}

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[kernel test robot]

Hi Jan,

I love your patch! Yet something to improve:

[auto build test ERROR on wsa/i2c/for-next]
[also build test ERROR on linux/master rafael-pm/linux-next linus/master v5.16 next-20220120]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Jan-Dabros/i2c-designware-Add-support-for-AMD-PSP-semaphore/20220120-081854
base:   https://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux.git i2c/for-next
config: i386-allyesconfig (https://download.01.org/0day-ci/archive/20220120/202201202353.tVXCQlqh-lkp@intel.com/config)
compiler: gcc-9 (Debian 9.3.0-22) 9.3.0
reproduce (this is a W=1 build):
        # https://github.com/0day-ci/linux/commit/f21a75f68baddffd9c50ffdc95d419ad7dbe3f68
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Jan-Dabros/i2c-designware-Add-support-for-AMD-PSP-semaphore/20220120-081854
        git checkout f21a75f68baddffd9c50ffdc95d419ad7dbe3f68
        # save the config file to linux build tree
        mkdir build_dir
        make W=1 O=build_dir ARCH=i386 SHELL=/bin/bash

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   drivers/i2c/busses/i2c-designware-amdpsp.c: In function 'psp_send_cmd':
>> drivers/i2c/busses/i2c-designware-amdpsp.c:130:2: error: implicit declaration of function 'writeq'; did you mean 'writel'? [-Werror=implicit-function-declaration]
     130 |  writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr);
         |  ^~~~~~
         |  writel
   cc1: some warnings being treated as errors


vim +130 drivers/i2c/busses/i2c-designware-amdpsp.c

   117	
   118	static int psp_send_cmd(struct psp_i2c_req *req)
   119	{
   120		struct psp_mbox *mbox = (struct psp_mbox *)mbox_iomem;
   121		union psp_mbox_cmd_reg cmd_reg = {0};
   122	
   123		if (psp_check_mbox_recovery(mbox))
   124			return -EIO;
   125	
   126		if (psp_wait_cmd(mbox))
   127			return -EBUSY;
   128	
   129		/* Fill address of command-response buffer */
 > 130		writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr);
   131	
   132		/* Write command register to trigger processing */
   133		cmd_reg.fields.mbox_cmd = PSP_I2C_REQ_BUS_CMD;
   134		writel(cmd_reg.val, &mbox->fields.val);
   135	
   136		if (psp_wait_cmd(mbox))
   137			return -ETIMEDOUT;
   138	
   139		if (psp_check_mbox_sts(mbox))
   140			return -EIO;
   141	
   142		return 0;
   143	}
   144	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

Re: [PATCH 0/2] i2c-designware: Add support for AMD PSP semaphore

[Hans de Goede]

Hi,

On 1/20/22 13:29, Jan Dąbroś wrote:
> Hi Hans,
> 
> 
> czw., 20 sty 2022 o 12:15 Hans de Goede <hdegoede@redhat.com> napisał(a):
>>
>> Hi Jan,
>>
>> On 1/20/22 01:16, Jan Dabros wrote:
>>> This patchset comprises support for new i2c-designware controller setup on some
>>> AMD Cezanne SoCs, where x86 is sharing i2c bus with PSP. PSP uses the same
>>> controller and acts as an i2c arbitrator there (x86 is leasing bus from it).
>>>
>>> First commit aims to improve generic i2c-designware code by adding extra locking
>>> on probe() and disable() paths. I would like to ask someone with access to
>>> boards which use Intel BayTrail(CONFIG_I2C_DESIGNWARE_BAYTRAIL) to verify
>>> behavior of my changes on such setup.
>>>
>>> Second commit adds support for new PSP semaphore arbitration mechanism.
>>> Implementation is similar to the one from i2c-designware-baytrail.c however
>>> there are two main differences:
>>> 1) Add new ACPI ID in order to protect against silent binding of the old driver
>>> to the setup with PSP semaphore. Extra flag ARBITRATION_SEMAPHORE added to this
>>> new _HID allows to recognize setup with PSP.
>>> 2) Beside acquire_lock() and release_lock() methods we are also applying quirks
>>> to the lock_bus() and unlock_bus() global adapter methods. With this in place
>>> all i2c clients drivers may lock i2c bus for a desired number of i2c
>>> transactions (e.g. write-wait-read) without being aware of that such bus is
>>> shared with another entity.
>>>
>>> This patchset is a follow-up to the RFC sent earlier on LKML [1], with review
>>> comments applied.
>>>
>>> Looking forward to some feedback.
>>>
>>> [1] https://lkml.org/lkml/2021/12/22/219
>>
>>
>> Thank you for your patch series.
>>
>> As you may have seen I've done a lot of work on the Bay Trail semaphore
>> thing. I also own several Bay Trail and Cherry Trail based devices which
>> use this setup.
>>
>> I'll add your patches to my personal WIP tree which I regularly run
>> on these devices and I'll report back if I notice any issues.
> 
> Thanks in advance, this will be really helpful! I don't have Bay
> Trail/Cherry Trail, so I've only tested that build of Bay Trail
> semaphore isn't broken.
> 
> I would like to point to new locks in i2c_dw_disable() method as
> something to be the most fragile and error-prone, will be great if you
> can verify this thoroughly. This function is invoked on both
> dw_i2c_driver.remove() and dw_i2c_plat_suspend() paths. Considering
> that Bay Trail semaphore means that i2c bus is shared with PMIC, I'm
> not sure whether all corner cases are secured especially on platform
> suspend.

You are right that the whole sharing of the bus to the PMIC between
the SoC's internal power-management microcontroller (P-Unit) and
the OS is a bit fragile (it really is a bit crazy design IMHO).

You are also right that disabling the controller on suspend
is a problem, because once everything is suspended and we hit
deeper power-saving states then the P-Unit actually needs the
controller to tell the PMIC to disable certain regulators; and
the P-Unit is not prepared for us having turned the controller off,
therefor dw_i2c_plat_suspend() looks like this:

static int dw_i2c_plat_suspend(struct device *dev)
{
        struct dw_i2c_dev *i_dev = dev_get_drvdata(dev);

        i_dev->suspended = true;

        if (i_dev->shared_with_punit)
                return 0;

	...


Note the shared_with_punit flag, so on the Bay Trail case
i2c_dw_disable() never gets called on suspend, so that should
not be an issue.

So all in all I don't really expect any problems, still thank
you for Cc-ing me.

Regards,

Hans



>> One remark, I notice that there are no AMD people in the Cc, it
>> would be good if you can find someone from AMD to look at this,
>> also see my remarks to the 2nd patch in my reply to that patch.
> 
> This was partially discussed with AMD folks and you are right that I
> should include someone from AMD to take a look at this. Thanks for all
> your comments!
> 
>> Regards,
>>
>> Hans
>>
>>
>>
>>
>>>
>>> Jan Dabros (2):
>>>   i2c: designware: Add missing locks
>>>   i2c: designware: Add AMD PSP I2C bus support
>>>
>>>  MAINTAINERS                                  |   1 +
>>>  drivers/acpi/acpi_apd.c                      |   1 +
>>>  drivers/i2c/busses/Kconfig                   |  10 +
>>>  drivers/i2c/busses/Makefile                  |   1 +
>>>  drivers/i2c/busses/i2c-designware-amdpsp.c   | 357 +++++++++++++++++++
>>>  drivers/i2c/busses/i2c-designware-baytrail.c |  10 +-
>>>  drivers/i2c/busses/i2c-designware-common.c   |  12 +
>>>  drivers/i2c/busses/i2c-designware-core.h     |  18 +-
>>>  drivers/i2c/busses/i2c-designware-master.c   |   6 +
>>>  drivers/i2c/busses/i2c-designware-platdrv.c  |  61 ++++
>>>  10 files changed, 469 insertions(+), 8 deletions(-)
>>>  create mode 100644 drivers/i2c/busses/i2c-designware-amdpsp.c
>>>
>>
> 

Re: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Andy Shevchenko]

On Thu, Jan 20, 2022 at 11:33:05PM +0800, kernel test robot wrote:

> If you fix the issue, kindly add following tag as appropriate
> Reported-by: kernel test robot <lkp@intel.com>
> 
> All errors (new ones prefixed by >>):
> 
>    drivers/i2c/busses/i2c-designware-amdpsp.c: In function 'psp_send_cmd':
> >> drivers/i2c/busses/i2c-designware-amdpsp.c:130:2: error: implicit declaration of function 'writeq'; did you mean 'writel'? [-Werror=implicit-function-declaration]
>      130 |  writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr);
>          |  ^~~~~~
>          |  writel
>    cc1: some warnings being treated as errors

Adding io-64-nonatomic-lo-hi.h after io.h should fix this.


-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dąbroś]

(...)

> > --- /dev/null
> > +++ b/drivers/i2c/busses/i2c-designware-amdpsp.c
> > @@ -0,0 +1,357 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +
> > +#include <asm/msr.h>
> > +#include <linux/i2c.h>
> > +#include <linux/psp-sev.h>
> > +
> > +#include "i2c-designware-core.h"
>
> So all the stuff starting here:
>
> > +
> > +#define MSR_AMD_PSP_ADDR     0xc00110a2
> > +#define PSP_MBOX_OFFSET              0x10570
> > +#define PSP_CMD_TIMEOUT_MS   500
> > +
> > +#define PSP_I2C_REQ_BUS_CMD          0x64
> > +#define PSP_I2C_REQ_RETRY_CNT                10
> > +#define PSP_I2C_REQ_RETRY_DELAY_MSEC 50
> > +#define PSP_I2C_REQ_STS_OK           0x0
> > +#define PSP_I2C_REQ_STS_BUS_BUSY     0x1
> > +#define PSP_I2C_REQ_STS_INV_PARAM    0x3
> > +
> > +union psp_req_buffer_hdr {
> > +     struct {
> > +             u32 total_size;
> > +             u32 status;
> > +     } __packed;
> > +     u64 hdr_val;
> > +};
> > +
> > +enum psp_i2c_req_type {
> > +     PSP_I2C_REQ_ACQUIRE,
> > +     PSP_I2C_REQ_RELEASE,
> > +     PSP_I2C_REQ_MAX,
> > +};
> > +
> > +struct psp_i2c_req {
> > +     union psp_req_buffer_hdr hdr;
> > +     enum psp_i2c_req_type type;
> > +} __packed __aligned(32);
> > +
> > +union psp_mbox_cmd_reg {
> > +     struct psp_mbox_cmd_fields {
> > +             u16 mbox_status;
> > +             u8 mbox_cmd;
> > +             u8 reserved:6;
> > +             u8 recovery:1;
> > +             u8 ready:1;
> > +     } __packed fields;
> > +     u32 val;
> > +};
> > +
> > +struct psp_mbox {
> > +     union psp_mbox_cmd_reg fields;
> > +     uintptr_t i2c_req_addr;
> > +} __packed;
> > +
> > +static DEFINE_MUTEX(psp_i2c_access_mutex);
> > +static unsigned long psp_i2c_sem_acquired;
> > +static void __iomem *mbox_iomem;
> > +static u32 psp_i2c_access_count;
> > +static bool psp_i2c_mbox_fail;
> > +static struct device *psp_i2c_dev;
> > +
> > +static int psp_get_mbox_addr(unsigned long *mbox_addr)
> > +{
> > +     unsigned long long psp_mmio;
> > +
> > +     if (rdmsrl_safe(MSR_AMD_PSP_ADDR, &psp_mmio))
> > +             return -EIO;
> > +
> > +     *mbox_addr = (unsigned long)(psp_mmio + PSP_MBOX_OFFSET);
> > +
> > +     return 0;
> > +}
> > +
> > +static int psp_mbox_probe(void)
> > +{
> > +     unsigned long mbox_addr;
> > +
> > +     if (psp_get_mbox_addr(&mbox_addr))
> > +             return -1;
> > +
> > +     mbox_iomem = ioremap(mbox_addr, sizeof(struct psp_mbox));
> > +     if (!mbox_iomem)
> > +             return -ENOMEM;
> > +
> > +     return 0;
> > +}
> > +
> > +/* Recovery field should be equal 0 to start sending commands */
> > +static int psp_check_mbox_recovery(struct psp_mbox *mbox)
> > +{
> > +     union psp_mbox_cmd_reg tmp = {0};
> > +
> > +     tmp.val = readl(&mbox->fields.val);
> > +     return !!tmp.fields.recovery;
> > +}
> > +
> > +static int psp_wait_cmd(struct psp_mbox *mbox)
> > +{
> > +     union psp_mbox_cmd_reg expected = { .val = 0 };
> > +     u32 tmp;
> > +
> > +     /* Expect mbox_cmd to be cleared and ready bit to be set by PSP */
> > +     expected.fields.ready = 1;
> > +
> > +     return readl_poll_timeout(&mbox->fields.val, tmp, (tmp == expected.val),
> > +                               0, 1000 * PSP_CMD_TIMEOUT_MS);
> > +}
> > +
> > +/* Status equal to 0 means that PSP succeed processing command */
> > +static int psp_check_mbox_sts(struct psp_mbox *mbox)
> > +{
> > +     union psp_mbox_cmd_reg cmd_reg = {0};
> > +
> > +     cmd_reg.val = readl(&mbox->fields.val);
> > +     return cmd_reg.fields.mbox_status;
> > +}
> > +
> > +static int psp_send_cmd(struct psp_i2c_req *req)
> > +{
> > +     struct psp_mbox *mbox = (struct psp_mbox *)mbox_iomem;
> > +     union psp_mbox_cmd_reg cmd_reg = {0};
> > +
> > +     if (psp_check_mbox_recovery(mbox))
> > +             return -EIO;
> > +
> > +     if (psp_wait_cmd(mbox))
> > +             return -EBUSY;
> > +
> > +     /* Fill address of command-response buffer */
> > +     writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr);
> > +
> > +     /* Write command register to trigger processing */
> > +     cmd_reg.fields.mbox_cmd = PSP_I2C_REQ_BUS_CMD;
> > +     writel(cmd_reg.val, &mbox->fields.val);
> > +
> > +     if (psp_wait_cmd(mbox))
> > +             return -ETIMEDOUT;
> > +
> > +     if (psp_check_mbox_sts(mbox))
> > +             return -EIO;
> > +
> > +     return 0;
> > +}
>
> Through here seems to all be generic code for accessing
> the AMD PSP. To me this seems like something which belongs
> in a separate AMD-PSP-mbox driver/lib, which can then be
> shared between other kernel drivers which may also want
> to access PSP.

I see your point clearly and actually it is not an accident that I've
put all PSP-mailbox methods in one "block". They are logically
different than the rest of i2c-adapter specific methods.

That being said, above PSP mailbox was created by AMD solely for the
purpose of i2c_arbitration. It has its own set of commands and
specific format of the command-response buffer. Thus it is not and it
won't be generic in the future. There are already upstreamed drivers
from AMD (under drivers/crypto/ccp/) which made use of PSP, however
their channel of communication looks completely different than the
very simple i2c_arbitration model implemented above.

Because of this I'm treating this as an i2c_semaphore-related code and
putting this in this module. In my opinion moving this into some
separate driver (which will be actually used only here) makes code
less clear. But let's also hear some voice from AMD.

>
> Sorta like the generic iosf_mbi_read() and
> iosf_mbi_write() functions from:
>
> arch/x86/platform/intel/iosf_mbi.c
>
> used on the Intel chips, which are also used outside of
> the I2C bus-locking code.
>
> This is also one of the reasons why I think it would be
> good to get some AMD folks involved in this, since they
> may be aware of other drivers which also need to access
> the PSP mbox.
>

Right, I'm adding mario.limonciello@amd.com to the CC, so that he can comment.

(...)

> > +/*
> > + * Locking methods are based on the default implementation from
> > + * drivers/i2c/i2c-core.base.c, but with psp acquire and release operations
> > + * added. With this in place we can ensure that i2c clients on the bus shared
> > + * with psp are able to lock HW access to the bus for arbitrary number of
> > + * operations - that is e.g. write-wait-read.
> > + */
> > +static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter,
> > +                                     unsigned int flags)
> > +{
> > +     psp_acquire_i2c_bus();
> > +     rt_mutex_lock_nested(&adapter->bus_lock, i2c_adapter_depth(adapter));
>
> This does not do what you think it does and you will still deadlock
> when things nest because of someone taking the bus-lock and then
> the main i2c-designware transfer function calling the acquire_lock
> callback.

I haven't used rt_mutex_lock_nested() with the intent to prevent me
from deadlock when i2c-designware calls acquire_lock with bus-lock
already taken. This is a method copied from
drivers/i2c/i2c-core-base.c (BTW, I have a typo in above comment).
This is the default implementation applied by i2c-core when particular
adapter doesn't register its own locking callbacks - thus it is called
for i2c-designware for all platforms.

In case of this driver internal i2c-designware acquire_lock() is equal
to psp_acquire_i2c_bus(). In other words, bus-level lock
i2c_adapter_dw_psp_lock_bus() is a superset of internal adapter's
acquire_lock().

In order to prevent deadlock which you are talking about, I'm using
reference lock counter inside psp_acquire_i2c_bus() thus it is safe to
invoke acquire_lock() when bus-lock is already taken.

>
> The _nested postfix is only for the lockdep lock-debugger, this
> actually turns into a regular mutex_lock when lockdep is not enabled:
>
> #ifdef CONFIG_DEBUG_LOCK_ALLOC
> extern void rt_mutex_lock_nested(struct rt_mutex *lock, unsigned int subclass);
> #define rt_mutex_lock(lock) rt_mutex_lock_nested(lock, 0)
> #else
> extern void rt_mutex_lock(struct rt_mutex *lock);
> #define rt_mutex_lock_nested(lock, subclass) rt_mutex_lock(lock)
> #endif
>
> The _nested postfix as such is only to tell the lockdep code that
> even though it seems we are trying to take the same mutex twice
> since in both cases it is of i2c_adapter.rt_mutex "lock class"
> that we are sure it is never the same i2c_adapter (but rather
> one which always gets called in a nested fashion from another
> i2c_adapter).
>
> IOW this only disables a false-positive lockdep warning, it does
> not allow taking the same mutex twice, you will still hang on
> the second mutex_lock call on the same lock.

Thanks for the technical background about rt_mutex_lock_nested. I
think we should keep using it as is, since as I wrote above I don't
have any reasoning to modify it here.

>
> Also I don't think you are allowed to use the bus_locking code
> like this. The i2c bus-locking code is intended to deal with
> busses which have muxes in them, where the mux must be set
> to the right branch of the bus to reach the client and then
> not be changed during the transfer to that client.
>
> So i2c-client drivers are never supposed to directly call
> the bus-locking functions.

I think you are not correct here. There are examples of i2c-clients
which are using i2c bus_locking for the purpose of locking adapter for
the bunch of i2c transactions.

As an example let's take drivers/char/tpm/tpm_tis_i2c_cr50.c. It
operates in write-wait-read model and there is i2c_lock_bus() call
used to ensure that bus won't be released -
https://github.com/torvalds/linux/blob/master/drivers/char/tpm/tpm_tis_i2c_cr50.c#L202.

Similar model is followed in drivers/char/tpm/tpm_i2c_infineon.c and
couple of other i2c-client drivers.

> This is why in the Bay Trail case we have i2c-drivers
> directly calling iosf_mbi_block_punit_i2c_access() and
> iosf_mbi_unblock_punit_i2c_access() to lock the bus
> for multiple i2c-transfers. We can get away with this there
> because the bus in question is only used to access the
> PMIC and that PMIC is only used on Bay Trail (and CHT)
> boards, so the PMIC drivers can just hard-code these
> calls.
>
> If you need to take the PSP I2C semaphore for multiple
> transfers in some generic drivers, then I guess that the
> i2c-subsys will need to get some new i2c_adapter callbacks
> to acquire / release the bus for i2c-controllers where
> the bus/controller is shared with some co-processor like
> in the PSP case.

This is exactly my intention to support generic i2c-clients drivers
without them being aware that i2c-adapter above is using some
semaphore/arbitration. Hopefully you can agree with me that currently
available bus_locking can be used and is enough for this purpose.

> Also note that iosf_mbi_block_punit_i2c_access() and
> iosf_mbi_unblock_punit_i2c_access() do their own
> ref/lock-counting to allow calling them multiple times and
> the first block call takes the bus and the last unblock
> call releases it.

This is exactly what I was talking about above and also implemented
within psp_acquire_i2c_bus() and psp_release_i2c_bus().

Best Regards,
Jan


>
> Regards,
>
> Hans
>
>
>
>
>
> > +}
> > +
> > +static int i2c_adapter_dw_psp_trylock_bus(struct i2c_adapter *adapter,
> > +                                       unsigned int flags)
> > +{
> > +     int ret;
> > +
> > +     ret = rt_mutex_trylock(&adapter->bus_lock);
> > +     if (!ret)
> > +             psp_acquire_i2c_bus();
> > +
> > +     return ret;
> > +}
> > +
> > +static void i2c_adapter_dw_psp_unlock_bus(struct i2c_adapter *adapter,
> > +                                       unsigned int flags)
> > +{
> > +     psp_release_i2c_bus();
> > +     rt_mutex_unlock(&adapter->bus_lock);
> > +}
> > +
> > +static const struct i2c_lock_operations i2c_dw_psp_lock_ops = {
> > +     .lock_bus = i2c_adapter_dw_psp_lock_bus,
> > +     .trylock_bus = i2c_adapter_dw_psp_trylock_bus,
> > +     .unlock_bus = i2c_adapter_dw_psp_unlock_bus,
> > +};
> > +
> > +int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev)
> > +{
> > +     if (!dev || !dev->dev)
> > +             return -ENODEV;
> > +
> > +     if (!(dev->flags & ARBITRATION_SEMAPHORE))
> > +             return -ENODEV;
> > +
> > +     /* Allow to bind only one instance of a driver */
> > +     if (!psp_i2c_dev)
> > +             psp_i2c_dev = dev->dev;
> > +     else
> > +             return -EEXIST;
> > +
> > +     if (psp_mbox_probe())
> > +             return -EIO;
> > +
> > +     dev_info(psp_i2c_dev, "I2C bus managed by AMD PSP\n");
> > +
> > +     /*
> > +      * Install global locking callbacks for adapter as well as internal i2c
> > +      * controller locks
> > +      */
> > +     dev->adapter.lock_ops = &i2c_dw_psp_lock_ops;
> > +     dev->acquire_lock = psp_acquire_i2c_bus;
> > +     dev->release_lock = psp_release_i2c_bus;
> > +
> > +     return 0;
> > +}
> > +
> > +/* Unmap area used as a mailbox with PSP */
> > +void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev)
> > +{
> > +     iounmap(mbox_iomem);
> > +}
> > diff --git a/drivers/i2c/busses/i2c-designware-baytrail.c b/drivers/i2c/busses/i2c-designware-baytrail.c
> > index c6a7a00e1d52..0c674542dd99 100644
> > --- a/drivers/i2c/busses/i2c-designware-baytrail.c
> > +++ b/drivers/i2c/busses/i2c-designware-baytrail.c
> > @@ -12,25 +12,25 @@
> >
> >  #include "i2c-designware-core.h"
> >
> > -int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
> > +int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev)
> >  {
> >       acpi_status status;
> >       unsigned long long shared_host = 0;
> >       acpi_handle handle;
> >
> >       if (!dev || !dev->dev)
> > -             return 0;
> > +             return -ENODEV;
> >
> >       handle = ACPI_HANDLE(dev->dev);
> >       if (!handle)
> > -             return 0;
> > +             return -ENODEV;
> >
> >       status = acpi_evaluate_integer(handle, "_SEM", NULL, &shared_host);
> >       if (ACPI_FAILURE(status))
> > -             return 0;
> > +             return -ENODEV;
> >
> >       if (!shared_host)
> > -             return 0;
> > +             return -ENODEV;
> >
> >       if (!iosf_mbi_available())
> >               return -EPROBE_DEFER;
> > diff --git a/drivers/i2c/busses/i2c-designware-core.h b/drivers/i2c/busses/i2c-designware-core.h
> > index 4b26cba40139..1d65212fddbd 100644
> > --- a/drivers/i2c/busses/i2c-designware-core.h
> > +++ b/drivers/i2c/busses/i2c-designware-core.h
> > @@ -227,6 +227,8 @@ struct reset_control;
> >   * @hs_lcnt: high speed LCNT value
> >   * @acquire_lock: function to acquire a hardware lock on the bus
> >   * @release_lock: function to release a hardware lock on the bus
> > + * @semaphore_idx: Index of table with semaphore type attached to the bus. It's
> > + *   -1 if there is no semaphore.
> >   * @shared_with_punit: true if this bus is shared with the SoCs PUNIT
> >   * @disable: function to disable the controller
> >   * @disable_int: function to disable all interrupts
> > @@ -285,6 +287,7 @@ struct dw_i2c_dev {
> >       u16                     hs_lcnt;
> >       int                     (*acquire_lock)(void);
> >       void                    (*release_lock)(void);
> > +     int                     semaphore_idx;
> >       bool                    shared_with_punit;
> >       void                    (*disable)(struct dw_i2c_dev *dev);
> >       void                    (*disable_int)(struct dw_i2c_dev *dev);
> > @@ -297,6 +300,7 @@ struct dw_i2c_dev {
> >
> >  #define ACCESS_INTR_MASK     BIT(0)
> >  #define ACCESS_NO_IRQ_SUSPEND        BIT(1)
> > +#define ARBITRATION_SEMAPHORE        BIT(2)
> >
> >  #define MODEL_MSCC_OCELOT    BIT(8)
> >  #define MODEL_BAIKAL_BT1     BIT(9)
> > @@ -310,6 +314,11 @@ struct dw_i2c_dev {
> >  #define AMD_UCSI_INTR_REG    0x474
> >  #define AMD_UCSI_INTR_EN     0xd
> >
> > +struct i2c_dw_semaphore_callbacks {
> > +     int     (*probe)(struct dw_i2c_dev *dev);
> > +     void    (*remove)(struct dw_i2c_dev *dev);
> > +};
> > +
> >  int i2c_dw_init_regmap(struct dw_i2c_dev *dev);
> >  u32 i2c_dw_scl_hcnt(u32 ic_clk, u32 tSYMBOL, u32 tf, int cond, int offset);
> >  u32 i2c_dw_scl_lcnt(u32 ic_clk, u32 tLOW, u32 tf, int offset);
> > @@ -370,9 +379,12 @@ static inline void i2c_dw_configure(struct dw_i2c_dev *dev)
> >  }
> >
> >  #if IS_ENABLED(CONFIG_I2C_DESIGNWARE_BAYTRAIL)
> > -extern int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev);
> > -#else
> > -static inline int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev) { return 0; }
> > +int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev);
> > +#endif
> > +
> > +#if IS_ENABLED(CONFIG_I2C_DESIGNWARE_AMDPSP)
> > +int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev);
> > +void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev);
> >  #endif
> >
> >  int i2c_dw_validate_speed(struct dw_i2c_dev *dev);
> > diff --git a/drivers/i2c/busses/i2c-designware-platdrv.c b/drivers/i2c/busses/i2c-designware-platdrv.c
> > index 2bd81abc86f6..5844a4df4141 100644
> > --- a/drivers/i2c/busses/i2c-designware-platdrv.c
> > +++ b/drivers/i2c/busses/i2c-designware-platdrv.c
> > @@ -51,6 +51,7 @@ static const struct acpi_device_id dw_i2c_acpi_match[] = {
> >       { "AMD0010", ACCESS_INTR_MASK },
> >       { "AMDI0010", ACCESS_INTR_MASK },
> >       { "AMDI0510", 0 },
> > +     { "AMDI0019", ACCESS_INTR_MASK | ARBITRATION_SEMAPHORE },
> >       { "APMC0D0F", 0 },
> >       { "HISI02A1", 0 },
> >       { "HISI02A2", 0 },
> > @@ -204,6 +205,64 @@ static const struct dmi_system_id dw_i2c_hwmon_class_dmi[] = {
> >       { } /* terminate list */
> >  };
> >
> > +static const struct i2c_dw_semaphore_callbacks i2c_dw_semaphore_cb_table[] = {
> > +#ifdef CONFIG_I2C_DESIGNWARE_BAYTRAIL
> > +     {
> > +             .probe = i2c_dw_baytrail_probe_lock_support,
> > +             .remove = NULL,
> > +     },
> > +#endif
> > +#ifdef CONFIG_I2C_DESIGNWARE_AMDPSP
> > +     {
> > +             .probe = i2c_dw_amdpsp_probe_lock_support,
> > +             .remove = i2c_dw_amdpsp_remove_lock_support,
> > +     },
> > +#endif
> > +     {
> > +             .probe = NULL,
> > +             .remove = NULL,
> > +     },
> > +};
> > +
> > +static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
> > +{
> > +     int ret;
> > +     int i;
> > +
> > +     dev->semaphore_idx = -1;
> > +
> > +     for (i = 0; i < ARRAY_SIZE(i2c_dw_semaphore_cb_table); i++) {
> > +             if (!i2c_dw_semaphore_cb_table[i].probe)
> > +                     continue;
> > +
> > +             ret = i2c_dw_semaphore_cb_table[i].probe(dev);
> > +             if (!ret) {
> > +                     dev->semaphore_idx = i;
> > +                     break;
> > +             } else if (ret == -ENODEV) {
> > +                     /*
> > +                      * If there is no semaphore device attached to this
> > +                      * controller, we shouldn't abort general i2c_controller
> > +                      * probe.
> > +                      */
> > +                     continue;
> > +             } else {
> > +                     return ret;
> > +             }
> > +     }
> > +
> > +     return 0;
> > +}
> > +
> > +static void i2c_dw_remove_lock_support(struct dw_i2c_dev *dev)
> > +{
> > +     if (dev->semaphore_idx < 0)
> > +             return;
> > +
> > +     if (i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove)
> > +             i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove(dev);
> > +}
> > +
> >  static int dw_i2c_plat_probe(struct platform_device *pdev)
> >  {
> >       struct i2c_adapter *adap;
> > @@ -334,6 +393,8 @@ static int dw_i2c_plat_remove(struct platform_device *pdev)
> >       pm_runtime_put_sync(&pdev->dev);
> >       dw_i2c_plat_pm_cleanup(dev);
> >
> > +     i2c_dw_remove_lock_support(dev);
> > +
> >       reset_control_assert(dev->rst);
> >
> >       return 0;
> >
>

Re: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dąbroś]

czw., 20 sty 2022 o 15:21 Andy Shevchenko
<andriy.shevchenko@linux.intel.com> napisał(a):
>
> On Thu, Jan 20, 2022 at 01:16:21AM +0100, Jan Dabros wrote:
> > Implement an I2C controller sharing mechanism between the host (kernel)
> > and PSP co-processor on some platforms equipped with AMD Cezanne SoC.
> >
> > On these platforms we need to implement "software" i2c arbitration.
> > Default arbitration owner is PSP and kernel asks for acquire as well
> > as inform about release of the i2c bus via mailbox mechanism.
> >
> >             +---------+
> >  <- ACQUIRE |         |
> >   +---------|   CPU   |\
> >   |         |         | \      +----------+  SDA
> >   |         +---------+  \     |          |-------
> > MAILBOX                   +--> |  I2C-DW  |  SCL
> >   |         +---------+        |          |-------
> >   |         |         |        +----------+
> >   +---------|   PSP   |
> >    <- ACK   |         |
> >             +---------+
> >
> >             +---------+
> >  <- RELEASE |         |
> >   +---------|   CPU   |
> >   |         |         |        +----------+  SDA
> >   |         +---------+        |          |-------
> > MAILBOX                   +--> |  I2C-DW  |  SCL
> >   |         +---------+  /     |          |-------
> >   |         |         | /      +----------+
> >   +---------|   PSP   |/
> >    <- ACK   |         |
> >             +---------+
> >
> > The solution is similar to i2c-designware-baytrail.c implementation, where
> > we are using a generic i2c-designware-* driver with a small "wrapper".
> >
> > In contrary to baytrail semaphore implementation, beside internal
> > acquire_lock() and release_lock() methods we are also applying quirks to
> > lock_bus() and unlock_bus() global adapter methods. With this in place
> > all i2c clients drivers may lock i2c bus for a desired number of i2c
> > transactions (e.g. write-wait-read) without being aware of that such bus
> > is shared with another entity.
> >
> > Modify i2c_dw_probe_lock_support() to select correct semaphore
> > implementation at runtime, since now we have more than one available.
> >
> > Configure new matching ACPI ID "AMDI0019" and register
> > ARBITRATION_SEMAPHORE flag in order to distinguish setup with PSP
> > arbitration.
>
> > Add new entry in MAINTAINERS file to cover new module.
>
> It's confusing. You added yourself as a reviewer for I2C DesignWare driver,
> which is great, but not described in the commit message.

Should I rephrase this sentence (to be more specific that I may be
helpful for reviewing amdpsp.c module) or rather you mean to exclude
drivers/i2c/busses/i2c-designware-amdpsp.c from the rest of
designware-* modules and create separate entry?

Actually initially I wasn't planning to modify MAINTAINERS (after all
I'm not an I2C DesignWare expert) until run checkpatch.pl which
recommended to do so when adding new file. Eventually for me it made
some sense since I have a platform equipped with AMD Cezanne SoC and I
will be able to review and test potential changes in
i2c-designware-amdpsp.c or in general around semaphore areas.

This may also work with different model, similar to how you pointed me
to Hans as an owner of Bay Trail platform who is acquinted with how
its i2c semaphore is working. I will simply remove myself from the
MAINTAINERS file and you can add me to the threads if there is
anything requiring my help.

Let me know which way is working for you. I just thought it is not
good to leave you alone with a new module which you cannot actually
test and don't have deep insight about how PSP-x86 communications
works.

>
> ...
>
> >       { "AMD0020", APD_ADDR(cz_uart_desc) },
>
> >       { "AMDI0020", APD_ADDR(cz_uart_desc) },
> >       { "AMDI0022", APD_ADDR(cz_uart_desc) },
> > +     { "AMDI0019", APD_ADDR(wt_i2c_desc) },
>
> This addition adds more chaos in the ordering (the group of AMDI should be
> after AMD as far as I can see here). Can you order the entries by IDs?

Sure.

>
> >       { "AMD0030", },
> >       { "AMD0040", APD_ADDR(fch_misc_desc)},
>
> ...
>
> > +#include <asm/msr.h>
>
> Usually linux/* followed by asm/*.
>
> > +#include <linux/i2c.h>
> > +#include <linux/psp-sev.h>
>
> types.h?

I need to take a deeper look at the headers included here, especially
considering errors pointed by kernel test robot. Not sure why I
haven't seen any issues on my setup.

>
> ...
>
> > +union psp_req_buffer_hdr {
> > +     struct {
> > +             u32 total_size;
> > +             u32 status;
> > +     } __packed;
>
> What does packet bring you here?

In this particular case binary-wise nothing, I can as well drop this.
It may be necessary if there are some changes in this structs fields
in future (e.g changing total_size to u8), since PSP expects every
field to be byte-aligned.

>
> > +     u64 hdr_val;
>
> And why does this not have the same alignment since it's also part of
> the union?

__packed is not about alignment of the whole struct/union but about
lack of padding between its fields. As above - in this particular case
with two u32 it doesn't matter.

>
> > +};
> > +
> > +enum psp_i2c_req_type {
> > +     PSP_I2C_REQ_ACQUIRE,
> > +     PSP_I2C_REQ_RELEASE,
>
> > +     PSP_I2C_REQ_MAX,
>
> Is MAX a terminator or not?
> If former, no comma.

ACK.

>
> > +};
> > +
> > +struct psp_i2c_req {
> > +     union psp_req_buffer_hdr hdr;
> > +     enum psp_i2c_req_type type;
>
> > +} __packed __aligned(32);
>
> Can you explain, what this means and how it's supposed to work?

This means that each instance of the struct should be aligned (32)
while at the same time no padding within members - thus this may
result in non-aligned addresses of members.

>
> > +union psp_mbox_cmd_reg {
> > +     struct psp_mbox_cmd_fields {
> > +             u16 mbox_status;
> > +             u8 mbox_cmd;
> > +             u8 reserved:6;
> > +             u8 recovery:1;
> > +             u8 ready:1;
>
> > +     } __packed fields;
>
> So, what is the __packed purpose here?

As in all above cases - considering current layout of members and
their sizes dropping `__packed` will not results in any errors.

However PSP expects all members os structs within shared buffers to be
byte-aligned, that's why I've added this attributes to be on the safe
side. If you think this doesn't make sense, I can remove them - in
(very unlikely) case of changes, one will need to add this specifier.

> > +     u32 val;
> > +};
> > +
> > +struct psp_mbox {
> > +     union psp_mbox_cmd_reg fields;
> > +     uintptr_t i2c_req_addr;
> > +} __packed;
>
> ...
>
> > +static int psp_mbox_probe(void)
> > +{
> > +     unsigned long mbox_addr;
> > +
> > +     if (psp_get_mbox_addr(&mbox_addr))
>
> > +             return -1;
>
> Use error code.

ACK.

>
> > +     mbox_iomem = ioremap(mbox_addr, sizeof(struct psp_mbox));
> > +     if (!mbox_iomem)
> > +             return -ENOMEM;
> > +
> > +     return 0;
> > +}
>
> ...
>
> > +     union psp_mbox_cmd_reg tmp = {0};
>
> > +     tmp.val = readl(&mbox->fields.val);
> > +     return !!tmp.fields.recovery;
>
> OK, I understood the purpose of unions, no, please use bitfield.h APIs.

OK.

> ...
>
> > +     struct psp_mbox *mbox = (struct psp_mbox *)mbox_iomem;
>
> Heck, no!

I need to get acquinted to the kernel-reviewer language:) Pardon my
ignorance, but just to be sure I get what you mean here:
I'm using global mbox_iomem to keep address of PSP mailbox in IO
memory. Casting this to struct psp_mbox layout here, to make access
more convenient.
Your point here is that:
1. I should move the assignment out from the variable declaration part
of this function;
2. I should use ioremap/iounmap each time in psp_send_cmd instead of
using it once in `probe` and unmap in `remove`?
I thought about this option as to be less effective performance-wise
(even though I can get rid of one global variable).
3. Something else?

> ...
>
> > +     /* Fill address of command-response buffer */
> > +     writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr);
>
> What does this voodoo mean?!

Basically I need to take physical address (__psp_pa) of request buffer
req and write this down into mailbox IO memory.
This should be spread into more lines with some comments, is this your point?

>
> ...
>
> > +     start = jiffies;
> > +     do {
> > +             if (psp_send_cmd(req)) {
> > +                     ret = -EIO;
> > +                     goto cleanup;
> > +             }
> > +
> > +             status = check_i2c_req_sts(req);
> > +             if (!status) {
> > +                     dev_dbg(psp_i2c_dev, "Request accepted by PSP after %ums\n",
> > +                             jiffies_to_msecs(jiffies - start));
> > +                     ret = 0;
> > +                     goto cleanup;
> > +             } else if (status == -EBUSY) {
> > +                     retry_cnt--;
> > +             } else {
> > +                     ret = -EIO;
> > +                     goto cleanup;
> > +             };
> > +
> > +             /* IF EBUSY, give PSP time to finish its i2c activities */
> > +             mdelay(PSP_I2C_REQ_RETRY_DELAY_MSEC);
> > +     } while (retry_cnt);
>
> NIH iopoll.h API(s).

I don't think macros avaialble in iopoll.h are suitable here.
Procedure above is not about simply reading some IO and waiting for
particular condition to be met with this particular value. Eventually
`psp_send_cmd()` invokes `psp_wait_cmd()` where I'm using
`readl_poll_timeout()`, so on lower level I'm making use of this API.
However here I don't see any obvious method how to incorporate
iopoll.h API to reach my goal.

> > +     ret = -ETIMEDOUT;
>
> ...
>
> > +     status = psp_send_i2c_req(PSP_I2C_REQ_ACQUIRE);
> > +     if (!status) {
>
> Handle errors first.
>
> ...
>
> > +             goto cleanup;
> > +     } else if (status == -ETIMEDOUT) {
>
> In this case it's redundant 'else'.

Addressing above two comments - what do you think about below:
if (status) {
      if (status == -ETIMEDOUT)
               dev_err(psp_i2c_dev, "Timed out waiting for PSP to
release I2C bus\n");
      else
               dev_err(psp_i2c_dev, "PSP communication error\n");

       dev_err(psp_i2c_dev, "PSP communication error\n");
       psp_i2c_mbox_fail = true;
       goto cleanup;
}

psp_i2c_sem_acquired = jiffies;
psp_i2c_access_count++;
(...)

> ...
>
> > +     /* Send a release command to PSP */
> > +     status = psp_send_i2c_req(PSP_I2C_REQ_RELEASE);
> > +     if (!status) {
> > +             dev_dbg(psp_i2c_dev, "PSP semaphore held for %ums\n",
> > +                     jiffies_to_msecs(jiffies - psp_i2c_sem_acquired));
> > +             goto cleanup;
> > +     } else if (status == -ETIMEDOUT) {
> > +             dev_err(psp_i2c_dev, "Timed out waiting for PSP to acquire I2C bus\n");
> > +     } else {
> > +             dev_err(psp_i2c_dev, "PSP communication error\n");
> > +     }
>
> As per above comments.

OK, will do similar as proposed above if this looks good for you.

> ...
>
> > +     int ret;
> > +
> > +     ret = rt_mutex_trylock(&adapter->bus_lock);
> > +     if (!ret)
>
>         if (ret)
>                 ...

ACK.

> > +             psp_acquire_i2c_bus();
> > +
> > +     return ret;
>
> ...
>
> > +     /* Allow to bind only one instance of a driver */
> > +     if (!psp_i2c_dev)
> > +             psp_i2c_dev = dev->dev;
> > +     else
> > +             return -EEXIST;
>
> As per above.

ACK.

> ...
>
> > +     if (psp_mbox_probe())
> > +             return -EIO;
>
> Why error code is hidden?

Will propagate error from psp_mbox_probe.

> ...
>
> > +     /*
> > +      * Install global locking callbacks for adapter as well as internal i2c
> > +      * controller locks
>
> Missed period.

ACK.

> > +      */
>
> ...
>
> >       { "AMD0010", ACCESS_INTR_MASK },
> >       { "AMDI0010", ACCESS_INTR_MASK },
> >       { "AMDI0510", 0 },
> > +     { "AMDI0019", ACCESS_INTR_MASK | ARBITRATION_SEMAPHORE },
>
> It's not in order.

ACK.

> ...
>
> > +static const struct i2c_dw_semaphore_callbacks i2c_dw_semaphore_cb_table[] = {
> > +#ifdef CONFIG_I2C_DESIGNWARE_BAYTRAIL
> > +     {
> > +             .probe = i2c_dw_baytrail_probe_lock_support,
>
> > +             .remove = NULL,
>
> See below.
>
> > +     },
> > +#endif
> > +#ifdef CONFIG_I2C_DESIGNWARE_AMDPSP
> > +     {
> > +             .probe = i2c_dw_amdpsp_probe_lock_support,
> > +             .remove = i2c_dw_amdpsp_remove_lock_support,
> > +     },
> > +#endif
>
> > +     {
> > +             .probe = NULL,
> > +             .remove = NULL,
> > +     },
>
> First of all, it should be terminating entry, so no comma.
> On top of that, no need to assign 0/NULL to static variables.
> So here, it will become as simple as
>
>         {}

ACK.

> > +};
>
> ...
>
> > +static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
> > +{
> > +     int ret;
> > +     int i;
> > +
> > +     dev->semaphore_idx = -1;
> > +
> > +     for (i = 0; i < ARRAY_SIZE(i2c_dw_semaphore_cb_table); i++) {
>
> > +             if (!i2c_dw_semaphore_cb_table[i].probe)
> > +                     continue;
>
> Huh?

Just to be sure I get your point.
Once I found terminating entry, I will get out of the loop and return
0 as there are no semaphores to be "applied". Actually I should
probably use `break;` here as there shouldn't be a case when certain
type of semaphore installs without `probe` being implemented.

Best Regards,
Jan

> > +     }
> > +
> > +     return 0;
> > +}
>
> --
> With Best Regards,
> Andy Shevchenko
>
>

Re: [PATCH 0/2] i2c-designware: Add support for AMD PSP semaphore

[Jan Dąbroś]

czw., 20 sty 2022 o 16:57 Hans de Goede <hdegoede@redhat.com> napisał(a):
>
> Hi,
>
> On 1/20/22 13:29, Jan Dąbroś wrote:
> > Hi Hans,
> >
> >
> > czw., 20 sty 2022 o 12:15 Hans de Goede <hdegoede@redhat.com> napisał(a):
> >>
> >> Hi Jan,
> >>
> >> On 1/20/22 01:16, Jan Dabros wrote:
> >>> This patchset comprises support for new i2c-designware controller setup on some
> >>> AMD Cezanne SoCs, where x86 is sharing i2c bus with PSP. PSP uses the same
> >>> controller and acts as an i2c arbitrator there (x86 is leasing bus from it).
> >>>
> >>> First commit aims to improve generic i2c-designware code by adding extra locking
> >>> on probe() and disable() paths. I would like to ask someone with access to
> >>> boards which use Intel BayTrail(CONFIG_I2C_DESIGNWARE_BAYTRAIL) to verify
> >>> behavior of my changes on such setup.
> >>>
> >>> Second commit adds support for new PSP semaphore arbitration mechanism.
> >>> Implementation is similar to the one from i2c-designware-baytrail.c however
> >>> there are two main differences:
> >>> 1) Add new ACPI ID in order to protect against silent binding of the old driver
> >>> to the setup with PSP semaphore. Extra flag ARBITRATION_SEMAPHORE added to this
> >>> new _HID allows to recognize setup with PSP.
> >>> 2) Beside acquire_lock() and release_lock() methods we are also applying quirks
> >>> to the lock_bus() and unlock_bus() global adapter methods. With this in place
> >>> all i2c clients drivers may lock i2c bus for a desired number of i2c
> >>> transactions (e.g. write-wait-read) without being aware of that such bus is
> >>> shared with another entity.
> >>>
> >>> This patchset is a follow-up to the RFC sent earlier on LKML [1], with review
> >>> comments applied.
> >>>
> >>> Looking forward to some feedback.
> >>>
> >>> [1] https://lkml.org/lkml/2021/12/22/219
> >>
> >>
> >> Thank you for your patch series.
> >>
> >> As you may have seen I've done a lot of work on the Bay Trail semaphore
> >> thing. I also own several Bay Trail and Cherry Trail based devices which
> >> use this setup.
> >>
> >> I'll add your patches to my personal WIP tree which I regularly run
> >> on these devices and I'll report back if I notice any issues.
> >
> > Thanks in advance, this will be really helpful! I don't have Bay
> > Trail/Cherry Trail, so I've only tested that build of Bay Trail
> > semaphore isn't broken.
> >
> > I would like to point to new locks in i2c_dw_disable() method as
> > something to be the most fragile and error-prone, will be great if you
> > can verify this thoroughly. This function is invoked on both
> > dw_i2c_driver.remove() and dw_i2c_plat_suspend() paths. Considering
> > that Bay Trail semaphore means that i2c bus is shared with PMIC, I'm
> > not sure whether all corner cases are secured especially on platform
> > suspend.
>
> You are right that the whole sharing of the bus to the PMIC between
> the SoC's internal power-management microcontroller (P-Unit) and
> the OS is a bit fragile (it really is a bit crazy design IMHO).
>
> You are also right that disabling the controller on suspend
> is a problem, because once everything is suspended and we hit
> deeper power-saving states then the P-Unit actually needs the
> controller to tell the PMIC to disable certain regulators; and
> the P-Unit is not prepared for us having turned the controller off,
> therefor dw_i2c_plat_suspend() looks like this:
>
> static int dw_i2c_plat_suspend(struct device *dev)
> {
>         struct dw_i2c_dev *i_dev = dev_get_drvdata(dev);
>
>         i_dev->suspended = true;
>
>         if (i_dev->shared_with_punit)
>                 return 0;
>
>         ...
>
>
> Note the shared_with_punit flag, so on the Bay Trail case
> i2c_dw_disable() never gets called on suspend, so that should
> not be an issue.

Thanks for pointing this! So actually the only path which is now (with
my patch) altered on the Bay Trails is unbinding driver from the
device which will call dw_i2c_driver.remove().

>
> So all in all I don't really expect any problems, still thank
> you for Cc-ing me.

Thanks a lot for your help with testing and reviewing my code.

>
> Regards,
>
> Hans
>
>
>
> >> One remark, I notice that there are no AMD people in the Cc, it
> >> would be good if you can find someone from AMD to look at this,
> >> also see my remarks to the 2nd patch in my reply to that patch.
> >
> > This was partially discussed with AMD folks and you are right that I
> > should include someone from AMD to take a look at this. Thanks for all
> > your comments!
> >
> >> Regards,
> >>
> >> Hans
> >>
> >>
> >>
> >>
> >>>
> >>> Jan Dabros (2):
> >>>   i2c: designware: Add missing locks
> >>>   i2c: designware: Add AMD PSP I2C bus support
> >>>
> >>>  MAINTAINERS                                  |   1 +
> >>>  drivers/acpi/acpi_apd.c                      |   1 +
> >>>  drivers/i2c/busses/Kconfig                   |  10 +
> >>>  drivers/i2c/busses/Makefile                  |   1 +
> >>>  drivers/i2c/busses/i2c-designware-amdpsp.c   | 357 +++++++++++++++++++
> >>>  drivers/i2c/busses/i2c-designware-baytrail.c |  10 +-
> >>>  drivers/i2c/busses/i2c-designware-common.c   |  12 +
> >>>  drivers/i2c/busses/i2c-designware-core.h     |  18 +-
> >>>  drivers/i2c/busses/i2c-designware-master.c   |   6 +
> >>>  drivers/i2c/busses/i2c-designware-platdrv.c  |  61 ++++
> >>>  10 files changed, 469 insertions(+), 8 deletions(-)
> >>>  create mode 100644 drivers/i2c/busses/i2c-designware-amdpsp.c
> >>>
> >>
> >
>

Re: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Hans de Goede]

Hi Jan,

On 1/21/22 10:59, Jan Dąbroś wrote:
> (...)
> 
>>> --- /dev/null
>>> +++ b/drivers/i2c/busses/i2c-designware-amdpsp.c
>>> @@ -0,0 +1,357 @@
>>> +// SPDX-License-Identifier: GPL-2.0
>>> +
>>> +#include <asm/msr.h>
>>> +#include <linux/i2c.h>
>>> +#include <linux/psp-sev.h>
>>> +
>>> +#include "i2c-designware-core.h"
>>
>> So all the stuff starting here:
>>
>>> +
>>> +#define MSR_AMD_PSP_ADDR     0xc00110a2
>>> +#define PSP_MBOX_OFFSET              0x10570
>>> +#define PSP_CMD_TIMEOUT_MS   500
>>> +
>>> +#define PSP_I2C_REQ_BUS_CMD          0x64
>>> +#define PSP_I2C_REQ_RETRY_CNT                10
>>> +#define PSP_I2C_REQ_RETRY_DELAY_MSEC 50
>>> +#define PSP_I2C_REQ_STS_OK           0x0
>>> +#define PSP_I2C_REQ_STS_BUS_BUSY     0x1
>>> +#define PSP_I2C_REQ_STS_INV_PARAM    0x3
>>> +
>>> +union psp_req_buffer_hdr {
>>> +     struct {
>>> +             u32 total_size;
>>> +             u32 status;
>>> +     } __packed;
>>> +     u64 hdr_val;
>>> +};
>>> +
>>> +enum psp_i2c_req_type {
>>> +     PSP_I2C_REQ_ACQUIRE,
>>> +     PSP_I2C_REQ_RELEASE,
>>> +     PSP_I2C_REQ_MAX,
>>> +};
>>> +
>>> +struct psp_i2c_req {
>>> +     union psp_req_buffer_hdr hdr;
>>> +     enum psp_i2c_req_type type;
>>> +} __packed __aligned(32);
>>> +
>>> +union psp_mbox_cmd_reg {
>>> +     struct psp_mbox_cmd_fields {
>>> +             u16 mbox_status;
>>> +             u8 mbox_cmd;
>>> +             u8 reserved:6;
>>> +             u8 recovery:1;
>>> +             u8 ready:1;
>>> +     } __packed fields;
>>> +     u32 val;
>>> +};
>>> +
>>> +struct psp_mbox {
>>> +     union psp_mbox_cmd_reg fields;
>>> +     uintptr_t i2c_req_addr;
>>> +} __packed;
>>> +
>>> +static DEFINE_MUTEX(psp_i2c_access_mutex);
>>> +static unsigned long psp_i2c_sem_acquired;
>>> +static void __iomem *mbox_iomem;
>>> +static u32 psp_i2c_access_count;
>>> +static bool psp_i2c_mbox_fail;
>>> +static struct device *psp_i2c_dev;
>>> +
>>> +static int psp_get_mbox_addr(unsigned long *mbox_addr)
>>> +{
>>> +     unsigned long long psp_mmio;
>>> +
>>> +     if (rdmsrl_safe(MSR_AMD_PSP_ADDR, &psp_mmio))
>>> +             return -EIO;
>>> +
>>> +     *mbox_addr = (unsigned long)(psp_mmio + PSP_MBOX_OFFSET);
>>> +
>>> +     return 0;
>>> +}
>>> +
>>> +static int psp_mbox_probe(void)
>>> +{
>>> +     unsigned long mbox_addr;
>>> +
>>> +     if (psp_get_mbox_addr(&mbox_addr))
>>> +             return -1;
>>> +
>>> +     mbox_iomem = ioremap(mbox_addr, sizeof(struct psp_mbox));
>>> +     if (!mbox_iomem)
>>> +             return -ENOMEM;
>>> +
>>> +     return 0;
>>> +}
>>> +
>>> +/* Recovery field should be equal 0 to start sending commands */
>>> +static int psp_check_mbox_recovery(struct psp_mbox *mbox)
>>> +{
>>> +     union psp_mbox_cmd_reg tmp = {0};
>>> +
>>> +     tmp.val = readl(&mbox->fields.val);
>>> +     return !!tmp.fields.recovery;
>>> +}
>>> +
>>> +static int psp_wait_cmd(struct psp_mbox *mbox)
>>> +{
>>> +     union psp_mbox_cmd_reg expected = { .val = 0 };
>>> +     u32 tmp;
>>> +
>>> +     /* Expect mbox_cmd to be cleared and ready bit to be set by PSP */
>>> +     expected.fields.ready = 1;
>>> +
>>> +     return readl_poll_timeout(&mbox->fields.val, tmp, (tmp == expected.val),
>>> +                               0, 1000 * PSP_CMD_TIMEOUT_MS);
>>> +}
>>> +
>>> +/* Status equal to 0 means that PSP succeed processing command */
>>> +static int psp_check_mbox_sts(struct psp_mbox *mbox)
>>> +{
>>> +     union psp_mbox_cmd_reg cmd_reg = {0};
>>> +
>>> +     cmd_reg.val = readl(&mbox->fields.val);
>>> +     return cmd_reg.fields.mbox_status;
>>> +}
>>> +
>>> +static int psp_send_cmd(struct psp_i2c_req *req)
>>> +{
>>> +     struct psp_mbox *mbox = (struct psp_mbox *)mbox_iomem;
>>> +     union psp_mbox_cmd_reg cmd_reg = {0};
>>> +
>>> +     if (psp_check_mbox_recovery(mbox))
>>> +             return -EIO;
>>> +
>>> +     if (psp_wait_cmd(mbox))
>>> +             return -EBUSY;
>>> +
>>> +     /* Fill address of command-response buffer */
>>> +     writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr);
>>> +
>>> +     /* Write command register to trigger processing */
>>> +     cmd_reg.fields.mbox_cmd = PSP_I2C_REQ_BUS_CMD;
>>> +     writel(cmd_reg.val, &mbox->fields.val);
>>> +
>>> +     if (psp_wait_cmd(mbox))
>>> +             return -ETIMEDOUT;
>>> +
>>> +     if (psp_check_mbox_sts(mbox))
>>> +             return -EIO;
>>> +
>>> +     return 0;
>>> +}
>>
>> Through here seems to all be generic code for accessing
>> the AMD PSP. To me this seems like something which belongs
>> in a separate AMD-PSP-mbox driver/lib, which can then be
>> shared between other kernel drivers which may also want
>> to access PSP.
> 
> I see your point clearly and actually it is not an accident that I've
> put all PSP-mailbox methods in one "block". They are logically
> different than the rest of i2c-adapter specific methods.
> 
> That being said, above PSP mailbox was created by AMD solely for the
> purpose of i2c_arbitration. It has its own set of commands and
> specific format of the command-response buffer. Thus it is not and it
> won't be generic in the future. There are already upstreamed drivers
> from AMD (under drivers/crypto/ccp/) which made use of PSP, however
> their channel of communication looks completely different than the
> very simple i2c_arbitration model implemented above.
> 
> Because of this I'm treating this as an i2c_semaphore-related code and
> putting this in this module. In my opinion moving this into some
> separate driver (which will be actually used only here) makes code
> less clear. But let's also hear some voice from AMD.

Since as you say this mailbox is special and only for i2c-arbitration,
keeping it inside this patch / .c file is fine.

> 
>>
>> Sorta like the generic iosf_mbi_read() and
>> iosf_mbi_write() functions from:
>>
>> arch/x86/platform/intel/iosf_mbi.c
>>
>> used on the Intel chips, which are also used outside of
>> the I2C bus-locking code.
>>
>> This is also one of the reasons why I think it would be
>> good to get some AMD folks involved in this, since they
>> may be aware of other drivers which also need to access
>> the PSP mbox.
>>
> 
> Right, I'm adding mario.limonciello@amd.com to the CC, so that he can comment.
> 
> (...)
> 
>>> +/*
>>> + * Locking methods are based on the default implementation from
>>> + * drivers/i2c/i2c-core.base.c, but with psp acquire and release operations
>>> + * added. With this in place we can ensure that i2c clients on the bus shared
>>> + * with psp are able to lock HW access to the bus for arbitrary number of
>>> + * operations - that is e.g. write-wait-read.
>>> + */
>>> +static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter,
>>> +                                     unsigned int flags)
>>> +{
>>> +     psp_acquire_i2c_bus();
>>> +     rt_mutex_lock_nested(&adapter->bus_lock, i2c_adapter_depth(adapter));
>>
>> This does not do what you think it does and you will still deadlock
>> when things nest because of someone taking the bus-lock and then
>> the main i2c-designware transfer function calling the acquire_lock
>> callback.
> 
> I haven't used rt_mutex_lock_nested() with the intent to prevent me
> from deadlock when i2c-designware calls acquire_lock with bus-lock
> already taken. This is a method copied from
> drivers/i2c/i2c-core-base.c (BTW, I have a typo in above comment).
> This is the default implementation applied by i2c-core when particular
> adapter doesn't register its own locking callbacks - thus it is called
> for i2c-designware for all platforms.
> 
> In case of this driver internal i2c-designware acquire_lock() is equal
> to psp_acquire_i2c_bus(). In other words, bus-level lock
> i2c_adapter_dw_psp_lock_bus() is a superset of internal adapter's
> acquire_lock().

Ah I missed that this is just mimicking the core functions +
an extra call to psp_acquire_i2c_bus().

I assumed that the dwc->acquire callback path was also taking
the mutex and I thought you had fallen for the _nested meaning
something different then it does, my bad.

> In order to prevent deadlock which you are talking about, I'm using
> reference lock counter inside psp_acquire_i2c_bus() thus it is safe to
> invoke acquire_lock() when bus-lock is already taken.

Ah good, that is pretty much is the same as what the Bay Trail code
is doing.

> 
>>
>> The _nested postfix is only for the lockdep lock-debugger, this
>> actually turns into a regular mutex_lock when lockdep is not enabled:
>>
>> #ifdef CONFIG_DEBUG_LOCK_ALLOC
>> extern void rt_mutex_lock_nested(struct rt_mutex *lock, unsigned int subclass);
>> #define rt_mutex_lock(lock) rt_mutex_lock_nested(lock, 0)
>> #else
>> extern void rt_mutex_lock(struct rt_mutex *lock);
>> #define rt_mutex_lock_nested(lock, subclass) rt_mutex_lock(lock)
>> #endif
>>
>> The _nested postfix as such is only to tell the lockdep code that
>> even though it seems we are trying to take the same mutex twice
>> since in both cases it is of i2c_adapter.rt_mutex "lock class"
>> that we are sure it is never the same i2c_adapter (but rather
>> one which always gets called in a nested fashion from another
>> i2c_adapter).
>>
>> IOW this only disables a false-positive lockdep warning, it does
>> not allow taking the same mutex twice, you will still hang on
>> the second mutex_lock call on the same lock.
> 
> Thanks for the technical background about rt_mutex_lock_nested. I
> think we should keep using it as is, since as I wrote above I don't
> have any reasoning to modify it here.

Ack, now that my misreading of the code has been cleared up
I agree.

>> Also I don't think you are allowed to use the bus_locking code
>> like this. The i2c bus-locking code is intended to deal with
>> busses which have muxes in them, where the mux must be set
>> to the right branch of the bus to reach the client and then
>> not be changed during the transfer to that client.
>>
>> So i2c-client drivers are never supposed to directly call
>> the bus-locking functions.
> 
> I think you are not correct here. There are examples of i2c-clients
> which are using i2c bus_locking for the purpose of locking adapter for
> the bunch of i2c transactions.
> 
> As an example let's take drivers/char/tpm/tpm_tis_i2c_cr50.c. It
> operates in write-wait-read model and there is i2c_lock_bus() call
> used to ensure that bus won't be released -
> https://github.com/torvalds/linux/blob/master/drivers/char/tpm/tpm_tis_i2c_cr50.c#L202.
> 
> Similar model is followed in drivers/char/tpm/tpm_i2c_infineon.c and
> couple of other i2c-client drivers.

Ah I see, interesting (live and learn).

But this is then combined with using the special __i2c_transfer()
function for the actual i2c reads/writes, since using the regular
i2c_transfer() function after already taking the lock would deadlock.

There is a similar unlocked raw __i2c_smbus_xfer(), but as the
comment in include/linux/i2c.h above the locked i2c_smbus_xfer() says:

/* This is the very generalized SMBus access routine. You probably do not
   want to use this, though; one of the functions below may be much easier,
   and probably just as fast.
   Note that we use i2c_adapter here, because you do not need a specific
   smbus adapter to call this function. */
s32 i2c_smbus_xfer(...);

So in this case a driver cannot use the usual
i2c_smbus_read_byte/word/byte_data/word_data() helpers and
the same for writes. Also using an i2c_regmap (which is used
in a ton of places like PMIC drivers) will not work this way.

So yes you can use i2c_bus_lock() for this; but only if all the
drivers where you want to do that limit themselves to
__i2c_transfer() and __i2c_smbus_xfer() calls and/or are
rewritten to only use those.
>> This is why in the Bay Trail case we have i2c-drivers
>> directly calling iosf_mbi_block_punit_i2c_access() and
>> iosf_mbi_unblock_punit_i2c_access() to lock the bus
>> for multiple i2c-transfers. We can get away with this there
>> because the bus in question is only used to access the
>> PMIC and that PMIC is only used on Bay Trail (and CHT)
>> boards, so the PMIC drivers can just hard-code these
>> calls.
>>
>> If you need to take the PSP I2C semaphore for multiple
>> transfers in some generic drivers, then I guess that the
>> i2c-subsys will need to get some new i2c_adapter callbacks
>> to acquire / release the bus for i2c-controllers where
>> the bus/controller is shared with some co-processor like
>> in the PSP case.
> 
> This is exactly my intention to support generic i2c-clients drivers
> without them being aware that i2c-adapter above is using some
> semaphore/arbitration. Hopefully you can agree with me that currently
> available bus_locking can be used and is enough for this purpose.

It can be used, but with limitations, see above.

> 
>> Also note that iosf_mbi_block_punit_i2c_access() and
>> iosf_mbi_unblock_punit_i2c_access() do their own
>> ref/lock-counting to allow calling them multiple times and
>> the first block call takes the bus and the last unblock
>> call releases it.
> 
> This is exactly what I was talking about above and also implemented
> within psp_acquire_i2c_bus() and psp_release_i2c_bus().

Right, I was to quick in skimming over your code when
I wrote down my concerns about there being a deadlock
there, sorry.

Regards,

Hans

RE: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Limonciello, Mario]

[Public]

+Thomas (ccp driver)
+Alex (amdgpu driver)

> 
> On 1/21/22 10:59, Jan Dąbroś wrote:
> > (...)
> >
> >>> --- /dev/null
> >>> +++ b/drivers/i2c/busses/i2c-designware-amdpsp.c
> >>> @@ -0,0 +1,357 @@
> >>> +// SPDX-License-Identifier: GPL-2.0
> >>> +
> >>> +#include <asm/msr.h>
> >>> +#include <linux/i2c.h>
> >>> +#include <linux/psp-sev.h>
> >>> +
> >>> +#include "i2c-designware-core.h"
> >>
> >> So all the stuff starting here:
> >>
> >>> +
> >>> +#define MSR_AMD_PSP_ADDR     0xc00110a2
> >>> +#define PSP_MBOX_OFFSET              0x10570
> >>> +#define PSP_CMD_TIMEOUT_MS   500
> >>> +
> >>> +#define PSP_I2C_REQ_BUS_CMD          0x64
> >>> +#define PSP_I2C_REQ_RETRY_CNT                10
> >>> +#define PSP_I2C_REQ_RETRY_DELAY_MSEC 50
> >>> +#define PSP_I2C_REQ_STS_OK           0x0
> >>> +#define PSP_I2C_REQ_STS_BUS_BUSY     0x1
> >>> +#define PSP_I2C_REQ_STS_INV_PARAM    0x3
> >>> +
> >>> +union psp_req_buffer_hdr {
> >>> +     struct {
> >>> +             u32 total_size;
> >>> +             u32 status;
> >>> +     } __packed;
> >>> +     u64 hdr_val;
> >>> +};
> >>> +
> >>> +enum psp_i2c_req_type {
> >>> +     PSP_I2C_REQ_ACQUIRE,
> >>> +     PSP_I2C_REQ_RELEASE,
> >>> +     PSP_I2C_REQ_MAX,
> >>> +};
> >>> +
> >>> +struct psp_i2c_req {
> >>> +     union psp_req_buffer_hdr hdr;
> >>> +     enum psp_i2c_req_type type;
> >>> +} __packed __aligned(32);
> >>> +
> >>> +union psp_mbox_cmd_reg {
> >>> +     struct psp_mbox_cmd_fields {
> >>> +             u16 mbox_status;
> >>> +             u8 mbox_cmd;
> >>> +             u8 reserved:6;
> >>> +             u8 recovery:1;
> >>> +             u8 ready:1;
> >>> +     } __packed fields;
> >>> +     u32 val;
> >>> +};
> >>> +
> >>> +struct psp_mbox {
> >>> +     union psp_mbox_cmd_reg fields;
> >>> +     uintptr_t i2c_req_addr;
> >>> +} __packed;
> >>> +
> >>> +static DEFINE_MUTEX(psp_i2c_access_mutex);
> >>> +static unsigned long psp_i2c_sem_acquired;
> >>> +static void __iomem *mbox_iomem;
> >>> +static u32 psp_i2c_access_count;
> >>> +static bool psp_i2c_mbox_fail;
> >>> +static struct device *psp_i2c_dev;
> >>> +
> >>> +static int psp_get_mbox_addr(unsigned long *mbox_addr)
> >>> +{
> >>> +     unsigned long long psp_mmio;
> >>> +
> >>> +     if (rdmsrl_safe(MSR_AMD_PSP_ADDR, &psp_mmio))
> >>> +             return -EIO;
> >>> +
> >>> +     *mbox_addr = (unsigned long)(psp_mmio + PSP_MBOX_OFFSET);
> >>> +
> >>> +     return 0;
> >>> +}
> >>> +
> >>> +static int psp_mbox_probe(void)
> >>> +{
> >>> +     unsigned long mbox_addr;
> >>> +
> >>> +     if (psp_get_mbox_addr(&mbox_addr))
> >>> +             return -1;
> >>> +
> >>> +     mbox_iomem = ioremap(mbox_addr, sizeof(struct psp_mbox));
> >>> +     if (!mbox_iomem)
> >>> +             return -ENOMEM;
> >>> +
> >>> +     return 0;
> >>> +}
> >>> +
> >>> +/* Recovery field should be equal 0 to start sending commands */
> >>> +static int psp_check_mbox_recovery(struct psp_mbox *mbox)
> >>> +{
> >>> +     union psp_mbox_cmd_reg tmp = {0};
> >>> +
> >>> +     tmp.val = readl(&mbox->fields.val);
> >>> +     return !!tmp.fields.recovery;
> >>> +}
> >>> +
> >>> +static int psp_wait_cmd(struct psp_mbox *mbox)
> >>> +{
> >>> +     union psp_mbox_cmd_reg expected = { .val = 0 };
> >>> +     u32 tmp;
> >>> +
> >>> +     /* Expect mbox_cmd to be cleared and ready bit to be set by PSP */
> >>> +     expected.fields.ready = 1;
> >>> +
> >>> +     return readl_poll_timeout(&mbox->fields.val, tmp, (tmp ==
> expected.val),
> >>> +                               0, 1000 * PSP_CMD_TIMEOUT_MS);
> >>> +}
> >>> +
> >>> +/* Status equal to 0 means that PSP succeed processing command */
> >>> +static int psp_check_mbox_sts(struct psp_mbox *mbox)
> >>> +{
> >>> +     union psp_mbox_cmd_reg cmd_reg = {0};
> >>> +
> >>> +     cmd_reg.val = readl(&mbox->fields.val);
> >>> +     return cmd_reg.fields.mbox_status;
> >>> +}
> >>> +
> >>> +static int psp_send_cmd(struct psp_i2c_req *req)
> >>> +{
> >>> +     struct psp_mbox *mbox = (struct psp_mbox *)mbox_iomem;
> >>> +     union psp_mbox_cmd_reg cmd_reg = {0};
> >>> +
> >>> +     if (psp_check_mbox_recovery(mbox))
> >>> +             return -EIO;
> >>> +
> >>> +     if (psp_wait_cmd(mbox))
> >>> +             return -EBUSY;
> >>> +
> >>> +     /* Fill address of command-response buffer */
> >>> +     writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr);
> >>> +
> >>> +     /* Write command register to trigger processing */
> >>> +     cmd_reg.fields.mbox_cmd = PSP_I2C_REQ_BUS_CMD;
> >>> +     writel(cmd_reg.val, &mbox->fields.val);
> >>> +
> >>> +     if (psp_wait_cmd(mbox))
> >>> +             return -ETIMEDOUT;
> >>> +
> >>> +     if (psp_check_mbox_sts(mbox))
> >>> +             return -EIO;
> >>> +
> >>> +     return 0;
> >>> +}
> >>
> >> Through here seems to all be generic code for accessing
> >> the AMD PSP. To me this seems like something which belongs
> >> in a separate AMD-PSP-mbox driver/lib, which can then be
> >> shared between other kernel drivers which may also want
> >> to access PSP.
> >
> > I see your point clearly and actually it is not an accident that I've
> > put all PSP-mailbox methods in one "block". They are logically
> > different than the rest of i2c-adapter specific methods.
> >
> > That being said, above PSP mailbox was created by AMD solely for the
> > purpose of i2c_arbitration. It has its own set of commands and
> > specific format of the command-response buffer. Thus it is not and it
> > won't be generic in the future. There are already upstreamed drivers
> > from AMD (under drivers/crypto/ccp/) which made use of PSP, however
> > their channel of communication looks completely different than the
> > very simple i2c_arbitration model implemented above.
> >

Can you please double confirm no other users for this mailbox and it's for
you only?  And if so is it only specific to this platform that no other users? 
At least on some UEFI AMD platforms that mailbox is defined for
communication with something else.  We might need some way to attest
from the firmware that it's safe to use.

The only mailbox that is defined for OS use across different silicon AFAIK is
the GPU driver mailbox.  It may be safer to use that, but I'm not sure if
GPU driver has come up early enough for your use.

> > Because of this I'm treating this as an i2c_semaphore-related code and
> > putting this in this module. In my opinion moving this into some
> > separate driver (which will be actually used only here) makes code
> > less clear. But let's also hear some voice from AMD.
> 
> Since as you say this mailbox is special and only for i2c-arbitration,
> keeping it inside this patch / .c file is fine.
> 
> >
> >>
> >> Sorta like the generic iosf_mbi_read() and
> >> iosf_mbi_write() functions from:
> >>
> >> arch/x86/platform/intel/iosf_mbi.c
> >>
> >> used on the Intel chips, which are also used outside of
> >> the I2C bus-locking code.
> >>
> >> This is also one of the reasons why I think it would be
> >> good to get some AMD folks involved in this, since they
> >> may be aware of other drivers which also need to access
> >> the PSP mbox.
> >>
> >
> > Right, I'm adding mario.limonciello@amd.com to the CC, so that he can
> comment.
> >
> > (...)
> >
> >>> +/*
> >>> + * Locking methods are based on the default implementation from
> >>> + * drivers/i2c/i2c-core.base.c, but with psp acquire and release operations
> >>> + * added. With this in place we can ensure that i2c clients on the bus shared
> >>> + * with psp are able to lock HW access to the bus for arbitrary number of
> >>> + * operations - that is e.g. write-wait-read.
> >>> + */
> >>> +static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter,
> >>> +                                     unsigned int flags)
> >>> +{
> >>> +     psp_acquire_i2c_bus();
> >>> +     rt_mutex_lock_nested(&adapter->bus_lock,
> i2c_adapter_depth(adapter));
> >>
> >> This does not do what you think it does and you will still deadlock
> >> when things nest because of someone taking the bus-lock and then
> >> the main i2c-designware transfer function calling the acquire_lock
> >> callback.
> >
> > I haven't used rt_mutex_lock_nested() with the intent to prevent me
> > from deadlock when i2c-designware calls acquire_lock with bus-lock
> > already taken. This is a method copied from
> > drivers/i2c/i2c-core-base.c (BTW, I have a typo in above comment).
> > This is the default implementation applied by i2c-core when particular
> > adapter doesn't register its own locking callbacks - thus it is called
> > for i2c-designware for all platforms.
> >
> > In case of this driver internal i2c-designware acquire_lock() is equal
> > to psp_acquire_i2c_bus(). In other words, bus-level lock
> > i2c_adapter_dw_psp_lock_bus() is a superset of internal adapter's
> > acquire_lock().
> 
> Ah I missed that this is just mimicking the core functions +
> an extra call to psp_acquire_i2c_bus().
> 
> I assumed that the dwc->acquire callback path was also taking
> the mutex and I thought you had fallen for the _nested meaning
> something different then it does, my bad.
> 
> > In order to prevent deadlock which you are talking about, I'm using
> > reference lock counter inside psp_acquire_i2c_bus() thus it is safe to
> > invoke acquire_lock() when bus-lock is already taken.
> 
> Ah good, that is pretty much is the same as what the Bay Trail code
> is doing.
> 
> >
> >>
> >> The _nested postfix is only for the lockdep lock-debugger, this
> >> actually turns into a regular mutex_lock when lockdep is not enabled:
> >>
> >> #ifdef CONFIG_DEBUG_LOCK_ALLOC
> >> extern void rt_mutex_lock_nested(struct rt_mutex *lock, unsigned int
> subclass);
> >> #define rt_mutex_lock(lock) rt_mutex_lock_nested(lock, 0)
> >> #else
> >> extern void rt_mutex_lock(struct rt_mutex *lock);
> >> #define rt_mutex_lock_nested(lock, subclass) rt_mutex_lock(lock)
> >> #endif
> >>
> >> The _nested postfix as such is only to tell the lockdep code that
> >> even though it seems we are trying to take the same mutex twice
> >> since in both cases it is of i2c_adapter.rt_mutex "lock class"
> >> that we are sure it is never the same i2c_adapter (but rather
> >> one which always gets called in a nested fashion from another
> >> i2c_adapter).
> >>
> >> IOW this only disables a false-positive lockdep warning, it does
> >> not allow taking the same mutex twice, you will still hang on
> >> the second mutex_lock call on the same lock.
> >
> > Thanks for the technical background about rt_mutex_lock_nested. I
> > think we should keep using it as is, since as I wrote above I don't
> > have any reasoning to modify it here.
> 
> Ack, now that my misreading of the code has been cleared up
> I agree.
> 
> >> Also I don't think you are allowed to use the bus_locking code
> >> like this. The i2c bus-locking code is intended to deal with
> >> busses which have muxes in them, where the mux must be set
> >> to the right branch of the bus to reach the client and then
> >> not be changed during the transfer to that client.
> >>
> >> So i2c-client drivers are never supposed to directly call
> >> the bus-locking functions.
> >
> > I think you are not correct here. There are examples of i2c-clients
> > which are using i2c bus_locking for the purpose of locking adapter for
> > the bunch of i2c transactions.
> >
> > As an example let's take drivers/char/tpm/tpm_tis_i2c_cr50.c. It
> > operates in write-wait-read model and there is i2c_lock_bus() call
> > used to ensure that bus won't be released -
> >
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c
> om%2Ftorvalds%2Flinux%2Fblob%2Fmaster%2Fdrivers%2Fchar%2Ftpm%2Ftpm_
> tis_i2c_cr50.c%23L202&amp;data=04%7C01%7Cmario.limonciello%40amd.com
> %7C1bdc742bc2a24f59b7d908d9dcc95438%7C3dd8961fe4884e608e11a82d994
> e183d%7C0%7C0%7C637783579554955840%7CUnknown%7CTWFpbGZsb3d8ey
> JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
> 3000&amp;sdata=fr0UEka%2BxYyPxqUG6oOZo%2Bc6pWgto2mD7hWA20YulVQ
> %3D&amp;reserved=0.
> >
> > Similar model is followed in drivers/char/tpm/tpm_i2c_infineon.c and
> > couple of other i2c-client drivers.
> 
> Ah I see, interesting (live and learn).
> 
> But this is then combined with using the special __i2c_transfer()
> function for the actual i2c reads/writes, since using the regular
> i2c_transfer() function after already taking the lock would deadlock.
> 
> There is a similar unlocked raw __i2c_smbus_xfer(), but as the
> comment in include/linux/i2c.h above the locked i2c_smbus_xfer() says:
> 
> /* This is the very generalized SMBus access routine. You probably do not
>    want to use this, though; one of the functions below may be much easier,
>    and probably just as fast.
>    Note that we use i2c_adapter here, because you do not need a specific
>    smbus adapter to call this function. */
> s32 i2c_smbus_xfer(...);
> 
> So in this case a driver cannot use the usual
> i2c_smbus_read_byte/word/byte_data/word_data() helpers and
> the same for writes. Also using an i2c_regmap (which is used
> in a ton of places like PMIC drivers) will not work this way.
> 
> So yes you can use i2c_bus_lock() for this; but only if all the
> drivers where you want to do that limit themselves to
> __i2c_transfer() and __i2c_smbus_xfer() calls and/or are
> rewritten to only use those.
> >> This is why in the Bay Trail case we have i2c-drivers
> >> directly calling iosf_mbi_block_punit_i2c_access() and
> >> iosf_mbi_unblock_punit_i2c_access() to lock the bus
> >> for multiple i2c-transfers. We can get away with this there
> >> because the bus in question is only used to access the
> >> PMIC and that PMIC is only used on Bay Trail (and CHT)
> >> boards, so the PMIC drivers can just hard-code these
> >> calls.
> >>
> >> If you need to take the PSP I2C semaphore for multiple
> >> transfers in some generic drivers, then I guess that the
> >> i2c-subsys will need to get some new i2c_adapter callbacks
> >> to acquire / release the bus for i2c-controllers where
> >> the bus/controller is shared with some co-processor like
> >> in the PSP case.
> >
> > This is exactly my intention to support generic i2c-clients drivers
> > without them being aware that i2c-adapter above is using some
> > semaphore/arbitration. Hopefully you can agree with me that currently
> > available bus_locking can be used and is enough for this purpose.
> 
> It can be used, but with limitations, see above.
> 
> >
> >> Also note that iosf_mbi_block_punit_i2c_access() and
> >> iosf_mbi_unblock_punit_i2c_access() do their own
> >> ref/lock-counting to allow calling them multiple times and
> >> the first block call takes the bus and the last unblock
> >> call releases it.
> >
> > This is exactly what I was talking about above and also implemented
> > within psp_acquire_i2c_bus() and psp_release_i2c_bus().
> 
> Right, I was to quick in skimming over your code when
> I wrote down my concerns about there being a deadlock
> there, sorry.
> 
> Regards,
> 
> Hans

Re: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Andy Shevchenko]

On Fri, Jan 21, 2022 at 11:20:19AM +0100, Jan Dąbroś wrote:
> czw., 20 sty 2022 o 15:21 Andy Shevchenko
> <andriy.shevchenko@linux.intel.com> napisał(a):
> > On Thu, Jan 20, 2022 at 01:16:21AM +0100, Jan Dabros wrote:

...

> > > Add new entry in MAINTAINERS file to cover new module.
> >
> > It's confusing. You added yourself as a reviewer for I2C DesignWare driver,
> > which is great, but not described in the commit message.
> 
> Should I rephrase this sentence (to be more specific that I may be
> helpful for reviewing amdpsp.c module) or rather you mean to exclude
> drivers/i2c/busses/i2c-designware-amdpsp.c from the rest of
> designware-* modules and create separate entry?
> 
> Actually initially I wasn't planning to modify MAINTAINERS (after all
> I'm not an I2C DesignWare expert) until run checkpatch.pl which
> recommended to do so when adding new file. Eventually for me it made
> some sense since I have a platform equipped with AMD Cezanne SoC and I
> will be able to review and test potential changes in
> i2c-designware-amdpsp.c or in general around semaphore areas.
> 
> This may also work with different model, similar to how you pointed me
> to Hans as an owner of Bay Trail platform who is acquinted with how
> its i2c semaphore is working. I will simply remove myself from the
> MAINTAINERS file and you can add me to the threads if there is
> anything requiring my help.
> 
> Let me know which way is working for you. I just thought it is not
> good to leave you alone with a new module which you cannot actually
> test and don't have deep insight about how PSP-x86 communications
> works.

You have a few options:
- leave it for us (but it probably won't go well in long-term as you noticed)
- add yourself as a reviewer (it doesn't require to review everything, but
  you will get all i2c DesignWare driver changes)
- add a new MAINTAINERS database entry where you can put yourself for that
  file even as a maintainaer

...

> > > +#include <asm/msr.h>
> >
> > Usually linux/* followed by asm/*.
> >
> > > +#include <linux/i2c.h>
> > > +#include <linux/psp-sev.h>
> >
> > types.h?
> 
> I need to take a deeper look at the headers included here, especially
> considering errors pointed by kernel test robot. Not sure why I
> haven't seen any issues on my setup.

The problem here is not so visible. Headers should be a compromise between
what is really used and what we may include for that. There are headers
that guaranteed to be included by others, otherwise it will be an implicit
dependency which is not good in cases of generic headers, such as types.h.

...

> > > +union psp_req_buffer_hdr {
> > > +     struct {
> > > +             u32 total_size;
> > > +             u32 status;
> > > +     } __packed;
> >
> > What does packet bring you here?
> 
> In this particular case binary-wise nothing, I can as well drop this.
> It may be necessary if there are some changes in this structs fields
> in future (e.g changing total_size to u8), since PSP expects every
> field to be byte-aligned.

_packed will make another thing which you probably won't need, it brings
the entire structure to be possible on unaligned addresses and then some
warnings from compiler may be issued even if there is no problem in the
flow.

Read this discussion: https://lore.kernel.org/linux-media/20220110224656.266536-1-sakari.ailus@linux.intel.com/

> > > +     u64 hdr_val;
> >
> > And why does this not have the same alignment since it's also part of
> > the union?
> 
> __packed is not about alignment of the whole struct/union

It's. See above.

> but about
> lack of padding between its fields. As above - in this particular case
> with two u32 it doesn't matter.

...

> > > +struct psp_i2c_req {
> > > +     union psp_req_buffer_hdr hdr;
> > > +     enum psp_i2c_req_type type;
> >
> > > +} __packed __aligned(32);
> >
> > Can you explain, what this means and how it's supposed to work?
> 
> This means that each instance of the struct should be aligned (32)
> while at the same time no padding within members - thus this may
> result in non-aligned addresses of members.

32 bytes? And on unaligned address at the same time.

...

> > > +union psp_mbox_cmd_reg {
> > > +     struct psp_mbox_cmd_fields {
> > > +             u16 mbox_status;
> > > +             u8 mbox_cmd;
> > > +             u8 reserved:6;
> > > +             u8 recovery:1;
> > > +             u8 ready:1;
> >
> > > +     } __packed fields;
> >
> > So, what is the __packed purpose here?
> 
> As in all above cases - considering current layout of members and
> their sizes dropping `__packed` will not results in any errors.
> 
> However PSP expects all members os structs within shared buffers to be
> byte-aligned, that's why I've added this attributes to be on the safe
> side. If you think this doesn't make sense, I can remove them - in
> (very unlikely) case of changes, one will need to add this specifier.

I guess you don't need them at all in this case in any of the data structure
you created here.

...

> > > +     struct psp_mbox *mbox = (struct psp_mbox *)mbox_iomem;
> >
> > Heck, no!
> 
> I need to get acquinted to the kernel-reviewer language:) Pardon my
> ignorance, but just to be sure I get what you mean here:
> I'm using global mbox_iomem to keep address of PSP mailbox in IO
> memory. Casting this to struct psp_mbox layout here, to make access
> more convenient.
> Your point here is that:
> 1. I should move the assignment out from the variable declaration part
> of this function;
> 2. I should use ioremap/iounmap each time in psp_send_cmd instead of
> using it once in `probe` and unmap in `remove`?
> I thought about this option as to be less effective performance-wise
> (even though I can get rid of one global variable).
> 3. Something else?

Casting an __iomem pointer to the some struct without keeping it. I believe
sparse should blow because of this.

...

> > > +     /* Fill address of command-response buffer */
> > > +     writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr);
> >
> > What does this voodoo mean?!
> 
> Basically I need to take physical address (__psp_pa) of request buffer
> req and write this down into mailbox IO memory.
> This should be spread into more lines with some comments, is this your point?

It needs much better comment explaining what is this address and its meaning
for the hardware and why you need physical address here (DMA?). For me it looks
like a voodoo. Ah, and not using phys_addr_t / dma_addr_t / etc type here, but
uintptr_t just adds a confusion.

...

> > > +     start = jiffies;
> > > +     do {
> > > +             if (psp_send_cmd(req)) {
> > > +                     ret = -EIO;
> > > +                     goto cleanup;
> > > +             }
> > > +
> > > +             status = check_i2c_req_sts(req);
> > > +             if (!status) {
> > > +                     dev_dbg(psp_i2c_dev, "Request accepted by PSP after %ums\n",
> > > +                             jiffies_to_msecs(jiffies - start));
> > > +                     ret = 0;
> > > +                     goto cleanup;
> > > +             } else if (status == -EBUSY) {
> > > +                     retry_cnt--;
> > > +             } else {
> > > +                     ret = -EIO;
> > > +                     goto cleanup;
> > > +             };
> > > +
> > > +             /* IF EBUSY, give PSP time to finish its i2c activities */
> > > +             mdelay(PSP_I2C_REQ_RETRY_DELAY_MSEC);
> > > +     } while (retry_cnt);
> >
> > NIH iopoll.h API(s).
> 
> I don't think macros avaialble in iopoll.h are suitable here.
> Procedure above is not about simply reading some IO and waiting for
> particular condition to be met with this particular value. Eventually
> `psp_send_cmd()` invokes `psp_wait_cmd()` where I'm using
> `readl_poll_timeout()`, so on lower level I'm making use of this API.
> However here I don't see any obvious method how to incorporate
> iopoll.h API to reach my goal.

You do not go to clean up if and only if status == -EBUSY, so here we have
a condition. the rest can be moved to a function that you wrap by
read_poll_timeout_atomic() (pay attention to the macro name).
Here is the question, btw, why _atomic()? I.o.w. why mdelay() and not msleep()?

> > > +     ret = -ETIMEDOUT;

...

> > Handle errors first.

> Addressing above two comments - what do you think about below:
> if (status) {
>       if (status == -ETIMEDOUT)
>                dev_err(psp_i2c_dev, "Timed out waiting for PSP to
> release I2C bus\n");
>       else
>                dev_err(psp_i2c_dev, "PSP communication error\n");

>        dev_err(psp_i2c_dev, "PSP communication error\n");

This dup message is not needed, otherwise fine to me.

>        psp_i2c_mbox_fail = true;
>        goto cleanup;
> }
> 
> psp_i2c_sem_acquired = jiffies;
> psp_i2c_access_count++;
> (...)

...

> > > +static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
> > > +{
> > > +     int ret;
> > > +     int i;
> > > +
> > > +     dev->semaphore_idx = -1;
> > > +
> > > +     for (i = 0; i < ARRAY_SIZE(i2c_dw_semaphore_cb_table); i++) {
> >
> > > +             if (!i2c_dw_semaphore_cb_table[i].probe)
> > > +                     continue;
> >
> > Huh?
> 
> Just to be sure I get your point.
> Once I found terminating entry, I will get out of the loop and return
> 0 as there are no semaphores to be "applied". Actually I should
> probably use `break;` here as there shouldn't be a case when certain
> type of semaphore installs without `probe` being implemented.

Yes, that's what I though, and rather using ARRAY_SIZE(), use a terminator you
provided.

Originally you have used two approaches which seems competing with each other:
- ARRAY_SIZE
- terminator entry

And on top of that so confusing continue on top of not-found ->probe() that
makes me think what the meaning of the entry that has set ->remove(), but no
->probe().

That said, I would rather see something like

	struct ... *p = &...;

	while (p->probe) {
		...
		p++;
	}

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Tom Lendacky]

On 1/21/22 11:38 AM, Limonciello, Mario wrote:
> [Public]
> 
> +Thomas (ccp driver)
> +Alex (amdgpu driver)
> 
>>
>> On 1/21/22 10:59, Jan Dąbroś wrote:

>>>>
>>>> Through here seems to all be generic code for accessing
>>>> the AMD PSP. To me this seems like something which belongs
>>>> in a separate AMD-PSP-mbox driver/lib, which can then be
>>>> shared between other kernel drivers which may also want
>>>> to access PSP.
>>>
>>> I see your point clearly and actually it is not an accident that I've
>>> put all PSP-mailbox methods in one "block". They are logically
>>> different than the rest of i2c-adapter specific methods.
>>>
>>> That being said, above PSP mailbox was created by AMD solely for the
>>> purpose of i2c_arbitration. It has its own set of commands and
>>> specific format of the command-response buffer. Thus it is not and it
>>> won't be generic in the future. There are already upstreamed drivers
>>> from AMD (under drivers/crypto/ccp/) which made use of PSP, however
>>> their channel of communication looks completely different than the
>>> very simple i2c_arbitration model implemented above.
>>>
> 
> Can you please double confirm no other users for this mailbox and it's for
> you only?  And if so is it only specific to this platform that no other users?
> At least on some UEFI AMD platforms that mailbox is defined for
> communication with something else.  We might need some way to attest
> from the firmware that it's safe to use.
> 
> The only mailbox that is defined for OS use across different silicon AFAIK is
> the GPU driver mailbox.  It may be safer to use that, but I'm not sure if
> GPU driver has come up early enough for your use.

The CCP/PSP driver will load as a PCIe device driver on this platform and 
will ioremap the MMIO space. Today, that driver doesn't reference those 
mailbox registers, and I don't know that there will be a need in the 
future. But if there is a need in the future, you could end up with a 
conflict between these two drivers.

Thanks,
Tom

> 
>>> Because of this I'm treating this as an i2c_semaphore-related code and
>>> putting this in this module. In my opinion moving this into some
>>> separate driver (which will be actually used only here) makes code
>>> less clear. But let's also hear some voice from AMD.
>>
>> Since as you say this mailbox is special and only for i2c-arbitration,
>> keeping it inside this patch / .c file is fine.
>>
>>>
>>>>
>>>> Sorta like the generic iosf_mbi_read() and
>>>> iosf_mbi_write() functions from:
>>>>
>>>> arch/x86/platform/intel/iosf_mbi.c
>>>>
>>>> used on the Intel chips, which are also used outside of
>>>> the I2C bus-locking code.
>>>>
>>>> This is also one of the reasons why I think it would be
>>>> good to get some AMD folks involved in this, since they
>>>> may be aware of other drivers which also need to access
>>>> the PSP mbox.
>>>>
>>>
>>> Right, I'm adding mario.limonciello@amd.com to the CC, so that he can
>> comment.
>>>
>>> (...)
>>>
>>>>> +/*
>>>>> + * Locking methods are based on the default implementation from
>>>>> + * drivers/i2c/i2c-core.base.c, but with psp acquire and release operations
>>>>> + * added. With this in place we can ensure that i2c clients on the bus shared
>>>>> + * with psp are able to lock HW access to the bus for arbitrary number of
>>>>> + * operations - that is e.g. write-wait-read.
>>>>> + */
>>>>> +static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter,
>>>>> +                                     unsigned int flags)
>>>>> +{
>>>>> +     psp_acquire_i2c_bus();
>>>>> +     rt_mutex_lock_nested(&adapter->bus_lock,
>> i2c_adapter_depth(adapter));
>>>>
>>>> This does not do what you think it does and you will still deadlock
>>>> when things nest because of someone taking the bus-lock and then
>>>> the main i2c-designware transfer function calling the acquire_lock
>>>> callback.
>>>
>>> I haven't used rt_mutex_lock_nested() with the intent to prevent me
>>> from deadlock when i2c-designware calls acquire_lock with bus-lock
>>> already taken. This is a method copied from
>>> drivers/i2c/i2c-core-base.c (BTW, I have a typo in above comment).
>>> This is the default implementation applied by i2c-core when particular
>>> adapter doesn't register its own locking callbacks - thus it is called
>>> for i2c-designware for all platforms.
>>>
>>> In case of this driver internal i2c-designware acquire_lock() is equal
>>> to psp_acquire_i2c_bus(). In other words, bus-level lock
>>> i2c_adapter_dw_psp_lock_bus() is a superset of internal adapter's
>>> acquire_lock().
>>
>> Ah I missed that this is just mimicking the core functions +
>> an extra call to psp_acquire_i2c_bus().
>>
>> I assumed that the dwc->acquire callback path was also taking
>> the mutex and I thought you had fallen for the _nested meaning
>> something different then it does, my bad.
>>
>>> In order to prevent deadlock which you are talking about, I'm using
>>> reference lock counter inside psp_acquire_i2c_bus() thus it is safe to
>>> invoke acquire_lock() when bus-lock is already taken.
>>
>> Ah good, that is pretty much is the same as what the Bay Trail code
>> is doing.
>>
>>>
>>>>
>>>> The _nested postfix is only for the lockdep lock-debugger, this
>>>> actually turns into a regular mutex_lock when lockdep is not enabled:
>>>>
>>>> #ifdef CONFIG_DEBUG_LOCK_ALLOC
>>>> extern void rt_mutex_lock_nested(struct rt_mutex *lock, unsigned int
>> subclass);
>>>> #define rt_mutex_lock(lock) rt_mutex_lock_nested(lock, 0)
>>>> #else
>>>> extern void rt_mutex_lock(struct rt_mutex *lock);
>>>> #define rt_mutex_lock_nested(lock, subclass) rt_mutex_lock(lock)
>>>> #endif
>>>>
>>>> The _nested postfix as such is only to tell the lockdep code that
>>>> even though it seems we are trying to take the same mutex twice
>>>> since in both cases it is of i2c_adapter.rt_mutex "lock class"
>>>> that we are sure it is never the same i2c_adapter (but rather
>>>> one which always gets called in a nested fashion from another
>>>> i2c_adapter).
>>>>
>>>> IOW this only disables a false-positive lockdep warning, it does
>>>> not allow taking the same mutex twice, you will still hang on
>>>> the second mutex_lock call on the same lock.
>>>
>>> Thanks for the technical background about rt_mutex_lock_nested. I
>>> think we should keep using it as is, since as I wrote above I don't
>>> have any reasoning to modify it here.
>>
>> Ack, now that my misreading of the code has been cleared up
>> I agree.
>>
>>>> Also I don't think you are allowed to use the bus_locking code
>>>> like this. The i2c bus-locking code is intended to deal with
>>>> busses which have muxes in them, where the mux must be set
>>>> to the right branch of the bus to reach the client and then
>>>> not be changed during the transfer to that client.
>>>>
>>>> So i2c-client drivers are never supposed to directly call
>>>> the bus-locking functions.
>>>
>>> I think you are not correct here. There are examples of i2c-clients
>>> which are using i2c bus_locking for the purpose of locking adapter for
>>> the bunch of i2c transactions.
>>>
>>> As an example let's take drivers/char/tpm/tpm_tis_i2c_cr50.c. It
>>> operates in write-wait-read model and there is i2c_lock_bus() call
>>> used to ensure that bus won't be released -
>>>
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c
>> om%2Ftorvalds%2Flinux%2Fblob%2Fmaster%2Fdrivers%2Fchar%2Ftpm%2Ftpm_
>> tis_i2c_cr50.c%23L202&data=04%7C01%7Cmario.limonciello%40amd.com
>> %7C1bdc742bc2a24f59b7d908d9dcc95438%7C3dd8961fe4884e608e11a82d994
>> e183d%7C0%7C0%7C637783579554955840%7CUnknown%7CTWFpbGZsb3d8ey
>> JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
>> 3000&sdata=fr0UEka%2BxYyPxqUG6oOZo%2Bc6pWgto2mD7hWA20YulVQ
>> %3D&reserved=0.
>>>
>>> Similar model is followed in drivers/char/tpm/tpm_i2c_infineon.c and
>>> couple of other i2c-client drivers.
>>
>> Ah I see, interesting (live and learn).
>>
>> But this is then combined with using the special __i2c_transfer()
>> function for the actual i2c reads/writes, since using the regular
>> i2c_transfer() function after already taking the lock would deadlock.
>>
>> There is a similar unlocked raw __i2c_smbus_xfer(), but as the
>> comment in include/linux/i2c.h above the locked i2c_smbus_xfer() says:
>>
>> /* This is the very generalized SMBus access routine. You probably do not
>>     want to use this, though; one of the functions below may be much easier,
>>     and probably just as fast.
>>     Note that we use i2c_adapter here, because you do not need a specific
>>     smbus adapter to call this function. */
>> s32 i2c_smbus_xfer(...);
>>
>> So in this case a driver cannot use the usual
>> i2c_smbus_read_byte/word/byte_data/word_data() helpers and
>> the same for writes. Also using an i2c_regmap (which is used
>> in a ton of places like PMIC drivers) will not work this way.
>>
>> So yes you can use i2c_bus_lock() for this; but only if all the
>> drivers where you want to do that limit themselves to
>> __i2c_transfer() and __i2c_smbus_xfer() calls and/or are
>> rewritten to only use those.
>>>> This is why in the Bay Trail case we have i2c-drivers
>>>> directly calling iosf_mbi_block_punit_i2c_access() and
>>>> iosf_mbi_unblock_punit_i2c_access() to lock the bus
>>>> for multiple i2c-transfers. We can get away with this there
>>>> because the bus in question is only used to access the
>>>> PMIC and that PMIC is only used on Bay Trail (and CHT)
>>>> boards, so the PMIC drivers can just hard-code these
>>>> calls.
>>>>
>>>> If you need to take the PSP I2C semaphore for multiple
>>>> transfers in some generic drivers, then I guess that the
>>>> i2c-subsys will need to get some new i2c_adapter callbacks
>>>> to acquire / release the bus for i2c-controllers where
>>>> the bus/controller is shared with some co-processor like
>>>> in the PSP case.
>>>
>>> This is exactly my intention to support generic i2c-clients drivers
>>> without them being aware that i2c-adapter above is using some
>>> semaphore/arbitration. Hopefully you can agree with me that currently
>>> available bus_locking can be used and is enough for this purpose.
>>
>> It can be used, but with limitations, see above.
>>
>>>
>>>> Also note that iosf_mbi_block_punit_i2c_access() and
>>>> iosf_mbi_unblock_punit_i2c_access() do their own
>>>> ref/lock-counting to allow calling them multiple times and
>>>> the first block call takes the bus and the last unblock
>>>> call releases it.
>>>
>>> This is exactly what I was talking about above and also implemented
>>> within psp_acquire_i2c_bus() and psp_release_i2c_bus().
>>
>> Right, I was to quick in skimming over your code when
>> I wrote down my concerns about there being a deadlock
>> there, sorry.
>>
>> Regards,
>>
>> Hans

Re: [PATCH 1/2] i2c: designware: Add missing locks

[Jan Dąbroś]

czw., 20 sty 2022 o 12:25 Hans de Goede <hdegoede@redhat.com> napisał(a):
>
> Hi,
>
> On 1/20/22 01:16, Jan Dabros wrote:
> > All accesses to controller's registers should be protected on
> > probe, disable and xfer paths. This is needed for i2c bus controllers
> > that are shared with but not controller by kernel.
> >
> > Signed-off-by: Jan Dabros <jsd@semihalf.com>
>
> Thanks, patch looks good to me:
>
> Reviewed-by: Hans de Goede <hdegoede@redhat.com>

Thanks!

Best Regards,
Jan

>
>
> Regards,
>
> Hans
>
>
> > ---
> >  drivers/i2c/busses/i2c-designware-common.c | 12 ++++++++++++
> >  drivers/i2c/busses/i2c-designware-master.c |  6 ++++++
> >  2 files changed, 18 insertions(+)
> >
> > diff --git a/drivers/i2c/busses/i2c-designware-common.c b/drivers/i2c/busses/i2c-designware-common.c
> > index bf2a4920638a..9f8574320eb2 100644
> > --- a/drivers/i2c/busses/i2c-designware-common.c
> > +++ b/drivers/i2c/busses/i2c-designware-common.c
> > @@ -578,7 +578,12 @@ int i2c_dw_set_fifo_size(struct dw_i2c_dev *dev)
> >        * Try to detect the FIFO depth if not set by interface driver,
> >        * the depth could be from 2 to 256 from HW spec.
> >        */
> > +     ret = i2c_dw_acquire_lock(dev);
> > +     if (ret)
> > +             return ret;
> > +
> >       ret = regmap_read(dev->map, DW_IC_COMP_PARAM_1, &param);
> > +     i2c_dw_release_lock(dev);
> >       if (ret)
> >               return ret;
> >
> > @@ -607,6 +612,11 @@ u32 i2c_dw_func(struct i2c_adapter *adap)
> >  void i2c_dw_disable(struct dw_i2c_dev *dev)
> >  {
> >       u32 dummy;
> > +     int ret;
> > +
> > +     ret = i2c_dw_acquire_lock(dev);
> > +     if (ret)
> > +             return;
> >
> >       /* Disable controller */
> >       __i2c_dw_disable(dev);
> > @@ -614,6 +624,8 @@ void i2c_dw_disable(struct dw_i2c_dev *dev)
> >       /* Disable all interrupts */
> >       regmap_write(dev->map, DW_IC_INTR_MASK, 0);
> >       regmap_read(dev->map, DW_IC_CLR_INTR, &dummy);
> > +
> > +     i2c_dw_release_lock(dev);
> >  }
> >
> >  void i2c_dw_disable_int(struct dw_i2c_dev *dev)
> > diff --git a/drivers/i2c/busses/i2c-designware-master.c b/drivers/i2c/busses/i2c-designware-master.c
> > index 9177463c2cbb..1a4b23556db3 100644
> > --- a/drivers/i2c/busses/i2c-designware-master.c
> > +++ b/drivers/i2c/busses/i2c-designware-master.c
> > @@ -905,7 +905,13 @@ int i2c_dw_probe_master(struct dw_i2c_dev *dev)
> >               irq_flags = IRQF_SHARED | IRQF_COND_SUSPEND;
> >       }
> >
> > +     ret = i2c_dw_acquire_lock(dev);
> > +     if (ret)
> > +             return ret;
> > +
> >       i2c_dw_disable_int(dev);
> > +     i2c_dw_release_lock(dev);
> > +
> >       ret = devm_request_irq(dev->dev, dev->irq, i2c_dw_isr, irq_flags,
> >                              dev_name(dev->dev), dev);
> >       if (ret) {
> >
>

Re: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dąbroś]

pt., 21 sty 2022 o 11:32 Hans de Goede <hdegoede@redhat.com> napisał(a):
>
> Hi Jan,
>
> On 1/21/22 10:59, Jan Dąbroś wrote:
> > (...)
> >
> >>> --- /dev/null
> >>> +++ b/drivers/i2c/busses/i2c-designware-amdpsp.c
> >>> @@ -0,0 +1,357 @@
> >>> +// SPDX-License-Identifier: GPL-2.0
> >>> +
> >>> +#include <asm/msr.h>
> >>> +#include <linux/i2c.h>
> >>> +#include <linux/psp-sev.h>
> >>> +
> >>> +#include "i2c-designware-core.h"
> >>
> >> So all the stuff starting here:
> >>
> >>> +
> >>> +#define MSR_AMD_PSP_ADDR     0xc00110a2
> >>> +#define PSP_MBOX_OFFSET              0x10570
> >>> +#define PSP_CMD_TIMEOUT_MS   500
> >>> +
> >>> +#define PSP_I2C_REQ_BUS_CMD          0x64
> >>> +#define PSP_I2C_REQ_RETRY_CNT                10
> >>> +#define PSP_I2C_REQ_RETRY_DELAY_MSEC 50
> >>> +#define PSP_I2C_REQ_STS_OK           0x0
> >>> +#define PSP_I2C_REQ_STS_BUS_BUSY     0x1
> >>> +#define PSP_I2C_REQ_STS_INV_PARAM    0x3
> >>> +
> >>> +union psp_req_buffer_hdr {
> >>> +     struct {
> >>> +             u32 total_size;
> >>> +             u32 status;
> >>> +     } __packed;
> >>> +     u64 hdr_val;
> >>> +};
> >>> +
> >>> +enum psp_i2c_req_type {
> >>> +     PSP_I2C_REQ_ACQUIRE,
> >>> +     PSP_I2C_REQ_RELEASE,
> >>> +     PSP_I2C_REQ_MAX,
> >>> +};
> >>> +
> >>> +struct psp_i2c_req {
> >>> +     union psp_req_buffer_hdr hdr;
> >>> +     enum psp_i2c_req_type type;
> >>> +} __packed __aligned(32);
> >>> +
> >>> +union psp_mbox_cmd_reg {
> >>> +     struct psp_mbox_cmd_fields {
> >>> +             u16 mbox_status;
> >>> +             u8 mbox_cmd;
> >>> +             u8 reserved:6;
> >>> +             u8 recovery:1;
> >>> +             u8 ready:1;
> >>> +     } __packed fields;
> >>> +     u32 val;
> >>> +};
> >>> +
> >>> +struct psp_mbox {
> >>> +     union psp_mbox_cmd_reg fields;
> >>> +     uintptr_t i2c_req_addr;
> >>> +} __packed;
> >>> +
> >>> +static DEFINE_MUTEX(psp_i2c_access_mutex);
> >>> +static unsigned long psp_i2c_sem_acquired;
> >>> +static void __iomem *mbox_iomem;
> >>> +static u32 psp_i2c_access_count;
> >>> +static bool psp_i2c_mbox_fail;
> >>> +static struct device *psp_i2c_dev;
> >>> +
> >>> +static int psp_get_mbox_addr(unsigned long *mbox_addr)
> >>> +{
> >>> +     unsigned long long psp_mmio;
> >>> +
> >>> +     if (rdmsrl_safe(MSR_AMD_PSP_ADDR, &psp_mmio))
> >>> +             return -EIO;
> >>> +
> >>> +     *mbox_addr = (unsigned long)(psp_mmio + PSP_MBOX_OFFSET);
> >>> +
> >>> +     return 0;
> >>> +}
> >>> +
> >>> +static int psp_mbox_probe(void)
> >>> +{
> >>> +     unsigned long mbox_addr;
> >>> +
> >>> +     if (psp_get_mbox_addr(&mbox_addr))
> >>> +             return -1;
> >>> +
> >>> +     mbox_iomem = ioremap(mbox_addr, sizeof(struct psp_mbox));
> >>> +     if (!mbox_iomem)
> >>> +             return -ENOMEM;
> >>> +
> >>> +     return 0;
> >>> +}
> >>> +
> >>> +/* Recovery field should be equal 0 to start sending commands */
> >>> +static int psp_check_mbox_recovery(struct psp_mbox *mbox)
> >>> +{
> >>> +     union psp_mbox_cmd_reg tmp = {0};
> >>> +
> >>> +     tmp.val = readl(&mbox->fields.val);
> >>> +     return !!tmp.fields.recovery;
> >>> +}
> >>> +
> >>> +static int psp_wait_cmd(struct psp_mbox *mbox)
> >>> +{
> >>> +     union psp_mbox_cmd_reg expected = { .val = 0 };
> >>> +     u32 tmp;
> >>> +
> >>> +     /* Expect mbox_cmd to be cleared and ready bit to be set by PSP */
> >>> +     expected.fields.ready = 1;
> >>> +
> >>> +     return readl_poll_timeout(&mbox->fields.val, tmp, (tmp == expected.val),
> >>> +                               0, 1000 * PSP_CMD_TIMEOUT_MS);
> >>> +}
> >>> +
> >>> +/* Status equal to 0 means that PSP succeed processing command */
> >>> +static int psp_check_mbox_sts(struct psp_mbox *mbox)
> >>> +{
> >>> +     union psp_mbox_cmd_reg cmd_reg = {0};
> >>> +
> >>> +     cmd_reg.val = readl(&mbox->fields.val);
> >>> +     return cmd_reg.fields.mbox_status;
> >>> +}
> >>> +
> >>> +static int psp_send_cmd(struct psp_i2c_req *req)
> >>> +{
> >>> +     struct psp_mbox *mbox = (struct psp_mbox *)mbox_iomem;
> >>> +     union psp_mbox_cmd_reg cmd_reg = {0};
> >>> +
> >>> +     if (psp_check_mbox_recovery(mbox))
> >>> +             return -EIO;
> >>> +
> >>> +     if (psp_wait_cmd(mbox))
> >>> +             return -EBUSY;
> >>> +
> >>> +     /* Fill address of command-response buffer */
> >>> +     writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr);
> >>> +
> >>> +     /* Write command register to trigger processing */
> >>> +     cmd_reg.fields.mbox_cmd = PSP_I2C_REQ_BUS_CMD;
> >>> +     writel(cmd_reg.val, &mbox->fields.val);
> >>> +
> >>> +     if (psp_wait_cmd(mbox))
> >>> +             return -ETIMEDOUT;
> >>> +
> >>> +     if (psp_check_mbox_sts(mbox))
> >>> +             return -EIO;
> >>> +
> >>> +     return 0;
> >>> +}
> >>
> >> Through here seems to all be generic code for accessing
> >> the AMD PSP. To me this seems like something which belongs
> >> in a separate AMD-PSP-mbox driver/lib, which can then be
> >> shared between other kernel drivers which may also want
> >> to access PSP.
> >
> > I see your point clearly and actually it is not an accident that I've
> > put all PSP-mailbox methods in one "block". They are logically
> > different than the rest of i2c-adapter specific methods.
> >
> > That being said, above PSP mailbox was created by AMD solely for the
> > purpose of i2c_arbitration. It has its own set of commands and
> > specific format of the command-response buffer. Thus it is not and it
> > won't be generic in the future. There are already upstreamed drivers
> > from AMD (under drivers/crypto/ccp/) which made use of PSP, however
> > their channel of communication looks completely different than the
> > very simple i2c_arbitration model implemented above.
> >
> > Because of this I'm treating this as an i2c_semaphore-related code and
> > putting this in this module. In my opinion moving this into some
> > separate driver (which will be actually used only here) makes code
> > less clear. But let's also hear some voice from AMD.
>
> Since as you say this mailbox is special and only for i2c-arbitration,
> keeping it inside this patch / .c file is fine.
>
> >
> >>
> >> Sorta like the generic iosf_mbi_read() and
> >> iosf_mbi_write() functions from:
> >>
> >> arch/x86/platform/intel/iosf_mbi.c
> >>
> >> used on the Intel chips, which are also used outside of
> >> the I2C bus-locking code.
> >>
> >> This is also one of the reasons why I think it would be
> >> good to get some AMD folks involved in this, since they
> >> may be aware of other drivers which also need to access
> >> the PSP mbox.
> >>
> >
> > Right, I'm adding mario.limonciello@amd.com to the CC, so that he can comment.
> >
> > (...)
> >
> >>> +/*
> >>> + * Locking methods are based on the default implementation from
> >>> + * drivers/i2c/i2c-core.base.c, but with psp acquire and release operations
> >>> + * added. With this in place we can ensure that i2c clients on the bus shared
> >>> + * with psp are able to lock HW access to the bus for arbitrary number of
> >>> + * operations - that is e.g. write-wait-read.
> >>> + */
> >>> +static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter,
> >>> +                                     unsigned int flags)
> >>> +{
> >>> +     psp_acquire_i2c_bus();
> >>> +     rt_mutex_lock_nested(&adapter->bus_lock, i2c_adapter_depth(adapter));
> >>
> >> This does not do what you think it does and you will still deadlock
> >> when things nest because of someone taking the bus-lock and then
> >> the main i2c-designware transfer function calling the acquire_lock
> >> callback.
> >
> > I haven't used rt_mutex_lock_nested() with the intent to prevent me
> > from deadlock when i2c-designware calls acquire_lock with bus-lock
> > already taken. This is a method copied from
> > drivers/i2c/i2c-core-base.c (BTW, I have a typo in above comment).
> > This is the default implementation applied by i2c-core when particular
> > adapter doesn't register its own locking callbacks - thus it is called
> > for i2c-designware for all platforms.
> >
> > In case of this driver internal i2c-designware acquire_lock() is equal
> > to psp_acquire_i2c_bus(). In other words, bus-level lock
> > i2c_adapter_dw_psp_lock_bus() is a superset of internal adapter's
> > acquire_lock().
>
> Ah I missed that this is just mimicking the core functions +
> an extra call to psp_acquire_i2c_bus().
>
> I assumed that the dwc->acquire callback path was also taking
> the mutex and I thought you had fallen for the _nested meaning
> something different then it does, my bad.
>
> > In order to prevent deadlock which you are talking about, I'm using
> > reference lock counter inside psp_acquire_i2c_bus() thus it is safe to
> > invoke acquire_lock() when bus-lock is already taken.
>
> Ah good, that is pretty much is the same as what the Bay Trail code
> is doing.
>
> >
> >>
> >> The _nested postfix is only for the lockdep lock-debugger, this
> >> actually turns into a regular mutex_lock when lockdep is not enabled:
> >>
> >> #ifdef CONFIG_DEBUG_LOCK_ALLOC
> >> extern void rt_mutex_lock_nested(struct rt_mutex *lock, unsigned int subclass);
> >> #define rt_mutex_lock(lock) rt_mutex_lock_nested(lock, 0)
> >> #else
> >> extern void rt_mutex_lock(struct rt_mutex *lock);
> >> #define rt_mutex_lock_nested(lock, subclass) rt_mutex_lock(lock)
> >> #endif
> >>
> >> The _nested postfix as such is only to tell the lockdep code that
> >> even though it seems we are trying to take the same mutex twice
> >> since in both cases it is of i2c_adapter.rt_mutex "lock class"
> >> that we are sure it is never the same i2c_adapter (but rather
> >> one which always gets called in a nested fashion from another
> >> i2c_adapter).
> >>
> >> IOW this only disables a false-positive lockdep warning, it does
> >> not allow taking the same mutex twice, you will still hang on
> >> the second mutex_lock call on the same lock.
> >
> > Thanks for the technical background about rt_mutex_lock_nested. I
> > think we should keep using it as is, since as I wrote above I don't
> > have any reasoning to modify it here.
>
> Ack, now that my misreading of the code has been cleared up
> I agree.
>
> >> Also I don't think you are allowed to use the bus_locking code
> >> like this. The i2c bus-locking code is intended to deal with
> >> busses which have muxes in them, where the mux must be set
> >> to the right branch of the bus to reach the client and then
> >> not be changed during the transfer to that client.
> >>
> >> So i2c-client drivers are never supposed to directly call
> >> the bus-locking functions.
> >
> > I think you are not correct here. There are examples of i2c-clients
> > which are using i2c bus_locking for the purpose of locking adapter for
> > the bunch of i2c transactions.
> >
> > As an example let's take drivers/char/tpm/tpm_tis_i2c_cr50.c. It
> > operates in write-wait-read model and there is i2c_lock_bus() call
> > used to ensure that bus won't be released -
> > https://github.com/torvalds/linux/blob/master/drivers/char/tpm/tpm_tis_i2c_cr50.c#L202.
> >
> > Similar model is followed in drivers/char/tpm/tpm_i2c_infineon.c and
> > couple of other i2c-client drivers.
>
> Ah I see, interesting (live and learn).
>
> But this is then combined with using the special __i2c_transfer()
> function for the actual i2c reads/writes, since using the regular
> i2c_transfer() function after already taking the lock would deadlock.

Correct. In other words, if i2c-client wants to block the bus/adapter
for more than one transaction it must use some special methods. This
isn't changed with my patchset. If one is using "normal"
i2c_transfer(), we should be on the safe side, nothing will change
from the i2c-client point of view. The same if one is using
__i2c_transfer().

>
> There is a similar unlocked raw __i2c_smbus_xfer(), but as the
> comment in include/linux/i2c.h above the locked i2c_smbus_xfer() says:
>
> /* This is the very generalized SMBus access routine. You probably do not
>    want to use this, though; one of the functions below may be much easier,
>    and probably just as fast.
>    Note that we use i2c_adapter here, because you do not need a specific
>    smbus adapter to call this function. */
> s32 i2c_smbus_xfer(...);
>
> So in this case a driver cannot use the usual
> i2c_smbus_read_byte/word/byte_data/word_data() helpers and
> the same for writes. Also using an i2c_regmap (which is used
> in a ton of places like PMIC drivers) will not work this way.

Right, however this behavior is not altered by my patch. I just wanted
to ensure that drivers which are already using i2c bus_locking will
still work as expected.

> So yes you can use i2c_bus_lock() for this; but only if all the
> drivers where you want to do that limit themselves to
> __i2c_transfer() and __i2c_smbus_xfer() calls and/or are
> rewritten to only use those.

My goal is to not modify current behavior, that is - we don't need to
modify clients' drivers and extra quirks applied by amdpsp semaphore
will be "transparent" for them. IOW, switch from generic
i2c-designware to i2c-designware-amdpsp should be invisible from the
i2c bus perspective for i2c-clients.

Best Regards,
Jan

> >> This is why in the Bay Trail case we have i2c-drivers
> >> directly calling iosf_mbi_block_punit_i2c_access() and
> >> iosf_mbi_unblock_punit_i2c_access() to lock the bus
> >> for multiple i2c-transfers. We can get away with this there
> >> because the bus in question is only used to access the
> >> PMIC and that PMIC is only used on Bay Trail (and CHT)
> >> boards, so the PMIC drivers can just hard-code these
> >> calls.
> >>
> >> If you need to take the PSP I2C semaphore for multiple
> >> transfers in some generic drivers, then I guess that the
> >> i2c-subsys will need to get some new i2c_adapter callbacks
> >> to acquire / release the bus for i2c-controllers where
> >> the bus/controller is shared with some co-processor like
> >> in the PSP case.
> >
> > This is exactly my intention to support generic i2c-clients drivers
> > without them being aware that i2c-adapter above is using some
> > semaphore/arbitration. Hopefully you can agree with me that currently
> > available bus_locking can be used and is enough for this purpose.
>
> It can be used, but with limitations, see above.
>
> >
> >> Also note that iosf_mbi_block_punit_i2c_access() and
> >> iosf_mbi_unblock_punit_i2c_access() do their own
> >> ref/lock-counting to allow calling them multiple times and
> >> the first block call takes the bus and the last unblock
> >> call releases it.
> >
> > This is exactly what I was talking about above and also implemented
> > within psp_acquire_i2c_bus() and psp_release_i2c_bus().
>
> Right, I was to quick in skimming over your code when
> I wrote down my concerns about there being a deadlock
> there, sorry.
>
> Regards,
>
> Hans
>

Re: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dąbroś]

Hi Mario, Tom,

+Nimesh (PSP)

pt., 21 sty 2022 o 20:18 Tom Lendacky <thomas.lendacky@amd.com> napisał(a):
>
> On 1/21/22 11:38 AM, Limonciello, Mario wrote:
> > [Public]
> >
> > +Thomas (ccp driver)
> > +Alex (amdgpu driver)
> >
> >>
> >> On 1/21/22 10:59, Jan Dąbroś wrote:
>
> >>>>
> >>>> Through here seems to all be generic code for accessing
> >>>> the AMD PSP. To me this seems like something which belongs
> >>>> in a separate AMD-PSP-mbox driver/lib, which can then be
> >>>> shared between other kernel drivers which may also want
> >>>> to access PSP.
> >>>
> >>> I see your point clearly and actually it is not an accident that I've
> >>> put all PSP-mailbox methods in one "block". They are logically
> >>> different than the rest of i2c-adapter specific methods.
> >>>
> >>> That being said, above PSP mailbox was created by AMD solely for the
> >>> purpose of i2c_arbitration. It has its own set of commands and
> >>> specific format of the command-response buffer. Thus it is not and it
> >>> won't be generic in the future. There are already upstreamed drivers
> >>> from AMD (under drivers/crypto/ccp/) which made use of PSP, however
> >>> their channel of communication looks completely different than the
> >>> very simple i2c_arbitration model implemented above.
> >>>
> >
> > Can you please double confirm no other users for this mailbox and it's for
> > you only?  And if so is it only specific to this platform that no other users?
> > At least on some UEFI AMD platforms that mailbox is defined for
> > communication with something else.  We might need some way to attest
> > from the firmware that it's safe to use.
> >
> > The only mailbox that is defined for OS use across different silicon AFAIK is
> > the GPU driver mailbox.  It may be safer to use that, but I'm not sure if
> > GPU driver has come up early enough for your use.
>
> The CCP/PSP driver will load as a PCIe device driver on this platform and
> will ioremap the MMIO space. Today, that driver doesn't reference those
> mailbox registers, and I don't know that there will be a need in the
> future. But if there is a need in the future, you could end up with a
> conflict between these two drivers.

Right, so I have confirmed this with AMD PSP firmware developers, that
this particular x86-PSP mailbox is created and will be reserved
_solely_ for the purpose of i2c arbitration (and thus this driver).
There is no intend to use it elsewhere or share with another users.

> Thanks,
> Tom
>
> >
> >>> Because of this I'm treating this as an i2c_semaphore-related code and
> >>> putting this in this module. In my opinion moving this into some
> >>> separate driver (which will be actually used only here) makes code
> >>> less clear. But let's also hear some voice from AMD.
> >>
> >> Since as you say this mailbox is special and only for i2c-arbitration,
> >> keeping it inside this patch / .c file is fine.
> >>
> >>>
> >>>>
> >>>> Sorta like the generic iosf_mbi_read() and
> >>>> iosf_mbi_write() functions from:
> >>>>
> >>>> arch/x86/platform/intel/iosf_mbi.c
> >>>>
> >>>> used on the Intel chips, which are also used outside of
> >>>> the I2C bus-locking code.
> >>>>
> >>>> This is also one of the reasons why I think it would be
> >>>> good to get some AMD folks involved in this, since they
> >>>> may be aware of other drivers which also need to access
> >>>> the PSP mbox.
> >>>>
> >>>
> >>> Right, I'm adding mario.limonciello@amd.com to the CC, so that he can
> >> comment.
> >>>
> >>> (...)
> >>>
> >>>>> +/*
> >>>>> + * Locking methods are based on the default implementation from
> >>>>> + * drivers/i2c/i2c-core.base.c, but with psp acquire and release operations
> >>>>> + * added. With this in place we can ensure that i2c clients on the bus shared
> >>>>> + * with psp are able to lock HW access to the bus for arbitrary number of
> >>>>> + * operations - that is e.g. write-wait-read.
> >>>>> + */
> >>>>> +static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter,
> >>>>> +                                     unsigned int flags)
> >>>>> +{
> >>>>> +     psp_acquire_i2c_bus();
> >>>>> +     rt_mutex_lock_nested(&adapter->bus_lock,
> >> i2c_adapter_depth(adapter));
> >>>>
> >>>> This does not do what you think it does and you will still deadlock
> >>>> when things nest because of someone taking the bus-lock and then
> >>>> the main i2c-designware transfer function calling the acquire_lock
> >>>> callback.
> >>>
> >>> I haven't used rt_mutex_lock_nested() with the intent to prevent me
> >>> from deadlock when i2c-designware calls acquire_lock with bus-lock
> >>> already taken. This is a method copied from
> >>> drivers/i2c/i2c-core-base.c (BTW, I have a typo in above comment).
> >>> This is the default implementation applied by i2c-core when particular
> >>> adapter doesn't register its own locking callbacks - thus it is called
> >>> for i2c-designware for all platforms.
> >>>
> >>> In case of this driver internal i2c-designware acquire_lock() is equal
> >>> to psp_acquire_i2c_bus(). In other words, bus-level lock
> >>> i2c_adapter_dw_psp_lock_bus() is a superset of internal adapter's
> >>> acquire_lock().
> >>
> >> Ah I missed that this is just mimicking the core functions +
> >> an extra call to psp_acquire_i2c_bus().
> >>
> >> I assumed that the dwc->acquire callback path was also taking
> >> the mutex and I thought you had fallen for the _nested meaning
> >> something different then it does, my bad.
> >>
> >>> In order to prevent deadlock which you are talking about, I'm using
> >>> reference lock counter inside psp_acquire_i2c_bus() thus it is safe to
> >>> invoke acquire_lock() when bus-lock is already taken.
> >>
> >> Ah good, that is pretty much is the same as what the Bay Trail code
> >> is doing.
> >>
> >>>
> >>>>
> >>>> The _nested postfix is only for the lockdep lock-debugger, this
> >>>> actually turns into a regular mutex_lock when lockdep is not enabled:
> >>>>
> >>>> #ifdef CONFIG_DEBUG_LOCK_ALLOC
> >>>> extern void rt_mutex_lock_nested(struct rt_mutex *lock, unsigned int
> >> subclass);
> >>>> #define rt_mutex_lock(lock) rt_mutex_lock_nested(lock, 0)
> >>>> #else
> >>>> extern void rt_mutex_lock(struct rt_mutex *lock);
> >>>> #define rt_mutex_lock_nested(lock, subclass) rt_mutex_lock(lock)
> >>>> #endif
> >>>>
> >>>> The _nested postfix as such is only to tell the lockdep code that
> >>>> even though it seems we are trying to take the same mutex twice
> >>>> since in both cases it is of i2c_adapter.rt_mutex "lock class"
> >>>> that we are sure it is never the same i2c_adapter (but rather
> >>>> one which always gets called in a nested fashion from another
> >>>> i2c_adapter).
> >>>>
> >>>> IOW this only disables a false-positive lockdep warning, it does
> >>>> not allow taking the same mutex twice, you will still hang on
> >>>> the second mutex_lock call on the same lock.
> >>>
> >>> Thanks for the technical background about rt_mutex_lock_nested. I
> >>> think we should keep using it as is, since as I wrote above I don't
> >>> have any reasoning to modify it here.
> >>
> >> Ack, now that my misreading of the code has been cleared up
> >> I agree.
> >>
> >>>> Also I don't think you are allowed to use the bus_locking code
> >>>> like this. The i2c bus-locking code is intended to deal with
> >>>> busses which have muxes in them, where the mux must be set
> >>>> to the right branch of the bus to reach the client and then
> >>>> not be changed during the transfer to that client.
> >>>>
> >>>> So i2c-client drivers are never supposed to directly call
> >>>> the bus-locking functions.
> >>>
> >>> I think you are not correct here. There are examples of i2c-clients
> >>> which are using i2c bus_locking for the purpose of locking adapter for
> >>> the bunch of i2c transactions.
> >>>
> >>> As an example let's take drivers/char/tpm/tpm_tis_i2c_cr50.c. It
> >>> operates in write-wait-read model and there is i2c_lock_bus() call
> >>> used to ensure that bus won't be released -
> >>>
> >> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c
> >> om%2Ftorvalds%2Flinux%2Fblob%2Fmaster%2Fdrivers%2Fchar%2Ftpm%2Ftpm_
> >> tis_i2c_cr50.c%23L202&amp;data=04%7C01%7Cmario.limonciello%40amd.com
> >> %7C1bdc742bc2a24f59b7d908d9dcc95438%7C3dd8961fe4884e608e11a82d994
> >> e183d%7C0%7C0%7C637783579554955840%7CUnknown%7CTWFpbGZsb3d8ey
> >> JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
> >> 3000&amp;sdata=fr0UEka%2BxYyPxqUG6oOZo%2Bc6pWgto2mD7hWA20YulVQ
> >> %3D&amp;reserved=0.
> >>>
> >>> Similar model is followed in drivers/char/tpm/tpm_i2c_infineon.c and
> >>> couple of other i2c-client drivers.
> >>
> >> Ah I see, interesting (live and learn).
> >>
> >> But this is then combined with using the special __i2c_transfer()
> >> function for the actual i2c reads/writes, since using the regular
> >> i2c_transfer() function after already taking the lock would deadlock.
> >>
> >> There is a similar unlocked raw __i2c_smbus_xfer(), but as the
> >> comment in include/linux/i2c.h above the locked i2c_smbus_xfer() says:
> >>
> >> /* This is the very generalized SMBus access routine. You probably do not
> >>     want to use this, though; one of the functions below may be much easier,
> >>     and probably just as fast.
> >>     Note that we use i2c_adapter here, because you do not need a specific
> >>     smbus adapter to call this function. */
> >> s32 i2c_smbus_xfer(...);
> >>
> >> So in this case a driver cannot use the usual
> >> i2c_smbus_read_byte/word/byte_data/word_data() helpers and
> >> the same for writes. Also using an i2c_regmap (which is used
> >> in a ton of places like PMIC drivers) will not work this way.
> >>
> >> So yes you can use i2c_bus_lock() for this; but only if all the
> >> drivers where you want to do that limit themselves to
> >> __i2c_transfer() and __i2c_smbus_xfer() calls and/or are
> >> rewritten to only use those.
> >>>> This is why in the Bay Trail case we have i2c-drivers
> >>>> directly calling iosf_mbi_block_punit_i2c_access() and
> >>>> iosf_mbi_unblock_punit_i2c_access() to lock the bus
> >>>> for multiple i2c-transfers. We can get away with this there
> >>>> because the bus in question is only used to access the
> >>>> PMIC and that PMIC is only used on Bay Trail (and CHT)
> >>>> boards, so the PMIC drivers can just hard-code these
> >>>> calls.
> >>>>
> >>>> If you need to take the PSP I2C semaphore for multiple
> >>>> transfers in some generic drivers, then I guess that the
> >>>> i2c-subsys will need to get some new i2c_adapter callbacks
> >>>> to acquire / release the bus for i2c-controllers where
> >>>> the bus/controller is shared with some co-processor like
> >>>> in the PSP case.
> >>>
> >>> This is exactly my intention to support generic i2c-clients drivers
> >>> without them being aware that i2c-adapter above is using some
> >>> semaphore/arbitration. Hopefully you can agree with me that currently
> >>> available bus_locking can be used and is enough for this purpose.
> >>
> >> It can be used, but with limitations, see above.
> >>
> >>>
> >>>> Also note that iosf_mbi_block_punit_i2c_access() and
> >>>> iosf_mbi_unblock_punit_i2c_access() do their own
> >>>> ref/lock-counting to allow calling them multiple times and
> >>>> the first block call takes the bus and the last unblock
> >>>> call releases it.
> >>>
> >>> This is exactly what I was talking about above and also implemented
> >>> within psp_acquire_i2c_bus() and psp_release_i2c_bus().
> >>
> >> Right, I was to quick in skimming over your code when
> >> I wrote down my concerns about there being a deadlock
> >> there, sorry.
> >>
> >> Regards,
> >>
> >> Hans

Re: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Hans de Goede]

Hi,

On 1/24/22 13:35, Jan Dąbroś wrote:
> pt., 21 sty 2022 o 11:32 Hans de Goede <hdegoede@redhat.com> napisał(a):
>>
>> Hi Jan,
>>
>> On 1/21/22 10:59, Jan Dąbroś wrote:

<snip>

>>>> Also I don't think you are allowed to use the bus_locking code
>>>> like this. The i2c bus-locking code is intended to deal with
>>>> busses which have muxes in them, where the mux must be set
>>>> to the right branch of the bus to reach the client and then
>>>> not be changed during the transfer to that client.
>>>>
>>>> So i2c-client drivers are never supposed to directly call
>>>> the bus-locking functions.
>>>
>>> I think you are not correct here. There are examples of i2c-clients
>>> which are using i2c bus_locking for the purpose of locking adapter for
>>> the bunch of i2c transactions.
>>>
>>> As an example let's take drivers/char/tpm/tpm_tis_i2c_cr50.c. It
>>> operates in write-wait-read model and there is i2c_lock_bus() call
>>> used to ensure that bus won't be released -
>>> https://github.com/torvalds/linux/blob/master/drivers/char/tpm/tpm_tis_i2c_cr50.c#L202.
>>>
>>> Similar model is followed in drivers/char/tpm/tpm_i2c_infineon.c and
>>> couple of other i2c-client drivers.
>>
>> Ah I see, interesting (live and learn).
>>
>> But this is then combined with using the special __i2c_transfer()
>> function for the actual i2c reads/writes, since using the regular
>> i2c_transfer() function after already taking the lock would deadlock.
> 
> Correct. In other words, if i2c-client wants to block the bus/adapter
> for more than one transaction it must use some special methods. This
> isn't changed with my patchset. If one is using "normal"
> i2c_transfer(), we should be on the safe side, nothing will change
> from the i2c-client point of view. The same if one is using
> __i2c_transfer().
> 
>>
>> There is a similar unlocked raw __i2c_smbus_xfer(), but as the
>> comment in include/linux/i2c.h above the locked i2c_smbus_xfer() says:
>>
>> /* This is the very generalized SMBus access routine. You probably do not
>>    want to use this, though; one of the functions below may be much easier,
>>    and probably just as fast.
>>    Note that we use i2c_adapter here, because you do not need a specific
>>    smbus adapter to call this function. */
>> s32 i2c_smbus_xfer(...);
>>
>> So in this case a driver cannot use the usual
>> i2c_smbus_read_byte/word/byte_data/word_data() helpers and
>> the same for writes. Also using an i2c_regmap (which is used
>> in a ton of places like PMIC drivers) will not work this way.
> 
> Right, however this behavior is not altered by my patch. I just wanted
> to ensure that drivers which are already using i2c bus_locking will
> still work as expected.

Ah I see, I thought that maybe you wanted to add extra i2c bus-locking
calls to some drivers which don't have them yet to ensure that the
AMD PSP semaphore would be hold over multiple transfers in
drivers which don't do this yet.

>> So yes you can use i2c_bus_lock() for this; but only if all the
>> drivers where you want to do that limit themselves to
>> __i2c_transfer() and __i2c_smbus_xfer() calls and/or are
>> rewritten to only use those.
> 
> My goal is to not modify current behavior, that is - we don't need to
> modify clients' drivers and extra quirks applied by amdpsp semaphore
> will be "transparent" for them. IOW, switch from generic
> i2c-designware to i2c-designware-amdpsp should be invisible from the
> i2c bus perspective for i2c-clients.

Ok, I believe everything is fine as is then. My worries where about
extending the i2c bus-locking to more drivers, but if there are no
plans for that then everything is good.

Regards,

Hans

RE: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Limonciello, Mario]

[Public]

> >
> > >>>>
> > >>>> Through here seems to all be generic code for accessing
> > >>>> the AMD PSP. To me this seems like something which belongs
> > >>>> in a separate AMD-PSP-mbox driver/lib, which can then be
> > >>>> shared between other kernel drivers which may also want
> > >>>> to access PSP.
> > >>>
> > >>> I see your point clearly and actually it is not an accident that I've
> > >>> put all PSP-mailbox methods in one "block". They are logically
> > >>> different than the rest of i2c-adapter specific methods.
> > >>>
> > >>> That being said, above PSP mailbox was created by AMD solely for the
> > >>> purpose of i2c_arbitration. It has its own set of commands and
> > >>> specific format of the command-response buffer. Thus it is not and it
> > >>> won't be generic in the future. There are already upstreamed drivers
> > >>> from AMD (under drivers/crypto/ccp/) which made use of PSP, however
> > >>> their channel of communication looks completely different than the
> > >>> very simple i2c_arbitration model implemented above.
> > >>>
> > >
> > > Can you please double confirm no other users for this mailbox and it's for
> > > you only?  And if so is it only specific to this platform that no other users?
> > > At least on some UEFI AMD platforms that mailbox is defined for
> > > communication with something else.  We might need some way to attest
> > > from the firmware that it's safe to use.
> > >
> > > The only mailbox that is defined for OS use across different silicon AFAIK is
> > > the GPU driver mailbox.  It may be safer to use that, but I'm not sure if
> > > GPU driver has come up early enough for your use.
> >
> > The CCP/PSP driver will load as a PCIe device driver on this platform and
> > will ioremap the MMIO space. Today, that driver doesn't reference those
> > mailbox registers, and I don't know that there will be a need in the
> > future. But if there is a need in the future, you could end up with a
> > conflict between these two drivers.
> 
> Right, so I have confirmed this with AMD PSP firmware developers, that
> this particular x86-PSP mailbox is created and will be reserved
> _solely_ for the purpose of i2c arbitration (and thus this driver).
> There is no intend to use it elsewhere or share with another users.

I've learned never to say never.  People move on to different roles and history
gets lost.  As it's exclusive to this use case of I2C arbitration it will probably still
be useful to leave in the comments nearby for our future selves what model/family
SOC this was introduced in case the number of mailboxes are extinguished some
day and this ends up being needed for a secondary purpose for some future SOC.

> 
> > Thanks,
> > Tom
> >
> > >
> > >>> Because of this I'm treating this as an i2c_semaphore-related code and
> > >>> putting this in this module. In my opinion moving this into some
> > >>> separate driver (which will be actually used only here) makes code
> > >>> less clear. But let's also hear some voice from AMD.
> > >>
> > >> Since as you say this mailbox is special and only for i2c-arbitration,
> > >> keeping it inside this patch / .c file is fine.
> > >>
> > >>>
> > >>>>
> > >>>> Sorta like the generic iosf_mbi_read() and
> > >>>> iosf_mbi_write() functions from:
> > >>>>
> > >>>> arch/x86/platform/intel/iosf_mbi.c
> > >>>>
> > >>>> used on the Intel chips, which are also used outside of
> > >>>> the I2C bus-locking code.
> > >>>>
> > >>>> This is also one of the reasons why I think it would be
> > >>>> good to get some AMD folks involved in this, since they
> > >>>> may be aware of other drivers which also need to access
> > >>>> the PSP mbox.
> > >>>>
> > >>>
> > >>> Right, I'm adding mario.limonciello@amd.com to the CC, so that he can
> > >> comment.
> > >>>
> > >>> (...)
> > >>>
> > >>>>> +/*
> > >>>>> + * Locking methods are based on the default implementation from
> > >>>>> + * drivers/i2c/i2c-core.base.c, but with psp acquire and release
> operations
> > >>>>> + * added. With this in place we can ensure that i2c clients on the bus
> shared
> > >>>>> + * with psp are able to lock HW access to the bus for arbitrary number
> of
> > >>>>> + * operations - that is e.g. write-wait-read.
> > >>>>> + */
> > >>>>> +static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter,
> > >>>>> +                                     unsigned int flags)
> > >>>>> +{
> > >>>>> +     psp_acquire_i2c_bus();
> > >>>>> +     rt_mutex_lock_nested(&adapter->bus_lock,
> > >> i2c_adapter_depth(adapter));
> > >>>>
> > >>>> This does not do what you think it does and you will still deadlock
> > >>>> when things nest because of someone taking the bus-lock and then
> > >>>> the main i2c-designware transfer function calling the acquire_lock
> > >>>> callback.
> > >>>
> > >>> I haven't used rt_mutex_lock_nested() with the intent to prevent me
> > >>> from deadlock when i2c-designware calls acquire_lock with bus-lock
> > >>> already taken. This is a method copied from
> > >>> drivers/i2c/i2c-core-base.c (BTW, I have a typo in above comment).
> > >>> This is the default implementation applied by i2c-core when particular
> > >>> adapter doesn't register its own locking callbacks - thus it is called
> > >>> for i2c-designware for all platforms.
> > >>>
> > >>> In case of this driver internal i2c-designware acquire_lock() is equal
> > >>> to psp_acquire_i2c_bus(). In other words, bus-level lock
> > >>> i2c_adapter_dw_psp_lock_bus() is a superset of internal adapter's
> > >>> acquire_lock().
> > >>
> > >> Ah I missed that this is just mimicking the core functions +
> > >> an extra call to psp_acquire_i2c_bus().
> > >>
> > >> I assumed that the dwc->acquire callback path was also taking
> > >> the mutex and I thought you had fallen for the _nested meaning
> > >> something different then it does, my bad.
> > >>
> > >>> In order to prevent deadlock which you are talking about, I'm using
> > >>> reference lock counter inside psp_acquire_i2c_bus() thus it is safe to
> > >>> invoke acquire_lock() when bus-lock is already taken.
> > >>
> > >> Ah good, that is pretty much is the same as what the Bay Trail code
> > >> is doing.
> > >>
> > >>>
> > >>>>
> > >>>> The _nested postfix is only for the lockdep lock-debugger, this
> > >>>> actually turns into a regular mutex_lock when lockdep is not enabled:
> > >>>>
> > >>>> #ifdef CONFIG_DEBUG_LOCK_ALLOC
> > >>>> extern void rt_mutex_lock_nested(struct rt_mutex *lock, unsigned int
> > >> subclass);
> > >>>> #define rt_mutex_lock(lock) rt_mutex_lock_nested(lock, 0)
> > >>>> #else
> > >>>> extern void rt_mutex_lock(struct rt_mutex *lock);
> > >>>> #define rt_mutex_lock_nested(lock, subclass) rt_mutex_lock(lock)
> > >>>> #endif
> > >>>>
> > >>>> The _nested postfix as such is only to tell the lockdep code that
> > >>>> even though it seems we are trying to take the same mutex twice
> > >>>> since in both cases it is of i2c_adapter.rt_mutex "lock class"
> > >>>> that we are sure it is never the same i2c_adapter (but rather
> > >>>> one which always gets called in a nested fashion from another
> > >>>> i2c_adapter).
> > >>>>
> > >>>> IOW this only disables a false-positive lockdep warning, it does
> > >>>> not allow taking the same mutex twice, you will still hang on
> > >>>> the second mutex_lock call on the same lock.
> > >>>
> > >>> Thanks for the technical background about rt_mutex_lock_nested. I
> > >>> think we should keep using it as is, since as I wrote above I don't
> > >>> have any reasoning to modify it here.
> > >>
> > >> Ack, now that my misreading of the code has been cleared up
> > >> I agree.
> > >>
> > >>>> Also I don't think you are allowed to use the bus_locking code
> > >>>> like this. The i2c bus-locking code is intended to deal with
> > >>>> busses which have muxes in them, where the mux must be set
> > >>>> to the right branch of the bus to reach the client and then
> > >>>> not be changed during the transfer to that client.
> > >>>>
> > >>>> So i2c-client drivers are never supposed to directly call
> > >>>> the bus-locking functions.
> > >>>
> > >>> I think you are not correct here. There are examples of i2c-clients
> > >>> which are using i2c bus_locking for the purpose of locking adapter for
> > >>> the bunch of i2c transactions.
> > >>>
> > >>> As an example let's take drivers/char/tpm/tpm_tis_i2c_cr50.c. It
> > >>> operates in write-wait-read model and there is i2c_lock_bus() call
> > >>> used to ensure that bus won't be released -
> > >>>
> > >>
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c
> %2F&data=04%7C01%7CMario.Limonciello%40amd.com%7Cf8cd037a72b
> 749fb003408d9df3f5ced%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0
> %7C637786285501955886%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
> MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata
> =s4N9vTWbuRUww0qvsRz16fV%2Fnj41SxyLOB%2BjGFjyOaA%3D&reserved
> =0
> > >>
> om%2Ftorvalds%2Flinux%2Fblob%2Fmaster%2Fdrivers%2Fchar%2Ftpm%2Ftpm_
> > >>
> tis_i2c_cr50.c%23L202&data=04%7C01%7Cmario.limonciello%40amd.com
> > >>
> %7C1bdc742bc2a24f59b7d908d9dcc95438%7C3dd8961fe4884e608e11a82d994
> > >>
> e183d%7C0%7C0%7C637783579554955840%7CUnknown%7CTWFpbGZsb3d8ey
> > >>
> JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
> > >>
> 3000&sdata=fr0UEka%2BxYyPxqUG6oOZo%2Bc6pWgto2mD7hWA20YulVQ
> > >> %3D&reserved=0.
> > >>>
> > >>> Similar model is followed in drivers/char/tpm/tpm_i2c_infineon.c and
> > >>> couple of other i2c-client drivers.
> > >>
> > >> Ah I see, interesting (live and learn).
> > >>
> > >> But this is then combined with using the special __i2c_transfer()
> > >> function for the actual i2c reads/writes, since using the regular
> > >> i2c_transfer() function after already taking the lock would deadlock.
> > >>
> > >> There is a similar unlocked raw __i2c_smbus_xfer(), but as the
> > >> comment in include/linux/i2c.h above the locked i2c_smbus_xfer() says:
> > >>
> > >> /* This is the very generalized SMBus access routine. You probably do not
> > >>     want to use this, though; one of the functions below may be much easier,
> > >>     and probably just as fast.
> > >>     Note that we use i2c_adapter here, because you do not need a specific
> > >>     smbus adapter to call this function. */
> > >> s32 i2c_smbus_xfer(...);
> > >>
> > >> So in this case a driver cannot use the usual
> > >> i2c_smbus_read_byte/word/byte_data/word_data() helpers and
> > >> the same for writes. Also using an i2c_regmap (which is used
> > >> in a ton of places like PMIC drivers) will not work this way.
> > >>
> > >> So yes you can use i2c_bus_lock() for this; but only if all the
> > >> drivers where you want to do that limit themselves to
> > >> __i2c_transfer() and __i2c_smbus_xfer() calls and/or are
> > >> rewritten to only use those.
> > >>>> This is why in the Bay Trail case we have i2c-drivers
> > >>>> directly calling iosf_mbi_block_punit_i2c_access() and
> > >>>> iosf_mbi_unblock_punit_i2c_access() to lock the bus
> > >>>> for multiple i2c-transfers. We can get away with this there
> > >>>> because the bus in question is only used to access the
> > >>>> PMIC and that PMIC is only used on Bay Trail (and CHT)
> > >>>> boards, so the PMIC drivers can just hard-code these
> > >>>> calls.
> > >>>>
> > >>>> If you need to take the PSP I2C semaphore for multiple
> > >>>> transfers in some generic drivers, then I guess that the
> > >>>> i2c-subsys will need to get some new i2c_adapter callbacks
> > >>>> to acquire / release the bus for i2c-controllers where
> > >>>> the bus/controller is shared with some co-processor like
> > >>>> in the PSP case.
> > >>>
> > >>> This is exactly my intention to support generic i2c-clients drivers
> > >>> without them being aware that i2c-adapter above is using some
> > >>> semaphore/arbitration. Hopefully you can agree with me that currently
> > >>> available bus_locking can be used and is enough for this purpose.
> > >>
> > >> It can be used, but with limitations, see above.
> > >>
> > >>>
> > >>>> Also note that iosf_mbi_block_punit_i2c_access() and
> > >>>> iosf_mbi_unblock_punit_i2c_access() do their own
> > >>>> ref/lock-counting to allow calling them multiple times and
> > >>>> the first block call takes the bus and the last unblock
> > >>>> call releases it.
> > >>>
> > >>> This is exactly what I was talking about above and also implemented
> > >>> within psp_acquire_i2c_bus() and psp_release_i2c_bus().
> > >>
> > >> Right, I was to quick in skimming over your code when
> > >> I wrote down my concerns about there being a deadlock
> > >> there, sorry.
> > >>
> > >> Regards,
> > >>
> > >> Hans

Re: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dąbroś]

czw., 20 sty 2022 o 18:12 Andy Shevchenko
<andriy.shevchenko@linux.intel.com> napisał(a):
>
> On Thu, Jan 20, 2022 at 11:33:05PM +0800, kernel test robot wrote:
>
> > If you fix the issue, kindly add following tag as appropriate
> > Reported-by: kernel test robot <lkp@intel.com>
> >
> > All errors (new ones prefixed by >>):
> >
> >    drivers/i2c/busses/i2c-designware-amdpsp.c: In function 'psp_send_cmd':
> > >> drivers/i2c/busses/i2c-designware-amdpsp.c:130:2: error: implicit declaration of function 'writeq'; did you mean 'writel'? [-Werror=implicit-function-declaration]
> >      130 |  writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr);
> >          |  ^~~~~~
> >          |  writel
> >    cc1: some warnings being treated as errors
>
> Adding io-64-nonatomic-lo-hi.h after io.h should fix this.

Correct, thanks! Actually io.h is directly included from
io-64-nonatomic-lo-hi.h.

Best Regards,
Jan


>
>
> --
> With Best Regards,
> Andy Shevchenko
>
>

Re: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dąbroś]

pt., 21 sty 2022 o 18:52 Andy Shevchenko
<andriy.shevchenko@linux.intel.com> napisał(a):
>
> On Fri, Jan 21, 2022 at 11:20:19AM +0100, Jan Dąbroś wrote:
> > czw., 20 sty 2022 o 15:21 Andy Shevchenko
> > <andriy.shevchenko@linux.intel.com> napisał(a):
> > > On Thu, Jan 20, 2022 at 01:16:21AM +0100, Jan Dabros wrote:
>
> ...
>
> > > > Add new entry in MAINTAINERS file to cover new module.
> > >
> > > It's confusing. You added yourself as a reviewer for I2C DesignWare driver,
> > > which is great, but not described in the commit message.
> >
> > Should I rephrase this sentence (to be more specific that I may be
> > helpful for reviewing amdpsp.c module) or rather you mean to exclude
> > drivers/i2c/busses/i2c-designware-amdpsp.c from the rest of
> > designware-* modules and create separate entry?
> >
> > Actually initially I wasn't planning to modify MAINTAINERS (after all
> > I'm not an I2C DesignWare expert) until run checkpatch.pl which
> > recommended to do so when adding new file. Eventually for me it made
> > some sense since I have a platform equipped with AMD Cezanne SoC and I
> > will be able to review and test potential changes in
> > i2c-designware-amdpsp.c or in general around semaphore areas.
> >
> > This may also work with different model, similar to how you pointed me
> > to Hans as an owner of Bay Trail platform who is acquinted with how
> > its i2c semaphore is working. I will simply remove myself from the
> > MAINTAINERS file and you can add me to the threads if there is
> > anything requiring my help.
> >
> > Let me know which way is working for you. I just thought it is not
> > good to leave you alone with a new module which you cannot actually
> > test and don't have deep insight about how PSP-x86 communications
> > works.
>
> You have a few options:
> - leave it for us (but it probably won't go well in long-term as you noticed)
> - add yourself as a reviewer (it doesn't require to review everything, but
>   you will get all i2c DesignWare driver changes)

I think this is the best option. So I will leave initial change to the
MAINTAINERS as is, but will put few more words of description into
commit message.

> - add a new MAINTAINERS database entry where you can put yourself for that
>   file even as a maintainaer
>
> ...
>
> > > > +#include <asm/msr.h>
> > >
> > > Usually linux/* followed by asm/*.
> > >
> > > > +#include <linux/i2c.h>
> > > > +#include <linux/psp-sev.h>
> > >
> > > types.h?
> >
> > I need to take a deeper look at the headers included here, especially
> > considering errors pointed by kernel test robot. Not sure why I
> > haven't seen any issues on my setup.
>
> The problem here is not so visible. Headers should be a compromise between
> what is really used and what we may include for that. There are headers
> that guaranteed to be included by others, otherwise it will be an implicit
> dependency which is not good in cases of generic headers, such as types.h.

ACK. I included couple of new headers in v2.

>
> ...
>
> > > > +union psp_req_buffer_hdr {
> > > > +     struct {
> > > > +             u32 total_size;
> > > > +             u32 status;
> > > > +     } __packed;
> > >
> > > What does packet bring you here?
> >
> > In this particular case binary-wise nothing, I can as well drop this.
> > It may be necessary if there are some changes in this structs fields
> > in future (e.g changing total_size to u8), since PSP expects every
> > field to be byte-aligned.
>
> _packed will make another thing which you probably won't need, it brings
> the entire structure to be possible on unaligned addresses and then some
> warnings from compiler may be issued even if there is no problem in the
> flow.
>
> Read this discussion: https://lore.kernel.org/linux-media/20220110224656.266536-1-sakari.ailus@linux.intel.com/

OK, I see your point - thanks for this pointer, something new for me.

>
> > > > +     u64 hdr_val;
> > >
> > > And why does this not have the same alignment since it's also part of
> > > the union?
> >
> > __packed is not about alignment of the whole struct/union
>
> It's. See above.

ACK.

>
> > but about
> > lack of padding between its fields. As above - in this particular case
> > with two u32 it doesn't matter.
>
> ...
>
> > > > +struct psp_i2c_req {
> > > > +     union psp_req_buffer_hdr hdr;
> > > > +     enum psp_i2c_req_type type;
> > >
> > > > +} __packed __aligned(32);
> > >
> > > Can you explain, what this means and how it's supposed to work?
> >
> > This means that each instance of the struct should be aligned (32)
> > while at the same time no padding within members - thus this may
> > result in non-aligned addresses of members.
>
> 32 bytes? And on unaligned address at the same time.

Yes, it was initial recommendation. I left this as is in v2, will work
with AMD to see if we can lower (get rid of) this requirement.

>
> ...
>
> > > > +union psp_mbox_cmd_reg {
> > > > +     struct psp_mbox_cmd_fields {
> > > > +             u16 mbox_status;
> > > > +             u8 mbox_cmd;
> > > > +             u8 reserved:6;
> > > > +             u8 recovery:1;
> > > > +             u8 ready:1;
> > >
> > > > +     } __packed fields;
> > >
> > > So, what is the __packed purpose here?
> >
> > As in all above cases - considering current layout of members and
> > their sizes dropping `__packed` will not results in any errors.
> >
> > However PSP expects all members os structs within shared buffers to be
> > byte-aligned, that's why I've added this attributes to be on the safe
> > side. If you think this doesn't make sense, I can remove them - in
> > (very unlikely) case of changes, one will need to add this specifier.
>
> I guess you don't need them at all in this case in any of the data structure
> you created here.

ACK, I removed unnecessary __packed attributes in v2.

> ...
>
> > > > +     struct psp_mbox *mbox = (struct psp_mbox *)mbox_iomem;
> > >
> > > Heck, no!
> >
> > I need to get acquinted to the kernel-reviewer language:) Pardon my
> > ignorance, but just to be sure I get what you mean here:
> > I'm using global mbox_iomem to keep address of PSP mailbox in IO
> > memory. Casting this to struct psp_mbox layout here, to make access
> > more convenient.
> > Your point here is that:
> > 1. I should move the assignment out from the variable declaration part
> > of this function;
> > 2. I should use ioremap/iounmap each time in psp_send_cmd instead of
> > using it once in `probe` and unmap in `remove`?
> > I thought about this option as to be less effective performance-wise
> > (even though I can get rid of one global variable).
> > 3. Something else?
>
> Casting an __iomem pointer to the some struct without keeping it. I believe
> sparse should blow because of this.

Oh right.. pretty obvious. Fixed.

>
> ...
>
> > > > +     /* Fill address of command-response buffer */
> > > > +     writeq((uintptr_t)__psp_pa((void *)req), &mbox->i2c_req_addr);
> > >
> > > What does this voodoo mean?!
> >
> > Basically I need to take physical address (__psp_pa) of request buffer
> > req and write this down into mailbox IO memory.
> > This should be spread into more lines with some comments, is this your point?
>
> It needs much better comment explaining what is this address and its meaning
> for the hardware and why you need physical address here (DMA?). For me it looks
> like a voodoo. Ah, and not using phys_addr_t / dma_addr_t / etc type here, but
> uintptr_t just adds a confusion.

ACK.

> ...
>
> > > > +     start = jiffies;
> > > > +     do {
> > > > +             if (psp_send_cmd(req)) {
> > > > +                     ret = -EIO;
> > > > +                     goto cleanup;
> > > > +             }
> > > > +
> > > > +             status = check_i2c_req_sts(req);
> > > > +             if (!status) {
> > > > +                     dev_dbg(psp_i2c_dev, "Request accepted by PSP after %ums\n",
> > > > +                             jiffies_to_msecs(jiffies - start));
> > > > +                     ret = 0;
> > > > +                     goto cleanup;
> > > > +             } else if (status == -EBUSY) {
> > > > +                     retry_cnt--;
> > > > +             } else {
> > > > +                     ret = -EIO;
> > > > +                     goto cleanup;
> > > > +             };
> > > > +
> > > > +             /* IF EBUSY, give PSP time to finish its i2c activities */
> > > > +             mdelay(PSP_I2C_REQ_RETRY_DELAY_MSEC);
> > > > +     } while (retry_cnt);
> > >
> > > NIH iopoll.h API(s).
> >
> > I don't think macros avaialble in iopoll.h are suitable here.
> > Procedure above is not about simply reading some IO and waiting for
> > particular condition to be met with this particular value. Eventually
> > `psp_send_cmd()` invokes `psp_wait_cmd()` where I'm using
> > `readl_poll_timeout()`, so on lower level I'm making use of this API.
> > However here I don't see any obvious method how to incorporate
> > iopoll.h API to reach my goal.
>
> You do not go to clean up if and only if status == -EBUSY, so here we have
> a condition. the rest can be moved to a function that you wrap by
> read_poll_timeout_atomic() (pay attention to the macro name).
> Here is the question, btw, why _atomic()? I.o.w. why mdelay() and not msleep()?

Right, this was another error - above code shouldn't be executed in
atomic context. I used read_poll_timeout() in v2.

> > > > +     ret = -ETIMEDOUT;
>
> ...
>
> > > Handle errors first.
>
> > Addressing above two comments - what do you think about below:
> > if (status) {
> >       if (status == -ETIMEDOUT)
> >                dev_err(psp_i2c_dev, "Timed out waiting for PSP to
> > release I2C bus\n");
> >       else
> >                dev_err(psp_i2c_dev, "PSP communication error\n");
>
> >        dev_err(psp_i2c_dev, "PSP communication error\n");
>
> This dup message is not needed, otherwise fine to me.

ACK.

>
> >        psp_i2c_mbox_fail = true;
> >        goto cleanup;
> > }
> >
> > psp_i2c_sem_acquired = jiffies;
> > psp_i2c_access_count++;
> > (...)
>
> ...
>
> > > > +static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
> > > > +{
> > > > +     int ret;
> > > > +     int i;
> > > > +
> > > > +     dev->semaphore_idx = -1;
> > > > +
> > > > +     for (i = 0; i < ARRAY_SIZE(i2c_dw_semaphore_cb_table); i++) {
> > >
> > > > +             if (!i2c_dw_semaphore_cb_table[i].probe)
> > > > +                     continue;
> > >
> > > Huh?
> >
> > Just to be sure I get your point.
> > Once I found terminating entry, I will get out of the loop and return
> > 0 as there are no semaphores to be "applied". Actually I should
> > probably use `break;` here as there shouldn't be a case when certain
> > type of semaphore installs without `probe` being implemented.
>
> Yes, that's what I though, and rather using ARRAY_SIZE(), use a terminator you
> provided.
>
> Originally you have used two approaches which seems competing with each other:
> - ARRAY_SIZE
> - terminator entry
>
> And on top of that so confusing continue on top of not-found ->probe() that
> makes me think what the meaning of the entry that has set ->remove(), but no
> ->probe().
>
> That said, I would rather see something like
>
>         struct ... *p = &...;
>
>         while (p->probe) {
>                 ...
>                 p++;
>         }

ACK.

> --
> With Best Regards,
> Andy Shevchenko
>
>

Re: [PATCH 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dąbroś]

śr., 26 sty 2022 o 20:36 Limonciello, Mario
<Mario.Limonciello@amd.com> napisał(a):
>
> [Public]
>
> > >
> > > >>>>
> > > >>>> Through here seems to all be generic code for accessing
> > > >>>> the AMD PSP. To me this seems like something which belongs
> > > >>>> in a separate AMD-PSP-mbox driver/lib, which can then be
> > > >>>> shared between other kernel drivers which may also want
> > > >>>> to access PSP.
> > > >>>
> > > >>> I see your point clearly and actually it is not an accident that I've
> > > >>> put all PSP-mailbox methods in one "block". They are logically
> > > >>> different than the rest of i2c-adapter specific methods.
> > > >>>
> > > >>> That being said, above PSP mailbox was created by AMD solely for the
> > > >>> purpose of i2c_arbitration. It has its own set of commands and
> > > >>> specific format of the command-response buffer. Thus it is not and it
> > > >>> won't be generic in the future. There are already upstreamed drivers
> > > >>> from AMD (under drivers/crypto/ccp/) which made use of PSP, however
> > > >>> their channel of communication looks completely different than the
> > > >>> very simple i2c_arbitration model implemented above.
> > > >>>
> > > >
> > > > Can you please double confirm no other users for this mailbox and it's for
> > > > you only?  And if so is it only specific to this platform that no other users?
> > > > At least on some UEFI AMD platforms that mailbox is defined for
> > > > communication with something else.  We might need some way to attest
> > > > from the firmware that it's safe to use.
> > > >
> > > > The only mailbox that is defined for OS use across different silicon AFAIK is
> > > > the GPU driver mailbox.  It may be safer to use that, but I'm not sure if
> > > > GPU driver has come up early enough for your use.
> > >
> > > The CCP/PSP driver will load as a PCIe device driver on this platform and
> > > will ioremap the MMIO space. Today, that driver doesn't reference those
> > > mailbox registers, and I don't know that there will be a need in the
> > > future. But if there is a need in the future, you could end up with a
> > > conflict between these two drivers.
> >
> > Right, so I have confirmed this with AMD PSP firmware developers, that
> > this particular x86-PSP mailbox is created and will be reserved
> > _solely_ for the purpose of i2c arbitration (and thus this driver).
> > There is no intend to use it elsewhere or share with another users.
>
> I've learned never to say never.  People move on to different roles and history
> gets lost.  As it's exclusive to this use case of I2C arbitration it will probably still
> be useful to leave in the comments nearby for our future selves what model/family
> SOC this was introduced in case the number of mailboxes are extinguished some
> day and this ends up being needed for a secondary purpose for some future SOC.

OK. I put a comment in v2. Please take a look.

>
> >
> > > Thanks,
> > > Tom
> > >
> > > >
> > > >>> Because of this I'm treating this as an i2c_semaphore-related code and
> > > >>> putting this in this module. In my opinion moving this into some
> > > >>> separate driver (which will be actually used only here) makes code
> > > >>> less clear. But let's also hear some voice from AMD.
> > > >>
> > > >> Since as you say this mailbox is special and only for i2c-arbitration,
> > > >> keeping it inside this patch / .c file is fine.
> > > >>
> > > >>>
> > > >>>>
> > > >>>> Sorta like the generic iosf_mbi_read() and
> > > >>>> iosf_mbi_write() functions from:
> > > >>>>
> > > >>>> arch/x86/platform/intel/iosf_mbi.c
> > > >>>>
> > > >>>> used on the Intel chips, which are also used outside of
> > > >>>> the I2C bus-locking code.
> > > >>>>
> > > >>>> This is also one of the reasons why I think it would be
> > > >>>> good to get some AMD folks involved in this, since they
> > > >>>> may be aware of other drivers which also need to access
> > > >>>> the PSP mbox.
> > > >>>>
> > > >>>
> > > >>> Right, I'm adding mario.limonciello@amd.com to the CC, so that he can
> > > >> comment.
> > > >>>
> > > >>> (...)
> > > >>>
> > > >>>>> +/*
> > > >>>>> + * Locking methods are based on the default implementation from
> > > >>>>> + * drivers/i2c/i2c-core.base.c, but with psp acquire and release
> > operations
> > > >>>>> + * added. With this in place we can ensure that i2c clients on the bus
> > shared
> > > >>>>> + * with psp are able to lock HW access to the bus for arbitrary number
> > of
> > > >>>>> + * operations - that is e.g. write-wait-read.
> > > >>>>> + */
> > > >>>>> +static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter,
> > > >>>>> +                                     unsigned int flags)
> > > >>>>> +{
> > > >>>>> +     psp_acquire_i2c_bus();
> > > >>>>> +     rt_mutex_lock_nested(&adapter->bus_lock,
> > > >> i2c_adapter_depth(adapter));
> > > >>>>
> > > >>>> This does not do what you think it does and you will still deadlock
> > > >>>> when things nest because of someone taking the bus-lock and then
> > > >>>> the main i2c-designware transfer function calling the acquire_lock
> > > >>>> callback.
> > > >>>
> > > >>> I haven't used rt_mutex_lock_nested() with the intent to prevent me
> > > >>> from deadlock when i2c-designware calls acquire_lock with bus-lock
> > > >>> already taken. This is a method copied from
> > > >>> drivers/i2c/i2c-core-base.c (BTW, I have a typo in above comment).
> > > >>> This is the default implementation applied by i2c-core when particular
> > > >>> adapter doesn't register its own locking callbacks - thus it is called
> > > >>> for i2c-designware for all platforms.
> > > >>>
> > > >>> In case of this driver internal i2c-designware acquire_lock() is equal
> > > >>> to psp_acquire_i2c_bus(). In other words, bus-level lock
> > > >>> i2c_adapter_dw_psp_lock_bus() is a superset of internal adapter's
> > > >>> acquire_lock().
> > > >>
> > > >> Ah I missed that this is just mimicking the core functions +
> > > >> an extra call to psp_acquire_i2c_bus().
> > > >>
> > > >> I assumed that the dwc->acquire callback path was also taking
> > > >> the mutex and I thought you had fallen for the _nested meaning
> > > >> something different then it does, my bad.
> > > >>
> > > >>> In order to prevent deadlock which you are talking about, I'm using
> > > >>> reference lock counter inside psp_acquire_i2c_bus() thus it is safe to
> > > >>> invoke acquire_lock() when bus-lock is already taken.
> > > >>
> > > >> Ah good, that is pretty much is the same as what the Bay Trail code
> > > >> is doing.
> > > >>
> > > >>>
> > > >>>>
> > > >>>> The _nested postfix is only for the lockdep lock-debugger, this
> > > >>>> actually turns into a regular mutex_lock when lockdep is not enabled:
> > > >>>>
> > > >>>> #ifdef CONFIG_DEBUG_LOCK_ALLOC
> > > >>>> extern void rt_mutex_lock_nested(struct rt_mutex *lock, unsigned int
> > > >> subclass);
> > > >>>> #define rt_mutex_lock(lock) rt_mutex_lock_nested(lock, 0)
> > > >>>> #else
> > > >>>> extern void rt_mutex_lock(struct rt_mutex *lock);
> > > >>>> #define rt_mutex_lock_nested(lock, subclass) rt_mutex_lock(lock)
> > > >>>> #endif
> > > >>>>
> > > >>>> The _nested postfix as such is only to tell the lockdep code that
> > > >>>> even though it seems we are trying to take the same mutex twice
> > > >>>> since in both cases it is of i2c_adapter.rt_mutex "lock class"
> > > >>>> that we are sure it is never the same i2c_adapter (but rather
> > > >>>> one which always gets called in a nested fashion from another
> > > >>>> i2c_adapter).
> > > >>>>
> > > >>>> IOW this only disables a false-positive lockdep warning, it does
> > > >>>> not allow taking the same mutex twice, you will still hang on
> > > >>>> the second mutex_lock call on the same lock.
> > > >>>
> > > >>> Thanks for the technical background about rt_mutex_lock_nested. I
> > > >>> think we should keep using it as is, since as I wrote above I don't
> > > >>> have any reasoning to modify it here.
> > > >>
> > > >> Ack, now that my misreading of the code has been cleared up
> > > >> I agree.
> > > >>
> > > >>>> Also I don't think you are allowed to use the bus_locking code
> > > >>>> like this. The i2c bus-locking code is intended to deal with
> > > >>>> busses which have muxes in them, where the mux must be set
> > > >>>> to the right branch of the bus to reach the client and then
> > > >>>> not be changed during the transfer to that client.
> > > >>>>
> > > >>>> So i2c-client drivers are never supposed to directly call
> > > >>>> the bus-locking functions.
> > > >>>
> > > >>> I think you are not correct here. There are examples of i2c-clients
> > > >>> which are using i2c bus_locking for the purpose of locking adapter for
> > > >>> the bunch of i2c transactions.
> > > >>>
> > > >>> As an example let's take drivers/char/tpm/tpm_tis_i2c_cr50.c. It
> > > >>> operates in write-wait-read model and there is i2c_lock_bus() call
> > > >>> used to ensure that bus won't be released -
> > > >>>
> > > >>
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c
> > %2F&amp;data=04%7C01%7CMario.Limonciello%40amd.com%7Cf8cd037a72b
> > 749fb003408d9df3f5ced%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0
> > %7C637786285501955886%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
> > MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata
> > =s4N9vTWbuRUww0qvsRz16fV%2Fnj41SxyLOB%2BjGFjyOaA%3D&amp;reserved
> > =0
> > > >>
> > om%2Ftorvalds%2Flinux%2Fblob%2Fmaster%2Fdrivers%2Fchar%2Ftpm%2Ftpm_
> > > >>
> > tis_i2c_cr50.c%23L202&amp;data=04%7C01%7Cmario.limonciello%40amd.com
> > > >>
> > %7C1bdc742bc2a24f59b7d908d9dcc95438%7C3dd8961fe4884e608e11a82d994
> > > >>
> > e183d%7C0%7C0%7C637783579554955840%7CUnknown%7CTWFpbGZsb3d8ey
> > > >>
> > JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
> > > >>
> > 3000&amp;sdata=fr0UEka%2BxYyPxqUG6oOZo%2Bc6pWgto2mD7hWA20YulVQ
> > > >> %3D&amp;reserved=0.
> > > >>>
> > > >>> Similar model is followed in drivers/char/tpm/tpm_i2c_infineon.c and
> > > >>> couple of other i2c-client drivers.
> > > >>
> > > >> Ah I see, interesting (live and learn).
> > > >>
> > > >> But this is then combined with using the special __i2c_transfer()
> > > >> function for the actual i2c reads/writes, since using the regular
> > > >> i2c_transfer() function after already taking the lock would deadlock.
> > > >>
> > > >> There is a similar unlocked raw __i2c_smbus_xfer(), but as the
> > > >> comment in include/linux/i2c.h above the locked i2c_smbus_xfer() says:
> > > >>
> > > >> /* This is the very generalized SMBus access routine. You probably do not
> > > >>     want to use this, though; one of the functions below may be much easier,
> > > >>     and probably just as fast.
> > > >>     Note that we use i2c_adapter here, because you do not need a specific
> > > >>     smbus adapter to call this function. */
> > > >> s32 i2c_smbus_xfer(...);
> > > >>
> > > >> So in this case a driver cannot use the usual
> > > >> i2c_smbus_read_byte/word/byte_data/word_data() helpers and
> > > >> the same for writes. Also using an i2c_regmap (which is used
> > > >> in a ton of places like PMIC drivers) will not work this way.
> > > >>
> > > >> So yes you can use i2c_bus_lock() for this; but only if all the
> > > >> drivers where you want to do that limit themselves to
> > > >> __i2c_transfer() and __i2c_smbus_xfer() calls and/or are
> > > >> rewritten to only use those.
> > > >>>> This is why in the Bay Trail case we have i2c-drivers
> > > >>>> directly calling iosf_mbi_block_punit_i2c_access() and
> > > >>>> iosf_mbi_unblock_punit_i2c_access() to lock the bus
> > > >>>> for multiple i2c-transfers. We can get away with this there
> > > >>>> because the bus in question is only used to access the
> > > >>>> PMIC and that PMIC is only used on Bay Trail (and CHT)
> > > >>>> boards, so the PMIC drivers can just hard-code these
> > > >>>> calls.
> > > >>>>
> > > >>>> If you need to take the PSP I2C semaphore for multiple
> > > >>>> transfers in some generic drivers, then I guess that the
> > > >>>> i2c-subsys will need to get some new i2c_adapter callbacks
> > > >>>> to acquire / release the bus for i2c-controllers where
> > > >>>> the bus/controller is shared with some co-processor like
> > > >>>> in the PSP case.
> > > >>>
> > > >>> This is exactly my intention to support generic i2c-clients drivers
> > > >>> without them being aware that i2c-adapter above is using some
> > > >>> semaphore/arbitration. Hopefully you can agree with me that currently
> > > >>> available bus_locking can be used and is enough for this purpose.
> > > >>
> > > >> It can be used, but with limitations, see above.
> > > >>
> > > >>>
> > > >>>> Also note that iosf_mbi_block_punit_i2c_access() and
> > > >>>> iosf_mbi_unblock_punit_i2c_access() do their own
> > > >>>> ref/lock-counting to allow calling them multiple times and
> > > >>>> the first block call takes the bus and the last unblock
> > > >>>> call releases it.
> > > >>>
> > > >>> This is exactly what I was talking about above and also implemented
> > > >>> within psp_acquire_i2c_bus() and psp_release_i2c_bus().
> > > >>
> > > >> Right, I was to quick in skimming over your code when
> > > >> I wrote down my concerns about there being a deadlock
> > > >> there, sorry.
> > > >>
> > > >> Regards,
> > > >>
> > > >> Hans

[PATCH v2 0/2] i2c-designware: Add support for AMD PSP semaphore

[Jan Dabros]

This patchset comprises support for new i2c-designware controller setup on some
AMD Cezanne SoCs, where x86 is sharing i2c bus with PSP. PSP uses the same
controller and acts as an i2c arbitrator there (x86 is leasing bus from it).

First commit aims to improve generic i2c-designware code by adding extra locking
on probe() and disable() paths. I would like to ask someone with access to
boards which use Intel BayTrail(CONFIG_I2C_DESIGNWARE_BAYTRAIL) to verify
behavior of my changes on such setup.

Second commit adds support for new PSP semaphore arbitration mechanism.
Implementation is similar to the one from i2c-designware-baytrail.c however
there are two main differences:
1) Add new ACPI ID in order to protect against silent binding of the old driver
to the setup with PSP semaphore. Extra flag ARBITRATION_SEMAPHORE added to this
new _HID allows to recognize setup with PSP.
2) Beside acquire_lock() and release_lock() methods we are also applying quirks
to the lock_bus() and unlock_bus() global adapter methods. With this in place
all i2c clients drivers may lock i2c bus for a desired number of i2c
transactions (e.g. write-wait-read) without being aware of that such bus is
shared with another entity.

This patchset is a follow-up to the RFC sent earlier on LKML [1], with review
comments applied.

Looking forward to some feedback.

[1] https://lkml.org/lkml/2021/12/22/219

v1 -> v2:
* Remove usage of unions
* Get rid of unnecessary __packed attributes
* Switch to use iopoll.h and bitfields.h APIs were applicable
* Follow the convention to check for the error first
* Reorder entries (includes, table entries) alphabetically
* Add necessary includes
* Add Kconfig dependency on X86_64
(above two fixes for "kernel test robot <lkp@intel.com>" issues)
* Modify probe() to use terminating entry for traversing through table
  instead of ARRAY_SIZE
* Fix typos in comments
* Rebase patchset

Jan Dabros (2):
  i2c: designware: Add missing locks
  i2c: designware: Add AMD PSP I2C bus support

 MAINTAINERS                                  |   1 +
 drivers/acpi/acpi_apd.c                      |   7 +-
 drivers/i2c/busses/Kconfig                   |  11 +
 drivers/i2c/busses/Makefile                  |   1 +
 drivers/i2c/busses/i2c-designware-amdpsp.c   | 373 +++++++++++++++++++
 drivers/i2c/busses/i2c-designware-baytrail.c |  10 +-
 drivers/i2c/busses/i2c-designware-common.c   |  12 +
 drivers/i2c/busses/i2c-designware-core.h     |  18 +-
 drivers/i2c/busses/i2c-designware-master.c   |   6 +
 drivers/i2c/busses/i2c-designware-platdrv.c  |  61 +++
 10 files changed, 489 insertions(+), 11 deletions(-)
 create mode 100644 drivers/i2c/busses/i2c-designware-amdpsp.c

-- 
2.35.0.rc0.227.g00780c9af4-goog

[PATCH v2 1/2] i2c: designware: Add missing locks

[Jan Dabros]

All accesses to controller's registers should be protected on
probe, disable and xfer paths. This is needed for i2c bus controllers
that are shared with but not controller by kernel.

Signed-off-by: Jan Dabros <jsd@semihalf.com>
Reviewed-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/i2c/busses/i2c-designware-common.c | 12 ++++++++++++
 drivers/i2c/busses/i2c-designware-master.c |  6 ++++++
 2 files changed, 18 insertions(+)

diff --git a/drivers/i2c/busses/i2c-designware-common.c b/drivers/i2c/busses/i2c-designware-common.c
index bf2a4920638a..9f8574320eb2 100644
--- a/drivers/i2c/busses/i2c-designware-common.c
+++ b/drivers/i2c/busses/i2c-designware-common.c
@@ -578,7 +578,12 @@ int i2c_dw_set_fifo_size(struct dw_i2c_dev *dev)
 	 * Try to detect the FIFO depth if not set by interface driver,
 	 * the depth could be from 2 to 256 from HW spec.
 	 */
+	ret = i2c_dw_acquire_lock(dev);
+	if (ret)
+		return ret;
+
 	ret = regmap_read(dev->map, DW_IC_COMP_PARAM_1, &param);
+	i2c_dw_release_lock(dev);
 	if (ret)
 		return ret;
 
@@ -607,6 +612,11 @@ u32 i2c_dw_func(struct i2c_adapter *adap)
 void i2c_dw_disable(struct dw_i2c_dev *dev)
 {
 	u32 dummy;
+	int ret;
+
+	ret = i2c_dw_acquire_lock(dev);
+	if (ret)
+		return;
 
 	/* Disable controller */
 	__i2c_dw_disable(dev);
@@ -614,6 +624,8 @@ void i2c_dw_disable(struct dw_i2c_dev *dev)
 	/* Disable all interrupts */
 	regmap_write(dev->map, DW_IC_INTR_MASK, 0);
 	regmap_read(dev->map, DW_IC_CLR_INTR, &dummy);
+
+	i2c_dw_release_lock(dev);
 }
 
 void i2c_dw_disable_int(struct dw_i2c_dev *dev)
diff --git a/drivers/i2c/busses/i2c-designware-master.c b/drivers/i2c/busses/i2c-designware-master.c
index 9177463c2cbb..1a4b23556db3 100644
--- a/drivers/i2c/busses/i2c-designware-master.c
+++ b/drivers/i2c/busses/i2c-designware-master.c
@@ -905,7 +905,13 @@ int i2c_dw_probe_master(struct dw_i2c_dev *dev)
 		irq_flags = IRQF_SHARED | IRQF_COND_SUSPEND;
 	}
 
+	ret = i2c_dw_acquire_lock(dev);
+	if (ret)
+		return ret;
+
 	i2c_dw_disable_int(dev);
+	i2c_dw_release_lock(dev);
+
 	ret = devm_request_irq(dev->dev, dev->irq, i2c_dw_isr, irq_flags,
 			       dev_name(dev->dev), dev);
 	if (ret) {
-- 
2.35.0.rc0.227.g00780c9af4-goog

[PATCH v2 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dabros]

Implement an I2C controller sharing mechanism between the host (kernel)
and PSP co-processor on some platforms equipped with AMD Cezanne SoC.

On these platforms we need to implement "software" i2c arbitration.
Default arbitration owner is PSP and kernel asks for acquire as well
as inform about release of the i2c bus via mailbox mechanism.

            +---------+
 <- ACQUIRE |         |
  +---------|   CPU   |\
  |         |         | \      +----------+  SDA
  |         +---------+  \     |          |-------
MAILBOX                   +--> |  I2C-DW  |  SCL
  |         +---------+        |          |-------
  |         |         |        +----------+
  +---------|   PSP   |
   <- ACK   |         |
            +---------+

            +---------+
 <- RELEASE |         |
  +---------|   CPU   |
  |         |         |        +----------+  SDA
  |         +---------+        |          |-------
MAILBOX                   +--> |  I2C-DW  |  SCL
  |         +---------+  /     |          |-------
  |         |         | /      +----------+
  +---------|   PSP   |/
   <- ACK   |         |
            +---------+

The solution is similar to i2c-designware-baytrail.c implementation, where
we are using a generic i2c-designware-* driver with a small "wrapper".

In contrary to baytrail semaphore implementation, beside internal
acquire_lock() and release_lock() methods we are also applying quirks to
lock_bus() and unlock_bus() global adapter methods. With this in place
all i2c clients drivers may lock i2c bus for a desired number of i2c
transactions (e.g. write-wait-read) without being aware of that such bus
is shared with another entity.

Modify i2c_dw_probe_lock_support() to select correct semaphore
implementation at runtime, since now we have more than one available.

Configure new matching ACPI ID "AMDI0019" and register
ARBITRATION_SEMAPHORE flag in order to distinguish setup with PSP
arbitration.

Add myself as a reviewer for I2C DesignWare in order to help with reviewing
and testing possible changes touching new i2c-designware-amdpsp.c module.

Signed-off-by: Jan Dabros <jsd@semihalf.com>
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: kernel test robot <lkp@intel.com>
---
 MAINTAINERS                                  |   1 +
 drivers/acpi/acpi_apd.c                      |   7 +-
 drivers/i2c/busses/Kconfig                   |  11 +
 drivers/i2c/busses/Makefile                  |   1 +
 drivers/i2c/busses/i2c-designware-amdpsp.c   | 373 +++++++++++++++++++
 drivers/i2c/busses/i2c-designware-baytrail.c |  10 +-
 drivers/i2c/busses/i2c-designware-core.h     |  18 +-
 drivers/i2c/busses/i2c-designware-platdrv.c  |  61 +++
 8 files changed, 471 insertions(+), 11 deletions(-)
 create mode 100644 drivers/i2c/busses/i2c-designware-amdpsp.c

diff --git a/MAINTAINERS b/MAINTAINERS
index f41088418aae..e0a7e05c5a41 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -18666,6 +18666,7 @@ SYNOPSYS DESIGNWARE I2C DRIVER
 M:	Jarkko Nikula <jarkko.nikula@linux.intel.com>
 R:	Andy Shevchenko <andriy.shevchenko@linux.intel.com>
 R:	Mika Westerberg <mika.westerberg@linux.intel.com>
+R:	Jan Dabros <jsd@semihalf.com>
 L:	linux-i2c@vger.kernel.org
 S:	Maintained
 F:	drivers/i2c/busses/i2c-designware-*
diff --git a/drivers/acpi/acpi_apd.c b/drivers/acpi/acpi_apd.c
index e7934ba79b02..ad245bbd965e 100644
--- a/drivers/acpi/acpi_apd.c
+++ b/drivers/acpi/acpi_apd.c
@@ -232,12 +232,13 @@ static const struct acpi_device_id acpi_apd_device_ids[] = {
 	/* Generic apd devices */
 #ifdef CONFIG_X86_AMD_PLATFORM_DEVICE
 	{ "AMD0010", APD_ADDR(cz_i2c_desc) },
-	{ "AMDI0010", APD_ADDR(wt_i2c_desc) },
 	{ "AMD0020", APD_ADDR(cz_uart_desc) },
-	{ "AMDI0020", APD_ADDR(cz_uart_desc) },
-	{ "AMDI0022", APD_ADDR(cz_uart_desc) },
 	{ "AMD0030", },
 	{ "AMD0040", APD_ADDR(fch_misc_desc)},
+	{ "AMDI0010", APD_ADDR(wt_i2c_desc) },
+	{ "AMDI0019", APD_ADDR(wt_i2c_desc) },
+	{ "AMDI0020", APD_ADDR(cz_uart_desc) },
+	{ "AMDI0022", APD_ADDR(cz_uart_desc) },
 	{ "HYGO0010", APD_ADDR(wt_i2c_desc) },
 #endif
 #ifdef CONFIG_ARM64
diff --git a/drivers/i2c/busses/Kconfig b/drivers/i2c/busses/Kconfig
index 42da31c1ab70..3ba9e7f9366e 100644
--- a/drivers/i2c/busses/Kconfig
+++ b/drivers/i2c/busses/Kconfig
@@ -553,6 +553,17 @@ config I2C_DESIGNWARE_PLATFORM
 	  This driver can also be built as a module.  If so, the module
 	  will be called i2c-designware-platform.
 
+config I2C_DESIGNWARE_AMDPSP
+	bool "AMD PSP I2C semaphore support"
+	depends on X86_64
+	depends on ACPI
+	depends on I2C_DESIGNWARE_PLATFORM
+	help
+	  This driver enables managed host access to the selected I2C bus shared
+	  between AMD CPU and AMD PSP.
+
+	  You should say Y if running on an AMD system equipped with the PSP.
+
 config I2C_DESIGNWARE_BAYTRAIL
 	bool "Intel Baytrail I2C semaphore support"
 	depends on ACPI
diff --git a/drivers/i2c/busses/Makefile b/drivers/i2c/busses/Makefile
index 1d00dce77098..752f47be3fc1 100644
--- a/drivers/i2c/busses/Makefile
+++ b/drivers/i2c/busses/Makefile
@@ -54,6 +54,7 @@ i2c-designware-core-y					+= i2c-designware-master.o
 i2c-designware-core-$(CONFIG_I2C_DESIGNWARE_SLAVE) 	+= i2c-designware-slave.o
 obj-$(CONFIG_I2C_DESIGNWARE_PLATFORM)			+= i2c-designware-platform.o
 i2c-designware-platform-y 				:= i2c-designware-platdrv.o
+i2c-designware-platform-$(CONFIG_I2C_DESIGNWARE_AMDPSP)	+= i2c-designware-amdpsp.o
 i2c-designware-platform-$(CONFIG_I2C_DESIGNWARE_BAYTRAIL) += i2c-designware-baytrail.o
 obj-$(CONFIG_I2C_DESIGNWARE_PCI)			+= i2c-designware-pci.o
 i2c-designware-pci-y					:= i2c-designware-pcidrv.o
diff --git a/drivers/i2c/busses/i2c-designware-amdpsp.c b/drivers/i2c/busses/i2c-designware-amdpsp.c
new file mode 100644
index 000000000000..9b600ca54f7c
--- /dev/null
+++ b/drivers/i2c/busses/i2c-designware-amdpsp.c
@@ -0,0 +1,373 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/bitfield.h>
+#include <linux/i2c.h>
+#include <linux/io-64-nonatomic-lo-hi.h>
+#include <linux/psp-sev.h>
+#include <linux/types.h>
+
+#include <asm/msr.h>
+
+#include "i2c-designware-core.h"
+
+#define MSR_AMD_PSP_ADDR	0xc00110a2
+#define PSP_MBOX_OFFSET		0x10570
+#define PSP_CMD_TIMEOUT_MS	500
+
+#define PSP_I2C_REQ_BUS_CMD		0x64
+#define PSP_I2C_REQ_RETRY_CNT		10
+#define PSP_I2C_REQ_RETRY_DELAY_USEC	(50 * 1000)
+#define PSP_I2C_REQ_STS_OK		0x0
+#define PSP_I2C_REQ_STS_BUS_BUSY	0x1
+#define PSP_I2C_REQ_STS_INV_PARAM	0x3
+
+#define PSP_MBOX_FIELDS_STS		GENMASK(15, 0)
+#define PSP_MBOX_FIELDS_CMD		GENMASK(23, 16)
+#define PSP_MBOX_FIELDS_RESERVED	GENMASK(29, 24)
+#define PSP_MBOX_FIELDS_RECOVERY	BIT(30)
+#define PSP_MBOX_FIELDS_READY		BIT(31)
+
+struct psp_req_buffer_hdr {
+	u32 total_size;
+	u32 status;
+};
+
+enum psp_i2c_req_type {
+	PSP_I2C_REQ_ACQUIRE,
+	PSP_I2C_REQ_RELEASE,
+	PSP_I2C_REQ_MAX
+};
+
+struct psp_i2c_req {
+	struct psp_req_buffer_hdr hdr;
+	enum psp_i2c_req_type type;
+} __aligned(32);
+
+struct psp_mbox {
+	u32 cmd_fields;
+	phys_addr_t i2c_req_addr;
+} __packed;
+
+static DEFINE_MUTEX(psp_i2c_access_mutex);
+static unsigned long psp_i2c_sem_acquired;
+static void __iomem *mbox_iomem;
+static u32 psp_i2c_access_count;
+static bool psp_i2c_mbox_fail;
+static struct device *psp_i2c_dev;
+
+/*
+ * Implementation of PSP-x86 i2c-arbitration mailbox introduced for AMD Cezanne
+ * family of SoCs.
+ */
+
+static int psp_get_mbox_addr(unsigned long *mbox_addr)
+{
+	unsigned long long psp_mmio;
+
+	if (rdmsrl_safe(MSR_AMD_PSP_ADDR, &psp_mmio))
+		return -EIO;
+
+	*mbox_addr = (unsigned long)(psp_mmio + PSP_MBOX_OFFSET);
+
+	return 0;
+}
+
+static int psp_mbox_probe(void)
+{
+	unsigned long mbox_addr;
+	int ret;
+
+	ret = psp_get_mbox_addr(&mbox_addr);
+	if (ret)
+		return ret;
+
+	mbox_iomem = ioremap(mbox_addr, sizeof(struct psp_mbox));
+	if (!mbox_iomem)
+		return -ENOMEM;
+
+	return 0;
+}
+
+
+/* Recovery field should be equal 0 to start sending commands */
+static int psp_check_mbox_recovery(struct psp_mbox __iomem *mbox)
+{
+	u32 tmp;
+
+	tmp = readl(&mbox->cmd_fields);
+
+	return FIELD_GET(PSP_MBOX_FIELDS_RECOVERY, tmp);
+}
+
+static int psp_wait_cmd(struct psp_mbox __iomem *mbox)
+{
+	u32 tmp, expected;
+
+	/* Expect mbox_cmd to be cleared and ready bit to be set by PSP */
+	expected = FIELD_PREP(PSP_MBOX_FIELDS_READY, 1);
+
+	return readl_poll_timeout(&mbox->cmd_fields, tmp, (tmp == expected),
+				  0, 1000 * PSP_CMD_TIMEOUT_MS);
+}
+
+/* Status equal to 0 means that PSP succeed processing command */
+static int psp_check_mbox_sts(struct psp_mbox __iomem *mbox)
+{
+	u32 cmd_reg;
+
+	cmd_reg = readl(&mbox->cmd_fields);
+
+	return FIELD_GET(PSP_MBOX_FIELDS_STS, cmd_reg);
+}
+
+static int psp_send_cmd(struct psp_i2c_req *req)
+{
+	struct psp_mbox __iomem *mbox = (struct psp_mbox __iomem *)mbox_iomem;
+	phys_addr_t req_addr;
+	u32 cmd_reg;
+
+	if (psp_check_mbox_recovery(mbox))
+		return -EIO;
+
+	if (psp_wait_cmd(mbox))
+		return -EBUSY;
+
+	/*
+	 * Fill mailbox with address of command-response buffer, which will be
+	 * used for sending i2c requests as well as reading status returned by
+	 * PSP. Use physical address of buffer, since PSP will map this region.
+	 */
+	req_addr = __psp_pa((void *)req);
+	writeq(req_addr, &mbox->i2c_req_addr);
+
+	/* Write command register to trigger processing */
+	cmd_reg = FIELD_PREP(PSP_MBOX_FIELDS_CMD, PSP_I2C_REQ_BUS_CMD);
+	writel(cmd_reg, &mbox->cmd_fields);
+
+	if (psp_wait_cmd(mbox))
+		return -ETIMEDOUT;
+
+	if (psp_check_mbox_sts(mbox))
+		return -EIO;
+
+	return 0;
+}
+
+/* Helper to verify status returned by PSP */
+static int check_i2c_req_sts(struct psp_i2c_req *req)
+{
+	int status;
+
+	status = readl(&req->hdr.status);
+
+	switch (status) {
+	case PSP_I2C_REQ_STS_OK:
+		return 0;
+	case PSP_I2C_REQ_STS_BUS_BUSY:
+		return -EBUSY;
+	case PSP_I2C_REQ_STS_INV_PARAM:
+	default:
+		return -EIO;
+	};
+}
+
+static int psp_send_check_i2c_req(struct psp_i2c_req *req)
+{
+	if (psp_send_cmd(req))
+		return -EIO;
+
+	return check_i2c_req_sts(req);
+}
+
+static int psp_send_i2c_req(enum psp_i2c_req_type i2c_req_type)
+{
+	struct psp_i2c_req *req;
+	unsigned long start;
+	int status, ret;
+
+	/* Allocate command-response buffer */
+	req = kzalloc(sizeof(*req), GFP_KERNEL);
+	if (!req)
+		return -ENOMEM;
+
+	req->hdr.total_size = sizeof(*req);
+	req->type = i2c_req_type;
+
+	start = jiffies;
+	ret = read_poll_timeout(psp_send_check_i2c_req, status,
+				(status != -EBUSY),
+				PSP_I2C_REQ_RETRY_DELAY_USEC,
+				PSP_I2C_REQ_RETRY_CNT * PSP_I2C_REQ_RETRY_DELAY_USEC,
+				0, req);
+	if (ret)
+		goto cleanup;
+
+	if (status) {
+		ret = status;
+		goto cleanup;
+	}
+
+	dev_dbg(psp_i2c_dev, "Request accepted by PSP after %ums\n",
+		jiffies_to_msecs(jiffies - start));
+
+cleanup:
+	kfree(req);
+	return ret;
+}
+
+static int psp_acquire_i2c_bus(void)
+{
+	int status;
+
+	mutex_lock(&psp_i2c_access_mutex);
+
+	/* Return early if mailbox malfunctioned */
+	if (psp_i2c_mbox_fail)
+		goto cleanup;
+
+	/*
+	 * Simply increment usage counter and return if PSP semaphore was
+	 * already taken by kernel.
+	 */
+	if (psp_i2c_access_count > 0) {
+		psp_i2c_access_count++;
+		goto cleanup;
+	};
+
+	status = psp_send_i2c_req(PSP_I2C_REQ_ACQUIRE);
+	if (status) {
+		if (status == -ETIMEDOUT)
+			dev_err(psp_i2c_dev, "Timed out waiting for PSP to release I2C bus\n");
+		else
+			dev_err(psp_i2c_dev, "PSP communication error\n");
+
+		dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n");
+		psp_i2c_mbox_fail = true;
+		goto cleanup;
+	}
+
+	psp_i2c_sem_acquired = jiffies;
+	psp_i2c_access_count++;
+
+cleanup:
+	mutex_unlock(&psp_i2c_access_mutex);
+	return 0;
+}
+
+static void psp_release_i2c_bus(void)
+{
+	int status;
+
+	mutex_lock(&psp_i2c_access_mutex);
+
+	/* Return early if mailbox was malfunctional */
+	if (psp_i2c_mbox_fail)
+		goto cleanup;
+
+	/*
+	 * If we are last owner of PSP semaphore, need to release aribtration
+	 * via mailbox.
+	 */
+	psp_i2c_access_count--;
+	if (psp_i2c_access_count > 0)
+		goto cleanup;
+
+	/* Send a release command to PSP */
+	status = psp_send_i2c_req(PSP_I2C_REQ_RELEASE);
+	if (status) {
+		if (status == -ETIMEDOUT)
+			dev_err(psp_i2c_dev, "Timed out waiting for PSP to acquire I2C bus\n");
+		else
+			dev_err(psp_i2c_dev, "PSP communication error\n");
+
+		dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n");
+		psp_i2c_mbox_fail = true;
+		goto cleanup;
+	}
+
+	dev_dbg(psp_i2c_dev, "PSP semaphore held for %ums\n",
+		jiffies_to_msecs(jiffies - psp_i2c_sem_acquired));
+
+cleanup:
+	mutex_unlock(&psp_i2c_access_mutex);
+}
+
+/*
+ * Locking methods are based on the default implementation from
+ * drivers/i2c/i2c-core-base.c, but with psp acquire and release operations
+ * added. With this in place we can ensure that i2c clients on the bus shared
+ * with psp are able to lock HW access to the bus for arbitrary number of
+ * operations - that is e.g. write-wait-read.
+ */
+static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter,
+					unsigned int flags)
+{
+	psp_acquire_i2c_bus();
+	rt_mutex_lock_nested(&adapter->bus_lock, i2c_adapter_depth(adapter));
+}
+
+static int i2c_adapter_dw_psp_trylock_bus(struct i2c_adapter *adapter,
+					  unsigned int flags)
+{
+	int ret;
+
+	ret = rt_mutex_trylock(&adapter->bus_lock);
+	if (ret)
+		return ret;
+
+	psp_acquire_i2c_bus();
+
+	return ret;
+}
+
+static void i2c_adapter_dw_psp_unlock_bus(struct i2c_adapter *adapter,
+					  unsigned int flags)
+{
+	psp_release_i2c_bus();
+	rt_mutex_unlock(&adapter->bus_lock);
+}
+
+static const struct i2c_lock_operations i2c_dw_psp_lock_ops = {
+	.lock_bus = i2c_adapter_dw_psp_lock_bus,
+	.trylock_bus = i2c_adapter_dw_psp_trylock_bus,
+	.unlock_bus = i2c_adapter_dw_psp_unlock_bus,
+};
+
+int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev)
+{
+	int ret;
+
+	if (!dev || !dev->dev)
+		return -ENODEV;
+
+	if (!(dev->flags & ARBITRATION_SEMAPHORE))
+		return -ENODEV;
+
+	/* Allow to bind only one instance of a driver */
+	if (psp_i2c_dev)
+		return -EEXIST;
+
+	psp_i2c_dev = dev->dev;
+
+	ret = psp_mbox_probe();
+	if (ret)
+		return ret;
+
+	dev_info(psp_i2c_dev, "I2C bus managed by AMD PSP\n");
+
+	/*
+	 * Install global locking callbacks for adapter as well as internal i2c
+	 * controller locks.
+	 */
+	dev->adapter.lock_ops = &i2c_dw_psp_lock_ops;
+	dev->acquire_lock = psp_acquire_i2c_bus;
+	dev->release_lock = psp_release_i2c_bus;
+
+	return 0;
+}
+
+/* Unmap area used as a mailbox with PSP */
+void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev)
+{
+	iounmap(mbox_iomem);
+}
diff --git a/drivers/i2c/busses/i2c-designware-baytrail.c b/drivers/i2c/busses/i2c-designware-baytrail.c
index c6a7a00e1d52..0c674542dd99 100644
--- a/drivers/i2c/busses/i2c-designware-baytrail.c
+++ b/drivers/i2c/busses/i2c-designware-baytrail.c
@@ -12,25 +12,25 @@
 
 #include "i2c-designware-core.h"
 
-int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
+int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev)
 {
 	acpi_status status;
 	unsigned long long shared_host = 0;
 	acpi_handle handle;
 
 	if (!dev || !dev->dev)
-		return 0;
+		return -ENODEV;
 
 	handle = ACPI_HANDLE(dev->dev);
 	if (!handle)
-		return 0;
+		return -ENODEV;
 
 	status = acpi_evaluate_integer(handle, "_SEM", NULL, &shared_host);
 	if (ACPI_FAILURE(status))
-		return 0;
+		return -ENODEV;
 
 	if (!shared_host)
-		return 0;
+		return -ENODEV;
 
 	if (!iosf_mbi_available())
 		return -EPROBE_DEFER;
diff --git a/drivers/i2c/busses/i2c-designware-core.h b/drivers/i2c/busses/i2c-designware-core.h
index 4b26cba40139..1d65212fddbd 100644
--- a/drivers/i2c/busses/i2c-designware-core.h
+++ b/drivers/i2c/busses/i2c-designware-core.h
@@ -227,6 +227,8 @@ struct reset_control;
  * @hs_lcnt: high speed LCNT value
  * @acquire_lock: function to acquire a hardware lock on the bus
  * @release_lock: function to release a hardware lock on the bus
+ * @semaphore_idx: Index of table with semaphore type attached to the bus. It's
+ *	-1 if there is no semaphore.
  * @shared_with_punit: true if this bus is shared with the SoCs PUNIT
  * @disable: function to disable the controller
  * @disable_int: function to disable all interrupts
@@ -285,6 +287,7 @@ struct dw_i2c_dev {
 	u16			hs_lcnt;
 	int			(*acquire_lock)(void);
 	void			(*release_lock)(void);
+	int			semaphore_idx;
 	bool			shared_with_punit;
 	void			(*disable)(struct dw_i2c_dev *dev);
 	void			(*disable_int)(struct dw_i2c_dev *dev);
@@ -297,6 +300,7 @@ struct dw_i2c_dev {
 
 #define ACCESS_INTR_MASK	BIT(0)
 #define ACCESS_NO_IRQ_SUSPEND	BIT(1)
+#define ARBITRATION_SEMAPHORE	BIT(2)
 
 #define MODEL_MSCC_OCELOT	BIT(8)
 #define MODEL_BAIKAL_BT1	BIT(9)
@@ -310,6 +314,11 @@ struct dw_i2c_dev {
 #define AMD_UCSI_INTR_REG	0x474
 #define AMD_UCSI_INTR_EN	0xd
 
+struct i2c_dw_semaphore_callbacks {
+	int	(*probe)(struct dw_i2c_dev *dev);
+	void	(*remove)(struct dw_i2c_dev *dev);
+};
+
 int i2c_dw_init_regmap(struct dw_i2c_dev *dev);
 u32 i2c_dw_scl_hcnt(u32 ic_clk, u32 tSYMBOL, u32 tf, int cond, int offset);
 u32 i2c_dw_scl_lcnt(u32 ic_clk, u32 tLOW, u32 tf, int offset);
@@ -370,9 +379,12 @@ static inline void i2c_dw_configure(struct dw_i2c_dev *dev)
 }
 
 #if IS_ENABLED(CONFIG_I2C_DESIGNWARE_BAYTRAIL)
-extern int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev);
-#else
-static inline int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev) { return 0; }
+int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev);
+#endif
+
+#if IS_ENABLED(CONFIG_I2C_DESIGNWARE_AMDPSP)
+int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev);
+void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev);
 #endif
 
 int i2c_dw_validate_speed(struct dw_i2c_dev *dev);
diff --git a/drivers/i2c/busses/i2c-designware-platdrv.c b/drivers/i2c/busses/i2c-designware-platdrv.c
index 2bd81abc86f6..7cb555d6c2e8 100644
--- a/drivers/i2c/busses/i2c-designware-platdrv.c
+++ b/drivers/i2c/busses/i2c-designware-platdrv.c
@@ -50,6 +50,7 @@ static const struct acpi_device_id dw_i2c_acpi_match[] = {
 	{ "808622C1", ACCESS_NO_IRQ_SUSPEND },
 	{ "AMD0010", ACCESS_INTR_MASK },
 	{ "AMDI0010", ACCESS_INTR_MASK },
+	{ "AMDI0019", ACCESS_INTR_MASK | ARBITRATION_SEMAPHORE },
 	{ "AMDI0510", 0 },
 	{ "APMC0D0F", 0 },
 	{ "HISI02A1", 0 },
@@ -204,6 +205,64 @@ static const struct dmi_system_id dw_i2c_hwmon_class_dmi[] = {
 	{ } /* terminate list */
 };
 
+static const struct i2c_dw_semaphore_callbacks i2c_dw_semaphore_cb_table[] = {
+#ifdef CONFIG_I2C_DESIGNWARE_BAYTRAIL
+	{
+		.probe = i2c_dw_baytrail_probe_lock_support,
+	},
+#endif
+#ifdef CONFIG_I2C_DESIGNWARE_AMDPSP
+	{
+		.probe = i2c_dw_amdpsp_probe_lock_support,
+		.remove = i2c_dw_amdpsp_remove_lock_support,
+	},
+#endif
+	{}
+};
+
+static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
+{
+	const struct i2c_dw_semaphore_callbacks *ptr;
+	int i = 0;
+	int ret;
+
+	ptr = i2c_dw_semaphore_cb_table;
+
+	dev->semaphore_idx = -1;
+
+	while (ptr->probe) {
+		ret = ptr->probe(dev);
+		if (ret) {
+			/*
+			 * If there is no semaphore device attached to this
+			 * controller, we shouldn't abort general i2c_controller
+			 * probe.
+			 */
+			if (ret == -ENODEV) {
+				i++;
+				ptr++;
+				continue;
+			} else {
+				return ret;
+			}
+		}
+
+		dev->semaphore_idx = i;
+		break;
+	}
+
+	return 0;
+}
+
+static void i2c_dw_remove_lock_support(struct dw_i2c_dev *dev)
+{
+	if (dev->semaphore_idx < 0)
+		return;
+
+	if (i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove)
+		i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove(dev);
+}
+
 static int dw_i2c_plat_probe(struct platform_device *pdev)
 {
 	struct i2c_adapter *adap;
@@ -334,6 +393,8 @@ static int dw_i2c_plat_remove(struct platform_device *pdev)
 	pm_runtime_put_sync(&pdev->dev);
 	dw_i2c_plat_pm_cleanup(dev);
 
+	i2c_dw_remove_lock_support(dev);
+
 	reset_control_assert(dev->rst);
 
 	return 0;
-- 
2.35.0.rc0.227.g00780c9af4-goog

Re: [PATCH v2 0/2] i2c-designware: Add support for AMD PSP semaphore

[Jan Dąbroś]

Hi,

Adding proper Andy's email address (and removing wrong one) in the
whole patchset. Sorry for noise!

Best Regards,
Jan


pt., 28 sty 2022 o 15:48 Jan Dabros <jsd@semihalf.com> napisał(a):
>
> This patchset comprises support for new i2c-designware controller setup on some
> AMD Cezanne SoCs, where x86 is sharing i2c bus with PSP. PSP uses the same
> controller and acts as an i2c arbitrator there (x86 is leasing bus from it).
>
> First commit aims to improve generic i2c-designware code by adding extra locking
> on probe() and disable() paths. I would like to ask someone with access to
> boards which use Intel BayTrail(CONFIG_I2C_DESIGNWARE_BAYTRAIL) to verify
> behavior of my changes on such setup.
>
> Second commit adds support for new PSP semaphore arbitration mechanism.
> Implementation is similar to the one from i2c-designware-baytrail.c however
> there are two main differences:
> 1) Add new ACPI ID in order to protect against silent binding of the old driver
> to the setup with PSP semaphore. Extra flag ARBITRATION_SEMAPHORE added to this
> new _HID allows to recognize setup with PSP.
> 2) Beside acquire_lock() and release_lock() methods we are also applying quirks
> to the lock_bus() and unlock_bus() global adapter methods. With this in place
> all i2c clients drivers may lock i2c bus for a desired number of i2c
> transactions (e.g. write-wait-read) without being aware of that such bus is
> shared with another entity.
>
> This patchset is a follow-up to the RFC sent earlier on LKML [1], with review
> comments applied.
>
> Looking forward to some feedback.
>
> [1] https://lkml.org/lkml/2021/12/22/219
>
> v1 -> v2:
> * Remove usage of unions
> * Get rid of unnecessary __packed attributes
> * Switch to use iopoll.h and bitfields.h APIs were applicable
> * Follow the convention to check for the error first
> * Reorder entries (includes, table entries) alphabetically
> * Add necessary includes
> * Add Kconfig dependency on X86_64
> (above two fixes for "kernel test robot <lkp@intel.com>" issues)
> * Modify probe() to use terminating entry for traversing through table
>   instead of ARRAY_SIZE
> * Fix typos in comments
> * Rebase patchset
>
> Jan Dabros (2):
>   i2c: designware: Add missing locks
>   i2c: designware: Add AMD PSP I2C bus support
>
>  MAINTAINERS                                  |   1 +
>  drivers/acpi/acpi_apd.c                      |   7 +-
>  drivers/i2c/busses/Kconfig                   |  11 +
>  drivers/i2c/busses/Makefile                  |   1 +
>  drivers/i2c/busses/i2c-designware-amdpsp.c   | 373 +++++++++++++++++++
>  drivers/i2c/busses/i2c-designware-baytrail.c |  10 +-
>  drivers/i2c/busses/i2c-designware-common.c   |  12 +
>  drivers/i2c/busses/i2c-designware-core.h     |  18 +-
>  drivers/i2c/busses/i2c-designware-master.c   |   6 +
>  drivers/i2c/busses/i2c-designware-platdrv.c  |  61 +++
>  10 files changed, 489 insertions(+), 11 deletions(-)
>  create mode 100644 drivers/i2c/busses/i2c-designware-amdpsp.c
>
> --
> 2.35.0.rc0.227.g00780c9af4-goog
>

Re: [PATCH v2 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dąbroś]

Hi,

Adding proper Andy's email address (and removing wrong one) in the
whole patchset. Sorry for noise!

Best Regards,
Jan

pt., 28 sty 2022 o 15:48 Jan Dabros <jsd@semihalf.com> napisał(a):
>
> Implement an I2C controller sharing mechanism between the host (kernel)
> and PSP co-processor on some platforms equipped with AMD Cezanne SoC.
>
> On these platforms we need to implement "software" i2c arbitration.
> Default arbitration owner is PSP and kernel asks for acquire as well
> as inform about release of the i2c bus via mailbox mechanism.
>
>             +---------+
>  <- ACQUIRE |         |
>   +---------|   CPU   |\
>   |         |         | \      +----------+  SDA
>   |         +---------+  \     |          |-------
> MAILBOX                   +--> |  I2C-DW  |  SCL
>   |         +---------+        |          |-------
>   |         |         |        +----------+
>   +---------|   PSP   |
>    <- ACK   |         |
>             +---------+
>
>             +---------+
>  <- RELEASE |         |
>   +---------|   CPU   |
>   |         |         |        +----------+  SDA
>   |         +---------+        |          |-------
> MAILBOX                   +--> |  I2C-DW  |  SCL
>   |         +---------+  /     |          |-------
>   |         |         | /      +----------+
>   +---------|   PSP   |/
>    <- ACK   |         |
>             +---------+
>
> The solution is similar to i2c-designware-baytrail.c implementation, where
> we are using a generic i2c-designware-* driver with a small "wrapper".
>
> In contrary to baytrail semaphore implementation, beside internal
> acquire_lock() and release_lock() methods we are also applying quirks to
> lock_bus() and unlock_bus() global adapter methods. With this in place
> all i2c clients drivers may lock i2c bus for a desired number of i2c
> transactions (e.g. write-wait-read) without being aware of that such bus
> is shared with another entity.
>
> Modify i2c_dw_probe_lock_support() to select correct semaphore
> implementation at runtime, since now we have more than one available.
>
> Configure new matching ACPI ID "AMDI0019" and register
> ARBITRATION_SEMAPHORE flag in order to distinguish setup with PSP
> arbitration.
>
> Add myself as a reviewer for I2C DesignWare in order to help with reviewing
> and testing possible changes touching new i2c-designware-amdpsp.c module.
>
> Signed-off-by: Jan Dabros <jsd@semihalf.com>
> Reported-by: kernel test robot <lkp@intel.com>
> Reported-by: kernel test robot <lkp@intel.com>
> ---
>  MAINTAINERS                                  |   1 +
>  drivers/acpi/acpi_apd.c                      |   7 +-
>  drivers/i2c/busses/Kconfig                   |  11 +
>  drivers/i2c/busses/Makefile                  |   1 +
>  drivers/i2c/busses/i2c-designware-amdpsp.c   | 373 +++++++++++++++++++
>  drivers/i2c/busses/i2c-designware-baytrail.c |  10 +-
>  drivers/i2c/busses/i2c-designware-core.h     |  18 +-
>  drivers/i2c/busses/i2c-designware-platdrv.c  |  61 +++
>  8 files changed, 471 insertions(+), 11 deletions(-)
>  create mode 100644 drivers/i2c/busses/i2c-designware-amdpsp.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index f41088418aae..e0a7e05c5a41 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -18666,6 +18666,7 @@ SYNOPSYS DESIGNWARE I2C DRIVER
>  M:     Jarkko Nikula <jarkko.nikula@linux.intel.com>
>  R:     Andy Shevchenko <andriy.shevchenko@linux.intel.com>
>  R:     Mika Westerberg <mika.westerberg@linux.intel.com>
> +R:     Jan Dabros <jsd@semihalf.com>
>  L:     linux-i2c@vger.kernel.org
>  S:     Maintained
>  F:     drivers/i2c/busses/i2c-designware-*
> diff --git a/drivers/acpi/acpi_apd.c b/drivers/acpi/acpi_apd.c
> index e7934ba79b02..ad245bbd965e 100644
> --- a/drivers/acpi/acpi_apd.c
> +++ b/drivers/acpi/acpi_apd.c
> @@ -232,12 +232,13 @@ static const struct acpi_device_id acpi_apd_device_ids[] = {
>         /* Generic apd devices */
>  #ifdef CONFIG_X86_AMD_PLATFORM_DEVICE
>         { "AMD0010", APD_ADDR(cz_i2c_desc) },
> -       { "AMDI0010", APD_ADDR(wt_i2c_desc) },
>         { "AMD0020", APD_ADDR(cz_uart_desc) },
> -       { "AMDI0020", APD_ADDR(cz_uart_desc) },
> -       { "AMDI0022", APD_ADDR(cz_uart_desc) },
>         { "AMD0030", },
>         { "AMD0040", APD_ADDR(fch_misc_desc)},
> +       { "AMDI0010", APD_ADDR(wt_i2c_desc) },
> +       { "AMDI0019", APD_ADDR(wt_i2c_desc) },
> +       { "AMDI0020", APD_ADDR(cz_uart_desc) },
> +       { "AMDI0022", APD_ADDR(cz_uart_desc) },
>         { "HYGO0010", APD_ADDR(wt_i2c_desc) },
>  #endif
>  #ifdef CONFIG_ARM64
> diff --git a/drivers/i2c/busses/Kconfig b/drivers/i2c/busses/Kconfig
> index 42da31c1ab70..3ba9e7f9366e 100644
> --- a/drivers/i2c/busses/Kconfig
> +++ b/drivers/i2c/busses/Kconfig
> @@ -553,6 +553,17 @@ config I2C_DESIGNWARE_PLATFORM
>           This driver can also be built as a module.  If so, the module
>           will be called i2c-designware-platform.
>
> +config I2C_DESIGNWARE_AMDPSP
> +       bool "AMD PSP I2C semaphore support"
> +       depends on X86_64
> +       depends on ACPI
> +       depends on I2C_DESIGNWARE_PLATFORM
> +       help
> +         This driver enables managed host access to the selected I2C bus shared
> +         between AMD CPU and AMD PSP.
> +
> +         You should say Y if running on an AMD system equipped with the PSP.
> +
>  config I2C_DESIGNWARE_BAYTRAIL
>         bool "Intel Baytrail I2C semaphore support"
>         depends on ACPI
> diff --git a/drivers/i2c/busses/Makefile b/drivers/i2c/busses/Makefile
> index 1d00dce77098..752f47be3fc1 100644
> --- a/drivers/i2c/busses/Makefile
> +++ b/drivers/i2c/busses/Makefile
> @@ -54,6 +54,7 @@ i2c-designware-core-y                                 += i2c-designware-master.o
>  i2c-designware-core-$(CONFIG_I2C_DESIGNWARE_SLAVE)     += i2c-designware-slave.o
>  obj-$(CONFIG_I2C_DESIGNWARE_PLATFORM)                  += i2c-designware-platform.o
>  i2c-designware-platform-y                              := i2c-designware-platdrv.o
> +i2c-designware-platform-$(CONFIG_I2C_DESIGNWARE_AMDPSP)        += i2c-designware-amdpsp.o
>  i2c-designware-platform-$(CONFIG_I2C_DESIGNWARE_BAYTRAIL) += i2c-designware-baytrail.o
>  obj-$(CONFIG_I2C_DESIGNWARE_PCI)                       += i2c-designware-pci.o
>  i2c-designware-pci-y                                   := i2c-designware-pcidrv.o
> diff --git a/drivers/i2c/busses/i2c-designware-amdpsp.c b/drivers/i2c/busses/i2c-designware-amdpsp.c
> new file mode 100644
> index 000000000000..9b600ca54f7c
> --- /dev/null
> +++ b/drivers/i2c/busses/i2c-designware-amdpsp.c
> @@ -0,0 +1,373 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +#include <linux/bitfield.h>
> +#include <linux/i2c.h>
> +#include <linux/io-64-nonatomic-lo-hi.h>
> +#include <linux/psp-sev.h>
> +#include <linux/types.h>
> +
> +#include <asm/msr.h>
> +
> +#include "i2c-designware-core.h"
> +
> +#define MSR_AMD_PSP_ADDR       0xc00110a2
> +#define PSP_MBOX_OFFSET                0x10570
> +#define PSP_CMD_TIMEOUT_MS     500
> +
> +#define PSP_I2C_REQ_BUS_CMD            0x64
> +#define PSP_I2C_REQ_RETRY_CNT          10
> +#define PSP_I2C_REQ_RETRY_DELAY_USEC   (50 * 1000)
> +#define PSP_I2C_REQ_STS_OK             0x0
> +#define PSP_I2C_REQ_STS_BUS_BUSY       0x1
> +#define PSP_I2C_REQ_STS_INV_PARAM      0x3
> +
> +#define PSP_MBOX_FIELDS_STS            GENMASK(15, 0)
> +#define PSP_MBOX_FIELDS_CMD            GENMASK(23, 16)
> +#define PSP_MBOX_FIELDS_RESERVED       GENMASK(29, 24)
> +#define PSP_MBOX_FIELDS_RECOVERY       BIT(30)
> +#define PSP_MBOX_FIELDS_READY          BIT(31)
> +
> +struct psp_req_buffer_hdr {
> +       u32 total_size;
> +       u32 status;
> +};
> +
> +enum psp_i2c_req_type {
> +       PSP_I2C_REQ_ACQUIRE,
> +       PSP_I2C_REQ_RELEASE,
> +       PSP_I2C_REQ_MAX
> +};
> +
> +struct psp_i2c_req {
> +       struct psp_req_buffer_hdr hdr;
> +       enum psp_i2c_req_type type;
> +} __aligned(32);
> +
> +struct psp_mbox {
> +       u32 cmd_fields;
> +       phys_addr_t i2c_req_addr;
> +} __packed;
> +
> +static DEFINE_MUTEX(psp_i2c_access_mutex);
> +static unsigned long psp_i2c_sem_acquired;
> +static void __iomem *mbox_iomem;
> +static u32 psp_i2c_access_count;
> +static bool psp_i2c_mbox_fail;
> +static struct device *psp_i2c_dev;
> +
> +/*
> + * Implementation of PSP-x86 i2c-arbitration mailbox introduced for AMD Cezanne
> + * family of SoCs.
> + */
> +
> +static int psp_get_mbox_addr(unsigned long *mbox_addr)
> +{
> +       unsigned long long psp_mmio;
> +
> +       if (rdmsrl_safe(MSR_AMD_PSP_ADDR, &psp_mmio))
> +               return -EIO;
> +
> +       *mbox_addr = (unsigned long)(psp_mmio + PSP_MBOX_OFFSET);
> +
> +       return 0;
> +}
> +
> +static int psp_mbox_probe(void)
> +{
> +       unsigned long mbox_addr;
> +       int ret;
> +
> +       ret = psp_get_mbox_addr(&mbox_addr);
> +       if (ret)
> +               return ret;
> +
> +       mbox_iomem = ioremap(mbox_addr, sizeof(struct psp_mbox));
> +       if (!mbox_iomem)
> +               return -ENOMEM;
> +
> +       return 0;
> +}
> +
> +
> +/* Recovery field should be equal 0 to start sending commands */
> +static int psp_check_mbox_recovery(struct psp_mbox __iomem *mbox)
> +{
> +       u32 tmp;
> +
> +       tmp = readl(&mbox->cmd_fields);
> +
> +       return FIELD_GET(PSP_MBOX_FIELDS_RECOVERY, tmp);
> +}
> +
> +static int psp_wait_cmd(struct psp_mbox __iomem *mbox)
> +{
> +       u32 tmp, expected;
> +
> +       /* Expect mbox_cmd to be cleared and ready bit to be set by PSP */
> +       expected = FIELD_PREP(PSP_MBOX_FIELDS_READY, 1);
> +
> +       return readl_poll_timeout(&mbox->cmd_fields, tmp, (tmp == expected),
> +                                 0, 1000 * PSP_CMD_TIMEOUT_MS);
> +}
> +
> +/* Status equal to 0 means that PSP succeed processing command */
> +static int psp_check_mbox_sts(struct psp_mbox __iomem *mbox)
> +{
> +       u32 cmd_reg;
> +
> +       cmd_reg = readl(&mbox->cmd_fields);
> +
> +       return FIELD_GET(PSP_MBOX_FIELDS_STS, cmd_reg);
> +}
> +
> +static int psp_send_cmd(struct psp_i2c_req *req)
> +{
> +       struct psp_mbox __iomem *mbox = (struct psp_mbox __iomem *)mbox_iomem;
> +       phys_addr_t req_addr;
> +       u32 cmd_reg;
> +
> +       if (psp_check_mbox_recovery(mbox))
> +               return -EIO;
> +
> +       if (psp_wait_cmd(mbox))
> +               return -EBUSY;
> +
> +       /*
> +        * Fill mailbox with address of command-response buffer, which will be
> +        * used for sending i2c requests as well as reading status returned by
> +        * PSP. Use physical address of buffer, since PSP will map this region.
> +        */
> +       req_addr = __psp_pa((void *)req);
> +       writeq(req_addr, &mbox->i2c_req_addr);
> +
> +       /* Write command register to trigger processing */
> +       cmd_reg = FIELD_PREP(PSP_MBOX_FIELDS_CMD, PSP_I2C_REQ_BUS_CMD);
> +       writel(cmd_reg, &mbox->cmd_fields);
> +
> +       if (psp_wait_cmd(mbox))
> +               return -ETIMEDOUT;
> +
> +       if (psp_check_mbox_sts(mbox))
> +               return -EIO;
> +
> +       return 0;
> +}
> +
> +/* Helper to verify status returned by PSP */
> +static int check_i2c_req_sts(struct psp_i2c_req *req)
> +{
> +       int status;
> +
> +       status = readl(&req->hdr.status);
> +
> +       switch (status) {
> +       case PSP_I2C_REQ_STS_OK:
> +               return 0;
> +       case PSP_I2C_REQ_STS_BUS_BUSY:
> +               return -EBUSY;
> +       case PSP_I2C_REQ_STS_INV_PARAM:
> +       default:
> +               return -EIO;
> +       };
> +}
> +
> +static int psp_send_check_i2c_req(struct psp_i2c_req *req)
> +{
> +       if (psp_send_cmd(req))
> +               return -EIO;
> +
> +       return check_i2c_req_sts(req);
> +}
> +
> +static int psp_send_i2c_req(enum psp_i2c_req_type i2c_req_type)
> +{
> +       struct psp_i2c_req *req;
> +       unsigned long start;
> +       int status, ret;
> +
> +       /* Allocate command-response buffer */
> +       req = kzalloc(sizeof(*req), GFP_KERNEL);
> +       if (!req)
> +               return -ENOMEM;
> +
> +       req->hdr.total_size = sizeof(*req);
> +       req->type = i2c_req_type;
> +
> +       start = jiffies;
> +       ret = read_poll_timeout(psp_send_check_i2c_req, status,
> +                               (status != -EBUSY),
> +                               PSP_I2C_REQ_RETRY_DELAY_USEC,
> +                               PSP_I2C_REQ_RETRY_CNT * PSP_I2C_REQ_RETRY_DELAY_USEC,
> +                               0, req);
> +       if (ret)
> +               goto cleanup;
> +
> +       if (status) {
> +               ret = status;
> +               goto cleanup;
> +       }
> +
> +       dev_dbg(psp_i2c_dev, "Request accepted by PSP after %ums\n",
> +               jiffies_to_msecs(jiffies - start));
> +
> +cleanup:
> +       kfree(req);
> +       return ret;
> +}
> +
> +static int psp_acquire_i2c_bus(void)
> +{
> +       int status;
> +
> +       mutex_lock(&psp_i2c_access_mutex);
> +
> +       /* Return early if mailbox malfunctioned */
> +       if (psp_i2c_mbox_fail)
> +               goto cleanup;
> +
> +       /*
> +        * Simply increment usage counter and return if PSP semaphore was
> +        * already taken by kernel.
> +        */
> +       if (psp_i2c_access_count > 0) {
> +               psp_i2c_access_count++;
> +               goto cleanup;
> +       };
> +
> +       status = psp_send_i2c_req(PSP_I2C_REQ_ACQUIRE);
> +       if (status) {
> +               if (status == -ETIMEDOUT)
> +                       dev_err(psp_i2c_dev, "Timed out waiting for PSP to release I2C bus\n");
> +               else
> +                       dev_err(psp_i2c_dev, "PSP communication error\n");
> +
> +               dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n");
> +               psp_i2c_mbox_fail = true;
> +               goto cleanup;
> +       }
> +
> +       psp_i2c_sem_acquired = jiffies;
> +       psp_i2c_access_count++;
> +
> +cleanup:
> +       mutex_unlock(&psp_i2c_access_mutex);
> +       return 0;
> +}
> +
> +static void psp_release_i2c_bus(void)
> +{
> +       int status;
> +
> +       mutex_lock(&psp_i2c_access_mutex);
> +
> +       /* Return early if mailbox was malfunctional */
> +       if (psp_i2c_mbox_fail)
> +               goto cleanup;
> +
> +       /*
> +        * If we are last owner of PSP semaphore, need to release aribtration
> +        * via mailbox.
> +        */
> +       psp_i2c_access_count--;
> +       if (psp_i2c_access_count > 0)
> +               goto cleanup;
> +
> +       /* Send a release command to PSP */
> +       status = psp_send_i2c_req(PSP_I2C_REQ_RELEASE);
> +       if (status) {
> +               if (status == -ETIMEDOUT)
> +                       dev_err(psp_i2c_dev, "Timed out waiting for PSP to acquire I2C bus\n");
> +               else
> +                       dev_err(psp_i2c_dev, "PSP communication error\n");
> +
> +               dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n");
> +               psp_i2c_mbox_fail = true;
> +               goto cleanup;
> +       }
> +
> +       dev_dbg(psp_i2c_dev, "PSP semaphore held for %ums\n",
> +               jiffies_to_msecs(jiffies - psp_i2c_sem_acquired));
> +
> +cleanup:
> +       mutex_unlock(&psp_i2c_access_mutex);
> +}
> +
> +/*
> + * Locking methods are based on the default implementation from
> + * drivers/i2c/i2c-core-base.c, but with psp acquire and release operations
> + * added. With this in place we can ensure that i2c clients on the bus shared
> + * with psp are able to lock HW access to the bus for arbitrary number of
> + * operations - that is e.g. write-wait-read.
> + */
> +static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter,
> +                                       unsigned int flags)
> +{
> +       psp_acquire_i2c_bus();
> +       rt_mutex_lock_nested(&adapter->bus_lock, i2c_adapter_depth(adapter));
> +}
> +
> +static int i2c_adapter_dw_psp_trylock_bus(struct i2c_adapter *adapter,
> +                                         unsigned int flags)
> +{
> +       int ret;
> +
> +       ret = rt_mutex_trylock(&adapter->bus_lock);
> +       if (ret)
> +               return ret;
> +
> +       psp_acquire_i2c_bus();
> +
> +       return ret;
> +}
> +
> +static void i2c_adapter_dw_psp_unlock_bus(struct i2c_adapter *adapter,
> +                                         unsigned int flags)
> +{
> +       psp_release_i2c_bus();
> +       rt_mutex_unlock(&adapter->bus_lock);
> +}
> +
> +static const struct i2c_lock_operations i2c_dw_psp_lock_ops = {
> +       .lock_bus = i2c_adapter_dw_psp_lock_bus,
> +       .trylock_bus = i2c_adapter_dw_psp_trylock_bus,
> +       .unlock_bus = i2c_adapter_dw_psp_unlock_bus,
> +};
> +
> +int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev)
> +{
> +       int ret;
> +
> +       if (!dev || !dev->dev)
> +               return -ENODEV;
> +
> +       if (!(dev->flags & ARBITRATION_SEMAPHORE))
> +               return -ENODEV;
> +
> +       /* Allow to bind only one instance of a driver */
> +       if (psp_i2c_dev)
> +               return -EEXIST;
> +
> +       psp_i2c_dev = dev->dev;
> +
> +       ret = psp_mbox_probe();
> +       if (ret)
> +               return ret;
> +
> +       dev_info(psp_i2c_dev, "I2C bus managed by AMD PSP\n");
> +
> +       /*
> +        * Install global locking callbacks for adapter as well as internal i2c
> +        * controller locks.
> +        */
> +       dev->adapter.lock_ops = &i2c_dw_psp_lock_ops;
> +       dev->acquire_lock = psp_acquire_i2c_bus;
> +       dev->release_lock = psp_release_i2c_bus;
> +
> +       return 0;
> +}
> +
> +/* Unmap area used as a mailbox with PSP */
> +void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev)
> +{
> +       iounmap(mbox_iomem);
> +}
> diff --git a/drivers/i2c/busses/i2c-designware-baytrail.c b/drivers/i2c/busses/i2c-designware-baytrail.c
> index c6a7a00e1d52..0c674542dd99 100644
> --- a/drivers/i2c/busses/i2c-designware-baytrail.c
> +++ b/drivers/i2c/busses/i2c-designware-baytrail.c
> @@ -12,25 +12,25 @@
>
>  #include "i2c-designware-core.h"
>
> -int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
> +int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev)
>  {
>         acpi_status status;
>         unsigned long long shared_host = 0;
>         acpi_handle handle;
>
>         if (!dev || !dev->dev)
> -               return 0;
> +               return -ENODEV;
>
>         handle = ACPI_HANDLE(dev->dev);
>         if (!handle)
> -               return 0;
> +               return -ENODEV;
>
>         status = acpi_evaluate_integer(handle, "_SEM", NULL, &shared_host);
>         if (ACPI_FAILURE(status))
> -               return 0;
> +               return -ENODEV;
>
>         if (!shared_host)
> -               return 0;
> +               return -ENODEV;
>
>         if (!iosf_mbi_available())
>                 return -EPROBE_DEFER;
> diff --git a/drivers/i2c/busses/i2c-designware-core.h b/drivers/i2c/busses/i2c-designware-core.h
> index 4b26cba40139..1d65212fddbd 100644
> --- a/drivers/i2c/busses/i2c-designware-core.h
> +++ b/drivers/i2c/busses/i2c-designware-core.h
> @@ -227,6 +227,8 @@ struct reset_control;
>   * @hs_lcnt: high speed LCNT value
>   * @acquire_lock: function to acquire a hardware lock on the bus
>   * @release_lock: function to release a hardware lock on the bus
> + * @semaphore_idx: Index of table with semaphore type attached to the bus. It's
> + *     -1 if there is no semaphore.
>   * @shared_with_punit: true if this bus is shared with the SoCs PUNIT
>   * @disable: function to disable the controller
>   * @disable_int: function to disable all interrupts
> @@ -285,6 +287,7 @@ struct dw_i2c_dev {
>         u16                     hs_lcnt;
>         int                     (*acquire_lock)(void);
>         void                    (*release_lock)(void);
> +       int                     semaphore_idx;
>         bool                    shared_with_punit;
>         void                    (*disable)(struct dw_i2c_dev *dev);
>         void                    (*disable_int)(struct dw_i2c_dev *dev);
> @@ -297,6 +300,7 @@ struct dw_i2c_dev {
>
>  #define ACCESS_INTR_MASK       BIT(0)
>  #define ACCESS_NO_IRQ_SUSPEND  BIT(1)
> +#define ARBITRATION_SEMAPHORE  BIT(2)
>
>  #define MODEL_MSCC_OCELOT      BIT(8)
>  #define MODEL_BAIKAL_BT1       BIT(9)
> @@ -310,6 +314,11 @@ struct dw_i2c_dev {
>  #define AMD_UCSI_INTR_REG      0x474
>  #define AMD_UCSI_INTR_EN       0xd
>
> +struct i2c_dw_semaphore_callbacks {
> +       int     (*probe)(struct dw_i2c_dev *dev);
> +       void    (*remove)(struct dw_i2c_dev *dev);
> +};
> +
>  int i2c_dw_init_regmap(struct dw_i2c_dev *dev);
>  u32 i2c_dw_scl_hcnt(u32 ic_clk, u32 tSYMBOL, u32 tf, int cond, int offset);
>  u32 i2c_dw_scl_lcnt(u32 ic_clk, u32 tLOW, u32 tf, int offset);
> @@ -370,9 +379,12 @@ static inline void i2c_dw_configure(struct dw_i2c_dev *dev)
>  }
>
>  #if IS_ENABLED(CONFIG_I2C_DESIGNWARE_BAYTRAIL)
> -extern int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev);
> -#else
> -static inline int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev) { return 0; }
> +int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev);
> +#endif
> +
> +#if IS_ENABLED(CONFIG_I2C_DESIGNWARE_AMDPSP)
> +int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev);
> +void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev);
>  #endif
>
>  int i2c_dw_validate_speed(struct dw_i2c_dev *dev);
> diff --git a/drivers/i2c/busses/i2c-designware-platdrv.c b/drivers/i2c/busses/i2c-designware-platdrv.c
> index 2bd81abc86f6..7cb555d6c2e8 100644
> --- a/drivers/i2c/busses/i2c-designware-platdrv.c
> +++ b/drivers/i2c/busses/i2c-designware-platdrv.c
> @@ -50,6 +50,7 @@ static const struct acpi_device_id dw_i2c_acpi_match[] = {
>         { "808622C1", ACCESS_NO_IRQ_SUSPEND },
>         { "AMD0010", ACCESS_INTR_MASK },
>         { "AMDI0010", ACCESS_INTR_MASK },
> +       { "AMDI0019", ACCESS_INTR_MASK | ARBITRATION_SEMAPHORE },
>         { "AMDI0510", 0 },
>         { "APMC0D0F", 0 },
>         { "HISI02A1", 0 },
> @@ -204,6 +205,64 @@ static const struct dmi_system_id dw_i2c_hwmon_class_dmi[] = {
>         { } /* terminate list */
>  };
>
> +static const struct i2c_dw_semaphore_callbacks i2c_dw_semaphore_cb_table[] = {
> +#ifdef CONFIG_I2C_DESIGNWARE_BAYTRAIL
> +       {
> +               .probe = i2c_dw_baytrail_probe_lock_support,
> +       },
> +#endif
> +#ifdef CONFIG_I2C_DESIGNWARE_AMDPSP
> +       {
> +               .probe = i2c_dw_amdpsp_probe_lock_support,
> +               .remove = i2c_dw_amdpsp_remove_lock_support,
> +       },
> +#endif
> +       {}
> +};
> +
> +static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
> +{
> +       const struct i2c_dw_semaphore_callbacks *ptr;
> +       int i = 0;
> +       int ret;
> +
> +       ptr = i2c_dw_semaphore_cb_table;
> +
> +       dev->semaphore_idx = -1;
> +
> +       while (ptr->probe) {
> +               ret = ptr->probe(dev);
> +               if (ret) {
> +                       /*
> +                        * If there is no semaphore device attached to this
> +                        * controller, we shouldn't abort general i2c_controller
> +                        * probe.
> +                        */
> +                       if (ret == -ENODEV) {
> +                               i++;
> +                               ptr++;
> +                               continue;
> +                       } else {
> +                               return ret;
> +                       }
> +               }
> +
> +               dev->semaphore_idx = i;
> +               break;
> +       }
> +
> +       return 0;
> +}
> +
> +static void i2c_dw_remove_lock_support(struct dw_i2c_dev *dev)
> +{
> +       if (dev->semaphore_idx < 0)
> +               return;
> +
> +       if (i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove)
> +               i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove(dev);
> +}
> +
>  static int dw_i2c_plat_probe(struct platform_device *pdev)
>  {
>         struct i2c_adapter *adap;
> @@ -334,6 +393,8 @@ static int dw_i2c_plat_remove(struct platform_device *pdev)
>         pm_runtime_put_sync(&pdev->dev);
>         dw_i2c_plat_pm_cleanup(dev);
>
> +       i2c_dw_remove_lock_support(dev);
> +
>         reset_control_assert(dev->rst);
>
>         return 0;
> --
> 2.35.0.rc0.227.g00780c9af4-goog
>

Re: [PATCH v2 2/2] i2c: designware: Add AMD PSP I2C bus support

[Andy Shevchenko]

On Fri, Jan 28, 2022 at 03:59:40PM +0100, Jan Dąbroś wrote:
> Hi,
> 
> Adding proper Andy's email address (and removing wrong one) in the
> whole patchset. Sorry for noise!

Thanks!

> pt., 28 sty 2022 o 15:48 Jan Dabros <jsd@semihalf.com> napisał(a):
> >
> > Implement an I2C controller sharing mechanism between the host (kernel)
> > and PSP co-processor on some platforms equipped with AMD Cezanne SoC.
> >
> > On these platforms we need to implement "software" i2c arbitration.
> > Default arbitration owner is PSP and kernel asks for acquire as well
> > as inform about release of the i2c bus via mailbox mechanism.
> >
> >             +---------+
> >  <- ACQUIRE |         |
> >   +---------|   CPU   |\
> >   |         |         | \      +----------+  SDA
> >   |         +---------+  \     |          |-------
> > MAILBOX                   +--> |  I2C-DW  |  SCL
> >   |         +---------+        |          |-------
> >   |         |         |        +----------+
> >   +---------|   PSP   |
> >    <- ACK   |         |
> >             +---------+
> >
> >             +---------+
> >  <- RELEASE |         |
> >   +---------|   CPU   |
> >   |         |         |        +----------+  SDA
> >   |         +---------+        |          |-------
> > MAILBOX                   +--> |  I2C-DW  |  SCL
> >   |         +---------+  /     |          |-------
> >   |         |         | /      +----------+
> >   +---------|   PSP   |/
> >    <- ACK   |         |
> >             +---------+
> >
> > The solution is similar to i2c-designware-baytrail.c implementation, where
> > we are using a generic i2c-designware-* driver with a small "wrapper".
> >
> > In contrary to baytrail semaphore implementation, beside internal
> > acquire_lock() and release_lock() methods we are also applying quirks to
> > lock_bus() and unlock_bus() global adapter methods. With this in place
> > all i2c clients drivers may lock i2c bus for a desired number of i2c
> > transactions (e.g. write-wait-read) without being aware of that such bus
> > is shared with another entity.
> >
> > Modify i2c_dw_probe_lock_support() to select correct semaphore
> > implementation at runtime, since now we have more than one available.
> >
> > Configure new matching ACPI ID "AMDI0019" and register
> > ARBITRATION_SEMAPHORE flag in order to distinguish setup with PSP
> > arbitration.
> >
> > Add myself as a reviewer for I2C DesignWare in order to help with reviewing
> > and testing possible changes touching new i2c-designware-amdpsp.c module.
> >
> > Signed-off-by: Jan Dabros <jsd@semihalf.com>

> > Reported-by: kernel test robot <lkp@intel.com>
> > Reported-by: kernel test robot <lkp@intel.com>

New feature can't be reported.
If you want to give a credit to CI, do it in changelog.

...

> > +       depends on X86_64

Not sure if it's better than using non-atomic IO helpers.
At least you can't run 32-bit kernels on that platforms
in order to get this functionality working. Doest it mean
those platforms do not have 32-bit compatibility mode
anymore?

...

> > +#include <linux/io-64-nonatomic-lo-hi.h>

Ah, this is not needed if you keep code running exclusively on 64-bit
platforms.

...

> > +struct psp_mbox {
> > +       u32 cmd_fields;

> > +       phys_addr_t i2c_req_addr;

But phys_addr_t is platform-dependent type. Perhaps you meant to use u64 here
always?

> > +} __packed;

...

> > +       struct psp_mbox __iomem *mbox = (struct psp_mbox __iomem *)mbox_iomem;

For void * pointers the cast is implied, i.o.w. it's not needed here.

...

> > +static int psp_send_check_i2c_req(struct psp_i2c_req *req)
> > +{
> > +       if (psp_send_cmd(req))

> > +               return -EIO;

Why is error code shadowed?

> > +       return check_i2c_req_sts(req);
> > +}

...

> > +cleanup:
> > +       mutex_unlock(&psp_i2c_access_mutex);
> > +       return 0;

Not sure I understand why we ignore all above errors here.

...

> > +       if (!dev || !dev->dev)
> > +               return -ENODEV;

At which circumstances may we get
	dev != NULL
	dev->dev == NULL
?

...

> >         if (!dev || !dev->dev)
> > -               return 0;
> > +               return -ENODEV;

I see the same here, perhaps Hans knows the answer :-)

...

> > +static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
> > +{
> > +       const struct i2c_dw_semaphore_callbacks *ptr;
> > +       int i = 0;
> > +       int ret;
> > +
> > +       ptr = i2c_dw_semaphore_cb_table;
> > +
> > +       dev->semaphore_idx = -1;
> > +
> > +       while (ptr->probe) {
> > +               ret = ptr->probe(dev);
> > +               if (ret) {

> > +                       /*
> > +                        * If there is no semaphore device attached to this
> > +                        * controller, we shouldn't abort general i2c_controller
> > +                        * probe.
> > +                        */
> > +                       if (ret == -ENODEV) {
> > +                               i++;
> > +                               ptr++;
> > +                               continue;
> > +                       } else {

Redundant 'else', but see below.

> > +                               return ret;
> > +                       }

May it be

	    if (ret != -ENODEV)
	        return ret;

	    i++;
	    ptr++;
	    continue;

?

> > +               }
> > +
> > +               dev->semaphore_idx = i;
> > +               break;
> > +       }
> > +
> > +       return 0;
> > +}

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v2 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dąbroś]

Aargh.. so if this won't be enough to use wrong email address in v2 -
I not used plain text above. Mailing list (understandably) aren't
happy with this, thus resending my answers to Andy.. Again sorry for
noise.


pt., 28 sty 2022 o 16:50 Andy Shevchenko
<andriy.shevchenko@linux.intel.com> napisał(a):
>
> On Fri, Jan 28, 2022 at 03:59:40PM +0100, Jan Dąbroś wrote:
> > Hi,
> >
> > Adding proper Andy's email address (and removing wrong one) in the
> > whole patchset. Sorry for noise!
>
> Thanks!
>
> > pt., 28 sty 2022 o 15:48 Jan Dabros <jsd@semihalf.com> napisał(a):
> > >
> > > Implement an I2C controller sharing mechanism between the host (kernel)
> > > and PSP co-processor on some platforms equipped with AMD Cezanne SoC.
> > >
> > > On these platforms we need to implement "software" i2c arbitration.
> > > Default arbitration owner is PSP and kernel asks for acquire as well
> > > as inform about release of the i2c bus via mailbox mechanism.
> > >
> > >             +---------+
> > >  <- ACQUIRE |         |
> > >   +---------|   CPU   |\
> > >   |         |         | \      +----------+  SDA
> > >   |         +---------+  \     |          |-------
> > > MAILBOX                   +--> |  I2C-DW  |  SCL
> > >   |         +---------+        |          |-------
> > >   |         |         |        +----------+
> > >   +---------|   PSP   |
> > >    <- ACK   |         |
> > >             +---------+
> > >
> > >             +---------+
> > >  <- RELEASE |         |
> > >   +---------|   CPU   |
> > >   |         |         |        +----------+  SDA
> > >   |         +---------+        |          |-------
> > > MAILBOX                   +--> |  I2C-DW  |  SCL
> > >   |         +---------+  /     |          |-------
> > >   |         |         | /      +----------+
> > >   +---------|   PSP   |/
> > >    <- ACK   |         |
> > >             +---------+
> > >
> > > The solution is similar to i2c-designware-baytrail.c implementation, where
> > > we are using a generic i2c-designware-* driver with a small "wrapper".
> > >
> > > In contrary to baytrail semaphore implementation, beside internal
> > > acquire_lock() and release_lock() methods we are also applying quirks to
> > > lock_bus() and unlock_bus() global adapter methods. With this in place
> > > all i2c clients drivers may lock i2c bus for a desired number of i2c
> > > transactions (e.g. write-wait-read) without being aware of that such bus
> > > is shared with another entity.
> > >
> > > Modify i2c_dw_probe_lock_support() to select correct semaphore
> > > implementation at runtime, since now we have more than one available.
> > >
> > > Configure new matching ACPI ID "AMDI0019" and register
> > > ARBITRATION_SEMAPHORE flag in order to distinguish setup with PSP
> > > arbitration.
> > >
> > > Add myself as a reviewer for I2C DesignWare in order to help with reviewing
> > > and testing possible changes touching new i2c-designware-amdpsp.c module.
> > >
> > > Signed-off-by: Jan Dabros <jsd@semihalf.com>
>
> > > Reported-by: kernel test robot <lkp@intel.com>
> > > Reported-by: kernel test robot <lkp@intel.com>
>
> New feature can't be reported.
> If you want to give a credit to CI, do it in changelog.

OK, will remove this.

> ...
>
> > > +       depends on X86_64
>
> Not sure if it's better than using non-atomic IO helpers.

There are 2 issues reported by kernel robot for my patchset:
1. Lack of <asm/msr.h>;
2. Missing declaration for 'writeq'.
Actually above was my idea to fix first issue, but please see below.

> At least you can't run 32-bit kernels on that platforms
> in order to get this functionality working. Doest it mean
> those platforms do not have 32-bit compatibility mode
> anymore?

Correct, I was focusing too much on my use case, where I'm building
only 64-bit. This isn't right. Furthermore I should rather use
dependency on CONFIG_X86_MSR which is better suited for ensuring above
msr.h header is present.

>
> ...
>
> > > +#include <linux/io-64-nonatomic-lo-hi.h>
>
> Ah, this is not needed if you keep code running exclusively on 64-bit
> platforms.

Will keep this, since switching to "depends on X86_MSR".

> ...
>
> > > +struct psp_mbox {
> > > +       u32 cmd_fields;
>
> > > +       phys_addr_t i2c_req_addr;
>
> But phys_addr_t is platform-dependent type. Perhaps you meant to use u64 here
> always?

Once I remove the "depends on X86_64" I believe this should be left
platform-dependent.

> > > +} __packed;
>
> ...
>
> > > +       struct psp_mbox __iomem *mbox = (struct psp_mbox __iomem *)mbox_iomem;
>
> For void * pointers the cast is implied, i.o.w. it's not needed here.

ACK.

> ...
>
> > > +static int psp_send_check_i2c_req(struct psp_i2c_req *req)
> > > +{
> > > +       if (psp_send_cmd(req))
>
> > > +               return -EIO;
>
> Why is error code shadowed?
>
> > > +       return check_i2c_req_sts(req);
> > > +}

Just as a side note - it wasn't modified in v2 when moving above to
psp_send_check_i2c_req(), but let me explain why I have introduced
this initially.

We have two means of timeouts in the context of this driver:
1. Timeout of internal mailbox, which means we cannot communicate with
a PSP for a programmed timeout. This timeout is encountered inside
psp_send_cmd().
2. Timeout of i2c arbitration - which means that we can communicate
with PSP, but PSP refuses to release i2c bus for too long. This
timeout is returned by psp_send_i2c_req() in case of error.
(side note: both error conditions are very unlikely to happen at runtime)

I wanted to clearly distinguish between these two and thus put all
errors around mailbox into "-EIO category", which is actually true.

> ...
>
> > > +cleanup:
> > > +       mutex_unlock(&psp_i2c_access_mutex);
> > > +       return 0;
>
> Not sure I understand why we ignore all above errors here.

Actually we are not ignoring them, since each error sets
"psp_i2c_mbox_fail = true;". This means that if there is any error on
x86-PSP interface, we are ignoring i2c-arbitration and just fall back
to normal (that is no-quirk) operation.

From the i2c-client perspective (who is eventually gathering error
code from above) I think we can claim that everything is fine, since
bus is granted to it. For developers there is an error message in case
some debug will be necessary.

> ...
>
> > > +       if (!dev || !dev->dev)
> > > +               return -ENODEV;
>
> At which circumstances may we get
>         dev != NULL
>         dev->dev == NULL
> ?
>
> ...
>
> > >         if (!dev || !dev->dev)
> > > -               return 0;
> > > +               return -ENODEV;
>
> I see the same here, perhaps Hans knows the answer :-)

Right, so I must admit that I simply used *-baytrail.c as a reference
and thinking that additional check shouldn't hurt us (always better
than not enough safety..). Looking more at this now -
`dw_i2c_plat_probe()` will boil-out earlier if `dev->dev == NULL`.
Should I remove this extra check in *-baytrail.c in the same commit?

> ...
>
> > > +static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
> > > +{
> > > +       const struct i2c_dw_semaphore_callbacks *ptr;
> > > +       int i = 0;
> > > +       int ret;
> > > +
> > > +       ptr = i2c_dw_semaphore_cb_table;
> > > +
> > > +       dev->semaphore_idx = -1;
> > > +
> > > +       while (ptr->probe) {
> > > +               ret = ptr->probe(dev);
> > > +               if (ret) {
>
> > > +                       /*
> > > +                        * If there is no semaphore device attached to this
> > > +                        * controller, we shouldn't abort general i2c_controller
> > > +                        * probe.
> > > +                        */
> > > +                       if (ret == -ENODEV) {
> > > +                               i++;
> > > +                               ptr++;
> > > +                               continue;
> > > +                       } else {
>
> Redundant 'else', but see below.
>
> > > +                               return ret;
> > > +                       }
>
> May it be
>
>             if (ret != -ENODEV)
>                 return ret;
>
>             i++;
>             ptr++;
>             continue;
>
> ?

Yes, looks good. Thanks!

Best Regards,
Jan

>
> > > +               }
> > > +
> > > +               dev->semaphore_idx = i;
> > > +               break;
> > > +       }
> > > +
> > > +       return 0;
> > > +}
>
> --
> With Best Regards,
> Andy Shevchenko
>
>

Re: [PATCH v2 2/2] i2c: designware: Add AMD PSP I2C bus support

[Andy Shevchenko]

On Mon, Jan 31, 2022 at 12:56:27PM +0100, Jan Dąbroś wrote:
> pt., 28 sty 2022 o 16:50 Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> napisał(a):
> > On Fri, Jan 28, 2022 at 03:59:40PM +0100, Jan Dąbroś wrote:
> > > pt., 28 sty 2022 o 15:48 Jan Dabros <jsd@semihalf.com> napisał(a):

...

> > > > +struct psp_mbox {
> > > > +       u32 cmd_fields;
> >
> > > > +       phys_addr_t i2c_req_addr;
> >
> > But phys_addr_t is platform-dependent type. Perhaps you meant to use u64
> > here
> > always?
> 
> Once I remove the "depends on X86_64" I believe this should be left
> platform-dependent.

If it's a protocol or HW layout, it may not be platform-dependent.

> > > > +} __packed;

...

> > > > +       if (psp_send_cmd(req))
> >
> > > > +               return -EIO;
> >
> > Why is error code shadowed?
> >
> 
> Just as a side note - it wasn't modified in v2 when moving above to
> psp_send_check_i2c_req(), but let me explain why I have introduced this
> initially.
> 
> We have two means of timeouts in the context of this driver:
> 1. Timeout of internal mailbox, which means we cannot communicate with a
> PSP for a programmed timeout. This timeout is encountered inside
> psp_send_cmd().
> 2. Timeout of i2c arbitration - which means that we can communicate with
> PSP, but PSP refuses to release i2c bus for too long. This timeout is
> returned by psp_send_i2c_req() in case of error.
> (side note: both error conditions are very unlikely to happen at runtime)
> 
> I wanted to clearly distinguish between these two and thus put all errors
> around mailbox into "-EIO category", which is actually true.

At very least this code needs more or less the above to be put as a comment.

...

> > > > +cleanup:
> > > > +       mutex_unlock(&psp_i2c_access_mutex);
> > > > +       return 0;
> >
> > Not sure I understand why we ignore all above errors here.
> >
> 
> Actually we are not ignoring them, since each error sets "psp_i2c_mbox_fail
> = true;". This means that if there is any error on x86-PSP interface, we
> are ignoring i2c-arbitration and just fall back to normal (that is
> no-quirk) operation.
> 
> From the i2c-client perspective (who is eventually gathering error code
> from above) I think we can claim that everything is fine, since bus is
> granted to it. For developers there is an error message in case some debug
> will be necessary.

Perhaps needs a comment (sorry, if I overlooked it).

...

> > > > +       if (!dev || !dev->dev)
> > > > +               return -ENODEV;
> >
> > At which circumstances may we get
> >         dev != NULL
> >         dev->dev == NULL
> > ?
> >
> > ...
> >
> > > >         if (!dev || !dev->dev)
> > > > -               return 0;
> > > > +               return -ENODEV;
> >
> > I see the same here, perhaps Hans knows the answer :-)
> 
> Right, so I must admit that I simply used *-baytrail.c as a reference and
> thinking that additional check shouldn't hurt us (always better than not
> enough safety..). Looking more at this now - `dw_i2c_plat_probe()` will
> boil-out earlier if `dev->dev == NULL`. Should I remove this extra check in
> *-baytrail.c in the same commit?

Maybe. Please, double check that it's not needed indeed.

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v2 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dąbroś]

pon., 31 sty 2022 o 14:32 Andy Shevchenko
<andriy.shevchenko@linux.intel.com> napisał(a):
>
> On Mon, Jan 31, 2022 at 12:56:27PM +0100, Jan Dąbroś wrote:
> > pt., 28 sty 2022 o 16:50 Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> > napisał(a):
> > > On Fri, Jan 28, 2022 at 03:59:40PM +0100, Jan Dąbroś wrote:
> > > > pt., 28 sty 2022 o 15:48 Jan Dabros <jsd@semihalf.com> napisał(a):
>
> ...
>
> > > > > +struct psp_mbox {
> > > > > +       u32 cmd_fields;
> > >
> > > > > +       phys_addr_t i2c_req_addr;
> > >
> > > But phys_addr_t is platform-dependent type. Perhaps you meant to use u64
> > > here
> > > always?
> >
> > Once I remove the "depends on X86_64" I believe this should be left
> > platform-dependent.
>
> If it's a protocol or HW layout, it may not be platform-dependent.

Right you are correct, will keep this as 64bit variable.

>
> > > > > +} __packed;
>
> ...
>
> > > > > +       if (psp_send_cmd(req))
> > >
> > > > > +               return -EIO;
> > >
> > > Why is error code shadowed?
> > >
> >
> > Just as a side note - it wasn't modified in v2 when moving above to
> > psp_send_check_i2c_req(), but let me explain why I have introduced this
> > initially.
> >
> > We have two means of timeouts in the context of this driver:
> > 1. Timeout of internal mailbox, which means we cannot communicate with a
> > PSP for a programmed timeout. This timeout is encountered inside
> > psp_send_cmd().
> > 2. Timeout of i2c arbitration - which means that we can communicate with
> > PSP, but PSP refuses to release i2c bus for too long. This timeout is
> > returned by psp_send_i2c_req() in case of error.
> > (side note: both error conditions are very unlikely to happen at runtime)
> >
> > I wanted to clearly distinguish between these two and thus put all errors
> > around mailbox into "-EIO category", which is actually true.
>
> At very least this code needs more or less the above to be put as a comment.

ACK

>
> ...
>
> > > > > +cleanup:
> > > > > +       mutex_unlock(&psp_i2c_access_mutex);
> > > > > +       return 0;
> > >
> > > Not sure I understand why we ignore all above errors here.
> > >
> >
> > Actually we are not ignoring them, since each error sets "psp_i2c_mbox_fail
> > = true;". This means that if there is any error on x86-PSP interface, we
> > are ignoring i2c-arbitration and just fall back to normal (that is
> > no-quirk) operation.
> >
> > From the i2c-client perspective (who is eventually gathering error code
> > from above) I think we can claim that everything is fine, since bus is
> > granted to it. For developers there is an error message in case some debug
> > will be necessary.
>
> Perhaps needs a comment (sorry, if I overlooked it).

ACK.

>
> ...
>
> > > > > +       if (!dev || !dev->dev)
> > > > > +               return -ENODEV;
> > >
> > > At which circumstances may we get
> > >         dev != NULL
> > >         dev->dev == NULL
> > > ?
> > >
> > > ...
> > >
> > > > >         if (!dev || !dev->dev)
> > > > > -               return 0;
> > > > > +               return -ENODEV;
> > >
> > > I see the same here, perhaps Hans knows the answer :-)
> >
> > Right, so I must admit that I simply used *-baytrail.c as a reference and
> > thinking that additional check shouldn't hurt us (always better than not
> > enough safety..). Looking more at this now - `dw_i2c_plat_probe()` will
> > boil-out earlier if `dev->dev == NULL`. Should I remove this extra check in
> > *-baytrail.c in the same commit?
>
> Maybe. Please, double check that it's not needed indeed.

Confirmed that this is unnecessary.

Best Regards,
Jan

>
> --
> With Best Regards,
> Andy Shevchenko
>
>

[PATCH v3 0/2] i2c-designware: Add support for AMD PSP semaphore

[Jan Dabros]

This patchset comprises support for new i2c-designware controller setup on some
AMD Cezanne SoCs, where x86 is sharing i2c bus with PSP. PSP uses the same
controller and acts as an i2c arbitrator there (x86 is leasing bus from it).

First commit aims to improve generic i2c-designware code by adding extra locking
on probe() and disable() paths. I would like to ask someone with access to
boards which use Intel BayTrail(CONFIG_I2C_DESIGNWARE_BAYTRAIL) to verify
behavior of my changes on such setup.

Second commit adds support for new PSP semaphore arbitration mechanism.
Implementation is similar to the one from i2c-designware-baytrail.c however
there are two main differences:
1) Add new ACPI ID in order to protect against silent binding of the old driver
to the setup with PSP semaphore. Extra flag ARBITRATION_SEMAPHORE added to this
new _HID allows to recognize setup with PSP.
2) Beside acquire_lock() and release_lock() methods we are also applying quirks
to the lock_bus() and unlock_bus() global adapter methods. With this in place
all i2c clients drivers may lock i2c bus for a desired number of i2c
transactions (e.g. write-wait-read) without being aware of that such bus is
shared with another entity.

This patchset is a follow-up to the RFC sent earlier on LKML [1], with review
comments applied.

Looking forward to some feedback.

[1] https://lkml.org/lkml/2021/12/22/219

v2 -> v3:
* Change X86_64 Kconfig dependency to X86_MSR
* Switch from phys_addr_t to u64 in mailbox struct definition
* Remove redundant guard in semaphores' probes
* Add comments about error propagation
* Move credits for kernel test robot into changelog

v1 -> v2:
* Remove usage of unions
* Get rid of unnecessary __packed attributes
* Switch to use iopoll.h and bitfields.h APIs were applicable
* Follow the convention to check for the error first
* Reorder entries (includes, table entries) alphabetically
* Add necessary includes
* Add Kconfig dependency on X86_64
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: kernel test robot <lkp@intel.com>
* Modify probe() to use terminating entry for traversing through table
  instead of ARRAY_SIZE
* Fix typos in comments
* Rebase patchset

Jan Dabros (2):
  i2c: designware: Add missing locks
  i2c: designware: Add AMD PSP I2C bus support

 MAINTAINERS                                  |   1 +
 drivers/acpi/acpi_apd.c                      |   7 +-
 drivers/i2c/busses/Kconfig                   |  11 +
 drivers/i2c/busses/Makefile                  |   1 +
 drivers/i2c/busses/i2c-designware-amdpsp.c   | 389 +++++++++++++++++++
 drivers/i2c/busses/i2c-designware-baytrail.c |  12 +-
 drivers/i2c/busses/i2c-designware-common.c   |  12 +
 drivers/i2c/busses/i2c-designware-core.h     |  18 +-
 drivers/i2c/busses/i2c-designware-master.c   |   6 +
 drivers/i2c/busses/i2c-designware-platdrv.c  |  60 +++
 10 files changed, 505 insertions(+), 12 deletions(-)
 create mode 100644 drivers/i2c/busses/i2c-designware-amdpsp.c

-- 
2.35.0.rc2.247.g8bbb082509-goog

[PATCH v3 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dabros]

Implement an I2C controller sharing mechanism between the host (kernel)
and PSP co-processor on some platforms equipped with AMD Cezanne SoC.

On these platforms we need to implement "software" i2c arbitration.
Default arbitration owner is PSP and kernel asks for acquire as well
as inform about release of the i2c bus via mailbox mechanism.

            +---------+
 <- ACQUIRE |         |
  +---------|   CPU   |\
  |         |         | \      +----------+  SDA
  |         +---------+  \     |          |-------
MAILBOX                   +--> |  I2C-DW  |  SCL
  |         +---------+        |          |-------
  |         |         |        +----------+
  +---------|   PSP   |
   <- ACK   |         |
            +---------+

            +---------+
 <- RELEASE |         |
  +---------|   CPU   |
  |         |         |        +----------+  SDA
  |         +---------+        |          |-------
MAILBOX                   +--> |  I2C-DW  |  SCL
  |         +---------+  /     |          |-------
  |         |         | /      +----------+
  +---------|   PSP   |/
   <- ACK   |         |
            +---------+

The solution is similar to i2c-designware-baytrail.c implementation, where
we are using a generic i2c-designware-* driver with a small "wrapper".

In contrary to baytrail semaphore implementation, beside internal
acquire_lock() and release_lock() methods we are also applying quirks to
lock_bus() and unlock_bus() global adapter methods. With this in place
all i2c clients drivers may lock i2c bus for a desired number of i2c
transactions (e.g. write-wait-read) without being aware of that such bus
is shared with another entity.

Modify i2c_dw_probe_lock_support() to select correct semaphore
implementation at runtime, since now we have more than one available.

Configure new matching ACPI ID "AMDI0019" and register
ARBITRATION_SEMAPHORE flag in order to distinguish setup with PSP
arbitration.

Add myself as a reviewer for I2C DesignWare in order to help with reviewing
and testing possible changes touching new i2c-designware-amdpsp.c module.

Signed-off-by: Jan Dabros <jsd@semihalf.com>
---
 MAINTAINERS                                  |   1 +
 drivers/acpi/acpi_apd.c                      |   7 +-
 drivers/i2c/busses/Kconfig                   |  11 +
 drivers/i2c/busses/Makefile                  |   1 +
 drivers/i2c/busses/i2c-designware-amdpsp.c   | 389 +++++++++++++++++++
 drivers/i2c/busses/i2c-designware-baytrail.c |  12 +-
 drivers/i2c/busses/i2c-designware-core.h     |  18 +-
 drivers/i2c/busses/i2c-designware-platdrv.c  |  60 +++
 8 files changed, 487 insertions(+), 12 deletions(-)
 create mode 100644 drivers/i2c/busses/i2c-designware-amdpsp.c

diff --git a/MAINTAINERS b/MAINTAINERS
index f41088418aae..e0a7e05c5a41 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -18666,6 +18666,7 @@ SYNOPSYS DESIGNWARE I2C DRIVER
 M:	Jarkko Nikula <jarkko.nikula@linux.intel.com>
 R:	Andy Shevchenko <andriy.shevchenko@linux.intel.com>
 R:	Mika Westerberg <mika.westerberg@linux.intel.com>
+R:	Jan Dabros <jsd@semihalf.com>
 L:	linux-i2c@vger.kernel.org
 S:	Maintained
 F:	drivers/i2c/busses/i2c-designware-*
diff --git a/drivers/acpi/acpi_apd.c b/drivers/acpi/acpi_apd.c
index e7934ba79b02..ad245bbd965e 100644
--- a/drivers/acpi/acpi_apd.c
+++ b/drivers/acpi/acpi_apd.c
@@ -232,12 +232,13 @@ static const struct acpi_device_id acpi_apd_device_ids[] = {
 	/* Generic apd devices */
 #ifdef CONFIG_X86_AMD_PLATFORM_DEVICE
 	{ "AMD0010", APD_ADDR(cz_i2c_desc) },
-	{ "AMDI0010", APD_ADDR(wt_i2c_desc) },
 	{ "AMD0020", APD_ADDR(cz_uart_desc) },
-	{ "AMDI0020", APD_ADDR(cz_uart_desc) },
-	{ "AMDI0022", APD_ADDR(cz_uart_desc) },
 	{ "AMD0030", },
 	{ "AMD0040", APD_ADDR(fch_misc_desc)},
+	{ "AMDI0010", APD_ADDR(wt_i2c_desc) },
+	{ "AMDI0019", APD_ADDR(wt_i2c_desc) },
+	{ "AMDI0020", APD_ADDR(cz_uart_desc) },
+	{ "AMDI0022", APD_ADDR(cz_uart_desc) },
 	{ "HYGO0010", APD_ADDR(wt_i2c_desc) },
 #endif
 #ifdef CONFIG_ARM64
diff --git a/drivers/i2c/busses/Kconfig b/drivers/i2c/busses/Kconfig
index 42da31c1ab70..5ea26a7aa1d5 100644
--- a/drivers/i2c/busses/Kconfig
+++ b/drivers/i2c/busses/Kconfig
@@ -553,6 +553,17 @@ config I2C_DESIGNWARE_PLATFORM
 	  This driver can also be built as a module.  If so, the module
 	  will be called i2c-designware-platform.
 
+config I2C_DESIGNWARE_AMDPSP
+	bool "AMD PSP I2C semaphore support"
+	depends on X86_MSR
+	depends on ACPI
+	depends on I2C_DESIGNWARE_PLATFORM
+	help
+	  This driver enables managed host access to the selected I2C bus shared
+	  between AMD CPU and AMD PSP.
+
+	  You should say Y if running on an AMD system equipped with the PSP.
+
 config I2C_DESIGNWARE_BAYTRAIL
 	bool "Intel Baytrail I2C semaphore support"
 	depends on ACPI
diff --git a/drivers/i2c/busses/Makefile b/drivers/i2c/busses/Makefile
index 1d00dce77098..752f47be3fc1 100644
--- a/drivers/i2c/busses/Makefile
+++ b/drivers/i2c/busses/Makefile
@@ -54,6 +54,7 @@ i2c-designware-core-y					+= i2c-designware-master.o
 i2c-designware-core-$(CONFIG_I2C_DESIGNWARE_SLAVE) 	+= i2c-designware-slave.o
 obj-$(CONFIG_I2C_DESIGNWARE_PLATFORM)			+= i2c-designware-platform.o
 i2c-designware-platform-y 				:= i2c-designware-platdrv.o
+i2c-designware-platform-$(CONFIG_I2C_DESIGNWARE_AMDPSP)	+= i2c-designware-amdpsp.o
 i2c-designware-platform-$(CONFIG_I2C_DESIGNWARE_BAYTRAIL) += i2c-designware-baytrail.o
 obj-$(CONFIG_I2C_DESIGNWARE_PCI)			+= i2c-designware-pci.o
 i2c-designware-pci-y					:= i2c-designware-pcidrv.o
diff --git a/drivers/i2c/busses/i2c-designware-amdpsp.c b/drivers/i2c/busses/i2c-designware-amdpsp.c
new file mode 100644
index 000000000000..a45e4faeb2c1
--- /dev/null
+++ b/drivers/i2c/busses/i2c-designware-amdpsp.c
@@ -0,0 +1,389 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/bitfield.h>
+#include <linux/i2c.h>
+#include <linux/io-64-nonatomic-lo-hi.h>
+#include <linux/psp-sev.h>
+#include <linux/types.h>
+
+#include <asm/msr.h>
+
+#include "i2c-designware-core.h"
+
+#define MSR_AMD_PSP_ADDR	0xc00110a2
+#define PSP_MBOX_OFFSET		0x10570
+#define PSP_CMD_TIMEOUT_MS	500
+
+#define PSP_I2C_REQ_BUS_CMD		0x64
+#define PSP_I2C_REQ_RETRY_CNT		10
+#define PSP_I2C_REQ_RETRY_DELAY_USEC	(50 * 1000)
+#define PSP_I2C_REQ_STS_OK		0x0
+#define PSP_I2C_REQ_STS_BUS_BUSY	0x1
+#define PSP_I2C_REQ_STS_INV_PARAM	0x3
+
+#define PSP_MBOX_FIELDS_STS		GENMASK(15, 0)
+#define PSP_MBOX_FIELDS_CMD		GENMASK(23, 16)
+#define PSP_MBOX_FIELDS_RESERVED	GENMASK(29, 24)
+#define PSP_MBOX_FIELDS_RECOVERY	BIT(30)
+#define PSP_MBOX_FIELDS_READY		BIT(31)
+
+struct psp_req_buffer_hdr {
+	u32 total_size;
+	u32 status;
+};
+
+enum psp_i2c_req_type {
+	PSP_I2C_REQ_ACQUIRE,
+	PSP_I2C_REQ_RELEASE,
+	PSP_I2C_REQ_MAX
+};
+
+struct psp_i2c_req {
+	struct psp_req_buffer_hdr hdr;
+	enum psp_i2c_req_type type;
+} __aligned(32);
+
+struct psp_mbox {
+	u32 cmd_fields;
+	u64 i2c_req_addr;
+} __packed;
+
+static DEFINE_MUTEX(psp_i2c_access_mutex);
+static unsigned long psp_i2c_sem_acquired;
+static void __iomem *mbox_iomem;
+static u32 psp_i2c_access_count;
+static bool psp_i2c_mbox_fail;
+static struct device *psp_i2c_dev;
+
+/*
+ * Implementation of PSP-x86 i2c-arbitration mailbox introduced for AMD Cezanne
+ * family of SoCs.
+ */
+
+static int psp_get_mbox_addr(unsigned long *mbox_addr)
+{
+	unsigned long long psp_mmio;
+
+	if (rdmsrl_safe(MSR_AMD_PSP_ADDR, &psp_mmio))
+		return -EIO;
+
+	*mbox_addr = (unsigned long)(psp_mmio + PSP_MBOX_OFFSET);
+
+	return 0;
+}
+
+static int psp_mbox_probe(void)
+{
+	unsigned long mbox_addr;
+	int ret;
+
+	ret = psp_get_mbox_addr(&mbox_addr);
+	if (ret)
+		return ret;
+
+	mbox_iomem = ioremap(mbox_addr, sizeof(struct psp_mbox));
+	if (!mbox_iomem)
+		return -ENOMEM;
+
+	return 0;
+}
+
+
+/* Recovery field should be equal 0 to start sending commands */
+static int psp_check_mbox_recovery(struct psp_mbox __iomem *mbox)
+{
+	u32 tmp;
+
+	tmp = readl(&mbox->cmd_fields);
+
+	return FIELD_GET(PSP_MBOX_FIELDS_RECOVERY, tmp);
+}
+
+static int psp_wait_cmd(struct psp_mbox __iomem *mbox)
+{
+	u32 tmp, expected;
+
+	/* Expect mbox_cmd to be cleared and ready bit to be set by PSP */
+	expected = FIELD_PREP(PSP_MBOX_FIELDS_READY, 1);
+
+	return readl_poll_timeout(&mbox->cmd_fields, tmp, (tmp == expected),
+				  0, 1000 * PSP_CMD_TIMEOUT_MS);
+}
+
+/* Status equal to 0 means that PSP succeed processing command */
+static int psp_check_mbox_sts(struct psp_mbox __iomem *mbox)
+{
+	u32 cmd_reg;
+
+	cmd_reg = readl(&mbox->cmd_fields);
+
+	return FIELD_GET(PSP_MBOX_FIELDS_STS, cmd_reg);
+}
+
+static int psp_send_cmd(struct psp_i2c_req *req)
+{
+	struct psp_mbox __iomem *mbox = mbox_iomem;
+	phys_addr_t req_addr;
+	u32 cmd_reg;
+
+	if (psp_check_mbox_recovery(mbox))
+		return -EIO;
+
+	if (psp_wait_cmd(mbox))
+		return -EBUSY;
+
+	/*
+	 * Fill mailbox with address of command-response buffer, which will be
+	 * used for sending i2c requests as well as reading status returned by
+	 * PSP. Use physical address of buffer, since PSP will map this region.
+	 */
+	req_addr = __psp_pa((void *)req);
+	writeq(req_addr, &mbox->i2c_req_addr);
+
+	/* Write command register to trigger processing */
+	cmd_reg = FIELD_PREP(PSP_MBOX_FIELDS_CMD, PSP_I2C_REQ_BUS_CMD);
+	writel(cmd_reg, &mbox->cmd_fields);
+
+	if (psp_wait_cmd(mbox))
+		return -ETIMEDOUT;
+
+	if (psp_check_mbox_sts(mbox))
+		return -EIO;
+
+	return 0;
+}
+
+/* Helper to verify status returned by PSP */
+static int check_i2c_req_sts(struct psp_i2c_req *req)
+{
+	int status;
+
+	status = readl(&req->hdr.status);
+
+	switch (status) {
+	case PSP_I2C_REQ_STS_OK:
+		return 0;
+	case PSP_I2C_REQ_STS_BUS_BUSY:
+		return -EBUSY;
+	case PSP_I2C_REQ_STS_INV_PARAM:
+	default:
+		return -EIO;
+	};
+}
+
+static int psp_send_check_i2c_req(struct psp_i2c_req *req)
+{
+	/*
+	 * Errors in x86-PSP i2c-arbitration protocol may occur at two levels:
+	 * 1. mailbox communication - PSP is not operational or some IO errors
+	 * with basic communication had happened;
+	 * 2. i2c-requests - PSP refuses to grant i2c arbitration to x86 for too
+	 * long.
+	 * In order to distinguish between these two in error handling code, all
+	 * errors on the first level (returned by psp_send_cmd) are shadowed by
+	 * -EIO.
+	 */
+	if (psp_send_cmd(req))
+		return -EIO;
+
+	return check_i2c_req_sts(req);
+}
+
+static int psp_send_i2c_req(enum psp_i2c_req_type i2c_req_type)
+{
+	struct psp_i2c_req *req;
+	unsigned long start;
+	int status, ret;
+
+	/* Allocate command-response buffer */
+	req = kzalloc(sizeof(*req), GFP_KERNEL);
+	if (!req)
+		return -ENOMEM;
+
+	req->hdr.total_size = sizeof(*req);
+	req->type = i2c_req_type;
+
+	start = jiffies;
+	ret = read_poll_timeout(psp_send_check_i2c_req, status,
+				(status != -EBUSY),
+				PSP_I2C_REQ_RETRY_DELAY_USEC,
+				PSP_I2C_REQ_RETRY_CNT * PSP_I2C_REQ_RETRY_DELAY_USEC,
+				0, req);
+	if (ret)
+		goto cleanup;
+
+	if (status) {
+		ret = status;
+		goto cleanup;
+	}
+
+	dev_dbg(psp_i2c_dev, "Request accepted by PSP after %ums\n",
+		jiffies_to_msecs(jiffies - start));
+
+cleanup:
+	kfree(req);
+	return ret;
+}
+
+static int psp_acquire_i2c_bus(void)
+{
+	int status;
+
+	mutex_lock(&psp_i2c_access_mutex);
+
+	/* Return early if mailbox malfunctioned */
+	if (psp_i2c_mbox_fail)
+		goto cleanup;
+
+	/*
+	 * Simply increment usage counter and return if PSP semaphore was
+	 * already taken by kernel.
+	 */
+	if (psp_i2c_access_count > 0) {
+		psp_i2c_access_count++;
+		goto cleanup;
+	};
+
+	status = psp_send_i2c_req(PSP_I2C_REQ_ACQUIRE);
+	if (status) {
+		if (status == -ETIMEDOUT)
+			dev_err(psp_i2c_dev, "Timed out waiting for PSP to release I2C bus\n");
+		else
+			dev_err(psp_i2c_dev, "PSP communication error\n");
+
+		dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n");
+		psp_i2c_mbox_fail = true;
+		goto cleanup;
+	}
+
+	psp_i2c_sem_acquired = jiffies;
+	psp_i2c_access_count++;
+
+	/*
+	 * In case of errors with PSP arbitrator psp_i2c_mbox_fail variable is
+	 * set above. As a consequence consecutive calls to acquire will bypass
+	 * communication with PSP. At any case i2c bus is granted to the caller,
+	 * thus always return success.
+	 */
+cleanup:
+	mutex_unlock(&psp_i2c_access_mutex);
+	return 0;
+}
+
+static void psp_release_i2c_bus(void)
+{
+	int status;
+
+	mutex_lock(&psp_i2c_access_mutex);
+
+	/* Return early if mailbox was malfunctional */
+	if (psp_i2c_mbox_fail)
+		goto cleanup;
+
+	/*
+	 * If we are last owner of PSP semaphore, need to release aribtration
+	 * via mailbox.
+	 */
+	psp_i2c_access_count--;
+	if (psp_i2c_access_count > 0)
+		goto cleanup;
+
+	/* Send a release command to PSP */
+	status = psp_send_i2c_req(PSP_I2C_REQ_RELEASE);
+	if (status) {
+		if (status == -ETIMEDOUT)
+			dev_err(psp_i2c_dev, "Timed out waiting for PSP to acquire I2C bus\n");
+		else
+			dev_err(psp_i2c_dev, "PSP communication error\n");
+
+		dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n");
+		psp_i2c_mbox_fail = true;
+		goto cleanup;
+	}
+
+	dev_dbg(psp_i2c_dev, "PSP semaphore held for %ums\n",
+		jiffies_to_msecs(jiffies - psp_i2c_sem_acquired));
+
+cleanup:
+	mutex_unlock(&psp_i2c_access_mutex);
+}
+
+/*
+ * Locking methods are based on the default implementation from
+ * drivers/i2c/i2c-core-base.c, but with psp acquire and release operations
+ * added. With this in place we can ensure that i2c clients on the bus shared
+ * with psp are able to lock HW access to the bus for arbitrary number of
+ * operations - that is e.g. write-wait-read.
+ */
+static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter,
+					unsigned int flags)
+{
+	psp_acquire_i2c_bus();
+	rt_mutex_lock_nested(&adapter->bus_lock, i2c_adapter_depth(adapter));
+}
+
+static int i2c_adapter_dw_psp_trylock_bus(struct i2c_adapter *adapter,
+					  unsigned int flags)
+{
+	int ret;
+
+	ret = rt_mutex_trylock(&adapter->bus_lock);
+	if (ret)
+		return ret;
+
+	psp_acquire_i2c_bus();
+
+	return ret;
+}
+
+static void i2c_adapter_dw_psp_unlock_bus(struct i2c_adapter *adapter,
+					  unsigned int flags)
+{
+	psp_release_i2c_bus();
+	rt_mutex_unlock(&adapter->bus_lock);
+}
+
+static const struct i2c_lock_operations i2c_dw_psp_lock_ops = {
+	.lock_bus = i2c_adapter_dw_psp_lock_bus,
+	.trylock_bus = i2c_adapter_dw_psp_trylock_bus,
+	.unlock_bus = i2c_adapter_dw_psp_unlock_bus,
+};
+
+int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev)
+{
+	int ret;
+
+	if (!dev)
+		return -ENODEV;
+
+	if (!(dev->flags & ARBITRATION_SEMAPHORE))
+		return -ENODEV;
+
+	/* Allow to bind only one instance of a driver */
+	if (psp_i2c_dev)
+		return -EEXIST;
+
+	psp_i2c_dev = dev->dev;
+
+	ret = psp_mbox_probe();
+	if (ret)
+		return ret;
+
+	dev_info(psp_i2c_dev, "I2C bus managed by AMD PSP\n");
+
+	/*
+	 * Install global locking callbacks for adapter as well as internal i2c
+	 * controller locks.
+	 */
+	dev->adapter.lock_ops = &i2c_dw_psp_lock_ops;
+	dev->acquire_lock = psp_acquire_i2c_bus;
+	dev->release_lock = psp_release_i2c_bus;
+
+	return 0;
+}
+
+/* Unmap area used as a mailbox with PSP */
+void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev)
+{
+	iounmap(mbox_iomem);
+}
diff --git a/drivers/i2c/busses/i2c-designware-baytrail.c b/drivers/i2c/busses/i2c-designware-baytrail.c
index c6a7a00e1d52..45774aa47c28 100644
--- a/drivers/i2c/busses/i2c-designware-baytrail.c
+++ b/drivers/i2c/busses/i2c-designware-baytrail.c
@@ -12,25 +12,25 @@
 
 #include "i2c-designware-core.h"
 
-int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
+int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev)
 {
 	acpi_status status;
 	unsigned long long shared_host = 0;
 	acpi_handle handle;
 
-	if (!dev || !dev->dev)
-		return 0;
+	if (!dev)
+		return -ENODEV;
 
 	handle = ACPI_HANDLE(dev->dev);
 	if (!handle)
-		return 0;
+		return -ENODEV;
 
 	status = acpi_evaluate_integer(handle, "_SEM", NULL, &shared_host);
 	if (ACPI_FAILURE(status))
-		return 0;
+		return -ENODEV;
 
 	if (!shared_host)
-		return 0;
+		return -ENODEV;
 
 	if (!iosf_mbi_available())
 		return -EPROBE_DEFER;
diff --git a/drivers/i2c/busses/i2c-designware-core.h b/drivers/i2c/busses/i2c-designware-core.h
index 4b26cba40139..1d65212fddbd 100644
--- a/drivers/i2c/busses/i2c-designware-core.h
+++ b/drivers/i2c/busses/i2c-designware-core.h
@@ -227,6 +227,8 @@ struct reset_control;
  * @hs_lcnt: high speed LCNT value
  * @acquire_lock: function to acquire a hardware lock on the bus
  * @release_lock: function to release a hardware lock on the bus
+ * @semaphore_idx: Index of table with semaphore type attached to the bus. It's
+ *	-1 if there is no semaphore.
  * @shared_with_punit: true if this bus is shared with the SoCs PUNIT
  * @disable: function to disable the controller
  * @disable_int: function to disable all interrupts
@@ -285,6 +287,7 @@ struct dw_i2c_dev {
 	u16			hs_lcnt;
 	int			(*acquire_lock)(void);
 	void			(*release_lock)(void);
+	int			semaphore_idx;
 	bool			shared_with_punit;
 	void			(*disable)(struct dw_i2c_dev *dev);
 	void			(*disable_int)(struct dw_i2c_dev *dev);
@@ -297,6 +300,7 @@ struct dw_i2c_dev {
 
 #define ACCESS_INTR_MASK	BIT(0)
 #define ACCESS_NO_IRQ_SUSPEND	BIT(1)
+#define ARBITRATION_SEMAPHORE	BIT(2)
 
 #define MODEL_MSCC_OCELOT	BIT(8)
 #define MODEL_BAIKAL_BT1	BIT(9)
@@ -310,6 +314,11 @@ struct dw_i2c_dev {
 #define AMD_UCSI_INTR_REG	0x474
 #define AMD_UCSI_INTR_EN	0xd
 
+struct i2c_dw_semaphore_callbacks {
+	int	(*probe)(struct dw_i2c_dev *dev);
+	void	(*remove)(struct dw_i2c_dev *dev);
+};
+
 int i2c_dw_init_regmap(struct dw_i2c_dev *dev);
 u32 i2c_dw_scl_hcnt(u32 ic_clk, u32 tSYMBOL, u32 tf, int cond, int offset);
 u32 i2c_dw_scl_lcnt(u32 ic_clk, u32 tLOW, u32 tf, int offset);
@@ -370,9 +379,12 @@ static inline void i2c_dw_configure(struct dw_i2c_dev *dev)
 }
 
 #if IS_ENABLED(CONFIG_I2C_DESIGNWARE_BAYTRAIL)
-extern int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev);
-#else
-static inline int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev) { return 0; }
+int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev);
+#endif
+
+#if IS_ENABLED(CONFIG_I2C_DESIGNWARE_AMDPSP)
+int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev);
+void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev);
 #endif
 
 int i2c_dw_validate_speed(struct dw_i2c_dev *dev);
diff --git a/drivers/i2c/busses/i2c-designware-platdrv.c b/drivers/i2c/busses/i2c-designware-platdrv.c
index 2bd81abc86f6..9973ac894a51 100644
--- a/drivers/i2c/busses/i2c-designware-platdrv.c
+++ b/drivers/i2c/busses/i2c-designware-platdrv.c
@@ -50,6 +50,7 @@ static const struct acpi_device_id dw_i2c_acpi_match[] = {
 	{ "808622C1", ACCESS_NO_IRQ_SUSPEND },
 	{ "AMD0010", ACCESS_INTR_MASK },
 	{ "AMDI0010", ACCESS_INTR_MASK },
+	{ "AMDI0019", ACCESS_INTR_MASK | ARBITRATION_SEMAPHORE },
 	{ "AMDI0510", 0 },
 	{ "APMC0D0F", 0 },
 	{ "HISI02A1", 0 },
@@ -204,6 +205,63 @@ static const struct dmi_system_id dw_i2c_hwmon_class_dmi[] = {
 	{ } /* terminate list */
 };
 
+static const struct i2c_dw_semaphore_callbacks i2c_dw_semaphore_cb_table[] = {
+#ifdef CONFIG_I2C_DESIGNWARE_BAYTRAIL
+	{
+		.probe = i2c_dw_baytrail_probe_lock_support,
+	},
+#endif
+#ifdef CONFIG_I2C_DESIGNWARE_AMDPSP
+	{
+		.probe = i2c_dw_amdpsp_probe_lock_support,
+		.remove = i2c_dw_amdpsp_remove_lock_support,
+	},
+#endif
+	{}
+};
+
+static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
+{
+	const struct i2c_dw_semaphore_callbacks *ptr;
+	int i = 0;
+	int ret;
+
+	ptr = i2c_dw_semaphore_cb_table;
+
+	dev->semaphore_idx = -1;
+
+	while (ptr->probe) {
+		ret = ptr->probe(dev);
+		if (ret) {
+			/*
+			 * If there is no semaphore device attached to this
+			 * controller, we shouldn't abort general i2c_controller
+			 * probe.
+			 */
+			if (ret != -ENODEV)
+				return ret;
+
+			i++;
+			ptr++;
+			continue;
+		}
+
+		dev->semaphore_idx = i;
+		break;
+	}
+
+	return 0;
+}
+
+static void i2c_dw_remove_lock_support(struct dw_i2c_dev *dev)
+{
+	if (dev->semaphore_idx < 0)
+		return;
+
+	if (i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove)
+		i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove(dev);
+}
+
 static int dw_i2c_plat_probe(struct platform_device *pdev)
 {
 	struct i2c_adapter *adap;
@@ -334,6 +392,8 @@ static int dw_i2c_plat_remove(struct platform_device *pdev)
 	pm_runtime_put_sync(&pdev->dev);
 	dw_i2c_plat_pm_cleanup(dev);
 
+	i2c_dw_remove_lock_support(dev);
+
 	reset_control_assert(dev->rst);
 
 	return 0;
-- 
2.35.0.rc2.247.g8bbb082509-goog

Re: [PATCH v3 2/2] i2c: designware: Add AMD PSP I2C bus support

[Andy Shevchenko]

On Wed, Feb 02, 2022 at 03:43:02PM +0100, Jan Dabros wrote:
> Implement an I2C controller sharing mechanism between the host (kernel)
> and PSP co-processor on some platforms equipped with AMD Cezanne SoC.
> 
> On these platforms we need to implement "software" i2c arbitration.
> Default arbitration owner is PSP and kernel asks for acquire as well
> as inform about release of the i2c bus via mailbox mechanism.
> 
>             +---------+
>  <- ACQUIRE |         |
>   +---------|   CPU   |\
>   |         |         | \      +----------+  SDA
>   |         +---------+  \     |          |-------
> MAILBOX                   +--> |  I2C-DW  |  SCL
>   |         +---------+        |          |-------
>   |         |         |        +----------+
>   +---------|   PSP   |
>    <- ACK   |         |
>             +---------+
> 
>             +---------+
>  <- RELEASE |         |
>   +---------|   CPU   |
>   |         |         |        +----------+  SDA
>   |         +---------+        |          |-------
> MAILBOX                   +--> |  I2C-DW  |  SCL
>   |         +---------+  /     |          |-------
>   |         |         | /      +----------+
>   +---------|   PSP   |/
>    <- ACK   |         |
>             +---------+
> 
> The solution is similar to i2c-designware-baytrail.c implementation, where
> we are using a generic i2c-designware-* driver with a small "wrapper".
> 
> In contrary to baytrail semaphore implementation, beside internal
> acquire_lock() and release_lock() methods we are also applying quirks to
> lock_bus() and unlock_bus() global adapter methods. With this in place
> all i2c clients drivers may lock i2c bus for a desired number of i2c
> transactions (e.g. write-wait-read) without being aware of that such bus
> is shared with another entity.
> 
> Modify i2c_dw_probe_lock_support() to select correct semaphore
> implementation at runtime, since now we have more than one available.
> 
> Configure new matching ACPI ID "AMDI0019" and register
> ARBITRATION_SEMAPHORE flag in order to distinguish setup with PSP
> arbitration.
> 
> Add myself as a reviewer for I2C DesignWare in order to help with reviewing
> and testing possible changes touching new i2c-designware-amdpsp.c module.

Side note, please start a new email thread for new version of the series.

Looks good enough to me, although a few minor comments below.
Up to you to address them now or later.
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

> Signed-off-by: Jan Dabros <jsd@semihalf.com>
> ---
>  MAINTAINERS                                  |   1 +
>  drivers/acpi/acpi_apd.c                      |   7 +-
>  drivers/i2c/busses/Kconfig                   |  11 +
>  drivers/i2c/busses/Makefile                  |   1 +
>  drivers/i2c/busses/i2c-designware-amdpsp.c   | 389 +++++++++++++++++++
>  drivers/i2c/busses/i2c-designware-baytrail.c |  12 +-
>  drivers/i2c/busses/i2c-designware-core.h     |  18 +-
>  drivers/i2c/busses/i2c-designware-platdrv.c  |  60 +++
>  8 files changed, 487 insertions(+), 12 deletions(-)
>  create mode 100644 drivers/i2c/busses/i2c-designware-amdpsp.c
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index f41088418aae..e0a7e05c5a41 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -18666,6 +18666,7 @@ SYNOPSYS DESIGNWARE I2C DRIVER
>  M:	Jarkko Nikula <jarkko.nikula@linux.intel.com>
>  R:	Andy Shevchenko <andriy.shevchenko@linux.intel.com>
>  R:	Mika Westerberg <mika.westerberg@linux.intel.com>
> +R:	Jan Dabros <jsd@semihalf.com>
>  L:	linux-i2c@vger.kernel.org
>  S:	Maintained
>  F:	drivers/i2c/busses/i2c-designware-*
> diff --git a/drivers/acpi/acpi_apd.c b/drivers/acpi/acpi_apd.c
> index e7934ba79b02..ad245bbd965e 100644
> --- a/drivers/acpi/acpi_apd.c
> +++ b/drivers/acpi/acpi_apd.c
> @@ -232,12 +232,13 @@ static const struct acpi_device_id acpi_apd_device_ids[] = {
>  	/* Generic apd devices */
>  #ifdef CONFIG_X86_AMD_PLATFORM_DEVICE
>  	{ "AMD0010", APD_ADDR(cz_i2c_desc) },
> -	{ "AMDI0010", APD_ADDR(wt_i2c_desc) },
>  	{ "AMD0020", APD_ADDR(cz_uart_desc) },
> -	{ "AMDI0020", APD_ADDR(cz_uart_desc) },
> -	{ "AMDI0022", APD_ADDR(cz_uart_desc) },
>  	{ "AMD0030", },
>  	{ "AMD0040", APD_ADDR(fch_misc_desc)},
> +	{ "AMDI0010", APD_ADDR(wt_i2c_desc) },
> +	{ "AMDI0019", APD_ADDR(wt_i2c_desc) },
> +	{ "AMDI0020", APD_ADDR(cz_uart_desc) },
> +	{ "AMDI0022", APD_ADDR(cz_uart_desc) },
>  	{ "HYGO0010", APD_ADDR(wt_i2c_desc) },
>  #endif
>  #ifdef CONFIG_ARM64
> diff --git a/drivers/i2c/busses/Kconfig b/drivers/i2c/busses/Kconfig
> index 42da31c1ab70..5ea26a7aa1d5 100644
> --- a/drivers/i2c/busses/Kconfig
> +++ b/drivers/i2c/busses/Kconfig
> @@ -553,6 +553,17 @@ config I2C_DESIGNWARE_PLATFORM
>  	  This driver can also be built as a module.  If so, the module
>  	  will be called i2c-designware-platform.
>  
> +config I2C_DESIGNWARE_AMDPSP
> +	bool "AMD PSP I2C semaphore support"
> +	depends on X86_MSR
> +	depends on ACPI
> +	depends on I2C_DESIGNWARE_PLATFORM
> +	help
> +	  This driver enables managed host access to the selected I2C bus shared
> +	  between AMD CPU and AMD PSP.
> +
> +	  You should say Y if running on an AMD system equipped with the PSP.
> +
>  config I2C_DESIGNWARE_BAYTRAIL
>  	bool "Intel Baytrail I2C semaphore support"
>  	depends on ACPI
> diff --git a/drivers/i2c/busses/Makefile b/drivers/i2c/busses/Makefile
> index 1d00dce77098..752f47be3fc1 100644
> --- a/drivers/i2c/busses/Makefile
> +++ b/drivers/i2c/busses/Makefile
> @@ -54,6 +54,7 @@ i2c-designware-core-y					+= i2c-designware-master.o
>  i2c-designware-core-$(CONFIG_I2C_DESIGNWARE_SLAVE) 	+= i2c-designware-slave.o
>  obj-$(CONFIG_I2C_DESIGNWARE_PLATFORM)			+= i2c-designware-platform.o
>  i2c-designware-platform-y 				:= i2c-designware-platdrv.o
> +i2c-designware-platform-$(CONFIG_I2C_DESIGNWARE_AMDPSP)	+= i2c-designware-amdpsp.o
>  i2c-designware-platform-$(CONFIG_I2C_DESIGNWARE_BAYTRAIL) += i2c-designware-baytrail.o
>  obj-$(CONFIG_I2C_DESIGNWARE_PCI)			+= i2c-designware-pci.o
>  i2c-designware-pci-y					:= i2c-designware-pcidrv.o
> diff --git a/drivers/i2c/busses/i2c-designware-amdpsp.c b/drivers/i2c/busses/i2c-designware-amdpsp.c
> new file mode 100644
> index 000000000000..a45e4faeb2c1
> --- /dev/null
> +++ b/drivers/i2c/busses/i2c-designware-amdpsp.c
> @@ -0,0 +1,389 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +#include <linux/bitfield.h>

IIRC this does not guarantee bits.h inclusion.

> +#include <linux/i2c.h>
> +#include <linux/io-64-nonatomic-lo-hi.h>
> +#include <linux/psp-sev.h>
> +#include <linux/types.h>
> +
> +#include <asm/msr.h>
> +
> +#include "i2c-designware-core.h"
> +
> +#define MSR_AMD_PSP_ADDR	0xc00110a2
> +#define PSP_MBOX_OFFSET		0x10570
> +#define PSP_CMD_TIMEOUT_MS	500
> +
> +#define PSP_I2C_REQ_BUS_CMD		0x64
> +#define PSP_I2C_REQ_RETRY_CNT		10
> +#define PSP_I2C_REQ_RETRY_DELAY_USEC	(50 * 1000)

_US is enough and possibly (50 * USEC_PER_MSEC).

> +#define PSP_I2C_REQ_STS_OK		0x0
> +#define PSP_I2C_REQ_STS_BUS_BUSY	0x1
> +#define PSP_I2C_REQ_STS_INV_PARAM	0x3
> +
> +#define PSP_MBOX_FIELDS_STS		GENMASK(15, 0)
> +#define PSP_MBOX_FIELDS_CMD		GENMASK(23, 16)
> +#define PSP_MBOX_FIELDS_RESERVED	GENMASK(29, 24)
> +#define PSP_MBOX_FIELDS_RECOVERY	BIT(30)
> +#define PSP_MBOX_FIELDS_READY		BIT(31)
> +
> +struct psp_req_buffer_hdr {
> +	u32 total_size;
> +	u32 status;
> +};
> +
> +enum psp_i2c_req_type {
> +	PSP_I2C_REQ_ACQUIRE,
> +	PSP_I2C_REQ_RELEASE,
> +	PSP_I2C_REQ_MAX
> +};
> +
> +struct psp_i2c_req {
> +	struct psp_req_buffer_hdr hdr;
> +	enum psp_i2c_req_type type;
> +} __aligned(32);

I forgot if this alignment is requirement due to DMA or so?
We may use ____cacheline_aligned in such case.

> +struct psp_mbox {
> +	u32 cmd_fields;
> +	u64 i2c_req_addr;
> +} __packed;
> +
> +static DEFINE_MUTEX(psp_i2c_access_mutex);
> +static unsigned long psp_i2c_sem_acquired;
> +static void __iomem *mbox_iomem;
> +static u32 psp_i2c_access_count;
> +static bool psp_i2c_mbox_fail;
> +static struct device *psp_i2c_dev;
> +
> +/*
> + * Implementation of PSP-x86 i2c-arbitration mailbox introduced for AMD Cezanne
> + * family of SoCs.
> + */
> +
> +static int psp_get_mbox_addr(unsigned long *mbox_addr)
> +{
> +	unsigned long long psp_mmio;
> +
> +	if (rdmsrl_safe(MSR_AMD_PSP_ADDR, &psp_mmio))
> +		return -EIO;
> +
> +	*mbox_addr = (unsigned long)(psp_mmio + PSP_MBOX_OFFSET);
> +
> +	return 0;
> +}
> +
> +static int psp_mbox_probe(void)
> +{
> +	unsigned long mbox_addr;
> +	int ret;
> +
> +	ret = psp_get_mbox_addr(&mbox_addr);
> +	if (ret)
> +		return ret;
> +
> +	mbox_iomem = ioremap(mbox_addr, sizeof(struct psp_mbox));
> +	if (!mbox_iomem)
> +		return -ENOMEM;
> +
> +	return 0;
> +}
> +
> +
> +/* Recovery field should be equal 0 to start sending commands */
> +static int psp_check_mbox_recovery(struct psp_mbox __iomem *mbox)
> +{
> +	u32 tmp;
> +
> +	tmp = readl(&mbox->cmd_fields);
> +
> +	return FIELD_GET(PSP_MBOX_FIELDS_RECOVERY, tmp);
> +}
> +
> +static int psp_wait_cmd(struct psp_mbox __iomem *mbox)
> +{
> +	u32 tmp, expected;
> +
> +	/* Expect mbox_cmd to be cleared and ready bit to be set by PSP */
> +	expected = FIELD_PREP(PSP_MBOX_FIELDS_READY, 1);
> +
> +	return readl_poll_timeout(&mbox->cmd_fields, tmp, (tmp == expected),
> +				  0, 1000 * PSP_CMD_TIMEOUT_MS);

0?!

> +}
> +
> +/* Status equal to 0 means that PSP succeed processing command */
> +static int psp_check_mbox_sts(struct psp_mbox __iomem *mbox)

  static u32 ...

So reader will know it won't return error codes.

> +{
> +	u32 cmd_reg;
> +
> +	cmd_reg = readl(&mbox->cmd_fields);
> +
> +	return FIELD_GET(PSP_MBOX_FIELDS_STS, cmd_reg);
> +}
> +
> +static int psp_send_cmd(struct psp_i2c_req *req)
> +{
> +	struct psp_mbox __iomem *mbox = mbox_iomem;
> +	phys_addr_t req_addr;
> +	u32 cmd_reg;
> +
> +	if (psp_check_mbox_recovery(mbox))
> +		return -EIO;
> +
> +	if (psp_wait_cmd(mbox))
> +		return -EBUSY;
> +
> +	/*
> +	 * Fill mailbox with address of command-response buffer, which will be
> +	 * used for sending i2c requests as well as reading status returned by
> +	 * PSP. Use physical address of buffer, since PSP will map this region.
> +	 */
> +	req_addr = __psp_pa((void *)req);
> +	writeq(req_addr, &mbox->i2c_req_addr);
> +
> +	/* Write command register to trigger processing */
> +	cmd_reg = FIELD_PREP(PSP_MBOX_FIELDS_CMD, PSP_I2C_REQ_BUS_CMD);
> +	writel(cmd_reg, &mbox->cmd_fields);
> +
> +	if (psp_wait_cmd(mbox))
> +		return -ETIMEDOUT;
> +
> +	if (psp_check_mbox_sts(mbox))
> +		return -EIO;
> +
> +	return 0;
> +}
> +
> +/* Helper to verify status returned by PSP */
> +static int check_i2c_req_sts(struct psp_i2c_req *req)
> +{
> +	int status;
> +
> +	status = readl(&req->hdr.status);
> +
> +	switch (status) {
> +	case PSP_I2C_REQ_STS_OK:
> +		return 0;
> +	case PSP_I2C_REQ_STS_BUS_BUSY:
> +		return -EBUSY;
> +	case PSP_I2C_REQ_STS_INV_PARAM:
> +	default:
> +		return -EIO;
> +	};
> +}
> +
> +static int psp_send_check_i2c_req(struct psp_i2c_req *req)
> +{
> +	/*
> +	 * Errors in x86-PSP i2c-arbitration protocol may occur at two levels:
> +	 * 1. mailbox communication - PSP is not operational or some IO errors
> +	 * with basic communication had happened;
> +	 * 2. i2c-requests - PSP refuses to grant i2c arbitration to x86 for too
> +	 * long.
> +	 * In order to distinguish between these two in error handling code, all
> +	 * errors on the first level (returned by psp_send_cmd) are shadowed by
> +	 * -EIO.
> +	 */
> +	if (psp_send_cmd(req))
> +		return -EIO;
> +
> +	return check_i2c_req_sts(req);
> +}
> +
> +static int psp_send_i2c_req(enum psp_i2c_req_type i2c_req_type)
> +{
> +	struct psp_i2c_req *req;
> +	unsigned long start;
> +	int status, ret;
> +
> +	/* Allocate command-response buffer */
> +	req = kzalloc(sizeof(*req), GFP_KERNEL);
> +	if (!req)
> +		return -ENOMEM;
> +
> +	req->hdr.total_size = sizeof(*req);
> +	req->type = i2c_req_type;
> +
> +	start = jiffies;
> +	ret = read_poll_timeout(psp_send_check_i2c_req, status,
> +				(status != -EBUSY),
> +				PSP_I2C_REQ_RETRY_DELAY_USEC,
> +				PSP_I2C_REQ_RETRY_CNT * PSP_I2C_REQ_RETRY_DELAY_USEC,
> +				0, req);
> +	if (ret)
> +		goto cleanup;

> +	if (status) {
> +		ret = status;
> +		goto cleanup;
> +	}

I would write it in this form:

	ret = status;
	if (ret)
		goto cleanup;

> +	dev_dbg(psp_i2c_dev, "Request accepted by PSP after %ums\n",
> +		jiffies_to_msecs(jiffies - start));
> +
> +cleanup:
> +	kfree(req);
> +	return ret;
> +}
> +
> +static int psp_acquire_i2c_bus(void)
> +{
> +	int status;
> +
> +	mutex_lock(&psp_i2c_access_mutex);
> +
> +	/* Return early if mailbox malfunctioned */
> +	if (psp_i2c_mbox_fail)
> +		goto cleanup;
> +
> +	/*
> +	 * Simply increment usage counter and return if PSP semaphore was
> +	 * already taken by kernel.
> +	 */
> +	if (psp_i2c_access_count > 0) {

' > 0' is redundant. It's unsigned.

> +		psp_i2c_access_count++;
> +		goto cleanup;
> +	};
> +
> +	status = psp_send_i2c_req(PSP_I2C_REQ_ACQUIRE);
> +	if (status) {
> +		if (status == -ETIMEDOUT)
> +			dev_err(psp_i2c_dev, "Timed out waiting for PSP to release I2C bus\n");
> +		else
> +			dev_err(psp_i2c_dev, "PSP communication error\n");
> +
> +		dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n");
> +		psp_i2c_mbox_fail = true;
> +		goto cleanup;
> +	}
> +
> +	psp_i2c_sem_acquired = jiffies;
> +	psp_i2c_access_count++;
> +
> +	/*
> +	 * In case of errors with PSP arbitrator psp_i2c_mbox_fail variable is
> +	 * set above. As a consequence consecutive calls to acquire will bypass
> +	 * communication with PSP. At any case i2c bus is granted to the caller,
> +	 * thus always return success.
> +	 */
> +cleanup:
> +	mutex_unlock(&psp_i2c_access_mutex);
> +	return 0;
> +}
> +
> +static void psp_release_i2c_bus(void)
> +{
> +	int status;
> +
> +	mutex_lock(&psp_i2c_access_mutex);
> +
> +	/* Return early if mailbox was malfunctional */
> +	if (psp_i2c_mbox_fail)
> +		goto cleanup;
> +
> +	/*
> +	 * If we are last owner of PSP semaphore, need to release aribtration
> +	 * via mailbox.
> +	 */
> +	psp_i2c_access_count--;

> +	if (psp_i2c_access_count > 0)

Ditto.

> +		goto cleanup;
> +
> +	/* Send a release command to PSP */
> +	status = psp_send_i2c_req(PSP_I2C_REQ_RELEASE);
> +	if (status) {
> +		if (status == -ETIMEDOUT)
> +			dev_err(psp_i2c_dev, "Timed out waiting for PSP to acquire I2C bus\n");
> +		else
> +			dev_err(psp_i2c_dev, "PSP communication error\n");
> +
> +		dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n");
> +		psp_i2c_mbox_fail = true;
> +		goto cleanup;
> +	}
> +
> +	dev_dbg(psp_i2c_dev, "PSP semaphore held for %ums\n",
> +		jiffies_to_msecs(jiffies - psp_i2c_sem_acquired));
> +
> +cleanup:
> +	mutex_unlock(&psp_i2c_access_mutex);
> +}
> +
> +/*
> + * Locking methods are based on the default implementation from
> + * drivers/i2c/i2c-core-base.c, but with psp acquire and release operations
> + * added. With this in place we can ensure that i2c clients on the bus shared
> + * with psp are able to lock HW access to the bus for arbitrary number of
> + * operations - that is e.g. write-wait-read.
> + */
> +static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter,
> +					unsigned int flags)
> +{
> +	psp_acquire_i2c_bus();
> +	rt_mutex_lock_nested(&adapter->bus_lock, i2c_adapter_depth(adapter));
> +}
> +
> +static int i2c_adapter_dw_psp_trylock_bus(struct i2c_adapter *adapter,
> +					  unsigned int flags)
> +{
> +	int ret;
> +
> +	ret = rt_mutex_trylock(&adapter->bus_lock);
> +	if (ret)
> +		return ret;
> +
> +	psp_acquire_i2c_bus();
> +
> +	return ret;
> +}
> +
> +static void i2c_adapter_dw_psp_unlock_bus(struct i2c_adapter *adapter,
> +					  unsigned int flags)
> +{
> +	psp_release_i2c_bus();
> +	rt_mutex_unlock(&adapter->bus_lock);
> +}
> +
> +static const struct i2c_lock_operations i2c_dw_psp_lock_ops = {
> +	.lock_bus = i2c_adapter_dw_psp_lock_bus,
> +	.trylock_bus = i2c_adapter_dw_psp_trylock_bus,
> +	.unlock_bus = i2c_adapter_dw_psp_unlock_bus,
> +};
> +
> +int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev)
> +{
> +	int ret;
> +
> +	if (!dev)
> +		return -ENODEV;
> +
> +	if (!(dev->flags & ARBITRATION_SEMAPHORE))
> +		return -ENODEV;
> +
> +	/* Allow to bind only one instance of a driver */
> +	if (psp_i2c_dev)
> +		return -EEXIST;
> +
> +	psp_i2c_dev = dev->dev;
> +
> +	ret = psp_mbox_probe();
> +	if (ret)
> +		return ret;
> +
> +	dev_info(psp_i2c_dev, "I2C bus managed by AMD PSP\n");
> +
> +	/*
> +	 * Install global locking callbacks for adapter as well as internal i2c
> +	 * controller locks.
> +	 */
> +	dev->adapter.lock_ops = &i2c_dw_psp_lock_ops;
> +	dev->acquire_lock = psp_acquire_i2c_bus;
> +	dev->release_lock = psp_release_i2c_bus;
> +
> +	return 0;
> +}
> +
> +/* Unmap area used as a mailbox with PSP */
> +void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev)
> +{
> +	iounmap(mbox_iomem);
> +}
> diff --git a/drivers/i2c/busses/i2c-designware-baytrail.c b/drivers/i2c/busses/i2c-designware-baytrail.c
> index c6a7a00e1d52..45774aa47c28 100644
> --- a/drivers/i2c/busses/i2c-designware-baytrail.c
> +++ b/drivers/i2c/busses/i2c-designware-baytrail.c
> @@ -12,25 +12,25 @@
>  
>  #include "i2c-designware-core.h"
>  
> -int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
> +int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev)
>  {
>  	acpi_status status;
>  	unsigned long long shared_host = 0;
>  	acpi_handle handle;
>  
> -	if (!dev || !dev->dev)
> -		return 0;
> +	if (!dev)
> +		return -ENODEV;
>  
>  	handle = ACPI_HANDLE(dev->dev);
>  	if (!handle)
> -		return 0;
> +		return -ENODEV;
>  
>  	status = acpi_evaluate_integer(handle, "_SEM", NULL, &shared_host);
>  	if (ACPI_FAILURE(status))
> -		return 0;
> +		return -ENODEV;
>  
>  	if (!shared_host)
> -		return 0;
> +		return -ENODEV;
>  
>  	if (!iosf_mbi_available())
>  		return -EPROBE_DEFER;
> diff --git a/drivers/i2c/busses/i2c-designware-core.h b/drivers/i2c/busses/i2c-designware-core.h
> index 4b26cba40139..1d65212fddbd 100644
> --- a/drivers/i2c/busses/i2c-designware-core.h
> +++ b/drivers/i2c/busses/i2c-designware-core.h
> @@ -227,6 +227,8 @@ struct reset_control;
>   * @hs_lcnt: high speed LCNT value
>   * @acquire_lock: function to acquire a hardware lock on the bus
>   * @release_lock: function to release a hardware lock on the bus
> + * @semaphore_idx: Index of table with semaphore type attached to the bus. It's
> + *	-1 if there is no semaphore.
>   * @shared_with_punit: true if this bus is shared with the SoCs PUNIT
>   * @disable: function to disable the controller
>   * @disable_int: function to disable all interrupts
> @@ -285,6 +287,7 @@ struct dw_i2c_dev {
>  	u16			hs_lcnt;
>  	int			(*acquire_lock)(void);
>  	void			(*release_lock)(void);
> +	int			semaphore_idx;
>  	bool			shared_with_punit;
>  	void			(*disable)(struct dw_i2c_dev *dev);
>  	void			(*disable_int)(struct dw_i2c_dev *dev);
> @@ -297,6 +300,7 @@ struct dw_i2c_dev {
>  
>  #define ACCESS_INTR_MASK	BIT(0)
>  #define ACCESS_NO_IRQ_SUSPEND	BIT(1)
> +#define ARBITRATION_SEMAPHORE	BIT(2)
>  
>  #define MODEL_MSCC_OCELOT	BIT(8)
>  #define MODEL_BAIKAL_BT1	BIT(9)
> @@ -310,6 +314,11 @@ struct dw_i2c_dev {
>  #define AMD_UCSI_INTR_REG	0x474
>  #define AMD_UCSI_INTR_EN	0xd
>  
> +struct i2c_dw_semaphore_callbacks {
> +	int	(*probe)(struct dw_i2c_dev *dev);
> +	void	(*remove)(struct dw_i2c_dev *dev);
> +};
> +
>  int i2c_dw_init_regmap(struct dw_i2c_dev *dev);
>  u32 i2c_dw_scl_hcnt(u32 ic_clk, u32 tSYMBOL, u32 tf, int cond, int offset);
>  u32 i2c_dw_scl_lcnt(u32 ic_clk, u32 tLOW, u32 tf, int offset);
> @@ -370,9 +379,12 @@ static inline void i2c_dw_configure(struct dw_i2c_dev *dev)
>  }
>  
>  #if IS_ENABLED(CONFIG_I2C_DESIGNWARE_BAYTRAIL)
> -extern int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev);
> -#else
> -static inline int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev) { return 0; }
> +int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev);
> +#endif
> +
> +#if IS_ENABLED(CONFIG_I2C_DESIGNWARE_AMDPSP)
> +int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev);
> +void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev);
>  #endif
>  
>  int i2c_dw_validate_speed(struct dw_i2c_dev *dev);
> diff --git a/drivers/i2c/busses/i2c-designware-platdrv.c b/drivers/i2c/busses/i2c-designware-platdrv.c
> index 2bd81abc86f6..9973ac894a51 100644
> --- a/drivers/i2c/busses/i2c-designware-platdrv.c
> +++ b/drivers/i2c/busses/i2c-designware-platdrv.c
> @@ -50,6 +50,7 @@ static const struct acpi_device_id dw_i2c_acpi_match[] = {
>  	{ "808622C1", ACCESS_NO_IRQ_SUSPEND },
>  	{ "AMD0010", ACCESS_INTR_MASK },
>  	{ "AMDI0010", ACCESS_INTR_MASK },
> +	{ "AMDI0019", ACCESS_INTR_MASK | ARBITRATION_SEMAPHORE },
>  	{ "AMDI0510", 0 },
>  	{ "APMC0D0F", 0 },
>  	{ "HISI02A1", 0 },
> @@ -204,6 +205,63 @@ static const struct dmi_system_id dw_i2c_hwmon_class_dmi[] = {
>  	{ } /* terminate list */
>  };
>  
> +static const struct i2c_dw_semaphore_callbacks i2c_dw_semaphore_cb_table[] = {
> +#ifdef CONFIG_I2C_DESIGNWARE_BAYTRAIL
> +	{
> +		.probe = i2c_dw_baytrail_probe_lock_support,
> +	},
> +#endif
> +#ifdef CONFIG_I2C_DESIGNWARE_AMDPSP
> +	{
> +		.probe = i2c_dw_amdpsp_probe_lock_support,
> +		.remove = i2c_dw_amdpsp_remove_lock_support,
> +	},
> +#endif
> +	{}
> +};
> +
> +static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
> +{
> +	const struct i2c_dw_semaphore_callbacks *ptr;
> +	int i = 0;
> +	int ret;
> +
> +	ptr = i2c_dw_semaphore_cb_table;
> +
> +	dev->semaphore_idx = -1;
> +
> +	while (ptr->probe) {
> +		ret = ptr->probe(dev);
> +		if (ret) {
> +			/*
> +			 * If there is no semaphore device attached to this
> +			 * controller, we shouldn't abort general i2c_controller
> +			 * probe.
> +			 */
> +			if (ret != -ENODEV)
> +				return ret;
> +
> +			i++;
> +			ptr++;
> +			continue;
> +		}
> +
> +		dev->semaphore_idx = i;
> +		break;
> +	}
> +
> +	return 0;
> +}
> +
> +static void i2c_dw_remove_lock_support(struct dw_i2c_dev *dev)
> +{
> +	if (dev->semaphore_idx < 0)
> +		return;
> +
> +	if (i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove)
> +		i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove(dev);
> +}
> +
>  static int dw_i2c_plat_probe(struct platform_device *pdev)
>  {
>  	struct i2c_adapter *adap;
> @@ -334,6 +392,8 @@ static int dw_i2c_plat_remove(struct platform_device *pdev)
>  	pm_runtime_put_sync(&pdev->dev);
>  	dw_i2c_plat_pm_cleanup(dev);
>  
> +	i2c_dw_remove_lock_support(dev);
> +
>  	reset_control_assert(dev->rst);
>  
>  	return 0;
> -- 
> 2.35.0.rc2.247.g8bbb082509-goog
> 

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v3 2/2] i2c: designware: Add AMD PSP I2C bus support

[Limonciello, Mario]

On 2/2/2022 08:43, Jan Dabros wrote:
> Implement an I2C controller sharing mechanism between the host (kernel)
> and PSP co-processor on some platforms equipped with AMD Cezanne SoC.
> 
> On these platforms we need to implement "software" i2c arbitration.
> Default arbitration owner is PSP and kernel asks for acquire as well
> as inform about release of the i2c bus via mailbox mechanism.
> 
>              +---------+
>   <- ACQUIRE |         |
>    +---------|   CPU   |\
>    |         |         | \      +----------+  SDA
>    |         +---------+  \     |          |-------
> MAILBOX                   +--> |  I2C-DW  |  SCL
>    |         +---------+        |          |-------
>    |         |         |        +----------+
>    +---------|   PSP   |
>     <- ACK   |         |
>              +---------+
> 
>              +---------+
>   <- RELEASE |         |
>    +---------|   CPU   |
>    |         |         |        +----------+  SDA
>    |         +---------+        |          |-------
> MAILBOX                   +--> |  I2C-DW  |  SCL
>    |         +---------+  /     |          |-------
>    |         |         | /      +----------+
>    +---------|   PSP   |/
>     <- ACK   |         |
>              +---------+
> 
> The solution is similar to i2c-designware-baytrail.c implementation, where
> we are using a generic i2c-designware-* driver with a small "wrapper".
> 
> In contrary to baytrail semaphore implementation, beside internal
> acquire_lock() and release_lock() methods we are also applying quirks to
> lock_bus() and unlock_bus() global adapter methods. With this in place
> all i2c clients drivers may lock i2c bus for a desired number of i2c
> transactions (e.g. write-wait-read) without being aware of that such bus
> is shared with another entity.
> 
> Modify i2c_dw_probe_lock_support() to select correct semaphore
> implementation at runtime, since now we have more than one available.
> 
> Configure new matching ACPI ID "AMDI0019" and register
> ARBITRATION_SEMAPHORE flag in order to distinguish setup with PSP
> arbitration.
> 
> Add myself as a reviewer for I2C DesignWare in order to help with reviewing
> and testing possible changes touching new i2c-designware-amdpsp.c module.
> 
> Signed-off-by: Jan Dabros <jsd@semihalf.com>
> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> ---
>   MAINTAINERS                                  |   1 +
>   drivers/acpi/acpi_apd.c                      |   7 +-
>   drivers/i2c/busses/Kconfig                   |  11 +
>   drivers/i2c/busses/Makefile                  |   1 +
>   drivers/i2c/busses/i2c-designware-amdpsp.c   | 389 +++++++++++++++++++
>   drivers/i2c/busses/i2c-designware-baytrail.c |  12 +-
>   drivers/i2c/busses/i2c-designware-core.h     |  18 +-
>   drivers/i2c/busses/i2c-designware-platdrv.c  |  60 +++
>   8 files changed, 487 insertions(+), 12 deletions(-)
>   create mode 100644 drivers/i2c/busses/i2c-designware-amdpsp.c
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index f41088418aae..e0a7e05c5a41 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -18666,6 +18666,7 @@ SYNOPSYS DESIGNWARE I2C DRIVER
>   M:	Jarkko Nikula <jarkko.nikula@linux.intel.com>
>   R:	Andy Shevchenko <andriy.shevchenko@linux.intel.com>
>   R:	Mika Westerberg <mika.westerberg@linux.intel.com>
> +R:	Jan Dabros <jsd@semihalf.com>
>   L:	linux-i2c@vger.kernel.org
>   S:	Maintained
>   F:	drivers/i2c/busses/i2c-designware-*
> diff --git a/drivers/acpi/acpi_apd.c b/drivers/acpi/acpi_apd.c
> index e7934ba79b02..ad245bbd965e 100644
> --- a/drivers/acpi/acpi_apd.c
> +++ b/drivers/acpi/acpi_apd.c
> @@ -232,12 +232,13 @@ static const struct acpi_device_id acpi_apd_device_ids[] = {
>   	/* Generic apd devices */
>   #ifdef CONFIG_X86_AMD_PLATFORM_DEVICE
>   	{ "AMD0010", APD_ADDR(cz_i2c_desc) },
> -	{ "AMDI0010", APD_ADDR(wt_i2c_desc) },
>   	{ "AMD0020", APD_ADDR(cz_uart_desc) },
> -	{ "AMDI0020", APD_ADDR(cz_uart_desc) },
> -	{ "AMDI0022", APD_ADDR(cz_uart_desc) },
>   	{ "AMD0030", },
>   	{ "AMD0040", APD_ADDR(fch_misc_desc)},
> +	{ "AMDI0010", APD_ADDR(wt_i2c_desc) },
> +	{ "AMDI0019", APD_ADDR(wt_i2c_desc) },

Did you already check with anyone in AMD about reserving this ID?

Considering this design is specific to arbitration used for Chromebooks 
I wonder if it is better to be GOOG ID?

I can see it both ways, but if you didn't already talk to anyone and the 
consensus is for it to be AMDI****, I will ask around internally about 
reserving it / making sure there are no future or existing conflicts.

> +	{ "AMDI0020", APD_ADDR(cz_uart_desc) },
> +	{ "AMDI0022", APD_ADDR(cz_uart_desc) },
>   	{ "HYGO0010", APD_ADDR(wt_i2c_desc) },
>   #endif
>   #ifdef CONFIG_ARM64
> diff --git a/drivers/i2c/busses/Kconfig b/drivers/i2c/busses/Kconfig
> index 42da31c1ab70..5ea26a7aa1d5 100644
> --- a/drivers/i2c/busses/Kconfig
> +++ b/drivers/i2c/busses/Kconfig
> @@ -553,6 +553,17 @@ config I2C_DESIGNWARE_PLATFORM
>   	  This driver can also be built as a module.  If so, the module
>   	  will be called i2c-designware-platform.
>   
> +config I2C_DESIGNWARE_AMDPSP
> +	bool "AMD PSP I2C semaphore support"
> +	depends on X86_MSR
> +	depends on ACPI
> +	depends on I2C_DESIGNWARE_PLATFORM
> +	help
> +	  This driver enables managed host access to the selected I2C bus shared
> +	  between AMD CPU and AMD PSP.
> +
> +	  You should say Y if running on an AMD system equipped with the PSP.
> +
>   config I2C_DESIGNWARE_BAYTRAIL
>   	bool "Intel Baytrail I2C semaphore support"
>   	depends on ACPI
> diff --git a/drivers/i2c/busses/Makefile b/drivers/i2c/busses/Makefile
> index 1d00dce77098..752f47be3fc1 100644
> --- a/drivers/i2c/busses/Makefile
> +++ b/drivers/i2c/busses/Makefile
> @@ -54,6 +54,7 @@ i2c-designware-core-y					+= i2c-designware-master.o
>   i2c-designware-core-$(CONFIG_I2C_DESIGNWARE_SLAVE) 	+= i2c-designware-slave.o
>   obj-$(CONFIG_I2C_DESIGNWARE_PLATFORM)			+= i2c-designware-platform.o
>   i2c-designware-platform-y 				:= i2c-designware-platdrv.o
> +i2c-designware-platform-$(CONFIG_I2C_DESIGNWARE_AMDPSP)	+= i2c-designware-amdpsp.o
>   i2c-designware-platform-$(CONFIG_I2C_DESIGNWARE_BAYTRAIL) += i2c-designware-baytrail.o
>   obj-$(CONFIG_I2C_DESIGNWARE_PCI)			+= i2c-designware-pci.o
>   i2c-designware-pci-y					:= i2c-designware-pcidrv.o
> diff --git a/drivers/i2c/busses/i2c-designware-amdpsp.c b/drivers/i2c/busses/i2c-designware-amdpsp.c
> new file mode 100644
> index 000000000000..a45e4faeb2c1
> --- /dev/null
> +++ b/drivers/i2c/busses/i2c-designware-amdpsp.c
> @@ -0,0 +1,389 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +#include <linux/bitfield.h>
> +#include <linux/i2c.h>
> +#include <linux/io-64-nonatomic-lo-hi.h>
> +#include <linux/psp-sev.h>
> +#include <linux/types.h>
> +
> +#include <asm/msr.h>
> +
> +#include "i2c-designware-core.h"
> +
> +#define MSR_AMD_PSP_ADDR	0xc00110a2
> +#define PSP_MBOX_OFFSET		0x10570
> +#define PSP_CMD_TIMEOUT_MS	500
> +
> +#define PSP_I2C_REQ_BUS_CMD		0x64
> +#define PSP_I2C_REQ_RETRY_CNT		10
> +#define PSP_I2C_REQ_RETRY_DELAY_USEC	(50 * 1000)
> +#define PSP_I2C_REQ_STS_OK		0x0
> +#define PSP_I2C_REQ_STS_BUS_BUSY	0x1
> +#define PSP_I2C_REQ_STS_INV_PARAM	0x3
> +
> +#define PSP_MBOX_FIELDS_STS		GENMASK(15, 0)
> +#define PSP_MBOX_FIELDS_CMD		GENMASK(23, 16)
> +#define PSP_MBOX_FIELDS_RESERVED	GENMASK(29, 24)
> +#define PSP_MBOX_FIELDS_RECOVERY	BIT(30)
> +#define PSP_MBOX_FIELDS_READY		BIT(31)
> +
> +struct psp_req_buffer_hdr {
> +	u32 total_size;
> +	u32 status;
> +};
> +
> +enum psp_i2c_req_type {
> +	PSP_I2C_REQ_ACQUIRE,
> +	PSP_I2C_REQ_RELEASE,
> +	PSP_I2C_REQ_MAX
> +};
> +
> +struct psp_i2c_req {
> +	struct psp_req_buffer_hdr hdr;
> +	enum psp_i2c_req_type type;
> +} __aligned(32);
> +
> +struct psp_mbox {
> +	u32 cmd_fields;
> +	u64 i2c_req_addr;
> +} __packed;
> +
> +static DEFINE_MUTEX(psp_i2c_access_mutex);
> +static unsigned long psp_i2c_sem_acquired;
> +static void __iomem *mbox_iomem;
> +static u32 psp_i2c_access_count;
> +static bool psp_i2c_mbox_fail;
> +static struct device *psp_i2c_dev;
> +
> +/*
> + * Implementation of PSP-x86 i2c-arbitration mailbox introduced for AMD Cezanne
> + * family of SoCs.
> + */
> +
> +static int psp_get_mbox_addr(unsigned long *mbox_addr)
> +{
> +	unsigned long long psp_mmio;
> +
> +	if (rdmsrl_safe(MSR_AMD_PSP_ADDR, &psp_mmio))
> +		return -EIO;
> +
> +	*mbox_addr = (unsigned long)(psp_mmio + PSP_MBOX_OFFSET);
> +
> +	return 0;
> +}
> +
> +static int psp_mbox_probe(void)
> +{
> +	unsigned long mbox_addr;
> +	int ret;
> +
> +	ret = psp_get_mbox_addr(&mbox_addr);
> +	if (ret)
> +		return ret;
> +
> +	mbox_iomem = ioremap(mbox_addr, sizeof(struct psp_mbox));
> +	if (!mbox_iomem)
> +		return -ENOMEM;
> +
> +	return 0;
> +}
> +
> +
> +/* Recovery field should be equal 0 to start sending commands */
> +static int psp_check_mbox_recovery(struct psp_mbox __iomem *mbox)
> +{
> +	u32 tmp;
> +
> +	tmp = readl(&mbox->cmd_fields);
> +
> +	return FIELD_GET(PSP_MBOX_FIELDS_RECOVERY, tmp);
> +}
> +
> +static int psp_wait_cmd(struct psp_mbox __iomem *mbox)
> +{
> +	u32 tmp, expected;
> +
> +	/* Expect mbox_cmd to be cleared and ready bit to be set by PSP */
> +	expected = FIELD_PREP(PSP_MBOX_FIELDS_READY, 1);
> +
> +	return readl_poll_timeout(&mbox->cmd_fields, tmp, (tmp == expected),
> +				  0, 1000 * PSP_CMD_TIMEOUT_MS);
> +}
> +
> +/* Status equal to 0 means that PSP succeed processing command */
> +static int psp_check_mbox_sts(struct psp_mbox __iomem *mbox)
> +{
> +	u32 cmd_reg;
> +
> +	cmd_reg = readl(&mbox->cmd_fields);
> +
> +	return FIELD_GET(PSP_MBOX_FIELDS_STS, cmd_reg);
> +}
> +
> +static int psp_send_cmd(struct psp_i2c_req *req)
> +{
> +	struct psp_mbox __iomem *mbox = mbox_iomem;
> +	phys_addr_t req_addr;
> +	u32 cmd_reg;
> +
> +	if (psp_check_mbox_recovery(mbox))
> +		return -EIO;
> +
> +	if (psp_wait_cmd(mbox))
> +		return -EBUSY;
> +
> +	/*
> +	 * Fill mailbox with address of command-response buffer, which will be
> +	 * used for sending i2c requests as well as reading status returned by
> +	 * PSP. Use physical address of buffer, since PSP will map this region.
> +	 */
> +	req_addr = __psp_pa((void *)req);
> +	writeq(req_addr, &mbox->i2c_req_addr);
> +
> +	/* Write command register to trigger processing */
> +	cmd_reg = FIELD_PREP(PSP_MBOX_FIELDS_CMD, PSP_I2C_REQ_BUS_CMD);
> +	writel(cmd_reg, &mbox->cmd_fields);
> +
> +	if (psp_wait_cmd(mbox))
> +		return -ETIMEDOUT;
> +
> +	if (psp_check_mbox_sts(mbox))
> +		return -EIO;
> +
> +	return 0;
> +}
> +
> +/* Helper to verify status returned by PSP */
> +static int check_i2c_req_sts(struct psp_i2c_req *req)
> +{
> +	int status;
> +
> +	status = readl(&req->hdr.status);
> +
> +	switch (status) {
> +	case PSP_I2C_REQ_STS_OK:
> +		return 0;
> +	case PSP_I2C_REQ_STS_BUS_BUSY:
> +		return -EBUSY;
> +	case PSP_I2C_REQ_STS_INV_PARAM:
> +	default:
> +		return -EIO;
> +	};
> +}
> +
> +static int psp_send_check_i2c_req(struct psp_i2c_req *req)
> +{
> +	/*
> +	 * Errors in x86-PSP i2c-arbitration protocol may occur at two levels:
> +	 * 1. mailbox communication - PSP is not operational or some IO errors
> +	 * with basic communication had happened;
> +	 * 2. i2c-requests - PSP refuses to grant i2c arbitration to x86 for too
> +	 * long.
> +	 * In order to distinguish between these two in error handling code, all
> +	 * errors on the first level (returned by psp_send_cmd) are shadowed by
> +	 * -EIO.
> +	 */
> +	if (psp_send_cmd(req))
> +		return -EIO;
> +
> +	return check_i2c_req_sts(req);
> +}
> +
> +static int psp_send_i2c_req(enum psp_i2c_req_type i2c_req_type)
> +{
> +	struct psp_i2c_req *req;
> +	unsigned long start;
> +	int status, ret;
> +
> +	/* Allocate command-response buffer */
> +	req = kzalloc(sizeof(*req), GFP_KERNEL);
> +	if (!req)
> +		return -ENOMEM;
> +
> +	req->hdr.total_size = sizeof(*req);
> +	req->type = i2c_req_type;
> +
> +	start = jiffies;
> +	ret = read_poll_timeout(psp_send_check_i2c_req, status,
> +				(status != -EBUSY),
> +				PSP_I2C_REQ_RETRY_DELAY_USEC,
> +				PSP_I2C_REQ_RETRY_CNT * PSP_I2C_REQ_RETRY_DELAY_USEC,
> +				0, req);
> +	if (ret)
> +		goto cleanup;
> +
> +	if (status) {
> +		ret = status;
> +		goto cleanup;
> +	}
> +
> +	dev_dbg(psp_i2c_dev, "Request accepted by PSP after %ums\n",
> +		jiffies_to_msecs(jiffies - start));
> +
> +cleanup:
> +	kfree(req);
> +	return ret;
> +}
> +
> +static int psp_acquire_i2c_bus(void)
> +{
> +	int status;
> +
> +	mutex_lock(&psp_i2c_access_mutex);
> +
> +	/* Return early if mailbox malfunctioned */
> +	if (psp_i2c_mbox_fail)
> +		goto cleanup;
> +
> +	/*
> +	 * Simply increment usage counter and return if PSP semaphore was
> +	 * already taken by kernel.
> +	 */
> +	if (psp_i2c_access_count > 0) {
> +		psp_i2c_access_count++;
> +		goto cleanup;
> +	};
> +
> +	status = psp_send_i2c_req(PSP_I2C_REQ_ACQUIRE);
> +	if (status) {
> +		if (status == -ETIMEDOUT)
> +			dev_err(psp_i2c_dev, "Timed out waiting for PSP to release I2C bus\n");
> +		else
> +			dev_err(psp_i2c_dev, "PSP communication error\n");
> +
> +		dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n");
> +		psp_i2c_mbox_fail = true;
> +		goto cleanup;
> +	}
> +
> +	psp_i2c_sem_acquired = jiffies;
> +	psp_i2c_access_count++;
> +
> +	/*
> +	 * In case of errors with PSP arbitrator psp_i2c_mbox_fail variable is
> +	 * set above. As a consequence consecutive calls to acquire will bypass
> +	 * communication with PSP. At any case i2c bus is granted to the caller,
> +	 * thus always return success.
> +	 */
> +cleanup:
> +	mutex_unlock(&psp_i2c_access_mutex);
> +	return 0;
> +}
> +
> +static void psp_release_i2c_bus(void)
> +{
> +	int status;
> +
> +	mutex_lock(&psp_i2c_access_mutex);
> +
> +	/* Return early if mailbox was malfunctional */
> +	if (psp_i2c_mbox_fail)
> +		goto cleanup;
> +
> +	/*
> +	 * If we are last owner of PSP semaphore, need to release aribtration
> +	 * via mailbox.
> +	 */
> +	psp_i2c_access_count--;
> +	if (psp_i2c_access_count > 0)
> +		goto cleanup;
> +
> +	/* Send a release command to PSP */
> +	status = psp_send_i2c_req(PSP_I2C_REQ_RELEASE);
> +	if (status) {
> +		if (status == -ETIMEDOUT)
> +			dev_err(psp_i2c_dev, "Timed out waiting for PSP to acquire I2C bus\n");
> +		else
> +			dev_err(psp_i2c_dev, "PSP communication error\n");
> +
> +		dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n");
> +		psp_i2c_mbox_fail = true;
> +		goto cleanup;
> +	}
> +
> +	dev_dbg(psp_i2c_dev, "PSP semaphore held for %ums\n",
> +		jiffies_to_msecs(jiffies - psp_i2c_sem_acquired));
> +
> +cleanup:
> +	mutex_unlock(&psp_i2c_access_mutex);
> +}
> +
> +/*
> + * Locking methods are based on the default implementation from
> + * drivers/i2c/i2c-core-base.c, but with psp acquire and release operations
> + * added. With this in place we can ensure that i2c clients on the bus shared
> + * with psp are able to lock HW access to the bus for arbitrary number of
> + * operations - that is e.g. write-wait-read.
> + */
> +static void i2c_adapter_dw_psp_lock_bus(struct i2c_adapter *adapter,
> +					unsigned int flags)
> +{
> +	psp_acquire_i2c_bus();
> +	rt_mutex_lock_nested(&adapter->bus_lock, i2c_adapter_depth(adapter));
> +}
> +
> +static int i2c_adapter_dw_psp_trylock_bus(struct i2c_adapter *adapter,
> +					  unsigned int flags)
> +{
> +	int ret;
> +
> +	ret = rt_mutex_trylock(&adapter->bus_lock);
> +	if (ret)
> +		return ret;
> +
> +	psp_acquire_i2c_bus();
> +
> +	return ret;
> +}
> +
> +static void i2c_adapter_dw_psp_unlock_bus(struct i2c_adapter *adapter,
> +					  unsigned int flags)
> +{
> +	psp_release_i2c_bus();
> +	rt_mutex_unlock(&adapter->bus_lock);
> +}
> +
> +static const struct i2c_lock_operations i2c_dw_psp_lock_ops = {
> +	.lock_bus = i2c_adapter_dw_psp_lock_bus,
> +	.trylock_bus = i2c_adapter_dw_psp_trylock_bus,
> +	.unlock_bus = i2c_adapter_dw_psp_unlock_bus,
> +};
> +
> +int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev)
> +{
> +	int ret;
> +
> +	if (!dev)
> +		return -ENODEV;
> +
> +	if (!(dev->flags & ARBITRATION_SEMAPHORE))
> +		return -ENODEV;
> +
> +	/* Allow to bind only one instance of a driver */
> +	if (psp_i2c_dev)
> +		return -EEXIST;
> +
> +	psp_i2c_dev = dev->dev;
> +
> +	ret = psp_mbox_probe();
> +	if (ret)
> +		return ret;
> +
> +	dev_info(psp_i2c_dev, "I2C bus managed by AMD PSP\n");
> +
> +	/*
> +	 * Install global locking callbacks for adapter as well as internal i2c
> +	 * controller locks.
> +	 */
> +	dev->adapter.lock_ops = &i2c_dw_psp_lock_ops;
> +	dev->acquire_lock = psp_acquire_i2c_bus;
> +	dev->release_lock = psp_release_i2c_bus;
> +
> +	return 0;
> +}
> +
> +/* Unmap area used as a mailbox with PSP */
> +void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev)
> +{
> +	iounmap(mbox_iomem);
> +}
> diff --git a/drivers/i2c/busses/i2c-designware-baytrail.c b/drivers/i2c/busses/i2c-designware-baytrail.c
> index c6a7a00e1d52..45774aa47c28 100644
> --- a/drivers/i2c/busses/i2c-designware-baytrail.c
> +++ b/drivers/i2c/busses/i2c-designware-baytrail.c
> @@ -12,25 +12,25 @@
>   
>   #include "i2c-designware-core.h"
>   
> -int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
> +int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev)
>   {
>   	acpi_status status;
>   	unsigned long long shared_host = 0;
>   	acpi_handle handle;
>   
> -	if (!dev || !dev->dev)
> -		return 0;
> +	if (!dev)
> +		return -ENODEV;
>   
>   	handle = ACPI_HANDLE(dev->dev);
>   	if (!handle)
> -		return 0;
> +		return -ENODEV;
>   
>   	status = acpi_evaluate_integer(handle, "_SEM", NULL, &shared_host);
>   	if (ACPI_FAILURE(status))
> -		return 0;
> +		return -ENODEV;
>   
>   	if (!shared_host)
> -		return 0;
> +		return -ENODEV;
>   
>   	if (!iosf_mbi_available())
>   		return -EPROBE_DEFER;
> diff --git a/drivers/i2c/busses/i2c-designware-core.h b/drivers/i2c/busses/i2c-designware-core.h
> index 4b26cba40139..1d65212fddbd 100644
> --- a/drivers/i2c/busses/i2c-designware-core.h
> +++ b/drivers/i2c/busses/i2c-designware-core.h
> @@ -227,6 +227,8 @@ struct reset_control;
>    * @hs_lcnt: high speed LCNT value
>    * @acquire_lock: function to acquire a hardware lock on the bus
>    * @release_lock: function to release a hardware lock on the bus
> + * @semaphore_idx: Index of table with semaphore type attached to the bus. It's
> + *	-1 if there is no semaphore.
>    * @shared_with_punit: true if this bus is shared with the SoCs PUNIT
>    * @disable: function to disable the controller
>    * @disable_int: function to disable all interrupts
> @@ -285,6 +287,7 @@ struct dw_i2c_dev {
>   	u16			hs_lcnt;
>   	int			(*acquire_lock)(void);
>   	void			(*release_lock)(void);
> +	int			semaphore_idx;
>   	bool			shared_with_punit;
>   	void			(*disable)(struct dw_i2c_dev *dev);
>   	void			(*disable_int)(struct dw_i2c_dev *dev);
> @@ -297,6 +300,7 @@ struct dw_i2c_dev {
>   
>   #define ACCESS_INTR_MASK	BIT(0)
>   #define ACCESS_NO_IRQ_SUSPEND	BIT(1)
> +#define ARBITRATION_SEMAPHORE	BIT(2)
>   
>   #define MODEL_MSCC_OCELOT	BIT(8)
>   #define MODEL_BAIKAL_BT1	BIT(9)
> @@ -310,6 +314,11 @@ struct dw_i2c_dev {
>   #define AMD_UCSI_INTR_REG	0x474
>   #define AMD_UCSI_INTR_EN	0xd
>   
> +struct i2c_dw_semaphore_callbacks {
> +	int	(*probe)(struct dw_i2c_dev *dev);
> +	void	(*remove)(struct dw_i2c_dev *dev);
> +};
> +
>   int i2c_dw_init_regmap(struct dw_i2c_dev *dev);
>   u32 i2c_dw_scl_hcnt(u32 ic_clk, u32 tSYMBOL, u32 tf, int cond, int offset);
>   u32 i2c_dw_scl_lcnt(u32 ic_clk, u32 tLOW, u32 tf, int offset);
> @@ -370,9 +379,12 @@ static inline void i2c_dw_configure(struct dw_i2c_dev *dev)
>   }
>   
>   #if IS_ENABLED(CONFIG_I2C_DESIGNWARE_BAYTRAIL)
> -extern int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev);
> -#else
> -static inline int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev) { return 0; }
> +int i2c_dw_baytrail_probe_lock_support(struct dw_i2c_dev *dev);
> +#endif
> +
> +#if IS_ENABLED(CONFIG_I2C_DESIGNWARE_AMDPSP)
> +int i2c_dw_amdpsp_probe_lock_support(struct dw_i2c_dev *dev);
> +void i2c_dw_amdpsp_remove_lock_support(struct dw_i2c_dev *dev);
>   #endif
>   
>   int i2c_dw_validate_speed(struct dw_i2c_dev *dev);
> diff --git a/drivers/i2c/busses/i2c-designware-platdrv.c b/drivers/i2c/busses/i2c-designware-platdrv.c
> index 2bd81abc86f6..9973ac894a51 100644
> --- a/drivers/i2c/busses/i2c-designware-platdrv.c
> +++ b/drivers/i2c/busses/i2c-designware-platdrv.c
> @@ -50,6 +50,7 @@ static const struct acpi_device_id dw_i2c_acpi_match[] = {
>   	{ "808622C1", ACCESS_NO_IRQ_SUSPEND },
>   	{ "AMD0010", ACCESS_INTR_MASK },
>   	{ "AMDI0010", ACCESS_INTR_MASK },
> +	{ "AMDI0019", ACCESS_INTR_MASK | ARBITRATION_SEMAPHORE },
>   	{ "AMDI0510", 0 },
>   	{ "APMC0D0F", 0 },
>   	{ "HISI02A1", 0 },
> @@ -204,6 +205,63 @@ static const struct dmi_system_id dw_i2c_hwmon_class_dmi[] = {
>   	{ } /* terminate list */
>   };
>   
> +static const struct i2c_dw_semaphore_callbacks i2c_dw_semaphore_cb_table[] = {
> +#ifdef CONFIG_I2C_DESIGNWARE_BAYTRAIL
> +	{
> +		.probe = i2c_dw_baytrail_probe_lock_support,
> +	},
> +#endif
> +#ifdef CONFIG_I2C_DESIGNWARE_AMDPSP
> +	{
> +		.probe = i2c_dw_amdpsp_probe_lock_support,
> +		.remove = i2c_dw_amdpsp_remove_lock_support,
> +	},
> +#endif
> +	{}
> +};
> +
> +static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev)
> +{
> +	const struct i2c_dw_semaphore_callbacks *ptr;
> +	int i = 0;
> +	int ret;
> +
> +	ptr = i2c_dw_semaphore_cb_table;
> +
> +	dev->semaphore_idx = -1;
> +
> +	while (ptr->probe) {
> +		ret = ptr->probe(dev);
> +		if (ret) {
> +			/*
> +			 * If there is no semaphore device attached to this
> +			 * controller, we shouldn't abort general i2c_controller
> +			 * probe.
> +			 */
> +			if (ret != -ENODEV)
> +				return ret;
> +
> +			i++;
> +			ptr++;
> +			continue;
> +		}
> +
> +		dev->semaphore_idx = i;
> +		break;
> +	}
> +
> +	return 0;
> +}
> +
> +static void i2c_dw_remove_lock_support(struct dw_i2c_dev *dev)
> +{
> +	if (dev->semaphore_idx < 0)
> +		return;
> +
> +	if (i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove)
> +		i2c_dw_semaphore_cb_table[dev->semaphore_idx].remove(dev);
> +}
> +
>   static int dw_i2c_plat_probe(struct platform_device *pdev)
>   {
>   	struct i2c_adapter *adap;
> @@ -334,6 +392,8 @@ static int dw_i2c_plat_remove(struct platform_device *pdev)
>   	pm_runtime_put_sync(&pdev->dev);
>   	dw_i2c_plat_pm_cleanup(dev);
>   
> +	i2c_dw_remove_lock_support(dev);
> +
>   	reset_control_assert(dev->rst);
>   
>   	return 0;

Re: [PATCH v3 2/2] i2c: designware: Add AMD PSP I2C bus support

[Andy Shevchenko]

On Wed, Feb 02, 2022 at 04:49:40PM -0600, Limonciello, Mario wrote:
> On 2/2/2022 08:43, Jan Dabros wrote:

...

> > +	{ "AMDI0019", APD_ADDR(wt_i2c_desc) },
> 
> Did you already check with anyone in AMD about reserving this ID?

Oh, it's a very good point! I was under impression that Jan operates on
allocated IDs...

> Considering this design is specific to arbitration used for Chromebooks I
> wonder if it is better to be GOOG ID?
> 
> I can see it both ways, but if you didn't already talk to anyone and the
> consensus is for it to be AMDI****, I will ask around internally about
> reserving it / making sure there are no future or existing conflicts.

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v3 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dąbroś]

Hi Andy, Mario,

czw., 3 lut 2022 o 11:43 Andy Shevchenko
<andriy.shevchenko@linux.intel.com> napisał(a):
>
> On Wed, Feb 02, 2022 at 04:49:40PM -0600, Limonciello, Mario wrote:
> > On 2/2/2022 08:43, Jan Dabros wrote:
>
> ...
>
> > > +   { "AMDI0019", APD_ADDR(wt_i2c_desc) },
> >
> > Did you already check with anyone in AMD about reserving this ID?
>
> Oh, it's a very good point! I was under impression that Jan operates on
> allocated IDs...

Yes, I received this ID from AMD.

>
> > Considering this design is specific to arbitration used for Chromebooks I
> > wonder if it is better to be GOOG ID?
> >
> > I can see it both ways, but if you didn't already talk to anyone and the
> > consensus is for it to be AMDI****, I will ask around internally about
> > reserving it / making sure there are no future or existing conflicts.

It was agreed with AMD that we will use "AMDI****" format. This ID is
already reserved by them.

Best Regards,
Jan

>
> --
> With Best Regards,
> Andy Shevchenko
>
>

Re: [PATCH v3 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dąbroś]

Hi Andy,

śr., 2 lut 2022 o 17:16 Andy Shevchenko
<andriy.shevchenko@linux.intel.com> napisał(a):
>
> On Wed, Feb 02, 2022 at 03:43:02PM +0100, Jan Dabros wrote:
> > Implement an I2C controller sharing mechanism between the host (kernel)
> > and PSP co-processor on some platforms equipped with AMD Cezanne SoC.
> >
> > On these platforms we need to implement "software" i2c arbitration.
> > Default arbitration owner is PSP and kernel asks for acquire as well
> > as inform about release of the i2c bus via mailbox mechanism.
> >
> >             +---------+
> >  <- ACQUIRE |         |
> >   +---------|   CPU   |\
> >   |         |         | \      +----------+  SDA
> >   |         +---------+  \     |          |-------
> > MAILBOX                   +--> |  I2C-DW  |  SCL
> >   |         +---------+        |          |-------
> >   |         |         |        +----------+
> >   +---------|   PSP   |
> >    <- ACK   |         |
> >             +---------+
> >
> >             +---------+
> >  <- RELEASE |         |
> >   +---------|   CPU   |
> >   |         |         |        +----------+  SDA
> >   |         +---------+        |          |-------
> > MAILBOX                   +--> |  I2C-DW  |  SCL
> >   |         +---------+  /     |          |-------
> >   |         |         | /      +----------+
> >   +---------|   PSP   |/
> >    <- ACK   |         |
> >             +---------+
> >
> > The solution is similar to i2c-designware-baytrail.c implementation, where
> > we are using a generic i2c-designware-* driver with a small "wrapper".
> >
> > In contrary to baytrail semaphore implementation, beside internal
> > acquire_lock() and release_lock() methods we are also applying quirks to
> > lock_bus() and unlock_bus() global adapter methods. With this in place
> > all i2c clients drivers may lock i2c bus for a desired number of i2c
> > transactions (e.g. write-wait-read) without being aware of that such bus
> > is shared with another entity.
> >
> > Modify i2c_dw_probe_lock_support() to select correct semaphore
> > implementation at runtime, since now we have more than one available.
> >
> > Configure new matching ACPI ID "AMDI0019" and register
> > ARBITRATION_SEMAPHORE flag in order to distinguish setup with PSP
> > arbitration.
> >
> > Add myself as a reviewer for I2C DesignWare in order to help with reviewing
> > and testing possible changes touching new i2c-designware-amdpsp.c module.
>
> Side note, please start a new email thread for new version of the series.

Sure, will do so starting with v4.

>
> Looks good enough to me, although a few minor comments below.
> Up to you to address them now or later.
> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

Thanks a lot! I will address your comments and send v4.

>
> > Signed-off-by: Jan Dabros <jsd@semihalf.com>

(...)

> > --- /dev/null
> > +++ b/drivers/i2c/busses/i2c-designware-amdpsp.c
> > @@ -0,0 +1,389 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +
> > +#include <linux/bitfield.h>
>
> IIRC this does not guarantee bits.h inclusion.

That is correct. There is a comment in bitfield.h with example of
GENMASK usage, but bits.h isn't actually included there.

>
> > +#include <linux/i2c.h>
> > +#include <linux/io-64-nonatomic-lo-hi.h>
> > +#include <linux/psp-sev.h>
> > +#include <linux/types.h>
> > +
> > +#include <asm/msr.h>
> > +
> > +#include "i2c-designware-core.h"
> > +
> > +#define MSR_AMD_PSP_ADDR     0xc00110a2
> > +#define PSP_MBOX_OFFSET              0x10570
> > +#define PSP_CMD_TIMEOUT_MS   500
> > +
> > +#define PSP_I2C_REQ_BUS_CMD          0x64
> > +#define PSP_I2C_REQ_RETRY_CNT                10
> > +#define PSP_I2C_REQ_RETRY_DELAY_USEC (50 * 1000)
>
> _US is enough and possibly (50 * USEC_PER_MSEC).

ACK.

>
> > +#define PSP_I2C_REQ_STS_OK           0x0
> > +#define PSP_I2C_REQ_STS_BUS_BUSY     0x1
> > +#define PSP_I2C_REQ_STS_INV_PARAM    0x3
> > +
> > +#define PSP_MBOX_FIELDS_STS          GENMASK(15, 0)
> > +#define PSP_MBOX_FIELDS_CMD          GENMASK(23, 16)
> > +#define PSP_MBOX_FIELDS_RESERVED     GENMASK(29, 24)
> > +#define PSP_MBOX_FIELDS_RECOVERY     BIT(30)
> > +#define PSP_MBOX_FIELDS_READY                BIT(31)
> > +
> > +struct psp_req_buffer_hdr {
> > +     u32 total_size;
> > +     u32 status;
> > +};
> > +
> > +enum psp_i2c_req_type {
> > +     PSP_I2C_REQ_ACQUIRE,
> > +     PSP_I2C_REQ_RELEASE,
> > +     PSP_I2C_REQ_MAX
> > +};
> > +
> > +struct psp_i2c_req {
> > +     struct psp_req_buffer_hdr hdr;
> > +     enum psp_i2c_req_type type;
> > +} __aligned(32);
>
> I forgot if this alignment is requirement due to DMA or so?
> We may use ____cacheline_aligned in such case.

I used some old code as a reference - this structure is mapped by PSP,
thus alignment applied here. I'm checking this with AMD whether it is
really needed or at least can be somehow limited - still waiting for
their response.
I propose to left this as is right now and eventually remove or make
more liberal in the future.

>
> > +struct psp_mbox {
> > +     u32 cmd_fields;
> > +     u64 i2c_req_addr;
> > +} __packed;
> > +
> > +static DEFINE_MUTEX(psp_i2c_access_mutex);
> > +static unsigned long psp_i2c_sem_acquired;
> > +static void __iomem *mbox_iomem;
> > +static u32 psp_i2c_access_count;
> > +static bool psp_i2c_mbox_fail;
> > +static struct device *psp_i2c_dev;
> > +
> > +/*
> > + * Implementation of PSP-x86 i2c-arbitration mailbox introduced for AMD Cezanne
> > + * family of SoCs.
> > + */
> > +
> > +static int psp_get_mbox_addr(unsigned long *mbox_addr)
> > +{
> > +     unsigned long long psp_mmio;
> > +
> > +     if (rdmsrl_safe(MSR_AMD_PSP_ADDR, &psp_mmio))
> > +             return -EIO;
> > +
> > +     *mbox_addr = (unsigned long)(psp_mmio + PSP_MBOX_OFFSET);
> > +
> > +     return 0;
> > +}
> > +
> > +static int psp_mbox_probe(void)
> > +{
> > +     unsigned long mbox_addr;
> > +     int ret;
> > +
> > +     ret = psp_get_mbox_addr(&mbox_addr);
> > +     if (ret)
> > +             return ret;
> > +
> > +     mbox_iomem = ioremap(mbox_addr, sizeof(struct psp_mbox));
> > +     if (!mbox_iomem)
> > +             return -ENOMEM;
> > +
> > +     return 0;
> > +}
> > +
> > +
> > +/* Recovery field should be equal 0 to start sending commands */
> > +static int psp_check_mbox_recovery(struct psp_mbox __iomem *mbox)
> > +{
> > +     u32 tmp;
> > +
> > +     tmp = readl(&mbox->cmd_fields);
> > +
> > +     return FIELD_GET(PSP_MBOX_FIELDS_RECOVERY, tmp);
> > +}
> > +
> > +static int psp_wait_cmd(struct psp_mbox __iomem *mbox)
> > +{
> > +     u32 tmp, expected;
> > +
> > +     /* Expect mbox_cmd to be cleared and ready bit to be set by PSP */
> > +     expected = FIELD_PREP(PSP_MBOX_FIELDS_READY, 1);
> > +
> > +     return readl_poll_timeout(&mbox->cmd_fields, tmp, (tmp == expected),
> > +                               0, 1000 * PSP_CMD_TIMEOUT_MS);
>
> 0?!

Yes, we are checking for readiness of PSP mailbox in a tight loop. We
would like to proceed further quickly as soon as this bit is set.
Actually checking this twice per every ACQUIRE&RELEASE - once before
sending command (to check whether PSP is ready to accept requests) and
second time after sending it. Do you think we should increase
@sleep_us value?

>
> > +}
> > +
> > +/* Status equal to 0 means that PSP succeed processing command */
> > +static int psp_check_mbox_sts(struct psp_mbox __iomem *mbox)
>
>   static u32 ...
>
> So reader will know it won't return error codes.

ACK/

>
> > +{
> > +     u32 cmd_reg;
> > +
> > +     cmd_reg = readl(&mbox->cmd_fields);
> > +
> > +     return FIELD_GET(PSP_MBOX_FIELDS_STS, cmd_reg);
> > +}
> > +
> > +static int psp_send_cmd(struct psp_i2c_req *req)
> > +{
> > +     struct psp_mbox __iomem *mbox = mbox_iomem;
> > +     phys_addr_t req_addr;
> > +     u32 cmd_reg;
> > +
> > +     if (psp_check_mbox_recovery(mbox))
> > +             return -EIO;
> > +
> > +     if (psp_wait_cmd(mbox))
> > +             return -EBUSY;
> > +
> > +     /*
> > +      * Fill mailbox with address of command-response buffer, which will be
> > +      * used for sending i2c requests as well as reading status returned by
> > +      * PSP. Use physical address of buffer, since PSP will map this region.
> > +      */
> > +     req_addr = __psp_pa((void *)req);
> > +     writeq(req_addr, &mbox->i2c_req_addr);
> > +
> > +     /* Write command register to trigger processing */
> > +     cmd_reg = FIELD_PREP(PSP_MBOX_FIELDS_CMD, PSP_I2C_REQ_BUS_CMD);
> > +     writel(cmd_reg, &mbox->cmd_fields);
> > +
> > +     if (psp_wait_cmd(mbox))
> > +             return -ETIMEDOUT;
> > +
> > +     if (psp_check_mbox_sts(mbox))
> > +             return -EIO;
> > +
> > +     return 0;
> > +}
> > +
> > +/* Helper to verify status returned by PSP */
> > +static int check_i2c_req_sts(struct psp_i2c_req *req)
> > +{
> > +     int status;
> > +
> > +     status = readl(&req->hdr.status);
> > +
> > +     switch (status) {
> > +     case PSP_I2C_REQ_STS_OK:
> > +             return 0;
> > +     case PSP_I2C_REQ_STS_BUS_BUSY:
> > +             return -EBUSY;
> > +     case PSP_I2C_REQ_STS_INV_PARAM:
> > +     default:
> > +             return -EIO;
> > +     };
> > +}
> > +
> > +static int psp_send_check_i2c_req(struct psp_i2c_req *req)
> > +{
> > +     /*
> > +      * Errors in x86-PSP i2c-arbitration protocol may occur at two levels:
> > +      * 1. mailbox communication - PSP is not operational or some IO errors
> > +      * with basic communication had happened;
> > +      * 2. i2c-requests - PSP refuses to grant i2c arbitration to x86 for too
> > +      * long.
> > +      * In order to distinguish between these two in error handling code, all
> > +      * errors on the first level (returned by psp_send_cmd) are shadowed by
> > +      * -EIO.
> > +      */
> > +     if (psp_send_cmd(req))
> > +             return -EIO;
> > +
> > +     return check_i2c_req_sts(req);
> > +}
> > +
> > +static int psp_send_i2c_req(enum psp_i2c_req_type i2c_req_type)
> > +{
> > +     struct psp_i2c_req *req;
> > +     unsigned long start;
> > +     int status, ret;
> > +
> > +     /* Allocate command-response buffer */
> > +     req = kzalloc(sizeof(*req), GFP_KERNEL);
> > +     if (!req)
> > +             return -ENOMEM;
> > +
> > +     req->hdr.total_size = sizeof(*req);
> > +     req->type = i2c_req_type;
> > +
> > +     start = jiffies;
> > +     ret = read_poll_timeout(psp_send_check_i2c_req, status,
> > +                             (status != -EBUSY),
> > +                             PSP_I2C_REQ_RETRY_DELAY_USEC,
> > +                             PSP_I2C_REQ_RETRY_CNT * PSP_I2C_REQ_RETRY_DELAY_USEC,
> > +                             0, req);
> > +     if (ret)
> > +             goto cleanup;
>
> > +     if (status) {
> > +             ret = status;
> > +             goto cleanup;
> > +     }
>
> I would write it in this form:
>
>         ret = status;
>         if (ret)
>                 goto cleanup;

ACK.

> > +     dev_dbg(psp_i2c_dev, "Request accepted by PSP after %ums\n",
> > +             jiffies_to_msecs(jiffies - start));
> > +
> > +cleanup:
> > +     kfree(req);
> > +     return ret;
> > +}
> > +
> > +static int psp_acquire_i2c_bus(void)
> > +{
> > +     int status;
> > +
> > +     mutex_lock(&psp_i2c_access_mutex);
> > +
> > +     /* Return early if mailbox malfunctioned */
> > +     if (psp_i2c_mbox_fail)
> > +             goto cleanup;
> > +
> > +     /*
> > +      * Simply increment usage counter and return if PSP semaphore was
> > +      * already taken by kernel.
> > +      */
> > +     if (psp_i2c_access_count > 0) {
>
> ' > 0' is redundant. It's unsigned.

ACK.

>
> > +             psp_i2c_access_count++;
> > +             goto cleanup;
> > +     };
> > +
> > +     status = psp_send_i2c_req(PSP_I2C_REQ_ACQUIRE);
> > +     if (status) {
> > +             if (status == -ETIMEDOUT)
> > +                     dev_err(psp_i2c_dev, "Timed out waiting for PSP to release I2C bus\n");
> > +             else
> > +                     dev_err(psp_i2c_dev, "PSP communication error\n");
> > +
> > +             dev_err(psp_i2c_dev, "Assume i2c bus is for exclusive host usage\n");
> > +             psp_i2c_mbox_fail = true;
> > +             goto cleanup;
> > +     }
> > +
> > +     psp_i2c_sem_acquired = jiffies;
> > +     psp_i2c_access_count++;
> > +
> > +     /*
> > +      * In case of errors with PSP arbitrator psp_i2c_mbox_fail variable is
> > +      * set above. As a consequence consecutive calls to acquire will bypass
> > +      * communication with PSP. At any case i2c bus is granted to the caller,
> > +      * thus always return success.
> > +      */
> > +cleanup:
> > +     mutex_unlock(&psp_i2c_access_mutex);
> > +     return 0;
> > +}
> > +
> > +static void psp_release_i2c_bus(void)
> > +{
> > +     int status;
> > +
> > +     mutex_lock(&psp_i2c_access_mutex);
> > +
> > +     /* Return early if mailbox was malfunctional */
> > +     if (psp_i2c_mbox_fail)
> > +             goto cleanup;
> > +
> > +     /*
> > +      * If we are last owner of PSP semaphore, need to release aribtration
> > +      * via mailbox.
> > +      */
> > +     psp_i2c_access_count--;
>
> > +     if (psp_i2c_access_count > 0)
>
> Ditto.

ACK.

Best Regards,
Jan

Re: [PATCH v3 2/2] i2c: designware: Add AMD PSP I2C bus support

[Andy Shevchenko]

On Mon, Feb 07, 2022 at 09:27:12AM +0100, Jan Dąbroś wrote:
> śr., 2 lut 2022 o 17:16 Andy Shevchenko
> <andriy.shevchenko@linux.intel.com> napisał(a):
> > On Wed, Feb 02, 2022 at 03:43:02PM +0100, Jan Dabros wrote:

...

> > > +struct psp_i2c_req {
> > > +     struct psp_req_buffer_hdr hdr;
> > > +     enum psp_i2c_req_type type;
> > > +} __aligned(32);
> >
> > I forgot if this alignment is requirement due to DMA or so?
> > We may use ____cacheline_aligned in such case.
> 
> I used some old code as a reference - this structure is mapped by PSP,
> thus alignment applied here. I'm checking this with AMD whether it is
> really needed or at least can be somehow limited - still waiting for
> their response.
> I propose to left this as is right now and eventually remove or make
> more liberal in the future.

Would be nice to clarify sooner than later.
In either case it needs a good comment.

...

> > > +     return readl_poll_timeout(&mbox->cmd_fields, tmp, (tmp == expected),
> > > +                               0, 1000 * PSP_CMD_TIMEOUT_MS);
> >
> > 0?!
> 
> Yes, we are checking for readiness of PSP mailbox in a tight loop. We
> would like to proceed further quickly as soon as this bit is set.
> Actually checking this twice per every ACQUIRE&RELEASE - once before
> sending command (to check whether PSP is ready to accept requests) and
> second time after sending it. Do you think we should increase
> @sleep_us value?

It depends on what you have in mind about hardware. I'm fine with either way,
but 0 has to be explained (in the comment).

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v3 2/2] i2c: designware: Add AMD PSP I2C bus support

[Wolfram Sang]

[-- Attachment #1: Type: text/plain, Size: 209 bytes --]


> Side note, please start a new email thread for new version of the series.

Yes, please. Also, neither me nor patchwork got patch 1 of this series.
So, please ensure the series is complete when you resend.


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v3 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dąbroś]

Hi,

pon., 7 lut 2022 o 15:25 Wolfram Sang <wsa@kernel.org> napisał(a):
>
>
> > Side note, please start a new email thread for new version of the series.
>
> Yes, please. Also, neither me nor patchwork got patch 1 of this series.
> So, please ensure the series is complete when you resend.

Right, I fixed this in v4.

Best Regards,
Jan

Re: [PATCH v3 2/2] i2c: designware: Add AMD PSP I2C bus support

[Jan Dąbroś]

pon., 7 lut 2022 o 12:42 Andy Shevchenko
<andriy.shevchenko@linux.intel.com> napisał(a):
>
> On Mon, Feb 07, 2022 at 09:27:12AM +0100, Jan Dąbroś wrote:
> > śr., 2 lut 2022 o 17:16 Andy Shevchenko
> > <andriy.shevchenko@linux.intel.com> napisał(a):
> > > On Wed, Feb 02, 2022 at 03:43:02PM +0100, Jan Dabros wrote:
>
> ...
>
> > > > +struct psp_i2c_req {
> > > > +     struct psp_req_buffer_hdr hdr;
> > > > +     enum psp_i2c_req_type type;
> > > > +} __aligned(32);
> > >
> > > I forgot if this alignment is requirement due to DMA or so?
> > > We may use ____cacheline_aligned in such case.
> >
> > I used some old code as a reference - this structure is mapped by PSP,
> > thus alignment applied here. I'm checking this with AMD whether it is
> > really needed or at least can be somehow limited - still waiting for
> > their response.
> > I propose to left this as is right now and eventually remove or make
> > more liberal in the future.
>
> Would be nice to clarify sooner than later.
> In either case it needs a good comment.

I confirmed that we can remove this alignment completely. Removed in v4.

Best Regards,
Jan

>
> ...
>
> > > > +     return readl_poll_timeout(&mbox->cmd_fields, tmp, (tmp == expected),
> > > > +                               0, 1000 * PSP_CMD_TIMEOUT_MS);
> > >
> > > 0?!
> >
> > Yes, we are checking for readiness of PSP mailbox in a tight loop. We
> > would like to proceed further quickly as soon as this bit is set.
> > Actually checking this twice per every ACQUIRE&RELEASE - once before
> > sending command (to check whether PSP is ready to accept requests) and
> > second time after sending it. Do you think we should increase
> > @sleep_us value?
>
> It depends on what you have in mind about hardware. I'm fine with either way,
> but 0 has to be explained (in the comment).
>
> --
> With Best Regards,
> Andy Shevchenko
>
>