* [PATCH v6 1/7] cdx: add the cdx bus driver
2023-01-26 10:46 [PATCH v6 0/7] add support for CDX bus Nipun Gupta
@ 2023-01-26 10:46 ` Nipun Gupta
2023-01-26 16:58 ` Randy Dunlap
2023-01-26 10:46 ` [PATCH v6 2/7] iommu/arm-smmu-v3: support ops registration for CDX bus Nipun Gupta
` (5 subsequent siblings)
6 siblings, 1 reply; 14+ messages in thread
From: Nipun Gupta @ 2023-01-26 10:46 UTC (permalink / raw)
To: robh+dt, krzysztof.kozlowski+dt, gregkh, rafael, eric.auger,
alex.williamson, cohuck, song.bao.hua, mchehab+huawei, maz,
f.fainelli, jeffrey.l.hugo, saravanak, Michael.Srba, mani,
yishaih, jgg, jgg, robin.murphy, will, joro, masahiroy,
ndesaulniers, linux-arm-kernel, linux-kbuild, linux-kernel,
devicetree
Cc: okaya, harpreet.anand, nikhil.agarwal, michal.simek, git, Nipun Gupta
Introduce AMD CDX bus, which provides a mechanism for scanning
and probing CDX devices. These devices are memory mapped on
system bus for Application Processors(APUs).
CDX devices can be changed dynamically in the Fabric and CDX
bus interacts with CDX controller to rescan the bus and
rediscover the devices.
Signed-off-by: Nipun Gupta <nipun.gupta@amd.com>
---
CDX bus module is kept as bool and not as a tristate as IOMMU
does not support buses with module compilation. Buses are provided
to IOMMU at compile time only.
Documentation/ABI/testing/sysfs-bus-cdx | 12 +
MAINTAINERS | 7 +
drivers/Kconfig | 2 +
drivers/Makefile | 1 +
drivers/cdx/Kconfig | 16 +
drivers/cdx/Makefile | 8 +
drivers/cdx/cdx.c | 408 ++++++++++++++++++++++++
drivers/cdx/cdx.h | 62 ++++
include/linux/cdx/cdx_bus.h | 147 +++++++++
include/linux/mod_devicetable.h | 15 +
scripts/mod/devicetable-offsets.c | 4 +
scripts/mod/file2alias.c | 12 +
12 files changed, 694 insertions(+)
create mode 100644 Documentation/ABI/testing/sysfs-bus-cdx
create mode 100644 drivers/cdx/Kconfig
create mode 100644 drivers/cdx/Makefile
create mode 100644 drivers/cdx/cdx.c
create mode 100644 drivers/cdx/cdx.h
create mode 100644 include/linux/cdx/cdx_bus.h
diff --git a/Documentation/ABI/testing/sysfs-bus-cdx b/Documentation/ABI/testing/sysfs-bus-cdx
new file mode 100644
index 000000000000..8c2425fdb6d9
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-bus-cdx
@@ -0,0 +1,12 @@
+What: /sys/bus/cdx/rescan
+Date: January 2023
+Contact: nipun.gupta@amd.com
+Description:
+ Writing a non-zero value to this file would cause rescan
+ of the bus and devices on the CDX bus. Any new devices would
+ be scanned and added to the list of linux devices and any
+ devices removed are also deleted from linux.
+
+ For example::
+
+ # echo 1 > /sys/bus/cdx/rescan
diff --git a/MAINTAINERS b/MAINTAINERS
index 7d4677efcf02..3b31df9f6bb9 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -964,6 +964,13 @@ Q: https://patchwork.kernel.org/project/linux-rdma/list/
F: drivers/infiniband/hw/efa/
F: include/uapi/rdma/efa-abi.h
+AMD CDX BUS DRIVER
+M: Nipun Gupta <nipun.gupta@amd.com>
+M: Nikhil Agarwal <nikhil.agarwal@amd.com>
+S: Maintained
+F: drivers/cdx/*
+F: include/linux/cdx/*
+
AMD CRYPTOGRAPHIC COPROCESSOR (CCP) DRIVER
M: Tom Lendacky <thomas.lendacky@amd.com>
M: John Allen <john.allen@amd.com>
diff --git a/drivers/Kconfig b/drivers/Kconfig
index 968bd0a6fd78..514ae6b24cb2 100644
--- a/drivers/Kconfig
+++ b/drivers/Kconfig
@@ -241,4 +241,6 @@ source "drivers/peci/Kconfig"
source "drivers/hte/Kconfig"
+source "drivers/cdx/Kconfig"
+
endmenu
diff --git a/drivers/Makefile b/drivers/Makefile
index 6e3d527ca0c7..bf2b09ed849d 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -190,3 +190,4 @@ obj-$(CONFIG_MOST) += most/
obj-$(CONFIG_PECI) += peci/
obj-$(CONFIG_HTE) += hte/
obj-$(CONFIG_DRM_ACCEL) += accel/
+obj-$(CONFIG_CDX_BUS) += cdx/
diff --git a/drivers/cdx/Kconfig b/drivers/cdx/Kconfig
new file mode 100644
index 000000000000..d134c8bfc7f8
--- /dev/null
+++ b/drivers/cdx/Kconfig
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# CDX bus configuration
+#
+# Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+#
+
+config CDX_BUS
+ bool "CDX Bus driver"
+ help
+ Driver to enable Composable DMA Transfer(CDX) Bus. CDX bus
+ exposes Fabric devices which uses composable DMA IP to the
+ APU. CDX bus provides a mechanism for scanning and probing
+ of CDX devices. CDX devices are memory mapped on system bus
+ for embedded CPUs. CDX bus uses CDX controller and firmware
+ to scan these CDX devices.
diff --git a/drivers/cdx/Makefile b/drivers/cdx/Makefile
new file mode 100644
index 000000000000..c5feb11fc718
--- /dev/null
+++ b/drivers/cdx/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for CDX
+#
+# Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+#
+
+obj-$(CONFIG_CDX_BUS) += cdx.o
diff --git a/drivers/cdx/cdx.c b/drivers/cdx/cdx.c
new file mode 100644
index 000000000000..795770522b01
--- /dev/null
+++ b/drivers/cdx/cdx.c
@@ -0,0 +1,408 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * CDX bus driver.
+ *
+ * Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+ */
+
+/*
+ * Architecture Overview
+ * =====================
+ * CDX is a Hardware Architecture designed for AMD FPGA devices. It
+ * consists of sophisticated mechanism for interaction between FPGA,
+ * Firmware and the APUs (Application CPUs).
+ *
+ * Firmware resides on RPU (Realtime CPUs) which interacts with
+ * the FPGA program manager and the APUs. The RPU provides memory-mapped
+ * interface (RPU if) which is used to communicate with APUs.
+ *
+ * The diagram below shows an overview of the CDX architecture:
+ *
+ * +--------------------------------------+
+ * | Application CPUs (APU) |
+ * | |
+ * | CDX device drivers|
+ * | Linux OS | |
+ * | CDX bus |
+ * | | |
+ * | CDX controller |
+ * | | |
+ * +-----------------------------|--------+
+ * | (discover, config,
+ * | reset, rescan)
+ * |
+ * +------------------------| RPU if |----+
+ * | | |
+ * | V |
+ * | Realtime CPUs (RPU) |
+ * | |
+ * +--------------------------------------+
+ * |
+ * +---------------------|----------------+
+ * | FPGA | |
+ * | +-----------------------+ |
+ * | | | | |
+ * | +-------+ +-------+ +-------+ |
+ * | | dev 1 | | dev 2 | | dev 3 | |
+ * | +-------+ +-------+ +-------+ |
+ * +--------------------------------------+
+ *
+ * The RPU firmware extracts the device information from the loaded FPGA
+ * image and implements a mechanism that allows the APU drivers to
+ * enumerate such devices (device personality and resource details) via
+ * a dedicated communication channel. RPU mediates operations such as
+ * discover, reset and rescan of the FPGA devices for the APU. This is
+ * done using memory mapped interface provided by the RPU to APU.
+ */
+
+#include <linux/init.h>
+#include <linux/irqdomain.h>
+#include <linux/kernel.h>
+#include <linux/of_device.h>
+#include <linux/slab.h>
+#include <linux/mm.h>
+#include <linux/xarray.h>
+#include <linux/cdx/cdx_bus.h>
+#include "cdx.h"
+
+/* Default DMA mask for devices on a CDX bus */
+#define CDX_DEFAULT_DMA_MASK (~0ULL)
+#define MAX_CDX_CONTROLLERS 16
+
+/* CDX controllers registered with the CDX bus */
+static DEFINE_XARRAY_ALLOC(cdx_controllers);
+
+/**
+ * cdx_unregister_device - Unregister a CDX device
+ * @dev: CDX device
+ * @data: This is always passed as NULL, and is not used in this API,
+ * but is required here as the bus_for_each_dev() API expects
+ * the passed function (cdx_unregister_device) to have this
+ * as an argument.
+ *
+ * Return: -errno on failure, 0 on success.
+ */
+static int cdx_unregister_device(struct device *dev,
+ void *data)
+{
+ struct cdx_device *cdx_dev = to_cdx_device(dev);
+
+ kfree(cdx_dev->driver_override);
+ cdx_dev->driver_override = NULL;
+ /*
+ * Do not free cdx_dev here as it would be freed in
+ * cdx_device_release() called from within put_device().
+ */
+ device_del(&cdx_dev->dev);
+ put_device(&cdx_dev->dev);
+
+ return 0;
+}
+
+static void cdx_unregister_devices(struct bus_type *bus)
+{
+ /* Reset all the devices attached to cdx bus */
+ bus_for_each_dev(bus, NULL, NULL, cdx_unregister_device);
+}
+
+/**
+ * cdx_match_one_device - Tell if a CDX device structure has a matching
+ * CDX device id structure
+ * @id: single CDX device id structure to match
+ * @dev: the CDX device structure to match against
+ *
+ * Return: matching cdx_device_id structure or NULL if there is no match.
+ */
+static inline const struct cdx_device_id *
+cdx_match_one_device(const struct cdx_device_id *id,
+ const struct cdx_device *dev)
+{
+ /* Use vendor ID and device ID for matching */
+ if ((id->vendor == CDX_ANY_ID || id->vendor == dev->vendor) &&
+ (id->device == CDX_ANY_ID || id->device == dev->device))
+ return id;
+ return NULL;
+}
+
+/**
+ * cdx_match_id - See if a CDX device matches a given cdx_id table
+ * @ids: array of CDX device ID structures to search in
+ * @dev: the CDX device structure to match against.
+ *
+ * Used by a driver to check whether a CDX device is in its list of
+ * supported devices. Returns the matching cdx_device_id structure or
+ * NULL if there is no match.
+ *
+ * Return: matching cdx_device_id structure or NULL if there is no match.
+ */
+static inline const struct cdx_device_id *
+cdx_match_id(const struct cdx_device_id *ids, struct cdx_device *dev)
+{
+ if (ids) {
+ while (ids->vendor || ids->device) {
+ if (cdx_match_one_device(ids, dev))
+ return ids;
+ ids++;
+ }
+ }
+ return NULL;
+}
+
+/**
+ * cdx_bus_match - device to driver matching callback
+ * @dev: the cdx device to match against
+ * @drv: the device driver to search for matching cdx device
+ * structures
+ *
+ * Return: true on success, false otherwise.
+ */
+static int cdx_bus_match(struct device *dev, struct device_driver *drv)
+{
+ struct cdx_device *cdx_dev = to_cdx_device(dev);
+ struct cdx_driver *cdx_drv = to_cdx_driver(drv);
+ const struct cdx_device_id *found_id = NULL;
+ const struct cdx_device_id *ids;
+
+ ids = cdx_drv->match_id_table;
+
+ /* When driver_override is set, only bind to the matching driver */
+ if (cdx_dev->driver_override && strcmp(cdx_dev->driver_override, drv->name))
+ return false;
+
+ found_id = cdx_match_id(ids, cdx_dev);
+ if (!found_id)
+ return false;
+
+ do {
+ /*
+ * In case override_only was set, enforce driver_override
+ * matching.
+ */
+ if (!found_id->override_only)
+ return true;
+ if (cdx_dev->driver_override)
+ return true;
+
+ ids = found_id + 1;
+ found_id = cdx_match_id(ids, cdx_dev);
+ } while (found_id);
+
+ return false;
+}
+
+static int cdx_probe(struct device *dev)
+{
+ struct cdx_driver *cdx_drv = to_cdx_driver(dev->driver);
+ struct cdx_device *cdx_dev = to_cdx_device(dev);
+ int error;
+
+ error = cdx_drv->probe(cdx_dev);
+ if (error < 0) {
+ dev_err_probe(dev, error, "%s failed\n", __func__);
+ return error;
+ }
+
+ return 0;
+}
+
+static void cdx_remove(struct device *dev)
+{
+ struct cdx_driver *cdx_drv = to_cdx_driver(dev->driver);
+ struct cdx_device *cdx_dev = to_cdx_device(dev);
+
+ if (cdx_drv && cdx_drv->remove)
+ cdx_drv->remove(cdx_dev);
+}
+
+static void cdx_shutdown(struct device *dev)
+{
+ struct cdx_driver *cdx_drv = to_cdx_driver(dev->driver);
+ struct cdx_device *cdx_dev = to_cdx_device(dev);
+
+ if (cdx_drv && cdx_drv->shutdown)
+ cdx_drv->shutdown(cdx_dev);
+}
+
+static int cdx_dma_configure(struct device *dev)
+{
+ struct cdx_device *cdx_dev = to_cdx_device(dev);
+ u32 input_id = cdx_dev->req_id;
+ int ret;
+
+ ret = of_dma_configure_id(dev, dev->parent->of_node, 0, &input_id);
+ if (ret && ret != -EPROBE_DEFER) {
+ dev_err(dev, "of_dma_configure_id() failed\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+static ssize_t rescan_store(struct bus_type *bus,
+ const char *buf, size_t count)
+{
+ unsigned long index, val = 0;
+ struct cdx_controller *cdx;
+
+ if (kstrtoul(buf, 0, &val) < 0)
+ return -EINVAL;
+
+ if (!val)
+ return -EINVAL;
+
+ /* Unregister all the devices on the bus */
+ cdx_unregister_devices(&cdx_bus_type);
+
+ /* Rescan all the devices */
+ xa_for_each(&cdx_controllers, index, cdx) {
+ int ret;
+
+ ret = cdx->ops->scan(cdx);
+ if (ret)
+ dev_err(cdx->dev, "cdx bus scanning failed\n");
+ }
+
+ return count;
+}
+static BUS_ATTR_WO(rescan);
+
+static struct attribute *cdx_bus_attrs[] = {
+ &bus_attr_rescan.attr,
+ NULL,
+};
+ATTRIBUTE_GROUPS(cdx_bus);
+
+struct bus_type cdx_bus_type = {
+ .name = "cdx",
+ .match = cdx_bus_match,
+ .probe = cdx_probe,
+ .remove = cdx_remove,
+ .shutdown = cdx_shutdown,
+ .dma_configure = cdx_dma_configure,
+ .bus_groups = cdx_bus_groups,
+};
+EXPORT_SYMBOL_GPL(cdx_bus_type);
+
+int __cdx_driver_register(struct cdx_driver *cdx_driver,
+ struct module *owner)
+{
+ int error;
+
+ cdx_driver->driver.owner = owner;
+ cdx_driver->driver.bus = &cdx_bus_type;
+
+ error = driver_register(&cdx_driver->driver);
+ if (error < 0) {
+ pr_err("driver_register() failed for %s: %d\n",
+ cdx_driver->driver.name, error);
+ return error;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(__cdx_driver_register);
+
+void cdx_driver_unregister(struct cdx_driver *cdx_driver)
+{
+ driver_unregister(&cdx_driver->driver);
+}
+EXPORT_SYMBOL_GPL(cdx_driver_unregister);
+
+static void cdx_device_release(struct device *dev)
+{
+ struct cdx_device *cdx_dev = to_cdx_device(dev);
+
+ kfree(cdx_dev);
+}
+
+int cdx_device_add(struct cdx_dev_params *dev_params)
+{
+ struct cdx_controller *cdx = dev_params->cdx;
+ struct device *parent = cdx->dev;
+ struct cdx_device *cdx_dev;
+ int ret;
+
+ cdx_dev = kzalloc(sizeof(*cdx_dev), GFP_KERNEL);
+ if (!cdx_dev)
+ return -ENOMEM;
+
+ /* Populate resource */
+ memcpy(cdx_dev->res, dev_params->res, sizeof(struct resource) *
+ dev_params->res_count);
+ cdx_dev->res_count = dev_params->res_count;
+
+ /* Populate CDX dev params */
+ cdx_dev->req_id = dev_params->req_id;
+ cdx_dev->vendor = dev_params->vendor;
+ cdx_dev->device = dev_params->device;
+ cdx_dev->bus_num = dev_params->bus_num;
+ cdx_dev->dev_num = dev_params->dev_num;
+ cdx_dev->cdx = dev_params->cdx;
+ cdx_dev->dma_mask = CDX_DEFAULT_DMA_MASK;
+
+ /* Initiaize generic device */
+ device_initialize(&cdx_dev->dev);
+ cdx_dev->dev.parent = parent;
+ cdx_dev->dev.bus = &cdx_bus_type;
+ cdx_dev->dev.dma_mask = &cdx_dev->dma_mask;
+ cdx_dev->dev.release = cdx_device_release;
+
+ /* Set Name */
+ dev_set_name(&cdx_dev->dev, "cdx-%02x:%02x",
+ ((cdx->id << CDX_CONTROLLER_ID_SHIFT) | (cdx_dev->bus_num & CDX_BUS_NUM_MASK)),
+ cdx_dev->dev_num);
+
+ ret = device_add(&cdx_dev->dev);
+ if (ret != 0) {
+ dev_err(&cdx_dev->dev,
+ "cdx device add failed: %d", ret);
+ goto fail;
+ }
+
+ return 0;
+fail:
+ /*
+ * Do not free cdx_dev here as it would be freed in
+ * cdx_device_release() called from put_device().
+ */
+ put_device(&cdx_dev->dev);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(cdx_device_add);
+
+int cdx_register_controller(struct cdx_controller *cdx)
+{
+ int ret;
+
+ ret = xa_alloc(&cdx_controllers, &cdx->id, cdx,
+ XA_LIMIT(0, MAX_CDX_CONTROLLERS - 1), GFP_KERNEL);
+ if (ret < 0) {
+ dev_err(cdx->dev,
+ "No free index available. Maximum controllers already registered\n");
+ cdx->id = (u8)MAX_CDX_CONTROLLERS;
+ return ret;
+ }
+
+ /* Scan all the devices */
+ cdx->ops->scan(cdx);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(cdx_register_controller);
+
+void cdx_unregister_controller(struct cdx_controller *cdx)
+{
+ if (cdx->id >= MAX_CDX_CONTROLLERS)
+ return;
+
+ device_for_each_child(cdx->dev, NULL, cdx_unregister_device);
+ xa_erase(&cdx_controllers, cdx->id);
+}
+EXPORT_SYMBOL_GPL(cdx_unregister_controller);
+
+static int __init cdx_bus_init(void)
+{
+ return bus_register(&cdx_bus_type);
+}
+postcore_initcall(cdx_bus_init);
diff --git a/drivers/cdx/cdx.h b/drivers/cdx/cdx.h
new file mode 100644
index 000000000000..c436ac7ac86f
--- /dev/null
+++ b/drivers/cdx/cdx.h
@@ -0,0 +1,62 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ * Header file for the CDX Bus
+ *
+ * Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+ */
+
+#ifndef _CDX_H_
+#define _CDX_H_
+
+#include <linux/cdx/cdx_bus.h>
+
+/**
+ * struct cdx_dev_params - CDX device parameters
+ * @cdx: CDX controller associated with the device
+ * @parent: Associated CDX controller
+ * @vendor: Vendor ID for CDX device
+ * @device: Device ID for CDX device
+ * @bus_num: Bus number for this CDX device
+ * @dev_num: Device number for this device
+ * @res: array of MMIO region entries
+ * @res_count: number of valid MMIO regions
+ * @req_id: Requestor ID associated with CDX device
+ */
+struct cdx_dev_params {
+ struct cdx_controller *cdx;
+ u16 vendor;
+ u16 device;
+ u8 bus_num;
+ u8 dev_num;
+ struct resource res[MAX_CDX_DEV_RESOURCES];
+ u8 res_count;
+ u32 req_id;
+};
+
+/**
+ * cdx_register_controller - Register a CDX controller and its ports
+ * on the CDX bus.
+ * @cdx: The CDX controller to register
+ *
+ * Return: -errno on failure, 0 on success.
+ */
+int cdx_register_controller(struct cdx_controller *cdx);
+
+/**
+ * cdx_unregister_controller - Unregister a CDX controller
+ * @cdx: The CDX controller to unregister
+ */
+void cdx_unregister_controller(struct cdx_controller *cdx);
+
+/**
+ * cdx_device_add - Add a CDX device. This function adds a CDX device
+ * on the CDX bus as per the device parameters provided
+ * by caller. It also creates and registers an associated
+ * Linux generic device.
+ * @dev_params: device parameters associated with the device to be created.
+ *
+ * Return: -errno on failure, 0 on success.
+ */
+int cdx_device_add(struct cdx_dev_params *dev_params);
+
+#endif /* _CDX_H_ */
diff --git a/include/linux/cdx/cdx_bus.h b/include/linux/cdx/cdx_bus.h
new file mode 100644
index 000000000000..d134e0104724
--- /dev/null
+++ b/include/linux/cdx/cdx_bus.h
@@ -0,0 +1,147 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ * CDX bus public interface
+ *
+ * Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+ *
+ */
+
+#ifndef _CDX_BUS_H_
+#define _CDX_BUS_H_
+
+#include <linux/device.h>
+#include <linux/list.h>
+#include <linux/mod_devicetable.h>
+
+#define MAX_CDX_DEV_RESOURCES 4
+#define CDX_ANY_ID (0xFFFF)
+#define CDX_CONTROLLER_ID_SHIFT 4
+#define CDX_BUS_NUM_MASK 0xF
+
+/* Forward declaration for CDX controller */
+struct cdx_controller;
+
+typedef int (*cdx_scan_cb)(struct cdx_controller *cdx);
+
+/**
+ * CDX_DEVICE_DRIVER_OVERRIDE - macro used to describe a CDX device with
+ * override_only flags.
+ * @vend: the 16 bit CDX Vendor ID
+ * @dev: the 16 bit CDX Device ID
+ * @driver_override: the 32 bit CDX Device override_only
+ *
+ * This macro is used to create a struct cdx_device_id that matches only a
+ * driver_override device.
+ */
+#define CDX_DEVICE_DRIVER_OVERRIDE(vend, dev, driver_override) \
+ .vendor = (vend), .device = (dev), .override_only = (driver_override)
+
+/**
+ * struct cdx_ops - Callbacks supported by CDX controller.
+ * @scan: scan the devices on the controller
+ */
+struct cdx_ops {
+ cdx_scan_cb scan;
+};
+
+/**
+ * struct cdx_controller: CDX controller object
+ * @dev: Linux device associated with the CDX controller.
+ * @priv: private data
+ * @id: Controller ID
+ * @ops: CDX controller ops
+ */
+struct cdx_controller {
+ struct device *dev;
+ void *priv;
+ u32 id;
+ struct cdx_ops *ops;
+};
+
+/**
+ * struct cdx_device - CDX device object
+ * @dev: Linux driver model device object
+ * @cdx: CDX controller associated with the device
+ * @vendor: Vendor ID for CDX device
+ * @device: Device ID for CDX device
+ * @bus_num: Bus number for this CDX device
+ * @dev_num: Device number for this device
+ * @res: array of MMIO region entries
+ * @res_attr: resource binary attribute
+ * @res_count: number of valid MMIO regions
+ * @dma_mask: Default DMA mask
+ * @flags: CDX device flags
+ * @req_id: Requestor ID associated with CDX device
+ * @driver_override: driver name to force a match; do not set directly,
+ * because core frees it; use driver_set_override() to
+ * set or clear it.
+ */
+struct cdx_device {
+ struct device dev;
+ struct cdx_controller *cdx;
+ u16 vendor;
+ u16 device;
+ u8 bus_num;
+ u8 dev_num;
+ struct resource res[MAX_CDX_DEV_RESOURCES];
+ u8 res_count;
+ u64 dma_mask;
+ u16 flags;
+ u32 req_id;
+ const char *driver_override;
+};
+
+#define to_cdx_device(_dev) \
+ container_of(_dev, struct cdx_device, dev)
+
+/**
+ * struct cdx_driver - CDX device driver
+ * @driver: Generic device driver
+ * @match_id_table: table of supported device matching Ids
+ * @probe: Function called when a device is added
+ * @remove: Function called when a device is removed
+ * @shutdown: Function called at shutdown time to quiesce the device
+ * @driver_managed_dma: Device driver doesn't use kernel DMA API for DMA.
+ * For most device drivers, no need to care about this flag
+ * as long as all DMAs are handled through the kernel DMA API.
+ * For some special ones, for example VFIO drivers, they know
+ * how to manage the DMA themselves and set this flag so that
+ * the IOMMU layer will allow them to setup and manage their
+ * own I/O address space.
+ */
+struct cdx_driver {
+ struct device_driver driver;
+ const struct cdx_device_id *match_id_table;
+ int (*probe)(struct cdx_device *dev);
+ int (*remove)(struct cdx_device *dev);
+ void (*shutdown)(struct cdx_device *dev);
+ bool driver_managed_dma;
+};
+
+#define to_cdx_driver(_drv) \
+ container_of(_drv, struct cdx_driver, driver)
+
+/* Macro to avoid include chaining to get THIS_MODULE */
+#define cdx_driver_register(drv) \
+ __cdx_driver_register(drv, THIS_MODULE)
+
+/**
+ * __cdx_driver_register - registers a CDX device driver
+ * @cdx_driver: CDX driver to register
+ * @owner: module owner
+ *
+ * Return: -errno on failure, 0 on success.
+ */
+int __must_check __cdx_driver_register(struct cdx_driver *cdx_driver,
+ struct module *owner);
+
+/**
+ * cdx_driver_unregister - unregisters a device driver from the
+ * CDX bus.
+ * @cdx_driver: CDX driver to register
+ */
+void cdx_driver_unregister(struct cdx_driver *cdx_driver);
+
+extern struct bus_type cdx_bus_type;
+
+#endif /* _CDX_BUS_H_ */
diff --git a/include/linux/mod_devicetable.h b/include/linux/mod_devicetable.h
index 549590e9c644..23445b163c2a 100644
--- a/include/linux/mod_devicetable.h
+++ b/include/linux/mod_devicetable.h
@@ -911,4 +911,19 @@ struct ishtp_device_id {
kernel_ulong_t driver_data;
};
+/**
+ * struct cdx_device_id - CDX device identifier
+ * @vendor: Vendor ID
+ * @device: Device ID
+ * @override_only: Match only when dev->driver_override is this driver.
+ *
+ * Type of entries in the "device Id" table for CDX devices supported by
+ * a CDX device driver.
+ */
+struct cdx_device_id {
+ __u16 vendor;
+ __u16 device;
+ __u32 override_only;
+};
+
#endif /* LINUX_MOD_DEVICETABLE_H */
diff --git a/scripts/mod/devicetable-offsets.c b/scripts/mod/devicetable-offsets.c
index c0d3bcb99138..62dc988df84d 100644
--- a/scripts/mod/devicetable-offsets.c
+++ b/scripts/mod/devicetable-offsets.c
@@ -262,5 +262,9 @@ int main(void)
DEVID(ishtp_device_id);
DEVID_FIELD(ishtp_device_id, guid);
+ DEVID(cdx_device_id);
+ DEVID_FIELD(cdx_device_id, vendor);
+ DEVID_FIELD(cdx_device_id, device);
+
return 0;
}
diff --git a/scripts/mod/file2alias.c b/scripts/mod/file2alias.c
index 91c2e7ba5e52..28da34ba4359 100644
--- a/scripts/mod/file2alias.c
+++ b/scripts/mod/file2alias.c
@@ -1452,6 +1452,17 @@ static int do_dfl_entry(const char *filename, void *symval, char *alias)
return 1;
}
+/* Looks like: cdx:vNdN */
+static int do_cdx_entry(const char *filename, void *symval,
+ char *alias)
+{
+ DEF_FIELD(symval, cdx_device_id, vendor);
+ DEF_FIELD(symval, cdx_device_id, device);
+
+ sprintf(alias, "cdx:v%08Xd%08Xd", vendor, device);
+ return 1;
+}
+
/* Does namelen bytes of name exactly match the symbol? */
static bool sym_is(const char *name, unsigned namelen, const char *symbol)
{
@@ -1531,6 +1542,7 @@ static const struct devtable devtable[] = {
{"ssam", SIZE_ssam_device_id, do_ssam_entry},
{"dfl", SIZE_dfl_device_id, do_dfl_entry},
{"ishtp", SIZE_ishtp_device_id, do_ishtp_entry},
+ {"cdx", SIZE_cdx_device_id, do_cdx_entry},
};
/* Create MODULE_ALIAS() statements.
--
2.17.1
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [PATCH v6 1/7] cdx: add the cdx bus driver
2023-01-26 10:46 ` [PATCH v6 1/7] cdx: add the cdx bus driver Nipun Gupta
@ 2023-01-26 16:58 ` Randy Dunlap
0 siblings, 0 replies; 14+ messages in thread
From: Randy Dunlap @ 2023-01-26 16:58 UTC (permalink / raw)
To: Nipun Gupta, robh+dt, krzysztof.kozlowski+dt, gregkh, rafael,
eric.auger, alex.williamson, cohuck, song.bao.hua,
mchehab+huawei, maz, f.fainelli, jeffrey.l.hugo, saravanak,
Michael.Srba, mani, yishaih, jgg, jgg, robin.murphy, will, joro,
masahiroy, ndesaulniers, linux-arm-kernel, linux-kbuild,
linux-kernel, devicetree
Cc: okaya, harpreet.anand, nikhil.agarwal, michal.simek, git
Better as:
On 1/26/23 02:46, Nipun Gupta wrote:
> diff --git a/Documentation/ABI/testing/sysfs-bus-cdx b/Documentation/ABI/testing/sysfs-bus-cdx
> new file mode 100644
> index 000000000000..8c2425fdb6d9
> --- /dev/null
> +++ b/Documentation/ABI/testing/sysfs-bus-cdx
> @@ -0,0 +1,12 @@
> +What: /sys/bus/cdx/rescan
> +Date: January 2023
> +Contact: nipun.gupta@amd.com
> +Description:
(also capitalize Linux)
Writing a non-zero value to this file causes rescan
of the bus and devices on the CDX bus. Any new devices are
scanned and added to the list of Linux devices and any
devices removed are also deleted from Linux.
> +
> + For example::
> +
> + # echo 1 > /sys/bus/cdx/rescan
--
~Randy
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH v6 2/7] iommu/arm-smmu-v3: support ops registration for CDX bus
2023-01-26 10:46 [PATCH v6 0/7] add support for CDX bus Nipun Gupta
2023-01-26 10:46 ` [PATCH v6 1/7] cdx: add the cdx bus driver Nipun Gupta
@ 2023-01-26 10:46 ` Nipun Gupta
2023-01-26 10:46 ` [PATCH v6 3/7] dt-bindings: bus: add CDX bus controller for versal net Nipun Gupta
` (4 subsequent siblings)
6 siblings, 0 replies; 14+ messages in thread
From: Nipun Gupta @ 2023-01-26 10:46 UTC (permalink / raw)
To: robh+dt, krzysztof.kozlowski+dt, gregkh, rafael, eric.auger,
alex.williamson, cohuck, song.bao.hua, mchehab+huawei, maz,
f.fainelli, jeffrey.l.hugo, saravanak, Michael.Srba, mani,
yishaih, jgg, jgg, robin.murphy, will, joro, masahiroy,
ndesaulniers, linux-arm-kernel, linux-kbuild, linux-kernel,
devicetree
Cc: okaya, harpreet.anand, nikhil.agarwal, michal.simek, git, Nipun Gupta
With new CDX bus supported for AMD FPGA devices on ARM
platform, the bus requires registration for the SMMU v3
driver.
Signed-off-by: Nipun Gupta <nipun.gupta@amd.com>
---
drivers/iommu/iommu.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index e3e401da53c6..4e8f2051a9b7 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -28,6 +28,7 @@
#include <linux/fsl/mc.h>
#include <linux/module.h>
#include <linux/cc_platform.h>
+#include <linux/cdx/cdx_bus.h>
#include <trace/events/iommu.h>
#include <linux/sched/mm.h>
#include <linux/msi.h>
@@ -129,6 +130,9 @@ static struct bus_type * const iommu_buses[] = {
#ifdef CONFIG_TEGRA_HOST1X_CONTEXT_BUS
&host1x_context_device_bus_type,
#endif
+#ifdef CONFIG_CDX_BUS
+ &cdx_bus_type,
+#endif
};
/*
--
2.17.1
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v6 3/7] dt-bindings: bus: add CDX bus controller for versal net
2023-01-26 10:46 [PATCH v6 0/7] add support for CDX bus Nipun Gupta
2023-01-26 10:46 ` [PATCH v6 1/7] cdx: add the cdx bus driver Nipun Gupta
2023-01-26 10:46 ` [PATCH v6 2/7] iommu/arm-smmu-v3: support ops registration for CDX bus Nipun Gupta
@ 2023-01-26 10:46 ` Nipun Gupta
2023-01-26 10:46 ` [PATCH v6 4/7] cdx: add MCDI protocol interface for firmware interaction Nipun Gupta
` (3 subsequent siblings)
6 siblings, 0 replies; 14+ messages in thread
From: Nipun Gupta @ 2023-01-26 10:46 UTC (permalink / raw)
To: robh+dt, krzysztof.kozlowski+dt, gregkh, rafael, eric.auger,
alex.williamson, cohuck, song.bao.hua, mchehab+huawei, maz,
f.fainelli, jeffrey.l.hugo, saravanak, Michael.Srba, mani,
yishaih, jgg, jgg, robin.murphy, will, joro, masahiroy,
ndesaulniers, linux-arm-kernel, linux-kbuild, linux-kernel,
devicetree
Cc: okaya, harpreet.anand, nikhil.agarwal, michal.simek, git, Nipun Gupta
Add CDX bus controller device tree bindings for versal-net
devices.
Signed-off-by: Nipun Gupta <nipun.gupta@amd.com>
---
.../bindings/bus/xlnx,versal-net-cdx.yaml | 68 +++++++++++++++++++
MAINTAINERS | 1 +
2 files changed, 69 insertions(+)
create mode 100644 Documentation/devicetree/bindings/bus/xlnx,versal-net-cdx.yaml
diff --git a/Documentation/devicetree/bindings/bus/xlnx,versal-net-cdx.yaml b/Documentation/devicetree/bindings/bus/xlnx,versal-net-cdx.yaml
new file mode 100644
index 000000000000..8452185b9d70
--- /dev/null
+++ b/Documentation/devicetree/bindings/bus/xlnx,versal-net-cdx.yaml
@@ -0,0 +1,68 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/bus/xlnx,versal-net-cdx.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: AMD CDX bus controller
+
+description: |
+ CDX bus controller for AMD devices is implemented to dynamically
+ detect CDX bus and devices on these bus using the firmware.
+ The CDX bus manages multiple FPGA based hardware devices, which
+ can support network, crypto or any other specialized type of
+ devices. These FPGA based devices can be added/modified dynamically
+ on run-time.
+
+ All devices on the CDX bus will have a unique streamid (for IOMMU)
+ and a unique device ID (for MSI) corresponding to a requestor ID
+ (one to one associated with the device). The streamid and deviceid
+ are used to configure SMMU and GIC-ITS respectively.
+
+ iommu-map property is used to define the set of stream ids
+ corresponding to each device and the associated IOMMU.
+
+ The MSI writes are accompanied by sideband data (Device ID).
+ The msi-map property is used to associate the devices with the
+ device ID as well as the associated ITS controller.
+
+ rproc property (xlnx,rproc) is used to identify the remote processor
+ with which APU (Application Processor Unit) interacts to find out
+ the bus and device configuration.
+
+maintainers:
+ - Nipun Gupta <nipun.gupta@amd.com>
+ - Nikhil Agarwal <nikhil.agarwal@amd.com>
+
+properties:
+ compatible:
+ const: xlnx,versal-net-cdx
+
+ iommu-map: true
+
+ msi-map: true
+
+ xlnx,rproc:
+ $ref: /schemas/types.yaml#/definitions/phandle
+ description:
+ phandle to the remoteproc_r5 rproc node using which APU interacts
+ with remote processor.
+
+required:
+ - compatible
+ - iommu-map
+ - msi-map
+ - xlnx,rproc
+
+additionalProperties: false
+
+examples:
+ - |
+ cdx {
+ compatible = "xlnx,versal-net-cdx";
+ /* define map for RIDs 250-259 */
+ iommu-map = <250 &smmu 250 10>;
+ /* define msi map for RIDs 250-259 */
+ msi-map = <250 &its 250 10>;
+ xlnx,rproc = <&remoteproc_r5>;
+ };
diff --git a/MAINTAINERS b/MAINTAINERS
index 3b31df9f6bb9..fe5c656f2738 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -968,6 +968,7 @@ AMD CDX BUS DRIVER
M: Nipun Gupta <nipun.gupta@amd.com>
M: Nikhil Agarwal <nikhil.agarwal@amd.com>
S: Maintained
+F: Documentation/devicetree/bindings/bus/xlnx,versal-net-cdx.yaml
F: drivers/cdx/*
F: include/linux/cdx/*
--
2.17.1
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v6 4/7] cdx: add MCDI protocol interface for firmware interaction
2023-01-26 10:46 [PATCH v6 0/7] add support for CDX bus Nipun Gupta
` (2 preceding siblings ...)
2023-01-26 10:46 ` [PATCH v6 3/7] dt-bindings: bus: add CDX bus controller for versal net Nipun Gupta
@ 2023-01-26 10:46 ` Nipun Gupta
2023-01-28 13:15 ` kernel test robot
2023-01-26 10:46 ` [PATCH v6 5/7] cdx: add cdx controller Nipun Gupta
` (2 subsequent siblings)
6 siblings, 1 reply; 14+ messages in thread
From: Nipun Gupta @ 2023-01-26 10:46 UTC (permalink / raw)
To: robh+dt, krzysztof.kozlowski+dt, gregkh, rafael, eric.auger,
alex.williamson, cohuck, song.bao.hua, mchehab+huawei, maz,
f.fainelli, jeffrey.l.hugo, saravanak, Michael.Srba, mani,
yishaih, jgg, jgg, robin.murphy, will, joro, masahiroy,
ndesaulniers, linux-arm-kernel, linux-kbuild, linux-kernel,
devicetree
Cc: okaya, harpreet.anand, nikhil.agarwal, michal.simek, git, Nipun Gupta
The MCDI (Management CPU Driver Interface) is used as a
protocol to communicate with the RPU firmware. It has
pre-defined set of messages for different message exchanges
between APU and RPU.
Signed-off-by: Puneet Gupta <puneet.gupta@amd.com>
Signed-off-by: Nipun Gupta <nipun.gupta@amd.com>
Signed-off-by: Tarak Reddy <tarak.reddy@amd.com>
---
drivers/cdx/Kconfig | 2 +
drivers/cdx/Makefile | 2 +-
drivers/cdx/controller/Kconfig | 20 +
drivers/cdx/controller/Makefile | 9 +
drivers/cdx/controller/bitfield.h | 88 +++
drivers/cdx/controller/mc_cdx_pcol.h | 590 +++++++++++++++++
drivers/cdx/controller/mcdi.c | 913 +++++++++++++++++++++++++++
drivers/cdx/controller/mcdi.h | 238 +++++++
8 files changed, 1861 insertions(+), 1 deletion(-)
create mode 100644 drivers/cdx/controller/Kconfig
create mode 100644 drivers/cdx/controller/Makefile
create mode 100644 drivers/cdx/controller/bitfield.h
create mode 100644 drivers/cdx/controller/mc_cdx_pcol.h
create mode 100644 drivers/cdx/controller/mcdi.c
create mode 100644 drivers/cdx/controller/mcdi.h
diff --git a/drivers/cdx/Kconfig b/drivers/cdx/Kconfig
index d134c8bfc7f8..ca4e959dbe2e 100644
--- a/drivers/cdx/Kconfig
+++ b/drivers/cdx/Kconfig
@@ -14,3 +14,5 @@ config CDX_BUS
of CDX devices. CDX devices are memory mapped on system bus
for embedded CPUs. CDX bus uses CDX controller and firmware
to scan these CDX devices.
+
+source "drivers/cdx/controller/Kconfig"
diff --git a/drivers/cdx/Makefile b/drivers/cdx/Makefile
index c5feb11fc718..0324e4914f6e 100644
--- a/drivers/cdx/Makefile
+++ b/drivers/cdx/Makefile
@@ -5,4 +5,4 @@
# Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
#
-obj-$(CONFIG_CDX_BUS) += cdx.o
+obj-$(CONFIG_CDX_BUS) += cdx.o controller/
diff --git a/drivers/cdx/controller/Kconfig b/drivers/cdx/controller/Kconfig
new file mode 100644
index 000000000000..785c71063b2a
--- /dev/null
+++ b/drivers/cdx/controller/Kconfig
@@ -0,0 +1,20 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# CDX controller configuration
+#
+# Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+#
+
+if CDX_BUS
+
+config MCDI_LOGGING
+ bool "MCDI Logging for the CDX controller"
+ depends on CDX_CONTROLLER
+ help
+ Enable MCDI Logging for
+ the CDX Controller for debug
+ purpose.
+
+ If unsure, say N.
+
+endif
diff --git a/drivers/cdx/controller/Makefile b/drivers/cdx/controller/Makefile
new file mode 100644
index 000000000000..0ce200678eda
--- /dev/null
+++ b/drivers/cdx/controller/Makefile
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Makefile for CDX controller drivers
+#
+# Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+#
+
+obj-$(CONFIG_CDX_CONTROLLER) += cdx-controller.o
+cdx-controller-objs := mcdi.o
diff --git a/drivers/cdx/controller/bitfield.h b/drivers/cdx/controller/bitfield.h
new file mode 100644
index 000000000000..65a0254e263d
--- /dev/null
+++ b/drivers/cdx/controller/bitfield.h
@@ -0,0 +1,88 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ * Copyright 2005-2006 Fen Systems Ltd.
+ * Copyright 2006-2013 Solarflare Communications Inc.
+ * Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+ */
+
+#ifndef CDX_BITFIELD_H
+#define CDX_BITFIELD_H
+
+/* Lowest bit numbers and widths */
+#define CDX_DWORD_LBN 0
+#define CDX_DWORD_WIDTH 32
+
+/* Specified attribute (e.g. LBN) of the specified field */
+#define CDX_VAL(field, attribute) field ## _ ## attribute
+/* Low bit number of the specified field */
+#define CDX_LOW_BIT(field) CDX_VAL(field, LBN)
+/* Bit width of the specified field */
+#define CDX_WIDTH(field) CDX_VAL(field, WIDTH)
+/* High bit number of the specified field */
+#define CDX_HIGH_BIT(field) (CDX_LOW_BIT(field) + CDX_WIDTH(field) - 1)
+
+/* A doubleword (i.e. 4 byte) datatype - little-endian in HW */
+struct cdx_dword {
+ __le32 cdx_u32;
+};
+
+/* Value expanders for printk */
+#define CDX_DWORD_VAL(dword) \
+ ((unsigned int)le32_to_cpu((dword).cdx_u32))
+
+/*
+ * Extract bit field portion [low,high) from the 32-bit little-endian
+ * element which contains bits [min,max)
+ */
+#define CDX_DWORD_FIELD(dword, field) \
+ (FIELD_GET(GENMASK(CDX_HIGH_BIT(field), CDX_LOW_BIT(field)), \
+ le32_to_cpu((dword).cdx_u32)))
+
+/*
+ * Creates the portion of the named bit field that lies within the
+ * range [min,max).
+ */
+#define CDX_INSERT_FIELD(field, value) \
+ (FIELD_PREP(GENMASK(CDX_HIGH_BIT(field), \
+ CDX_LOW_BIT(field)), value))
+
+/*
+ * Creates the portion of the named bit fields that lie within the
+ * range [min,max).
+ */
+#define CDX_INSERT_FIELDS(field1, value1, \
+ field2, value2, \
+ field3, value3, \
+ field4, value4, \
+ field5, value5, \
+ field6, value6, \
+ field7, value7) \
+ (CDX_INSERT_FIELD(field1, (value1)) | \
+ CDX_INSERT_FIELD(field2, (value2)) | \
+ CDX_INSERT_FIELD(field3, (value3)) | \
+ CDX_INSERT_FIELD(field4, (value4)) | \
+ CDX_INSERT_FIELD(field5, (value5)) | \
+ CDX_INSERT_FIELD(field6, (value6)) | \
+ CDX_INSERT_FIELD(field7, (value7)))
+
+#define CDX_POPULATE_DWORD(dword, ...) \
+ (dword).cdx_u32 = cpu_to_le32(CDX_INSERT_FIELDS(__VA_ARGS__))
+
+/* Populate a dword field with various numbers of arguments */
+#define CDX_POPULATE_DWORD_7 CDX_POPULATE_DWORD
+#define CDX_POPULATE_DWORD_6(dword, ...) \
+ CDX_POPULATE_DWORD_7(dword, CDX_DWORD, 0, __VA_ARGS__)
+#define CDX_POPULATE_DWORD_5(dword, ...) \
+ CDX_POPULATE_DWORD_6(dword, CDX_DWORD, 0, __VA_ARGS__)
+#define CDX_POPULATE_DWORD_4(dword, ...) \
+ CDX_POPULATE_DWORD_5(dword, CDX_DWORD, 0, __VA_ARGS__)
+#define CDX_POPULATE_DWORD_3(dword, ...) \
+ CDX_POPULATE_DWORD_4(dword, CDX_DWORD, 0, __VA_ARGS__)
+#define CDX_POPULATE_DWORD_2(dword, ...) \
+ CDX_POPULATE_DWORD_3(dword, CDX_DWORD, 0, __VA_ARGS__)
+#define CDX_POPULATE_DWORD_1(dword, ...) \
+ CDX_POPULATE_DWORD_2(dword, CDX_DWORD, 0, __VA_ARGS__)
+#define CDX_SET_DWORD(dword) \
+ CDX_POPULATE_DWORD_1(dword, CDX_DWORD, 0xffffffff)
+
+#endif /* CDX_BITFIELD_H */
diff --git a/drivers/cdx/controller/mc_cdx_pcol.h b/drivers/cdx/controller/mc_cdx_pcol.h
new file mode 100644
index 000000000000..0d4005d8da08
--- /dev/null
+++ b/drivers/cdx/controller/mc_cdx_pcol.h
@@ -0,0 +1,590 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ * Driver for AMD network controllers and boards
+ *
+ * Copyright (C) 2021, Xilinx, Inc.
+ * Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+ */
+
+#ifndef MC_CDX_PCOL_H
+#define MC_CDX_PCOL_H
+
+/* The current version of the MCDI protocol. */
+#define MCDI_PCOL_VERSION 2
+
+/*
+ * Each MCDI request starts with an MCDI_HEADER, which is a 32bit
+ * structure, filled in by the client.
+ *
+ * 0 7 8 16 20 22 23 24 31
+ * | CODE | R | LEN | SEQ | Rsvd | E | R | XFLAGS |
+ * | | |
+ * | | \--- Response
+ * | \------- Error
+ * \------------------------------ Resync (always set)
+ *
+ * The client writes its request into MC shared memory, and rings the
+ * doorbell. Each request is completed either by the MC writing
+ * back into shared memory, or by writing out an event.
+ *
+ * All MCDI commands support completion by shared memory response. Each
+ * request may also contain additional data (accounted for by HEADER.LEN),
+ * and some responses may also contain additional data (again, accounted
+ * for by HEADER.LEN).
+ *
+ * Some MCDI commands support completion by event, in which any associated
+ * response data is included in the event.
+ *
+ * The protocol requires one response to be delivered for every request; a
+ * request should not be sent unless the response for the previous request
+ * has been received (either by polling shared memory, or by receiving
+ * an event).
+ */
+
+/** Request/Response structure */
+#define MCDI_HEADER_OFST 0
+#define MCDI_HEADER_CODE_LBN 0
+#define MCDI_HEADER_CODE_WIDTH 7
+#define MCDI_HEADER_RESYNC_LBN 7
+#define MCDI_HEADER_RESYNC_WIDTH 1
+#define MCDI_HEADER_DATALEN_LBN 8
+#define MCDI_HEADER_DATALEN_WIDTH 8
+#define MCDI_HEADER_SEQ_LBN 16
+#define MCDI_HEADER_SEQ_WIDTH 4
+#define MCDI_HEADER_RSVD_LBN 20
+#define MCDI_HEADER_RSVD_WIDTH 1
+#define MCDI_HEADER_NOT_EPOCH_LBN 21
+#define MCDI_HEADER_NOT_EPOCH_WIDTH 1
+#define MCDI_HEADER_ERROR_LBN 22
+#define MCDI_HEADER_ERROR_WIDTH 1
+#define MCDI_HEADER_RESPONSE_LBN 23
+#define MCDI_HEADER_RESPONSE_WIDTH 1
+#define MCDI_HEADER_XFLAGS_LBN 24
+#define MCDI_HEADER_XFLAGS_WIDTH 8
+/* Request response using event */
+#define MCDI_HEADER_XFLAGS_EVREQ 0x01
+/* Request (and signal) early doorbell return */
+#define MCDI_HEADER_XFLAGS_DBRET 0x02
+
+/* Maximum number of payload bytes */
+#define MCDI_CTL_SDU_LEN_MAX_V2 0x400
+
+#define MCDI_CTL_SDU_LEN_MAX MCDI_CTL_SDU_LEN_MAX_V2
+
+/*
+ * The MC can generate events for two reasons:
+ * - To advance a shared memory request if XFLAGS_EVREQ was set
+ * - As a notification (link state, i2c event), controlled
+ * via MC_CMD_LOG_CTRL
+ *
+ * Both events share a common structure:
+ *
+ * 0 32 33 36 44 52 60
+ * | Data | Cont | Level | Src | Code | Rsvd |
+ * |
+ * \ There is another event pending in this notification
+ *
+ * If Code==CMDDONE, then the fields are further interpreted as:
+ *
+ * - LEVEL==INFO Command succeeded
+ * - LEVEL==ERR Command failed
+ *
+ * 0 8 16 24 32
+ * | Seq | Datalen | Errno | Rsvd |
+ *
+ * These fields are taken directly out of the standard MCDI header, i.e.,
+ * LEVEL==ERR, Datalen == 0 => Reboot
+ *
+ * Events can be squirted out of the UART (using LOG_CTRL) without a
+ * MCDI header. An event can be distinguished from a MCDI response by
+ * examining the first byte which is 0xc0. This corresponds to the
+ * non-existent MCDI command MC_CMD_DEBUG_LOG.
+ *
+ * 0 7 8
+ * | command | Resync | = 0xc0
+ *
+ * Since the event is written in big-endian byte order, this works
+ * providing bits 56-63 of the event are 0xc0.
+ *
+ * 56 60 63
+ * | Rsvd | Code | = 0xc0
+ *
+ * Which means for convenience the event code is 0xc for all MC
+ * generated events.
+ */
+
+/*
+ * the errno value may be followed by the (0-based) number of the
+ * first argument that could not be processed.
+ */
+#define MC_CMD_ERR_ARG_OFST 4
+
+/* MC_CMD_ERR MCDI error codes. */
+/* Operation not permitted. */
+#define MC_CMD_ERR_EPERM 0x1
+/* Non-existent command target */
+#define MC_CMD_ERR_ENOENT 0x2
+/* assert() has killed the MC */
+#define MC_CMD_ERR_EINTR 0x4
+/* I/O failure */
+#define MC_CMD_ERR_EIO 0x5
+/* Already exists */
+#define MC_CMD_ERR_EEXIST 0x6
+/* Try again */
+#define MC_CMD_ERR_EAGAIN 0xb
+/* Out of memory */
+#define MC_CMD_ERR_ENOMEM 0xc
+/* Caller does not hold required locks */
+#define MC_CMD_ERR_EACCES 0xd
+/* Resource is currently unavailable (e.g. lock contention) */
+#define MC_CMD_ERR_EBUSY 0x10
+/* No such device */
+#define MC_CMD_ERR_ENODEV 0x13
+/* Invalid argument to target */
+#define MC_CMD_ERR_EINVAL 0x16
+/* No space */
+#define MC_CMD_ERR_ENOSPC 0x1c
+/* Read-only */
+#define MC_CMD_ERR_EROFS 0x1e
+/* Broken pipe */
+#define MC_CMD_ERR_EPIPE 0x20
+/* Out of range */
+#define MC_CMD_ERR_ERANGE 0x22
+/* Non-recursive resource is already acquired */
+#define MC_CMD_ERR_EDEADLK 0x23
+/* Operation not implemented */
+#define MC_CMD_ERR_ENOSYS 0x26
+/* Operation timed out */
+#define MC_CMD_ERR_ETIME 0x3e
+/* Link has been severed */
+#define MC_CMD_ERR_ENOLINK 0x43
+/* Protocol error */
+#define MC_CMD_ERR_EPROTO 0x47
+/* Bad message */
+#define MC_CMD_ERR_EBADMSG 0x4a
+/* Operation not supported */
+#define MC_CMD_ERR_ENOTSUP 0x5f
+/* Address not available */
+#define MC_CMD_ERR_EADDRNOTAVAIL 0x63
+/* Not connected */
+#define MC_CMD_ERR_ENOTCONN 0x6b
+/* Operation already in progress */
+#define MC_CMD_ERR_EALREADY 0x72
+/* Stale handle. The handle references resource that no longer exists */
+#define MC_CMD_ERR_ESTALE 0x74
+/* Resource allocation failed. */
+#define MC_CMD_ERR_ALLOC_FAIL 0x1000
+/* V-adaptor not found. */
+#define MC_CMD_ERR_NO_VADAPTOR 0x1001
+/* EVB port not found. */
+#define MC_CMD_ERR_NO_EVB_PORT 0x1002
+/* V-switch not found. */
+#define MC_CMD_ERR_NO_VSWITCH 0x1003
+/* Too many VLAN tags. */
+#define MC_CMD_ERR_VLAN_LIMIT 0x1004
+/* Bad PCI function number. */
+#define MC_CMD_ERR_BAD_PCI_FUNC 0x1005
+/* Invalid VLAN mode. */
+#define MC_CMD_ERR_BAD_VLAN_MODE 0x1006
+/* Invalid v-switch type. */
+#define MC_CMD_ERR_BAD_VSWITCH_TYPE 0x1007
+/* Invalid v-port type. */
+#define MC_CMD_ERR_BAD_VPORT_TYPE 0x1008
+/* MAC address exists. */
+#define MC_CMD_ERR_MAC_EXIST 0x1009
+/* Slave core not present */
+#define MC_CMD_ERR_SLAVE_NOT_PRESENT 0x100a
+/* The datapath is disabled. */
+#define MC_CMD_ERR_DATAPATH_DISABLED 0x100b
+/* The requesting client is not a function */
+#define MC_CMD_ERR_CLIENT_NOT_FN 0x100c
+/*
+ * The requested operation might require the command to be passed between
+ * MCs, and thetransport doesn't support that. Should only ever been seen over
+ * the UART.
+ */
+#define MC_CMD_ERR_NO_PRIVILEGE 0x1013
+/*
+ * Workaround 26807 could not be turned on/off because some functions
+ * have already installed filters. See the comment at
+ * MC_CMD_WORKAROUND_BUG26807. May also returned for other operations such as
+ * sub-variant switching.
+ */
+#define MC_CMD_ERR_FILTERS_PRESENT 0x1014
+/* The clock whose frequency you've attempted to set doesn't exist */
+#define MC_CMD_ERR_NO_CLOCK 0x1015
+/*
+ * Returned by MC_CMD_TESTASSERT if the action that should have caused an
+ * assertion failed to do so.
+ */
+#define MC_CMD_ERR_UNREACHABLE 0x1016
+/*
+ * This command needs to be processed in the background but there were no
+ * resources to do so. Send it again after a command has completed.
+ */
+#define MC_CMD_ERR_QUEUE_FULL 0x1017
+/*
+ * The operation could not be completed because the PCIe link has gone
+ * away. This error code is never expected to be returned over the TLP
+ * transport.
+ */
+#define MC_CMD_ERR_NO_PCIE 0x1018
+/*
+ * The operation could not be completed because the datapath has gone
+ * away. This is distinct from MC_CMD_ERR_DATAPATH_DISABLED in that the
+ * datapath absence may be temporary
+ */
+#define MC_CMD_ERR_NO_DATAPATH 0x1019
+/* The operation could not complete because some VIs are allocated */
+#define MC_CMD_ERR_VIS_PRESENT 0x101a
+/*
+ * The operation could not complete because some PIO buffers are
+ * allocated
+ */
+#define MC_CMD_ERR_PIOBUFS_PRESENT 0x101b
+
+/***********************************/
+/*
+ * MC_CMD_CDX_BUS_ENUM_BUSES
+ * CDX bus hosts devices (functions) that are implemented using the Composable
+ * DMA subsystem and directly mapped into the memory space of the FGPA PSX
+ * Application Processors (APUs). As such, they only apply to the PSX APU side,
+ * not the host (PCIe). Unlike PCIe, these devices have no native configuration
+ * space or enumeration mechanism, so this message set provides a minimal
+ * interface for discovery and management (bus reset, FLR, BME) of such
+ * devices. This command returns the number of CDX buses present in the system.
+ */
+#define MC_CMD_CDX_BUS_ENUM_BUSES 0x1
+#define MC_CMD_CDX_BUS_ENUM_BUSES_MSGSET 0x1
+#undef MC_CMD_0x1_PRIVILEGE_CTG
+
+#define MC_CMD_0x1_PRIVILEGE_CTG SRIOV_CTG_ADMIN
+
+/* MC_CMD_CDX_BUS_ENUM_BUSES_IN msgrequest */
+#define MC_CMD_CDX_BUS_ENUM_BUSES_IN_LEN 0
+
+/* MC_CMD_CDX_BUS_ENUM_BUSES_OUT msgresponse */
+#define MC_CMD_CDX_BUS_ENUM_BUSES_OUT_LEN 4
+/*
+ * Number of CDX buses present in the system. Buses are numbered 0 to
+ * BUS_COUNT-1
+ */
+#define MC_CMD_CDX_BUS_ENUM_BUSES_OUT_BUS_COUNT_OFST 0
+#define MC_CMD_CDX_BUS_ENUM_BUSES_OUT_BUS_COUNT_LEN 4
+
+/***********************************/
+/*
+ * MC_CMD_CDX_BUS_ENUM_DEVICES
+ * Enumerate CDX bus devices on a given bus
+ */
+#define MC_CMD_CDX_BUS_ENUM_DEVICES 0x2
+#define MC_CMD_CDX_BUS_ENUM_DEVICES_MSGSET 0x2
+#undef MC_CMD_0x2_PRIVILEGE_CTG
+
+#define MC_CMD_0x2_PRIVILEGE_CTG SRIOV_CTG_ADMIN
+
+/* MC_CMD_CDX_BUS_ENUM_DEVICES_IN msgrequest */
+#define MC_CMD_CDX_BUS_ENUM_DEVICES_IN_LEN 4
+/*
+ * Bus number to enumerate, in range 0 to BUS_COUNT-1, as returned by
+ * MC_CMD_CDX_BUS_ENUM_BUSES_OUT
+ */
+#define MC_CMD_CDX_BUS_ENUM_DEVICES_IN_BUS_OFST 0
+#define MC_CMD_CDX_BUS_ENUM_DEVICES_IN_BUS_LEN 4
+
+/* MC_CMD_CDX_BUS_ENUM_DEVICES_OUT msgresponse */
+#define MC_CMD_CDX_BUS_ENUM_DEVICES_OUT_LEN 4
+/*
+ * Number of devices present on the bus. Devices on the bus are numbered 0 to
+ * DEVICE_COUNT-1. Returns EAGAIN if number of devices unknown or if the target
+ * devices are not ready (e.g. undergoing a bus reset)
+ */
+#define MC_CMD_CDX_BUS_ENUM_DEVICES_OUT_DEVICE_COUNT_OFST 0
+#define MC_CMD_CDX_BUS_ENUM_DEVICES_OUT_DEVICE_COUNT_LEN 4
+
+/***********************************/
+/*
+ * MC_CMD_CDX_BUS_GET_DEVICE_CONFIG
+ * Returns device identification and MMIO/MSI resource data for a CDX device.
+ * The expected usage is for the caller to first retrieve the number of devices
+ * on the bus using MC_CMD_BUS_ENUM_DEVICES, then loop through the range (0,
+ * DEVICE_COUNT - 1), retrieving device resource data. May return EAGAIN if the
+ * number of exposed devices or device resources change during enumeration (due
+ * to e.g. a PL reload / bus reset), in which case the caller is expected to
+ * restart the enumeration loop. MMIO addresses are specified in terms of bus
+ * addresses (prior to any potential IOMMU translation). For versal-net, these
+ * are equivalent to APU physical addresses. Implementation note - for this to
+ * work, the implementation needs to keep state (generation count) per client.
+ */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG 0x3
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_MSGSET 0x3
+#undef MC_CMD_0x3_PRIVILEGE_CTG
+
+#define MC_CMD_0x3_PRIVILEGE_CTG SRIOV_CTG_ADMIN
+
+/* MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_IN msgrequest */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_IN_LEN 8
+/* Device bus number, in range 0 to BUS_COUNT-1 */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_IN_BUS_OFST 0
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_IN_BUS_LEN 4
+/* Device number relative to the bus, in range 0 to DEVICE_COUNT-1 for that bus */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_IN_DEVICE_OFST 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_IN_DEVICE_LEN 4
+
+/* MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT msgresponse */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_LEN 88
+/* 16-bit Vendor identifier, compliant with PCI-SIG VendorID assignment. */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_VENDOR_ID_OFST 0
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_VENDOR_ID_LEN 2
+/* 16-bit Device ID assigned by the vendor */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_DEVICE_ID_OFST 2
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_DEVICE_ID_LEN 2
+/*
+ * 16-bit Subsystem Vendor ID, , compliant with PCI-SIG VendorID assignment.
+ * For further device differentiation, as required. 0 if unused.
+ */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_SUBSYS_VENDOR_ID_OFST 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_SUBSYS_VENDOR_ID_LEN 2
+/*
+ * 16-bit Subsystem Device ID assigned by the vendor. For further device
+ * differentiation, as required. 0 if unused.
+ */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_SUBSYS_DEVICE_ID_OFST 6
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_SUBSYS_DEVICE_ID_LEN 2
+/* 24-bit Device Class code, compliant with PCI-SIG Device Class codes */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_DEVICE_CLASS_OFST 8
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_DEVICE_CLASS_LEN 3
+/* 8-bit vendor-assigned revision */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_DEVICE_REVISION_OFST 11
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_DEVICE_REVISION_LEN 1
+/* Reserved (alignment) */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_RESERVED_OFST 12
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_RESERVED_LEN 4
+/* MMIO region 0 base address (bus address), 0 if unused */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_BASE_OFST 16
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_BASE_LEN 8
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_BASE_LO_OFST 16
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_BASE_LO_LEN 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_BASE_LO_LBN 128
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_BASE_LO_WIDTH 32
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_BASE_HI_OFST 20
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_BASE_HI_LEN 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_BASE_HI_LBN 160
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_BASE_HI_WIDTH 32
+/* MMIO region 0 size, 0 if unused */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_SIZE_OFST 24
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_SIZE_LEN 8
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_SIZE_LO_OFST 24
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_SIZE_LO_LEN 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_SIZE_LO_LBN 192
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_SIZE_LO_WIDTH 32
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_SIZE_HI_OFST 28
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_SIZE_HI_LEN 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_SIZE_HI_LBN 224
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_SIZE_HI_WIDTH 32
+/* MMIO region 1 base address (bus address), 0 if unused */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_BASE_OFST 32
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_BASE_LEN 8
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_BASE_LO_OFST 32
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_BASE_LO_LEN 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_BASE_LO_LBN 256
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_BASE_LO_WIDTH 32
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_BASE_HI_OFST 36
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_BASE_HI_LEN 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_BASE_HI_LBN 288
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_BASE_HI_WIDTH 32
+/* MMIO region 1 size, 0 if unused */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_SIZE_OFST 40
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_SIZE_LEN 8
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_SIZE_LO_OFST 40
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_SIZE_LO_LEN 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_SIZE_LO_LBN 320
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_SIZE_LO_WIDTH 32
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_SIZE_HI_OFST 44
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_SIZE_HI_LEN 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_SIZE_HI_LBN 352
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_SIZE_HI_WIDTH 32
+/* MMIO region 2 base address (bus address), 0 if unused */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_BASE_OFST 48
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_BASE_LEN 8
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_BASE_LO_OFST 48
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_BASE_LO_LEN 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_BASE_LO_LBN 384
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_BASE_LO_WIDTH 32
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_BASE_HI_OFST 52
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_BASE_HI_LEN 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_BASE_HI_LBN 416
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_BASE_HI_WIDTH 32
+/* MMIO region 2 size, 0 if unused */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_SIZE_OFST 56
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_SIZE_LEN 8
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_SIZE_LO_OFST 56
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_SIZE_LO_LEN 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_SIZE_LO_LBN 448
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_SIZE_LO_WIDTH 32
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_SIZE_HI_OFST 60
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_SIZE_HI_LEN 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_SIZE_HI_LBN 480
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_SIZE_HI_WIDTH 32
+/* MMIO region 3 base address (bus address), 0 if unused */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_BASE_OFST 64
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_BASE_LEN 8
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_BASE_LO_OFST 64
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_BASE_LO_LEN 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_BASE_LO_LBN 512
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_BASE_LO_WIDTH 32
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_BASE_HI_OFST 68
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_BASE_HI_LEN 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_BASE_HI_LBN 544
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_BASE_HI_WIDTH 32
+/* MMIO region 3 size, 0 if unused */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_SIZE_OFST 72
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_SIZE_LEN 8
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_SIZE_LO_OFST 72
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_SIZE_LO_LEN 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_SIZE_LO_LBN 576
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_SIZE_LO_WIDTH 32
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_SIZE_HI_OFST 76
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_SIZE_HI_LEN 4
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_SIZE_HI_LBN 608
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_SIZE_HI_WIDTH 32
+/* MSI vector count */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MSI_COUNT_OFST 80
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_MSI_COUNT_LEN 4
+/* Requester ID used by device (SMMU StreamID, GIC ITS DeviceID) */
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_REQUESTER_ID_OFST 84
+#define MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_REQUESTER_ID_LEN 4
+
+/***********************************/
+/*
+ * MC_CMD_CDX_DEVICE_RESET
+ * After this call completes, device DMA and interrupts are quiesced, devices
+ * logic is reset in a hardware-specific way and DMA bus mastering is disabled.
+ */
+#define MC_CMD_CDX_DEVICE_RESET 0x6
+#define MC_CMD_CDX_DEVICE_RESET_MSGSET 0x6
+#undef MC_CMD_0x6_PRIVILEGE_CTG
+
+#define MC_CMD_0x6_PRIVILEGE_CTG SRIOV_CTG_ADMIN
+
+/* MC_CMD_CDX_DEVICE_RESET_IN msgrequest */
+#define MC_CMD_CDX_DEVICE_RESET_IN_LEN 8
+/* Device bus number, in range 0 to BUS_COUNT-1 */
+#define MC_CMD_CDX_DEVICE_RESET_IN_BUS_OFST 0
+#define MC_CMD_CDX_DEVICE_RESET_IN_BUS_LEN 4
+/* Device number relative to the bus, in range 0 to DEVICE_COUNT-1 for that bus */
+#define MC_CMD_CDX_DEVICE_RESET_IN_DEVICE_OFST 4
+#define MC_CMD_CDX_DEVICE_RESET_IN_DEVICE_LEN 4
+
+/*
+ * MC_CMD_CDX_DEVICE_RESET_OUT msgresponse: The device is quiesced and all
+ * pending device initiated DMA has completed.
+ */
+#define MC_CMD_CDX_DEVICE_RESET_OUT_LEN 0
+
+/***********************************/
+/*
+ * MC_CMD_CDX_DEVICE_CONTROL_SET
+ * If BUS_MASTER is set to disabled, device DMA and interrupts are quiesced.
+ * Pending DMA requests and MSI interrupts are flushed and no further DMA or
+ * interrupts are issued after this command returns. If BUS_MASTER is set to
+ * enabled, device is allowed to initiate DMA. Whether interrupts are enabled
+ * also depends on the value of MSI_ENABLE bit. Note that, in this case, the
+ * device may start DMA before the host receives and processes the MCDI
+ * response. MSI_ENABLE masks or unmasks device interrupts only. Note that for
+ * interrupts to be delivered to the host, both BUS_MASTER and MSI_ENABLE needs
+ * to be set. MMIO_REGIONS_ENABLE enables or disables host accesses to device
+ * MMIO regions. Note that an implementation is allowed to permanently set this
+ * bit to 1, in which case MC_CMD_CDX_DEVICE_CONTROL_GET will always return 1
+ * for this bit, regardless of the value set here.
+ */
+#define MC_CMD_CDX_DEVICE_CONTROL_SET 0x7
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_MSGSET 0x7
+#undef MC_CMD_0x7_PRIVILEGE_CTG
+
+#define MC_CMD_0x7_PRIVILEGE_CTG SRIOV_CTG_ADMIN
+
+/* MC_CMD_CDX_DEVICE_CONTROL_SET_IN msgrequest */
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_IN_LEN 12
+/* Device bus number, in range 0 to BUS_COUNT-1 */
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_IN_BUS_OFST 0
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_IN_BUS_LEN 4
+/* Device number relative to the bus, in range 0 to DEVICE_COUNT-1 for that bus */
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_IN_DEVICE_OFST 4
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_IN_DEVICE_LEN 4
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_IN_FLAGS_OFST 8
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_IN_FLAGS_LEN 4
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_IN_BUS_MASTER_ENABLE_OFST 8
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_IN_BUS_MASTER_ENABLE_LBN 0
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_IN_BUS_MASTER_ENABLE_WIDTH 1
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_IN_MSI_ENABLE_OFST 8
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_IN_MSI_ENABLE_LBN 1
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_IN_MSI_ENABLE_WIDTH 1
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_IN_MMIO_REGIONS_ENABLE_OFST 8
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_IN_MMIO_REGIONS_ENABLE_LBN 2
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_IN_MMIO_REGIONS_ENABLE_WIDTH 1
+
+/* MC_CMD_CDX_DEVICE_CONTROL_SET_OUT msgresponse */
+#define MC_CMD_CDX_DEVICE_CONTROL_SET_OUT_LEN 0
+
+/***********************************/
+/*
+ * MC_CMD_CDX_DEVICE_CONTROL_GET
+ * Returns device DMA, interrupt and MMIO region access control bits. See
+ * MC_CMD_CDX_DEVICE_CONTROL_SET for definition of the available control bits.
+ */
+#define MC_CMD_CDX_DEVICE_CONTROL_GET 0x8
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_MSGSET 0x8
+#undef MC_CMD_0x8_PRIVILEGE_CTG
+
+#define MC_CMD_0x8_PRIVILEGE_CTG SRIOV_CTG_ADMIN
+
+/* MC_CMD_CDX_DEVICE_CONTROL_GET_IN msgrequest */
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_IN_LEN 8
+/* Device bus number, in range 0 to BUS_COUNT-1 */
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_IN_BUS_OFST 0
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_IN_BUS_LEN 4
+/* Device number relative to the bus, in range 0 to DEVICE_COUNT-1 for that bus */
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_IN_DEVICE_OFST 4
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_IN_DEVICE_LEN 4
+
+/* MC_CMD_CDX_DEVICE_CONTROL_GET_OUT msgresponse */
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_OUT_LEN 4
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_OUT_FLAGS_OFST 0
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_OUT_FLAGS_LEN 4
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_OUT_BUS_MASTER_ENABLE_OFST 0
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_OUT_BUS_MASTER_ENABLE_LBN 0
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_OUT_BUS_MASTER_ENABLE_WIDTH 1
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_OUT_MSI_ENABLE_OFST 0
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_OUT_MSI_ENABLE_LBN 1
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_OUT_MSI_ENABLE_WIDTH 1
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_OUT_MMIO_REGIONS_ENABLE_OFST 0
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_OUT_MMIO_REGIONS_ENABLE_LBN 2
+#define MC_CMD_CDX_DEVICE_CONTROL_GET_OUT_MMIO_REGIONS_ENABLE_WIDTH 1
+
+/***********************************/
+/* MC_CMD_V2_EXTN - Encapsulation for a v2 extended command */
+#define MC_CMD_V2_EXTN 0x7f
+
+/* MC_CMD_V2_EXTN_IN msgrequest */
+#define MC_CMD_V2_EXTN_IN_LEN 4
+/* the extended command number */
+#define MC_CMD_V2_EXTN_IN_EXTENDED_CMD_LBN 0
+#define MC_CMD_V2_EXTN_IN_EXTENDED_CMD_WIDTH 15
+#define MC_CMD_V2_EXTN_IN_UNUSED_LBN 15
+#define MC_CMD_V2_EXTN_IN_UNUSED_WIDTH 1
+/* the actual length of the encapsulated command */
+#define MC_CMD_V2_EXTN_IN_ACTUAL_LEN_LBN 16
+#define MC_CMD_V2_EXTN_IN_ACTUAL_LEN_WIDTH 10
+#define MC_CMD_V2_EXTN_IN_UNUSED2_LBN 26
+#define MC_CMD_V2_EXTN_IN_UNUSED2_WIDTH 2
+/* Type of command/response */
+#define MC_CMD_V2_EXTN_IN_MESSAGE_TYPE_LBN 28
+#define MC_CMD_V2_EXTN_IN_MESSAGE_TYPE_WIDTH 4
+/*
+ * enum: MCDI command directed to versal-net. MCDI responses of this type
+ * are not defined.
+ */
+#define MC_CMD_V2_EXTN_IN_MCDI_MESSAGE_TYPE_PLATFORM 0x2
+
+#endif /* MC_CDX_PCOL_H */
diff --git a/drivers/cdx/controller/mcdi.c b/drivers/cdx/controller/mcdi.c
new file mode 100644
index 000000000000..1a9f1f6e80ae
--- /dev/null
+++ b/drivers/cdx/controller/mcdi.c
@@ -0,0 +1,913 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Management-Controller-to-Driver Interface
+ *
+ * Copyright 2008-2013 Solarflare Communications Inc.
+ * Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+ */
+#include <linux/delay.h>
+#include <linux/slab.h>
+#include <linux/io.h>
+#include <linux/spinlock.h>
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/ethtool.h>
+#include <linux/if_vlan.h>
+#include <linux/timer.h>
+#include <linux/list.h>
+#include <linux/pci.h>
+#include <linux/device.h>
+#include <linux/version.h>
+#include <linux/rwsem.h>
+#include <linux/vmalloc.h>
+#include <net/netevent.h>
+#include <linux/log2.h>
+#include <linux/net_tstamp.h>
+#include <linux/wait.h>
+
+#include "bitfield.h"
+#include "mcdi.h"
+
+struct cdx_mcdi_copy_buffer {
+ struct cdx_dword buffer[DIV_ROUND_UP(MCDI_CTL_SDU_LEN_MAX, 4)];
+};
+
+#ifdef CONFIG_MCDI_LOGGING
+#define LOG_LINE_MAX (1024 - 32)
+#endif
+
+static void cdx_mcdi_cancel_cmd(struct cdx_mcdi *cdx, struct cdx_mcdi_cmd *cmd);
+static void cdx_mcdi_wait_for_cleanup(struct cdx_mcdi *cdx);
+static int cdx_mcdi_rpc_async_internal(struct cdx_mcdi *cdx,
+ struct cdx_mcdi_cmd *cmd,
+ unsigned int *handle);
+static void cdx_mcdi_start_or_queue(struct cdx_mcdi_iface *mcdi,
+ bool allow_retry);
+static void cdx_mcdi_cmd_start_or_queue(struct cdx_mcdi_iface *mcdi,
+ struct cdx_mcdi_cmd *cmd);
+static void cdx_mcdi_cmd_start_or_queue_ext(struct cdx_mcdi_iface *mcdi,
+ struct cdx_mcdi_cmd *cmd);
+static bool cdx_mcdi_complete_cmd(struct cdx_mcdi_iface *mcdi,
+ struct cdx_mcdi_cmd *cmd,
+ struct cdx_dword *outbuf,
+ int len,
+ struct list_head *cleanup_list);
+static void cdx_mcdi_timeout_cmd(struct cdx_mcdi_iface *mcdi,
+ struct cdx_mcdi_cmd *cmd,
+ struct list_head *cleanup_list);
+static void cdx_mcdi_cmd_work(struct work_struct *context);
+static void cdx_mcdi_mode_fail(struct cdx_mcdi *cdx, struct list_head *cleanup_list);
+static void _cdx_mcdi_display_error(struct cdx_mcdi *cdx, unsigned int cmd,
+ size_t inlen, int raw, int arg, int rc);
+
+static bool cdx_cmd_cancelled(struct cdx_mcdi_cmd *cmd)
+{
+ return cmd->state == MCDI_STATE_RUNNING_CANCELLED;
+}
+
+static void cdx_mcdi_cmd_release(struct kref *ref)
+{
+ kfree(container_of(ref, struct cdx_mcdi_cmd, ref));
+}
+
+static unsigned int cdx_mcdi_cmd_handle(struct cdx_mcdi_cmd *cmd)
+{
+ return cmd->handle;
+}
+
+static void _cdx_mcdi_remove_cmd(struct cdx_mcdi_iface *mcdi,
+ struct cdx_mcdi_cmd *cmd,
+ struct list_head *cleanup_list)
+{
+ /* if cancelled, the completers have already been called */
+ if (cdx_cmd_cancelled(cmd))
+ return;
+
+ if (cmd->completer) {
+ list_add_tail(&cmd->cleanup_list, cleanup_list);
+ ++mcdi->outstanding_cleanups;
+ kref_get(&cmd->ref);
+ }
+}
+
+static void cdx_mcdi_remove_cmd(struct cdx_mcdi_iface *mcdi,
+ struct cdx_mcdi_cmd *cmd,
+ struct list_head *cleanup_list)
+{
+ list_del(&cmd->list);
+ _cdx_mcdi_remove_cmd(mcdi, cmd, cleanup_list);
+ cmd->state = MCDI_STATE_FINISHED;
+ kref_put(&cmd->ref, cdx_mcdi_cmd_release);
+ if (list_empty(&mcdi->cmd_list))
+ wake_up(&mcdi->cmd_complete_wq);
+}
+
+static unsigned long cdx_mcdi_rpc_timeout(struct cdx_mcdi *cdx, unsigned int cmd)
+{
+ if (!cdx->mcdi_ops->mcdi_rpc_timeout)
+ return MCDI_RPC_TIMEOUT;
+ else
+ return cdx->mcdi_ops->mcdi_rpc_timeout(cdx, cmd);
+}
+
+int cdx_mcdi_init(struct cdx_mcdi *cdx)
+{
+ struct cdx_mcdi_iface *mcdi;
+ int rc = -ENOMEM;
+
+ cdx->mcdi = kzalloc(sizeof(*cdx->mcdi), GFP_KERNEL);
+ if (!cdx->mcdi)
+ goto fail;
+
+ mcdi = cdx_mcdi_if(cdx);
+ mcdi->cdx = cdx;
+
+#ifdef CONFIG_MCDI_LOGGING
+ mcdi->logging_buffer = kmalloc(LOG_LINE_MAX, GFP_KERNEL);
+ if (!mcdi->logging_buffer)
+ goto fail2;
+#endif
+ mcdi->workqueue = alloc_ordered_workqueue("mcdi_wq", 0);
+ if (!mcdi->workqueue)
+ goto fail3;
+ mutex_init(&mcdi->iface_lock);
+ mcdi->mode = MCDI_MODE_EVENTS;
+ INIT_LIST_HEAD(&mcdi->cmd_list);
+ init_waitqueue_head(&mcdi->cmd_complete_wq);
+
+ mcdi->new_epoch = true;
+
+ return 0;
+fail3:
+#ifdef CONFIG_MCDI_LOGGING
+ kfree(mcdi->logging_buffer);
+fail2:
+#endif
+ kfree(cdx->mcdi);
+ cdx->mcdi = NULL;
+fail:
+ return rc;
+}
+
+void cdx_mcdi_finish(struct cdx_mcdi *cdx)
+{
+ struct cdx_mcdi_iface *mcdi;
+
+ mcdi = cdx_mcdi_if(cdx);
+ if (!mcdi)
+ return;
+
+ cdx_mcdi_wait_for_cleanup(cdx);
+
+#ifdef CONFIG_MCDI_LOGGING
+ kfree(mcdi->logging_buffer);
+#endif
+
+ destroy_workqueue(mcdi->workqueue);
+ kfree(cdx->mcdi);
+ cdx->mcdi = NULL;
+}
+
+static bool cdx_mcdi_flushed(struct cdx_mcdi_iface *mcdi, bool ignore_cleanups)
+{
+ bool flushed;
+
+ mutex_lock(&mcdi->iface_lock);
+ flushed = list_empty(&mcdi->cmd_list) &&
+ (ignore_cleanups || !mcdi->outstanding_cleanups);
+ mutex_unlock(&mcdi->iface_lock);
+ return flushed;
+}
+
+/* Wait for outstanding MCDI commands to complete. */
+static void cdx_mcdi_wait_for_cleanup(struct cdx_mcdi *cdx)
+{
+ struct cdx_mcdi_iface *mcdi = cdx_mcdi_if(cdx);
+
+ if (!mcdi)
+ return;
+
+ wait_event(mcdi->cmd_complete_wq,
+ cdx_mcdi_flushed(mcdi, false));
+}
+
+int cdx_mcdi_wait_for_quiescence(struct cdx_mcdi *cdx,
+ unsigned int timeout_jiffies)
+{
+ struct cdx_mcdi_iface *mcdi = cdx_mcdi_if(cdx);
+ DEFINE_WAIT_FUNC(wait, woken_wake_function);
+ int rc = 0;
+
+ if (!mcdi)
+ return -EINVAL;
+
+ flush_workqueue(mcdi->workqueue);
+
+ add_wait_queue(&mcdi->cmd_complete_wq, &wait);
+
+ while (!cdx_mcdi_flushed(mcdi, true)) {
+ rc = wait_woken(&wait, TASK_IDLE, timeout_jiffies);
+ if (rc != 0)
+ continue;
+ break;
+ }
+
+ remove_wait_queue(&mcdi->cmd_complete_wq, &wait);
+
+ if (rc > 0)
+ rc = 0;
+ else if (rc == 0)
+ rc = -ETIMEDOUT;
+
+ return rc;
+}
+
+static u8 cdx_mcdi_payload_csum(const struct cdx_dword *hdr, size_t hdr_len,
+ const struct cdx_dword *sdu, size_t sdu_len)
+{
+ u8 *p = (u8 *)hdr;
+ u8 csum = 0;
+ int i;
+
+ for (i = 0; i < hdr_len; i++)
+ csum += p[i];
+
+ p = (u8 *)sdu;
+ for (i = 0; i < sdu_len; i++)
+ csum += p[i];
+
+ return ~csum & 0xff;
+}
+
+static void cdx_mcdi_send_request(struct cdx_mcdi *cdx,
+ struct cdx_mcdi_cmd *cmd)
+{
+ struct cdx_mcdi_iface *mcdi = cdx_mcdi_if(cdx);
+ const struct cdx_dword *inbuf = cmd->inbuf;
+ size_t inlen = cmd->inlen;
+ struct cdx_dword hdr[2];
+ size_t hdr_len;
+ bool not_epoch;
+ u32 xflags;
+#ifdef CONFIG_MCDI_LOGGING
+ char *buf;
+#endif
+
+ if (!mcdi)
+ return;
+#ifdef CONFIG_MCDI_LOGGING
+ buf = mcdi->logging_buffer; /* page-sized */
+#endif
+
+ mcdi->prev_seq = cmd->seq;
+ mcdi->seq_held_by[cmd->seq] = cmd;
+ mcdi->db_held_by = cmd;
+ cmd->started = jiffies;
+
+ not_epoch = !mcdi->new_epoch;
+ xflags = 0;
+
+ /* MCDI v2 */
+ WARN_ON(inlen > MCDI_CTL_SDU_LEN_MAX_V2);
+ CDX_POPULATE_DWORD_7(hdr[0],
+ MCDI_HEADER_RESPONSE, 0,
+ MCDI_HEADER_RESYNC, 1,
+ MCDI_HEADER_CODE, MC_CMD_V2_EXTN,
+ MCDI_HEADER_DATALEN, 0,
+ MCDI_HEADER_SEQ, cmd->seq,
+ MCDI_HEADER_XFLAGS, xflags,
+ MCDI_HEADER_NOT_EPOCH, not_epoch);
+ CDX_POPULATE_DWORD_3(hdr[1],
+ MC_CMD_V2_EXTN_IN_EXTENDED_CMD, cmd->cmd,
+ MC_CMD_V2_EXTN_IN_ACTUAL_LEN, inlen,
+ MC_CMD_V2_EXTN_IN_MESSAGE_TYPE,
+ MC_CMD_V2_EXTN_IN_MCDI_MESSAGE_TYPE_PLATFORM);
+ hdr_len = 8;
+
+#ifdef CONFIG_MCDI_LOGGING
+ if (!WARN_ON_ONCE(!buf)) {
+ const struct cdx_dword *frags[] = { hdr, inbuf };
+ const size_t frag_len[] = { hdr_len, round_up(inlen, 4) };
+ int bytes = 0;
+ int i, j;
+
+ for (j = 0; j < ARRAY_SIZE(frags); j++) {
+ const struct cdx_dword *frag;
+
+ frag = frags[j];
+ for (i = 0;
+ i < frag_len[j] / 4;
+ i++) {
+ /*
+ * Do not exceed the internal printk limit.
+ * The string before that is just over 70 bytes.
+ */
+ if ((bytes + 75) > LOG_LINE_MAX) {
+ pr_info("MCDI RPC REQ:%s \\\n", buf);
+ bytes = 0;
+ }
+ bytes += snprintf(buf + bytes,
+ LOG_LINE_MAX - bytes, " %08x",
+ le32_to_cpu(frag[i].cdx_u32));
+ }
+ }
+
+ pr_info("MCDI RPC REQ:%s\n", buf);
+ }
+#endif
+ hdr[0].cdx_u32 |= (__force __le32)(cdx_mcdi_payload_csum(hdr, hdr_len, inbuf, inlen) <<
+ MCDI_HEADER_XFLAGS_LBN);
+ cdx->mcdi_ops->mcdi_request(cdx, hdr, hdr_len, inbuf, inlen);
+
+ mcdi->new_epoch = false;
+}
+
+static int cdx_mcdi_errno(struct cdx_mcdi *cdx, unsigned int mcdi_err)
+{
+ switch (mcdi_err) {
+ case 0:
+ case MC_CMD_ERR_QUEUE_FULL:
+ return mcdi_err;
+ case MC_CMD_ERR_EPERM:
+ return -EPERM;
+ case MC_CMD_ERR_ENOENT:
+ return -ENOENT;
+ case MC_CMD_ERR_EINTR:
+ return -EINTR;
+ case MC_CMD_ERR_EAGAIN:
+ return -EAGAIN;
+ case MC_CMD_ERR_EACCES:
+ return -EACCES;
+ case MC_CMD_ERR_EBUSY:
+ return -EBUSY;
+ case MC_CMD_ERR_EINVAL:
+ return -EINVAL;
+ case MC_CMD_ERR_ERANGE:
+ return -ERANGE;
+ case MC_CMD_ERR_EDEADLK:
+ return -EDEADLK;
+ case MC_CMD_ERR_ENOSYS:
+ return -EOPNOTSUPP;
+ case MC_CMD_ERR_ETIME:
+ return -ETIME;
+ case MC_CMD_ERR_EALREADY:
+ return -EALREADY;
+ case MC_CMD_ERR_ENOSPC:
+ return -ENOSPC;
+ case MC_CMD_ERR_ENOMEM:
+ return -ENOMEM;
+ case MC_CMD_ERR_ENOTSUP:
+ return -EOPNOTSUPP;
+ case MC_CMD_ERR_ALLOC_FAIL:
+ return -ENOBUFS;
+ case MC_CMD_ERR_MAC_EXIST:
+ return -EADDRINUSE;
+ case MC_CMD_ERR_NO_EVB_PORT:
+ return -EAGAIN;
+ default:
+ return -EPROTO;
+ }
+}
+
+static void cdx_mcdi_process_cleanup_list(struct cdx_mcdi *cdx,
+ struct list_head *cleanup_list)
+{
+ struct cdx_mcdi_iface *mcdi = cdx_mcdi_if(cdx);
+ unsigned int cleanups = 0;
+
+ if (!mcdi)
+ return;
+
+ while (!list_empty(cleanup_list)) {
+ struct cdx_mcdi_cmd *cmd =
+ list_first_entry(cleanup_list,
+ struct cdx_mcdi_cmd, cleanup_list);
+ cmd->completer(cdx, cmd->cookie, cmd->rc,
+ cmd->outbuf, cmd->outlen);
+ list_del(&cmd->cleanup_list);
+ kref_put(&cmd->ref, cdx_mcdi_cmd_release);
+ ++cleanups;
+ }
+
+ if (cleanups) {
+ bool all_done;
+
+ mutex_lock(&mcdi->iface_lock);
+ CDX_WARN_ON_PARANOID(cleanups > mcdi->outstanding_cleanups);
+ all_done = (mcdi->outstanding_cleanups -= cleanups) == 0;
+ mutex_unlock(&mcdi->iface_lock);
+ if (all_done)
+ wake_up(&mcdi->cmd_complete_wq);
+ }
+}
+
+static void _cdx_mcdi_cancel_cmd(struct cdx_mcdi_iface *mcdi,
+ unsigned int handle,
+ struct list_head *cleanup_list)
+{
+ struct cdx_mcdi_cmd *cmd;
+
+ list_for_each_entry(cmd, &mcdi->cmd_list, list)
+ if (cdx_mcdi_cmd_handle(cmd) == handle) {
+ switch (cmd->state) {
+ case MCDI_STATE_QUEUED:
+ case MCDI_STATE_RETRY:
+ pr_debug("command %#x inlen %zu cancelled in queue\n",
+ cmd->cmd, cmd->inlen);
+ /* if not yet running, properly cancel it */
+ cmd->rc = -EPIPE;
+ cdx_mcdi_remove_cmd(mcdi, cmd, cleanup_list);
+ break;
+ case MCDI_STATE_RUNNING:
+ case MCDI_STATE_RUNNING_CANCELLED:
+ case MCDI_STATE_FINISHED:
+ default:
+ /* invalid state? */
+ WARN_ON(1);
+ }
+ break;
+ }
+}
+
+static void cdx_mcdi_cancel_cmd(struct cdx_mcdi *cdx, struct cdx_mcdi_cmd *cmd)
+{
+ struct cdx_mcdi_iface *mcdi = cdx_mcdi_if(cdx);
+ LIST_HEAD(cleanup_list);
+
+ if (!mcdi)
+ return;
+
+ mutex_lock(&mcdi->iface_lock);
+ cdx_mcdi_timeout_cmd(mcdi, cmd, &cleanup_list);
+ mutex_unlock(&mcdi->iface_lock);
+ cdx_mcdi_process_cleanup_list(cdx, &cleanup_list);
+}
+
+struct cdx_mcdi_blocking_data {
+ struct kref ref;
+ bool done;
+ wait_queue_head_t wq;
+ int rc;
+ struct cdx_dword *outbuf;
+ size_t outlen;
+ size_t outlen_actual;
+};
+
+static void cdx_mcdi_blocking_data_release(struct kref *ref)
+{
+ kfree(container_of(ref, struct cdx_mcdi_blocking_data, ref));
+}
+
+static void cdx_mcdi_rpc_completer(struct cdx_mcdi *cdx, unsigned long cookie,
+ int rc, struct cdx_dword *outbuf,
+ size_t outlen_actual)
+{
+ struct cdx_mcdi_blocking_data *wait_data =
+ (struct cdx_mcdi_blocking_data *)cookie;
+
+ wait_data->rc = rc;
+ memcpy(wait_data->outbuf, outbuf,
+ min(outlen_actual, wait_data->outlen));
+ wait_data->outlen_actual = outlen_actual;
+ /* memory barrier */
+ smp_wmb();
+ wait_data->done = true;
+ wake_up(&wait_data->wq);
+ kref_put(&wait_data->ref, cdx_mcdi_blocking_data_release);
+}
+
+static int cdx_mcdi_rpc_sync(struct cdx_mcdi *cdx, unsigned int cmd,
+ const struct cdx_dword *inbuf, size_t inlen,
+ struct cdx_dword *outbuf, size_t outlen,
+ size_t *outlen_actual, bool quiet)
+{
+ struct cdx_mcdi_blocking_data *wait_data;
+ struct cdx_mcdi_cmd *cmd_item;
+ unsigned int handle;
+ int rc;
+
+ if (outlen_actual)
+ *outlen_actual = 0;
+
+ wait_data = kmalloc(sizeof(*wait_data), GFP_KERNEL);
+ if (!wait_data)
+ return -ENOMEM;
+
+ cmd_item = kmalloc(sizeof(*cmd_item), GFP_KERNEL);
+ if (!cmd_item) {
+ kfree(wait_data);
+ return -ENOMEM;
+ }
+
+ kref_init(&wait_data->ref);
+ wait_data->done = false;
+ init_waitqueue_head(&wait_data->wq);
+ wait_data->outbuf = outbuf;
+ wait_data->outlen = outlen;
+
+ kref_init(&cmd_item->ref);
+ cmd_item->quiet = quiet;
+ cmd_item->cookie = (unsigned long)wait_data;
+ cmd_item->completer = &cdx_mcdi_rpc_completer;
+ cmd_item->cmd = cmd;
+ cmd_item->inlen = inlen;
+ cmd_item->inbuf = inbuf;
+
+ /* Claim an extra reference for the completer to put. */
+ kref_get(&wait_data->ref);
+ rc = cdx_mcdi_rpc_async_internal(cdx, cmd_item, &handle);
+ if (rc) {
+ kref_put(&wait_data->ref, cdx_mcdi_blocking_data_release);
+ goto out;
+ }
+
+ if (!wait_event_timeout(wait_data->wq, wait_data->done,
+ cdx_mcdi_rpc_timeout(cdx, cmd)) &&
+ !wait_data->done) {
+ pr_err("MC command 0x%x inlen %zu timed out (sync)\n",
+ cmd, inlen);
+
+ cdx_mcdi_cancel_cmd(cdx, cmd_item);
+
+ wait_data->rc = -ETIMEDOUT;
+ wait_data->outlen_actual = 0;
+ }
+
+ if (outlen_actual)
+ *outlen_actual = wait_data->outlen_actual;
+ rc = wait_data->rc;
+
+out:
+ kref_put(&wait_data->ref, cdx_mcdi_blocking_data_release);
+
+ return rc;
+}
+
+static bool cdx_mcdi_get_seq(struct cdx_mcdi_iface *mcdi, unsigned char *seq)
+{
+ *seq = mcdi->prev_seq;
+ do {
+ *seq = (*seq + 1) % ARRAY_SIZE(mcdi->seq_held_by);
+ } while (mcdi->seq_held_by[*seq] && *seq != mcdi->prev_seq);
+ return !mcdi->seq_held_by[*seq];
+}
+
+static int cdx_mcdi_rpc_async_internal(struct cdx_mcdi *cdx,
+ struct cdx_mcdi_cmd *cmd,
+ unsigned int *handle)
+{
+ struct cdx_mcdi_iface *mcdi = cdx_mcdi_if(cdx);
+ LIST_HEAD(cleanup_list);
+
+ if (!mcdi) {
+ kref_put(&cmd->ref, cdx_mcdi_cmd_release);
+ return -ENETDOWN;
+ }
+
+ if (mcdi->mode == MCDI_MODE_FAIL) {
+ kref_put(&cmd->ref, cdx_mcdi_cmd_release);
+ return -ENETDOWN;
+ }
+
+ cmd->mcdi = mcdi;
+ INIT_WORK(&cmd->work, cdx_mcdi_cmd_work);
+ INIT_LIST_HEAD(&cmd->list);
+ INIT_LIST_HEAD(&cmd->cleanup_list);
+ cmd->rc = 0;
+ cmd->outbuf = NULL;
+ cmd->outlen = 0;
+
+ queue_work(mcdi->workqueue, &cmd->work);
+ return 0;
+}
+
+static void cdx_mcdi_cmd_start_or_queue_ext(struct cdx_mcdi_iface *mcdi,
+ struct cdx_mcdi_cmd *cmd)
+{
+ struct cdx_mcdi *cdx = mcdi->cdx;
+ u8 seq;
+
+ if (!mcdi->db_held_by &&
+ cdx_mcdi_get_seq(mcdi, &seq)) {
+ cmd->seq = seq;
+ cmd->reboot_seen = false;
+ cdx_mcdi_send_request(cdx, cmd);
+ cmd->state = MCDI_STATE_RUNNING;
+ } else {
+ cmd->state = MCDI_STATE_QUEUED;
+ }
+}
+
+static void cdx_mcdi_cmd_start_or_queue(struct cdx_mcdi_iface *mcdi,
+ struct cdx_mcdi_cmd *cmd)
+{
+ cdx_mcdi_cmd_start_or_queue_ext(mcdi, cmd);
+}
+
+/* try to advance other commands */
+static void cdx_mcdi_start_or_queue(struct cdx_mcdi_iface *mcdi,
+ bool allow_retry)
+{
+ struct cdx_mcdi_cmd *cmd, *tmp;
+
+ list_for_each_entry_safe(cmd, tmp, &mcdi->cmd_list, list)
+ if (cmd->state == MCDI_STATE_QUEUED ||
+ (cmd->state == MCDI_STATE_RETRY && allow_retry))
+ cdx_mcdi_cmd_start_or_queue(mcdi, cmd);
+}
+
+void cdx_mcdi_process_cmd(struct cdx_mcdi *cdx, struct cdx_dword *outbuf, int len)
+{
+ struct cdx_mcdi_iface *mcdi;
+ struct cdx_mcdi_cmd *cmd;
+ LIST_HEAD(cleanup_list);
+ unsigned int respseq;
+
+ if (!len || !outbuf) {
+ pr_err("Got empty MC response\n");
+ return;
+ }
+
+ mcdi = cdx_mcdi_if(cdx);
+ if (!mcdi)
+ return;
+
+ respseq = CDX_DWORD_FIELD(outbuf[0], MCDI_HEADER_SEQ);
+
+ mutex_lock(&mcdi->iface_lock);
+ cmd = mcdi->seq_held_by[respseq];
+
+ if (cmd) {
+ if (cmd->state == MCDI_STATE_FINISHED) {
+ mutex_unlock(&mcdi->iface_lock);
+ kref_put(&cmd->ref, cdx_mcdi_cmd_release);
+ return;
+ }
+
+ cdx_mcdi_complete_cmd(mcdi, cmd, outbuf, len, &cleanup_list);
+ } else {
+ pr_err("MC response unexpected for seq : %0X\n", respseq);
+ }
+
+ mutex_unlock(&mcdi->iface_lock);
+
+ cdx_mcdi_process_cleanup_list(mcdi->cdx, &cleanup_list);
+}
+
+static void cdx_mcdi_cmd_work(struct work_struct *context)
+{
+ struct cdx_mcdi_cmd *cmd =
+ container_of(context, struct cdx_mcdi_cmd, work);
+ struct cdx_mcdi_iface *mcdi = cmd->mcdi;
+
+ mutex_lock(&mcdi->iface_lock);
+
+ cmd->handle = mcdi->prev_handle++;
+ list_add_tail(&cmd->list, &mcdi->cmd_list);
+ cdx_mcdi_cmd_start_or_queue_ext(mcdi, cmd);
+
+ mutex_unlock(&mcdi->iface_lock);
+}
+
+/*
+ * Returns true if the MCDI module is finished with the command.
+ * (examples of false would be if the command was proxied, or it was
+ * rejected by the MC due to lack of resources and requeued).
+ */
+static bool cdx_mcdi_complete_cmd(struct cdx_mcdi_iface *mcdi,
+ struct cdx_mcdi_cmd *cmd,
+ struct cdx_dword *outbuf,
+ int len,
+ struct list_head *cleanup_list)
+{
+ size_t resp_hdr_len, resp_data_len;
+ struct cdx_mcdi *cdx = mcdi->cdx;
+ unsigned int respcmd, error;
+ bool completed = false;
+ int rc;
+
+ /* ensure the command can't go away before this function returns */
+ kref_get(&cmd->ref);
+
+ respcmd = CDX_DWORD_FIELD(outbuf[0], MCDI_HEADER_CODE);
+ error = CDX_DWORD_FIELD(outbuf[0], MCDI_HEADER_ERROR);
+
+ if (respcmd != MC_CMD_V2_EXTN) {
+ resp_hdr_len = 4;
+ resp_data_len = CDX_DWORD_FIELD(outbuf[0], MCDI_HEADER_DATALEN);
+ } else {
+ resp_data_len = 0;
+ resp_hdr_len = 8;
+ if (len >= 8)
+ resp_data_len =
+ CDX_DWORD_FIELD(outbuf[1], MC_CMD_V2_EXTN_IN_ACTUAL_LEN);
+ }
+
+ if ((resp_hdr_len + resp_data_len) > len) {
+ pr_warn("Incomplete MCDI response received %d. Expected %zu\n",
+ len, (resp_hdr_len + resp_data_len));
+ resp_data_len = 0;
+ }
+
+#ifdef CONFIG_MCDI_LOGGING
+ if (!WARN_ON_ONCE(!mcdi->logging_buffer)) {
+ char *log = mcdi->logging_buffer;
+ int i, bytes = 0;
+ size_t rlen;
+
+ WARN_ON_ONCE(resp_hdr_len % 4);
+
+ rlen = resp_hdr_len / 4 + DIV_ROUND_UP(resp_data_len, 4);
+
+ for (i = 0; i < rlen; i++) {
+ if ((bytes + 75) > LOG_LINE_MAX) {
+ pr_info("MCDI RPC RESP:%s \\\n", log);
+ bytes = 0;
+ }
+ bytes += snprintf(log + bytes, LOG_LINE_MAX - bytes,
+ " %08x", le32_to_cpu(outbuf[i].cdx_u32));
+ }
+
+ pr_info("MCDI RPC RESP:%s\n", log);
+ }
+#endif
+
+ if (error && resp_data_len == 0) {
+ /* MC rebooted during command */
+ rc = -EIO;
+ } else {
+ if (WARN_ON_ONCE(error && resp_data_len < 4))
+ resp_data_len = 4;
+ if (error) {
+ rc = CDX_DWORD_FIELD(outbuf[resp_hdr_len / 4], CDX_DWORD);
+ if (!cmd->quiet) {
+ int err_arg = 0;
+
+ if (resp_data_len >= MC_CMD_ERR_ARG_OFST + 4) {
+ int offset = (resp_hdr_len + MC_CMD_ERR_ARG_OFST) / 4;
+
+ err_arg = CDX_DWORD_VAL(outbuf[offset]);
+ }
+
+ _cdx_mcdi_display_error(cdx, cmd->cmd,
+ cmd->inlen, rc, err_arg,
+ cdx_mcdi_errno(cdx, rc));
+ }
+ rc = cdx_mcdi_errno(cdx, rc);
+ } else {
+ rc = 0;
+ }
+ }
+
+ /* free doorbell */
+ if (mcdi->db_held_by == cmd)
+ mcdi->db_held_by = NULL;
+
+ if (cdx_cmd_cancelled(cmd)) {
+ list_del(&cmd->list);
+ kref_put(&cmd->ref, cdx_mcdi_cmd_release);
+ completed = true;
+ } else if (rc == MC_CMD_ERR_QUEUE_FULL) {
+ cmd->state = MCDI_STATE_RETRY;
+ } else {
+ cmd->rc = rc;
+ cmd->outbuf = outbuf + DIV_ROUND_UP(resp_hdr_len, 4);
+ cmd->outlen = resp_data_len;
+ cdx_mcdi_remove_cmd(mcdi, cmd, cleanup_list);
+ completed = true;
+ }
+
+ /* free sequence number and buffer */
+ mcdi->seq_held_by[cmd->seq] = NULL;
+
+ cdx_mcdi_start_or_queue(mcdi, rc != MC_CMD_ERR_QUEUE_FULL);
+
+ /* wake up anyone waiting for flush */
+ wake_up(&mcdi->cmd_complete_wq);
+
+ kref_put(&cmd->ref, cdx_mcdi_cmd_release);
+
+ return completed;
+}
+
+static void cdx_mcdi_timeout_cmd(struct cdx_mcdi_iface *mcdi,
+ struct cdx_mcdi_cmd *cmd,
+ struct list_head *cleanup_list)
+{
+ struct cdx_mcdi *cdx = mcdi->cdx;
+
+ pr_err("MC command 0x%x inlen %zu state %d timed out after %u ms\n",
+ cmd->cmd, cmd->inlen, cmd->state,
+ jiffies_to_msecs(jiffies - cmd->started));
+
+ cmd->rc = -ETIMEDOUT;
+ cdx_mcdi_remove_cmd(mcdi, cmd, cleanup_list);
+
+ cdx_mcdi_mode_fail(cdx, cleanup_list);
+}
+
+/**
+ * cdx_mcdi_rpc - Issue an MCDI command and wait for completion
+ * @cdx: NIC through which to issue the command
+ * @cmd: Command type number
+ * @inbuf: Command parameters
+ * @inlen: Length of command parameters, in bytes. Must be a multiple
+ * of 4 and no greater than %MCDI_CTL_SDU_LEN_MAX_V1.
+ * @outbuf: Response buffer. May be %NULL if @outlen is 0.
+ * @outlen: Length of response buffer, in bytes. If the actual
+ * response is longer than @outlen & ~3, it will be truncated
+ * to that length.
+ * @outlen_actual: Pointer through which to return the actual response
+ * length. May be %NULL if this is not needed.
+ *
+ * This function may sleep and therefore must be called in process
+ * context.
+ *
+ * Return: A negative error code, or zero if successful. The error
+ * code may come from the MCDI response or may indicate a failure
+ * to communicate with the MC. In the former case, the response
+ * will still be copied to @outbuf and *@outlen_actual will be
+ * set accordingly. In the latter case, *@outlen_actual will be
+ * set to zero.
+ */
+int cdx_mcdi_rpc(struct cdx_mcdi *cdx, unsigned int cmd,
+ const struct cdx_dword *inbuf, size_t inlen,
+ struct cdx_dword *outbuf, size_t outlen,
+ size_t *outlen_actual)
+{
+ return cdx_mcdi_rpc_sync(cdx, cmd, inbuf, inlen, outbuf, outlen,
+ outlen_actual, false);
+}
+
+/**
+ * cdx_mcdi_rpc_async - Schedule an MCDI command to run asynchronously
+ * @cdx: NIC through which to issue the command
+ * @cmd: Command type number
+ * @inbuf: Command parameters
+ * @inlen: Length of command parameters, in bytes
+ * @outlen: Length to allocate for response buffer, in bytes
+ * @complete: Function to be called on completion or cancellation.
+ * @cookie: Arbitrary value to be passed to @complete.
+ *
+ * This function does not sleep and therefore may be called in atomic
+ * context. It will fail if event queues are disabled or if MCDI
+ * event completions have been disabled due to an error.
+ *
+ * If it succeeds, the @complete function will be called exactly once
+ * in process context, when one of the following occurs:
+ * (a) the completion event is received (in process context)
+ * (b) event queues are disabled (in the process that disables them)
+ */
+int
+cdx_mcdi_rpc_async(struct cdx_mcdi *cdx, unsigned int cmd,
+ const struct cdx_dword *inbuf, size_t inlen,
+ cdx_mcdi_async_completer *complete, unsigned long cookie)
+{
+ struct cdx_mcdi_cmd *cmd_item =
+ kmalloc(sizeof(struct cdx_mcdi_cmd) + inlen, GFP_ATOMIC);
+
+ if (!cmd_item)
+ return -ENOMEM;
+
+ kref_init(&cmd_item->ref);
+ cmd_item->quiet = true;
+ cmd_item->cookie = cookie;
+ cmd_item->completer = complete;
+ cmd_item->cmd = cmd;
+ cmd_item->inlen = inlen;
+ /* inbuf is probably not valid after return, so take a copy */
+ cmd_item->inbuf = (struct cdx_dword *)(cmd_item + 1);
+ memcpy(cmd_item + 1, inbuf, inlen);
+
+ return cdx_mcdi_rpc_async_internal(cdx, cmd_item, NULL);
+}
+
+static void _cdx_mcdi_display_error(struct cdx_mcdi *cdx, unsigned int cmd,
+ size_t inlen, int raw, int arg, int rc)
+{
+ pr_err("MC command 0x%x inlen %d failed rc=%d (raw=%d) arg=%d\n",
+ cmd, (int)inlen, rc, raw, arg);
+}
+
+/*
+ * Set MCDI mode to fail to prevent any new commands, then cancel any
+ * outstanding commands.
+ * Caller must hold the mcdi iface_lock.
+ */
+static void cdx_mcdi_mode_fail(struct cdx_mcdi *cdx, struct list_head *cleanup_list)
+{
+ struct cdx_mcdi_iface *mcdi = cdx_mcdi_if(cdx);
+
+ if (!mcdi)
+ return;
+
+ mcdi->mode = MCDI_MODE_FAIL;
+
+ while (!list_empty(&mcdi->cmd_list)) {
+ struct cdx_mcdi_cmd *cmd;
+
+ cmd = list_first_entry(&mcdi->cmd_list, struct cdx_mcdi_cmd,
+ list);
+ _cdx_mcdi_cancel_cmd(mcdi, cdx_mcdi_cmd_handle(cmd), cleanup_list);
+ }
+}
diff --git a/drivers/cdx/controller/mcdi.h b/drivers/cdx/controller/mcdi.h
new file mode 100644
index 000000000000..63933ede33ed
--- /dev/null
+++ b/drivers/cdx/controller/mcdi.h
@@ -0,0 +1,238 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ * Copyright 2008-2013 Solarflare Communications Inc.
+ * Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+ */
+
+#ifndef CDX_MCDI_H
+#define CDX_MCDI_H
+
+#include <linux/mutex.h>
+#include <linux/kref.h>
+
+#include "bitfield.h"
+#include "mc_cdx_pcol.h"
+
+#ifdef DEBUG
+#define CDX_WARN_ON_ONCE_PARANOID(x) WARN_ON_ONCE(x)
+#define CDX_WARN_ON_PARANOID(x) WARN_ON(x)
+#else
+#define CDX_WARN_ON_ONCE_PARANOID(x) do {} while (0)
+#define CDX_WARN_ON_PARANOID(x) do {} while (0)
+#endif
+
+/**
+ * enum cdx_mcdi_mode - MCDI transaction mode
+ * @MCDI_MODE_EVENTS: wait for an mcdi response callback.
+ * @MCDI_MODE_FAIL: we think MCDI is dead, so fail-fast all calls
+ */
+enum cdx_mcdi_mode {
+ MCDI_MODE_EVENTS,
+ MCDI_MODE_FAIL,
+};
+
+#define MCDI_RPC_TIMEOUT (10 * HZ)
+#define MCDI_RPC_LONG_TIMEOU (60 * HZ)
+#define MCDI_RPC_POST_RST_TIME (10 * HZ)
+
+#define MCDI_BUF_LEN (8 + MCDI_CTL_SDU_LEN_MAX)
+
+/**
+ * enum cdx_mcdi_cmd_state - State for an individual MCDI command
+ * @MCDI_STATE_QUEUED: Command not started and is waiting to run.
+ * @MCDI_STATE_RETRY: Command was submitted and MC rejected with no resources,
+ * as MC have too many outstanding commands. Command will be retried once
+ * another command returns.
+ * @MCDI_STATE_RUNNING: Command was accepted and is running.
+ * @MCDI_STATE_RUNNING_CANCELLED: Command is running but the issuer cancelled
+ * the command.
+ * @MCDI_STATE_FINISHED: Processing of this command has completed.
+ */
+
+enum cdx_mcdi_cmd_state {
+ MCDI_STATE_QUEUED,
+ MCDI_STATE_RETRY,
+ MCDI_STATE_RUNNING,
+ MCDI_STATE_RUNNING_CANCELLED,
+ MCDI_STATE_FINISHED,
+};
+
+/**
+ * struct cdx_mcdi - CDX MCDI Firmware interface, to interact
+ * with CDX controller.
+ * @mcdi: MCDI interface
+ * @mcdi_ops: MCDI operations
+ */
+struct cdx_mcdi {
+ /* MCDI interface */
+ struct cdx_mcdi_data *mcdi;
+ const struct cdx_mcdi_ops *mcdi_ops;
+};
+
+struct cdx_mcdi_ops {
+ void (*mcdi_request)(struct cdx_mcdi *cdx,
+ const struct cdx_dword *hdr, size_t hdr_len,
+ const struct cdx_dword *sdu, size_t sdu_len);
+ unsigned int (*mcdi_rpc_timeout)(struct cdx_mcdi *cdx, unsigned int cmd);
+};
+
+typedef void cdx_mcdi_async_completer(struct cdx_mcdi *cdx,
+ unsigned long cookie, int rc,
+ struct cdx_dword *outbuf,
+ size_t outlen_actual);
+
+/**
+ * struct cdx_mcdi_cmd - An outstanding MCDI command
+ * @ref: Reference count. There will be one reference if the command is
+ * in the mcdi_iface cmd_list, another if it's on a cleanup list,
+ * and a third if it's queued in the work queue.
+ * @list: The data for this entry in mcdi->cmd_list
+ * @cleanup_list: The data for this entry in a cleanup list
+ * @work: The work item for this command, queued in mcdi->workqueue
+ * @mcdi: The mcdi_iface for this command
+ * @state: The state of this command
+ * @inlen: inbuf length
+ * @inbuf: Input buffer
+ * @quiet: Whether to silence errors
+ * @reboot_seen: Whether a reboot has been seen during this command,
+ * to prevent duplicates
+ * @seq: Sequence number
+ * @started: Jiffies this command was started at
+ * @cookie: Context for completion function
+ * @completer: Completion function
+ * @handle: Command handle
+ * @cmd: Command number
+ * @rc: Return code
+ * @outlen: Length of output buffer
+ * @outbuf: Output buffer
+ */
+struct cdx_mcdi_cmd {
+ struct kref ref;
+ struct list_head list;
+ struct list_head cleanup_list;
+ struct work_struct work;
+ struct cdx_mcdi_iface *mcdi;
+ enum cdx_mcdi_cmd_state state;
+ size_t inlen;
+ const struct cdx_dword *inbuf;
+ bool quiet;
+ bool reboot_seen;
+ u8 seq;
+ unsigned long started;
+ unsigned long cookie;
+ cdx_mcdi_async_completer *completer;
+ unsigned int handle;
+ unsigned int cmd;
+ int rc;
+ size_t outlen;
+ struct cdx_dword *outbuf;
+ /* followed by inbuf data if necessary */
+};
+
+/**
+ * struct cdx_mcdi_iface - MCDI protocol context
+ * @cdx: The associated NIC
+ * @iface_lock: Serialise access to this structure
+ * @outstanding_cleanups: Count of cleanups
+ * @cmd_list: List of outstanding and running commands
+ * @workqueue: Workqueue used for delayed processing
+ * @cmd_complete_wq: Waitqueue for command completion
+ * @db_held_by: Command the MC doorbell is in use by
+ * @seq_held_by: Command each sequence number is in use by
+ * @prev_handle: The last used command handle
+ * @mode: Poll for mcdi completion, or wait for an mcdi_event
+ * @prev_seq: The last used sequence number
+ * @new_epoch: Indicates start of day or start of MC reboot recovery
+ * @logging_buffer: Buffer that may be used to build MCDI tracing messages
+ * @logging_enabled: Whether to trace MCDI
+ */
+struct cdx_mcdi_iface {
+ struct cdx_mcdi *cdx;
+ /* Serialise access */
+ struct mutex iface_lock;
+ unsigned int outstanding_cleanups;
+ struct list_head cmd_list;
+ struct workqueue_struct *workqueue;
+ wait_queue_head_t cmd_complete_wq;
+ struct cdx_mcdi_cmd *db_held_by;
+ struct cdx_mcdi_cmd *seq_held_by[16];
+ unsigned int prev_handle;
+ enum cdx_mcdi_mode mode;
+ u8 prev_seq;
+ bool new_epoch;
+#ifdef CONFIG_MCDI_LOGGING
+ bool logging_enabled;
+ char *logging_buffer;
+#endif
+};
+
+/**
+ * struct cdx_mcdi_data - extra state for NICs that implement MCDI
+ * @iface: Interface/protocol state
+ * @fn_flags: Flags for this function, as returned by %MC_CMD_DRV_ATTACH.
+ */
+struct cdx_mcdi_data {
+ struct cdx_mcdi_iface iface;
+ u32 fn_flags;
+};
+
+static inline struct cdx_mcdi_iface *cdx_mcdi_if(struct cdx_mcdi *cdx)
+{
+ return cdx->mcdi ? &cdx->mcdi->iface : NULL;
+}
+
+int cdx_mcdi_init(struct cdx_mcdi *cdx);
+void cdx_mcdi_finish(struct cdx_mcdi *cdx);
+
+void cdx_mcdi_process_cmd(struct cdx_mcdi *cdx, struct cdx_dword *outbuf, int len);
+int cdx_mcdi_rpc(struct cdx_mcdi *cdx, unsigned int cmd,
+ const struct cdx_dword *inbuf, size_t inlen,
+ struct cdx_dword *outbuf, size_t outlen, size_t *outlen_actual);
+int cdx_mcdi_rpc_async(struct cdx_mcdi *cdx, unsigned int cmd,
+ const struct cdx_dword *inbuf, size_t inlen,
+ cdx_mcdi_async_completer *complete,
+ unsigned long cookie);
+int cdx_mcdi_wait_for_quiescence(struct cdx_mcdi *cdx,
+ unsigned int timeout_jiffies);
+
+/*
+ * We expect that 16- and 32-bit fields in MCDI requests and responses
+ * are appropriately aligned, but 64-bit fields are only
+ * 32-bit-aligned.
+ */
+#define MCDI_DECLARE_BUF(_name, _len) struct cdx_dword _name[DIV_ROUND_UP(_len, 4)] = {{0}}
+#define _MCDI_PTR(_buf, _offset) \
+ ((u8 *)(_buf) + (_offset))
+#define MCDI_PTR(_buf, _field) \
+ _MCDI_PTR(_buf, MC_CMD_ ## _field ## _OFST)
+#define _MCDI_CHECK_ALIGN(_ofst, _align) \
+ ((void)BUILD_BUG_ON_ZERO((_ofst) & ((_align) - 1)), \
+ (_ofst))
+#define _MCDI_DWORD(_buf, _field) \
+ ((_buf) + (_MCDI_CHECK_ALIGN(MC_CMD_ ## _field ## _OFST, 4) >> 2))
+
+#define MCDI_BYTE(_buf, _field) \
+ ((void)BUILD_BUG_ON_ZERO(MC_CMD_ ## _field ## _LEN != 1), \
+ *MCDI_PTR(_buf, _field))
+#define MCDI_WORD(_buf, _field) \
+ ((void)BUILD_BUG_ON_ZERO(MC_CMD_ ## _field ## _LEN != 2), \
+ le16_to_cpu(*(__force const __le16 *)MCDI_PTR(_buf, _field)))
+#define MCDI_SET_DWORD(_buf, _field, _value) \
+ CDX_POPULATE_DWORD_1(*_MCDI_DWORD(_buf, _field), CDX_DWORD, _value)
+#define MCDI_DWORD(_buf, _field) \
+ CDX_DWORD_FIELD(*_MCDI_DWORD(_buf, _field), CDX_DWORD)
+#define MCDI_POPULATE_DWORD_1(_buf, _field, _name1, _value1) \
+ CDX_POPULATE_DWORD_1(*_MCDI_DWORD(_buf, _field), \
+ MC_CMD_ ## _name1, _value1)
+#define MCDI_SET_QWORD(_buf, _field, _value) \
+ do { \
+ CDX_POPULATE_DWORD_1(_MCDI_DWORD(_buf, _field)[0], \
+ CDX_DWORD, (u32)(_value)); \
+ CDX_POPULATE_DWORD_1(_MCDI_DWORD(_buf, _field)[1], \
+ CDX_DWORD, (u64)(_value) >> 32); \
+ } while (0)
+#define MCDI_QWORD(_buf, _field) \
+ (CDX_DWORD_FIELD(_MCDI_DWORD(_buf, _field)[0], CDX_DWORD) | \
+ (u64)CDX_DWORD_FIELD(_MCDI_DWORD(_buf, _field)[1], CDX_DWORD) << 32)
+
+#endif /* CDX_MCDI_H */
--
2.17.1
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [PATCH v6 4/7] cdx: add MCDI protocol interface for firmware interaction
2023-01-26 10:46 ` [PATCH v6 4/7] cdx: add MCDI protocol interface for firmware interaction Nipun Gupta
@ 2023-01-28 13:15 ` kernel test robot
0 siblings, 0 replies; 14+ messages in thread
From: kernel test robot @ 2023-01-28 13:15 UTC (permalink / raw)
To: Nipun Gupta, robh+dt, krzysztof.kozlowski+dt, gregkh, rafael,
eric.auger, alex.williamson, cohuck, song.bao.hua,
mchehab+huawei, maz, f.fainelli, jeffrey.l.hugo, saravanak,
Michael.Srba, mani, yishaih, jgg, jgg, robin.murphy, will, joro,
masahiroy, ndesaulniers, linux-arm-kernel, linux-kbuild,
linux-kernel, devicetree
Cc: oe-kbuild-all, okaya, harpreet.anand, nikhil.agarwal
Hi Nipun,
I love your patch! Perhaps something to improve:
[auto build test WARNING on next-20230127]
[cannot apply to masahiroy-kbuild/for-next masahiroy-kbuild/fixes robh/for-next joro-iommu/next linus/master v6.2-rc5 v6.2-rc4 v6.2-rc3 v6.2-rc5]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Nipun-Gupta/cdx-add-the-cdx-bus-driver/20230128-161622
patch link: https://lore.kernel.org/r/20230126104630.15493-5-nipun.gupta%40amd.com
patch subject: [PATCH v6 4/7] cdx: add MCDI protocol interface for firmware interaction
reproduce:
make versioncheck
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
versioncheck warnings: (new ones prefixed by >>)
INFO PATH=/opt/cross/clang/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
/usr/bin/timeout -k 100 3h /usr/bin/make W=1 --keep-going HOSTCC=gcc-11 CC=gcc-11 -j32 ARCH=x86_64 versioncheck
find ./* \( -name SCCS -o -name BitKeeper -o -name .svn -o -name CVS -o -name .pc -o -name .hg -o -name .git \) -prune -o \
-name '*.[hcS]' -type f -print | sort \
| xargs perl -w ./scripts/checkversion.pl
./drivers/accessibility/speakup/genmap.c: 13 linux/version.h not needed.
./drivers/accessibility/speakup/makemapdata.c: 13 linux/version.h not needed.
>> ./drivers/cdx/controller/mcdi.c: 20 linux/version.h not needed.
./drivers/net/ethernet/qlogic/qede/qede.h: 10 linux/version.h not needed.
./drivers/net/ethernet/qlogic/qede/qede_ethtool.c: 7 linux/version.h not needed.
./drivers/scsi/cxgbi/libcxgbi.h: 27 linux/version.h not needed.
./drivers/scsi/mpi3mr/mpi3mr.h: 32 linux/version.h not needed.
./drivers/scsi/qedi/qedi_dbg.h: 14 linux/version.h not needed.
./drivers/soc/tegra/cbb/tegra-cbb.c: 19 linux/version.h not needed.
./drivers/soc/tegra/cbb/tegra194-cbb.c: 26 linux/version.h not needed.
./drivers/soc/tegra/cbb/tegra234-cbb.c: 27 linux/version.h not needed.
./drivers/staging/media/atomisp/include/linux/atomisp.h: 23 linux/version.h not needed.
./samples/trace_events/trace_custom_sched.c: 11 linux/version.h not needed.
./sound/soc/codecs/cs42l42.c: 14 linux/version.h not needed.
./tools/lib/bpf/bpf_helpers.h: 289: need linux/version.h
./tools/perf/tests/bpf-script-example.c: 60: need linux/version.h
./tools/perf/tests/bpf-script-test-kbuild.c: 21: need linux/version.h
./tools/perf/tests/bpf-script-test-prologue.c: 49: need linux/version.h
./tools/perf/tests/bpf-script-test-relocation.c: 51: need linux/version.h
./tools/testing/selftests/bpf/progs/dev_cgroup.c: 9 linux/version.h not needed.
./tools/testing/selftests/bpf/progs/netcnt_prog.c: 3 linux/version.h not needed.
./tools/testing/selftests/bpf/progs/test_map_lock.c: 4 linux/version.h not needed.
./tools/testing/selftests/bpf/progs/test_send_signal_kern.c: 4 linux/version.h not needed.
./tools/testing/selftests/bpf/progs/test_spin_lock.c: 4 linux/version.h not needed.
./tools/testing/selftests/bpf/progs/test_tcp_estats.c: 37 linux/version.h not needed.
./tools/testing/selftests/wireguard/qemu/init.c: 27 linux/version.h not needed.
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH v6 5/7] cdx: add cdx controller
2023-01-26 10:46 [PATCH v6 0/7] add support for CDX bus Nipun Gupta
` (3 preceding siblings ...)
2023-01-26 10:46 ` [PATCH v6 4/7] cdx: add MCDI protocol interface for firmware interaction Nipun Gupta
@ 2023-01-26 10:46 ` Nipun Gupta
2023-01-28 12:44 ` kernel test robot
2023-01-28 18:26 ` kernel test robot
2023-01-26 10:46 ` [PATCH v6 6/7] cdx: add rpmsg communication channel for CDX Nipun Gupta
2023-01-26 10:46 ` [PATCH v6 7/7] cdx: add device attributes Nipun Gupta
6 siblings, 2 replies; 14+ messages in thread
From: Nipun Gupta @ 2023-01-26 10:46 UTC (permalink / raw)
To: robh+dt, krzysztof.kozlowski+dt, gregkh, rafael, eric.auger,
alex.williamson, cohuck, song.bao.hua, mchehab+huawei, maz,
f.fainelli, jeffrey.l.hugo, saravanak, Michael.Srba, mani,
yishaih, jgg, jgg, robin.murphy, will, joro, masahiroy,
ndesaulniers, linux-arm-kernel, linux-kbuild, linux-kernel,
devicetree
Cc: okaya, harpreet.anand, nikhil.agarwal, michal.simek, git, Nipun Gupta
CDX controller uses MCDI interface as a protocol to
communicate with the RPU firmware and registers the
detected CDX devices on the CDX bus. It also uses
RPMsg as the communication channel with the Firmware.
Signed-off-by: Nipun Gupta <nipun.gupta@amd.com>
Signed-off-by: Puneet Gupta <puneet.gupta@amd.com>
Signed-off-by: Abhijit Gangurde <abhijit.gangurde@amd.com>
Signed-off-by: Nikhil Agarwal <nikhil.agarwal@amd.com>
---
drivers/cdx/controller/Kconfig | 9 ++
drivers/cdx/controller/Makefile | 2 +-
drivers/cdx/controller/cdx_controller.c | 187 ++++++++++++++++++++++++
drivers/cdx/controller/mcdi_functions.c | 125 ++++++++++++++++
drivers/cdx/controller/mcdi_functions.h | 50 +++++++
5 files changed, 372 insertions(+), 1 deletion(-)
create mode 100644 drivers/cdx/controller/cdx_controller.c
create mode 100644 drivers/cdx/controller/mcdi_functions.c
create mode 100644 drivers/cdx/controller/mcdi_functions.h
diff --git a/drivers/cdx/controller/Kconfig b/drivers/cdx/controller/Kconfig
index 785c71063b2a..17f9c6be2fe1 100644
--- a/drivers/cdx/controller/Kconfig
+++ b/drivers/cdx/controller/Kconfig
@@ -7,6 +7,15 @@
if CDX_BUS
+config CDX_CONTROLLER
+ tristate "CDX bus controller"
+ help
+ CDX controller drives the CDX bus. It interacts with
+ firmware to get the hardware devices and registers with
+ the CDX bus. Say Y to enable the CDX hardware driver.
+
+ If unsure, say N.
+
config MCDI_LOGGING
bool "MCDI Logging for the CDX controller"
depends on CDX_CONTROLLER
diff --git a/drivers/cdx/controller/Makefile b/drivers/cdx/controller/Makefile
index 0ce200678eda..f7437c882cc9 100644
--- a/drivers/cdx/controller/Makefile
+++ b/drivers/cdx/controller/Makefile
@@ -6,4 +6,4 @@
#
obj-$(CONFIG_CDX_CONTROLLER) += cdx-controller.o
-cdx-controller-objs := mcdi.o
+cdx-controller-objs := cdx_controller.o mcdi.o mcdi_functions.o
diff --git a/drivers/cdx/controller/cdx_controller.c b/drivers/cdx/controller/cdx_controller.c
new file mode 100644
index 000000000000..ec578422f87e
--- /dev/null
+++ b/drivers/cdx/controller/cdx_controller.c
@@ -0,0 +1,187 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * CDX host controller driver for AMD versal-net platform.
+ *
+ * Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+ */
+
+#include <linux/of_platform.h>
+#include <linux/cdx/cdx_bus.h>
+
+#include "../cdx.h"
+#include "mcdi_functions.h"
+#include "mcdi.h"
+
+static unsigned int cdx_mcdi_rpc_timeout(struct cdx_mcdi *cdx, unsigned int cmd)
+{
+ return MCDI_RPC_TIMEOUT;
+}
+
+static void cdx_mcdi_request(struct cdx_mcdi *cdx,
+ const struct cdx_dword *hdr, size_t hdr_len,
+ const struct cdx_dword *sdu, size_t sdu_len)
+{
+ /*
+ * This will get updated by rpmsg APIs, with RPMSG introduction
+ * in CDX controller as a transport layer.
+ */
+}
+
+static const struct cdx_mcdi_ops mcdi_ops = {
+ .mcdi_rpc_timeout = cdx_mcdi_rpc_timeout,
+ .mcdi_request = cdx_mcdi_request,
+};
+
+static int cdx_scan_devices(struct cdx_controller *cdx)
+{
+ struct cdx_mcdi *cdx_mcdi = cdx->priv;
+ u8 bus_num, dev_num, num_cdx_bus;
+ int ret;
+
+ /* MCDI FW Read: Fetch the number of CDX buses on this controller */
+ ret = cdx_mcdi_get_num_buses(cdx_mcdi);
+ if (ret < 0) {
+ dev_err(cdx->dev,
+ "Get number of CDX buses failed: %d\n", ret);
+ return ret;
+ }
+ num_cdx_bus = (u8)ret;
+
+ for (bus_num = 0; bus_num < num_cdx_bus; bus_num++) {
+ u8 num_cdx_dev;
+
+ /* MCDI FW Read: Fetch the number of devices present */
+ ret = cdx_mcdi_get_num_devs(cdx_mcdi, bus_num);
+ if (ret < 0) {
+ dev_err(cdx->dev,
+ "CDX bus %d has no devices: %d\n", bus_num, ret);
+ continue;
+ }
+ num_cdx_dev = (u8)ret;
+
+ for (dev_num = 0; dev_num < num_cdx_dev; dev_num++) {
+ struct cdx_dev_params dev_params;
+
+ /* MCDI FW: Get the device config */
+ ret = cdx_mcdi_get_dev_config(cdx_mcdi, bus_num,
+ dev_num, &dev_params);
+ if (ret) {
+ dev_err(cdx->dev,
+ "CDX device config get failed for %d(bus):%d(dev), %d\n",
+ bus_num, dev_num, ret);
+ continue;
+ }
+ dev_params.cdx = cdx;
+
+ /* Add the device to the cdx bus */
+ ret = cdx_device_add(&dev_params);
+ if (ret) {
+ dev_err(cdx->dev, "registering cdx dev: %d failed: %d\n",
+ dev_num, ret);
+ continue;
+ }
+
+ dev_dbg(cdx->dev, "CDX dev: %d on cdx bus: %d created\n",
+ dev_num, bus_num);
+ }
+ }
+
+ return 0;
+}
+
+static struct cdx_ops cdx_ops = {
+ .scan = cdx_scan_devices,
+};
+
+static int xlnx_cdx_probe(struct platform_device *pdev)
+{
+ struct cdx_controller *cdx;
+ struct cdx_mcdi *cdx_mcdi;
+ int ret;
+
+ cdx_mcdi = kzalloc(sizeof(*cdx_mcdi), GFP_KERNEL);
+ if (!cdx_mcdi)
+ return -ENOMEM;
+
+ /* Store the MCDI ops */
+ cdx_mcdi->mcdi_ops = &mcdi_ops;
+ /* MCDI FW: Initialize the FW path */
+ ret = cdx_mcdi_init(cdx_mcdi);
+ if (ret) {
+ dev_err_probe(&pdev->dev, ret, "MCDI Initialization failed\n");
+ goto mcdi_init_fail;
+ }
+
+ cdx = kzalloc(sizeof(*cdx), GFP_KERNEL);
+ if (!cdx) {
+ ret = -ENOMEM;
+ goto cdx_alloc_fail;
+ }
+ platform_set_drvdata(pdev, cdx);
+
+ cdx->dev = &pdev->dev;
+ cdx->priv = cdx_mcdi;
+ cdx->ops = &cdx_ops;
+
+ return 0;
+
+cdx_alloc_fail:
+ cdx_mcdi_finish(cdx_mcdi);
+mcdi_init_fail:
+ kfree(cdx_mcdi);
+
+ return ret;
+}
+
+static int xlnx_cdx_remove(struct platform_device *pdev)
+{
+ struct cdx_controller *cdx = platform_get_drvdata(pdev);
+ struct cdx_mcdi *cdx_mcdi = cdx->priv;
+
+ kfree(cdx);
+
+ cdx_mcdi_finish(cdx_mcdi);
+ kfree(cdx_mcdi);
+
+ return 0;
+}
+
+static const struct of_device_id cdx_match_table[] = {
+ {.compatible = "xlnx,versal-net-cdx",},
+ { },
+};
+
+MODULE_DEVICE_TABLE(of, cdx_match_table);
+
+static struct platform_driver cdx_pdriver = {
+ .driver = {
+ .name = "cdx-controller",
+ .pm = NULL,
+ .of_match_table = cdx_match_table,
+ },
+ .probe = xlnx_cdx_probe,
+ .remove = xlnx_cdx_remove,
+};
+
+static int __init cdx_controller_init(void)
+{
+ int ret;
+
+ ret = platform_driver_register(&cdx_pdriver);
+ if (ret < 0)
+ pr_err("platform_driver_register() failed: %d\n", ret);
+
+ return ret;
+}
+
+static void __exit cdx_controller_exit(void)
+{
+ platform_driver_unregister(&cdx_pdriver);
+}
+
+module_init(cdx_controller_init);
+module_exit(cdx_controller_exit);
+
+MODULE_AUTHOR("AMD Inc.");
+MODULE_DESCRIPTION("CDX controller for AMD devices");
+MODULE_LICENSE("GPL");
diff --git a/drivers/cdx/controller/mcdi_functions.c b/drivers/cdx/controller/mcdi_functions.c
new file mode 100644
index 000000000000..3940a2c7919c
--- /dev/null
+++ b/drivers/cdx/controller/mcdi_functions.c
@@ -0,0 +1,125 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+ */
+
+#include <linux/module.h>
+
+#include "mcdi.h"
+#include "mcdi_functions.h"
+
+int cdx_mcdi_get_num_buses(struct cdx_mcdi *cdx)
+{
+ MCDI_DECLARE_BUF(outbuf, MC_CMD_CDX_BUS_ENUM_BUSES_OUT_LEN);
+ size_t outlen;
+ int rc;
+
+ rc = cdx_mcdi_rpc(cdx, MC_CMD_CDX_BUS_ENUM_BUSES, NULL, 0,
+ outbuf, sizeof(outbuf), &outlen);
+ if (rc)
+ return rc;
+
+ if (outlen != MC_CMD_CDX_BUS_ENUM_BUSES_OUT_LEN)
+ return -EIO;
+
+ return MCDI_DWORD(outbuf, CDX_BUS_ENUM_BUSES_OUT_BUS_COUNT);
+}
+
+int cdx_mcdi_get_num_devs(struct cdx_mcdi *cdx, int bus_num)
+{
+ MCDI_DECLARE_BUF(outbuf, MC_CMD_CDX_BUS_ENUM_DEVICES_OUT_LEN);
+ MCDI_DECLARE_BUF(inbuf, MC_CMD_CDX_BUS_ENUM_DEVICES_IN_LEN);
+ size_t outlen;
+ int rc;
+
+ MCDI_SET_DWORD(inbuf, CDX_BUS_ENUM_DEVICES_IN_BUS, bus_num);
+
+ rc = cdx_mcdi_rpc(cdx, MC_CMD_CDX_BUS_ENUM_DEVICES, inbuf, sizeof(inbuf),
+ outbuf, sizeof(outbuf), &outlen);
+ if (rc)
+ return rc;
+
+ if (outlen != MC_CMD_CDX_BUS_ENUM_DEVICES_OUT_LEN)
+ return -EIO;
+
+ return MCDI_DWORD(outbuf, CDX_BUS_ENUM_DEVICES_OUT_DEVICE_COUNT);
+}
+
+int cdx_mcdi_get_dev_config(struct cdx_mcdi *cdx,
+ u8 bus_num, u8 dev_num,
+ struct cdx_dev_params *dev_params)
+{
+ MCDI_DECLARE_BUF(outbuf, MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_LEN);
+ MCDI_DECLARE_BUF(inbuf, MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_IN_LEN);
+ struct resource *res = &dev_params->res[0];
+ size_t outlen;
+ u32 req_id;
+ int rc;
+
+ MCDI_SET_DWORD(inbuf, CDX_BUS_GET_DEVICE_CONFIG_IN_BUS, bus_num);
+ MCDI_SET_DWORD(inbuf, CDX_BUS_GET_DEVICE_CONFIG_IN_DEVICE, dev_num);
+
+ rc = cdx_mcdi_rpc(cdx, MC_CMD_CDX_BUS_GET_DEVICE_CONFIG, inbuf, sizeof(inbuf),
+ outbuf, sizeof(outbuf), &outlen);
+ if (rc)
+ return rc;
+
+ if (outlen != MC_CMD_CDX_BUS_GET_DEVICE_CONFIG_OUT_LEN)
+ return -EIO;
+
+ dev_params->bus_num = bus_num;
+ dev_params->dev_num = dev_num;
+
+ req_id = MCDI_DWORD(outbuf, CDX_BUS_GET_DEVICE_CONFIG_OUT_REQUESTER_ID);
+ dev_params->req_id = req_id;
+
+ dev_params->res_count = 0;
+ if (MCDI_QWORD(outbuf, CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_SIZE) != 0) {
+ res[dev_params->res_count].start =
+ MCDI_QWORD(outbuf, CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_BASE);
+ res[dev_params->res_count].end =
+ MCDI_QWORD(outbuf, CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_BASE) +
+ MCDI_QWORD(outbuf,
+ CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION0_SIZE) - 1;
+ res[dev_params->res_count].flags = IORESOURCE_MEM;
+ dev_params->res_count++;
+ }
+
+ if (MCDI_QWORD(outbuf, CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_SIZE) != 0) {
+ res[dev_params->res_count].start =
+ MCDI_QWORD(outbuf, CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_BASE);
+ res[dev_params->res_count].end =
+ MCDI_QWORD(outbuf, CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_BASE) +
+ MCDI_QWORD(outbuf,
+ CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION1_SIZE) - 1;
+ res[dev_params->res_count].flags = IORESOURCE_MEM;
+ dev_params->res_count++;
+ }
+
+ if (MCDI_QWORD(outbuf, CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_SIZE) != 0) {
+ res[dev_params->res_count].start =
+ MCDI_QWORD(outbuf, CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_BASE);
+ res[dev_params->res_count].end =
+ MCDI_QWORD(outbuf, CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_BASE) +
+ MCDI_QWORD(outbuf,
+ CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION2_SIZE) - 1;
+ res[dev_params->res_count].flags = IORESOURCE_MEM;
+ dev_params->res_count++;
+ }
+
+ if (MCDI_QWORD(outbuf, CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_SIZE) != 0) {
+ res[dev_params->res_count].start =
+ MCDI_QWORD(outbuf, CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_BASE);
+ res[dev_params->res_count].end =
+ MCDI_QWORD(outbuf, CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_BASE) +
+ MCDI_QWORD(outbuf,
+ CDX_BUS_GET_DEVICE_CONFIG_OUT_MMIO_REGION3_SIZE) - 1;
+ res[dev_params->res_count].flags = IORESOURCE_MEM;
+ dev_params->res_count++;
+ }
+
+ dev_params->vendor = MCDI_WORD(outbuf, CDX_BUS_GET_DEVICE_CONFIG_OUT_VENDOR_ID);
+ dev_params->device = MCDI_WORD(outbuf, CDX_BUS_GET_DEVICE_CONFIG_OUT_DEVICE_ID);
+
+ return 0;
+}
diff --git a/drivers/cdx/controller/mcdi_functions.h b/drivers/cdx/controller/mcdi_functions.h
new file mode 100644
index 000000000000..6bf5a4a0778f
--- /dev/null
+++ b/drivers/cdx/controller/mcdi_functions.h
@@ -0,0 +1,50 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ * Header file for MCDI FW interaction for CDX bus.
+ *
+ * Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+ */
+
+#ifndef CDX_MCDI_FUNCTIONS_H
+#define CDX_MCDI_FUNCTIONS_H
+
+#include "mcdi.h"
+#include "../cdx.h"
+
+/**
+ * cdx_mcdi_get_num_buses - Get the total number of buses on
+ * the controller.
+ * @cdx: pointer to MCDI interface.
+ *
+ * Return: total number of buses available on the controller,
+ * <0 on failure
+ */
+int cdx_mcdi_get_num_buses(struct cdx_mcdi *cdx);
+
+/**
+ * cdx_mcdi_get_num_devs - Get the total number of devices on
+ * a particular bus of the controller.
+ * @cdx: pointer to MCDI interface.
+ * @bus_num: Bus number.
+ *
+ * Return: total number of devices available on the bus, <0 on failure
+ */
+int cdx_mcdi_get_num_devs(struct cdx_mcdi *cdx, int bus_num);
+
+/**
+ * cdx_mcdi_get_dev_config - Get configuration for a particular
+ * bus_num:dev_num
+ * @cdx: pointer to MCDI interface.
+ * @bus_num: Bus number.
+ * @dev_num: Device number.
+ * @dev_params: Pointer to cdx_dev_params, this is populated by this
+ * device with the configuration corresponding to the provided
+ * bus_num:dev_num.
+ *
+ * Return: 0 total number of devices available on the bus, <0 on failure
+ */
+int cdx_mcdi_get_dev_config(struct cdx_mcdi *cdx,
+ u8 bus_num, u8 dev_num,
+ struct cdx_dev_params *dev_params);
+
+#endif /* CDX_MCDI_FUNCTIONS_H */
--
2.17.1
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [PATCH v6 5/7] cdx: add cdx controller
2023-01-26 10:46 ` [PATCH v6 5/7] cdx: add cdx controller Nipun Gupta
@ 2023-01-28 12:44 ` kernel test robot
2023-01-28 18:26 ` kernel test robot
1 sibling, 0 replies; 14+ messages in thread
From: kernel test robot @ 2023-01-28 12:44 UTC (permalink / raw)
To: Nipun Gupta, robh+dt, krzysztof.kozlowski+dt, gregkh, rafael,
eric.auger, alex.williamson, cohuck, song.bao.hua,
mchehab+huawei, maz, f.fainelli, jeffrey.l.hugo, saravanak,
Michael.Srba, mani, yishaih, jgg, jgg, robin.murphy, will, joro,
masahiroy, ndesaulniers, linux-arm-kernel, linux-kbuild,
linux-kernel, devicetree
Cc: oe-kbuild-all, okaya, harpreet.anand, nikhil.agarwal
Hi Nipun,
I love your patch! Yet something to improve:
[auto build test ERROR on next-20230127]
[cannot apply to masahiroy-kbuild/for-next masahiroy-kbuild/fixes robh/for-next joro-iommu/next linus/master v6.2-rc5 v6.2-rc4 v6.2-rc3 v6.2-rc5]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Nipun-Gupta/cdx-add-the-cdx-bus-driver/20230128-161622
patch link: https://lore.kernel.org/r/20230126104630.15493-6-nipun.gupta%40amd.com
patch subject: [PATCH v6 5/7] cdx: add cdx controller
config: sh-allmodconfig (https://download.01.org/0day-ci/archive/20230128/202301282010.JreX4WjQ-lkp@intel.com/config)
compiler: sh4-linux-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/f412d73800370b8e51d1be454e651d3c4ff796a8
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Nipun-Gupta/cdx-add-the-cdx-bus-driver/20230128-161622
git checkout f412d73800370b8e51d1be454e651d3c4ff796a8
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=sh olddefconfig
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=sh SHELL=/bin/bash drivers/cdx/controller/
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
All error/warnings (new ones prefixed by >>):
drivers/cdx/controller/cdx_controller.c: In function 'xlnx_cdx_probe':
>> drivers/cdx/controller/cdx_controller.c:102:20: error: implicit declaration of function 'kzalloc' [-Werror=implicit-function-declaration]
102 | cdx_mcdi = kzalloc(sizeof(*cdx_mcdi), GFP_KERNEL);
| ^~~~~~~
>> drivers/cdx/controller/cdx_controller.c:102:18: warning: assignment to 'struct cdx_mcdi *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
102 | cdx_mcdi = kzalloc(sizeof(*cdx_mcdi), GFP_KERNEL);
| ^
>> drivers/cdx/controller/cdx_controller.c:115:13: warning: assignment to 'struct cdx_controller *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
115 | cdx = kzalloc(sizeof(*cdx), GFP_KERNEL);
| ^
>> drivers/cdx/controller/cdx_controller.c:131:9: error: implicit declaration of function 'kfree'; did you mean 'kvfree'? [-Werror=implicit-function-declaration]
131 | kfree(cdx_mcdi);
| ^~~~~
| kvfree
cc1: some warnings being treated as errors
vim +/kzalloc +102 drivers/cdx/controller/cdx_controller.c
95
96 static int xlnx_cdx_probe(struct platform_device *pdev)
97 {
98 struct cdx_controller *cdx;
99 struct cdx_mcdi *cdx_mcdi;
100 int ret;
101
> 102 cdx_mcdi = kzalloc(sizeof(*cdx_mcdi), GFP_KERNEL);
103 if (!cdx_mcdi)
104 return -ENOMEM;
105
106 /* Store the MCDI ops */
107 cdx_mcdi->mcdi_ops = &mcdi_ops;
108 /* MCDI FW: Initialize the FW path */
109 ret = cdx_mcdi_init(cdx_mcdi);
110 if (ret) {
111 dev_err_probe(&pdev->dev, ret, "MCDI Initialization failed\n");
112 goto mcdi_init_fail;
113 }
114
> 115 cdx = kzalloc(sizeof(*cdx), GFP_KERNEL);
116 if (!cdx) {
117 ret = -ENOMEM;
118 goto cdx_alloc_fail;
119 }
120 platform_set_drvdata(pdev, cdx);
121
122 cdx->dev = &pdev->dev;
123 cdx->priv = cdx_mcdi;
124 cdx->ops = &cdx_ops;
125
126 return 0;
127
128 cdx_alloc_fail:
129 cdx_mcdi_finish(cdx_mcdi);
130 mcdi_init_fail:
> 131 kfree(cdx_mcdi);
132
133 return ret;
134 }
135
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [PATCH v6 5/7] cdx: add cdx controller
2023-01-26 10:46 ` [PATCH v6 5/7] cdx: add cdx controller Nipun Gupta
2023-01-28 12:44 ` kernel test robot
@ 2023-01-28 18:26 ` kernel test robot
1 sibling, 0 replies; 14+ messages in thread
From: kernel test robot @ 2023-01-28 18:26 UTC (permalink / raw)
To: Nipun Gupta, robh+dt, krzysztof.kozlowski+dt, gregkh, rafael,
eric.auger, alex.williamson, cohuck, song.bao.hua,
mchehab+huawei, maz, f.fainelli, jeffrey.l.hugo, saravanak,
Michael.Srba, mani, yishaih, jgg, jgg, robin.murphy, will, joro,
masahiroy, ndesaulniers, linux-arm-kernel, linux-kbuild,
linux-kernel, devicetree
Cc: llvm, oe-kbuild-all, okaya, harpreet.anand, nikhil.agarwal
Hi Nipun,
I love your patch! Yet something to improve:
[auto build test ERROR on next-20230127]
[cannot apply to masahiroy-kbuild/for-next masahiroy-kbuild/fixes robh/for-next joro-iommu/next linus/master v6.2-rc5 v6.2-rc4 v6.2-rc3 v6.2-rc5]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Nipun-Gupta/cdx-add-the-cdx-bus-driver/20230128-161622
patch link: https://lore.kernel.org/r/20230126104630.15493-6-nipun.gupta%40amd.com
patch subject: [PATCH v6 5/7] cdx: add cdx controller
config: powerpc-buildonly-randconfig-r004-20230129 (https://download.01.org/0day-ci/archive/20230129/202301290233.80xUAwYL-lkp@intel.com/config)
compiler: clang version 16.0.0 (https://github.com/llvm/llvm-project 4196ca3278f78c6e19246e54ab0ecb364e37d66a)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install powerpc cross compiling tool for clang build
# apt-get install binutils-powerpc-linux-gnu
# https://github.com/intel-lab-lkp/linux/commit/f412d73800370b8e51d1be454e651d3c4ff796a8
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Nipun-Gupta/cdx-add-the-cdx-bus-driver/20230128-161622
git checkout f412d73800370b8e51d1be454e651d3c4ff796a8
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=powerpc olddefconfig
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=powerpc SHELL=/bin/bash drivers/cdx/controller/
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
>> drivers/cdx/controller/cdx_controller.c:102:13: error: call to undeclared function 'kzalloc'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
cdx_mcdi = kzalloc(sizeof(*cdx_mcdi), GFP_KERNEL);
^
>> drivers/cdx/controller/cdx_controller.c:102:11: error: incompatible integer to pointer conversion assigning to 'struct cdx_mcdi *' from 'int' [-Wint-conversion]
cdx_mcdi = kzalloc(sizeof(*cdx_mcdi), GFP_KERNEL);
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> drivers/cdx/controller/cdx_controller.c:115:6: error: incompatible integer to pointer conversion assigning to 'struct cdx_controller *' from 'int' [-Wint-conversion]
cdx = kzalloc(sizeof(*cdx), GFP_KERNEL);
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> drivers/cdx/controller/cdx_controller.c:131:2: error: call to undeclared function 'kfree'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
kfree(cdx_mcdi);
^
drivers/cdx/controller/cdx_controller.c:141:2: error: call to undeclared function 'kfree'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
kfree(cdx);
^
5 errors generated.
--
>> drivers/cdx/controller/mcdi.c:272:2: error: call to undeclared function 'FIELD_PREP'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
CDX_POPULATE_DWORD_7(hdr[0],
^
drivers/cdx/controller/bitfield.h:72:30: note: expanded from macro 'CDX_POPULATE_DWORD_7'
#define CDX_POPULATE_DWORD_7 CDX_POPULATE_DWORD
^
drivers/cdx/controller/bitfield.h:69:32: note: expanded from macro 'CDX_POPULATE_DWORD'
(dword).cdx_u32 = cpu_to_le32(CDX_INSERT_FIELDS(__VA_ARGS__))
^
drivers/cdx/controller/bitfield.h:60:3: note: expanded from macro 'CDX_INSERT_FIELDS'
(CDX_INSERT_FIELD(field1, (value1)) | \
^
drivers/cdx/controller/bitfield.h:46:3: note: expanded from macro 'CDX_INSERT_FIELD'
(FIELD_PREP(GENMASK(CDX_HIGH_BIT(field), \
^
>> drivers/cdx/controller/mcdi.c:635:12: error: call to undeclared function 'FIELD_GET'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
respseq = CDX_DWORD_FIELD(outbuf[0], MCDI_HEADER_SEQ);
^
drivers/cdx/controller/bitfield.h:38:3: note: expanded from macro 'CDX_DWORD_FIELD'
(FIELD_GET(GENMASK(CDX_HIGH_BIT(field), CDX_LOW_BIT(field)), \
^
drivers/cdx/controller/mcdi.c:692:12: error: call to undeclared function 'FIELD_GET'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
respcmd = CDX_DWORD_FIELD(outbuf[0], MCDI_HEADER_CODE);
^
drivers/cdx/controller/bitfield.h:38:3: note: expanded from macro 'CDX_DWORD_FIELD'
(FIELD_GET(GENMASK(CDX_HIGH_BIT(field), CDX_LOW_BIT(field)), \
^
3 errors generated.
--
>> drivers/cdx/controller/mcdi_functions.c:25:9: error: call to undeclared function 'FIELD_GET'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
return MCDI_DWORD(outbuf, CDX_BUS_ENUM_BUSES_OUT_BUS_COUNT);
^
drivers/cdx/controller/mcdi.h:223:2: note: expanded from macro 'MCDI_DWORD'
CDX_DWORD_FIELD(*_MCDI_DWORD(_buf, _field), CDX_DWORD)
^
drivers/cdx/controller/bitfield.h:38:3: note: expanded from macro 'CDX_DWORD_FIELD'
(FIELD_GET(GENMASK(CDX_HIGH_BIT(field), CDX_LOW_BIT(field)), \
^
>> drivers/cdx/controller/mcdi_functions.c:35:2: error: call to undeclared function 'FIELD_PREP'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
MCDI_SET_DWORD(inbuf, CDX_BUS_ENUM_DEVICES_IN_BUS, bus_num);
^
drivers/cdx/controller/mcdi.h:221:2: note: expanded from macro 'MCDI_SET_DWORD'
CDX_POPULATE_DWORD_1(*_MCDI_DWORD(_buf, _field), CDX_DWORD, _value)
^
drivers/cdx/controller/bitfield.h:84:2: note: expanded from macro 'CDX_POPULATE_DWORD_1'
CDX_POPULATE_DWORD_2(dword, CDX_DWORD, 0, __VA_ARGS__)
^
drivers/cdx/controller/bitfield.h:82:2: note: expanded from macro 'CDX_POPULATE_DWORD_2'
CDX_POPULATE_DWORD_3(dword, CDX_DWORD, 0, __VA_ARGS__)
^
note: (skipping 5 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
drivers/cdx/controller/bitfield.h:69:32: note: expanded from macro 'CDX_POPULATE_DWORD'
(dword).cdx_u32 = cpu_to_le32(CDX_INSERT_FIELDS(__VA_ARGS__))
^
drivers/cdx/controller/bitfield.h:60:3: note: expanded from macro 'CDX_INSERT_FIELDS'
(CDX_INSERT_FIELD(field1, (value1)) | \
^
drivers/cdx/controller/bitfield.h:46:3: note: expanded from macro 'CDX_INSERT_FIELD'
(FIELD_PREP(GENMASK(CDX_HIGH_BIT(field), \
^
drivers/cdx/controller/mcdi_functions.c:45:9: error: call to undeclared function 'FIELD_GET'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
return MCDI_DWORD(outbuf, CDX_BUS_ENUM_DEVICES_OUT_DEVICE_COUNT);
^
drivers/cdx/controller/mcdi.h:223:2: note: expanded from macro 'MCDI_DWORD'
CDX_DWORD_FIELD(*_MCDI_DWORD(_buf, _field), CDX_DWORD)
^
drivers/cdx/controller/bitfield.h:38:3: note: expanded from macro 'CDX_DWORD_FIELD'
(FIELD_GET(GENMASK(CDX_HIGH_BIT(field), CDX_LOW_BIT(field)), \
^
drivers/cdx/controller/mcdi_functions.c:59:2: error: call to undeclared function 'FIELD_PREP'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
MCDI_SET_DWORD(inbuf, CDX_BUS_GET_DEVICE_CONFIG_IN_BUS, bus_num);
^
drivers/cdx/controller/mcdi.h:221:2: note: expanded from macro 'MCDI_SET_DWORD'
CDX_POPULATE_DWORD_1(*_MCDI_DWORD(_buf, _field), CDX_DWORD, _value)
^
drivers/cdx/controller/bitfield.h:84:2: note: expanded from macro 'CDX_POPULATE_DWORD_1'
CDX_POPULATE_DWORD_2(dword, CDX_DWORD, 0, __VA_ARGS__)
^
drivers/cdx/controller/bitfield.h:82:2: note: expanded from macro 'CDX_POPULATE_DWORD_2'
CDX_POPULATE_DWORD_3(dword, CDX_DWORD, 0, __VA_ARGS__)
^
note: (skipping 5 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
drivers/cdx/controller/bitfield.h:69:32: note: expanded from macro 'CDX_POPULATE_DWORD'
(dword).cdx_u32 = cpu_to_le32(CDX_INSERT_FIELDS(__VA_ARGS__))
^
drivers/cdx/controller/bitfield.h:60:3: note: expanded from macro 'CDX_INSERT_FIELDS'
(CDX_INSERT_FIELD(field1, (value1)) | \
^
drivers/cdx/controller/bitfield.h:46:3: note: expanded from macro 'CDX_INSERT_FIELD'
(FIELD_PREP(GENMASK(CDX_HIGH_BIT(field), \
^
drivers/cdx/controller/mcdi_functions.c:73:11: error: call to undeclared function 'FIELD_GET'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
req_id = MCDI_DWORD(outbuf, CDX_BUS_GET_DEVICE_CONFIG_OUT_REQUESTER_ID);
^
drivers/cdx/controller/mcdi.h:223:2: note: expanded from macro 'MCDI_DWORD'
CDX_DWORD_FIELD(*_MCDI_DWORD(_buf, _field), CDX_DWORD)
^
drivers/cdx/controller/bitfield.h:38:3: note: expanded from macro 'CDX_DWORD_FIELD'
(FIELD_GET(GENMASK(CDX_HIGH_BIT(field), CDX_LOW_BIT(field)), \
^
5 errors generated.
vim +/kzalloc +102 drivers/cdx/controller/cdx_controller.c
95
96 static int xlnx_cdx_probe(struct platform_device *pdev)
97 {
98 struct cdx_controller *cdx;
99 struct cdx_mcdi *cdx_mcdi;
100 int ret;
101
> 102 cdx_mcdi = kzalloc(sizeof(*cdx_mcdi), GFP_KERNEL);
103 if (!cdx_mcdi)
104 return -ENOMEM;
105
106 /* Store the MCDI ops */
107 cdx_mcdi->mcdi_ops = &mcdi_ops;
108 /* MCDI FW: Initialize the FW path */
109 ret = cdx_mcdi_init(cdx_mcdi);
110 if (ret) {
111 dev_err_probe(&pdev->dev, ret, "MCDI Initialization failed\n");
112 goto mcdi_init_fail;
113 }
114
> 115 cdx = kzalloc(sizeof(*cdx), GFP_KERNEL);
116 if (!cdx) {
117 ret = -ENOMEM;
118 goto cdx_alloc_fail;
119 }
120 platform_set_drvdata(pdev, cdx);
121
122 cdx->dev = &pdev->dev;
123 cdx->priv = cdx_mcdi;
124 cdx->ops = &cdx_ops;
125
126 return 0;
127
128 cdx_alloc_fail:
129 cdx_mcdi_finish(cdx_mcdi);
130 mcdi_init_fail:
> 131 kfree(cdx_mcdi);
132
133 return ret;
134 }
135
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH v6 6/7] cdx: add rpmsg communication channel for CDX
2023-01-26 10:46 [PATCH v6 0/7] add support for CDX bus Nipun Gupta
` (4 preceding siblings ...)
2023-01-26 10:46 ` [PATCH v6 5/7] cdx: add cdx controller Nipun Gupta
@ 2023-01-26 10:46 ` Nipun Gupta
2023-01-26 10:46 ` [PATCH v6 7/7] cdx: add device attributes Nipun Gupta
6 siblings, 0 replies; 14+ messages in thread
From: Nipun Gupta @ 2023-01-26 10:46 UTC (permalink / raw)
To: robh+dt, krzysztof.kozlowski+dt, gregkh, rafael, eric.auger,
alex.williamson, cohuck, song.bao.hua, mchehab+huawei, maz,
f.fainelli, jeffrey.l.hugo, saravanak, Michael.Srba, mani,
yishaih, jgg, jgg, robin.murphy, will, joro, masahiroy,
ndesaulniers, linux-arm-kernel, linux-kbuild, linux-kernel,
devicetree
Cc: okaya, harpreet.anand, nikhil.agarwal, michal.simek, git,
Abhijit Gangurde, Nipun Gupta
From: Abhijit Gangurde <abhijit.gangurde@amd.com>
RPMsg is used as a transport communication channel. This
change introduces RPMsg driver and integrates it with the
CDX controller.
Signed-off-by: Abhijit Gangurde <abhijit.gangurde@amd.com>
Signed-off-by: Nipun Gupta <nipun.gupta@amd.com>
---
drivers/cdx/controller/Kconfig | 1 +
drivers/cdx/controller/Makefile | 2 +-
drivers/cdx/controller/cdx_controller.c | 31 +++-
drivers/cdx/controller/cdx_controller.h | 30 ++++
drivers/cdx/controller/cdx_rpmsg.c | 202 ++++++++++++++++++++++++
drivers/cdx/controller/mcdi.h | 9 ++
6 files changed, 270 insertions(+), 5 deletions(-)
create mode 100644 drivers/cdx/controller/cdx_controller.h
create mode 100644 drivers/cdx/controller/cdx_rpmsg.c
diff --git a/drivers/cdx/controller/Kconfig b/drivers/cdx/controller/Kconfig
index 17f9c6be2fe1..aea3ac86d3aa 100644
--- a/drivers/cdx/controller/Kconfig
+++ b/drivers/cdx/controller/Kconfig
@@ -9,6 +9,7 @@ if CDX_BUS
config CDX_CONTROLLER
tristate "CDX bus controller"
+ select RPMSG
help
CDX controller drives the CDX bus. It interacts with
firmware to get the hardware devices and registers with
diff --git a/drivers/cdx/controller/Makefile b/drivers/cdx/controller/Makefile
index f7437c882cc9..f071be411d96 100644
--- a/drivers/cdx/controller/Makefile
+++ b/drivers/cdx/controller/Makefile
@@ -6,4 +6,4 @@
#
obj-$(CONFIG_CDX_CONTROLLER) += cdx-controller.o
-cdx-controller-objs := cdx_controller.o mcdi.o mcdi_functions.o
+cdx-controller-objs := cdx_controller.o cdx_rpmsg.o mcdi.o mcdi_functions.o
diff --git a/drivers/cdx/controller/cdx_controller.c b/drivers/cdx/controller/cdx_controller.c
index ec578422f87e..eeba4a74aadd 100644
--- a/drivers/cdx/controller/cdx_controller.c
+++ b/drivers/cdx/controller/cdx_controller.c
@@ -8,6 +8,7 @@
#include <linux/of_platform.h>
#include <linux/cdx/cdx_bus.h>
+#include "cdx_controller.h"
#include "../cdx.h"
#include "mcdi_functions.h"
#include "mcdi.h"
@@ -21,10 +22,7 @@ static void cdx_mcdi_request(struct cdx_mcdi *cdx,
const struct cdx_dword *hdr, size_t hdr_len,
const struct cdx_dword *sdu, size_t sdu_len)
{
- /*
- * This will get updated by rpmsg APIs, with RPMSG introduction
- * in CDX controller as a transport layer.
- */
+ cdx_rpmsg_send(cdx, hdr, hdr_len, sdu, sdu_len);
}
static const struct cdx_mcdi_ops mcdi_ops = {
@@ -32,6 +30,19 @@ static const struct cdx_mcdi_ops mcdi_ops = {
.mcdi_request = cdx_mcdi_request,
};
+void cdx_rpmsg_post_probe(struct cdx_controller *cdx)
+{
+ /* Register CDX controller with CDX bus driver */
+ if (cdx_register_controller(cdx))
+ dev_err(cdx->dev, "Failed to register CDX controller\n");
+}
+
+void cdx_rpmsg_pre_remove(struct cdx_controller *cdx)
+{
+ cdx_unregister_controller(cdx);
+ cdx_mcdi_wait_for_quiescence(cdx->priv, MCDI_RPC_TIMEOUT);
+}
+
static int cdx_scan_devices(struct cdx_controller *cdx)
{
struct cdx_mcdi *cdx_mcdi = cdx->priv;
@@ -123,8 +134,18 @@ static int xlnx_cdx_probe(struct platform_device *pdev)
cdx->priv = cdx_mcdi;
cdx->ops = &cdx_ops;
+ ret = cdx_setup_rpmsg(pdev);
+ if (ret) {
+ if (ret != -EPROBE_DEFER)
+ dev_err(&pdev->dev, "Failed to register CDX RPMsg transport\n");
+ goto cdx_rpmsg_fail;
+ }
+
+ dev_info(&pdev->dev, "Successfully registered CDX controller with RPMsg as transport\n");
return 0;
+cdx_rpmsg_fail:
+ kfree(cdx);
cdx_alloc_fail:
cdx_mcdi_finish(cdx_mcdi);
mcdi_init_fail:
@@ -138,6 +159,8 @@ static int xlnx_cdx_remove(struct platform_device *pdev)
struct cdx_controller *cdx = platform_get_drvdata(pdev);
struct cdx_mcdi *cdx_mcdi = cdx->priv;
+ cdx_destroy_rpmsg(pdev);
+
kfree(cdx);
cdx_mcdi_finish(cdx_mcdi);
diff --git a/drivers/cdx/controller/cdx_controller.h b/drivers/cdx/controller/cdx_controller.h
new file mode 100644
index 000000000000..43b7c742df87
--- /dev/null
+++ b/drivers/cdx/controller/cdx_controller.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ * Header file for the CDX Controller
+ *
+ * Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+ */
+
+#ifndef _CDX_CONTROLLER_H_
+#define _CDX_CONTROLLER_H_
+
+#include <linux/cdx/cdx_bus.h>
+#include "mcdi_functions.h"
+
+void cdx_rpmsg_post_probe(struct cdx_controller *cdx);
+
+void cdx_rpmsg_pre_remove(struct cdx_controller *cdx);
+
+int cdx_rpmsg_send(struct cdx_mcdi *cdx_mcdi,
+ const struct cdx_dword *hdr, size_t hdr_len,
+ const struct cdx_dword *sdu, size_t sdu_len);
+
+void cdx_rpmsg_read_resp(struct cdx_mcdi *cdx_mcdi,
+ struct cdx_dword *outbuf, size_t offset,
+ size_t outlen);
+
+int cdx_setup_rpmsg(struct platform_device *pdev);
+
+void cdx_destroy_rpmsg(struct platform_device *pdev);
+
+#endif /* _CDX_CONT_PRIV_H_ */
diff --git a/drivers/cdx/controller/cdx_rpmsg.c b/drivers/cdx/controller/cdx_rpmsg.c
new file mode 100644
index 000000000000..2be9bb279f81
--- /dev/null
+++ b/drivers/cdx/controller/cdx_rpmsg.c
@@ -0,0 +1,202 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Platform driver for CDX bus.
+ *
+ * Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+ */
+
+#include <linux/rpmsg.h>
+#include <linux/remoteproc.h>
+#include <linux/of_platform.h>
+#include <linux/cdx/cdx_bus.h>
+#include <linux/module.h>
+
+#include "../cdx.h"
+#include "cdx_controller.h"
+#include "mcdi_functions.h"
+#include "mcdi.h"
+
+static struct rpmsg_device_id cdx_rpmsg_id_table[] = {
+ { .name = "mcdi_ipc" },
+ { },
+};
+MODULE_DEVICE_TABLE(rpmsg, cdx_rpmsg_id_table);
+
+int cdx_rpmsg_send(struct cdx_mcdi *cdx_mcdi,
+ const struct cdx_dword *hdr, size_t hdr_len,
+ const struct cdx_dword *sdu, size_t sdu_len)
+{
+ unsigned char *send_buf;
+ int rc;
+
+ send_buf = kzalloc(hdr_len + sdu_len, GFP_KERNEL);
+ if (!send_buf)
+ return -ENOMEM;
+
+ memcpy(send_buf, hdr, hdr_len);
+ memcpy(send_buf + hdr_len, sdu, sdu_len);
+
+ rc = rpmsg_send(cdx_mcdi->ept, send_buf, hdr_len + sdu_len);
+ kfree(send_buf);
+
+ return rc;
+}
+
+static int cdx_attach_to_rproc(struct platform_device *pdev)
+{
+ struct device_node *r5_core_node;
+ struct cdx_controller *cdx_c;
+ struct cdx_mcdi *cdx_mcdi;
+ struct device *dev;
+ struct rproc *rp;
+ int ret;
+
+ dev = &pdev->dev;
+ cdx_c = platform_get_drvdata(pdev);
+ cdx_mcdi = cdx_c->priv;
+
+ r5_core_node = of_parse_phandle(dev->of_node, "xlnx,rproc", 0);
+ if (!r5_core_node) {
+ dev_err(&pdev->dev, "xlnx,rproc: invalid phandle\n");
+ return -EINVAL;
+ }
+
+ rp = rproc_get_by_phandle(r5_core_node->phandle);
+ if (!rp) {
+ ret = -EPROBE_DEFER;
+ goto pdev_err;
+ }
+
+ /* Attach to remote processor */
+ ret = rproc_boot(rp);
+ if (ret) {
+ dev_err(&pdev->dev, "Failed to attach to remote processor\n");
+ rproc_put(rp);
+ goto pdev_err;
+ }
+
+ cdx_mcdi->r5_rproc = rp;
+pdev_err:
+ of_node_put(r5_core_node);
+ return ret;
+}
+
+static void cdx_detach_to_r5(struct platform_device *pdev)
+{
+ struct cdx_controller *cdx_c;
+ struct cdx_mcdi *cdx_mcdi;
+
+ cdx_c = platform_get_drvdata(pdev);
+ cdx_mcdi = cdx_c->priv;
+
+ rproc_detach(cdx_mcdi->r5_rproc);
+ rproc_put(cdx_mcdi->r5_rproc);
+}
+
+static int cdx_rpmsg_cb(struct rpmsg_device *rpdev, void *data,
+ int len, void *priv, u32 src)
+{
+ struct cdx_controller *cdx_c = dev_get_drvdata(&rpdev->dev);
+ struct cdx_mcdi *cdx_mcdi = cdx_c->priv;
+
+ if (len > MCDI_BUF_LEN)
+ return -EINVAL;
+
+ cdx_mcdi_process_cmd(cdx_mcdi, (struct cdx_dword *)data, len);
+
+ return 0;
+}
+
+static void cdx_rpmsg_post_probe_work(struct work_struct *work)
+{
+ struct cdx_controller *cdx_c;
+ struct cdx_mcdi *cdx_mcdi;
+
+ cdx_mcdi = container_of(work, struct cdx_mcdi, work);
+ cdx_c = dev_get_drvdata(&cdx_mcdi->rpdev->dev);
+ cdx_rpmsg_post_probe(cdx_c);
+}
+
+static int cdx_rpmsg_probe(struct rpmsg_device *rpdev)
+{
+ struct rpmsg_channel_info chinfo = {0};
+ struct cdx_controller *cdx_c;
+ struct cdx_mcdi *cdx_mcdi;
+
+ cdx_c = (struct cdx_controller *)cdx_rpmsg_id_table[0].driver_data;
+ cdx_mcdi = cdx_c->priv;
+
+ chinfo.src = RPMSG_ADDR_ANY;
+ chinfo.dst = rpdev->dst;
+ strscpy(chinfo.name, cdx_rpmsg_id_table[0].name,
+ strlen(cdx_rpmsg_id_table[0].name));
+
+ cdx_mcdi->ept = rpmsg_create_ept(rpdev, cdx_rpmsg_cb, NULL, chinfo);
+ if (!cdx_mcdi->ept) {
+ dev_err_probe(&rpdev->dev, -ENXIO,
+ "Failed to create ept for channel %s\n",
+ chinfo.name);
+ return -EINVAL;
+ }
+
+ cdx_mcdi->rpdev = rpdev;
+ dev_set_drvdata(&rpdev->dev, cdx_c);
+
+ schedule_work(&cdx_mcdi->work);
+ return 0;
+}
+
+static void cdx_rpmsg_remove(struct rpmsg_device *rpdev)
+{
+ struct cdx_controller *cdx_c = dev_get_drvdata(&rpdev->dev);
+ struct cdx_mcdi *cdx_mcdi = cdx_c->priv;
+
+ flush_work(&cdx_mcdi->work);
+ cdx_rpmsg_pre_remove(cdx_c);
+
+ rpmsg_destroy_ept(cdx_mcdi->ept);
+ dev_set_drvdata(&rpdev->dev, NULL);
+}
+
+static struct rpmsg_driver cdx_rpmsg_driver = {
+ .drv.name = KBUILD_MODNAME,
+ .id_table = cdx_rpmsg_id_table,
+ .probe = cdx_rpmsg_probe,
+ .remove = cdx_rpmsg_remove,
+ .callback = cdx_rpmsg_cb,
+};
+
+int cdx_setup_rpmsg(struct platform_device *pdev)
+{
+ struct cdx_controller *cdx_c;
+ struct cdx_mcdi *cdx_mcdi;
+ int ret;
+
+ /* Attach to remote processor */
+ ret = cdx_attach_to_rproc(pdev);
+ if (ret)
+ return ret;
+
+ cdx_c = platform_get_drvdata(pdev);
+ cdx_mcdi = cdx_c->priv;
+
+ /* Register RPMsg driver */
+ cdx_rpmsg_id_table[0].driver_data = (kernel_ulong_t)cdx_c;
+
+ INIT_WORK(&cdx_mcdi->work, cdx_rpmsg_post_probe_work);
+ ret = register_rpmsg_driver(&cdx_rpmsg_driver);
+ if (ret < 0) {
+ dev_err(&pdev->dev,
+ "Failed to register cdx RPMsg driver: %d\n", ret);
+ cdx_detach_to_r5(pdev);
+ }
+
+ return ret;
+}
+
+void cdx_destroy_rpmsg(struct platform_device *pdev)
+{
+ unregister_rpmsg_driver(&cdx_rpmsg_driver);
+
+ cdx_detach_to_r5(pdev);
+}
diff --git a/drivers/cdx/controller/mcdi.h b/drivers/cdx/controller/mcdi.h
index 63933ede33ed..a4d750816bd2 100644
--- a/drivers/cdx/controller/mcdi.h
+++ b/drivers/cdx/controller/mcdi.h
@@ -62,11 +62,20 @@ enum cdx_mcdi_cmd_state {
* with CDX controller.
* @mcdi: MCDI interface
* @mcdi_ops: MCDI operations
+ * @r5_rproc : R5 Remoteproc device handle
+ * @rpdev: RPMsg device
+ * @ept: RPMsg endpoint
+ * @work: Post probe work
*/
struct cdx_mcdi {
/* MCDI interface */
struct cdx_mcdi_data *mcdi;
const struct cdx_mcdi_ops *mcdi_ops;
+
+ struct rproc *r5_rproc;
+ struct rpmsg_device *rpdev;
+ struct rpmsg_endpoint *ept;
+ struct work_struct work;
};
struct cdx_mcdi_ops {
--
2.17.1
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v6 7/7] cdx: add device attributes
2023-01-26 10:46 [PATCH v6 0/7] add support for CDX bus Nipun Gupta
` (5 preceding siblings ...)
2023-01-26 10:46 ` [PATCH v6 6/7] cdx: add rpmsg communication channel for CDX Nipun Gupta
@ 2023-01-26 10:46 ` Nipun Gupta
2023-01-26 16:55 ` Randy Dunlap
6 siblings, 1 reply; 14+ messages in thread
From: Nipun Gupta @ 2023-01-26 10:46 UTC (permalink / raw)
To: robh+dt, krzysztof.kozlowski+dt, gregkh, rafael, eric.auger,
alex.williamson, cohuck, song.bao.hua, mchehab+huawei, maz,
f.fainelli, jeffrey.l.hugo, saravanak, Michael.Srba, mani,
yishaih, jgg, jgg, robin.murphy, will, joro, masahiroy,
ndesaulniers, linux-arm-kernel, linux-kbuild, linux-kernel,
devicetree
Cc: okaya, harpreet.anand, nikhil.agarwal, michal.simek, git, Nipun Gupta
Create sysfs entry for CDX devices.
Sysfs entries provided in each of the CDX device detected by
the CDX controller
- vendor id
- device id
- remove
- reset of the device.
- driver override
Signed-off-by: Puneet Gupta <puneet.gupta@amd.com>
Signed-off-by: Nipun Gupta <nipun.gupta@amd.com>
Signed-off-by: Tarak Reddy <tarak.reddy@amd.com>
---
Documentation/ABI/testing/sysfs-bus-cdx | 44 +++++++
drivers/cdx/cdx.c | 145 ++++++++++++++++++++++++
drivers/cdx/controller/cdx_controller.c | 19 ++++
drivers/cdx/controller/mcdi_functions.c | 14 +++
drivers/cdx/controller/mcdi_functions.h | 11 ++
include/linux/cdx/cdx_bus.h | 27 +++++
6 files changed, 260 insertions(+)
diff --git a/Documentation/ABI/testing/sysfs-bus-cdx b/Documentation/ABI/testing/sysfs-bus-cdx
index 8c2425fdb6d9..69bdbc891743 100644
--- a/Documentation/ABI/testing/sysfs-bus-cdx
+++ b/Documentation/ABI/testing/sysfs-bus-cdx
@@ -10,3 +10,47 @@ Description:
For example::
# echo 1 > /sys/bus/cdx/rescan
+
+What: /sys/bus/cdx/devices/.../vendor
+Date: January 2023
+Contact: nipun.gupta@amd.com
+Description:
+ Vendor ID for this CDX device. Vendor ID is 16 bit
+ identifier which is specific to the device manufacturer.
+ Combination of Vendor ID and Device ID identifies a device.
+
+What: /sys/bus/cdx/devices/.../device
+Date: January 2023
+Contact: nipun.gupta@amd.com
+Description:
+ Device ID for this CDX device. Device ID is a 16 bit
+ identifier to identify a device type within the range
+ of a device manufacturer.
+ Combination of Vendor ID and Device ID identifies a device.
+
+What: /sys/bus/cdx/devices/.../reset
+Date: January 2023
+Contact: nipun.gupta@amd.com
+Description:
+ Writing a non-zero value to this file would reset the CDX
+ device. On resetting the device, the corresponding driver
+ would be notified twice, once before the device is being
+ reset, and again after the reset has been complete.
+
+ For example::
+
+ # echo 1 > /sys/bus/cdx/.../reset
+
+What: /sys/bus/cdx/devices/.../remove
+Date: January 2023
+Contact: tarak.reddy@amd.com
+Description:
+ Writing a non-zero value to this file would remove the
+ corrosponding device from the CDX bus. If the device is
+ to be reconfigured in the Hardware, the device can be
+ removed, so that the device driver does not access the
+ device while it is being reconfigured.
+
+ For example::
+
+ # echo 1 > /sys/bus/cdx/devices/.../remove
diff --git a/drivers/cdx/cdx.c b/drivers/cdx/cdx.c
index 795770522b01..88f71227f4ee 100644
--- a/drivers/cdx/cdx.c
+++ b/drivers/cdx/cdx.c
@@ -72,6 +72,39 @@
/* CDX controllers registered with the CDX bus */
static DEFINE_XARRAY_ALLOC(cdx_controllers);
+/**
+ * cdx_dev_reset - Reset a CDX device
+ * @dev: CDX device
+ *
+ * Return: -errno on failure, 0 on success.
+ */
+int cdx_dev_reset(struct device *dev)
+{
+ struct cdx_device *cdx_dev = to_cdx_device(dev);
+ struct cdx_controller *cdx = cdx_dev->cdx;
+ struct cdx_device_config dev_config;
+ struct cdx_driver *cdx_drv;
+ int ret;
+
+ cdx_drv = to_cdx_driver(dev->driver);
+ /* Notify driver that device is being reset */
+ if (cdx_drv->reset_prepare)
+ cdx_drv->reset_prepare(cdx_dev);
+
+ dev_config.type = CDX_DEV_RESET_CONF;
+ ret = cdx->ops->dev_configure(cdx, cdx_dev->bus_num,
+ cdx_dev->dev_num, &dev_config);
+ if (ret)
+ dev_err(dev, "cdx device reset failed\n");
+
+ /* Notify driver that device reset is complete */
+ if (cdx_drv->reset_done)
+ cdx_drv->reset_done(cdx_dev);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(cdx_dev_reset);
+
/**
* cdx_unregister_device - Unregister a CDX device
* @dev: CDX device
@@ -238,6 +271,117 @@ static int cdx_dma_configure(struct device *dev)
return 0;
}
+/* show configuration fields */
+#define cdx_config_attr(field, format_string) \
+static ssize_t \
+field##_show(struct device *dev, struct device_attribute *attr, char *buf) \
+{ \
+ struct cdx_device *cdx_dev = to_cdx_device(dev); \
+ return sysfs_emit(buf, format_string, cdx_dev->field); \
+} \
+static DEVICE_ATTR_RO(field)
+
+cdx_config_attr(vendor, "0x%04x\n");
+cdx_config_attr(device, "0x%04x\n");
+
+static ssize_t remove_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ unsigned long val = 0;
+
+ if (kstrtoul(buf, 0, &val) < 0)
+ return -EINVAL;
+
+ if (!val)
+ return -EINVAL;
+
+ if (device_remove_file_self(dev, attr)) {
+ int ret;
+
+ ret = cdx_unregister_device(dev, NULL);
+ if (ret)
+ return ret;
+ }
+
+ return count;
+}
+static DEVICE_ATTR_WO(remove);
+
+static ssize_t reset_store(struct device *dev, struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ unsigned long val = 0;
+ int ret = 0;
+
+ if (kstrtoul(buf, 0, &val) < 0)
+ return -EINVAL;
+
+ if (!val)
+ return -EINVAL;
+
+ ret = cdx_dev_reset(dev);
+ if (ret)
+ return ret;
+
+ return count;
+}
+static DEVICE_ATTR_WO(reset);
+
+static ssize_t driver_override_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct cdx_device *cdx_dev = to_cdx_device(dev);
+ const char *old = cdx_dev->driver_override;
+ char *driver_override;
+ char *cp;
+
+ if (WARN_ON(dev->bus != &cdx_bus_type))
+ return -EINVAL;
+
+ if (count >= (PAGE_SIZE - 1))
+ return -EINVAL;
+
+ driver_override = kstrndup(buf, count, GFP_KERNEL);
+ if (!driver_override)
+ return -ENOMEM;
+
+ cp = strchr(driver_override, '\n');
+ if (cp)
+ *cp = '\0';
+
+ if (strlen(driver_override)) {
+ cdx_dev->driver_override = driver_override;
+ } else {
+ kfree(driver_override);
+ cdx_dev->driver_override = NULL;
+ }
+
+ kfree(old);
+
+ return count;
+}
+
+static ssize_t driver_override_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct cdx_device *cdx_dev = to_cdx_device(dev);
+
+ return sysfs_emit(buf, "%s\n", cdx_dev->driver_override);
+}
+static DEVICE_ATTR_RW(driver_override);
+
+static struct attribute *cdx_dev_attrs[] = {
+ &dev_attr_remove.attr,
+ &dev_attr_reset.attr,
+ &dev_attr_vendor.attr,
+ &dev_attr_device.attr,
+ &dev_attr_driver_override.attr,
+ NULL,
+};
+ATTRIBUTE_GROUPS(cdx_dev);
+
static ssize_t rescan_store(struct bus_type *bus,
const char *buf, size_t count)
{
@@ -280,6 +424,7 @@ struct bus_type cdx_bus_type = {
.shutdown = cdx_shutdown,
.dma_configure = cdx_dma_configure,
.bus_groups = cdx_bus_groups,
+ .dev_groups = cdx_dev_groups,
};
EXPORT_SYMBOL_GPL(cdx_bus_type);
diff --git a/drivers/cdx/controller/cdx_controller.c b/drivers/cdx/controller/cdx_controller.c
index eeba4a74aadd..bb986bf8e29a 100644
--- a/drivers/cdx/controller/cdx_controller.c
+++ b/drivers/cdx/controller/cdx_controller.c
@@ -43,6 +43,24 @@ void cdx_rpmsg_pre_remove(struct cdx_controller *cdx)
cdx_mcdi_wait_for_quiescence(cdx->priv, MCDI_RPC_TIMEOUT);
}
+static int cdx_configure_device(struct cdx_controller *cdx,
+ u8 bus_num, u8 dev_num,
+ struct cdx_device_config *dev_config)
+{
+ int ret = 0;
+
+ switch (dev_config->type) {
+ case CDX_DEV_RESET_CONF:
+ ret = cdx_mcdi_reset_device(cdx->priv, bus_num, dev_num);
+ break;
+ default:
+ dev_err(cdx->dev, "Invalid device configuration flag\n");
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
static int cdx_scan_devices(struct cdx_controller *cdx)
{
struct cdx_mcdi *cdx_mcdi = cdx->priv;
@@ -102,6 +120,7 @@ static int cdx_scan_devices(struct cdx_controller *cdx)
static struct cdx_ops cdx_ops = {
.scan = cdx_scan_devices,
+ .dev_configure = cdx_configure_device,
};
static int xlnx_cdx_probe(struct platform_device *pdev)
diff --git a/drivers/cdx/controller/mcdi_functions.c b/drivers/cdx/controller/mcdi_functions.c
index 3940a2c7919c..673b3896411e 100644
--- a/drivers/cdx/controller/mcdi_functions.c
+++ b/drivers/cdx/controller/mcdi_functions.c
@@ -123,3 +123,17 @@ int cdx_mcdi_get_dev_config(struct cdx_mcdi *cdx,
return 0;
}
+
+int cdx_mcdi_reset_device(struct cdx_mcdi *cdx, u8 bus_num, u8 dev_num)
+{
+ MCDI_DECLARE_BUF(inbuf, MC_CMD_CDX_DEVICE_RESET_IN_LEN);
+ int rc;
+
+ MCDI_SET_DWORD(inbuf, CDX_DEVICE_RESET_IN_BUS, bus_num);
+ MCDI_SET_DWORD(inbuf, CDX_DEVICE_RESET_IN_DEVICE, dev_num);
+
+ rc = cdx_mcdi_rpc(cdx, MC_CMD_CDX_DEVICE_RESET, inbuf, sizeof(inbuf),
+ NULL, 0, NULL);
+
+ return rc;
+}
diff --git a/drivers/cdx/controller/mcdi_functions.h b/drivers/cdx/controller/mcdi_functions.h
index 6bf5a4a0778f..7440ace5539a 100644
--- a/drivers/cdx/controller/mcdi_functions.h
+++ b/drivers/cdx/controller/mcdi_functions.h
@@ -47,4 +47,15 @@ int cdx_mcdi_get_dev_config(struct cdx_mcdi *cdx,
u8 bus_num, u8 dev_num,
struct cdx_dev_params *dev_params);
+/**
+ * cdx_mcdi_reset_device - Reset cdx device represented by bus_num:dev_num
+ * @cdx: pointer to MCDI interface.
+ * @bus_num: Bus number.
+ * @dev_num: Device number.
+ *
+ * Return: 0 on success, <0 on failure
+ */
+int cdx_mcdi_reset_device(struct cdx_mcdi *cdx,
+ u8 bus_num, u8 dev_num);
+
#endif /* CDX_MCDI_FUNCTIONS_H */
diff --git a/include/linux/cdx/cdx_bus.h b/include/linux/cdx/cdx_bus.h
index d134e0104724..35ef41d8a61a 100644
--- a/include/linux/cdx/cdx_bus.h
+++ b/include/linux/cdx/cdx_bus.h
@@ -21,8 +21,20 @@
/* Forward declaration for CDX controller */
struct cdx_controller;
+enum {
+ CDX_DEV_RESET_CONF,
+};
+
+struct cdx_device_config {
+ u8 type;
+};
+
typedef int (*cdx_scan_cb)(struct cdx_controller *cdx);
+typedef int (*cdx_dev_configure_cb)(struct cdx_controller *cdx,
+ u8 bus_num, u8 dev_num,
+ struct cdx_device_config *dev_config);
+
/**
* CDX_DEVICE_DRIVER_OVERRIDE - macro used to describe a CDX device with
* override_only flags.
@@ -39,9 +51,12 @@ typedef int (*cdx_scan_cb)(struct cdx_controller *cdx);
/**
* struct cdx_ops - Callbacks supported by CDX controller.
* @scan: scan the devices on the controller
+ * @dev_configure: configuration like reset, master_enable,
+ * msi_config etc for a CDX device
*/
struct cdx_ops {
cdx_scan_cb scan;
+ cdx_dev_configure_cb dev_configure;
};
/**
@@ -101,6 +116,8 @@ struct cdx_device {
* @probe: Function called when a device is added
* @remove: Function called when a device is removed
* @shutdown: Function called at shutdown time to quiesce the device
+ * @reset_prepare: Function called before is reset to notify driver
+ * @reset_done: Function called after reset is complete to notify driver
* @driver_managed_dma: Device driver doesn't use kernel DMA API for DMA.
* For most device drivers, no need to care about this flag
* as long as all DMAs are handled through the kernel DMA API.
@@ -115,6 +132,8 @@ struct cdx_driver {
int (*probe)(struct cdx_device *dev);
int (*remove)(struct cdx_device *dev);
void (*shutdown)(struct cdx_device *dev);
+ void (*reset_prepare)(struct cdx_device *dev);
+ void (*reset_done)(struct cdx_device *dev);
bool driver_managed_dma;
};
@@ -144,4 +163,12 @@ void cdx_driver_unregister(struct cdx_driver *cdx_driver);
extern struct bus_type cdx_bus_type;
+/**
+ * cdx_dev_reset - Reset CDX device
+ * @dev: device pointer
+ *
+ * Return: 0 for success, -errno on failure
+ */
+int cdx_dev_reset(struct device *dev);
+
#endif /* _CDX_BUS_H_ */
--
2.17.1
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [PATCH v6 7/7] cdx: add device attributes
2023-01-26 10:46 ` [PATCH v6 7/7] cdx: add device attributes Nipun Gupta
@ 2023-01-26 16:55 ` Randy Dunlap
2023-01-27 11:17 ` Nipun Gupta
0 siblings, 1 reply; 14+ messages in thread
From: Randy Dunlap @ 2023-01-26 16:55 UTC (permalink / raw)
To: Nipun Gupta, robh+dt, krzysztof.kozlowski+dt, gregkh, rafael,
eric.auger, alex.williamson, cohuck, song.bao.hua,
mchehab+huawei, maz, f.fainelli, jeffrey.l.hugo, saravanak,
Michael.Srba, mani, yishaih, jgg, jgg, robin.murphy, will, joro,
masahiroy, ndesaulniers, linux-arm-kernel, linux-kbuild,
linux-kernel, devicetree
Cc: okaya, harpreet.anand, nikhil.agarwal, michal.simek, git
Hi--
On 1/26/23 02:46, Nipun Gupta wrote:
> diff --git a/Documentation/ABI/testing/sysfs-bus-cdx b/Documentation/ABI/testing/sysfs-bus-cdx
> index 8c2425fdb6d9..69bdbc891743 100644
> --- a/Documentation/ABI/testing/sysfs-bus-cdx
> +++ b/Documentation/ABI/testing/sysfs-bus-cdx
> @@ -10,3 +10,47 @@ Description:
> For example::
>
> # echo 1 > /sys/bus/cdx/rescan
> +
> +What: /sys/bus/cdx/devices/.../vendor
> +Date: January 2023
> +Contact: nipun.gupta@amd.com
> +Description:
> + Vendor ID for this CDX device. Vendor ID is 16 bit
> + identifier which is specific to the device manufacturer.
> + Combination of Vendor ID and Device ID identifies a device.
> +
> +What: /sys/bus/cdx/devices/.../device
> +Date: January 2023
> +Contact: nipun.gupta@amd.com
> +Description:
> + Device ID for this CDX device. Device ID is a 16 bit
> + identifier to identify a device type within the range
> + of a device manufacturer.
> + Combination of Vendor ID and Device ID identifies a device.
> +
> +What: /sys/bus/cdx/devices/.../reset
> +Date: January 2023
> +Contact: nipun.gupta@amd.com
> +Description:
Reads better without "would":
> + Writing a non-zero value to this file would reset the CDX> + device. On resetting the device, the corresponding driver
> + would be notified twice, once before the device is being
> + reset, and again after the reset has been complete.
Writing a non-zero value to this file resets the CDX
device. On resetting the device, the corresponding driver
is notified twice, once before the device is being
reset, and again after the reset has been complete.
> +
> + For example::
> +
> + # echo 1 > /sys/bus/cdx/.../reset
> +
> +What: /sys/bus/cdx/devices/.../remove
> +Date: January 2023
> +Contact: tarak.reddy@amd.com
> +Description:
> + Writing a non-zero value to this file would remove the
> + corrosponding device from the CDX bus. If the device is
> + to be reconfigured in the Hardware, the device can be
> + removed, so that the device driver does not access the
> + device while it is being reconfigured.
and: (also fix typo of corresponding)
Writing a non-zero value to this file removes the
corresponding device from the CDX bus. If the device is
to be reconfigured in the Hardware, the device can be
removed, so that the device driver does not access the
device while it is being reconfigured.
> +
> + For example::
> +
> + # echo 1 > /sys/bus/cdx/devices/.../remove
--
~Randy
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v6 7/7] cdx: add device attributes
2023-01-26 16:55 ` Randy Dunlap
@ 2023-01-27 11:17 ` Nipun Gupta
0 siblings, 0 replies; 14+ messages in thread
From: Nipun Gupta @ 2023-01-27 11:17 UTC (permalink / raw)
To: Randy Dunlap, robh+dt, krzysztof.kozlowski+dt, gregkh, rafael,
eric.auger, alex.williamson, cohuck, song.bao.hua,
mchehab+huawei, maz, f.fainelli, jeffrey.l.hugo, saravanak,
Michael.Srba, mani, yishaih, jgg, jgg, robin.murphy, will, joro,
masahiroy, ndesaulniers, linux-arm-kernel, linux-kbuild,
linux-kernel, devicetree
Cc: okaya, harpreet.anand, nikhil.agarwal, michal.simek, git
On 1/26/2023 10:25 PM, Randy Dunlap wrote:
> Hi--
>
> On 1/26/23 02:46, Nipun Gupta wrote:
>> diff --git a/Documentation/ABI/testing/sysfs-bus-cdx b/Documentation/ABI/testing/sysfs-bus-cdx
>> index 8c2425fdb6d9..69bdbc891743 100644
>> --- a/Documentation/ABI/testing/sysfs-bus-cdx
>> +++ b/Documentation/ABI/testing/sysfs-bus-cdx
>> @@ -10,3 +10,47 @@ Description:
>> For example::
>>
>> # echo 1 > /sys/bus/cdx/rescan
>> +
>> +What: /sys/bus/cdx/devices/.../vendor
>> +Date: January 2023
>> +Contact: nipun.gupta@amd.com
>> +Description:
>> + Vendor ID for this CDX device. Vendor ID is 16 bit
>> + identifier which is specific to the device manufacturer.
>> + Combination of Vendor ID and Device ID identifies a device.
>> +
>> +What: /sys/bus/cdx/devices/.../device
>> +Date: January 2023
>> +Contact: nipun.gupta@amd.com
>> +Description:
>> + Device ID for this CDX device. Device ID is a 16 bit
>> + identifier to identify a device type within the range
>> + of a device manufacturer.
>> + Combination of Vendor ID and Device ID identifies a device.
>> +
>> +What: /sys/bus/cdx/devices/.../reset
>> +Date: January 2023
>> +Contact: nipun.gupta@amd.com
>> +Description:
>
> Reads better without "would":
>
>> + Writing a non-zero value to this file would reset the CDX> + device. On resetting the device, the corresponding driver
>> + would be notified twice, once before the device is being
>> + reset, and again after the reset has been complete.
>
>
> Writing a non-zero value to this file resets the CDX
> device. On resetting the device, the corresponding driver
> is notified twice, once before the device is being
> reset, and again after the reset has been complete.
>
>> +
>> + For example::
>> +
>> + # echo 1 > /sys/bus/cdx/.../reset
>> +
>> +What: /sys/bus/cdx/devices/.../remove
>> +Date: January 2023
>> +Contact: tarak.reddy@amd.com
>> +Description:
>> + Writing a non-zero value to this file would remove the
>> + corrosponding device from the CDX bus. If the device is
>> + to be reconfigured in the Hardware, the device can be
>> + removed, so that the device driver does not access the
>> + device while it is being reconfigured.
>
> and: (also fix typo of corresponding)
Thanks for the review. Will fix and update these.
Regards
Nipun
>
> Writing a non-zero value to this file removes the
> corresponding device from the CDX bus. If the device is
> to be reconfigured in the Hardware, the device can be
> removed, so that the device driver does not access the
> device while it is being reconfigured.
>
>> +
>> + For example::
>> +
>> + # echo 1 > /sys/bus/cdx/devices/.../remove
>
> --
> ~Randy
^ permalink raw reply [flat|nested] 14+ messages in thread